Install NPM Packages Automatically for Node.js on Windows Azure Web Site
- by Shaun
In one of my previous post I described and demonstrated how to use NPM packages in Node.js and Windows Azure Web Site (WAWS). In that post I used NPM command to install packages, and then use Git for Windows to commit my changes and sync them to WAWS git repository. Then WAWS will trigger a new deployment to host my Node.js application. Someone may notice that, a NPM package may contains many files and could be a little bit huge. For example, the “azure” package, which is the Windows Azure SDK for Node.js, is about 6MB. Another popular package “express”, which is a rich MVC framework for Node.js, is about 1MB. When I firstly push my codes to Windows Azure, all of them must be uploaded to the cloud. Is that possible to let Windows Azure download and install these packages for us? In this post, I will introduce how to make WAWS install all required packages for us when deploying. Let’s Start with Demo Demo is most straightforward. Let’s create a new WAWS and clone it to my local disk. Drag the folder into Git for Windows so that it can help us commit and push. Please refer to this post if you are not familiar with how to use Windows Azure Web Site, Git deployment, git clone and Git for Windows. And then open a command windows and install a package in our code folder. Let’s say I want to install “express”. And then created a new Node.js file named “server.js” and pasted the code as below. 1: var express = require("express");
2: var app = express();
3:
4: app.get("/", function(req, res) {
5: res.send("Hello Node.js and Express.");
6: });
7:
8: console.log("Web application opened.");
9: app.listen(process.env.PORT);
If we switch to Git for Windows right now we will find that it detected the changes we made, which includes the “server.js” and all files under “node_modules” folder. What we need to upload should only be our source code, but the huge package files also have to be uploaded as well. Now I will show you how to exclude them and let Windows Azure install the package on the cloud.
First we need to add a special file named “.gitignore”. It seems cannot be done directly from the file explorer since this file only contains extension name. So we need to do it from command line. Navigate to the local repository folder and execute the command below to create an empty file named “.gitignore”. If the command windows asked for input just press Enter.
1: echo > .gitignore
Now open this file and copy the content below and save.
1: node_modules
Now if we switch to Git for Windows we will found that the packages under the “node_modules” were not in the change list. So now if we commit and push, the “express” packages will not be uploaded to Windows Azure.
Second, let’s tell Windows Azure which packages it needs to install when deploying. Create another file named “package.json” and copy the content below into that file and save.
1: {
2: "name": "npmdemo",
3: "version": "1.0.0",
4: "dependencies": {
5: "express": "*"
6: }
7: }
Now back to Git for Windows, commit our changes and push it to WAWS.
Then let’s open the WAWS in developer portal, we will see that there’s a new deployment finished. Click the arrow right side of this deployment we can see how WAWS handle this deployment. Especially we can find WAWS executed NPM.
And if we opened the log we can review what command WAWS executed to install the packages and the installation output messages. As you can see WAWS installed “express” for me from the cloud side, so that I don’t need to upload the whole bunch of the package to Azure.
Open this website and we can see the result, which proved the “express” had been installed successfully.
What’s Happened Under the Hood
Now let’s explain a bit on what the “.gitignore” and “package.json” mean.
The “.gitignore” is an ignore configuration file for git repository. All files and folders listed in the “.gitignore” will be skipped from git push. In the example below I copied “node_modules” into this file in my local repository. This means, do not track and upload all files under the “node_modules” folder. So by using “.gitignore” I skipped all packages from uploading to Windows Azure.
“.gitignore” can contain files, folders. It can also contain the files and folders that we do NOT want to ignore. In the next section we will see how to use the un-ignore syntax to make the SQL package included.
The “package.json” file is the package definition file for Node.js application. We can define the application name, version, description, author, etc. information in it in JSON format. And we can also put the dependent packages as well, to indicate which packages this Node.js application is needed.
In WAWS, name and version is necessary. And when a deployment happened, WAWS will look into this file, find the dependent packages, execute the NPM command to install them one by one. So in the demo above I copied “express” into this file so that WAWS will install it for me automatically.
I updated the dependencies section of the “package.json” file manually. But this can be done partially automatically. If we have a valid “package.json” in our local repository, then when we are going to install some packages we can specify “--save” parameter in “npm install” command, so that NPM will help us upgrade the dependencies part.
For example, when I wanted to install “azure” package I should execute the command as below. Note that I added “--save” with the command.
1: npm install azure --save
Once it finished my “package.json” will be updated automatically.
Each dependent packages will be presented here. The JSON key is the package name while the value is the version range. Below is a brief list of the version range format. For more information about the “package.json” please refer here.
Format
Description
Example
version
Must match the version exactly.
"azure": "0.6.7"
>=version
Must be equal or great than the version.
"azure": ">0.6.0"
1.2.x
The version number must start with the supplied digits, but any digit may be used in place of the x.
"azure": "0.6.x"
~version
The version must be at least as high as the range, and it must be less than the next major revision above the range.
"azure": "~0.6.7"
*
Matches any version.
"azure": "*"
And WAWS will install the proper version of the packages based on what you defined here. The process of WAWS git deployment and NPM installation would be like this.
But Some Packages…
As we know, when we specified the dependencies in “package.json” WAWS will download and install them on the cloud. For most of packages it works very well. But there are some special packages may not work. This means, if the package installation needs some special environment restraints it might be failed.
For example, the SQL Server Driver for Node.js package needs “node-gyp”, Python and C++ 2010 installed on the target machine during the NPM installation. If we just put the “msnodesql” in “package.json” file and push it to WAWS, the deployment will be failed since there’s no “node-gyp”, Python and C++ 2010 in the WAWS virtual machine.
For example, the “server.js” file.
1: var express = require("express");
2: var app = express();
3:
4: app.get("/", function(req, res) {
5: res.send("Hello Node.js and Express.");
6: });
7:
8: var sql = require("msnodesql");
9: var connectionString = "Driver={SQL Server Native Client 10.0};Server=tcp:tqy4c0isfr.database.windows.net,1433;Database=msteched2012;Uid=shaunxu@tqy4c0isfr;Pwd=P@ssw0rd123;Encrypt=yes;Connection Timeout=30;";
10: app.get("/sql", function (req, res) {
11: sql.open(connectionString, function (err, conn) {
12: if (err) {
13: console.log(err);
14: res.send(500, "Cannot open connection.");
15: }
16: else {
17: conn.queryRaw("SELECT * FROM [Resource]", function (err, results) {
18: if (err) {
19: console.log(err);
20: res.send(500, "Cannot retrieve records.");
21: }
22: else {
23: res.json(results);
24: }
25: });
26: }
27: });
28: });
29:
30: console.log("Web application opened.");
31: app.listen(process.env.PORT);
The “package.json” file.
1: {
2: "name": "npmdemo",
3: "version": "1.0.0",
4: "dependencies": {
5: "express": "*",
6: "msnodesql": "*"
7: }
8: }
And it failed to deploy to WAWS.
From the NPM log we can see it’s because “msnodesql” cannot be installed on WAWS.
The solution is, in “.gitignore” file we should ignore all packages except the “msnodesql”, and upload the package by ourselves. This can be done by use the content as below. We firstly un-ignored the “node_modules” folder. And then we ignored all sub folders but need git to check each sub folders. And then we un-ignore one of the sub folders named “msnodesql” which is the SQL Server Node.js Driver.
1: !node_modules/
2:
3: node_modules/*
4: !node_modules/msnodesql
For more information about the syntax of “.gitignore” please refer to this thread.
Now if we go to Git for Windows we will find the “msnodesql” was included in the uncommitted set while “express” was not. I also need remove the dependency of “msnodesql” from “package.json”.
Commit and push to WAWS. Now we can see the deployment successfully done.
And then we can use the Windows Azure SQL Database from our Node.js application through the “msnodesql” package we uploaded.
Summary
In this post I demonstrated how to leverage the deployment process of Windows Azure Web Site to install NPM packages during the publish action. With the “.gitignore” and “package.json” file we can ignore the dependent packages from our Node.js and let Windows Azure Web Site download and install them while deployed.
For some special packages that cannot be installed by Windows Azure Web Site, such as “msnodesql”, we can put them into the publish payload as well.
With the combination of Windows Azure Web Site, Node.js and NPM it makes even more easy and quick for us to develop and deploy our Node.js application to the cloud.
Hope this helps,
Shaun
All documents and related graphics, codes are provided "AS IS" without warranty of any kind.
Copyright © Shaun Ziyan Xu. This work is licensed under the Creative Commons License.
