Tag Archives: npm

Using npm config

Intro

I think most people (I don’t interact with most people so I don’t know really) don’t know about the various wonders npm does (or tries to do). One of them is that you can totally use npm as a build tool, an alternative to Grunt or Gulp.

In this post I’m going to discuss how you can use npm as a configuration system on top of your module written in (probably) node.js.

But first, let me discuss how people have been addressing the problem of configuration.

Popular config systems

Environment variables

Of course the most popular option would be to use environment variables. There are different ways to set environment variables in various operating systems but once set, these are passed onto your program whenever it is run. In node.js you can access an environment variable MY_AWESOME_VAR using process.env["MY_VAR"]. Environment variables can be considered per-user and thus has a consequence: conflicts can occur between programs. Some programs address this by adding a prefix. Many don’t.

.env files

Since setting environment variables on an operating system can be tedious, and have a huge scope with many possible conflicts (per-user scope), some have come up with .env files. Dot env files are files that you usually put on the root of your module directory, and when you run your module, it will import the .env file to your environment variable store. That way, you can both read from your OS’ environment variables and possibly override them in the .env file to provide a per-directory configuration. Packages enabling this functionality include dotenv.

Runtime command-line arguments

Of course you can always enable configuration setting via command-line arguments passed to your program. Just like mongo -u user -p=pass. This provides a very specific scope, specific to your program’s running instance. But if you’re typing this manually, it could be tedious. I mean, who here are still typing their usernames and passwords whenever they git push? Anyone?

From a database

What? Really? That’s a thing? I didn’t know that…

The awesome: npm config

The basics

What most people don’t know is that they could do all of these without using any dependencies, using just npm. (Except for the database thing. That’s so stupid… Why do that?)

Okay, let me clear this out of the way first. You’re probably reading this because you didn’t understand how npm config works, but here are all of the relevant documentation:

  • npm-config – Tells you what typing npm config should do, but not exactly why or how?
  • misc/config – Has some info but the bulk is npm’s own config
  • npmrc – Fair enough.

What I’m gonna teach here is how to create a configuration system that will be specific to your own module, and nothing else.

Supposing we’re done, let’s show first how you would read a configured variable such as password in a node.js app:

process.env.npm_package_config_password

Well, that’s easy enough. But wait. Aren’t you curious to know what all of that does? I know I am.

  • npm is just a prefix to all of the environment variables npm inserts. You know, to avoid conflicts.
  • package is a prefix to package-specific variables. If you have a package.json file (you should) a bunch of things from that will appear in your process.env prefixed with npm_package.
  • config is, specifically, your package-specific config. Which I am discussing right now.
  • password is the name of the config var.
  • _ (underscore) is the separator npm uses.

Suppose you are a module author authoring the module awesome-point-of-sale that requires a connection to a MySQL database. From the above you would know what to pass to create a connection:

const MySQL = require('mysql')
let connection = MySQL.createConnection({
  host: process.env.npm_config_package_host,
  user: process.env.npm_config_package_user,
  password: process.env.npm_config_package_pass,
  database: process.env.npm_config_package_db
})

Now, what you want to do is to ask your users to configure their system so they would have the correct configuration. One thing they can do is to set a per-user config:

npm config set awesome-point-of-sale:host 192.168.0.123

This will save the said config inside a .npmrc file in your $HOME directory. On Windows, this is the %USERPROFILE% directory. Don’t be shocked why they do this right there, it’s standard practice, really.

Alternatively, they can open a text editor using:

npm config edit

They would then append the following line in .env format:

awesome-point-of-sale:host=192.168.0.123

There are other places where you could put .npmrc config files. See the npmrc docs for details. With this it is possible to set config vars in different scopes: per-system, per-user, and per-directory. Hint: Try npm config edit -g!

Now, how would npm insert all this stuff in your program?

You should totally run your program using npm run-script. Google “npm as build tool” for ideas. (Although I’m considering blogging it myself. I have some personal opinions.)

And that’s it! We’re done. Wait, not yet? Yes, there’s more to it than it seems.

Defaults

Yes, definitely! You can set your project defaults in your config. Suppose you are able to run a MySQL database locally. We can use that.

package.json:

{
  "config": {
    "host": "localhost",
    "user": "admin",
    "pass": "",
    "db": "pos"
  }
  ...
}

Grouped configs

What if you want to group your config vars such that there’s a separate config for how to connect to your MySQL database and how to start your server? What you can do is group your configs into objects:

package.json:

{
  "config": {
    "mysql": {
      "host": "localhost",
      "user": "admin",
      ...
    },
    "server": {
      "host": "0.0.0.0",
      "port": 80
    }
  }
  ...
}

Now how would this reflect in your program? What npm does is it uses the underscore character (_) to indicate the path to the variable:

let serverHost = process.env.npm_package_config_server_host
let serverPort = process.env.npm_package_config_server_port
let mysqlHost = process.env.npm_package_config_mysql_host

How will users set this in the config though? The same way:

npm config set awesome-point-of-sale:server_host 0.0.0.0

And that’s a wrap!

Whoo! Aren’t you glad that’s over? Although if you have an absurd module name such as awesome-point-of-sale you might want to consider adding some config functionality. Something like npm init, even. (Whoa, that’s a really good suggestion right there that I could use myself.)

Why the MEAN stack is mean (bad!)

While I have been browsing for freelancer gigs, I can’t help but notice employers actually requiring construction of a MEAN stack-based website. And this is actually bad! Companies and employers should not require or fully rely on a specific methodology to do things—after all, all roads lead to Rome. (For lack of a better saying? Help me on this one, I suck at writing.)

M: The story of the database

So you’re a company and you decide to move towards node.js development. After all, node.js is awesome and all. From a PHP-based workspace, you’d probably be using MySQL for your projects. Well guess what: MySQL is still a perfectly valid database solution for node.js apps. If instead you’re from a Windows environment, you’re still in luck: SQL Server is still a perfectly valid database solution for node.js apps. Every database was constructed with specific needs, after all. One such need is the deprecation of a query language to interact with the database. Thus arose NoSQL databases, such as MongoDB. Uhmm. Yep. Feel free to go on with MongoDB if your heart tells you to do so.

E: The story of the webserver

Ah. Express. I can tell you, I have used Express in the early days. But did anyone tell you how hard it is to reorder its varying middlewares? People of the node.js land have been constantly switching between middleware-based chaining and configuration-setting time and again. Alas, for Express, it was very hard to do so. Still, Express is solid and established in the node.js community as a standard webserver. But, have you heard the story of a certain company who got fed up tweaking Express and ended up developing their own webserver module? Yes, Express is good, but are you sure that’s want you want? What if you want streaming? What if you only just needed Connect? What if you just want to serve up some static files? Did you really need Express?

A: The story of the view

And now we temporarily depart node.js land and set sail to the far-off land of Bower. Back at the port, did you ask yourself: what’s Bower? Did we really need Bower? Yes, as an npm fanboy I can put a big black no to that. But other front-end developers had a need for other things, hence Bower. Bower’s functionality is still valid though. Throw in a bunch of files and a bower.json, voila, a Bower package. (Which is their own standard… package.json is from CommonJS.) But seriously, let’s get back to topic. I haven’t worked on Angular before, but I can more or less see what it can do from its home page. But really, Angular is just another tool that transforms data into a front-end view—a markup language that your web browser will understand. And to that regard, there are many modules on npm that already do just that. Of course I should mention my favorite, Jade. There’s Handlebars, Coffeecup, (I probably missed a lot of popular ones here…) that are awesome. And they are all equally valid options to render a view on the front-end.

NB: At the time of writing I am aware that Angular has landed onto node.js land. However, the MEAN stack has been popular before that and required Bower for it to be used.

N: The story of the executable

Let’s face it, companies: PHP sucks. Programmers don’t want to delve into this suck-hole. That’s why Facebook invented the Hack programming language and HHVM. Most PHP applications are single-threaded in nature and built with being run in Apache in mind. PHP did not have a webserver utility of its own until such a time when it’s not even relevant anymore. And imagine if someone opens up your website in 10,000 tabs in one fell swoop. You’d actually see a swarm of php.exes running in your operating system. Have you heard of bug #29992? I have struggled with it, and if you’re still a PHP developer, learn how to live with this nightmare: it’s a wontfix.

So a recap of why people love node.js: It is asynchronous in nature, and it uses the awesome V8 Javascript engine. Though limited to a single-thread, that should not stop you from forking it into other processes, which you should really do on another CPU or server. That said, IIS has been ahead in the asynchronicity game, though I don’t imagine Linux fanboys becoming Windows fanboys. Did I mention that node.js is cross-platform?

Problems coming from other full-stack methodologies

Yeah, we all know what the LAMP stack is. Linux, Apache, MySQL, PHP, we get it. The LAMP stack really just had one major problem:

Windows programmers.

Yep. It makes sense, right? You may be a PHP programmer, but in the end you’ll encounter Linux-only things and shit. And that just won’t work in a Windows environment. Another problem I experienced is Perl. Perl never really clicked to me. I didn’t know how it worked.

Either way, MEAN stack or not, I like that people are moving away from PHP. End the tyranny of PHP!

Pros

Having a standard way of doing things is great for portability and consistency. It will also be easier to find programmers who are experts on doing such things.

Cons

It has been hard for freelancers like me to find clients who don’t require the MEAN stack in their projects. node.js programmers should be free to npm install whatever shit they want to install.

Let me leave you with something to comtemplate with: meanjs.org’s package.json.

Why I like node.js

Months ago I stumbled across this and though, meh, it’s just another one of those dumb programming shenanigans. Over the last few weeks, I got to try to program with it. But before we go there, let’s stop and define what node.js is really.

Node.js is a program. It exposes a framework (an API), on top of which you can program using Javascript. Other than that, it boasts of this framework having an asynchronous nature, which we all know is better than synchronous sh*t.

I have crafted this mini table for a quick comparison. NB: These are all purely my opinion.

Programming language Javascript! (No, node.js is not a programming language!) Visual Basic or C# Freaking Java!
Program (compiler or interpreter) nodejs.exe I don’t know, MSBuild? Freaking javac!
Framework I don’t know, did they give it a name? The glorious .NET Framework Freaking JRE?!
Your friendly neighborhood IDE PhpStorm, Freaking NetBeans?!??!?!, Visual Studio (!) Visual Studio Freaking NetBeans?!??!?! Or Eclipse?
Community resource pool (?) Plugins? npm Nuget ????

Now that’s done, let’s have my answers: why like node.js?

  1. It’s freaking Javascript! It’s a language we all know and love! (It’s a C-based language, which most of us know, but scare most beginners, but hey, you can always alleviate that if you’re into language transformers like CoffeeScript.) It’s the language your server tells your client user agent to do! Now, it’s the language you write your server program in! That’s right:
  2. It’s your freaking own server program! Yes, unlike Apache and PHP, which are separate web server and interpreter, node.js allows you to be free and write your own! Although this creates an issue for hosting providers that might not want to give their customers that much freedom. Although there are some who are willing to give developers a chance. Note that node.js doesn’t have a built-in web server of its own, but it has an HTTP library which can get you started. And there are always libraries that can simplify your intended task.
  3. Yes, and with npm, installing third-party tools are a breeze. Just like apt, it takes care of all the dependencies your third-party library might require.
  4. It’s based on V8, the same Javascript engine used by the awesome browser Chrome! If you’re not familiar with what V8 does, here the thing: V8 compiles Javascript code into freaking native machine code. What this means is that when it’s done compiling, your code will run blazingly fast once it starts!
  5. It’s already in the intro, but it’s based on a freaking asynchronous API! What this means is that when some code reads a file, that code can wait for that I/O operation to complete while other code consumes the precious CPU cycles! This can be really useful especially in the website hosting context where many users can connect simultaneously to your website.
  6. Write a small script and it should just work, whether it’s a simple console.log("Hello World");, a demo to show to your students (given you’re a teacher? Probably…), or just to show off your m4d w3b scr4p1ng sk1ll2. Because Javascript is a freaking scripting language!
  7. CREATE YOUR OWN GAME SERVER! THIS IS NOT POSSIBLE USING FREAKING APACHE+PHP!!!!! NO DATABASES REQUIRED!

As node has many good things, it also has some bad shortcomings. Let’s take a short look at that:

  1. This really is for JS and not node: I am really lamenting the lack of an “await” keyword for JS. It would be really useful since node has an asynchronous API. Right now the closest we have for that is Q.async + ES6′ yield.
  2. Yeah, we all know Javascript, but only because it’s an ages-old programming language. There are a couple more advanced programming languages out there, Visual Basic being my (past? :O </3 oh no!) favorite, Ruby (psssh… only Taric uses this), and Python (Who uses Python anyway? I don’t know… NASA? Freaking Google?) that addresses many of the older languages’ shortcomings. But as these are actively developed, so are the oldies. I do hope Javascript catches up one day. Fingers crossed.

And that’s it! This concludes my fanboyish post about the freakishingly awesome node.js.