Earlier today I had a major “Aha!” moment, the kind that you get every once in a while at random times about things you weren’t thinking about. I was walking through Macy’s and I thought, “Oh, so a npm module that you require is just a JavaScript object.” Everything suddenly became much simpler and clearer to me, and server-side JavaScript, CommonJS, and npm all boiled down to plain old JavaScript objects and functions. While this is a bit of a simplification of the entire process, once I had that idea in my head, I felt curious enough to figure out how the require function works exactly.

Acquiring require intelligence

The first place I usually check for things that have to do with Node core functionality is the docs. Usually once I find the docs, I read intensely for 5-10 minutes, start to space off, check my twitter, and then come back completely lost. However, this time, the documentation was very interesting and really easy to read, I suggest you check it out: modules docs. I then try to read someone’s plain English explanation of it, which can be a bit more digestible than docs. The third chapter of Professional Node.js does a pretty great job of this, and I was feeling pretty awesome about the ins and outs of the require function. One thing I learned is the following fun fact:

FUN FACT: you can turn .json files into JavaScript objects simply by using the require function, e.g. require('./locations.json'), or even just require('./locations');.

After I read through both of those great resources, I decide to dig a little bit deeper. I found the Node.js repository on Github and started sniffing around source code for the exact implementation of the require function. I find this kind of source code sniffing to be the most fruitful: when you are only looking to understand one part of the system and can trace that part all the way to its roots. As I was sifting through the 1,181 instances of the require function being called, I had a strange thought that is big enough for the next section title:

How does Node “require” the module that defines “require”?

Yeah, it blew my mind too. It turns out, there is a fun fact about the way Node.js defines the module system:

FUN FACT: Node.js defines the module system twice, once with a minimal system for native modules and again for the general module system as a whole.

The implementation for the native module system only takes 64 lines of code! You can click that link and read through it in five minutes. The require method of NativeModule essential boils down to running NativeModule.compile() and returning NativeModule.exports. The entirety of the NativeModule.compile() method can be seen here:

NativeModule.getSource = function(id) {
  return NativeModule._source[id];
}
NativeModule.wrap = function(script) {
  return NativeModule.wrapper[0] + script + NativeModule.wrapper[1];
};
NativeModule.wrapper = [
  '(function (exports, require, module, __filename, __dirname) { ',
  '\n});'
];
NativeModule.prototype.compile = function() {
  var source = NativeModule.getSource(this.id);
  source = NativeModule.wrap(source);
  var fn = runInThisContext(source, { filename: this.filename });
  fn(this.exports, NativeModule.require, this, this.filename);
  this.loaded = true;
};

So when compiling a native module, Node looks it up in the _source dictionary, wraps it in a function providing require, module and __dirname, and then calls that function with the module’s reference.

I was pretty impressed with this implementation, but I still wanted to know how the entire system worked. I kept digging until I found the node/lib/modules.js file, which defined the whole implementation of the CommonJS module system for Node. You can read along here.

It took longer to read this file, but I did stumble upon one sneaky tidbit on line 415 that no one ever told me about:

for (var k in global) {
  sandbox[k] = global[k];
}

Here’s your third and final fun fact:

FUN FACT Node.js has a global object that you can assign and re-assign values to that will persist in all of your modules. This is VERY frowned upon, and can lead to dangerous consequences.

I played around with the global object for a while, and found out just how dangerous it can be for side-effects. Specifically, if someone wanted to take advantage of the number of times you misspell require as reqiure, they could add the following code to their module and wait until you run the code with a misspelling:

global.reqiure = function() {
  setInterval(function() {
    console.log('You have been hacked! Get spellcheck!');
  }, 100);
};

Thankfully, I found that you cannot rewrite the implementation of require, but I am now pretty scared of what evil-minded module writers could do by slipping in references to the global object in their module. This reinforces two big life lessons in the node community:

1. LIFE LESSON #1 Never use global
2. LIFE LESSON #2 Always read through the source of your third-party dependencies. Double check that they did not use global.

That concludes my adventure into the Node source for today. See you tomorrow!

---

Tweet

---