Node.js Which control flow suits your project?

1 January 2013 4 minutes to read

It isn't a secret that Node.js has a problem with flow control. And there are many solutions that you can find in the internet. Most of them are simple wrappers for the default functionality. Another portion comes from the community of Node.js itself. Like this set of code standards[1] according nesting. Or personal feeling of a programmer about the count of nested callbacks.

One of these solutions is more than enough for a small project[2], but in a big project it will cause a problem. In the case of application's lifecycle, you'll spend many human hours to "read → realize" the code each time you change something that was not created by you.

I'll make a quick look at 3 different solutions for the control flow organization of Node.js and we will use the simplest example of the image upload process(not a perfect example for sure). Let’s say a use-case looks like this: a user wants to upload an image to the server, check the image extension and store the file in the filesystem (or CDN). Every project has such functionality.

For the first example we'll use a good enough library "Async.js". It contains many different methods, and from all this set we choose "async.waterfall" - it is what we need. Let me introduce our rabbit:

  async.waterfall( // waterfall example
    [
      function (callback) {
        im.identify(image.path, callback);
      },


      function (meta, callback) {
        extension = meta.format.toLowerCase();
        if (['jpeg', 'png'].indexOf(extension) === -1) {
          return callback('validations.ext');
        }
        fs.rename(image.path, newFilePath, callback);
      },


      function (callback) {
        im.resize(
          {srcPath: newFilePath, dstPath: newFilePathResized, width: 200},
          callback
        );
      }
    ],
    function (err) { // result
      if (err) {
        console.log(err);
      }
    }
  );

As you can see it is impossible to understand what is going on here without diving into linked functions. Moreover you can't describe what is the "meta" in the 2nd function. And to find it out, you should analyze other functions before you can start to implement a new business logic.

Similar situation is with "node-seq". It provides wrapper and additional cookies to handle the waterfall in different ways. But in common it looks like the previous example. Everytime your eyes will jump from bottom to top. Everytime.

Besides, you should learn this library. Because a more complex use-case makes this solution hard to understand without diving into diving into linked functions.

  Seq() // Seq example
    .seq(function () {
      im.identify(image.path, this);
    })
    .seq(function (meta) {
      extension = meta.format.toLowerCase();


      if (['jpeg', 'png'].indexOf(extension) === -1) {
        return this('validations.ext');
      }


      fs.rename(image.path, newFilePath, this);
    })
    .seq(function () {
      im.resize(
        {srcPath: newFilePath, dstPath: newFilePathResized, width: 200}, this
      );
    })
    .catch(function (err) { // error
      console.error(err);
    });

Some years ago, Marcel Laverdet introduced "node-fibers". This library adds support of coroutine/fiber to Node.js. Nowadays this idea is used by the "Meteor.js" project. And though in those times it made much noise but wasn’t viewed seriously by programmers, now it presents an adequate solution of day-to-day issues.

But we are talking about the flow control and the next example contains the code with "node-sync" library that uses "node-fibers".

  Sync( // sync example
    function() {
      var meta = im.identify.sync(null, image.path);


      if (['jpeg', 'png'].indexOf(meta.format.toLowerCase()) === -1) {
        throw new Error(meta.format);
      }


      fs.rename.sync(null, image.path, newFilePath);


      im.resize.sync(
        null, {srcPath: newFilePath, dstPath: newFilePathResized, width: 200}
      );


      return true;
    },
    function callback (err, result) { // result
      if (err) {
        console.log(err);
      }
    }
  );

Nice thing is that we have step-by-step interpretation. We can add something before or after the rename function. We can use the "throw" method. And we can return data without dancing. Bad side of this implementation is that library extends the function prototype and JavaScript has no interfaces to handle this situation. So, you must be sure that the library your are trying to work with has clear vision of object and function creation. First "null" (context) is annoying as well.

Performance

And, how much time does it take? It depends. The current test depends on much things. For example, each iteration makes copy, rename, resize steps. All of them require good amount of I/O time. Important thing is to realise that both Async.js (wrapper) and node-sync (Fibers) have almost similar values of ops/sec. Though the "node-seq" solution provides many features for data handling.

benchmark
2.4 GHz Intel Core i7, 12 GB 1333 MHz DDR3, SSD 240GB

Having all pro’s and contra’s before me, I’d conclude that all 3 ways don’t solve the problem in general. The first one is too complicated for comprehension, the second one contains excessive functional, the third one just isn’t ready for the moment to be used on a production server.

I think that Node.js community needs some time to cleanup all it has. That is why we are using "Async.js" for the current project. And “node-sync” (Fibers) for the next project. But if your team is ready for contribution to opensource, start with Fibers right now.