Tag Archive : nodejs

/ nodejs

NodeJS Timers

December 9, 2015 | Article | No Comments

A timer is a specialized type of clock for measuring time intervals. It is used for deploying routine action.

Node implements the timers API which also found in web browsers.

setTimeout

setTimeout let us to schedule an arbitrary function to be executed in the future. For example:

var timeout = 2000;    // 2 seconds
setTimeout(function() {
    console.log('time out!');
}, timeout);

The code above will register a function to be called when the timeout expires. As in any place in JavaScript, we can pass in an inline function, the name of a function or a variable which value is a function.

If we set the timeout to be 0 (zero), the function we pass gets executed some time after the stack clears, but with no waiting. This can be used to, for instance schedule a function that does not need to be executed immediately. This was a trick sometimes used on browser JavaScript. Another alternative we can use is process.nextTick() which is more efficient.

clearTimeout

A timer schedule can be disabled after it is scheduled. To clear it, we need a timeout handle which is returned by function setTimeout.

var timeoutHandle = setTimeout(function() { 
    console.log('Groaaarrrr!!!');
}, 1000);
clearTimeout(timeoutHandle);

If you look carefully, the timeout will never execute because we clear it just after we set it.

Another example:

var timeoutA = setTimeout(function() {
    console.log('timeout A');
}, 2000);

var timeoutB = setTimeout(function() {
    console.log('timeout B');
    clearTimeout(timeoutA);
}, 1000);

Which timeoutA will never be executed.

There are two timers above, A with timeout 2 seconds and B with timeout 1 second. The timeoutB (which fires first) unschedule timeoutA so timeout never executes and the program exits right after the timeoutB is executed.

setInterval

Set interval is similar to set timeout, but schedules a given function to run every X seconds.

var period = 1000; // 1 second
var interval = setInterval(function() {
  console.log('tick');
}, period);

That code will indefinitely keep the console logging ‘tick’ unless we terminate Node.

clearInterval

To terminate schedule set by setInterval, the procedure we do is similar to what we did to setTimeout. We need interval handler returned by setInterval and do it like this:

var interval = setInterval(...);
clearInterval(interval);

process.nextTick

A callback function can also be scheduled to run on next run of the event loop. To do so, we use:

process.nextTick(function() {
    // This runs on the next event loop
    console.log('yay!');
});

This method is preferred to setTimeout(fn, 0) because it is more efficient.

On each loop, the event loop executes the queued I/O events sequentially by calling associated callbacks. If, on any of the callbacks you take too long, the event loop won’t be processing other pending I/O events meanwhile (blocking). This can lead to waiting customers or tasks. When executing something that may take too long, we can delay execution until the next event loop, so waiting events will be processed meanwhile. It’s like going to the back of the line on a waiting line.

To escape the current event loop, we can use process.nextTick() like this:

process.nextTick(function() {
    // do something
});

This will delay processing that is not necessary to do immediately to the next event loop.

For instance, we need to remove a file, but perhaps we don’t need to do it before replying to the client. So we could do something like this:

stream.on('data', funciton(data) {
    stream.end('my response');
    process.nextTick(function() {
        fs.unlink('path/to/file');
    });
});

Let’s say we want to schedule a function that does some I/O – like parsing a log file -to execute periodically, and we want to guarantee that no two of those functions are executing at the same time. The best way is not to use a setInterval, since we don’t have that guarantee. the interval will fire no matter if the function has finished it’s duty or not.

Supposing there is an asynchronous function called “async” that performs some IO and that gets a callback to be invoked when finished, and we want to call it every second:

var interval = 1000;
setInterval(function() {
    async(function() {
        console.log('async is done');
    }
}, interval);

If any two async() calls can’t overlap, it is better off using tail recursion like this:

var interval = 1000;
(function schedule() {
    setTimeout(function() {
        async(function() {
            console.log('async is done!');
            schedule();
        });
    }, interval);
})();

Here we declare schedule() and invoking it immediately after we are declaring it.

This function schedules another function to execute within one second. The other function will then call async() and only when async is done we schedule a new one by calling schedul() again, this time inside the schedule function. This way we can be sure that no two calls to async execute simultaneously in this context.

The difference is that we probably won’t have async called every second (unless async takes to time to execute), but we will have it called 1 second after the last one finished.

NodeJS Event Emitter

December 9, 2015 | Article | No Comments

Many objects can emit events, in NodeJS of course. For instance a TCP server can emit a ‘connect’ event every time a client connects, or a file stream request can emit a ‘data’ event.

Connecting an Event

One can listen for events. If you are familiar with other event-driven programming, you will know that there must be a function or method “addListener” where you have to pass a callback. Every time the event is triggered, for example ‘data’ event is triggered every time there is some data available to read, then your callback is called.

In NodeJS, here is how we can achieve that:

var fs = require('fs');      // get the fs module
var readStream = fs.createReadStream('file.txt');
readStream.on('data', function(data) {
    console.log(data);
});
readStream.on('end', function(data) {
    console.log('file ended');
});

Here on readStream object we are binding two event: ‘data’ and ‘end’. We pass callback function to handle each of these cases. All are available uniquely to handle events from readStream object.

We can either pass in an anonymous function (as we are doing here), or a function name for a function available on the current scope, or even a variable containing a function.

Only Connect Once

There is a case where we only want to handle an event once and the rest we give up for it. It means, we only interests in first event only. We want to listen for event exactly once, no more or less.

There are two ways to do it: using .once() method or make sure we remove the callback once we are called.

The first on is the simplest way. We use .once() to tell NodeJS that we are only interested in handling first event occurred.

object.once('event', function() {
    // Callback body
});

Other way is

function evtListener() {
    // Function body
    object.removeListener('event', evtListener);
}
object.on('event', evtListener);

Here we use removeListener() which will be discussed more in next section.

On two above samples, make sure you pass appropriate callback i.e. providing appropriate argument number. The event also should be specified.

Removing Callback from Certain Event

Though we have use it in previous section, we will discuss it again here.

To remove a callback we need the object in which we will remove the callback from it and also the event name. Note that this is a pair which we should provide. We can’t provide only one of it.

function evtListener() {
    // Function body
    object.removeListener('event', evtListener);
}
object.on('event', evtListener);

The removeListener belongs to the EventEmitter pattern. It accepts the event name and the function is should remove.

Removing All Callback from Certain Event

If you ever need to, removing all listener for an event from an Event Emitter is possible. We can use:

object.removeAllListener('event');

Creating Self-Defined Event

One can use this Event-Emitter patter throughout application. The way we do is creating a pseudo-class and make it inherit from the EventEmitter.

var EventEmitter = require('events').EventEmitter,
    util         = require('util');

// Here is the MyClass constructor
var MyClass = function(option1, option2) {
    this.option1 = option1;
    this.option2 = option2;
}

util.inherits(MyClass, EventEmitter);

util.inherits() is setting up the prototype chain so that we get the EventEmitter prototype methods available on MyClass instance.

That way, instances of MyClass can emit events:

MyClass.prototype.someMethod = function() {
    this.emit('custom event', 'some arguments');
}

Which emitting an event named ‘custom event’, sending also some data (in this case is “some arguments”).

A clients of MyClass instance can listen to “custom events” event by:

var myInstance = new MyClass(1,2);
myInstance.on('custom event', function() {
    console.log('got a custom event!');
});

NodeJS Buffers

December 9, 2015 | Article | No Comments

Natively, JavaScript is not very good at handling binary data. Therefore, NodeJS adds a native buffer implementation and still using JavaScript’s way to manipulate it. The class here is the standard way in Node to transport data.

Generally, buffer can be passed on every Node API which require data to be sent. Also, when receiving data on a callback, we get a buffer (except when we specify stream encoding, in which we get a string).

Create a Buffer

The default encoding format in NodeJS is UTF-8. To create a Buffer from UTF-8 string, we can do:

var buff = new Buffer('Hello World');

A new buffer can also be created from other encoding format. As long as we specify the encoding format in second argument, there is no problem:

var buf = new Buffer('8b76fde713ce', 'base64');

Accepted encodings are: “ascii”, “utf8”, and “base64”.

We can also create a new empty buffer by specify the size:

var buf = new Buffer(1024);

Accessing Buffer

Accessing a buffer is like accessing an array of string. We use [] to access individual ‘character’.

buf[20] = 127;   // Set byte 20 to 127

Format Conversion

A data held on a buffer using an encoding format can be converted to other encoding format.

var str  = buf.toString('utf8');     // UTF-8
var str1 = buf.toString('base64');   // Base64
var str2 = buf.toString('ascii');    // ASCII

When you don’t specify the encoding, Node will assume we are going to use UTF-8. If you need specific encoding, pass it as the argument.

Slice a Buffer

A buffer can be sliced into a smaller buffer by using the appropriately named slice() method.

var buffer = new Buffer('A buffer with UTF-8 encoded string');
var slice = buffer.slice(10,20);

On above code, we slice the original buffer that has 34 bytes into a new buffer that has 10 bytes equal to the 10th to 20th bytes of original buffer.

Note that the slice function does not create new buffer memory, it uses the original untouched buffer underneath.

Copy from Buffer

We can copy a part of a buffer into another pre-allocated buffer by:

var buffer = new Buffer('A buffer with UTF-8 encoded string');
var slice = new Buffer(10);
var targetStart = 0,
    sourceStart = 10,
    sourceEnd = 20;

buffer.copy(slice, targetStart, sourceStart, sourceEnd);

It should be self-explained. Here we copy part of buffer into slice, but only data on positions 10 through 20.

NodeJS Utilities: About I/O and Debugging

December 9, 2015 | Article | No Comments

Utilities, like implied by the name, is utilities employed for some distinct purpose. These utilities are provided by NodeJS through global objects.

console

Node provides a global “console” object to which we can output strings. However, the output is classified into some mode, according to the output stream it print the output to.

.log()

If you want to print data to stdout, it’s as simple as writing following code:

console.log("Hello World!");

Which will print “Hello World!”. The data (string in that case) is streamed out after formatting it. It is mainly used for simple output and instead of string we can also output an object, like this:

var a = {1: true, 2: false};
console.log(a);    // => {'1': true, '2': false}

We can also use string interpolation to print out things, like:

var a = {1: true, 2: false};
console.log('This is a number: %d, and this is a stirng: %s, ' +
            'and this is an object outputted as JSON: %j',
            42, 'Hello', a);

Which in turns print:

This is a number: 42, and this is a stirng: Hello, and this is an object outputted as JSON: {"1": true, "2": false}

If you are familiar with C or C++, you might find it similar with C’s printf() function. The placeholder is similar, you use %d for number (integer and floating point number), %s for string, and %j for JSON which is not exists in C’s formatting.

.warn()

If you want to print out to stderr, you can do this:

console.warn("Warning!!");

.trace()

And to print stack trace, you can do:

console.trace();

For stack trace, you will be presented by the current stack condition.

util

util is a module, which bundles some functions. To use this module, we need to include the util module:

var util = require('util');

.log()

var util = require('util');
util.log('Hello');

Similar to console.log(), however it’s slightly different. The util.log() will print current timestamp and given string in which build a line like this: Mar 17:11:09 – Hello

.inspect()

There is also a handy function, inspect(), which is nice for quick debugging by inspecting and printing an object properties.

var util = require('util');
var a = {1: true, 2: false};
console.log(util.inspect(a));

We can give more arguments to util.inspect() which are in following format:

util.inspect(object, showHidden, depth = 2, showColors);

showHidden is argument which inspect non-enumerable properties if it is turned on. This properties are belong to the object prototype chain, not the object itself. The depth, third argument, is the default depth on the object graph it should show. This is useful when inspecting large objects. To recurse indefinitely, pass a null value.

An important note: util.inspect keeps track of the visited objects. If there is a circular dependencies, a string “[Circular]” will appear on the outputted string.

NodeJS API Quick Tour

December 9, 2015 | Article | No Comments

Most programming language has shipped what it call as standard library. In NodeJS, there is a predefined library which we call API. All APIs are exactly a module, which can be included on source code.

Node provides a platform API that covers some aspects:

  1. process
  2. filesystem
  3. networking
  4. utilities

Note that we won’t cover all the object available, nor cover a material in details. For that purpose, you should go to http://nodejs.org/api/ instead.

Our list is also built only for stable modules. There are some unstable modules which may change later, for example crypto. We won’t cover that here.

[ Process ]

Node allows programmer to analyze process (environment variables, etc) and manage external processes. The involved modules are:

process

Inquire the current process to know the PID, environment variables, platform, memory usage, etc.

child_process

Spawn and kill new processes, execute commands and pipe their outputs

[ File System ]

Low-level API is also provided to manipulate files and directory. All the API is influenced by POSIX style.

fs

This is used for file manipulation, including: create, remove, load, write, and read files. This modules also used for create read and write streams.

path

Normalize and join file paths. It can also be used for checking whether a file exists or is a directory.

[ Networking ]

Used for networking purpose such as connecting, sending and receiving information over network.

net

This module is used for creating a TCP server or client.

dgram

This module is used for manipulating UDP packets, including receiving and send UDP packets.

http

Create an HTTP server or client. It is also a more specific version of net module.

tls

Transport Layer Security (TLS) is a successor of Secure Socket Layer (SSL) protocol. Node uses OpenSSL to provide TLS and/or SSL encrypted stream communication.

https

Implementing http over TLS/SSL.

dns

This module implement asynchronous DNS resolution.

[ Utilities ]

Various utilities for NodeJS.

util

A module which bundles various utility functions.

Introduction to NodeJS Modules

December 9, 2015 | Article | 1 Comment

NodeJS is different with client-side JavaScript, aside from the platform it run on.

Client-side Javascript has a bad reputation because of the common namespace shared by all scripts, which can lead to conflicts and security leaks. Node however, implements CommonJS modules standard. Node use modules where each module is separated from other modules. This way, each module have separate namespace and could be exported only on the desired properties.

Import a Module

Import / include an existing module is easy. One can use a function require() to do so.

var module = require('module_name');

Which import module_name and name it as module. The module is either standard API provided by NodeJS, a module installed by npm, or simply user-defined module. Therefore, we can also use relative notation like this to import module:

var module = require("/absolute/path/to/module_name");
var module2 = require("./relative/path/to/module_name");

Where the first declaration will fetch module using absolute filepath and the second one will use path relative to current directory.

The fetched object is treated like other object. It has a name (variable name) and allocated in memory.

Modules are loaded only once per process. If you have several require calls to the same modules, Node caches the require calls if it resolves to the same files.

How Node Resolves Module Path

Core Modules

There are list of core modules, which Node includes in the distribution binary. It is called standard API. When we require this module, Node will just returns that module and the require() ends.

Modules with Path (Absolute or Relative)

If the module path is supplied (absolute or relative), node tries to load the modules as a file. If it does not succeed, it tries to load the module as a directory.

When loading module as a file, if a file exists, Node just loads it as Javascript text. If not, it tries doing the same by appending “.js” extension to the given path. Again, if this is not succeed, Node tries to append “.node” extension and load it as a binary add-on.

When loading module as a directory, it will do some procedures. If appending “/package.json” is a file, Node will try loading the package definition and look for a “main” field. It then try to load it as a file. If unsuccessful, Node will try to load it by appending “/index” to it.

Installed Module

Third party module is installed using NPM.

If the module path does not begin with “.” or “/” or if loading it with complete or relative paths does not work, Node tries to load the module as a module that was previously installed. It adds “/node_modules” to the current directory and tries to load the module from there. If it does not succeed it tries adding “/node_modules” to the parent directory and load the modules from there. This will be repeated until the module is found or nothing found after the root directory.

This means we can put our Node modules to our app directory and Node will find those.

Understanding Node Event Loop

December 9, 2015 | Article | No Comments

Event I/O programming using node js is simple. Node has pulled speed and scalability on the fingertips of the common programmers.

But the event loop approach comes with a price, even though you are not aware of it. Here we will discuss about how node works, technically the node event loop and what we do’s and don’ts on top of it.

Event-Queue Processing Loop

Concurrent is true nature of Node and it is achieved by event loop.

Event loop can be thought as a loop that processes an event queue. Interesting events happen and go in a queue, waiting for their turn. Then, there is an event loop popping out these events, one by one, and invoking the associated callback functions, one at a time. The event loop pops one event out of the queue and invokes the associated callback. When the callback returns the event loop pops the next event and invokes the associated callback function. When the event queue is empty, the event loop waits for new events if there are some pending calls or servers listening, or just quits of there are none.

For example, we will create a new file as hello.js and write these:

setTimeout(function() {
    console.log('World!');
}, 2000);
console.log('Hello');

run it using the node command line tool.

node hello.js

You should see the word “Hello” written out first and then the word “World!” will come out 2 seconds later. If we see the code, the word “World!” appears first but it is not the case when we execute the code. Remember that Node is using event loop. When we declare anonymous function that print out “World!”, this function is not executed yet. The function is passed in as the first argument to a setTimeout() call, which schedules this function to run in 2000 miliseconds (2 seconds) later. Then, next statement is executed which will prints out “Hello”. Two seconds later, the setTimeout is execute as the timeout event is occurred. This means the word “World!” is printed out.

So, the first argument to the setTimeout call is a function we call as a “callback”. It’s a function which will be called later, when the event we set out to listen to occurs.

After our callback is invoked, Node understands that there is nothing more to do and exits.

Callbacks that Generate Events

In above example, we only use one-time execution. We can keep Node busy and keep on scheduling callbacks by using this patterns:

(function schedule() {
  setTimeout(function() {
    console.log('HelloWorld!');
    schedule();
  }, 10000);
})();

If you look it, it is similar to recursive function, but it is not.

We trap the whole thing inside a function named “schedule” and invoking it immediately after declaration. This function will schedule a callback to execute in 1 seconds. This callback, when invoked, will print “Hello World!” and then run schedule again.

On every callback, we registering a new one to be invoked one second later, never letting Node finish.

Not Blocking

The primary concern and main use case for an event loop is to create highly scalable servers. Since an event loop runs in a single thread, it only processes the next event when the callback finishes. If you could see the call stack of a busy Node application you would see it going up and down really fast, invoking callbacks and piling up the next event in line. But for this to work well you have to clear the event loop as fast as you can.

There are two main categories of things that can block the event loop: synchronous I/O and big loops.

Node API is not all asynchronous. Some parts of it are synchronous like, for instance, some file operations. Don’t worry, they are very well marked: they always terminate in “Sync” – like fs.readFileSync – , and they should not be used, or used only when initializing. On a working server you should never use a blocking I/O function inside a callback, since you’re blocking the event loop and preventing other callbacks – probably belonging to other client connections – from being served.

The second category of blocking scenario is when a giant loop is performed. A giant loop in this term is a loop which take a lot of time like iterating over thousands of objects or doing complex time-taking operations in memory.

Let see an example:

var open = false;

setTimeout(function() {
    open = true;
}, 1000);

while(!open) {
    // wait
}

console.log('opened!');

You would expect the code to work, seeing the setTimeout() will surely be executed on next 1 second. However, this never happens as the node will never execute the timeout callback because the vent loop is stuck on the while loop, never giving the timeout event.

Introduction to NPM (Node Package Manager)

December 9, 2015 | Article | 1 Comment

NPM (Node Package Manager) or simply npm is a simple application to install node programs. NPM also support a feature which will indicate dependencies and package starting file.

In this article, we will discuss about how to use npm, from a basic command to a rather advance one.

Getting Help

npm has a lot of feature, therefore let’s start by knowing how to get a help. NPM has a lot of help documentation about all of its command. Getting a help is as simple as typing this command on your terminal:

npm help

If you have a problem with any of npm command/feature, you can append –help on the command, leading to a documentation specific to that command. For example, both are valid command:

npm help install
npm install --help

Filtering Packages

npm ls [filter]

NPM can see the list of all packages and their version (npm ls with no filter), or filter by a tag (npm filter tag). For examples:

Installed Package

npm ls installed

This will list all installed package.

Stable Package

npm ls stable

This will list all stable package available (installed or not).

Search by Content

npm ls name

This way, we can have all packages that have “name” inside its name or tags.

Search by Version

npm ls @1.0

This is used to query packages specified by its version.

Install a Package

NPM is used to install,update, and remove a package. That’s what a package managers do.

When we want to install something, we can use following command and let NPM install the latest version of it to our machine. Here we will have “pkg_name” installed on our system.

npm install pkg_name

You can also install from a tarball, a folder, or an url to a tarball. If you run npm install without arguments, it tries to install the current folder. For example, we will install express:

npm install express

Or we can also install a specific version of a package by:

npm install [email protected]

We can also install the latest of a package within a version range we specify. For instance:

npm install [email protected]">=0.1.0"

More filters can be used to select a specific packages.

Update Packages

npm update [pkg_name]

The update command does a few things:

  1. Search the registry for new versions of all the packages installed (if pkg_name is omitted) or specific packages.
  2. If there’s a newer version, then install it
  3. Point dependent packages at the new version, if it satisfies their dependency
  4. Remove old versions, if no other package names them as a dependency.

Basically, update behaves a lot like a “standard” package manager’s update command, except that it also checks to make sure that the new version isn’t going to break anything before it points stuff at it.

Remove Packages

This command will uninstall a package. If the version are omitted, then all the found version are removed.

npm rm pkg_name[@version] [pkg_name2[@version]]

Read Package Information

To view all of a package information, we can do like this:

npm view pkg_name[@version]

Social media & sharing icons powered by UltimatelySocial