Have you been using Node.js for a while and only until now have you heard about the word pipe? You tried checking the documentation but still can’t figure out what it means or does? In this article, I will clarify those doubts by explaining what .pipe or piping is and how to use it in Node.js. To ensure you understand the article, previous knowledge about streams is strongly recommended.
In this pipe implementation, we are going to create a simple HTTP server that will read data from a file and send the response to the client.
1.
Let’s start by creating the HTTP server using the
http
package that returns some data.
Or another option is to open a new tab http://localhost:8080/. Once you make the request, you should receive “hello!”.
We are going to pause for a second. Let’s recall the anatomy of an HTTP transaction. An HTTP transaction is made of a server, created by the method createServer which in itself is an EventEmitter. When an HTTP request hits the server, node calls the request handler using the req and res objects, which are request and response respectively, for dealing with the transaction.
The req or request object is an instance of the IncomingMessage object. The IncomingMessage object is a child object of a ReadableStream.
The res or response object is an instance of the ServerResponse object. The ServerResponse object is a child object of a WriteableStream.
Therefore, we know we have a writeable and a readable stream.
2. We are going to create a data.txt file in the same directory folder, and save some information. For the sake of making things clear, I will save the following text: “This is data from the data.txt file”.
3. Remove the existing logic from the event handler.
4. We are going to read the content of the data.txt file using the fs package using fs.createReadStream. The fs.createReadStream will return a ReadableStream. We are going to use that ReadableStream to pipe or pass the data from data.txt file to the response object, which is a WriteableStream.
consthttp=require('http');constfs=require('fs');http.createServer(function(req,res){// generete readable stream to read content of data.txtconstreadStream=fs.createReadStream(__dirname+'/data.txt');// pass readable stream data, which are the content of data.txt, to the // response object, which is a writeable streamreadStream.pipe(res);}).listen(8080);http.createServer(function(req,res){constreadStream=fs.createReadStream(__dirname+'/data.txt');readStream.on('data',function(chunk){console.log('this is the data from file',chunk);readStream.pause();console.log('on pause: readable flowing',readStream.readableFlowing);readStream.resume();console.log('on resume: readable flowing',readStream.readableFlowing);res.write('Hello!')res.end();}).listen(8080);
If you run the example above, you will only read data from the data.txt file whenever the readable stream flowing mode is set to true which is enabled by using the resume() method. If the flowing mode is set to false, it will never read the content of the data.txt file.
However, when using the pipe method the flowing mode will automatically be set to true ensuring the data is passed from one stream to another. We can confirm this if we try to pause the flowing mode prior to piping both streams.
consthttp=require('http');constfs=require('fs');http.createServer(function(req,res){// generete readable stream to read content of data.txtconstreadStream=fs.createReadStream(__dirname+'/data.txt');readStream.on('data',function(chunk){console.log('this is the data from file',chunk);readStream.pause();console.log('on pause: readable flowing',readStream.readableFlowing);readStream.pipe(res);}).listen(8080);
If you have never heard of the word “pipe” or “piping” when working with streams, there is a slight chance you could find the wrong information if you go to Node.js documentation and start finding for the word “pipe”. When you do a quick search, you will find two options.
If you find the first option, you will notice it is an event listener that writeable streams can set when a readable stream uses the pipe method to pass the data from one stream to another. The event pipe is only available on writeable streams. We are going to use our simple server API to demonstrate the event pipes.
consthttp=require('http');constfs=require('fs');http.createServer(function(req,res){constreadStream=fs.createReadStream(__dirname+'/data.txt');// setting pipe event listener before triggering the pipe method in the readable stream// otherwise, the pipe event listener won't be triggered if set after triggering the pipe methodres.on('pipe',function(src){console.log('Triggered the pipe event listener whenever a source readable stream pipes the writeable stream');readStream.pipe(res);}).listen(8080);
In other words, calling the pipe method on the readable stream causes the pipe event listener to be triggered on the writeable stream.
It is important to mention to define the pipe event listener prior to calling the pipe method from the readable stream. Attempting to call the pipe method prior to setting the event listener in the writeable stream won’t work.
Streams are one of the most powerful and fundamental concepts of Node.js applications. They allow us to handle data in a more efficient way as pieces of data can be transported in smaller chunks preventing you from running out of memory and maintaining good performance in your applications.
Therefore, using the pipe method is an effective and easy solution to push data between streams. In that way, we avoid storing too much data that needs to be manipulated or modified all at the same time. Also, the code will be shorter, elegant, and easy to follow.
Once unpublished, all posts by arealesramirez will become hidden and only accessible to themselves.
If arealesramirez is not suspended, they can still re-publish their posts from their dashboard.
