Node.js is designed to handle large amounts of data efficiently, making it an ideal choice for working with streams. Streams allow you to process data piece by piece instead of loading the entire content into memory. This is particularly useful for handling large files or real-time data processing. In this blog, we will explore how streams work in Node.js with practical examples of readable and writable streams.

What Are Streams in Node.js?

Streams in Node.js are objects that let you read or write data in chunks instead of loading everything at once. They are based on EventEmitter and can handle data efficiently in a non-blocking way. There are four types of streams in Node.js:

Readable Streams – Used for reading data.

Writable Streams – Used for writing data.

Duplex Streams – Used for both reading and writing.

Transform Streams – Used for modifying or transforming data during the read/write process.

Readable Streams: Handling Large Files

A readable stream is used to read data from a source, such as a file, and send it in chunks. This is particularly useful when dealing with large files that cannot fit into memory all at once.

Example: Reading a Large File

const fs = require(‘fs’);

const readableStream = fs.createReadStream(‘largeFile.txt’, ‘utf8’);

readableStream.on(‘data’, (chunk) => {

console.log(‘Received chunk:’, chunk);

});

readableStream.on(‘end’, () => {

console.log(‘Finished reading file’);

});

readableStream.on(‘error’, (err) => {

console.error(‘Error reading file:’, err);

});

Explanation:

fs.createReadStream() creates a stream to read a file.

The data event fires whenever a chunk of data is available.

The end event signals that the entire file has been read.

The error event handles any errors that occur during reading.

Writable Streams: Writing Large Files

A writable stream allows you to write data in chunks instead of writing everything at once. This is useful when processing large amounts of data.

Example: Writing Data to a File

const fs = require(‘fs’);

const writableStream = fs.createWriteStream(‘output.txt’);

writableStream.write(‘Hello, this is the first line!\n’);

writableStream.write(‘Here is another line of text.\n’);

writableStream.end(() => {

console.log(‘Finished writing to file’);

});

Explanation:

fs.createWriteStream() creates a writable stream.

.write() method writes data to the file in chunks.

.end() finalizes the stream and closes the file.

Piping Streams: Efficient Data Transfer

One of the best features of streams is piping, which allows direct data transfer between readable and writable streams without manually managing the data flow.

Example: Copying a Large File

const fs = require(‘fs’);

const readableStream = fs.createReadStream(‘source.txt’);

const writableStream = fs.createWriteStream(‘destination.txt’);

readableStream.pipe(writableStream);

writableStream.on(‘finish’, () => {

console.log(‘File copied successfully’);

});

Explanation:

.pipe() connects the readable stream to the writable stream.

The file content is transferred in chunks, reducing memory usage.

Handling Real-Time Data with Streams

Streams are also useful for real-time data processing, such as handling HTTP requests.

Example: Streaming a File in an HTTP Server

const http = require(‘http’);

const fs = require(‘fs’);

const server = http.createServer((req, res) => {

const readableStream = fs.createReadStream(‘largeFile.txt’);

res.writeHead(200, { ‘Content-Type’: ‘text/plain’ });

readableStream.pipe(res);

});

server.listen(3000, () => {

console.log(‘Server is running on port 3000’);

});

Explanation:

When a user makes an HTTP request, the server streams a file instead of sending it all at once.

The .pipe(res) sends the file content chunk by chunk, improving performance.

Conclusion

Streams in Node.js provide a powerful way to handle large files and real-time data efficiently. By using readable and writable streams, along with piping, you can significantly improve performance and reduce memory consumption. Whether you’re working with files or handling HTTP requests, leveraging streams ensures smooth and efficient data processing.

If you are looking for any services related to Website Development, App Development, Digital Marketing and SEO, just email us at nchouksey@manifestinfotech.com or Skype id: live:76bad32bff24d30d

𝐅𝐨𝐥𝐥𝐨𝐰 𝐔𝐬:

𝐋𝐢𝐧𝐤𝐞𝐝𝐢𝐧: linkedin.com/company/manifestinfotech

𝐅𝐚𝐜𝐞𝐛𝐨𝐨𝐤: facebook.com/manifestinfotech/

𝐈𝐧𝐬𝐭𝐚𝐠𝐫𝐚𝐦: instagram.com/manifestinfotech/

𝐓𝐰𝐢𝐭𝐭𝐞𝐫: twitter.com/Manifest_info

#NodeJS #Streams #JavaScript #BackendDevelopment #WebDevelopment