Efficient Data Handling with Node.js Streams: Writeable Streams

Code examples for utilizing NodeJS Streams can be found in my GitHub repository.

What are Writeable Streams? Link to heading

In Node.js, a Writable Stream is an abstraction for a destination to which data can be written. They are instances of the stream.Writable class, an extension of the EventEmitter class, and emit events that can be listened to.

Writable Streams excel in efficiently handling large amounts of data, as they allow data to be written piece by piece, rather than buffering the entire data set into memory before writing.

Implementing a Writable Stream Link to heading

To implement a custom Writable Stream, you also have two options:

Option 1: Extending the stream.Writable class: Link to heading

const { Writable } = require('stream');

class MyWritableStream extends Writable {
  _write(chunk, encoding, callback) {
    process.stdout.write(`Writing: ${chunk.toString()}\n`);
    callback();
  }
}

const myStream = new MyWritableStream();
myStream.write('Hello!\n');
myStream.write('Goodbye!\n');

Option 2: Using the alternative method approach: Link to heading

const { Writable } = require('stream');

const myStream = new Writable({
  write(chunk, encoding, callback) {
    process.stdout.write(`Writing: ${chunk.toString()}\n`);
    callback();
  }
});

myStream.write('Hello!\n');
myStream.write('Goodbye!\n');

In both approaches, the custom Writable Stream writes each received chunk of data to the standard output, prepended with “Writing: “.

Common Writable Streams Link to heading

Both the class extension approach and the alternative method approach achieve the same goal of implementing a custom Readable or Writable Stream. You can choose the approach that best fits your coding style and preference.

Creating a Writable Stream in Node.js is straightforward. There are numerous built-in methods that return a Writable Stream, each designed to handle a different type of data destination. Here are some examples, stemming off of the same examples we gave in the Readable post:

File System Streams (fs.createWriteStream()): These streams are used for writing data to files. They allow you to write large amounts of data piece by piece, reducing memory consumption and improving performance.

const fs = require('fs');
const writeStream = fs.createWriteStream('output.txt');
writeStream.write('Hello, World!');
writeStream.end();

HTTP Response Streams (http.ServerResponse): In an HTTP server, each response is a writable stream. You can write the response body as a stream of data, which is particularly useful for serving large files.

const http = require('http');

const server = http.createServer((req, res) => {
    res.write('Hello, World!');
    res.end();
});

Zlib Streams (zlib.createGzip(), zlib.createDeflate()): These streams are used for compressing data. You can pipe a file stream or any other readable stream into a zlib stream to compress data on the fly.

const fs = require('fs');
const zlib = require('zlib');

const readStream = fs.createReadStream('input.txt');
const gzip = zlib.createGzip();
const writeStream = fs.createWriteStream('input.txt.gz');
readStream.pipe(gzip).pipe(writeStream);

Crypto Streams (crypto.createCipheriv()): These streams are used for encrypting data. You can pipe a file stream or any other readable stream into a crypto stream to encrypt data on the fly.

const fs = require('fs');
const crypto = require('crypto');

const algorithm = 'aes-256-cbc';
const key = crypto.randomBytes(32);
const iv = crypto.randomBytes(16);

const readStream = fs.createReadStream('input.txt');
const cipher = crypto.createCipheriv(algorithm, key, iv);
const writeStream = fs.createWriteStream('encrypted.txt');
readStream.pipe(cipher).pipe(writeStream);

In each of these examples, we create a Writable Stream and write data to it. When we’re done writing data, we call end() to signal that no more data will be written to the stream.

That’s it for this post! In the next post, we’ll take a look at Duplex Streams.