Learning Node.js - File I/O and Basic Streams

#Introduction

In episode 3 we read files with the Sync versions, which block the main thread. For a server application, that's not an option. Node.js provides two proper approaches: the Promise API from fs/promises for regular file operations, and streams for large files or continuously flowing data.

Episode 5 covers both. You'll see fs/promises with clean async/await, the memory-efficient stream concept, pipeline for chaining streams, and readline for processing log files line by line — a very common real-world pattern.

#File I/O with fs/promises

#Clean Async/Await

The node:fs/promises module provides all fs functions as Promises, without the Sync suffix and without nested callbacks:

Read and write with fs/promises

import { readFile, writeFile, mkdir } from "node:fs/promises";
 
await mkdir("data", { recursive: true });
await writeFile("data/catatan.txt", "Belajar stream");
const isi = await readFile("data/catatan.txt", "utf8");
console.log(isi);

writeFile and readFile return Promises, so they can be used with await. Because ESM supports top-level await (episode 4), the example above can run directly. The recursive: true option on mkdir creates the folder without erroring if it already exists.

#Handling Errors with Try/Catch

Unlike callbacks, which use the error-first pattern, the Promise version uses a standard try/catch:

Try/catch for file I/O

import { readFile } from "node:fs/promises";
 
try {
  const isi = await readFile("tidak-ada.txt", "utf8");
  console.log(isi);
} catch (err) {
  console.error("Gagal membaca:", err.message);
}

await readFile("tidak-ada.txt", "utf8") throws an ENOENT error when the file isn't found, and the catch block catches it. This pattern is far easier to read than nested callbacks — we'll discuss errors in more detail in episode 6.

#The Basics of Streams

#Why Streams Matter

Reading a 2 gigabyte file with readFile will put its entire contents in memory — an OOM risk. Streams solve this: data is read in small pieces (chunks), processed, then released. Memory stays low no matter the file size.

Streams in Node.js come in four types:

• Readable: the data source, e.g. createReadStream.
• Writable: the data destination, e.g. createWriteStream.
• Duplex: two-way, e.g. a TCP socket.
• Transform: processes data in the middle, e.g. zlib.createGzip.

#Copying Files with Pipeline

pipeline from node:stream/promises chains streams while correctly handling errors and closure:

Copy a file via stream

import { createReadStream, createWriteStream } from "node:fs";
import { pipeline } from "node:stream/promises";
 
await pipeline(
  createReadStream("akses.log"),
  createWriteStream("salinan.log"),
);
console.log("Selesai streaming");

pipeline(createReadStream("akses.log"), createWriteStream("salinan.log")) flows data from source to destination in small chunks. Use pipeline rather than the .pipe() method directly — pipeline ensures the flow is cleaned up and errors are propagated correctly.

#Processing Data with Transform and Readline

#Chaining a Transform Stream

To process each chunk in the middle of the flow, insert a Transform stream:

Stream with transform

import { createReadStream, createWriteStream } from "node:fs";
import { Transform } from "node:stream";
import { pipeline } from "node:stream/promises";
 
const uppercase = new Transform({
  transform(chunk, enc, callback) {
    callback(null, chunk.toString().toUpperCase());
  },
});
 
await pipeline(
  createReadStream("data.txt"),
  uppercase,
  createWriteStream("hasil.txt"),
);

The Transform object above turns every chunk into uppercase before passing it to the destination. Many transforms can be inserted between the source and destination streams — this is the foundation of compression, encryption, and log parsing pipelines.

#Readline for Line-by-Line Processing

When you need to process a file line by line — such as logs or CSV — node:readline is the right choice:

Process a file line by line

import { createReadStream } from "node:fs";
import { createInterface } from "node:readline";
 
const rl = createInterface({
  input: createReadStream("data.txt"),
});
 
for await (const baris of rl) {
  console.log("Baris:", baris);
}

createInterface({ input: createReadStream("data.txt") }) wraps the file stream into a line interface, and the for await loop processes each line one by one. This pattern is widely used for log aggregation and bulk data import.

#When to Choose readFile vs Stream

#A Practical Guide

The rule of choice is quite simple:

• Small files read once: use readFile from fs/promises.
• Large files or data of unknown size: use streams.
• Files processed line by line: use readline.
• Processing each chunk: insert a Transform in pipeline.

In the real world, log servers, file uploaders, and media streaming almost always use streams. Knowing when to switch from readFile to streams is one of the things that separates beginner code from production code.

#Closing

Here's what to take away:

• fs/promises provides file I/O with clean async/await.
• Streams process data in small chunks so memory stays low.
• pipeline chains streams while handling errors.
• Transform modifies data in the middle of the flow.
• readline processes files line by line with a for await loop.
• Small files use readFile; large files and flowing data use streams.

In the next episode, episode 6, we'll discuss error handling and debugging in Node — try/catch patterns, error-first callbacks, error events, uncaught exceptions, and debugging with the inspector and DevTools. These are skills that save you every day as a backend developer.