Async I/O and events · Lesson 8 of 18
Streams and backpressure
Process large data chunk by chunk instead of buffering it all in memory.
readFile must buffer an entire file. A stream moves small chunks, starting work early while keeping memory bounded. Readables produce, Writables consume, Transforms modify, and pipeline connects them with error handling and cleanup.
Readable chunks
Transform
Backpressure
Writable
const { Readable, Transform, Writable } = require("node:stream")
const { pipeline } = require("node:stream/promises")
async function main() {
const out = []
await pipeline(
Readable.from(["sea", "code"]),
new Transform({ transform(chunk, _enc, cb) { cb(null, chunk.toString().toUpperCase()) } }),
new Writable({ write(chunk, _enc, cb) { out.push(chunk.toString()); cb() } })
)
console.log(out.join(" "))
}
main().catch(console.error)Backpressure means a consumer tells a producer to slow down when its buffer fills. pipe and pipeline manage it. If manually writing and write() returns false, wait for drain. For files use createReadStream and createWriteStream.
Quick question
Why is a stream better for a 10GB file?
Exercise
Use Readable.from and consume sea, code, academy with for-await, joining by dashes. Print sea-code-academy.
Expected output: sea-code-academy
const { Readable } = require("node:stream")
async function main() {
const parts = []
const input = Readable.from(["sea", "code", "academy"])
console.log(parts.join("-"))
}
main()Learn more from
Played 0 of 2 games
Sign in to save your progress and earn XP.