Skip to content

Worker Threads

Child processes give you isolation but each one spins up a whole V8 instance That's expensive - hundreds of MB per process , seconds to start Worker threads run inside the same process , share the same V8 heap , and start in milliseconds They're built for CPU work that would paralyze the Event Loop

When to Use Worker Threads

Worker threads are for CPU-intensive operations - NOT for I/O

// GOOD use case - CPU-bound work
// Image processing, data transformation, crypto, compression

// BAD use case - I/O work (use async I/O instead)
// File reads, network requests, database queries

Node's I/O is already non-blocking via the Event Loop and libuv's thread pool Throwing a worker at a file read wastes memory and complexity - the Event Loop handles it fine Reach for workers when you have heavy synchronous computation that blocks everything else

worker_threads Module

const { Worker, parentPort, workerData, isMainThread } = require('worker_threads')

Worker - creates a new worker thread running a script parentPort - message port on the worker side to communicate with parent workerData - data passed from parent to worker at creation (structured clone) isMainThread - boolean to check if code runs in main or worker

// main.js
const { Worker } = require('worker_threads')
const path = require('path')

function runInWorker(workerFile, data) {
  return new Promise((resolve, reject) => {
    const worker = new Worker(path.resolve(__dirname, workerFile), {
      workerData: data
    })

    worker.on('message', resolve)
    worker.on('error', reject)
    worker.on('exit', (code) => {
      if (code !== 0) {
        reject(new Error(`Worker stopped with exit code ${code}`))
      }
    })
  })
}

// Usage
const result = await runInWorker('./fib-worker.js', { n: 45 })
console.log('fib result:', result)

// fib-worker.js - runs in the worker thread
const { parentPort, workerData } = require('worker_threads')

function fib(n) {
  if (n <= 1) return n
  return fib(n - 1) + fib(n - 2)
}

const result = fib(workerData.n)
parentPort.postMessage(result)

Communication - Messages and Transferables

Messages between threads use structured clone algorithm - same as postMessage in browsers No prototype chains , no functions , no Symbols - just plain data

// main.js
const { Worker } = require('worker_threads')

const worker = new Worker('./processor.js', { workerData: { batchSize: 1000 } })

worker.on('message', (msg) => {
  if (msg.type === 'progress') {
    console.log(`Progress: ${msg.percent}%`)
  } else if (msg.type === 'done') {
    console.log('Result:', msg.data)
    worker.terminate()
  }
})

worker.postMessage({ cmd: 'start', input: largeArray })

// processor.js
const { parentPort, workerData } = require('worker_threads')

parentPort.on('message', async (msg) => {
  if (msg.cmd === 'start') {
    const total = msg.input.length
    let processed = 0

    for (const item of msg.input) {
      // do CPU work
      processed++
      if (processed % 100 === 0) {
        parentPort.postMessage({
          type: 'progress',
          percent: Math.round((processed / total) * 100)
        })
      }
    }

    parentPort.postMessage({ type: 'done', data: 'all processed' })
  }
})

Transferables - move ArrayBuffer ownership to another thread without copying Useful for large binary data (images , video frames)

// Transfer large buffer without copying
const sharedBuffer = new SharedArrayBuffer(1024 * 1024 * 100) // 100MB
worker.postMessage({ buffer: sharedBuffer }, [sharedBuffer])
// After transfer, main thread can't access sharedBuffer anymore

SharedArrayBuffer - Shared Memory

const { Worker } = require('worker_threads')

// Create shared buffer
const sharedBuffer = new SharedArrayBuffer(4) // 4 bytes - one Int32
const sharedArray = new Int32Array(sharedBuffer)

const worker = new Worker('./counter-worker.js')
worker.postMessage({ shared: sharedBuffer })

// Wait for worker to finish
Atomics.wait(sharedArray, 0, 0)
console.log('Final count:', sharedArray[0])

// counter-worker.js
const { parentPort, workerData } = require('worker_threads')

parentPort.on('message', (msg) => {
  const array = new Int32Array(msg.shared)

  for (let i = 0; i < 1000000; i++) {
    Atomics.add(array, 0, 1)
  }

  Atomics.store(array, 0, array[0])
  Atomics.notify(array, 0)
  parentPort.postMessage('done')
})

SharedArrayBuffer lets threads share memory without copying BUT you need Atomics for synchronization Data races corrupt your state faster than you can debug

Thread Pool Sizing

const os = require('os')
const { Worker } = require('worker_threads')

const POOL_SIZE = os.cpus().length

class WorkerPool {
  constructor(workerFile, poolSize = os.cpus().length) {
    this.workers = []
    this.queue = []
    this.active = 0

    for (let i = 0; i < poolSize; i++) {
      const worker = new Worker(workerFile)
      worker.on('message', () => this._onComplete(worker))
      worker.on('error', () => this._onComplete(worker))
      this.workers.push(worker)
    }
  }

  exec(data) {
    return new Promise((resolve, reject) => {
      this.queue.push({ data, resolve, reject })
      this._processQueue()
    })
  }

  _processQueue() {
    if (this.queue.length === 0) return
    if (this.active >= this.workers.length) return

    const worker = this.workers[this.active]
    const task = this.queue.shift()
    this.active++
    worker.postMessage(task.data)
    worker.once('message', (result) => {
      task.resolve(result)
    })
  }

  _onComplete(worker) {
    this.active--
    this._processQueue()
  }
}

os.cpus().length is the magic number - matches your CPU core count More workers than cores causes context switching overhead , not parallelism For I/O-heavy work you don't need workers (libuv's thread pool handles it)

worker_threads vs cluster vs child_process

Feature worker_threads cluster child_process
Memory Shared heap Separate Separate
Start time ~5ms ~50ms ~100ms+
IPC Structured clone Built-in round-robin stdio / message
Use case CPU work HTTP load balancing Isolated tasks
Crash isolation No (same process) Per-worker Per-process
Number of instances Many (lightweight) Few (heavy) Few (heavy)

worker_threads for CPU parallelism inside your process cluster for scaling HTTP servers across CPU cores child_process for running non-Node binaries or total isolation

Security - Thread Safety and Isolation

// DANGER - shared state without synchronization
const shared = new SharedArrayBuffer(4)
const view = new Int32Array(shared)

// Thread 1
view[0] = view[0] + 1  // read-modify-write - not atomic!

// Thread 2
view[0] = view[0] + 1  // both threads read same value - one increment lost

Race conditions with SharedArrayBuffer corrupt data silently Always use Atomics operations for shared memory access

// SAFE - atomic operations
Atomics.add(view, 0, 1)   // thread-safe increment
Atomics.load(view, 0)     // thread-safe read
Atomics.store(view, 0, 5) // thread-safe write

Workers can't be killed cleanly worker.terminate() kills the thread immediately - no cleanup , no finally blocks , no graceful shutdown If the worker held a file handle or database connection , it's gone

// Graceful shutdown pattern
worker.on('message', (msg) => {
  if (msg === 'shutdown') {
    worker.postMessage('cleanup')
    setTimeout(() => worker.terminate(), 5000) // force kill after timeout
  }
})

Workers share process memory space A segfault in one worker crashes the entire Node process Native addons in workers can corrupt shared heap Treat worker code with the same security scrutiny as main thread code

Prerequisites

next -> adv_03_native_addons.md