Skip to content

Core 01 fs

Core 01 - File System (fs)

Basic Idea

Everything in Node touches the filesystem at some point config files , logs , uploads , databases - they all live on disk The fs module is your gate to all of it

fs vs fs.promises

Node gives you three ways to use fs

const fs = require('fs') // callbacks, old school
const fsPromises = require('fs').promises // promises, modern
// or
const fs = require('fs/promises') // node 14+ direct import

Callbacks feel like 2012 but they're still everywhere in legacy code Promises are what you should write today

// callback style - wtf pyramid
fs.readFile('/etc/passwd', (err, data) => {
  if (err) throw err
  console.log(data.toString())
})

// promise style - clean
async function readConfig() {
  try {
    const data = await fs.readFile('/etc/config.json', 'utf-8')
    return JSON.parse(data)
  } catch (err) {
    console.error('config exploded:', err.message)
  }
}

Reading Files

Three ways to read , each for different scenarios

const fs = require('fs/promises')
const { createReadStream } = require('fs')

// whole file in memory - fine for configs , bad for 4K videos
const small = await fs.readFile('config.json', 'utf-8')

// stream it for big stuff - memory stays happy
const stream = createReadStream('server.log', { highWaterMark: 64 * 1024 })
stream.on('data', chunk => {
  process.stdout.write(chunk.length + ' bytes\n')
})
stream.on('end', () => console.log('done reading'))

// sync version - blocks everything , use at startup only
const data = fs.readFileSync('/etc/hosts', 'utf-8')

readFile loads everything into RAM For files bigger than 100MB , use createReadStream or your process will OOM and die

Writing Files

// async write - overwrites by default
await fs.writeFile('output.txt', 'hello from the other side')

// append - adds to end
await fs.appendFile('access.log', `${new Date().toISOString()} request\n`)

// streaming write
const { createWriteStream } = require('fs')
const ws = createWriteStream('large-output.bin')
for (let i = 0; i < 10000; i++) {
  ws.write(Buffer.alloc(1024, i % 256))
}
ws.end()

writeFile with a large buffer will spike memory If you're writing 500MB of data , stream it or watch your RSS climb like a rocket

Directory and File Operations

// create directories - recursive creates parents too
await fs.mkdir('logs/2025/01', { recursive: true })

// read directory contents
const files = await fs.readdir('./uploads')
console.log('files:', files)

// delete stuff - recursive for dirs
await fs.rm('./temp', { recursive: true, force: true })

// file metadata
const stats = await fs.stat('config.json')
console.log({
  size: stats.size,
  isFile: stats.isFile(),
  isDirectory: stats.isDirectory(),
  created: stats.birthtime,
  modified: stats.mtime
})

File Descriptors

// low-level fd operations
const fd = await fs.open('data.bin', 'r+')
const buf = Buffer.alloc(1024)
const { bytesRead } = await fd.read(buf, 0, 1024, 0)
console.log('read', bytesRead, 'bytes')
await fd.close() // don't forget this or you'll leak fds

Node manages most fds through the higher-level APIs But if you open raw fds and don't close them , you'll hit EMFILE (too many open files) real quick

Watching Files

// watch for changes - fs.watch is preferred
const watcher = fs.watch('./config', (eventType, filename) => {
  console.log(`${filename} was ${eventType}d - reload config`)
})

// older API , not recommended
fs.watchFile('config.json', { interval: 500 }, (curr, prev) => {
  console.log('file changed at', curr.mtime)
})

fs.watch is more efficient - uses OS notifications (inotify on Linux , FSEvents on macOS) fs.watchFile polls every interval ms , wastes CPU cycles

// DANGER - path traversal
app.get('/files/:name', (req, res) => {
  const data = fs.readFileSync(`/var/data/files/${req.params.name}`)
  // what if req.params.name is '../../etc/passwd'?
})

// safe - resolve and check prefix
const path = require('path')
const safeDir = '/var/data/files/'
app.get('/files/:name', (req, res) => {
  const requested = path.resolve(safeDir, req.params.name)
  if (!requested.startsWith(safeDir)) {
    return res.status(403).send('nope')
  }
  // now it's safe
})

Path traversal is still in the OWASP Top 10 Always canonicalize paths and verify they stay within your intended directory Symlink attacks: if an attacker can create symlinks in the target directory , they can redirect your write operation to /etc/cron.d/ and own your box

// prevent symlink attacks - check realpath first
async function safeWrite(dir, filename, content) {
  const target = path.resolve(dir, filename)
  const dirReal = await fs.realpath(dir)
  const targetReal = await fs.realpath(path.dirname(target))
  if (!targetReal.startsWith(dirReal)) {
    throw new Error('path escape attempted')
  }
  await fs.writeFile(target, content)
}

Summary

  • Use fs/promises for modern code , callbacks only in maintenance hell
  • Stream big files , buffer small ones
  • Always validate and canonicalize user-supplied paths
  • Close your file descriptors - you're not in C land but fds still leak

Prerequisites

next -> core_02_path.md