iwantcoding.com
🔥 Daily 👥 Rooms 🏆 Top Log in Sign up

Buffer

Buffer is Node’s raw binary container — fixed-size, byte-addressable. Streams produce Buffers; crypto consumes them; HTTP bodies arrive as them. Convert to/from strings carefully (encoding!).

Allocate, slice, encode, hex

EXAMPLE
import { Buffer } from 'node:buffer';

// 1) Create
const a = Buffer.alloc(8);                       // zeroed, length 8
const b = Buffer.allocUnsafe(8);                  // FAST but contents uninitialised
const c = Buffer.from('hello');                   // from a string (utf8)
const d = Buffer.from('aGVsbG8=', 'base64');     // from base64
const e = Buffer.from([0x48, 0x69]);              // from an array of bytes
const f = Buffer.from(new Uint8Array([0x68, 0x69])); // from a typed array

console.log(a.length);                            // 8
console.log(c.toString('utf8'));                  // 'hello'
console.log(c.toString('hex'));                   // '68656c6c6f'
console.log(c.toString('base64'));                // 'aGVsbG8='

// 2) Indexing (UInt8)
c[0] = 0x48;
console.log(c[0]);                                // 72

// 3) Read / write typed values
const buf = Buffer.alloc(8);
buf.writeUInt32BE(42, 0);                         // bytes 0-3: 32-bit big-endian
buf.writeFloatLE(3.14, 4);                        // bytes 4-7: float little-endian
const n = buf.readUInt32BE(0);                    // 42
const f1 = buf.readFloatLE(4);                    // 3.14

// 4) Concatenate (zero-copy)
const combined = Buffer.concat([a, b, c]);

// 5) Slice — view into the same memory
const view = c.subarray(0, 3);                    // 'hel' — shares memory!
view[0] = 0x4a;                                   // mutates `c` too: now 'Jello'

// 6) Copy — independent
const copy = Buffer.alloc(c.length);
c.copy(copy);

// 7) Compare + equal
Buffer.compare(c, copy);                          // 0 if equal
c.equals(copy);                                   // true

// 8) Search
c.indexOf('llo');                                 // 2
c.includes('ell');                                // true

// 9) Hex + base64 — common conversions
const hex = c.toString('hex');
const back = Buffer.from(hex, 'hex');
const b64 = c.toString('base64');
const back2 = Buffer.from(b64, 'base64');

// base64url for URL-safe (e.g. JWTs)
const safe = c.toString('base64url');
const back3 = Buffer.from(safe, 'base64url');

// 10) UTF-8 caveats
const emoji = Buffer.from('hi 🎉');
emoji.length;                                     // 8 (NOT 4 — emoji = 4 UTF-8 bytes)
emoji.toString();                                 // 'hi 🎉'

// Slicing in the middle of a multi-byte char produces invalid UTF-8
const broken = emoji.subarray(0, 5);              // cuts emoji in half
broken.toString('utf8');                          // 'hi \ufffd' (replacement char)

// Decode safely across chunks (e.g. streams)
import { StringDecoder } from 'node:string_decoder';
const decoder = new StringDecoder('utf8');
decoder.write(emoji.subarray(0, 4));              // 'hi '
decoder.write(emoji.subarray(4));                  // '🎉'

// 11) Real use cases

// Reading binary files
import { readFile } from 'node:fs/promises';
const png = await readFile('image.png');           // Buffer
const header = png.subarray(0, 8).toString('hex');
// 89504e470d0a1a0a = PNG signature

// HTTP body chunks → Buffer
app.post('/upload', async (req, res) => {
    const chunks = [];
    for await (const c of req) chunks.push(c);
    const body = Buffer.concat(chunks);
    // process binary upload
    res.end();
});

// Crypto wants Buffers
import crypto from 'node:crypto';
const hash = crypto.createHash('sha256').update(png).digest();   // Buffer
console.log(hash.toString('hex'));

// 12) Conversion to/from Uint8Array
const u8 = new Uint8Array(c.buffer, c.byteOffset, c.byteLength);
const back4 = Buffer.from(u8.buffer, u8.byteOffset, u8.byteLength);

// Buffer extends Uint8Array — pass to APIs expecting Uint8Array directly.

// 13) Common bugs
//   • Using subarray expecting a copy → original mutates if you write
//   • Slicing UTF-8 in the middle of a multi-byte char
//   • Confusing length (bytes) with character count
//   • Buffer.allocUnsafe + forgetting to overwrite → leaks recycled memory data
//   • Comparing Buffers with === → reference, not contents (use .equals or .compare)

// 14) Performance
//   • Buffer.allocUnsafe + manual fill > Buffer.alloc when you write the whole thing
//   • Buffer.concat reuses memory; cheaper than Array.join('').toBuffer
//   • Pre-size buffers when length is known
//   • Stream binary data; don't read entire 5GB files into one Buffer

Why it matters

Buffer is binary; String is text with an encoding. Treat them as different things, decode at the boundary (e.g. StringDecoder for streams), and don’t use .length as a character count for anything past ASCII.

Tip: Tweak the snippet with Try it Yourself », then sit the quiz at the bottom of the page.

Example

Example
const buf = Buffer.from('hello', 'utf8');
console.log(buf.toString('hex'));   // 68656c6c6f
Try it Yourself »

Discussion

Loading…