Collectives™ on Stack Overflow

Find centralized, trusted content and collaborate around the technologies you use most.

Learn more about Collectives

Teams

Q&A for work

Connect and share knowledge within a single location that is structured and easy to search.

Learn more about Teams a good example would be writing a library that worked with File's in browsers and also for NodeJS files? fbstj Jan 18, 2012 at 18:36 Another reason is that a float takes too many bytes of RAM when stored in an Array . So to store many floats you need Float32Array where it takes 4 bytes. And if you want quick serialization of those floats to a file you need a Buffer , as serializing to JSON takes ages. nponeccop Nov 13, 2013 at 13:32 const file = fs.readFileSync(filePath); , so how do I use this?... 30 minutes later, wow I miss C. William Entriken May 27, 2020 at 4:02

Instances of Buffer are also instances of Uint8Array in node.js 4.x and higher. Thus, the most efficient solution is to access the buffer's own .buffer property directly, as per https://stackoverflow.com/a/31394257/1375574 . The Buffer constructor also takes an ArrayBufferView argument if you need to go the other direction.

Note that this will not create a copy, which means that writes to any ArrayBufferView will write through to the original Buffer instance.

In older versions, node.js has both ArrayBuffer as part of v8, but the Buffer class provides a more flexible API. In order to read or write to an ArrayBuffer, you only need to create a view and copy across.

From Buffer to ArrayBuffer:

function toArrayBuffer(buffer) {
  const arrayBuffer = new ArrayBuffer(buffer.length);
  const view = new Uint8Array(arrayBuffer);
  for (let i = 0; i < buffer.length; ++i) {
    view[i] = buffer[i];
  return arrayBuffer;

From ArrayBuffer to Buffer:

function toBuffer(arrayBuffer) {
  const buffer = Buffer.alloc(arrayBuffer.byteLength);
  const view = new Uint8Array(arrayBuffer);
  for (let i = 0; i < buffer.length; ++i) {
    buffer[i] = view[i];
  return buffer;
                I'd also recommend you to optimize this by copying integers when possible using DataView. Until size&0xfffffffe, copy 32-bit integers, then, if there's 1 byte remaining, copy 8-bit integer, if 2 bytes, copy 16-bit integer, and if 3 bytes, copy 16-bit and 8-bit integer.
– Triang3l
                Feb 13, 2013 at 17:25
                Have tested Buffer -> ArrayBuffer with a module intended for browser use and it is working brilliantly. Thanks!
– pospi
                May 26, 2014 at 2:49
                ‘The slice() method returns a new ArrayBuffer whose contents are a copy of this ArrayBuffer's bytes from begin, inclusive, up to end, exclusive.’ - MDN ArrayBuffer.prototype.slice()
– Константин Ван
                Jan 23, 2018 at 19:01

No dependencies, fastest, Node.js 4.x and later

Buffers are Uint8Arrays, so you just need to slice (copy) its region of the backing ArrayBuffer.

// Original Buffer
let b = Buffer.alloc(512);
// Slice (copy) its segment of the underlying ArrayBuffer
let ab = b.buffer.slice(b.byteOffset, b.byteOffset + b.byteLength);

The slice and offset stuff is required because small Buffers (less than 4 kB by default, half the pool size) can be views on a shared ArrayBuffer. Without slicing, you can end up with an ArrayBuffer containing data from another Buffer. See explanation in the docs.

If you ultimately need a TypedArray, you can create one without copying the data:

// Create a new view of the ArrayBuffer without copying
let ui32 = new Uint32Array(b.buffer, b.byteOffset, b.byteLength / Uint32Array.BYTES_PER_ELEMENT);

No dependencies, moderate speed, any version of Node.js

Use Martin Thomson's answer, which runs in O(n) time. (See also my replies to comments on his answer about non-optimizations. Using a DataView is slow. Even if you need to flip bytes, there are faster ways to do so.)

Dependency, fast, Node.js ≤ 0.12 or iojs 3.x

You can use https://www.npmjs.com/package/memcpy to go in either direction (Buffer to ArrayBuffer and back). It's faster than the other answers posted here and is a well-written library. Node 0.12 through iojs 3.x require ngossen's fork (see this).

Use ngossen's fork: github.com/dcodeIO/node-memcpy/pull/6. See also my new answer if you're using node 4+. – ZachB Dec 2, 2015 at 19:09 @K._ Those properties are inherited from TypedArray: developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/… and developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/… – ZachB Jan 23, 2018 at 20:31 Late addition: @trevnorris says "starting in [V8] 4.3 Buffers are backed by Uint8Array", so possibly this is faster now... – ChrisV Jun 21, 2015 at 20:38 This only works because instances of Buffer are also instances of Uint8Array in Node.js 4.x and higher. For lower Node.js versions you have to implement a toArrayBuffer function. – Benny Code Mar 28, 2017 at 21:36

1. A Buffer is just a view for looking into an ArrayBuffer.

A Buffer, in fact, is a FastBuffer, which extends (inherits from) Uint8Array, which is an octet-unit view (“partial accessor”) of the actual memory, an ArrayBuffer.

  📜/lib/buffer.js#L65-L73 Node.js 9.4.0
class FastBuffer extends Uint8Array {
  constructor(arg1, arg2, arg3) {
    super(arg1, arg2, arg3);
FastBuffer.prototype.constructor = Buffer;
internalBuffer.FastBuffer = FastBuffer;
Buffer.prototype = FastBuffer.prototype;

2. The size of an ArrayBuffer and the size of its view may vary.

Reason #1: Buffer.from(arrayBuffer[, byteOffset[, length]]).

With Buffer.from(arrayBuffer[, byteOffset[, length]]), you can create a Buffer with specifying its underlying ArrayBuffer and the view's position and size.

const test_buffer = Buffer.from(new ArrayBuffer(50), 40, 10);
console.info(test_buffer.buffer.byteLength); // 50; the size of the memory.
console.info(test_buffer.length); // 10; the size of the view.

Reason #2: FastBuffer's memory allocation.

It allocates the memory in two different ways depending on the size.

  • If the size is less than the half of the size of a memory pool and is not 0 (“small”): it makes use of a memory pool to prepare the required memory.
  • Else: it creates a dedicated ArrayBuffer that exactly fits the required memory.
  •   📜/lib/buffer.js#L306-L320 Node.js 9.4.0
    function allocate(size) {
      if (size <= 0) {
        return new FastBuffer();
      if (size < (Buffer.poolSize >>> 1)) {
        if (size > (poolSize - poolOffset))
          createPool();
        var b = new FastBuffer(allocPool, poolOffset, size);
        poolOffset += size;
        alignPool();
        return b;
      } else {
        return createUnsafeBuffer(size);
      📜/lib/buffer.js#L98-L100 Node.js 9.4.0
    
    function createUnsafeBuffer(size) {
      return new FastBuffer(createUnsafeArrayBuffer(size));
    

    What do you mean by a “memory pool?”

    A memory pool is a fixed-size pre-allocated memory block for keeping small-size memory chunks for Buffers. Using it keeps the small-size memory chunks tightly together, so prevents fragmentation caused by separate management (allocation and deallocation) of small-size memory chunks.

    In this case, the memory pools are ArrayBuffers whose size is 8 KiB by default, which is specified in Buffer.poolSize. When it is to provide a small-size memory chunk for a Buffer, it checks if the last memory pool has enough available memory to handle this; if so, it creates a Buffer that “views” the given partial chunk of the memory pool, otherwise, it creates a new memory pool and so on.

    You can access the underlying ArrayBuffer of a Buffer. The Buffer's buffer property (that is, inherited from Uint8Array) holds it. A “small” Buffer's buffer property is an ArrayBuffer that represents the entire memory pool. So in this case, the ArrayBuffer and the Buffer varies in size.

    const zero_sized_buffer = Buffer.allocUnsafe(0);
    const small_buffer = Buffer.from([0xC0, 0xFF, 0xEE]);
    const big_buffer = Buffer.allocUnsafe(Buffer.poolSize >>> 1);
    // A `Buffer`'s `length` property holds the size, in octets, of the view.
    // An `ArrayBuffer`'s `byteLength` property holds the size, in octets, of its data.
    console.info(zero_sized_buffer.length); /// 0; the view's size.
    console.info(zero_sized_buffer.buffer.byteLength); /// 0; the memory..'s size.
    console.info(Buffer.poolSize); /// 8192; a memory pool's size.
    console.info(small_buffer.length); /// 3; the view's size.
    console.info(small_buffer.buffer.byteLength); /// 8192; the memory pool's size.
    console.info(Buffer.poolSize); /// 8192; a memory pool's size.
    console.info(big_buffer.length); /// 4096; the view's size.
    console.info(big_buffer.buffer.byteLength); /// 4096; the memory's size.
    console.info(Buffer.poolSize); /// 8192; a memory pool's size.
    

    3. So we need to extract the memory it “views.”

    An ArrayBuffer is fixed in size, so we need to extract it out by making a copy of the part. To do this, we use Buffer's byteOffset property and length property, which are inherited from Uint8Array, and the ArrayBuffer.prototype.slice method, which makes a copy of a part of an ArrayBuffer. The slice()-ing method herein was inspired by @ZachB.

    const test_buffer = Buffer.from(new ArrayBuffer(10));
    const zero_sized_buffer = Buffer.allocUnsafe(0);
    const small_buffer = Buffer.from([0xC0, 0xFF, 0xEE]);
    const big_buffer = Buffer.allocUnsafe(Buffer.poolSize >>> 1);
    function extract_arraybuffer(buf)
        // You may use the `byteLength` property instead of the `length` one.
        return buf.buffer.slice(buf.byteOffset, buf.byteOffset + buf.length);
    // A copy -
    const test_arraybuffer = extract_arraybuffer(test_buffer); // of the memory.
    const zero_sized_arraybuffer = extract_arraybuffer(zero_sized_buffer); // of the... void.
    const small_arraybuffer = extract_arraybuffer(small_buffer); // of the part of the memory.
    const big_arraybuffer = extract_arraybuffer(big_buffer); // of the memory.
    console.info(test_arraybuffer.byteLength); // 10
    console.info(zero_sized_arraybuffer.byteLength); // 0
    console.info(small_arraybuffer.byteLength); // 3
    console.info(big_arraybuffer.byteLength); // 4096
    

    4. Performance improvement

    If you're to use the results as read-only, or it is okay to modify the input Buffers' contents, you can avoid unnecessary memory copying.

    const test_buffer = Buffer.from(new ArrayBuffer(10));
    const zero_sized_buffer = Buffer.allocUnsafe(0);
    const small_buffer = Buffer.from([0xC0, 0xFF, 0xEE]);
    const big_buffer = Buffer.allocUnsafe(Buffer.poolSize >>> 1);
    function obtain_arraybuffer(buf)
        if(buf.length === buf.buffer.byteLength)
            return buf.buffer;
        } // else:
        // You may use the `byteLength` property instead of the `length` one.
        return buf.subarray(0, buf.length);
    // Its underlying `ArrayBuffer`.
    const test_arraybuffer = obtain_arraybuffer(test_buffer);
    // Just a zero-sized `ArrayBuffer`.
    const zero_sized_arraybuffer = obtain_arraybuffer(zero_sized_buffer);
    // A copy of the part of the memory.
    const small_arraybuffer = obtain_arraybuffer(small_buffer);
    // Its underlying `ArrayBuffer`.
    const big_arraybuffer = obtain_arraybuffer(big_buffer);
    console.info(test_arraybuffer.byteLength); // 10
    console.info(zero_sized_arraybuffer.byteLength); // 0
    console.info(small_arraybuffer.byteLength); // 3
    console.info(big_arraybuffer.byteLength); // 4096
                    This is all well and good... but did you actually answer OP's question? If you did, it's buried...
    – Tustin2121
                    May 14, 2018 at 16:42
                    Great answer!  In obtain_arraybuffer: buf.buffer.subarray does not seem to exist. Did you mean buf.buffer.slice here?
    – Marcin Czenko
                    Jul 2, 2018 at 17:41
                    @everydayproductive Thank you. As you can see in the edit history, I actually used ArrayBuffer.prototype.slice and later modified it to Uint8Array.prototype.subarray. Oh, and I did it wrong. Probably got a bit confused back then. It's all good right now thanks to you.
    – Константин Ван
                    Jul 8, 2018 at 7:18
    

    Use the following excellent npm package: to-arraybuffer.

    Or, you can implement it yourself. If your buffer is called buf, do this:

    buf.buffer.slice(buf.byteOffset, buf.byteOffset + buf.byteLength)
    

    You can think of an ArrayBuffer as a typed Buffer.

    An ArrayBuffer therefore always needs a type (the so-called "Array Buffer View"). Typically, the Array Buffer View has a type of Uint8Array or Uint16Array.

    There is a good article from Renato Mangini on converting between an ArrayBuffer and a String.

    I have summarized the essential parts in a code example (for Node.js). It also shows how to convert between the typed ArrayBuffer and the untyped Buffer.

    function stringToArrayBuffer(string) {
      const arrayBuffer = new ArrayBuffer(string.length);
      const arrayBufferView = new Uint8Array(arrayBuffer);
      for (let i = 0; i < string.length; i++) {
        arrayBufferView[i] = string.charCodeAt(i);
      return arrayBuffer;
    function arrayBufferToString(buffer) {
      return String.fromCharCode.apply(null, new Uint8Array(buffer));
    const helloWorld = stringToArrayBuffer('Hello, World!'); // "ArrayBuffer" (Uint8Array)
    const encodedString = new Buffer(helloWorld).toString('base64'); // "string"
    const decodedBuffer = Buffer.from(encodedString, 'base64'); // "Buffer"
    const decodedArrayBuffer = new Uint8Array(decodedBuffer).buffer; // "ArrayBuffer" (Uint8Array)
    console.log(arrayBufferToString(decodedArrayBuffer)); // prints "Hello, World!"
    

    A Buffer is a view for an ArrayBuffer. You can get to the internal wrapped ArrayBuffer using the buffer property.

    This is shared memory, and no copying is required.

    const arrayBuffer = theBuffer.buffer
    

    If you want a copy of the data, create another Buffer from the original Buffer (NOT from the wrapped ArrayBuffer), and then reference its wrapped ArrayBuffer.

    const newArrayBuffer = Buffer.from(theBuffer).buffer
    

    For reference, going the other direction, from an ArrayBuffer to a Buffer

    const arrayBuffer = getArrayBuffer()
    const sharedBuffer = Buffer.from(arrayBuffer)
    const copiedBuffer = Buffer.from(sharedBuffer)
    const copiedArrayBuffer = copiedBuffer.buffer
    

    Now there is a very useful npm package for this: buffer https://github.com/feross/buffer

    It tries to provide an API that is 100% identical to node's Buffer API and allow:

  • convert typed array to buffer: https://github.com/feross/buffer#convert-typed-array-to-buffer
  • convert buffer to typed array: https://github.com/feross/buffer#convert-buffer-to-typed-array
  • and few more.

    I tried the above for a Float64Array and it just did not work.

    I ended up realising that really the data needed to be read 'INTO' the view in correct chunks. This means reading 8 bytes at a time from the source Buffer.

    Anyway this is what I ended up with...

    var buff = new Buffer("40100000000000004014000000000000", "hex");
    var ab = new ArrayBuffer(buff.length);
    var view = new Float64Array(ab);
    var viewIndex = 0;
    for (var bufferIndex=0;bufferIndex<buff.length;bufferIndex=bufferIndex+8)            {
        view[viewIndex] = buff.readDoubleLE(bufferIndex);
        viewIndex++;
                    That's why Martin Thomson's answer uses Uint8Array -- it is agnostic to the size of the elements. The Buffer.read* methods are all slow, also.
    – ZachB
                    Jul 13, 2015 at 22:06
                    Multiple typed array views can reference the same ArrayBuffer using the same memory. Each value in a Buffer is one byte, so you need to put it into an array with element size of 1 byte. You can use Martin's method, then make a new Float64Array using the same arraybuffer in the constructor.
    – ZachB
                    Jul 15, 2015 at 1:32
    

    This Proxy will expose the buffer as any of the TypedArrays, without any copy. :

    https://www.npmjs.com/package/node-buffer-as-typedarray

    It only works on LE, but can be easily ported to BE. Also, never got to actually test how efficient this is.

    While this link may answer the question, it is better to include the essential parts of the answer here and provide the link for reference. Link-only answers can become invalid if the linked page changes – koceeng Feb 20, 2017 at 2:35 My wording might not sound very official, but it does provide enough information to recreate the solution. The solution relies on JavaScript Proxy Object to wrap a native NodeJS Buffer with getters and setters used by TypedArrays. This makes the Buffer instance compatible with any library that requires Typed Array interface. This is the answer original poster was hoping for, but feel free to dismiss it as it doesn't fit your academic/corporate lingo. See if I care. – Dlabz Feb 20, 2017 at 13:48

    Surprisingly, in my case (Electron: sending buffer to renderer as ArrayBuffer) this simply worked

    function bufferToArrayBuffer(buffer: Buffer): ArrayBuffer {
      return buffer.buffer as ArrayBuffer;
    

    and is like 100x faster than

    function bufferToArrayBuffer(buf) {
        const ab = new ArrayBuffer(buf.length);
        const view = new Uint8Array(ab);
        for (let i = 0; i < buf.length; ++i) {
            view[i] = buf[i];
        return ab;
    

    NodeJS, at one point (I think it was v0.6.x) had ArrayBuffer support. I created a small library for base64 encoding and decoding here, but since updating to v0.7, the tests (on NodeJS) fail. I'm thinking of creating something that normalizes this, but till then, I suppose Node's native Buffer should be used.

    This looks like a specialized (and slow) serialization-based method, not a generic method for converting to/from Buffer/ArrayBuffer? – ZachB Dec 2, 2015 at 19:10 toArrayBuffer(new Buffer([1,2,3])) -> ['01', '02', '03'] -- this is returning an array of strings, not integers/bytes. – ZachB Dec 4, 2015 at 3:23 In that case it's the same as stackoverflow.com/a/19544002/1218408, and still without necessary the byte offset checks in stackoverflow.com/a/31394257/1218408. – ZachB Dec 4, 2015 at 3:25

    Thanks for contributing an answer to Stack Overflow!

    • Please be sure to answer the question. Provide details and share your research!

    But avoid

    • Asking for help, clarification, or responding to other answers.
    • Making statements based on opinion; back them up with references or personal experience.

    To learn more, see our tips on writing great answers.