Streaming API Responses

🎨 Front End

Oct 11, 2024

2 min read

What is Streaming?

Essentially it’s a way of receiving data in chunks rather than getting the full response from the API in one go.

Chunked Transfer Encoding

The chunks are sent via Chunked Transfer Encoding, which is known as a data transfer mechanism within HTTP.

The data stream is split into chunks that are sent by the server. When the transfer of data is completed it is known as, chunked.

It is an efficient way to reduce latency and memory on both server and client side.

Server Side Rendering

Due to the nature of chunking, you are likely to have an async function of some kind to produce the chunks within your application.

This could mean creating a server via Node.js with Express or another back-end microservice.

Another way around this could be to use a full-stack framework like Next.js.

How Does it Differ from Traditional Response?

Large data can be processed in a more efficient way. If you are building an application that has data constantly changing and you need to send a POST/PUT to a back-end it can be very helpful. The most obvious use case is a chatbot however it can also work for anything that requires update for things like graphs, uploading data and other visualisations.

Example Code

const express = require('express');
const app = express();
const port = 3000;

app.get('/stream', (req, res) => {
  // Set the appropriate headers for streaming
  res.setHeader('Content-Type', 'application/json');
  res.setHeader('Transfer-Encoding', 'chunked');

  // Simulate a large dataset
  const largeDataset = Array.from({ length: 1000 }, (_, i) => ({ id: i, value: `Item ${i}` }));

  // Function to send data in chunks
  const sendChunk = (chunk) => {
    res.write(JSON.stringify(chunk) + '\n');
  };

  // Start the response
  res.write('[\n');

  // Send data in chunks
  let currentIndex = 0;
  const chunkSize = 100;

  function sendNextChunk() {
    if (currentIndex < largeDataset.length) {
      const chunk = largeDataset.slice(currentIndex, currentIndex + chunkSize);
      sendChunk(chunk);
      currentIndex += chunkSize;
      // Use setImmediate to prevent blocking the event loop
      setImmediate(sendNextChunk);
    } else {
      // End the response
      res.write(']');
      res.end();
    }
  }

  // Start sending chunks
  sendNextChunk();
});

app.listen(port, () => {
  console.log(`Server running at http://localhost:${port}`);
});

umma.dev