Building a Video Streaming Platform with AWS S3, HLS, and Node.js

By the end of this guide, you'll have the skills to:

- Store video chunks securely in AWS S3.

- Stream videos dynamically to users using HLS.

- Enable users to upload videos, automatically process them into HLS chunks, and store them in S3.

Ready to get started? Let's dive in!

Step 1: Storing Video Chunks in AWS S3

To stream videos, we first need to upload .ts (MPEG transport stream) files and a .m3u8 playlist to an AWS S3 bucket. Follow these steps to upload your files:

Why AWS S3?

AWS S3 provides a scalable, reliable, and secure storage solution, making it perfect for handling large video files. Using the AWS CLI simplifies the upload process and integrates easily into automation pipelines.

Step-by-Step Instructions:

1. Open your terminal.

2. Run the following commands to upload the video chunks and playlist:

aws s3 cp output0.ts s3://your-bucket-name/path-to-folder/
aws s3 cp playlist.m3u8 s3://your-bucket-name/path-to-folder/

While we’re using the AWS CLI here, you could also use the AWS Management Console or SDKs (e.g., AWS SDK for JavaScript) if you prefer a graphical interface or code-based interaction.

Once uploaded, your files will be securely stored and accessible for streaming. Let’s move to setting up the backend!

Step 2: Streaming the Video with Express.js

Now that our video chunks are in S3, we need a backend to fetch and serve them to the client. We’ll use Node.js with Express.js to handle this.

Backend Setup:

1. Create a new Node.js project and install the necessary packages:

npm init -y
npm install express @aws-sdk/client-s3

2. Add the following code to your server.js file:

import express from 'express';
import { S3Client, GetObjectCommand } from '@aws-sdk/client-s3';
import stream from 'stream';

const app = express();
const s3 = new S3Client({ region: 'your-region' });

// Endpoint to serve the HLS playlist
app.get('/play/playlist.m3u8', async (req, res) => {
  try {
    const command = new GetObjectCommand({
      Bucket: 'your-bucket-name',
      Key: 'path-to-folder/playlist.m3u8',
    });
    const s3Response = await s3.send(command);
    s3Response.Body.pipe(res);
  } catch (err) {
    console.error('Error fetching playlist:', err);
    res.status(500).send('Error fetching playlist');
  }
});

// Endpoint to serve video chunks
app.get('/play/video/:segment', async (req, res) => {
  try {
    const command = new GetObjectCommand({
      Bucket: 'your-bucket-name',
      Key: `path-to-folder/${req.params.segment}`,
    });
    const s3Response = await s3.send(command);
    s3Response.Body.pipe(res);
  } catch (err) {
    console.error('Error fetching video segment:', err);
    res.status(500).send('Error fetching video segment');
  }
});

const PORT = 3000;
app.listen(PORT, () => {
  console.log(`Server is running on http://localhost:${PORT}`);
});

What’s Happening Here?

- Playlist Endpoint: Fetches and streams the .m3u8 file from S3.

- Video Chunks Endpoint: Dynamically fetches .ts chunks based on the client’s request.

Once the backend is running, we can serve video content to the client. Let’s bring it all together on the frontend.

Step 3: Playing the Video on the Client-Side

To play the HLS video stream in the browser, we’ll use the <video> HTML tag and the HLS.js library for compatibility with all modern browsers.

Implementation:

1. Create an index.html file with the following content:

<!DOCTYPE html>
<html lang="en">
<head>
 <meta charset="UTF-8">
 <meta name="viewport" content="width=device-width, initial-scale=1.0">
 <title>Video Streaming</title>
 <script src="https://cdn.jsdelivr.net/npm/hls.js@latest"></script>
</head>
<body>
 <h1>Video Streaming Example</h1>
 <video id="videoPlayer" controls width="720" autoplay></video>

 <script>
  const video = document.getElementById('videoPlayer');
  const videoSrc = 'http://localhost:3000/play/playlist.m3u8';

  if (Hls.isSupported()) {
   const hls = new Hls();
   hls.loadSource(videoSrc);
   hls.attachMedia(video);
  } else if (video.canPlayType('application/vnd.apple.mpegurl')) {
   video.src = videoSrc;
  } else {
   console.error('This browser does not support HLS streaming');
  }
 </script>
</body>
</html>

Key Points:

- The browser fetches the .m3u8 playlist, which directs it to download the .ts chunks for smooth playback.

- The HLS.js library ensures compatibility with browsers that don’t natively support HLS.

Now you have a working video player! Let’s take it a step further by allowing users to upload videos.

Step 4: Allowing Users to Upload Videos

What if users want to upload their own videos? We can process those videos into HLS chunks and store them in S3.

Backend for Uploading and Processing:

1. Install the required packages:

npm install multer fluent-ffmpeg

2. Update your server.js file with the following:

import multer from 'multer';
import ffmpeg from 'fluent-ffmpeg';
import { PutObjectCommand } from '@aws-sdk/client-s3';
import fs from 'fs';

const upload = multer({ dest: 'uploads/' });

app.post('/upload', upload.single('video'), (req, res) => {
  const inputPath = req.file.path;
  const outputPath = 'processed/';

  ffmpeg(inputPath)
    .output(`${outputPath}playlist.m3u8`)
    .outputOptions([
      '-hls_time 10',
      '-hls_list_size 0',
      '-f hls',
    ])
    .on('end', async () => {
      try {
        // Upload .m3u8 and .ts files to S3
        const filesToUpload = fs.readdirSync(outputPath);
        for (const file of filesToUpload) {
          const command = new PutObjectCommand({
            Bucket: 'your-bucket-name',
            Key: `path-to-folder/${file}`,
            Body: fs.createReadStream(`${outputPath}${file}`),
          });
          await s3.send(command);
        }
        res.send('Video uploaded and processed successfully!');
      } catch (err) {
        console.error('Error uploading to S3:', err);
        res.status(500).send('Error uploading video.');
      }
    })
    .on('error', (err) => {
      console.error('Error processing video:', err);
      res.status(500).send('Error processing video.');
    })
    .run();
});

Workflow:

1. Users upload a video via the /upload endpoint.

2. FFmpeg processes the video into HLS chunks.

3. The processed files are automatically uploaded to S3.

Final Thoughts

Congratulations! You’ve built a fully functional video streaming platform. Here's a recap of what we accomplished:

- Secure Storage: Used AWS S3 for reliable and scalable storage.

- Seamless Streaming: Enabled dynamic streaming with HLS.

- User Uploads: Automated video processing and storage with FFmpeg and Node.js.

This architecture is scalable and can serve as the foundation

🚀Darmawan

Labs