Custom Video Elements with Javascript and Agora Web SDK

In today’s multimedia-driven web, integrating live audio and video streaming into web applications is increasingly commonplace. However, developers often encounter restrictions with standard video integrations that lack flexibility or fail to meet specific user interface requirements. The Agora Web SDK provides a powerful set of tools for real-time engagement, allowing not only for easy embedding of live voice and video functionalities but also for complete customization of video elements using JavaScript/TypeScript.

Whether you are a beginner eager to explore real-time video functionalities, or an advanced developer looking to create highly customized video interfaces, this guide is tailored for you. The Agora Web SDK offers convenient APIs that can automatically embed a video element within a specified <div/>, but there are scenarios where developers need a bit more control over the video tracks.

In this guide we’ll dive into how to manipulate audio and video tracks created by the Agora Web SDK, customize video player controls, and integrate interactive elements to enhance user engagement.

Prerequisites

Before starting, ensure you have the following:

• A basic understanding of HTML, CSS, and JavaScript.
• Node.js installed on your computer.
• A code editor — I like to use VSCode.
• A developer account with Agora.io.

Project Setup

Before we dive into the code, this guide assumes you have a project setup with an HTML file linked to a JavaScript file. This guide will focus solely on integrating the Agora Web SDK and setting up the custom video element.

If you haven’t already, you need to add the Agora Web SDK to your project. Open your terminal, navigate to your project directory, and run the following command:

npm install agora-rtc-sdk-ng

Creating the Custom Video Element

The Agora Web SDK makes it straightforward to manage audio and video tracks separately, giving you the flexibility to create a custom video element tailored to your application’s needs. Below, I’ll walk you through the steps to set up and control a video stream.

Step 1: Initialize the SDK and Create Audio and Video Tracks

First, import AgoraRTC and then create an asynchronous function that handles the setup. Use .createMicrophoneAndCameraTracks() to initialize the audio and video tracks, wrap this in a try-catch to handle any errors that might occur during initialization

import AgoraRTC from 'agora-rtc-sdk-ng';

async function initializeVideo() {
    try {
        // Create audio and video tracks with specific configurations
        const [audioTrack, videoTrack] = await AgoraRTC.createMicrophoneAndCameraTracks({
            audioConfig: 'music_standard', 
            videoConfig: '360p_7'
        });

        // Proceed to creating and configuring the video element
        setupVideoElement(audioTrack, videoTrack, true);
    } catch (error) {
        console.error('Failed to initialize media tracks:', error);
    }
}

Step 2: Create and Configure the Video Element

Next, we’ll create setupVideoElement() to create a video element, configure its attributes, and set its source to use the audio and video tracks. Once we set up the <video /> element, we’ll append it to the DOM and set it up to play automatically once the media metadata is loaded.

function setupVideoElement(audioTrack, videoTrack, isRemote) {
    const videoFromStream = document.createElement('video');
    videoFromStream.id = 'local-video-stream';
    videoFromStream.setAttribute('playsinline', 'true'); // avoid fullscreen on mobile browsers
    videoFromStream.setAttribute('webkit-playsinline', 'true'); // for playback in webkit browsers

    // Set the source object of the video element to include our tracks
    const tracks = [videoTrack.getMediaStreamTrack()]
    if (isRemote) { 
      // only include the audio track if it's a remote user
      tracks.push(audioTrack.getMediaStreamTrack())
    }
    videoFromStream.srcObject = new MediaStream(tracks);

    videoFromStream.controls = false; // we choose not to display controls
    videoFromStream.height = 300; // set the height
    videoFromStream.width = 500; // set the width

    // Append the video element to the document's body
    document.body.appendChild(videoFromStream);

    // Automatically play the video when it is ready
    videoFromStream.onloadedmetadata = () => {
        videoFromStream.play();
    };
}

This function is setup to differentiate between the local and remote videos, because you wouldn’t want to play the local audio as it would cause an echo, but for remote users you want to add the audio track.

Step 3: Start the Video

Finally, you need to call the initializeVideo function when your application is ready to start processing the video. This could be after the DOM is fully loaded, or after some user interaction based on your application's logic.

document.addEventListener('DOMContentLoaded', initializeVideo);

Next Steps

That’s it! Not so difficult right? Now with this knowledge, you’re ready to integrate and manipulate media streams in your Agora-powered web apps, enabling you to offer enhanced interactive experiences to your users.

Here are a few next steps to keep you going on your journey with Agora:

1. Explore Agora’s other SDKs such as real-time messaging with Agora Signaling, or robust in-app chat features with Agora Chat.
2. Make your live video communication secure by implementing authentication mechanisms, end-to-end encryption, and other security measures.
3. Explore Agora’s scalability features. If your application anticipates a large number of users, understand how to scale your application to accommodate a growing user base while maintaining performance and reliability.

Other Resources

• Dive into the Agora Documentation to better understand the features and capabilities offered by Agora SDK. Explore the API reference, sample codes, and best practices.
• Be part of the Agora developer community: Join the conversation on X(Twitter), or LinkedIn to share experiences, and stay updated on the latest developments. Need support? Reach out via StackOverflow for advice on your implementation.
• Continue your developer journey with more guides from Agora’s Medium publication.

In Plain English 🚀

Thank you for being a part of the In Plain English community! Before you go:

• Be sure to clap and follow the writer ️👏️️
• Follow us: X | LinkedIn | YouTube | Discord | Newsletter
• Visit our other platforms: Stackademic | CoFeed | Venture | Cubed
• More content at PlainEnglish.io