Using The Getusermedia API With The HTML5 Video And Canvas Elements

 getUserMedia API

By Ian Devlin

One of the really cool things that has come out of the W3C is a web API that gives you the ability to access a webcam’s video and/or audio through the browser. This is the getUserMedia API. Here we will take a quick look at the API and how we can use it. In addition, we will see how we can use this API with the HTML5 video and canvas elements.

The getUserMedia API

The API itself is quite simple to use, you simply ask the browser to obtain a connection to the user’s webcam (with the user’s permission of course!) and you either get the connection or you don’t.

navigator.getUserMedia = (navigator.getUserMedia || 
                          navigator.webkitGetUserMedia || 
                          navigator.mozGetUserMedia || 
                          navigator.msGetUserMedia);
   if (navigator.getUserMedia) {
      navigator.getUserMedia(
         {
            video:true,
            audio:false
         },        
         function(stream) { /* do something */ },
         function(error) { /* do something */ }
      );
   }
   else {
      alert('Sorry, the browser you are using doesn\'t support getUserMedia');
      return;
    }

The code snippet above shows you how you might go about accessing a webcam. At the moment various vendor prefixes might be required so we provide them just in case. This example also requests access to the video stream only, and not the audio. The default values for either video or audio is false.

If the video stream is successfully returned, then our success callback is called otherwise our error callback is executed. The error contains a code that can have one of the following values:

PERMISSION_DENIED
The user denied us permission to access their camera

NOT_SUPPORTED_ERROR
The requested access to the camera is not supported (e.g. if we want audio and the browser doesn’t support that)

MANDATORY_UNSATISFIED_ERROR
No media tracks of the types specified (e.g. audio or video) were found.

It’s that simple, there’s not much more to the API itself. Let’s now take a look at our example where we actually use the stream that the getUserMedia API returns.

Using the getUserMedia API with HTML5 <video>

For the purposes of our example, let’s setup a few things. Our HTML is quite short and the important bits look as follows:

<video id='v'></video>

Our JavaScript puts everything in a listener for the DOMContentLoaded event. The first part defines some variables and initializes them and then we attempt to gain access to the user’s webcam using code that we’ve already seen:

window.addEventListener('DOMContentLoaded', function() {
   var v = document.getElementById('v');
   navigator.getUserMedia = (navigator.getUserMedia || 
                             navigator.webkitGetUserMedia || 
                             navigator.mozGetUserMedia || 
                             navigator.msGetUserMedia);
   if (navigator.getUserMedia) {
      // Request access to video only
      navigator.getUserMedia(
         {
            video:true,
            audio:false
         },        
         function(stream) {
            var url = window.URL || window.webkitURL;
            v.src = url ? url.createObjectURL(stream) : stream;
            v.play();
         },
         function(error) {
            alert('Something went wrong. (error code ' + error.code + ')');
            return;
         }
      );
   }
   else {
      alert('Sorry, the browser you are using doesn\'t support getUserMedia');
      return;
   }
}

The interesting bit here is contained within the success callback function where we assign the camera’s stream to the src of our video element. The camera picture is now displayed within our HTML page’s video element.

We could of course leave it at that, as it shows how the getUserMedia API can work and interact with the HTML5 video element. But we can take it a bit further.

Using the getUserMedia API with HTML5 <video> and <canvas>

Now that we have the camera streaming in our HTML5 video element, we’re going to hide the video element and copy its contents to a HTML canvas element so we can manipulate it.

To start off, we need to add canvas and button elements, so our HTML now looks as follows:

<video id='v'></video>
<canvas id='c'></canvas>
<button id='grey'>Toggle Greyness</button>

We “hide” the video element by simply doing the following with CSS:

video { 
   position:absolute;
   visibility:hidden;
}

We also need to add some more variables to our JavaScript initialization which now looks as follows:

var isStreaming = false,
    v = document.getElementById('v'),
    c = document.getElementById('c'),
    grey = document.getElementById('grey');
    con = c.getContext('2d');
    w = 600, 
    h = 420,
    greyscale = false;

Next, we need to set up an event listener on the canplay event (raised when the video is available to play) to initialize the canvas:

v.addEventListener('canplay', function(e) {
   if (!isStreaming) {
      // videoWidth isn't always set correctly in all browsers
      if (v.videoWidth > 0) h = v.videoHeight / (v.videoWidth / w);
      c.setAttribute('width', w);
      c.setAttribute('height', h);
      // Reverse the canvas image
      con.translate(w, 0);
      con.scale(-1, 1);
      isStreaming = true;
   }
}, false);

When the video stream starts to play, we need to copy the video image to the canvas element. We do so by waiting for the play event and then using setInterval to call our copying code every 33 milliseconds:

v.addEventListener('play', function() {
   // Every 33 milliseconds copy the video image to the canvas
   setInterval(function() {
      if (v.paused || v.ended) return;
      con.fillRect(0, 0, w, h);
      con.drawImage(v, 0, 0, w, h);
      if (greyscale) goingGrey();
   }, 33);
}, false);

Our code also contains a single button for turning the video stream grey, so we also need to define a click event handler for that, and also the function that turns the canvas contents grey.

grey.addEventListener('click', function() { greyscale = !greyscale; }, false);
 
var goingGrey = function() {
   var imageData = con.getImageData(0, 0, w, h);
   var data = imageData.data;
   for (var i = 0; i < data.length; i += 4) {
      var bright = 0.34 * data[i] + 0.5 * data[i + 1] + 0.16 * data[i + 2];
      data[i] = bright;
      data[i + 1] = bright;
      data[i + 2] = bright;
   }
   con.putImageData(imageData, 0, 0);
}

The code for copying the video’s contents into the HTML5 canvas and the  goingGrey() function both use HTML5’s Canvas 2D Context API.

This code now asks the user for access to their video camera, once given, it streams it on screen to a HTML5 canvas element via a HTML5 video element. The user can then toggle the video image to be greyscale by simply pressing a button.

You can see it in action on the demo page.

Of course other interactions with the canvas’ image are possible, applying complicated filters etc., but this gives you an idea of how these technologies can be used together to build a complete application.