What are Media Capture and Streams and How Do I Use Them?

This article is part of a web development series from Microsoft. Thank you for supporting the partners who make SitePoint possible.

When using web technologies to develop apps, be it on the web or mobile, there are times when you would want to use local multimedia devices, such as microphones or video cameras. An example would be to allow users to stream or take photos of themselves from the local video camera. To give you a little background, audio/video capture and streaming on the web had been largely dependent on browser plugins (Flash or Silverlight). With HTML5 and browsers pulling the plug on browser plugins, HTML5, hailed as the savior, brought to the web access to device capabilities, from Geolocation (GPS) and WebGL (GPU) to the Web Audio API (audio hardware), amongst many others.

These powerful features expose high level JavaScript APIs that talk to the system's underlying hardware capabilities.

Let’s start with HTML Media Capture which, per specification, is defined as a form extension that facilitates the access to a device's media capture mechanism, which can be a camera, or microphone or even a file within the upload control.

At its heart the Media Capture extends the HTMLInputElement interface with a capture attribute. A basic example would be: .

This capture attribute makes the request to use the media capture tool (camera, microphone, etc.) for capturing media on the spot.

Here is simple declarative example to illustrate its use. The following shows an HTML form using capture alongside the accept attribute, which gives hints on the preferred MIME type for the user to capture media.

<input type="file" accept="image/*" capture>

HTML Media Capture extension was specifically designed to be simple and declarative, and covers a subset of the media capture functionality of the web platform. Nevertheless, this HTML specification does not provide detailed author control over capture nor does it allow access to real-time media streams from the hosting device. HTML Media Capture was the first shot at standardizing media capture on the web. It reuses the file input element and works by overloading it and adding new values for the accept parameter. So basically, it works like a charm but it only allows you to record a media file or take a snapshot in time. Where Media Capture fell short was the ability to achieve real-time effects such as rendering live webcam data to a <canvas> element and apply some WebGL filters on it.

And thus, we have Media Capture and Streams.

Media Capture and Streams is actually a set of JavaScript APIs. These APIs allow local media (audio and video) to be requested from a platform. In other words, it provides access to the user's local audio and video input/output devices.

More specifically, we have the MediaStream API, which provides the means to control where multimedia stream data is consumed, and provides some control over the devices that produce the media. Additionally, the MediaStream API exposes information about the devices that are able to capture and render media.

Why is it important? Here’s a history lesson for the future generations that might take this for granted. The media (Audio/Video) capture capability has been the "Nirvana" of web development for some time. Historically, we had to rely on browser plugins (Flash or Silverlight) to achieve this. Then came HTML5 to the rescue. HTML5 brought powerful features that allow access to device hardware natively from Geolocation (GPS) to WebGL (GPU) and much more. These features which are now baked into the browser expose high level JavaScript APIs that sit on top of the device’s hardware capabilities.

So, why would we use it becomes obvious.

One of the most important methods in this API is getUserMedia() and it’s the gateway into that set of APIs. getUserMedia() provides the means to access the user's local camera/microphone stream.

Nevertheless, Feature detecting is the best way to check for its support, either directly if(navigator.getUserMedia) or using modernizer if(Modernizr.getusermedia).

Continue reading %What are Media Capture and Streams and How Do I Use Them?%

by Rami Sarieddine via SitePoint