Discover the new easier way to develop Kurento video applications

JavaScript - Magic Mirror

This web application extends the Hello World Tutorial, adding media processing to the basic WebRTC loopback.

Running this example

First of all, install Kurento Media Server: Installation Guide. Start the media server and leave it running in the background.

Install Node.js, Bower, and a web server in your system:

curl -sL https://deb.nodesource.com/setup_8.x | sudo -E bash -
sudo apt-get install -y nodejs
sudo npm install -g bower
sudo npm install -g http-server

Here, we suggest using the simple Node.js http-server, but you could use any other web server.

You also need the source code of this tutorial. Clone it from GitHub, then start the web server:

git clone https://github.com/Kurento/kurento-tutorial-js.git
cd kurento-tutorial-js/kurento-magic-mirror/
git checkout master
bower install
http-server -p 8443 --ssl --cert keys/server.crt --key keys/server.key

Note that HTTPS is required by browsers to enable WebRTC, so the web server must use SSL and a certificate file. For instructions, check Configure JavaScript applications to use HTTPS. For convenience, this tutorial already provides dummy self-signed certificates (which will cause a security warning in the browser).

When your web server is up and running, use a WebRTC compatible browser (Firefox, Chrome) to open the tutorial page:

  • If KMS is running in your local machine:

    https://localhost:8443/
    
  • If KMS is running in a remote machine:

    https://localhost:8443/index.html?ws_uri=ws://{KMS_HOST}:8888/kurento
    

Note

By default, this tutorial works out of the box by using non-secure WebSocket (ws://) to establish a client connection between the browser and KMS. This only works for localhost. It will fail if the web server is remote.

If you want to run this tutorial from a remote web server, then you have to do 3 things:

  1. Configure Secure WebSocket in KMS. For instructions, check Securing Kurento Media Server.
  2. In index.js, change the ws_uri to use Secure WebSocket (wss:// instead of ws://) and the correct KMS port (8433 instead of 8888).
  3. As explained in the link from step 1, if you configured KMS to use Secure WebSocket with a self-signed certificate you now have to browse to https://{KMS_HOST}:8433/kurento and click to accept the untrusted certificate.

Understanding this example

This application uses computer vision and augmented reality techniques to add a funny hat on top of detected faces. The following picture shows a screenshot of the demo running in a web browser:

Kurento Magic Mirror Screenshot: WebRTC with filter in loopback

Kurento Magic Mirror Screenshot: WebRTC with filter in loopback

The interface of the application (an HTML web page) is composed by two HTML5 video tags: one for the video camera stream (the local client-side stream) and other for the mirror (the remote stream). The video camera stream is sent to the Kurento Media Server, processed and then is returned to the client as a remote stream.

To implement this, we need to create a Media Pipeline composed by the following Media Element s:

  • WebRtcEndpoint: Provides full-duplex (bidirectional) WebRTC capabilities.
  • FaceOverlay filter: Computer vision filter that detects faces in the video stream and puts an image on top of them. In this demo the filter is configured to put a Super Mario hat).

The media pipeline implemented is illustrated in the following picture:

WebRTC with filter in loopback Media Pipeline

WebRTC with filter in loopback Media Pipeline

The complete source code of this demo can be found in GitHub.

JavaScript Logic

This demo follows a Single Page Application architecture (SPA). The interface is the following HTML page: index.html. This web page links two Kurento JavaScript libraries:

  • kurento-client.js : Implementation of the Kurento JavaScript Client.
  • kurento-utils.js : Kurento utility library aimed to simplify the WebRTC management in the browser.

In addition, these two JavaScript libraries are also required:

  • Bootstrap : Web framework for developing responsive web sites.
  • jquery.js : Cross-platform JavaScript library designed to simplify the client-side scripting of HTML.
  • adapter.js : WebRTC JavaScript utility library maintained by Google that abstracts away browser differences.
  • ekko-lightbox : Module for Bootstrap to open modal images, videos, and galleries.
  • demo-console : Custom JavaScript console.

The specific logic of this demo is coded in the following JavaScript page: index.js. In this file, there is a function which is called when the green button labeled as Start in the GUI is clicked.

var startButton = document.getElementById("start");

startButton.addEventListener("click", function() {
   var options = {
     localVideo: videoInput,
     remoteVideo: videoOutput
   };

   webRtcPeer = kurentoUtils.WebRtcPeer.WebRtcPeerSendrecv(options, function(error) {
      if(error) return onError(error)
      this.generateOffer(onOffer)
   });

   [...]
}

The function WebRtcPeer.WebRtcPeerSendrecv abstracts the WebRTC internal details (i.e. PeerConnection and getUserStream) and makes possible to start a full-duplex WebRTC communication, using the HTML video tag with id videoInput to show the video camera (local stream) and the video tag videoOutput to show the remote stream provided by the Kurento Media Server.

Inside this function, a call to generateOffer is performed. This function accepts a callback in which the SDP offer is received. In this callback we create an instance of the KurentoClient class that will manage communications with the Kurento Media Server. So, we need to provide the URI of its WebSocket endpoint. In this example, we assume it’s listening in port 8888 at the same host than the HTTP serving the application.

[...]

var args = getopts(location.search,
{
  default:
  {
    ws_uri: 'ws://' + location.hostname + ':8888/kurento',
    ice_servers: undefined
  }
});

[...]

kurentoClient(args.ws_uri, function(error, client){
  [...]
};

Once we have an instance of kurentoClient, the following step is to create a Media Pipeline, as follows:

client.create("MediaPipeline", function(error, _pipeline){
   [...]
});

If everything works correctly, we have an instance of a media pipeline (variable pipeline in this example). With this instance, we are able to create Media Elements. In this example we just need a WebRtcEndpoint and a FaceOverlayFilter. Then, these media elements are interconnected:

pipeline.create('WebRtcEndpoint', function(error, webRtcEp) {
  if (error) return onError(error);

  setIceCandidateCallbacks(webRtcPeer, webRtcEp, onError)

  webRtcEp.processOffer(sdpOffer, function(error, sdpAnswer) {
    if (error) return onError(error);

    webRtcPeer.processAnswer(sdpAnswer, onError);
  });
  webRtcEp.gatherCandidates(onError);

  pipeline.create('FaceOverlayFilter', function(error, filter) {
    if (error) return onError(error);

    filter.setOverlayedImage(args.hat_uri, -0.35, -1.2, 1.6, 1.6,
    function(error) {
      if (error) return onError(error);

    });

    client.connect(webRtcEp, filter, webRtcEp, function(error) {
      if (error) return onError(error);

      console.log("WebRtcEndpoint --> filter --> WebRtcEndpoint");
    });
  });
});

Note

The TURN and STUN servers to be used can be configured simple adding the parameter ice_servers to the application URL, as follows:

https://localhost:8443/index.html?ice_servers=[{"urls":"stun:stun1.example.net"},{"urls":"stun:stun2.example.net"}]
https://localhost:8443/index.html?ice_servers=[{"urls":"turn:turn.example.org","username":"user","credential":"myPassword"}]

Dependencies

The dependencies of this demo has to be obtained using Bower. The definition of these dependencies are defined in the bower.json file, as follows:

"dependencies": {
   "kurento-client": "6.14.0",
   "kurento-utils": "6.14.0"
}

Note

We are in active development. You can find the latest version of Kurento JavaScript Client at Bower.