Saturday, March 8, 2014

Content synchronization with HTML5 videos Part 1

Hi folks. With this post I will share the details about an Extension:Elearnware which I created for Media-wiki.
Main technical aspect of this extension is to synchronize HTML5 videos with other content such as text and images. I named it as ELearnware because it can be used to create and view video lectures synchronized with slides. I have used popcorn JavaScript library and  a Cross-Browser Rich Text Editor to implement this extension.

This extension has two main parts. In MediaWiki perspective, we can say that there are two special pages called video editor and viewer. for this extension. Interaction between these page are like this.

First you upload an HTML5 video (.mp4 , .webm , .ogg). Then you select that video to sample and add content. Then you will be moved to special  page called video editor. In that page you sample the video. Even though it appears as sampling its not like that. Whenever you press the sample button, a JavaScript will save the time value of the current frame. Then it will take a snapshot of the current frame and draw it in the HTML5 canvas.   Whenever you click on one of those snapshots, you can seek your video to the time value that is associated with that snapshot. This will happen by using the JavaScript array which stores the time values. HTML 5 canvas click event will be used to identify the frame you clicked.

Video editor special page






















Apart from the array which stores time values, there will be an array to store content for each sample. Whenever you click on a snapshot,  content related to that snapshot will be loaded to the text area.
If you change from one snapshot to another, first script will store the content of the sample which you were in to the array. It will be saved in HTML format. Then script will load the content associated with that new frame from array to the text area. Finally, when  you are saving the content, it will be added to a HTML hidden field and will be posted to the server. Following is the JavaScript I used to sample the video.

                              



In my next post I will talk about what's happening in Viewer special page.