Proposals/Audio and video recordings

Idea

It would be nice to be able to record audio and video directly into Mahara.

Browser support

The standard that the web seems to be triangulating on is the HTML 5 "getUserMedia/Stream" API: [1]. I've successfully used the demo in Chromium, Firefox, and Chrome for Android. Additionally, the web says that Edge (Microsoft) supports it. You can try some demos of it here: https://github.com/webrtc/samples#getusermedia

Unfortunately, there is currently *no* way to record audio and video directly in the browser on iOS. This is a rapidly-evolving area of standardization, so it may become available in the next year or two, but it's not there now.

What does PoodLL do?

There is a Moodle plugin called PoodLL, which allows the recording of audio and video in the browser, for Moodle. It uses two systems:

1. If Flash is available, it uses a flash object to access the camera and microphone.
2. If Flash is not available, it presents a standard HTML <input type="file"> file uploader object, with the additional flag "accept=audio/*" or "accept=video/*", which prompts a mobile browser to open its applications that can share or create audio & video files.
3. In iOS, "accept=audio/*" is not supported, so PoodLL gives you an "accept=video/*" instead and converts the video file into an audio file on the server-side after it is uploaded. It can either use a file converter webservice provided by PoodLL, or it can convert them using ffmpeg on the web server.
4. PoodLL uses a CSS hack to superimpose a nicer-looking button on top of the standard HTML file upload button.

Implementation idea

So, with all that info, and keeping in mind that we still need to clarify some requirements, the rough implementation plan would be as follows:

• We enhance the existing "filebrowser" Pieform element to have additional controls for the recording of audio & video using getUserMedia(). (Ideally we'll use some existing wrapper library around getUserMedia() for better UI, cross-browser compatibility, and to make it easier to adapt when browser standards change)
• We use browser sniffing to detect whether getUserMedia() will be present, and if it's not, we fall back to an <input type="file" accept=...> element.
• Because iOS doesn't support uploading audio files, we add server-side file conversion for this use-case. Uploaded files that need to be converted will be put into a queue, and a cron job will convert them, using ffmpeg. While implementing this, it may also be a good idea to add automatic format conversion for all uploaded audio/video files into formats that have optimal cross-browser compatibility.
• We create a new TinyMCE plugin (or 2 separate ones) for embedding audio & video files into rich text areas. This will be based on Mahara's existing image embed TinyMCE plugin, which uses the pieform filebrowser (which we'll have already extended to support recording audio/video). It will place a widget into the TinyMCE display area, which can use the same HTML5 player as our embedded media block, to display the audio & video in a playable manner. This will additionally require adding backend code to handle permissions on embedded audio/video, which will be based on the existing systems for permissions for embedded images.

Mahara Mobile

Additional possibilities open up if we consider the Mahara native app. We're currently in the process of migrating this from a native Android app (MaharaDroid) into a cross-platform Apache Cordova app (working name "Mahara Mobile"). If we do that, then we could:

• Record audio & video directly in the Mahara Mobile app. There is a Cordova plugin for this, and evidence of people successfully using it in iOS and Android.
• Expand Mahara Mobile to support importing audio & video from other applications (like it currently does with images)
  • Android: This is easy to do in Android. You just fire off an "Intent" that asks for audio or video, and the Android OS will open up all the applications that have registered as being able to provide that. It may also be possible to access the device's media storage area where music & ringtone files are stored.
  • iOS: This is more complicated in iOS. Well, for video it's easy: you can ask the system to access the camera and gallery apps. But there is no similar system for audio. One possibility is to make it a general filesystem access call, in which case we could access files recorded by other apps if they make them available. Another possibility is to use one of the third-party audio-sharing SDKs like AudioCopy or AudioBus. However, those SDKs are meant more for music composition & mixing applications, and may be too heavy-duty for Mahara Mobile.

Further links

Discussion around audio and video recording in general on mahara.org Discussion around PoodLL on mahara.org