Latest revision as of 17:43, 11 July 2020

Idea

It would be nice to be able to record audio and video directly into Mahara.

Browser support

The standard that the web seems to be triangulating on is the HTML 5 "getUserMedia/Stream" API: [1]. I've successfully used the demo in Chromium, Firefox, and Chrome for Android. Additionally, the web says that Edge (Microsoft) supports it. You can try some demos of it here: https://github.com/webrtc/samples#getusermedia

Unfortunately, there is currently *no* way to record audio and video directly in the browser on iOS. This is a rapidly-evolving area of standardization, so it may become available in the next year or two, but it's not there now.

What does PoodLL do?

There is a Moodle plugin called PoodLL, which allows the recording of audio and video in the browser, for Moodle. It uses two systems:

If Flash is available, it uses a flash object to access the camera and microphone.
If Flash is not available, it presents a standard HTML <input type="file"> file uploader object, with the additional flag "accept=audio/*" or "accept=video/*", which prompts a mobile browser to open its applications that can share or create audio & video files.
In iOS, "accept=audio/*" is not supported, so PoodLL gives you an "accept=video/*" instead and converts the video file into an audio file on the server-side after it is uploaded. It can either use a file converter webservice provided by PoodLL, or it can convert them using ffmpeg on the web server.
PoodLL uses a CSS hack to superimpose a nicer-looking button on top of the standard HTML file upload button.

Implementation idea

General

So, with all that info, and keeping in mind that we still need to clarify some requirements, the rough implementation plan by AaaronW would be as follows:

To the filepicker and the "File -> Contents" screen, we add "Record video" and "Record audio" buttons near the current "Upload" button/drag-drop.
We use OS-sniffing to customize these buttons depending on the user's browser. On Mobile they'll say "Record", and on Desktop they'll say "Upload", to reflect what will probably happen on the user's computer when they click them.
These buttons will use <input type="file" accept="audio/*"> and "video/*" tags. Browsers will respond to this by either launching an audio/video capture application, or opening a file upload dialog that tries to filter to just audio/video files. (Demo: https://jsfiddle.net/dipish/WWqR5/ )
iOS doesn't support <input type="file" accept="audio/*"> so it needs special treatment, which I'll describe further down.
Once the file is uploaded, we put it into a conversion queue. A cron job looks through that queue and makes a "display copy" of the file suitable for display in a web browser via HTML5. For audio, this will be MP3 format, and for video it'll be H.264 MP4. The conversion will be done via the command-line utilities "avconv" or "ffmpeg". The path to these commands will be configurable by a config setting (similar to the old pathtozip config). If it's not present, conversion won't be done. This code can be written in an extensible manner so that new conversion programs for different operating systems can be added later, and the admin can choose which one to use.
We treat the "display copy" much like we treat image thumbnails. Its only purpose is as a workaround for web browser limitations, not an end in itself. So we don't ask the user for permission to do it, we don't count it against their file quota, and we delete or create them as needed.
To limit potential disk over-usage due to a/v display copies, we can make admin settings to set a maximum size of file that will be converted. We can also use high-compression settings on the converted files to limit the amount of space they use. We might also consider only doing the conversion once a file is placed into a block... but I prefer doing the conversion right away, so that we can display the file on its artefact detail page.
Because the "display copy" is only a sort of thumbnail, we retain the original file as well. The "Download" link for the file gives the user the original file, not the display copy.

Audio in iOS

In addition to the above:

If we sniff that the user is on iOS, for the "Record audio" button we'll actually serve an "accept=video/*" button.
When an iOS user clicks this "Record audio" button, it will launch their video recorder/selector.
Once they record and upload a video, on the server side we will note that they were trying to record audio, and so we will convert the file to an audio file, change its name in the file area to end with the correct file suffix, and discard the original video file.
In testing on my local machine, I found that it takes about 1 second to convert each minute of video. So for small files (5 minutes or less) it would provide the best user interface if we just converted them immediately, as part of the upload script.
For larger files, we'll need to put them into the conversion cron queue. This means there may be several minutes during which the file has been uploaded but is not yet "usable" as an audio file. I propose that during this period we list the file in the user's content area (so they don't get confused and try to upload it again), but makes its name indicate "Pending conversion" or something similar, so that they know it's not yet usable. To avoid needing special display code to gray out Pending files, we could simply say that if they're downloaded, it serves an empty, zero-length file. Once the conversion queue converts the file, the "Pending" automatically leaves the filename, and download attempts correctly get the converted file.

Recording audio/video in-browser, not yet

There is an HTML5 API for accessing the camera and microphone from a web browser, the "navigate.getUserMedia()" API. But this technology does not yet seem to be mature. Tellingly, none of the major audio/video sharing websites use it (such as YouTube). And crucially, it doesn't work in iOS Safari.

The API is also very low-level, and would require additional extensive coding to add features needed for a good user experience, like time codes, indications of file size used and available remaining memory, the ability to trim the file after recording, etc.

The state of the audio API seems to be more mature than the video API at this point. There is an excellent demo available at https://webaudiodemos.appspot.com/AudioRecorder/index.html of an audio recorder that shows a sound wave as the audio is recorded. However, as mentioned, it doesn't work in iOS.

So for now, our users will get a better user experience from relying on recording programs outside the browser.

Mahara Mobile

Additional possibilities open up if we consider the Mahara native app. We're currently in the process of migrating this from a native Android app (MaharaDroid) into a cross-platform Apache Cordova app (working name "Mahara Mobile"). If we do that, then we could:

Record audio & video directly in the Mahara Mobile app. There is a Cordova plugin for this, and evidence of people successfully using it in iOS and Android.
Expand Mahara Mobile to support importing audio & video from other applications (like it currently does with images)
- Android: This is easy to do in Android. You just fire off an "Intent" that asks for audio or video, and the Android OS will open up all the applications that have registered as being able to provide that. It may also be possible to access the device's media storage area where music & ringtone files are stored.
- iOS: This is more complicated in iOS. Well, for video it's easy: you can ask the system to access the camera and gallery apps. But there is no similar system for audio. One possibility is to make it a general filesystem access call, in which case we could access files recorded by other apps if they make them available. Another possibility is to use one of the third-party audio-sharing SDKs like AudioCopy or AudioBus. However, those SDKs are meant more for music composition & mixing applications, and may be too heavy-duty for Mahara Mobile.

Proposals/Audio and video recordings: Difference between revisions

From Mahara Wiki

< Proposals