Multimedia has become technically easier to create (e.g. recording lectures) but while users can easily bookmark, search, link to, or tag (i.e. classify) the WHOLE of a podcast or video recording available on the web they cannot easily find, or associate their notes or resources with, PART of that recording. As an analogy, users would clearly find a text book difficult to use if it had no contents page, index or page numbers. Therefore the growing amount of knowledge available in multimedia format has yet to achieve the level of interconnection and manipulation achieved for text documents via the World Wide Web and so realize the exciting opportunities for learning that can occur in "Web 2.0" and "Social Software" environments.
The provision of synchronized text captions (subtitles) and images with audio and video enables all their communication qualities and strengths to be available as appropriate for different contexts, content, tasks, learning styles, learning preferences and learning differences. Text can reduce the memory demands of spoken language; speech can better express subtle emotions; while images can communicate moods, relationships and complex information holistically. Deaf learners and non-native speakers may also be particularly disadvantaged if multimedia involving speech is not captioned.
The planned MACFOB software aims to achieve the following:
A prototype annotation system that meets these requirements has been under development at Southampton to add to the synchronised multimedia captions, images, and slides demonstrated using ViaScribe, the facility for users to also create tags and notes and bookmarks and links . The interfacing of other speech recognition engines (e.g. Dragon, Microsoft) with the system is also under investigation. The system design would allow the future incorporation of other specific types of annotation such as audio, video, animations, etc. However these were not identified as a current user need and can also be achieved with the prototype system through linking to external files. The system has built on the user interface knowledge gained in developing a real-time editing system that enables an operator to select and correct errors in the speech recognition captions as they are displayed. MACFOB will build on this prototype and the experience with ViaScribe to allow users to search, browse and add synchronised annotations (notes, tags, bookmarks, images/slides, links, and captions) to recordings and any existing user annotations.