The ability to distribute interactive networked multimedia services depends upon having a suitable medium for authoring and delivery of content. When this project began, there were few tools for authoring in an Internet environment that allowed adequate Quality of Service (multiple streams of real-time, bandwidth-intensive media, e.g., good quality motion video with synchronized sound). ScriptX appeared to be the most promising authoring environment for developing networked interactive multimedia for the World Wide Web (WWW). ScriptX (Version 1.0) from Kaleida Labs, Inc. was available for the Microsoft Windows and Macintosh platforms. It combined the ability to create scripts which were compiled into platform independent bytecode that could be distributed and executed by the Kaleida Media Player (KMP), a virtual machine or software interpreter for pre-compiled "machine-code"[5]. ScriptX also included a library of multimedia tools such as timers, drawing tools and external media importers for bitmaps, audio (AIFF and WAV), and video (Quicktime and VFW). The WWW and ScriptX each had limitations with respect to four requirements for networked multimedia: Data Flow, Synchronization, Media Integration and User Interactivity. However, combining the ScriptX platform with the WWW to create an authoring environment for networked interactive multimedia leveraged the features of both architectures to overcome their individual limitations. [13]

Accomplishments

MIT/CAES and Dartmouth/IML each developed ScriptX-based authoring environments for creating interactive multimedia services and then used the resulting tool kits to produce and deliver prototype LAN based interactive multimedia services deliverable via the WWW.

At MIT/CAES, work began with the development of the tutorial "Introduction to Machine Tools". This effort provided a basic demonstration of WWW capabilities to deliver video materials over the network. There were graphics, sections of video and hypertext links to short video clips (about 30 seconds long). This early work led to the conclusion that more sophisticated authoring and delivery capabilities were needed.

The next MIT/CAES effort was a prototype ScriptX-Web authoring environment which included extensions and enhancements to provide additional capability including: background fetching, non-linear synchronization and orchestration, both static and dynamic, and specially formatted text-based media. A sample interactive tutorial was developed in the prototype authoring environment to demonstrate the successful implementation of the usage of both the original ScriptX features and the extensions. The sample course was developed at CAES using materials taken from the video course "A New American TQM: Revolutions in Management." [14]

The Dartmouth/IML team began their experiments in interactive networked multimedia by modifying their in-house authoring language, 5L, from DOS/Laserdisc (analog video) to Windows/Macintosh/digital video (CD-ROM and networks using QuickTime-wrapped Cinepak encoding). A wide range of combinations of video window sizes, frame rates, and data rates was examined to optimize training impact vs. hardware and network limitations. A program originally developed for DOS/Laserdisc was converted to Windows/Mac, using content-sourcing from CD-ROM and local area network. Results were very positive for CD-ROM; network-based delivery of content was deemed promising, but not yet adequate for use health care training due to long latencies, interruptions in video stream, and loss of synchronization between audio and video. This effort also provided experience with digital video production, including digitization, use of green-screen videography with video/graphics compositing in post-production, non-linear editing, and data compression.

"Preventive Medicine in the Combat Theater" is a public health training program, providing an immersive, highly experiential "reflective practicum" in dealing with infectious disease outbreaks. It makes extensive use of motion video, navigational interfaces, audio ambiance loops, and text documents. As such, it places great technical and performance demands on supporting network infrastructure. Development of PMCT for network and CD-ROM/network use began with conversion of all video to digital formats, using techniques developed in earlier NMIS work (Cinepak/QuickTime, 1/6 screen, 15 frames/sec, data rates not to exceed 150Kbytes/sec.). All interfaces were redesigned and reauthored to make use of new graphics technologies, techniques, and tools (original graphics were raster-based geometric objects).

Given our initial experience with 5L, and heated development of net-based multimedia tools in the commercial sector, a variety of off-the-shelf authoring environments and tools was examined. As a result, Director 6.0 was selected as the software environment for authoring PMCT, extended with a variety of off-the-shelf and custom plugins. NetShow was used to stream video from a Silicon Graphics Origin 200 media server and a NetPower Windows-NT server on a local area network. The SGI server in this test used HTTP and the NT server used UDP. In both cases performance was considered excellent with regard to video quality (good resolution, no interruptions in service) and audio synchronization; latency was considered tolerable. This performance was sustained as long as network traffic was low and bandwidth was sufficient for video streaming (ca. 150KBytes/sec. - determined during video encoding). The PMCT prototype also runs on CD-ROM with excellent performance. It is likely that Director 6/NetShow will provide the functionality needed to deliver PMCT in a wide-area-network environment, and it is clear that it can do so using network/CD-ROM hybrid approaches.

Future Directions

In recent months there has been intense activity in development of these tools within the private sector, including commercially-available authoring environments that allow Web-based delivery of courseware and tools for streaming motion video and/or audio. Having explored many alternatives, the health care component of NMIS has settled on development using common Web development tools (enhanced HTML), Macromedia Director 6.0, and Microsoft NetShow for streaming media. Additional software extension "plugins" for Director will be acquired commercially or written by IML in C++ as needed to extend the functionality of Director. In general, media will be "wrapped" in QuickTime, using Cinepak and/or MPEG I codecs. In addition, a variety of other media formats will be used, appropriate to the media concerned (audio, graphics). This solution is likely to be widely applicable by a broad range of developers having varying levels and types of skill, expertise, and authoring and development tools. It is also likely to integrate well with commonly available Web development tools.

As a result of expanded scope of work, the health-related section of NMIS will continue through February, 1998. Dartmouth/IML will develop an initial implementation of a prototype Digital Multimedia Distance Learning System (DLS), containing the following major elements: (1) A generic world-wide-web front end giving end-users access to all training material available in the digital library (in collaboration with, and providing close support of, ongoing development at ATSC); (2) software infrastructure and APIs necessary to deliver interactive media (including motion video and audio streaming, graphics, and text) delivered in real-time over the network, with specifications for the development for future training material; and (3) an asset management infrastructure allowing content developers and training administrators to add new applications in the future in a standardized and user-friendly manner. Element (2) above can be considered a "middleware" layer that hybrid multimedia learning applications will be built upon, to ensure operability with the library server hardware and software platform. This will consist largely of commercially available client and server software elements; however we will acquire or develop appropriate software "wrappers" as required to make these elements inter-operate as an integrated environment. At the same time, there is a need to avoid proprietary or platform-specific solutions (especially on the client side), and to allow for future expansion and "openness" of the software architecture.

As an example, the specification would state that training applications making use of streaming video be allowed to make use of MPEG, QuickTime, or NetShow media formats, and the server be capable of delivering such streams using appropriate network protocols. Other application and media formats based on commercial products such as Macromedia Director may also be included in the specification as appropriate and feasible. Another aspect of the middleware infrastructure is client and server-side support for automatic network/CD-ROM hybrid or broadband (network only, no local CD-ROM required) delivery selection, and automatic download of appropriate client applications and helpers on session startup. PMCT will demonstrate methods of courseware integration using this middleware layer.

PMCT will be redesigned so that it can link to the DLS and a Web-site dedicated to the program, in three ways: 1) automatically (without user intervention) to download updates to the program or its content, 2) user may link to the Web-site without leaving the program, and 3) exit the program temporarily to access the Web via a standard browser (e.g., Netscape Navigator, Microsoft Internet Explorer). New software tools will be adopted, adapted, or developed as required, initially fully to implement the new program, later to refine it and optimize DLS performance, whether content is delivered via optical disc/Internet hybrid or network only (broadband service).

The program will be developed so that it will adjust automatically to available bandwidth: dialup at 28.8 KBPS (or greater) and broadband. If the former, the program will automatically gain access to a Web-site dedicated to the program; query for, and download, any updates to the program or its content, which will be cached on the hard disk of the client computer system; the program will then automatically merge data streams (from local hard disk, local CD-ROM or DVD-ROM, and network/modem) as required to present the program's content to the user. If access is via broadband connection, all content is accessed in real time from an appropriate server on the network; there is no requirement for local CD-or DVD-ROM.

A prototype DLS Web-site to support the PMCT program will be developed, using appropriate combinations of commercial and in-house software development tools. The Web-site will provide essential functionality in primitive form: the ability to be linked to a program and multimedia content delivered on CD-ROM. The program may link to the Web-site in the three ways outlined above. Following this initial development phase, the PMCT-related prototypes will be refined and a new application will be developed and added to the DLS. This application will take a middle ground between the "high-end," video-intensive exemplar (Regimental Surgeon) and programs that are essentially direct, "page turner" conversions of existing text documents. As such, the program will serve as a model for effective use of interactive media appropriate to development teams of moderate capacity, working with moderate resources. This program will be developed in conjunction with the Army Training Support Center to promote transfer of technology from Dartmouth and the NMIS project to Army and National Guard development efforts; the extent of this collaboration, and consequent transfer of multimedia development methods and techniques will be driven by the availability of ATSC developers, who may have competing commitments.

We will continue to develop the middleware software infrastructure specifications, beyond what is implementable in the prototype. Such development will look toward the future and will leverage existing or future, de facto and ordained, open standards in multimedia content delivery. We will identify deficiencies in current software offerings and outline specific areas where additional software development would be necessary for facilitating interoperability of existing standards with the ATSC server hardware/software platform and/or supporting emerging standards.

The work will also involve a detailed evaluation of trends in technology development and deployment that may affect use and effectiveness of the DLS; these include speech recognition and immersive environments (visual, aural, tactile). Once developed, the completed DLS, hybrid and network only, will undergo a cycle of evaluation-refinement-reevaluation. There will be two types of evaluation, end-user (formative and summative evaluations) and technical (system performance). The Department of Instructional Technology, School of Education, University of Georgia, has been subcontracted to perform the former.