CA2260993A1 - Non-linear editing system for home entertainment environments - Google Patents

Abstract

A non-linear editing system for home audio and video applications includes a compression/decompression engine, a high capacity storage device and a media editor that provides point and click audio and video editing functionality, including recording, playback and special effects, such as real time gamma correction, color effects, 2D effects and real time fades, using a time-line system. The compression/decompression engine includes electronic circuitry designed to implement high speed data compression and decompression using JPEG, MPEG, or wavelet techniques. The high capacity storage device typically comprises internal and external non-linear magnetic storage devices, such as Enhanced IDE or SCSI hard drives, although other non-linear storage devices, such as magneto-optical or optical disk drives may be used. Similarly, the system includes input/output (I/O) capability for video in composite NTSC or PAL formats including SVHS resolutions, analog and digital stereo audio in 16-bit CD format, multimedia inputs, such as clip art and synchronized audio/video, from CD-ROMs and DVDs, a Musical Instrument Digital Interface (MIDI) and Internet connectivity through strandard telephone lines using a modem and through cable TV lines. The system does not require the use of a computing device, such as a personal computer, to perform its non-linear editing functions.

Description

W 098/06098 PCTrUS97/13055 Non-lineu Editing System for Home Entertainment Environments Bc k~ r, ' of the Invention Field of the Invention The invention relates to non linear editing systems and, more particularly, to consumer systems for the storage, editing and retrieval of audiolvisual ;"Il,rl ~; n Description of ths Related Tq I v-Linear video editing systems, such as those for videotape and photoy, r~- film, are old in the art. In contrast, current personal computer (PCI systems, such as those of the Apple6"1,~ 'G~hC and Intel~ a,ul,;l~ .,d, offer non-linear video editing systems. Non linear editing on computer oriented systems involves digitizing analog media data recorded from a linear source, such as videotape or film, and storiny the digitized media data on a storage device, such as a magnetic disk drive. Once digitized, the r !, - editing system permits rapid access to media data at any point in the linear sequence for subsequent Illdn, _' 'iun of media data portions into any order.
For example, non linear editing systems enable the combination of audio clips with other video clips and the formation of new clips using portions of other clips.
The non-linear video editing capability typically resides in a plug-in card for the NuBus or PCI t r ~ - bus of a M? Iosl, architecture PC or the ISA, EISA or PCI e.., ~ bus of an Intel a(L' ~ PC. These non-linear video editiny systems typically use compression i ' . - d ~ d by the Joint Photographic Experts Group ~JPEG) or the Motion Picture Experts Group (MPEG). For example, in U.S. Patent No. 5,577,190, Peters discloses a media editing system that receives, di~itizes, stores and edits audio and video source material, using JPEG compression, for later manipulation by a computer, such as an Apple Macintosh. Similarly, in U.S. Patent No. 5,508,940, Rossmere, et al., disclose a multimedia random access audiolvideo editing system including a main control unit that receives data and commands over a SCSI bus from a personal computer having an analog llO board coupled to audio and video processing boards using JPEG col"~,r. Moreover, Reber et al. disclose a system and method for the " t of media data in a non-linear editing system that enables dynamic linking of digitized media data with a specific source reference at run time in U.S. Patent No. 5,584,006. Lastly, in U.S. Patent No. 5,438,423, Lynch, et al., disclose a system and method for continuously storing the video content of a program, using JPEG or MPEG
con"." in a recirculating random access buffer having sufficient capacity to store a - ~ -t~ ~ti~ portion of the program.
Unfortunately, consumers currently have no cost effective alt llali.~S for enhancement of their camcorder movies and di~ital pictures without having to incur substantial costs to purchase a personal computer with a high ,.s ' I n computer ~qraphics monitor, assoc;alLd add-in cards and software for non-linear editing. In addition, , ' non-linear editing systems are designed for expert users, such as a, ull ' movie editor, to edit a larae number of unrelated movie clips stored on the same linear film or videotape. Thus, conventional non linear editing system tools are complex and require a high degree of manual t~ a~,; and CG.Ifi~, alion to produce a final edited result. In contrast, consumers often capture closely related events, such as vacations and birthday parties, W 098/06098 PCTrUS9711305 on videotape using their ce-~ovd( ~. To edit these camcorder tapes, consumers require easy to use non-linear editing systems that facilitate editing without a high degree of computer or editing skill and time spent to configure plug-in cards and software. Similarly, . ~a 1,~5 currently have no viable, cost effective means for incorporating non-linear editing ~",cli r ~ily into their home entertainment components because currently available non-linear editing 5 systems are speci~i 'Iy adapted for personal computer plug-in a,ur' :ions and functionality, instead of home cate,i ""ant er ,-rf ls. ComiG.,I r,;~or,! - editing systems, such as Adobea' P~ , provide user interfaces designed for rendering on high l~sc': - nor i.,lb,ldcLd computer graphics monitors. Although computer graphics monitors are viewed from a distance of one to two feet, cGr,sll"~ must often zoom-in on portions of the user interface to perform non linear editing functions. In Gontrast, ccr,~u~l n~ ' television sets having low resolution 10 i"l~llaced displays, co~ . Drly available in home entertainmentc..~i,u" ~ "s, render poor quality images of these user interfaces designed for high 1~ s ' i r graphics monitors. To . , ,d matters, consumers often view their television sets from a distance of several feet, where a poor quality rendition of a user interface severely impedes its use.
Consumers require a nor. ! --- editing system adapted for use with LGm~c.~ ' television sets.
Lastly, current personal computer based non-linear editing systems are sl,eci~i -'Iy adapted to an RGB (red, 15 green and blue) color space used for r i l~,ldccd computer graphics monitors or CRTs. Ilowever, RGB is a poor choice for ,ep;, !.9 real world images because equal bandwidths are needed to describe each color -l 1, -r ~1 while the human eye is more sensitive to luminance Ithe Y or black and white -~, t) than color u""pc. ts (the U and V c pr JrS). Similarly, equal b..-~ 11 provides the same pixel depth and display ,l~s-'l i for each coior {o, ~.,1, which dldll,a.iL lly increases the cost of non-linear editing systems due to the need for expensive 20 dual port video RAMs, instead of low cost s~. ~". qu DRAMs, to support the high t. 'v~;dlh " ~ Is.
Moreover, G.~,..i ' non-linear editing systems increase system cost by using additional dedicated processors, such as audio DSPs or the CPU of the personal computer, for audio editing functions. In summary, consumers have a substantial unmet need for cost effective, non linear editing systems for home audio and video arr'il elions that do not require the use or purchase of an e.~ n,;.;.~ personal computer.
Summarv of the Invention The present invention provides an ~ Jn . -' solution for the i s ~s dG and editing of hy~.~.",ad;d, such as motion pictures, music, animation and photographs, into a wide variety of present and future platforms. By eliminating the need for â costly personal computer, the present invention enables the :. or,~. dliur, of cor,~anl home entertainment components, such as VCRs, ca".cGrdl ~ and compact disc players, into an economical, stand 30 alone, - - 1 J, h,~ ,.-,li,. editing system. The present invention allows co" ~ to capture h1".,,1' from real-time on-line sources, such as l~?dl-: radio and t~ pay per view cablelsatellite television services and the World Wide Web portion of the Internet, as well as off-line sources, such as video cassette tapes, laserdiscs, DVDs and compact discs. Analog hypermedia is digitized and may be compressed for storage. Consumers may replay the captured hypermedia in addition to s~,'~,..li.~,ly capturing and manipulating hypermedia portions, or clips, using the 35 graphical user interface (GUI) of the present invention. Captured clips appear as icons on the GUI and GOr .
may combine captured clips by manipulating their It;,pL.,Ii.~ icons to effect a wide variety of editing ~L. _ . ~. such W 098/06098 PCTrUS97/1305S

as fades, dissolves, wipes, and animated effects. Moreover, consumers may also use the point, click, drag and drop functionality of the GUI to integrate captured clips onto a timeline to form a motion picture clip. In a similar manner, consumers may edit and ;I,cG.r ale still photographs, audio, text and other data for r ~r ~l into a clip. Lastly, the present invention provides for a wide variety of output mediums for the edited results, such as television sets, ~ 5 color printers, videotapes, OVOs, computer displays and audio speakers.One aspect of the present invention includes a non linear editing system comprising a remote control to provide command signals, a main unit, 1~ pc .e to the command signals, having a r~n' - editor to receive, access, and edit hypermedia and a direct access storage device to capture the hypermedia, and an output device in communication with the main unit to receive and play the hr~.~..l"Edia.
Another aspect of the present invention includes a non-linear editor comprising a bus; a processor, in c- , -i ~- with the bus, to control access to the bus; a memory in con ~ I with the pluGes~ol~ a compression engine, in communication with the bus, to compress and decompress hypermedia; a storage, in i - with the bus, to capture and provide direct access to the ca , bssEd hl~., ' and a media editor, in communication with the s ., I r engine and the bus, the media editor and the compression engine receiving the hypermedia, wherein the media editor manipulates the hypermedia.
Yet another aspect of the present invention includes a method of operating a non-linear editing system comprising the steps of capturing hypermedia, automatically loading the plt~ captured h~ and providing a storyboard to manipulate the, b~'"L Jy captured hypermedia.
Lastly, yet another aspect of the present invention includes a method of editing hypermedia in a non linear editing system comprising the steps of capturing a plurality of hypermedia portions to a ~ ba- d and autL i 'Iy providing a transition between a pair of hypermedia portions within the plurality of hypermedia portions on the storyboard.
Brief Descri~tion of the Drawinas Figure 1 is a block diagram illustrating an example of an environment for practicing the present invention.
Figure 2a is a front side elevational view of an embodiment of a remote control of the present invention.
Figure 2b is a top plan view of the remote control shown in Figure 2a.
Figure 3 is a block diagram illustrating the major functional units of the remote control shown in Figure 2.
Figure 4a is a front side ek,~ài' ~I view of an embodiment of a main unit of the present invention.
Figure 4b is a cutaway front side cl~ ~~' view of an embodiment of an open a lcl panel of the main unit of Figure 4a.
Figure 4c is a rear side elevational view of the main unit shown in Figure 4a.
Figure 5 is a block diagram illustrating the major ~1 : n-' units of a main unit of the present invention.
Figure 6 is a block diagram illustrating the al~ of a media editor of the present invention.
Figure 7 is a flowchart illustrating a method of operating the non linear editing system of the present invention.
Fi~ure 8 is a ~'1. ' l 111 t.ai ~ a method of manipulating hypermedia of the present invention.

.. . ..

WO 98/06098 PCTrUS97/1305 Figure 9 illustrates an embodiment of a graphical user interface for the capture of h~
Figure 10 illustrates an embodiment of a graphical user interface having a storyboard.
Figure 11 illustrates an embodiment of a graphical user interface for the creation of a transition.
Figure 12 illustrates an embodiment of a graphical user interface for the creation of graphics.
Figure 13 is a task flow diagram illustrating the tasks a consumer can perform using the graphical user interface of the present invention.
Figure 14 is a diagram illustrating the display flow of the graphical user interface of Figure 9.
Detailed DescriPtion of the Preferred Embodiments The following detailed dml i, liun of the presently preferred embodiments presents a desc,i,ui .. of certain 10 specific embodiments to assist in ulld~..al ~'lil9 the claims. However, one may practice the present invention in a multitude of different embodiments as defined and covered by the claims.
For co".~ -e the description comprises four sections: the Non Linear Editing System, the Media Editor, Operation of the Non-Linear Editing System and Summary. The first section describes the nDa ' ,ear editing system of the present invention, the next section provides an overview of the media editor of the present invention, the 15 following section describes the operation of the nonlinear editing system and the remaining section s~ ,a,iL~s advantageous features of the present invention.
The term h~,u~ refers to the hlll,.JIL - of text, graphics, sound, video, and other data, or any combination into a primarily assoc;ali~ system for information prt~"tal n, storage and retrieval. For example, h~ ,l"~did includes motion pictures, music, animation and photographs. Hypermedia ~...;I~. Is enable users to 20 make assa;aliuns between tûpics. For example, a hypermedia; rt~ i' on navigation may include a55oc;ai , or links, to such topics as aal~. lQ t, bird migration, geography, satellites and radar. Multimedia, the combination of sound, graphics, animation and video 1~ ,lla6ùn, is related to h~ .l ' in that hypermedia combines the elements of multimedia with a ;uc ' to link the information.
I. Non-Linear Editing System Figure 1 illustrates an C,l~ I tor practicing the present invention. A non-linear editing system 100 of the present invention communicates with a network 102. Network devices may include computing devices, cable and satellite TV tuners, Web television sets, wireless phones and hllu.l~ài kiosks, among others. Computing devices communicating with the network 102 may include clients, such as a network computer and a mobile computer 110, and servers 112, such as a wide area network 114 including a plurality of local area networks 116, each having assoG;dlod computing devices. To obtain h~ ,dia, such as television programming and Internet data, network devices communicate with the network 102 using a network c .ac~liun 104. Network Coll..~t ~5 104 may include wired links, such as those c~ 1LI~,d to the public switched t~'~ph e network (PSTN) and a cable television provider, and wireless links 106, such as cellular phones, mobile computer cellular digital packet data (CDPO) modems and satellite dishes 108.
The non-linear editing system 100 includes an output device 120, such as a home television set, a main unit 122, in communication with the output device 120, providing the non linear editing functions and a remote W O 98/06098 PCTAUS97/1305 ~5-control 124. The non linear editing system may also include hypermedia recording devices 126, such as a video cassette recorder (VCR), video camera, camcorder or digital video disk (~VD) recorder. The remote control 124 communicates all editing and playback functions to the main unit 122 for display on the output device 120, L~l.. "~
using wireless l~ hni, s, such as infrared or radio waves. However, the remote control 124 may also communicate 5 with the main unit 122 using a wired cr- P 6uil.
Figure 2a illustrates a front side view of one presently preferred embodiment of the remote control 124.
Referring now to the top view of Figure 2b, the remote control 124 includes a keyboard 130 having a plurality of keys or buttons to communicate editing functions to the main unit 122 (Figure 11. The keyboard 130 provides for 9f .. .. of alphanumeric Lha~acle,s, such as those used in home video titles, flying text, logos or graphics. The remote control 124 may also provide special keys 132, such as cut, paste and help buttons, and a pointing device 134, such as a mouse, touch pad, track ball or light pen. The pointing device 134 provides for control of the on screen cursor on the viewing device 120 (Figure 1) to select, scroll or move along a selected path. Moreover, the remote control 124 may also include a; vl ' 1l1., wheel 136 enabling rapid selection of an individual frame of a motion picture. In a preferred embodiment, the remote control 124 includes a con6~ .'1 device capable of operating as a pointing device 134 and a ioglshuttle wheel 136. Using software, a user may configure the pressure sensitive buttons 134, 136 of the co..';~, " devices to operate either as a pointing device or as a jogtshuttle wheel. When operating as a pointing device, the on screen cursor acce!~..di )~ is P~r crli"r~dl to the pressure of user exertion on the button 134, 136. Advantageously, a configurable device enables proficient operation of the pointing device and the; ~' h~ wheel by both left handed and right handed users. Lastly, the remote control 124 may 20 accept power from a portable power source, such as ll,.,hal~ '' and nona~cha,b "e batteries, as well as AC
power from a wall socket.
Referring now to Figure 3, a block diagram illustrates the major functional units of the remote control 124.
The remote control 124 includes a keyboard processor 142, such as the UR5HCSPI keyboard controller available from USAR Systems, which receives power from a power system 146, such as a battery or AC power source, and ,c;cce s electrical signals representing user commands from a keyboard 144. The keyboard u,uce.. ~ùl 142 may also accept electrical signals ,., ~O,Ili,.s user rr ~ '~ from a pointing device 148 and a ioglshuttle wheel 150.
Lastly, the keyboard ,c oc sDr 142 may provide its output signal to a wireless transmitter 152, which provides a wireless signal for communication with the main unit 122 (Figure 1).
Figure 4a illustrates a front view of one presently preferred embodiment of the main unit 122 of the present invention. Figure 4b provides a cutaway view of an embodiment of an open -~ : panel 160 of the main unit of Figure 4a. The connector panel 160 includes a video signal input 162, such as an RCA ,~.cL,.lacl~ and a S video receptacle, an audio signal input 164, such as left and right channel RCA ,~ceplaLk~3~ a m;~"., ' Dre input 166 and a headphone input 168 having a volume adjustment. Lastly, Figure 4c illustrates a rear view of an embodiment of the main unit 122. In a preferred embodiment, the main unit 122 includes a SCSI port 172, an RS232 serial port 174 and a parallel port 176. The main unit 122 may also include a Universal Serial Bus (USB) ce ~ lor, a PC Card c nn lor and an IrDA compliant infrared port for communication with with various network devices. Moreover, the W 098t06098 PCTGUS9711305 main unit 122 p,~fel3hl~ includes input and output cir -~:ora for coaxial cable 178, a co, Q 1~VidQo input and output connoclor 180 for video signals from a VCR or video recorder, S video input and output conno~,tol~ 182, leftlright channel audio output connectors 184 for a television set, leftlright channel audio output connectors 186 for a VCR, and leftlright channel audio input ce i 5 188 for an auxiliary audio component, such as a compact 5 disc player.
Referring now to Figure 5, a block diagram " : dles the arch;lu.,lul~ of the non-linear editor 200. The non-linear editor 200 receives hypermedia input 202 from a variety of on line and off line sources in a wide variety of formats. For example, the no.. ! ~ editor may receive separate analog YUV video signals and stereo audio signals from a VCR or multiplexed MPEG data paGkets from a digital satellite ~ .. Thus, the non-linear editor 200 may include an optional decoder 204 having an analog to digital converter to convert the hypermedia input 202 into CL r-t ~ digital data formats, such as CCIR 601 and CCIR 656 compliant bit streams or serial digital audio bit streams, for editing and storage. For example, the decoder 204 may convert encoded Sl~ldl M ' ~ analog signal into decoded descrambled compatible digital data formats having separate audio and video cl,~"prr l~ for suhsequent use. Similarly, the non linear editor 200 may include an optional encoder 206 having a digital to analog converter to convert compatible digital data format into a hr~ output 208. For example, the encoder 206 may convert a compatible digital data format, such as a 24 bit RGB color bitmap file, into a h~l "~ d output 208 including an analog YUV signal for an ;..I~,laced television display with a separate s~,-' uri~ d analog stereo signal for home speakers or headphones.
The non linear editor 200 includes a media editor 210 in coml il _i with a compression engine 212, which compresses and decompresses data, a bus 214, and an optional media buffer 216 for temporary storage. The non linear editor 200 likewise includes a pr.ress~r 218 in r ., cliùll with the bus 214, a memory 220 and a storage 222. The media editor 210 and the compression engine 212 simuli ~~ '~ receive the h~,u~"u. ' input 202. In this manner, the compression engine 212 compresses and transfers the hypermedia input 202 through the bus 214 for storing in the storage 222 while the media editor 210, which performs non linear editing functions, -d~a,.l geoL ~ provides for the output and !- ~ ~C~r of the h~prrl ' input 202. The processor 218 administers access to the bus 214 and to the storage 222 and uses the memory 220 as a local storage area for program ;nsl,u~,liuns and as a buffer for data. Thus, the media editor 210 coml ~s with the processor 218 and the storage 222 via the bus 214. The processor 218 also supports comm~ atiun le,faces (not shown~, such as SCSI ports, serial ports, parallel ports, IrDA compliant infrared ports, USB ports, Musical Instrument Digital b,lL.IaceslMlDI~, PCMCIAIPC Card l~,fac6s~ IEEE 1394 Firewire and Smart Card illlu,~ ees for additional network and hypermedia devices.
In one presently preferred embodiment, the compression engine 212 provides data compression and decompression using wavelet technigues. In another preferred embodiment, the compression engine 212 provides data compression and decompression using discrete cosine llall~f~ IDCT) t ~r' , ~ s such as JPEG, MPEG 1, MPEG 2 and MPEG4. Similarly, the p,.ce 218 p,ef-,;F"~ comprises a cDr.3,.; - ' embedded controller, such as a Hitachi SH7032, IBM PowerPC 403 or an Intel i860, although a general purpose m;",o, uctsior may be used. In addition, the memory 220 may ,l~e~.abl~ be embodied as one or more of various types of E!~ L. memory, such as DRAM, SRAM, ROM, EPROM, EEPROM and flash memory. Similarly, the storage 222 may, ~ .tly be embodied as one or more of various types of direct access non volatile storage devices, such as magnetic, magneto optic and optical disk drives. As is well known in the i ' Ir' ~r, the bus 214 comprises a set of cl k to,~, or wires, for the transfer of data and addresses among the compression engine 212, the media editor 210, the processor 218 and the storage 2Z2. In one presently preferred embodiment, the bus 214 operates at 33MHz and transfers 32 bits of ;"fo" in each cycle.
The dual paths of the hypermedia input 202 enable simultaneous viewing and editing of l"fl, di~ during capture. For example, a hypermedia input 202 comprising an MPEG compressed digital video bit stream is captured 10 directly to the storage 222 by transfer through the bus 214 without . ~ ' ~ication by the compression engine 212 since the ~"~,e"".,Jia input 202, an MPEG bit stream, is already c ., t~scd. In parallel, the media editor 210 concurrently receives the hypermedia input 202, determines the hypermedia input 202 is a o ~ scd MPEG bit stream, and communicates with the compression engine 212 to decompress the MPEG bit stream for editing and output to an output device, such as a television set. In contrast to ccn~.,.,t; ' personal computer e~r ~ )r bus 15 devices, the dual paths of the present invention eliminate the need to transfer large amounts of data from the storage 222 through the bus 214 to the media editor 210 during hypermedia capture. As a oo ~, nce the dual path design likewise reduces the need for additional memory, bar,d~, ;"' and, I,cessor computing power, thereby reducing system cost substantially.
Il. The Media Editor Referring now to Figure 6, a block diagram illustrates the architecture of the media editor 210. As d; ,- csPd above, hypermedia input 202 IFigure 5) is simultaneously compressed to the storage 222 while streaming to the media editor 210 via a video controller 252. To prevent displayed images from jumping when viewed, the video controller 252 s;, h.~n;.ei, the video output signals i ', ' .ll~ of the input video signals. Thus, the input h~r lal and input vertical ~ liua signals fed across lines 254, 256 are not used to control the output 25 h i~onldl and output vertical synchronization signals fed across lines 258, 260. However, ~, d I frame synchronization often results in a "tearing" distortion of unmanipulated, or pass through, video images. The present invention eliminates "tearing" distortions through a design whereby hypermedia input 202 and hypermedia output 208 share the same master pixel clock 250. Thus, the media editor 210 produces a stable video output from the video portion of a hypermedia input 202, which may include unstable and distorted feeds from multiple, 1 ~' ~~ d, 30 real-time video sources.
The video controller 252 provides the video portion 262 of the h~ ' input 202 to a programmable low-pass filter 264, such as a Faraday FL676 or a Micro Linear ML6420, to remove high G~, 1 aliasing components so as to form a filtered video signal 266. In turn, the filtered video signal 266 is fed to a programmable video scaler 268, such as a Phillips SAA 7140A or SAA 7186. The filtered video signal 266 improves the output 35 quality of the video scaler 268, which enables resizing of a video image frame to a selected size for viewing and editing. In contrast to the RGB color signals of the prior art, the video scaler 268 outputs a scaled YUV video signal .. _ , ... .

W098/06098 PCTrUS97/1305 ~ r i~i,,g a luminance signal 270 and two color difference signals 272, 274. A gamma corrector 276 receives the signals 270, 272, 274 and performs gamma cG"e~ r for the non linear illumination of cathode ray tube (CRT) displays in addition to various color effects, such as posl~ alion, negative imagery, tints and en' ~ .~,.uents. In a preferred embodiment, the gamma corrector 276 is implemented as a lookup table in semiconductor memory. The resulting gamma corrected signals 278 are fed to a first-in, first-out lFlFO) buffer 280, then to an input frame controller 282, which assembles gamma corrected signals 278 into a video frame 284. A memory controller 286 receives and transfers the video frame 284 to the media buffer 216 (Figure 5) for rapid access by the media editor 210. In contrast to CGIl~ ' memory c ~I' s for PC based non-linear editing systems, which access multiple devices using a wide variety of fetches, such as cache burst, OMA, and CPU ;,,~ cliuil or data fetches, the memory 10 controller 286 of the present invention is ~ c~e. ably optimized for multiple s~ lidl fetches of large memory blocks.
To perform a transition between a first and a second sequence of frames using a selected first frame and a selected second frame, each representing their lcspc~,li.c frame s~ 3~, the input frame controller 282 stores the first frame selected from the first frame sequence in a first location of the media buffer 216. Similarly, the input frame controller 282 stores the second frame selected from the second frame sequence in a second location 15 of the media buffer 216. In response to the consumer's input, the non-linear editor 200 (Figure 5) creates an alpha frame, which describes how to combine the first and second frames so as to form the llc~l1sil n, and stores the alpha frame in a third location of the media buffer 216. An alpha frame is a video frame including an alpha value for each pixel, instead of a pixel value ,l, esc"li.lg a color in a co,,~c,,lional video frame. The alpha value defines a mix level between Cl,." pr ' g pixels of the first and second frames. A SCRAM engine 288 retrieves the first 20 frame, the second frame and the alpha frame from the media buffer 216. Upon retrieving these frames, the SCRAM
engine 288 forms a transition frame, on a pixelated basis, by combining the pixels of the first, second and alpha frames according to the equation:
Transition Frame pixel ~ [(First Frame pixel * Alpha Frame pixel) + Second Frame pixel * 11 -Alpha Frame pixel)I, where each pixel in an alpha frame is normalized to a value between zero and one inclusive. Lastly, the SCRAM
engine 288 provides the transition frame to the memory controller 286. The memory controller Z86 may transfer the transition frame to the media buffer 216 lFigure 5) or to the compression engine 212 (Figure 5), which compresses the transition frame data for subsequent transfer to the bus 214 (Figure 5) for capture in the storage 30 222 (Figure 5). In a preferred en bo~ I, the media editor 210 includes the Applied Magic Video Fl"cessol.
A consumer may set the pixel values for any of the first, second and alpha frames equal to a constant, indicating a single color for the entire frame. In response, the SCRAM engine 288 does not retrieve the frames, but instead uses the specified constant values to calculate the transition frame. For example, the transfer of an alpha frame typically requires a 10 M]9 b ~ SEC bandwidth and the transfer of a first or second video frame typically 35 requires a 20 '' grbytelsec bandwidth. To reduce bandwidth and media buffer memory, the SCRAM engine 288 uses a single constant value for Ili ri'inn~, such as a dissolve or fade between a first and second frame , ,e, requiring the same constant value for each pixel of the alpha frame. Thus, instead of using 10 ~0$~'1yte~1~c,, of W O 98/06098 PCTrUS97/13055 .9.
b? h ;'11l to transfer alpha frame data, the SCRAM engine 288 uses only 30 byteslsec for NTSC video or 25 byteslsec for PAL video. Similarly, for a fade to black, the SCRAM engine 288 uses single constant values for both the alpha frame and the second frame to avoid the use of a further 20 ~ arbl, Ltsl~dr. of b3rd~ ;dlll to transfer pixel data for both the alpha frame and the second frame. Moreover, in a preferred embodiment, ~ in the b3r ~ ;'lh 1~ s of the present invention enable the use of a media buffer 216 IFigure 5) comprising inexpensive ORAM, instead of expensive dual port video RAM (VRAM).
To play back video l~al,~;liùns, an alpha playback controller 290 communicates a fetch request to the memory controller 286 to retrieve selected alpha frames from the media buffer 216 (Figure 5). The alpha playback controller 290 provides the retrieved alpha frames to a 'video Arithmetic Logic Unit (ALU) 292 to combine with the 10 first and second frames so as to form transition frames. In a similar manner, a first playback controller 294 and a second playback controller 296 ' ~L~ly retrieve the first and second frames from the media buffer 216 through the memory controller 286. The retrieved frames are buffered in cu"l, ~nding FlFOs 2g8, 300, 302 for ready access by the Video ALU 292. The video ALU 292 uses the retrieved alpha frames to combine the first and second frames retrieved so as to form a transition frame for output to an encoder 206 (Figure 5) or to a viewing 15 device. Moreover, for unmodified video, the present invention likewise reduces playback bE dl~;dlll and eliminates Ul 1nl e ~ y memory fetches by disabling the second playback controller 296 and the alpha playback controller 290 to free up 30 '~ga~r~ cc of bandwidth for other system uses. A similar 30 ~1 v ~I,"s~c bandwidth reduction would result from fade to black during playback. The alpha playback controller 290 enables more complex playback features, such as real-time rendering of flying or scrolling text overlays, moving pre rendered graphical objets onto 20 an image from a variety of hypermedia sources, and providing on screen controls on top of the playing video. The Video ALU 292 receives input from three real time video paths, the master pixel clock 250, an alpha path having mixing information for two full color video input paths, and serves as the primary video output of the media editor 210.
The media editor 210 also provides for the playback, capture and mixing of multiple audio channels, each 25 channel having a left and right input with individual volume controls. An audio codec interface 304 receives the audio portion of the hypermedia input 202 (Figure 5) and outputs the audio portion of the hypermedia output 208 (Figure 51. The audio codec interface 304 buffers audio samples for playback of all audio channels to an audio write FIFO 306. An audio write controller 308 transfers audio samples retrieved from the audio write FIFO 306 to the memory controller 286 for storage in contiguous memory locations in the media buffer 216 (Figure 5) or in the 30 storage 222 (Figure 5) by transfer through the bus 214. To play back captured audio samples, an audio read controller 310 retrieves selected audio samples from the media buffer 216 (Figure 5) or the storage 222 (Figure 5) by issuing a fetch request to the memory controller 286. The audio read controller 310 buffers retrieved audio samples in an audio read FIFO 312, which communicates with an audio mixer 314, wherein volume information for each channel is added to audio samples retrieved from the audio read FIFO 312. Lastly, the audio mixer 314 35 provides a mixed audio sample having channel volume information to the audio codec interface 304, which s~ ,~r~ 5 the mixed audio sample with video for output using the master pixel clock 250.

.

W098/06098 PCTrUS97/1305 The memory controller 286 manages the interface between the media editor 210 and the media buffer 216.
In a preferred embodiment, the media buffer 216 cv ~ ises dynamic RAM IDRAM) memory, instead of costly dual port video RAM IVRAMI. To satisfy DRAM timing and access constraints, the memory controller 286 provides dli ~alcd burst access to the media buffer 216 from the media editor 210. As multiple cc.,lr," a within the media S editor 210 access the media buffer 216, the memory controller 286 uses bank ;.,t~d~; 9 to improve access performance by avoiding page misses. Bank ~ a~ , exploits the performance of the DRAM memory. By allocating video frames in a selected order, the memory controller 286 can perform fetches without page misses.
One embodiment of a bank ;l~ 6,~y scheme which eliminates page misses and optimizes use of available memory b. '~ ;dlh is as follows:
1. Activate memory bank O with a row address from the alpha playback controller 290;
2. Burst read the alpha frame data while activating Bank 1 with a row address from the second playback controller 296 for second frame data;
3. Interrupt the transfer to the alpha playback controller 290 after 'x' cycles to transfer second frame data to the second playback controller 296; activate bank O again with a row address from the first playback controller 294;

4. Burst read second frame data for '2x' cycles;

5. Burst read first frame data for '2x' cycles.
Note that a bank hll,,rl,,a~; ,, scheme optimized to avoid page misses is depr d I upon the number of concurrent active memory accesses. In a preferred embodiment, the memory controller 286 locates video frame data within the media buffer 216 so as to avoid page misses when retrieving video frame data. Moreover, the memory controller 286 pdoriliL~a access to the media buffer 216. For example, when the alpha playback controller 290, the first playback Gontroller 294, the second playback controller 296 and the input frame controller 282 require realtime access, the memory controller 296 sets the access priorities so as to ensure that each controller receives its data when needed. The memory controller 286 is r ~ programmable so that the priority of access for each op.,.dE --' mode is s.,l~.tr" Lastly, due to the relatively small bandwidth for audio access as compared to video access, the audio read controller 308 and the audio write controller 310 may access the media buffer 216 at all times.
~r Jt of the Non-linear Editin~ System A c~ may use the n-l - editing system 100 (Figure 1) to capture, manipulate, interact with, transport and play back hjp ~ from a plurality of concurrent, independent hypermedia sources. Figure 7 illustrates a method ot operating the non-linear editing system 100. At state 360, a consumer initializes the system 100 by powering on the main unit 122 IFigure 1). In a preferred em~--' 1, the consumer d, ~ses the "On"
button of the remote control 124 (Figure 11. During initialization, the system 100 may perform a power on self test (POST) to determine if all of its components, such as the memory 220 IFigure 5), are functioning properly.
Additionally, the, .~ Dr 218 ~Figure 5) loads an ~ bLdd~d operating system, I"e~.dbl~ the ~' ~. ar~ OS9000 W 098/06~98 PCTAUS97/13055 ~11 for the IBM PowerPC 403, and software inal~Lclions for the GUI and non-linear editing functions from the memory 220. In a preferred embodiment, the software ;~ for the non linear editing system 100 are stored in a programmable control ROM portion of the memory 220. In contrast to prior art systems, r l-; L ly captured data is automatically loaded at state 362. Moreover, at state 362, the system 100 is preferably restored to its condition 5 immediately prior to the last power down. Adva i ~ , a consumer may proceed with all tasks in progress as if the system 100 had not been powered down. At state 364, a determination is made whether to capture new data. If new data is not desired, the system 100 provides a àl rybo7 d having p.u.;ously captured data at state 372, wherein a consumer may edit and play back the storyboard. A alolybba~ll is a ~ , lidl llr bs~.~lal of a clip having video, audio, stills, effects and other hypermedia,, er : ~d as a picture icon. The sequence runs from left to right in a top to bottom order. If new data is desired, the system 100 checks the storage 222 ~Figure 51 for available space at state 366. If there is no available space in the storage 222, the system proceeds to state 368 to make space available. Otherwise, the system 100 captures hypermedia at state 370 and proceeds to state 372 where the system 100 a..l ~ provides a sl"r~bl- d having the captured h~3~ ' Lastly, at state 374, a consumer may create a copy of a storyboard to a ,.c~ i' '' DVD or videotape.
Referring now to Figure 8, a l;~- I,hall illustrates a method for manipulating hypermedia. At state 380, a consumer captures hypermedia to a storyboard. The all,l~bc d often includes a plurality of captured hypermedia portions, or clips. At state 382, the consumer manipulates the captured hypermedia on the storyboard. The non-linear editing system 100 (Figure 1) preferably includes selected default transitions between each pair of clips dul ~ Ul,ally. For example, consumers manipulate clips by enhancing them with graphics, text and audio otatiu"~.
Consumers may likewise enhance C~L~ mail (e-mail) by excerpting, editing and attaching the edited clips to their e-mail sng Consumers also manipulate clips by compiling a sequence of clips and creating the transitions between each pair of clips. Additionally, consumers may manipulate clips by ~ ~D alin9 digitized photographs, s~ music and other forms of digital h~ captured from the Internet via the Internet browser functionality ;.,cor~ altd into the non-linear editing system 100. For example, a consumer may manipulate a clip by excerpting a video still or a digital photograph from a clip for placement on a World Wide Web page or print out as a photograph or postcard. At state 384, the consumer may modify the default transitions selected by the non linear editing system 100. Moreover, at state 386, the system 100 enables consumers to add overlays, such as graphical and audio anr,o;: r of clips. Lastly, consumers may play back their storyboards at state 388 or copy their storyboards at state 390. Consumers I efo,i ~'y copy the hypermedia rl pre"""t~d by ~L--rda to videotape machines, such as a VCR or camcorder.
Figure 9 illustrates an embodiment of a graphical user interface 400 useful for the capture of hypermedia.
The capture GUI 400 provides a plurality of tabs 402 enabling selection of various functions of the non-linear editing system 100. In capture mode, the capture GUI 400 r ~te. ' ~y includes a shot tab 404 displaying icons 406 representing, ~ captured clips. The capture GUI 400 likewise includes a display window 408 to display an image from a clip l~le,l ' by a selected icon 410. Moreover, the capture GUI 400 includes a control panel 412 displaying a plurality of buttons 414 representing capture il : such as record, play, stop and pause. Similarly, W O 98/06098 PCTrUS97/13055 the captl~re GUI 400 provides tor the selection of capture ~ r 6~ using at least one drop down menu 416. For example, using the drop down menu 416, a consumer may instruct the non linear editing system 100 to capture from a VCR to Storyboard 1 as illustrated in Figure g. The capture GUI 400 may also include a selection box 418 to specify hy~ mrdiq data types to capture as well as an entry box 420 to specify a clip name. The capture GUI 400 may additionally include a quality window 422 having a slide bar 424 and a slider 426. The consumer selects the capture quality by manipulating the slider 426 within the slide bar 424. Lastly, the capture GUI 400 includes an icon bar 428 having icons representing the different modes of the non linear editing system 100. The consumer manipulates the capture GUI 400 using the point and click functionality of the pointing device 134 (Figure 2b~ to select and -ni,: ' le items within the capture GUI 400 in coordination with the keyboard 130 (Figure 2b) to provide 10 keyboard inputs as needed.
Figure 10 illustrates an embodiment of a sl. yboa,d GUI 440. As discussed ,GI~Id . ~y, the storyboard GUI
440 includes a shot tab 404 displaying icons 406 1, t~,e"li"., p.e~; ~'y captured clips. The slD~yb- d GUI 440 likewise includes a display window 408 to display an image from a clip le~ n~cd by a selected icon 410. In addition, the storyboard GUI 440 includes a plurality of slcryl-oa,d tabs 442, each tab ~u,l,nc;~g a selected 15 storyboard having a sequence of clips with Ir- s For example, a vacation tab 444 includes a vacation storyboard 446 having six clips, cc,l.~, ding to the image icons, and five lla" 'i( ., one transition between each pair of clips. The vacation ~lul~L- d 446 includes a time ruler 448 having a graphics bar 450, a first audio bar 452 and a second audio bar 454. The bars 450,452,454 indicate the name and duration of the hypermedia portion c~,rlt~erding to the clip ll, es~"~lod by the icon above the bar. Each of the 5l~ ~bc- d tabs 442 includes a scroll 20 bar 456 to scroll down the window illustrating the storyboard CGIIl, ' 9 to the selected tab. To add a clip to a storyboard, the consumer selects the shots tab 404 and one of the si ybO~,d tabs 442 of interest, such as the vacation tab 444, to display the si ~ yb~ d446. The consumer then selects, drags and drops a clip icon 410 onto the storyboard 446 as shown in Figure 10. The non linear editing system 100 automatically provides a default transition on the slD.ybc~ d immediately prior to the newly added clip 410. Lastly, the sl rybGalll GUI 440 includes 25 a button panel 458 having a plurality of buttons or1 , ' ~, to a wide variety of editing functions for the ~iJryb~d446~ such as save, print, cut, copy and paste. A consumer may edit a ~tu~y~ d446 using the point and click functionality of the ~;lrl,bo ,I GUI 440 in conjunction with the functions c( I~_r_ ding to buttons comprising the button panel 458.
The storyboard GUI 440 preferably displays the sequence of items on a sl-.l yLGa"l instead of their absolute 30 time scales. Thus, icons of a single size represent each captured clip, regardless of its actual duration. However, other data on the storyboard ~ d with a captured clip is le~,l d by an indicator having a size and location CGIl~,: ding to the temporal l.!a ', with the captured clip. For example, an audio clip having a duration of half the length of a captured clip appears on a first 452 or second 454 audio bar under the icon of the captured clip with an indicator bar half the length of the icon. Moreover, for a half length audio clip srl ' u i71~d to begin 35 at one quarter of the clip duration, the indicator bar appears indented one quarter of the icon width from both sides, GGIIl ,)C ' ~ to a start at one quarter clip duration and a termination at three quarter clip duration. Thus, the time W 098/06098 PCTrUS97/13055 ruler 448 ot the present invention, referred to as a cartoon timeline, illustrates the e~ lial relationship of events.
Ad~,ar,~L~, Du~'y/ the cartoon timeline metaphor r' 3te~ the need for consumers to ~oom in on various portions of the~i ybur .Itoidentifyandmanipulateclips,therebyimprovingtheusabilityofas~J,~L- dGUI440. Similarly, the cartoon timeline metaphor of the present invention eliminates the need for a high resolution computer graphics S monitor as a high quality image of the ~LorybGard GUI 440 can be rendered on a cc . liunal television set.
Moreover, the cartoon timeline metaphor eliminates the need to render a minute sliver of an icon, r~"tS,!.~i v a captured clip of extremely short duration, on a low resolution television screen for consumer viewing from relatively large dii.i Lastly, in contrast to prior art systems, the cartoon timeline metaphor of the present invention illustrates the r~': r ', between associated data, such as audio and graphical l s, and the captured clips 10 through indicator bars, which are relative in nature as opposed to absolute in time.
Figure 11 illustrates an embodiment of a transition 6UI 470. As shown in the storyboard GUI 440 (Figure 10~, the transition GUI 470 includes a vacation tab 444 having a vacation sloryb. - ~1 446 and a button panel 458 having a plurality of buttons corresponding to editing functions. In the transition GUI 470, the l, - ~inns tab 472 is selected to display a plurality of icons 474, each icon ,l, res~ , a predetermined transition effect. As before, 15 to change a ~ the consumer selects, drags and drops a desired transition icon 476 on the t ybGalll 446 at a location between a pair of clips. The consumer may likewise edit the lr - :iùns of a .t rybc d 446 using the point and click functionality of the transition GUI 470 in conjunction with the functions co.,., ding to button comprising the button panel 458. Note that the ll ~r! s tab 472 includes a scroll bar to scroll down the window illustrating the icons Cb.l~, _-ding to various transition effects.
Figure 12 illustrates an embodiment of a graphics GUI 490. The graphics GUI 490 includes a graphics tab 492 having a scroll bar 494 and a plurality of icons 496 representing various graphics overlays, such as color, titles, and text on color. The graphics GUI 490 similarly includes a display window 408 to display an image for graphical editing. The graphics GUI 490 also includes an image edit window 498 having a plurality of image editing tool buttons 500, such as a line tool, a box tool, a text tool, cut, copy and paste, about the periphery of the image edit 25 window 498. As before, a consumer uses the point and click functionality of the remote control 124 lFigure 2b) to select and manipulate the image editing tool buttons 500. Moreover, the graphics GUI 490 includes a features portion 502. The features portion 502 includes an name box 504, wherein the consumer may select a name to identify the edited graphic and a plurality of tabs 506, each tab having property selections for some of the image editing tool buttons 500. For example, a color tab 508 includes selection boxes 510 to select color features of lines, 30 fills and shadows. Lastly, the features portion 502 includes a pair of slider bars 512, each having a slider 514, for the selection of color gradients. Operation of the graphics editor is similar to conventional graphics editors, such âS MacPaint and Paintbrush.
Each major element of the GUI is derived from a list of consumer tasks. Figure 13 is a task flow diagram 550 illustrating the manipulations a consumer may perform with the non-linear editing system 100. Each of the 35 illustrated tasks is intra-cyclical in nature, meaning that a consumer may perform the task over and over until satisfied with the results. Note that illustrated tasks are also inter cyclical in that a consumer may pass forward WO98106098 PCTrUS97113055 ~14-and back to other tasks. As discussed prl,. 'y, the software functionality for carrying out the tasks is located in the p.uc~ ;rr 218 ~Figure 51, which loads the software hlallucl - from the memory 220 lFigure 51. At initialization, the system 7 ~,'icP'ly loads ~.",~;olisl~ captured l,~,.e"..edid portions from the storage 222 IFigure 5) and places them into a ~lo,yLoard so that a consumer may create an edited slur~,bc. ~1. At task 552, a consumer may capture, or digitize for storage, hypermedia portions, or shots, from a hypermedia source. Ultimately, the h~.er",~dia shots are stored in the storage 222 IFigure 5). During this task, a consumer may perform some "rough"
editing by electing not to digitize selected portions that they do not wish to include in the finished master. At task 554, the consumer places captured shots on the si ybQP d. The storyboard may be thought of as a working portion of the media buffer 216 (Figure 5). As discussed p,~. 'y, shots are automatically placed into the slùryboa,J in 10 the order in which they were captured. Similarly, when shots are placed into the storyboard, 1- : 5 between shots are aul li"ally g~n~ d. At task 556, the consumer may perform "fine" editing of shots, such as moving, duplicating, removing and manipulating the duration of shots. For example, in a storyboard having 5 shots with 4 ;"l~"~,csed Lr ;itions a consumer may move the first shot to the end, whereby the system allt~ dl -'Iy removes the transition between the first and second shots and adds a default transition between the end shot and the first 15 shot. The consumer may likewise modify the Irdllsiti~.as at task 558 and annotate shots with overlays, such as graphics and audio, at task 560. Moreover, the consumer may playback the sl~ yboa,-l at task 562 or may save a copy of the storyboard at state 564. When the consumer has completed all desired tasks on this ~tl.,yboa,ll, the consumer may create another ~IDIyL~ d at task 566.
Figure 14 is a diagram illustrating the GUI display flow. The GUI is implemented by a non-linear or event 20 based software application. In traditional user interface d~c' r 11~ the diagram would represent the menu structure of the ap,:'lr liJn. However, in a presently preferred embodiment, the first layer 580 identifies the major system functions lu,~b~ led by icons 428 ~Figure 9). The second layer 582 identify further functions and features cG,,espù,,ding to the major system functions of the first layer 580. The capture function 584 provides tools to capture from a variety of hypermedia sources, such as a ca",e del 586, a VCR 588 and a digital camera 590. The 25 storyboard function 592 provides tools to edit, playback and record shots. StoryLcard tools include graphics 594, Il -: ns596 and effects 598. The paint box function 600 provides additional editing tools for drawing 602, character generation 604, video underlay 606 and still photo underlay 608. The Internet function 610 includes tools for e-mail 612 and a browser plug-in 614, wherein commercially available browsers, such as Internet Explorer and Netscape Navigator, may be ;.,lu~, al~d. A consumer may add, remove, change or olho~ e manipulate hyperlinks 30 using the browser. The setup function 618 enables configuration of the system by providing tools to indicate ~f..l ee5 620 and software installationtremoval tools 622. The extras function 624 provides extendibility tools to ~ r.: and extend system f, r~ Lastly, the help function 628 provides tools to access system help documentation.
All objects, such as shots, audio, graphics and 1~ 3r . . can be m- ~i, ' tcd by initiating a details display from the button panel 458 (Figure 101 after selecting the desired object from the ~i ybu( d. The consumer can play the edited hypermedia in a playback area or full screen at any time during the editing process, and when W 098/06098 PCTAUS97tl3055 -15 satisfied, the consumer may record the edited h~,J,I ~i; to an output device, such as a leco ,' "t DVD or VCR, by selecting the Select All button on the ~loil~bc- d display and then selecting the Master button. Thus, rather than a delivery only system for h~ " ' . the present invention allows the consumer to, for example, download h~,c,,,~EJia from the Internet and edit or manipulate it through the storyboard program.
IV. Summary The present invention provides a complete, ~ ' ' economical system that enables consumers to create, Illdll, I tl, edit, view and output hypermedia in their homes. In contrast to the prior art, the non linear editing system of the present invention does not require a consumer to incur substantial expenses to purchase a personal computer having l~--r- ' bus add-in cards and software to capture and edit hypermedia in their homes. Similarly, 10 unlike current personal c~ n,~u s, the present invention is adapted for communication with c~n~c..lional home o,.le, i . ",..l components, such as a VCR, t~k.. stereo, camcorder and digital still camera. Moreover, the non-linear editing system of the present invention provides for a wide variety of hypermedia output mediums, such as computer displays, t~h,. lr, printers, I,;Jeola~.es, DVDs, and audio speakers.
In addition, the present invention ?~V. I geously overcomes the expense and complexity of prior art non-linear editing systems by providing an inexpensive and easy to learn system for home use. Unlike cor.. ' nonlinear editing systems designed for ~ ~oPc.,sional use, the present invention I
"ai -'Iy loads prtv ~ captured data onto the storyboard for editing. Similarly, the present invention aui 1 'Iy provides default 11 r between each pair of clips on the ~I r~bc d. The present invention thus simplifies and facilitates the editing of home movies by enabling consumers to immediately manipulate captured clips, instead of undertaking complicated 20 tasks to configure the sl ~Lle d as required by ~ ' non-linear editing systems. The present invention likewise provides novel methods of data handling to reduce t '~ ;dlh requirements and leverages the availability of low cost c~ to perform the functions of the costly, speciali~ed components of the prior art. The media editor of the present invention combines the functions of various prior art software and hardware components into a novel dllhi- LIUIC~ which is integrated onto a low cost application specific t~ dll.d circuit IASIC). The novel 25 architecture of the present invention permits the use of incA~ , DRAM to create hypermedia effects using multiple video and analog channels simultaneously. In contrast to costly prior art VRAM buffers, the media editor of the present invention provides for priority based : 16 . ud access to a DRAM based media buffer that delivers full video bandwidth. Additionally, the system accepts a wide variety of hypermedia data types and stores them in the same stora~qe, enabling the mixing of audio, video, text, music and other data types usin~q common mixing engines. Moreover, the system allows for the capture of multiple hypermedia channels with simultaneous playback and editing. Furthermore, because the hypermedia output is not :'(, ' l on the hypermedia input synchlr hypermedia inputs may be removed, attached and switched on the fly, resulting in different hypermedia input a~ - S without any effect on the timing or quality of the h,l ,.. 'i~ output. Lastly, audio is ~r" ' IJ.~;~cd to video by reference to a master pixel clock, which is also used to prevent "tearing" and other Ji..l~" liu..~.
The invention may be embodied in other specific forms without departin~q from its spirit or essential characl~ iL~. Thus, the described embodiments are, in all respects, merely illustrative and not la~lliLIi.C.

.

~ O 98/06098 PCTAUS97/13055 l6 ~oreover, the appended claims, rather than the foregoing dcsc,i~Jli"n, serve to define the scope of the invention.
~hese claims embrace within their scope all changes which come within their meaning and range of equivalency.

Claims (39)

WHAT IS CLAIMED IS:

1. A real time, non linear editing system, comprising:
a remote control to provide command signals;
a main unit, responsive to the command signals, having a non linear editor processing subsystem and a direct access storage device, wherein the non-linear editor processing subsystem receives hypermedia via an input port, and enables editing operations, such as fades, dissolves, wipes or animated effects, on a video portion of the hypermedia in a YUV color coding scheme, and wherein the direct access storage device stores the hypermedia; and a viewing device that receives the hypermedia from the main unit and displays at least the viewable portions of the received hypermedia.

2. The non linear editing system of Claim 1, wherein the remote control comprises:
a power system;
a keyboard having at least one button; and a keyboard processor in communication with the keyboard and receiving power from the power system, wherein the keyboard processor receives and converts input from the keyboard into the command signals.

3. The non linear editing system of Claim 2, further comprising at least one pointing device in communication with the keyboard processor, wherein the keyboard processor receives and converts input from the at least one pointing device into the command signals.

4. The non-linear editing system of Claim 3, wherein the at least one pointing device may be configured to operate as a jog/shuttle wheel and wherein the keyboard processor receives and converts input from the jog/shuttle wheel into the command signals.

5. The non linear editing system of Claim 2, further comprising a jog/shuttle wheel in communication with the keyboard processor, wherein the keyboard processor receives and converts input from the jog/shuttle wheel into the command signals.

6. The non-linear editing system of Claim 2, further comprising a wireless transmitter in communication with the keyboard processor, wherein the keyboard processor provides the command signals to the wireless transmitter for conversion into a wireless command signal.

7. The non-linear editing system of Claim 1, wherein the viewing device comprises a television set or a computer display.

8. The non-linear editing system of Claim 1, wherein the system additionally comprises a videotape recorder, or at least one speaker, or a recordable CO-ROM, or a DVD, or a printer.

9. The non-linear editing system of Claim 1, wherein the main unit communicates with a network to access the hypermedia.

10. The non linear editing system of Claim 9, wherein the network provides access to a real time hypermedia source or an off line hypermedia source.

11. The non linear editing system of Claim 9, wherein the network comprises a broadcast television network, or a cable television network, or a satellite television network, or the World Wide Web portion of the Internet.

12. The non-linear editing system of Claim 1, wherein the main unit accesses the hypermedia from a videotape player, or a radio tuner, or a compact disc player, or a CD-ROM.

13. The non-linear editing system of Claim 1, wherein the main unit includes a communication port.

14. The non linear editing system of Claim 13, wherein the communication port comprises a RS-232 serial port, or a parallel port, or an infrared port, or a PCMCIA port, or a USB port, or a SCSI port.

15. The non-linear editing system of Claim 1, wherein the hypermedia is in a compressed digital data format.

16. A real time non-linear editor, comprising:
a bus;
a processor, in communication with the bus, to control access to the bus;
a compression engine, in communication with the bus, to compress and decompress video data;
a persistent storage, in communication with the bus, to capture and provide direct access to the compressed video data;
a media editor, in communication with the compression engine and the bus, the media editor and the compression engine receiving the video data, wherein the media editor operates on the video data in a YUV color space; and a media buffer in communication with the media editor, wherein the media buffer comprises single port random access memory.

17. The non-linear editor of Claim 16, further comprising a decoder in communication with the compression engine and the media editor, wherein the decoder receives and decodes a video input.

18. The non-linear editor of Claim 17, wherein the decoder includes an analog to digital converter.

19. The non linear editor of Claim 16, further comprising an encoder in communication with the media editor, wherein the encoder receives and encodes the video data from the media editor.

20. The non linear editor of Claim 19, wherein the encoder includes a digital to analog converter.

21. The non-linear editor of Claim 16, wherein the media buffer is controlled by a memory controller.

22. The non-linear editor of Claim 21, wherein the memory controller utilizes bank interleaving.

23. The non-linear editor of Claim 16, wherein the processor comprises an embedded controller.

24. The non-linear editor of Claim 16, wherein the compression engine provides MPEG-1 compression and decompression, or MPEG-2 compression and decompression, or wavelet compression and decompression, or JPEG
compression and decompression.

25. The non-linear editor of Claim 16, wherein the storage comprises a magnetic disk drive, or a DVD
device, or a magneto optic disk drive, or a rewriteable optical disk drive.

26. A method of operating a real-time, non-linear editing system having a memory, comprising the steps of:
storing a plurality of hypermedia portions, which include video, to a persistent storage;
powering down the non linear editing system;
powering up the non linear editing system;
automatically loading the previously stored hypermedia from the persistent storage to the memory;
and providing a storyboard to permit manipulation of the previously stored hypermedia including editing operations such as fades, dissolves, wipes or animated effects.

27. The method of Claim 26, further comprising the step of initializing the non linear editing system.

28. The method of Claim 26, further comprising the step of making a copy of the storyboard.

29. The method of Claim 26, further comprising the steps of:
determining if the storage has space to store a new hypermedia portion;
allocating space for the new hypermedia portion in the storage if the storage does not have space;
storing the new hypermedia portion to the storage; and automatically placing the new hypermedia portion on the storyboard.

30. The method of Claim 26, wherein the step of automatically loading the previously stored hypermedia includes providing a default transition between a pair of previously stored hypermedia portions.

31. A method of editing hypermedia in a real time, non linear editing system including a viewing device and a memory, comprising the steps of:
storing a plurality of hypermedia portions, which include video, to a persistent storage;
automatically placing the hypermedia portions on a storyboard displayed on the viewing device;
and automatically providing a transition, such as a cut, dissolve or fade, between a first hypermedia portion and a second hypermedia portion as the second hypermedia portion is placed on the storyboard.

32. The method of Claim 31, wherein a transition frame is formed on a pixelated basis according to the following equation: Transition Frame pixel = [(First Frame pixel from a first hypermedia portion ~ Alpha Frame pixel) + Second Frame pixel from a second hypermedia portion ~ (1 - Alpha Frame pixel)], where the alpha frame pixel is from an alpha frame stored in the memory and is normalized to a value between zero and one inclusive.

33. The method of Claim 31, further comprising the step of manipulating at least one of the plurality of hypermedia portions.

34. The method of Claim 31, further comprising the step of modifying the transition.

35. The method of Claim 31, further comprising the step of adding an overlay to at least one of the plurality of hypermedia portions.

36. The method of Claim 31, further comprising the step of playing back the storyboard.

37. The method of Claim 31, further comprising the step of copying the storyboard.

38. The method of Claim 31, additionally comprising digitizing one of the hypermedia portions before staring the portion to the persistent storage.

39. The method of Claim 31, wherein a default automatic transition comprises a cut.