WO1998034395A2

WO1998034395A2 - Compression/decompression of stream of video data

Info

Publication number: WO1998034395A2
Application number: PCT/US1998/000817
Authority: WO
Inventors: Alan Chung; Ralf Berger
Original assignee: Media 100, Inc.
Priority date: 1997-02-04
Filing date: 1998-01-16
Publication date: 1998-08-06
Also published as: WO1998034395A3

Abstract

An adaptive method of lossless compression of video data (46) including evaluating a block of consecutive video data values, selecting one mode of a plurality of possible lossless data compression modes based upon the evaluation of the block, compressing the video data values (50) to the selected mode to result in compressed video data values for the block, and adding an indication of the selected mode to the compressed video data values for the block.

Description

COMPRESSION/DECOMPRESSION OF STREAM OF VIDEO DATA Background of the Invention The invention relates to compression/decompression bf a stream of video data and is particularly directed to compression/decompression of video effects data.

In creating some effects on video, coefficients are used on a pixel-by-pixel basis to define how the raw video data are modified. E.g., moving mattes can be created by 8-bit or 10-bit per pixel coefficients, and alpha channel effects can be created by 16-bit per channel coefficients. The coefficients change on a field-by-field basis, and a stream of the video effects coefficients are inputted in parallel with a stream of video data to an ef ects processor to modify the video to have the desired effect. When implemented on a desktop computer-based system, significant amounts of bus bandwidth can be required to provide the stream of video effects data.

Summary of the Invention

In one aspect, the invention features, in general, adaptively losslessly compressing video data by evaluating a block of consecutive video data values, selecting one mode of a plurality of possible lossless data compression modes based upon the evaluation of the block, compressing the video data values according to the selected mode, and adding an indication of the selected mode to the resulting compressed video data values.

In another aspect, the invention features, in general, losslessly decompressing adaptively compressed video data values by receiving a block of compressed video data values including an indication of the selected mode of lossless compression used to generate the compressed video data values, selecting one mode of a plurality of possible lossless data decompression modes based upon the indication of the selected mode of lossless compression for the block, and decompressing the block of compressed video data values according to the selected mode to result in decompressed video data. Certain implementations of the invention may include one or more of the following features. In certain implementations, the video data are grouped in fields or frames having one or more segments corresponding to the blocks already mentioned. The segments are received in sequence. At least some of the segments in a field or frame are compressed by different modes. The video data that are compressed are video effects coefficients used to modify video. Each field or frame has one or more segments and a global header including a vertical resolution value, a horizontal resolution value, and a sample width value.

In certain implementations, the lossless compression is achieved using one or more of the processes of modified differential pulse code modulation (MDPCM) , occurrence table coding, and run length encoding, depending upon the selected mode. The compressed video values for a segment include s segment header and one or more segment codewords. The segment header defines the mode of compression, and includes information relating to the compression process, e.g., an initial value for a MDPCM compression technique and the number of bits in MDPCM symbols. The segment header can also include occurrence table values used in an occurrence table coding compression technique. The segment codewords include the compressed video data values, a count, and a mode value defining the meaning of the count and the symbols.

Embodiments of the invention may include one or more of the following advantages. By compressing video effects coefficients, bus bandwidth is saved, freeing up bus bandwidth for other purposes and permitting the storage, transfer and processing of greater amounts of video data and effects data. This also facilitates processing of multiple streams of effects data for multiple video streams at one time.

Other advantages and features of the invention will be apparent from the following description of the preferred embodiment and from the claims.

Brief Description of the Drawings

Fig. 1 is a block diagram of a video editing system.

Fig. 2 is a block diagram showing an adaptive, three-stage process for loss-less compression of video effects data.

Fig. 3 is flow chart showing steps in the Fig. 2 process.

Fig. 4 is a block diagram showing an adaptive process for loss-less decompression of compressed video effects data.

Fig. 5 is a diagram illustrating an 8-bit modified differential pulse code modulation process used in the Fig. 2 process.

Fig. 6 is a diagram of an occurrence table used to determine occurrences of codewords generated in the Fig. 5 process.

Fig. 7 is a diagram of a subset of the frequently occurring codewords of the Fig. 6 occurrence table. Fig. 8 is a diagram showing the format of compressed video effects data.

Fig. 9 is a diagram showing the format of a global header of the compressed video effects data of Fig. 8. Fig. 10 is a diagram showing the format of a segment header of the compressed video effects data of Fig. 8.

Fig. 11 is a diagram showing the format of a segment codeword of the compressed video effects data of Fig. 8.

Figs. 11A-11D are diagrams showing formats for different types of segment codewords.

Description of the Preferred Embodiments Referring to Fig. 1, there is shown computer-based video editing system 10 including general purpose host computer 12 and peripheral video editing card 14 used to provide video editing capabilities for computer 12. Computer 12 can, for example, be a personal computer with a peripheral component interface (PCI; as defined by

Revision 2.1 of PCI Local Bus Special Interest Group, PO Box 14070, Portland, Oregon 97214) and an NT operating system. Computer 12 includes central processing unit (CPU) 16, main memory unit 17, system bus 18, PCI I/O bus 19, and peripheral I/O bridge interface unit 20 connected between PCI I/O bus 19 and system bus 18. Computer 12 also includes computer display monitor 21 connected to PCI I/O bus 19 via peripheral I/O display controller 22 and host storage disks 23 connected to PCI I/O bus 19 via peripheral I/O disk controller 2.4. Video editing card 14 is also connected to PCI I/O bus 19, which is defined by the above-referenced standard.

Video editing card 14 includes local interface bus 25, which is connected to PCI I/O bus 19 via peripheral I/O interface circuit 26, and video bus 27, used to transfer decompressed video data to four video decoders 28. Peripheral I/O interface circuit 26 (e.g., available from Intel under the i960 RP trade designation) is used to transfer data between PCI I/O bus 19 and local interface bus 25. Flash memory 29, dynamic random access memory (DRAM) 29A, video encoder circuit 31, and frame cache memory controller 32 are connected to local interface bus 25. Flash memory 29 and DRAM 29A provide on-board storage for programs and coefficients used by various components on board 14. Frame cache memory controller 32 controls all reads and writes to video frame memory 33, which has 32 Mbytes of memory addressed into the address space of PCI I/O bus 19. Frame cache memory controller 32 also transfers data read from memory 33 to video bus 27. Video decoders 28 and effects decoder 30 are connected to video effects mixer 34, which is connected to video output port 35 for outputting source video in SMPTE 259 format. Video card 14 also includes a video input port 36 for receiving video in SMPTE 259 format and providing it to video encoder 31. Video encoder 31 and video decoder 28 carry out MPEG encoding and decoding (ISO/IEC standard nos. 11172, 13818) . The output of video encoder 31 and the inputs to video decoders 28 are decoded video in the form of MPEG I-frames or MPEG IBP-frames. Video editing card 14 also has video effects input port 39 for receiving external video effects coefficients in SMPTE 259 format and providing them to real time hardware video effects encoding processor 41.

Video effects mixer 34 receives one or more streams 40 of raw digital video and one or more streams 42 of video effects coefficients and outputs one or more streams of modified video 44. Each video effects coefficient stream 42 includes a stream of multi-bit values of the effect's coefficients, e.g., 8- or 10-bit moving matte coefficients or 16-bit alpha channel coefficients. There is one coefficient for each pixel in a field, and the coefficients are received for video pixels in the same order that video pixels are received, i.e., one horizontal scan line after another in interlaced or noninterlaced order.

Fig. 2 describes a three-stage process for compressing video effects coefficients 60. The three- stage process can be implemented in hardware or in software, and the compressed video effects coefficients 65 can be stored on disk 23 or sent to peripheral video editing card 14 and decompressed at a hardware video effects decoder 30 and passed to video effects mixer 34. Compressed video effects coefficients 65 can also be decompressed in software.

Fig. 3 is a flow chart describing the steps and decisions made in deciding which stages of the Fig. 2 process are used for a given segment of video effects coefficients 60.

The three stages include modified differential pulse code modulation stage (MDPCM) 46, occurrence table coding stage (OTC) 48, and run length encoding stage (RLE) 50. OTC 48 employs a subset of occurrence table (SOT) 52, which is based upon occurrence symbol table (OST) 54. OST 54 is generated during MDPCM 46, and SOT 52 is generated from OST 54 if it is decided to use OTC 48.

Table 1 below describes four modes involving different combinations of MDPCM 46, OTC 48, and RLE 50 and one mode that outputs the raw data without compression. Table l also shows hex values used to describe the mode chosen. The mode used in compression of a segment of video effects coefficients must of course also be used in decompression of the coded effects coefficients. Other combinations beyond those shown on Table 1 can be used and other processes beyond MDPCM, OTC, and RLE can be included. Table 1

MDPCM 46

Referring to Fig. 2, in MDPCM 46, a horizontal scan line of effects coefficients 60 is processed at one time. (Alternatively another minimum size for a

"segment," described below, can be processed at one time.) Consecutive video effects coefficients 60 are compared, and the unsigned differences between the consecutive values are outputted at 62. Occurrence symbol table 54 (Figs. 2 and 6) is created as differences 62 are generated. Both the original coefficients at 60 and the differences output at 62 are saved, as indicated at Step 200 (Fig. 3) .

Fig. 5 shows unit circle 56 used to do determine unsigned differences 62. In general, a circle with 2^N units is needed for N-bit coefficients 60. Unit circle 56 shown in Fig. 5 would be used for 8-bit coefficients 60. In a segment, the first coefficient 60 is considered the initial condition, I₀, which is saved to provide a reference for the unsigned differences that follow at 62. The differences are determined by locating the reference value on unit circle 56 and going along unit circle 56 in a clockwise direction to the value of the next coefficient. The number of discrete points that are counted in going to the next coefficient value is the output of MDPCM 46 at 62. The number of occurrences of each of the difference values at 62 is tabulated in OST 54 (Figs. 2 and 6) .

If the deviation of the differences at 62 is small, it may be possible to represent the original sample with a reduced bit length. Thus, in subsequent processing of the values at 62 , a reduced number of bits will be used. The analysis of differences and the decision as to use of MDPCM 46 are carried out in Steps 202, 204 (Fig. 3). E.g., if the difference values at 62 are not larger than 16, then 4-bit values can be used in subsequent processing. If the differences values at 62 still require 8 bits, then there will not be any data reduction in use of MDPCM 46; in this case there is an option to use either the original values 60 (Step 205 on Fig. 2) or the difference values at 62 in subsequent processing.

OCT 48

As already noted, occurrence symbol table 54 (Figs. 2 and 6) tabulates the occurrences of each of the difference values at 62 in a horizontal scan line of coefficient values. The most frequent entries of OST 54 are evaluated in Step 206. If a small number of difference values represent a significant number of the input data, e.g. , if 16 difference values account for 75% or even 50% of the input coefficient values, and the difference values have at least five bits, then there can be further reduction in the amount of data by creating SOT 52 (Figs. 2 and 7) and using it in OTC 48. If it is decided to use OTC 48 for the 16 most frequent difference values at 62 (Step 208) , then SOT 52 is constructed (Step 210) . In this case difference values stored at 62 are coded using SOT 52 to provide the output values at 64 (Step 210) . While subset of occurrence table 52 as shown in Fig. 7 has 16 values, if a large percentage of differences values can be described in a smaller table 52, then a smaller table, e.g., one with 8, 4, or 2 values, resulting in 3-, 2-, or 1-bit symbols at 64 (Fig. 2) , is used. If, on the other hand, it is determined at Step 208 that there would not be reduction in data using subset of occurrence table 52, then the difference values at 62 will instead be used in subsequent processing (Step 212) .

RLE 50

RLE 50 (Fig. 2) is performed on the original values at 60 (without MDPCM 46 or OTC 48) , the difference values at 62 (without OTC 48) , or the symbol values 64 (with OTC 48) ; the values input to RLE 50 (original values 60, differences 62, or symbols 64) are saved (Step 214) . RLE 50 involves comparing an input value with the prior input value to determine if there is a repeat. The number of unique values in a row are counted, and the number of repeats in a row are counted. While RLE 50 can result in reducing the volume of data, if the data do not repeat in consecutive values, there can be an increase in the volume of data. In Step 216, the volume of the output data at 65 is compared with the volume of the input data to see if there is a reduction in data by use of RLE 50 (Step 218) . If there is no reduction (or is an increase in data) , then the original input data (either differences 62 or symbols 64) are output (Step 220) . If there is a reduction in data, then the RLE encoded values are output at 65 (Step 222) .

The same general procedure is then carried out for the next line of video effects coefficients at 60. One exception is the creation of subset of occurrence table 52 in Step 210. In this step, the SOT 52 that had been created for the prior line of coefficients is considered for possible use with the data stored in the present OST 54 for the present line of coefficients. If the prior SOT 52 can be used, it is used in OTC 48, and the compressed effects values at 65 are combined with common header information.

The compressed values at 65 (Fig. 2) and other information, as described with reference to Figs. 8-10, 11A-11D, are stored at host disk 23 or transferred to video editing card 14 for immediate use.

Format of Compressed Effects Data

Referring to Fig. 8, the compressed output for a field of effects coefficients includes a single global header 70 and one or more segments 72. Each segment 72 includes a segment header 74 and a segment codeword 76. Referring to Fig. 9, the format of global header 70 is shown. Global header 70 includes 12-bit horizontal resolution value 78, 12-bit vertical resolution value 80 and 8-bit sample width value 82. Horizontal resolution value 78 defines how many pixels are in a horizontal scan line, and vertical resolution value 80 defines how many scan lines are in a field. Sample width 82 defines the number of bits in the input video effects coefficients, e.g., 8- or 10-bits for moving matte coefficients or 16- bits for alpha channel coefficients.

Referring to Fig. 10, it is seen that segment header 74 includes 4-bit compression mode 84 defining which of the modes of Table 1 described above is employed, 4-bit symbol width 86 (defining the number of bits used in the difference values at 62) , 16-bit initial value, I₀, for the first value in a segment going through MDPCM 46, unused 4-bits 90, 4-bit OTC size value 92 defining the number of entries in the subset of occurrence table 52 (also defining the bit width for the values at 64) , and OTC entries 53 of SOT 52 (Fig. 7) . Referring to Fig. 11, the format for segment codewords 76 is shown. It includes a 2-bit segment compression mode value 94, a 6-bit count 96, and the actual symbol values 98. Figs. 11A-11D describe the different segment compression modes (0-3) and the meanings of count 96 and symbol 98 in each of the modes. Figs. 11A and 11B describe cases where OTC 48 is not used; Figs. 11C and 11D describe cases where OTC 48 is used, and symbol 98 is the OTC-coded symbol at 64. In Figs. 11A and 11D, count 96 indicates the repeat count for the associated symbol 98, and in Figs. 11B and lie, count 96 indicates the number of unique symbols 98 that follow.

Decompression of Compressed Video Effects Coefficients

The decompression of the compressed video effects coefficients is the reverse of the compression process employed for the particular segment. Thus Fig. 4 shows compressed coefficients at 100, decompressed coefficients at 101, and three stages: run length decoding stage (RLD) 102, occurrence table decoding stage 104, and modified differential pulse code demodulation stage (MDPCD) 106. For a field of compressed coefficients as shown in Fig. 8, the information in global header 70 is used to define the expected size of the data to be reconstructed for the field, and segments 72 are then processed. As each segment 72 arrives at video effects decoder 30 (Fig. 1) , segment header 74 is used to set up video effects decoder 30 to handle the actual segment codewords 76 that follow. In segment header 74, compression mode value 84 defines which data compression/decompression mode of Table 1 has been employed in compressing the particular segment 72, i.e., which of RLE 50, OTC 48, and MDPCM 46, if any, has been used. This defines which reverse processes will thus be carried out in decoding. OTC size 92 defines the size of SOT 52 to be used in occurrence table decoding 104, and OTC entries 53 define the actual entries to be used in occurrence table decoding 104. Symbol width 86 defines the width of values prior to MDPCD (and also defines the width of OTC entries 53) . MDPCM initial value 88 is used in MDPCD 106. Depending upon the mode, one or more of run length decoding 102, occurrence table decoding 104, and MDPCD 106 are carried out on segment codewords 76. After decompressing, the volume of reconstructed video effects coefficients at 101 can then be compared with the information in the global header 70 to confirm that there has not been a loss or other disruption in the data.

Other embodiments of the invention are within the scope of the appended claims.

Claims

What is claimed is:

1. An adaptive method of lossless compression of video data comprising evaluating a block of consecutive video data values, selecting one mode of a plurality of possible lossless data compression modes based upon the evaluating of said block, compressing said video data values according to the selected mode to result in compressed video data values for said block, and adding an indication of the selected mode to said compressed video data values for said block.

2. The method of claim 1 wherein said video data values are grouped in fields or frames having one or more segments, and said block is a said segment.

3. The method of claim 2 wherein there are a plurality of said segments, and further comprising receiving said segments in sequence and performing said evaluating, selecting, compressing, and adding for said segments as they are received.

4. The method of claim 2. wherein at least some of the segments in a field or frame are compressed by different modes.

5. The method of claim 1 wherein said video data values are video effects coefficients used to modify video, and said compressed video data values are compressed video effects coefficients.

6. The method of claim 5 wherein said video effects coefficients are grouped in fields or frames having one or more segments and a global header, said block being a said segment, and further comprising receiving said segments in sequence and performing said evaluating, selecting, compressing, and adding for said segments as they are received.

7. The method of claim 6 wherein said global header includes a vertical resolution value, a horizontal resolution value, and a sample width value.

8. The method of claim 6 wherein said compressing includes one or more of the processes of modified differential pulse code modulation, occurrence table coding, and run length encoding, depending upon the selected mode.

9. The method of claim 8 wherein one said selected mode includes modified differential pulse code modulation.

10. The method of claim 8 wherein one said selected mode includes modified differential pulse code modulation and run length encoding.

11. The method of claim 8 wherein one said selected mode includes modified differential pulse code modulation and occurrence table coding.

12. The method of claim 8 wherein one said selected mode includes modified differential pulse code modulation, occurrence table coding, and run length encoding.

13. The method of claim 2 further comprising adding a segment header and one or more segment codewords per segment, said segment header defining said mode, said segment codewords including said compressed video data values.

14. The method of claim 13 wherein said segment header includes information relating to the compression process.

15. The method of claim 13 wherein each segment codeword includes one or more said compressed data values.

16. The method of claim 14 wherein said segment header specifies an initial value for a modified differential pulse code modulation (MDPCM) compression technique and defines the number of bits in MDPCM symbols.

17. The method of claim 14 wherein said segment header includes occurrence table values used in an occurrence table coding compression technique.

18. The method of claim 15 wherein said segment codewords include a count.

19. The method of claim 18 wherein each segment codeword includes a mode value defining the meaning of the count and the symbols.

20. A method of lossless decompression of adaptively compressed video data values comprising receiving a block of compressed video data values including an indication of the selected mode of lossless compression used to generate said compressed video data values in said block, selecting one mode of a plurality of possible lossless data decompression modes based upon said indication of the selected mode of lossless compression for said block, and decompressing said block of compressed video data values according to the selected mode to result in decompressed video data values.

21. The method of claim 20 wherein said compressed video data values are grouped in fields or frames having one or more segments, and said block is a said segment.

22. The method of claim 21 wherein there are a plurality of said segments, and further comprising receiving said segments in sequence and performing said selecting and decompressing for said segments as they are received.

23. The method of claim 22 wherein at least some of the segments in a field or frame have been compressed by different modes.

24. The method of claim 20 wherein said compressed video data values are compressed video effects coefficients used to modify video.

25. The method of claim 24 wherein said compressed video effects coefficients are grouped in fields or frames having one or more segments and a global header, and said block is a said segment.

26. The method of claim 25 wherein there are a plurality of said segments, and further comprising receiving said segments in sequence and performing said selecting and decompressing for said segments as they are received.

27. The method of claim 25 wherein said global header includes a vertical resolution value, a horizontal resolution value, and a sample width value.

28. The method of claim 25 wherein said decompressing includes one or more of the processes of modified differential pulse code demodulation, occurrence table decoding, and run length decoding, depending upon the selected mode.

29. The method of claim 28 wherein one said selected mode includes modified differential pulse code demodu1ation.

30. The method of claim.28 wherein one said selected mode includes modified differential pulse code demodulation and run length decoding.

31. The method of claim 28 wherein one said selected mode includes modified differential pulse code demodulation and occurrence table decoding.

32. The method of claim 28 wherein one said selected mode includes modified differential pulse code demodulation, occurrence table decoding, and run length decoding.

33. The method of claim 21 further comprising receiving a segment header and one or more segment codewords per segment, said segment header defining said mode, said segment codewords including said decompressed video data values.

34. The method of claim 33 wherein said segment header includes information relating to the compression process.

35. The method of claim 33 wherein each said segment codeword includes one or more said compressed data values.

36. The method of claim 34 wherein said segment header specifies an initial value for a modified differential pulse code demodulation (MDPCD) decompression technique and defines the number of bits in MDPCD symbols.

37. The method of claim.34 wherein said segment header includes occurrence table values used in an occurrence table decoding compression technique.

38. The method of claim 35 wherein said segment codewords include a count.

39. The method of claim 38 wherein each segment codeword includes a mode value defining the meaning of the count and the symbols.

40. Apparatus for lossless compression of video data comprising a data processor that evaluates a block of consecutive video data values, a selector that selects one mode of a plurality of possible lossless data compression modes based upon the evaluating of said block by said data processor, and a compressor that compresses said video data values according to the selected mode to result in compressed video data values, and adds an indication of the selected mode to said compressed video data values.

41. Apparatus for lossless decompression of adaptively compressed video data values comprising a source of a block of compressed video data values including an indication of the selected mode of lossless compression used to generate said compressed video data values in said block, a selector that selects one mode of a plurality of possible lossless data decompression modes based upon said indication of the selected mode of lossless compression for said block, and a decompressor that decompresses said block of compressed video data values according to the selected mode to result in decompressed video data.