WO2000051349A1 - Active techniques for video transmission and playback - Google Patents

Active techniques for video transmission and playback Download PDF

Info

Publication number
WO2000051349A1
WO2000051349A1 PCT/US2000/004491 US0004491W WO0051349A1 WO 2000051349 A1 WO2000051349 A1 WO 2000051349A1 US 0004491 W US0004491 W US 0004491W WO 0051349 A1 WO0051349 A1 WO 0051349A1
Authority
WO
WIPO (PCT)
Prior art keywords
sub
program
programs
files
decoding
Prior art date
Application number
PCT/US2000/004491
Other languages
French (fr)
Inventor
Prathima Agrawal
Jyh-Cheng Chen
David Famolari
Original Assignee
Telcordia Technologies, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Telcordia Technologies, Inc. filed Critical Telcordia Technologies, Inc.
Publication of WO2000051349A1 publication Critical patent/WO2000051349A1/en

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/435—Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
    • H04N21/4351—Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream involving reassembling additional data, e.g. rebuilding an executable program from recovered modules
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/235—Processing of additional data, e.g. scrambling of additional data or processing content descriptors
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/434—Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream
    • H04N21/4348—Demultiplexing of additional data and video streams
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/435—Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/443—OS processes, e.g. booting an STB, implementing a Java virtual machine in an STB or power management in an STB
    • H04N21/4431—OS processes, e.g. booting an STB, implementing a Java virtual machine in an STB or power management in an STB characterized by the use of Application Program Interface [API] libraries
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/81—Monomedia components thereof
    • H04N21/8166—Monomedia components thereof involving executable data, e.g. software
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/81—Monomedia components thereof
    • H04N21/8166—Monomedia components thereof involving executable data, e.g. software
    • H04N21/8193—Monomedia components thereof involving executable data, e.g. software dedicated tools, e.g. video decoder software or IPMP tool

Definitions

  • This invention relates generally to video communications, and, more
  • the encoded output appearing on lead 121 is formatted into packets 131 by data
  • Transmitter processor 135 controls the interactions of encoder 120 with
  • program memory 125 and also provides the necessary control information so as to form
  • packets 131 are transmitted via packet network 140 as propagating
  • extractor 150 to produce, on lead 151, the received counterpart of compressed output 121
  • decoder 160 In order to reproduce frame 111, it is necessary for decoder 160 to have
  • decoding software in decoder 160 so as to display the video stream.
  • receiver 102 receives a
  • decoding software either because decoder 160 has not been loaded with the
  • the subject matter of the present invention relates to: (a) encapsulating the
  • node used interchangeably with switch or router
  • a packet is composed of a header part and a payload
  • the output destination port for the packet is computed as usual.
  • the (possibly modified) packet is transmitted to its next-hop node.
  • ACPI Active Processing Function Identifier
  • BCZ proposes that an alternative to placing the ACPI in the network header itself is to define a
  • BCZ immediately rules out the use of the packet itself for
  • a node which can be performed in the node A node which has the computational power is
  • AP active processor
  • header indicate the priority of particular packets (for example, I, P, and B frames, as
  • This priority is used in the AP to decide which packet should be
  • receiving terminals the ability to play any type of encoded video stream without having the associated decoding software pre-loaded, thus creating a highly flexible and dynamic
  • the programming code to decode encoded video is bundled with the encoded
  • decoding sub-programs (c) propagating a sequence of the sub-files and the sub-
  • a method for transceiving a real-time frame includes the following procedure:
  • decoding sub-programs (c) propagating a sequence of active packets composed of the sub-files and the sub-programs; (d) detecting the sequence of active packets to determine
  • FIG. 1 depicts a conventional system for propagating packets representative of
  • FIG. 2 depicts the hierarchy of the various streams used in the MPEG-2 standard
  • FIGS. 3 A, 3B, and 3C depict, respectively: the components of a frame; the spatial
  • FIGS. 4A, 4B, and 4C depict the composition of MPEG-2 packets, at high level
  • FIG. 5 depicts a MPEG-2 Transport Stream and the relation of the packets in the
  • FIG. 6 illustrates an example of a conventional protocol stack for encapsulating
  • FIG. 7 is a high-level block diagram of an illustrative embodiment of the real-time
  • FIG. 8 illustrates the "active activation" as part of the application layer of the
  • FIG. 9 depicts the contents of an active packet, including the active packet header
  • FIG. 10 illustrates the generation of an active packet transport stream in a manner
  • FIG. 11 depicts representative contents of the data memory and program memory
  • FIG. 12 is a flow diagram for generating an active packet in the transmitter of
  • FIG. 7 in accordance with the present invention.
  • FIG. 13 is a flow diagram for assigning and associating the data packet identifier
  • FIG. 14 is a flow diagram for receiving active packets to generate a reproduced
  • FIG. 15 is a high-level block diagram of another illustrative embodiment of the
  • FIG. 16 is a high-level block diagram of an illustrative embodiment of the non-
  • FIG. 17 is a flow diagram for the operation of the system of FIG. 16;
  • FIG. 18 is a high-level block diagram of another illustrative embodiment of the
  • FIG. 19 is a flow diagram for the operation of the system of FIG. 18.
  • Video transmission is typically broadband in nature. For example,
  • HDTV High Definition Television
  • FIG. 2 illustrates the protocols levels that
  • data packetizer 130 of FIG. 1 generate so as to transport a digital video frame from the
  • each frame 110 received by encoder 120 is transformed into a compressed video digital representation of the frame
  • packetizer 130 generally includes elements, corresponding to the MPEG-2 standard, for
  • PES packetized elementary stream
  • the encoded video is processed by element 210 by
  • element 220 adds timing information (e.g., the Presentation Time
  • element 230 adds system information as well as a clock reference (e.g., the Program
  • the video source which produces a sequence of frames, with
  • a NTSC-compatible source generates 30 frames per
  • Intraframe coding an I-frame as used below exploits the spatial
  • An I-frame contains full picture information.
  • the spatial domain In the spatial domain,
  • each video frame is divided into a "block", for example, of 8 X 8 pixels.
  • DCT Cosine Transform
  • coefficients are quantized to a user-defined level. Four blocks are grouped to create a
  • MB Microblock
  • a slice is the basic transmission and resynchronization unit for the video stream.
  • the coding for an I-frame is based on spatial
  • the purpose of the P-frame is to reduce temporal
  • prediction and backward prediction can be used as the motion compensated prediction.
  • FIG. 3B summarizes I, P, and B frames.
  • frame 301 is shown as being composed of block 3011,
  • Frames 302-307 are, respectively, presumed to be B,P,
  • FIG. 3B that is, the sequential order 0, 1, 2, ... 6 (that is, the Presentation Time Stamp), FIG. 3B
  • FIG. 3B shows how the set of seven frames
  • FIG. 3C summarizes the various layers utilized by the MPEG-2
  • each packet contains high-level information and data
  • sequence layer 320 (which is oftentimes referred to as the packet
  • sequence layer 320 is composed
  • 322 is composed of Slice layer 323; layer 323 is composed of Macroblock layer 324; and,
  • layer 324 is composed of Block layer 325.
  • JPEG standard used to encode "still” images can be used for video systems by treating
  • each frame as a still image.
  • the coding scheme for each frame in JPEG is similar to
  • MPEG-2 video conferencing and is similar to MPEG-2 but only has I and P frames.
  • MPEG-1 also defines I, P, and B frames, and the coding algorithms are similar.
  • MPEG-2 however,
  • the H.263 standard is a
  • Data packetizer 130 of FIG. 1
  • MPEG-2 packets composing a transport stream is fully elucidated.
  • stream 400 of MPEG-2 packets is depicted
  • each packet further comprises a header/payload pair, namely, header 401 and its
  • Header 401 is generally four-bytes long, and payload 402 is 184-bytes.
  • Stream 400 is
  • data packetizer 130 of FIG. 1 emitted by data packetizer 130 of FIG. 1 and, in the terminology of MPEG-2, is a
  • each header contains at least eight fields;
  • FIG. 4B an illustrative value, in decimal, for each field is shown below its name, and
  • PID Packet Identifier
  • payload is composed of components of the compressed video (or in other applications,
  • payload 402 is shown as providing Program Association Table
  • PAT 402 lists information such as: PID 16 is associated with Program 0 (reference
  • PID 110 is associated with Program 1 (reference numeral 423), PID 210 is
  • a 'Program' is a collection of streams within a common timebase, which
  • FIG. 5 A more detailed view of Transport Stream 400 is shown in FIG. 5 at the
  • a PID 1 packet by definition in the MPEG-2 standard, contains a
  • Conditional Access Table which includes scrambling information as well as an
  • EMM Entitlement Management Message
  • a PID 16 packet by definition, always has a payload of listing the
  • NIT Network Information Table
  • a PID 0 packet provides the Program Association Table.
  • Transport Stream 400 commences propagation with a packet having
  • PID 0 As shown in FIG. 4C, and as repeated in FIG. 5, the PID 0 packet transmits PAT
  • NIT 510 PID 110 is linked to Program Map Table (PMT) 520, and PID 210 is linked to
  • PMT 530 PMT's 520 and 530 appear as the payloads in the two packets following the
  • a PMT provides more detailed referencing information to further
  • PMT 520 lists, as identified by row 521, the Stream Identifier, the Type of
  • a packet associated with Stream 1 of Program 1 is of
  • the packet containing this video has PID 111. Also, the packet
  • PMT 530 is arranged similarly to PMT 520, and it is seen that, for
  • encoder 120 By way of reiteration, the function of encoder 120 may be aptly described
  • 130 may be aptly described as a process of appending information to the encoded video
  • Elements 120, 125 and 130 operate under control of
  • FIG. 6 illustrates an example of the pictorial representation of FIG. 6
  • the stream handled by the MPEG-2 standard is packet 601 of FIG. 6. Starting at the top
  • Transport layer 620 encapsulates each
  • Network layer 630 further encapsulates the result of
  • this layer effects, for example, routing or re ⁇
  • Data Link layer 640 accomplishes error control and access control and further
  • Physical layer 650 engenders the actual transmission at the bit-level.
  • data extractor 150 of FIG. 1 The purpose of data extractor 150 of FIG. 1 is that of ultimately recovering
  • Block layer 325 of FIG. 3C so that these blocks, being representative of the
  • sequence of video frames may be decompressed by decoder 160.
  • decoder 160 may be a sequence of video frames.
  • the identification of the video stream portions e.g., PID 111 of the transport stream 400; (3) with reference to FIG. 3C, processing the hierarchical layer
  • FIG. 7 shows the elements of the present
  • transmitter 701 generates
  • Receiver 702 only needs to extract the code from the active packets to obtain
  • receiver 702 is capable of displaying any type of encoded video
  • system 700 in accordance with the
  • present invention is composed of: (a) transmitter 701; (b) receiver 702; and (c) network
  • Transmitter 701 is further composed of: (i) the components of transmitter
  • transmitter processor 135 (ii) decoding program 725 augmenting program memory 125;
  • packetizer 735 rather than being propagated directly by network 140 of FIG. 1 ; in
  • decoding memory 725 provides a second input to active packetizer 735.
  • output of active packetizer 735 is the stream of active packets 731.
  • Receiver 702 is composed of: (i) data and program separator 740; (ii) data
  • program memory 750 also coupled to separator 740; (v) decoder 160, as in FIG. 1; and
  • receiver processor 755 which controls the elements of receiver 702. Decoder 160 has
  • processor 755 As inputs, in addition to processor 755: (a) the data output of data extractor 150, and (b)
  • decoding program software stored in decoding program memory 750.
  • FIG. 8 is a modification of FIG. 6 in accordance with the present
  • the generation of the protocol stack of FIG. 8 is patterned after the technique by which the protocol stack of FIG. 6 is generated, namely, by adding certain
  • sub-layer 810 appends certain information to each MPEG-2 packet 400, namely, active
  • header 820 is added before packet 400, and program packet 830 is added after packet 400
  • Active packet header 820 is composed of: (a) active packet identifier
  • program packet pointer 915 which points to the initial location of program packet
  • packet header 931 which includes DPID 914, and packet payload 933.
  • payload 933 conveys the decoding program which will be used by receiver 702
  • DPID 914 serves as
  • the decoding program code need only be sent once, with the first data packet having a
  • decoding program is used to decode successive packets. On the other hand, in other applications, it may be necessary to send
  • FIG. 9 is illustrative of the most general case. Also, it may be necessary, because of
  • packet identifier such as PPID 932
  • PPID 932 packet identifier 932
  • segments of a complete program can be assigned and identified by the same PPID to
  • Active Packet Stream 1000 of FIGJ0 which transports active packets 731
  • FIG. 7 is formed in a manner analogous to the formation of Transport Stream 400 of
  • FIGS. 4 and 5 that is, the formation of the Active Packet Stream 1000 encapsulates the
  • APID 1 and APID 16 (CAT/EMM data, and NIT table 1010, respectively)
  • APID is shorthand for Active Packet IDentifier.
  • the first stream is a stream of Active Packet IDentifier.
  • Each APID is
  • receiver 702 utilized by receiver 702 to compose a video stream originated from the same video
  • receiver 702 processes the incoming active packet
  • processor 740 prior to processing the active packet, processor 740 also removes all headers
  • each active packet e.g., from FIG. 8, the Ethernet, IP, UDP, and RTP
  • the data payload portion of the active packet is
  • data memory 745 stored in data memory 745 in a form illustrated in the top-half of FIG.11, that is, data
  • memory 745 generally stores the data itself (e.g. , '01000101111 ...') as well as the
  • program part of the active packet is stored in program memory 750 in a form
  • program memory 750 generally stores
  • next two entries in memory 750 are representative of a program that cannot
  • payload contains, illustratively, the first part of the binary decoding algorithm for the data
  • program payload header 931 may incorporate a
  • Data extractor 150 removes from memory 745, illustratively, only the
  • Receiver processor 755 controls the overall processing of receiver 702,
  • one main function is that of ensuring that the appropriate program to decode a given
  • decoder 160 is a code that specifies the code by decoder 160.
  • Flow diagram 1200 of FIG. 12 is representative of high-level processing
  • processing block 1210 As then invoked by processing block 1210, the incoming real ⁇
  • decoding program and processing block 1220 is used to specify the decoding program
  • program is used for all frames in a given video stream. If the program has not already
  • the program packet is formed by processing block 1250; the formation includes adding
  • this is the MPEG-2 transport stream.
  • processing block 1270 the active packet is generated; each active packet is transported as part of active packet stream 1000 as explained above. It is clear that flow diagram is but
  • transmitter 701 one illustrative embodiment of the flow carried out by transmitter 701; thus, for instance,
  • encoding by block 1210 and specifying by block 1220 can be completed in parallel.
  • Flow diagram 1300 of FIG. 13 depicts the details of forming the active
  • DPID is assigned to each data packet and a corresponding program packet identifier
  • PPID if any, is assigned to the program packet. It is necessary to correlate the DPID and
  • processing block 1340 is executed to generate the active packet
  • processing block 1350 formed in processing block 1350 by combining the active packet header, the data packet,
  • Flow diagram 1400 of FIG. 14 is representative of the processing effected
  • processing block 702 receives from receiver 702 on incoming active packet stream 1000.
  • processing block 702 processes the processing block incoming active packet stream 1000.
  • the active header information (such as the data packet and program packet pointers).
  • processing block 1420 is invoked to remove the data payload for storage in 745.
  • processing block 1430 is removed by processing block 1430, with
  • processing by decision block 1440 determines if the program for the given
  • DPID is in memory - because of the presentation time stamp may be different than the
  • this program is loaded into decoder 160 under control of
  • extractor 150 and this data is decoded in decoder 160.
  • system 700 of FIG. 7 is modified as per the
  • I-frame is divided into blocks according to the setting in the transmitter, that is, a
  • DCT Discrete Cosine Transform
  • the program to decompress each block is filled into the program packets.
  • transmitter processor 1535 parameters for encoding and segmentation are managed by transmitter processor 1535,
  • transmitter processor 135 of FIG. 7 which is essentially identical to transmitter processor 135 of FIG. 7, but in this case
  • processor 1535 explicitly depicts the accessibility of the "parameters" in forming
  • active packets in active packetizer 735 may be necessary to only use
  • receiver 1502 operates as follows. As
  • the data packets contain compressed data for each block.
  • receiver processor 1555 to assemble the complete decoding program which is then
  • program memory 750 stores the parameters that are detected in the program
  • desegmentation is effected by device 1560, that is: the program to decode the block is
  • the program is executed to desegment or re-group these blocks to generate
  • receiver processor 1555 which may be part of receiver processor 1555, until all related frames are restored.
  • the receiver need not have knowledge to differentiate the
  • Stored video, or non-real time video corresponds to a video stream that
  • FIG. 16 depicts overall non-real time video system
  • transmitter 1601 As being composed of transmitter 1601 and receiver 1602.
  • transmitter 1601 In transmitter 1601,
  • encoding deducer 1615 determines the algorithm used to compress stored video 1610.
  • This algorithm has a decoding counterpart which is stored, upon its determination, in
  • decoding program memory 1626 which is, for example, an adjunct to program memory
  • program memory 1625 contains software for controlling encoding deducer 1615.
  • Segmentation device 1620 partitions the stored video data file into data sub-files
  • active packets 731 For transmission as data packets in active packets 731.
  • active packets e.g., a packet for transmission as data packets in active packets 731.
  • packets 731) are assembled by active packetizer 735, the deduced program code used to
  • decode the video data is injected into the program packet portion of the active packets
  • receiver 1602 is essentially the same as receiver 1502
  • Program information and the data are extracted from the active packets via data and program separator 740, wherein data is stored in data memory 745 and the
  • decoding program is stored in program memory 750.
  • the data is extracted by data
  • processor 1655 provides the overall control of the other components of receiver 1602.
  • Flow diagram 1700 of FIG. 17 depicts the high-level operation of system 1600.
  • processing block 1710 is invoked to determine the decoding program which
  • the data file is partitioned into sub-files
  • block 1740 is invoked to detect each of the propagated, partitioned sub-files and its
  • processing block 1750 is operated to decode each
  • FIG. 16 allows playback of the stored
  • transmitter 1601 needs to find how each block, MB, and slice in the stored video file
  • the receiver would therefore process the
  • system 1800 composed of
  • transmitter 1801 and receiver 1802 which transceives stored compressed file 1610 to a
  • transmitter 1801 encoding
  • deducer 1815 is used to determine the overall encoding program producing compressed
  • the encoding program has a decoding counterpart which will be transmitted to
  • the deduced decoding program is stored in memory as
  • decoding program 1826 this memory is, for example, part of program memory 1825.
  • Program memory 1825 under control of transmitter processor 1835, stores software used
  • Segmentation device 1820 partitions the stored video
  • the decoding program is also partitioned into
  • packetizer 735 forms active packets from the independently generated sub-files and sub ⁇
  • Processor 1855 can store the parameters
  • the sub-files are extracted from data memory 745 by data extractor 150,
  • Flow diagram 1900 of FIG. 19 depicts the high-level operation of system
  • processing block 1910 is invoked to determine the decoding program which corresponds to the encoding algorithm used to encode the stored compressed data
  • program are, independently, partitioned into sub-files and sub-programs. Then, as is
  • Processing block 1940 is invoked to detect the sub-files and the sub-programs.
  • processing block 1950 is executed to
  • processing block 1760 is operated to decode
  • the receiver does not need to have the knowledge to differentiate the different schemes
  • the active packets can instruct the receiver how to
  • program information into active packets increases packet size and increases the network
  • decoding than decoding.
  • the following techniques can be used to reduce the packet size and traffic
  • the coding of video can be done in hardware, and custom hardware is
  • VLSI chips available for the implementation of video coding.
  • This hardware may be dedicated coders or programmable video signal
  • Dedicated coders provide limited programmability and have architecture or
  • program information with video data can be initiated by the receiver. Therefore when a
  • receiver requests a video file or stream that is encoded by a standard that the receiver
  • the receiver can instruct the sender to
  • the second method to reduce the packet size and traffic load can be done
  • the receiver can inform transmitter that part or all of

Abstract

Encoding a real-time and non real-time video stream (Fig. 10), encapsulating the encoded video stream along with the decoding software corresponding to the encoding program into active packets at the Application and Services layer, and transmitting active the packets (131) to the receiving terminal, which then supplies the receiver with the necessary decoding software to play any type of encoded video stream without having the associated decoding software pre-loaded.

Description

ACTIVE TECHNIQUES FOR VIDEO TRANSMISSION AND PLAYBACK
CROSS-REFERENCE TO RELATED APPLICATION
This application is a non-provisional application of provisional application
Serial No. 60/121,532 filed February 25, 1999.
BACKGROUND OF THE DISCLOSURE
1. Field of the Invention
This invention relates generally to video communications, and, more
particularly, to methodologies and concomitant circuitry for dynamically injecting video
coding software into a transmitted video stream so that receivers are capable of playing
video encoded in any format.
2. Description of the Background
Currently there are many different types of video coding standards which
can only be utilized for video playback at a receiver if the receiver has the correct
software pre-loaded. As the number of different encoding techniques grows and existing
video encoding software is updated, the receiver must load all the new and updated
software in order to play current video streams.
Representative of the conventional arrangement to transmit video frames
via packets through a packet network is the high-level depiction of video system 100 shown in FIG. 1. Each video frame produced by a standard source (not shown), as
exemplified by frame 110 serving as the input to transmitter 101, is compressed by
encoder 120 with reference to an encoding program stored in program memory 125, and
the encoded output appearing on lead 121 is formatted into packets 131 by data
packetizer 130. Transmitter processor 135 controls the interactions of encoder 120 with
program memory 125, and also provides the necessary control information so as to form
packets 131. In turn, packets 131 are transmitted via packet network 140 as propagating
packets 132 which are detected by receiver 102, where the packets are processed by data
extractor 150 to produce, on lead 151, the received counterpart of compressed output 121
in transmitter 101. The resulting data stream on lead 151 is decompressed by decoder
160 to produce received frame 111, a reproduced version of original frame 110.
In order to reproduce frame 111, it is necessary for decoder 160 to have
available decoding software which corresponds to the encoding software of encoder 120.
Conventionally this is accomplished by the prior loading and storing of the corresponding
decoding software in decoder 160 so as to display the video stream. Unfortunately from
the point of view of loading and storage, there are many different video coding standards,
including MPEG-1, MPEG-2, MPEG-4, MPEG-7, JPEG, H.261, and H.263. The
standards also keep evolving. Thus, it is sometimes the case that receiver 102 receives a
video stream 132 which cannot be played back because decoder 160 lacks suitable
decoding software, either because decoder 160 has not been loaded with the
commensurate software or the decoder is not compatible with the older or newer version
of compressed video. Currently users of system 100 are responsible for installing each unique piece of software that may be required in decoder 160 in order to decode a
particular video stream.
The subject matter of the present invention relates to: (a) encapsulating the
appropriate video decoding software, including the encoding algorithms, via transmitter
101; (b) bundling the decoding software with the actual video packet streams; and (c)
transmitting the encoded video along with specific decoding instructions to receiver 102.
This provides any properly equipped receiving terminal with the ability to play any type
of encoded video stream without having the associated decoding software pre-loaded,
thus creating a highly flexible and dynamic video transmission environment. The
methodology and concomitant circuitry of the present inventive subject matter engenders
what is referred to as "active techniques" for video.
Recently, the notion of "active networking" has been introduced; active
networking is intended to effect a significant change on the historical network paradigm,
namely, a change from a passive carrier of analog/digital signals to a more general
computational ability associated with network components, and has especially been
applied to switches and/or routers used to provide telecommunications services.
However, such efforts to this point in time have been devoted more to outlining the
benefits that such a paradigm could achieve, without elucidating specifics of such an
approach except in a few special cases.
For example, the paper entitled "On Active Networking and Congestion"
as authored by Bhattacharjee, Calvert, and Zegura (BCZ) in January, 1996 and published as Georgia Institute of Technology Technical report GIT-CC-96/02, focuses on applying
active networking concepts to handling network congestion. In BCZ, the model of what
happens when a packet arrives at a node (used interchangeably with switch or router) is as
follows -- for purposes of discussion, a packet is composed of a header part and a payload
part:
(1) The output destination port for the packet is computed as usual.
(2) If a packet contains a valid Active Processing Function Identifier (ACPI), it is
sent to an active processor and processing continues; otherwise, it is transmitted as usual.
(3) The function specified in the ACPI is computed, using the packet's association
descriptor and user data as inputs.
(4) If the result of the function is transformed data (e.g., reduced length), the
packet's network-level header and ACPI are recomputed as necessary; the node's state is
updated as required by the specified function.
(5) The (possibly modified) packet is transmitted to its next-hop node.
It is extremely important to reiterate that the above procedure requires an
Active Processing Function Identifier (ACPI) to differentiate between conventional
processing and additional, that is, active processing. As BCZ further point out, the
obvious place to put the ACPI is in the same header used to switch the packet. However,
BCZ concludes that such an approach is unacceptable for at least two reasons. First, the
approach does not work for ATM or any other technology where the switched unit is too
small to accommodate additional overhead of the ACPI. And second, the approach is not
backward-compatible, requiring that all network protocols become "active-aware". BCZ proposes that an alternative to placing the ACPI in the network header itself is to define a
"generic" location for the ACPI function, sufficiently high in the protocol stack that the
additional processing overhead is not prohibitive, but sufficiently low in the protocol
stack to allow its location by switching nodes without too much knowledge of higher-
level protocols. Thus, BCZ immediately rules out the use of the packet itself for
differentiating between conventional and active processing. However, use of the packet
(either the header, payload, or both) overcomes what BCZ deems to be unacceptable, that
is, use of the packet itself eliminates additional packet overhead, and network protocols
need not be "active-aware".
Moreover, in the BCZ approach, there is no program portion in the packet.
Programs are embedded into the node. There is only a set of predefined computations
which can be performed in the node. A node which has the computational power is
called an active processor (AP). Header information in each packet specifies which
computation is to be performed on it. For example, for MPEG packets, the fields in the
header indicate the priority of particular packets (for example, I, P, and B frames, as
further discussed below). This priority is used in the AP to decide which packet should be
dropped to avoid congestion.
Consequently, the prior art is devoid of teachings or suggestions relating
to: encapsulating the appropriate video decoding algorithms and software, bundling them
with the actual video streams, and transmitting the encoded video along with specific
decoding instructions to the receiving terminal, which then allows properly equipped
receiving terminals the ability to play any type of encoded video stream without having the associated decoding software pre-loaded, thus creating a highly flexible and dynamic
video transmission environment.
SUMMARY OF THE INVENTION
Shortcomings and limitations of the prior art are obviated, in accordance
with the present invention, by a methodology and concomitant circuitry wherein,
generally, the programming code to decode encoded video is bundled with the encoded
video in the same propagation stream so that the appropriate decoding program is readily
available without the need to configure the receiver beforehand.
Broadly, in accordance with one method aspect of the present invention, a
method for transceiving a real-time video frame includes the following procedure:
(a) determining a decoding program for the compressed data file; (b) independently
partitioning the compressed data file into data sub-files and the decoding program into
decoding sub-programs; (c) propagating a sequence of the sub-files and the sub-
programs; (d) detecting the sequence to determine the sub-files and the sub-programs;
and (e)
combining the sub-files to reproduce the compressed data file and combining the sub¬
programs to reproduce the decoding program.
Broadly, in accordance with another method aspect of the present
invention, a method for transceiving a real-time frame includes the following procedure:
(a) determining a decoding program for the compressed data file; (b) independently
partitioning the compressed data file into data sub-files and the decoding program into
decoding sub-programs; (c) propagating a sequence of active packets composed of the sub-files and the sub-programs; (d) detecting the sequence of active packets to determine
the sub-files and the sub-programs; and (e) combining the sub-files to reproduce the
compressed data file and combining the sub-programs to reproduce the decoding
program.
BRIEF DESCRIPTION OF THE DRAWINGS
The teachings of the present invention can be readily understood by
considering the following detailed description in conjunction with the accompanying
drawings, in which:
FIG. 1 depicts a conventional system for propagating packets representative of
video frame information;
FIG. 2 depicts the hierarchy of the various streams used in the MPEG-2 standard;
FIGS. 3 A, 3B, and 3C depict, respectively: the components of a frame; the spatial
and temporal encoding, both for the MPEG-2 standard; and the layer structure for the
MPEG-2 standard;
FIGS. 4A, 4B, and 4C depict the composition of MPEG-2 packets, at high level
and in detail, including a payload conveying a Program Association Table;
FIG. 5 depicts a MPEG-2 Transport Stream and the relation of the packets in the
Transport Stream to the Program Association Table, the Program Map Table, and the
Network Information Table;
FIG. 6 illustrates an example of a conventional protocol stack for encapsulating
packets over the Internet;
FIG. 7 is a high-level block diagram of an illustrative embodiment of the real-time
video system of the present invention; FIG. 8 illustrates the "active activation" as part of the application layer of the
conventional protocol stack, this part being used to encapsulate the encoded video frame
and bundle the program used to generate the encoded video frame for self-decoding at the
receiver;
FIG. 9 depicts the contents of an active packet, including the active packet header,
the data payload, and the program payload;
FIG. 10 illustrates the generation of an active packet transport stream in a manner
analogous to generation of the transport stream of FIG. 5;
FIG. 11 depicts representative contents of the data memory and program memory
of the receiver of FIG. 7 in accordance with the present invention;
FIG. 12 is a flow diagram for generating an active packet in the transmitter of
FIG. 7 in accordance with the present invention;
FIG. 13 is a flow diagram for assigning and associating the data packet identifier,
and program packet identifier, if necessary, in an active packet;
FIG. 14 is a flow diagram for receiving active packets to generate a reproduced
version of the input video frame in the receiver of FIG. 7;
FIG. 15 is a high-level block diagram of another illustrative embodiment of the
real-time video system of the present invention;
FIG. 16 is a high-level block diagram of an illustrative embodiment of the non-
real time video system of the present invention;
FIG. 17 is a flow diagram for the operation of the system of FIG. 16;
FIG. 18 is a high-level block diagram of another illustrative embodiment of the
non-real time video system of the present invention; and FIG. 19 is a flow diagram for the operation of the system of FIG. 18.
To facilitate understanding, identical reference numerals have been used,
where possible, to designate identical elements that are common to the figures.
DETAILED DESCRIPTION
To fully appreciate the import of the signal processing system of the
present invention, as well as to gain an appreciation for the underlying operational
principles of the present invention, it is instructive to first present, in overview fashion, a
high-level description of a conventional system for transmitting a sequence of video
frames. This overview also serves to introduce terminology so as to facilitate the more
detailed description of an illustrative embodiment in accordance with the present
invention. Following this overview, a description of the illustrative embodiment is then
elucidated.
Overview of a Conventional Video System
Video transmission is typically broadband in nature. For example,
compressed High Definition Television (HDTV) requires a bandwidth between 20-to-40
Mbps. Compression algorithms at the source reduce the required bandwidth for the
transmission medium and yet maintain reasonable video quality at the receiver.
The pictorial representation of FIG. 2 illustrates the protocols levels that
data packetizer 130 of FIG. 1 generate so as to transport a digital video frame from the
source to the receiver for MPEG-2, which standard is taken as representative of modern
techniques for transporting video signals. In particular, each frame 110 received by encoder 120 is transformed into a compressed video digital representation of the frame,
which is referred to as the "encoded video" of the frame and appears on lead 121. Data
packetizer 130 generally includes elements, corresponding to the MPEG-2 standard, for
generating: (a) an elementary stream (ES) from the encoded video (element 210); (b) a
packetized elementary stream (PES) from the elementary stream (element 220); and (c) a
transport stream from one or more PESs (element 230) to derive the MPEG-2 packets 131
ready for transport over network 140. The encoded video is processed by element 210 by
adding header information that is used to reconstruct the frames at the receiving end.
However, no timing or system data is present at this point in the packaging of the
encoded video. Next, element 220 adds timing information (e.g., the Presentation Time
Stamp (PTS) and the Decode Time Stamp (DTS), as discussed below in more detail) so
as to inform the receiver as to the time to decode as well as the order to decode. Finally,
element 230 adds system information as well as a clock reference (e.g., the Program
Clock Reference (PCR)) for timing information. Thus, in a generic sense, data packetizer
130 transforms the encoded video to the transport stream which contains all necessary
information to re-transform the transport stream to derive the encoded video.
Encoding of Video
Numerous algorithms for video encoding have been devised to effect
compression at the source. The video source which produces a sequence of frames, with
frame 110 in FIG. 1 being representative of one such frame, generates a specified number
of frames per second; for example, a NTSC-compatible source generates 30 frames per
second (fps), and a video telephony source generates 10 fps. Generally, there are two coding techniques for compression of video information, namely, "intraframe" and
"interframe" coding. Intraframe coding (an I-frame as used below) exploits the spatial
redundancy within a frame, while interframe coding exploits temporal redundancy
between frames. An I-frame contains full picture information. In the spatial domain,
each video frame is divided into a "block", for example, of 8 X 8 pixels. The Discrete
Cosine Transform (DCT), for example, is applied to these blocks, and the DCT
coefficients are quantized to a user-defined level. Four blocks are grouped to create a
"macroblock" (MB). A number of row-wise adjacent MBs are grouped to create a "
slice". A slice is the basic transmission and resynchronization unit for the video stream.
In the temporal domain, there are three frames for MPEG-2: intra (I),
predictive (P), and bidirectional (B) frames. The coding for an I-frame is based on spatial
redundancy as described above. The purpose of the P-frame is to reduce temporal
redundancy by motion estimation. This is accomplished by searching for a best-match
MB from the previous I or P frame, for every MB in the current frame. The "motion
compensated predictive error", the difference between this MB and the best-match MB,
becomes part of the information transmitted to the receiving end. For a B frame, the best-
match MB search from both the previous and next I or P frame is effected. Both forward
prediction and backward prediction can be used as the motion compensated prediction.
Since the B frame needs the information from the previous and the next I or P frames, the
processing order is not identical to the natural sequence of the video frame. FIG. 3 A
summarizes the foregoing discussion with respect to the arrangement of blocks (310),
microblocks (311), and slices (312) for the given video frame 110. FIG. 3B summarizes I, P, and B frames. In FIG. 3B, frame 301 is shown as being composed of block 3011,
macroblock 3012, and slice 3013. Frames 302-307 are, respectively, presumed to be B,P,
B, P, B, and I frames. Whereas the displaying order is the natural order of the frames,
that is, the sequential order 0, 1, 2, ... 6 (that is, the Presentation Time Stamp), FIG. 3B
also highlights the fact that the processing order (that is, the Decode Time Stamp) for
encoding and decoding of the frames, namely, 0, 2, 1, 4, 3, 6, and 5, is different than the
displaying order at the receiver. Finally, video sequences are sent as a group of pictures
(GOP) which contains several frames; FIG. 3B shows how the set of seven frames
constitute a GOP. FIG. 3C summarizes the various layers utilized by the MPEG-2
standard so as to convert a frame into data suitably arranged for transmission as packets
131 of FIG. 1. In particular, each packet contains high-level information and data
representative of sequence layer 320 (which is oftentimes referred to as the packet
"payload", as discussed in more detail shortly). In turn, sequence layer 320 is composed
of Group of Pictures layer 321; layer 321 is composed of Picture/Frame layer 322; layer
322 is composed of Slice layer 323; layer 323 is composed of Macroblock layer 324; and,
finally, layer 324 is composed of Block layer 325.
This completes the high-level view of MPEG-2 encoding. Other
standards are now briefly discussed, and compared to MPEG-2 where appropriate. The
JPEG standard used to encode "still" images can be used for video systems by treating
each frame as a still image. The coding scheme for each frame in JPEG is similar to
intraframe coding of MPEG-2. The H.261 standard was developed for videophone and
video conferencing and is similar to MPEG-2 but only has I and P frames. MPEG-1 also defines I, P, and B frames, and the coding algorithms are similar. MPEG-2, however,
has some enhanced features, such as interlaced video manipulation, scaleability,
compatibility, error resilience, and so forth. Like MPEG-2, the H.263 standard is a
hybrid of interframe prediction to reduce the temporal redundancy and intraframe coding.
However, additional frame modes are defined for coding.
Packetizing of the Compressed Video
The foregoing portion of this overview discussed the coding aspects of the
various standards at the source, that is, how the video is compressed to a low enough bit
rate so that the video can make economic use of the available transmission bandwidth, as
well as the packetizing aspects of device 130 at high-level.
The next part of the overview discusses the creation of the transport stream
encapsulating the encoded video output of the encoder, as alluded to in FIG. 2, again for
the MPEG-2 standard as being representative of the encapsulating aspects of the other
coding standards. In particular, it is necessary to format the compressed video into
packets at the source which conform to the MPEG-2 standard for eventual propagation
and ultimate detection of the full video in the receiver. Data packetizer 130 of FIG. 1
accomplishes the generation of packets 131 conforming to, as exemplary, the MPEG-2
standard. The description follows the approach of first describing the contents of a
MPEG-2 packet, followed by a description how the sequence of packets is transformed to
produce the transport stream, so that the connection between the encoded video and
MPEG-2 packets composing a transport stream is fully elucidated. With reference to FIG. 4A, stream 400 of MPEG-2 packets is depicted
wherein each packet further comprises a header/payload pair, namely, header 401 and its
accompanying payload 402, header 403 and its accompanying payload 404, and so forth.
Header 401 is generally four-bytes long, and payload 402 is 184-bytes. Stream 400 is
emitted by data packetizer 130 of FIG. 1 and, in the terminology of MPEG-2, is a
Transport Stream.
As further shown in FIG. 4B, each header contains at least eight fields; in
FIG. 4B an illustrative value, in decimal, for each field is shown below its name, and
below that value is the bit-length of the field. Of particular importance for the present
invention, as discussed in detail later, is PID (Packet Identifier) field 411. In addition, the
payload is composed of components of the compressed video (or in other applications,
audio, data, and teletext/closed captioning), as well as referencing information, as
arranged according to the layered structure of FIG. 3C.
In FIG. 4C, payload 402 is shown as providing Program Association Table
(PAT), which is the payload associated with PID = 0 (hereafter, PID 0) and which is
composed of entries identified by reference numerals 422, 423, 424, .... In particular,
PAT 402 lists information such as: PID 16 is associated with Program 0 (reference
numeral 422), PID 110 is associated with Program 1 (reference numeral 423), PID 210 is
associated with Program 2 (reference numeral 424), and so forth. In MPEG-2
terminology, a 'Program' is a collection of streams within a common timebase, which
certainly pertains to the sequence of video frames in accordance with the present
invention. (Note that the definition of Program is broader than that used in standard TV
broadcasting wherein a program is a collection of streams not only with a common timebase, but also with a common start and end time. The MPEG-2 definition subsumes
the broadcast definition).
A more detailed view of Transport Stream 400 is shown in FIG. 5 at the
bottom wherein Stream 400 has combined the packets with the following PID's into a
single stream: 0, 110, 210, 1, 16, 111, 212, 111, 211, 111, 113, 213, .... The manner of
interleaving these individual packets is covered in the sequel.
A PID 1 packet, by definition in the MPEG-2 standard, contains a
Conditional Access Table (CAT), which includes scrambling information as well as an
Entitlement Management Message (EMM) that labels each program as to its accessibility
by user's equipment. A PID 16 packet, by definition, always has a payload of listing the
Network Information Table (NIT) of the originating system's name, location, and similar
identifying information. Again, a PID 0 packet provides the Program Association Table.
Transport Stream 400 commences propagation with a packet having
PID 0. As shown in FIG. 4C, and as repeated in FIG. 5, the PID 0 packet transmits PAT
402. The further breakdown of the PAT is shown in FIG. 5, wherein PID 16 is linked to
NIT 510, PID 110 is linked to Program Map Table (PMT) 520, and PID 210 is linked to
PMT 530. PMT's 520 and 530 appear as the payloads in the two packets following the
initiating PID 0 packet. A PMT provides more detailed referencing information to further
define the mapping between the encoded video stream and the actual packets prepared for
transmission, and is used at the receiving end to properly decode the Transport Stream.
PMT 520 lists, as identified by row 521, the Stream Identifier, the Type of
signal (e.g., video, audio, data), and a PID assigned to that type by the source. In particular, as depicted by row 522, a packet associated with Stream 1 of Program 1 is of
type 'Video", and the packet containing this video has PID 111. Also, the packet
associated with Stream 1 of type 'Data" has PID 113. Moreover, PMT 520 exemplifies
that another Stream 2, of type 'Audio" having PID 112, has been combined with Stream
1. With reference to Transport Stream 400, it is seen that PID 111 packets appear in the
sixth, eighth, and tenth time slots of Stream 400. Also, a PID 113 packet appears in the
eleventh time slot.
PMT 530 is arranged similarly to PMT 520, and it is seen that, for
example, PID 212 associated with Stream 2 of Program 2, of type 'Audio", appears in the
seventh time slot of Transport Stream 400.
By way of reiteration, the function of encoder 120 may be aptly described
as a process of applying an encoding algorithm to a sequence of incoming video frames,
the algorithm typically being effected by software resident program memory 125, to
produce a digital representation of the sequence of video frames, this digital
representation be called the encoded video. In addition, the function of data packetizer
130 may be aptly described as a process of appending information to the encoded video
to produce a series of packets, conforming to a standard, prepared for transmission over
an intervening packet network. Elements 120, 125 and 130 operate under control of
transmitter processor 135.
Propagation of Transport Stream over the Internet The pictorial representation of FIG. 6 illustrates an example of the
conventional manner for transmitting an encoded video stream, for which MPEG-2 is
taken as representative of other techniques, over the Internet using "protocol stack" 600.
The stream handled by the MPEG-2 standard is packet 601 of FIG. 6. Starting at the top
and working down in stack 600, "Applications & Services" level 610 produces the
encoded video stream, namely, packet 400. "Transport" layer 620 encapsulates each
packet in the MPEG-2 stream by appending headers, which are conventionally the RTP
and UDP headers 602; this layer effects, for example, end-to-end delivery, connection
management, and flow control. "Network" layer 630 further encapsulates the result of
layer 620 by appending the IP header 603; this layer effects, for example, routing or re¬
routing. "Data Link" layer 640 accomplishes error control and access control and further
encapsulates the result of layer 630 by appending an Ethernet header 604 for instance.
"Physical" layer 650 engenders the actual transmission at the bit-level. In accordance
with the present invention, an "active activation" aspect of the "Application & Services'
layer of the protocol stack will be treated explicitly, as described in the sequel.
Decoding of Video
The purpose of data extractor 150 of FIG. 1 is that of ultimately recovering
the blocks in Block layer 325 of FIG. 3C so that these blocks, being representative of the
sequence of video frames, may be decompressed by decoder 160. To accomplish this, a
number of intermediate steps are required, namely: (1) with reference to FIG. 6, removal
of the various "stack" headers such as the Ethernet, IP, UDP, and RTP headers; (2) with
reference to FIG. 5, the identification of the video stream portions (e.g., PID 111) of the transport stream 400; (3) with reference to FIG. 3C, processing the hierarchical layer
structure to obtain each block 325 by decomposing hierarchy starting with the sequence
layer 320 and working down to the macroblock layer 324; and, (4) recovering the
Program Clock Reference, Presentation Time Stamp, and Decode Time Stamp to re-
sequence the frames in the correct position according to the reference clock.
Illustrative Embodiment in Accordance with the Present Invention
The method of the present inventive subject matter leverages active
networking techniques by employing so-called "active packets" to carry decoding
software representative of the code necessary to reconstruct the video frames of the
associated compressed video data transport stream.
1.) Real-time Video
With reference to FIG. 7, which shows the elements of the present
invention overlaid on the conventional system of FIG. 1, transmitter 701 generates and
emits active packets for a sequence of video frames, again with frame 110 being
exemplary. Receiver 702 only needs to extract the code from the active packets to obtain
the appropriate program for decoding the video data stream to produce received frames,
with frame 111 being representative of one received frame. Then the video stream can be
decoded and displayed using the program instructions that are extracted from the active
packets. In this manner, receiver 702 is capable of displaying any type of encoded video
stream.
- ii In particular, referring to FIG. 7, system 700 in accordance with the
present invention is composed of: (a) transmitter 701; (b) receiver 702; and (c) network
140 interconnecting transmitter 701 and receiver 702.
Transmitter 701 is further composed of: (i) the components of transmitter
101 of FIG. 1, namely, encoder 120, program memory 125, data packetizer 130, and
transmitter processor 135; (ii) decoding program 725 augmenting program memory 125;
and (iii) "active" packetizer 735, which operates under control of transmitter processor
135. Packets appearing on lead 131 from data packetizer 130 are the same as produced
by system 100 of FIG. 1. However, packets 131 now serve as one input to active
packetizer 735 rather than being propagated directly by network 140 of FIG. 1 ; in
addition, decoding memory 725 provides a second input to active packetizer 735. The
output of active packetizer 735 is the stream of active packets 731.
Receiver 702 is composed of: (i) data and program separator 740; (ii) data
memory 745 coupled to separator 740; (iii) data extractor 150, as in FIG. 1; (iv) decoding
program memory 750, also coupled to separator 740; (v) decoder 160, as in FIG. 1; and
(vi) receiver processor 755 which controls the elements of receiver 702. Decoder 160 has
as inputs, in addition to processor 755: (a) the data output of data extractor 150, and (b)
decoding program software stored in decoding program memory 750.
Transmitter 701
The process of creating active packets 731 by transmitter 701 is described
with reference to FIG. 8, which is a modification of FIG. 6 in accordance with the present
invention. In fact, the generation of the protocol stack of FIG. 8 is patterned after the technique by which the protocol stack of FIG. 6 is generated, namely, by adding certain
information to the data produced by the next higher level in the stack. Thus, Application
and Services layer 610 has appended to it "active activation" sub-layer 810. Activation
sub-layer 810 appends certain information to each MPEG-2 packet 400, namely, active
header 820 is added before packet 400, and program packet 830 is added after packet 400
to form, in combination, each active packet 840 - an exemplary packet in packet stream
731 of FIG. 7.
The manner in which information is added to packet 400 is shown in more
detail in FIG. 9. Active packet header 820 is composed of: (a) active packet identifier
(APID) 912; (b) data packet pointer 913, which points to the initial location of data
packet 400; (c) data packet identifier (DPID) 914, which identifies the data packet; and
(d) program packet pointer 915, which points to the initial location of program packet
830. In turn, the format of program packet 830, as shown in the lower portion of FIG. 9,
is composed of packet header 931, which includes DPID 914, and packet payload 933. In
particular, payload 933 conveys the decoding program which will be used by receiver 702
to decode the data packet having the associated DPID 914. Thus, DPID 914 serves as
the link between data packet 400 and program packet 830 used to decode the data packet.
In MPEG-2 applications, it is usually the case that the program to decode
data in an MPEG-2 transport stream is the same from data packet-to-data packet. Thus,
the decoding program code need only be sent once, with the first data packet having a
given DPID. All other data packets are then assigned the same DPID, and the single
decoding program is used to decode successive packets. On the other hand, in other applications, it may be necessary to send
decoding program code with more than the first data packet, so the active packet structure
of FIG. 9 is illustrative of the most general case. Also, it may be necessary, because of
packet size limitations, to spread the decoding program over a number of active packets,
and then re-assemble the decoding programming code from a number of active packets to
produce the complete decoding program. In this case, to effect re-assembly, a program
packet identifier (PPID), such as PPID 932, will aid in the re-assembly process, that is,
segments of a complete program can be assigned and identified by the same PPID to
accomplish re-assembly of the complete code.
Active Packet Stream
Active Packet Stream 1000 of FIGJ0, which transports active packets 731
of FIG. 7, is formed in a manner analogous to the formation of Transport Stream 400 of
FIGS. 4 and 5, that is, the formation of the Active Packet Stream 1000 encapsulates the
standard MPEG-2 packets with header and payload information which identifies, as
needed, the Program Association Table 1011, the Program Map Tables 1020 and 1030, as
well as the APID 1 and APID 16 (CAT/EMM data, and NIT table 1010, respectively)
wherein APID is shorthand for Active Packet IDentifier. For example, the first stream
associated with PMT 1020 transports a video stream having APID 1111. Each APID is
utilized by receiver 702 to compose a video stream originated from the same video
source.
Receiver 702 As alluded to above, receiver 702 processes the incoming active packet
stream to separate the data payload from the program payload in separator 740. Of
course, prior to processing the active packet, processor 740 also removes all headers
encapsulating each active packet (e.g., from FIG. 8, the Ethernet, IP, UDP, and RTP
headers) to first derive the active packet. The data payload portion of the active packet is
stored in data memory 745 in a form illustrated in the top-half of FIG.11, that is, data
memory 745 generally stores the data itself (e.g. , '01000101111 ...') as well as the
associated DPID (e.g., 2111) for the data transported by each data payload. In addition,
the program part of the active packet is stored in program memory 750 in a form
illustrated in the bottom-half of FIG. 11, that is, program memory 750 generally stores
the program itself (e.g., binary of decoding algorithm for data with DPID 2111), as well
as the associated DPID and PPID, if necessary (e.g., PPID 2112 associated with DPID
2111). The next two entries in memory 750 are representative of a program that cannot
be encompassed by a single program payload, so two program payloads are required, as
indicated by the same PPID 2114 for the second and third row entries. The first program
payload contains, illustratively, the first part of the binary decoding algorithm for the data
with DPID 2113, whereas the second program payload conveys the remainder of the
program code for decoding the data with DPID 2113. To convey the fact that the
program is split between two payloads, program payload header 931 may incorporate a
"flag" indicative of the split program. Thus, it is clear that an active packet may have the
following forms: (a) a data payload plus a program payload; (b) a data payload only
(whenever the same program is used to decode, and it is sent with the first data payload);
and (c) a program payload for programs too large to fit into a single program payload. Data extractor 150 removes from memory 745, illustratively, only the
"raw" data (e.g., '01000101111 ...") for processing by decoder 160.
Receiver processor 755 controls the overall processing of receiver 702,
and one main function is that of ensuring that the appropriate program to decode a given
data payload is completely present in memory 750 before invoking the decoding effected
by decoder 160.
Flow Diagrams for Transmitter 701
Flow diagram 1200 of FIG. 12 is representative of high-level processing
effected by transmitter 701. With reference to FIG. 12, processing is initiated by
processing block 1201. As then invoked by processing block 1210, the incoming real¬
time video stream is encoded. With every encoding program, there is an associated
decoding program, and processing block 1220 is used to specify the decoding program
(e.g., if the encoding is accomplished by the Discrete Cosine Transform, the decoding
will utilize the Inverse Discrete Cosine Transform). Next, decision block 1230 is tested
to determine if the program has already been sent, which may occur if the same decoding
program is used for all frames in a given video stream. If the program has not already
been sent, then the program is retrieved via processing by block 1240. Once retrieved,
the program packet is formed by processing block 1250; the formation includes adding
the DPID, PPID, and the "flag" to indicate if the complete program is in this payload.
Once the data is encoded by block 1210, then the transport stream is formed by block
1260 - illustratively, this is the MPEG-2 transport stream. Finally, as indicated by
processing block 1270, the active packet is generated; each active packet is transported as part of active packet stream 1000 as explained above. It is clear that flow diagram is but
one illustrative embodiment of the flow carried out by transmitter 701; thus, for instance,
encoding by block 1210 and specifying by block 1220 can be completed in parallel.
Flow diagram 1300 of FIG. 13 depicts the details of forming the active
packet. In particular, as indicated by processing block 1310, a data packet identifier
DPID is assigned to each data packet and a corresponding program packet identifier
PPID, if any, is assigned to the program packet. It is necessary to correlate the DPID and
PPID, and this step is completed by processing block 1320. Next, as evidenced by
processing block 1330, the DPID is entered into the program packet to achieve the
coupling between the data payload and program payload necessary to engender decoding
at the receiver. Then processing block 1340 is executed to generate the active packet
header corresponding to the given data packet and program packet; the header
information includes the data and program pointers, if any. Finally, the active packet is
formed in processing block 1350 by combining the active packet header, the data packet,
and the program packet.
Flow Diagram for Receiver 702
Flow diagram 1400 of FIG. 14 is representative of the processing effected
by receiver 702 on incoming active packet stream 1000. In particular, processing block
1410 removes all transport headers (such as the Ethernet, IP, UDP, and RTP) as well as
the active header information (such as the data packet and program packet pointers).
Then processing block 1420 is invoked to remove the data payload for storage in 745. Similarly, the corresponding program payload is removed by processing block 1430, with
the resulting program payload being stored in memory 750. Before decoding can
commence, processing by decision block 1440 determines if the program for the given
DPID is in memory - because of the presentation time stamp may be different than the
decode time stamp. It is also necessary to ensure that the complete program is available
for decoding, which is the function of decision block 1450. Once the program to decode
a given data payload is present, this program is loaded into decoder 160 under control of
processor 755, as summarized by processing block 1460. Finally, the associated data
corresponding to the decoding program is extracted from data memory 745 by data
extractor 150 and this data is decoded in decoder 160.
Generalizations
Whereas the description to this point has focused on the transmission and
reception of real-time video, it has been presumed both the encoding of and transporting
of the video frame, that is, packetizing of the encoded video, has been explicitly specified
so as to transport a sequence of video frames, such as by the MPEG-2 standard.
However, there are video codecs (encoder-decoder combinations) whereby only the
encoding technique is explicitly specified, and the formation of the propagating stream is
determined by the system developer.
In the former case, parameters relating to decoding such as quantizer scale
of the device effecting video encoding, motion vectors for related to I, P, and B frames,
frame type, and so forth, are explicitly conveyed by header information in the packet. In the latter case, it is necessary to supply such decoding information with
the program payload. Thus, for these cases, system 700 of FIG. 7 is modified as per the
following description.
With reference to FIG. 15, for those video codecs which do not specific
how the video data should be transported, packetization (now referred to as
segmentation for purposes of distinguishing over the prior inventive aspect) and
encoding are combined in the process of generating active packets; this merging of
encoding and segmentation is depicted by element 1520 of FIG. 15. As before, each
I-frame is divided into blocks according to the setting in the transmitter, that is, a
frame is segmented according to the setting of each block. A compression algorithm,
for example the Discrete Cosine Transform (DCT), is then applied to these blocks.
The program to decompress each block is filled into the program packets. The
programs to assemble the program packets and to desegment the data packets along
with required decoding parameters are also encapsulated into the program packets; the
parameters for encoding and segmentation are managed by transmitter processor 1535,
which is essentially identical to transmitter processor 135 of FIG. 7, but in this case
processor 1535 explicitly depicts the accessibility of the "parameters" in forming
active packets in active packetizer 735. Once again, it may be necessary to only use
the same inversion program, for instance IDCT, if all subsequent blocks use the same
compression algorithm - then it is only the necessary parameters and coefficients for
decompression that have to be injected into the program packets of active packets.
At receiver end in FIG. 15, receiver 1502 operates as follows. As
before, the data and program portions of active packets are separated in separator 740. The data packets contain compressed data for each block. The program to decode a
block is stored in program memory 750; if the program to decode a block is
encapsulated into more than one active packet, the program packets are then compiled
by receiver processor 1555 to assemble the complete decoding program which is then
stored in program memory 750. Also, the parameters that are detected in the program
packets are, for example, managed by receiver processor 1555. Next, decoding and
desegmentation is effected by device 1560, that is: the program to decode the block is
executed to obtain the restored data of each block, and when all blocks of a frame are
collected, the program is executed to desegment or re-group these blocks to generate
the original uncompressed frame. This frame is then displayed if this frame has no
relationship with preceding or subsequent frames. Otherwise, this frame is stored in
memory, which may be part of receiver processor 1555, until all related frames are restored.
Moreover, although there are different approaches for real-time video
transmission and reception, the receiver need not have knowledge to differentiate the
different techniques performed at the transmitter. Information in the active packets
themselves can instruct the receiver how to reconstruct and display the video.
2. Stored Video - Illustrative Embodiment 1
Stored video, or non-real time video, corresponds to a video stream that
has been operated on by a video standard to produce a compressed file corresponding to the video stream. Therefore, the encoding/packetizing or encoding/segmentation cannot
be performed together, as in the real-time scenarios discussed above, because the
compression of the complete video stream has already been effected. Thus, the
partitioning of the compressed file to generate the data packets for encapsulation into
active packets is performed independent of the compression technique. However, the
program for decoding the compressed file at the receiver must be determined first by
requiring the transmitter to deduce the encoding algorithm which was used to produce the
stored compressed file. These aspects of this illustrative embodiment are summarized by
the high-level block diagram of FIG. 16 which depicts overall non-real time video system
1600 as being composed of transmitter 1601 and receiver 1602. In transmitter 1601,
encoding deducer 1615 determines the algorithm used to compress stored video 1610.
This algorithm has a decoding counterpart which is stored, upon its determination, in
decoding program memory 1626 which is, for example, an adjunct to program memory
1625; program memory 1625 contains software for controlling encoding deducer 1615.
Parameters determined from the deduction process are managed by transmitter processor
1635. Segmentation device 1620 partitions the stored video data file into data sub-files
for transmission as data packets in active packets 731. When the active packets (e.g.,
packets 731) are assembled by active packetizer 735, the deduced program code used to
decode the video data is injected into the program packet portion of the active packets
along with the discerned parameters.
At the receiver end, receiver 1602 is essentially the same as receiver 1502
of FIG. 15, with some minor variations in the operation of receiver 1602, as now
described. Program information and the data are extracted from the active packets via data and program separator 740, wherein data is stored in data memory 745 and the
decoding program is stored in program memory 750. The data is extracted by data
extractor 150, and this data is then decoded and desegmented by the processing of
element 1660 with reference to the associated program. This results in a reproduced
version of the original video stream designated as received video 1611. Receiver
processor 1655 provides the overall control of the other components of receiver 1602.
Flow Diagram for System 1600
Flow diagram 1700 of FIG. 17 depicts the high-level operation of system 1600.
Initially, processing block 1710 is invoked to determine the decoding program which
corresponds to the encoding algorithm used to encode the stored compressed data file.
Next, as carried out by processing block 1720, the data file is partitioned into sub-files
with reference to the deduced decoding program as discerned from the compressed data
file. Then, as is evidenced by processing block 1730, each of the sub-files and its
associated decoding program its propagated over, for instance, network 140. Processing
block 1740 is invoked to detect each of the propagated, partitioned sub-files and its
associated decoding program. Finally, processing block 1750 is operated to decode each
detected partitioned sub-file with reference to its decoding program.
The illustrative embodiment of FIG. 16 allows playback of the stored
video in receiver 1602 without waiting for the reception of the complete stored video file
1610. However, since the compression of stored video has been performed before the partitioning, that is, segmentation, this technique can suffer from overhead because
transmitter 1601 needs to find how each block, MB, and slice in the stored video file
1610 are encoded, and then inject the segmented data with the associated decoding
program into the same active packet.
3. Stored Video - Illustrative Embodiment 2
If it is not possible to associate a decoding program with each data sub¬
file due to complexity or other reasons, it may be necessary to arrange the transmitter to
inject program information which is not the decoding program specified for the particular
video data portion in the same active packet. The receiver would therefore process the
complete sequence of active packets to accumulate all the stored data as well as the
complete decoding program so as to reproduce a received version of the original stored
compressed data file.
With reference to FIG. 18, there is shown system 1800 composed of
transmitter 1801 and receiver 1802 which transceives stored compressed file 1610 to a
reproduced version 1611 of the original compressed file. In transmitter 1801, encoding
deducer 1815 is used to determine the overall encoding program producing compressed
file 1610. The encoding program has a decoding counterpart which will be transmitted to
receiver 1802 to effect decoding. The deduced decoding program is stored in memory as
decoding program 1826; this memory is, for example, part of program memory 1825.
Program memory 1825, under control of transmitter processor 1835, stores software used
to deduce the encoding algorithm and generate the decoding program. Parameters
required for decoding are also deduced while determining the encoding algorithm; such parameters are stored in transmitter processor 1835 for eventual packaging with the
program part of the active packets. Segmentation device 1820 partitions the stored video
data file into data sub-files for transmission as data packets in active packets 731.
Independently of the segmentation process, the decoding program is also partitioned into
data sub-programs which become the program portion of the active packets. Active
packetizer 735 forms active packets from the independently generated sub-files and sub¬
programs.
In receiver 1802, data and program separator 740 forwards the data sub¬
files to data memory 745 and the sub-programs to decoding program memory 750, both
under control of receiver processor 1855. Processor 1855 can store the parameters
transmitted as part of the program portion of active packets. Once all active packets have
been received, the sub-files are extracted from data memory 745 by data extractor 150,
and then processed by desegmentation device 1860 to recover a contiguous data file
which is a reconstructed version of original data file 1610. The reconstructed version,
which is the output of device 1860, is then decoded via decoder 1865 to yield received
video 1611, which corresponds to the original video stream which was encoded to obtain
stored video file 1610. Since this is not a real-time application, the delay caused by
waiting for the complete transmission of the stored data file should be acceptable.
Flow Diagram for System 1800
Flow diagram 1900 of FIG. 19 depicts the high-level operation of system
1800. Initially, processing block 1910 is invoked to determine the decoding program which corresponds to the encoding algorithm used to encode the stored compressed data
file. Next, as carried out by processing block 1720, the data file and the decoding
program are, independently, partitioned into sub-files and sub-programs. Then, as is
evidenced by processing block 1930, the sub-files and the sub-programs are used for
forming active packets, and the active packets are propagated, for instance, over network
140. Processing block 1940 is invoked to detect the sub-files and the sub-programs.
Once all of the active packets are received, then processing block 1950 is executed to
combine the sub-files into a desegmented detected data file and to combine the sub¬
programs into a decoding program. Finally, processing block 1760 is operated to decode
the detected data file with reference to the decoding program.
Although there are different approaches for real-time and stored video,
the receiver does not need to have the knowledge to differentiate the different schemes
performed in the transmitter. The active packets can instruct the receiver how to
reconstruct and playback the video.
In addition, although the active technique disclosed herein has the
advantage of playing back any type of video without specific decoders, the embedding of
program information into active packets increases packet size and increases the network
traffic load. However, most of the overhead of video coding is on the encoding rather
than decoding. The following techniques can be used to reduce the packet size and traffic
load. The coding of video can be done in hardware, and custom hardware is
available for the implementation of video coding. Some VLSI chips have been designed
specially for DCT, motion estimation, or entropy coding which can be used as an
auxiliary for video codecs. For those receivers equipped with these chips, they can
inform the transmitters that the programs for these computations are not necessary. The
transmitters, therefore, embed only the parameters used for these computations rather
than the actual program code into the active packets. This reduces the packet size
significantly.
Hardware implementation of video coding standards has also recently
become available. This hardware may be dedicated coders or programmable video signal
processors. Dedicated coders provide limited programmability and have architecture or
control logic for a specific video coding standard. Programmable processors, however,
offer much more flexibility in code design and implementation although they require
larger silicon area and dissipate more power than dedicated processors. This bundling of
program information with video data can be initiated by the receiver. Therefore when a
receiver requests a video file or stream that is encoded by a standard that the receiver
already supports in hardware or loaded software, the receiver can instruct the sender to
not include program information. This will reduced unnecessary transmission of
decoding information.
The second method to reduce the packet size and traffic load can be done
by caching. The programs for DCT, motion estimation, entropy coding, or even the
specific video codecs can be cached in the receivers or a proxy. Similar to the hardware implementation described above, the receiver can inform transmitter that part or all of
the decoding programs are not required if the programs have been cached. Thus, packet
size can be reduced as well.
Although various embodiments which incorporate the teachings of the
present invention have been shown and described in detail herein, those skilled in the art
can readily devise many other varied embodiments that still incorporate these teachings.

Claims

1. A method for transceiving a compressed data file comprising the steps
of
determining a decoding program for the compressed data file,
independently partitioning the compressed data file into data sub-files and
the decoding program into decoding sub-programs,
propagating a sequence of the sub-files and the sub-programs,
detecting the sequence to determine the sub-files and the sub-programs,
and
combining the sub-files to reproduce the compressed data file and
combining the sub-programs to reproduce the decoding program.
2. The method as recited in claim 1 further including the step of decoding the
reproduced compressed data file with the reproduced decoding program.
3. The method as recited in claim 2 wherein the step of propagating includes the
step of generating active packets, each active packet including one of the sub-files and
one of the sub-programs.
4. The method as recited in claim 3 wherein the step of generating active packets
includes the step of adding a header to each active packet.
5. The method as recited in claim 4 wherein the header includes a data identifier
and a program identifier, the step of detecting includes the step of detecting the data
identifier and the program identifier, and the step of combining includes the step of
arranging each of the detected sub-files with reference to the data identifier and each of
the detected sub-programs with reference to the program identifier.
6. A method for transceiving a compressed data file comprising the steps of
determining a decoding program for the compressed data file,
independently partitioning the compressed data file into data sub-files and
the decoding program into decoding sub-programs,
propagating a sequence of the sub-files and the sub-programs,
detecting the sequence to determine the sub-files and the sub-programs,
combining the sub-files to reproduce the compressed data file and
combining the sub-programs to reproduce the decoding program, and
decoding the reproduced compressed data file with the reproduced
decoding program.
7. A method for transceiving a compressed data file comprising the steps of
determining a decoding program for the compressed data file,
independently partitioning the compressed data file into data sub-files and
the decoding program into decoding sub-programs, propagating a sequence of active packets composed of the sub-files and
the sub-programs,
detecting the sequence of active packets to determine the sub-files and the
sub-programs, and
combining the sub-files to reproduce the compressed data file and
combining the sub-programs to reproduce the decoding program.
8. The method as recited in claim 7 further including the step of decoding the
reproduced compressed data file with the reproduced decoding program.
9. The method as recited in claim 8 wherein the step of propagating includes the
step of generating the active packets so that each active packet includes one of the sub-
files and one of the sub-programs.
10. The method as recited in claim 9 wherein the step of generating each active
packet includes the step of adding a header to each active packet.
11. The method as recited in claim 10 wherein the header includes a data
identifier and a program identifier, the step of detecting includes the step of detecting the
data identifier and the program identifier, and the step of combining includes the step of
arranging each of the detected sub-files with reference to the data identifier and each of
the detected sub-programs with reference to the program identifier.
12. A method for transceiving a compressed data file comprising the steps of
determining a decoding program for the compressed data file,
independently partitioning the compressed data file into data sub-files and
the decoding program into decoding sub-programs,
propagating a sequence of active packets, each active packet including one
of the sub-files and one of the sub-programs,
detecting the sequence of active packets to determine the sub-files and the
sub-programs,
combining the sub-files to reproduce the compressed data file and
combining the sub-programs to reproduce the decoding program; and
decoding reproduced compressed data file with the reproduced decoding
program.
13. The method as recited in claim 12 wherein the step of propagating active
packets includes the step of adding a header to each of the active packets.
14. The method as recited in claim 13 wherein the header includes a data
identifier and a program identifier, the step of detecting includes the step of detecting the
data identifier and the program identifier, and the step of combining includes the step of
arranging each of the detected sub-files with reference to the data identifier and each of
the detected sub-programs with reference to the program identifier.
15. A method for transmitting a compressed data file comprising the steps of
determining a decoding program for the compressed data file,
independently partitioning the compressed data file into data sub-files and
the decoding program into decoding sub-programs, and
propagating a sequence of active packets composed of the sub-files and
the sub-programs.
16. A method for receiving a compressed data file processed by partitioning the
data file into sub-files and by independently partitioning a decoding program for the
compressed file into sub-programs, and by propagating a sequence of active packets
wherein each of the active packets includes one of the sub-files and one of the sub-
programs, the method comprising the steps of
detecting the sequence to determine the sub-files and the sub-programs,
combining the sub-files to reproduce the compressed data file and
combining the sub-programs to reproduce the decoding program, and
decoding the reproduced compressed data file with the reproduced
decoding program.
17. A system for transceiving a compressed data file over a path comprising
an encoder deducer for determining a decoding program for the
compressed data file, partitioning means, coupled to the encoder deducer, for independently
partitioning the compressed data file into data sub-files and the decoding program into
decoding sub-programs,
propagating means, coupled to the partitioning means and the encoder
deducer, for generating and propagating onto the path a sequence of active packets
composed of the sub-files and the sub-programs,
a detector, coupled to the path, for detecting the active packets to
determine the sub-files and the sub-programs,
combiner means, responsive to the detector, for combining the sub-files to
reproduce the compressed data file and combining the sub-programs to reproduce the
decoding program, and
a detector, couple to the combiner means, for decoding the reproduced
compressed data file with the reproduced decoding program.
18. Circuitry for transmitting a compressed data file onto a path comprising
an encoder deducer for determining a decoding program for the
compressed data file,
partitioning means, coupled to the encoder deducer, for independently
partitioning the compressed data file into data sub-files and the decoding program into
decoding sub-programs, and means, coupled to the partitioning means and the encoder deducer, for
generating and propagating onto the path a sequence of active packets composed of the
sub-files and the sub-programs.
19. Circuitry for receiving over a path a compressed data file processed by
partitioning the data file into sub-files and by independently partitioning a decoding
program for the compressed file into sub-programs, and by propagating a sequence of
active packets wherein each of the active packets includes one of the sub-files and one of
the sub-programs, the circuitry comprising
a detector, coupled to the path, for detecting the sequence to determine the
sub-files and the sub-programs,
combiner means, responsive to the detector, for combining the sub-files to
reproduce the compressed data file and combining the sub-programs to reproduce the
decoding program, and
a decoder, coupled to the combiner means, for decoding the reproduced
compressed data file with the reproduced decoding program.
PCT/US2000/004491 1999-02-25 2000-02-23 Active techniques for video transmission and playback WO2000051349A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US12153299P 1999-02-25 1999-02-25
US60/121,532 1999-02-25

Publications (1)

Publication Number Publication Date
WO2000051349A1 true WO2000051349A1 (en) 2000-08-31

Family

ID=22397296

Family Applications (2)

Application Number Title Priority Date Filing Date
PCT/US2000/004491 WO2000051349A1 (en) 1999-02-25 2000-02-23 Active techniques for video transmission and playback
PCT/US2000/004508 WO2000051353A1 (en) 1999-02-25 2000-02-23 Active techniques for video transmission and playback

Family Applications After (1)

Application Number Title Priority Date Filing Date
PCT/US2000/004508 WO2000051353A1 (en) 1999-02-25 2000-02-23 Active techniques for video transmission and playback

Country Status (2)

Country Link
US (2) US6480537B1 (en)
WO (2) WO2000051349A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1267569A3 (en) * 2001-06-15 2004-02-18 Deutsche Telekom AG End-user device and method for using services offered on a telecommunication network
US7054805B2 (en) * 2002-02-28 2006-05-30 Broadcom Corporation Method and system for allocating memory during encoding of a datastream

Families Citing this family (96)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB9822087D0 (en) * 1998-10-09 1998-12-02 Snell & Wilcox Ltd Improvements in data compression
FI107000B (en) * 1999-02-17 2001-05-15 Nokia Mobile Phones Ltd Title compression in real-time services
JP2001189713A (en) * 1999-12-28 2001-07-10 Toshiba Corp Data transmitter and data transmission method
KR20010075754A (en) * 2000-01-17 2001-08-11 구자홍 method for capture data in television and composition method of the same
GB2359209A (en) * 2000-02-09 2001-08-15 Motorola Ltd Apparatus and methods for video distribution via networks
US20020026636A1 (en) * 2000-06-15 2002-02-28 Daniel Lecomte Video interfacing and distribution system and method for delivering video programs
US6965646B1 (en) * 2000-06-28 2005-11-15 Cisco Technology, Inc. MPEG file format optimization for streaming
US7313593B1 (en) * 2000-10-24 2007-12-25 International Business Machines Corporation Method and apparatus for providing full duplex and multipoint IP audio streaming
JP3699910B2 (en) * 2000-10-31 2005-09-28 株式会社東芝 Data transmission apparatus, data transmission method and program
JP4457506B2 (en) * 2001-02-27 2010-04-28 ソニー株式会社 Packetizing apparatus and packetizing method
US8091112B1 (en) * 2001-02-28 2012-01-03 Keen Personal Technologies, Inc. System and a method for transmitting and receiving a program with improved efficiency
US7277958B2 (en) * 2001-03-12 2007-10-02 Edgestream, Inc. Re-assembly of streaming files from separate connections
JP2002318598A (en) * 2001-04-20 2002-10-31 Toshiba Corp Device and method for information reproduction, and medium, device, method, and program for information recording
US7124086B2 (en) * 2001-05-30 2006-10-17 Olympus Corporation Data reproducing apparatus and data reproducing system for reproducing contents stored on a removable recording medium
US7496092B2 (en) * 2001-06-04 2009-02-24 Cisco Technology, Inc. System and method for allocating packet identifiers in a transport stream in a subscriber network
JP4549610B2 (en) * 2001-11-08 2010-09-22 ソニー株式会社 COMMUNICATION SYSTEM, COMMUNICATION METHOD, TRANSMISSION DEVICE AND METHOD, RECEPTION DEVICE AND METHOD, AND PROGRAM
EP1311124A1 (en) * 2001-11-13 2003-05-14 Matsushita Electric Industrial Co., Ltd. Selective protection method for images transmission
WO2003052612A1 (en) * 2001-12-15 2003-06-26 Thomson Licensing S.A. System and method for delivering data streams of multiple data types at different priority levels
US20050224902A1 (en) * 2002-02-06 2005-10-13 Ramsey Craig C Wireless substrate-like sensor
US7289230B2 (en) * 2002-02-06 2007-10-30 Cyberoptics Semiconductors, Inc. Wireless substrate-like sensor
US8284844B2 (en) 2002-04-01 2012-10-09 Broadcom Corporation Video decoding system supporting multiple standards
AU2003237388A1 (en) 2002-06-07 2003-12-22 Thomson Licensing S.A. Method and apparatus for controlling the distribution of digitally encoded data in a network
US20030233464A1 (en) * 2002-06-10 2003-12-18 Jonathan Walpole Priority progress streaming for quality-adaptive transmission of data
US20030236904A1 (en) * 2002-06-19 2003-12-25 Jonathan Walpole Priority progress multicast streaming for quality-adaptive transmission of data
US6788690B2 (en) * 2002-06-27 2004-09-07 Nokia Corporation Packet identifier search filtering
US7707317B2 (en) * 2002-07-01 2010-04-27 Prolifiq Software Inc. Adaptive electronic messaging
US7966374B2 (en) * 2002-07-01 2011-06-21 Profiliq Software Inc. Adaptive media messaging, such as for rich media messages incorporating digital content
US20040190629A1 (en) * 2002-07-19 2004-09-30 Cooper Jeffrey Allen System and method for broadcast of independently encoded signals on atsc channels
FR2845544B1 (en) * 2002-10-03 2005-01-14 Cit Alcatel ACTIVE RESOURCE RESERVATION SUPPORT IN A COMMUNICATION NETWORK
JP3513148B1 (en) 2002-10-11 2004-03-31 株式会社エヌ・ティ・ティ・ドコモ Moving picture coding method, moving picture decoding method, moving picture coding apparatus, moving picture decoding apparatus, moving picture coding program, and moving picture decoding program
US7386013B1 (en) * 2003-01-03 2008-06-10 Juniper Networks, Inc. Systems and methods for compressing packet headers
FR2849980B1 (en) * 2003-01-15 2005-04-08 Medialive METHOD FOR THE DISTRIBUTION OF VIDEO SEQUENCES, DECODER AND SYSTEM FOR THE IMPLEMENTATION OF THIS PRODUCT
FR2853786B1 (en) * 2003-04-11 2005-08-05 Medialive METHOD AND EQUIPMENT FOR DISTRIBUTING DIGITAL VIDEO PRODUCTS WITH A RESTRICTION OF CERTAIN AT LEAST REPRESENTATION AND REPRODUCTION RIGHTS
FR2854530B1 (en) * 2003-05-02 2005-07-22 Medialive METHOD AND DEVICE FOR SECURING THE TRANSMISSION, RECORDING AND VISUALIZATION OF DIGITAL AUDIOVISUAL EMPTY STREAMS
US7817721B2 (en) * 2003-05-15 2010-10-19 Lsi Corporation Posting status data in digital transport stream processing
KR100585933B1 (en) * 2003-08-20 2006-06-01 한국전자통신연구원 System ? Method for Digital Multimedia Broadcasting
CN1868213B (en) * 2003-09-02 2010-05-26 索尼株式会社 Content receiving apparatus, video/audio output timing control method, and content providing system
EP1672927A4 (en) * 2003-09-04 2010-06-16 Nec Corp Motion image data converting method, apparatus and program
US7911946B2 (en) * 2003-11-17 2011-03-22 General Instrument Corporation Method and apparatuses for using packet data to manage a data stream in a broadband communications system
CN1914807A (en) * 2004-02-10 2007-02-14 松下电器产业株式会社 Encoding/decoding device
US8189679B2 (en) * 2004-09-02 2012-05-29 Sony Corporation Content receiving apparatus, method of controlling video-audio output timing and content providing system
US20060075449A1 (en) * 2004-09-24 2006-04-06 Cisco Technology, Inc. Distributed architecture for digital program insertion in video streams delivered over packet networks
US7870590B2 (en) * 2004-10-20 2011-01-11 Cisco Technology, Inc. System and method for fast start-up of live multicast streams transmitted over a packet network
US20060090186A1 (en) * 2004-10-21 2006-04-27 Santangelo Bryan D Programming content capturing and processing system and method
US7650031B2 (en) * 2004-11-23 2010-01-19 Microsoft Corporation Method and system for detecting black frames in a sequence of frames
JP4487761B2 (en) * 2004-12-21 2010-06-23 カシオ計算機株式会社 Data output device and program
JP4559927B2 (en) * 2005-07-14 2010-10-13 パナソニック株式会社 Communication data processing apparatus and method
US20070083608A1 (en) * 2005-09-19 2007-04-12 Baxter Robert A Delivering a data stream with instructions for playback
US7680047B2 (en) * 2005-11-22 2010-03-16 Cisco Technology, Inc. Maximum transmission unit tuning mechanism for a real-time transport protocol stream
US8130841B2 (en) * 2005-12-29 2012-03-06 Harris Corporation Method and apparatus for compression of a video signal
US7893697B2 (en) 2006-02-21 2011-02-22 Cyberoptics Semiconductor, Inc. Capacitive distance sensing in semiconductor processing tools
WO2007098149A2 (en) 2006-02-21 2007-08-30 Cyberoptics Semiconductor, Inc. Capacitive distance sensing in semiconductor processing tools
US7965771B2 (en) 2006-02-27 2011-06-21 Cisco Technology, Inc. Method and apparatus for immediate display of multicast IPTV over a bandwidth constrained network
CN101336454B (en) * 2006-03-06 2011-04-13 三菱电机株式会社 Compressed encoded data reproducing apparatus and method for decoding and reproducing compressed encoded data in such apparatus
US8218654B2 (en) * 2006-03-08 2012-07-10 Cisco Technology, Inc. Method for reducing channel change startup delays for multicast digital video streams
KR100755714B1 (en) * 2006-05-03 2007-09-05 삼성전자주식회사 Apparatus and method for executing codec upgrade
KR100818924B1 (en) * 2006-05-03 2008-04-04 삼성전자주식회사 Apparatus and method for creating DSP code through combing of software module
US8326927B2 (en) * 2006-05-23 2012-12-04 Cisco Technology, Inc. Method and apparatus for inviting non-rich media endpoints to join a conference sidebar session
US20080117966A1 (en) * 2006-08-10 2008-05-22 Topiwala Pankaj N Method and compact apparatus for video capture and transmission with a common interface
US8358763B2 (en) * 2006-08-21 2013-01-22 Cisco Technology, Inc. Camping on a conference or telephony port
US8031701B2 (en) 2006-09-11 2011-10-04 Cisco Technology, Inc. Retransmission-based stream repair and stream join
US8823933B2 (en) * 2006-09-29 2014-09-02 Cyberoptics Corporation Substrate-like particle sensor
US8121277B2 (en) * 2006-12-12 2012-02-21 Cisco Technology, Inc. Catch-up playback in a conferencing system
US8769591B2 (en) * 2007-02-12 2014-07-01 Cisco Technology, Inc. Fast channel change on a bandwidth constrained network
US7778793B2 (en) * 2007-03-12 2010-08-17 Cyberoptics Semiconductor, Inc. Wireless sensor for semiconductor processing systems
US20080252490A1 (en) * 2007-04-10 2008-10-16 Chiluk David R Merging A Codec With A Digital Media File and Playing A Digital Media File On A Playback Device
US20080253369A1 (en) 2007-04-16 2008-10-16 Cisco Technology, Inc. Monitoring and correcting upstream packet loss
US8634310B2 (en) * 2007-06-26 2014-01-21 Qualcomm Incorporated Methods and apparatus for improved program acquisition for use with MPEG-2 based systems
US8787153B2 (en) * 2008-02-10 2014-07-22 Cisco Technology, Inc. Forward error correction based data recovery with path diversity
US20090219985A1 (en) * 2008-02-28 2009-09-03 Vasanth Swaminathan Systems and Methods for Processing Multiple Projections of Video Data in a Single Video File
US8363548B1 (en) * 2008-12-12 2013-01-29 Rockstar Consortium Us Lp Method and system for packet discard precedence for video transport
KR20100094709A (en) * 2009-02-19 2010-08-27 (주)휴맥스 Device and method for encoding/decoding
CN102484729B (en) * 2009-04-07 2016-08-24 Lg电子株式会社 Broadcasting transmitter, radio receiver and 3D video data handling procedure thereof
US9183543B2 (en) * 2010-02-19 2015-11-10 Prolifiq Software Inc. Tracking digital content objects
EP3905680B1 (en) 2011-01-13 2022-07-20 Canon Kabushiki Kaisha Image coding apparatus, image coding method, and program, and image decoding apparatus, image decoding method, and program
US10114660B2 (en) * 2011-02-22 2018-10-30 Julian Michael Urbach Software application delivery and launching system
FR2972588A1 (en) 2011-03-07 2012-09-14 France Telecom METHOD FOR ENCODING AND DECODING IMAGES, CORRESPONDING ENCODING AND DECODING DEVICE AND COMPUTER PROGRAMS
FR2977111A1 (en) 2011-06-24 2012-12-28 France Telecom METHOD FOR ENCODING AND DECODING IMAGES, CORRESPONDING ENCODING AND DECODING DEVICE AND COMPUTER PROGRAMS
KR101885852B1 (en) 2011-09-29 2018-08-08 삼성전자주식회사 Method and apparatus for transmitting and receiving content
US9432704B2 (en) * 2011-11-06 2016-08-30 Akamai Technologies Inc. Segmented parallel encoding with frame-aware, variable-size chunking
US9892188B2 (en) * 2011-11-08 2018-02-13 Microsoft Technology Licensing, Llc Category-prefixed data batching of coded media data in multiple categories
US9015555B2 (en) 2011-11-18 2015-04-21 Cisco Technology, Inc. System and method for multicast error recovery using sampled feedback
US10079710B2 (en) * 2012-02-16 2018-09-18 Brightcove, Inc. System and method for dynamic file availability during encoding
PL3515073T3 (en) * 2012-03-26 2021-03-08 Jvckenwood Corporation Image coding device, image coding method, and image coding program
RU2668399C9 (en) 2012-04-13 2019-06-19 ДжейВиСи КЕНВУД КОРПОРЕЙШН Picture decoding device, picture decoding method and reception device
WO2013179675A1 (en) * 2012-05-31 2013-12-05 株式会社Ｊｖｃケンウッド Image encoding device, image encoding method, image encoding program, transmission device, transmission method, transmission program, image decoding device, image decoding method, image decoding program, reception device, reception method, and reception program
GB2517359B (en) * 2012-07-31 2019-09-11 Hewlett Packard Entpr Dev Lp Wireless access point cell ID insertion in frame header
US9749321B2 (en) 2013-01-22 2017-08-29 Prolifiq Software Inc. System for multi-point publication syndication
US9628528B2 (en) 2013-07-19 2017-04-18 Electronics And Telecommunications Research Institute Apparatus and method for providing content
US20150085915A1 (en) * 2013-09-25 2015-03-26 Jay C.-C. Kuo Method and system for automatically encoding video with uniform throughput
US9485456B2 (en) 2013-12-30 2016-11-01 Akamai Technologies, Inc. Frame-rate conversion in a distributed computing system
US9654804B2 (en) * 2014-09-03 2017-05-16 Vigor Systems Inc. Replacing video frames in a transport stream
CN108024075B (en) * 2016-10-28 2019-10-11 原相科技股份有限公司 Global shutter high dynamic range pixel and image sensor
EP3563575B1 (en) * 2016-12-29 2024-04-10 British Telecommunications public limited company Transmission parameter control for segment delivery
US11368745B2 (en) 2018-07-05 2022-06-21 Dolby International Ab Processing media data structures
US11606574B2 (en) * 2019-05-31 2023-03-14 Apple Inc. Efficient coding of source video sequences partitioned into tiles

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5502497A (en) * 1991-08-28 1996-03-26 Hitachi, Ltd. Television broadcasting method and system enabling picture broadcasting from the transmitting equipment to the receiving equipment using alternative broadcasting system standards
US5517250A (en) * 1995-02-28 1996-05-14 General Instrument Corporation Of Delaware Acquisition of desired data from a packetized data stream and synchronization thereto

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH05122701A (en) * 1991-10-25 1993-05-18 Canon Inc Charging information processing unit
US5325126A (en) * 1992-04-01 1994-06-28 Intel Corporation Method and apparatus for real time compression and decompression of a digital motion video signal
US5623262A (en) * 1994-08-17 1997-04-22 Apple Computer, Inc. Multi-word variable length encoding and decoding
US5574505A (en) * 1995-05-16 1996-11-12 Thomson Multimedia S.A. Method and apparatus for operating a transport stream encoder to produce a stream of packets carrying data representing a plurality of component signals
US5598415A (en) * 1995-08-04 1997-01-28 General Instrument Corporation Of Delaware Transmission of high rate isochronous data in MPEG-2 data streams
US5742623A (en) * 1995-08-04 1998-04-21 General Instrument Corporation Of Delaware Error detection and recovery for high rate isochronous data in MPEG-2 data streams
JP3183155B2 (en) * 1996-03-18 2001-07-03 株式会社日立製作所 Image decoding apparatus and image decoding method
JPH10145773A (en) * 1996-11-14 1998-05-29 Toshiba Corp Method for ciphering animation data, computer system applying the method and dynamic image data encoding/ decoding device
US6018816A (en) * 1997-04-04 2000-01-25 Canon Kabushiki Kaisha Information processing system and method, image processing system and method, information processing apparatus and computer readable memory

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5502497A (en) * 1991-08-28 1996-03-26 Hitachi, Ltd. Television broadcasting method and system enabling picture broadcasting from the transmitting equipment to the receiving equipment using alternative broadcasting system standards
US5517250A (en) * 1995-02-28 1996-05-14 General Instrument Corporation Of Delaware Acquisition of desired data from a packetized data stream and synchronization thereto

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1267569A3 (en) * 2001-06-15 2004-02-18 Deutsche Telekom AG End-user device and method for using services offered on a telecommunication network
US7793327B2 (en) 2001-06-15 2010-09-07 Deutsche Telekom Ag Terminal device and method for using different services offered via a telecommunications network
US7054805B2 (en) * 2002-02-28 2006-05-30 Broadcom Corporation Method and system for allocating memory during encoding of a datastream

Also Published As

Publication number Publication date
US6480537B1 (en) 2002-11-12
US6570926B1 (en) 2003-05-27
WO2000051353A1 (en) 2000-08-31

Similar Documents

Publication Publication Date Title
US6480537B1 (en) Active techniques for video transmission and playback
US7502070B2 (en) Method and apparatus for processing a data series including processing priority data
US8780999B2 (en) Assembling multiview video coding sub-BITSTREAMS in MPEG-2 systems
US7197070B1 (en) Efficient systems and methods for transmitting compressed video data having different resolutions
US5847771A (en) Digital entertainment terminal providing multiple digital pictures
AU2010259914B2 (en) Multiview video coding over MPEG-2 systems
US6483543B1 (en) System and method for transcoding multiple channels of compressed video streams using a self-contained data unit
US6671323B1 (en) Encoding device, encoding method, decoding device, decoding method, coding system and coding method
US8948241B2 (en) Signaling characteristics of an MVC operation point
US7586924B2 (en) Apparatus and method for coding an information signal into a data stream, converting the data stream and decoding the data stream
US20040247122A1 (en) Processing multiple encrypted transport streams
MXPA01006991A (en) Sending progressive video sequences.
JPH1188856A (en) Transmission protocol converting system and catv network connecting system using protocol converting device
US20040190629A1 (en) System and method for broadcast of independently encoded signals on atsc channels
Chen et al. Active techniques for real-time video transmission and playback
Joseph et al. The grand alliance transport system for terrestrial HDTV transmission
Mitchell et al. Mpeg-2 overview
MXPA06009109A (en) Resizing of buffer in encoder and decoder

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): CA JP

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE

121 Ep: the epo has been informed by wipo that ep was designated in this application
DFPE Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101)
122 Ep: pct application non-entry in european phase