US20050094728A1 - Embedding supplemental data in a digital video signal - Google Patents
Embedding supplemental data in a digital video signal Download PDFInfo
- Publication number
- US20050094728A1 US20050094728A1 US11/005,260 US526004A US2005094728A1 US 20050094728 A1 US20050094728 A1 US 20050094728A1 US 526004 A US526004 A US 526004A US 2005094728 A1 US2005094728 A1 US 2005094728A1
- Authority
- US
- United States
- Prior art keywords
- picture
- gop
- pictures
- video signal
- mpeg
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T1/00—General purpose image data processing
- G06T1/0021—Image watermarking
- G06T1/0085—Time domain based watermarking, e.g. watermarks spread over several images
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/46—Embedding additional information in the video signal during the compression process
- H04N19/467—Embedding additional information in the video signal during the compression process characterised by the embedded information being invisible, e.g. watermarking
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/238—Interfacing the downstream path of the transmission network, e.g. adapting the transmission rate of a video stream to network bandwidth; Processing of multiplex streams
- H04N21/2389—Multiplex stream processing, e.g. multiplex stream encrypting
- H04N21/23892—Multiplex stream processing, e.g. multiplex stream encrypting involving embedding information at multiplex stream level, e.g. embedding a watermark at packet level
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
- H04N21/835—Generation of protective data, e.g. certificates
- H04N21/8358—Generation of protective data, e.g. certificates involving watermark
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/76—Television signal recording
- H04N5/91—Television signal processing therefor
- H04N5/913—Television signal processing therefor for scrambling ; for copy protection
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/24—Systems for the transmission of television signals using pulse code modulation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2201/00—General purpose image data processing
- G06T2201/005—Image watermarking
- G06T2201/0051—Embedding of the watermark in the spatial domain
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2201/00—General purpose image data processing
- G06T2201/005—Image watermarking
- G06T2201/0053—Embedding of the watermark in the coding stream, possibly without decoding; Embedding of the watermark in the compressed domain
-
- G—PHYSICS
- G11—INFORMATION STORAGE
- G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
- G11B20/00—Signal processing not specific to the method of recording or reproducing; Circuits therefor
- G11B20/00086—Circuits for prevention of unauthorised reproduction or copying, e.g. piracy
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/76—Television signal recording
- H04N5/91—Television signal processing therefor
- H04N5/913—Television signal processing therefor for scrambling ; for copy protection
- H04N2005/91307—Television signal processing therefor for scrambling ; for copy protection by adding a copy protection signal to the video signal
- H04N2005/91328—Television signal processing therefor for scrambling ; for copy protection by adding a copy protection signal to the video signal the copy protection signal being a copy management signal, e.g. a copy generation management signal [CGMS]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/76—Television signal recording
- H04N5/91—Television signal processing therefor
- H04N5/913—Television signal processing therefor for scrambling ; for copy protection
- H04N2005/91307—Television signal processing therefor for scrambling ; for copy protection by adding a copy protection signal to the video signal
- H04N2005/91335—Television signal processing therefor for scrambling ; for copy protection by adding a copy protection signal to the video signal the copy protection signal being a watermark
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/76—Television signal recording
- H04N5/91—Television signal processing therefor
- H04N5/913—Television signal processing therefor for scrambling ; for copy protection
- H04N2005/91307—Television signal processing therefor for scrambling ; for copy protection by adding a copy protection signal to the video signal
- H04N2005/91342—Television signal processing therefor for scrambling ; for copy protection by adding a copy protection signal to the video signal the copy protection signal being an authentication signal
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/76—Television signal recording
- H04N5/91—Television signal processing therefor
- H04N5/913—Television signal processing therefor for scrambling ; for copy protection
- H04N2005/91357—Television signal processing therefor for scrambling ; for copy protection by modifying the video signal
- H04N2005/91364—Television signal processing therefor for scrambling ; for copy protection by modifying the video signal the video signal being scrambled
Definitions
- the invention relates to a method and arrangement for embedding supplemental data in a digital video signal.
- the invention also relates to an arrangement for decoding the embedded supplemental data.
- Video and audio signals are increasingly transmitted and recorded in a digitally encoded form, for example, an MPEG bit stream.
- a digitally encoded form for example, an MPEG bit stream.
- supplemental data in the signal for example, a watermark to classify the signal as authentic program material.
- Watermarking digital signals is particularly useful in copy protection applications.
- the watermark can effectively take the form of a single bit indicating that the signal constitutes copy protected material, or a multi-bit code representing the originator of the material.
- the invention provides a method of embedding supplemental data in a video signal comprising the step of encoding the video signal in groups of pictures comprising an intraframe (I) coded picture and a series of predictively (P) and bidirectionally predictively (B) coded pictures, characterized by encoding the video signal in such a manner that the pattern of picture coding types in a group of pictures (GOP) represents a supplemental data value.
- a method of embedding supplemental data in a video signal comprising the step of encoding the video signal in groups of pictures comprising an intraframe (I) coded picture and a series of predictively (P) and bidirectionally predictively (B) coded pictures, characterized by encoding the video signal in such a manner that the pattern of picture coding types in a group of pictures (GOP) represents a supplemental data value.
- I intraframe
- P predictively
- B bidirectionally predictively
- a watermark can easily be detected.
- the picture coding types are accommodated in the picture headers of an MPEG bit stream and can easily be read.
- changing the picture coding type in a picture header to remove the watermark renders the picture data no longer compliant with the coding standard.
- the MPEG bit stream can no longer be decoded by a compliant decoder.
- the relevant picture must be transcoded to comply with the new picture coding type, e.g., by decoding the picture and encoding it again.
- the supplemental data value is represented by a given pattern of B and P picture coding types in a GOP, for example, by the position of a BPP pattern in a GOP.
- changing a picture coding type also changes its reference to other pictures within the GOP and, consequently, ripples through the remainder of the GOP.
- a substantial number of pictures in the GOP must now be transcoded rather than a single picture.
- a P picture can be transcoded into an I picture without requiring other pictures to be transcoded as well.
- the I picture must then be encoded with the low amount of bits used for the P picture. This affects the quality of the I picture as well as any P picture referring to this I picture. Consequently, a watermark cannot be removed from a GOP without either transcoding the remainder of the GOP or suffering severe decrease in quality for the remainder of the GOP.
- FIGS. 1-4 show examples of GOP structures of an MPEG-encoded video signal to illustrate the method of embedding supplemental data in accordance with the invention
- FIG. 5 shows an example of assigning different supplemental data values to respective positions of a BPP pattern in a GOP
- FIG. 6 shows a schematic diagram of an arrangement for embedding supplemental data in an MPEG video signal in accordance with the invention
- FIG. 7 shows a flowchart illustrating the operation of a control circuit which is shown in FIG. 6 ;
- FIG. 8 shows a schematic diagram of an arrangement for decoding supplemental data embedded in an MPEG-encoded video signal in accordance with the invention.
- an MPEG encoder encodes pictures in accordance with one of three different coding methods. Some pictures are autonomously encoded, i.e., without any reference to another picture in the video sequence. These pictures are denoted intraframe coded pictures or I pictures. Other pictures are predictively encoded, using a motion-compensated previous picture as a reference (prediction) image. They are denoted P pictures. The previous picture to which a P picture refers, may be an I picture or another P picture. Yet other pictures are bi-directionally predictively encoded. They refer to a previous as well as a future I or P picture, and are denoted B pictures.
- the amount of bits required to represent a picture is most for I pictures, less for P pictures and least for B pictures.
- the amount of compression and the quality of the decoded video sequence largely depends on the performance of the motion estimation process in the encoder.
- Motion estimation is the most complicated and computational intensive operation of an MPEG encoder. It is this operation which will make professional MPEG encoders far superior over cost-effective consumer encoders for a long time.
- a parameter, picture_type in each picture header of an MPEG video bit stream, describes how the relevant picture has been encoded. If the picture coding type is I, the decoder reconstructs the picture completely from the received picture data. If the picture coding type is P, the decoder reconstructs the picture from the received picture data and an already displayed I or P picture. If the picture coding type is B, it is reconstructed on the basis of a preceding as well as a succeeding I or P picture. It should be noted that the parameter picture_type implicitly specifies the reference picture(s), i.e., a P picture refers to the most recent I or P picture, a B picture refers to the most recent and the next I or P picture.
- a series of an I picture and consecutive B and P pictures are called a Group of Pictures (GOP).
- GOP Group of Pictures
- an encoder is free to choose the optimum sequence of I, B and P picture coding types.
- Only a few GOP-structures are used in practice:
- MPEG encoders currently under development optimize the GOP structure a little further than the conventional sequences listed above, usually in that an I picture is chosen when a hard scene change occurs.
- FIG. 1 shows an example of the commonly used IBBPBBP . . . structure of a GOP comprising 12 pictures 1 , 2 , . . . 12 .
- the arrows shown in the Figure point to the relevant reference picture(s).
- B pictures 2 and 3 have been encoded using I picture 1 and P picture 4 as prediction
- B pictures 5 and 6 have been encoded using P picture 4 and P picture 7 as reference, etc.
- P picture 4 has been encoded using I picture 1 as prediction
- P picture 7 has been encoded using the P picture 4 as prediction, etc.
- the pictures are shown in display order. The transmission order is different because for decoding a B picture, the decoder must already have a future P or I picture at its disposal.
- the commonly used GOP structure comprises a plurality of BBP patterns.
- the pattern BPP is rarely used in a GOP. In a preferred embodiment of the present invention, it is this pattern which is used to represent watermark data.
- FIG. 2 shows a GOP including such a BPP pattern (the pictures 5 , 6 and 7 ).
- a watermark in the form of a BPP pattern can easily be detected because the picture coding type is included in the respective picture header.
- the decoder will decode picture 7 with reference to P pictures 6 and 10 , whereas the encoder made reference to picture 6 only.
- the pictures 8 , 9 and 10 will neither be decoded correctly because their original references to P picture 7 have been changed into references to P picture 6 .
- parameter picture_type of P picture 6 or P picture 7 be changed from P into I because, in that case, a predictively encoded picture is then interpreted as an autonomously encoded picture (pixel differences are interpreted as pixels).
- the relevant picture has to be transcoded, i.e., decoded into the pixel domain and encoded again in accordance with its modified picture coding type. That is not attractive for a hacker because, as mentioned before, high-quality encoding involves complicated motion estimation circuitry, unless a severe degradation of quality is accepted.
- picture_type parameter has been changed.
- Pictures referring to the modified picture are to be re-encoded as well. For example, if picture 7 in FIG. 2 is transcoded from a P picture into a B picture as shown in FIG.
- the B pictures 8 , 9 and the P picture 10 will also have to be transcoded because their references have changed.
- the effect of transcoding ripples through the remainder of the GOP unless the relevant P picture is transcoded into an I picture.
- the I picture must be compressed into the same (relatively low) number of bits as originally spent to the P picture.
- a GOP 20 which starts with the BPP pattern represents a sync code to indicate the start of a message.
- a GOP 21 with the BPP pattern after a single P represents a binary supplemental data value “0”.
- a GOP 22 with the BPP pattern after two Ps represents a binary “1”.
- the reference numeral 23 denotes an MPEG-encoded video signal segment with an embedded message “0110 . . . ”.
- each GOP conveys a supplemental data value (most of the GOPs have the common IBBPBBP . . . structure) in view of the fact that watermarking affects the encoding efficiency. It is notably advantageous to embed a supplemental data value in every nth GOP (n being a predetermined integer) to assist a watermark detector in identifying the relevant GOPs and to reduce the false alarm rate.
- the GOPs in video signal 23 have variable lengths. Not only may the number of pictures in a GOP vary, the number of bits per picture also depends largely on the image contents.
- FIG. 6 shows a schematic diagram of an arrangement for embedding supplemental data in an MPEG video signal in accordance with the invention.
- the arrangement includes a conventional MPEG video encoder 30 and a control circuit 40 .
- the MPEG video encoder 30 is shown in more details, however, only to an extent necessary to understand the invention. More particularly, the encoder comprises a subtractor 31 which receives a video signal x to be encoded, and subtracts therefrom a prediction signal ⁇ circumflex over (x) ⁇ 1 . The difference signal is subjected to discrete cosine transform, quantization and variable-length coding (in the Figure collectively denoted 32 ).
- the MPEG-encoded output signal y is transmitted to a receiver or stored on a storage medium (not shown). It is also locally decoded by a local decoder 33 and, through an adder 34 , applied to a motion estimation and compensation circuit 35 .
- Said motion estimation and compensation circuit provides a forward predicted picture and a bi-directional predicted picture.
- the three MPEG encoding modes (I, P, B) are symbolized in FIG. 6 by a selection switch 36 which selects the prediction signal ⁇ circumflex over (x) ⁇ 2 applied to the subtractor 31 .
- the selection switch has three input terminals denoted I, P and B. If the I terminal is selected, the prediction signal 3 is zero which results in that the input signal x will autonomously be encoded. If the P terminal is selected, the forward-predicted picture is applied to the subtractor. If the B terminal is selected, the bi-directional-predicted picture is applied to the subtractor.
- the selected prediction signal ⁇ circumflex over (x) ⁇ 4 is also fed back into the motion estimation and compensation circuit 35 through the adder 34 .
- the current encoding mode (I, P, B) is controlled by the control circuit 40 which controls the selection switch 36 through a picture coding type signal PT in accordance with a received watermark message w to be embedded.
- FIG. 7 shows a flowchart illustrating the operation of said control circuit.
- the MPEG encoder produces the conventional GOP structure.
- the next watermark symbol w i of the watermark message w to be embedded is read in a step 52 .
- FIG. 8 shows a schematic diagram of an arrangement for decoding supplemental data embedded in an MPEG-encoded video signal in accordance with the invention.
- the arrangement comprises a GOP detector 61 , a picture header detector 62 , a window generator 63 , a gate 64 , a shift register 65 and a look-up table 66 .
- the picture header detector 62 detects the presence of a predetermined 32-bit picture_start_code (the hexadecimal value 00000100) in the MPEG signal and applies a picture header signal PHDR to the window generator 63 .
- the window generator 63 generates a timing window W.
- the window W opens the gate 64 each time an MPEG parameter picture_coding type is received and causes said parameter to be written into the shift register 65 .
- the parameter indicates whether the current picture is intraframe coded (I), predictively coded (P) or bi-directionally predictively coded (B).
- the GOP detector 61 detects the presence of a further predetermined 32-bit group_start_code (the hexadecimal value 000001B8) in the MPEG signal which indicates the start of a group of pictures. In response thereto, the detector activates the look-up table 66 to convert the current pattern of picture coding types PTRN in shift register 65 into a supplemental data value w i . In addition, the GOP detector resets the shift register 65 so as to start collecting the pattern of picture coding types for the next GOP.
- An MPEG-encoded video signal includes groups of pictures (GOPs), each GOP comprising an intraframe coded (I) picture and a series of predictively encoded (P) pictures and bi-directionally predictively (B) pictures.
- GOPs groups of pictures
- I intraframe coded
- P predictively encoded
- B bi-directionally predictively
- the GOP structure IBBPBBP . . . is used.
- the video signal is watermarked by forcing the MPEG encoder to produce a GOP structure which does normally not occur, e.g., a GOP including a BPP sequence. Different symbol values can be assigned to different positions of the BPP sequence in the GOP.
Abstract
An MPEG-encoded video signal includes groups of pictures (GOPs), each GOP having an intraframe coded (I) picture and a series of predictively encoded (P) pictures and bi-directionally predictively (B) pictures. Usually, the GOP structure IBBPBBP . . . is used. However, in order to embed a watermark in the MPEG-encoded video signal, the MPEG encoder is forced to produce a GOP structure which does not normally occur, e.g., a GOP including a BPP sequence. Different symbol values can be assigned to different positions of the BPP sequence in the GOP.
Description
- 1. Field of the Invention
- The invention relates to a method and arrangement for embedding supplemental data in a digital video signal. The invention also relates to an arrangement for decoding the embedded supplemental data.
- 2. Description of the Related Art
- Video and audio signals are increasingly transmitted and recorded in a digitally encoded form, for example, an MPEG bit stream. There is a growing need to accommodate supplemental data in the signal, for example, a watermark to classify the signal as authentic program material. Watermarking digital signals is particularly useful in copy protection applications. The watermark can effectively take the form of a single bit indicating that the signal constitutes copy protected material, or a multi-bit code representing the originator of the material.
- In the known MPEG standard for audio and video compression, a copy protection bit has been defined for that purpose. However, a disadvantage of this known method is that the protection bit can easily be modified to circumvent the copyright protection mechanism.
- It is an object of the invention to provide a method and arrangement for embedding a watermark in a video signal in such a manner that the embedded watermark can easily be detected but is difficult to remove.
- To this end, the invention provides a method of embedding supplemental data in a video signal comprising the step of encoding the video signal in groups of pictures comprising an intraframe (I) coded picture and a series of predictively (P) and bidirectionally predictively (B) coded pictures, characterized by encoding the video signal in such a manner that the pattern of picture coding types in a group of pictures (GOP) represents a supplemental data value.
- With the invention, it is achieved that a watermark can easily be detected. The picture coding types are accommodated in the picture headers of an MPEG bit stream and can easily be read. However, changing the picture coding type in a picture header to remove the watermark renders the picture data no longer compliant with the coding standard. The MPEG bit stream can no longer be decoded by a compliant decoder. The relevant picture must be transcoded to comply with the new picture coding type, e.g., by decoding the picture and encoding it again.
- It should be noted that the general idea of generating a predetermined sequence of I, P and B pictures in an MPEG signal so as to mark a digital video signal has also been proposed in Applicant's International Patent Application WO 97/13248, corresponding to U.S. Pat. No. 6,131,161 (Attorney Docket No. PHN 15,391). However, this application was published after the priority date of the present invention and fails to disclose the representation of a supplemental data value by a pattern of picture coding types within a group of pictures.
- Preferably, the supplemental data value is represented by a given pattern of B and P picture coding types in a GOP, for example, by the position of a BPP pattern in a GOP. Herewith, it is achieved that changing a picture coding type also changes its reference to other pictures within the GOP and, consequently, ripples through the remainder of the GOP. To remove a watermark, a substantial number of pictures in the GOP must now be transcoded rather than a single picture. There is one exception, i.e., a P picture can be transcoded into an I picture without requiring other pictures to be transcoded as well. However, the I picture must then be encoded with the low amount of bits used for the P picture. This affects the quality of the I picture as well as any P picture referring to this I picture. Consequently, a watermark cannot be removed from a GOP without either transcoding the remainder of the GOP or suffering severe decrease in quality for the remainder of the GOP.
-
FIGS. 1-4 show examples of GOP structures of an MPEG-encoded video signal to illustrate the method of embedding supplemental data in accordance with the invention; -
FIG. 5 shows an example of assigning different supplemental data values to respective positions of a BPP pattern in a GOP; -
FIG. 6 shows a schematic diagram of an arrangement for embedding supplemental data in an MPEG video signal in accordance with the invention; -
FIG. 7 shows a flowchart illustrating the operation of a control circuit which is shown inFIG. 6 ; and -
FIG. 8 shows a schematic diagram of an arrangement for decoding supplemental data embedded in an MPEG-encoded video signal in accordance with the invention. - First, the basic principles of MPEG which are essential to the watermarking method in accordance with the invention will be briefly described.
- To achieve efficient video compression, an MPEG encoder encodes pictures in accordance with one of three different coding methods. Some pictures are autonomously encoded, i.e., without any reference to another picture in the video sequence. These pictures are denoted intraframe coded pictures or I pictures. Other pictures are predictively encoded, using a motion-compensated previous picture as a reference (prediction) image. They are denoted P pictures. The previous picture to which a P picture refers, may be an I picture or another P picture. Yet other pictures are bi-directionally predictively encoded. They refer to a previous as well as a future I or P picture, and are denoted B pictures.
- Generally, the amount of bits required to represent a picture is most for I pictures, less for P pictures and least for B pictures. The amount of compression and the quality of the decoded video sequence largely depends on the performance of the motion estimation process in the encoder. Motion estimation is the most complicated and computational intensive operation of an MPEG encoder. It is this operation which will make professional MPEG encoders far superior over cost-effective consumer encoders for a long time.
- In order to inform an MPEG decoder whether a received picture is an I, P or B picture, a parameter, picture_type, in each picture header of an MPEG video bit stream, describes how the relevant picture has been encoded. If the picture coding type is I, the decoder reconstructs the picture completely from the received picture data. If the picture coding type is P, the decoder reconstructs the picture from the received picture data and an already displayed I or P picture. If the picture coding type is B, it is reconstructed on the basis of a preceding as well as a succeeding I or P picture. It should be noted that the parameter picture_type implicitly specifies the reference picture(s), i.e., a P picture refers to the most recent I or P picture, a B picture refers to the most recent and the next I or P picture.
- A series of an I picture and consecutive B and P pictures are called a Group of Pictures (GOP). According to the MPEG standard, an encoder is free to choose the optimum sequence of I, B and P picture coding types. However, only a few GOP-structures are used in practice:
-
- IPPP . . . is used by cheap encoders that do not have access to large amounts of memory;
- IBPBP . . . is used by more advanced encoders; and
- IBBPBBP . . . is commonly used by professional encoders.
- MPEG encoders currently under development optimize the GOP structure a little further than the conventional sequences listed above, usually in that an I picture is chosen when a hard scene change occurs.
-
FIG. 1 shows an example of the commonly used IBBPBBP . . . structure of a GOP comprising 12pictures P picture 4 as prediction, B pictures 5 and 6 have been encoded usingP picture 4 andP picture 7 as reference, etc. Similarly,P picture 4 has been encoded using I picture 1 as prediction,P picture 7 has been encoded using theP picture 4 as prediction, etc. Note that the pictures are shown in display order. The transmission order is different because for decoding a B picture, the decoder must already have a future P or I picture at its disposal. - As
FIG. 1 shows, the commonly used GOP structure comprises a plurality of BBP patterns. The pattern BPP is rarely used in a GOP. In a preferred embodiment of the present invention, it is this pattern which is used to represent watermark data.FIG. 2 shows a GOP including such a BPP pattern (thepictures - A watermark in the form of a BPP pattern can easily be detected because the picture coding type is included in the respective picture header. However, it is impossible to remove the watermark by merely changing the parameter picture_type. For example, if the parameter picture_type of
picture 6 inFIG. 2 is changed from P into B as shown inFIG. 3 , a decoder will decode this picture with reference toP picture 4 andP picture 7, whereas the encoder usedP picture 4 as the prediction image only.Picture 7 will neither be correctly decoded because it now refers toP picture 4 whereas the encoder referred topicture 6. Needless to say that the decoder will fail, or at least produce erroneous results. - Similarly, if the watermark is removed by changing the parameter picture_type of
picture 7 inFIG. 2 from P into B as shown inFIG. 4 , the decoder will decodepicture 7 with reference toP pictures picture 6 only. Thepictures P picture 7 have been changed into references toP picture 6. - Neither can the parameter picture_type of
P picture 6 orP picture 7 be changed from P into I because, in that case, a predictively encoded picture is then interpreted as an autonomously encoded picture (pixel differences are interpreted as pixels). - Accordingly, in order to remove a watermark, the relevant picture has to be transcoded, i.e., decoded into the pixel domain and encoded again in accordance with its modified picture coding type. That is not attractive for a hacker because, as mentioned before, high-quality encoding involves complicated motion estimation circuitry, unless a severe degradation of quality is accepted. In this respect, it is to be noted that not only the picture whose picture_type parameter has been changed has to be re-encoded. Pictures referring to the modified picture are to be re-encoded as well. For example, if
picture 7 inFIG. 2 is transcoded from a P picture into a B picture as shown inFIG. 4 , the B pictures 8, 9 and theP picture 10 will also have to be transcoded because their references have changed. Thus, the effect of transcoding ripples through the remainder of the GOP, unless the relevant P picture is transcoded into an I picture. However, in that case, the I picture must be compressed into the same (relatively low) number of bits as originally spent to the P picture. - The occurrence of two consecutive P pictures rarely occurs by accident. The number of false alarms (watermark detected in a non-watermarked signal) is thus limited. To further reduce the false alarm possibility, a requirement can be imposed on the maximal amount of GOPs between two watermarked GOPs. For example, a video stream is specified to be copyright protected if watermarked GOPs occur in small enough intervals.
- The above-described concept of watermarking a GOP allows messages of any length to be embedded in an MPEG video signal. To this end, different supplemental data values are assigned to different positions of the BPP pattern in the GOP. A first example thereof is shown in
FIG. 5 . In this example, aGOP 20 which starts with the BPP pattern represents a sync code to indicate the start of a message. AGOP 21 with the BPP pattern after a single P represents a binary supplemental data value “0”. AGOP 22 with the BPP pattern after two Ps represents a binary “1”. Thereference numeral 23 denotes an MPEG-encoded video signal segment with an embedded message “0110 . . . ”. Note that not each GOP conveys a supplemental data value (most of the GOPs have the common IBBPBBP . . . structure) in view of the fact that watermarking affects the encoding efficiency. It is notably advantageous to embed a supplemental data value in every nth GOP (n being a predetermined integer) to assist a watermark detector in identifying the relevant GOPs and to reduce the false alarm rate. Note also that the GOPs invideo signal 23 have variable lengths. Not only may the number of pictures in a GOP vary, the number of bits per picture also depends largely on the image contents. - It will be appreciated that the alphabet of supplemental data values can be further enlarged. For example, six different message symbols 0-5 may be assigned to GOP structures in accordance with the following Table I:
TABLE I Symbol GOP structure 0 IBPPBBPBB . . . 1 IBBPPBPBB . . . 2 IBBPBPPBB . . . 3 IBBPBBPPB . . . 4 IBBPBBPBPPBB . . . 5 IBBPBBPBBPPB . . . -
FIG. 6 shows a schematic diagram of an arrangement for embedding supplemental data in an MPEG video signal in accordance with the invention. The arrangement includes a conventionalMPEG video encoder 30 and acontrol circuit 40. TheMPEG video encoder 30 is shown in more details, however, only to an extent necessary to understand the invention. More particularly, the encoder comprises asubtractor 31 which receives a video signal x to be encoded, and subtracts therefrom a prediction signal {circumflex over (x)} 1. The difference signal is subjected to discrete cosine transform, quantization and variable-length coding (in the Figure collectively denoted 32). The MPEG-encoded output signal y is transmitted to a receiver or stored on a storage medium (not shown). It is also locally decoded by alocal decoder 33 and, through anadder 34, applied to a motion estimation andcompensation circuit 35. Said motion estimation and compensation circuit provides a forward predicted picture and a bi-directional predicted picture. - The three MPEG encoding modes (I, P, B) are symbolized in
FIG. 6 by aselection switch 36 which selects the prediction signal {circumflex over (x)} 2 applied to thesubtractor 31. The selection switch has three input terminals denoted I, P and B. If the I terminal is selected, theprediction signal 3 is zero which results in that the input signal x will autonomously be encoded. If the P terminal is selected, the forward-predicted picture is applied to the subtractor. If the B terminal is selected, the bi-directional-predicted picture is applied to the subtractor. The selected prediction signal {circumflex over (x)} 4 is also fed back into the motion estimation andcompensation circuit 35 through theadder 34. - The current encoding mode (I, P, B) is controlled by the
control circuit 40 which controls theselection switch 36 through a picture coding type signal PT in accordance with a received watermark message w to be embedded.FIG. 7 shows a flowchart illustrating the operation of said control circuit. In this example, it is assumed that a watermark symbol wi (wi=0 . . . 5) is to be embedded in every 8th GOP in accordance with Table I described hereinabove. In astep 50, it is determined whether the current GOP is the 8th GOP. If that is not the case, the control signal PT=IBBPBBP . . . is generated in astep 51. In response thereto, the MPEG encoder produces the conventional GOP structure. However, for every 8th GOP, the next watermark symbol wi of the watermark message w to be embedded is read in astep 52. Then, the GOP structure assigned to said symbol is looked-up in a memory in which Table I is stored, and the corresponding control signal PT=I . . . BPP . . . is generated in astep 53. -
FIG. 8 shows a schematic diagram of an arrangement for decoding supplemental data embedded in an MPEG-encoded video signal in accordance with the invention. The arrangement comprises aGOP detector 61, apicture header detector 62, awindow generator 63, agate 64, ashift register 65 and a look-up table 66. Thepicture header detector 62 detects the presence of a predetermined 32-bit picture_start_code (the hexadecimal value 00000100) in the MPEG signal and applies a picture header signal PHDR to thewindow generator 63. In response thereto, thewindow generator 63 generates a timing window W. The window W opens thegate 64 each time an MPEG parameter picture_coding type is received and causes said parameter to be written into theshift register 65. The parameter indicates whether the current picture is intraframe coded (I), predictively coded (P) or bi-directionally predictively coded (B). TheGOP detector 61 detects the presence of a further predetermined 32-bit group_start_code (the hexadecimal value 000001B8) in the MPEG signal which indicates the start of a group of pictures. In response thereto, the detector activates the look-up table 66 to convert the current pattern of picture coding types PTRN inshift register 65 into a supplemental data value wi. In addition, the GOP detector resets theshift register 65 so as to start collecting the pattern of picture coding types for the next GOP. - In summary, a method of embedding a watermark in an MPEG-encoded video signal is disclosed. An MPEG-encoded video signal includes groups of pictures (GOPs), each GOP comprising an intraframe coded (I) picture and a series of predictively encoded (P) pictures and bi-directionally predictively (B) pictures. Usually, the GOP structure IBBPBBP . . . is used. In accordance with the invention, the video signal is watermarked by forcing the MPEG encoder to produce a GOP structure which does normally not occur, e.g., a GOP including a BPP sequence. Different symbol values can be assigned to different positions of the BPP sequence in the GOP.
Claims (5)
1-8. (canceled)
9. An encoded video signal with embedded supplemental data, comprising groups of intraframe (I), predictively (P) and bidirectionally predictively (B) coded pictures, characterized in that supplemental data values are represented by respective predetermined patterns of picture coding types (I, B and P) in a group of pictures.
10. The signal as claimed in claim 9 , wherein the supplemental data value is represented by the position of a BPP pattern in a group of pictures.
11. A storage medium stored thereon an encoded video signal with embedded supplemental data as claimed in claim 9 .
12. A storage medium having stored thereon an encoded video signal with embedded supplemental data as claimed in claim 10.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US11/005,260 US20050094728A1 (en) | 1997-01-13 | 2004-12-06 | Embedding supplemental data in a digital video signal |
Applications Claiming Priority (7)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP97200058 | 1997-01-13 | ||
EP97200058.2 | 1997-01-13 | ||
EP97202612 | 1997-08-26 | ||
EP97202612.4 | 1997-08-26 | ||
US09/142,549 US6850567B1 (en) | 1997-01-13 | 1998-01-12 | Embedding supplemental data in a digital video signal |
PCT/IB1998/000026 WO1998031152A2 (en) | 1997-01-13 | 1998-01-12 | Embedding supplemental data in a digital video signal |
US11/005,260 US20050094728A1 (en) | 1997-01-13 | 2004-12-06 | Embedding supplemental data in a digital video signal |
Related Parent Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US09/142,549 Continuation US6850567B1 (en) | 1997-01-13 | 1998-01-12 | Embedding supplemental data in a digital video signal |
PCT/IB1998/000026 Continuation WO1998031152A2 (en) | 1997-01-13 | 1998-01-12 | Embedding supplemental data in a digital video signal |
Publications (1)
Publication Number | Publication Date |
---|---|
US20050094728A1 true US20050094728A1 (en) | 2005-05-05 |
Family
ID=26146041
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US09/142,549 Expired - Fee Related US6850567B1 (en) | 1997-01-13 | 1998-01-12 | Embedding supplemental data in a digital video signal |
US11/005,260 Abandoned US20050094728A1 (en) | 1997-01-13 | 2004-12-06 | Embedding supplemental data in a digital video signal |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US09/142,549 Expired - Fee Related US6850567B1 (en) | 1997-01-13 | 1998-01-12 | Embedding supplemental data in a digital video signal |
Country Status (8)
Country | Link |
---|---|
US (2) | US6850567B1 (en) |
EP (1) | EP0891674A2 (en) |
JP (1) | JP2000513167A (en) |
KR (1) | KR20000064585A (en) |
CN (1) | CN1134172C (en) |
AU (1) | AU728966B2 (en) |
BR (1) | BR9804764A (en) |
WO (1) | WO1998031152A2 (en) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20090087110A1 (en) * | 2007-09-28 | 2009-04-02 | Dolby Laboratories Licensing Corporation | Multimedia coding and decoding with additional information capability |
US20110150085A1 (en) * | 2009-12-21 | 2011-06-23 | Qualcomm Incorporated | Temporal and spatial video block reordering in a decoder to improve cache hits |
US20110208286A1 (en) * | 2006-07-07 | 2011-08-25 | Abbott Cardiovascular Systems Inc. | Catheter assembly and method for treating bifurcations |
Families Citing this family (32)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR100599017B1 (en) * | 1996-12-12 | 2006-12-13 | 소니 가부시끼 가이샤 | Image data compression device and method |
JPH11220736A (en) | 1998-01-30 | 1999-08-10 | Canon Inc | Image processor and, method and recording medium thereof |
US6971011B1 (en) * | 1998-03-04 | 2005-11-29 | Koninklijke Philips Electronics N.V. | Watermark detection |
JP3266569B2 (en) * | 1998-07-29 | 2002-03-18 | 日本電気株式会社 | Image attribute change system using digital watermark data |
ID26575A (en) * | 1999-01-13 | 2001-01-18 | Koninkl Philips Electronics Nv | SAVE ADDITIONAL DATA IN ONE SIGNAL PROVIDED BY CODE |
JP4705297B2 (en) * | 1999-10-12 | 2011-06-22 | シヴォリューティオン ベー. フェー. | Preliminary information embedding method and system, and decoding method and system |
US6449380B1 (en) * | 2000-03-06 | 2002-09-10 | Intel Corporation | Method of integrating a watermark into a compressed image |
US6687384B1 (en) * | 2000-03-27 | 2004-02-03 | Sarnoff Corporation | Method and apparatus for embedding data in encoded digital bitstreams |
GB2362531A (en) * | 2000-05-15 | 2001-11-21 | Nokia Mobile Phones Ltd | Indicating the temporal order of reference frames in a video sequence |
DE60114638T2 (en) | 2000-08-16 | 2006-07-20 | Dolby Laboratories Licensing Corp., San Francisco | MODULATION OF ONE OR MORE PARAMETERS IN A PERCEPTIONAL AUDIO OR VIDEO CODING SYSTEM IN RESPONSE TO ADDITIONAL INFORMATION |
JP2002330434A (en) * | 2001-04-27 | 2002-11-15 | Pioneer Electronic Corp | Decoder and decoding method, re-encoder and re-encoding method, information recording medium, and decoding program and re-encoding program |
DE10151111A1 (en) * | 2001-10-15 | 2003-05-08 | Deutsche Telekom Ag | Identification of digital sound or image sequences |
US7003035B2 (en) | 2002-01-25 | 2006-02-21 | Microsoft Corporation | Video coding methods and apparatuses |
ES2573317T3 (en) * | 2002-02-01 | 2016-06-07 | Koninklijke Philips N.V. | Apparatus and medium for recording support, and method, for recording a sequence of video data signals |
US20040001546A1 (en) | 2002-06-03 | 2004-01-01 | Alexandros Tourapis | Spatiotemporal prediction for bidirectionally predictive (B) pictures and motion vector prediction for multi-picture reference motion compensation |
US7154952B2 (en) | 2002-07-19 | 2006-12-26 | Microsoft Corporation | Timestamp-independent motion vector prediction for predictive (P) and bidirectionally predictive (B) pictures |
US7079578B2 (en) * | 2002-10-28 | 2006-07-18 | Scopus Network Technologies Ltd. | Partial bitstream transcoder system for compressed digital video bitstreams |
US7382969B2 (en) * | 2003-02-19 | 2008-06-03 | Sony Corporation | Method and system for preventing the unauthorized copying of video content |
US7409002B2 (en) * | 2003-09-30 | 2008-08-05 | Intel Corporation | Signal modulation |
JP4573511B2 (en) * | 2003-10-15 | 2010-11-04 | ルネサスエレクトロニクス株式会社 | Transcoder, transcoding method, data recording apparatus and method, and broadcast data receiving / recording apparatus |
KR101276847B1 (en) | 2006-01-12 | 2013-06-18 | 엘지전자 주식회사 | Processing multiview video |
JP5192393B2 (en) | 2006-01-12 | 2013-05-08 | エルジー エレクトロニクス インコーポレイティド | Multi-view video processing |
CN101371571B (en) * | 2006-01-12 | 2013-06-19 | Lg电子株式会社 | Processing multiview video |
DE102006005052A1 (en) * | 2006-02-03 | 2007-08-09 | Siemens Ag | Method and device for generating at least one user-specifically marked data stream, method and device for detecting a user-specifically marked data stream and a user-specifically marked data stream |
EP2052546A4 (en) | 2006-07-12 | 2010-03-03 | Lg Electronics Inc | A method and apparatus for processing a signal |
ATE550879T1 (en) * | 2007-12-19 | 2012-04-15 | Dolby Lab Licensing Corp | ADAPTIVE MOTION ESTIMATION |
US10075670B2 (en) | 2008-09-30 | 2018-09-11 | Entropic Communications, Llc | Profile for frame rate conversion |
US8347344B2 (en) * | 2008-12-18 | 2013-01-01 | Vmware, Inc. | Measuring remote video playback performance with embedded encoded pixels |
PT2413535E (en) * | 2010-07-30 | 2013-01-25 | Deutsche Telekom Ag | Method for estimating the type of the group of picture structure of a plurality of video frames in a video stream |
KR20150058324A (en) * | 2013-01-30 | 2015-05-28 | 인텔 코포레이션 | Content adaptive entropy coding for next generation video |
GB201317424D0 (en) | 2013-10-02 | 2013-11-13 | Givaudan Sa | Improvements in or relating to organic compounds |
KR20180093441A (en) | 2017-02-13 | 2018-08-22 | 주식회사 마크애니 | Watermark embedding apparatus and method through image structure conversion |
Citations (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5293229A (en) * | 1992-03-27 | 1994-03-08 | Matsushita Electric Corporation Of America | Apparatus and method for processing groups of fields in a video data compression system |
US5440345A (en) * | 1992-07-17 | 1995-08-08 | Kabushiki Kaisha Toshiba | High efficient encoding/decoding system |
US5646997A (en) * | 1994-12-14 | 1997-07-08 | Barton; James M. | Method and apparatus for embedding authentication information within digital data |
US5757910A (en) * | 1993-04-06 | 1998-05-26 | Goldstar Co., Ltd. | Apparatus for preventing illegal copying of a digital broadcasting signal |
US5809139A (en) * | 1996-09-13 | 1998-09-15 | Vivo Software, Inc. | Watermarking method and apparatus for compressed digital video |
US5859920A (en) * | 1995-11-30 | 1999-01-12 | Eastman Kodak Company | Method for embedding digital information in an image |
US5889868A (en) * | 1996-07-02 | 1999-03-30 | The Dice Company | Optimization methods for the insertion, protection, and detection of digital watermarks in digitized data |
US6131161A (en) * | 1995-10-04 | 2000-10-10 | U.S. Philips Corporation | Marking a digitally encoded video and/or audio signal |
US6957350B1 (en) * | 1996-01-30 | 2005-10-18 | Dolby Laboratories Licensing Corporation | Encrypted and watermarked temporal and resolution layering in advanced television |
Family Cites Families (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5329365A (en) * | 1993-07-07 | 1994-07-12 | Rca Thomson Licensing Corporation | Method and apparatus for providing compressed non-interlaced scanned video signal |
BE1007808A3 (en) * | 1993-11-30 | 1995-10-24 | Philips Electronics Nv | Apparatus for encoding a video signal. |
JP3137022B2 (en) * | 1997-02-24 | 2001-02-19 | 日本電気株式会社 | Video encoding device |
US6034746A (en) * | 1997-10-27 | 2000-03-07 | International Business Machines Corporation | System and method for inserting data into a digital audio/video data stream |
-
1998
- 1998-01-12 CN CNB98800027XA patent/CN1134172C/en not_active Expired - Fee Related
- 1998-01-12 JP JP10529168A patent/JP2000513167A/en active Pending
- 1998-01-12 WO PCT/IB1998/000026 patent/WO1998031152A2/en not_active Application Discontinuation
- 1998-01-12 BR BR9804764A patent/BR9804764A/en not_active IP Right Cessation
- 1998-01-12 EP EP98900023A patent/EP0891674A2/en not_active Withdrawn
- 1998-01-12 AU AU79955/98A patent/AU728966B2/en not_active Ceased
- 1998-01-12 KR KR1019980707209A patent/KR20000064585A/en not_active Application Discontinuation
- 1998-01-12 US US09/142,549 patent/US6850567B1/en not_active Expired - Fee Related
-
2004
- 2004-12-06 US US11/005,260 patent/US20050094728A1/en not_active Abandoned
Patent Citations (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5293229A (en) * | 1992-03-27 | 1994-03-08 | Matsushita Electric Corporation Of America | Apparatus and method for processing groups of fields in a video data compression system |
US5440345A (en) * | 1992-07-17 | 1995-08-08 | Kabushiki Kaisha Toshiba | High efficient encoding/decoding system |
US5757910A (en) * | 1993-04-06 | 1998-05-26 | Goldstar Co., Ltd. | Apparatus for preventing illegal copying of a digital broadcasting signal |
US5646997A (en) * | 1994-12-14 | 1997-07-08 | Barton; James M. | Method and apparatus for embedding authentication information within digital data |
US6131161A (en) * | 1995-10-04 | 2000-10-10 | U.S. Philips Corporation | Marking a digitally encoded video and/or audio signal |
US5859920A (en) * | 1995-11-30 | 1999-01-12 | Eastman Kodak Company | Method for embedding digital information in an image |
US6957350B1 (en) * | 1996-01-30 | 2005-10-18 | Dolby Laboratories Licensing Corporation | Encrypted and watermarked temporal and resolution layering in advanced television |
US5889868A (en) * | 1996-07-02 | 1999-03-30 | The Dice Company | Optimization methods for the insertion, protection, and detection of digital watermarks in digitized data |
US5809139A (en) * | 1996-09-13 | 1998-09-15 | Vivo Software, Inc. | Watermarking method and apparatus for compressed digital video |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20110208286A1 (en) * | 2006-07-07 | 2011-08-25 | Abbott Cardiovascular Systems Inc. | Catheter assembly and method for treating bifurcations |
US20090087110A1 (en) * | 2007-09-28 | 2009-04-02 | Dolby Laboratories Licensing Corporation | Multimedia coding and decoding with additional information capability |
US8229159B2 (en) | 2007-09-28 | 2012-07-24 | Dolby Laboratories Licensing Corporation | Multimedia coding and decoding with additional information capability |
US8571256B2 (en) | 2007-09-28 | 2013-10-29 | Dolby Laboratories Licensing Corporation | Multimedia coding and decoding with additional information capability |
US20110150085A1 (en) * | 2009-12-21 | 2011-06-23 | Qualcomm Incorporated | Temporal and spatial video block reordering in a decoder to improve cache hits |
US9877033B2 (en) * | 2009-12-21 | 2018-01-23 | Qualcomm Incorporated | Temporal and spatial video block reordering in a decoder to improve cache hits |
Also Published As
Publication number | Publication date |
---|---|
AU728966B2 (en) | 2001-01-25 |
CN1242907A (en) | 2000-01-26 |
AU7995598A (en) | 1998-08-03 |
BR9804764A (en) | 1999-08-17 |
WO1998031152A2 (en) | 1998-07-16 |
CN1134172C (en) | 2004-01-07 |
JP2000513167A (en) | 2000-10-03 |
US6850567B1 (en) | 2005-02-01 |
WO1998031152A3 (en) | 2000-08-17 |
KR20000064585A (en) | 2000-11-06 |
EP0891674A2 (en) | 1999-01-20 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US6850567B1 (en) | Embedding supplemental data in a digital video signal | |
USRE42505E1 (en) | Moving-picture signal coding and/or decoding system resistant to transmission error | |
EP1995964B1 (en) | Decoding variable-length encoded data | |
EP1351509A2 (en) | Differential encoding (DPCM) for video data | |
US5532837A (en) | Digital video signal processing apparatus | |
US7103104B1 (en) | Embedding auxiliary data in an information signal | |
JPH10506505A (en) | Synchronization method for continuous digital video | |
JP3270861B2 (en) | Decoding device for block transform code |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |