WO2001026376A1

WO2001026376A1 - Apparatus for encoding and decoding codes

Info

Publication number: WO2001026376A1
Application number: PCT/GB2000/003782
Authority: WO
Inventors: Jia Hong Yin
Original assignee: Central Research Laboratories Limited
Priority date: 1999-10-02
Filing date: 2000-10-02
Publication date: 2001-04-12
Also published as: GB9923212D0; AU7541300A

Abstract

A method and apparatus for encoding code into, and decoding code from, a series of stored images comprising : means for obtaining sub-codes, and means for inserting/recovering said sub-codes into/from a series of data fields or frames of an image. The data fields are identified by a sync signal, whereby an identification code (watermark) is derivable from a combination of said sub-codes and the sub-codes are derivable from said identification code.

Description

Apparatus for Encodine and Decoding Codes

This invention relates to an apparatus for, and method of encoding code into, and decoding code from, a seπes of stored images. More particularly, it relates to an apparatus for, and method of, encoding and decoding an electronic watermark into, or from, a video signal. The video signal may be generated for transmission or exist at reception or be stored on a data medium, such as a magnetic tape, CD. DVD or other medium.

The technique where codes are inserted into images, for example video transmissions or video clips or stills for transmission via a telecommunication link, is often referred to as electronic ^■watermarking^' as an analogy to the type of watermarks found in bank notes or bonds to veπfy their authenticity Codes are inserted, for example, by a copyright owner or licensee of the images. There are a number of known schemes for inserting codes into so called 'sync^' peπods of a video More recently it has been proposed to insert watermark codes into the image itself, but in such a manner that the code cannot readilv be detected by the human eye.

Published International Patent Application WO-A-9514289 discloses a technique of embedding an identification or watermark code throughout an image, by modulating a digitised version of the image with a small noise signal The technique descπbed suffers from the disadvantage of requiπng the oπginal image for code identification

Published International Patent Application VVO-A-9520291 discloses a method of hiding copyπght related messages within a digital data work. The method relies on commonly occurring patterns or sequences of data in the work acting as signposts to target data elements These elements are modified according to certain rules. The disclosed method suffers from a lack of robustness to signal degradation, for example duπng compression.

Another example of an apparatus employing watermarking for use in video signals is descπbed in the present Applicant's published International Patent Application No WO-A- 9841018 EP-A-0 901 282 descπbes a method for recording and reproducing watermark information in video signals. The watermark information is embedded into the video signals by alteπng the luminance values of selected pixels to specific predetermined values. Those pixels with luminance within the permissible alteration range, believed as invisible alteration range, is calculated depending upon a luminance signal, deπved from a visual feature. Since the coded luminance signals may have vaπed before decoding, a reference signal is recorded at a specific position m the video signal while embedded watermark information. In the process of decoding, the reference signal must be obtained m order to correct the luminance levels of images, so that the embedded watermark information can be detected efficiently.

The aforementioned encoding systems and techniques have to a greater or lesser extent been reasonably successful. However, as increasing numbers of authors and copyπght owners each demand a unique watermark there is an underlying problem of unique code availability So as to meet the demand, author identification codes or data words are becoming increasingly longer and are requiπng commensurate amounts of bits in order to meet this demand. Although this itself does not present a problem, as data words require more bits, there is a greater πsk that they may become visible or less robust. The present invention arose to overcome this and problems associated with the aforementioned arrangements.

According to an aspect of the present invention there is provided an apparatus for encoding code into and decoding code from a seπes of stored images compπsing: means for obtaining sub-codes, means for inserting/recoveπng said sub-codes into/from a seπes of data fields, said data fields being identified by a sync signal, whereby an identification code (w atermark) is deπvable from a combination of said sub-codes and said sub-codes are deπvable from said identification code.

A related method is also provided according to a second aspect of the invention' Video signals contain a sequence of discrete fields or frames of images; between two of which there is a fly-back interval containing vertical 'sync' pulse signals, test signals as well as other data such as teletext signals (in analog video). Other headers may also be stored in this 'svnc^' signal or 'sync' pulse in digital video. Existing techniques of encoding "watermarks" into video invisibly embed unique codes representative ot identification or ownership of these images

An advantage of the invention is that in order to achieve invisibility and robustness of embedded codes in images, even in the conditions where long identification or watermark codes are applied, code information contained in each field or frame of image only uses a small number of bits or sub-codes Thus, a full identification code does not have to be embedded into any single field of image if interlaced, or in one frame if non-interlaced, but can be embedded several fields or frames

By producing a plurality of sub-codes from a watermark and inserting sub-codes into a plurality of frames, the total amount of bits inserted in each frame is less Consequently there is less πsk of a watermark becoming visible or being corrupted by, say, data filteπng and/or compression transforms Means is preferably provided for extracting or generating the sub-codes and arranging them into a complete code.

Preferably sub-codes are inserted into adjacent frames of a sequence. However, sub-codes may be inserted into every second or third frame or in every odd or even frame. The exact number or sequence relationship of frames used is immateπal. However, what is important is that there is a suitable marker which is ideally a 'sync' pulse or signal at some predetermined location w ithin the sequence of sub-codes

Identification codes to be embedded in videos can be a seπes of letters, digits, or a combination of both. Large numbers of such unique identification codes may be needed for applications such as digital or analog television broadcast monitoπng, and such an identification code, with many digits or letters, may be embedded into video images

Sub-codes may be generated or reassembled into an identification code in the order they appear hidden in the seπes of frames; or they may be juxtaposed according to a predetermined pattern

It may be thought that because sub-codes in successive frames are separated, then there is a greater πsk of comφtion However, this is not the case because automatic data checking may be provided. A separate means can be included for this purpose Alternatively a checking feature can be included in an algoπthm. This is relatively straightforward to achieve because if sub-codes are repeated many times, (each seπes of repetitions being bounded by two ^'sync' pulses) one set of sub-codes is required in order to achieve authentication Consequently if a corrupted sub-code is detected, then the sub-code can be obtained and checked duπng one or more subsequent seπes of image frames.

Preferably , four bits of code are embedded into one field of image to represent one digit of a watermark. Alternatively, eight bits of code can be embedded into one field of image to represent one letter, (either capital letter or small letter) If four bits are used to represent sub-code information in one field, two fields (i.e. one frame) of images can host eight bits, w hich may be used to represent a letter.

λs mentioned above, a sync code may be used to indicate the beginning of a code and the end of a code, which is inserted into one frame. Preferably the first sync code is embedded in one frame when encoding starts. After this sync code, a full code is embedded in consecutive images. A second sync code indicating the end of the identification or watermark code is then inserted. This second sync code may indicate the beginning of a next code. Sync code can be any one or more numbers from, for example 10 (1010) to 15 d i l l )

Sync codes may also be used to indicate the type of codes as digits or letters For instance, 10 may be used as sync code for digit codes, 1 1 for small letter codes, and 12 for capital letter codes.

It only either digits or letters are used as an identification code, one sync code can be used to indicate the beginning of the code and the end of the code. If an identification code or watermark is used with a combination of digits and letters, more than one type of sync code is needed to indicate the beginning of a code and the end of the code, and also to indicate letter status and digit status This enables sub-codes detected by a decoder to be reassembled coπectlv to the oπginal watermark The identification or watermark may be a seπes of digits, letters either capital letters or small letters, or a combination of both digits and letters. The types of identification codes to be used may depend upon user needs or code standards

If only digits are used for identification codes, the codes can be any numbers ranging from 1 to a number with n digits (where n is an integer). For instance, if numbers in a range from 1 to 2,000,000.000 are used which have 10 digits, there are 2,000,000,000 identification codes available. If each digit is represented by four bits, the numbers within the range of 0 (0000) to 9 ( 1001 ) in the four bits may be used to represent the code. If letters are used as codes they may be defined by two digit numbers.

The process of embedding watermark or identification codes into images may be earned out starting at certain pixel or pixel block of the images in a specific manner The process of decoding should in general follow the same method but in reverse However, images with embedded identification codes may be transmitted through different communication channels and converters for example Analog to Digital (ADC) or Digital to Analog Converters (DAC), data compressors, or program editors. As a result of such transmission and modification, images to the decoder may become distorted or shifted from the images in the encoder A problem has been that it is unknown which pixel/block the process of decoding should start in shifted images, and it is usually very difficult or even impossible to judge the detected codes coπectly

According to a second aspect of the invention there is provided a method of decoding code from a seπes of stored images, compπsing the steps of detecting codes, by searching for a sync code and identifying the end or beginning of an identification code, removing subcodes and combining sub-codes, according to a predetermined order, so as to obtain an identification code.

The sync codes embedded into images are known By a technique of auto-searching through pixel by pixel, or block by block, in an image, the sync codes are detected.

Whenever a sync code is detected, the starting point of the process of decoding can be used as a reference point which indicates the shift position of the images in the decoder from the images in the encoder This reference point is used as compensation ot shift position to decode subsequent images for detection of watermark codes.

Embodiments of the invention are now descπbed with reference to the Figures in which :

Figure 1 shows diagrammatically a seπes of interlaced images, and how a plurality of subcodes are used to produce a watermark and are embedded into the images;

Figure 2 shows an example of a flow diagram illustrating the invention in the decoder, and

Figures 3 to 5 show examples of images, in which watermarks have been inserted

Referring to Figure 1, a watermark code indicated diagramatically by the numerals: '918273', is split into six sub-codes 9, 1 , 8, 2, 7 and 3 A sync pulse, indicated by 10, is used to mark the end and beginning of the watermark Figure 1 shows diagrammatically how the code is embedded.

If any one digit of a full code is inserted into one field of image for interlaced video, two digits can be inserted into one frame of image The full identification code is inserted into a sequence of consecutive images. The maximum number of frames for a full code of n digits in the example is n/2

At the beginning of the sequence, the sync code is embedded into 2 fields of images (1 frame) This is followed by the Watermark code 918273, a digit being inserted into each of six subsequent fields After this, another sync code is embedded, indicating the end of the complete Watermark code and the beginning of the next watermark code

To decode coπectly the coded images which may have been shifted in transmission from encoder to decoder, a decoder with auto-synchronisation is used. The decoder firstly finds the shift position of pixels of images to be decoded from an oπginal image in the encoder. This is done, for example, by searching for a sync code. This is also known as search- decoding Embedded codes are then decoded in the following consecutive seπes of images with compensation of the shift position. If there is no sync code detected duπng decoding for a certain number of frames, the process ot search-decoding starts again to find a more exact shift position. This method therefore copes with the different shift or distortion of different portions of programs which are provided by vaπous program resources.

The process of decoding with auto-synchronisation is shown by the flow chart in Figure 2. When the process of decoding starts, the process of search-decoding is first implemented. By finding the sync code, a shift position of the images in the decoder from the coded images in the encoder, can be deπved. The shift position is then used to compensate the shifted image in the process of decoding. The process of search-decoding is implemented from the start image. The process of search-decoding starts from the top of the image. If the sync code is detected, the shift position of pixels is estimated, and then the process of decoding is followed If no sync code is detected, the process of search-decoding is to shift one pixel and decode the image again The shifting is earned out by way of pixel by pixel or by way of "coarse" (block by block) to ^"fine" (pixel by pixel) searching until the sync code is found. The process of search-decoding stops as long as the sync code is found. If no svnc code can be found in this image, the next consecutive image is used for finding the sync code. This process continues until the sync code is found.

In the process of decoding, a sequence of consecutive images are decoded with compensation of the shift position. Detected code data from an image is only a part of an identification code. The code data from individual images has to be assembled as a full code in the process of Code Recovery whenever a sync code is detected which indicates the end of the codes. The number of consecutive images in which no sync code is detected, is counted as number of frames NF, in the process of count frames. If NF is larger than a pre-defined threshold, which indicates the process of decoding is not working properly, the process of search-decoding has to be implemented to find another shift position. If "NF" number is not larger than the pre-defined threshold, the process of decoding either continues or ends.

The process of decoding contains two stages: compensation of shift position and then code detection. The compensation of shift position is implemented by adding extra lines for vertical shifting at the beginning of each image and/or pixels to each line for hoπzontal shifting at the beginning of each line, or ignonng some lines for vertical shifting at the beginning of each image, and some pixels of each line for hoπzontal shifting at the beginning of each line, according to the shift position.

Figure 3 is an original coded image. A shifted image is shown in Figure 4 which is shifted from Figure 3 by 50 lines up and 50 pixels left respectively. The shift is just for the purpose of presentation, and the shift may be smaller or larger in practical situations. Figure 5 represents an image after compensation which shifts the shifted image Figure 4 back to its original position, and from which embedded code can be detected.

The invention has been described by way of examples only and variation may be made to the embodiments without departing from the scope of the invention.

Claims

Apparatus for encoding code into and decoding code from a senes of stored images compπsing means for obtaining sub-codes, means for lnserting/recoveπng said sub-codes into/from a seπes of data fields, said data fields being identified by a sync signal, whereby an identification code (watermark) is deπvable from a combination of said sub-codes and said sub-codes are deπvable from said identification code

Apparatus according to Claim 1 wherein means is provided for obtaining a plurality of sub-codes from a watermark and inserting each sub-code into a plurality of frames

Apparatus according to Claim 1 or 2 wherein means is provided for extracting or generating the sub-codes and arranging them into a complete code

Apparatus according to any preceding claim w herein means is provided for inserting sub-codes into adjacent frames of a sequence However, sub-codes may be inserted into every second or third frame The exact number of frames used is immateπal However, w hat is important is that at the end of a sequence of subcodes there is a suitable marker which is ideally a 'sync' pulse or signal

Apparatus according to any claims 1 to 3 wherein a means is provided for inserting sub-codes into a selected sequence of fields

Apparatus according to any one of claims 1 to 3 w herein a means is prov ided for inserting sub-codes into a selected sequence of frames

Apparatus according to any preceding claim wherein means for providing subcodes presents four bits of code to represent one digit of a watermark

8 Apparatus according to any preceding claim wherein means is provided tor detecting sync codes to indicate the beginning or end of a sequence of frames.

9. A method of decoding code from a seπes of stores images, compπsing the steps of detecting codes, by searching for a sync code and identifying the end or beginning of an identification code: removing sub-codes and combining subcodes, according to a predetermined order, so as to obtain an identification code or watermark.

10 Apparatus for carrying out the method according to claim 9 wherein there is provided a search means for searching the pixels of an image to locate the, or each sync code.

1 1 Apparatus according to claim 10 wherein the search means compπses means for searching blocks of pixels.

12. Apparatus according to claim 10 wherein an image shifting means is provided which is operable, in response to locating one or more sync codes to shift the displaced image vertically by inserting or removing one or more lines of the displayed image.

13 Apparatus according to claim 12 wherein the image shifting means is operable in response to locating one or more sync codes to shift the displayed image hoπzontally by inserting or removing pixels in one or more lines of the displayed image.

14. Apparatus according to claim 10 wherein the search means is operable to decode sub-codes from one or more selected fields.

15. Apparatus according to claim 10 wherein the search means is operable to decode sub-codes from one or more selected frames.

16. Apparatus according to any one of claims 10 to 15 wherein a code assembler means is provided which in response to detecting sub-codes, decodes the subcodes and assembles the sub-codes to obtain a complete watermark code.

17. Apparatus according to claim 16 wherein there is provided means for presenting the decoded watermark code.

18. Apparatus substantially as herein described with reference to the Figures.

19. A method substantially as herein described.