WO2002089056A1

WO2002089056A1 - Watermarking with coefficient predistortion

Info

Publication number: WO2002089056A1
Application number: PCT/GB2002/001845
Authority: WO
Inventors: David Roger Bull; Cedric Nishan Canagarajah; Dominique Winne; Henry David Knowles
Original assignee: University Of Bristol
Priority date: 2001-04-25
Filing date: 2002-04-23
Publication date: 2002-11-07
Also published as: GB0110151D0; GB2374996A; WO2002089056A8

Abstract

The present invention provides a scheme for inserting and extracting an invisible fragile watermark in an image/video sequence, in order to verify whether a watermarked image has been tampered with. According to one aspect of the invention, there is provided a method of watermarking a digital image, comprising: obtaining digital image coefficients; forming a plurality of arrays of image coefficients, each array comprising a predetermined number of image coefficients, the image coefficients comprising coefficients with first, second and third values; and for each array determining a binary value as a watermark component bit; determining a quantization parameter, based on the separation between the first and the second values; selecting a plurality of values to represent ones and zeros, the separation between successive selected values being determined on the basis of the quantization parameters; and modifying the third value, such that it takes the closest one of the selected values which represents the determined watermark component bit. According to another aspect of the present invention, there is provided a method of extracting a watermark embedded in that way.

Description

WATERMARKING WITH COEFFICIENT PREDISTORTION

This invention relates to watermarking, and in particular to a system for embedding and extracting a fragile watermark in a digital object, such as a digital image .

More specifically, the invention relates to techniques for inserting and extracting fragile watermarks in an image, for the purposes of image authentication, allowing a user to determine whether and how a watermarked image has been tampered with.

A watermark is a visible or invisible structure in an image, which- can be recovered after it has been embedded. A digital watermark is a digital pattern inserted into a digital creation, such as a digital image. The process of inserting a watermark into' a digital image (embedding procedure) can be done directly in the spatial or transformed domain. The watermark can be inserted by altering certain coefficients in a way which minimises the resulting distortion of the image. The imperceptibility of the watermark is the first line of defence, since, if an image is not visibly watermarked, it is more difficult to avoid the watermark, by tampering with the image undetectably .

Most watermarking techniques fall into one of two main categories, namely,the robust type, for copyright protection, and the fragile type, for authentication applications .

Robust watermarks must be embedded in the image so that it is virtually impossible, or at least difficult, to remove the watermark without visibly damaging the image. Such a watermark must be resistant to several image-processing techniques, such as cropping, scaling, filtering, compression/decompression, etc.

Fragile watermarks are designed to detect any unauthorised alteration of the image signal. Fragile watermarks may be used, for example, in connection with images generated by digital cameras to provide a basis for determining whether or not an image has been tampered with after its creation. The requirements of a fragile watermarking system change, depending on the data type and application.

An effective authentication scheme should have some or all of the following desirable features: to be able to determine whether an image has been altered or not, and to be able to locate any alteration made on the image, even without having the original image data; ^"*'" ^' to be able to integrate the authentication data with host image data, rather than as a separate data file; the embedded authentication data should be invisible under normal viewing conditions; to be able to extract information concerning the method of attack; to be able to restore the attacked area in the watermarked image.

US-6,064,764 proposes a fragile digital watermark embedding and extraction system where the insertion process is accomplished by embedding the bits of a digital signature of a hash function of the image in the frequency coefficients of the image. This procedure results in an imperceptible watermark, but the properties of hash functions prevent the localisation or quality assessment of the changes that have been made to the image .

EP-A2-0953938 describes an invisible digital watermark insertion technique for ownership verification and/or authentication purposes. The watermark is a combination of a given watermark bitmap and the hashed image. The extreme localisation is accomplished by embedding the digital signature in the Least Significant Bit (LSB) band of the image. However, the method cannot distinguish malicious changes from innocent image processing operations.

The paper "Digital watermarking using multiresolution wavelet decomposition" , Deepa Kundur and Dimitrios Hatzinakos, Proc . IEEE International Conference on Acoustics, Speech and Signal Processing, Seattle, Washington, May 1998, vol. 5, pp 2969-2972, discloses a robust watermarking technique.

The present invention seeks to provide a scheme for inserting and extracting an invisible fragile watermark in an image/video sequence, in order to verify whether a watermarked image has been tampered with.

According to one aspect of the invention, there is provided, a method of watermarking a digital image, comprising: obtaining digital image coefficients; forming a plurality of arrays of image coefficients, each array comprising a predetermined number of image coefficients, the image coefficients comprising coefficients with first, second and third values; and for each array: determining a binary value as a watermark component bit; _f determining a quantization parameter, based on the separation between the first and second values ; selecting a plurality of values to represent ones and zeros, the separation between successive ' selected values being determined on the basis of the quantization parameter; and modifying the third value, such that it takes the closest one of the selected values which represents the determined watermark component bit . The use of a quantization parameter, which is based on the separation between the first and second values as defined above, has the advantage that the watermark can be made fragile, while minimizing the amount of distortion which is introduced.

The first and second values are preferably the extreme values of the values of the coefficients.

According to a second aspect of the present invention, there is provided a method of extracting a watermark embedded in that way.

According to a third aspect of the present invention, there' is provided method of watermarking a digital image, comprising: obtaining digital image coefficients; forming a plurality of arrays of image coefficients, each array comprising a predetermined number of image coefficients, the image coefficients having two extreme values, and at least one inner value ; and for each array: determining a binary value as a watermark component bit; predistorting at least one of the extreme values of the image coefficients, if necessary, to ensure a minimum separation between the extreme values; * selecting a plurality of values between the extreme values to represent ones and zeros; and modifying the inner value, or at least one of the inner values, of the array, such that it takes the closest one of the selected values which ' represents the determined watermark component bit . The use of the predistortion step, defined above, has the advantage that the watermarking performance is improved in plain low image density areas of the image. For a better understanding of the invention, and to show how it may be put into effect, reference will now be made to the accompanying drawings .

Figure 1 is a flow chart illustrating a method of embedding a watermark, in accordance with the invention.

Figure 2 illustrates a quantization process used as part of the method of Figure 1.

Figure 3 illustrates a predistortion procedure used as part of the method of Figure 1.

Figure 4 is' a flow chart illustrating a method of extracting a watermark, in accordance with the invention.

Firstly, Figure 1 illustrates an embodiment of the invention in which a watermark is embedded in an image.

The algorithm described herein is a modification of an existing robust watermarking scheme of Kundur and Hatzinakos .

In step 101, the procedure starts with a set of image data. The image data may be in any form. For example, the image data may be a set of luminance and chrominance values associated with respective pixels of an image. The embedding of the watermark is advantageously carried put close to the source of the image. For example, the watermarking procedure can be carried out within a digital camera, for example.

The method of embedding the watermark into a digital image signal can be carried out in the spatial or transformed domain, depending on the application. Thus; step 102 shows a transformation procedure. The transformation may be of any type, and may for example involve performing a digital cosine transform (DCT) , a Fourier transform, a DFT, DST, Walsh, Hadamard, Hartley, or wavelet transform. Thus, when referring herein to coefficients, these can represent pixel values for a greyscale or coloured image, or DCT, wavelet or any transformed coefficients of the digital signal .

Next, in step 103, the coefficients of the image are divided into blocks. After watermarking, each block will contain a string of binary watermark values, and the block size represents the detection resolution. The smaller the block size, the finer the detection resolution.

In step 104, several arrays are generated inside each block. The- total number of arrays in each block will determine the number of watermark bits which can be embedded in each block. Each array contains three of the coefficients of the block, the coefficients forming each array depending on a secret key. Thus, the secret key acts as a seed number for a random number generator. The random number which is generated then controls the way in which the arrays of coefficients are formed.

The random number also provides a secret key which is used in the same way in the next block, and so on. This means that the watermark is effectively embedded in different blocks in different ways. This provides a defence against an attempt to tamper with an image by forming a collage of blocks from different images or from different parts of the same image.

The initial key, which governs the creation of the secret keys used in the respective blocks, is stored in the device in which the watermark is embedded, such that it can be retrieved only by an authorised person. .'In step 105, the watermark bit string is embedded in the selected block by modifying the values of the coefficients in each array, according to an algorithm to be described below. In steps 106 and 107, the watermarked image is obtained, by replacing the modified coefficients in their original position and then by calculating the inverse of any transform performed in step 102.

The basis of the algorithm which is used is described in the paper: "Digital watermarking using multiresolution wavelet decomposition" , by Kundur and Hatzinakos, cited above. Their technique is used for robust watermarking.

The basis of the algorithm used herein is that the available image data coefficients are divided into arrays, each array preferably comprising three coefficients, namely the smallest coefficient, f_s, the middle coefficient, f_m, and the largest coefficient, f_x .

The range between the smallest coefficient, f_s, and the largest coefficient, f_1; is divided into a number of equally sized bins, and the value of the middle coefficient, f_ra, is modified, based on a relationship between the bins and the watermark bit which is to be embedded .

In accordance with the preferred embodiment of the present invention, the bin width δ is calculated from the values of the smallest coefficient, f_g and the largest coefficient, f_{l r} according to the following formula :

where Q is a variable that defines the number of

I bins between f_x and f_g.

By contrast with the prior art robust watermarking system, in the illustrated embodiment of the present invention, the value of Q is determined by the difference value (f_x - f_s) .

- Figure 2 illustrates the way in which the gap between the smallest and largest coefficient values is divided into bins . After sorting the three coefficients of the array into ascending order, the gap between them is divided into a number, Q, of bins, each of width δ. The value which is δ/4 from the left-hand end of each bin is then assigned to the value 0, as indicated by the dashed lines in Figure 2, while the va.lue which is 3δ/4 from the left-hand end (i.e. δ/4 from the right-hand end) of each bin is assigned to the value 1, as indicated by the solid lines in Figure 2.

Then, depending on whether the watermark bit for that array is a 0 or a 1, the value of the middle coefficient f_m is modified to the closest value assigned to that bit value, that is, the closest dashed or solid line in Figure 2.

As a result,- the maximum amount of distortion which can be introduced by this modification is δ/2.

As mentioned above, the value of Q, the number of bins, is a content dependent quantization parameter. The user sets a maximum permitted modification of f_m, and the value of Q is set to an integer number determined by this maximum permitted modification and by the difference between f_λ and f_s .

For example, if the maximum permitted modification is set to be 10, and the difference between f_λ and f_g is 100, then a value of Q=5 will be chosen, since, then:

10 = δ/2 = (100 / 5) /2.

The value of the quantization parameter, Q, will be increased to 6 if the difference between f_x and f₃ is higher than 100.

> Using a content dependent quantization parameter has the effect that, when the image data allows it, a larger value of Q, and a correspondingly smaller value of δ, can be used. This increases the fragility of the watermarking scheme, and hence its ability to detect tampering with the image, while minimizing the distortion introduced by the watermarking system.

It will be noted that this watermarking process uses three coefficients to embed one watermarking bit

(although arrays can include more than three coefficients if desired) . This allows a watermark of 21 bits to be embedded in an 8x8 block of 64 coefficients .

Thus, the watermark is embedded in the image itself rather than appended, and thus provides a more flexible and convenient tool for subsequently investigating the image integrity.

In the example described above, the range between the smallest coefficient and the largest coefficient is divided into bins, and the middle coefficient is modified. However, it is also possible to divide the range between the- middle coefficient and one of the two outer coefficients into bins, and then to extend bins of the same size so that they include the other outer coefficient, and then to modify that other outer coefficient in the same way as described above. When embedding a watermark in the wavelet transform domain, the array can be generated from the three different detail coefficients at a given position (m, n) . For example, the array of three coefficients is combined from the HL, HH and LH coefficients.

The secret key, described earlier, determines how the coefficients are grouped to form the arrays of three coefficients. In some cases, some coefficients can be used several times in different arrays. If any coefficients are to be used more than once, it is preferable that these should be very large or small coefficients, since it is only the median value coefficients that are modified.

Also, the same watermark bit could be applied to more than one array, and detected separately in each array. This technique could advantageously improve the robustness of the watermark to white Gaussian noise attack.

The watermark embedding process can be carried out by a general purpose computer, operating under the control of suitable software, or by another hardware device, such as a DSP or an ASIC, or other integrated circuit .

Figure 3 illustrates a further modification to the prior art robust watermarking scheme, in order to improve its performance in plain low image density areas, which are often difficult to watermark. By predistorting these areas, the algorithm introduces some high frequency noise in the more or less constant wavelet coefficients. This is generally unnoticeable, not least because the original image is not available to compare with- the watermarked image, and the introduced high frequency noise is spread over the whole of the plain area. This predistortion function ensures a user defined minimum difference between f_x and f_g/ before embedding the binary watermark value.

Thus, the process of Figure 3 can be carried out as an initial part of step 105 of Figure 1.

The predistortion procedure starts at step 301, and, in step 302 a counter i is initialised and a user defined minimum difference value, min_diff, is set, in this case to an exemplary value of 10.

In step 303, it is determined whether a predistortion is necessary, that is, whether the difference between f_x and f_s is less than the user defined minimum difference value. If it is not, no predistortion is necessary, and the process ends (step 304) .

If it is determined that predistortion is required, this is calculated in a loop of steps 305 and 306, and then, in step 307, the largest coefficient, f_x, is increased by an amount sufficient to make f_x equal to f_g plus the preset minimum difference value. For instance, if the difference between f_λ and f_s is 6 then the value of f_λ is increased by 4 to achieve the minimum difference of 10. The predistortion procedure then ends, and the process returns to step 105 in the watermarking procedure .

Figure 4 illustrates the watermarking extraction procedure, as carried out in a detector. For example, the detection procedure can be performed in a suitably programmed device.

The procedure starts at step 401, and, in step 402, calculates the coefficients of the image with the necessary transform and initialises a counter value i .

In step 403, if several arrays contained one watermark bit -to. -improve the robustness of the watermark as described earlier, the coefficients are divided in blocks. The detected watermark bit can then be obtained from a weighted estimation of the results obtained from different arrays, as described in more detail below.

In step 404, the arrays containing the three coefficients are formed. This is controlled by a secret key function, which uniquely creates the correct arrays, using the same secret key used in the watermark embedding process. Thus, it is necessary to retrieve the key that was used in the watermark insertion process, in order to be able to determine the grouping of the coefficient arrays, and in order to extract the embedded watermark.

Thus, the secret key function controls the security of the authentication process even if the embedding and extraction algorithms are publicly known.

The following steps apply to each array.

In step 405, the given array is sorted in ascending order and the quantization parameter Q is^'' extracted from the difference between f_x and f_a. Then the bin size δ is calculated using the formula given previously. Once f_s and δ are known, the detected watermark binary value can be extracted from the position of f_m relative to f_x and f_s . Specifically:

Steps 407 and 406 are looped by incrementing the counter i to locate the bin in which f_ra lies, namely the value of i which allows: f_m e [f₃ + (i-l)δ , f₃ + i.δ]

Then, by comparison of the relative position of f_m inside that interval, we obtain the extracted watermark value .

Thus, as set out in steps 408, 409 and 410 of Figure 4, if: f_m -(f_g + (i-l)δ) < (f₃ + i.δ)-f_m , then W = 0, otherwise,..W. = 1.

Thus, step 408 determines the relative position of f_m and pinpoints the extracted binary value for this array.

This extraction process is repeated for all of the arrays of three coefficients, until all of the embedded bits have been extracted. The extracted bits then represent the extracted watermark, unless the arrays were grouped together to carry the same bit .

If multiple arrays carry the same bit, a weighted decision determines the extracted watermark bit value.

In that case, the process for extraction of the watermark bit is as described above, except that, instead of the thresholding step 408, a value v (where 0 ≤ v ≤ 1) is determined, based on the position of f_ra, relative to the nearest value which is assigned to the value 0 and the nearest value which is assigned to the value 1.

For example, if the determined position of f_m is equidistant between the nearest value which is assigned to the value 0 and the nearest value which is assigned to the value 1, then a value v of 0.5 is determined. To illustrate the result of this process, if a watermark bit was embedded in three arrays, and the three coefficients extracted from these arrays give values of v, based on their relative positions as described above, of: 0.6, 0.6 and 0.1, then a weighted decision results in an extracted value of 0. Thus: round( (0.6+0.6+0.1) /3) - 0.

By contrast, following the process as shown exactly in Figure 4, and then taking the average of the extracted (separately rounded) bits is a 1:- round(0.6) - 1 round (0.6) - 1 round (0.1) - 0 average = 1.

After the extraction process, the extracted watermark bit values can be compared with the originally embedded watermark. This comparison locates any tampered area, if the extracted bit does not match the embedded bit on the given position.

The location of the array which includes the non- matching bit indicates the tampered region of the image. Since it is advantageous to be able to locate the illegally tampered area precisely, the detection resolution should preferably be as high as possible.

The watermark can be embedded in the secret key information or standard built in the detector, as the watermark is not the most important factor of the detector.

The watermark extraction and comparison process can be carried out by a general purpose computer, operating under the control of suitable software, or by another hardware device, such as a DSP or an ASIC, or other integrated circuit .

. The correct formation of the arrays of coefficients plays an important role in the detection process .

If a watermarked image is resized or cropped, then the extraction procedure will also return an output that indicates that the whole image was changed. This would result in a random detector output if the watermarked image is resized or cropped.

For colour images, the watermark can be embedded in every plane or just a specific plane.

In the case of a video sequence, each frame or just some selected frames, e.g.: I-frames, can be watermarked, depending on the application and the ability to detect attacks which involve frame reordering and frame removal .

This fragile watermarking scheme can be used in combination with; a robust watermark algorithm to form a hybrid watermark system. The hybrid watermark can then be used to pinpoint changes accurately, as well as to distinguish malicious tampering from innocent operations .

Claims

1. A method of watermarking a digital image, comprising: obtaining digital image coefficients; forming a plurality of arrays of image coefficients, each array comprising a predetermined number of image coefficients, the image coefficients comprising coefficients with first, second and third values; and for each array: determining a binary value as a watermark component-,>,bit ; determining a quantization parameter, based on the separation between the first and second values; selecting a plurality of values to represent ones and zeros, the separation between successive selected values being determined on the basis of the quantization parameter; and modifying the third value, such that it takes the closest one of the selected values which represents the determined watermark component bit.

2. A method as claimed in claim 1, wherein the quantization parameter is determined on the basis of a preset maximum separation of the selected values.

3. A method as claimed in claim 2, wherein the preset maximum separation of the selected values is user selectable.

4. A method as claimed in claim 1, 2 or 3, wherein the first and second values are the smallest and the largest values of coefficients in the array.

.

5. A method as claimed in any preceding claim, wherein the image coefficients comprise transform coefficients .

6. A method as claimed in any preceding claim, wherein the image coefficients comprise bit values.

7. A computer system, programmed to carry out a method as claimed in any one of claims 1 to .

8. A hardware device, adapted to carry out a method as claimed in any one of claims 1 to 6.

9. An image generation device, comprising: means for generating a digital image; and means for carrying out a method as claimed in any one of claims 1 to 6.

10. A computer software product, containing code for carrying out a method as claimed in any one of claims 1 to 6.

11. A method of detecting a watermark in a digital image, the method comprising: obtaining digital image coefficients; forming a plurality of arrays of image coefficients, each array comprising a predetermined number of image coefficients, the image coefficients comprising coefficients having first, second and third values; and for each array: determining a binary value as an actual watermark component bit; determining a quantization parameter, based on the separation between the first and second values; selecting a plurality of values to represent ones and zeros, the separation between successive selected values being determined on the basis of the quantization parameter; and determining which of the selected values is closest to the third value; determining a detected watermark component bit, based on whether the closest one of the selected values to the third value represents a one or a zero; and comparing the detected watermark component bit and the actual watermark component bit .

12. A method as claimed in claim 11, further comprising : displaying said digital image, distinguishing between regions of the image in which the detected watermark component bit matches the actual watermark component bit, and regions of the image in which the detected watermark component bit does not match the actual watermark component bit .

13. A method as claimed in claim 11 or 12, wherein the digital image is a frame of a video sequence . ^--- - ^'

14. A computer system, programmed to carry out a method as claimed in any one of claims 11 to 12.

15. A hardware device, adapted to carry out a method as claimed in any one of claims 11 to 12.

16. A computer software product, containing code for carrying out a method as claimed in any one of claims 11 to 12.

17. A method of watermarking a digital image, comprising : obtaining digital image coefficients; forming a plurality of arrays of image coefficients, each array comprising a predetermined number of image coefficients, the image coefficients having two extreme values, and at least one inner value ; and for each array: determining a binary value as a watermark component bit; predistorting at least one of the extreme -'values of the image coefficients, if necessary, to ensure a minimum separation between the extreme values ; selecting a plurality of values between the extreme values to represent ones and zeros; and modifying the inner value, or at least one of the inner values, of the array, such that it takes the closest one of the selected values which represents the determined watermark component bit .

18. A method as claimed in claim 17, wherein the number of selected values, and the separation between successive selected values, are determined on the basis of the separation between the extreme values, and on the basis of a preset maximum separation of the selected values.

19. A method as claimed in claim 18, wherein the preset maximum*, separation of the selected values is user selectable.

20. A method as claimed in claim 17, 18 or 19, wherein each array comprises three image' coefficients.

21. A method as claimed in any of claims 17-20, wherein the image coefficients comprise transform coefficients .

22. A method as claimed in any of claims 17-21, wherein the image coefficients comprise bit values.

23. A computer system, programmed to carry out a method as claimed in any one of claims 17 to 22.

24. A hardware device, adapted to carry out a method as claimed in any one of claims 17 to 22.

25. An image generation device, comprising: means for generating a digital image; and means for carrying out a method as claimed in any one of claims 17 to 22.

26. A computer software product, containing code for carrying out a method as claimed in any one of claims 17 to 22.