US20060133647A1 - Watermarking - Google Patents

Watermarking Download PDF

Info

Publication number
US20060133647A1
US20060133647A1 US11/328,413 US32841306A US2006133647A1 US 20060133647 A1 US20060133647 A1 US 20060133647A1 US 32841306 A US32841306 A US 32841306A US 2006133647 A1 US2006133647 A1 US 2006133647A1
Authority
US
United States
Prior art keywords
watermark
picture
data
values
sequence
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US11/328,413
Inventor
Oliver Werner
Andrew Mason
Richard Salmon
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
British Broadcasting Corp
Original Assignee
British Broadcasting Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from GBGB9906299.4A external-priority patent/GB9906299D0/en
Priority claimed from GBGB9926530.8A external-priority patent/GB9926530D0/en
Priority claimed from GB9930353A external-priority patent/GB2348071B/en
Application filed by British Broadcasting Corp filed Critical British Broadcasting Corp
Priority to US11/328,413 priority Critical patent/US20060133647A1/en
Publication of US20060133647A1 publication Critical patent/US20060133647A1/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/46Embedding additional information in the video signal during the compression process
    • H04N19/467Embedding additional information in the video signal during the compression process characterised by the embedded information being invisible, e.g. watermarking

Definitions

  • the invention relates to embedding a watermark into either a still picture or a sequence of moving pictures; this may be used to assist in detection of copying or identification of the originator of material or for transmitting hidden data.
  • a watermark is a mark or data sequence that is embedded substantially invisibly into a picture for helping to identify the originator or the intended recipient of the picture or to detect tampering.
  • the invention provides a method as set out in claim 1 .
  • each watermark value By making use of a plurality of adjustment factors for each watermark value, each of which adjustment factors is a function of a local estimate of visibility of the watermark within the picture and which is a function of the picture sample values (and substantially independent of the watermark values), it has been found that, surprisingly, the watermark information can be reliably embedded in the picture without causing unacceptable distortion to the picture.
  • the embedded watermark value may then change the local mean or DC-values of the subset of pixels in which it is embedded, rendering detection simple and reliable.
  • this method may result in a watermarked picture in which watermark information is barely present or even not present at all in certain regions of the picture where the estimate of visibility suggests that the presence of the watermark is likely to cause visible distortions to the picture.
  • the magnitude of adjustment factors is determined from the picture sample values based on an estimate of visibility, preferably from the local variance. This enables the watermark values to be concealed effectively.
  • the sign of the adjustment factors is preferably a function of the watermark values, the watermark values preferably comprising a binary sequence of 0 and 1, being encoded as positive and negative signs respectively, or vice versa.
  • a robust coding scheme is provided, the magnitude of the change not being critical in detecting the watermark and hence being adjustable to allow the watermark to be kept substantially invisible.
  • the magnitude may be adjusted in steps; this may increase the available data capacity but may increase visibility, reduce robustness or increase complexity.
  • measures of visibility may be determined from the watermarked picture which should correspond substantially to the originally determined measures of visibility
  • decoding may disregard regions where a visibility estimate suggests that no data will be encoded. Weighted filtering may be employed at a decoder, based on an estimate of visibility.
  • a three level system in which zero adjustment is employed could be used; this may not be so advantageous for simple watermarking as zero adjustments would not assist in correlation, but could be used for data samples carried within the watermark (as discussed below) to increase the data capacity.
  • the adjustment factors may be combined with picture values by adding (which term is intended to encompass weighted addition or subtraction); this is simple to implement but effective, but more complex combination such as averaging may be employed.
  • the picture samples are preferably substantially adjacent. Small translations and distortions will tend to leave at least some of each subset of samples sufficiently unaffected for a determination of the watermark value for that subset.
  • a grid may be defined, preferably substantially rectangular for ease of processing, although hexagonal or other grid shapes may be employed, and the samples in each region of the grid may be assigned To a subset. In this way, when it comes to decoding, a slight misalignment of the grid will normally not prevent the majority of samples within a decoding grid from carrying the correct value and permitting correct decoding,
  • the grid preferably corresponds to blocks or groups of blocks of the coding algorithm. This may enable efficient processing and may also ensure that the watermark is reliably carried (more so than if individual watermark values were assigned to pixels in different blocks).
  • each subset comprises a block of at least about 4 by 4 samples (or 1 6 samples if non-rectangular grouping is used), this provides a much higher degree of robustness against a variety of attacks than a comparative example in which significantly fewer (or only one) picture samples per watermark value are employed.
  • blocks of at least about 8 by 8 samples are used, more preferably, at least for broadcast quality images, blocks of at least about 8 by 16 (preferably 8 vertically, 16 samples horizontally).
  • a restriction condition is applied to the choosing of watermarks from the available watermarks. This may reduce the amount of data that can be carried but may increase robustness or error tolerance or detection.
  • the watermarks are subdivided into subsets (for example 16 watermarks are sub-divided into 4 subsets of 4 marks) and a restriction condition related to the subsets is employed, for example exactly one (or in certain less preferred cases another predetermined number, for example in the case of larger subsets) watermark is chosen from each subset or each of a predetermined number of subsets (for example 1 mark from each of 3 of 4 subsets of 4 marks).
  • a measure of errors or the reliability of the data can be obtained based on the fact that there should be a given number (preferably one) of watermarks in each subset so if detection yields no marks or a reasonable probability of more than one mark in a subset, it can be assumed that the data is noisy or unreliable.
  • Watermark values may be assigned to substantially the whole of a picture. This may increase the dimension of the watermark and make unauthorised copying and detection more difficult. However, a repeating watermark may be used, or certain portions of the picture may be left blank.
  • the watermark may comprise a substantially static component and a variable component, the static component enabling the watermark to be positively identified, and the variable component carrying additional information, for example one or more of picture (or programme) title, date, author, originator, intended recipient, copying permissions, equipment or recording or coding conditions, user definable data and the like.
  • additional information for example one or more of picture (or programme) title, date, author, originator, intended recipient, copying permissions, equipment or recording or coding conditions, user definable data and the like.
  • data may be carried with the watermark.
  • a separate hidden data stream may be carried, some of the watermark assisting in alignment and framing and the remainder carrying user data. If the application is such that the source and framing of the sequence can be guaranteed, then no static watermark may be needed for synchronisation, and the whole of the watermark may in fact comprise variable user information.
  • the watermark preferably comprises a pseudo random pattern; this makes it harder for an unauthorised person to detect or apply the watermark.
  • a logo or regular pattern may be employed; this may simplify identification, for example visually from a difference picture.
  • the watermarked picture may be communicated or stored together with data facilitating identification of the watermark, as set out in claims 31 to 33 .
  • the invention further provides a method of testing for the presence of a watermark as set out in claim 34 and the preferred features set out in claims 35 to 44 .
  • the invention further provides methods of creating, embedding and detecting data-carrying watermarks, methods of marking moving pictures, applications of marked pictures, computer program products and apparatus for implementing any of the methods described above or below and further aspects and preferred features are set out in the other independent and dependent claims respectively, and may also be found in the following description of a preferred embodiment.
  • FIG. 1 shows a general outline of a watermarking system
  • FIG. 2 illustrates partition of a picture into square cells
  • FIG. 3 schematically illustrates an example of calculation of variance for current sample s n and its neighbouring samples that belong To B n;
  • FIG. 4 illustrates partition of a picture into square cells that carry either a watermark sample or a data bit
  • FIG. 5 illustrates sub-division of a set of 16 watermarks into 4 subsets of 4
  • FIG. 6 shows an 8 ⁇ 8 sample block for carrying data
  • FIG. 7 shows the blocks of FIG. 6 tiled m ⁇ n
  • FIG. 8 shows the tiled blocks of FIG. 7 forming a data watermark spread across a picture.
  • the watermark w is embedded into the original signal, resulting in the watermarked signal.
  • the watermarked signal may be changed by friendly attacks that are caused by transmission techniques e.g. data compression or by hostile attacks that deliberately attempt to remove the watermark. Therefore, the signal ⁇ w instead of S w feeds the input of the watermark-detector.
  • the detector outputs a binary decision which indicates if a given watermark is present in the input signal or not.
  • restricted watermarking means that the original signal is also needed as an input to the watermark-detector.
  • watermark detection is restricted to users who are in possession of the original signal
  • Unrestricted watermarking means that the original signal is not needed during detection. Both cases will be addressed.
  • the watermark is a zero-mean (to prevent a change in global mean (average brightness) of the signal after watermarking) white noise random signal that is chosen independently from the original signal and difficult-to-predict for an attacker.
  • the role of the watermark is similar to the role of the secret key in a symmetric crypto-system
  • the seed-number of the bit generator can depend on the value of a hash function that is applied to the samples of the original signal for providing authentication.
  • the original picture s is partitioned into cells. This is shown in FIG. 2 for a square cell shape. There is no restriction on the cell shape and any other tiling of the x/y-plane can be used just as well, for example a hexagonal grid.
  • the cell shape and the cell size are parameters that are kept secret together with the watermark w.
  • C k denote the set of indices that select the samples of the k-th cell.
  • the magnitude depends on a visibility measure and determines by how much the amplitude of a sample can be changed without creating a visible distortion.
  • the local variance can be calculated in a small window of 7 ⁇ 7 samples that is centred at the current sample position, see FIG. 3 .
  • B n denote the set of indices that select the samples in the neighbourhood including the current index n.
  • var ⁇ ( n ) ( 1 ⁇ B n ⁇ ⁇ ⁇ l ⁇ B n ⁇ s z ⁇ ( l ) ) - ( 1 ⁇ B n ⁇ ⁇ ⁇ l ⁇ B n ⁇ s ⁇ ( l ) ) 2 ( 3 )
  • Eqn. (5) is one example for the calculation of the magnitude ⁇ 2 (n) and more sophisticated models of the human visual system can be applied in combination with the embedding method that is specified in eq. (2).
  • the received signal ⁇ hacek over (s) ⁇ w is partitioned into cells of appropriate shape and size corresponding to the embedding procedure. For each cell a sample mean value is calculated, hereafter called DC-value.
  • the coefficients ⁇ 1 are the same for each picture and can be calculated by linear regression for minimising the mean squared error between DC(k) and DC pred (k). However, in terms of computational complexity a simpler prediction method is to average the DC-values of the neighbouring cells.
  • the detector decides upon the presence of the watermark in the signal ⁇ hacek over (s) ⁇ w if the magnitude of the correlation value exceeds a threshold,
  • the sign sgn(corr) of the correlation value signals one hidden data bit if the presence of the watermark is detected. As no watermark information is embedded into flat areas the detector has the option to exclude flat areas during the evaluation of eq. (9).
  • This property improves significantly the robustness against attacks that re-sample the picture including geometric attacks that introduce an unnoticeable amount of distortion.
  • the cell size and the cell shape can be adapted for improving the robustness against specific types of geometric distortions. Rotation, cropping or scaling by a noticeable amount can be handled by hypothesis testing which however results in a computational intensive search for maximum correlation.
  • a common method for increasing the capacity of the hidden data channel is to partition the picture into sub-pictures and to apply the above watermarking method to each sub-picture.
  • Another method for increasing the data capacity shall now be exemplified for a square cell shape, see FIG. 4 .
  • the white cells are used for the watermarking method as described above. Firstly, the watermark is detected from the white cells at the receiver. This also allows synchronisation and alignment of the sampling grid. Secondly, one data bit is detected from each dark cell as follows. For the k-th cell the DC-values of eqs. (6)-(8) are calculated.
  • the data bit is embedded similar to eqs (2)-(5), the watermark sample is replaced with the antipodal data bit in eq. (2).
  • the detector has the option to evaluate the data bit only if th ,min ⁇
  • ⁇ DC (k) (DC (k) ⁇ DC pred (k)) ⁇ th ,max this can be taken into account during embedding.
  • Each payload is represented by a combination of three watermarks. Although there could be fewer (for example 2) or more watermarks combined to increase the data capacity, we have found that, surprisingly, by combining exactly three watermarks from a number (ideally a defined set) of substantially independent watermarks, an optimum result may be achieved in terms of reliability of detection and increase in data payload. Thus, there is a total of N ⁇ (N ⁇ 1) ⁇ (N ⁇ 2) /6 possible combinations (neglecting trivial or redundant combinations in which two or three watermarks are the same or the order of combination is altered [(which cannot be detected using simple combination of watermarks with the picture]).
  • the payload may be most advantageously increased by selecting a reasonably large number of independent watermarks in this set or “library” and then combining exactly three of these in any one picture (rather than combining larger numbers of watermarks), the maximum size of the library being dependent on the ease with which watermarks can be distinguished, so varying with picture size.
  • the actual payload is represented by the three watermarks w 1 , w 2 , w 3 .
  • One could then generate a watermark w, by calculating the average value of w 1 , w 2 and W 3 . (4) w [w 1 +w 2 +w 3 ]/3
  • the watermark w is embedded in the DC-values of the picture in the usual way.
  • For detection of the payload one generates the watermark w 1 (10) w′ w 1 +. . . +w N and cross-correlates the watermark w′ with the DC-values of the pictures in the usual way. If a correlation-peak is detected, the payload is retrieved by cross-correlating each watermark w 1 , . . . , w N separately with the DC-values and selecting the three watermarks with the largest correlation peaks.
  • the patterns are independent and satisfy the above rules, it may be desirable in some cases to use patterns which are not truly independent but have a low level of cross correlation (this may increase the number of patterns that can be used or simplify pattern selection); this may make it harder to detect each component reliably, but this may be useful in certain applications, for example where it is intended that the data embedded should be well concealed or “fragile” (i.e. easily corrupted).
  • the above provides a method of combining three bipolar watermarks to produce a single bipolar watermark with the property that the product of the combined watermark with each of the constituent marks has an expectation value of 1 ⁇ 2.
  • the available watermarks may be sub-divided into subsets.
  • the set is portioned into 4 subsets of 4 marks and exactly one mark is chosen from each of 3 subsets, as schematically illustrated in FIG. 5 .
  • the following procedure may be used to extract the data conveyed by the watermarks.
  • Each peak conveys 2 bits of data, and the set of three conveys 2 bits of data.
  • a particular advantage is that a soft decision threshold can be used, when detection falls below the threshold.
  • the threshold can be varied until exactly 3 peaks are detected, with one in each subset, the fact that each mark is in a different subset serving as a check (if the three marks giving the highest peaks are not in different subsets, an error can be assumed),
  • the following method may be employed to scatter the payload data across the picture.
  • the payload for example 64 bits having values +1 or ⁇ 1, may be configured as a block of 8 ⁇ 8 bits (as shown in FIG. 6 ), which may then be tiled m ⁇ n times to form a data array as in FIG. 7 .
  • Bi is one bit in this 8 ⁇ 8 block.
  • each occurrence of payload data Bi is convolved with a pseudo-random sequence of length m ⁇ n corresponding to a predetermined key. This is done for each of the 8 ⁇ 8 payload data bits. Each resulting value in this (8 ⁇ m) ⁇ (8 ⁇ n) array is then used as a watermark.
  • Convolution of the data is most preferably performed by a multiplication (considering the data to be signed + or ⁇ ) or equivalently an XOR [or XNOR] operation (considering the input data to be unsigned).
  • the watermark is ideally a zero mean, as indicated above, so if the watermark is generated from a logical XOR, the binary values 0 and 1 would in fact be applied to the picture as bipolar values, with 1 corresponding to +1 and 0 corresponding to ⁇ 1 (or vice versa). It will be noted that the absolute amounts to be added to or subtracted from each pixel value may in fact vary from pixel to pixel, based on a local estimate of visibility, as described above; however, whilst this is highly preferred, the method of generating a watermark which carries data can be used in conjunction with another method of embedding the watermark.
  • Each bit of the resulting watermark is applied to a block of pixels in a picture (ideally 4 ⁇ 4, which is found to give optimum results in terms of enabling each data bit to be reliably detected and allowing a large key and data payload size, but other block sizes and shapes may be used) such that the array is spread over the whole or part of the extent of the picture, as shown in FIG. 8 .
  • this gives 18 ⁇ 22.5 blocks of 8 ⁇ 8 data bits each spread over a block of 4 ⁇ 4 pixels; this allows 405 bits for the key (the geometric arrangement is not critical, so the half blocks can be split over 2 lines). This arrangement is advantageous as the relatively large key allows reliable detection, but still allows a useful data payload.
  • 16 ⁇ 8 blocks may carry 128 bits with a 202 bit key and 16 ⁇ 6 blocks may carry 256 bits, allowing 101 bits for the key; such a key may still be reliably detected in many cases.
  • the watermark is applied to a signal as it is transmitted, such a key should be detectable at a receiver, for example to enable programme identification information to be decoded reliably.
  • the blocks need not be square, nor even rectangular (though these are most convenient for efficient packing of data, and less vulnerable to corruption of data by resizing operations) or even regular, but any shape which can be applied many times over the picture and interlocking or disjointed, but preferably non-overlapping shapes, may be used.
  • the method of embedding the watermark may be considered in three ways as follows:-
  • the blocks are then assembled to make a single rectangular array in some convenient manner, and each point in that array is expanded to cover, in our example, say 4 ⁇ 4 individual pixels in the picture (for a picture of normal European TV resolution, 720 ⁇ 576).
  • This watermark may then be combined with other watermarks, as discussed below.
  • the picture area is to be considered as groups of say a ⁇ b pixels. In our example, this is 4 ⁇ 4.
  • the data block is replicated n ⁇ m times across the entire picture, such that each element of an 8 ⁇ m by 8 ⁇ n (180 ⁇ 144) array corresponds to a 4 ⁇ 4 cluster of picture pixels. Special arrangements must be made for edge effects if n or m are not integers.
  • Each point in the resulting array is expanded to cover, in our example, 4 ⁇ 4 individual pixels in the picture (for a picture of normal European TV resolution, 720 ⁇ 576).
  • This watermark may then be combined with other watermarks, as discussed below.
  • Each data bit in the 8 ⁇ 8 data block is replicated n ⁇ m times, to create a set of 64 m ⁇ n arrays.
  • the resulting array may have varying number of data points on each line, or may be considered as a set of 64 405-element linear arrays.
  • the 64 arrays are then interleaved (every 8 positions vertically and horizontally, or in some other way) to make a single rectangular array of size 180 ⁇ 144 (in our example), and each point in that array is expanded to cover, in our example, 4 ⁇ 4 individual pixels in the picture (for a picture of normal European TV resolution, 720 ⁇ 576).
  • This watermark may then be combined with other watermarks, as discussed below.
  • the resulting watermark may be applied as a single watermark to the picture, and detected as for other watermarks by cross correlation with the key.
  • a further registration watermark is included, at a known position with respect to the date-carrying watermark. It is desirable, as discussed above that the registration watermark is substantially orthogonal to the data carrying watermark. It will be appreciated that the data carrying watermark varies with the data, and it is not practicable to test every possible data set for orthogonality of the resulting data-carrying watermark with the registration watermark. It is, however, possible to test each (405 bit or whatever size) key segment for degree of correlation with the corresponding (i.e. co-located) segment of the registration watermark to ensure a low correlation for each segment.
  • this may comprise:
  • A) Perform local averaging over each 4 ⁇ 4 (or whatever was used) block of pixels to extract DC prediction values (as described above, see particularly equation 8 above and related description).
  • DC prediction values as described above, see particularly equation 8 above and related description.
  • This feature may be applied independently, in particular to the basic watermarking technique mentioned above.
  • the key values can be re-correlated, using the determined data values, to give a further measure of reliability. If the starting position in the picture was not certain, the process may be repeated for different positions or offsets, to determine the position which gives the maximum correlation.
  • each key (say 405 bits) can be viewed as a key segment of a larger key (64*1405 bits).
  • Using a different key for each bit ensures that the data bits can be distinguished and are not decoded in the wrong sequence due to shifts in the picture, and also avoids repeating patterns in the watermark, thereby reducing detectability.
  • an additional registration watermark is provided, as discussed below, to facilitate registration: once the position of the known, higher dimension watermark is determined accurately, the data can be reliably detected.
  • the detection process can be viewed in three ways, as follows:-
  • the picture frame is processed creating a DC prediction error matrix of dimensions 22.5 ⁇ 8 by 18 ⁇ 8. This is broken down into 405 individual 8 ⁇ 8 blocks, each of which is multiplied by the pseudo-random sequence with which it was originally convolved. Averaging the 405 blocks will produce a single 8 ⁇ 8 correlation matrix, with positive or negative values in each cell of the matrix corresponding to the sign of the original data.
  • the picture frame is processed creating a DC prediction error matrix of dimensions 180 by 144. This is multiplied by the same pseudo-random sequence as that with which it was originally convolved. Averaging every eighth value along every eighth line (405 points) for each position of an 8 ⁇ 8 array will produce an 8 ⁇ 8 correlation matrix, with positive or negative values in each cell of the matrix corresponding to the sign of the original data.
  • the picture frame is processed creating a DC prediction error matrix of dimensions 180 by 144. This is broken down into 64 individual m ⁇ n arrays by the inverse process by which they were assembled into a single array, and each of them is multiplied by the pseudo-random sequence with which it was originally convolved. Averaging the 405 values in each of the 64 blocks will produce 64 correlation values, with positive or negative values corresponding to the sign of the original data.
  • the watermark may be used as one of several (most preferably three) watermarks w k , in the manner described above.
  • the watermark may be combined with one or more fixed watermarks or further data carrying watermarks.
  • the different data carrying watermarks are preferably convolved with substantially orthogonal keys.
  • a single fixed watermark selected from a small group of fixed watermarks.
  • a preferred implementation has one registration watermark, preferably selected from a relatively small set of possible watermarks, the choice of watermark encoding a few bits of information and two data-carrying watermarks, preferably each using a fixed key and encoding typically 64 bits of data each.
  • references to convolving data with a key as used herein are not limited to the multiplying and XOR operations described but apply more generally to combination in which the resulting watermark is a function of both the data and the key, particularly any form of combining wherein the data can be extracted from the resulting watermark using the key.
  • the watermark position is moved at a data change, as discussed below.
  • a key portion of the watermark is changed when the data is changed, for example a key sequence may be stepped through.
  • the key sequence may be generated from a pseudo random sequence, preferably having a seed value, preferably using a Blum, Blum, Shub random number generator.
  • One or more seed(s) for the random number generator may be communicated to a decoder, preferably embedded in a picture by a method disclosed herein and one or more seed or feature of the algorithm may be stored in the decoder so that the decoder may follow the sequence but an unauthorised party not knowing the seed and algorithm cannot easily do so.
  • the invention provides a method of detecting a watermark in a sequence of moving pictures comprising determining an expected position of the watermark and thereafter detecting the watermark based on the expected position, wherein the expected position is re-determined following a shot change or a change in picture content above a threshold,
  • the position of the watermark may be moved, particularly if the watermark is a simple pattern such as a chequerboard or the like.
  • the position in the picture can be determined by correlation with a fixed watermark.
  • the position, or more preferably (this is more rugged), relative movement may itself be used to encode information, for example with up, down, left, right being assigned to 2 bit code pairs, or 3 bits if diagonal movement is encoded.
  • the distance moved may be used to encode further information, although this may be less rugged, particularly if the picture is processed by an effects processor.
  • Each of these features relating to movement may be provided independently or in combination with other features.
  • the watermark is moved, preferably randomly (preferably based on a random noise generator rather than a pseudo random sequence generator) substantially at each shot change, or whenever a measure of accumulated change in picture content exceeds a threshold.
  • the data carried is changed at the same time as the watermark is moved, preferably every shot change. Such changes can assist in reducing detectability by unauthorised persons.
  • a watermark or portions of a watermark may be applied to blocks and then move with the blocks.
  • decoding such a picture will normally require either some information from the original picture or complex image processing algorithms (to identify the objects to which the watermark is attached). This may nonetheless be useful when the picture is being compared to an original version of the picture, for example to detect copying as it will be hard for a pirate to identify and remove the watermark. This feature may be independently provided.
  • the process of averaging many pictures together to produce a single picture has the effect of reducing the variance of the picture luminance if the pictures are time-varying. If the watermark is static the result is that the amplitude of the watermark becomes larger relative to the variance of the picture luminance. If a sufficiently large number of watermarked pictures are averaged, the dominant feature will be the watermark. In practice this might be of the order of 6000 pictures—4 minutes at television rates. This property may be undesirable since it may assist in revealing the ‘secret key’ of the watermark. Another problem that may arise when pictures that contain motion are watermarked is that the static watermark may become more visible when the picture to which it is applied is moving.
  • the subjective effect is that of seeing the picture through a glass screen containing imperfections. This may be undesirable for two reasons, because it represents a reduction in the quality of the signals that are produced and because it gives a clue about the nature of the watermarking process that might be useful to malicious persons, in a similar manner to the problem of averaging mentioned above.
  • the average motion of the picture is determined; this may be achieved by any of a number of known methods, such as phase correlation or block matching, each of which have known relative advantages and disadvantages and the implementation is not limited to any particular method.
  • the strength of the watermark is preferably reduced in areas where the actual motion does not match the average motion.
  • the process of determining whether the average motion is a good estimate of the actual motion in a particular part of the picture can be done in several ways and the embodiment is not limited to any particular method.
  • One example of a more accurate method is to divide the picture into smaller blocks and perform a block-matching or correlation process on pairs of blocks. This kind of method is typically used in MPEG-2 video coders to calculate motion vectors.
  • An example of another method, which is less accurate, but computationally far simpler, is simply to compare luminance levels, pixel-by-pixel, between an actual picture and a picture predicted using a motion estimate. If there is a significant difference, then it is assumed that the motion estimate was not accurate.
  • Detection typically relies on averaging several pictures and account must be taken of the average motion before this. Before the picture sequence is added Together, each picture must preferably be shifted to undo the average motion relative to the first picture. Again, the precise method of determining average motion is not critical but it is very highly desirable that the process by which the average motion estimate is made is the same as in the embedding process; otherwise errors may be introduced. A specific implementation of the above will now be described in yet further detail.
  • a spatially shifted watermark W k+1 is calculated by applying a cyclic shift to W k according to the motion estimate V k
  • the error, E k+1 in the estimated picture is calculated.
  • this may be achieved by applying the same shift to P k to give P ⁇ k+1 , end calculating the difference between pixel luminance values in the estimate P ⁇ k+1 and the actual P k+1 .
  • Another, more advanced, method for calculating the error E k+1 is to compare for each pixel the accuracy of the global shift with the estimate of a ‘true’ motion estimator.
  • the function f is preferably chosen to assist in concealing the watermark in areas where the motion estimate is not accurate and is preferably, but not necessarily, a non-linear function of the error signal.
  • the function should also be readily implementable and one simple example is the use of a linear function (C ⁇
  • the function f may also take into account the occurrence of a shot change or of a still picture displayed over several frame periods; various functions may be chosen depending on the degree to which concealment is required and the computational power available to implement the function.
  • a Corresponding Detection Process is as follows:
  • V k the average motion, from P k to picture P k+1 is measured. As above, this may conveniently be achieved by calculating the cross-correlation function of picture P k with P k+1 , although other methods may be used in both cases.
  • a spatially shifted picture P′ k+2 is calculated by applying a cyclic shift opposite to (v k +v k+1 ) to P k+2
  • the cross-correlation function of the average picture P′′ with the watermark W k is calculated as for a static watermark (it will be noted that this may employ or share some hardware or software with the cross-correlation used to determine motion)
  • the uses of the watermark fall broadly into two categories, one is identification, the other is authentication.
  • Identification associates the signal with descriptive information. In general, the association is made by using the watermark to convey a unique identification number that points to a record in a database holding more information.
  • Authentication establishes the credibility of a signal, To identify a signal the watermark should be difficult to remove, should be robust against distortions and should be difficult to perceive. To authenticate a signal the watermark should be difficult to create (or imitate), should be sensitive to distortions but can be easy to perceive.
  • the system described above is particularly useful for identification, but, as discussed, may be adapted for either application. There are many scenarios in which the identity of the signal is of significance. Some of these are as follows:
  • a broadcaster who creates a television programme usually retains rights over the programme. Limited rights to the programme might be sold to another broadcaster permitting some use of the programme—a defined number of broadcasts in a defined geographical area perhaps. The broadcaster owning the rights would like know that the rights granted are not exceeded. If a watermark is added to the programme before it is sold, monitoring of broadcasts around the world can generate reports of detection of the broadcast of the programme by virtue of detecting the watermark. The record of the detections can be analysed to find out whether the rights granted have been exceeded.
  • Copyright infringements that might be found in this way would include the transmission of a programme more times than were agreed, transmission of the programme in more geographical regions than were agreed, or the theft of the programme (either by recording or retransmission of a legitimate broadcast) and use by someone unauthorised.
  • a watermark can be made specific to an individual copy of the programme. Such a watermark is often called a fingerprint. This can be used to identify the recipient of the programme of which the rights were infringed as well as the originator.
  • a watermark can also be used to control the duplication of material. If a recording system is used that will not record signals in which it detects a particular watermark, then this provides a means of preventing illegitimate copies of the recording being made by that system. A related application would use an authenticating watermark to prevent replay of signals unless the mark were detectable in the signals.
  • the processes that are involved in producing television programmes are often complex and numerous. To keep track of all the programme material during this procedure can be difficult, The descriptive information, called meta-data, associated with the picture or sound signal (such as what it is, where it came from, where it has been used, where it will be used) can easily become dissociated from the signal itself. A simple example would occur when a self-adhesive label carrying a unique identifying programme number falls off the side of a video-cassette. An identifying watermark embedded in the pictures can be used to relate the pictures back to the descriptive information. Data embedded may comprise a SMPTE Unique Material Identifier (UMID) or a subset of the data defined therein or a proprietary programme identifier such as a British Broadcasting Corporation Audio-visual number.
  • UMID SMPTE Unique Material Identifier
  • a proprietary programme identifier such as a British Broadcasting Corporation Audio-visual number.
  • the stage of the production process at which such a watermark is embedded depends of the information tracking requirements.
  • the watermark could be embedded by a camera and could consist of a unique identifier which conveys the time of day and geographical location as well as a serial number corresponding to the camera. This would allow the plethora of constituent video clips that are combined to make a typical television programme to be identified and traced back to their source at any stage of the process, even, given sufficient robustness, after broadcast and re-recording.
  • every piece of equipment, on detecting the watermark could update a database that records the use of all programme material. This can be used to check that contractual agreements relating to rights are not being breached, or to ensure that when material is used anyone entitled to payment for its use is paid.

Abstract

A watermarketing system and method is proposed for still or moving pictures in which a watermark is embedded robustly and simply into DC-values, but without causing unacceptable visible picture degradation. The watermark is secure and can be readily detected, and the method can be used to convey additional data. Also disclosed is a method for watermarking a sequence of pictures in which the visibility of artifacts is reduced. The watermark may carry data and may be used to label a programme.

Description

  • The invention relates to embedding a watermark into either a still picture or a sequence of moving pictures; this may be used to assist in detection of copying or identification of the originator of material or for transmitting hidden data. A watermark is a mark or data sequence that is embedded substantially invisibly into a picture for helping to identify the originator or the intended recipient of the picture or to detect tampering.
  • It has been considered to modulate a watermark onto sample mean values, also known as DC-values, but this has tended to result in unacceptable picture distortion.
  • Yeung and Mintzer [Journal of Electronic Imaging 7(3), 578-591 (July 1998)] proposed a watermarking system in which a watermark is applied to DC coefficients of a JPEG compressed image, This is stated to produce satisfactory results for an RGB image but there was some noticeable degradation (blockiness) observed when the watermark was applied to luminance values. Investigations by the inventor have suggested that such a technique is unsuited to high quality image reproduction or broadcast quality motion video marking as the artifacts tend to become too visible, and one is generally dealing with a luminance signal rather than an RGB signal.
  • Qiao and Nahrstedt [IEEE International Conference on Multimedia Computing and Systems, Austin Tex. USA, 28th June-1 st July 1998] proposed a watermarking technique in which a watermark is applied to DCT coefficients in the transform domain. The technique is suitable for MPEG coded video. This disclosure notes that problems can arise when a watermark is applied to both DC and AC coefficients and suggests that no watermark should be applied to the DC coefficients. A drawback of this method is that the watermark must be applied in the transform domain and this may require transformation of an image, thereby increasing the complexity of the method.
  • For the above reasons, attempts have concentrated largely on spread spectrum techniques or embedding information into frequency bands other than DC values. The watermark is usually modulated onto a low frequency band. This makes the watermark more robust against attacks that are based on low pass signal processing such as data compression or digital filtering. This is in line with the properties of the human visual system. As a consequence, an attack on a low frequency band for corrupting the watermark information runs the risk of introducing visible distortions into the picture. However, for similar reasons, embedding the watermark may present problems; it may either require complex and difficult to reproduce algorithms, may not embed the data reliably, or may introduce visible distortions.
  • A number of patents assigned to Digimarc Corporation having similar disclosures, of which US-A-5832119 is a representative example, disclose a watermarking method in which a periodically repeating picture is filtered to remove low and high frequency components to leave only mid-band components and added to a picture. In addition to the drawbacks identified above for techniques of this nature, a specific drawback with this method which the inventor has identified is that the watermark may become visible with certain picture types, particularly in flat areas of the picture. Due to the complexity of the method and hence processing required, the method is unsuited, in practical terms, to marking of moving video sequences, particularly of broadcast quality.
  • The inventor has appreciated that, whilst attempts to modulate a watermark signal onto DC values have generally been unsuccessful for the reasons mentioned above, the use of DC values might offer a robust and simple watermarking system as compared to conventional techniques, if these difficulties could be overcome.
  • According to a first aspect, the invention provides a method as set out in claim 1.
  • By making use of a plurality of adjustment factors for each watermark value, each of which adjustment factors is a function of a local estimate of visibility of the watermark within the picture and which is a function of the picture sample values (and substantially independent of the watermark values), it has been found that, surprisingly, the watermark information can be reliably embedded in the picture without causing unacceptable distortion to the picture. The embedded watermark value may then change the local mean or DC-values of the subset of pixels in which it is embedded, rendering detection simple and reliable.
  • It will be appreciated that this method may result in a watermarked picture in which watermark information is barely present or even not present at all in certain regions of the picture where the estimate of visibility suggests that the presence of the watermark is likely to cause visible distortions to the picture.
  • Preferably the magnitude of adjustment factors is determined from the picture sample values based on an estimate of visibility, preferably from the local variance. This enables the watermark values to be concealed effectively. There will be a plurality of values calculated for each subset, to take into account picture variation, and there is preferably an independently determined adjustment value for each picture sample, although calculation of neighbouring adjustment factors may involve some overlap to reduce calculation.
  • The sign of the adjustment factors is preferably a function of the watermark values, the watermark values preferably comprising a binary sequence of 0 and 1, being encoded as positive and negative signs respectively, or vice versa. By changing the sign, a robust coding scheme is provided, the magnitude of the change not being critical in detecting the watermark and hence being adjustable to allow the watermark to be kept substantially invisible. Alternatively, the magnitude may be adjusted in steps; this may increase the available data capacity but may increase visibility, reduce robustness or increase complexity.
  • In a development, since measures of visibility may be determined from the watermarked picture which should correspond substantially to the originally determined measures of visibility, it should be possible at a decoder to determine the available room for data and thus to employ dynamic allocation of watermark values or data values by encoding more data (perhaps 2 or even 3 bits) in a region where the visibility estimate suggests that larger adjustments can be tolerated. Similarly, decoding may disregard regions where a visibility estimate suggests that no data will be encoded. Weighted filtering may be employed at a decoder, based on an estimate of visibility.
  • As an alternative to a binary system using positive and negative signs, a three level system in which zero adjustment is employed could be used; this may not be so advantageous for simple watermarking as zero adjustments would not assist in correlation, but could be used for data samples carried within the watermark (as discussed below) to increase the data capacity.
  • The adjustment factors may be combined with picture values by adding (which term is intended to encompass weighted addition or subtraction); this is simple to implement but effective, but more complex combination such as averaging may be employed.
  • Where it is desired for the watermark to be robust, for example to carry data or to make the watermark difficult to delete so that the source can be identified, as indicated above, the picture samples are preferably substantially adjacent. Small translations and distortions will tend to leave at least some of each subset of samples sufficiently unaffected for a determination of the watermark value for that subset. A grid may be defined, preferably substantially rectangular for ease of processing, although hexagonal or other grid shapes may be employed, and the samples in each region of the grid may be assigned To a subset. In this way, when it comes to decoding, a slight misalignment of the grid will normally not prevent the majority of samples within a decoding grid from carrying the correct value and permitting correct decoding,
  • When the picture is to be coded or compressed by an algorithm which partitions the picture into blocks, for example JPEG or MPEG coding, the grid preferably corresponds to blocks or groups of blocks of the coding algorithm. This may enable efficient processing and may also ensure that the watermark is reliably carried (more so than if individual watermark values were assigned to pixels in different blocks).
  • It has been found that if each subset comprises a block of at least about 4 by 4 samples (or 1 6 samples if non-rectangular grouping is used), this provides a much higher degree of robustness against a variety of attacks than a comparative example in which significantly fewer (or only one) picture samples per watermark value are employed. Preferably, blocks of at least about 8 by 8 samples (or similar size, at least about 64 samples if non-rectangular grids) are used, more preferably, at least for broadcast quality images, blocks of at least about 8 by 16 (preferably 8 vertically, 16 samples horizontally).
  • In a preferred development, a restriction condition is applied to the choosing of watermarks from the available watermarks. This may reduce the amount of data that can be carried but may increase robustness or error tolerance or detection. In a preferred embodiment, the watermarks are subdivided into subsets (for example 16 watermarks are sub-divided into 4 subsets of 4 marks) and a restriction condition related to the subsets is employed, for example exactly one (or in certain less preferred cases another predetermined number, for example in the case of larger subsets) watermark is chosen from each subset or each of a predetermined number of subsets (for example 1 mark from each of 3 of 4 subsets of 4 marks). By applying such a restriction, a measure of errors or the reliability of the data can be obtained based on the fact that there should be a given number (preferably one) of watermarks in each subset so if detection yields no marks or a reasonable probability of more than one mark in a subset, it can be assumed that the data is noisy or unreliable.
  • Watermark values may be assigned to substantially the whole of a picture. This may increase the dimension of the watermark and make unauthorised copying and detection more difficult. However, a repeating watermark may be used, or certain portions of the picture may be left blank.
  • The watermark may comprise a substantially static component and a variable component, the static component enabling the watermark to be positively identified, and the variable component carrying additional information, for example one or more of picture (or programme) title, date, author, originator, intended recipient, copying permissions, equipment or recording or coding conditions, user definable data and the like. Looked at another way, data may be carried with the watermark. In the case of a moving picture sequence, a separate hidden data stream may be carried, some of the watermark assisting in alignment and framing and the remainder carrying user data. If the application is such that the source and framing of the sequence can be guaranteed, then no static watermark may be needed for synchronisation, and the whole of the watermark may in fact comprise variable user information.
  • In certain cases, it may be desirable to make the watermark “fragile”, so that processing of the data can be detected readily by measuring degradation of the watermark; this may be used for authentication of original copies. This may be achieved by scattering the picture sample values of each subset over the picture, and by increasing the number of watermark values and decreasing the number of pixels per value, At the extreme, the method may be modified to use only a single picture sample per watermark value, but this will normally require use of the original picture to detect the watermark reliably.
  • The watermark preferably comprises a pseudo random pattern; this makes it harder for an unauthorised person to detect or apply the watermark. However, in certain applications, a logo or regular pattern may be employed; this may simplify identification, for example visually from a difference picture.
  • When the watermark is embedded into a moving sequence, it has been found that, surprisingly, although the watermark may be almost impossible to detect when the sequence is viewed frame by frame or in slow motion, artifacts may become visible when the sequence is viewed at normal speed. Investigation has found that this is due to movement in the picture causing the effectively static grid corresponding to assignment of watermark values to appear in. a similar manner to a dirty window overlaid on the picture. Further investigations have revealed that other known watermarking techniques, when examined closely, are prone to similar problems. The prior art does not address these unforseen problems which are peculiar to moving sequences.
  • Further aspects of the invention provide methods as set out in claims 20, 25 and 26. These may alleviate the problems of watermarking a moving picture and may be employed in conjunction with the first aspect, or independently. Preferred features of these aspects are set out in claims 21 to 30.
  • The watermarked picture may be communicated or stored together with data facilitating identification of the watermark, as set out in claims 31 to 33.
  • The invention further provides a method of testing for the presence of a watermark as set out in claim 34 and the preferred features set out in claims 35 to 44.
  • The invention further provides methods of creating, embedding and detecting data-carrying watermarks, methods of marking moving pictures, applications of marked pictures, computer program products and apparatus for implementing any of the methods described above or below and further aspects and preferred features are set out in the other independent and dependent claims respectively, and may also be found in the following description of a preferred embodiment.
  • An embodiment of the invention will now be described by way of example, with reference to the accompanying drawings in which:-
  • FIG. 1 shows a general outline of a watermarking system;
  • FIG. 2 illustrates partition of a picture into square cells;
  • FIG. 3 schematically illustrates an example of calculation of variance for current sample s n and its neighbouring samples that belong To B n;
  • FIG. 4 illustrates partition of a picture into square cells that carry either a watermark sample or a data bit;
  • FIG. 5 illustrates sub-division of a set of 16 watermarks into 4 subsets of 4;
  • FIG. 6 shows an 8×8 sample block for carrying data;
  • FIG. 7 shows the blocks of FIG. 6 tiled m×n;
  • FIG. 8 shows the tiled blocks of FIG. 7 forming a data watermark spread across a picture.
  • Referring to FIG. 1, the watermark w is embedded into the original signal, resulting in the watermarked signal. The watermarked signal may be changed by friendly attacks that are caused by transmission techniques e.g. data compression or by hostile attacks that deliberately attempt to remove the watermark. Therefore, the signal ŝw instead of Sw feeds the input of the watermark-detector. The detector outputs a binary decision which indicates if a given watermark is present in the input signal or not.
  • In this description restricted watermarking means that the original signal is also needed as an input to the watermark-detector. In this case watermark detection is restricted to users who are in possession of the original signal, Unrestricted watermarking means that the original signal is not needed during detection. Both cases will be addressed.
  • The watermark is a zero-mean (to prevent a change in global mean (average brightness) of the signal after watermarking) white noise random signal that is chosen independently from the original signal and difficult-to-predict for an attacker. The role of the watermark is similar to the role of the secret key in a symmetric crypto-system A pseudo random bit generator can be used for obtaining the antipodal series w(1), . . . w(k), . . . w(K) with
    w 2(k)=1, k=1, . . . K.   (1)
  • In restricted watermarking the seed-number of the bit generator can depend on the value of a hash function that is applied to the samples of the original signal for providing authentication.
  • The original picture s is partitioned into cells. This is shown in FIG. 2 for a square cell shape. There is no restriction on the cell shape and any other tiling of the x/y-plane can be used just as well, for example a hexagonal grid. The cell shape and the cell size are parameters that are kept secret together with the watermark w. Let Ck denote the set of indices that select the samples of the k-th cell. The watermark is modulated onto the sample values as
    s w(n)=s(n)+w(k)·α2(n), Ck.   (2)
  • The magnitude depends on a visibility measure and determines by how much the amplitude of a sample can be changed without creating a visible distortion. For example the local variance can be calculated in a small window of 7×7 samples that is centred at the current sample position, see FIG. 3. Let Bn denote the set of indices that select the samples in the neighbourhood including the current index n.
  • The variance is calculated as var ( n ) = ( 1 B n · l B n s z ( l ) ) - ( 1 B n · l B n s ( l ) ) 2 ( 3 )
  • Averaging over all samples gives the mean value var _ = 1 N · n = 1 N var ( n ) ( 4 )
  • Additionally, the positive modulation index q is introduced for allowing a global control of the energy of the modulated watermark, resulting in the magnitude α 2 ( n ) = ( 2 · var ( n ) + var _ var ( n ) + 2 · var _ 0 if var ( n ) > th flat var ( n ) th flat ( 5 )
  • As a consequence of eq. (5) no watermark information is embedded into flat areas that are detected with a threshold thflat.
  • Eqn. (5) is one example for the calculation of the magnitude α2(n) and more sophisticated models of the human visual system can be applied in combination with the embedding method that is specified in eq. (2).
  • After transmission and possible attacks the received signal {hacek over (s)}w is partitioned into cells of appropriate shape and size corresponding to the embedding procedure. For each cell a sample mean value is calculated, hereafter called DC-value. The DC-value of the k-th cell is calculated as DC ( k ) = 1 C k · n C k s w ( n ) , ( 6 )
  • In restricted watermarking a corresponding DC-value is calculated from the original signal s. DC ori ( k ) = 1 C k · n C k s ( n ) , ( 7 )
    and in unrestricted watermarking a prediction value is calculated from the DC-values specified in eq. (6), DC pred ( k ) = 1 C k · l l = k β l · DC ( l ) ( 8 )
  • The coefficients β1 are the same for each picture and can be calculated by linear regression for minimising the mean squared error between DC(k) and DCpred(k). However, in terms of computational complexity a simpler prediction method is to average the DC-values of the neighbouring cells.
  • In restricted watermarking ΔDC(k)=(DC(k)−DCpred(k)) and in unrestricted watermarking ΔDC(k)=(DC(k)−DCori(k)) is correlated with the watermark, corr = ( 1 K · k = 1 K Δ DC ( k ) · w ( k ) ) - ( 1 K · k = 1 K Δ DC ( k ) ) · ( 1 K · k = 1 K w ( k ) ) 1 - ( 1 K · k = 1 K w ( k ) )
  • The detector decides upon the presence of the watermark in the signal {hacek over (s)}w if the magnitude of the correlation value exceeds a threshold, |corr|≧thdetect. The sign sgn(corr) of the correlation value signals one hidden data bit if the presence of the watermark is detected. As no watermark information is embedded into flat areas the detector has the option to exclude flat areas during the evaluation of eq. (9).
  • In restricted watermarking it is easier to align the sampling grid of the received signal {hacek over (s)}w relative to the sampling grid that was used for embedding the watermark. This can be done by comparing {hacek over (s)}w with the original signal s which also allows compensation for geometric distortions such as scaling or rotation. In unrestricted watermarking the sampling grids can be aligned by a search for maximum correlation among a set of horizontal and vertical offset values that are applied to the sampling grid of {hacek over (s)}w. As every watermark sample is spread over one cell perfect alignment of the sampling grids is not needed for obtaining a good correlation. This property improves significantly the robustness against attacks that re-sample the picture including geometric attacks that introduce an unnoticeable amount of distortion. Additionally, the cell size and the cell shape can be adapted for improving the robustness against specific types of geometric distortions. Rotation, cropping or scaling by a noticeable amount can be handled by hypothesis testing which however results in a computational intensive search for maximum correlation.
  • A common method for increasing the capacity of the hidden data channel is to partition the picture into sub-pictures and to apply the above watermarking method to each sub-picture. Another method for increasing the data capacity shall now be exemplified for a square cell shape, see FIG. 4. The white cells are used for the watermarking method as described above. Firstly, the watermark is detected from the white cells at the receiver. This also allows synchronisation and alignment of the sampling grid. Secondly, one data bit is detected from each dark cell as follows. For the k-th cell the DC-values of eqs. (6)-(8) are calculated. In restricted watermarking, the sign of the difference
    ΔDC (k)=(DC(k)−DCori (k))
    signals the data bit and in unrestricted watermarking the sign of the difference
    ΔDC (k)=(DC (k)−DCpred (k))
    signals the data bit. At the transmitter the data bit is embedded similar to eqs (2)-(5), the watermark sample is replaced with the antipodal data bit in eq. (2). As no information is embedded into flat areas and prediction can fail in local areas of the picture the detector has the option to evaluate the data bit only if
    th ,min<|ΔDC (k)=(DC (k)−DCpred(k))<th ,max
    this can be taken into account during embedding.
  • The above method significantly increases the gross data rate. However, robustness is lost in comparison with the watermark that is carried on the white cells, Therefore, error correcting codes are applied to the data that is carried on the dark cells.
  • We have described above methods of increasing data rate. In addition to the prediction technique already outlined we will describe another way of increasing the payload. Instead of using one pseudo-random bit pattern as the watermark w a plurality of N substantially statistically independent patterns w1, . . . , wN is used, The patterns are preferably statistically independent with the following properties:
    (1) w 2 1 =. . . =w 2 N=1   (1)
    (2) E[w 1 ]=. . . =E[w N]=O, where E is the expectation operator
    (3) E[w k =w n ]=O if k≠n
  • Each payload is represented by a combination of three watermarks. Although there could be fewer (for example 2) or more watermarks combined to increase the data capacity, we have found that, surprisingly, by combining exactly three watermarks from a number (ideally a defined set) of substantially independent watermarks, an optimum result may be achieved in terms of reliability of detection and increase in data payload. Thus, there is a total of N·(N−1)·(N−2) /6 possible combinations (neglecting trivial or redundant combinations in which two or three watermarks are the same or the order of combination is altered [(which cannot be detected using simple combination of watermarks with the picture]). For example, if N=16 there are 560 combinations, and one can carry a payload of 9 information bits, plus reserved combinations for other signalling, Surprisingly, we have found that the payload may be most advantageously increased by selecting a reasonably large number of independent watermarks in this set or “library” and then combining exactly three of these in any one picture (rather than combining larger numbers of watermarks), the maximum size of the library being dependent on the ease with which watermarks can be distinguished, so varying with picture size. In order to ease the notation let us further assume that the actual payload is represented by the three watermarks w1, w2, w3. One could then generate a watermark w, by calculating the average value of w1, w2 and W3.
    (4) w=[w 1 +w 2 +w 3]/3
  • However, this combination has the property that the expectation value of the product of any of the individual watermarks with the combined watermark is ⅓.
    (5) E[w·w 1 ]=E[w·w 2 ]=E[w·w 3]=⅓
  • A more advantageous way of combining w1, w2 and w3 is,
    (6) w=[w 1 +w 2 +w 3 ·w 1 ·w 2 ·w 3]/2
  • The watermark w then has the following properties,
    (7) w 2=1
    (8) E[w]=O
    (9) E[w·w 1 ]=E[w·w 2 ]=E[w·w 3]=½
  • The watermark w is embedded in the DC-values of the picture in the usual way. For detection of the payload one generates the watermark w1
    (10) w′=w 1 +. . . +w N
    and cross-correlates the watermark w′ with the DC-values of the pictures in the usual way. If a correlation-peak is detected, the payload is retrieved by cross-correlating each watermark w1, . . . , wN separately with the DC-values and selecting the three watermarks with the largest correlation peaks. Although, as mentioned above, it is greatly preferred if the patterns are independent and satisfy the above rules, it may be desirable in some cases to use patterns which are not truly independent but have a low level of cross correlation (this may increase the number of patterns that can be used or simplify pattern selection); this may make it harder to detect each component reliably, but this may be useful in certain applications, for example where it is intended that the data embedded should be well concealed or “fragile” (i.e. easily corrupted). The above provides a method of combining three bipolar watermarks to produce a single bipolar watermark with the property that the product of the combined watermark with each of the constituent marks has an expectation value of ½.
  • To increase the reliability of data embedding, the available watermarks may be sub-divided into subsets, To recap, the method described above gave a possible 560 combinations, based on the binomial co-efficient ( 16 3 ) = 16 ! 3 ! · ( 16 - 3 ) = 560
    which, as mentioned, is just over 29.
  • In a modified proposal, the set is portioned into 4 subsets of 4 marks and exactly one mark is chosen from each of 3 subsets, as schematically illustrated in FIG. 5. This gives as a number of possible combinations ( 4 3 ) · ( 4 1 ) 3 = 256
    which is exactly 28 so exactly 8 bits can be carried, with increased robustness, the first binomial coefficient giving the number of ways of choosing 3 subsets from 4 and the second giving the number of ways of choosing 1 from 4 watermarks in a subset.
  • Although the number of bits of information that is conveyed has been reduced, a detection strategy can be used that can give increased confidence in the reliability of any data detected,
  • When correlation detection is used, the presence of a peak, over a certain threshold, in the correlation surface of the watermark and the picture (after pre-processing), indicates that the mark has been detected.
  • The following procedure may be used to extract the data conveyed by the watermarks.
  • 1) Check that exactly three subsets have been used.
      • Cross-correlate the picture with each of the 16 watermarks in turn. If there are more or fewer than three cross-correlation functions with peaks above the detection threshold then no data can be recovered.
  • 2) Inspect the distribution of the three peaks.
      • If there are three peaks, they should each belong to a different subset of 4 of the 16 possible watermarks. If more than one mark in a subset has a cross-correlation peak above the threshold then no data can be recovered.
  • 3) Each peak conveys 2 bits of data, and the set of three conveys 2 bits of data.
  • A particular advantage is that a soft decision threshold can be used, when detection falls below the threshold. For example, the three highest peaks that are derived from different subsets could be used. That is, the threshold can be varied until exactly 3 peaks are detected, with one in each subset, the fact that each mark is in a different subset serving as a check (if the three marks giving the highest peaks are not in different subsets, an error can be assumed),
  • Higher Data Rate Watermarking
  • A variety of methods with differing data capacities have been discussed. A further embodiment which may allow the amount of data carried to be increased will now be described, with reference to FIGS. 6-8.
  • To convey a data payload within a video watermark, such as the system described previously, the following method may be employed to scatter the payload data across the picture.
  • The payload, for example 64 bits having values +1 or −1, may be configured as a block of 8×8 bits (as shown in FIG. 6), which may then be tiled m×n times to form a data array as in FIG. 7. Bi is one bit in this 8×8 block.
  • In the watermark encoder each occurrence of payload data Bi is convolved with a pseudo-random sequence of length m×n corresponding to a predetermined key. This is done for each of the 8×8 payload data bits. Each resulting value in this (8×m)×(8×n) array is then used as a watermark. Of course, where different payload sizes and shapes and different tiling patterns are used, the bits of the payload will be convolved with the bits of the key in an appropriate fashion. Convolution of the data is most preferably performed by a multiplication (considering the data to be signed + or −) or equivalently an XOR [or XNOR] operation (considering the input data to be unsigned). The process may be explained as follows:-
    Bipolar multiplication Unipolar XOR
    Data Key Watermark Data Key Watermark
    +1 +1 +1 1 1 0
    +1 −1 −1 1 0 1
    −1 +1 −1 0 1 1
    −1 −1 +1 0 0 0
  • The watermark is ideally a zero mean, as indicated above, so if the watermark is generated from a logical XOR, the binary values 0 and 1 would in fact be applied to the picture as bipolar values, with 1 corresponding to +1 and 0 corresponding to −1 (or vice versa). It will be noted that the absolute amounts to be added to or subtracted from each pixel value may in fact vary from pixel to pixel, based on a local estimate of visibility, as described above; however, whilst this is highly preferred, the method of generating a watermark which carries data can be used in conjunction with another method of embedding the watermark.
  • Each bit of the resulting watermark is applied to a block of pixels in a picture (ideally 4×4, which is found to give optimum results in terms of enabling each data bit to be reliably detected and allowing a large key and data payload size, but other block sizes and shapes may be used) such that the array is spread over the whole or part of the extent of the picture, as shown in FIG. 8. In a most preferred example, for a television picture of 576 lines by 720 pixels, this gives 18×22.5 blocks of 8×8 data bits each spread over a block of 4×4 pixels; this allows 405 bits for the key (the geometric arrangement is not critical, so the half blocks can be split over 2 lines). This arrangement is advantageous as the relatively large key allows reliable detection, but still allows a useful data payload. Of course, other payload sizes and configurations may be used, for example 16×8 blocks may carry 128 bits with a 202 bit key and 16×6 blocks may carry 256 bits, allowing 101 bits for the key; such a key may still be reliably detected in many cases. For example if the watermark is applied to a signal as it is transmitted, such a key should be detectable at a receiver, for example to enable programme identification information to be decoded reliably. The blocks need not be square, nor even rectangular (though these are most convenient for efficient packing of data, and less vulnerable to corruption of data by resizing operations) or even regular, but any shape which can be applied many times over the picture and interlocking or disjointed, but preferably non-overlapping shapes, may be used. To summarise, the method of embedding the watermark may be considered in three ways as follows:-
  • 1) Block by Block
  • The data block is replicated n×m times, so in our example we can consider a set of 405 8×8 data blocks all containing identical data. 405 individual pseudo-random sequences of length 8×8=64 (having values +1 and −1) are then convolved with the data in each block.
  • The blocks are then assembled to make a single rectangular array in some convenient manner, and each point in that array is expanded to cover, in our example, say 4×4 individual pixels in the picture (for a picture of normal European TV resolution, 720×576). This watermark may then be combined with other watermarks, as discussed below.
  • 2) Whole Picture
  • The picture area is to be considered as groups of say a×b pixels. In our example, this is 4×4. The data block is replicated n×m times across the entire picture, such that each element of an 8×m by 8×n (180×144) array corresponds to a 4×4 cluster of picture pixels. Special arrangements must be made for edge effects if n or m are not integers. In our example a single pseudo-random sequence of length 8×8×22.5×18=25920 (having values +1 and −1) is then convolved with the 180×144 array.
  • Each point in the resulting array is expanded to cover, in our example, 4×4 individual pixels in the picture (for a picture of normal European TV resolution, 720×576). This watermark may then be combined with other watermarks, as discussed below.
  • 3)Data Bit by Data Bit
  • Each data bit in the 8×8 data block is replicated n×m times, to create a set of 64 m×n arrays. To take account of the non-integer value of m or n, the resulting array may have varying number of data points on each line, or may be considered as a set of 64 405-element linear arrays. 64 individual pseudo-random sequences of length 22.5×18=405 (having values +1 and −1) are then convolved with each array.
  • The 64 arrays are then interleaved (every 8 positions vertically and horizontally, or in some other way) to make a single rectangular array of size 180×144 (in our example), and each point in that array is expanded to cover, in our example, 4×4 individual pixels in the picture (for a picture of normal European TV resolution, 720×576). This watermark may then be combined with other watermarks, as discussed below.
  • The resulting watermark may be applied as a single watermark to the picture, and detected as for other watermarks by cross correlation with the key.
  • It will be appreciated that the three examples mentioned above may be applied to different watermark and data payload sizes.
  • Most preferably, using the principles discussed above, and, as discussed further below, a further registration watermark is included, at a known position with respect to the date-carrying watermark. It is desirable, as discussed above that the registration watermark is substantially orthogonal to the data carrying watermark. It will be appreciated that the data carrying watermark varies with the data, and it is not practicable to test every possible data set for orthogonality of the resulting data-carrying watermark with the registration watermark. It is, however, possible to test each (405 bit or whatever size) key segment for degree of correlation with the corresponding (i.e. co-located) segment of the registration watermark to ensure a low correlation for each segment. Since the data merely affects the sign of each segment, if the correlation is low, it will remain low whatever the data. Thus, in the ideal case where the correlation between each key segment and each corresponding watermark segment is exactly zero, this will remain true whatever the data, and the sum of the correlations, being the correlation between the complete data-carrying watermark and registration watermark, will also be zero. It is important to note that simply correlating a complete data carrying watermark (for example with dummy date all 1 s) with the complete registration watermark will not give a reliable test as some segments may cancel by chance, but would not cancel if the data were different.
  • Turning to the method of detection, this may comprise:-
  • A) Perform local averaging over each 4×4 (or whatever was used) block of pixels to extract DC prediction values (as described above, see particularly equation 8 above and related description). In a preferred implementation, we have found that performing 2×2 averaging, determining DC prediction values for each 2×2 block, and then forming a single average DC prediction value for each 4×4 block may yield better results. This feature may be applied independently, in particular to the basic watermarking technique mentioned above.
  • B) For each data bit position within each 8×8 block (or whatever shape or size was used), determine the corresponding n×m block (or whatever shape was used) or sequence of key bits to correlate with the predetermined key (or one of a number of predetermined keys see below). A strong positive or negative correlation gives a Positive or negative value for the data bit accordingly (or vice versa) and failure to correlate above a threshold indicates that the picture may be corrupted, or the data unreliable.
  • As an optional check, the key values can be re-correlated, using the determined data values, to give a further measure of reliability. If the starting position in the picture was not certain, the process may be repeated for different positions or offsets, to determine the position which gives the maximum correlation.
  • The key used most preferably varies for each data bit. Looked at another way, each key (say 405 bits) can be viewed as a key segment of a larger key (64*1405 bits). Using a different key for each bit ensures that the data bits can be distinguished and are not decoded in the wrong sequence due to shifts in the picture, and also avoids repeating patterns in the watermark, thereby reducing detectability. On the other hand, in the event that the data is shifted, the data will then become undetectable without searching for a registration point. In a preferred implementation, an additional registration watermark is provided, as discussed below, to facilitate registration: once the position of the known, higher dimension watermark is determined accurately, the data can be reliably detected. As with the embedding process, the detection process can be viewed in three ways, as follows:-
  • 1) Block by Block
  • In the detector the picture frame is processed creating a DC prediction error matrix of dimensions 22.5×8 by 18×8. This is broken down into 405 individual 8×8 blocks, each of which is multiplied by the pseudo-random sequence with which it was originally convolved. Averaging the 405 blocks will produce a single 8×8 correlation matrix, with positive or negative values in each cell of the matrix corresponding to the sign of the original data.
  • 2)Whole picture
  • In the detector the picture frame is processed creating a DC prediction error matrix of dimensions 180 by 144. This is multiplied by the same pseudo-random sequence as that with which it was originally convolved. Averaging every eighth value along every eighth line (405 points) for each position of an 8×8 array will produce an 8×8 correlation matrix, with positive or negative values in each cell of the matrix corresponding to the sign of the original data.
  • 3)Data Bit by Data Bit
  • In the detector the picture frame is processed creating a DC prediction error matrix of dimensions 180 by 144. This is broken down into 64 individual m×n arrays by the inverse process by which they were assembled into a single array, and each of them is multiplied by the pseudo-random sequence with which it was originally convolved. Averaging the 405 values in each of the 64 blocks will produce 64 correlation values, with positive or negative values corresponding to the sign of the original data.
  • It will be seen that all three embedding and detection methods are equivalent and have equivalent effects on a picture, but may differ in the way in which they are implemented in hardware or software (all aspects and features of the invention may be implemented in either or a combination of both), for example in the loop structure of a software implementation or the processing layout of a hardware implementation.
  • To improve accuracy and reliability of detection, it is useful to include a further watermark, the content of which is fixed, or which has a larger fixed (key) content and smaller data content, for example the watermark may be used as one of several (most preferably three) watermarks wk, in the manner described above. As stated, the watermark may be combined with one or more fixed watermarks or further data carrying watermarks. In the latter case, the different data carrying watermarks are preferably convolved with substantially orthogonal keys. In one example, a single fixed watermark, selected from a small group of fixed watermarks. Thus, applying the principles described above for encoding data by including exactly three from sixteen watermarks, ideally subdivided into four subsets of four watermarks (or whatever other numbers are chosen), in this case, the keys rather than complete watermarks are chosen as indicated above and the watermarks themselves are generated by convolving the keys with further data. Thus, choosing 3 from 4 subsets of 4 keys allows 8 bits to be encoded, applying the principles mentioned above. In addition, each key is convolved with an 8×8 block of 64 bits of data, giving a total of 3*64+8=200 bits. It will be appreciated that certain information will be carried more robustly than other information, and the principle of carrying information with differing degrees of robustness may be independently provided, using other methods for encoding the data. A preferred implementation has one registration watermark, preferably selected from a relatively small set of possible watermarks, the choice of watermark encoding a few bits of information and two data-carrying watermarks, preferably each using a fixed key and encoding typically 64 bits of data each.
  • References to convolving data with a key as used herein are not limited to the multiplying and XOR operations described but apply more generally to combination in which the resulting watermark is a function of both the data and the key, particularly any form of combining wherein the data can be extracted from the resulting watermark using the key.
  • The above described methods for carrying hidden data and watermarking can be applied to both still and moving pictures. In the latter case motion imposes an additional problem in terms of attacks and visibility of the watermark, and the spatial model of eqs. (3)-(5) may have to be enhanced to a spatio-temporal model, resulting in motion-compensating embedding and detection. A simpler method is to embed the same static watermark only in every n-th (for example every second or third) picture of a moving sequence and/or to alternate between different static watermarks and/or grid patterns.
  • It is preferable in the case of a moving picture to change the data carried by the watermark only at certain prespecified points, preferably when a shot change is detected or when an accumulated change in picture content exceeds a threshold. Preferably, the watermark position is moved at a data change, as discussed below. Preferably a key portion of the watermark is changed when the data is changed, for example a key sequence may be stepped through. The key sequence may be generated from a pseudo random sequence, preferably having a seed value, preferably using a Blum, Blum, Shub random number generator. One or more seed(s) for the random number generator may be communicated to a decoder, preferably embedded in a picture by a method disclosed herein and one or more seed or feature of the algorithm may be stored in the decoder so that the decoder may follow the sequence but an unauthorised party not knowing the seed and algorithm cannot easily do so.
  • To detect such a watermark, it is necessary to search for the position of the watermark at each shot change. In a detection method which may be provided independently, the invention provides a method of detecting a watermark in a sequence of moving pictures comprising determining an expected position of the watermark and thereafter detecting the watermark based on the expected position, wherein the expected position is re-determined following a shot change or a change in picture content above a threshold,
  • To reduce the visibility of a watermark in a sequence of moving pictures, it may be desirable to invert the sign of the watermark between pictures each time it is embedded, or according to a predetermined or pseudo-random sequence; in this way the mark will tend to average to zero (not exactly due to picture content modulation) and so will be less visible. This feature may be provided independently or in combination with other features.
  • The position of the watermark may be moved, particularly if the watermark is a simple pattern such as a chequerboard or the like. The position in the picture can be determined by correlation with a fixed watermark. The position, or more preferably (this is more rugged), relative movement may itself be used to encode information, for example with up, down, left, right being assigned to 2 bit code pairs, or 3 bits if diagonal movement is encoded. The distance moved may be used to encode further information, although this may be less rugged, particularly if the picture is processed by an effects processor. Each of these features relating to movement may be provided independently or in combination with other features.
  • In one preferred implementation, which may be provided independently, the watermark is moved, preferably randomly (preferably based on a random noise generator rather than a pseudo random sequence generator) substantially at each shot change, or whenever a measure of accumulated change in picture content exceeds a threshold. In the case of a data carrying watermark, preferably the data carried is changed at the same time as the watermark is moved, preferably every shot change. Such changes can assist in reducing detectability by unauthorised persons.
  • To hide the watermark better, it may be “attached” to a moving object and may move with the object. For example, using algorithms similar to MPEG-2 algorithms to assign motion vectors to blocks (indeed the same coding when the picture itself is to be MPEG coded), a watermark or portions of a watermark may be applied to blocks and then move with the blocks. A drawback, however, is that decoding such a picture will normally require either some information from the original picture or complex image processing algorithms (to identify the objects to which the watermark is attached). This may nonetheless be useful when the picture is being compared to an original version of the picture, for example to detect copying as it will be hard for a pirate to identify and remove the watermark. This feature may be independently provided.
  • To explain the watermarking of a moving sequence in more detail, the process of averaging many pictures together to produce a single picture has the effect of reducing the variance of the picture luminance if the pictures are time-varying. If the watermark is static the result is that the amplitude of the watermark becomes larger relative to the variance of the picture luminance. If a sufficiently large number of watermarked pictures are averaged, the dominant feature will be the watermark. In practice this might be of the order of 6000 pictures—4 minutes at television rates. This property may be undesirable since it may assist in revealing the ‘secret key’ of the watermark. Another problem that may arise when pictures that contain motion are watermarked is that the static watermark may become more visible when the picture to which it is applied is moving. The subjective effect is that of seeing the picture through a glass screen containing imperfections. This may be undesirable for two reasons, because it represents a reduction in the quality of the signals that are produced and because it gives a clue about the nature of the watermarking process that might be useful to malicious persons, in a similar manner to the problem of averaging mentioned above.
  • Above, motion-compensated embedding is proposed; a motion-compensation method will now be described in more detail in which the static watermark pattern is moved to follow the average motion of the contents of the picture, The features to be described, as well as the general feature of the watermark following motion in a picture, may be employed independently of other features and the features are independent of the precise implementation method used.
  • To implement this method, as a first step, the average motion of the picture is determined; this may be achieved by any of a number of known methods, such as phase correlation or block matching, each of which have known relative advantages and disadvantages and the implementation is not limited to any particular method. The strength of the watermark is preferably reduced in areas where the actual motion does not match the average motion.
  • As will be appreciated, the process of determining whether the average motion is a good estimate of the actual motion in a particular part of the picture can be done in several ways and the embodiment is not limited to any particular method. One example of a more accurate method is to divide the picture into smaller blocks and perform a block-matching or correlation process on pairs of blocks. This kind of method is typically used in MPEG-2 video coders to calculate motion vectors. An example of another method, which is less accurate, but computationally far simpler, is simply to compare luminance levels, pixel-by-pixel, between an actual picture and a picture predicted using a motion estimate. If there is a significant difference, then it is assumed that the motion estimate was not accurate.
  • Detection typically relies on averaging several pictures and account must be taken of the average motion before this. Before the picture sequence is added Together, each picture must preferably be shifted to undo the average motion relative to the first picture. Again, the precise method of determining average motion is not critical but it is very highly desirable that the process by which the average motion estimate is made is the same as in the embedding process; otherwise errors may be introduced. A specific implementation of the above will now be described in yet further detail.
  • An Exemplary Embedding Process is as follows:
  • 1) a watermark Wk is embedded in picture Pk
  • 2) the average motion, vk, from Pk to picture Pk+1 is measured. This may conveniently be achieved by calculating the cross-correlation function of picture Pk and Pk+1; this has the benefit of being relatively simple to implement and readily applicable in the step of detection of the watermark, Another benefit is that the apparatus may already include hardware accelerators (or optimised code) for calculating cross correlation of at least a portion of a picture, for other reasons.
  • 3) a spatially shifted watermark Wk+1, is calculated by applying a cyclic shift to Wk according to the motion estimate Vk
  • 4) the error, Ek+1 in the estimated picture is calculated. One way this may be achieved by applying the same shift to Pk to give P k+1, end calculating the difference between pixel luminance values in the estimate P k+1 and the actual Pk+1. Another, more advanced, method for calculating the error Ek+1 is to compare for each pixel the accuracy of the global shift with the estimate of a ‘true’ motion estimator.
  • 5) the watermark Wk+1 is then modulated according to the error signal Ek+1 to generate W′k+1=Wk+1·f(Ek+1).
  • The function f is preferably chosen to assist in concealing the watermark in areas where the motion estimate is not accurate and is preferably, but not necessarily, a non-linear function of the error signal. However, the function should also be readily implementable and one simple example is the use of a linear function (C−|Ek+1|) where C is a constant equal to the possible range of the error E. The function f may also take into account the occurrence of a shot change or of a still picture displayed over several frame periods; various functions may be chosen depending on the degree to which concealment is required and the computational power available to implement the function.
  • 6) the modulated, shifted watermark W′k+1 is embedded into the picture Pk+1 substantially as described in detail above for static pictures. This process is then repeated starting at step 2).
  • A Corresponding Detection Process is as follows:
  • 1) the average motion, Vk, from Pk to picture Pk+1 is measured. As above, this may conveniently be achieved by calculating the cross-correlation function of picture Pk with Pk+1, although other methods may be used in both cases.
  • 2) a spatially shifted version of Pk+1, P′k+1 is calculated by applying a cyclic shift opposite to vk
  • 3) the spatially shifted picture P′k+1 is added to Pk
  • 4) the average motion, vk+1, from picture Pk+1, to picture Pk+2 is measured by calculating the correlation function of picture Pk+1 with Pk+2
  • 5) a spatially shifted picture P′k+2 is calculated by applying a cyclic shift opposite to (vk+vk+1) to Pk+2
  • 6) the spatially shifted picture P′k+2 is added to the sum of Pk and Pk+1 Steps 4), 5), and 6) are repeated over the number of pictures, N, required for an adequate level of detection to produce a motion compensated average picture P″
    P″=(P k +P′ k+1 +P′ k+2 +. . . +P′ N+k−1)/N
  • 7) the cross-correlation function of the average picture P″ with the watermark Wk is calculated as for a static watermark (it will be noted that this may employ or share some hardware or software with the cross-correlation used to determine motion)
  • The use of motion-compensated watermarking thus makes embedded watermarks difficult to detect by unauthorised individuals and reduces picture degradation.
  • Preferred Applications
  • The uses of the watermark fall broadly into two categories, one is identification, the other is authentication. Identification associates the signal with descriptive information. In general, the association is made by using the watermark to convey a unique identification number that points to a record in a database holding more information. Authentication establishes the credibility of a signal, To identify a signal the watermark should be difficult to remove, should be robust against distortions and should be difficult to perceive. To authenticate a signal the watermark should be difficult to create (or imitate), should be sensitive to distortions but can be easy to perceive. The system described above is particularly useful for identification, but, as discussed, may be adapted for either application. There are many scenarios in which the identity of the signal is of significance. Some of these are as follows:
  • Copyright Protection:
  • Somebody who has created something of artistic worth might wish to be rewarded for the use of their creation. A broadcaster who creates a television programme usually retains rights over the programme. Limited rights to the programme might be sold to another broadcaster permitting some use of the programme—a defined number of broadcasts in a defined geographical area perhaps. The broadcaster owning the rights would like know that the rights granted are not exceeded. If a watermark is added to the programme before it is sold, monitoring of broadcasts around the world can generate reports of detection of the broadcast of the programme by virtue of detecting the watermark. The record of the detections can be analysed to find out whether the rights granted have been exceeded. Copyright infringements that might be found in this way would include the transmission of a programme more times than were agreed, transmission of the programme in more geographical regions than were agreed, or the theft of the programme (either by recording or retransmission of a legitimate broadcast) and use by someone unauthorised. To identify the route by which a programme came to be used illegitimately, a watermark can be made specific to an individual copy of the programme. Such a watermark is often called a fingerprint. This can be used to identify the recipient of the programme of which the rights were infringed as well as the originator.
  • Copy Protection:
  • As a step further on from detection of copyright infringements, a watermark can also be used to control the duplication of material. If a recording system is used that will not record signals in which it detects a particular watermark, then this provides a means of preventing illegitimate copies of the recording being made by that system. A related application would use an authenticating watermark to prevent replay of signals unless the mark were detectable in the signals.
  • Production Meta-Data Tracking:
  • The processes that are involved in producing television programmes are often complex and numerous. To keep track of all the programme material during this procedure can be difficult, The descriptive information, called meta-data, associated with the picture or sound signal (such as what it is, where it came from, where it has been used, where it will be used) can easily become dissociated from the signal itself. A simple example would occur when a self-adhesive label carrying a unique identifying programme number falls off the side of a video-cassette. An identifying watermark embedded in the pictures can be used to relate the pictures back to the descriptive information. Data embedded may comprise a SMPTE Unique Material Identifier (UMID) or a subset of the data defined therein or a proprietary programme identifier such as a British Broadcasting Corporation Audio-visual number. Considerations pursuant to the invention suggest that, for most purposes, a 32 bit number may be adequate. However, 48 bits is preferable and a 64 bit identifier enables maximum flexibility of encoding for a variety of producers, users and purposes. Since it has not hitherto been easy to embed such amounts of data in a picture, prior art considerations have concentrated on packing data into a small identifier, rather than increasing the data payload.
  • The stage of the production process at which such a watermark is embedded depends of the information tracking requirements. Conceivably, the watermark could be embedded by a camera and could consist of a unique identifier which conveys the time of day and geographical location as well as a serial number corresponding to the camera. This would allow the plethora of constituent video clips that are combined to make a typical television programme to be identified and traced back to their source at any stage of the process, even, given sufficient robustness, after broadcast and re-recording.
  • At each stage of the production process, including final broadcast, every piece of equipment, on detecting the watermark, could update a database that records the use of all programme material. This can be used to check that contractual agreements relating to rights are not being breached, or to ensure that when material is used anyone entitled to payment for its use is paid.
  • Each feature disclosed herein may be independently provided, unless otherwise stated. The appended abstract is incorporated herein by reference.

Claims (75)

1. A method of embedding a watermark signal comprising a series of watermark values in a picture signal comprising a series of picture sample values, the method comprising adjusting picture sample values based on watermark values, characterised in that adjusting comprises, for each watermark value:
combining the watermark value with a respective subset of the picture sample values using a plurality of adjustment factors, each adjustment factor being based on a local estimate of the visibility of the watermark at a corresponding picture sample location.
2. A method according to claim 1 wherein the magnitude of each adjustment factor is a function of the picture sample values, preferably based on the localised variance of the picture sample values.
3. A method according to claim 1, wherein the sign of each adjustment factor is a function of the watermark values.
4. A method according to claim 1, wherein combining comprises adding an adjustment factor to each picture sample value.
5. A method according to claim 1, wherein the picture sample locations for each said subset corresponding to a given watermark value are substantially adjacent.
6. A method according to claim 5, wherein a grid dividing the picture into a plurality of regions is defined and wherein each said subset comprises picture samples corresponding to a respective region of the grid.
7. A method according to claim 6, wherein the grid is substantially rectangular.
8. A method according to claim 7, wherein each region corresponds to a block of a coding process, for example MPEG or JPEG compression.
9. A method according to claim 1, wherein watermark values are assigned to substantially all of said regions of the picture.
10. A method according to claim 1, wherein watermark values are assigned to a first group of said regions and sample values of a data sequence are assigned to a second group of said regions.
11. A method according to claim 1, wherein the picture sample locations for each subset are scattered throughout the picture.
12. A method according to claim 1, wherein the watermark signal comprises a pseudo-random sequence.
13. A method according to claim 1, wherein the watermark signal comprises a regular pattern.
14. A method according to claim 1, wherein the watermark signal has a substantially zero mean whereby the global mean of the picture sample values in the picture is substantially unaffected by embedding of the watermark.
15. A method according to claim 1, wherein combining is arranged to change the local mean of the picture sample values included in each said subset, the sign of the change being determined by the corresponding watermark value.
16. A method according to claim 1, wherein each subset contains at least 16 picture sample values.
17. A method according to claim 1 wherein the adjustment factors are a function of a global modulation index variable whereby the energy of the watermark can be controlled.
18. A method according to claim 1 wherein said adjustment factors are assigned a value substantially equal to zero for regions having a measure of variance below a predetermined threshold.
19. A method according to claim 1, for embedding a watermark in a sequence of pictures corresponding to a motion video sequence, wherein the subsets to which watermark values are applied vary from picture to picture.
20. A method of embedding a watermark within a sequence of pictures corresponding to a motion video sequence wherein the watermark is combined with picture sample values characterised in that the method of combining varies from picture to picture to reduce the appearance of static artifacts in the sequence.
21. A method according to claim 20, for embedding a watermark in a sequence of pictures corresponding to a motion video sequence, wherein applying the watermark to the sequence of pictures includes compensating for motion between pictures.
22. A method according to claim 21, wherein motion compensation includes estimating average motion between pictures.
23. A method according to claim 22, wherein applying the watermark includes determining at least a local measure of accuracy of the estimate of average motion.
24. A method according to claim 23, wherein the watermark is applied so as to be reduced in visibility in areas of the picture (in the case of a local measure of accuracy) or in pictures (in the case of a global measure of accuracy) where said measure indicates that said estimate is inaccurate.
25. A method of embedding a watermark within a sequence of pictures corresponding to a motion video sequence, wherein the watermark is combined with picture sample values characterised in that the method of combining includes generating a motion-compensated version of the watermark to reduce the appearance of static artifacts in the sequence.
26. A method of embedding a watermark within a sequence of pictures corresponding to a motion video sequence, wherein the watermark is combined with picture sample values characterised in that the method of combining includes estimating average motion between the pictures and combining the watermark to reduce the appearance of static artifacts in the sequence.
27. A method according to claim 26, wherein a measure of the accuracy of said estimate is determined and the strength of the watermark is varied as a function of said measure.
28. A method according to claim 20, wherein the watermark is embedded in some but not all pictures of the sequence, preferably wherein at most one in two pictures are watermarked.
29. A method according to claim 20, wherein the pattern of picture sample values with which watermark values are combined varies from picture to picture.
30. A method according to claim 29, wherein embedding the watermark includes defining a grid dividing the picture into regions and wherein at least one characteristic of the grid, for example shape, size or alignment, is varied between pictures of the sequence.
31. A method according to claim 1 further comprising communicating or storing the watermarked picture together with information to assist in identifying the watermark.
32. A method according to claim 31 wherein said information comprises a series of local mean values, each mean value corresponding to the local mean of a subset of picture sample values prior to application of a watermark value.
33. A method according to claim 32, wherein the information is compressed, preferably JPEG compressed.
34. A method of testing for the presence of a watermark embedded in a picture signal by a method according to claim 1, comprising receiving the picture signal;
enhancing the watermark content of the received picture signal using local mean picture values;
correlating the picture signal or a processed signal derived therefrom with a watermark signal;
outputting an estimate of the presence of the watermark based on the results of said correlation.
35. (canceled)
36. A method according to claim 34 wherein enhancing, includes filtering based on received reference picture information.
37. A method according to claim 36 wherein the received reference picture information comprises reference local mean values indicative of local mean values of subsets of picture samples in the picture prior to watermarking, or compressed information from which said reference local mean values can be derived.
38. A method according to claim 37 wherein processing includes estimating local mean values indicative of local mean values of subsets of picture samples in the picture prior to watermarking.
39. A method according to claim 37, wherein processing includes subtracting the reference or estimated local mean values from local mean values determined for the received picture signal to give a difference signal.
40. A method according to claim 37, wherein a grid is defined dividing the received signal into regions corresponding to allocation of watermark values, wherein local mean values are determined for each of said regions.
41. A method according to claim 34, further comprising deriving a series of data values from the received picture signal.
42. A method according to claim 39, wherein the data sample values are determined based on the sign of the difference signal in regions corresponding to allocation of data values.
43. A method according to claim 34, wherein said correlating is performed for a plurality of offsets and the offset giving the maximum correlation is determined to give a measure of the position of the watermark within the picture.
44. A method according to claim 34, wherein correlating is performed taking into account possible effects of picture processing operations, for example rotation, scaling, shifting, cropping or re-sampling operations.
45-51. (canceled)
52. A method according to claim 34 for detecting a watermark embedded by a method wherein the watermark is derived from a combination of a number of substantially independent watermarks or wherein a number of substantially independent watermarks are embedded in each picture, wherein an estimate of the presence of each of said number of substantially independent watermarks is produced.
53. A method according to claim 52, wherein said number of substantially independent watermarks comprises a subset selected from a defined set of substantially independent watermarks, wherein the picture is cross correlated with a composite watermark derived from a sum of each of the watermarks of said defined set.
54. A method according to claim 53 wherein, at least in the event of cross correlation with said composite watermark yielding a positive result, the picture is cross correlated with each of the watermarks of said defined set.
55. A method according to claim 54, wherein the watermark is derived from a combination of three watermarks selected from a defined set of substantially independent watermarks, wherein the three watermarks giving the greatest cross-correlation values are identified.
56. A method according to claim 34, including estimating the cumulative average motion in a sequence of pictures.
57. A method according to claim 34, further comprising computing a motion-compensated average picture taking into account the average motion in the pictures.
58. A method according to claim 57, wherein the watermark is cross-correlated with the motion-compensated average picture.
59. A method of detecting a motion-compensated watermark comprising:
estimating the cumulative average motion in a sequence of pictures;
computing an average picture taking into account the average motion in the pictures;
calculating the cross-correlation function of the motion-compensated average picture and the watermark.
60. A watermarked picture, a sequence of pictures, a signal or data storage means containing a picture having a watermark embedded therein by a method according claim 1.
61. Apparatus or a computer program product arranged to perform a method according to claim 1.
62. A method of decoding data in a picture signal comprising determining local mean values of picture samples corresponding to regions of the picture in which data is carried; comparing said local mean values to estimated or reference local mean values for said regions in the absence of the data, and determining a data value from the result of each comparison, wherein preferably the data value is determined from at least the sign of the difference between the determined local mean value and the estimated or reference local mean value.
63. A method of embedding data comprising a series of data values in a picture comprising a series of picture values comprising defining a plurality of subsets of the picture values, one subset for each data value, and adding an adjustment factor to each picture value in each subset, a first component, preferably the magnitude, of each adjustment factor being a function of an estimate of the visibility of embedded data at the picture value location and being variable between the picture values of each subset, a second component, preferably the sign, of the adjustment factor being determined by the data value and being substantially constant for the picture values of each subset.
64. A method of embedding a watermark signal comprising a series of watermark values in a picture signal comprising a series of picture sample values the method comprising adjusting picture sample values based on watermark values including generating the watermark by convolving a key with a repeated data sequence to produce a data-carrying watermark.
65. A method of generating a watermark encoding data to be applied to a picture comprising convolving a key comprising a plurality of bits with a plurality of bits of data.
66. A method according to claim 65, wherein each data bit is convolved with substantially an entire key segment having a predetermined length.
67. A method according to claim 66, wherein a different key segment is convolved with each data bit.
68. A method according to claim 66, wherein each bit of the watermark is applied to a plurality of bits of the picture, preferably a block of at least 4 bits of the picture.
69. A method according to claim 64, wherein a registration watermark is applied to the picture in addition to the data carrying watermark, to facilitate decoding of data.
70. A method of embedding a watermark in a moving picture comprising changing the watermark or moving the watermark, preferably substantially randomly, at a shot change, or following detection of an accumulated change in picture content above a predetermined threshold.
71. A method of embedding a data-carrying watermark in a moving picture comprising changing the data carried by the watermark at a shot change, or following detection of an accumulated change in picture content above a predetermined threshold.
72. A method of embedding a data-carrying watermark in a moving picture comprising moving the watermark when the data content of the watermark changes, preferably at a shot change or following detection of an accumulated change in picture content above a predetermined threshold.
73. A method of labeling a frame of a moving picture signal comprising embedding an identifier of at least 64bits into the picture, by a method according to claim 1.
74. A method according to claim 73, for use in labeling a broadcast signal or signal to be distributed wherein the identifier encodes at least one of: the originator of the material, an authorised recipient and a material identifier.
75. A method according to claim 73, for use in labeling a source material in a studio, preferably implemented in a camera or recording device, wherein the identifier includes at least one of: an identifier of the source of the material, preferably including at least one of an identifier of a camera, recording time, location, conditions, and a user-definable label.
76. A method of decoding a data-carrying watermark embedded in a picture, by a method according to claim 64 comprising:
setting a positional reference, based on the position of a registration watermark;
estimating watermark values at a plurality of picture locations, preferably by determining local average values;
based on the estimated watermark values at locations corresponding to each data bit and a key value corresponding to the location, determining a value for each data bit.
77. A method according to claim 76, wherein watermark values are determined for a plurality of locations for each data bit and wherein the value for each data bit is determined by averaging the product of an estimated watermark value and a key value for each of said plurality of locations.
78. A method of detecting a watermark in a sequence of moving pictures comprising determining an expected position of the watermark and thereafter detecting the watermark based on the expected position, wherein the expected position is re-determined following a shot change or a change in picture content above a threshold.
79. (canceled)
80. A method of embedding data in a picture comprising:
generating a data-carrying watermark having a plurality of watermark values by convolving a set of data comprising a plurality of bits of data with a key comprising a plurality of bits;
applying the watermark to the picture by combining each watermark value with a plurality of picture values based on a local estimate of the visibility of the watermark
81. A method according to claim 80 for embedding a data stream in a sequence of pictures wherein sets of data are generated at intervals from the data stream and each set is embedded in a plurality of pictures.
US11/328,413 1999-03-18 2006-01-10 Watermarking Abandoned US20060133647A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US11/328,413 US20060133647A1 (en) 1999-03-18 2006-01-10 Watermarking

Applications Claiming Priority (10)

Application Number Priority Date Filing Date Title
GBGB9906299.4A GB9906299D0 (en) 1999-03-19 1999-03-18 Watermarking
GB9906299.4 1999-03-18
GBGB9926530.8A GB9926530D0 (en) 1999-03-18 1999-11-09 Watermarking
GB9926530.8 1999-11-09
GB9930353A GB2348071B (en) 1999-03-18 1999-12-22 Watermarking
GB9930353.9 1999-12-22
US09/936,816 US7130443B1 (en) 1999-03-18 2000-03-20 Watermarking
PCT/GB2000/001033 WO2000056058A1 (en) 1999-03-18 2000-03-20 Watermarking
WOPCT/GB00/01033 2000-03-20
US11/328,413 US20060133647A1 (en) 1999-03-18 2006-01-10 Watermarking

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
US09/936,816 Continuation US7130443B1 (en) 1999-03-18 2000-03-20 Watermarking

Publications (1)

Publication Number Publication Date
US20060133647A1 true US20060133647A1 (en) 2006-06-22

Family

ID=27269676

Family Applications (2)

Application Number Title Priority Date Filing Date
US09/936,816 Expired - Fee Related US7130443B1 (en) 1999-03-18 2000-03-20 Watermarking
US11/328,413 Abandoned US20060133647A1 (en) 1999-03-18 2006-01-10 Watermarking

Family Applications Before (1)

Application Number Title Priority Date Filing Date
US09/936,816 Expired - Fee Related US7130443B1 (en) 1999-03-18 2000-03-20 Watermarking

Country Status (9)

Country Link
US (2) US7130443B1 (en)
EP (1) EP1195047B1 (en)
AT (1) ATE270019T1 (en)
AU (1) AU3308400A (en)
CA (1) CA2368136A1 (en)
DE (1) DE60011773T2 (en)
GB (1) GB2351409B (en)
IL (1) IL145469A0 (en)
WO (1) WO2000056058A1 (en)

Cited By (33)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050100671A1 (en) * 2003-11-06 2005-05-12 Yoko Kawada Electronic watermarking apparatus and method
US20050207651A1 (en) * 2004-03-16 2005-09-22 Sony Corporation System and method for efficiently performing a pattern matching procedure
US20080205698A1 (en) * 2007-02-28 2008-08-28 Canon Kabushiki Kaisha Image processing apparatus and image processing method
US20090303126A1 (en) * 2008-06-05 2009-12-10 Atul Jain Multicode aperture transmitter/receiver
US20100254569A1 (en) * 2007-12-13 2010-10-07 Thomson Licensing Method and apparatus for inserting a removable visible watermark in an image and method and apparatus for removing such watermarks
US20110135089A1 (en) * 2008-08-12 2011-06-09 Sk Telecom Co., Ltd. Image encrypting/decrypting system and method
US20120213402A1 (en) * 2009-10-29 2012-08-23 Gauss Research Laboratory, Inc. Digital Watermarking
US20120324506A1 (en) * 2007-09-14 2012-12-20 Yahoo! Inc. Restoring program information for clips of broadcast programs shared online
US20130170695A1 (en) * 2010-08-27 2013-07-04 Fujitsu Limited Digital watermark embedding apparatus, digital watermark embedding method, and digital watermark detection apparatus
WO2013109934A1 (en) 2012-01-20 2013-07-25 Digimarc Corporation Shared secret arrangements and optical data transfer
US20140201772A1 (en) * 2009-05-29 2014-07-17 Zeev Neumeier Systems and methods for addressing a media database using distance associative hashing
US8958408B1 (en) 2008-06-05 2015-02-17 The Boeing Company Coded aperture scanning
US9008315B2 (en) 2012-01-20 2015-04-14 Digimarc Corporation Shared secret arrangements and optical data transfer
CN105144141A (en) * 2013-03-15 2015-12-09 康格尼蒂夫媒体网络公司 Systems and methods for addressing a media database using distance associative hashing
US20150373582A1 (en) * 2014-06-24 2015-12-24 Qualcomm Incorporated Dynamic bandwidth management for load-based equipment in unlicensed spectrum
US9330427B2 (en) 2010-11-04 2016-05-03 Digimarc Corporation Smartphone-based methods and systems
US9838753B2 (en) 2013-12-23 2017-12-05 Inscape Data, Inc. Monitoring individual viewing of television events using tracking pixels and cookies
US9898793B2 (en) 2013-05-08 2018-02-20 Digimarc Corporation Methods and arrangements involving substrate marking
US9906834B2 (en) 2009-05-29 2018-02-27 Inscape Data, Inc. Methods for identifying video segments and displaying contextually targeted content on a connected television
US9955192B2 (en) 2013-12-23 2018-04-24 Inscape Data, Inc. Monitoring individual viewing of television events using tracking pixels and cookies
US10080062B2 (en) 2015-07-16 2018-09-18 Inscape Data, Inc. Optimizing media fingerprint retention to improve system resource utilization
US10116972B2 (en) 2009-05-29 2018-10-30 Inscape Data, Inc. Methods for identifying video segments and displaying option to view from an alternative source and/or on an alternative device
US10169455B2 (en) 2009-05-29 2019-01-01 Inscape Data, Inc. Systems and methods for addressing a media database using distance associative hashing
US10192138B2 (en) 2010-05-27 2019-01-29 Inscape Data, Inc. Systems and methods for reducing data density in large datasets
US10375451B2 (en) 2009-05-29 2019-08-06 Inscape Data, Inc. Detection of common media segments
US10405014B2 (en) 2015-01-30 2019-09-03 Inscape Data, Inc. Methods for identifying video segments and displaying option to view from an alternative source and/or on an alternative device
US10482349B2 (en) 2015-04-17 2019-11-19 Inscape Data, Inc. Systems and methods for reducing data density in large datasets
US10873788B2 (en) 2015-07-16 2020-12-22 Inscape Data, Inc. Detection of common media segments
US10902048B2 (en) 2015-07-16 2021-01-26 Inscape Data, Inc. Prediction of future views of video segments to optimize system resource utilization
US10949458B2 (en) 2009-05-29 2021-03-16 Inscape Data, Inc. System and method for improving work load management in ACR television monitoring system
US10983984B2 (en) 2017-04-06 2021-04-20 Inscape Data, Inc. Systems and methods for improving accuracy of device maps using media viewing data
US11049094B2 (en) 2014-02-11 2021-06-29 Digimarc Corporation Methods and arrangements for device to device communication
US11308144B2 (en) 2015-07-16 2022-04-19 Inscape Data, Inc. Systems and methods for partitioning search indexes for improved efficiency in identifying media segments

Families Citing this family (50)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1215908A3 (en) 2000-12-07 2005-05-04 Sony United Kingdom Limited Apparatus for detecting and recovering embedded data
US7663670B1 (en) * 2001-02-09 2010-02-16 Digital Imaging Systems Gmbh Methods and systems for embedding camera information in images
US7061510B2 (en) * 2001-03-05 2006-06-13 Digimarc Corporation Geo-referencing of aerial imagery using embedded image identifiers and cross-referenced data sets
WO2002091374A1 (en) * 2001-05-08 2002-11-14 Koninklijke Philips Electronics N.V. Watermarking
DE60219789T2 (en) * 2001-06-21 2008-06-12 Koninklijke Philips Electronics N.V. IMPACTS AND DETECTION OF A WATERMARK IN MOTION PICTURES
EP1271876A1 (en) * 2001-06-25 2003-01-02 Koninklijke Philips Electronics N.V. Transmitting device and method of enhanced rendering
US6685567B2 (en) * 2001-08-08 2004-02-03 Igt Process verification
GB2379295A (en) * 2001-08-31 2003-03-05 Sony Uk Ltd A system for distributing audio/video material to a potential buyer
GB2383219A (en) * 2001-12-13 2003-06-18 Sony Uk Ltd Marking material using a two part watermark
GB2383220B (en) * 2001-12-13 2005-11-30 Sony Uk Ltd Data processing apparatus and method
US7567721B2 (en) * 2002-01-22 2009-07-28 Digimarc Corporation Digital watermarking of low bit rate video
JP3754403B2 (en) * 2002-07-26 2006-03-15 株式会社東芝 Digital watermark detection method and apparatus
JP3976183B2 (en) * 2002-08-14 2007-09-12 インターナショナル・ビジネス・マシーンズ・コーポレーション Content receiving apparatus, network system, and program
US7577841B2 (en) * 2002-08-15 2009-08-18 Digimarc Corporation Watermark placement in watermarking of time varying media signals
WO2004034325A1 (en) * 2002-10-09 2004-04-22 Koninklijke Philips Electronics N.V. Localisation of image tampering
JP4023324B2 (en) * 2003-02-04 2007-12-19 株式会社日立製作所 Watermark embedding and image compression unit
CN1853414A (en) * 2003-09-12 2006-10-25 皇家飞利浦电子股份有限公司 Methods and apparatus for tamper detection in watermarking systems
KR100576802B1 (en) * 2003-10-29 2006-05-10 한국전자통신연구원 The Method for embedding and extracting blindly watermarks by using wavelet transform and HVS
GB0403330D0 (en) * 2004-02-14 2004-03-17 Koninkl Philips Electronics Nv Watermark detection
GB0403327D0 (en) * 2004-02-14 2004-03-17 Koninkl Philips Electronics Nv Watermark detection
KR20070032674A (en) * 2004-06-08 2007-03-22 코닌클리케 필립스 일렉트로닉스 엔.브이. Compensation for Watermark Irregularities Caused by Moved Objects
US7281174B1 (en) * 2004-10-01 2007-10-09 Rockwell Collins, Inc. Diversity code combining scheme for turbo coded systems
GB2421133A (en) * 2004-12-09 2006-06-14 Sony Uk Ltd Registering a water marked image by calculating distortion vector estimates
JP4519678B2 (en) * 2005-02-21 2010-08-04 株式会社東芝 Digital watermark detection method and apparatus, digital watermark embedding method and apparatus
JP4118279B2 (en) * 2005-03-11 2008-07-16 株式会社東芝 Digital watermark detection apparatus and method
EP1891626A1 (en) * 2005-06-03 2008-02-27 Koninklijke Philips Electronics N.V. Homomorphic encryption for secure watermarking
US20070021195A1 (en) * 2005-06-24 2007-01-25 Campbell Steven M Gaming system file authentication
RU2008130059A (en) * 2005-12-22 2010-01-27 Конинклейке Филипс Электроникс Н.В. (Nl) EFFECTIVE CREATION OF PROTECTIVE JUDICIAL WATER SIGNS
US8265145B1 (en) * 2006-01-13 2012-09-11 Vbrick Systems, Inc. Management and selection of reference frames for long term prediction in motion estimation
US7840085B2 (en) * 2006-04-06 2010-11-23 Qualcomm Incorporated Electronic video image stabilization
US8766995B2 (en) * 2006-04-26 2014-07-01 Qualcomm Incorporated Graphics system with configurable caches
US8884972B2 (en) 2006-05-25 2014-11-11 Qualcomm Incorporated Graphics processor with arithmetic and elementary function units
US8869147B2 (en) 2006-05-31 2014-10-21 Qualcomm Incorporated Multi-threaded processor with deferred thread output control
WO2007145954A2 (en) * 2006-06-07 2007-12-21 Wms Gaming Inc. Processing metadata in wagering game systems
US8644643B2 (en) * 2006-06-14 2014-02-04 Qualcomm Incorporated Convolution filtering in a graphics processor
EP2036326A1 (en) 2006-06-17 2009-03-18 Thomson Licensing System and method for analyzing and marking a film
US8766996B2 (en) 2006-06-21 2014-07-01 Qualcomm Incorporated Unified virtual addressed register file
US8041145B2 (en) 2006-11-17 2011-10-18 The Invention Science Fund I, Llc Distortion compensated imaging
JP2008258807A (en) * 2007-04-03 2008-10-23 Toshiba Corp Electronic watermark detector, video reproducing device, video duplicating device and electronic watermark detection program
JP5063690B2 (en) * 2007-05-28 2012-10-31 三菱電機株式会社 Digital watermark embedding apparatus and method, and digital watermark detection apparatus and method
US8401331B2 (en) * 2007-12-06 2013-03-19 Alcatel Lucent Video quality analysis using a linear approximation technique
US8059909B2 (en) * 2008-04-29 2011-11-15 Sony Corporation Adaptive generation of irregular spatial sub-sampling for images
US8059908B2 (en) * 2008-04-29 2011-11-15 Sony Corporation Adaptive area of influence filter for irregular spatial sub-sampled images
US8055087B2 (en) * 2008-04-29 2011-11-08 Sony Corporation Sample level variation for spatial sub-sampled images
US9602819B2 (en) * 2011-01-31 2017-03-21 Apple Inc. Display quality in a variable resolution video coder/decoder system
WO2013137613A1 (en) * 2012-03-12 2013-09-19 Samsung Electronics Co., Ltd. Method and apparatus for determining content type of video content
CN104866080B (en) * 2014-02-24 2020-08-18 腾讯科技(深圳)有限公司 Screen content display method and system
US9959586B1 (en) * 2016-12-13 2018-05-01 GoAnimate, Inc. System, method, and computer program for encoding and decoding a unique signature in a video file as a set of watermarks
US10789667B2 (en) * 2017-06-15 2020-09-29 Treatstock Inc. Method and apparatus for digital watermarking of three dimensional object
CN111583087B (en) * 2020-05-06 2023-09-05 杭州小影创新科技股份有限公司 Image water-jet printing method

Citations (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5636292A (en) * 1995-05-08 1997-06-03 Digimarc Corporation Steganography methods employing embedded calibration data
US5710834A (en) * 1995-05-08 1998-01-20 Digimarc Corporation Method and apparatus responsive to a code signal conveyed through a graphic image
US5721788A (en) * 1992-07-31 1998-02-24 Corbis Corporation Method and system for digital image signatures
US5745604A (en) * 1993-11-18 1998-04-28 Digimarc Corporation Identification/authentication system using robust, distributed coding
US5748763A (en) * 1993-11-18 1998-05-05 Digimarc Corporation Image steganography system featuring perceptually adaptive and globally scalable signal embedding
US5748783A (en) * 1995-05-08 1998-05-05 Digimarc Corporation Method and apparatus for robust information coding
US5765152A (en) * 1995-10-13 1998-06-09 Trustees Of Dartmouth College System and method for managing copyrighted electronic media
US5822436A (en) * 1996-04-25 1998-10-13 Digimarc Corporation Photographic products and methods employing embedded information
US5832119A (en) * 1993-11-18 1998-11-03 Digimarc Corporation Methods for controlling systems using control signals embedded in empirical data
US5841978A (en) * 1993-11-18 1998-11-24 Digimarc Corporation Network linking method using steganographically embedded data objects
US5841886A (en) * 1993-11-18 1998-11-24 Digimarc Corporation Security system for photographic identification
US5850481A (en) * 1993-11-18 1998-12-15 Digimarc Corporation Steganographic system
US5862260A (en) * 1993-11-18 1999-01-19 Digimarc Corporation Methods for surveying dissemination of proprietary empirical data
US5862206A (en) * 1996-09-12 1999-01-19 Cabletron Systems, Inc. Method and apparatus for performing raw cell status report frequency mitigation on transmit in a network node
US5960081A (en) * 1997-06-05 1999-09-28 Cray Research, Inc. Embedding a digital signature in a video sequence
US6064764A (en) * 1998-03-30 2000-05-16 Seiko Epson Corporation Fragile watermarks for detecting tampering in images
US6154571A (en) * 1998-06-24 2000-11-28 Nec Research Institute, Inc. Robust digital watermarking
US6185312B1 (en) * 1997-01-28 2001-02-06 Nippon Telegraph And Telephone Corporation Method for embedding and reading watermark-information in digital form, and apparatus thereof
US6209094B1 (en) * 1998-10-14 2001-03-27 Liquid Audio Inc. Robust watermark method and apparatus for digital signals

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5889868A (en) * 1996-07-02 1999-03-30 The Dice Company Optimization methods for the insertion, protection, and detection of digital watermarks in digitized data
US5848155A (en) 1996-09-04 1998-12-08 Nec Research Institute, Inc. Spread spectrum watermark for embedded signalling
CA2227381C (en) * 1997-02-14 2001-05-29 Nec Corporation Image data encoding system and image inputting apparatus
EP0860997B1 (en) * 1997-02-24 2008-12-10 Nec Corporation Digital data encode system
EP0905967A1 (en) 1997-09-26 1999-03-31 Digital Copyright Technologies AG Method for generating digital watermarks and for exchanging data containing digital watermarks
GB9801131D0 (en) 1998-01-20 1998-03-18 Nds Ltd Improvements in or relating to watermarks
JP2003529225A (en) 1998-01-20 2003-09-30 ディジマーク コーポレイション Multiple watermark technology
EP1028585A1 (en) 1999-01-21 2000-08-16 Nec Corporation A method for inserting and detecting electronic watermark data into a digital image and a device for the same
GB2347295A (en) 1999-02-11 2000-08-30 Central Research Lab Ltd Encoding and decoding of watermarks into moving images using edge detection
US6792130B1 (en) 2000-12-13 2004-09-14 Eastman Kodak Company System and method for embedding a watermark signal that contains message data in a digital image

Patent Citations (24)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5721788A (en) * 1992-07-31 1998-02-24 Corbis Corporation Method and system for digital image signatures
US5809160A (en) * 1992-07-31 1998-09-15 Digimarc Corporation Method for encoding auxiliary data within a source signal
US5748763A (en) * 1993-11-18 1998-05-05 Digimarc Corporation Image steganography system featuring perceptually adaptive and globally scalable signal embedding
US5841886A (en) * 1993-11-18 1998-11-24 Digimarc Corporation Security system for photographic identification
US5832119C1 (en) * 1993-11-18 2002-03-05 Digimarc Corp Methods for controlling systems using control signals embedded in empirical data
US5862260A (en) * 1993-11-18 1999-01-19 Digimarc Corporation Methods for surveying dissemination of proprietary empirical data
US5850481A (en) * 1993-11-18 1998-12-15 Digimarc Corporation Steganographic system
US5768426A (en) * 1993-11-18 1998-06-16 Digimarc Corporation Graphics processing system employing embedded code signals
US5745604A (en) * 1993-11-18 1998-04-28 Digimarc Corporation Identification/authentication system using robust, distributed coding
US5850481C1 (en) * 1993-11-18 2002-07-16 Digimarc Corp Steganographic system
US5832119A (en) * 1993-11-18 1998-11-03 Digimarc Corporation Methods for controlling systems using control signals embedded in empirical data
US5841978A (en) * 1993-11-18 1998-11-24 Digimarc Corporation Network linking method using steganographically embedded data objects
US5710834A (en) * 1995-05-08 1998-01-20 Digimarc Corporation Method and apparatus responsive to a code signal conveyed through a graphic image
US5636292A (en) * 1995-05-08 1997-06-03 Digimarc Corporation Steganography methods employing embedded calibration data
US5748783A (en) * 1995-05-08 1998-05-05 Digimarc Corporation Method and apparatus for robust information coding
US5636292C1 (en) * 1995-05-08 2002-06-18 Digimarc Corp Steganography methods employing embedded calibration data
US5765152A (en) * 1995-10-13 1998-06-09 Trustees Of Dartmouth College System and method for managing copyrighted electronic media
US5822436A (en) * 1996-04-25 1998-10-13 Digimarc Corporation Photographic products and methods employing embedded information
US5862206A (en) * 1996-09-12 1999-01-19 Cabletron Systems, Inc. Method and apparatus for performing raw cell status report frequency mitigation on transmit in a network node
US6185312B1 (en) * 1997-01-28 2001-02-06 Nippon Telegraph And Telephone Corporation Method for embedding and reading watermark-information in digital form, and apparatus thereof
US5960081A (en) * 1997-06-05 1999-09-28 Cray Research, Inc. Embedding a digital signature in a video sequence
US6064764A (en) * 1998-03-30 2000-05-16 Seiko Epson Corporation Fragile watermarks for detecting tampering in images
US6154571A (en) * 1998-06-24 2000-11-28 Nec Research Institute, Inc. Robust digital watermarking
US6209094B1 (en) * 1998-10-14 2001-03-27 Liquid Audio Inc. Robust watermark method and apparatus for digital signals

Cited By (60)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050100671A1 (en) * 2003-11-06 2005-05-12 Yoko Kawada Electronic watermarking apparatus and method
US20050207651A1 (en) * 2004-03-16 2005-09-22 Sony Corporation System and method for efficiently performing a pattern matching procedure
US7634139B2 (en) * 2004-03-16 2009-12-15 Sony Corporation System and method for efficiently performing a pattern matching procedure
US20080205698A1 (en) * 2007-02-28 2008-08-28 Canon Kabushiki Kaisha Image processing apparatus and image processing method
US8363883B2 (en) * 2007-02-28 2013-01-29 Canon Kabushiki Kaisha Image processing apparatus and image processing method
US9036717B2 (en) * 2007-09-14 2015-05-19 Yahoo! Inc. Restoring program information for clips of broadcast programs shared online
US20120324506A1 (en) * 2007-09-14 2012-12-20 Yahoo! Inc. Restoring program information for clips of broadcast programs shared online
US20100254569A1 (en) * 2007-12-13 2010-10-07 Thomson Licensing Method and apparatus for inserting a removable visible watermark in an image and method and apparatus for removing such watermarks
US8635161B2 (en) * 2007-12-13 2014-01-21 Thomson Licensing Method and apparatus for inserting a removable visible watermark in an image and method and apparatus for removing such watermarks
US20090303126A1 (en) * 2008-06-05 2009-12-10 Atul Jain Multicode aperture transmitter/receiver
US8958408B1 (en) 2008-06-05 2015-02-17 The Boeing Company Coded aperture scanning
US8509205B2 (en) * 2008-06-05 2013-08-13 The Boeing Company Multicode aperture transmitter/receiver
US20110135089A1 (en) * 2008-08-12 2011-06-09 Sk Telecom Co., Ltd. Image encrypting/decrypting system and method
CN102119533A (en) * 2008-08-12 2011-07-06 Sk电信有限公司 Image encrypting/decrypting system and method
KR101151107B1 (en) 2008-08-12 2012-06-01 에스케이플래닛 주식회사 Moving picture encryption system and method
US8948384B2 (en) 2008-08-12 2015-02-03 Sk Planet Co., Ltd. Image encrypting/decrypting system and method
US9055335B2 (en) * 2009-05-29 2015-06-09 Cognitive Networks, Inc. Systems and methods for addressing a media database using distance associative hashing
US10820048B2 (en) 2009-05-29 2020-10-27 Inscape Data, Inc. Methods for identifying video segments and displaying contextually targeted content on a connected television
US11080331B2 (en) 2009-05-29 2021-08-03 Inscape Data, Inc. Systems and methods for addressing a media database using distance associative hashing
US10169455B2 (en) 2009-05-29 2019-01-01 Inscape Data, Inc. Systems and methods for addressing a media database using distance associative hashing
US10185768B2 (en) 2009-05-29 2019-01-22 Inscape Data, Inc. Systems and methods for addressing a media database using distance associative hashing
US11272248B2 (en) 2009-05-29 2022-03-08 Inscape Data, Inc. Methods for identifying video segments and displaying contextually targeted content on a connected television
US10949458B2 (en) 2009-05-29 2021-03-16 Inscape Data, Inc. System and method for improving work load management in ACR television monitoring system
US9906834B2 (en) 2009-05-29 2018-02-27 Inscape Data, Inc. Methods for identifying video segments and displaying contextually targeted content on a connected television
US10116972B2 (en) 2009-05-29 2018-10-30 Inscape Data, Inc. Methods for identifying video segments and displaying option to view from an alternative source and/or on an alternative device
US20140201772A1 (en) * 2009-05-29 2014-07-17 Zeev Neumeier Systems and methods for addressing a media database using distance associative hashing
US10375451B2 (en) 2009-05-29 2019-08-06 Inscape Data, Inc. Detection of common media segments
US10271098B2 (en) 2009-05-29 2019-04-23 Inscape Data, Inc. Methods for identifying video segments and displaying contextually targeted content on a connected television
US20120213402A1 (en) * 2009-10-29 2012-08-23 Gauss Research Laboratory, Inc. Digital Watermarking
US8934663B2 (en) * 2009-10-29 2015-01-13 Optimark, L.L.C. Digital watermarking
US10192138B2 (en) 2010-05-27 2019-01-29 Inscape Data, Inc. Systems and methods for reducing data density in large datasets
US20130170695A1 (en) * 2010-08-27 2013-07-04 Fujitsu Limited Digital watermark embedding apparatus, digital watermark embedding method, and digital watermark detection apparatus
US8712094B2 (en) * 2010-08-27 2014-04-29 Fujitsu Limited Digital watermark embedding apparatus, digital watermark embedding method, and digital watermark detection apparatus
US9330427B2 (en) 2010-11-04 2016-05-03 Digimarc Corporation Smartphone-based methods and systems
US9847976B2 (en) 2012-01-20 2017-12-19 Digimarc Corporation Shared secret arrangements and optical data transfer
WO2013109934A1 (en) 2012-01-20 2013-07-25 Digimarc Corporation Shared secret arrangements and optical data transfer
US8879735B2 (en) 2012-01-20 2014-11-04 Digimarc Corporation Shared secret arrangements and optical data transfer
US9008315B2 (en) 2012-01-20 2015-04-14 Digimarc Corporation Shared secret arrangements and optical data transfer
CN105144141A (en) * 2013-03-15 2015-12-09 康格尼蒂夫媒体网络公司 Systems and methods for addressing a media database using distance associative hashing
US9898793B2 (en) 2013-05-08 2018-02-20 Digimarc Corporation Methods and arrangements involving substrate marking
US9955192B2 (en) 2013-12-23 2018-04-24 Inscape Data, Inc. Monitoring individual viewing of television events using tracking pixels and cookies
US9838753B2 (en) 2013-12-23 2017-12-05 Inscape Data, Inc. Monitoring individual viewing of television events using tracking pixels and cookies
US10284884B2 (en) 2013-12-23 2019-05-07 Inscape Data, Inc. Monitoring individual viewing of television events using tracking pixels and cookies
US10306274B2 (en) 2013-12-23 2019-05-28 Inscape Data, Inc. Monitoring individual viewing of television events using tracking pixels and cookies
US11039178B2 (en) 2013-12-23 2021-06-15 Inscape Data, Inc. Monitoring individual viewing of television events using tracking pixels and cookies
US11049094B2 (en) 2014-02-11 2021-06-29 Digimarc Corporation Methods and arrangements for device to device communication
US20150373582A1 (en) * 2014-06-24 2015-12-24 Qualcomm Incorporated Dynamic bandwidth management for load-based equipment in unlicensed spectrum
US10080159B2 (en) * 2014-06-24 2018-09-18 Qualcomm Incorporated Dynamic bandwidth management for load-based equipment in unlicensed spectrum
US10945006B2 (en) 2015-01-30 2021-03-09 Inscape Data, Inc. Methods for identifying video segments and displaying option to view from an alternative source and/or on an alternative device
US10405014B2 (en) 2015-01-30 2019-09-03 Inscape Data, Inc. Methods for identifying video segments and displaying option to view from an alternative source and/or on an alternative device
US11711554B2 (en) 2015-01-30 2023-07-25 Inscape Data, Inc. Methods for identifying video segments and displaying option to view from an alternative source and/or on an alternative device
US10482349B2 (en) 2015-04-17 2019-11-19 Inscape Data, Inc. Systems and methods for reducing data density in large datasets
US10674223B2 (en) 2015-07-16 2020-06-02 Inscape Data, Inc. Optimizing media fingerprint retention to improve system resource utilization
US10080062B2 (en) 2015-07-16 2018-09-18 Inscape Data, Inc. Optimizing media fingerprint retention to improve system resource utilization
US10902048B2 (en) 2015-07-16 2021-01-26 Inscape Data, Inc. Prediction of future views of video segments to optimize system resource utilization
US10873788B2 (en) 2015-07-16 2020-12-22 Inscape Data, Inc. Detection of common media segments
US11308144B2 (en) 2015-07-16 2022-04-19 Inscape Data, Inc. Systems and methods for partitioning search indexes for improved efficiency in identifying media segments
US11451877B2 (en) 2015-07-16 2022-09-20 Inscape Data, Inc. Optimizing media fingerprint retention to improve system resource utilization
US11659255B2 (en) 2015-07-16 2023-05-23 Inscape Data, Inc. Detection of common media segments
US10983984B2 (en) 2017-04-06 2021-04-20 Inscape Data, Inc. Systems and methods for improving accuracy of device maps using media viewing data

Also Published As

Publication number Publication date
GB2351409A (en) 2000-12-27
GB2351409B (en) 2003-10-22
EP1195047A1 (en) 2002-04-10
CA2368136A1 (en) 2000-09-21
ATE270019T1 (en) 2004-07-15
EP1195047B1 (en) 2004-06-23
AU3308400A (en) 2000-10-04
US7130443B1 (en) 2006-10-31
DE60011773T2 (en) 2005-07-07
IL145469A0 (en) 2002-06-30
GB0006696D0 (en) 2000-05-10
WO2000056058A1 (en) 2000-09-21
DE60011773D1 (en) 2004-07-29

Similar Documents

Publication Publication Date Title
US7130443B1 (en) Watermarking
Swanson et al. Multiresolution scene-based video watermarking using perceptual models
US6563936B2 (en) Spatio-temporal channel for images employing a watermark and its complement
US7302577B2 (en) Data processing apparatus and method
US6553127B1 (en) Method and apparatus for selective block processing
US6792130B1 (en) System and method for embedding a watermark signal that contains message data in a digital image
US6771795B1 (en) Spatio-temporal channel for image watermarks or data
US7203335B2 (en) System and method for extracting a watermark signal in a digital image sequence
EP2070328B1 (en) Watermark detection method for broadcasting
US20080226125A1 (en) Method of Embedding Data in an Information Signal
Muharemagic et al. Survey of watermarking techniques and applications
US20050259820A1 (en) Temporally distributed watermarking for image sequences
US20030135736A1 (en) Data processing apparatus and method
US7284129B2 (en) Data processing apparatus and method
Goffin et al. Low-cost perceptive digital picture watermarking method
US7263615B2 (en) Apparatus and method for detecting embedded watermarks
US20030123658A1 (en) Data processing apparatus and method
EP1465404A2 (en) Watermarking
GB2386279A (en) Watermarking using a plurality of independent watermarks
GB2348071A (en) Watermarking
Delaigle et al. Low cost watermarking based on a human visual model
Agung Watermarking and Content Protection for Digital Images and Video
Muharemagic et al. Multimedia security: watermarking techniques

Legal Events

Date Code Title Description
STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION