WO2016172585A1

WO2016172585A1 - Efficient and secure watermark screening in media servers

Info

Publication number: WO2016172585A1
Application number: PCT/US2016/028988
Authority: WO
Inventors: Rade Petrovic; Joseph M. Winograd; Jian Zhao; Dean Angelico
Original assignee: Verance Corporation
Priority date: 2015-04-24
Filing date: 2016-04-22
Publication date: 2016-10-27

Abstract

Methods, devices and systems enable watermark detection and content management in a media server. The described technology reduces the cost of implementation of watermark screening components in a media server and provides an efficient architecture that reduces processing loads and makes efficient use of memory resources. The disclosed techniques further thwart attempts that are directed circumvention watermark screening of content in a media server in order to gain unauthorized access to multimedia content.

Description

EFFICIENT AND SECURE WATERMARK SCREENING IN MEDIA SERVERS

CROSS REFERENCE TO RELATED APPLICATIONS

[0001] This patent application claims the benefit of U.S. Provisional Patent

Application No. 62/152,642, filed April 24, 2015. The entire content of the before-mentioned patent application is incorporated by reference as part of the disclosure of tins application.

FIELD OF INVENTION

[0002] The present invention generally relates to the field of content management.

More particularly, the disclosed embodiments relate to improved watermark screening in media servers.

BACKGROUND

[0003] Digital watermarks have been proposed and used for copyright protection of multimedia content, such a multimedia content, or portions thereof, may contain only one type of content, including, but not limited to, a still image, a video sequence or an audio clip, while in other scenarios, the multimedia content, or portions thereof, may contain two or more types of content such as audiovisual content and a wide range of metadata. In a typical watermarking scenario an auxiliary information signal is hidden within a host content in such a way that it is substantially impe ceptible, and at the same time, difficult to remove without damaging the host content. The auxiliary information that is hidden within the host content can then allow content management to be carried out to varying degrees.

[0004] In some scenarios, content management includes, but is not limited to, the management of the use of content in accordance with one or more policies. For example, the auxiliary information may merely convey thai the host content is not allowed to be copied (i.e., a "no copy allowed" watermark). Once extracted and interpreted by a compliant device, copying of the content is prevented. A compliant device can include, but is not limited to, a device that performs content screening to ascertain whether not the use of the content conforms, or is otherwise operates in a manner consistent, with a content use policy. Content use (or the uses of content) can include, but is not limited to, operations involving content such as playback, copy, record, transfer, stream, or other operations.

[0005] Additionally, or alternatively, the embedded auxiliary information can identify the rightful owner(s), and/or author(s) of the content or can provide a code associated with the content or other content identifying information. The auxiliary information can also be used for other applications, such as to monitor the usage of the embedded host content, resolve ownership disputes, keep track of royalties, and the like. SUMMARY OF CERTAIN EMBODIMENTS

[0006] The disclosed embodiments relate to watermark screening in a media server and to techniques that reduce the processing load in a media server by intermittently screening multiple contents. Moreover, by implementing particular content selection, content switching and data management techni ues in the media server, attempts obtaining a media content from the media server without a proper authorization are thwarted.

[0007] For example, one aspect of the disclosed embodiments relates to a method for intermittent watermark screening in a media server that includes providing an indication to a watermark extractor that a plurality of media contents from a plurality of content sources are identified for being output from the media server. The watermark extractor can, for example, be implemented in-part using electronic circuits. The above note method further includes selecting a first media content from of the plurality of media contents, and performing watermark extraction on a segment of the first media content. The method also includes switching to a second media content from the plurality of media contents based on an indication provided by the watermark extractor, and performing watermark extraction on a segment of the second media content. Such a method includes evaluating the extracted watermarks from at least the first media content or the second media content to determine whe ther or not the ex tracted watennarks impose a restriction on outputting one or more of the plurality of media contents, and upon a determination that the extracted watermarks impose a restriction on outputting one or more of the plurality of media contents, selectively controlling output of the plurality of media contents from the media server.

BRIEF DESCRIPTION OF THE DRAWINGS

[0008] FIG. 1 illustrates an exemplary system in which the disclosed embodiments can be implemented.

[0009] FIG. 2 illustrates a set of operations that can be carried out to perform watermark screening at a media server in accordance with an exemplary embodiment.

[0010] FIG. 3 is a block diagram of a media server in accordance with an exemplary embodiment.

[0011] FIG. 4 illustrates a block diagram of an exemplary device that can

accommodate the disclosed embodiments.

DETAILED DESCRIPTION

[0012] In the following description, for purposes of explanation and not limi tation, details and descriptions are set forth in order to provide a thorough understanding of the disclosed embodiments. However, it will be apparent to those skilled in the art that the present invention may be practiced in other embodiments that depart from these details and descriptions.

[0013] Additionally, in the subject description, the word "exemplary" is used to mean serving as an example, instance, or illustration. Any embodiment or design described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments or designs. Rather, use of tlie word exemplary is intended to present concepts in a concrete manner.

[0014] Some media servers are capable of outputting multiple audiovisual contents simultaneously. One or more of those outputs can cany tlie content in compressed format. Also, the outputs can carry multiplexed audio, video and/or data streams. In some cases, the content is also output in encrypted form, where the encryption may have originated outside tlie media server. In these scenarios, if a media server has to perform watermark detection (e.g., for copy protection purposes or in Automatic Content Recognition (ACR) applications), it may be need to do simultaneous decryption, demultiplexing, and/or decompression of each of the outputs, and then perform watermark detection on all of them . These operations require significant processing power and thus limit tlie maximum number of simultaneous content outputs. Implementing multiple watermark extractors (i.e., one extractor for each content source) in the media server is not a preferred solution because it increases the cost of implementation, it can affect system reliability (e.g., due to having to maintain multiple extractor components), and it can increase the security risk to the sy stem. These issues can be further exacerbated when the number of content sources are large.

[0015] Tlie disclosed embodiments relate to techniques that reduce the processing load in media servers by intermittently screening multiple contents, while ensuring that no security loopholes are left behind to be exploited for unauthorized content use. In For example, by allowing a single detector (and single decoder, demultiplexer, and/or decryptor) to handle multiple streams in a secure manner, processing loads are reduced by intermittently screening multiple contents.

[0016] In some embodiments, it is assumed that the content is much longer than the watermark duration, and that the detection of only a subset of embedded watermarks is enough to enable tlie proper copyright enforcement policy or to establish the extent of the content usage (e.g., how long has tlie content been accessed). A typical example of a watermarking system that meets this assumption is Advanced Access Content System (AACS) Adopter Agreement (available at

http://www.aacsla.com/Iicense/AACS Adopter Agrmt 090605.pdf), which is incorporated herein by reference. In this agreement, it is required, for example, that a "Secondary Enforcement Logic^"' in "Short Form Content" enforcement is acti vated if Trusted Source Marks are found in 13 out of 18 sequential 50-second screening intervals. This means that even a "short fonn content" is expected to be at least 15 minutes long, while watermarks need to be detectable on a much smaller scale. Furthermore, AACS enforcement action for

Trusted Source Mark includes muting the audio for at least 15 seconds, and it takes multiple enforcement actions to effectively protect a copyrighted content.

[0017] It should be noted that if a content is output at a rate that corresponds to content rendering rate over a period of time, say one minute, the amount of content output is substantially equal to am ount of content rendered over the same time interval in the normal playback conditions. In this document, the term content rendering rate refers to the rate at which a conieni is rendered to produce a content for presentation at normal or nominal playback speed. Moreover, in this document, the term content screening is used to refer to analysis of a content, or a component or portion thereof, in order to determine whether or not the watermarks that are detected from the content satisfy a particular content management or content use policy. In order to perform content screening, the locations and/or values of watermarks within the content are needed. As such, watermark extraction, which refers to examination of the content to determine the locations and/or values of the embedded watermarks, is a necessary operation that is performed prior to, or simultaneously with, conducting watermark screening. As such, in this document, the term watermark screening is sometimes used to denote content screening operations that are based on the detected watermarks from a host content.

[0018] In some scenarios, a content can be output at rate that is different from its rendering rate. For example, it is known that in content streaming scenarios, content is initially output at rate faster than its rendering rate in order to create a content buffer at the receiving device. Alternately, if a user chooses to activate a fast-forward or a slow-down playback mode, the content output rate can be increased or decreased, accordingly.

Furthermore, if a content is output for the purpose of making a copy (including a "move" operation), the content output rate is independent from the content rendering rate, and mostly depends on the transmission channel throughput.

[0019] FIG. 1 illustrates an exemplary system in which the disclosed embodiments can be implemented. In particular, a Media Server 102 receives can receive different types of content from various sources, such as from a communication network 116 (e.g., Internet, cloud, a settop box, another device, etc.), from local storage 110, or remote storage 112. The media server 102 also includes the proper Input/Output 108 ports, and associated circuits, to recei ve content, control signals, or other types of data, and to communicate control signals, content, metadata, etc. to other devices, such as Output Device 1, Output Device 2, ... Output Device K, Some of the output devices 114 may he able to communicate with the Media Server 102 bi-directionally, while some Output Devices 1 14 may only be capable of unidirectional communication. Communication of content, control information, metadata and other types of data among various devices in FIG. 1 may be carried out via wired or wireless transmission protocols and techniques.

[0020] FIG. 1 also shows some other the components of the Media Server 102, including the Content Manipulation Components 104. Content Manipulation Components 1 4 can carryout content processing operations, including but not limited to, encryption and decryption, multiplexing and demultiplexing, transcoding, formatting, rate conversion, compression and decompression, and others. Watermark Extraction, Content Screening and Control Components 106 (shown as a block, but may be broken up into subcomponents) are responsible for extraction of watermarks, assessment of content use policies, and application of enforcement actions in compliance with content use policies and extracted watermarks.

[0021] In a typical scenario, a user initiates a request to the media server for receiving one or more contents at corresponding output device(s). In such a case, a compliant media server may need to assess copy protection status (or, more generally, content management status) of ail of the requested contents. In some implementations, in order to reduce the processing load on a media server, multiple output contents from the media center are intermittently screened for the presence of watermarks. That is, instead of processing the entire lengths of all outputs, only certain sections of each output can be examined in a predetermined or random sequence. In such implementations, an attacker may try to exploit intermittent screening of a media server outputs by initiating multiple outputs. In this scenario, most of the outputs act as diversion outputs that are not actually used (e.g., not rendered or copied), while one output is illicitly used. For example, the diversion outputs may contain a watermarked content, causing the extractor to process the contents to collect watermarks and implement an "enforcement logic,^" such as the one described above in connection with the Trusted Source Mark. In the mean time, due to the large amount of processing and the presence of multiple contents, the extractor may not be able to perform content screening for the output that is intended for use. Alternatively, diversion outputs may be partially marked and partially unmarked, thus causing the extractors to spend time and processing cycles on diversionary outputs in order to find additional watermarks. [0022] In order to overcome various attacks while maintaining as low processing load as possible, in some embodiments the selection of which content to screen at any given moment is guided by the history of watermark detections and enforcements on various outputs. This can be achieved, for example, by maintaining the history of watermark detections and enforcement events, and making decisions regarding when to switch between contents and what content to choose next. Such a history can, for example, be maintained by the watermark extractors, or by another entity that can make the information available to the extractors.

[0023] In order to implement the extractor-driven output switching, the media server needs to inform, the extractor about activation and deactivation of its outputs. For example the media server can associate each output with a bit in a binary vector. When an output is activated, the associated bit is set to a first value (e.g., a "one") and the binary "output" vector is passed to extractor. If the output is deactivated, the associated bit is set to a second valued (e.g., a "zero") and again passed to the extractor. When the extractor chooses to switch to another content that is output by the media server, it passes an indicator that identifies the next selected content output for screening. For example, it can pass a binary vector to the media server which has a first value (e.g., a "one") in a single position of a binary "selected output" vector, where each position in the selected output vector identifies one of the media server outputs. Each content selection event starts a content screening session; such a session ends when the next content selection event occurs.

[0024] In some embodiments, the media server has a limit on how much content can be output at a rate faster than content rendering rate, without watermark screening. This limit prevents an attacker from outputting the entire content (or a large portion of the content) before any watermark screening takes place. For example, in order to accommodate initial buffering in a content streaming scenario, the media server can allow a content to be output at a high rate for up to an "initial interval," say 60 seconds, measured based on nominal content rendering timeline. This initial interval can be measured relatively coarsely, such as with a precision of +/- 10%. Once the content is output at the faster rate for the initial interval without screening, the content output rate is set to the content rendering rate. This way, the outputs that are used for rendering will not be interrupted by the watermark screening process because the initial interval accommodates the buffering and processing latencies that may be associated with content delivery. However, such a limitation on outputting high content at high rates can slow down other operations such as copying of the content.

[0025] In some embodiments, when a new content output is selected for screening, the media server immediately provides the content to the extractor starting from the currently output content location, or within a small amount of delay , say five seconds on content rendering timeline. While the content is screened by the extractor, the output may be permitted to progress (e.g., be presented to on a display, copied, etc.) beyond the exact location within the content that is being currently screened, but no more than duration of the initial interval described above, or the transfer interval described below .

[0026] In some embodiments, the content is provided to the extractor as sequential content chunks. In such embodiments, the next content chunk is provided to the extractor after the previous chunk has been processed by the watermark extractor. For example, the media server can provide content chunks to the extractor at a predefined rate so that the extractor can measure the elapsed time by counting the number of content chunks that it has received. This arrangement is suitable when all outputs of the media server are running substantially at the same rate, e.g., at content rendering rates. Note that in this case, if there are no content chunks available for screening, the media server can pass empty or "null" chunks to the extractor at the predefined rate, as long as the extractor is still active, thus allowing the extractor to maintain its estimation of the current time on the content rendering timeline. It is important for the extractor to keep track of the current time because many copy control schemes (such as the one associated with the above-described Trusted Source Marks) rely on detection of watermarks in a particular number of time in tervals of predefined lengths.

[0027] Alternatively, the media server can provide content chunks to the extractor at a variable rate, e.g., as soon as the previous content chunk is processed, or as soon as the next chunk becomes available for screening. This arrangement is suitable when some outputs are operating at a high speed (such as in copying operations). Such an arrangement can also enable screening of a larger number of simultaneous outputs compared to the above described fixed-rate arrangement, and/or can allow a more flexible content handling arrangement. However, the media server has to repeatedly inform the extractor about the current time. In one example, the media server passes the current time information with every content chunk that is passes to the extractor. This current time information can have relatively coarse granularity; for example, it can be accurate to within one second, and it can be measured with respect to any starting point in time, e.g., it can be a count of the number of seconds from the start of extractor activation . The source for the current time information can be a time measuring mechanism that has a measurement precision (i.e., maximum error of measured quantity) that is better than +/- 0.1%. For example, a hardware based counter that counts the number of events using a clock signal a real-time clock, or other timer devices, can be used.

[0028] In some embodiments, in order to simplify content handling and access, in particular when content can only be accessed sequentially (and not randomly), a maximum content amount that is screened per content selection session is specified. This way, the media server can use a screening buffer to save the maximum amount of the content that is currently being output. Such a buffering operation can be carried out at one or more locations in the content processing chain. For example, in FIG. 3 (which will be described in further detail below), the screening buffer can be positioned either at the input or at the output of Content Transformation unit 308. Note that a content screening session can end before the entire buffer is screened. For example, with reference to FIG. 3, placement of the buffer at the input of unit 308 can increase the amount of required memory for implementing the buffer, but can save processing cycles, as compared to an implementation in which the buffer is placed at the output of unit 308. In particular, because content transformations may include extracting audio from AV mix, channel count reduction, resampling and filtering (all of which reduce the require memory for storing the transformed content). Further, the buffer contents may be discarded if the extractor finds a watermark before the end of the buffer. However, if the content can be accessed randomly, it may be unnecessary to buffer the content for screening. Instead, the content that was already output can be accessed upon a request from the extractor. For example, the content can be locally saved, e.g., on a solid- state drive or RAM, and access to the content for screening can be conducted independently from playback access. In a random access mode, the extractor can, for example, request a particular section of the content (e.g., from time, ti , to time, t_?. on the media timeline).

However, if assess is only sequential, e.g., from an optical disc, the media player may not be able to access different sections of the content randomly during playback. As such, buffering the content enables intermittent screening operations to continue without compromising the effectiveness of the screening operations.

[0029] In some implementations, watermark screening operations can be executed at a rate that is faster than content output rate from the media server. In such implementations, the media server can provide future sections of the content to the extractor (e.g., content sections beyond those that are being currently rendered). However, if provision of future content sections is not possible (e.g., if content access is limited to sequential access only), the media server can buffer a predefined amount (e.g., 25 seconds of content rendering timeline) of the most recently output content that has not been previously screened, before allowing this output to be screened. In this case, at the end of the screening session (e.g. when extractor sends "transfer interval" information to the media server as defined below) the media server can send a "buffer ready" vector. A particular value (e.g., a "one" value) in both the "output" and "buffer ready" vectors can identify outputs that are selectable for screening. Note that in some embodiments, separate physical memory components may be used to implement the disclosed buffering functionality, while in other embodiments, the same physical buffer unit may be used to fulfill the buffering requirements at different stages of watermark extraction or watermark screening.

[0030] In some embodiments, the extractor keeps screening the selected content for up to predefined content amount (e.g., 25 seconds of content rendering timeline), or until a watermark is extracted from the content, or until the currently screened output becomes inactive, whichever comes first. Then the extractor can close the screening session and make the new content output selection as described below.

[0031] When the media server has random access to the content and no buffering is implemented, then the screening can begin from the currently output content on the selected output, provided that it hasn't been already screened. If the currently output content has been screened, then the screening may begin from the beginning of the unscreened content.

[0032] Alternatively, the screening can commence from the beginning of the buffered content. Whenever the screening session ends, as defined below, the currently screened buffer can be discarded and filled up again starting with the current output.

[0033] In some embodiments, the extractor maintains a history of previous watermark detections. For example, in a scenario that involves the AACS rules for "Secondary Enforcement Logic" for "Short Form Content," a binary history vector with at least 18 bits is needed to keep track of at least 18 screening intervals. Similar history vectors may be needed for other types of watermarks. When a watermark that signifies "Secondary Enforcement Logic" for "Short Form Content" is detected, and the information from the media server indicates that the content is not protected with a trusted protection, the bit in the appropriate history vector is changed to a particular value (e.g., to a "one," assuming that at initialization all of bits are set to "zero"). Then, the extractor checks if the enforcement rule 'Trusted Source Mark is found in 13 out of 18 sequential 50-second screening intervals" is met, and if so, the enforcement action is initiated for the output that is currently screened.

[0034] When the screening of a selected output ends, the extractor returns a "transfer interval" value to the media server. The transfer interval represent the amount of previously- screened content that can be output by the media server at faster than content rendering rate. For example, if the transfer interval value has a particular value (e.g., a zero), this indicates that an enforceable watermark is detected during the screening, and that content output rate must be set to the rendering rate. If no enforceable watermark is detected, the transfer interval will be different from the first value (e.g., is larger that zero and somewhere in the range between one minute and 19 minutes). The media server is then allowed to output the screened content at any rate, but only up to the amount that matches the transfer interval. If the content output reaches the transfer interval boundary without another watermark screening and followed by transfer interval specification , the content output rate is set to the content rendering rate.

[0035] In some embodiments, the extractor maintains only one history vector for "Secondary Enforcement Logic" for "Short Form Content," regardless of the number and activity of media server outputs. This way, the history- vector can be populated by watermark detections that originate from different content outputs, and the enforcement criteria is satisfied based on ail content outputs. This feature is helpful in reducing the processing load on the extractor in cases where multiple marked streams are simultaneously outputted.

Without this feature, the extractor may be overloaded and incapable of screening each content output to collect enough watermarks to meet the enforcement rule for each output.

[0036] In some embodiments, the extractor keeps track of previous detections of enforceable watermarks on each active output. For example, for each active output, the extractor saves a time stamp for the last detected enforceable watermark, as well as the watermark' pay load. If the output is deactivated, the extractor clears the time stamp associated with the output and the detected pay load. If the output is still active, but the user changes the content that is sent to this output, the media server informs the extractor that such a change has occurred, causing the extractor to clear the saved information associated with this output.

[0037] When a screening session ends, the extractor checks the saved enforceable watermarks associated with the screened output. If there is an enforceable watermark detected on this output, then the extractor passes the transfer interval to the media server with a first value (e.g., zero) that indicates the absence of enforceable watermarks. Otherwise the extractor selects a second value (e.g., a random value) for the transfer interval in the range [a, b], and provides it to the media server. In one example related to the Secondary Enforcement Logic" for "Short Form Content," a is selected to be I min, and b is selected to be 19 minutes. This interval can be measured relatively coarsely by the media server, for example, with a precision of +/- 10%.

[0038] In some embodiments, when the content screening session ends, the extractor reviews the saved data before selecting which output is to be screened next. In this process, the extractor can group the selectable outputs according to their priority ranks, as described in more detail below. In the case of a media server with random content access capability, ail active outputs are selectable, while in cases where screening from a buffer is taking place, only those active outputs with a full buffer (e.g., "buffer ready" flag equal to "one") are selectable.

[0039] In some embodiments, the top priority is assigned to one or more outputs that meet ail of the following conditions:

1. An enforceable watermark detection on the output was made within a predefined time interval (e.g., within the last 10 minutes),

2. The detected payload from the output corresponds to the history vector for which the enforcement condition is met, and

3. A previous enforcement action is nonexistent or is more than 15 seconds old.

[0040] The first condition is indicative that it is likely that this output is used for playback that is subject to enforcements. The second condition indicates that every new watermark detection will likely result in enforcements and the third condition signifies that some previous enforcements have expired, and that it would take a new watermark detection to renew the enforcement.

[0041] The second priority group can consist of all outputs that meet conditions I and

2 above, but the previous enforcement action was less than 15 seconds old. The third priority group consists of all outputs for which only the first condition above is met. All other outputs are classified as the fourth priority group. It should be noted that in the above description, 15 seconds is used as an example to facilitate the understanding of the disclosed embodiments. It is understood, however, that such a predetermined interval can be set to other values to conform to the requirements of the watermarking system, and content protection policies.

[0042] After assigning the priority status to all active outputs, the extractor selects an output in a random manner for screening, where the probability of output selection depends on its priority status. For example, the probability of the selection of outputs in the first priority group can be twice the probability for outputs in the second priority group, four times the probability of the third priority group and eight times the pro bability of the fourtli priority group.

[0043] FIG. 2 illustrates a set of operations that can be carried out to perform intermittent watermark screening at a media server in accordance with an exemplary embodiment. At 202, an indication is provided to a watermark extractor that a plurality of media contents from a plurality of content sources are identified for being output from the media server. In some implementations, the watermark extractor is implemented in-part usmg electronic circuits. At 204, a first media content is selected from of the plurality of media contents, and watermark extraction is performed on a segment of the first media content. At 206, based on an indication that is received from the watermark extractor, the selection is switched to a second media content from of the plurality of media contents, and watermark extraction is performed on a segment of the second media content. At 208, the extracted watermarks from at least the first media content or the second media content are evaluated to determine whether or not the extracted watermarks impose a restriction on outputting one or more of the plurality of media contents. At 210, upon a determination that the extracted watermarks impose a restriction on outputting one or more of the plurality of media contents, output of the plurality of media contents is selectively controlled from the media server. The selection of the media contents may continue for one or more additional media contents as shown in operation 212. At 212, a segment of each of the one or more additional media contents is subjected to watermark extraction, and the watermark extraction results are evaluated at 208. Switching to the one or more additional media contents can continue until all of the plurality of media contents have been subjected to watermark extraction.

[0044] In one exemplary embodiment, the watermark extractor is the only watermark extractor that is implemented in the media server. In another exemplary embodiment, selectively controlling output of the plurality of media contents from the media server includes reducing an output rate of at least one of the plurality of media contents. For instance, reducing the output rate of the at least one of the plurality of media contents can include setting the output rate of the at least one of the plurality of media contents to the rendering rate of the at least one of the plurality of media contents. In another exemplary embodiment, the operations for intermittent watermark screening at the media server further include determining the current time relative to a rendering timeline as part of performing watermark extraction on the first or the second media contents. In this exemplary embodiment, determination of whether or not the extracted watermarks impose a restriction on outputting one or more of the plurality of media contents includes using the current time to determine whether or not the extracted watermarks conform to a particular pattern of watermark detections in a predetermined number of time intervals.

[0045] According to one exemplary embodiment, the operations for intermittent watermark screening at the media server also include storing information regarding the extracted watermarks and restrictions placed on one or more of the plurality of media contents, and, upon completion of watermark extraction on the segment of the first media content, analyzing the stored information to determine which of the plurality of media contents is to be selected as the second media content. In another embodiment, the above note operations can further include assigning a priority value to each of the plurality of the media contents based the analyzing of the stored information, and selecting a particular one of the plurality of the media contents that (a) has a highest priority value, and (b) has not been subjected to watermark extraction, as the second media content.

[0046] In yet another exemplary embodiment, the operations for intermittent watermark screening at the media server further include enabling one or more of the plurality of media contents to be output at a rate higher than content rendering rate for an initial interval, regardless of the determination that the extracted watermarks impose a restriction on outputting one or more of the plurality of media contents. In another exemplary embodiment, upon the determination that the extracted watermarks impose a restriction on outputting one or more of the plurality of media contents, the above operation include modifying one or more of the plurality of media contents according to one or more of the following enforcement actions: muting one or more outputs of the media server, stopping one or more outputs of the media server, including a message indicative of the enforcement action at one or more outputs of the media server, or changing a rate at which content is output at one or more outputs of the media server.

[0047] In still another exemplary embodiment, performing watermark extraction on the first or the second media contents stops upon (a) completion of watermark extraction for up to predefined content amount, (b) extraction of a valid watermark, or (c) the media content under watermark extraction is no longer identified for being output from the media server. In one exemplary embodiment, the operations for intermittent watermark screening at the media server further include, prior to switching to the second media content, determining a transfer interval that represents an amount of the first media content that can be output by the media server at faster than content rendering rate, and enabling output of the first media content by the media server at faster than rendering rate of the second media content for the duration of the transfer interval. In another exemplary embodiment, upon a determination that at least one watermark has been extracted from the first content, a zero value is assigned to the transfer interval; whereas, upon a determination that no watermark has been extracted from the first content, a non-zero value within a predetermined range of time values is assigned to the transfer interval.

[0048] According to one exemplary embodiment, performing watermark extraction on the segment of the first media content or on the segment of the second media content results in detection of a watermark value that is used for content identification. In another exemplary embodiment, (a) a determination to select, or (b) an instant in time to switch to, the second content is based on watermark detection results obtained from the first media content, or any one of the plurality of media contents that are designated for watermark extraction.

[0049] FIG. 3 is a block diagram of a media server architecture that can be used for copy control in accordance with an example embodiment. In particular, this example illustrates a media server using watermark extraction for copyright enforcements. The media server 300 (only a portion of which is illustrated in FIG. 3) processes the media from multiple sources 302, such as Source 1, Source 2, ... Source n. The sources may include sources external to the media server 300, such as those from a set top box, or sources internal to the media server, such as from a hard drive, a DVD player, etc. Signals from each of the sources 302 are directed to a switch 304 and to a corresponding control unit 306. The signals that are provided from each source 302 to each control 306 may be un syn chron ized with sign als that are provided from each source 302 to the switch 302. For example, the signals may reach the switch 304 ahead of those that reach from the control units 306, or may lag behind them. But such an offset in timing delivery should be no more than the transfer interval described above. The control units 306, shown in FIG. 3 as Control Unit 1, Control Unit 2, Control Unit n, receive content from Source 1, Source 2, ... Source n, respectively, and also control the speed of the content output. Content signals are sent through the switch 304 to the content transformation unit 308. Content transformations may include one or more of components that implement features such as content decryption, demultiplexing (e.g., separation of audio and video components of the content), decoding (e.g., transformation from perceptually compressed signal into linear pulse code modulated, LPCM, signal), resampling, mixing (e.g., changing the number of audio channels), etc. Note that in some embodiments, watermarks are only embedded in a portion of the content (e.g., only in audio portion). In such embodiments, content transformations can be carried out for only the portion of the content that carries the watermarks. Note also that, in some implementations, content transformations can be performed elsewhere for the purpose of playback. In such implementations, the number and extent of content transformations in unit 308 can be reduced or completely eliminated. [0050] A watermark extractor unit 310 is coupled to the Content Transformation Unit

308, receives the content from the content transformation unit 308, and extracts watermark data. The watermark extractor 310 is further configured to conduct content screening, that is, to assess whether or not the detected watermarks conform to a particular content use or content management policy. Based on the histoiy of content source selections and watermark extractions, the watermark extractor 310 controls the switch 304 to make a new content source selection as described above. Furthermore, the watermark extractor 310 informs the enforcement action unit 312 about the payload and timing of extracted watermarks on a particular content output, and enforcement action unit 312 decides what kind of enforcement action is needed, if any. Based on the instructions from the enforcement action unit 312, the control units 306 modify the content that is sent to the corresponding output units 314, such as Output I, Output I, ... Output n. For example, Output l 's audio can be muted, and a message can be displayed advising the user why the audio is muted. Alternatively , both the audio and video can be stopped, and a message can be displayed explaining to user why such an enforcement action has happened. Furthermore, the watermark extraction unit 310 informs the enforcement action unit 312 about its selection of the transfer interval for a particular source, which can be used to control how much of the content can be allowed to go out without screening before the content output speed is reduced to the rendering speed.

[0051] It should be noted that a number of buffers and signals that are exchanged between the various components of FIG. 3 are not explicitly shown to simplify the diagram. The buffers, signal exchanges and other aspects of the media server have been previously described and can be readily incorporated into FIG. 3. In addition, some the depicted components can be broken up into subcomponents, or can be combined together, to satisfy the desired cost or design needs.

[0052] A configuration similar to the one described in FIG . 3 can be used when watermarks are used for ACR applications. In this case, the enforcement action unit 312 can be labeled as an ACR action unit. This unit controls the content output speed by issuing specific instructions to the control units 306 in order to ensure reliable and accurate recognition of marked content usage. The ACR action unit can also have the responsibility to report the collected data regarding content usage according to data reporting protocols.

[0053] U sing the abo ve architectures, a method for watermark screening in a media server can be implemented that includes receiving a plurality of media con tents in a media server, where at least one of the media contents contains an embedded watermark. This method further includes selecting one of the plurality of media contents, where the selecting includes m£il<ing a selection decision based on an output from a watermark extractor unit. The method additionally includes performing watermark extraction on the selected media content using the watermark extractor unit, and controlling the output of the media server to selectively output a media output that is determined by the watermark extractor unit,

[0054] In one exemplary embodiment, the method for watermark screening in a media server further includes receiving an output from the watermark extractor unit in an enforcement action unit, sending an enforcement signal to a control unit, and controlling the output of the media server using a control unit based on the output of the watermark extractor unit. In another exemplar ' embodiment, the method includes prior to performing watermark extraction, transferring the selected content to a content transformation unit, and transforming the content in the content transformation unit, where the tra sforming includes at least one of: decryption, demultiplexing, decoding, resampling, and mixing.

[0055] In another embodiment, the above noted method further includes storing a history of watermark detection and enforcement events, where the selecting is based on the history of watermark detection. In one exemplary embodiment, the selecting is based on enforcement events. In another exemplary embodiment, storing the history comprises storing a time stamp for the last detected enforceable watermark, and storing the payload of the last detected enforceable watermark. In yet another embodiment, the controlling the output of the media server comprises prioritizing the outputs abased on a stored history of watermark detections. In still another exemplary embodiment, the above noted method further includes determining a transfer interval value and using the transfer interval value to determine the media server output screening rate.

[0056] The exemplary architecture that is shown in FIG. 3 can be used to implement a media server that includes a plurality content sources, and a switch that is coupled to each of the plurality of con tent sources to allow multimedia content from each of the plurality of content sources to be routed to another component of the media servers in response to a signal or command on a control input. Such a media server includes a watermark extractor that is coupled to the switch to receive, one-at-a-time, each of a plurality of media contents corresponding to the plurality of content sources. The media server also includes one or more control units that are coupled to the watermark extractor to receive an indication based on an output of the watermark extractor. The one or more control units are further coupled to one or more output ports of the media server. The one or more control units are further coupled to tlie plurality of sources to control a flow of each of the plurality of the media contents from the plurality of content sources to the one or more output ports based on the output of the watermark extractor.

[0057] In some embodiments, the above noted media server further includes an enforcement action unit coupled to the watermark extractor to receive one or both of watermark extraction results or content screening results from the watermark extractor. In such embodiments, the enforcement action unit is further coupled to each of the one or more control units to provide the indication based on the output of the watermark extractor for imposing a restriction on outputting one or more of the plurality of the media contents to the one or more output ports.

[0058] In some embodiments, the media server also includes a content transformation unit coupled to the switch to receive each of the plurality of media contents and to process each of the plurality of media contents. Such a content transformation unit is further coupled to the watermark extractor to provide each of the plurality of media contents, subsequent to processing thereof, to the watermark extractor. For example, the content transformation unit can include one or more of the following components: a decryption component, a demultiplexing component, a decoding component, a resampling component, or a mixing component.

[0059] In some embodiments, the media server includes a memory buffer located in a path between the content transformation unit and the watermark extractor to temporarily store each of the plurality of media contents after processing thereof by the content transformation unit. In some embodiments, the media server includes a memory buffer located in a path between the switch and the watermark extractor to temporarily store each of the plurality of media contents one-at-a-time prior to retrieval by the watermark extractor. In one exemplary embodiment, the switch is configured to receive a control signal from the watermark extractor that is indicative of which of the plurality of media contents is to be routed to the watermark extractor.

[0060] It is understood that the various embodiments of the present invention may be implemented individually, or collectively, in devices comprised of various hardware and/or software modules and components. These devices, for example, may comprise a processor, a memory unit, an interface that are communicatively connected to each other, and may range from desktop and/or laptop computers, to consumer electronic devices such as media players, mobile devices and the like. For example, FIG. 3 illustrates a block diagram of a device 1 00 within which the various disclosed embodiments may be implemented. The device 1000 comprises at least one processor 1002 and/or controller, at least one memoiy 1004 unit that is in communication with the processor 1002, and at least one communication unit 1006 that enables the exchange of data and information, directly or indirectly, through the

communication link 1008 with other entities, devices and networks. The communication unit 1 06 may provide wired and/or wireless communication capabilities in accordance with one or more communication protocols, and therefore it may comprise the proper

transmitter/receiver antennas, circuitry and ports, as well as the encoding/decoding capabilities that may be necessary for proper transmission and/or reception of data and other information.

[0061] Referring back to FIG. 3 the de vice 1000 and the like may be implemented in software, hardware, firmware, or combinations thereof. Similarly, the various components or sub-components within each module may be implemented in software, hardware or firmware. The connectivity between the modules and/or components within the modules may be provided using any one of the comiectivity methods and media that is known in the art, including, but not limited to, communications over the Internet, wired, or wireless networks using the appropriate protocols.

[0062] Various embodiments described herein are described in the general context of methods or processes, which may be implemented in one embodiment by a computer program product, embodied in a computer-readable medium, including computer-executable instructions, such as program code, executed by computers in networked environments. A computer-readable medium may include removable and non-remo able storage devices including, but not limited to, Read Only Memory (ROM), Random Access Memory (RAM), compact discs (CDs), digital versatile discs (DVD), etc. Therefore, the computer-readable media that is described in the present application comprises non-transitory storage media. Generally, program modules may include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types.

Computer-executable instructions, associated data structures, and program modules represent examples of program code for executing steps of the methods disclosed herein. The

particular sequence of such executable instructions or associated data stractures represents examples of corresponding acts for implementing the functions described in such steps or processes.

[0063] The foregoing description of embodiments has been presented for purposes of illustration and description. The foregoing description is not intended to be exhaustive or to limit embodiments of the present invention to the precise form disclosed, and modifications and variations are possible in light of the above teachings or may be acquired from practice of various embodiments. The embodiments discussed herein were chosen and described in order to explain the principles and the nature of various embodiments and its practical application to enable one skilled in the art to utilize the present invention in various embodiments and with various modifications as are suited to the particular use contemplated , The features of the embodiments described herein may be combined in all possible combinations of methods, apparatus, modules, systems, and computer program products.

Claims

WHAT IS CLAIMED IS:

1. A method for intermittent watermark screening in a media server, comprising:

providing an indication to a watermark extractor that is implemented in-part using electronic circuits that a plurality of media contents from a plurality of content sources are identified for being output from the media server;

selecting a first media content from of the plurality of media contents, and performing watermark extraction on a segment of the first media content;

switching to a second media content from the plurality of media contents based on an indication provided by the watermark extractor, and performing watermark extraction on a segment of the second media content;

evaluating the extracted watermarks from at least the first media content or the second media content to determine whether or not the extracted watermarks impose a restriction on outputting one or more of the plurality of media contents; and

upon a determination that the extracted watermarks impose a restriction on outputting one or more of the plurality of media contents, selectively controlling output of the plurality of media contents from the media server.

2. The method of claim 1, further comprising:

after switching to a second media content from the plurality of media contents and performing watermark extraction on a segment of the second media content, switching to one or more additional media contents from the plurality of media contents based on one or more additional indications provided by the watermark extractor, and performing watermark extraction on a segment of each of the one or more additional media contents, and

evaluating the extracted watermarks from, the first media content, the second media content, or the one or more additional media contents to make the determination as to whether or not the extracted watermarks impose a restriction on outputting one or more of the plurality of media contents.

3. The method of claim 2, wherein switching to the one or more additional media contents continues until all of the plurality of media contents have been subjected to watermark extraction.

4. The method of claim 1, wherein the watermark extractor is the only watermark extractor that is implemented in the media server.

5. The method of claim 1 , wherein selectively controlling output of the plurality of media contents from the media server includes reducing an output rate of at least one of the plurality of media contents.

6. The method of claim 5, wherein reducing the output rate of the at least one of the plurality of media contents includes setting the output rate of the at least one of the plurality of media contents to the rendering rate of the at least one of the plurality of media contents.

7. The method of claim 1, further comprising:

determining the current time relative to a rendering timeline as part of performing watermark extraction on the first or the second media contents, wherein:

determination of whether or not the extracted watermarks impose a restriction on outputting one or more of the plurality of media contents includes using the current time to determine whether or not the extracted watermarks conform to a particular pattern of watermark detections in a predetermined number of time intervals.

8. The method of claim 1, further comprising:

storing information regarding the extracted watermarks and restrictions placed on one or more of the plurality of media contents, and

upon completion of watermark extraction on the segment of the first media content, analyzing the stored information to determine which of the plurality of media contents is to be selected as the second media content.

9. The method of claim 8, further comprising:

assigning a pri ority value to each of the plurality of the media contents based the analyzing of the stored information, and

selecting a particular one of the plurality of the media contents that (a) has a highest priority value, and (b) has not been subjected to watermark extraction, as the second media content.

10. The method of claim 1, further comprising:

enabling one or more of the plurality of media contents to be output at a rate higher than content rendering rate for an initial interval, regardless of the determination that the extracted watermarks impose a restriction on outputting one or more of the plurality of media contents.

11. The method of claim 1, wherein upon the determination that the extracted watermarks impose a restriction on outputting one or more of the plurality of media contents, modifying one or more of the plurality of media contents according to one or more of the following enforcement actions:

mutmg one or more outputs of the media server;

stopping one or more outputs of the media server;

including a message indicative of the enforcement action at one or more outputs of the media server; or

changing a rate at which content is output at one or more outputs of the media server.

12. The method of claim. 1, wherein performing watermark extraction on the first or the second media contents stops upon (a) completion of watermark extraction for up to predefined content amount, (b) extraction of a valid watermark, or (c) the media content under watermark extraction is no longer identified for being output from the media server.

13. The method of claim 1, further comprising:

prior to switching to the second media content, determining a transfer interval that represents an amount of the fi rst content that can be output by the media server at faster than content rendering rate, and

enabling output of the first media content by the media server at faster than rendering rate of the first media content for the duration of the transfer interval.

14. The method of claim 13, wherein:

upon a determination that at least one watermark has been extracted from the first content, assigning a zero value to the transfer interval, and

upon a determination that no watermark has been extracted from the first content, assigning a non-zero value within a predetermined range of time values to the transfer interval.

15. The method of claim 1, wherein performing watermark extraction on the segment of the first media content or on the segment of the second media content results in detection of a watermark value that is used for content identification,

16. The method of claim 1, wherein (a) a determination to select, or (b) an instant in time to switch to, the second content is based on watermark detection results obtained from the first media content, or any one of the plurality of media contents that are designated for watermark extraction.

17. A media server device, comprising:

a processor; and

a memory including processor executable code, the processor executable code, when executed by the processor, cause the media server device to:

provide an indication to a watennark extractor that a plurality of media contents from a plurality of content sources are identified for being output from the media server;

select a first media content from of the plurality of media contents, and perform watermark extraction on a segment of the fi rst media content;

switch to a second media content from the plurality of media contents based on an indication provided by the watermark extractor, and perform watermark extraction on a segment of the second media content;

evaluate the extracted watermarks from at least the fi rst media content or the second media content to determine whetlier or not the extracted watermarks impose a restriction on outputtmg one or more of the plurality of media contents; and

upon a detennination that the extracted watermarks impose a restriction on outputting one or more of the plurality of media contents, selectively control output of the plurality of media contents from the media server.

18. The media server device of claim 17, the processor executable code, when executed by the processor, further configures the media server device to:

after switching to a second media content from the plurality of media contents and performing watermark extraction on a segment of the second media content, switch to one or more additional media contents from the plurality of media contents based on one or more additional indications provided by the watermark extractor, and perform watennark extraction on a segment of each of the one or more additional media contents, and evaluate the extracted watermarks from the first media content, the second media content, or the one or more additional media contents to make the determination as to whether or not the extracted watermarks impose a restriction on outputting one or more of the plurality of media contents.

19. The media server device of claim 18, wherein the processor executable code, when executed by the processor, further configures the media server device to continue switching to the one or more additional media contents until all of the plurality of media contents have been subjected to watermark extraction.

20, The media server device of claim 17, wherein the watermark extractor is the only watermark extractor that is implemented in the media server.

21. The media server device of claim. 17, wherein selective control of output of the plurality of media contents from the media server includes a reduction of an output rate of at least one of the plurality of media contents.

22, The media server device of claim 21, wherein reduction of the output rate of the at least one of the plurality of media contents includes setting the output rate of the at least one of the plurality of media contents to the rendering rate of the at least one of the plurality of media contents.

23. The media server device of claim 17, wherein the processor executable code, when executed by the processor, further configures the media server device to:

determine the current time relative to a rendering timeline as part of performing watermark extraction on the first or the second media contents, wherein:

determination as to whether or not the extracted watermarks impose a restriction on outputting one or more of the plurality of media contents includes using the current tim e to determine whether or not the extracted watermarks conform to a particular pattern of watermark detections in a predetermined number of time intervals.

24, The media server device of claim 17, wherein the processor executable code, when executed by the processor, further configures the media server device to: store information regarding the extracted watermarks and restrictions placed on one or more of the plurality of media contents, and

upon completion of watermark extraction on the segment of the first media content, analyze the stored information to determine which of the plurality of media contents is to be selected as the second media content.

25. The media server device of claim. 24, wherein the processor executable code, when executed by the processor, further configures the media server device to:

assign a priority value to each of the plurality of the media contents based analysis of the stored information, and

select a particular one of the plurality of the media contents that (a) has a highest priority value, and (b) has not been subjected to watermark extraction, as the second media content.

26. The media server device of claim 17, wherein the processor executable code, when executed by the processor, further configures the media server device to:

enable one or more of the plurality of media contents to be output at a rate higher than content rendering rate for an initial interval, regardless of the determination that the extracted watermarks impose a restriction on outputting one or more of the plurality of media contents.

27. The media server device of claim 17, wherein the processor executable code, when executed by the processor, further configures the media server device to:

upon the determination that the extracted watermarks impose a restriction on outputting one or more of the plurality of media contents, modify one or more of the plurality of media contents according to one or more of the following enforcement actions:

muting one or more outputs of the media server;

stopping one or more outputs of the media server;

28. The media server device of claim 17, wherein the processor executable code, when executed by the processor, further configures the media server device to stop watermark extraction on the first or the second media contents upon (a) completion of watermark extraction for up to predefined content amount, (b) extraction of a valid watermark, or (c) the media content under watermark extraction is no longer identified for being output from the media server.

29. The media server device of claim 17, wherein the processor executable code, when executed by the processor, further configures the media server device to:

prior to switching to the second media content, determine a transfer interval that represents an amount of the first media content that can be output by the media sen/er at faster than content rendering rate, and

enable output of the second media content by the media server at faster than rendering rate of the first media content for the duration of the transfer interval .

30. The media server device of claim 17, the processor executable code, when executed by the processor, further configures the media server device to:

upon a determination that at least one watermark has been extracted from the first content, assign a zero value to the transfer interval, and

upon a detennination that no watermark has been extracted from, the first content, assign a non-zero value within a predetermined range of time values to the transfer interval.

31. The media server device of claim 17, wherein performing watermark extraction on the segment of the fi rst media content or on the segment of the secon d media content results in detection of a watermark value that is used for content identification.

32. The media server device of claim 17, wherein (a) a determination to select, or (b) an instant in time to switch to, the second content is based on watermark detection results obtained from the first media content, or any one of the plurality of media contents that are designated for watermark extraction.

33. A computer program product embodied on one or more non-transitory computer- readable media, comprising:

program code for providing an indication to a watermark extractor that a plurality of media contents from a plurality of content sources are identified for being output from the media server; program code for selecting a first media content from of the plurality of media contents, and performing watermark extraction on a segment of the first media content; program code for switching to a second media content from the plurality of media contents based on an indication provided by the watermark extractor, and performing watermark extraction on a segment of the second media content;

program code for evaluating the extracted watermarks from at least the first media content or the second media content to determine whether or not the extracted watermarks impose a restriction on outputting one or more of the plurality of media contents; and program code for upon a determination that the extracted watermarks impose a restriction on outputting one or more of the plurality of media contents, selectively controlling output of the plurality' of media contents from the media server.