1
FILTERING IN PICTURE COLORIZATION
This application is a continuation-in-part of prior application Ser. No. 08/523,015 filed Sep. 1, 1995 now abandoned. 5
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention relates to computerized picture 10 reprocessing techniques and, more particularly, to the modification of colors of digitized picture stock by the application of mathematical models which are representative of photographic filters.
2. Description of Related Art 15 Film colorization, that is, colorizing black-and-white
motion pictures, turned the film industry on its side in the mid-1980s. With less than adequate color selection and limited hardware and software capabilities, early attempts at colorizing notable black-and-white- film classics such as 20 "Casablanca" and "The Big Sleep" produced less than favorable results, resulting in muddy hues that didn't always stick to the objects they were meant to color. Indeed, many film purists likened colorization to vandalism and defacement. However, in the 1990s a demand was created by the 25 skyrocketing cost of producing new movies and television shows coupled with the burgeoning demand for movies and television shows to fill up time slots on the 500 or so cable channels, a demand which has been an incentive for colorizers to advance their craft to much higher levels of 30 quality.
Colorizers have also applied their craft to more varied fields, fields which do not necessarily involve original black-and-white picture stock. For example, in the past if a director of a picture were unhappy with the color of a 35 particular shot, the director would have had to reshoot the shot, which would have incurred high production costs. Further, commercial artists and advertisers may desire to intensify particular aspects of television commercials to be more appealing to consumers of target markets. Other spe- 40 cial color effects may also be desired for a particular film, video, or television show, particularly music videos which are often intended for the less conservative teenage and young adult audience.
45
Other applications for the modification of digitized picture stock include the modification of the color of an entire digitized frame of the stock. For example, if a particular shot is shot during the day and the director later desires to have
the shot be a night shot, then the entire shot would have to „
& 50 be reshot, incurring high costs. Furthermore, if a director of
photography mistakenly used an improper filter or no filter
at all for a particular shot, the shot would have to be
completely redone with the desired filtered on the camera.
Accordingly, it is an object of the present invention to 55
provide filtering techniques for modifying digitized picture
stock.
SUMMARY OF THE INVENTION
Picture filtering technology of the present invention pro- 60 vides a method for modifying the color of digitized picture stock with mathematical filtering techniques. Generally speaking, picture filtering provides a method for modifying colors of a frame of picture stock by firstly digitizing the frame and storing the digitized frame in a computer. Two 65 reference pictures are then selected from a picture or film library which are substantially identical, with one of the
2
pictures having been shot with a filter and the other picture having been shot without a filter. The two reference pictures are digitized so that each set of resultant digital data or pixels corresponds to the set of pixels from the other reference picture. A quantitative differencing is then performed on a pixel-by-pixel basis, thereby determining the difference in, for example, the luminance value of each unfiltered pixels with each corresponding filtered pixel. In other words, for every variation of luminance value of the unfiltered picture there is then a corresponding quantitative difference value which, when multiplied with (or performed with some other arithmetic function) the unfiltered value, will yield the corresponding filtered value.
Accordingly, upon completing the quantitative differencing, a mathematical model is generated which is substantially a mathematical representation of the photographic filter used in the reference pictures. The mathematical model may be a function of any number of color variables, but in many applications, luminance values suffice for accurate filtering of images. The mathematical model is then applied to the digitized frame of the shot to be filtered by "feeding" the digital data through the mathematical model, thereby yielding "filtered" digital data. This process is analogous to light passing through a photographic filter, which light is filtered and recorded upon the exposed film. The filtered frame may then be viewed and further modified as desired. The change effected by the mathematical model may then be interpolated through other frames of the shot or applied to as many frames as desired.
One of the advantages of the filtering technology of the present invention is that if, for example, a director is unhappy with the overall color effect of a particular shot, rather than incur the great production cost of reshooting the shot, the shot may be digitized and "ran through" a mathematical model which is representative of a desired photographic filter, thereby filtering the frame or shot as desired. The shot may then be converted back to a desired form of picture stock.
In addition to other features, the filtering technology of the present invention has one feature which allows a user to propagate or interpolate any changes effected by the filter mathematical model through other frames of the shot, thereby saving the user much time and effort.
One of the features of the technology is that any reference pictures may be used, either from a film or picture library as mentioned or as developed by an individual director or colorist to meet a particular application, in order to generate the mathematical model which is representative of a desired filter.
Other aspects, advantages, and features of the picture filtering technology of the present invention will become apparent to those skilled in the art from a reading of the following detailed description with reference to the accompanying drawings.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 is a block diagram of a multiple workstation computer recolorization network illustrating certain principles of the present invention;
FIG. 2 is a block diagram illustrating a colorization process for picture stock, particularly showing the creation of a picture database;
FIG. 3 is a block diagram illustrating a frame interpolation process used in a colorization process according to the present invention;
FIG. 4 is a schematic diagram illustrating a frame interpolation process of the present invention;
3
FIG. 5 is a block diagram of a recolorization process illustrating principles of the present invention;
FIG. 6 is a block diagram of a filtering process illustrating principles of the present invention; and
FIG. 7 is a schematic diagram of a step in the filtering process of the present invention.
DETAILED DESCRIPTION OF THE
PREFERRED EMBODIMENTS
Referring to the drawings, exemplary embodiments of the present invention are shown, illustrating the principles of picture recolorization. Upon reading the following detailed description with reference to the accompanied drawings, those skilled in the film and colorization arts will come to realize various alternative and modified embodiments of those exemplified and described herein. This description provides a foundation of picture recolorization from which these alternatives and modifications stem. Accordingly, rather than provide an exhaustive description of all possible preferred embodiments envisioned by the inventors, the principles of the present invention are exemplified with only the embodiments illustrated by the attached drawings and elucidated by the following description.
FIG. 1 generally shows a multiple workstation computer network 10 including a central processor unit 12 such as a mainframe computer in communication with a data storage unit 14 and a plurality of terminals or workstations 16. Each of the workstations 16 may have any combination of the user interface devices available on the market, but it is preferable for each workstation 16 to include at least a keyboard with a mouse and a video display. Digitization pads and the like may also be employed in the workstations 16. The data storage unit 14 may take on any desired form available on the market, but as colorization processes require large amounts of data storage space, the data storage unit 14 should be capable of storing data on the magnitude of thousands of megabytes (or gigabytes) or millions of megabytes (or terabytes). The market currently provides either magnetic tape storage systems, magnetic disk systems, or optical disc systems which are capable of storing such voluminous capacity. It follows that it is preferable for the main processor 12 to have data compression capability to efficiently handle this large amount of data. Furthermore, each workstation 16 as well as the processor 12 preferably has dedicated random-access memory (RAM) capability for further efficient use of the picture recolorization process disclosed herein.
In a commercial implementation of the picture recolorization technology of the present invention, the color-oncolor network 10 may be broken down or segmented into dedicated function groups or "work bays" in which personnel performing similar tasks in the recolorization process are located. For example, colorists, that is, artists or other skilled animators who are experts on color, may be assigned a certain number of workstations 16; users who are skilled in the task of masking or drawing polygons around objects to be recolored may be assigned to a number of workstations 16; and users who are skilled in the algorithmic function of interpolating, which is an efficiency function of estimating or fitting color and mask data to unedited frames of a film, may be assigned to further workstations 16. In any case, any number of defined function workstations 16 may be manipulated by colorists and users in an efficient orchestration of the recolorization technology disclosed herein.
At this point a number of definitions of terms in the art will be given in order to allow those people not specifically
4
skilled in the art to understand more fully the principles set forth herein. The concept of color is defined by a combination of the three following qualities: hue, which indicates the gradation of color or the attribute of colors that permits them
5 to be classed as red, yellow, green, blue, or an intermediate color between any contiguous pair of these colors; intensity, which relates to the density or brightness qualities of a color; and saturation, which relates to chromatic purity (i.e., freedom from dilution with white) or the degree of difference from the gray having the same lightness. Additional words used in the art of colorization include: luminance, which relates to the black-and-white aspect of a frame; and chrominance, which is the hue and saturation component of a color. When speaking of movies or films and videos, s picture indicates a generic term for any motion picture including movie, film, video, or the like; frame refers to a single instantaneous exposure of film of the series of exposures on a length of a picture; and shot refers to an unedited or continuous sequence of frames between edited breaks or
2Q cuts of the picture (i.e., "scenes" in a picture) or, in other words, an unbroken view of a given scene from a single camera perspective.
More industry-specific terminology includes: diffusion, which relates to the blending or grading of color at the
25 border of two differently color objects; precedence, which determines which objects in a frame are more forward or rearward (i.e., closer or farther from a view's perspective) than other objects; and baseplane which is the background plane or the most rearward object to be masked in a frame.
30 The definition of the concept called colorspace is somewhat more complicated than those already given. Color television and color computer monitors (i.e., display units and monitors) normally operate in RGB colorspace, RGB standing for the additive primary colors red, green, and blue.
35 These three colors correspond to the three "guns" of color displays and, in addition, roughly correspond to the three types of color receptors in the human eye. As colorization processes add color to existing monochromatic images or modify color of polychromatic images as set forth herein,
40 the colorspace known as "YCrCb" is preferably chosen for internal representation and manipulation because YCrCb colorspace separates luminance information from chrominance information.
In YCrCb colorspace, "Y" represents the monochrome or
45 the luminance portion of the image, and "Cr" and "Cb" respectively represent the red portion and the blue portion of the color image, which are read as "red chrominance" and "blue chrominance." [The color green is not stored because green can be algebraically computed from the other three
50 colors, which is known in the art.] In order to visualize the concept of YCrCb colorspace more clearly, if the Cr-Cb space were displayed in two-dimensional Cartesian coordinates, gray would be in the center (0,0) with the Cr and Cb values both equal to zero. The further from the origin
55 that a point may move (i.e., the Y or luminance value) the more progressive the intensity of the color would become, with the hue of the color being defined as the angle with the origin as its vertex. In addition, a color recipe is a set of luminance points, which includes at least black and white,
60 for any given substantially homogeneous color area or mask. Having provide these basic colorization terms and concepts, picture recolorization technology according to the present invention generally entails a method for modifying the color of existing polychromatic or color picture stock
65 with luminance-to-chrominance mapping techniques. Generally speaking, picture recolorization provides a method for modifying colors of a frame of polychromatic picture stock
|
