WO2001029817A1

WO2001029817A1 - Improving image display quality by adaptive subpixel rendering

Info

Publication number: WO2001029817A1
Application number: PCT/US2000/029037
Authority: WO
Inventors: Anwar M. Ghuloum; Pak Y. Tam; Xiaolong Ouyang; Edmund Y Lam
Original assignee: Intensys Corporation
Priority date: 1999-10-19
Filing date: 2000-10-19
Publication date: 2001-04-26
Also published as: JP2003512653A; AU1097601A; EP1171868A1

Abstract

A method for displaying an image comprises receiving pixel data representing an image, determining an image feature in the received pixel data, and generating a subpixel drive correction signal based on the determined image. The generated subpixel drive correction signal can be based on display attributes, and can be used to generate a pixel drive signal.

Description

IMPROVING IMAGE DISPLAY QUALITY BY ADAPTIVE SUBPIXEL

RENDERING

PRIORITY

This application claims priority to Provisional Application No. 60/160,397, filed on October 19, 1999 , which is incorporated herein. FIELD OF THE INVENTION

The present invention relates to displaying images. In particular, the present invention relates to improving image-display quality by adaptive subpixel rendering.

BACKGROUND

Multicolor displays typically use pixels that are constructed from discrete red, green and blue subpixels. One example of such multicolor displays, liquid crystal displays (LCDs) have three spatially-offset color components including red, green and blue subpixels. One example of color components in an LCD screen is shown in

Figure 1, which consists of rectangular areas formed by arranged the red, green and blue components in a horizontal line. As can be seen from Figure 1, pixel 101 in this example consists of red subpixel 101a, green subpixel 101b and blue subpixel 101c. Of course, other arrangements are possible, such as those described by S. Tsuruta, et al., in "Color Pixel Arrangement Evaluation for LC-TV" (1985 International Display

Research Conference, pp. 24-26).

Each of the color components can be individually controlled by an LCD controller to produce different shades of color. Because each set of the color components belongs to a single pixel, however, in many image processing algorithms, including image rendering or rasterization, the subpixels are treated as if they co- locate in a single coordinate with many similar attributes. This can lead to jagged lines and inferior image display quality. Figure 2 shows an example of a diagonal line displayed using a typical pixel-rendering scheme. As can be seen from this picture, the line is jagged. Ways exist to address this problem, including methods of anti- aliasing. A typical anti-aliased diagonal line is shown in Figure 3. Using antialiasing, pixels adjacent to the blackened pixel are displayed in various shades of gray, thereby allowing the eye to smooth the line. Unfortunately, anti-aliasing diminishes image sharpness, and can result in an unclear, or blurry image. Other methods exist for improving image displays, but each of these methods have undesirable drawbacks. One method, for example, is shown in U.S. Patent No. 5,341 ,153, entitled "Method and Apparatus for Displaying a Multicolor Image." The invention in that patent attempts to deal with the problem of single-colored subpixels. That invention, however, attempts to correct for subpixel displacement, rather than using the displacement as a characteristic to improve a displayed image.

Another method is found in U.S. Patent No. 6,028,646, titled "Color Image Enhancement Device for Video Display Appliance." The invention in that patent attempts to provide a color-image enhancement device for improving the sharpness of the color image by adjusting the image's saturation. For the purposes of that invention, however, the subpixels are assumed to co-locate, and color space transformation is performed to convert the image's representation from red, green and blue to luminance, hue and saturation. As such, the advantages of using display attributes are typically not explored. Thus, because of the drawbacks in typical pixel displays, a need is shown for improved image rendering that can use subpixels to increase image clarity and resolution. SUMMARY OF THE INVENTION.

To alleviate the problems in known systems, displaying images using adaptive subpixel rendering is discussed herein.

Embodiments of the present invention allow for improved display quality by employing methods and devices that perform adaptive subpixel rendering. In one embodiment of the present invention, a method for displaying an image comprises receiving pixel data representing an image, determining an image feature in the received pixel data, and generating a subpixel drive correction signal based on the determined image. In another embodiment of the present invention, the subpixel drive correction signal is based on an attribute of a display device. BRIEF DESCRIPTION OF THE DRAWINGS

Figure 1 is a representation of a known subpixel structure. Figure 2 is a representation of an example of a diagonal line displayed using a typical pixel-rendering scheme.

Figure 3 is a representation of an example of a diagonal line displayed using an anti-aliasing scheme.

Figure 4 is a flow chart of an embodiment of the present invention. Figure 5 is a flow chart of calculations, according to an embodiment of the present invention.

Figure 6 is a block diagram of an apparatus according to an embodiment of the present invention, including a processor, a memory and a port. Figure 7 is a block diagram of another apparatus embodiment of the present invention, including an I/O Buffer, and Address Counter, a RAM, a Timing Generator, a MPU Interface, a Command Decoder, a Subpixel Feature Enhancer, and an Apparatus Driver.

Figure 8 is a representation of a black vertical line, illustrating an embodiment of the present invention.

Figure 9 is a representation of a diagonal line, according to an embodiment of the present invention. DETAILED DESCRIPTION

Embodiments of the present invention provide improved image-display quality using adaptive subpixel rendering. If desired, embodiments of the present invention can use attributes of a display device to provide the improved image-display quality.

As a contextual example, but not the only example, consider a multicolor display constructed from discrete red, green and blue components, such as a Liquid Crystal Display (LCD). Figure 4 is a flow chart of an embodiment of the present invention. In this embodiment, pixel data representing an image is received at step 401. At step 402, an image feature in the received pixel data is determined. Using the determined image feature, at step 403, a subpixel drive correction signal is generated. Once the subpixel drive correction signal is generated, at step 404, a drive signal based on the subpixel drive correction signal is generated.

In one embodiment of the present invention, the subpixel drive correction signal can be based on an attribute or attributes of a given display device. For example, but not the only example, a display device can be an LCD. The attributes can include, but are not limited to, subpixel configurations such as the relative position of the subpixels, the size of the subpixels, the shape of the subpixels, the geometry of the subpixels and the orientation of the subpixels. For example, referring back to Figure 1, as can be seen, the red, green and blue subpixels have a spatial offset from one another. These subpixel spatial offsets can be used to increase the spatial resolution of the image by a factor of N, where N is the number of color components for a given pixel.

The finite size and shape of the subpixels are also attributes that can be used to enhance the image because they determine the size and shape of the image details that can best be represented by the subpixels.

The geometry and the orientation are both concerned with the arrangement of the subpixels. Different arrangements allow our eyes to perceive sharper details at certain angles and frequencies, when the image is rendered at a subpixel level.

In another embodiment of the present invention, the attribute of "intensity mapping" is used to enhance the image. Because the color components can be controlled individually by a pixel drive signal, by treating them as separate subpixels it is possible to use dynamic intensity mapping to further improve image quality.

In one embodiment of the present invention, the pixel data received at step 401 can include subpixel data, and can be rendered or rasterized data. For the purposes of the present invention, the term "rasterization" means determining pixel values from input geometric primitives.

The image feature detection performed at step 402 can be performed in any known way. In one embodiment of the present invention, to make use of the subpixel attributes, after the information of the rendered image is passed on to, for example, but not the only example, a display controller, it is passed through a few stages of calculation. Figure 5 is a flow chart of such calculations, according to an embodiment of the present invention. At step 501, an NxM square, or window, of pixels is identified or input, where N and M can include, for example (but not the only example) from 3 to 9 pixels. One skilled in the art will realize, of course, that the values of N and M can be arbitrarily small or arbitrarily large, depending on the image feature detection to be performed, or depending on the end result that is desired. The range "3 to 9 pixels" is only an example to illustrate what is meant by "an NxM square."

At step 502, an edge in the input image block is detected, and at step 503, an edge metric is calculated. The edge metric can include, but is not limited to, information about the edge location, the edge orientation, and contrast. The detection and calculation can be performed by any appropriate operation. For example, but not the only example, edge detection and calculation can be performed by using the Sobel or Laplacian filter for edge detection. Other examples are described in Gonzalez and Woods, Digital Image Processing (Addison-Wesley, 1992), the relevant portions of which are hereby incorporated. The calculations can be performed on, for example, but not the only example, a single instruction, multiple data (SIMD) array processor. At step 504, pixel and subpixel attributes are combined with the edge metric. At step 505, the image features can be rendered or rasterized appropriately with subpixel accuracy for any particular color attribute or component geometry in a display device. This rendering or rasterization can be performed in software using any appropriate algorithm.

One skilled in the art will recognize that the order of the flow chart above, and all the flow charts in this document (including all claims), is not meant to imply a necessary order. For the purposes of the present invention, the steps of the flow charts and claims in this document can be performed in any order practicable.

Figure 6 is a block diagram of an apparatus according to an embodiment of the present invention. In one embodiment of the present invention, processor 601 is coupled to memory 602 and port 603, and memory 602 stores instructions adapted to be executed by processor to perform a method embodiment of the present invention. For example, but not the only example, memory 602 stores instructions adapted to be executed by processor 601 to receive a rasterized image, detect a feature in the rasterized image, and then enhance the detected feature on a subpixel level. For the purposes of this application, memory includes any computer-readable medium capable of storing instructions adapted to be executed by a processor. Some examples of such media include, but are not limited to, RAM, ROM, floppy disks, CDROM, magnetic tape, hard drives, and any other device that can store digital information. In one embodiment, the instructions are stored on the medium in a compressed and/or encrypted format. As used herein, the phrase "adapted to be executed by a processor" is meant to encompass instructions stored in a compressed and or encrypted format, as well as instructions that have to be compiled or installed by an installer before being executed by the processor.

Figure 7 is a block diagram of another apparatus embodiment of the present invention. In this block diagram, I/O Buffer 701 is coupled to Address Counter 702.

Address Counter 702 is coupled to RAM 703, which in turn is coupled to Timing Generator 704, Command Decoder 705, and Subpixel Feature Enhancer 706. Subpixel Feature Enhancer 706 and Timing Generator 704 are both coupled to Apparatus Driver 707, and Command Decoder 705 is coupled to MPU Interface 708. I/O Buffer 701 can receive rendered or rasterized data representing an image. Address Counter 702 can direct which of the data in I/O Buffer 701 is to be transferred to RAM 703 in the controller. Timing Generator 704 can provide a timing pattern for the entire controller, or some portion thereof. MPU Interface 708 can direct external commands to Command Decoder 705. wherein the commands can be subsequently directed to data stored in RAM 703.

In Subpixel Feature Enhancer 706. data representing an image block can be fed from RAM 703. Subpixel Feature Enhancer 706 can then determine particular image features in the received pixel data, such as the existence of an edge. If a particular feature is identified, the characteristics of the feature are also computed in

Subpixel Feature Enhancer 706, such as the orientation and contrast of the edge. Taking into account the attributes of the display controller, Subpixel Feature Enhancer 706 can generate a subpixel drive correction signal to enhance the detected features. The resulting enhanced image data are fed to the display driver which interacts with the actual image display.

The following examples are comparisons of various image renderings according to embodiments of the present invention, as compared with image renderings without applying embodiments of the present invention.

Consider Figure 8, which displays the simple case of a black vertical line, with a width of exactly one pixel wide. As can be seen in 8(a), the vertical line is displaced by one-third with respect to the pixel grid. When the line is rendered on a pixel level, considering the situation where white corresponds to maximum intensity and black corresponds to minimum intensity, the intensity of the left column is reduced to two- third while the intensity of the second column is reduced to one-third, to give the impression to the eye that together they form a black vertical line, as shown in 8(b).

If such an image is displayed on an LCD with an arrangement of the subpixels similar to the subpixels found in Figure 1 , the individual subpixel intensities are adjusted as in 8(c), where the red, green, and blue subpixels corresponding to the pixels in the left column are all reduced by two-thirds, and those corresponding to the pixels in the right column are all reduced by one-third. One skilled in the art will appreciate that, for the sake of clarity in the diagram, the red, green, and blue subpixels that correspond to the white pixels in the background are not shown. The image sharpness, however, can be significantly enhanced with subpixel level rendering, taking into account the subpixel attributes. The result is illustrated in 8(d), where the blue subpixels corresponding to the pixels in the left column and the red and green subpixels corresponding to the pixels in the right column are turned to minimum intensity. Because eyes would average a set of red, green, and blue pixels together, such a scheme would produce a black vertical line without color artifacts. Such a subpixel rendering scheme is also helpful for edges tilted at an acute or obtuse angle with respect to the horizontal or vertical grid. Conventionally, as discussed in the BACKGROUND, above, such a line is rendered using a standard anti-aliasing operation performed to preserve object edges. Conventional anti-aliasing schemes typically gray out the whole of some neighboring pixels to enhance the quality of the rendered fine line. This is schematically illustrated in Figures 2 and 3, as discussed above, and described in details in J. Foley et al. Computer Graphics: Principles and Practice, Second Edition in C (Addison-Wesley, 1995). In figure 2, one can see a tilted line rendered without anti-aliasing, while Figure 3 shows the same line rendered with traditional anti-aliasing. By taking into account the display attributes, however, the image quality can be improved in a way similar to the way the vertical line was improved in Figure 8. This is illustrated in figure 9. At 9(a), one can see the conventional anti-aliasing scheme together with the color components. When the algorithm calls for a graying of the neighborhood pixels, all 3 color components are reduced in intensity by the same amount. At 9(b), on the other hand, one can see the subpixel anti-aliasing scheme. At neighborhood locations of the fine line that requires diminished intensity, one can selectively reduce the intensities of the red, green, or blue components to minimize the jaggedness and color distortions of the rendered line. This results in a smoother and clearer line than would be available from the conventional method. As one skilled in the art will appreciate, subpixel-level feature enhancement schemes for image display are not limited to anti-aliasing applications. For example, but not the only example, another particular advantageous application is for low-end displays, such as palm-top organizers and mobile phones, in which maximum resolution texts should be displayed with as small a number of pixels as possible. For such devices, screen resolution is usually very limited. Thus, if the image rendering from the host computer is performed on a pixel level, many fine lines may be lost or difficult to represent. By using subpixel rendering, on the other hand, and taking into account the display attributes, resolution can be increased by up to N times, where N is the number of color components for a given pixel. Using similar techniques as described above, fine lines can be represented more precisely with subpixel level rendering.

The present invention has been described in terms of several embodiments solely for the purpose of illustration. Persons skilled in the art will recognize from this description that the invention is not limited to the embodiments described, but may be practiced with modifications and alterations limited only by the spirit and scope of the appended claims.

Claims

WHAT IS CLAIMED IS 1. A method for displaying an image, the method comprising: (a) receiving pixel data representing an image; (b) determining an image feature in the received pixel data; and (c) generating a subpixel drive correction signal based on the determined image.

2. The method of claim 1 , wherein said generating a subpixel drive correction signal is further based on an attribute of a display device.

3. The method of claim 2, wherein the attribute is a subpixel configuration.

4. The method of claim 2, wherein the attribute is a relative position of subpixels within a pixel.

5. The method of claim 2, wherein the attribute is a geometry of subpixels within a pixel.

6. The method of claim 2, wherein the attribute is an orientation of subpixels within a pixel.

7. The method of claim 2, wherein the attribute is a size of subpixels within a pixel.

8. The method of claim 1 , further comprising: (a) generating a drive signal based on the subpixel drive correction signal.

9. The method of claim 2, further comprising: (a) generating a drive signal based on the subpixel drive correction signal.

10. A computer readable medium storing instructions adapted to be executed on a processor, the instructions comprising: (a) receiving pixel data representing an image; (b) determining an image feature in the received pixel data; and (c) generating a subpixel drive correction signal based on the determined image.

11. The computer readable medium of claim 10, wherein the instruction for generating a subpixel drive correction signal is further based on an attribute of a display device.

12. The computer readable medium of claim 1 1, wherein the attribute is a subpixel configuration.

13. The computer readable medium of claim 1 1, wherein the attribute is a relative position of subpixels within a pixel.

14. The computer readable medium of claim 1 1, wherein the attribute is a geometry of subpixels within a pixel.

15. The computer readable medium of claim 11, wherein the attribute is an orientation of subpixels within a pixel.

16. The computer readable medium of claim 11, wherein the attribute is a size of subpixels within a pixel.

17. The computer readable medium of claim 10, further comprising: (a) generating a drive signal based on the subpixel drive correction signal.

18. The computer readable medium of claim 11 , further comprising: (a) generating a drive signal based on the subpixel drive correction signal.

19. An apparatus for displaying an image, the apparatus comprising: (a) means for receiving pixel data representing an image; (b) means for determining an image feature in the received pixel data; and (c) means for generating a subpixel drive correction signal based on the determined image.

20. The apparatus of claim 19, wherein said means for generating a subpixel drive correction signal is further based on an attribute of a display device.

21. The apparatus of claim 19, further comprising: (a) means for generating a drive signal based on the subpixel drive correction signal.

22. The apparatus of claim 20, further comprising: (a) means for generating a drive signal based on the subpixel drive correction signal.

23. A method for displaying an image, the method comprising: (a) receiving a rasterized image; (b) detecting a feature in the rasterized image; and (c) enhancing the detected feature on a subpixel level.

24. The method of claim 23, wherein said enhancing is based on an attribute of a display device.

25. A computer readable medium storing instructions adapted to be executed on a processor, the instructions comprising: (a) receiving a rasterized image; (b) detecting a feature in the rasterized image; and (c) enhancing the detected feature on a subpixel level.

26. The computer readable medium of claim 25, wherein said enhancing is based on an attribute of a display device.

27. An apparatus for displaying images, the apparatus comprising: (a) means for receiving a rasterized image; (b) means for detecting a feature in the rasterized image; and (c) means for enhancing the detected feature on a subpixel level.

28. The apparatus of claim 27, wherein said means for enhancing is based on an attribute of a display device.