WO2002032132A1

WO2002032132A1 - Architecture for multiple pixel formats

Info

Publication number: WO2002032132A1
Application number: PCT/EP2001/011566
Authority: WO
Inventors: Jens Rennert; Ralph Escherich
Original assignee: Koninklijke Philips Electronics N.V.
Priority date: 2000-10-07
Filing date: 2001-10-04
Publication date: 2002-04-18

Abstract

An interface (200) between a frame memory (110) and a display processing system (120) is provided wherein the number of bits-per-pixel is continuously programmable to any integer value between 1 and an upper limit, nominally 32 bits-per-pixel, without regard to the underlying structure of the processors that provide or use the pixel data. Data is read from memory (110) in fixed increments, based on the underlying structure of the processor, thereby providing an efficient interface with conventional processing systems, and subsequently processed based on the programmed number of bits-per-pixel. The architecture also allows for a programmable order of bits and components within each pixel to provide a programmable interface with other devices and components within a graphics or video processing system. Each component of the pixel format is allocated to one of a number of channels, nominally four, for subsequent processing and rendering to a display system.

Description

Architecture for Multiple Pixel Formats

BACKGROUND OF THE INVENTION

1. Field of the Invention

This invention relates to the field of video and graphics processing, and in particular to a generic system architecture that facilitates the use of a variety of pixel formats

2. Description of Related Art

Video and graphic systems display images based on pixel data that is formatted in a predefined manner. For example, a common format for color graphic pixels is eight bits each of red, green, and blue (RGB) component values, organized as a 24-bit word, or as a 32-bit word with eight null bits, or eight bits that are associated with another aspect of the pixel value, such as texture. A common format for black and white graphics is a single 8- bit component that specifies the intensity, or gray-scale value of each pixel. A common format for video data is a luminance (Y) component, and a pair of chrominance (UN) components, with a specified number of bits being associated with each component. To display an image corresponding to the pixel data, a display system is configured to appropriately interpret the contents of the pixel data, corresponding to one or more pixel formats.

A variety of techniques have been proposed for accommodating multiple pixel formats within a given system architecture. Generally, these techniques are based upon the architecture of the processing system, rather than an architecture that is designed to accommodate graphics and video data processing. For example, in a byte and word oriented computer architecture, the pixel data generally conforms to byte and word, or multi-word, boundaries. The aforementioned 32-bit pixel structure, with potentially 8 unused bits, is based on a 8-bit byte, 32-bit word architecture. Conventionally, the interface between devices and components is based on the byte, word, or double-word sizes used by the processing system. By constraining the pixel data to be an integer multiple of a common byte, word, or double-word size, the complexity of the transfer of pixel data to and from memory devices, or other devices, is greatly simplified. As advances are made in graphics and video processing, different formats can be expected to be developed to optimize the processing or storage of image data. For example, eight bits of color component values may be deemed insufficient for high-quality graphics processing, or techniques may be developed that use relative component values, rather than absolute values, to reduce storage requirements, or to provide higher color resolution with the same number of bits.

Each format change requires a subsequent re-design of the graphics or video processor, to add the format to the list of supported formats. Depending upon the degree of change, a format change often requires a substantial architectural change. To minimize the amount of change required, re-designed systems often use components of prior systems, and are designed to accommodate the change at the expense of efficiency. For example, if a 6-bit color encoding format becomes standard, an existing 8-bit systems might be redesigned to place the 6-bits of each color into the processing system as 8-bit words, with two bits unused, to minimize the scope of the redesign. In such an embodiment, however, the storage and communication advantages gained by the 6-bit encoding are not achieved. In this example, 25% of the storage used for holding the pixel data for presentation to the display, commonly termed the frame buffer, is not utilized. Even a relatively minor change, such as a change to the mapping of components within each pixel, from, for example, a red-green-blue format to a green-red-blue format, could also require a substantial re-layout of integrated circuits or printed circuit boards that are configured for red-green-blue (RGB) formatted data. Such a change in the order of bits or components within each pixel also introduces compatibility problems with other devices or components with the graphics or video processing system that are designed for a given bit or component ordering.

BRIEF SUMMARY OF THE INVENTION

It is an object of this invention to provide a system architecture that is generally compatible with current and future pixel formats. It is a further object of this invention to provide a system architecture that can be easily reconfigured to be compatible with a variety of pixel formats. To this end, the invention provides what is defined by the independent claims. Advantageous embodiments are defined by the dependent claims.

A preferred embodiment provides a system architecture in which the number of bits-per-pixel is continuously programmable to any integer value between 1 and an upper limit, nominally 32 bits-per-pixel, without regard to the byte, word, or double- word boundaries of the processing systems that provide or use the pixel data. Data is read from memory in fixed increments, thereby providing an efficient interface with conventional processing systems, and subsequently processed based on the programmed number of bits- per-pixel. The architecture also allows for a programmable order of bits and components within each pixel to provide a programmable interface with other devices and components within a graphics or video processing system. Each component of the pixel format is allocated to one of a number of channels, nominally four, for subsequent processing and rendering to a display system.

BRIEF DESCRIPTION OF THE DRAWINGS The invention is explained in further detail, and by way of example, with reference to the accompanying drawings wherein:

FIG. 1 illustrates an example block diagram of a video or graphics processing and display system in accordance with this invention.

FIG. 2 illustrates an example block diagram of a programmable system interface for transforming pixel data of varying formats in accordance with this invention.

FIG. 3 illustrates an example flow diagram of a programmable system interface for transforming pixel data of varying formats in accordance with this invention.

Throughout the drawings, the same reference numerals indicate similar or corresponding features or functions.

DETAILED DESCRIPTION OF THE INVENTION

FIG. 1 illustrates an example block diagram of a video or graphics processing and display system 100 in accordance with this invention. The system 100 includes a first processing system 110 that creates an array of picture elements (pixels) in a frame memory 115, and a second processing system 120 that renders these pixels to a display device 125. In accordance with this invention, the pixels are communicated from the first processing system 110 to the second processing system 120 via a programmable system interface 200. Although the first and second processing systems 110, 120 are illustrated as being separate entities, these processes may also be integrated within a single device. Encoded information, modulated signals, and so on are received by a device, such as a television receiver or computer, processed, and subsequently displayed.

The first processing system 110 may be a video system, such as a television receiver, an MPEG decoder, a video card in a personal computer, a set-top box, and so on. Alternatively, or additionally, the system 110 may be a graphics program that is running on a personal computer, a game device, a memory device, or any other source of image information that is arranged in pixel form. Generally, the format of the values that comprise the picture element varies, depending upon the type of system 110. For example, video information, such as used by the PAL (Phase Alternation Line), NTSC (National Television System Committee), and SECAM (Sequential Couleur Avec Memoire), comprises a luminance value (Y), and two chrominance values (U, and V), and the format of the video information is referred to as "YUV". Other standards, such as YIQ, YDbDr, and YCbCr are also used. Within particular standard formats, alternative encodings are used. The YCbCr standard includes, for example, different sampling formats: 4:4:4, 4:2:2, 4:1:1, and 4:2:0. These formats describe the ratio of Y:Cb:Cr components forming the pixel. For example, the 4:1:1 format contains four Y samples for every Cb and Cr sample, each of the Y samples corresponding to a pixel. The chrominance value for each pixel is determined via an interpolation of the provided Cb and Cr chrominance values. Generally, eight bits are allocated for each sub-pixel component Y, Cb, Cr, and therefore the number of bits-per-pixel for the 4:4:4, 4:2:2, 4:1:1, and 4:2:0 formats are 24, 16, 12, and 12, respectively. Pixels generated by graphics programs and applications are generally encoded as the values of red, green, and blue components that form the color of the pixel, and the format is referred to as "RGB". A "16-bit" pixel value typically contains three 5-bit R, G, and B values, the remaining bit being unused. A "24-bit" pixel value contains three 8-bit R, G, and B values. A "32-bit" pixel value generally contains three 8-bit R, G, and B values, and an 8-bit "alpha" (A) value, typically representing a parameter that is used to blend overlapping pixel values with variable weighting factors, or, some other ancillary value, such as a texture value.

The second processing system 120 is illustrated in FIG. 1 as being a display processing system, such as a display driver on a personal computer, or a video drive circuit in a television display. As would be evident to one of ordinary skill in the art, the processing system 120 may be any processing system that accepts pixel values as input. The processing system 120 may be, for example, a video mixer, a video or graphic sealer, an MPEG encoder, a video recorder, a picture-in-picture overlay generator, an interlacer/deinterlacer, and so on. Conventionally, such a processing system 120 accepts a limited number of input formats, often only one input format.

Conventionally, because the first processing system 110 is often software- based, and the second processing system 120 is often hardware-based, the processing by the first processing system 110 includes a transformation of the received information into the format required by the second processing system 120, prior to storage of the pixel information in the frame memory. As noted above, a change of format, such as the format of the pixel information that is stored in the frame memory of the first processing system 110, or the format of the pixel information that is used by the second processing system 120, often requires a substantial design change to either the system 110 or the system 120, or both. Due, in general, to the ease of modifying the system 110 compared to modifying the system 120, this often leads to general system inefficiencies, such as the use of more frame memory than would be required if both systems 110, 120 employed a common format, redundant processing, and so on.

In accordance with this invention, the interface system 200 provides the necessary pixel format transformation between a first processing system 110 and a second processing system 120. In this manner, each system 110, 120 can be optimized, independent of pixel-format-interchange considerations. For example, a high definition display system 120 may be designed to efficiently process 24-bit YUN pixel data. A computer game may use an internal 9-bit Green-Red-Blue format that is optimized to achieve high speed at a moderate resolution. The high speed may also be dependent upon having multiple frames of images in the frame memory at the same time. In a conventional system, the speed may be compromised by the need to convert the pixel data into a 24-bit YUN, in software, before storing it into the frame memory 115, and further compromised by limiting the number of frames that can be stored in the available frame memory space 115. By providing an interface system 200, the optimized internal 9-bit pixel format of the computer game can be maintained, while providing the required 24-bit YUV pixel format to the display system 120. In like manner, the same computer game can be used efficiently with a low-definition portable display device that is optimized to use a 16-bit RGB display format, merely by modifying the transformation that is effected via the interface system 200. FIG. 2 illustrates an example block diagram, and FIG. 3 illustrates an example flow diagram, of a programmable system interface 200 for transforming pixel data of varying formats in accordance with this invention. For convenience, references to items in FIG. 2 begin with the numeral "2", and references to items in FIG. 3 begin with the numeral "3". For data access efficiency, the memory bus interface 210 of the interface 200 reads a fixed number of bits from the frame memory 115 (of FIG. 1) with each data read, at 310, independent of the number of bits allocated per pixel in the frame memory 115. As in a conventional memory access, the fixed number of bits are read from the memory 115 in response to a data request DReq to the processor 110 (of FIG. 1) containing the memory 115, typically via a memory-access-sub-processor (not shown) within the processor 110. The fixed number of bits for each memory access is provided as a parameter to the process 310, depending upon the memory access capability provided by the processor 110.

At 320, the order of bits that are read from the memory 115 is reversed, as required, via the endianess converter 220. As is known in the art, some systems store bits in computer memory from least significant bit (LSB) to most significant bit (MSB), while other systems store bits in computer memory from MSB to LSB. (The terms "big-endian" and "little-endian" are used to distinguish between a word that has the MSB at its end, and a word that has the LSB at its end, respectively.) In a preferred embodiment, one convention, either MSB-LSB, or LSB-MSB is used in the system interface 200, and the endianess converter 220 effects a conversion to this convention, if the data in the frame memory 115 uses the opposite convention. The memory endianess is communicated as a parameter to the process 320 to control this conversion.

At 324, the fixed number of bits, with appropriate endianess, are stored in a buffer, or register, and at 330, a pixel's worth of data is removed from the buffer, via a pixel sheer 230, based on a parameter corresponding to the number of bits-per-pixel. In a preferred embodiment, this parameter is communicated from the processing system 110 to the interface system 200, based on the application that is currently running on the processing system 110. If the interface 200 is embedded in a fixed format system, the number of bits-per-pixel is provided via, for example, via a ROM, or switch or jumper settings. When a redesign of the fixed-format processing system 110 provides a different number of bits-per-pixel, the change is accommodated via a change to the ROM, switches, or jumpers, rather than a change to the processing system 120.

If the number of bits-per-pixel, N, is not in the buffer, at 328, another memory read is performed, via the loop back to block 310. This loop is repeated until there are at least N bits in the buffer.

The pixel sheer 230 extracts the N bits from the buffer using any of a variety of techniques common in the art. For example, the buffer can be configured as a circular buffer, having a read pointer and a write pointer. A write command includes the bits-per- memory-read parameter that stores the fixed number of bits into the buffer, starting at the address indicated by the write pointer, then advances the write pointer by the fixed number of bits. A read command includes the bits-per-pixel parameter, and provides this number of bits- per-pixel to the subsequent block 240, then advances the read pointer by this number of bits- per-pixel. The read pointer and write pointer effect a circular buffer operation by cycling back to zero when the end of the buffer is reached. Other slicing techniques, including shifting bits through a shift register that contains the data read from the memory, are common in the art.

The bits-per-pixel parameter is any integer between 1 and a maximum number of bits-per-pixel. The maximum number of bits-per-pixel determines the resources required to implement this invention, such as the size of the buffer that is used to hold the bits from the memory. Preferably, the buffer is configured to allow the storage of twice the number of bits-per-memory-read required to contain the maximum number of bits-per-pixel. For example, if the maximum number of bits-per-pixel is 30 bits, and the number of bits-per- memory-read is 16 bits, two reads (32-bits) are required to contain the maximum number of bits-per-pixel, and the buffer is configured to contain 64 bits (twice 32). If the number of bits- per-memory-read is 32 bits, one read (32-bits) is required, and the buffer is similarly configured to contain 64 bits. In a preferred embodiment of the invention, the maximum number of bits-per-pixel is 32, and the fixed number of bits-per-memory read 64 bits, and the buffer size is 128 bits. The component slicer 240 segregates the pixel bits into sub-pixel components, such as the alpha (A), red (R), green (G), and blue (B) components of an ARGB pixel format, or the Y, Cb, Cr components of a YCbCr pixel format, at 340. A bits-per-component parameter serves to identify the size of each component in the pixel that is currently being processed. In a preferred embodiment of this invention, the maximum bits-per-component for each component is at least half the maximum bits-per-pixel parameter, or 16 bits per component. A channel swap device 250 reorders the components into a standard format used by the interface 200, at 350. For example, if the interface 200 outputs pixel information in an R-G-B format, and the data is stored in the memory in G-R-B order, the swap device 250 effects the appropriate reordering of the G-R-B components into R-G-B components. In a preferred embodiment of this invention, at least four channels are provided by the component slicer 240, and each of these four components can be placed in any order relative to each other by the component swap device 250. As would be evident to one of ordinary skill in the art, the function of blocks 340-350 may be integrated into a single re-ordering element that receives re-ordering parameters that indicate the location and size of each component in the current pixel, and outputs the components in a known, fixed, order.

The format block 260 effects the required reformatting of the pixel data that is presented in a known, fixed, ordering into the pixel data format required by the second processor 120, at 360. Often, the only difference between the pixel values from the channel swap device 250 and the required format for the second processor is the number of bits-per- pixel. In these situations, the formatter effects an "expansion" or "compression" of the pixels in each channel, via a logical shift of the pixel values in each channel. Also common, the pixel data may be encoded in a signed form, and the format block 260 provides a two's- complement function to one or more channels. In a basic form of an embodiment of this invention, the fundamental encoding

(e.g. RGB, YUV, etc.) of the data in the first processing system 110 is assumed to be the same as the encoding required by the second processing system 120, wherein the differences are limited to the number of bits used for encoding each component, and/or the order of the components within each encoding. As evident to one of ordinary skill in the art, the elements 240-260 may be embodied as a single integrated element that provides the functions of blocks 340-360.

In a more advanced embodiment of this invention, the format device 260 is configured to effect a transformation of the fundamental encoding. The conversion from RGB values to and from YUV, YIQ, YDbDr, YCbCr, etc. can be effected using mathematical equations that are provided by each standard. In like manner, buffering and interpolation can be provided, to effect a change among the specific Y:Cb:Cr sampling formats, and other formats. Because the input to the format device 260 is standardized with regard to the component ordering, and because the basic format device 260 includes expansion and complementing functions to provide a standard configuration of each fundamental encoding, the task of converting from one standard encoding to another standard encoding is simplified, even though the input from the processing system 110 may be non- standard. In a preferred embodiment, the more advanced transformation functions are embodied in a programmable component, such as a Programmable Logic Array (PLA) or a Digital Signal Processor (DSP), so that future standards for input to a display processing system 120 can be supported via a change of programming of the format device 260. The formatter 260 provides the appropriately formatted pixel component values to the second processor 120, at 365.

The next pixel value is processed, via a loop back to the decision block 328. If a sufficient number (N) of bits are available in the buffer, N bits are extracted. Otherwise, one or more fixed-bit reads from the memory 115 are effected until at least N bits are available, via the loop 310-328, and the above process is repeated.

The foregoing merely illustrates the principles of the invention. It will thus be appreciated that those skilled in the art will be able to devise various arrangements which, although not explicitly described or shown herein, embody the principles of the invention and are thus within the scope of the following claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps other than those listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the device claim enumerating several means, several of these means can be embodied by one and the same item of hardware. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

Claims

CLAIMS:

1. An interface system (200) comprising: a buffer (210) that is configured to receive a fixed number of bits from a first processing system (110) a pixel slicer (230), operably coupled to the buffer (210), that is configured to segregate the fixed number of bits into one or more pixels having a programmable number of bits-per-pixel, the programmable number of bits-per-pixel, the programmable number of bits- per-pixel being an integer between one and a defined upper limit, and a demultiplexer (240), operably coupled to the pixel slicer (230), that is configured to partition each of the one or more pixels into a plurality of pixel components.

2. The interface system (200) of claim 1, wherein the pixel slicer (230) is further configured to initiate a data request to the first processing system (110), to load the buffer (210) with a subsequent fixed number of bits, when the buffer (210) contains less than the programmable number of bits-per-pixel.

3. The interface system (200) of claim 1, further including an endianess converter (220), operably coupled to the buffer (210), that is configured to recorder the fixed number of bits from the first processing system (110).

4. The interface system (200) of claim 1, further including a channel swapper (250), operably coupled to the demultiplexer (240), that is configured to recorder the plurality of pixel components into predefined pixel channels.

5. A System (100) for processing graphics and/or video data, comprising: a first processing system (110) that is configured to accommodate a variety of pixel formats, and to store pixel data in a memory system (115) in the variety of pixel formats, the memory system (115) being configured to transfer a fixed number of bits upon receipt of a read command, a second processing system (120) that is configured to receive components of pixel data in a particular form that facilities rendering an image corresponding to the pixel data, the particular form of the second processing system (120) differing from at least one of the variety of pixel formats, and an interface system (200), operably coupled between the first processing system (110) and the second processing system (120), that is configured to receive the fixed number of bits from the memory system (115) corresponding to the pixel data and to provide therefrom the components of the pixel data in the particular form used by the second processing system (120).

6. A method of processing variable width pixel data via a fixed width memory access, comprising: receiving (310) a fixed number of bits corresponding to the fixed width memory acces, slicing (330) the fixed number of bits into one or more pixels, based on a given number of bits-per pixel, initiating (328) another memory access when the slicing of the fixed number of bits for a next pixel provides fewer than the given number of bits-per pixel, and partitioning (340) each pixel into one or more pixel channels to facilitate communication to a rendering device.

7. The method of claim 6, further including reformatting (360) component values in one or more of the one or more pixel channels for communication to the rendering device.

8. The method of claim 7, wherein reformatting (360) the component values includes at least one of: transformation from a color- value format of component values to a luminance- chrominance format of component values, and a transformation from a luminance-chrominance format of component values to a color-value format of component values.