WO1999018719A1

WO1999018719A1 - Method of and apparatus for transmitting scaled and compressed raw ccd video data from a video camera

Info

Publication number: WO1999018719A1
Application number: PCT/US1998/020674
Authority: WO
Inventors: Michael Shinsky; Michael Yukelson; Leonid Milenky
Original assignee: Sony Electronics Inc.
Priority date: 1997-10-07
Filing date: 1998-09-28
Publication date: 1999-04-15
Also published as: AU9781898A

Abstract

Raw CCD (12) video data is transmitted from a video camera (100) to a host computer (200). The video camera does not include a digital signal processing circuit for processing the raw video data into the appropriate video format. The raw CCD video data is processed and converted into the appropriate video format for display by the host computer after it is received from the video camera. Before transmission from the video camera, the raw video data for each frame is separated into four original color planes, each of which is transmitted separately to the host computer. The raw video data is preferably separated by writing the frame data into a memory (104) and then reading the data for each color plane separately from the memory. Each of the original color planes consists of its own color component and can be scaled and compressed separately, as appropriate before transmission from the video camera to the host computer. At the host computer, when the separated, scaled and compressed video data is received, it is decompressed. After the decompression of the received data, the decompressed data is then combined from the four original color planes into the raw video data representing the frame output from the CCD. The host computer then converts the received raw video data into the appropriate video format for display (36).

Description

METHOD OF AND APPARATUS FOR TRANSMITTING SCALED AND COMPRESSED RAW CCD VIDEO DATA FROM A VIDEO CAMERA

FIELD OF THE INVENTION: The present invention relates to the field of transmission of video data. More particularly, the present invention relates to the field of transmission, scaling and compression of video data from a charge-coupled device.

BACKGROUND OF THE INVENTION: A schematic block diagram of a configuration including a video camera and a host computer is illustrated in Figure 1. The video camera 10 includes a charge-coupled device (CCD) and is coupled to the host computer 20 for providing video data from the video camera 10 to the host computer 20. Within the video camera 10, the CCD 12 is coupled to a timing chip 14 which provides a clocking signal to the CCD 12. The CCD 12 is also coupled to a sample and hold and analog-to-digital converter circuit 16. The CCD 12 provides image data at a rate determined by the timing chip 14 to the sample and hold and analog-to-digital converter circuit 16. This image data is then sampled and converted into a digital format by the sample and hold and analog-to-digital converter circuit 16. The sample and hold and analog-to-digital converter circuit 16 is coupled to a digital signal processing (DSP) circuit 18. The DSP circuit 18 receives the digital data from the sample and hold and analog-to-digital converter circuit 16 and converts it into an appropriate video format, such as RGB, YC_RC_B, NTSC, PAL or any other appropriate format. The DSP circuit 18 is then coupled to an interface circuit 19 for providing the video data for transmission from the video camera 10 to a device coupled to the video camera 10. In the configuration illustrated in Figure 1, the video camera 10 is coupled to a host computer 20 through which the video data transmitted from the video camera 10 can be displayed on an associated display 36, saved and/or transmitted to another device. The interface circuit 19 of the video camera 10 is coupled to an interface circuit 28 of the host computer 20 by a bus or cable for transmitting the video data from the video camera 10 to the host computer 20. The host computer system 20, illustrated in Figure 1, is exemplary only and includes a central processor unit (CPU) 42, a main memory 30, a video graphics adapter (VGA) card 22, a mass storage device 32 and an interface circuit 28, all coupled together by a conventional bidirectional system bus 34. The mass storage device 32 may include both fixed and removable media using any one or more of magnetic, optical or magneto-optical storage technology or any other available mass storage technology. The system bus 34 contains an address bus for addressing any portion of the memory 30. The system bus 34 also includes a data bus for transferring data between and among the CPU

42, the main memory 30, the VGA card 22, the mass storage device 32 and the interface circuit 28.

The host computer system 20 is also coupled to a number of peripheral input and output devices including the keyboard 38, the mouse 40 and the associated display 36. The keyboard 38 is coupled to the CPU 42 for allowing a user to input data and control commands into the computer system 20. A conventional mouse 40 is coupled to the keyboard 38 for manipulating graphic images on the display 36 as a cursor control device.

The VGA card 22 interfaces between the components within the computer system 20 and the display 36. The VGA card 22 converts data received from the components within the computer system 20 into signals which are used by the display 36 to generate images for display.

In the configuration illustrated in Figure 1, the data read out from the CCD 12 is provided to the sample and hold and analog-to-digital converter circuit 16 where it is converted into a digital format. This digital data is raw video data representing the data read out from the CCD 12. This digital data from the sample and hold and analog-to- digital converter circuit 16 is provided to the DSP circuit 18 where it is converted into the appropriate video data format before it is transmitted from the video camera 10. As described above, the appropriate video format can include RGB, YC_RC_B, NTSC, PAL or any other appropriate format. Assuming eight (8) bit resolution per each color component or raw data, data in the RGB format requires twenty-four (24) bits per pixel and therefore three times as much bandwidth for transmission as the raw video data. Correspondingly, data in the YC_RC_B (4:2:2) format requires sixteen (16) bits per pixel and therefore one and a half to two times as much bandwidth for transmission compared to the raw video data. In systems with ever increasing image size, pixel density and limited transmission bandwidth capabilities, it is desirable to transmit the raw video data from a video camera because it is the format requiring the lowest data rate and correspondingly, the least bandwidth. Typical systems which receive video data from devices such as a video camera, however, are not equipped to process the raw video data and convert it into the appropriate format for display. Accordingly, transmission of the raw video data from a

CCD camera to a receiving device is not used in typical systems.

The CCD 12 typically includes a Yellow-Cyan-Magenta-Green mosaic color filter, as is well known in the art. Using this mosaic filter, the CCD 12 captures color images and outputs data representing the color images. The color image data from the CCD 12 is combined into a tile structure, as illustrated in Figure 2. This tile structure 50 provides raw video data representing the image captured by the CCD 12. From this tile structure 50, the luminance and chrominance components for the video data are obtained. Within the tile structure 50, pixels representing different colors are arranged adjacent to each other.

The raw video data in the tile structure of the CCD color space cannot be scaled or compressed without violating the tile structure of the frame represented by the raw video data. Compression requires a correlation of data between adjacent pixels, either horizontally or vertically. Data in the tile structure of the CCD color space does include some correlation. However, in its regular format, direct manipulation of raw data pixels in the tile structure of the CCD color space will lead to severe color and luminance errors, due to the adjacent relationship of colored pixels of different colors within the tile structure.

What is needed is a method of and apparatus for transmitting raw video data in a format which can be scaled and compressed. What is further needed is a method of and apparatus for transmitting raw video data to a receiving device which can appropriately convert and process the received data into the proper video format.

SUMMARY OF THE INVENTION:

Raw CCD video data is transmitted from a video camera to a host computer. The video camera does not include a digital signal processing circuit for processing the raw video data into the appropriate video format. The raw CCD video data is processed and converted into the appropriate video format for display by the host computer after it is received from the video camera. Before transmission from the video camera, the raw video data for each frame is separated into four original color planes, each of which is transmitted separately to the host computer. The raw video data is preferably separated by writing the frame data into a memory and then reading the data for each color plane separately from the memory. Each of the original color planes consists of its own color component and can be scaled and compressed separately, as appropriate, before transmission from the video camera to the host computer. At the host computer, when the separated, scaled and compressed video data is received, it is decompressed. After decompression of the received data, the decompressed data is then combined from the four original color planes into the raw video data representing the frame output from the CCD. The host computer then converts the received raw video data into the appropriate video format for display.

BRIEF DESCRIPTION OF THE DRAWINGS:

Figure 1 illustrates a schematic block diagram of a configuration including a CCD video camera and a host computer of the prior art.

Figure 2 illustrates a tile structure of a frame of color pixels read out from a CCD.

Figure 3 illustrates a schematic block diagram of a configuration of the preferred embodiment of the present invention including a CCD video camera and a host computer.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT:

A video camera of the present invention includes the capability to transmit compressed and scaled raw CCD video data representing frames of images captured by the CCD. The video camera does not include a DSP circuit for processing and converting the raw video data into the appropriate video format. The raw video data is transmitted from the video camera to a host computer where it is processed and converted into the appropriate video format for display and/or transmission by the host computer. Before transmission from the video camera, the raw video data for each frame is separated into four original color planes, each of which is transmitted separately to the host computer. The four original color planes are (Cyan + Magenta), (Green + Yellow), (Magenta + Yellow) and (Cyan + Green). The four original color planes represent raw video data.

The raw video data is preferably separated by writing the frame data into a memory and then reading the data for the pixels of each color plane separately. Each of the color planes consists of its own color component and can be scaled and compressed separately before transmission from the video camera to the host computer. At the host computer, when the separated, scaled and compressed video data is received, it is decompressed and the four color planes for each frame are reconstructed. The data is then combined from the four original color planes into the raw video data representing the frame output from the

CCD. The host computer then converts the received raw video data into the appropriate video format for display, storage and/or transmission.

A schematic block diagram of a configuration of the preferred embodiment of the present invention, including a CCD video camera and a host computer, is illustrated in Figure 3. The video camera 100 includes the CCD 12 and is coupled to the host computer

200 for providing video data from the video camera 100 to the host computer 200. The video camera 100 does not include a DSP circuit for processing and converting the raw video data into an appropriate video format such as RGB, YC_RC_B, NTSC or PAL. Within the video camera 100, the CCD 12, the timing chip 14 and the sample and hold and analog-to-digital converter circuit 16 are all in the same configuration and perform the same functions as described above with respect to the video camera 10, illustrated in Figure 1. As described above, the CCD 12 provides image data to the sample and hold and analog-to-digital converter circuit 16 representing images captured by the CCD 12. This image data is then sampled and converted into a digital format by the sample and hold and analog-to-digital converter circuit 16. The sample and hold and analog-to-digital converter circuit 16 is coupled to the universal serial bus (USB) interface circuit 102. The digital raw video data from the sample and hold and analog-to-digital converter circuit 16 is provided to the USB interface circuit 102.

Within the video camera 100, the USB interface circuit 102 is coupled to the memory circuit 104. Preferably, the memory circuit 104 is a dynamic random access memory. Alternatively, the memory circuit 104 is any appropriate memory circuit or device. The raw video data from the sample and hold and analog-to-digital converter circuit 16 is written into the memory circuit 104 from the USB interface circuit 102. After the data for a complete video frame is written into the memory circuit 104, the raw video data is read from the memory circuit 104 for each color plane by separately reading the pixel data corresponding to each color plane. After the pixel data for each color plane in the frame is read from the memory 104, the pixel data within the color plane is scaled and compressed as necessary for the particular implementation. The scaling and compression of the color planes is achieved by any appropriate available technique and scheme that would be apparent to those skilled in the art. Each color plane consists of the same color component and therefore there is a correlation between adjoining pixels, allowing the pixel data of each separate color plane to be scaled and compressed without destroying the tile structure of the video data.

Once each color plane is scaled and compressed, if scaling and compression is required by the system, the data representing each separate color plane is transmitted to the host computer 200. Preferably, the USB 106 is used to transmit the data from the USB interface circuit 102 in the video camera 100 to the USB interface circuit 202 in the host computer 200. Alternatively, any other appropriate connection can be used to transmit data from the video camera 100 to the host computer 200. When received at the host computer 200, the data is preferably written into the main memory 30 and then processed by the host computer 200. The host computer 200 includes the software and components necessary to process and convert the received data. The received data is decompressed by the host computer 200, if necessary. The decompressed data representing the four original color planes of the frame is then combined into the tile structure of the raw video data, representing the frame image output from the CCD 12. This raw video data is then processed and converted by the host computer 200 into the appropriate video format, e.g., RGB, YC_RC_B, NTSC, PAL or any other appropriate format. After this conversion, the data is then displayed by the host computer 36, saved in the mass storage device 32 or transmitted to another device, as appropriate.

When received by the host computer 200, the four original color planes are preferably decompressed separately, before they are combined into the tile structure of the frame of raw video data. Once the original color planes are combined into this tile structure, the host computer 200 then performs the appropriate steps to convert raw video data into the appropriate video format. This is the conversion that was performed by the DSP 18 in the video camera of Figure 1. The appropriate steps for this conversion depend on the video format to which the data is being converted and are well known by those skilled in the art. Preferably, the conversion from raw video data to the appropriate video format is performed by software resident in the host computer 200. Alternatively, any other appropriate conversion implementation can be used, as will be apparent to those skilled in the art. While the preferred receiver of the raw video data is the host computer 200, this data can also be transmitted to, received and converted by any other appropriately configured device.

In the preferred embodiment of the present invention, the original frame tile structure is written into and read from the memory 104 in order to separate the frame into the four original color planes. As will be apparent to those skilled in the art, there are many other methods and implementations which can be used to separate the raw video data into the four original color planes.

The present invention has been described in terms of specific embodiments incorporating details to facilitate the understanding of principles of construction and operation of the invention. Such reference herein to specific embodiments and details thereof is not intended to limit the scope of the claims appended hereto. It will be apparent to those skilled in the art that modifications may be made in the embodiment chosen for illustration without departing from the spirit and scope of the invention. Specifically, while the preferred embodiment of the present invention receives image data from a charge-coupled device, it should be apparent to those skilled in the art that this image data can alternatively be received from any appropriate imaging device, including a CMOS device.

Claims

C L A I M SWe Claim:

1. A method of transmitting raw video data from an imaging device to a receiving device comprising the steps of: a. receiving pixel data representing pixels within a frame of data captured by the imaging device; b. separating the pixel data into a plurality of color planes, each color plane representing a single color component within the frame of data; and c. transmitting the plurality of color planes separately to the receiving device.

2. The method as claimed in claim 1 further comprising the steps of: a. combining the separately transmitted plurality of color planes into a received and combined frame of data at the receiving device; and b. converting the received and combined frame of data into an appropriate video format.

3. The method as claimed in claim 2 wherein the step of separating the pixel data into the plurality of color planes is completed by storing the pixel data into a memory and reading out the pixel data from the memory separately for each color plane.

4. The method as claimed in claim 3 further comprising the step of scaling each of the plurality of color planes before the color planes are transmitted.

5. The method as claimed in claim 3 further comprising the step of compressing each of the plurality of color planes before the color planes are transmitted.

6. The method as claimed in claim 2 wherein the appropriate video format includes RGB, YC_RC_B, NTSC and PAL formats.

7. The method as claimed in claim 1 wherein the pixel data is received in a digital format.

8. The method as claimed in claim 1 wherein the imaging device is a charge- coupled device.

9. An apparatus for transmitting raw video data from an imaging device to a receiving device comprising: a. a receiving circuit for receiving pixel data representing pixels within a frame of data captured by the imaging device; b. a separating circuit coupled to the receiving circuit for separating the pixel data into a plurality of color planes, each color plane representing a single color component within the frame of data; and c. a transmitting circuit coupled to the separating circuit for transmitting the plurality of color planes separately to the receiving device.

10. The apparatus as claimed in claim 9 wherein the separating circuit includes a memory for storing the pixel data and reading the pixel data from the memory in each of the plurality of color planes separately.

11. The apparatus as claimed in claim 10 wherein the transmitting circuit includes the ability to scale and compress each of the plurality of color planes separately before the color planes are transmitted.

12. The apparatus as claimed in claim 11 wherein the receiving device further comprises: a. means for combining the separately transmitted plurality of color planes into a received and combined frame of data; and b. means for converting the received and combined frame of data into an appropriate video format.

13. The apparatus as claimed in claim 12 wherein the imaging device is a charge-coupled device.

14. A method of transmitting raw video data from an imaging device to a receiving device comprising the steps of: a. receiving pixel data representing pixels within a frame of data captured by the imaging device; b. separating the pixel data into a plurality of color planes, each color plane representing a single color component within the frame of data; c. reducing each of the separate color planes thereby forming reduced color planes; and d. transmitting the reduced color planes separately to the receiving device.

15. The method as claimed in claim 14 further comprising the steps of: a. combining the separately transmitted reduced color planes into a received and combined frame of data at the receiving device; and b. converting the received and combined frame of data into an appropriate video format.

16. The method as claimed in claim 15 wherein the step of separating the pixel data into a plurality of color planes is completed by storing the pixel data into a memory and reading out the pixel data from the memory separately for each color plane.

17. The method as claimed in claim 16 wherein the step of reducing includes separately scaling each of the plurality of color planes.

18. The method as claimed in claim 17 wherein the step of reducing further includes separately compressing each of the plurality of color planes.

19. The method as claimed in claim 16 wherein the step of reducing includes separately compressing each of the plurality of color planes.

20. The method as claimed in claim 14 wherein the imaging device is a charge- coupled device.

21. A video camera for capturing and transmitting raw video data to a receiving device comprising: a. an imaging device for capturing frame images; b. a receiving circuit coupled to the imaging device for receiving pixel data representing pixels within the frame images captured by the imaging device; c. a separating circuit coupled to the receiving circuit for separating the pixel data into a plurality of color planes, each of the plurality of color planes representing a single color component within the frame of data; and d. a transmitting circuit coupled to the separating circuit for transmitting the plurality of color planes separately to the receiving device.

22. The video camera as claimed in claim 21 wherein the separating circuit includes a memory for storing the pixel data and reading the pixel data from the memory separately for each of the plurality of color planes.

23. The video camera as claimed in claim 22 wherein the transmitting circuit includes the ability to separately scale and compress the plurality of color planes before the plurality of color planes are transmitted.

24. The video camera as claimed in claim 23 wherein the receiving device is a host computer and includes the ability to combine the separately transmitted color planes into a received and combined frame of data and convert the received and combined frame of data into an appropriate video format.

25. The video camera as claimed in claim 24 wherein the imaging device is a charge-coupled device.