US20050008230A1 - Image processing apparatus - Google Patents
Image processing apparatus Download PDFInfo
- Publication number
- US20050008230A1 US20050008230A1 US10/811,840 US81184004A US2005008230A1 US 20050008230 A1 US20050008230 A1 US 20050008230A1 US 81184004 A US81184004 A US 81184004A US 2005008230 A1 US2005008230 A1 US 2005008230A1
- Authority
- US
- United States
- Prior art keywords
- data
- image
- image processing
- buffer memory
- processing unit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T1/00—General purpose image data processing
- G06T1/60—Memory management
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
- H04N1/21—Intermediate information storage
- H04N1/2104—Intermediate information storage for one or a few pictures
- H04N1/2112—Intermediate information storage for one or a few pictures using still video cameras
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
- H04N1/21—Intermediate information storage
- H04N1/2104—Intermediate information storage for one or a few pictures
- H04N1/2112—Intermediate information storage for one or a few pictures using still video cameras
- H04N1/2137—Intermediate information storage for one or a few pictures using still video cameras with temporary storage before final recording, e.g. in a frame buffer
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
- H04N1/32—Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device, e.g. between a still-image camera and its memory or between a still-image camera and a printer device
- H04N1/32358—Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device, e.g. between a still-image camera and its memory or between a still-image camera and a printer device using picture signal storage, e.g. at transmitter
- H04N1/32491—Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device, e.g. between a still-image camera and its memory or between a still-image camera and a printer device using picture signal storage, e.g. at transmitter alternate storage in and retrieval from two parallel memories, e.g. using ping-pong buffers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/76—Television signal recording
- H04N5/765—Interface circuits between an apparatus for recording and another apparatus
- H04N5/77—Interface circuits between an apparatus for recording and another apparatus between a recording apparatus and a television camera
- H04N5/772—Interface circuits between an apparatus for recording and another apparatus between a recording apparatus and a television camera the recording apparatus and the television camera being placed in the same enclosure
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N2101/00—Still video cameras
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N2201/00—Indexing scheme relating to scanning, transmission or reproduction of documents or the like, and to details thereof
- H04N2201/32—Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device, e.g. between a still-image camera and its memory or between a still-image camera and a printer device
- H04N2201/3285—Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device, e.g. between a still-image camera and its memory or between a still-image camera and a printer device using picture signal storage, e.g. at transmitter
- H04N2201/329—Storage of less than a complete document page or image frame
- H04N2201/3292—Storage of less than a complete document page or image frame of one or two complete lines
Definitions
- the present invention relates to an image processing apparatus.
- FIGS. 7 and 8 each schematically show a flow of image data captured by a conventional digital camera.
- An interlace sensor 2 is used as a CCD sensor for image capturing in FIG. 7
- a progressive sensor 18 is used in FIG. 8 .
- image processing operation of a conventional digital camera will be discussed below.
- the interlace sensor 2 first converts captured information as light about a subject into a digital signal which is then outputted as image data.
- the outputted digital signal is subjected to noise control by correlated dual sampling, A/D conversion into a digital signal and the like at a signal processing unit 3 , and is thereafter sent to a main processor 4 .
- the main processor 4 comprises a sensor processing unit (hereinafter referred to as “SPU”) 6 for performing signal processing such as white balance control on the received image data.
- SPU sensor processing unit
- the processed image data is stored in a main memory 5 .
- the interlace sensor 2 outputs the captured image of one frame in two fields, namely, odd and even fields.
- image processing using the SPU 6 and later units requires peripheral pixel information, and hence, information only about the odd field or about the even field is not responsive to the image processing.
- subsequent process steps are put on hold until the interlace sensor 2 outputs the image data of two fields to reproduce the image data of one frame as original image data 16 on the main memory 5 .
- a real-time processing unit (hereinafter referred to as “RPU”) 7 performs signal processing such as interpolation, color space conversion and pseudo color suppression on the image data written to a line memory 8 from the main memory 5 .
- the image data processed by the RPU 7 is thereafter sent to an image compression and expansion unit 13 through line buffers A 24 and B 25 on the main memory 5 .
- the line buffers A and B each normally have a capacity of 8 lines.
- the RPU 7 continues to process subsequent image data read from the original image data 16 , which processed data is then sent to the line buffer B 25 .
- the image data sent in advance to the line buffer A 24 is concurrently sent to the image compression and expansion unit 13 and is subjected to processing therein.
- the RPU 7 continues to be operative to send subsequent image data to the empty line buffer A 24 , concurrently with which the image data sent in advance to the line buffer B 25 is sent to the image compression and expansion unit 13 . That is, the two line buffers A 24 and B 25 become alternately operative.
- the image data read from the line buffers A 24 and B 25 are subjected to compression into a predetermined format such as JPEG at the image compression and expansion unit 13 , and are thereafter sent as compressed image data 17 to the main memory 5 .
- the compressed image data 17 is subjected to processing by an image display unit 19 and a signal conversion unit 20 , whereby an image is displayed on a display device 21 , or the compressed image data 17 is stored in a memory card through a memory card interface.
- the image display unit 19 uses a display buffer 29 as a working buffer memory.
- the progressive sensor 18 as a CCD sensor for image capturing outputs the captured image of one frame at a time, and hence, generation of the original image data 16 on the main memory 5 is not required. Except for this, image processing using the progressive sensor 18 follows the same flow as the foregoing image processing using the interlace sensor 2 .
- the CCD sensors 2 and 18 each have a growing number of pixels per horizontal line, reaching such a length that exceeds the horizontal line length of the line memory 8 of the RPU 7 in some cases. On the occurrence of such excess, regardless of whether the CCD sensor 2 or 18 is used, the image data outputted from the CCD sensor 2 or 18 cannot be directly processed by the RPU 7 . In response, even in the use of the progressive sensor 18 , the original image data 16 is generated once on the main memory 5 , and thereafter the subsequent process steps are followed. More specifically, with reference to FIG.
- the original image data 16 is divided into left and right regions to avoid excess of the pixels per horizontal line of the original image data 16 over the horizontal line length of the line memory 8 of the RPU 7 .
- the left region including regional data P 1 through P (n/2) is processed first to generate a region 27 including regional data Q 1 through Q (n/2) .
- the right region including regional data P (n/2)+1 through P n is processed to generate a region 28 including regional data Q (n/2)+1 through Q n .
- two regional data R 1 and R 2 on the left and right are synthesized to form intermediate data 26 of one frame on the main memory 5 which is then processed by the image compression and expansion unit 13 .
- image data transmission should be repeated between the main processor 4 and the main memory 5 through a bus 14 . More specifically, a digital camera using the interlace sensor 2 requires five transmissions through the bus 14 as shown in FIG. 7 . Even a digital camera using the progressive sensor 18 requires three transmissions.
- the CCD sensors 2 and 18 have a horizontal line of pixels reaching a length that exceeds the horizontal line length of the line memory 8 of the RPU 7 , in which case the regional data R 1 and R 2 on the left and right separately processed are synthesized into the intermediate data 26 which is then held in the main memory 5 .
- the main memory 5 should be larger in capacity than required in the case in which the CCD sensors 2 and 18 contain pixels per horizontal line which do not reach the horizontal line length of the line memory 8 of the RPU 7 .
- the foregoing data transmissions are handled by a DMA control unit 15 which realizes high-speed DMA transmission through the bus 14 with no intervention of a CPU.
- the CCD sensors 2 and 18 are a 5-megapixel-class CCD sensor with 2560 pixels high by 1920 pixels wide each storing 16 bits of information, for example, the image data outputted from the CCD sensor 2 or 18 contains information as much as about 10 megabytes.
- repetition of transmission of such high-volume data causes heavily places a burden on a bus band, thus leading to increase in power consumption, lower processing speed, and the like.
- the present invention is intended for an image processing apparatus in which an image processing unit writes image data to a buffer memory, and a compression unit reads the image data from the buffer memory to perform compression thereon.
- the image processing apparatus comprises: an image processing part for performing image processing on captured data of an image; and a storage unit provided outside the image processing part.
- the image processing part includes: an image processing unit for performing a predetermined process on the captured data to obtain image data, and outputting the image data; a buffer memory for storing the image data sent from the image processing unit; and a compression unit for compressing the image data read from the buffer memory.
- the buffer memory includes two buffer memories.
- the image processing apparatus further comprises a control unit for controlling read and writing to and from these two buffer memories.
- the control unit is operative in such a manner that while the image processing unit writes image data to either one of the two buffer memories, the compression unit selectively reads image data previously stored in the other one of the two buffer memories.
- FIG. 1 schematically shows the configuration of a digital camera using an interlace sensor according to a first preferred embodiment of the present invention
- FIG. 2 schematically shows the configuration of a digital camera using a progressive sensor according to the first preferred embodiment
- FIG. 3 schematically shows the configuration of a digital camera according to a second preferred embodiment of the present invention
- FIG. 4 schematically shows the configuration of a digital camera according to a third preferred embodiment of the present invention.
- FIG. 5 schematically shows the configuration of a digital camera according to a fourth preferred embodiment of the present invention.
- FIG. 6 shows process flow of regional data according to the fourth preferred embodiment
- FIG. 7 schematically shows the configuration of a conventional digital camera using an interlace sensor
- FIG. 8 schematically shows the configuration of a conventional digital camera using a progressive sensor
- FIG. 9 shows conventional process flow of regional data.
- FIG. 1 schematically shows a digital camera comprising an image processing apparatus 1 according to a first preferred embodiment of the present invention.
- arrows indicate flow of image data captured by the digital camera.
- the image processing apparatus 1 comprises a main processor 4 and a main memory 5 .
- the image processing apparatus 1 is connected through a signal processing unit 3 to an interlace sensor 2 .
- the main processor 4 comprises an SPU 6 , an RPU 7 including a line memory 8 , an image compression and expansion unit 13 , a local buffer A 9 (first buffer memory) and a local buffer B 10 (second buffer memory) serving to transmit data between the RPU 7 and the image compression and expansion unit 13 , a write control unit 11 and a read control unit 12 for selecting which one of the two local buffers A 9 or B 10 to use, a bus 14 for data transmission, and a DMA control unit 15 for controlling data transmission through the bus 14 between the main processor 4 and the main memory 5 .
- the main memory 5 serves to temporarily store original image data 16 generated in the main processor 4 by data processing of odd and even fields sent from the interlace sensor 2 through the signal processing unit 3 , and compressed image data 17 sent from the image compression and expansion unit 13 .
- the digital camera of the first preferred embodiment has the same configuration as that of the conventional digital camera.
- the main processor 4 further comprises a CPU for various types of operations, an external memory card for storing captured image data, hardware serving as an interface for storage into the external memory card, and a storage device for storing a software program.
- the SPU 6 receives image data from the interlace sensor 2 through the signal processing unit 3 to generate the original image data 16 on the main memory 5 .
- the RPU 7 stores image data in the line memory 8 read from the original image data 16 and performs signal processing thereon such as interpolation, color space conversion and pseudo color suppression.
- the digital camera of the first preferred embodiment sends the processed image data to the local buffers A 9 and B 10 in the main processor 4 , not to the line buffers A 24 and B 25 in the main memory 5 .
- the processed image data of 8 lines is sent to the local buffer A 9 .
- the RPU 7 thereafter continues to process subsequent image data of 8 lines read from the original image data 16 , and sends the processed image data to the local buffer B 10 .
- the image data stored in the local buffer A 9 is sent to the image compression and expansion unit 13 for compression into a predetermined format such as JPEG.
- the compressed image data is thereafter sent to the main memory 5 .
- the RPU 7 still continues to process subsequent image data of 8 lines read from the original image data 16 and send the processed image data to the empty line buffer A 9 .
- the image data stored in the local buffer B 10 is concurrently sent to the image compression and expansion unit 13 for compression into a predetermined format such as JPEG.
- the compressed image data is thereafter sent to the main memory 5 . That is, while the RPU 7 sends image data either to the local buffer A 9 or to B 10 , image data stored either in the local buffer B 10 or in A 9 is sent to the image compression and expansion unit 13 to be compressed therein. These operations are repeatedly performed, whereby the compressed image data 17 is generated on the main memory 5 .
- FIG. 2 shows the configuration of a digital camera using a progressive sensor 18 as a CCD sensor. Except for the progressive sensor 18 , the digital camera shown in FIG. 2 has the same configuration as that of the digital camera of FIG. 1 . In FIG. 2 , constituent elements which are operative in the same manner as those of FIG. 1 are designated by the same reference numerals.
- the operation of the digital camera using the progressive sensor 18 is the same as that of the digital camera using the interlace sensor 2 , with exception that no original image data 16 is required, and data from the SPU 6 is directly sent to the RPU 7 to be processed therein.
- the write and read control units 11 and 12 select which one of the two local buffers A 9 or B 10 to use by making connection to the RPU 7 and the image compression and expansion unit 13 .
- the line buffers A 9 and B 10 are only required to be not less than image data processed at one time by the RPU 7 in horizontal line length. More specifically, as long as the local buffers A 9 and B 10 have a horizontal line not less than one horizontal line of pixels of the CCD sensors 2 and 18 , the local buffers A 9 and B 10 may be the same as or shorter than in horizontal line length the line memory 8 of the RPU 7 .
- the horizontal line length of the local buffers A 9 and B 10 may contain 2600 pixels or 4096 pixels, as long as it is not less than the length of 2560 pixels.
- the use of the local buffers A 9 and B 10 reduces the number of DMA transmissions of image data between the main processor 4 and the main memory 5 under control by the DMA control unit 15 until the compressed image data 17 is generated. That is, the conventional digital camera requires five transmissions in the use of the interlace sensor 2 and three transmissions in the use of the progressive sensor 18 as shown in FIGS. 7 and 8 , respectively, whereas with reference to FIGS. 1 and 2 , the digital camera of the first preferred embodiment only requires three transmissions and one transmission in the use of the interlace sensor 2 and the progressive sensor 18 , respectively. A burden placed on a bus band is lightened accordingly, whereby high-speed image processing is realized.
- the first preferred embodiment further advantageously serves to reduce the frequency of use of the main memory 5 generally operating at a higher voltage than the main processor 4 as well as to reduce the number of operations of the DMA control unit 15 , thereby reducing power consumption of the image processing apparatus 1 as a whole. Additionally, provision of two local buffers allows continuous process flow from the SPU 6 to the image compression and expansion unit 13 , thereby advantageously realizing high-speed image processing.
- the first preferred embodiment allows real-time data processing flow from the SPU 6 to the image compression and expansion unit 13 by way of the local buffers A 9 and B 10 without generating the original image data 16 once on the main memory 5 as shown in FIG. 2 .
- the CCD sensors 2 and 18 have a horizontal line of pixels that exceeds in length the horizontal line length of the local buffers A 9 and B 10 , however, data sent from the RPU 7 cannot completely be stored in the local buffers A 9 and B 10 , whereby such sequential processing becomes impossible.
- FIG. 3 schematically shows a digital camera comprising an image processing apparatus 1 according to the second preferred embodiment.
- arrows indicate flow of image data captured by the digital camera.
- the image processing apparatus 1 of the second preferred embodiment has the same configuration as that of the first preferred embodiment, whereas the local buffers A 9 and B 10 are operative in different ways between the first and second preferred embodiments.
- constituent elements which are operative in the same manner as those of FIGS. 1 and 2 are designated by the same reference numerals.
- the local buffers A 9 and B 10 are overflowed with data sent from the RPU 7 which prevents process flow.
- the two local buffers A 9 and B 10 together serve as one continuous buffer memory as shown in FIG. 3 .
- the horizontal line length of the local buffers A 9 and B 10 becomes responsive to data up to 5200 pixels per horizontal line which is twice the horizontal line length of each one of the local buffers A 9 and B 10 .
- the use of the local buffers A 9 and B 10 as one continuous buffer memory does not allow the RPU 7 and the image compression and expansion unit 13 to follow sequential process flow by means of the alternate use of the two local buffers A 9 and B 10 as discussed in the first preferred embodiment.
- a control signal is transmitted between the RPU 7 and the image compression and expansion unit 13 .
- the RPU 7 first processes image data of one line read from the original image data 16 , and sends the processed image data to the buffer memory as a unity of the two local buffers A 9 and B 10 .
- the RPU 7 stops the operation.
- the image compression and expansion unit 13 serves to process the data read from the buffer memory.
- the image compression and expansion unit 13 sends a control signal to the RPU 7 notifying completion of data readout.
- the RPU 7 again processes image data read from the original image data 16 , and sends the processed image data to the buffer memory.
- the second preferred embodiment is applicable even in the use of the progressive sensor 18 as a CCD sensor. More specifically, data from the SPU 6 is not directly sent to the RPU 7 , but is sent to the main memory 5 once to generate the original image data 16 .
- the two local buffers A 9 and B 10 are conceptually shown as one buffer memory in physical connection in a horizontal direction.
- the actual use of the local buffers A 9 and B 10 is such that under control by a software program, for example, the RPU 7 sends data to the local buffer A 9 , and overflow data from the local buffer A 9 is sequentially sent to the local buffer B 10 .
- This process flow is also followed in reading to the image compression and expansion unit 13 .
- the write and read control units 11 and 12 are collectively controlled, the second preferred embodiment is feasible in the configurations shown in FIGS. 1 and 2 . That is, the configurations of FIGS. 1 and 2 can be subjected to the process flows of the first and second preferred embodiments.
- the second preferred embodiment has been described in the use of the two local buffers A 9 and B 10 as discussed in the first preferred embodiment.
- one local buffer may be provided in the image processing apparatus 1 that has the same capacity as that of the line memory 8 of the RPU 7 .
- the second preferred embodiment is also feasible by the use of such a local buffer and the foregoing control signal.
- the local buffers A 9 and B 10 serve to transmit data from the RPU 7 to the image compression and expansion unit 13 .
- the local buffers A 9 and B 10 may be operative for another processing performed in the main processor 4 .
- the local buffers A 9 and B 10 are operative for image display on a display device 21 .
- a digital camera may use an electronic viewfinder or a liquid crystal display.
- image data sent from the CCD sensor 2 or 18 passes through the signal processing unit 3 and the SPU 6 , and is thereafter sent through the bus 14 to the image display unit 19 for conversion.
- the converted image data is further sent through the signal conversion unit 20 to the display device 21 such as an electronic viewfinder or a liquid crystal display, whereby an image is reproduced on the display device 21 .
- reproduction of the subject should be continuously made on the display device 21 , and should not be limited to a capturing period. Accordingly, data should be continuously sent to the image display unit 19 , thus heavily placing a burden on the bus 14 serving for this data transmission.
- the local buffers A 9 and B 10 serve to transmit data to the image display unit 19 to lighten a burden on a bus band.
- FIG. 4 schematically shows a digital camera comprising an image processing apparatus 1 according to the third preferred embodiment.
- arrows indicate flow 45 of image data captured by the digital camera.
- Constituent elements which are operative in the same manner as those of FIGS. 1, 2 and 3 are designated by the same reference numerals.
- Image data sent from the CCD sensor 2 or 18 is subjected to the processing at the SPU 6 .
- the processed image data is thereafter sent to the RPU 7 after generation of the original image data 16 in the use of the interlace sensor 2 , whereas in the use of the progressive sensor 18 , it is directly sent to the RPU 7 .
- the captured image data from the RPU 7 is sent through the local buffers A 9 and B 10 to the image compression and expansion unit 13 and to the image display unit 19 .
- the captured image data is subjected to the processing at the image compression and expansion unit 13 during image capturing, whereas in visual recognition of a subject to be captured, it is subjected to the processing at the image display unit 19 .
- image data sent from the RPU 7 during visual recognition is skipped with 240 pixels high by 320 pixels wide, for example.
- the skipped image data is subjected to the sequential process flow as discussed in the first preferred embodiment including writing from the RPU 7 and reading to the image display unit 19 by means of alternate use of the two local buffers A 9 and B 10 under control by the write and read control units 11 and 12 .
- Continuity between the RPU 7 and the local buffers A 9 and B 10 , and continuity between the local buffers A 9 and B 10 , and the image compression and expansion unit 13 and the image display unit 19 , are controlled by purpose-built hardware and a purpose-built software program having a switching function of continuity such as the write and read control units 11 and 12 .
- the output from the RPU 7 is sent to the local buffers A 9 and B 10 , and the output from the local buffers A 9 and B 10 is selectively sent either to the image compression and expansion unit 13 or to the image display unit 19 . That is, the local buffers A 9 and B 10 are selectively available either to the image compression and expansion unit 13 or to the image display unit 19 . More specifically, between the image compression and expansion unit 13 and the image display unit 19 , the local buffers A 9 and B 10 are allocated to the one placing a heavier burden on the bus 14 .
- data from the RPU 7 is directly sent through the local buffers A 9 and B 10 to the image display unit 19 without data transmission through the bus 14 for image reproduction as required in the conventional digital camera.
- a burden placed on a bus band is lightened accordingly, to thereby reduce power consumption.
- the lightened burden on a bus band further advantageously leads to enhanced processing speed of the image processing apparatus 1 as a whole.
- the original image data 16 is divided into left and right regions. These left and right regions are separately processed at the RPU 7 by means of the line memory 8 to generate the intermediate data 26 on the main memory 5 .
- the intermediate data 26 is subsequently subjected to the processing at the image compression and expansion unit 13 to generate the compressed image data 17 .
- Such process flow can handle the CCD sensors 2 and 18 having a horizontal line of pixels exceeding in length the horizontal line length of the line memory 8 of the RPU 7 , whereas a region for storing the high-volume intermediate data 26 is required in the main memory 5 that contains pixels corresponding in number to those of the CCD sensors 2 and 18 .
- a region for storing the high-volume intermediate data 26 is required in the main memory 5 that contains pixels corresponding in number to those of the CCD sensors 2 and 18 .
- FIG. 5 schematically shows a digital camera comprising an image processing apparatus 1 according to the fourth preferred embodiment.
- arrows indicate flow of image data captured by the digital camera.
- FIG. 6 is an explanatory view sequentially showing generation of data on the main memory 5 . With reference to FIGS. 5 and 6 , the operation of the digital camera according to the fourth preferred embodiment will be discussed below.
- the original image data 16 is generated once in the main memory 5 regardless of whether the CCD sensor 2 or 18 is used.
- the original image data 16 is divided and is then subjected to the processing at the RPU 7 by means of the line memory 8 .
- the original image data 16 is divided into left and right regions in two columns. The left and right regions are each further horizontally divided into regions of 8 lines.
- intermediate data R 1 ( 23 ) of 8 lines is generated in the main memory 5 that has a horizontal line length of pixels corresponding in number to those of the horizontal line of the CCD sensors 2 and 18 .
- the image compression and expansion unit 13 handles data on a block unit basis of 8 pixels high by 8 pixels wide, and hence, is capable of handling data R 1 to generate data S 1 . These process steps are sequentially performed until the compressed image data 17 is generated.
- the RPU 7 processes the regional data including P 1 , P 2 , . . . , and the resultant data R 1 , R 2 , . . . are subjected to the processing at the image compression and expansion unit 13 .
- the CCD sensors 2 and 18 have a horizontal line of pixels exceeding in length the horizontal line length of the line memory 8 , process flow continues without waiting for generation of intermediate data of one frame on the main memory 5 as required in the conventional digital camera.
- the fourth preferred embodiment further advantageously lowers working buffer capacity in the main memory 5 . Still further, provision of two lines buffers A 22 and B 23 in the main memory 5 , and alternate use of these buffers A 22 and B 23 as discussed with reference to the local buffers A 9 and B 10 in the first and second preferred embodiments realize image processing with still higher speed.
- the CCD sensors 2 and 18 may have a horizontal line of pixels exceeding in length the line memory 8 of the RPU 7 as a result of higher pixel resolution of the CCD sensors 2 and 18 , in which case the fourth preferred embodiment devises reading of image data from the original image data 16 to realize image processing.
- the fourth preferred embodiment requires a region for storing intermediate data in the main memory 5 , thereby placing a heavier burden on the bus 14 and causing higher power consumption than the first and second preferred embodiments using the local buffers A 9 and B 10 .
- a fifth preferred embodiment of the present invention as a combination of the second and fourth preferred embodiments is responsive to these problems.
- the fifth preferred embodiment employs the one shown in FIG. 3 discussed in the second preferred embodiment.
- image processing only the reading flow of the original image data 16 discussed in the fourth preferred embodiment with reference to FIG. 6 is adopted.
- the original image data 16 generated on the main memory 5 after passing through the CCD sensor 2 or 18 and the SPU 6 is read as the regional data P 1 and P 2 in this order as shown in FIG. 6 .
- the regional data P 1 and P 2 thereby read are subjected to the processing at the RPU 7 by means of the line memory 8 .
- the processed data are sent to the local buffers A 9 and B 10 , not to the line buffer A 22 in the main memory 5 .
- the data R 1 ( 23 ) shown in FIG. 6 is generated in the local buffers A 9 and B 10 .
- the RPU 7 stops reading of the original image data 16 .
- the image compression and expansion unit 13 reads the data from the local buffers A 9 and B 10 and performs compression thereon.
- the compressed data is thereafter sent as the compressed image data 17 to the main memory 5 .
- the data S 1 shown in FIG. 6 is generated as the compressed image data 17 on the main memory 5 .
- the image compression and expansion unit 13 sends a control signal notifying completion of data readout.
- the RPU 7 continues to read the regional data P 3 and P 4 from the original image data 16 , and performs image processing thereon.
- the processed data are sent to the local buffers A 9 and B 10 .
- the fifth preferred embodiment has been discussed as a combination of the fourth and second preferred embodiments. As long as the CCD sensors 2 and 18 have a horizontal line of pixels that does not exceed in length the horizontal line length of the local buffers A 9 and B 10 , the first preferred embodiment may alternatively be combined with the fourth preferred embodiment. Still alternatively, a combination of the third and fourth preferred embodiments will be applicable.
- the use of the local buffers A 9 and B 10 of the second preferred embodiment combined with the fourth preferred embodiment eliminates the need for generating intermediate data on the main memory 5 .
- Such combination further advantageously lightens a burden to be placed on the bus 14 and reduces power consumption.
Abstract
Description
- 1.Field of the Invention
- The present invention relates to an image processing apparatus.
- 2.Description of the Background Art
- In response to a recent trend towards higher pixel resolution of a digital camera, higher transmission speed and higher image processing speed of captured image data have been required.
FIGS. 7 and 8 each schematically show a flow of image data captured by a conventional digital camera. Aninterlace sensor 2 is used as a CCD sensor for image capturing inFIG. 7 , whereas aprogressive sensor 18 is used inFIG. 8 . With reference toFIGS. 7 and 8 , image processing operation of a conventional digital camera will be discussed below. - With reference to
FIG. 7 , when a CCD sensor for image capturing is theinterlace sensor 2, theinterlace sensor 2 first converts captured information as light about a subject into a digital signal which is then outputted as image data. The outputted digital signal is subjected to noise control by correlated dual sampling, A/D conversion into a digital signal and the like at asignal processing unit 3, and is thereafter sent to amain processor 4. Themain processor 4 comprises a sensor processing unit (hereinafter referred to as “SPU”) 6 for performing signal processing such as white balance control on the received image data. Next, the processed image data is stored in amain memory 5. - The
interlace sensor 2 outputs the captured image of one frame in two fields, namely, odd and even fields. However, image processing using theSPU 6 and later units requires peripheral pixel information, and hence, information only about the odd field or about the even field is not responsive to the image processing. In response, subsequent process steps are put on hold until theinterlace sensor 2 outputs the image data of two fields to reproduce the image data of one frame asoriginal image data 16 on themain memory 5. - After the image data of one frame is reproduced as the
original image data 16, a real-time processing unit (hereinafter referred to as “RPU”) 7 performs signal processing such as interpolation, color space conversion and pseudo color suppression on the image data written to aline memory 8 from themain memory 5. The image data processed by theRPU 7 is thereafter sent to an image compression andexpansion unit 13 through line buffers A24 and B25 on themain memory 5. For compression into the JPEG (Joint Photographic Experts Group) format, the line buffers A and B each normally have a capacity of 8 lines. - More specifically, after the processed image data is sent to the line buffer A24, the
RPU 7 continues to process subsequent image data read from theoriginal image data 16, which processed data is then sent to the line buffer B25. The image data sent in advance to the line buffer A24 is concurrently sent to the image compression andexpansion unit 13 and is subjected to processing therein. The RPU 7 continues to be operative to send subsequent image data to the empty line buffer A24, concurrently with which the image data sent in advance to the line buffer B25 is sent to the image compression andexpansion unit 13. That is, the two line buffers A24 and B25 become alternately operative. - The image data read from the line buffers A24 and B25 are subjected to compression into a predetermined format such as JPEG at the image compression and
expansion unit 13, and are thereafter sent ascompressed image data 17 to themain memory 5. Thecompressed image data 17 is subjected to processing by animage display unit 19 and asignal conversion unit 20, whereby an image is displayed on adisplay device 21, or thecompressed image data 17 is stored in a memory card through a memory card interface. Theimage display unit 19 uses adisplay buffer 29 as a working buffer memory. - Unlike the
interlace sensor 2, theprogressive sensor 18 as a CCD sensor for image capturing outputs the captured image of one frame at a time, and hence, generation of theoriginal image data 16 on themain memory 5 is not required. Except for this, image processing using theprogressive sensor 18 follows the same flow as the foregoing image processing using theinterlace sensor 2. - As a result of higher pixel resolution, the
CCD sensors line memory 8 of theRPU 7 in some cases. On the occurrence of such excess, regardless of whether theCCD sensor CCD sensor RPU 7. In response, even in the use of theprogressive sensor 18, theoriginal image data 16 is generated once on themain memory 5, and thereafter the subsequent process steps are followed. More specifically, with reference toFIG. 9 , theoriginal image data 16 is divided into left and right regions to avoid excess of the pixels per horizontal line of theoriginal image data 16 over the horizontal line length of theline memory 8 of theRPU 7. The left region including regional data P1 through P(n/2) is processed first to generate aregion 27 including regional data Q1 through Q(n/2). Next, the right region including regional data P(n/2)+1 through Pn is processed to generate aregion 28 including regional data Q(n/2)+1 through Qn. Finally, two regional data R1 and R2 on the left and right are synthesized to formintermediate data 26 of one frame on themain memory 5 which is then processed by the image compression andexpansion unit 13. - As discussed, for compression of a subject image captured by a digital camera into a predetermined format such as JPEG to be displayed on a display device or stored in a memory card, image data transmission should be repeated between the
main processor 4 and themain memory 5 through abus 14. More specifically, a digital camera using theinterlace sensor 2 requires five transmissions through thebus 14 as shown inFIG. 7 . Even a digital camera using theprogressive sensor 18 requires three transmissions. - As discussed, the
CCD sensors line memory 8 of theRPU 7, in which case the regional data R1 and R2 on the left and right separately processed are synthesized into theintermediate data 26 which is then held in themain memory 5. As a result, themain memory 5 should be larger in capacity than required in the case in which theCCD sensors line memory 8 of theRPU 7. - The foregoing data transmissions are handled by a
DMA control unit 15 which realizes high-speed DMA transmission through thebus 14 with no intervention of a CPU. However, when theCCD sensors CCD sensor CCD sensors - The present invention is intended for an image processing apparatus in which an image processing unit writes image data to a buffer memory, and a compression unit reads the image data from the buffer memory to perform compression thereon.
- According to the present invention, the image processing apparatus comprises: an image processing part for performing image processing on captured data of an image; and a storage unit provided outside the image processing part. The image processing part includes: an image processing unit for performing a predetermined process on the captured data to obtain image data, and outputting the image data; a buffer memory for storing the image data sent from the image processing unit; and a compression unit for compressing the image data read from the buffer memory.
- Accordingly, high-speed processing of the image data is realized without using the storage unit. Reduction in power consumption is allowed as well.
- According to one aspect of the present invention, the buffer memory includes two buffer memories. The image processing apparatus further comprises a control unit for controlling read and writing to and from these two buffer memories. The control unit is operative in such a manner that while the image processing unit writes image data to either one of the two buffer memories, the compression unit selectively reads image data previously stored in the other one of the two buffer memories. As a result, the image processing unit and the compression unit are both allowed to perform sequential image processing without the need for waiting for completion of data transmission to the buffer memories.
- It is therefore an object of the present invention to provide an image processing apparatus realizing high-speed image processing with low power consumption.
- These and other objects, features, aspects and advantages of the present invention will become more apparent from the following detailed description of the present invention when taken in conjunction with the accompanying drawings.
-
FIG. 1 schematically shows the configuration of a digital camera using an interlace sensor according to a first preferred embodiment of the present invention; -
FIG. 2 schematically shows the configuration of a digital camera using a progressive sensor according to the first preferred embodiment; -
FIG. 3 schematically shows the configuration of a digital camera according to a second preferred embodiment of the present invention; -
FIG. 4 schematically shows the configuration of a digital camera according to a third preferred embodiment of the present invention; -
FIG. 5 schematically shows the configuration of a digital camera according to a fourth preferred embodiment of the present invention; -
FIG. 6 shows process flow of regional data according to the fourth preferred embodiment; -
FIG. 7 schematically shows the configuration of a conventional digital camera using an interlace sensor; -
FIG. 8 schematically shows the configuration of a conventional digital camera using a progressive sensor; and -
FIG. 9 shows conventional process flow of regional data. -
FIG. 1 schematically shows a digital camera comprising animage processing apparatus 1 according to a first preferred embodiment of the present invention. InFIG. 1 , arrows indicate flow of image data captured by the digital camera. With reference toFIG. 1 , theimage processing apparatus 1 comprises amain processor 4 and amain memory 5. Theimage processing apparatus 1 is connected through asignal processing unit 3 to aninterlace sensor 2. - The
main processor 4 comprises anSPU 6, anRPU 7 including aline memory 8, an image compression andexpansion unit 13, a local buffer A9 (first buffer memory) and a local buffer B10 (second buffer memory) serving to transmit data between theRPU 7 and the image compression andexpansion unit 13, awrite control unit 11 and aread control unit 12 for selecting which one of the two local buffers A9 or B10 to use, abus 14 for data transmission, and aDMA control unit 15 for controlling data transmission through thebus 14 between themain processor 4 and themain memory 5. - The
main memory 5 serves to temporarily storeoriginal image data 16 generated in themain processor 4 by data processing of odd and even fields sent from theinterlace sensor 2 through thesignal processing unit 3, andcompressed image data 17 sent from the image compression andexpansion unit 13. - Except for the two local buffers A9 and B10 provided between the
RPU 7 and the image compression andexpansion unit 13, and operation of themain memory 5 as a line buffer, the digital camera of the first preferred embodiment has the same configuration as that of the conventional digital camera. - Though not specifically shown, the
main processor 4 further comprises a CPU for various types of operations, an external memory card for storing captured image data, hardware serving as an interface for storage into the external memory card, and a storage device for storing a software program. - The operation of the
image processing apparatus 1 having the foregoing configuration will be discussed below. - In the
main processor 4, theSPU 6 receives image data from theinterlace sensor 2 through thesignal processing unit 3 to generate theoriginal image data 16 on themain memory 5. Next, theRPU 7 stores image data in theline memory 8 read from theoriginal image data 16 and performs signal processing thereon such as interpolation, color space conversion and pseudo color suppression. These process steps are the same as those followed by the conventional digital camera discussed with reference toFIG. 7 . Unlike the conventional digital camera, the digital camera of the first preferred embodiment sends the processed image data to the local buffers A9 and B10 in themain processor 4, not to the line buffers A24 and B25 in themain memory 5. - More specifically, the processed image data of 8 lines is sent to the local buffer A9. The
RPU 7 thereafter continues to process subsequent image data of 8 lines read from theoriginal image data 16, and sends the processed image data to the local buffer B10. Concurrent with transmission of the image data from theRPU 7 to the local buffer B10, the image data stored in the local buffer A9 is sent to the image compression andexpansion unit 13 for compression into a predetermined format such as JPEG. The compressed image data is thereafter sent to themain memory 5. TheRPU 7 still continues to process subsequent image data of 8 lines read from theoriginal image data 16 and send the processed image data to the empty line buffer A9. The image data stored in the local buffer B10 is concurrently sent to the image compression andexpansion unit 13 for compression into a predetermined format such as JPEG. The compressed image data is thereafter sent to themain memory 5. That is, while theRPU 7 sends image data either to the local buffer A9 or to B10, image data stored either in the local buffer B10 or in A9 is sent to the image compression andexpansion unit 13 to be compressed therein. These operations are repeatedly performed, whereby thecompressed image data 17 is generated on themain memory 5. -
FIG. 2 shows the configuration of a digital camera using aprogressive sensor 18 as a CCD sensor. Except for theprogressive sensor 18, the digital camera shown inFIG. 2 has the same configuration as that of the digital camera ofFIG. 1 . InFIG. 2 , constituent elements which are operative in the same manner as those ofFIG. 1 are designated by the same reference numerals. The operation of the digital camera using theprogressive sensor 18 is the same as that of the digital camera using theinterlace sensor 2, with exception that nooriginal image data 16 is required, and data from theSPU 6 is directly sent to theRPU 7 to be processed therein. - The write and read
control units RPU 7 and the image compression andexpansion unit 13. - The line buffers A9 and B10 are only required to be not less than image data processed at one time by the
RPU 7 in horizontal line length. More specifically, as long as the local buffers A9 and B10 have a horizontal line not less than one horizontal line of pixels of theCCD sensors line memory 8 of theRPU 7. By way of example, when theCCD sensors line memory 8 of theRPU 7 has a horizontal line length with 4096 pixels, the horizontal line length of the local buffers A9 and B10 may contain 2600 pixels or 4096 pixels, as long as it is not less than the length of 2560 pixels. - As discussed, the use of the local buffers A9 and B10 reduces the number of DMA transmissions of image data between the
main processor 4 and themain memory 5 under control by theDMA control unit 15 until thecompressed image data 17 is generated. That is, the conventional digital camera requires five transmissions in the use of theinterlace sensor 2 and three transmissions in the use of theprogressive sensor 18 as shown inFIGS. 7 and 8 , respectively, whereas with reference toFIGS. 1 and 2 , the digital camera of the first preferred embodiment only requires three transmissions and one transmission in the use of theinterlace sensor 2 and theprogressive sensor 18, respectively. A burden placed on a bus band is lightened accordingly, whereby high-speed image processing is realized. The first preferred embodiment further advantageously serves to reduce the frequency of use of themain memory 5 generally operating at a higher voltage than themain processor 4 as well as to reduce the number of operations of theDMA control unit 15, thereby reducing power consumption of theimage processing apparatus 1 as a whole. Additionally, provision of two local buffers allows continuous process flow from theSPU 6 to the image compression andexpansion unit 13, thereby advantageously realizing high-speed image processing. - Under the conditions that the
progressive sensor 18 is used as a CCD sensor, and theprogressive sensor 18 has a horizontal line of pixels that does not exceed in length the horizontal line of the local buffers A9 and B10, the first preferred embodiment allows real-time data processing flow from theSPU 6 to the image compression andexpansion unit 13 by way of the local buffers A9 and B10 without generating theoriginal image data 16 once on themain memory 5 as shown inFIG. 2 . When theCCD sensors RPU 7 cannot completely be stored in the local buffers A9 and B10, whereby such sequential processing becomes impossible. - In a second preferred embodiment of the present invention, it will be discussed what process a digital camera follows to handle the case in which the
CCD sensors -
FIG. 3 schematically shows a digital camera comprising animage processing apparatus 1 according to the second preferred embodiment. InFIG. 3 , arrows indicate flow of image data captured by the digital camera. Theimage processing apparatus 1 of the second preferred embodiment has the same configuration as that of the first preferred embodiment, whereas the local buffers A9 and B10 are operative in different ways between the first and second preferred embodiments. InFIG. 3 , constituent elements which are operative in the same manner as those ofFIGS. 1 and 2 are designated by the same reference numerals. - As an example, assuming that the
CCD sensors line memory 8 of theRPU 7 contains 4096 pixels per horizontal line, and that the local buffers A9 and B10 each have 2600 pixels per horizontal line responsive only to a 5-megapixel-class CCD with 2560 pixels per horizontal line, the local buffers A9 and B10 are overflowed with data sent from theRPU 7 which prevents process flow. In response, in the second preferred embodiment, the two local buffers A9 and B10 together serve as one continuous buffer memory as shown inFIG. 3 . As a result, the horizontal line length of the local buffers A9 and B10 becomes responsive to data up to 5200 pixels per horizontal line which is twice the horizontal line length of each one of the local buffers A9 and B10. - The use of the local buffers A9 and B10 as one continuous buffer memory does not allow the
RPU 7 and the image compression andexpansion unit 13 to follow sequential process flow by means of the alternate use of the two local buffers A9 and B10 as discussed in the first preferred embodiment. In response, a control signal is transmitted between theRPU 7 and the image compression andexpansion unit 13. - More specifically, the
RPU 7 first processes image data of one line read from theoriginal image data 16, and sends the processed image data to the buffer memory as a unity of the two local buffers A9 and B10. At the time when data transmission is completed, theRPU 7 stops the operation. The image compression andexpansion unit 13 serves to process the data read from the buffer memory. After receipt of all the data in the buffer memory, the image compression andexpansion unit 13 sends a control signal to theRPU 7 notifying completion of data readout. On receipt of this control signal, theRPU 7 again processes image data read from theoriginal image data 16, and sends the processed image data to the buffer memory. - The second preferred embodiment is applicable even in the use of the
progressive sensor 18 as a CCD sensor. More specifically, data from theSPU 6 is not directly sent to theRPU 7, but is sent to themain memory 5 once to generate theoriginal image data 16. - In
FIG. 3 , the two local buffers A9 and B10 are conceptually shown as one buffer memory in physical connection in a horizontal direction. The actual use of the local buffers A9 and B10 is such that under control by a software program, for example, theRPU 7 sends data to the local buffer A9, and overflow data from the local buffer A9 is sequentially sent to the local buffer B10. This process flow is also followed in reading to the image compression andexpansion unit 13. When the write and readcontrol units FIGS. 1 and 2 . That is, the configurations ofFIGS. 1 and 2 can be subjected to the process flows of the first and second preferred embodiments. - The second preferred embodiment has been described in the use of the two local buffers A9 and B10 as discussed in the first preferred embodiment. As the simplest way to use a buffer memory, one local buffer may be provided in the
image processing apparatus 1 that has the same capacity as that of theline memory 8 of theRPU 7. The second preferred embodiment is also feasible by the use of such a local buffer and the foregoing control signal. - As discussed, even when the
CCD sensors CCD sensors RPU 7 and the image compression andexpansion unit 13 does not require themain memory 5 and thebus 14. A burden placed on a bus band is lightened accordingly, whereby high-speed image processing and reduction in power consumption are realized. - In the first and second preferred embodiments, the local buffers A9 and B10 serve to transmit data from the
RPU 7 to the image compression andexpansion unit 13. The local buffers A9 and B10 may be operative for another processing performed in themain processor 4. In a third preferred embodiment of the present invention, the local buffers A9 and B10 are operative for image display on adisplay device 21. - For visual recognition of a subject to be captured, a digital camera may use an electronic viewfinder or a liquid crystal display. With reference to
FIGS. 7 and 8 , in the conventional way of such visual recognition, image data sent from theCCD sensor signal processing unit 3 and theSPU 6, and is thereafter sent through thebus 14 to theimage display unit 19 for conversion. The converted image data is further sent through thesignal conversion unit 20 to thedisplay device 21 such as an electronic viewfinder or a liquid crystal display, whereby an image is reproduced on thedisplay device 21. - For the purpose of visual recognition of a subject to be captured, reproduction of the subject should be continuously made on the
display device 21, and should not be limited to a capturing period. Accordingly, data should be continuously sent to theimage display unit 19, thus heavily placing a burden on thebus 14 serving for this data transmission. - In response, in the third preferred embodiment, the local buffers A9 and B10 serve to transmit data to the
image display unit 19 to lighten a burden on a bus band. -
FIG. 4 schematically shows a digital camera comprising animage processing apparatus 1 according to the third preferred embodiment. InFIG. 4 , arrows indicate flow 45 of image data captured by the digital camera. Constituent elements which are operative in the same manner as those ofFIGS. 1, 2 and 3 are designated by the same reference numerals. - Image data sent from the
CCD sensor SPU 6. The processed image data is thereafter sent to theRPU 7 after generation of theoriginal image data 16 in the use of theinterlace sensor 2, whereas in the use of theprogressive sensor 18, it is directly sent to theRPU 7. The captured image data from theRPU 7 is sent through the local buffers A9 and B10 to the image compression andexpansion unit 13 and to theimage display unit 19. The captured image data is subjected to the processing at the image compression andexpansion unit 13 during image capturing, whereas in visual recognition of a subject to be captured, it is subjected to the processing at theimage display unit 19. During visual recognition of a subject to be captured, display on thedisplay device 21 is required to be at such a level that allows recognition of a subjected to be captured, and does not require data corresponding to millions of pixels sent from theCCD sensor RPU 7 during visual recognition is skipped with 240 pixels high by 320 pixels wide, for example. The skipped image data is subjected to the sequential process flow as discussed in the first preferred embodiment including writing from theRPU 7 and reading to theimage display unit 19 by means of alternate use of the two local buffers A9 and B10 under control by the write and readcontrol units - Continuity between the
RPU 7 and the local buffers A9 and B10, and continuity between the local buffers A9 and B10, and the image compression andexpansion unit 13 and theimage display unit 19, are controlled by purpose-built hardware and a purpose-built software program having a switching function of continuity such as the write and readcontrol units - The output from the
RPU 7 is sent to the local buffers A9 and B10, and the output from the local buffers A9 and B10 is selectively sent either to the image compression andexpansion unit 13 or to theimage display unit 19. That is, the local buffers A9 and B10 are selectively available either to the image compression andexpansion unit 13 or to theimage display unit 19. More specifically, between the image compression andexpansion unit 13 and theimage display unit 19, the local buffers A9 and B10 are allocated to the one placing a heavier burden on thebus 14. - As discussed, data from the
RPU 7 is directly sent through the local buffers A9 and B10 to theimage display unit 19 without data transmission through thebus 14 for image reproduction as required in the conventional digital camera. A burden placed on a bus band is lightened accordingly, to thereby reduce power consumption. The lightened burden on a bus band further advantageously leads to enhanced processing speed of theimage processing apparatus 1 as a whole. - As discussed in the description of the background art with reference to
FIG. 9 , in response to the case in which theCCD sensors line memory 8 of theRPU 7, theoriginal image data 16 is divided into left and right regions. These left and right regions are separately processed at theRPU 7 by means of theline memory 8 to generate theintermediate data 26 on themain memory 5. Theintermediate data 26 is subsequently subjected to the processing at the image compression andexpansion unit 13 to generate thecompressed image data 17. Such process flow can handle theCCD sensors line memory 8 of theRPU 7, whereas a region for storing the high-volumeintermediate data 26 is required in themain memory 5 that contains pixels corresponding in number to those of theCCD sensors original image data 16 is processed without requiring such high-volume intermediate data. -
FIG. 5 schematically shows a digital camera comprising animage processing apparatus 1 according to the fourth preferred embodiment. InFIG. 5 , arrows indicate flow of image data captured by the digital camera. Constituent elements which are operative in the same manner as those ofFIGS. 1, 2 , 3 and 4 designated by the same reference numerals.FIG. 6 is an explanatory view sequentially showing generation of data on themain memory 5. With reference toFIGS. 5 and 6 , the operation of the digital camera according to the fourth preferred embodiment will be discussed below. - When the
CCD sensors line memory 8 of theRPU 7, and thus process flow cannot sequentially proceed from theSPU 6 to theRPU 7, theoriginal image data 16 is generated once in themain memory 5 regardless of whether theCCD sensor original image data 16 is divided and is then subjected to the processing at theRPU 7 by means of theline memory 8. With reference toFIG. 6 , theoriginal image data 16 is divided into left and right regions in two columns. The left and right regions are each further horizontally divided into regions of 8 lines. Hence, assuming that theCCD sensors RPU 7 becomes operative even when theline memory 8 has a capacity of as small as 4096 pixels. Regional data P1 shown inFIG. 6 after being subjected to the processing at theRPU 7 is written as data Q1 to a line buffer A22 in themain memory 5. Unlike the conventional way which subsequently processes regional data P3, regional data P2 is subsequently processed and is added as data Q2 to the line buffer A22 on themain memory 5. As a result, intermediate data R1 (23) of 8 lines is generated in themain memory 5 that has a horizontal line length of pixels corresponding in number to those of the horizontal line of theCCD sensors expansion unit 13 handles data on a block unit basis of 8 pixels high by 8 pixels wide, and hence, is capable of handling data R1 to generate data S1. These process steps are sequentially performed until thecompressed image data 17 is generated. - As discussed, the
RPU 7 processes the regional data including P1, P2, . . . , and the resultant data R1, R2, . . . are subjected to the processing at the image compression andexpansion unit 13. Even when theCCD sensors line memory 8, process flow continues without waiting for generation of intermediate data of one frame on themain memory 5 as required in the conventional digital camera. The fourth preferred embodiment further advantageously lowers working buffer capacity in themain memory 5. Still further, provision of two lines buffers A22 and B23 in themain memory 5, and alternate use of these buffers A22 and B23 as discussed with reference to the local buffers A9 and B10 in the first and second preferred embodiments realize image processing with still higher speed. - The
CCD sensors line memory 8 of theRPU 7 as a result of higher pixel resolution of theCCD sensors original image data 16 to realize image processing. However, the fourth preferred embodiment requires a region for storing intermediate data in themain memory 5, thereby placing a heavier burden on thebus 14 and causing higher power consumption than the first and second preferred embodiments using the local buffers A9 and B10. A fifth preferred embodiment of the present invention as a combination of the second and fourth preferred embodiments is responsive to these problems. - As a circuit configuration of a digital camera, the fifth preferred embodiment employs the one shown in
FIG. 3 discussed in the second preferred embodiment. As to image processing, only the reading flow of theoriginal image data 16 discussed in the fourth preferred embodiment with reference toFIG. 6 is adopted. - More specifically, the
original image data 16 generated on themain memory 5 after passing through theCCD sensor SPU 6 is read as the regional data P1 and P2 in this order as shown inFIG. 6 . The regional data P1 and P2 thereby read are subjected to the processing at theRPU 7 by means of theline memory 8. The processed data are sent to the local buffers A9 and B10, not to the line buffer A22 in themain memory 5. At this stage, the data R1 (23) shown inFIG. 6 is generated in the local buffers A9 and B10. At the time when data transmission is completed, theRPU 7 stops reading of theoriginal image data 16. Next, the image compression andexpansion unit 13 reads the data from the local buffers A9 and B10 and performs compression thereon. The compressed data is thereafter sent as thecompressed image data 17 to themain memory 5. As this stage, the data S1 shown inFIG. 6 is generated as thecompressed image data 17 on themain memory 5. At the time when data readout from the local buffers A9 and B10 is completed, the image compression andexpansion unit 13 sends a control signal notifying completion of data readout. On receipt of this control signal, theRPU 7 continues to read the regional data P3 and P4 from theoriginal image data 16, and performs image processing thereon. The processed data are sent to the local buffers A9 and B10. These process steps are repeatedly performed, whereby only thecompressed image data 17 is generated on themain memory 5 without generating theintermediate data 26. - The fifth preferred embodiment has been discussed as a combination of the fourth and second preferred embodiments. As long as the
CCD sensors - As discussed, the use of the local buffers A9 and B10 of the second preferred embodiment combined with the fourth preferred embodiment eliminates the need for generating intermediate data on the
main memory 5. Such combination further advantageously lightens a burden to be placed on thebus 14 and reduces power consumption. - While the invention has been shown and described in detail, the foregoing description is in all aspects illustrative and not restrictive. It is therefore understood that numerous modifications and variations can be devised without departing from the scope of the invention.
Claims (11)
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US13/230,068 US8531546B2 (en) | 2003-03-31 | 2011-09-12 | Image processing apparatus having a buffer memory for image data storage |
US13/944,173 US9082205B2 (en) | 2003-03-31 | 2013-07-17 | Image processing apparatus for processing image data to produce compressed image data and display data |
US13/944,238 US9082206B2 (en) | 2003-03-31 | 2013-07-17 | Image processing apparatus having a buffer memory for image data storage |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2003-093075 | 2003-03-31 | ||
JP2003093075A JP4245139B2 (en) | 2003-03-31 | 2003-03-31 | Image processing device |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/230,068 Division US8531546B2 (en) | 2003-03-31 | 2011-09-12 | Image processing apparatus having a buffer memory for image data storage |
Publications (1)
Publication Number | Publication Date |
---|---|
US20050008230A1 true US20050008230A1 (en) | 2005-01-13 |
Family
ID=33405953
Family Applications (4)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/811,840 Abandoned US20050008230A1 (en) | 2003-03-31 | 2004-03-30 | Image processing apparatus |
US13/230,068 Expired - Lifetime US8531546B2 (en) | 2003-03-31 | 2011-09-12 | Image processing apparatus having a buffer memory for image data storage |
US13/944,238 Expired - Lifetime US9082206B2 (en) | 2003-03-31 | 2013-07-17 | Image processing apparatus having a buffer memory for image data storage |
US13/944,173 Expired - Lifetime US9082205B2 (en) | 2003-03-31 | 2013-07-17 | Image processing apparatus for processing image data to produce compressed image data and display data |
Family Applications After (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/230,068 Expired - Lifetime US8531546B2 (en) | 2003-03-31 | 2011-09-12 | Image processing apparatus having a buffer memory for image data storage |
US13/944,238 Expired - Lifetime US9082206B2 (en) | 2003-03-31 | 2013-07-17 | Image processing apparatus having a buffer memory for image data storage |
US13/944,173 Expired - Lifetime US9082205B2 (en) | 2003-03-31 | 2013-07-17 | Image processing apparatus for processing image data to produce compressed image data and display data |
Country Status (2)
Country | Link |
---|---|
US (4) | US20050008230A1 (en) |
JP (1) | JP4245139B2 (en) |
Cited By (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20060101116A1 (en) * | 2004-10-28 | 2006-05-11 | Danny Rittman | Multifunctional telephone, walkie talkie, instant messenger, video-phone computer, based on WiFi (Wireless Fidelity) and WiMax technology, for establishing global wireless communication, network and video conferencing via the internet |
US20060140498A1 (en) * | 2004-12-28 | 2006-06-29 | Fujitsu Limited | Apparatus and method for processing an image |
US20060146076A1 (en) * | 2004-12-08 | 2006-07-06 | Chung-Hsun Huang | Image processing module with less line buffers |
US20070065022A1 (en) * | 2005-09-22 | 2007-03-22 | Nec Electronics Corporation | Image signal processing apparatus and method |
US20080316221A1 (en) * | 2007-06-19 | 2008-12-25 | Aas Eric F | Down-sampled image display |
US20130083233A1 (en) * | 2011-10-04 | 2013-04-04 | Sony Corporation | Image pickup unit |
CN103140873A (en) * | 2010-09-29 | 2013-06-05 | 英泰克普拉斯有限公司 | Method and device for transmitting/receiving image data at high speed |
TWI413943B (en) * | 2009-02-11 | 2013-11-01 | Silicon Motion Inc | Image processing system and method thereof |
US10674154B2 (en) * | 2009-10-05 | 2020-06-02 | Beamr Imaging Ltd | Apparatus and methods for recompression of digital images |
Families Citing this family (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP4925271B2 (en) | 2006-08-21 | 2012-04-25 | 株式会社メガチップス | Continuous imaging method for single-lens reflex digital camera |
EP2302845B1 (en) | 2009-09-23 | 2012-06-20 | Google, Inc. | Method and device for determining a jitter buffer level |
US8856212B1 (en) | 2011-02-08 | 2014-10-07 | Google Inc. | Web-based configurable pipeline for media processing |
TW201242346A (en) * | 2011-04-12 | 2012-10-16 | Altek Corp | Image capturing device and image processing method thereof |
JP5103678B2 (en) * | 2011-04-27 | 2012-12-19 | 株式会社メガチップス | Continuous imaging method for single-lens reflex digital camera |
TWI520598B (en) * | 2012-05-23 | 2016-02-01 | 晨星半導體股份有限公司 | Image processing apparatus and image processing method |
US8917336B2 (en) * | 2012-05-31 | 2014-12-23 | Apple Inc. | Image signal processing involving geometric distortion correction |
US8928680B1 (en) | 2012-07-10 | 2015-01-06 | Google Inc. | Method and system for sharing a buffer between a graphics processing unit and a media encoder |
CN103414877A (en) * | 2013-08-01 | 2013-11-27 | 天津天地伟业数码科技有限公司 | Image frame multiplication processing method based on high-speed memory dynamic burst length |
Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5694166A (en) * | 1991-09-04 | 1997-12-02 | Canon Kabushiki Kaisha | Video camera with slower than standard read-out from CCD |
US6002446A (en) * | 1997-02-24 | 1999-12-14 | Paradise Electronics, Inc. | Method and apparatus for upscaling an image |
US6137534A (en) * | 1997-07-10 | 2000-10-24 | Flashpoint Technology, Inc. | Method and apparatus for providing live view and instant review in an image capture device |
US6233010B1 (en) * | 1990-11-20 | 2001-05-15 | St. Clair Intellectual Property Consultants, Inc. | Electronic still video camera with direct personal computer (PC) compatible digital format output |
US20010019362A1 (en) * | 2000-02-22 | 2001-09-06 | Kenji Nakamura | Digital camera and recording medium |
US6400471B1 (en) * | 1999-02-11 | 2002-06-04 | Flashpoint Technology, Inc. | Flexible architecture for image processing |
US6563535B1 (en) * | 1998-05-19 | 2003-05-13 | Flashpoint Technology, Inc. | Image processing system for high performance digital imaging devices |
US6847388B2 (en) * | 1999-05-13 | 2005-01-25 | Flashpoint Technology, Inc. | Method and system for accelerating a user interface of an image capture unit during play mode |
Family Cites Families (14)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPS61164377A (en) | 1985-01-16 | 1986-07-25 | Sanyo Electric Co Ltd | Coding and decoding system |
JPH05128232A (en) | 1991-11-08 | 1993-05-25 | Mitsubishi Electric Corp | Filing method for image and image processor |
US5450599A (en) * | 1992-06-04 | 1995-09-12 | International Business Machines Corporation | Sequential pipelined processing for the compression and decompression of image data |
JPH06245013A (en) | 1993-02-19 | 1994-09-02 | Murata Mach Ltd | Facsimile equipment |
US7123772B2 (en) * | 2001-03-28 | 2006-10-17 | Texas Instruments Incorporated | Image compression by differences within a strip |
JPH09245157A (en) | 1996-03-12 | 1997-09-19 | Canon Inc | Image forming device |
US5745603A (en) * | 1996-08-26 | 1998-04-28 | Hewlett-Packard Company | Two dimensional context model obtained without a line buffer for arithmetic coding |
KR100314801B1 (en) * | 1998-12-22 | 2002-01-15 | 박종섭 | Apparatus for panning and scaling window in image sensor |
JP3532781B2 (en) * | 1999-02-12 | 2004-05-31 | 株式会社メガチップス | Image processing circuit of image input device |
JP3961736B2 (en) | 2000-02-04 | 2007-08-22 | 株式会社リコー | Image processing device |
JP2002125105A (en) | 2000-10-13 | 2002-04-26 | Ricoh Co Ltd | Image processor, image processing method, and computer- readable storage medium with program stored to execute the method |
JP2002237951A (en) | 2001-02-09 | 2002-08-23 | Matsushita Electric Ind Co Ltd | Video signal processing circuit |
JP4112187B2 (en) * | 2001-03-01 | 2008-07-02 | 富士フイルム株式会社 | Image processing method, apparatus, and program |
JP2003244529A (en) * | 2002-02-20 | 2003-08-29 | Minolta Co Ltd | Digital camera |
-
2003
- 2003-03-31 JP JP2003093075A patent/JP4245139B2/en not_active Expired - Lifetime
-
2004
- 2004-03-30 US US10/811,840 patent/US20050008230A1/en not_active Abandoned
-
2011
- 2011-09-12 US US13/230,068 patent/US8531546B2/en not_active Expired - Lifetime
-
2013
- 2013-07-17 US US13/944,238 patent/US9082206B2/en not_active Expired - Lifetime
- 2013-07-17 US US13/944,173 patent/US9082205B2/en not_active Expired - Lifetime
Patent Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6233010B1 (en) * | 1990-11-20 | 2001-05-15 | St. Clair Intellectual Property Consultants, Inc. | Electronic still video camera with direct personal computer (PC) compatible digital format output |
US5694166A (en) * | 1991-09-04 | 1997-12-02 | Canon Kabushiki Kaisha | Video camera with slower than standard read-out from CCD |
US6002446A (en) * | 1997-02-24 | 1999-12-14 | Paradise Electronics, Inc. | Method and apparatus for upscaling an image |
US6137534A (en) * | 1997-07-10 | 2000-10-24 | Flashpoint Technology, Inc. | Method and apparatus for providing live view and instant review in an image capture device |
US6563535B1 (en) * | 1998-05-19 | 2003-05-13 | Flashpoint Technology, Inc. | Image processing system for high performance digital imaging devices |
US6400471B1 (en) * | 1999-02-11 | 2002-06-04 | Flashpoint Technology, Inc. | Flexible architecture for image processing |
US6847388B2 (en) * | 1999-05-13 | 2005-01-25 | Flashpoint Technology, Inc. | Method and system for accelerating a user interface of an image capture unit during play mode |
US20010019362A1 (en) * | 2000-02-22 | 2001-09-06 | Kenji Nakamura | Digital camera and recording medium |
Cited By (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20060101116A1 (en) * | 2004-10-28 | 2006-05-11 | Danny Rittman | Multifunctional telephone, walkie talkie, instant messenger, video-phone computer, based on WiFi (Wireless Fidelity) and WiMax technology, for establishing global wireless communication, network and video conferencing via the internet |
US20060146076A1 (en) * | 2004-12-08 | 2006-07-06 | Chung-Hsun Huang | Image processing module with less line buffers |
US8269805B2 (en) * | 2004-12-08 | 2012-09-18 | Himax Technologies Limited | Image processing module with less line buffers |
US7583851B2 (en) * | 2004-12-28 | 2009-09-01 | Fujitsu Microelectronics Limited | Apparatus and method for processing an image |
US20060140498A1 (en) * | 2004-12-28 | 2006-06-29 | Fujitsu Limited | Apparatus and method for processing an image |
US20070065022A1 (en) * | 2005-09-22 | 2007-03-22 | Nec Electronics Corporation | Image signal processing apparatus and method |
US20080316221A1 (en) * | 2007-06-19 | 2008-12-25 | Aas Eric F | Down-sampled image display |
TWI413943B (en) * | 2009-02-11 | 2013-11-01 | Silicon Motion Inc | Image processing system and method thereof |
US10674154B2 (en) * | 2009-10-05 | 2020-06-02 | Beamr Imaging Ltd | Apparatus and methods for recompression of digital images |
CN103140873A (en) * | 2010-09-29 | 2013-06-05 | 英泰克普拉斯有限公司 | Method and device for transmitting/receiving image data at high speed |
US20130176443A1 (en) * | 2010-09-29 | 2013-07-11 | Intekplus Co., Ltd | Method and device for transmitting/receiving image data at high speed |
US9218050B2 (en) * | 2010-09-29 | 2015-12-22 | Intekplus Co., Ltd. | Method and device for transmitting/receiving image data at high speed |
US20130083233A1 (en) * | 2011-10-04 | 2013-04-04 | Sony Corporation | Image pickup unit |
Also Published As
Publication number | Publication date |
---|---|
JP4245139B2 (en) | 2009-03-25 |
US20130300756A1 (en) | 2013-11-14 |
US8531546B2 (en) | 2013-09-10 |
US20130300898A1 (en) | 2013-11-14 |
JP2004304387A (en) | 2004-10-28 |
US9082206B2 (en) | 2015-07-14 |
US20120002080A1 (en) | 2012-01-05 |
US9082205B2 (en) | 2015-07-14 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US9082206B2 (en) | Image processing apparatus having a buffer memory for image data storage | |
JP4993856B2 (en) | Image conversion device, direct memory access device for image conversion, and camera interface supporting image conversion | |
KR101128167B1 (en) | Image display device | |
JP4263190B2 (en) | Video composition circuit | |
US6593965B1 (en) | CCD data pixel interpolation circuit and digital still camera equipped with it | |
EP1531399A2 (en) | Method and system for preventing data transfer bus bottlenecks through image compression | |
US7023455B2 (en) | Apparatus and method for processing video signal | |
US6356306B1 (en) | Digital camera capable of converting a progressive scan signal into an interlace scan signal | |
JP2000092375A (en) | Signal processing unit and its control method | |
JP6722278B2 (en) | Image processing device | |
US6661452B1 (en) | Digital camera capable of decreasing a required memory capacity | |
US7391932B2 (en) | Apparatus and method for selecting image to be displayed | |
JP4158245B2 (en) | Signal processing device | |
US8085345B2 (en) | Image processing apparatus and image processing method | |
CN113658049A (en) | Image transposition method, equipment and computer readable storage medium | |
JP3178665B2 (en) | Image size conversion method and device therefor | |
JP2000224540A (en) | Picture file device | |
JP2002101310A (en) | Filter processing unit and method | |
US20100254618A1 (en) | Method for Accessing Image Data and Related Apparatus | |
JP2000092349A (en) | Image processor | |
JP4211572B2 (en) | Imaging device | |
JP2007124075A (en) | Image processing apparatus and method, and image recording apparatus and method | |
JP2004328405A (en) | Moving image recording and reproducing device, its still image compression recording method, and program | |
JP4439746B2 (en) | Information processing device | |
JP2002237953A (en) | Image data processing unit and its method, and camera system |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: MEGA CHIPS COPORATION, JAPAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:SASAKI, GEN;REEL/FRAME:015161/0450 Effective date: 20040322 |
|
AS | Assignment |
Owner name: MEGA CHIPS CORPORATION, JAPAN Free format text: RECORD TO CORRECT THE ASSIGNEE'S NAME AND ADDRESS ON AN ASSIGNMENT PREVIOUSLY RECORDED ON REEL/FRAME 015161/0450;ASSIGNOR:SASAKI, GEN;REEL/FRAME:015856/0728 Effective date: 20040322 |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |