WO2007049899A1

WO2007049899A1 - Method of controlling automatic exposure of image sensor

Info

Publication number: WO2007049899A1
Application number: PCT/KR2006/004342
Authority: WO
Inventors: Sung Su Lee
Original assignee: Pixelplus Co., Ltd
Priority date: 2005-10-24
Filing date: 2006-10-24
Publication date: 2007-05-03
Also published as: KR20070044257A; KR100725837B1

Abstract

A method of controlling automatic exposure of an image sensor is provided. The method includes (a) calculating current average brightness by dividing a whole image into several zones and giving weights to the divided zones, respectively, (b) comparing and checking the current average brightness and target brightness, calculating modified target brightness by reflecting a weight based on a value of exposure control speed, (d) calculating the amount of exposure to be applied to the next frame using the modified target brightness, (e) calculating a charge integration time and a value of a gain to be applied to the next frame based on the amount of the exposure, and (f) calculating a converted gain using the value of the gain to be applied to the next frame for the application to a practical amplification circuit.

Description

METHOD OF CONTROLLING AUTOMATIC EXPOSURE OF

IMAGE SENSOR

Technical Field

[1] The present invention relates to an image sensor, and more particularly, to a method of controlling exposure of an image sensor.

Background Art

[2] The term 'exposure' according to the present invention conceptually includes both charge integration time and a gain. The charge integration time is a time interval between a time point when a pixel starts receiving light after the pixel is reset and a time point when the amount of accumulated charge is read. The gain indicates degree of amplifying the charge generated in proportion to the charge integration time by an analog/digital method. When the light is sufficient, generally the gain is set to one, and the exposure control is performed only by adjusting the charge integration time. However, under an environment in which the light is insufficient, the gain greater than one is applied additionally to obtain an image having proper brightness, since an image having sufficient brightness cannot be obtained even when the exposure time is maximized.

[3] FlG. 1 is a diagram illustrating relation between an image sensor and an automatic exposure controller.

[4] Referring to FlG. 1, the automatic exposure controller receives image data from the image sensor, processes the image data, and sends determined adequate charge integration time and a gain to the sensor. Disclosure of Invention Technical Problem

[5] The present invention relates to an exposure control algorithm in which average brightness of an image can reach a target value. The object of the present invention is to provide a method in which complicated exposure control functions such as controlling flicker of a fluorescent lamp and changing in a frame height are performed in a simple and consistent way using only the amount of exposure as a variable. Technical Solution

[6] According to an aspect of the present invention, there is provided a method of controlling automatic exposure of an image sensor comprising:

[7] (a) calculating current average brightness by dividing a whole image into several zones and giving weights to the divided zones, respectively; (b) comparing and checking the current average brightness and target brightness; (c) calculating modified target brightness by reflecting a weight based on a value of exposure control speed; (d) calculating the amount of exposure to be applied to the next frame using the modified target brightness; (e) calculating a charge integration time and a value of a gain to be applied to the next frame based on the amount of the exposure; and (f) calculating a converted gain using the value of the gain to be applied to the next frame for the application to a practical amplification circuit.

Brief Description of the Drawings

[8] The above and other features and advantages of the present invention will become more apparent by describing in detail exemplary embodiments thereof with reference to the attached drawings in which: [9] FlG. 1 is a diagram illustrating relation between an image sensor and an automatic exposure controller; [10] FlG. 2 is a flowchart illustrating an algorithm of automatic exposure control according to the present invention;

[11] FlG. 3 is a diagram illustrating the whole image divided into two zones;

[12] FlG. 4 is a graph illustrating a basic assumption that the average brightness is in linear proportion to the exposure; [13] FlG. 5 is a graph illustrating a final value of the target exposure under the basic assumption illustrated in FlG. 4; [14] FlG. 6 is a graph illustrating relation between the amount of the exposure to be applied to the next image and the charge integration time and the gain; [15] FlG. 7 is a graph illustrating an embodiment of the gain conversion; and

[16] FIGS. 8 to 9 illustrate the change in the frame speed by changing the frame height.

Best Mode for Carrying Out the Invention [17] Hereinafter, the present will be described in detail with reference to accompanying drawings. [18] FlG. 2 is a flowchart illustrating an algorithm of automatic exposure control according to the present invention. The automatic exposure control includes calculating average brightness (SlO), checking an error of the automatic exposure

(S20), calculating target brightness to which a weight according to exposure control speed is reflected (S30), calculating the amount of the exposure to be applied to the next frame (S40), calculating charge integration time and a gain to be applied to the next image (S50), and converting the gain (S60). [19] The method of controlling automatic exposure according to the present invention will now be explained in reference to FlG. 2. [20] In the calculating the average brightness (SlO), the whole image is divided into several zones for the exposure control. [21] FlG. 3 is a diagram illustrating the whole image divided into two zones.

[22] Average brightness values of central and peripheral zones 12 and 14 are calculated by dividing total brightness of each zone by the areas of each zone, respectively. The average brightness is obtained by giving a weight to the average brightness value of the central zone 12.

[23] [Equation 1]

[24]

Y mean _c =( / iY_c )/(Ucenter ) [25]

T mean _J? ⁼( / Y _pΛl ⁽ ttperiphery Λ

[26]

Y mean = Y mean c X c .weight + Y. mean . p X ( 1 -c .weight)

[27] Here, Y_mean_c and Y_mean_p mean the average brightness of the central and peripheral zones 12 and 14, respectively. Y_mean means a weighted average value of the average brightness values of the central and peripheral zones 12 and 14. #center and #periphery mean areas, which are numbers of pixels, of the central and peripheral zones 12 and 14, respectively.

[28] As the weight given to the central zone 12 increases, the central zone 12 becomes more referenced for calculating the exposure. In other words, when the central zone 12 is dark although the peripheral zone is bright, the exposure is adjusted to brighten the central zone 12. On the contrary, when the central zone 12 is bright although the peripheral zone is dark, the exposure is controlled to lessen the brightness of the central zone 12.

[29] Although a case when the image is divided into two zones is explained only according to an embodiment of the present invention, the present invention can be easily applicable to a case when the image is divided into three more zones.

[30] In the checking an error of the automatic exposure error (S20), when the value obtained from dividing the absolute value of difference between the current average brightness and the target brightness by the target brightness is less than an allowable error, the automatic exposure control is not performed. This can be represented by Equation 2.

[31] [Equation 2]

[32]

I Y mean - Y. tar get I

Skip AEC if < Lock .threshold

Y .target [33] Here, Y_mean means the current average brightness calculated above. Y_target means the target brightness, and Lock_threshold means the allowable error.

[34] When the current average brightness is close enough to the target brightness, the automatic exposure control is not performed.

[35] In the checking an error of the automatic exposure (S20), when there is a difference between the current average brightness and the target brightness, the calculation of the target brightness to which the weight according to the exposure control speed is reflected is to be performed.

[36] In the calculation of the target brightness (S30) to which the weight according to the exposure control speed is reflected, the target brightness is calculated by weighting the exposure control speed which indicates how quickly the target can be reached.

[37] FlG. 4 is a graph illustrating a basic assumption that the average brightness is in linear proportion to the exposure. When the amount of the exposure is increased, the average brightness increases. On the contrary, when the amount of the exposure is decreased, the average brightness decreases.

[38] FlG. 5 is a graph illustrating a final value of the target exposure under the basic assumption illustrated in FlG. 4.

[39] Referring to FlG. 5, when the current amount of the exposure 1, the current average brightness 2, and the target brightness 3 are known, the amount of the exposure 4 which should be applied to reach the target brightness 3 can be obtained. In other words, the final amount of the exposure is determined by multiplying the current amount of the exposure to the ratio of the target brightness and the current brightness.

[40] [Equation 3]

[41]

Y Jar get

N .exposure = exposure X

Y mean

[42] Here, N_exposure is the amount of exposure to be applied to reach the target brightness at once, and exposure is the amount of exposure applied to the current image. However, the speed of the exposure control should be properly adjusted to avoid deterioration of the quality of the image and a possibility of oscillation due to abrupt exposure control. The exposure control speed is a variable to be used for adjusting the speed of the exposure control. In other words, the target brightness to which the weight according to the exposure control speed is reflected can be calculated using Equation 4.

[43] [Equation 4] Y .target. w ⁼ speed X Y target + ( 1 - speed) X Y mean

[45] Here, Y_target_w means the target brightness to which the weight according to the exposure control speed is reflected, and speed means the value of the exposure control speed. Y_target means the previous target brightness, and Y_mean means the current average brightness.

[46] For example, when the value of the exposure control speed is 0.5, a median value of the current average brightness and the target brightness becomes new target brightness. The exposure control speed can be changed by determining the value of the exposure control speed in the range of 0 to 1.

[47] After the new target brightness is determined, the amount of the next exposure to be applied to the next frame is calculated in the calculating the amount of exposure for the next frame. The amount of the exposure for the next frame can be calculated according to Equation 5 using the results of Equations 1 and 3.

[48] [Equation 5]

[49]

Y target w

N .exposure = exposure X

Y mean

[50] Here, N_exposure means the amount of exposure to be applied to the next image, and exposure means the amount of exposure applied to the current image. Y_target_w means the target brightness to which the exposure control speed is reflected.

[51] When the amount of exposure to be applied to the next image is obtained, the calculating of the next charge integration time and the next gain is followed.

[52] The next charge integration time and the next gain can be calculated using

Equation 6.

[53] [Equation 6]

[54]

N Time = M X f .period

[55]

N .exposure

N. gain —

N Time

[56] Here, N_Time means the charge integration time to be applied to the next image.

The maximal time for the charge integration is duration of one frame. When the charge integration time needs to be extended further, the size of the image frame should be increased to extend the duration time for one frame. The maximal charge integration time that a user designates is represented as Max_Time. The automatic exposure controller should control the charge integration time within the range of the maximal charge integration time. f_period means a flicker brightness period of a fluorescent lamp. To acquire an image without any flicker, the charge integration time should be adjusted to be times the flicker brightness period. M is a maximal integer value which satisfies the conditions that N_Time is smaller than or equal to max_Time, and that N_Time is smaller than or equal to N_exposure. N_exposure is the amount of exposure to be applied to the next frame which has been calculated previously. N_gain means the gain to be applied to the next frame.

[57] The exposure N_exposure conceptually contains the charge integration time

N_time and the gain N_gain. So, when the amount of the exposure is more than the maximal charge integration time, the gain is calculated based on the remaining value after the maximal charge integration time is subtracted from the amount of the exposure. The gain should be increased to have more exposure than the amount of the exposure at which the charge integration time is maximized.

[58] When the charge integration time is controlled, flicker of light should be considered. It is well known that when the charge integration time is arbitrary set under an alternate current lamp such as a fluorescent lamp, cross stripes caused by repetition of brightness and darkness may occur. Accordingly, the charge integration time should be set to times a brightness period of the lamp to avoid the flicker of the light.

[59] The charge integration time should be smaller than the maximal exposure time and the amount of the exposure simultaneously and also times the flicker period, since an image without any flicker can be acquired when the charge integration time is set to times the flicker period. Under a lamp without any flicker, the flicker period may be set to one. And then, the charge integration time can have continuous values such as one, two, three, four, etc in proportion to the amount of the exposure.

[60] When the charge integration time is set to times the flicker period, blinking occurs during the exposure control due to a big difference of brightness between the images exposed at N period and (N+ 1) or (N-I) period. In addition, proper brightness may not be acquired. Accordingly, to acquire smooth images, the gain together with the charge integration time should be changed to avoid the problems.

[61] FlG. 6 is a graph illustrating relation between the amount of the exposure to be applied to the next image and the charge integration time and the gain. The gain to be applied to the next frame can be calculated using the amount of the exposure and charge integration time to be applied to the next image.

[62] The gain applied to the next frame will now be explained in detail for each time interval of the charge integration time to be applied to the next image in reference to FIG. 6.

[63] In time interval 1, the amount of exposure is increased slowly from zero, and the amount of the exposure is smaller than one flicker period. At this time interval, the amount of the exposure becomes the charge integration time. In other words, when the environment is too bright, if the exposure time is set to one flicker period to avoid the flicker, a supersaturated bright image is acquired. So, the brightness of the image is controlled so that the image has proper brightness, although the flicker occurs, while the basic premise that the exposure time is times f_period is discarded. At this interval, the gain is fixed to one.

[64] In time interval 2, the amount of the exposure is greater than or equal to one flicker period (l*f_period) while the amount of the exposure is smaller than two flicker periods (2*f_period), and the charge integration time is fixed to one flicker period (l*f_period). At this interval, the gain is a ratio of the amount of the exposure and the charge integration time and can be calculated using Equation 7, since the charge integration time is one flicker period (l*f_period).

[65] [Equation 7]

[66]

1 f .period ^f-P ^riod < gain <

1/ .period \f .period

[67]

1 < gain <2

[68] As a result, the gain has a value more than or equals to one and less than two.

[69] In time interval 3, the amount of the exposure is greater than or equal to two flicker periods (2*f_period) while the amount of the exposure is smaller than three flicker periods (3*f_period), and the charge integration time is fixed to two flicker periods (2*f_period). In this case, the gain can be calculated as explained above, and the gain has a value more than or equals to one and less than 1.5. In other words, the gain can be calculated using Equation 8.

[70] [Equation 8]

[71]

2 j .period 3 f_p eriod 2/ .period 2/ ^'.period

[72]

1 < gain <3/2 [73] The charge integration time and gain for intervals 4 to 6 can be calculated using the same method explained above.

[74] In time interval 7, of which the amount of the exposure is continuous from that of time interval 6, the amount of the exposure is greater than or equal to five flicker periods (5*f_period), and the charge integration time is five flicker periods (5*f_period). In this case, the gain can be calculated using Equation 9.

[75] [Equation 9]

[76]

_ exposure gain

Sf period

[77] After the gain is calculated as explained above, the conversion of the gain is started.

[78] FIG. 7 is a graph illustrating an embodiment of the gain conversion. The conversion from a linear gain to a non-linear gain is illustrated in FIG. 7.

[79] The gain N_gain to be applied to the next frame calculated above has a linear value. In other words, when the gain has a value of 2.5, the gain becomes 2.5 times. However, it is common to implement the gain in an exponential form or in a gain curve having a different form using a semiconductor circuit. Accordingly, for the implementation of the exposure control using a practical amplification circuit, it is required to convert the linear gain acquired from the previous step to a final gain C_gain.

[80] The exposure variables defined above such as exposure and time are in the unit of a pixel clock cycle. In other words, 'exposure=100' means exposing during while data for 100 pixels are read. Accordingly, when the size of an image frame is given as (frame_width, frame_height), the maximal charge integration time becomes frame_width x frame_height.

[81] Factors determining frame speed are a frame width, a frame height, and a clock speed. Accordingly, to change the frame speed, the frame width, the frame height, or the clock speed may be changed.

[82] In FIGS. 8 to 9, the change in the frame speed by changing the frame height is illustrated.

[83] The whole frame of which height is changeable can be divided into an active window in which a real image is appeared and a timing frame which is invisible.

[84] The amount of exposure having a size smaller than or equal to the area of the configured basic frame can be adjusted within the basic frame without changing the size, that is the area of the frame. However, when a value greater than the area of the frame (frame width x frame height) is set as the maximal charge integration time, the charge integration time should be longer than the area of the basic frame if the amount of the exposure becomes more than the area of the frame. However, the charge integration time which the amount of the exposure indicates cannot be maintained by exposing only for the duration of the basic frame. In this case, the frame height should be lengthened to increase the time required to read data for one frame.

[85] When the amount of the exposure increases to be larger than the designated maximal charge integration time, the area of the frame is fixed to a value which can accommodate the maximal charge integration time, and the remaining exposure is used to form the gain. The gain can be calculated using Equation 9.

[86] [Equation 9]

[87]

_ exposure Craϊn — max time

[88] Referring to FIG. 9, in time interval 1, the amount of the exposure is less than the area of the frame, and the frame speed is fixed to a maximal value, that is the frame height has the minimal value. In time interval 2, the amount of the exposure is larger than or equal to the area of the frame and smaller than the maximal charge integration time. In time interval 2, the frame height is changed according to the amount of the exposure to change the frame speed.

[89] In the time interval 3, the amount of the exposure is larger than or equal to the maximal charge integration time, and the frame speed is fixed to the minimal value, that is the frame height has the maximal value. Only the gain is increased as the amount of the exposure increases, in time interval 3.

[90] The frame speed can be changed by changing the frame width in a similar way as was explained for changing in the frame speed by changing in the frame height.

[91] While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. Industrial Applicability

[92] According to a method of controlling automatic exposure, an image sensor can have a proper brightness using proper combination of exposure time and gain, and accordingly the image quality of the image sensor can be improved. In addition, a complex algorithm such as removing flicker of a fluorescent lamp and changing in the frame speed can be easily implemented by controlling only a single variable of the amount of the exposure according to the present invention.

Claims

[1] A method of controlling automatic exposure of an image sensor comprising:

(a) calculating current average brightness by dividing a whole image into several zones and giving weights to the divided zones, respectively;

(b) comparing and checking the current average brightness and target brightness;

(c) calculating modified target brightness by reflecting a weight based on a value of exposure control speed;

(d) calculating the amount of exposure to be applied to the next frame using the modified target brightness;

(e) calculating a charge integration time and a value of a gain to be applied to the next frame based on the amount of the exposure; and

(f) calculating a converted gain using the value of the gain to be applied to the next frame for the application to a practical amplification circuit.

[2] The method of claim 1, wherein the (a) comprises calculating the current average brightness by calculating average brightness values of central and peripheral zones by dividing total brightness of each zone by each area and giving a weight to the average brightness value of the central zone.

[3] The method of claim 1, wherein the (b) comprises not performing the automatic exposure control when a ratio of difference between the current average brightness and the target brightness is less than an allowable error ratio.

[4] The method of claim 1, wherein the (c) comprises calculating the modified target brightness by reflecting the weight based on the value of the exposure control speed using the following equation, and wherein Y_target_w means the modified target brightness to which the weight based on the value of the exposure control speed is applied, speed means the value of the exposure control speed, Y_target means the previous target brightness, and Y_mean means the current average brightness.

Y Jar get w = speed X Y .target + (1 - speed) X Y mean

[5] The method of claim 1, wherein the (d) comprises calculating the exposure to be applied to the next frame using the following equation, and wherein exposure means the amount of exposure applied to the current image, N_exposure means the amount of exposure to be applied to the next image, Y_mean means a value representing the brightness of the current image, and Y_target means a value representing the target brightness. Y Jar get

N exposure = exposure X

Y mean

[6] The method of claim 5, wherein the (d) comprises calculating the exposure to be applied to the next image and the modified target brightness using the following equations for controlling the exposure control speed, wherein speed as a variable designating the exposure control speed has a value between zero to one.

Y .target w

N .exposure = exposure X

Y mean

Y .target w = speed X Y. target + (1 - speed) X Y. mean

[7] The method of claim 1, wherein the (e) comprises calculating the charge integration time and the value of the gain to be applied to the next frame using the following equations, and wherein N_Time means the charge integration time to be applied to the next image, M means the possible maximal integer value, f_period means a flicker brightness period, N_gain means the gain to be applied to the next frame, and N_exposure means the amount of the exposure to be applied to the next frame which has been calculated previously.

N Time ⁼ Ad X f .period

N .exposure

N .gain —

N Time

[8] The method of claim 7, wherein the exposure time is acquired by lengthening the height of the frame to extend the duration of the frame when the charge integration time to be applied to the next image is larger than the duration of one frame.