CN101518054B - A hand jitter reduction system for cameras - Google Patents
A hand jitter reduction system for cameras Download PDFInfo
- Publication number
- CN101518054B CN101518054B CN2007800354470A CN200780035447A CN101518054B CN 101518054 B CN101518054 B CN 101518054B CN 2007800354470 A CN2007800354470 A CN 2007800354470A CN 200780035447 A CN200780035447 A CN 200780035447A CN 101518054 B CN101518054 B CN 101518054B
- Authority
- CN
- China
- Prior art keywords
- gain
- time
- exposure
- automatic exposure
- normal mode
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
- H04N23/682—Vibration or motion blur correction
- H04N23/684—Vibration or motion blur correction performed by controlling the image sensor readout, e.g. by controlling the integration time
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/70—Circuitry for compensating brightness variation in the scene
- H04N23/72—Combination of two or more compensation controls
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/70—Circuitry for compensating brightness variation in the scene
- H04N23/73—Circuitry for compensating brightness variation in the scene by influencing the exposure time
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/70—Circuitry for compensating brightness variation in the scene
- H04N23/76—Circuitry for compensating brightness variation in the scene by influencing the image signals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N2101/00—Still video cameras
Abstract
A camera system in normal mode and hand jitter reduction (hjr) mode may comprise generating a first exposure time-gain product by multiplying the normal mode exposure time with the normal mode gain. It may further comprise modifying the normal mode exposure time and gain and multiplying these modified parameters to generate a second exposure time-gain product for a hjr mode that reduces the difference between the first exposure time-gain product and the second exposure time-gain product. To reduce the difference the normal mode frame rate may also be modified. Operation of a camera in normal mode may be in response to a sensed light level being above a threshold. The hjr mode may be selected by the user while the camera is operating. The hjr mode may be used in response to a sensed light level being lower than the threshold.
Description
The crosscorrelation application case
The application's case is advocated the rights and interests of the 60/760th, No. 768 U.S. Provisional Application case that is entitled as " hand jitter reduction system design (HAND JITTERREDUCTION SYSTEM DESIGN) " of application on January 19th, 2006.The present invention with all with the application's case in the 11/534th of being entitled as of co-applications on the 25th September in 2006 " hand that compensation rotatablely moves is shaken minimizing (HANDJITTER REDUCTION COMPENSATING FOR ROTATIONAL MOTION) "; No. 935 (attorney docket 060268) common patent application case co-pending is relevant with the 11/534th, No. 808 (attorney docket 060193) the common patent application case co-pending that is entitled as " the hand shake that is used for the compensated linear displacement reduces (HAND JITTER REDUCTION FOR COMPENSATING FOR LINEAR DISPLACEMENT) ".
Technical field
The present invention relates to digital picture and handle, and more particularly, relate to a kind of hand jitter reduction system that is used for camera.
Background technology
To the demand of multimedia application in the mobile communication with surprising speed increment.Nowadays, the user can send from the internet and receive still image and download images and video, on mobile unit or mobile phone, watching.The integrated growth trend that further help in mobile communication multimedia function property of digital camera in the mobile unit.
The given limited amount resource like battery capacity, processing power and transmission speed that is associated with mobile unit needs effective digital imagery treatment technology to support multimedia function.This requires to reduce the exploitation that the computational complexity that is used for multimedia application is kept the upper hardware and software of picture quality simultaneously.The exploitation of this type of hardware and software causes the lower power consumption and the longer stand-by time of mobile unit.
An aspect of digital imagery process relate to remove from picture fuzzy.Possibly cause fuzzy by the hand shake.Mobile the cause hand of user's hand when taking digital picture with camera shaken.Even the user does not recognize mobile, hand also possibly constantly move.Said mobile phase is to less, if but mobile phase is bigger for the time for exposure, and digital picture possibly blur so.Object or person in the picture can look and move.Also maybe by object/person when the pictures taken move cause fuzzy.Also possibly cause fuzzy by the restriction of the optical system that is used to capture picture.
According under the condition, for example used digital camera takes a long time the registration picture in the mobile unit at low light.Slight the moving that the longer time for exposure has increased that hand produces can cause the possibility of bluring.Similarly, the longer time for exposure has increased the probability that moving of object/person can be bigger with respect to the time for exposure.
The current technology that is used to compensate mobile camera moving relates to uses miniature gyroscope or other mechanical device.As if said technology all do not have acceptable manner and come especially to shine under the condition at low light with digital form compensation mobile camera moving.Need be to be suitable for moving the fuzzy quantity in effective processing a resource shrinkage digital picture of using under all conditions.
Summary of the invention
The details of one or more configurations of statement in accompanying drawing and the following description content.From describe content, graphic and claims, will understand other features, objects and advantages.
A kind of comprise that hand shake reduces the camera system of (hjr) minimizing pattern and normal mode can be through the time for exposure be multiply by generation exposure time-gain product mutually and operates with gaining.Can there be normal mode exposure time-gain sum of products hjr exposure mode time-gain product.The exposure time-gain product of normal exposure time and normal gain in the normal mode can be stored in the table when needed or use.Exposure time-gain product in the hjr pattern also can be stored in the table.Can through revise in (add, deduct, multiply by or divided by) normal mode the time for exposure-clauses and subclauses in the product table (or clauses and subclauses) produce the exposure time-gain product in the hjr pattern.So, can not necessarily need independent table, but the equivalent exposure time-gain product in the hjr pattern can compare with the exposure time-gain product in the normal mode.When in the hjr pattern, operating, change parameter to reduce poor between exposure time-gain product and the exposure time-gain product in the normal mode in the hjr pattern.As long as the imageing sensor in the camera system can be higher than minimum average light level, said difference just can reduce.In the hjr pattern, can not satisfy in any district of lighting level of minimum average amount, possibly can not in the hjr pattern, keep and exposure time-gain product identical in normal mode at imageing sensor.The operation of camera in normal mode can be higher than threshold value in response to institute's sensed light level.The operation of camera in the hjr pattern can be selected by the user.Can be lower than said threshold value and use the hjr pattern in response to institute's sensed light level.
Description of drawings
By instance rather than with the mode of restriction various configurations are described in the accompanying drawing.
Fig. 1 is the block diagram of a configuration of explanation digital imagery process.
Fig. 2 is functional block diagram of a configuration of a front end image processing module in the explanation digital image processing system.
Fig. 3 is functional block diagram of another configuration of the front end image processing module in the explanation digital image processing system.
Fig. 4 A is that explanation is to the curve chart of normal mode time for exposure to lighting level.
Fig. 4 B is that explanation is to the curve chart of normal mode gain to lighting level.
Fig. 5 A is that shake reduces the curve chart of exposure mode time to lighting level to hand in explanation.
Fig. 5 B is that shake reduces the curve chart of modal gain to lighting level to hand in explanation.
Fig. 6 is that explanation produces the flow chart through the method for revising the automatic exposure parameter.
Fig. 7 explains flow chart how in normal mode and hand shake minimizing pattern, to operate camera system.
Fig. 8 is the automatic exposure parameter in the camera system is revised in explanation in hand shake minimizing pattern a flow chart.
Embodiment
This paper uses word " exemplary " to represent " as instance, example or explanation ".Any configuration, scheme, design or the calibration that this paper is described as " exemplary " all may not be interpreted as and compare with other configuration, scheme, design or calibration is preferred or favourable.In general, this paper describes and reduces in the digital picture because the technology of bluring that hand shake and/or the illumination condition lower than normal illumination condition cause.The technology of calibrating the camera that can in normal mode and hand shake minimizing pattern, operate is also described.
In conventional camera apparatus, when the user took snapshot (current accomplish through pressing button), only a frame was used to produce picture usually.The method of using an above frame to produce picture gets nowhere usually, because it produces poor outcome.Utilize conventional camera apparatus, picture possibly move moving of producing owing to user's oneself hand and blur, and these hands move and are called the hand shake, and conventional camera apparatus also receives the required time quantum challenge of exposure image.According under the condition, increase the time for exposure at low light usually.It is observable because low light shines the noisiness that condition causes that the increase time for exposure has increased the user, and increase the possibility that the hand shake will produce blurred picture.Current, camera apparatus can contain the hand shake that miniature gyroscope produces with the compensation user.Yet, have many challenges of when being placed on gyroscope on the mobile unit, facing.Even when overcoming these whens challenge, digital hand jitter reduce technology can with have gyrostatic device combination and use.Current camera apparatus also can be at low light according to scalar gain under the condition.Regrettably, increase gain simply and amplified the noise that exists owing to low light levels.The result normally has the picture of poor quality.Similarly, the digital compensation for the hand shake does not always provide suitable result.Yet,, existing possibly reduce the hand shake and reduce than the noise under the light conditions through the technology that discloses among the present invention.
Fig. 1 is the block diagram that explanation is suitable for being integrated into the digital imagery process of the camera apparatus in the mobile unit.Mobile unit can be radio telephone, PDA(Personal Digital Assistant), laptop computer, or any other portable radio.Lens (not shown) can be used for focusing an image on the imageing sensor 102 in the image sensor module 104.In a kind of configuration, image sensor module 104 can have the memory that is used for storage gain and exposure parameter.Image sensor module 104 also can have and is used for revising gain and automatic exposure parameter control driver.In another configuration, image sensor module 104 can be coupled to integrated circuit (for example, travelling carriage modulator-demodulator (MSM
TM)) or have be used to store and revise the gain and the memory of automatic exposure parameter and/or other module of Control Driver.Imageing sensor 102 can be charge coupled device (CCD), complementary metal oxide semiconductors (CMOS) (CMOS) imageing sensor, or any other suitable imageing sensor.In at least a configuration of imageing sensor 102, semiconductor array is used in the different pixels place capture light of image.The colorful optical filter array (CFA) (not shown) that is positioned at imageing sensor 102 the place aheads can be used for single color (that is, red, green or blue) is delivered to each semiconductor.Modal CFA is RGB and CMYG pattern.Image sensor module 104 can drive or control chart image-position sensor 102 gains and or the time for exposure to revise.
With before taking snapshot and producing digital picture, preview mode can be captured the series of frames that is produced by imageing sensor 102 at user's pressing button.The subdivision of entire frame or frame is called as image or (being called interchangeably) picture.For illustration purposes, discussing image processing expediently is series of frames.But it should be understood that when using front end image processing module 106 and need not handle entire frame.In addition, frame sequence also is called as crossfire.Can said crossfire be provided to front end image processing module 106, it is removed mosaic so that obtain full RGB resolution as the input to still image and video compressor 108 at this place.When crossfire passes through front end image processing module 106, in preview mode, can collect the statistics that produces the frame of digital picture about auxiliary.These statisticss can be (but being not limited to) exposure metrics, white balance metrics and focus metrics.
Front end image processing module 106 can be with the various signal feedback that help control chart image-position sensor 102 in image sensor module 104.Still image and video compressor 108 can use the JPEG compression, or any other suitable compression algorithm.Automatic exposure control module 110 can receive and the proportional value of being handled by front end image processing module 106 of lighting level, and itself and the optical target of being stored are compared, so that at least one in the function of auxiliary front end image processing module 106.Image through the resume module in the front end image processing module 106 is the part of digital frame.Also can crossfire be sent to the view finder that can be arranged in display module 112.In preview mode, can be used for controlling automatic exposure from the preview decision of display module 112.
Have that preview mode in the mobile unit of digital camera can be used for normal mode or the hand shake reduces (hjr) pattern.The user can be via menu or is manually selected hjr pattern (being shown as hjr among Fig. 1 selects) through the user interface.Can confirm for example gain, automatic exposure time, frame rate and the automatic exposure parameters such as frame number that will handle in moment after taking snapshot and producing digital picture at user's pressing button.Collected statistics can be used for confirming the automatic exposure parameter in normal mode and hjr pattern, during snapshot, used.Therefore, after user's pressing button, the hjr pattern can be different with image processing between the normal mode.Before user's pressing button, even selected the hjr pattern, preview mode is also as its such image of handling in normal mode.
Fig. 2 is functional block diagram of the configuration of a front end image processing module 106a in the explanation digital image processing system.Front end image processing module 106a can be used for compensating human visual system's response and the sensor signal that produces by imageing sensor 102 between difference.Can use various treatment technologies to proofread and correct these difference, said treatment technology comprises (by way of example) black correction and lens slippage 202, goes mosaic module 204, white balance and colour correction 206, gamma regulate 208, and color conversion 210.These processes are expressed as independent processing module in Fig. 2, but or can use shared hardware or software platform to carry out.In addition, these modules can comprise a plurality of image processing modules of carrying out identical function, allow whereby different images is carried out said function concurrently.
After the color conversion module processes frame, can image three-colo(u)r component (Y, Cb and Cr) be sent shake control module 212 in one's hands.Can be with presenting in the shake control module 212 in one's hands from the various parameters of automatic exposure control module.Hand shake control module 212 can be used for a plurality of purposes.Hand shake control module 212 can be confirmed the image processing that snapshot takes place afterwards.Hand shake control module 212 can detect the value of hjr selection and determine whether that needing to carry out the hand shake reduces (hjr).Although the user has selected the hjr pattern, hand shake control module 212 can be confirmed the image processing as accomplishing in the normal mode can take place.Hand shake control module 212 can confirm to take place the image processing in the hjr pattern.In the hjr pattern, producing the digital picture image processing can comprise and capture single frame or a plurality of frame.If hand shake control module 212 is confirmed to capture a plurality of frames, through after the hjr control module, can the parameter how many frames frame can handle together with indication noise minimizing/frame registration module 214 be sent to noise minimizing/frame registration module 214 so.If will handle single frame, can the noise minimizing take place through using noise to reduce module 215 so on single frame.It can be Baeyer (bayer) filter that noise reduces module, or other similar filter.If will handle a plurality of frames, noise minimizing/frame registration module 214 available buffers are shaken the frame number numf of control module 212 appointments by hand so, and it is carried out the frame registration.According to how many frames and lighting level, the purpose of a plurality of frame registrations can be used for the purposes that noise reduces and/or blur minimizing.A plurality of frame registrations can be accomplished by frame registration module 216.
If hand shake control module 212 is confirmed image processing and as in the normal mode, taking place, although the user selects the hjr pattern so, also can not used noise minimizing/frame registration module 214, and can use the output from (for example) color correction module 210.Confirm what image processing (image processing in image processing in the normal mode or the hjr pattern) according to hand shake control module 212, can use signal (sel) to select 217 outputs of which multiplexer to send to post-processing module 218.Can the output of post-processing module 218 be sent to static and image/video compressor reducer 108 and/or display module 112.
Except output selects signal (sel) and frame number to be used for the noise minimizing/frame registration; Also exportable other parameter of hand shake control module 212: new automatic exposure frame rate (ae fr_new), new automatic exposure gain (ae gain_new), new automatic exposure time (ae time_new) and pending frame number (numf).Can these parameters be sent to image sensor module 104 with control chart image-position sensor 102.Hand is shaken the also exportable digital gain of control module 212 and can be applied to image sensor module 104 any module place afterwards.As an instance, can during white-balance/color correction module 206, apply digital gain.
One of ordinary skill in the art will recognize that though describe pixel usually, sub-pixel or a plurality of pixel also can be used as the input that gets among the front end image processing module 106a.In addition, the subclass of these picture contents or other form: also can be with sending shake control module in one's hands through the pixel of RGB and space-frequency conversion, for example hand is shaken control module 212.So, functional another configuration of capturing the front end image processing module in the digital image processing system is described among Fig. 3.
In Fig. 3, moved hand shake control module 212, noise minimizing/frame registration module 214 and multiplexer 217 and be inserted in black correction and lens slippage module 202 and go between the mosaic module 204.This configuration displaying hand shake control module 212 that is embedded among the front end image processing module 106b can be operated R, G and B picture content, and not only to Y, Cb and Cr operation.In general, hand shake control module 212 can be operated in any subsequent module and before any display module 112 or still image and video compressor module 108 after imageing sensor 102 capture light.
Can calibrate normal mode and hand shake minimizing pattern through creating at least one automatic exposure time-gain table.Automatic exposure time-gain table can have time for exposure row and gain column.Clauses and subclauses in the time for exposure row can multiply by mutually with the clauses and subclauses in the gain column and produce the exposure time-gain product.Light level value can be represented in each the row clauses and subclauses or the index that get in automatic exposure time-gain table, can each lighting level be mapped to the automatic exposure index that gets in automatic exposure time-gain table.Automatic exposure time-gain table can have various operating spaces, for example those districts that in Fig. 4 A, Fig. 4 B, Fig. 5 A and Fig. 5 B, point out.In all four figure (Fig. 4 A, Fig. 4 B, Fig. 5 A and Fig. 5 B), exist in four possible lighting level districts that explain on the axis of abscissas, R1, R2, R3 and R4.The separation in district can be by different boundary (border _ a, border _ b and border _ c) control.The lighting level that the leftmost side of axis of abscissas is showed begin higher and then be reduced to low, such as the rightmost side of axis of abscissas displaying.Exposure time-gain product (that is (time for exposure) * (gain)) should be intended to not reduce in each district.So, the reduction of time for exposure or gain can cause gaining or the increase of time for exposure.Can calibrate automatic exposure time-gain table according to the setting and/or the characteristic of sensor type, camera.Also can use an above frame rate to calibrate said table.In addition, can said table be stored in the memory that is positioned on the integrated circuit (for example, front end image processing module 106 and still image and video compressor module 108) that carries out main image processing.Also can said table be stored in the memory that is not positioned on the integrated circuit that carries out main image processing.Under two kinds of situation, whether no matter be positioned on the integrated circuit identical with main image processing, said table all can be coupled to image sensor module 104.
Fig. 4 A be explanation to the curve chart of normal mode time for exposure with (detect) lighting level, and Fig. 4 B is explanation to the curve chart of normal mode gain to (detect) lighting level.Because lighting level be height among the R1 in the district, so the time for exposure is shorter, i.e. low value.Similarly, in district R1, gain is in nominal value, and is promptly lower with respect to other district.For the object that slowly moves or when user's hand move very little (that is, about 5ms) time, can have the minimal advantage of the fuzzy and/or noise minimizing among the correction zone R1.Even will carry out fuzzy and/or the noise minimizing, its eyes for the user also are sightless.Therefore, border _ a is positioned at fuzzy and/or noise to be reduced for around the visible point of user, but promptly allows user's insight to see fuzzy and/or the time for exposure that noise reduces and the combination of gain.Disengagement zone R1 can change according to different users and/or different cameras with border _ a of district R2.Also can border _ a be regarded as image processing and the threshold value between the image processing in the hjr pattern in the normal mode.
In district R1 and R2, camera can be in (for example, the fr1) operation down of a frame rate.Shown in Fig. 4 A, along with lighting level reduces, in district R1, the time for exposure increases.Time for exposure increases in district R1.Time for exposure can continue to increase till the time for exposure reaches maximum for frame rate 1 (fr1) in district R2.Under this lighting level (wherein the time for exposure at first is in maximum for frame rate 1 (fr1)), gain (shown in Fig. 4 B) can begin to increase.Gain can continue to increase till the predetermined value that is provided with between alignment epoch is in the point that has the acceptable noise level.If be increased to maximum, noise level maybe be unacceptable for the said given time for exposure so.For normal mode, can border _ b be expressed as time for exposure under the frame rate 1 (fr1) and be in the point that maximum and gain are in the predetermined value that wherein has the acceptable noise level.Disengagement zone R2 can change according to different cameras with border _ b of district R3.
Need to carry out as far as possible continuously being converted to another district from a district.Therefore, the exposure time-gain product of the leftmost side or identical with it among the exposure time-gain product accessible area R3 of the rightmost side among the district R2.Because the time for exposure is in maximum (in district R2 shown in the rightmost side), thus frame rate can be on the border _ reduction of b place.The reduction of frame rate (that is, making frame rate be changed to fr2 from fr1) allows the time for exposure to increase.Time for exposure can reach maximum (as distinguishing shown in the R3 among Fig. 4 B) for new frame rate (frame rate 2 fr2).In order to keep distinguishing the continuity of exposure time-gain product between R2 and the district R3, the gain among the reduction district R3 is so that the increase of time for exposure skew.Gain can increase till it reaches maximum.Although can apply digital gain, the gain of using in the normal mode is generally simulation.In normal mode, border _ c is that analog gain reaches peaked point.On the border _ and the c place, in normal mode, can the lighting level of correspondence be stored as the optical target of checking in hand shake minimizing pattern.
Can calibrate hand shake minimizing (hjr) pattern through creating another automatic exposure time-gain table.Can carry out subtraction, addition, division or multiplication to produce required " equivalence " automatic exposure time-gain table in the hand shake minimizing pattern to normal mode automatic exposure time-gain table clauses and subclauses.Characteristic in the row (time for exposure and gain) of Fig. 5 A and Fig. 5 B another automatic exposure time-gain table of explanation or " equivalence " automatic exposure time-gain table.When in the hjr pattern, taking snapshot, the automatic exposure time-gain table through equivalence is the normal mode when being in preview mode to the change of time for exposure and gain.Preview mode uses the characteristic and the automatic exposure parameter of normal mode.
To be explanation reduce the curve chart of exposure mode time to (detect) lighting level to hand shake to Fig. 5 A.To be explanation reduce the curve chart of modal gain to (detect) lighting level to hand shake to Fig. 5 B.Reduce in (hjr) pattern in hand shake, target be intended to keep with normal mode in identical exposure time-gain product.So, [time for exposure in the hjr pattern] * [gain in the hjr pattern] should be intended to identical with [time for exposure in the normal mode] * [gain in the normal mode].As before mentioned, there is the minimal advantage of attempting to reduce the hand shake among the district R1, because it maybe be invisible for human eye.In district R1, the gain in the hjr pattern and time for exposure can with normal mode in identical.Therefore, the image processing that in normal mode, in district R1, takes place also can take place in district R1 when being in the hjr pattern.Yet in district R2, R3 and R4, image processing takes place in the hjr pattern.The image processing that reduces the hand shake to be taking place through revising the automatic exposure parameter, and the image processing that need not reduce the hand shake takes place with unmodified automatic exposure parameter.
Slight the moving that longer time for exposure increases that hand produces can cause the possibility of bluring.Therefore, in case eyes in district R2, can detect fuzzy and/or noise reduces, the time for exposure just should reduce.Therefore, the time for exposure (shown in Fig. 5 A) that reduces among the district R2 will cause the possibility of bluring to reduce the hand shake.In order to offset the minimizing of time for exposure, increase the gain (shown in Fig. 5 B) among the district R2.Note, in the hjr pattern, keep with normal mode in the identical time for exposure and the shape of gain in distinguishing R2.One of difference in district R2 between normal mode and the hjr pattern is from the time for exposure of normal mode and gain and hjr pattern skew is arranged.In the hjr pattern, the time for exposure is in lower skew than in normal mode, and the ratio of gains is in higher skew in normal mode.Yet the exposure time-gain product is equivalent between normal mode and hjr pattern in the district.
As before mentioned, in normal mode or hjr pattern, but the preview series of frames.Preview mode uses the characteristic and (unmodified) automatic exposure parameter of normal mode.In the hjr pattern, the automatic exposure time-gain table of equivalence is the normal mode when being in preview mode.Therefore, when increasing or reduce time for exposure, gain or frame rate, it is with respect to the value in the preview mode.In the hjr pattern, can through inspection exposure time-gain product when reach with normal mode in the identical value of exposure time-gain product confirm border _ b.In the hjr pattern, if lighting level the district R3 in, frame rate can increase the amount greater than frame rate fr2 so.Why this can be because in preview mode, in district R3, frame rate is fr2 (being lower than the speed of fr1).For instance, if frame rate fr1 is 30 frame per seconds (fps), and frame rate fr2 is 15fps, and the frame rate of 15fps can increase a certain amount L in the hjr pattern so, up to frame rate fr1.In this example, L is 15fps.Therefore, in the hjr pattern, frame rate (for example) is increased to 30fps from 15fps allows the time for exposure among district R3, to be maximized to its value in district R2 of normal mode.Time for exposure is the reduction in district R3 that can cause gaining of the increase among the R3 in the district.If in district R1, R2 and R3, use analog gain, owing to the cause of the gain offset that in the hjr pattern, increases (in the leftmost side of district R2), the analog gain among the district R3 can be saturated before border _ c so, promptly reaches maximum (like the rightmost side finding of district R3).Operate in order to exceed maximum analog gain, can add digital gain to analog gain.At at Fig. 5 B is explained digital gain between district R3 and district R4.
Can the lighting level at border in normal mode _ c place be stored as predetermined optical target, in the hjr pattern, to check.In the hjr pattern,, possibly there is not enough light to supply imageing sensor 102 to produce the minimum average light level that is used for imageing sensor 102 so if lighting level is lower than the optical target of the optical target of storing.Can be through light value (brightness and/or colourity) the phase adduction at each pixel place be confirmed minimum average light level divided by the sum of all pixels order.The other type that calculating is used for the minimum average light level of imageing sensor 102 is to abandon all pixels that are lower than a certain threshold value in calculating.For instance, do not use the pixel that is lower than value 10, and use residual pixel (being higher than a certain threshold value) to calculate minimum average light level.Usually, use the brightness light value, but also can use chromatic value.For illustration purposes, discuss brightness value.In the hjr pattern, if brightness value is lower than predetermined luminance (light) target, light (brightness) level will be in district R4 so.
If confirmed that lighting level is lower than luma target in the hjr pattern, need not so to change and in preview mode, be the frame rate of fr2.Yet, can reduce owing to the amount of light and regulate the time for exposure, and can increase the time for exposure.Can the time for exposure be increased to the maximum that is allowed for frame rate fr2.In order to offset the increase of time for exposure, digital gain on the border _ the c place reduces, and can in district R4, constantly increase.
In district R1, R2 and R3, normal mode is intended to identical with exposure time-gain product between the hjr pattern.That is, should reduce difference so that difference between the two approaching as far as possible zero.In district R4, imageing sensor 102 possibly not satisfy the lighting level of minimum average amount.Therefore, can apply digital gain to increase the gain stage among the district R4.
Fig. 6 is that explanation produces the flow chart through the method for revising the automatic exposure parameter.As illustrated in the flow chart, can be in capture frame 600 after the snapshot.Can the last frame of handling in the preview mode be stored in the memory.Capture and mean from the memory search frame and can be used for producing digital picture.Under the present frame speed and memory storage capabilities on the mobile unit, can exist in the memory after the snapshot and be less than ten frames.But be not when producing digital picture, possibly need each frame in the memory.Usually, three institute's capture frame can be used for producing digital picture at the most.In order after snapshot, to produce through revising the automatic exposure parameter, camera can be in the hjr pattern.Therefore, in another embodiment, getting into hjr pattern 602 can be prior to capture frame after snapshot.But in Fig. 6, get into hjr pattern 602 and be illustrated as after capture frame after snapshot.In case be in the hjr pattern, the lighting level that is detected just with from the unmodified automatic exposure parameter correlation of normal mode joins 604.Can be under the lighting level that is detected from the gain and time for exposure 606 of memory search unmodified.Can accomplish the calculating 608 of the exposure time-gain product in the normal mode through multiplying each other with gain from the time for exposure of normal mode.Can calculate the exposure time-gain product in the hjr pattern from the automatic exposure parameter of normal mode through revising, these automatic exposure parameters can be gain, time for exposure and/or frame rate.Can revise the automatic exposure parameter to reduce differing from 610 between exposure time-gain product and the exposure time-gain product in the normal mode in the hjr pattern.Therefore, exist corresponding to the exposure time-gain product in the hjr pattern of the exposure time-gain product in the normal mode.On the ideal, said correspondence should be identical.Yet, because potential the lighting level that detects is slightly different between potential computational accuracy difference when multiplying each other and hjr pattern and the normal mode, so said correspondence can be inequality.Difference between the exposure time-gain product in hjr pattern and the normal mode reduces just enough as much as possible.In addition, in the district in the hjr pattern (the for example district 4 of preceding text), imageing sensor 102 possibly not satisfy the lighting level of minimum average amount.Therefore, possibly be able to not in the hjr pattern, keep with normal mode in identical exposure time-gain product.
Be that flow chart how in normal mode and hand shake minimizing pattern, to operate camera system is described among Fig. 7.In order to produce image, first action is a capture frame 700 after snapshot.In case capture, just carry out inspection to find out the hjr pattern of selecting whether.If still non-selected hjr pattern can be transmitted unmodified automatic exposure parameter so with control chart image-position sensor 704.These parameters can be automatic exposure frame rate (ae fr), automatic exposure time (ae time), automatic exposure gain (aegain), measured brightness, and luma target.If selected the hjr pattern, the modification 706 of automatic exposure parameter takes place so when needed.In next action, carry out inspection to find out that lighting level whether in district R1 708, promptly exists in the district of identical (or much at one) parameter in normal mode.If said lighting level can be transmitted unmodified automatic exposure parameter with control chart image-position sensor 704 so greater than the lighting level (that is, lighting level is in district R1) at border _ a place.Although selected the hjr pattern, the image processing in the normal mode can take place.In subsequent action, after transmission unmodified automatic exposure parameter is with the control chart image-position sensor, can continue the processing 710 of digital picture.If said lighting level is lower than the lighting level (inspection as in the decision block 708 is performed) at border _ a place, can uses so through revising automatic exposure parameter and frame number (numf) and digital gain and come control chart image-position sensor 711.Through revising the automatic exposure parameter can be new automatic exposure frame rate (ae fr_new), new automatic exposure time (ae time_new), new automatic exposure gain (ae gain_new), measured brightness, and luma target.Can check the number numf 712 that will handle with the frame that produces digital picture.If numf is greater than 1, but a plurality of frames 714 of registration so.But the motion between the frame registration compensated frame (level, vertical or angled).It can increase the frame registration intensity of gained image afterwards.If numf is greater than 2, but also aid in noise minimizing of frame registration so.If numf is 1 and has selected hjr, the minimizing 716 of noise so also can take place.A kind of possible mode that reduces the noise in the frame is to use Baeyer to picture content to filter.After the noise minimizing takes place, can continue the processing 710 of digital picture.It should be noted that and to shake the frame 702,704,706 and 711 that finds in the control module 212 among Fig. 7 at hand.
Fig. 8 is the automatic exposure parameter in the camera system is revised in explanation in hand shake minimizing pattern a flow chart.Although the selection of hjr pattern possibly take place, but still can use the image processing in the normal mode.Carry out inspection by decision block 800, if lighting level greater than the lighting level at border _ a place, so pending frame number numf with the generation digital picture is set to 1802, and uses normal image processing.In the case, the automatic exposure parameter is not made amendment.If lighting level less than the lighting level at border _ a place, can be used the image processing in the hjr pattern so.Pending frame number numf with the generation digital picture is set to 1804.As illustrated among Fig. 5 A and Fig. 5 B, can the time for exposure be reduced a certain amount M 806.Exist M can wider range, yet, be in or about 50% value has been showed good result.Gain also can increase a certain amount K 808.Because the exposure time-gain product in the hjr pattern is intended to keep the exposure time-gain product in the normal mode, thus K be in or about 200% value exhibition reduce the hand shake.
If the gain that increases surpasses the maximum analog gain of imageing sensor, can use the selection 810 of the minimum value of the gain that has increased K between gaining with maximum (simulation) so.Also can in like the district of R2, use it in the configuration of use digital gain therein.Can the ratio 812 of gain that has increased K and new analog gain (ae_gain_new) be compared with constant C 1.C1 is relevant with the maximum analog gain of transducer, and typical value can be 1.Select C1 and increased the minimum value 814 between the ratio of gain and new analog gain of K, and can apply digital gain at said minimum value place.
Some different configurations and technology have been described.Said technology can be improved and from image, remove fuzzy with the longer time for exposure.Said technology is shaken with the auxiliary hand that reduces of in fact any digital device that configuration also can be pictures taken.Said technology and configuration may be implemented in hardware, software, firmware or its any combination.If be implemented in the software; So said technology can be directed to the computer-readable media that comprises computer readable program code (also can be described as computer code) with configuration, one or more in the said computer readable program code method that the execution preceding text are mentioned when carrying out in the device in pictures taken.
Said computer readable program code can computer-readable instruction stored in form on memory.Under said situation, for example the processor of DSP can be carried out and be stored in the instruction in the memory so that carry out one or more in the technology described herein.In some cases, said technology can be carried out by DSP, and said DSP calls the various nextport hardware component NextPorts that multiply each other of for example time for exposure and gain to produce the exposure time-gain product.The exposure time-gain product that is disclosed may be implemented in one or more microprocessors, one or more application-specific integrated circuit (ASIC)s (ASIC) and one or more field programmable gate arrays (FPGA) or a certain other hardware-combination of software.These the technology be configured in the scope of appended claims.
Claims (25)
1. device that is used for carries out image processing, it comprises:
Be used for discerning the device of the lighting level of at least one institute's capture images;
Be used to detect the device that hand is shaken the selection of minimizing pattern;
Be used for dynamically selecting the warp of hand shake minimizing pattern to revise the automatic exposure parameter said at least one institute's capture images is carried out the device of image processing, wherein said hand is shaken having and the identical or approaching identical exposure time-gain product of the exposure time-gain product of normal mode automatic exposure parameter through revising the automatic exposure parameter of minimizing pattern.
2. device according to claim 1, gain and the time for exposure wherein said comprising through modification automatic exposure parameter.
3. device according to claim 1, wherein said warp are revised the automatic exposure parameter and are further comprised frame rate.
4. device that comprises the carries out image processing of hand shake minimizing pattern, it comprises:
Be used to detect the device of lighting level;
In response to the lighting level of said detection being used to reduce the device of hand shake, single frames noise minimizing when the said hand shake of said minimizing comprises when said lighting level is in first scope in and the multiframe noise minimizing in said lighting level is in second scope time;
Be used for dynamically selecting the warp of hand shake minimizing pattern to revise the automatic exposure parameter at least one capture images is carried out the device of image processing, wherein said hand is shaken having and the identical or approaching identical exposure time-gain product of the exposure time-gain product of normal mode automatic exposure parameter through revising the automatic exposure parameter of minimizing pattern; And
Wherein said first scope is between border a and border b;
Said second scope surpasses border b;
Said border a is that image processing and the said hand in the said normal mode trembled the threshold value between the image processing in the minimizing pattern;
Said border b is in maximum and gain is in the point that existence can be accepted the predetermined value of noise level the time for exposure under one frame rate.
5. device according to claim 4, wherein said minimizing hand shake comprises modified automatic exposure parameter.
6. device according to claim 5, wherein said modified automatic exposure parameter is gain, time for exposure and frame rate.
7. device according to claim 4, wherein said multiframe noise minimizing alleviates fuzzy through the multiframe registration.
8. it is that Baeyer filters that device according to claim 4, wherein said single frames noise reduce.
9. device according to claim 4 wherein when the lighting level of said detection is higher than the lighting level of employed minimum when removing fuzzy in the digital picture, carries out the image processing in the hand shake minimizing pattern.
10. device according to claim 8, the lighting level of wherein said detection is mapped to the automatic exposure index.
11. device according to claim 4, wherein said first scope, first frame rate reaches maximum, and as response, said first frame rate reduces and increases the time for exposure to produce second frame rate.
12. device according to claim 11, wherein gain response is in time for exposure of said increase and reduce.
13. device according to claim 4, wherein said second scope comprises two scopes, the 3rd scope and the 4th scope.
14. device according to claim 13, second border between wherein said the 3rd scope and said the 4th scope is to be confirmed by the said lighting level that does not satisfy luma target.
15. device according to claim 14 wherein applies digital gain in said the 3rd scope and said the 4th scope.
16. device according to claim 4, wherein said first scope is between border a and border b.
The b 17. device according to claim 4, wherein said second scope are beyond the boundary.
18. an integrated circuit, it is suitable for being coupled to imageing sensor, with in response to the modified automatic exposure parameter that is produced by said integrated circuit, shakes and shake the hand that reduces the digital picture of capturing in the minimizing pattern at hand, and said integrated circuit comprises:
Be used to discern the device of at least one institute's capture images;
Be used to detect the device that hand is shaken the selection of minimizing pattern;
Be used to discern the device of the lighting level that is associated with said at least one institute's capture images;
Be used for said lighting level is mapped to corresponding gain of normal mode and the device of time for exposure;
Be used for calculating the device of the first exposure time-gain product from corresponding gain and time for exposure; And
Be used for producing modified automatic exposure parameter to reduce the device of hand shake in response to the said exposure time-gain product of said calculating, wherein said modified automatic exposure parameter has and the identical or approaching identical exposure time-gain product of the exposure time-gain product of normal mode automatic exposure parameter.
19. integrated circuit according to claim 18, wherein said being used for produces through revising the device of automatic exposure parameter in response to the said exposure time-gain product of said calculating, comprises and revising said gain and time for exposure.
20. further comprising, integrated circuit according to claim 19, the wherein said device that is used to produce modified automatic exposure parameter calculate the second exposure time-gain product.
21. integrated circuit according to claim 18, wherein said integrated circuit are travelling carriage modulator-demodulator (MSM
TM).
22. a method that produces modified automatic exposure parameter, it comprises:
After snapshot, capture at least one frame;
Get into hand and tremble the minimizing pattern;
The lighting level and the normal mode automatic exposure parameter correlation that are detected are joined;
From memory search time for exposure and gain;
Time for exposure and the gain calculating first exposure time-gain product from said retrieval; And
Produce modified automatic exposure parameter based on the said first exposure time-gain product, wherein said modified automatic exposure parameter has and the identical or approaching identical exposure time-gain product of the exposure time-gain product of said normal mode automatic exposure parameter.
23. method according to claim 22 wherein saidly produces modified automatic exposure parameter based on the said first exposure time-gain product and comprises: multiply by, deduct, add and divided by the time for exposure and the gain of said retrieval.
24. method according to claim 22, gain and the time for exposure wherein said comprising through modification automatic exposure parameter.
25. method according to claim 24, wherein said warp are revised the automatic exposure parameter and are further comprised frame rate.
Applications Claiming Priority (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US11/534,993 | 2006-09-25 | ||
| US11/534,993 US8120658B2 (en) | 2006-01-19 | 2006-09-25 | Hand jitter reduction system for cameras |
| PCT/US2007/060809 WO2008039551A1 (en) | 2006-09-25 | 2007-01-20 | A hand jitter reduction system for cameras |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101518054A CN101518054A (en) | 2009-08-26 |
| CN101518054B true CN101518054B (en) | 2012-11-21 |
Family
ID=39230515
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2007800354470A Active CN101518054B (en) | 2006-09-25 | 2007-01-20 | A hand jitter reduction system for cameras |
Country Status (5)
| Country | Link |
|---|---|
| EP (1) | EP2070312A1 (en) |
| JP (2) | JP5507251B2 (en) |
| KR (1) | KR101120966B1 (en) |
| CN (1) | CN101518054B (en) |
| WO (1) | WO2008039551A1 (en) |
Families Citing this family (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8416319B2 (en) | 2008-09-22 | 2013-04-09 | Freedom Scientific, Inc. | Systems and methods for imaging objects |
| CN101510960B (en) * | 2009-03-26 | 2014-06-25 | 北京中星微电子有限公司 | Mobile phone camera shooting method and apparatus |
| US8659670B2 (en) * | 2009-04-20 | 2014-02-25 | Qualcomm Incorporated | Motion information assisted 3A techniques |
| EP2712174B1 (en) * | 2012-09-21 | 2019-11-20 | BlackBerry Limited | Methods and devices for controlling camera image capture |
| KR101933454B1 (en) * | 2012-09-25 | 2018-12-31 | 삼성전자주식회사 | Photograph image generating method, apparatus therof, and medium storing program source thereof |
| US9646190B2 (en) * | 2013-12-27 | 2017-05-09 | Mitsubishi Electric Corporation | Two-dimensional code reading device |
| US9641759B2 (en) * | 2014-07-10 | 2017-05-02 | Lg Electronics Inc. | Mobile terminal and controlling method thereof |
| CN104219448B (en) * | 2014-08-28 | 2015-12-02 | 小米科技有限责任公司 | Image pickup method and device |
| CN105472263B (en) * | 2014-09-12 | 2018-07-13 | 聚晶半导体股份有限公司 | Image acquisition method and the image capture equipment for using the method |
| FR3027143B1 (en) * | 2014-10-10 | 2016-11-11 | Parrot | MOBILE APPARATUS, IN PARTICULAR ROTATING SAIL DRONE, PROVIDED WITH A VIDEO CAMERA DELIVERING IMAGE SEQUENCES DYNAMICALLY CORRECTED BY THE "WOBBLE" EFFECT |
| KR102176230B1 (en) * | 2017-07-14 | 2020-11-09 | 주식회사 엘지화학 | Method for analyzing polymer layer |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5745808A (en) * | 1995-08-21 | 1998-04-28 | Eastman Kodak Company | Camera exposure control system using variable-length exposure tables |
| JP2000224470A (en) * | 1999-02-02 | 2000-08-11 | Minolta Co Ltd | Camera system |
| US6693673B1 (en) * | 1998-07-28 | 2004-02-17 | Minolta Co., Ltd. | Digital camera for adjusting charge accumulation time in response to object distance |
Family Cites Families (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH1141515A (en) * | 1997-07-24 | 1999-02-12 | Minolta Co Ltd | Still camera |
| JPH11215432A (en) * | 1997-11-19 | 1999-08-06 | Ricoh Co Ltd | Digital camera |
| JPH11215431A (en) * | 1998-01-27 | 1999-08-06 | Hitachi Denshi Ltd | Television camera |
| JP2001045361A (en) * | 1999-07-30 | 2001-02-16 | Hitachi Ltd | Image pickup device |
| JP2003259199A (en) * | 2002-03-05 | 2003-09-12 | Sony Corp | Camera signal processor and exposure range expanding method thereof |
| JP2003322906A (en) * | 2002-04-30 | 2003-11-14 | Olympus Optical Co Ltd | Camera |
| JP3934506B2 (en) * | 2002-08-06 | 2007-06-20 | オリンパス株式会社 | Imaging system and image processing program |
| US7295232B2 (en) * | 2003-01-15 | 2007-11-13 | Canon Kabushiki Kaisha | Camera and program |
| JP2004221992A (en) * | 2003-01-15 | 2004-08-05 | Canon Inc | Imaging device and program |
| JP2004304252A (en) * | 2003-03-28 | 2004-10-28 | Minolta Co Ltd | Imaging systems |
| JP2004357202A (en) * | 2003-05-30 | 2004-12-16 | Canon Inc | Photographing apparatus |
| JP2005286790A (en) * | 2004-03-30 | 2005-10-13 | Fuji Photo Film Co Ltd | Mobile terminal device |
| JP4677748B2 (en) * | 2004-09-15 | 2011-04-27 | 株式会社ニコン | Imaging device |
| WO2006070046A1 (en) * | 2004-12-29 | 2006-07-06 | Nokia Corporation | Exposure of digital imaging |
-
2007
- 2007-01-20 KR KR1020097008384A patent/KR101120966B1/en active IP Right Grant
- 2007-01-20 EP EP07710242A patent/EP2070312A1/en not_active Ceased
- 2007-01-20 WO PCT/US2007/060809 patent/WO2008039551A1/en active Application Filing
- 2007-01-20 JP JP2009530477A patent/JP5507251B2/en active Active
- 2007-01-20 CN CN2007800354470A patent/CN101518054B/en active Active
-
2012
- 2012-04-18 JP JP2012094648A patent/JP5612017B2/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5745808A (en) * | 1995-08-21 | 1998-04-28 | Eastman Kodak Company | Camera exposure control system using variable-length exposure tables |
| US6693673B1 (en) * | 1998-07-28 | 2004-02-17 | Minolta Co., Ltd. | Digital camera for adjusting charge accumulation time in response to object distance |
| JP2000224470A (en) * | 1999-02-02 | 2000-08-11 | Minolta Co Ltd | Camera system |
Also Published As
| Publication number | Publication date |
|---|---|
| JP2010504718A (en) | 2010-02-12 |
| KR20090057461A (en) | 2009-06-05 |
| JP2012182806A (en) | 2012-09-20 |
| KR101120966B1 (en) | 2012-03-05 |
| EP2070312A1 (en) | 2009-06-17 |
| WO2008039551A1 (en) | 2008-04-03 |
| CN101518054A (en) | 2009-08-26 |
| JP5612017B2 (en) | 2014-10-22 |
| JP5507251B2 (en) | 2014-05-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101518054B (en) | A hand jitter reduction system for cameras | |
| US20190028654A1 (en) | Digital camera and display method of digital camera | |
| JP6185213B2 (en) | Imaging apparatus, image processing method of imaging apparatus, and program | |
| CN101212571B (en) | Image capturing apparatus and focusing method | |
| US9237319B2 (en) | Imaging device and automatic focus adjustment method | |
| US7576787B2 (en) | Image-pickup device, and device and method for correcting defective pixel | |
| JP2006222672A (en) | Method and device for controlling white balance and imaging device | |
| JP2007053499A (en) | White balance control unit and imaging apparatus | |
| US11184523B2 (en) | Imaging apparatus with phase difference detecting element | |
| WO2014080718A1 (en) | Image pick-up device and focus control method | |
| US8860876B2 (en) | Image display using flash reach information | |
| US20180262691A1 (en) | Imaging device and control method therefor | |
| JP2007052061A (en) | Imaging device, focusing operation control method, and electronic information apparatus | |
| US20110187903A1 (en) | Digital photographing apparatus for correcting image distortion and image distortion correcting method thereof | |
| JP2013031010A (en) | Imaging apparatus, imaging method, and program | |
| KR20110072213A (en) | Image correction method | |
| JP2007174160A (en) | Imaging apparatus | |
| JP2009282436A (en) | Liquid crystal display device and liquid crystal display method | |
| US20140055666A1 (en) | Image capture apparatus and method of controlling the same | |
| JP2007189639A (en) | Digital camera | |
| JP6103570B2 (en) | Imaging apparatus and control method thereof | |
| TWI394439B (en) | A digital still camera with previewing function by picture in picture and method thereof | |
| JP6810298B2 (en) | Image alignment aids, methods and programs and imaging devices | |
| JP2008071150A (en) | Image processor, image processing program, and photographing device | |
| JP4876642B2 (en) | Imaging apparatus and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |