CN102073993A - Camera self-calibration-based jittering video deblurring method and device - Google Patents
Camera self-calibration-based jittering video deblurring method and device Download PDFInfo
- Publication number
- CN102073993A CN102073993A CN 201010612677 CN201010612677A CN102073993A CN 102073993 A CN102073993 A CN 102073993A CN 201010612677 CN201010612677 CN 201010612677 CN 201010612677 A CN201010612677 A CN 201010612677A CN 102073993 A CN102073993 A CN 102073993A
- Authority
- CN
- China
- Prior art keywords
- image
- blurred
- blurred image
- camera
- calibration
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Abstract
The invention provides a camera self-calibration-based jittering video deblurring method and a camera self-calibration-based jittering video deblurring device. The method comprises the following steps: A, calculating the initial point diffuse function and deblurred image of a blurred image; B, performing self-calibration on the deblurred image to obtain the internal and external parameters of a camera; C, calculating a depth map according to the deblurred image and the internal and external parameters; D, estimating the intra-frame motion of the camera according to the initial point diffuse function and the depth map; E, optimizing intra-frame motion and the deblurred image according to a probability model to obtain the final intra-frame motion and the final deblurred image; and F, executing the steps from A to E circularly to obtain the deblurred video of a jittering video. In the method, depth-associated video image sequence deblurring is realized by a camera self-calibration technique, so the video image sequence has a better deblurring effect and the image deblurring efficiency is improved.
Description
Technical field
The present invention relates to computer vision and digital video image process field, particularly a kind of shake video deblurring method and device based on camera self-calibration.
Background technology
The deblurring of shake video is a kind of video image processing technology.Along with the development of picture pick-up device, the price of various apparatuss for making a video recording reduces significantly, and individual picture pick-up device and various hand-held picture pick-up device are popularized in a large number, cause a large amount of appearance of vedio data.And because image blurring the becoming that the motion of video camera causes when taking reduces one of video image quality principal element, therefore, many image deblurring algorithms are suggested, to be used to repair fuzzy image.Existing algorithm adopts the consistent point spread function hypothesis in space mostly, and the fuzzy core of promptly supposing on the image to be had a few is identical.Yet in fact the point spread function of each point is relevant with motion and the degree of depth of this some place corresponding scenery of video camera in the time shutter on the image.Extreme example is that the scenery in the infinite distance can't thicken because of the translation motion of video camera.Therefore, adopt overall consistent point spread function to make and have, even can cause the violent fringe region of change in depth ringing to occur than some regional deblurring poor effect in the image of big depth range.Therefore, how can especially to shake video image deblurring, and how to make the deblurring better effects if of image become current social problem demanding prompt solution to video image.
Summary of the invention
Purpose of the present invention is intended to solve at least one of above-mentioned technological deficiency.
The present invention be directed to the deblurring poor effect of existing shake video deblurring method, and a kind of shake video image deblurring method and the device based on camera self-calibration that propose.
For achieving the above object, one aspect of the present invention proposes a kind of shake video image deblurring method based on camera self-calibration, and may further comprise the steps: A. is according to the initial point spread function and the de-blurred image of blurred picture in the described shake video of blind deconvolution algorithm computation; B. described de-blurred image is carried out from demarcating to obtain the inside and outside parameter of video camera; C. according to the depth image of described de-blurred image and the described de-blurred image of the inside and outside calculation of parameter of described video camera; D. estimate the intraframe motion of described video camera according to described initial point spread function and described depth image; E. optimize described intraframe motion and described de-blurred image according to probability model, to obtain final intraframe motion and final de-blurred image; And F. circulation carries out A and finishes dealing with until all two field pictures of described shake video to E, obtains the deblurring video of described shake video.
In an embodiment of the present invention, described steps A further comprises: described blurred picture is carried out down-sampling to reduce the blur level of described blurred picture; According to the sparse constraint condition of convolution kernel, adopt the Richardson-Lucy algorithm that described blurred picture is carried out the blind deconvolution computing, to obtain described de-blurred image and convolution kernel; Described convolution kernel is carried out up-sampling, and as initial value described blurred picture is carried out the blind deconvolution computing, with blurred picture after being optimized and convolution kernel with described convolution kernel.
In an embodiment of the present invention, described step B further comprises: according to the gaussian filtering equation described de-blurred image is carried out filtering, to obtain the denoising image; Detect the unique point of described denoising image according to KLT feature point tracking algorithm; The inside and outside parameter of described denoising image is demarcated in employing based on the quadric camera self-calibration algorithm of absolute antithesis.
In an embodiment of the present invention, described step C further comprises: the initial depth figure that calculates described de-blurred image according to adjacent front and back two two field pictures of belief propagation algorithm and described de-blurred image; Adopt the average drifting algorithm that described de-blurred image is carried out color and cut apart, described initial depth figure is carried out match according to carve information; Adopt bundle collection adjustment algorithm that described initial depth figure is optimized, to guarantee the consistance of the degree of depth.
In an embodiment of the present invention, described step D further comprises: D1. estimates the motion of described video camera in the time shutter according to described initial point spread function; D2. according to motion and the described depth image point spread function that obtain the pixel of described de-blurred image of described video camera in the described time shutter; D3. the execution in step that circulates D1 until the operation of finishing each pixel in the described de-blurred image, obtains initial video camera intraframe motion to step D2.
In an embodiment of the present invention, described step e further comprises: E1. sets up the Bayesian probability model according to the gradient of described de-blurred image and the prior distribution of noise; E2. adopt belief propagation algorithm that de-blurred image is optimized; E3. adopt the Levenberg-Marquard algorithm that described intraframe motion is optimized; E4. the execution in step that circulates E2 and step e 3 pre-determined numbers are to obtain final camera motion model and final de-blurred image.
In an embodiment of the present invention, describedly according to the gaussian filtering equation described de-blurred image is carried out filtering, wherein, described gaussian filtering equation is:
Wherein, w is described denoising image, and x, y are the coordinate of described denoising image, and A is the normalization coefficient of described gaussian filtering equation, and σ is the standard deviation of described gaussian filtering equation.
In an embodiment of the present invention, described Bayesian probability model is:
p(I
l，E(t)，D|I
b)κp(I
b|I
l，E(t)，D)p(E(t)|I
l，D)p(I
l，D)，
Wherein, I
lBe described final de-blurred image, E (t) is the t intraframe motion parameter of described video camera constantly, and D is the depth information of described depth image, I
bBe described blurred picture.
The present invention proposes a kind of shake video image deblurring device based on camera self-calibration on the other hand, and comprising: computing module is used for initial point spread function and de-blurred image according to blind deconvolution algorithm computation blurred picture; Demarcating module is used for described de-blurred image is carried out from demarcating to obtain the inside and outside parameter of video camera; The depth image generation module is used for the depth image according to described de-blurred image and the described de-blurred image of the inside and outside calculation of parameter of described video camera; The intraframe motion generation module is used for estimating according to described initial point spread function and described depth image the intraframe motion of described video camera; And optimal module, be used for optimizing described intraframe motion and described de-blurred image, to obtain final intraframe motion and final de-blurred image according to probability model.
Aspect that the present invention adds and advantage part in the following description provide, and part will become obviously from the following description, or recognize by practice of the present invention.
Description of drawings
Above-mentioned and/or additional aspect of the present invention and advantage are from obviously and easily understanding becoming the description of embodiment below in conjunction with accompanying drawing, wherein:
Fig. 1 is the process flow diagram based on the shake video deblurring method of camera self-calibration of the embodiment of the invention; And
Fig. 2 is the structural drawing based on the shake video deblurring device of camera self-calibration of the embodiment of the invention.
Embodiment
Describe whole embodiment of the present invention below in detail, the example of described embodiment is shown in the drawings, and wherein identical from start to finish or similar label is represented identical or similar elements or the element with identical or similar functions.Below by the embodiment that is described with reference to the drawings is exemplary, only is used to explain the present invention, and can not be interpreted as limitation of the present invention.
The present invention be directed to a kind of shake video deblurring method and device that existing method proposes deblurring poor effect with big depth range image based on camera self-calibration.Understand for the method to the embodiment of the invention has more clearly, below just in conjunction with the accompanying drawings the workflow and the principle of work of the method and apparatus of the embodiment of the invention are done detailed description.
As shown in Figure 1, be the process flow diagram based on the shake video deblurring method of camera self-calibration of the embodiment of the invention.In specific embodiments of the invention, this method may further comprise the steps:
Step S101, A. is according to the initial point spread function and the de-blurred image of blind deconvolution algorithm computation blurred picture.
Particularly, in one embodiment of the invention, at first this blurred picture is carried out down-sampling, purpose is to reduce the blur level of this blurred picture, and for next using this image of blind deconvolution algorithm process to reduce difficulty.Then, after this image is carried out deblurring, adopt the Richardson-Lucy algorithm that this blurred picture is carried out the blind deconvolution computing, thereby obtain image and convolution kernel after the deblurring of this blurred picture, here need to prove, when the utilization blind deconvolution was tried to achieve this convolution kernel, prerequisite was for to use this constraint condition of sparse constraint to convolution kernel, that is: each pixel on this blurred picture is to have a part of weighting summation in the corresponding picture rich in detail surrounding pixel to obtain.
Step S102, B. carries out from demarcating to obtain the inside and outside parameter of video camera described de-blurred image.
Particularly, in one embodiment of the invention, 1, at first, the image after using the gaussian filtering equation to this deblurring carries out filtering to reduce the influence of picture noise to the camera self-calibration method that next carries out, more specifically, this gaussian filtering equation is:
In this equation, w is described denoising image, and x, y are the coordinate of described denoising image, and A is the normalization coefficient of described gaussian filtering equation, and σ is the standard deviation of described gaussian filtering equation, by this equation this de-blurred image is carried out denoising.
2, then, adopt KLT feature point tracking algorithm to detect the unique point of this denoising image and follow the tracks of unique point in this denoising image.
3, adopt the ransac algorithm that the fundamental matrix of this denoising image and its each two field picture of front and back is carried out Robust Estimation then, and the unique point deletion of exterior point will be classified as in the estimation procedure, because these matching characteristic points do not satisfy that the fundamental matrix calculate describes to utmost point geometrical constraint, so be considered to error matching points.
4, subsequently, according to the robust features point matching relationship that obtains in 3 this denoising image is carried out projective reconstruction, restructuring procedure more detailed step be: at first determine two width of cloth initialization views, set up world coordinate system, rebuild according to the triangle projection relation then and obtain 3 corresponding dimension space coordinates of matching characteristic point, and circulation adds new picture, all joins in the projective reconstruction until all pictures, thereby image scene has been carried out reconstruct.
5, adopt linear calibration's algorithm directly to calibrate the transformation matrix that projective space is rebuild to tolerance.With absolute antithesis quadric surface
Be decomposed into SS
T, wherein S is the companion matrix of 4 λ 3, like this can basis
Linear restriction will be converted into to the linear restriction of confidential reference items matrix K, SS can be proved in theory S
TObtain
Have orthotropicity, and its order is 3.
6, S is mended rows of vectors, the 4 rank square formations that obtain are reversible, and this square formation contrary is exactly this projection to be rebuild a transformation matrix that tolerance is rebuild of conversion.
7, the tolerance that conversion is obtained is rebuild the video camera projection matrix under the meaning, carries out RQ and decomposes, and obtaining the confidential reference items matrix K of video camera respectively and taking each two field picture is the rotation matrix R translation vector T of video camera and the essential matrix that calculates video camera
(the whole attitude informations that wherein comprise video camera) finish camera self-calibration.In a preferred embodiment of the invention, the intrinsic parameter of described video camera is 5, and outer parameter is 6.
Step S103, C. is according to the depth image of described de-blurred image and the described de-blurred image of the inside and outside calculation of parameter of described video camera.
Particularly, in one embodiment of the invention, at first adopt adjacent front and back two two field pictures of belief propagation algorithm and this de-blurred image to calculate the initial depth figure that changes de-blurred image.Then adopt the average drifting algorithm that this de-blurred image is carried out color and cut apart, and the pairing initial depth figure of this de-blurred image is carried out match according to carve information.At last, have consistance, in specific embodiments of the invention, adopt bundle collection adjustment algorithm that the depth image of all de-blurred image is carried out combined optimization in order to ensure the degree of depth of each two field picture.
Step S104, D. estimate the intraframe motion of described video camera according to described initial point spread function and described depth image.
Particularly, in one embodiment of the invention, this step S104 comprises:
Step 1 is estimated the motion of this video camera in the time shutter according to the initial global point spread function that obtains among the step S101;
Step 2, the depth image that obtains among motion in the time shutter that obtains in step 1 according to this video camera and the step S103 obtains the point spread function of each pixel of corresponding de-blurred image, compare with initial global point spread function, the point spread function of each pixel is more accurate.
Step 3, circulation execution in step 1 until the operation of finishing each pixel in this de-blurred image, obtain the motion of initial video camera in this frame to step 2.
More specifically, can regard on the de-blurred image corresponding point of each pixel in three-dimensional scenic on the point spread function as along with the movement locus of motion imaging on the CCD imaging plane of video camera in the time shutter.Can be designated as
Wherein, x represents this pixel coordinate on de-blurred image, and d represents that this puts the corresponding degree of depth, t express time, x ' be illustrated in t constantly on the de-blurred image pixel coordinate be x, the degree of depth is the pixel coordinate of d imaging on the video camera imaging plane.According to the video camera projection model, can try to achieve corresponding video camera at t moment kinematic parameter by the initial point spread function with method of addition.Wherein, in a preferred embodiment of the invention, this projection model is the pinhole imaging system model.Certainly; those skilled in the art will know that; this pinhole imaging system model is only specifically used as one of one embodiment of the invention; also can use other projection model; as: the quadrature imaging model; affine projection models etc., these conversion and changes based on inventive concept all should be classified protection scope of the present invention as.
At first, choose the unique point of texture-rich (can select the bigger point of SIFT proper vector mould value) conduct on every side with reference to point on de-blurred image, in a preferred embodiment of the invention, the number of reference point is more than or equal to 9.
Then, according to the initial point spread function that obtains before, suppose that t coordinate constantly is x ', order
Can try to achieve the increment of this moment video camera according to following equation with respect to the video camera attitude E of de-blurred image correspondence
Adopt the RANSAC algorithm to utilize following formula can try to achieve the increment of video camera in t relative movement parameters constantly
At last, adopt the SVD decomposition algorithm, with video camera in t actual motion parameter constantly
Decompose and obtain rotation matrix R and translation vector T.
Step S105, E. optimizes described intraframe motion and described de-blurred image according to probability model, to obtain final intraframe motion and final de-blurred image.Particularly, in one embodiment of the invention, step S105 may further comprise the steps:
Step 1 is set up the Bayesian probability model according to the gradient of this de-blurred image and the prior distribution of noise;
Step 2 adopts belief propagation algorithm that this de-blurred image is optimized according to the Bayesian probability model that obtains in the step 1;
Step 3 adopts the Levenberg-Marquard algorithm that this intraframe motion is optimized according to described Bayesian probability model;
Step 4, circulation execution in step 2 and step 3 pre-determined number, thus obtain final camera motion model and final de-blurred image.
More specifically, set up the Bayesian probability model according to the gradient of this de-blurred image and the prior distribution of noise.Wherein, the representation of this Bayesian probability model is shown below: p (I
l, E (t), D|I
b) κ p (I
b| I
l, E (t), D) p (E (t) | I
l, D) p (I
l, D) 1)
In this Bayesian probability model, I
lBe described final de-blurred image, E (t) is the t intraframe motion parameter of described video camera constantly, and D is the depth information of described depth image, I
bBe described blurred picture.1) levoform (p (I in the formula
l, E (t), D|I
b)) be illustrated in given blurred picture I
bCondition under picture rich in detail to be asked, video camera intraframe motion and scene depth probability distribution, according to the maximum a posteriori criterion, require this 1) formula obtains maximization, calculate for convenience, to 1) formula gets negative logarithm.Right formula (p (I
b| I
l, E (t), D) p (E (t) | I
l, D) p (I
l, D)) in first p (I
b| I
l, E (t) D) is defined as:
In the following formula
Wherein,
Because the motion E (t) and the I of video camera
l, D is separate, thus second p of following formula right-hand member (E (t) | I
l, D) become p (E (t)), can suppose even distribution in the reality, can ignore this influence.The 3rd p (I of following formula right-hand member
l, D) be degree of depth constraint, can decompose as follows:
Wherein
Be the depth map energy function, can use the depth map derivation algorithm that it is optimized,
Wherein, k, a, b represent the parameter of prior distribution, can get for general pattern
Adopt belief propagation algorithm that de-blurred image is optimized I
l
Adopt the Levenberg-Marquard algorithm that kinematic parameter between camera frame is optimized;
Two steps operation on the loop iteration, convergence obtains final camera motion model and de-blurred image.
Step S106, the F. circulation is carried out A and is finished dealing with until all two field pictures of described shake video to E, obtains the deblurring video of described shake video.
Another aspect of the present invention also proposes a kind of shake video deblurring device based on camera self-calibration, as shown in Figure 2, is the structural drawing based on the shake video deblurring device of camera self-calibration of the embodiment of the invention.In specific embodiments of the invention, should comprise computing module 201, demarcating module 202, depth image generation module 203, intraframe motion generation module 204 and optimal module 205 based on the shake video deblurring device 200 of camera self-calibration.Wherein, computing module 201 act as initial point spread function and de-blurred image according to blind deconvolution algorithm computation blurred picture, acting as of demarcating module 202 carried out from demarcating to obtain in the video camera described de-blurred image, outer parameter, acting as of depth image generation module 203 according in described de-blurred image and the described video camera, the depth image of the described de-blurred image of outer calculation of parameter, intraframe motion generation module 204 act as the intraframe motion of estimating described video camera according to described initial point spread function and described depth image, described intraframe motion and described de-blurred image are optimized in acting as according to probability model of optimal module 205, to obtain final intraframe motion and final de-blurred image.
Particularly, in one embodiment of the invention, computing module 201 at first carries out down-sampling to reduce the blur level of described blurred picture to described blurred picture, then according to the sparse constraint condition of convolution kernel, adopt the Richardson-Lucy algorithm that described blurred picture is carried out the blind deconvolution computing, to obtain described de-blurred image and convolution kernel, at last described convolution kernel is carried out up-sampling, and as initial value described blurred picture is carried out the blind deconvolution computing with described convolution kernel, with blurred picture after being optimized and convolution kernel.
In one embodiment of the invention, demarcating module 202 at first carries out filtering according to the gaussian filtering equation to described de-blurred image, to obtain the denoising image, detect the unique point of described denoising image then according to KLT feature point tracking algorithm, adopt the inside and outside parameter of demarcating described denoising image based on the quadric camera self-calibration algorithm of absolute antithesis at last.
In one embodiment of the invention, depth image generation module 203 at first calculates the initial depth figure of described de-blurred image according to adjacent front and back two two field pictures of belief propagation algorithm and described de-blurred image, adopting the average drifting algorithm that described de-blurred image is carried out color then cuts apart, described initial depth figure is carried out match according to carve information, adopt bundle collection adjustment algorithm that described initial depth figure is optimized at last, to guarantee the consistance of the degree of depth.
In one embodiment of the invention, intraframe motion generation module 204 comprises step 1, estimate the motion of described video camera in the time shutter according to described initial point spread function, step 2, according to motion and the described depth image point spread function that obtain the pixel of described de-blurred image of described video camera in the described time shutter, step 3, circulation execution in step 1 until the operation of finishing each pixel in the described de-blurred image, obtain initial video camera intraframe motion to step 2.
In one embodiment of the invention, optimal module 205 comprises step 1, set up the Bayesian probability model according to the gradient of described de-blurred image and the prior distribution of noise, step 2, adopt belief propagation algorithm that de-blurred image is optimized according to described Bayesian probability model, step 3, adopt the Levenberg-Marquard algorithm that described intraframe motion is optimized according to described Bayesian probability model, step 4, circulation execution in step 2 and step 3 pre-determined number are to obtain final camera motion model and final de-blurred image.
Shake video deblurring method and device by embodiment of the invention proposition based on camera self-calibration, this method is by camera self-calibration technical calibration camera parameters, and then try to achieve depth image, and according to the three-dimensional coordinate of each pixel in the image and each pixel of motion calculation point spread function separately of video camera, thereby realized the relevant video image deblurring of the degree of depth, than compared the better blur effect that becomes according to the method deblurring of entire frame image overall point spread function in the past, and this method realizes simple, and this installs easy operating.
Although illustrated and described embodiments of the invention, for the ordinary skill in the art, be appreciated that without departing from the principles and spirit of the present invention and can carry out multiple variation, modification, replacement and modification that scope of the present invention is by claims and be equal to and limit to these embodiment.
Claims (14)
1. the shake video deblurring method based on camera self-calibration is characterized in that, may further comprise the steps:
A. according to the initial point spread function and the de-blurred image of blurred picture in the described shake video of blind deconvolution algorithm computation;
B. described de-blurred image is carried out from demarcating to obtain the inside and outside parameter of video camera;
C. according to the depth image of described de-blurred image and the described de-blurred image of the inside and outside calculation of parameter of described video camera;
D. estimate the intraframe motion of described video camera according to described initial point spread function and described depth image;
E. optimize described intraframe motion and described de-blurred image according to probability model, to obtain final intraframe motion and final de-blurred image; And
F. circulation execution A finishes dealing with until all two field pictures of described shake video to E, obtains the deblurring video of described shake video.
2. the shake video deblurring method based on camera self-calibration as claimed in claim 1 is characterized in that described steps A further comprises:
Described blurred picture is carried out down-sampling to reduce the blur level of described blurred picture;
According to the sparse constraint condition of convolution kernel, adopt the Richardson-Lucv algorithm that described blurred picture is carried out the blind deconvolution computing, to obtain described de-blurred image and convolution kernel;
Described convolution kernel is carried out up-sampling, and as initial value described blurred picture is carried out the blind deconvolution computing, with blurred picture after being optimized and convolution kernel with described convolution kernel.
3. the shake video deblurring method based on camera self-calibration as claimed in claim 1 is characterized in that described step B further comprises:
According to the gaussian filtering equation described de-blurred image is carried out filtering, to obtain the denoising image;
Detect the unique point of described denoising image according to KLT feature point tracking algorithm;
The inside and outside parameter of described denoising image is demarcated in employing based on the quadric camera self-calibration algorithm of absolute antithesis.
4. the shake video deblurring method based on camera self-calibration as claimed in claim 1 is characterized in that described step C further comprises:
Calculate the initial depth figure of described de-blurred image according to adjacent front and back two two field pictures of belief propagation algorithm and described de-blurred image;
Adopt the average drifting algorithm that described de-blurred image is carried out color and cut apart, described initial depth figure is carried out match according to carve information;
Adopt bundle collection adjustment algorithm that described initial depth figure is optimized.
5. the shake video deblurring method based on camera self-calibration as claimed in claim 1 is characterized in that described step D further comprises:
D1. estimate the motion of described video camera in the time shutter according to described initial point spread function;
D2. according to motion and the described depth image point spread function that obtain the pixel of described de-blurred image of described video camera in the described time shutter;
D3. the execution in step that circulates D1 until the operation of finishing each pixel in the described de-blurred image, obtains initial video camera intraframe motion to step D2.
6. the shake video deblurring method based on camera self-calibration as claimed in claim 1 is characterized in that described step e further comprises:
E1. set up the Bayesian probability model according to the gradient of described de-blurred image and the prior distribution of noise;
E2. adopt belief propagation algorithm that de-blurred image is optimized according to described Bayesian probability model;
E3. adopt the Levenberg-Marquard algorithm that described intraframe motion is optimized according to described Bayesian probability model;
E4. the execution in step that circulates E2 and step e 3 pre-determined numbers are to obtain final camera motion model and final de-blurred image.
7. the shake video deblurring method based on camera self-calibration as claimed in claim 3 is characterized in that, describedly according to the gaussian filtering equation described de-blurred image is carried out filtering, and wherein, described gaussian filtering equation is:
Wherein, w is described denoising image, and x, y are the coordinate of described denoising image, and A is the normalization coefficient of described gaussian filtering equation, and σ is the standard deviation of described gaussian filtering equation.
8. the shake video deblurring method based on camera self-calibration as claimed in claim 6 is characterized in that, described Bayesian probability model is:
p(I
l，E(t)，D|I
b)∝p(I
b|I
l，E(t)，D)p(E(t)|I
l，D)p(I
l，D)，
Wherein, I
lBe described final de-blurred image, E (t) is the t intraframe motion parameter of described video camera constantly, and D is the depth information of described depth image, I
bBe described blurred picture.
9. the shake video deblurring device based on camera self-calibration is characterized in that, comprising:
Computing module is used for initial point spread function and de-blurred image according to the described shake video of blind deconvolution algorithm computation blurred picture;
Demarcating module is used for described de-blurred image is carried out from demarcating to obtain the inside and outside parameter of video camera;
The depth image generation module is used for the depth image according to described de-blurred image and the described de-blurred image of the inside and outside calculation of parameter of described video camera;
The intraframe motion generation module is used for estimating according to described initial point spread function and described depth image the intraframe motion of described video camera; And
Optimal module is used for optimizing described intraframe motion and described de-blurred image according to probability model, to obtain final intraframe motion and final de-blurred image.
10. the shake video deblurring device based on camera self-calibration as claimed in claim 9, it is characterized in that, described computing module carries out down-sampling to reduce the blur level of described blurred picture to described blurred picture, and according to the sparse constraint condition of convolution kernel, adopt the Richardson-Lucy algorithm that described blurred picture is carried out the blind deconvolution computing, to obtain described de-blurred image and convolution kernel, and described convolution kernel carried out up-sampling, and as initial value described blurred picture is carried out the blind deconvolution computing with described convolution kernel, with blurred picture after being optimized and convolution kernel.
11. the shake video deblurring device based on camera self-calibration as claimed in claim 9, it is characterized in that, described demarcating module carries out filtering according to the gaussian filtering equation to described de-blurred image, to obtain the denoising image, and detect the unique point of described denoising image, and adopt the inside and outside parameter of demarcating described denoising image based on the quadric camera self-calibration algorithm of absolute antithesis according to KLT feature point tracking algorithm.
12. the shake video deblurring device based on camera self-calibration as claimed in claim 9, it is characterized in that, described depth image generation module calculates the initial depth figure of described de-blurred image according to adjacent front and back two two field pictures of belief propagation algorithm and described de-blurred image, and adopt the average drifting algorithm that described de-blurred image is carried out color to cut apart, described initial depth figure is carried out match according to carve information, and adopt bundle collection adjustment algorithm that described initial depth figure is optimized, to guarantee the consistance of the degree of depth.
13. the shake video deblurring device based on camera self-calibration as claimed in claim 9, it is characterized in that, described intraframe motion generation module is estimated the motion of described video camera in the time shutter according to described initial point spread function, and according to motion and the described depth image point spread function that obtain the pixel of described de-blurred image of described video camera in the described time shutter; And the circulation execution obtains initial video camera intraframe motion until the operation of finishing each pixel in the described de-blurred image.
14. the shake video deblurring device based on camera self-calibration as claimed in claim 9 is characterized in that the execution of described optimal module comprises the steps:
Step 1 is set up the Bayesian probability model according to the gradient of described de-blurred image and the prior distribution of noise;
Step 2 adopts belief propagation algorithm that de-blurred image is optimized according to described Bayesian probability model;
Step 3 adopts the Levenberg-Marquard algorithm that described intraframe motion is optimized according to described Bayesian probability model;
Step 4, circulation execution in step 2 and step 3 pre-determined number are to obtain final camera motion model and final de-blurred image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2010106126772A CN102073993B (en) | 2010-12-29 | 2010-12-29 | Camera self-calibration-based jittering video deblurring method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2010106126772A CN102073993B (en) | 2010-12-29 | 2010-12-29 | Camera self-calibration-based jittering video deblurring method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102073993A true CN102073993A (en) | 2011-05-25 |
| CN102073993B CN102073993B (en) | 2012-08-22 |
Family
ID=44032523
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2010106126772A Active CN102073993B (en) | 2010-12-29 | 2010-12-29 | Camera self-calibration-based jittering video deblurring method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102073993B (en) |
Cited By (22)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102436639A (en) * | 2011-09-02 | 2012-05-02 | 清华大学 | Image acquiring method for removing image blurring and image acquiring system |
| CN102509294A (en) * | 2011-11-08 | 2012-06-20 | 清华大学深圳研究生院 | Single-image-based global depth estimation method |
| CN102609977A (en) * | 2012-01-12 | 2012-07-25 | 浙江大学 | Depth integration and curved-surface evolution based multi-viewpoint three-dimensional reconstruction method |
| CN102663718A (en) * | 2012-03-19 | 2012-09-12 | 清华大学 | Method and system for deblurring of gloablly inconsistent image |
| CN102750679A (en) * | 2012-06-28 | 2012-10-24 | 西安电子科技大学 | Blind deblurring method for image quality evaluation |
| CN103413277A (en) * | 2013-08-19 | 2013-11-27 | 南京邮电大学 | Blind camera shake deblurring method based on L0 sparse prior |
| CN103426144A (en) * | 2012-05-17 | 2013-12-04 | 佳能株式会社 | Method and device for deblurring image having perspective distortion |
| CN104867111A (en) * | 2015-03-27 | 2015-08-26 | 北京理工大学 | Block-blur-kernel-set-based heterogeneous video blind deblurring method |
| CN104932868A (en) * | 2014-03-17 | 2015-09-23 | 联想(北京)有限公司 | Data processing method and electronic device |
| CN104954631A (en) * | 2014-03-25 | 2015-09-30 | 腾讯科技（深圳）有限公司 | Video processing method, device and system |
| CN106683174A (en) * | 2016-12-23 | 2017-05-17 | 上海斐讯数据通信技术有限公司 | 3D reconstruction method, apparatus of binocular visual system, and binocular visual system |
| CN106920229A (en) * | 2017-01-22 | 2017-07-04 | 北京奇艺世纪科技有限公司 | Image obscuring area automatic testing method and system |
| CN107369135A (en) * | 2017-06-22 | 2017-11-21 | 广西大学 | A kind of micro imaging system three-dimensional point spread function space size choosing method based on Scale invariant features transform algorithm |
| CN107590781A (en) * | 2017-08-17 | 2018-01-16 | 天津大学 | Adaptive weighted TGV image deblurring methods based on primal dual algorithm |
| CN109151281A (en) * | 2018-09-26 | 2019-01-04 | 中国计量大学 | A kind of pixel aperture offset camera obtaining depth information |
| CN110062164A (en) * | 2019-04-22 | 2019-07-26 | 深圳市商汤科技有限公司 | Method of video image processing and device |
| CN111445404A (en) * | 2020-03-23 | 2020-07-24 | 上海数迹智能科技有限公司 | Phase deblurring method based on dual-frequency sum probability model |
| CN113077395A (en) * | 2021-03-26 | 2021-07-06 | 东北大学 | Deblurring method for large-size sample image under high-power optical microscope |
| CN113222863A (en) * | 2021-06-04 | 2021-08-06 | 中国铁道科学研究院集团有限公司 | High-speed railway operation environment based video self-adaptive deblurring method and device |
| CN113409200A (en) * | 2020-03-17 | 2021-09-17 | 通用汽车环球科技运作有限责任公司 | System and method for image deblurring in a vehicle |
| CN115047890A (en) * | 2022-08-17 | 2022-09-13 | 北京大学 | Unmanned ship control method, unmanned ship control device and computer-readable storage medium |
| CN115396644A (en) * | 2022-07-21 | 2022-11-25 | 贝壳找房(北京)科技有限公司 | Video fusion method and device based on multi-segment external parameter data |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP3251127B2 (en) * | 1993-08-05 | 2002-01-28 | ソニー・ユナイテッド・キングダム・リミテッド | Video data processing method |
| US20070098292A1 (en) * | 2005-10-28 | 2007-05-03 | Texas Instruments Incorporated | Digital Camera and Method |
| US7379612B2 (en) * | 2004-12-16 | 2008-05-27 | The Regents Of The University Of California, Santa Cruz | Dynamic reconstruction of high-resolution video from color-filtered low-resolution video-to-video super-resolution |
| CN101406041A (en) * | 2006-05-08 | 2009-04-08 | 三菱电机株式会社 | Method for reducing blur in an image of a scene and method for deblurring an image of a scene |
| CN101727663A (en) * | 2008-10-13 | 2010-06-09 | 索尼株式会社 | Method and system for image deblurring |
-
2010
- 2010-12-29 CN CN2010106126772A patent/CN102073993B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP3251127B2 (en) * | 1993-08-05 | 2002-01-28 | ソニー・ユナイテッド・キングダム・リミテッド | Video data processing method |
| US7379612B2 (en) * | 2004-12-16 | 2008-05-27 | The Regents Of The University Of California, Santa Cruz | Dynamic reconstruction of high-resolution video from color-filtered low-resolution video-to-video super-resolution |
| US20070098292A1 (en) * | 2005-10-28 | 2007-05-03 | Texas Instruments Incorporated | Digital Camera and Method |
| CN101406041A (en) * | 2006-05-08 | 2009-04-08 | 三菱电机株式会社 | Method for reducing blur in an image of a scene and method for deblurring an image of a scene |
| CN101727663A (en) * | 2008-10-13 | 2010-06-09 | 索尼株式会社 | Method and system for image deblurring |
Non-Patent Citations (2)
| Title |
|---|
| 《Appeared in ACM Transactions on Graphics》 20090831 Amit Agrawal等 Invertible Motion Blur in Video 第28卷, 第3期 2 * |
| 《信号处理》 20100831 李沛秦 等 一种面向目标区域的快速去模糊算法 第26卷, 第8期 2 * |
Cited By (37)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102436639A (en) * | 2011-09-02 | 2012-05-02 | 清华大学 | Image acquiring method for removing image blurring and image acquiring system |
| CN102436639B (en) * | 2011-09-02 | 2013-12-04 | 清华大学 | Image acquiring method for removing image blurring and image acquiring system |
| CN102509294B (en) * | 2011-11-08 | 2013-09-25 | 清华大学深圳研究生院 | Single-image-based global depth estimation method |
| CN102509294A (en) * | 2011-11-08 | 2012-06-20 | 清华大学深圳研究生院 | Single-image-based global depth estimation method |
| CN102609977B (en) * | 2012-01-12 | 2014-07-16 | 浙江大学 | Depth integration and curved-surface evolution based multi-viewpoint three-dimensional reconstruction method |
| CN102609977A (en) * | 2012-01-12 | 2012-07-25 | 浙江大学 | Depth integration and curved-surface evolution based multi-viewpoint three-dimensional reconstruction method |
| CN102663718B (en) * | 2012-03-19 | 2015-06-24 | 清华大学 | Method and system for deblurring of gloablly inconsistent image |
| CN102663718A (en) * | 2012-03-19 | 2012-09-12 | 清华大学 | Method and system for deblurring of gloablly inconsistent image |
| CN103426144A (en) * | 2012-05-17 | 2013-12-04 | 佳能株式会社 | Method and device for deblurring image having perspective distortion |
| CN103426144B (en) * | 2012-05-17 | 2016-05-11 | 佳能株式会社 | For making the method and apparatus of the image deblurring with perspective distortion |
| CN102750679B (en) * | 2012-06-28 | 2014-07-09 | 西安电子科技大学 | Blind deblurring method for image quality evaluation |
| CN102750679A (en) * | 2012-06-28 | 2012-10-24 | 西安电子科技大学 | Blind deblurring method for image quality evaluation |
| CN103413277B (en) * | 2013-08-19 | 2017-04-05 | 南京邮电大学 | Blind camera shake deblurring method based on L0 sparse priors |
| CN103413277A (en) * | 2013-08-19 | 2013-11-27 | 南京邮电大学 | Blind camera shake deblurring method based on L0 sparse prior |
| CN104932868A (en) * | 2014-03-17 | 2015-09-23 | 联想(北京)有限公司 | Data processing method and electronic device |
| CN104954631B (en) * | 2014-03-25 | 2018-02-27 | 腾讯科技（深圳）有限公司 | A kind of method for processing video frequency, device and system |
| CN104954631A (en) * | 2014-03-25 | 2015-09-30 | 腾讯科技（深圳）有限公司 | Video processing method, device and system |
| CN104867111A (en) * | 2015-03-27 | 2015-08-26 | 北京理工大学 | Block-blur-kernel-set-based heterogeneous video blind deblurring method |
| CN104867111B (en) * | 2015-03-27 | 2017-08-25 | 北京理工大学 | A kind of blind deblurring method of non-homogeneous video based on piecemeal fuzzy core collection |
| CN106683174A (en) * | 2016-12-23 | 2017-05-17 | 上海斐讯数据通信技术有限公司 | 3D reconstruction method, apparatus of binocular visual system, and binocular visual system |
| CN106920229A (en) * | 2017-01-22 | 2017-07-04 | 北京奇艺世纪科技有限公司 | Image obscuring area automatic testing method and system |
| CN106920229B (en) * | 2017-01-22 | 2021-01-05 | 北京奇艺世纪科技有限公司 | Automatic detection method and system for image fuzzy area |
| CN107369135A (en) * | 2017-06-22 | 2017-11-21 | 广西大学 | A kind of micro imaging system three-dimensional point spread function space size choosing method based on Scale invariant features transform algorithm |
| CN107590781B (en) * | 2017-08-17 | 2020-11-27 | 天津大学 | Self-adaptive weighted TGV image deblurring method based on original dual algorithm |
| CN107590781A (en) * | 2017-08-17 | 2018-01-16 | 天津大学 | Adaptive weighted TGV image deblurring methods based on primal dual algorithm |
| CN109151281A (en) * | 2018-09-26 | 2019-01-04 | 中国计量大学 | A kind of pixel aperture offset camera obtaining depth information |
| CN110062164A (en) * | 2019-04-22 | 2019-07-26 | 深圳市商汤科技有限公司 | Method of video image processing and device |
| CN113409200B (en) * | 2020-03-17 | 2023-09-29 | 通用汽车环球科技运作有限责任公司 | System and method for image deblurring in a vehicle |
| CN113409200A (en) * | 2020-03-17 | 2021-09-17 | 通用汽车环球科技运作有限责任公司 | System and method for image deblurring in a vehicle |
| CN111445404A (en) * | 2020-03-23 | 2020-07-24 | 上海数迹智能科技有限公司 | Phase deblurring method based on dual-frequency sum probability model |
| CN113077395A (en) * | 2021-03-26 | 2021-07-06 | 东北大学 | Deblurring method for large-size sample image under high-power optical microscope |
| CN113077395B (en) * | 2021-03-26 | 2023-10-24 | 东北大学 | Deblurring method for large-size sample image under high-power optical microscope |
| CN113222863A (en) * | 2021-06-04 | 2021-08-06 | 中国铁道科学研究院集团有限公司 | High-speed railway operation environment based video self-adaptive deblurring method and device |
| CN113222863B (en) * | 2021-06-04 | 2024-04-16 | 中国铁道科学研究院集团有限公司 | Video self-adaptive deblurring method and device based on high-speed railway operation environment |
| CN115396644B (en) * | 2022-07-21 | 2023-09-15 | 贝壳找房(北京)科技有限公司 | Video fusion method and device based on multi-section external reference data |
| CN115396644A (en) * | 2022-07-21 | 2022-11-25 | 贝壳找房(北京)科技有限公司 | Video fusion method and device based on multi-segment external parameter data |
| CN115047890A (en) * | 2022-08-17 | 2022-09-13 | 北京大学 | Unmanned ship control method, unmanned ship control device and computer-readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102073993B (en) | 2012-08-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102073993B (en) | Camera self-calibration-based jittering video deblurring method and device | |
| EP2291999B1 (en) | Image deconvolution using color priors | |
| CN102436639B (en) | Image acquiring method for removing image blurring and image acquiring system | |
| CN112132958B (en) | Underwater environment three-dimensional reconstruction method based on binocular vision | |
| CN109978774B (en) | Denoising fusion method and device for multi-frame continuous equal exposure images | |
| Šindelář et al. | Image deblurring in smartphone devices using built-in inertial measurement sensors | |
| US20140307982A1 (en) | Multi-frame super-resolution of image sequence with arbitrary motion patterns | |
| Im et al. | High quality structure from small motion for rolling shutter cameras | |
| US9253415B2 (en) | Simulating tracking shots from image sequences | |
| US9342873B1 (en) | Tile-based optical flow | |
| Hervieu et al. | Stereoscopic image inpainting: distinct depth maps and images inpainting | |
| GB2536430A (en) | Image noise reduction | |
| Singh et al. | Weighted least squares based detail enhanced exposure fusion | |
| US9860441B1 (en) | Tile-based digital image correspondence | |
| JP2009111921A (en) | Image processing device and image processing method | |
| CN104318518A (en) | Projection-onto-convex-sets image reconstruction method based on SURF matching and edge detection | |
| Saleem et al. | A non-reference evaluation of underwater image enhancement methods using a new underwater image dataset | |
| CN109712230B (en) | Three-dimensional model supplementing method and device, storage medium and processor | |
| Kriener et al. | Accelerating defocus blur magnification | |
| Mohan | Adaptive super-resolution image reconstruction with lorentzian error norm | |
| Zhang et al. | Motion-blurring free integral imaging reconstruction via parallax information | |
| Yue et al. | High-dimensional camera shake removal with given depth map | |
| Fan et al. | Non-aligned supervision for real image dehazing | |
| van Dam et al. | Landmark-based model-free 3d face shape reconstruction from video sequences | |
| Lee et al. | Disparity Refinement with Guided Filtering of Soft 3D Cost Function in Multi-view Stereo System |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |