CN104361561A - Raindrop detection and removal method based on logarithmic image processing - Google Patents

Abstract

The invention discloses a raindrop detection and removal method based on logarithmic image processing. The method includes 1, mapping the incident light intensity to a gray tone space according to the logarithmic transformation principle; 2, performing gray tone constraint, performing nonlinear transformation in the gray tone space, setting a gray tone threshold, and detecting whether a current frame pixel is a raindrop or not by detecting the gray tone values of three frames before and after; 3, if so, adopting the pixel as a raindrop pixel; if not, adopting the pixel as a background pixel; 4, removing the raindrop of the raindrop pixel by the frame difference method, and acquiring the raindrop-removed pixel. According to the method, the logarithmic image processing meets image physical characteristics and human visual perception rule better, the raindrop detecting effect is fine, the raindrop-removed image is enhanced through logarithmic transformation, and the image quality of the low gray value portion is improved significantly.

Description

A kind of video raindrop based on Logarithmic image processing detect and minimizing technology

Technical field

The present invention relates to technical field of computer vision, particularly relate to a kind of video raindrop based on Logarithmic image processing and detect and minimizing technology.

Background technology

The present invention is mainly used in the Postprocessing technique in computer vision, and outdoor vision system is widely used in the fields such as military and national defense, medical skill, intelligent transportation, Industry Control, contacts also more and more closer with our life.But inclement weather has had a strong impact on the performance of outdoor vision system, so carry out pre-service to video image, eliminates the impact of various weather conditions, be absolutely necessary for a round-the-clock outdoor vision system.Raindrop cause very large impact for the video quality that the rainy day absorbs due to the characteristic such as optics, physics of its complexity, go rain technology not only can recover the video image affected by raindrop, and being conducive to the further process of image, the performance comprised based on technology such as the target detection of image, identification, tracking, segmentation and monitoring improves.Video image goes rain technology to become the indispensable guardian technique of computer vision field.

In the last few years about raindrop in video image detect with the research of removing more and more extensive.Starik etc. 2003 propose median method the earliest and carry out rain, and author thinks in sequence of video images, and raindrop are only present in several frame the impact of pixel, therefore directly can be averaged on frame of video and just can obtain not by the original image that raindrop affect.This method simple and fast, but only could obtain ideal effect when the force of rain is little.Garg and Nayar sets up raindrop model (K.Garg and S.K.Nayar at first, " Detection andremoval of rain from videos; " in Proc.IEEE Conf.Comput.Vis.Pattern Recognit., Jun.2004, vol.1, pp.528 – 535), then propose method (the K.Garg and S.K.Nayar detecting and remove, " Photorealistic rendering of rain streaks; " ACM Trans.Graph., vol.25, no.3, pp.996 – 1002, Jul.2006; K.Garg and S.K.Nayar, " Vision and rain; " Int.J.Comput.Vis., vol.75, no.1, pp.3 – 27, Oct.2007) have studied dynamics and the optical characteristics of rain in literary composition, propose a kind of frame difference method and carry out raindrop detection, before and after utilizing, frame image information carries out the method for raindrop removal.Distinguish whether be raindrop by the pixel in the same raindrop of matching at the linear ratio of the luminance difference and background luminance that affect front and back by raindrop, but the usual area of raindrop is less, matching is subject to noise, and the method also needs the known shooting time shutter simultaneously.People (the Zhang X P such as Zhang in 2006, Li H, Qi Y Y, Leow W K, NgT K.Rain removal in video by combining temporal and chromatic properties.In:Proceedings of the 2006International Conference on Multimedia and Expo.Toronto, Canada:IEEE, 2006.461 464) method proposing K-means cluster detects raindrop, and the chromatic characteristic adding raindrop is to reduce the error of detection, experiment effect is better, but utilizes the method for cluster to distinguish raindrop and background at whole video, counting yield is not high, can not carry out real-time process.People (Barnum P C, Narasimhan S G, Kanade T.Analysis of rain and snow infrequency space.Internatio-nal Journal of Computer Vision, 2010,86 (2 such as Barnum in 2007 3): 256 274) obtain the typical frequency domain character of sleet based on world model, do three-dimensional Fourier transform to rainfall video sequence, eliminate raindrop, then inverse transformation is to video image at frequency domain, but this method requires that sleet has higher decline rate, the scope of application is limited.Brewer N in 2008 etc. suppose the known time shutter, under the isoparametric prerequisite of focal length, the physical features utilizing rain is proposed, like rain line length breadth ratio, raindrop area etc. realizes detection (the Brewer N of raindrop, Liu N J.Using the shape characteristics of rain to identify andremove rain from video.In:Proceeding of the 2008Joint IAPR InternationalWorkshop on Structural, Syntactic, and Statistical Pattern Recognition.Berlin, Hei-delberg:Springer-Verlag, 2008, 5342:451 458), but due to the impact of noise, interference can be caused to parameters such as rain line length breadth ratios, thus cause to carry out raindrop detection accurately.

There is the method for removing rain based on single image afterwards, gone rain algorithm to be wherein a kind of comparatively novel algorithm based on single image, not only can remove rain to single image, also may be used on video and go in the middle of rain, so usable range is wider.(the Fu Y H such as Yu-Hsiang Fu, Kang L W, Lin C W, et al.Single-frame-based rain removal via image decomposition.In:Proceeding of 2011IEEEInternatio nal Conference on Acoustics, Speech and Signal Processing (ICASSP) .Prague, Czech:IEEE Press, 2011:1453-1456.) and (the Kang L W such as Li-Wei Kang, Lin CW, Fu Y H.Automatic single-image-based rain streaks removal via imagedecomposition.Image Processing, IEEE Transactions on, 2012, 21 (4): 1742-1755.) propose and use the method for picture breakdown to carry out single image to remove rain, (the Huang D A such as De-An Huang, KangL W, Yang M C, et al.Context-aware single image rain removal.In:Proceeding of2012IEEE International Conference on Multimedia and Expo (ICME) .Melbourne, Australia:IEEE Press, 2012:164-169.) propose and remove rain by context aware, (the George J such as Jaina George, Bhavani S, Jaya J.Certain explorations on removal of rain streaks usingmorphological component analysis.International Journal of Engineering Research & Technology.2013,2 (2) .) propose to use the method for morphology constituent analysis to carry out rain, (the Chen D Y such as Duan-Yu Chen, Chen C C, Kang L W.Visual depth guided image rain streaks removalvia sparse coding.In:Proceeding of2012International Symposium on IntelligentSignal Processing and Communications Systems.New Taipei, Taiwan:IEEE, 2012:151-156.) by guiding filtering and sparse coding to carry out rain.

But prior art has following shortcoming:

Adopt real arithmetic rule, do not meet human eye visual perception rule and image physical property;

Process for dark images part is poor, and contrast is not obvious.

Therefore, for above-mentioned technical matters, be necessary to provide a kind of video raindrop based on Logarithmic image processing to detect and minimizing technology.

Summary of the invention

In view of this, a kind of video raindrop based on Logarithmic image processing have been the object of the present invention is to provide to detect and minimizing technology, input picture based on gray scale under conventional model is converted to the image based on gray tone, sets up gray tone constraint condition, more can highlight raindrop information.

In order to achieve the above object, the technical scheme that provides of the embodiment of the present invention is as follows:

Video raindrop based on Logarithmic image processing detect and a minimizing technology, and described method comprises:

S1, according to log-transformation principle, incident intensity is mapped to gray tone space;

S2, carry out gray tone constraint, carry out nonlinear conversion in gray tone space, and set gray tone threshold value, detect whether current frame pixel point is raindrop by the grey tone pitch of three two field pictures before and after detecting;

S3, if so, then list this pixel in raindrop pixel; If not, then this pixel is listed in quiet and secluded pixel;

S4, utilize frame difference method to carry out raindrop removal to raindrop pixel, obtain rain pixel.

As a further improvement on the present invention, the mapping function in described step S1, incident intensity being mapped to gray tone space is:

$f(x,y)=M(1-\frac{F(x,y)}{F_{\max }}),$

Wherein, f (x, y) is gray tone function, and F (x, y) is incident intensity, F _maxfor the intensity value of human visual system.

As a further improvement on the present invention, the computation rule of described mapping function f (x, y) is:

Addition: $f(x,y)\⊕g(x,y)=f(x,y)+g(x,y)-\frac{f(x,y)g(x,y)}{M};$

Subtraction: $f(x,y)\mathrm{\Θg}(x,y)=M\frac{f(x,y)-g(x,y)}{M-g(x,y)}$

Inverse operations: $\mathrm{\Θf}(x,y)=-M\frac{f(x,y)}{M-f(x,y)};$

Absolute value operation:

As a further improvement on the present invention, described gray tone space E and real number space R is isomorphism, and mapping relations are:

As a further improvement on the present invention, in described step S2 current frame pixel point to be the testing conditions of raindrop pixel be:

F _n<f _n-1aMP.AMp.Amp f _n<f _n+1; And

|f _nΘf _n-1| _E>T _f&|f _nΘf _n+1| _E>T _f，

Wherein, f _n-1, f _n, f _n+1for the grey tone pitch of continuous three two field pictures, T _ffor gray tone threshold value.

As a further improvement on the present invention, also comprise after described step S4:

Again carry out Logarithmic image processing, strengthen picture contrast.

The present invention has following beneficial effect:

Logarithmic image processing meets image physical characteristics and human eye visual perception rule more, and raindrop Detection results is better;

Image after being removed by raindrop uses log-transformation to strengthen, and the picture quality for low gray-scale value part is improved comparatively obvious.

Accompanying drawing explanation

In order to be illustrated more clearly in the embodiment of the present invention or technical scheme of the prior art, be briefly described to the accompanying drawing used required in embodiment or description of the prior art below, apparently, the accompanying drawing that the following describes is only some embodiments recorded in the present invention, for those of ordinary skill in the art, under the prerequisite not paying creative work, other accompanying drawing can also be obtained according to these accompanying drawings.

Fig. 1 is that the video raindrop that the present invention is based on Logarithmic image processing detect the schematic flow sheet with minimizing technology.

Fig. 2 is the sequential gray tone change oscillogram of certain pixel in the present invention one specific embodiment rainfall scene.

Embodiment

Technical scheme in the present invention is understood better in order to make those skilled in the art person, below in conjunction with the accompanying drawing in the embodiment of the present invention, technical scheme in the embodiment of the present invention is clearly and completely described, obviously, described embodiment is only the present invention's part embodiment, instead of whole embodiments.Based on the embodiment in the present invention, those of ordinary skill in the art, not making the every other embodiment obtained under creative work prerequisite, should belong to the scope of protection of the invention.

Shown in ginseng Fig. 1, the invention discloses a kind of video raindrop based on Logarithmic image processing to detect and minimizing technology, be different from the real arithmetic rule extensively adopted in prior art to process brightness of image, make raindrop Detection results better, the picture contrast after recovering is stronger.The present invention includes following steps:

S1, according to log-transformation principle, incident intensity is mapped to gray tone space;

S2, carry out gray tone constraint, carry out nonlinear conversion in gray tone space, and set gray tone threshold value, detect whether current frame pixel point is raindrop by the grey tone pitch of three two field pictures before and after detecting;

S3, if so, then list this pixel in raindrop pixel; If not, then this pixel is listed in quiet and secluded pixel;

S4, utilize frame difference method to carry out raindrop removal to raindrop pixel, weighting reconstruct reduces image blurring, the information dropout that raindrop edge causes, and obtains rain pixel.

Further, also comprise after step S4:

Again carry out Logarithmic image processing, strengthen picture contrast.

Below in conjunction with embodiment, the invention will be further described.

Logarithmic image processing has been proved to be and has more met human eye visual perception rule, may be used in the process of bounded luminance picture, the nonlinear characteristic that image itself has can better be reflected, have in fields such as image enhaucament, rim detection, Iamge Segmentation and apply comparatively widely.Traditional luminance picture is by gray tone function mistake in Logarithmic image processing framework! Do not find Reference source.Characterize.An incident intensity mistake! Do not find Reference source.With the transformational relation of gray tone function f (x, y) such as formula shown in (1):

$f(x,y)=M(1-\frac{F(x,y)}{F_{\max }})---\left(1\right)$

Wherein, a mistake! Do not find Reference source.For the intensity value of human visual system, the arithmetic operation of image all needs to redefine under Logarithmic image processing framework.

A gray tone function mistake! Do not find Reference source.And mistake! Do not find Reference source.Addition be defined as follows shown in formula (2):

$f(x,y)\&CirclePlus;g(x,y)=f(x,y)+g(x,y)-\frac{f(x,y)g(x,y)}{M}---\left(2\right)$

As can be seen here, two the gray tone function sums be defined within [0, M] are still within the scope of this, and this and general luminance picture addition may cause the situation of overflowing different.In addition, the computing of subtraction, inverse operations and absolute value operation is defined as follows respectively:

Inverse operations: $\mathrm{\&Theta;f}(x,y)=-M\frac{f(x,y)}{M-f(x,y)}---\left(3\right)$

Subtraction: $f(x,y)\mathrm{\&Theta;g}(x,y)=M\frac{f(x,y)-g(x,y)}{M-g(x,y)}---\left(4\right)$

Absolute value operation:

In formula (5), subscript E represents gray tone space, and as can be seen from formula (4), gray tone subtraction has nonlinear characteristic, and gray tone subtraction is not only relevant with both linear difference, is also subject to by the Nonlinear Adjustment subtracting gray tone functional value.

Logarithmic image processing theoretical proof, gray tone space E and real number space R is isomorphism, and its mapping relations can represent such as formula shown in (6):

In the isomorphism real number space of gray tone function, common arithmetic operation can be utilized to realize the computing of gray tone function, the addition as gray tone function first can utilize formula (7), then, and recycling inverse function by results conversion to gray tone space, so greatly can simplify the computing of gray tone.

Because raindrop can reflect large-scale ambient light, thus scene blocked by raindrop after brightness often higher than the background luminance that this place is original; On the other hand, Modeling on Rain Drops Falling Velocity is very fast, and thus same location of pixels is blocked by raindrop at adjacent two frames seldom simultaneously, and the change of its series brightness is in dither state.Due to the monotonous descending function (formula (1)) that gray tone is brightness, therefore, the pixel in raindrop scene, when being blocked by raindrop, gray tone declines, and then keeps real background gray tone when not being blocked, also in dither state.

Ginseng Figure 2 shows that the sequential gray tone change oscillogram of certain pixel in the present invention one specific embodiment rainfall scene.Accordingly, utilize continuous three two field pictures can detect the raindrop of present frame, define such as formula shown in (8) and formula (9).

f _n<f _n-1&f _n<f _n+1(8)

|f _nΘf _n-1| _E>T _f&|f _nΘf _n+1| _E>T _f(9)

Wherein, f _n-1, f _n, f _n+1for the grey tone pitch of continuous three two field pictures, T _ffor gray tone threshold value.As previously mentioned, computing | f Θ g| _ecan be by substitute, in LIP theory, this represents the Euclidean distance of gray tone f and g in fact.

Adopt gray tone functional operation and directly do not adopt the advantage of luminance picture computing to be, luminance picture computing such as luminance subtraction be homogenous linear operation in whole brightness range, and gray tone functional operation is if gray tone distance is with image grey tone pitch Automatic adjusument, there is nonlinear characteristic, be consistent with human eye vision rule, raindrop information can be highlighted.

After detecting raindrop, utilize frame difference method to carry out the removal of raindrop, adopt the video information of continuous three frames, consider that the pixel value taking out two frames before and after it compares, recover the pixel covered by raindrop with the mean value of front and back two frame.

In sum, the present invention has following beneficial effect:

Logarithmic image processing meets image physical characteristics and human eye visual perception rule more, and raindrop Detection results is better;

Image after being removed by raindrop uses log-transformation to strengthen, and the picture quality for low gray-scale value part is improved comparatively obvious.

To those skilled in the art, obviously the invention is not restricted to the details of above-mentioned one exemplary embodiment, and when not deviating from spirit of the present invention or essential characteristic, the present invention can be realized in other specific forms.Therefore, no matter from which point, all should embodiment be regarded as exemplary, and be nonrestrictive, scope of the present invention is limited by claims instead of above-mentioned explanation, and all changes be therefore intended in the implication of the equivalency by dropping on claim and scope are included in the present invention.Any Reference numeral in claim should be considered as the claim involved by limiting.

In addition, be to be understood that, although this instructions is described according to embodiment, but not each embodiment only comprises an independently technical scheme, this narrating mode of instructions is only for clarity sake, those skilled in the art should by instructions integrally, and the technical scheme in each embodiment also through appropriately combined, can form other embodiments that it will be appreciated by those skilled in the art that.

Claims (6)

1. the video raindrop based on Logarithmic image processing detect and a minimizing technology, and it is characterized in that, described method comprises:

S1, according to log-transformation principle, incident intensity is mapped to gray tone space;

S2, carry out gray tone constraint, carry out nonlinear conversion in gray tone space, and set gray tone threshold value, detect whether current frame pixel point is raindrop by the grey tone pitch of three two field pictures before and after detecting;

S3, if so, then list this pixel in raindrop pixel; If not, then this pixel is listed in quiet and secluded pixel;

S4, utilize frame difference method to carry out raindrop removal to raindrop pixel, obtain rain pixel.

2. method according to claim 1, is characterized in that, the mapping function in described step S1, incident intensity being mapped to gray tone space is:

$f(x,y)=M(1-\frac{F(x,y)}{F_{\max }}),$

Wherein, f (x, y) is gray tone function, and F (x, y) is incident intensity, F _maxfor the intensity value of human visual system.

3. method according to claim 2, is characterized in that, the computation rule of described mapping function f (x, y) is:

Addition: $f(x,y)\&CirclePlus;g(x,y)=f(x,y)+g(x,y)-\frac{f(x,y)g(x,y)}{M};$

Subtraction: $f(x,y)\mathrm{\&Theta;g}(x,y)=M\frac{f(x,y)-g(x,y)}{M-g(x,y)};$

Inverse operations: $\mathrm{\&Theta;f}(x,y)=-M\frac{f(x,y)}{M-f(x,y)};$

Absolute value operation: e represents gray tone space.

4. method according to claim 3, is characterized in that, described gray tone space E and real number space R is isomorphism, and mapping relations are:

5. method according to claim 3, is characterized in that, in described step S2, to be the testing conditions of raindrop pixel be current frame pixel point:

F _n< f _n-1aMP.AMp.Amp f _n< f _n+1; And

Wherein, f _n-1, f _n, f _n+1for the grey tone pitch of continuous three two field pictures, T _ffor gray tone threshold value.

6. method according to claim 1, is characterized in that, also comprises after described step S4: again carry out Logarithmic image processing, strengthens picture contrast.