CN102663775A - Target tracking method oriented to video with low frame rate - Google Patents

Abstract

The invention discloses a target tracking method oriented to a video with a low frame rate. The method comprises the following steps: (1), representing a target region by a method integrating a dominant color and a space distribution characteristic thereof; (2), employing a cross color ratio-based matching criterion to carry out similarity matching on a candidate region and the target region; (3), employing a parameter integrogram-based fitness function to characterizing a matching degree of a sample particle and a target template; and (4), utilizing an annealing particle swarm optimization framework with simulation of biological swarm intelligence to search abrupt motions caused by a video with a low frame rate. According to the invention, the effective target tracking method is realized; and moreover, experimental results show that the provided method, compared with other classical low frame rate tracking methods, has good effectiveness and robustness.

Description

Method for tracking target towards low frame-rate video

Technical field

The present invention relates to computer vision field, especially a kind of method for tracking target towards low frame-rate video.

Background technology

In the machine vision applications of a lot of reality, available resource all be restricted (for example: embedded visible system).So, in the process of video acquisition, will produce some low frame per second (Low Frame Rate) video, concrete reason has following two kinds: 1) because the time-delay of video acquisition hardware or the restriction of transmission bandwidth make video acquisition process or transmission course fall frame; 2), thereby improve processing speed or reduce memory space because the restriction of CPU processing power or memory capacity is carried out down-sampling to the video data of being gathered on time dimension.

Because low frame-rate video data images displayed p.s. data are less than 10 frames, cause the apparent or very big sudden change of existence of moving of target object in its continuous images frame.Yet mostly classical track algorithm all is based on the continuity hypothesis of dbjective state, i.e. motion of hypothetical target object and apparent variation in consecutive image interframe are very little.For example, particle filter algorithm is employed on the basis of tracking results of former frame and predicts the particle in the current frame image; Based on the track algorithm of iteration optimization, too will be like KLT (Kanade-Lucas-Tomasi), template matching algorithm and average drifting algorithm based on the tracking results of former frame initial value as the present frame iteration optimization.Therefore, the track algorithm of above-mentioned classics can't obtain gratifying result in the application of low frame-rate video.

Directly less both at home and abroad to the research of target following under the low frame-rate video; Wherein people such as Porikli expands the average drifting algorithm; On the basis of background subtraction, the motion marking area adopted the average drifting algorithm of multinuclear, thereby overcome by caused motion uncontinuity of low frame per second and uncertain problem.People such as Li propose the problem that a kind of cascade particle filtering algorithm solves target travel sudden change under the low frame-rate video; This algorithm is learnt the characteristic of different sequential on the cycle through detecting device; And the response of detecting device is dissolved into particle filter algorithm as significance distribution, realize effective tracking to low frame per second people face video.The possibility zone passage background subtraction of people's target travels such as Carrano obtains, and combines following four kinds of characteristics then: relative coefficient, mean pixel intensity, velocity contrast and differential seat angle, realize the coupling in object module and motion possibility zone.In addition, people such as Zhang adopt the difference diagram of consecutive image interframe to come the motion sudden change of target of prediction, utilize prediction result to instruct the sample communication process in the particle filter then.Generally speaking, domestic and international existing research all is the motion sudden change of low frame-rate video being regarded as target object.Yet in practical application, what low frame-rate video was introduced is not only the motion sudden change of target object, also is accompanied by the apparent great variety of target.Therefore, realize under the low frame-rate video robust tracking, need set about from apparent model and two aspects of motion search framework of target object simultaneously, make it meet the characteristic of hanging down frame-rate video.

Summary of the invention

In order to overcome the apparent problem of suddenling change with motion of target that low frame-rate video is brought, the present invention's proposition sets about setting up the Target Tracking System towards low frame-rate video of a robust from following three aspects: 1) object representation; 2) Model Matching; 3) motion search.At first, the apparent model that propose to merge main color and space distribution thereof with intersect the matching criterior of color-ratio, make the model apparent sudden change with light of processing target effectively; Secondly, propose method for searching motion, thereby the motion sudden change is followed the tracks of effectively based on biological group intelligence.

In order to realize above-mentioned purpose, the technical scheme below the present invention has adopted:

Set up the Target Tracking System towards low frame-rate video of a robust, comprise the steps:

1. merge the apparent and renewal of target of main color and space distribution information thereof, mainly contain following steps:

The first step is transformed into the rgI color space with the target area pixel from RGB, the Euclidean distance of calculating pixel color again, thus set up the weight map matrix between the pixel of target area;

Second step, utilize the graph structure between pixel of setting up, adopt leading clustering algorithm can obtain the main color mode of target area successively to pixel, when the residual pixel number less than certain threshold value, then cluster end, and think that residual pixel is a noise;

In the 3rd step, extract the space distribution information that comprises weight, average and variance that drops on all pixels in each main color mode, and centralization processing (in order to eliminate the influence of absolute position) need be carried out in the locus of all pixels;

In the 4th step, after each frame is followed the tracks of end, main color mode and space distribution information thereof are upgraded;

2. after " main color+space " expression of accomplishing the target area template, when tracker obtains a candidate region, need carry out the similarity coupling to candidate region and object module.The present invention has just proposed the matching criterior based on the intersection color-ratio, and basic step is following:

The first step utilizes nearest neighbor algorithm to confirm the main color mode of candidate region on the main color mode of given To Template, calculates the cross-ratio matrix between each autonomous color mode again;

In second step,, calculates quadratic sum between two matrixes apart from as the matching error of color mode according to the different cross-ratio matrixes of being set up;

The 3rd step, according to the space distribution information of main color in the To Template, the space matching error of corresponding color pattern in the calculated candidate zone, and merge with the color-match error;

3. before using annealing particle group optimizing framework, need adaptive value function of definition to characterize the matching degree of sample particles and To Template.For fear of the repeat region of computed image repeatedly in the adaptive value evaluation procedure, the present invention designs a kind of adaptive value quick calculation method based on parameter integral figure, and concrete performing step is following:

The first step is confirmed the maximum region that potential sample particles can cover according to the maximum constraints speed parameter in the particle swarm optimization algorithm;

In second step, confirm according to the main color mode and the nearest neighbor algorithm of To Template which main color mode is each pixel belong in this image-region, thereby obtain the label of entire image area pixel;

In the 3rd step, to each pixel, write down its label and spatial parameter and form one 5 dimensional vector, and calculate the integrogram of this 5 dimensional vector;

In the 4th step, for the corresponding region of given sample particles, search this regional model parameter, and calculate the adaptive value of this particle according to matching criterior according to integrogram;

4. for by the caused mutation movement of low frame-rate video, the present invention designs a kind of annealing particle group optimizing framework of simulating biological group intelligence and searches for.Its key step is following:

The first step is carried out the propagation at random on the sequential to the individual optimal particle that the previous frame image is followed the tracks of after restraining, to improve the diversity of sample particles, owing to the compactness of individual optimal particle, so need not resample to particle;

In second step,, in iterative process, need to upgrade individual optimal particle and colony's optimal particle of population, in order to instruct the evolution iterative process of population through the adaptive value evaluation to the sample particles optimization iteration of annealing;

The 3rd step, the particle of evolving is carried out convergence judge, and the output tracking result;

The invention has the beneficial effects as follows:

1, track algorithm proposed by the invention is a kind of method towards the target following of hanging down frame-rate video;

2, the present invention proposes a kind of target apparent model that merges main color and space distribution information thereof; The leading clustering algorithm of employing obtains the main color mode of target area, and this algorithm can determine the number of class automatically according to " compactness " of sample, produces the cluster classification of clear layer; Thereby the color mode that keeps tool identification; The influence of cancelling noise pixel, and the calculated amount of this algorithm is little, satisfies the real-time demand easily.Extract the distribution pattern of main color in the locus simultaneously, adaptability that both can the retaining color model has improved the identification of model again.

3, the present invention proposes a kind of matching criterior of intersecting color-ratio; This criterion is than other criterions (like Bhattacharyya coefficient, Kullback-Leibler divergence etc.) robust more, and two aspects below the concrete manifestation: 1) this criterion adopts color-ratio can effectively eliminate the influence of illumination variation; 2) each color mode is considered to come in to every other color mode during computed range, can alleviate the influence of ground unrest pixel and shield portions pixel.

4. the annealing particle group optimizing framework of the present invention's proposition is a process based on the distribution importance sampling; Through two steps of thick sampling and thin sampling image observation information is joined in the sampling process; Thereby the sampled result that obtains is approached the result of Direct Sampling in best significance distribution, has also well solved the search problem of unexpected motion.

5. the present invention proposes a kind of adaptive value quick calculation method based on integrogram and has avoided in evaluation procedure the repeatedly repeat region of computed image.Each pixel only needs once coupling to calculate, even the iterations of motion search has increased, also can not increase computation complexity.

Description of drawings

Below in conjunction with accompanying drawing and embodiment the present invention is further specified.

Fig. 1 is the general frame of tracker of the present invention;

Embodiment

Through embodiment the present invention is carried out concrete description below; Only be used for the present invention is further specified; Can not be interpreted as the qualification to protection domain of the present invention, the technician in this field can make some nonessential improvement and adjustment to the present invention according to the content of foregoing invention.

As shown in Figure 1, Fig. 1 is a general frame of the present invention.The present invention is a kind of method for tracking target towards low frame-rate video, and the hardware and the programming language of the concrete operation of method of the present invention do not limit, and can accomplish with any language, and other mode of operation repeats no more for this reason.

Embodiments of the invention adopt one to have the Pentium 4 computing machine of 3.2G hertz central processing unit and 1G byte of memory and worked out working routine with the Matlab language; Realized method of the present invention, the method for tracking target towards low frame-rate video of the present invention may further comprise the steps:

The target that calculate to merge main color and space distribution information thereof is apparent, intersect color-ratio matching criterior structure, simulation biological group intelligence annealing particle group optimizing framework renewal and judgement, based on the adaptive value of integrogram module such as calculating fast, concrete steps are described below:

(1) at first by formula with the target area pixel

r＝R/(R+G+B)，g＝G/(R+G+B)，I＝(R+G+B)/3

Carry out the transformation of color space; And then the weight map between the definition pixel is following:

W _i，j＝||f _i-f _j|| ²

F wherein _i=(r _i, g _i, I _i), i, j represent i pixel and j pixel respectively.

Method through leading clustering just can obtain main color mode

successively again

(2) calculating drops on the space distribution pattern of all pixels of l color mode: average,

Variance is distinguished as follows:

${\mathrm{\μ}}_l=\frac{{\mathrm{\Σ}}_i{p}_i\mathrm{\δ}(L\left(f_i\right)-l)}{{\mathrm{\Σ}}_i\mathrm{\δ}(L\left(f_i\right)-l)};$ ${\mathrm{\Σ}}_l=\frac{{\mathrm{\Σ}}_i(p_i-u_l){(p_i-u_l)}^T\mathrm{\δ}(L\left(f_i\right)-l)}{{\mathrm{\Σ}}_i\mathrm{\δ}(L\left(f_i\right)-l)};$

Wherein, p _iIt is the position of pixel i; δ () is the Kronecker function, is

Thereby can know; Object region will be expressed as

wherein,

be the weight of l main color mode.

(3) confirm the number of each color mode interior pixel of candidate region according to nearest neighbor algorithm

The cross-ratio matrix that can get between each autonomous color mode does

Wherein, Represent main color mode c _iAnd c _jRelation.

Compute histograms U again ^TAnd U ^CSimilarity M _v,

Promptly $M_v(U^T,U^C)=\underset{v=1}{\overset{k}{\mathrm{\Σ}}}{(\frac{u_l^T}{u_v^T}-\frac{u_l^C}{u_v^C})}^2/{(\frac{1}{u_v^T}+\frac{1}{u_v^C})}^2;$

Wherein, When and

levels off to zero the time,

with orthogonalization in case its instability problem.

Thereby can get the color-match error do

$M_v(U^T,U^C)=\underset{v=1}{\overset{k}{\mathrm{\Σ}}}{M}_v(U^T,U^C)=\underset{v=1}{\overset{k}{\mathrm{\Σ}}}(1-\frac{u_v^T{u}_v^C}{{(u_v^T+u_v^C)}^2}{\left|\right|U^T+U^C\left|\right|}_2^2)$

(4) merge and can get according to the space distribution information of main color in the To Template matching error of corresponding color pattern in the calculated candidate zone again, and with the color-match error:

$S(U^T,U^C)=\underset{l=1}{\overset{k}{\mathrm{\Σ}}}{w}_l\exp \{-\frac{1}{2}{(u_l^T-u_l^C)}^T{({\mathrm{\Σ}}_l^T+{\mathrm{\Σ}}_l^C)}^{-1}(u_l^T-u_l^C)\}$

(5) annealing particle group optimizing framework is the Gauss that on particle group optimizing frame foundation, the revises particle swarm optimization algorithm of annealing, and concrete equation is following:

v ^i，n+1＝|r ₁|(p ⁱ-x ^i，n)+|r ₂|(g-x ^i，n)+ε

x ^i，n+1＝x ^i，n+v ^i，n+1

Wherein, | r ₁| with | r ₂| be the random value of the independent sampling during gaussian probability distributes; ε is absorbed in local minimum zero-mean Gaussian noise in order to stop to search for, and its covariance ∑ _εAlso change, i.e. ∑ at adaptive annealing way _ε=∑ e ^-cn, wherein, ∑ is the covariance matrix that predefined transfer distributes; N is an iterations; C is the annealing factor.

The adaptive value of each particle is calculated f (x ^{I, n+1}) be with display model p (o ^{I, n+1}| x ^{I, n+1}) represent as follows: f (x ^{I, n+1})=p (o ^{I, n+1}| x ^{I, n+1})

$p\left(o^{i,n+1}\right|x^{i,n+1})=\exp \left\{\frac{1}{{2\mathrm{\σ}}_1^2}(1-S(U^T,U^C))\right\}*\exp \left\{\frac{1}{{2\mathrm{\σ}}_2^2}M(U^T,U^C)\right\}$

Wherein,

is the outward appearance covariance parameter about the influence of weighing space length and cross-ratio similarity.

In addition, individual optimal particle and colony's optimal particle will be upgraded by following formula after adaptive value is calculated:

$g=\arg \underset{p^i}{\max }f\left(p^i\right)$

(6) by the particle swarm optimization algorithm, the maximum speed limit parameter?

Identify potential at time t +1 sample particles can cover a maximum image area of?

Again each particle is set up the vector of one 5 dimension, promptly

$F_i=\{L\left(i\right),{\left(p_i\right)}_x,{\left(p_i\right)}_y,{\left(p_i\right)}_x^2,{\left(p_i\right)}_y^2\}$

Wherein, L (i) is the color mode that belongs to pixel i; (p _i) _x, (p _i) _yBe respectively the x of pixel i, the coordinate figure of y axle.Calculate the integrogram of l color mode again, and the value in the i position is in this integrogram:

Above-mentioned integrogram representes that respectively the i position belongs to the number of pixels of l color mode, and these pixels are in the average of locus, and variance.

For the corresponding region of given sample particles, search this regional model parameter according to above-mentioned integrogram, and calculate the adaptive value of this particle according to matching criterior.

Claims (8)

1. the method for tracking target towards low frame-rate video is characterized in that, may further comprise the steps:

(1) in the object representation process that merges main color and space distribution thereof, to color of pixel space, target area be transformed into the rgI space from RGB, adopt the main color mode of the method acquisition pixel of leading clustering again;

(2) calculate for the space distribution information that comprises weight, average and variance that drops on all pixels in each main color mode, and merge with color mode information and upgrade;

(3) the main color mode to given To Template utilizes nearest neighbor algorithm to confirm the main color mode of candidate region, and then calculates cross-ratio matrix and the matching error of color mode between each main color mode;

(4) merge according to the space matching error of corresponding color pattern in the space distribution information calculated candidate zone of main color in the To Template, and with the color-match error;

(5) calculate the wherein label of each pixel in fixed maximum image zone, and itself and spatial parameter are constituted 5 dimensional vectors, calculate the integrogram of this 5 dimensional vector again;

(6) search the model parameter of the corresponding region of given sample particles according to integrogram, and calculate the adaptive value of this particle according to matching criterior;

(7) according to the motion search framework of annealing particle group optimizing to the sample particles optimization iteration of annealing; Also to utilize the adaptive value evaluation to upgrade the individual optimal particle and all optimal particle of population simultaneously, then the particle of evolving carried out convergence and judge and the output tracking result.

2. the method for tracking target towards low frame-rate video according to claim 1 is characterized in that described step (1) specifically comprises following substep:

At first, the conversion in target area pixel color space;

Secondly, the Euclidean distance of calculating pixel color is set up the weight map matrix between pixel thus;

At last, obtain main color mode successively by graph structure.

3. the method for tracking target towards low frame-rate video according to claim 1 is characterized in that described step (2) specifically comprises following substep:

At first, from the space distribution pattern of weight, average, three aspect calculating pixels of variance, and the position of pixel also wants centralization to handle;

Secondly, space distribution information and distribution of color information are merged and upgrade.

4. the method for tracking target towards low frame-rate video according to claim 1 is characterized in that described step (3) specifically comprises following substep:

At first, confirm the main color mode of candidate region;

Secondly, calculate the cross-ratio matrix between each autonomous color mode;

At last, the quadratic sum distance between two matrixes of calculating is as the matching error of color mode.

5. the method for tracking target towards low frame-rate video according to claim 1 is characterized in that described step (4) specifically comprises following substep:

At first, the space matching error of corresponding color pattern in the calculated candidate zone;

Secondly, color-match error and space matching error are merged.

6. the method for tracking target towards low frame-rate video according to claim 1 is characterized in that described step (5) specifically comprises following substep:

At first, confirm the maximum image zone that potential sample particles can cover;

Secondly, by the main color mode of each pixel in the maximum image zone can this area pixel label;

At last, calculate the integrogram of 5 dimensional vectors that constitute by label and spatial parameter.

7. the method for tracking target towards low frame-rate video according to claim 1 is characterized in that described step (6) specifically comprises following substep:

At first, search the model parameter of the corresponding region of given sample particles according to integrogram;

Secondly, calculate the adaptive value of this particle according to matching criterior.

8. the method for tracking target towards low frame-rate video according to claim 1 is characterized in that described step (7) specifically comprises following substep:

At first, the individual optimal particle after the convergence of former frame image is carried out the propagation at random on the sequential;

Secondly, to the sample particles optimization iteration of annealing, also to upgrade the individual optimal particle of population and colony's optimal particle simultaneously through the adaptive value evaluation;

At last, the particle of evolving being carried out convergence judges.