CN101098461B - Full shelter processing method of video target tracking - Google Patents

Abstract

The invention relates to computer vision and mode analysis technical field, in particular to a full-shield processing method of video target track. In video target track process, target is usually shielded by other objects for some time. Therefore, the invention provides a method for effectively detecting the reshow time of target and on-time catching target. The method selects one best match in each several frames when the target is all shielded, as possible reshow target, and uses the different characters at backward match between the shield object and the reshow target to check if the match is the real reshow target, and when the match is the real one, confirms the frame of the match as reshow time, and the position of the best match as reshow position of target. The tests of real video flow have proved that effectiveness of the inventive method.

Description

Full shelter processing method in a kind of video frequency object tracking

Technical field

The invention belongs to computer vision and pattern analysis technical field, be specifically related to the full shelter processing method in a kind of video frequency object tracking.

Background technology

Target following has a wide range of applications in man-machine interaction, supervision automatically, video frequency searching, Traffic monitoring and automobile navigation.The task of target following is to determine how much states of target in each frame of video flowing, comprises position, size and orientation etc.Owing to do not limit the outward appearance of tracked target, and the outward appearance of target can change in tracing process, adds the interference of complicated prospect and background, and method for tracking target is faced with lot of challenges, is one of research focus of computer vision field.

Method for tracking target can be divided three classes, and a class is a tracking (point tracking) ^[1,2], second class is that nuclear is followed the tracks of (kernel tracking) ^[3～7,10], the 3rd class is that silhouette is followed the tracks of (silhouette tracking) ^[8,9]The method for tracking target that the present invention proposes belongs to the nuclear tracking.This method characterizes target with display model (that is template), and the geological information of target in each frame described with affine transformation parameter usually ^[10]

For the nuclear tracking, one of maximum challenge is exactly processing target this problem that is blocked how ^[3,5-7]Why this problem is difficult to solve is because target and shelter can be any outward appearances, and the time of blocking also can be arbitrarily.Block and can be divided into two types: partial occlusion with block entirely.Many documents ^[3,5,11]All provided and solved the scheme of partial occlusion, but do not provided the method that processing is blocked entirely.

Handling the key of blocking entirely is the moment how effective detection of a target reappears and in time catches target once more.A simple method is to establish a similarity threshold, in case image-region that finds and the similarity between the template surpass this threshold value, thinks that then target has reappeared.The problem of this method is that similarity threshold is difficult to determine, because the different needed threshold values of tracking video flowing is widely different.This problem has been evaded by seek the image-region that mates most with template in the duration of predesignating in document [6] and [7].Yet this method can only be handled and block the situation of duration less than its regulation duration entirely.In addition, if shelter is very similar to target appearance, this method can be disturbed.

List of references

[1]C.Rasmussen，and?G.Hager.Probabilistic?data?association?methods?for?trackingcomplex?visual?objects.IEEE?Trans.on?Pattern?Analysis?and?Machine?Intelligence，23(6)：560-576，2001.

[2]C.Hue，J.L.Cadre，P.Prez.Sequential?Monte?Carlo?methods?for?multiple?targettracking?and?data?fusion.IEEE?Trans.on?Signal?Processing，50(2)：309-325，2002.

[3]A.D.Jepson，D.J.Fleet，and?T.F.EI-Maraghi.Robust?online?appearance?model?forvisual?tracking.IEEE.Trans.on?Pattern?Analysis?and?Machine?Intelligence，25(10)：1296-1311，2003.

[4]D.Comaniciu，V.Ramesh，and?P.Meer.Kernel-based?object?tracking，IEEETrans.on?Pattern?Analysis?and?Machine?Intelligence，25(5)：564-577，2003

[5]S.K.Zhou，R.Chellappa，and?B.Moghaddam.Visual?tracking?and?recognitionusing?appearance-adaptive?models?for?particle?filters.IEEE?Trans.on?ImageProcessing，13(11)：1491-1506，2004.

[6]H.T.Nguyen，M.Worring，and?R.van?den?Boomgaard.Occlusion?robust?adaptivetemplate?tracking.Proc.IEEE?Int’l?Conf.Computer?Vision，1：678-683，2001.

[7]H.T.Nguyen，and?A.W.M.Smeulders.Fast?occluded?object?tracking?by?a?robustappearance?filter.IEEE?Trans.on?Pattern?Analysis?and?Machine?Intelligence，26(8)：1099-1104，2004.

[8]Y.Chen，Y.Rui，and?T.Huang.Jpdaf?based?HMM?for?real-time?contour?tracking.Proc.IEEE?Conf.on?Computer?Vision?and?Pattern?Recognition，1：543-550，2001.

[9]A.Yilmaz，X.Li，and?M.Shan.Contour?based?object?tracking?with?occlusionhandling?in?video?acquired?using?mobile?cameras.IEEE?Trans.on?PatternAnalysis?and?Machine?Intelligence，26(11)：1531-1536，2004.

[10]S.Baker，and?I.Matthews.Lucas-Kanade?20?years?on：a?unifying?framework.Int’l?Journal?Computer?Vision，53(3)：221-255，2004.

[11]J.Pan，and?B.Hu.Robust?occlusion?handling?in?object?tracking.IEEE?CVPRWorkshop?on?OTCBVS?2007，to?be?published.

Summary of the invention

The objective of the invention is to propose the full shelter processing method in a kind of video frequency object tracking.

Key of the present invention is the moment how effective detection of a target reappears and in time catches target once more.

When target during by partial occlusion, the method tracking target that the present invention adopts document [11] to propose.When the target surface ratio of blocking surpasses certain threshold value r (general desirable 0.8-0.95), think that target has entered entirely to block that the moment that the full shelter processing method detection of a target that promptly adopts the present invention to propose reappears is also caught target when occurring.

Specifically, target enter block entirely after, with template in every frame by coordinate transform ^[7]Searching has the image-region of optimum Match, and chooses an optimum Match as candidate target every the K frame; To this candidate target through a proof procedure; If passed through checking, then this candidate target is exactly the target that reappears, and the position at this candidate target place then is the position at the target place that reappears; If by checking, the optimum Match that then continues next K frame is chosen and is verified.

Because the existence of proof procedure, the value of K can be less, is 2-6 for example, and needn't be taken as 20 or 25 frames as document [6] or document [7].Less K value can in time be found the target that reappears.The key here is how to design an effective proof procedure.The present invention utilizes shelter and the target that just reappeared at the different qualities during to coupling behind the work, checks whether this coupling really is the target that reappears.The following specifically describes method of the present invention.

Comprise in the frame group of K frame the coordinate conversion parameter a of optimum Match at k _mAnd frame number n _mObtain by following formula:

$(a_m,n_m)\underset{a,n}{\arg \min }\frac{1}{N}\underset{x\∈{\mathrm{\Ω}}_T}{\mathrm{\Σ}}\left|I_n\right[\mathrm{\φ}(x;a)]-\hat{T}\left(x\right)|---\left(1\right)$

I wherein _nRepresent the n two field picture,

The expression template, N is the number of pixel in the template, φ (x; A) be the arbitrary coordinate conversion of depending on parameter a, Ω _TRepresent all of template pixel, the span of frame number n is as follows:

$n\∈{\{n_c+(k-1)K+i\}}_{i=1}^K---\left(2\right)$

N wherein _cBe the frame number of target when just having been blocked fully.When operation (1) formula, the initial search point of each frame obtains in the following way: if there is priori (for example can only occur from some specific image-region) in the zone that may occur target, then initial search point can be taken in these image-regions; Otherwise, when target is blocked entirely, the speed of target is not carried out Kalman filtering, and obtain target and just entered speed when blocking entirely, later hypothetical target is made linear uniform motion, and then the initial search point of every frame is exactly that the initial search point of previous frame adds target velocity.Captive probability during initially searching multiple spot and choose strategy and can increase target and occur like this.

After obtaining the coordinate conversion parameter and frame number of optimum Match, optimum Match T _LBObtain by following formula:

$T_{\mathrm{LB}}\left(x\right)=I_{n_m}\left[\mathrm{\φ}(x;a_m)\right].---\left(3\right)$

Note e _tBe T _LBAnd the matching error between the template, promptly

$e_t=\frac{1}{N}\underset{x\∈{\mathrm{\Ω}}_T}{\mathrm{\Σ}}|T_{\mathrm{LB}}\left(x\right)-\hat{T}\left(x\right)|---\left(4\right)$

In order to verify T _LBReally be the target that reappears, with T _LB(be not

) in preceding P frame, do the reverse estimation, and calculate average reverse compensating error e _b:

$\stackrel{\‾}{e_b}=\frac{1}{p}\underset{n=n_m-p}{\overset{n_m-1}{\mathrm{\Σ}}}\left\{\underset{a}{\min }\frac{1}{N}\underset{x\∈{\mathrm{\Ω}}_T}{\mathrm{\Σ}}|I_n\right[\mathrm{\φ}(x;a)]-T_{\mathrm{LB}}\left(x\right)|\}---\left(5\right)$

Wherein the value of P is as follows:

P＝min{n _m-n _c，3K} (6)

T _LBWhether be the target that reappears really, can be by comparing e _tWith e _bJudge.If target still is blocked, T then _LBIn fact be positioned on shelter or other object, they generally are positioned at prospect in preceding P frame, thereby T _LBGenerally can in preceding P frame, find coupling preferably.Simultaneously, because T _LBBe not target, the coupling between it and the template is far short of what is expected.So, do not finish e if block entirely also _tAlways than e _bGreatly.

Conversely, if T _LBReally be the target that reappears, then owing to survey once every the K frame, obviously most of times of target in preceding P frame all also do not recover from block.So, T _LBAll can't find coupling preferably in most of picture frames in preceding P frame, thereby cause e _bBigger.Simultaneously, as long as target appearance is in that to block phase change entirely little, e _tGenerally less.Therefore, if T _LBReally be the target that reappears, then e _bGenerally can be greater than e _t

Based on the above discussion, optimum Match T when following formula is set up _LBBy verifying:

e _t-e _b＜δ·log[n _m-n _c] (7)

Wherein δ is a less positive number, general desirable 2＜δ＜4, and its value is adjusted according to the severe degree that target appearance changes.Object variations Shaoxing opera is strong, and the δ value is big more.

At T _LBAfter checking,, therefore need to insert the transitional period of a s frame because the sub-fraction target may still be blocked.In the transitional period, follow the tracks of with the masterplate matching process of standard, but template is not upgraded.After transitional period finishes, with the method continuation tracking target of document [11].

Description of drawings

Fig. 1: the inventive method compares to the performance of document [7] method when shelter of processing short-term more similar to target blocked entirely.The 1st row and the 2nd row have shown the tracking situation of the inventive method, and the 3rd row and the 4th row have shown the tracking situation of document [7] method.Tracking results is shown with the white rectangle frame table of center band cross.The image that shows is taken from the 53rd, 64,68,71,74,77,82 of video flowing, with 90 frames.

Fig. 2: the inventive method is handled stronger the blocking entirely for a long time of ambient interferences.Tracking results is shown with the white rectangle frame table of center band cross.The image that shows is taken from the 1st, 100,104,167,215,294,299 of video flowing, with 362 frames.

Embodiment

We have compared the inventive method and the existing method of blocking disposal ability have entirely been arranged on a large amount of outdoor scene video flowings ^[6,7]Performance.Parameter value in the inventive method is as follows: block proportion threshold value r=0.85; Detect frame group length K=3; Positive number δ=2.7.Above parameter setting all is suitable for for nearly all test video stream.Hypothetical target is made linear uniform motion when choosing initial search point.

Block entirely less than the short-term of 20 frames for blocking duration, the inventive method performance is obviously because the method in document [6] or [7], be embodied in the inventive method and can detect the target that reappears in time, and remake judgement after needn't waiting until 20 frames or 25 frames, saved a large amount of amounts of calculation.The more important thing is that because the inventive method has authentication mechanism, even shelter is very similar to target appearance, the inventive method still can be not disturbed, and the method for document [6] or [7] can be thought shelter by mistake target in this case.A typical example is shown among Fig. 1.In Fig. 1, tracked girl's face is closely similar with the face as another girl of shelter, fails to such an extent as to the method for document [7] is disturbed.Although the optimum Match that the inventive method is selected when target girl's face is blocked entirely is another girl's a face, another girl's face can't be by follow-up authentication mechanism, thereby not to be mistaken as be the target that reappears.When the 77th frame target girl's face reappeared, optimum Match was just passed through checking, thereby has detected the moment and position that target reappears timely and accurately.

Surpass blocking entirely for a long time of 25 frames for blocking duration, the method for document [6] or [7] is bound to fail, and the inventive method then can be handled equally well.A typical example is shown among Fig. 2.In Fig. 2, there are 150 frame left and right sides targets to be blocked fully.In the so long time, although there is the similar background object of outward appearance to have (for example finger), the inventive method does not have error in judgement one time, and the 294th frame target is caught it at once again once occurring.

To sum up, the inventive method can effectively be surveyed the finish time of blocking entirely of any duration, and has the very strong anti-prospect or the interference capability of background object.

Claims (3)

1. the full shelter processing method in the video frequency object tracking, it is characterized in that target enter block entirely after, in every frame, seek image-region with template, and choose an optimum Match as candidate target every the K frame with optimum Match by coordinate transform; This candidate target is verified; If passed through checking, then this candidate target is exactly the target that reappears, and the position at this candidate target place then is the position at the target place that reappears; If by checking, the optimum Match that then continues next K frame is chosen and is verified; Here K gets 2-6; Wherein, described checking is by comparing the error e between optimum Match and the template _tAnd matching error is estimated in the average reverse of optimum Match

Verify whether this optimum Match is exactly the target that reappears; E wherein _tBe calculated as follows:

$e_t=\frac{1}{N}\underset{x\∈{\mathrm{\Ω}}_T}{\mathrm{\Σ}}|T_{\mathrm{LB}}\left(x\right)-\hat{T}\left(x\right)|---\left(4\right)$

Be calculated as follows:

${\stackrel{\‾}{e}}_b=\frac{1}{P}\underset{n=n_m-P}{\overset{n_m-1}{\mathrm{\Σ}}}\left\{\underset{a}{\min }\frac{1}{N}\underset{x\∈{\mathrm{\Ω}}_T}{\mathrm{\Σ}}\right|I_n\left[\mathrm{\φ}(x;a)\right]-T_{\mathrm{LB}}\left(x\right)\left|\right\}---\left(5\right)$

Wherein the value of P is as follows:

P＝min{n _m-n _c，3K} (6)；

Optimum Match T when following formula is set up _LBBy checking, promptly optimum Match is exactly the target that reappears:

$e_t-{\stackrel{\&OverBar;}{e}}_b<\mathrm{\&delta;}\&CenterDot;\log [n_m-n_c]---\left(7\right)$

2＜δ＜4 wherein, its value is adjusted according to the severe degree that target appearance changes;

K comprises the coordinate conversion parameter a of the image-region that mates most with template in the frame group of K frame _mWith place frame number n _mProvide by following formula:

$(a_m,n_m)=\underset{a,n}{\arg \min }\frac{1}{N}\underset{x\&Element;{\mathrm{\&Omega;}}_T}{\mathrm{\&Sigma;}}\left|I_n\right[\mathrm{\&phi;}(x;a)]-\hat{T}\left(x\right)|---\left(1\right),$

I wherein _nRepresent the n two field picture,

The expression template, T _LBThe expression optimum Match, N is the number of pixel in the template, φ (x; A) be the arbitrary coordinate conversion of depending on parameter a, Ω _TRepresent all of template pixel, the span of frame number n is as follows:

$n\&Element;{\{n_c+(k-1)K+i\}}_{i=1}^K---\left(2\right)$

N wherein _cBe the frame number of target when just having been blocked fully.

2. the full shelter processing method in the video frequency object tracking according to claim 1, it is characterized in that when operation type (1) is searched the image-region that mates most with template, the initial search point of each frame obtains in the following way: if there is priori in the zone that may occur target, then initial search point is taken in these image-regions; Otherwise, when target is blocked entirely, the speed of target is not carried out Kalman filtering, and obtain target and just entered speed when blocking entirely, later hypothetical target is made linear uniform motion, and then the initial search point of every frame is exactly that the initial search point of previous frame adds target velocity.

3. the full shelter processing method in the video frequency object tracking according to claim 2 is characterized in that k comprises in the frame group of K frame and template optimum Match T _LBObtain by following formula:

$T_{\mathrm{LB}}\left(x\right)=I_{n_m}\left[\mathrm{\&phi;}(x;a_m)\right]---\left(3\right).$

Images