CA2026290C - Moving object detecting system - Google Patents
Moving object detecting systemInfo
- Publication number
- CA2026290C CA2026290C CA002026290A CA2026290A CA2026290C CA 2026290 C CA2026290 C CA 2026290C CA 002026290 A CA002026290 A CA 002026290A CA 2026290 A CA2026290 A CA 2026290A CA 2026290 C CA2026290 C CA 2026290C
- Authority
- CA
- Canada
- Prior art keywords
- features
- image
- feature
- background
- region
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
Abstract
Abstract of the Disclosure A moving object detecting system includes an image acquiring section, a feature extracting section, a background detecting section, a prediction parameter calculating section, a region estimating section, and a moving object determining section. The image acquiring section has a mobile imaging system and acquires image frames sequentially obtained upon movement of the imaging system. The feature extracting section extracts features of the acquired image frames. The background feature detecting section detects a background feature from the features. The prediction parameter calculating section obtains prediction parameters for predicting a motion of the background region upon movement of the imaging system in accordance with a positional rela-tionship between the correlated background features.
The region estimating section estimates a region where features detected from image frames obtained by the mobile imaging system may have been present in an image frame of the immediately preceding frame by using the prediction parameters. The moving object determining section determines whether a feature corresponding to the given feature is present in the estimation region, thereby checking the presence/absence of the moving object.
The region estimating section estimates a region where features detected from image frames obtained by the mobile imaging system may have been present in an image frame of the immediately preceding frame by using the prediction parameters. The moving object determining section determines whether a feature corresponding to the given feature is present in the estimation region, thereby checking the presence/absence of the moving object.
Description
2~2~29~
The present invention relates to a moving ob;ect detecting system for effectively detecting a moving ob~ect in a given environment from environmental image frames momently obtained by a mobile imaging system lncorporated ln a moving unit such as a mobile robot.
One of the lmportant techniques for realizlng a state monitor apparatu~, a mobile robot, or the like is a technique for detecting a movlng ob~ect in a glven environment from envlronmental image frames from moment to moment obtained by an imaging system which ls movable, Thls technique for detecting the moving ob~ect from a moving image is based on a so-called motion analysls. In recent years, extensive studies have been made on this detection technique in the fields of computer visions. ~his technique, however, is not applied to practlcal apparatuses or fleld works.
Image processing for detectlng such a movlng ob~ect is called a gradlent-based scheme for generally detectlng spatlal and temporal changes in image den-sities of moving image frames as an ob~ect of interest.In addition to this scheme, a feature-based scheme is available in which features such as edges of a moving image ti.e.~ an ob~ect of interest) are extracted, and these features are correlated to each other between image frames having a time difference, thereby esti-mating a displacement and hence detecting the moving ob;ect.
. . ~ -, , , 2~2~2~
Although the former scheme, i.e., the gradient-based scheme has an advantage in that a fine displace-ment can be attained with respect to the entire image, since spatial and temporal changes in image densities s are detected by differentiations, detection precision is greatly degraded in the presence of noise in the image.
In addltion, in principle, accurate estimation values cannot be obtained in a region having a small change in denslty or a boundary region of the object.
To the contrary, in the latter scheme, i.e., the feature-based scheme, although portions of lnterest sub~ected to displacement measurements are dispersed in a distribution since features in an image are used, a problem such as degradation of detection preclslon caused by noise can be greatly reduced. In the feature-based scheme, it is not easy to correlate the features to each other between lmage rames havlng a time difference. ~his correspondence processing requires a long period of time if it is accurately per-formed in accordance with its theory. For this reason,the moving ob;ect in an environment monitored by a moving image cannot be detected at high speed.
In the conventional image processing techniques, the moving object in its environment cannot be accurately detected at high speed from image frames momently detected by a mobile imaging system.
It is an object of the present invention to provide ~?~29~
a moving object detecting system which can easily, pre-cisely, quickly detect a moving ob;ect in an environment from environmental image frames momently obtained by a mobile imaging system.
The moving object detectlng system accordlng to the present inventlon has a feature extractlng sectlon for sequentially extracting eatures from image frames momently and sequentlally imaged by a mobile imaging system. The moving object detecting system detects background features included in a background reglon of the image frames from the features extracted by the feature extracting section. For example, parameters for predicting a motion of the background region in the lmage upon movement of the imaging system are obtained from a positional relationship between the corresponding background features between a plurallty of lmage frames in, e.g, an inltlal period of movement. The moving ob~ect detecting system estimates a region where features detected from image frames obtained by the mobile imaging system may have been present at imme-diately preceding moments (if a given feature belongs to the background) on the basis of the prediction para-meters and detects whether a feature corresponding to the given feature belongs to the estimated region of presence, thereby detecting the presence/absence of the moving object.
In the system of the present invention, the 2~2~2~
features apparently belonging to the background are correlated to each other from an image obtained in the initial period of movement, and the prediction parame-ters are obtained from this positional relationship.
The system then determines whether the corresponding feature is present within the estimated reglon based on the predictlon parameters and whether the feature belongs to the background, thereby detectlng the moving ob~ect ln the image.
In the movlng object detecting system according to the present invention, the prediction parameters are estimated from motion of the known background at the start of movement of the mobile imaging system. On the basis of the predictlon parameters, those positions ln the immedlately preceding frame which correspond to the respectlve features obtained from the moving image frame are estimated, and only those features which do not have corresponding features at the estimatad posltions in the immediately preceding frame are extracted. Therefore, the moving ob;ect in the moving image can be effi-ciently, accurately searched at high speed.
This invention can be more fully understood from the following detailed descrlption when taken in con-~unction with the accompanying drawings, in which:
Fig. l is a block diagram illustrating an arrange- -ment of a moving object detecting system according to the first embodiment of the present invention;
- 52~2l~2~
Fig. 2 is a block diagram illustrating an arrange-ment of a moving ob;ect detecting system according to the second embodiment of the present invention;
Figs. 3A and 3B are views illustrating input image frames which have changes in backgrounds upon movement of an imaging system and which are used to explaln the system 9hown in Fig. 2;
Fig. 4 is a view showing directlons of spatlal dif-ferentiations of input image frames so as to explain the system shown ln Flg. 2;
Flg. 5 ls a vlew for explaining a coordinate system whlch ls used to explain the system of Fig. 2 and which is associated with a motion of a camera serving as an imaglng system; and Flg. 6 is a vlew showing estimation llnes for features used for explalnlng the apparatus of Fig. 2.
A moving ob~ect detectlng system according to the first embodiment o the present inventlon will be described with reference to the accompanying drawings.
The moving object detecting system shown in Fig. 1 comprises an image input section 1, a feature detecting section 2, a background feature detecting section 3, a prediction parameter calculating section 4, an estima-tion line calculating section 5, a feature searching section 6, and a moving object determining section 7.
The image input section 1 comprises an imaging system mounted on a moving system such as a mobile robot .
- 6 2~2~29a and momently image frames and inputs moving environmen-tal image frames to the system. Upon movement of the moving system, frames of image information sequentially imaged and input from the image input section whose imaglng positlon ls sequentially changed are supplled to ~he feature extractlng sectlon 2. The feature extractlng sectlon 2 sequantially extracts reglons havlng large spatlal dlfferentlal values as features.
The background feature detectlng sectlon 3 detects background features of a plurallty of image frames which are lnput by the image input section l in an initial perlod of movement, from pieces of feature information respectively extracted from the plurality of image frames. The background features are correlated to each other between the plurality of image frames. That is, the background feature detectlng section 3 processes lnput lmage frames in the lnltial parlod of movement as lnltial lmage frames servlng as criteria of subsequent lmage processing and designates its background region to correlate features to each other between the plurallty of image frames in this background region. The predlc-tion parameter calculating section 4 calculates parame-ters for predicting a motion of the background region movlng in the input image frames upon movement of the image input section l. These parameters are 15 parame-ters representing changes ln the imaglng system (to be described in detail later in association with the second - 7 - 2~2~29~
embodiment).
As for the features detected in units of input image frames by the feature extracting section 2 on the -basis of input image frames momently and sequentially input by the image input section 1, the estimation line calculating section 5 estimates (reverse-predicts) a reglon o presence o~ the glven ~ature obtalned at an lmmedlately preceding timing to be a line region ~to be described later) on the basis of the parameters obtained by the prediction parameter calculating section 4 and the positions of the features in the input image frames.
This region of presence of the feature is performed in units of features extracted from the input lmage frames.
The feature searching section 6 checks whether a lS correspondlng feature ls present ln an estlmated region of presence of each feature of the immedlately preceding timing obtained by the estimation line calculating sec-tion 5. If the corresponding featura is not detected from the lmage of the previous frame, the feature infor-mation is output to the moving ob~ect determiningsection 7. Pieces of feature information having no correspondence with features in the image of the pre-vious frame are acquired by the moving ob~ect deter-mining section 7. The moving object determining section 7 checks the number of features and their positional proximities. The feature count and proximity data are detected as moving object information in the ' - 8 - 2~2 ~2 corresponding input image.
In the moving object detecting system having the above arrangement, the following series of operations are basically performed.
(a) Regions where features extracted from sequen-tlally input image frames may have been present ln image frames obtained at the immedlately precedlng tlmlngs are estimated (i the features belong to the background)~
(b) When features correspondlng the estlmated reglons are not found, the extracted features are detected as new features of the image frames.
(c) Informatlon of the new features ls processed to dstect informatlon of a movlng ob~ect appearlng in an lmage.
Slnce the estlmated reglons of the features of the image frames obtalned at the immedlately precedlng timings are obtalned as llne regions from the predlctlon parameters representing the background motion obtalned in accordance with a relationship between background features correlated to each other from the lmage frames obtained at, e.g., the initial period of movement, the correlation processing (i~e~ processing for determining whether the features are present ln the estlmated regions) can be effectively, easily performed. The processing speed of determination processing can be sufficiently increased.
For this reason, the moving object appearing in 9 2~2~
image frames sequentially obtained by the mobile imaging system can be accurately detected at high speed.
The moving ob;ect detecting system according to the present invention may be arranged as a system s corresponding to the sections shown in Fig. 1 or a comblnatlon of clrcult blocks. ~owever, the second embodiment exemplifles a practical system having substantially the same functlon as described above.
Flg. 2 shows an arrangement of a moving ob~ect detecting system according to the second embodiment.
Referring to Fig. 2, a portion corresponding to the image input section 1 shown in Fig. 1 comprises an ITV (industrial television) camera 11, an A/D
(analog-to-dlgital) converter 12, and an ITV interface 13.
The ITV camera 11 is mounted on the moving body and ls used to sequentlally image and lnput envlronmental lmage frames obtained upon movement of the moving body.
The A/D converter 12 convèrts the image signal obtained by the ITV camera 11 into digital data. The ITV inter-face 13 is an interface for sequentially storing the converted digital image data into an image memory 15 through an image bus 14.
The image memory 15 has a memory capacity capable of storing a plurality of image frames. The read/write access of the image memory 15 is properly controlled in response to a control signal supplied through a control - lo ~2~2~
bus 16 connected to the image memory 15.
In the moving environmental image frames sequen-tlally imaged by the ITV camera ll, as shown in Figs. 3A
and 3B, stationary backgrounds are changed in the s obtained image frames in their imaging environments upon changes in lmaging position upon movement of the ITV camera 11. The movlng ob~ect detecting syst~m is used to efflciently detect a movlng ob~ect ~an image component except for the background) from envlronmental lmage frames accompanying displacements of lmage information in the background.
A portlon corresponding to the feature detecting section 2 for detectlng features from image frames sequentially obtained by and input from the ITV camera lS 11 in the image lnput section comprlses a spatial dif-ferentiator 21, a threshold clrcuit 22, and a feature position detecting circuit 23, all o which are con-nected to the lmage memory 15 through the image bus 14.
This portion also comprises a feature table memory 24 connected to the above constituent circuits through the control bus 16.
The spatial differentiator 21 calculates difference values for a local region of an image stored in the image memory in radial directions (eight directions) along predetermined fourth axes at equal angular inter-vals, as shown in Fig. 4, thereby performing spatial differentiation. The threshold circuit 22 compares 2 ~
a threshold value with the difference values of the respective directions calculated by the spatial dif-ferentiator 21, thereby detecting features.
More specifically, when a difference value of each direction in a portion of an image exceeds a predeter-mlned threshold value, this portlon ls detected as a feature image portion having a largé spatial dlferen-tlal value. Feature detection information is supplied to the feature position detecting circuit 23, and the posltlon of the feature on the image is calculated.
Pleces of feature detection position information are sequentlally stored as a feature table in the feature table memory 24.
Thls feature detection processing is performed in units of lmage frames sequentlally obtained and stored in the image memory 15. Pleces of feature detectlon posltlon information obtained from each image are classified, and the classifiad pieces of information are written in the feature table.
A portion corresponding to the background feature detecting section 3 comprises an image monitor 31, a region designating circuit 32, a feature detecting circuit 33, and a background feature table memory 34.
The image monitor 31 is connected to the image bus 14 to monitor an image stored in the image memory 15.
The region designating circuit 32 causes the image moni-tor 31 to sequentially display image frames sequentially - 12 2~2~2~
obtained and input by the ITv camera 11 in the image input section at, e.g., the start of movement. The image monitor 31 designates a stationary background in the image environment as, e.g., a rectangular region upon operator's operation with respect to these dl9played environmental image frames at the start of movement. More speciically, the ope~ator uses the region deslgnating circuit 32 to designate a known background (a stationary image portion with respect to the environment) from the environmental image obtained upon the start of movement. The background image is set ln the environmental image. The background is lnteractively deslgnated by properly changing the positions, sizes and number of rectangular regions.
Of a plurality of features stored as a feature table in the feature table memory 24, the feature detecting circuit 33 selectively extracts features belonging to the designated background region. Pieces of feature information thus obtained are stored in the background feature table memory 34 as background feature table information serving as the basis of feature corre-lation processing.
This background feature detection processing is similarly performed for the next frame image obtained upon the start of movement, and the obtained information is stored in the background feature table memory 34.
A portion corresponding to the prediction parameter 202~29~
calculating section 4 comprises a feature selecting cir-cuit 41, a prediction parameter calculating circuit 42, and a prediction parameter table memory 43.
The feature selecting circuit 41 correlates the background features in an image frame obtained at the start of movement and stored ln the background feature table in the backg~ound feature table memory 34 and the background features in an lmage frame obtalned upon the start of movement under the assumption that a large change does not occur between these image frames. It is also posslble to interactively correlate the background features by using the image monitor 31 and the region designating circuit 32. The prediction parameter calcu-lating circuit 42 calculates parameters representing a displacement of the background on the image in accor-dance with the displacement of the moving system, i.e., displacement of the imaglng system (the ITV camera 11 in the imàge input section) which causes displacements of features in accordance with a positional relationship between the correlated background features between the two image frames obtained at the time of movement and upon the start of movement. The obtained parameters are written in a prediction parameter table stored in the prediction parameter table memory 43.
The parameters representing the movement of the background are defined as follows when a coordinate system associated with the position of the imaging ... . -2~2~
system (ITV camera 11) is defined, e.g., as shown in Fig. 5:
That is, a translational component of a motion of the imaging system is given as:
(-TX, -Ty, -Tz) and lts rotational component is given as:
( -nx, -ny, -nz ) Then, the motion of a stationary ob~ect (i.e., the background) in the envlronment with respect to the camera coordinate system is given as follows when its rotational component is given to be sufficiently small:
Xt+l ( Xt+l ~ Yt+l ~ Zt+l ) , R-XX(xt, Yt, Zt) + T ... (1) where ~ 1 -Qy nz-R - nz 1 -nx _-ny nX l_ T , (Tx, Ty, Tz) A pro~ection position (xt, Yt) on the Xt image, i.e., the coordinates are given as follows:
Xt = Xt/zt~ Yt = Yt/Zt ... (2) Assumlng that:
TX/zt = TX
TY/Zt = Ty TZ/Zt = Tz 2~2~9~
the following equations are derived from equations (1 and (2):
Xt+l = xt - n zYt + ny +Tx 1 - n yxt + QXYt + TZ
Xt+l ~ --1 - n yxt + n XYt ~ Tz When the above equatlons are solved with respect to the point (Xt, Yt)~ the following positional relatlonship can be obtalned:
Xt ' ~(Xt+l~ Yt+l) + (l/Zt)~(Xt+l~ Yt+l) Yt = ~(xt+l, Yt+l) + (l/Zt)~(Xt+l~ Yt+l) ... (3) where ~ Dxt+l + EYt+l + F
Axt+l +~BYt+l + C
~ = GXt+l + HYt+l + I
Axt+l + BYt+l + C
~ Jxt+l + Kyt+l + L
Axt+l + Byt+l + C
'}' MXt+l + Nyt+l + o Axt+l + BYt+l + C
The fifteen parameters A to o have the following meanings:
A = ny + nxnz B = -QX + nyQZ
- 16 ~2~29~
C = 1 + nz2 D = 1 + nX2 E = S2z + QXnY
F = -nz + nXnZ
G= -QZ + QXnY
H - 1 + ny2 I ~ nX ~ nynz J = Tz - nxTy K = nXTX + nZTZ
L = - ( TX - nzTy ) M = - ( nyTy ~ nzTz ) N = Tz + QYTX
O = -Ty + QZTX
The flfteen parameters A to 0 are defined as pre-diction parameters representing features between image frames obtained by the mobile imaging system. When these parameters are obtalned by the predlction para-meter calculating circuit 42, displacements of features extracted from image frames between a plurality of image frames can be predicted.
More specifically, for example, the fifteen predic-tion parameters A to o correlate the background features of two image frames (at least eight features of each image) at the time of movement and upon the start of movement of the imaging system under the assumption that a z initial value (i.e., a distance between the background and the imaging system prior to movement) is - 172~2~29~
known. These pieces of position information are substi-tuted to obtain the correlated background features.
When zt is eliminated from equations (3), the following equation is obtained:
~(Xt+l~ Yt+l)Yt ~ ~Xt+l~ Yt+l)Xt + ~xt+1, Yt+~ Xt+l~ Yt+l) -~xt~1, Yt+~ Xt~1~ Yt~1) - 0 ... ~4) This equation represents a line.
Equation ~4) representing the line indicates that a point Pt~xt, Yt) in the immediately preceding frame ls present in the region ~estimated line) represented by equation ~4) if the feature Pt+l~Xt+l~ Yt+l) ln the image belongs to a stationary object with respect to the environment. When the feature is found in the image, the line represented by equation ~4) ln the immediately preceding frame is searched to determine whether the corresponding feature is present, thereby determining whether the feature belongs to the sta-tionary object with respect to the environment. That is, when the corresponding feature is present on the line represented by equation ~4), the feature of interest belongs to the stationary background object with respect to the environment. However, when the feature corresponding to the feature of interest is not present, the feature of interest belongs to a new object - 182~2~2~
whlch is not the background.
The estimation line coefficient calculating circuit 51 constituting a portion corresponding to the estima-tion line calculating portion 5 obtains position infor-mation of the feature extracted from the moving imageframe in accordance with the feature table in the feature table memory 24 and ca~culates ~he coefflclents ~ , and ~ in accordance with the 15 prediction parameters stored in the prediction parameter table memory 43, thereby obtaining an estimation line repre-senting the region in which each feature belongs.
A portion corresponding to the feature searching section 6 comprises a feature candidate selecting clr-cult 61, a slmllarity calculatlng circuit 62, a feature determinlng circuit 63, and a moving object candidate table 64.
As for a plurallty o eatures extracted rom the moving image frame, the feature candidate selecting circuit 61 extracts as feature candidates features of the immediately preceding frame which are stored in the feature table memory 24 and are present on the estimation line. That is, an estimation line (i~e~
an estimation region) in which correlated features of the immediately preceding frame are present is obtained for the features extracted from the moving image frame, as shown in Fig. 6, and it is determined whether the features are present on this estimation line. All 2~2~290 features present on the estimation line are obtained as features which can be correlated to the features of the current frame.
The similarity calculating circuit 62 calculates a slmilarity between a feature of the image and its neigh-boring partial image region in the immediately preceding frams and a feature and its neighborlng partlal image region in the current frame. Thls similarity is obtained as follows:
lQ S(Li~ - Ri~)2/(2Li~ 2Ri;) where Li~ and Ri~ are image densities of the correlated features and neighboring regions between the image of the current frame and the image of the immediately preceding frame.
The feature determlning circuit 63 compares the obtained similarity with a predetermined threshold value and determlnes whether the image feature~ are suficiently similar to each other. That is, the correlated features are defined as similar image features from all the features of the image of the immediately preceding frame. Feature for which corre-lated features cannot be detected in the immediately preceding frame by this determination processing are determined to be new features. The information of such new features is stored as a moving body candidate table in the moving body candidate table 64.
A portion corresponding to the moving body ~2~
determining section 7 comprlses a neighboring feature detecting circuit 71 and a moving object determining circuit 72.
The neighboring feature detecting circuit 71 extracts features, the distance values of which are smaller than a predetermined threshold value, from the feature9 stored in the moving ob~ect ~andidate table 64.
The moving ob~ect determlnlng circuit 72 determines the number of features and a ratio of the area of the features to the area of the entire image and detects that features which are determined not to belong to the background belong to a moving ob~ect entering the environment.
As described above, in the moving ob~ect detectlng system of the present lnventlon, regularity of motion of the background upon movement of the lmaglng system is taken into conslderation. The re~ion of movement of features upon movement of the imaging system is esti-mated to obtaln feature correlatlon between a plurality of image frames. In additlon, the region of movement of the features is estimated as a line region in accordance with the prediction parameters determined on the basis of the motion data of the background in the initial image frames. For this reason, the moving object can be accurately and quickly detected by very simple pro-cessing for verifying whether features corresponding to the prediction region on the line are present.
202~29~
The moving object in the image can be efficiently detected as information-of features which cannot be correlated while background features correlated to the estimation line are sequentially excluded from the image S frames sequentially obtained by the mobile imaging system. For example, this technique ls very effective to realize a mobile robot or the like.
The present invention is not llmited to the par-tlcular embodiment described above. For example, when the lmaglng system is sub~ected to only a translational motlon, rotatlon of the coordinate system need not be taken into consideration. Therefore, the prediction parameters can be much slmplified, and the operations required for calculatlng a predlctlon llne are further simplifled. When generatlon of predlctlon llnes for a plurallty of features obtalned from an lmage frame are concurrently performed, a processing speed can be increased. Other changes and modifications may be made without departing from the spirit and scope of the inventlon.
The present invention relates to a moving ob;ect detecting system for effectively detecting a moving ob~ect in a given environment from environmental image frames momently obtained by a mobile imaging system lncorporated ln a moving unit such as a mobile robot.
One of the lmportant techniques for realizlng a state monitor apparatu~, a mobile robot, or the like is a technique for detecting a movlng ob~ect in a glven environment from envlronmental image frames from moment to moment obtained by an imaging system which ls movable, Thls technique for detecting the moving ob~ect from a moving image is based on a so-called motion analysls. In recent years, extensive studies have been made on this detection technique in the fields of computer visions. ~his technique, however, is not applied to practlcal apparatuses or fleld works.
Image processing for detectlng such a movlng ob~ect is called a gradlent-based scheme for generally detectlng spatlal and temporal changes in image den-sities of moving image frames as an ob~ect of interest.In addition to this scheme, a feature-based scheme is available in which features such as edges of a moving image ti.e.~ an ob~ect of interest) are extracted, and these features are correlated to each other between image frames having a time difference, thereby esti-mating a displacement and hence detecting the moving ob;ect.
. . ~ -, , , 2~2~2~
Although the former scheme, i.e., the gradient-based scheme has an advantage in that a fine displace-ment can be attained with respect to the entire image, since spatial and temporal changes in image densities s are detected by differentiations, detection precision is greatly degraded in the presence of noise in the image.
In addltion, in principle, accurate estimation values cannot be obtained in a region having a small change in denslty or a boundary region of the object.
To the contrary, in the latter scheme, i.e., the feature-based scheme, although portions of lnterest sub~ected to displacement measurements are dispersed in a distribution since features in an image are used, a problem such as degradation of detection preclslon caused by noise can be greatly reduced. In the feature-based scheme, it is not easy to correlate the features to each other between lmage rames havlng a time difference. ~his correspondence processing requires a long period of time if it is accurately per-formed in accordance with its theory. For this reason,the moving ob;ect in an environment monitored by a moving image cannot be detected at high speed.
In the conventional image processing techniques, the moving object in its environment cannot be accurately detected at high speed from image frames momently detected by a mobile imaging system.
It is an object of the present invention to provide ~?~29~
a moving object detecting system which can easily, pre-cisely, quickly detect a moving ob;ect in an environment from environmental image frames momently obtained by a mobile imaging system.
The moving object detectlng system accordlng to the present inventlon has a feature extractlng sectlon for sequentially extracting eatures from image frames momently and sequentlally imaged by a mobile imaging system. The moving object detecting system detects background features included in a background reglon of the image frames from the features extracted by the feature extracting section. For example, parameters for predicting a motion of the background region in the lmage upon movement of the imaging system are obtained from a positional relationship between the corresponding background features between a plurallty of lmage frames in, e.g, an inltlal period of movement. The moving ob~ect detecting system estimates a region where features detected from image frames obtained by the mobile imaging system may have been present at imme-diately preceding moments (if a given feature belongs to the background) on the basis of the prediction para-meters and detects whether a feature corresponding to the given feature belongs to the estimated region of presence, thereby detecting the presence/absence of the moving object.
In the system of the present invention, the 2~2~2~
features apparently belonging to the background are correlated to each other from an image obtained in the initial period of movement, and the prediction parame-ters are obtained from this positional relationship.
The system then determines whether the corresponding feature is present within the estimated reglon based on the predictlon parameters and whether the feature belongs to the background, thereby detectlng the moving ob~ect ln the image.
In the movlng object detecting system according to the present invention, the prediction parameters are estimated from motion of the known background at the start of movement of the mobile imaging system. On the basis of the predictlon parameters, those positions ln the immedlately preceding frame which correspond to the respectlve features obtained from the moving image frame are estimated, and only those features which do not have corresponding features at the estimatad posltions in the immediately preceding frame are extracted. Therefore, the moving ob;ect in the moving image can be effi-ciently, accurately searched at high speed.
This invention can be more fully understood from the following detailed descrlption when taken in con-~unction with the accompanying drawings, in which:
Fig. l is a block diagram illustrating an arrange- -ment of a moving object detecting system according to the first embodiment of the present invention;
- 52~2l~2~
Fig. 2 is a block diagram illustrating an arrange-ment of a moving ob;ect detecting system according to the second embodiment of the present invention;
Figs. 3A and 3B are views illustrating input image frames which have changes in backgrounds upon movement of an imaging system and which are used to explaln the system 9hown in Fig. 2;
Fig. 4 is a view showing directlons of spatlal dif-ferentiations of input image frames so as to explain the system shown ln Flg. 2;
Flg. 5 ls a vlew for explaining a coordinate system whlch ls used to explain the system of Fig. 2 and which is associated with a motion of a camera serving as an imaglng system; and Flg. 6 is a vlew showing estimation llnes for features used for explalnlng the apparatus of Fig. 2.
A moving ob~ect detectlng system according to the first embodiment o the present inventlon will be described with reference to the accompanying drawings.
The moving object detecting system shown in Fig. 1 comprises an image input section 1, a feature detecting section 2, a background feature detecting section 3, a prediction parameter calculating section 4, an estima-tion line calculating section 5, a feature searching section 6, and a moving object determining section 7.
The image input section 1 comprises an imaging system mounted on a moving system such as a mobile robot .
- 6 2~2~29a and momently image frames and inputs moving environmen-tal image frames to the system. Upon movement of the moving system, frames of image information sequentially imaged and input from the image input section whose imaglng positlon ls sequentially changed are supplled to ~he feature extractlng sectlon 2. The feature extractlng sectlon 2 sequantially extracts reglons havlng large spatlal dlfferentlal values as features.
The background feature detectlng sectlon 3 detects background features of a plurallty of image frames which are lnput by the image input section l in an initial perlod of movement, from pieces of feature information respectively extracted from the plurality of image frames. The background features are correlated to each other between the plurality of image frames. That is, the background feature detectlng section 3 processes lnput lmage frames in the lnltial parlod of movement as lnltial lmage frames servlng as criteria of subsequent lmage processing and designates its background region to correlate features to each other between the plurallty of image frames in this background region. The predlc-tion parameter calculating section 4 calculates parame-ters for predicting a motion of the background region movlng in the input image frames upon movement of the image input section l. These parameters are 15 parame-ters representing changes ln the imaglng system (to be described in detail later in association with the second - 7 - 2~2~29~
embodiment).
As for the features detected in units of input image frames by the feature extracting section 2 on the -basis of input image frames momently and sequentially input by the image input section 1, the estimation line calculating section 5 estimates (reverse-predicts) a reglon o presence o~ the glven ~ature obtalned at an lmmedlately preceding timing to be a line region ~to be described later) on the basis of the parameters obtained by the prediction parameter calculating section 4 and the positions of the features in the input image frames.
This region of presence of the feature is performed in units of features extracted from the input lmage frames.
The feature searching section 6 checks whether a lS correspondlng feature ls present ln an estlmated region of presence of each feature of the immedlately preceding timing obtained by the estimation line calculating sec-tion 5. If the corresponding featura is not detected from the lmage of the previous frame, the feature infor-mation is output to the moving ob~ect determiningsection 7. Pieces of feature information having no correspondence with features in the image of the pre-vious frame are acquired by the moving ob~ect deter-mining section 7. The moving object determining section 7 checks the number of features and their positional proximities. The feature count and proximity data are detected as moving object information in the ' - 8 - 2~2 ~2 corresponding input image.
In the moving object detecting system having the above arrangement, the following series of operations are basically performed.
(a) Regions where features extracted from sequen-tlally input image frames may have been present ln image frames obtained at the immedlately precedlng tlmlngs are estimated (i the features belong to the background)~
(b) When features correspondlng the estlmated reglons are not found, the extracted features are detected as new features of the image frames.
(c) Informatlon of the new features ls processed to dstect informatlon of a movlng ob~ect appearlng in an lmage.
Slnce the estlmated reglons of the features of the image frames obtalned at the immedlately precedlng timings are obtalned as llne regions from the predlctlon parameters representing the background motion obtalned in accordance with a relationship between background features correlated to each other from the lmage frames obtained at, e.g., the initial period of movement, the correlation processing (i~e~ processing for determining whether the features are present ln the estlmated regions) can be effectively, easily performed. The processing speed of determination processing can be sufficiently increased.
For this reason, the moving object appearing in 9 2~2~
image frames sequentially obtained by the mobile imaging system can be accurately detected at high speed.
The moving ob;ect detecting system according to the present invention may be arranged as a system s corresponding to the sections shown in Fig. 1 or a comblnatlon of clrcult blocks. ~owever, the second embodiment exemplifles a practical system having substantially the same functlon as described above.
Flg. 2 shows an arrangement of a moving ob~ect detecting system according to the second embodiment.
Referring to Fig. 2, a portion corresponding to the image input section 1 shown in Fig. 1 comprises an ITV (industrial television) camera 11, an A/D
(analog-to-dlgital) converter 12, and an ITV interface 13.
The ITV camera 11 is mounted on the moving body and ls used to sequentlally image and lnput envlronmental lmage frames obtained upon movement of the moving body.
The A/D converter 12 convèrts the image signal obtained by the ITV camera 11 into digital data. The ITV inter-face 13 is an interface for sequentially storing the converted digital image data into an image memory 15 through an image bus 14.
The image memory 15 has a memory capacity capable of storing a plurality of image frames. The read/write access of the image memory 15 is properly controlled in response to a control signal supplied through a control - lo ~2~2~
bus 16 connected to the image memory 15.
In the moving environmental image frames sequen-tlally imaged by the ITV camera ll, as shown in Figs. 3A
and 3B, stationary backgrounds are changed in the s obtained image frames in their imaging environments upon changes in lmaging position upon movement of the ITV camera 11. The movlng ob~ect detecting syst~m is used to efflciently detect a movlng ob~ect ~an image component except for the background) from envlronmental lmage frames accompanying displacements of lmage information in the background.
A portlon corresponding to the feature detecting section 2 for detectlng features from image frames sequentially obtained by and input from the ITV camera lS 11 in the image lnput section comprlses a spatial dif-ferentiator 21, a threshold clrcuit 22, and a feature position detecting circuit 23, all o which are con-nected to the lmage memory 15 through the image bus 14.
This portion also comprises a feature table memory 24 connected to the above constituent circuits through the control bus 16.
The spatial differentiator 21 calculates difference values for a local region of an image stored in the image memory in radial directions (eight directions) along predetermined fourth axes at equal angular inter-vals, as shown in Fig. 4, thereby performing spatial differentiation. The threshold circuit 22 compares 2 ~
a threshold value with the difference values of the respective directions calculated by the spatial dif-ferentiator 21, thereby detecting features.
More specifically, when a difference value of each direction in a portion of an image exceeds a predeter-mlned threshold value, this portlon ls detected as a feature image portion having a largé spatial dlferen-tlal value. Feature detection information is supplied to the feature position detecting circuit 23, and the posltlon of the feature on the image is calculated.
Pleces of feature detection position information are sequentlally stored as a feature table in the feature table memory 24.
Thls feature detection processing is performed in units of lmage frames sequentlally obtained and stored in the image memory 15. Pleces of feature detectlon posltlon information obtained from each image are classified, and the classifiad pieces of information are written in the feature table.
A portion corresponding to the background feature detecting section 3 comprises an image monitor 31, a region designating circuit 32, a feature detecting circuit 33, and a background feature table memory 34.
The image monitor 31 is connected to the image bus 14 to monitor an image stored in the image memory 15.
The region designating circuit 32 causes the image moni-tor 31 to sequentially display image frames sequentially - 12 2~2~2~
obtained and input by the ITv camera 11 in the image input section at, e.g., the start of movement. The image monitor 31 designates a stationary background in the image environment as, e.g., a rectangular region upon operator's operation with respect to these dl9played environmental image frames at the start of movement. More speciically, the ope~ator uses the region deslgnating circuit 32 to designate a known background (a stationary image portion with respect to the environment) from the environmental image obtained upon the start of movement. The background image is set ln the environmental image. The background is lnteractively deslgnated by properly changing the positions, sizes and number of rectangular regions.
Of a plurality of features stored as a feature table in the feature table memory 24, the feature detecting circuit 33 selectively extracts features belonging to the designated background region. Pieces of feature information thus obtained are stored in the background feature table memory 34 as background feature table information serving as the basis of feature corre-lation processing.
This background feature detection processing is similarly performed for the next frame image obtained upon the start of movement, and the obtained information is stored in the background feature table memory 34.
A portion corresponding to the prediction parameter 202~29~
calculating section 4 comprises a feature selecting cir-cuit 41, a prediction parameter calculating circuit 42, and a prediction parameter table memory 43.
The feature selecting circuit 41 correlates the background features in an image frame obtained at the start of movement and stored ln the background feature table in the backg~ound feature table memory 34 and the background features in an lmage frame obtalned upon the start of movement under the assumption that a large change does not occur between these image frames. It is also posslble to interactively correlate the background features by using the image monitor 31 and the region designating circuit 32. The prediction parameter calcu-lating circuit 42 calculates parameters representing a displacement of the background on the image in accor-dance with the displacement of the moving system, i.e., displacement of the imaglng system (the ITV camera 11 in the imàge input section) which causes displacements of features in accordance with a positional relationship between the correlated background features between the two image frames obtained at the time of movement and upon the start of movement. The obtained parameters are written in a prediction parameter table stored in the prediction parameter table memory 43.
The parameters representing the movement of the background are defined as follows when a coordinate system associated with the position of the imaging ... . -2~2~
system (ITV camera 11) is defined, e.g., as shown in Fig. 5:
That is, a translational component of a motion of the imaging system is given as:
(-TX, -Ty, -Tz) and lts rotational component is given as:
( -nx, -ny, -nz ) Then, the motion of a stationary ob~ect (i.e., the background) in the envlronment with respect to the camera coordinate system is given as follows when its rotational component is given to be sufficiently small:
Xt+l ( Xt+l ~ Yt+l ~ Zt+l ) , R-XX(xt, Yt, Zt) + T ... (1) where ~ 1 -Qy nz-R - nz 1 -nx _-ny nX l_ T , (Tx, Ty, Tz) A pro~ection position (xt, Yt) on the Xt image, i.e., the coordinates are given as follows:
Xt = Xt/zt~ Yt = Yt/Zt ... (2) Assumlng that:
TX/zt = TX
TY/Zt = Ty TZ/Zt = Tz 2~2~9~
the following equations are derived from equations (1 and (2):
Xt+l = xt - n zYt + ny +Tx 1 - n yxt + QXYt + TZ
Xt+l ~ --1 - n yxt + n XYt ~ Tz When the above equatlons are solved with respect to the point (Xt, Yt)~ the following positional relatlonship can be obtalned:
Xt ' ~(Xt+l~ Yt+l) + (l/Zt)~(Xt+l~ Yt+l) Yt = ~(xt+l, Yt+l) + (l/Zt)~(Xt+l~ Yt+l) ... (3) where ~ Dxt+l + EYt+l + F
Axt+l +~BYt+l + C
~ = GXt+l + HYt+l + I
Axt+l + BYt+l + C
~ Jxt+l + Kyt+l + L
Axt+l + Byt+l + C
'}' MXt+l + Nyt+l + o Axt+l + BYt+l + C
The fifteen parameters A to o have the following meanings:
A = ny + nxnz B = -QX + nyQZ
- 16 ~2~29~
C = 1 + nz2 D = 1 + nX2 E = S2z + QXnY
F = -nz + nXnZ
G= -QZ + QXnY
H - 1 + ny2 I ~ nX ~ nynz J = Tz - nxTy K = nXTX + nZTZ
L = - ( TX - nzTy ) M = - ( nyTy ~ nzTz ) N = Tz + QYTX
O = -Ty + QZTX
The flfteen parameters A to 0 are defined as pre-diction parameters representing features between image frames obtained by the mobile imaging system. When these parameters are obtalned by the predlction para-meter calculating circuit 42, displacements of features extracted from image frames between a plurality of image frames can be predicted.
More specifically, for example, the fifteen predic-tion parameters A to o correlate the background features of two image frames (at least eight features of each image) at the time of movement and upon the start of movement of the imaging system under the assumption that a z initial value (i.e., a distance between the background and the imaging system prior to movement) is - 172~2~29~
known. These pieces of position information are substi-tuted to obtain the correlated background features.
When zt is eliminated from equations (3), the following equation is obtained:
~(Xt+l~ Yt+l)Yt ~ ~Xt+l~ Yt+l)Xt + ~xt+1, Yt+~ Xt+l~ Yt+l) -~xt~1, Yt+~ Xt~1~ Yt~1) - 0 ... ~4) This equation represents a line.
Equation ~4) representing the line indicates that a point Pt~xt, Yt) in the immediately preceding frame ls present in the region ~estimated line) represented by equation ~4) if the feature Pt+l~Xt+l~ Yt+l) ln the image belongs to a stationary object with respect to the environment. When the feature is found in the image, the line represented by equation ~4) ln the immediately preceding frame is searched to determine whether the corresponding feature is present, thereby determining whether the feature belongs to the sta-tionary object with respect to the environment. That is, when the corresponding feature is present on the line represented by equation ~4), the feature of interest belongs to the stationary background object with respect to the environment. However, when the feature corresponding to the feature of interest is not present, the feature of interest belongs to a new object - 182~2~2~
whlch is not the background.
The estimation line coefficient calculating circuit 51 constituting a portion corresponding to the estima-tion line calculating portion 5 obtains position infor-mation of the feature extracted from the moving imageframe in accordance with the feature table in the feature table memory 24 and ca~culates ~he coefflclents ~ , and ~ in accordance with the 15 prediction parameters stored in the prediction parameter table memory 43, thereby obtaining an estimation line repre-senting the region in which each feature belongs.
A portion corresponding to the feature searching section 6 comprises a feature candidate selecting clr-cult 61, a slmllarity calculatlng circuit 62, a feature determinlng circuit 63, and a moving object candidate table 64.
As for a plurallty o eatures extracted rom the moving image frame, the feature candidate selecting circuit 61 extracts as feature candidates features of the immediately preceding frame which are stored in the feature table memory 24 and are present on the estimation line. That is, an estimation line (i~e~
an estimation region) in which correlated features of the immediately preceding frame are present is obtained for the features extracted from the moving image frame, as shown in Fig. 6, and it is determined whether the features are present on this estimation line. All 2~2~290 features present on the estimation line are obtained as features which can be correlated to the features of the current frame.
The similarity calculating circuit 62 calculates a slmilarity between a feature of the image and its neigh-boring partial image region in the immediately preceding frams and a feature and its neighborlng partlal image region in the current frame. Thls similarity is obtained as follows:
lQ S(Li~ - Ri~)2/(2Li~ 2Ri;) where Li~ and Ri~ are image densities of the correlated features and neighboring regions between the image of the current frame and the image of the immediately preceding frame.
The feature determlning circuit 63 compares the obtained similarity with a predetermined threshold value and determlnes whether the image feature~ are suficiently similar to each other. That is, the correlated features are defined as similar image features from all the features of the image of the immediately preceding frame. Feature for which corre-lated features cannot be detected in the immediately preceding frame by this determination processing are determined to be new features. The information of such new features is stored as a moving body candidate table in the moving body candidate table 64.
A portion corresponding to the moving body ~2~
determining section 7 comprlses a neighboring feature detecting circuit 71 and a moving object determining circuit 72.
The neighboring feature detecting circuit 71 extracts features, the distance values of which are smaller than a predetermined threshold value, from the feature9 stored in the moving ob~ect ~andidate table 64.
The moving ob~ect determlnlng circuit 72 determines the number of features and a ratio of the area of the features to the area of the entire image and detects that features which are determined not to belong to the background belong to a moving ob~ect entering the environment.
As described above, in the moving ob~ect detectlng system of the present lnventlon, regularity of motion of the background upon movement of the lmaglng system is taken into conslderation. The re~ion of movement of features upon movement of the imaging system is esti-mated to obtaln feature correlatlon between a plurality of image frames. In additlon, the region of movement of the features is estimated as a line region in accordance with the prediction parameters determined on the basis of the motion data of the background in the initial image frames. For this reason, the moving object can be accurately and quickly detected by very simple pro-cessing for verifying whether features corresponding to the prediction region on the line are present.
202~29~
The moving object in the image can be efficiently detected as information-of features which cannot be correlated while background features correlated to the estimation line are sequentially excluded from the image S frames sequentially obtained by the mobile imaging system. For example, this technique ls very effective to realize a mobile robot or the like.
The present invention is not llmited to the par-tlcular embodiment described above. For example, when the lmaglng system is sub~ected to only a translational motlon, rotatlon of the coordinate system need not be taken into consideration. Therefore, the prediction parameters can be much slmplified, and the operations required for calculatlng a predlctlon llne are further simplifled. When generatlon of predlctlon llnes for a plurallty of features obtalned from an lmage frame are concurrently performed, a processing speed can be increased. Other changes and modifications may be made without departing from the spirit and scope of the inventlon.
Claims (14)
1. A moving object detecting system comprising:
image acquiring means, having an imaging system, for acquiring image frames sequentially obtained by said imaging system;
feature extracting means for extracting features of each image frame acquired by said image acquiring means;
background feature detecting means for detecting background features belonging to a background region of the image frame from the features extracted by said feature extracting means;
prediction parameter calculating means for obtaining prediction parameters for predicting a motion of the background region in the image frame upon move-ment of said imaging system in accordance with a posi-tional relationship between the correlated background features between a plurality of image frames on the basis of the background features detected by said background feature detecting means;
region estimating means for estimating a region of a given feature corresponding to the feature of an image frame of an immediately preceding frame by using the prediction parameters obtained by said prediction para-meter calculating means when the given feature detected from the image frame obtained by said imaging system is assumed as a feature belonging to a background; and moving object determining means for determining whether a feature corresponding to the given feature is present in the estimation region obtained by said region estimating means.
image acquiring means, having an imaging system, for acquiring image frames sequentially obtained by said imaging system;
feature extracting means for extracting features of each image frame acquired by said image acquiring means;
background feature detecting means for detecting background features belonging to a background region of the image frame from the features extracted by said feature extracting means;
prediction parameter calculating means for obtaining prediction parameters for predicting a motion of the background region in the image frame upon move-ment of said imaging system in accordance with a posi-tional relationship between the correlated background features between a plurality of image frames on the basis of the background features detected by said background feature detecting means;
region estimating means for estimating a region of a given feature corresponding to the feature of an image frame of an immediately preceding frame by using the prediction parameters obtained by said prediction para-meter calculating means when the given feature detected from the image frame obtained by said imaging system is assumed as a feature belonging to a background; and moving object determining means for determining whether a feature corresponding to the given feature is present in the estimation region obtained by said region estimating means.
2. A system according to claim 1, wherein said imaging system includes a television camera.
3. A system according to claim 1, wherein said imaging system is mounted on a moving unit.
4. A system according to claim 3, wherein said image acquiring means comprises means for acquiring image frames sequentially obtained with movement of said imaging system.
5. A system according to claim 1, wherein said feature extracting means includes means for calculating spatial differential values of image information in a plurality of directions and extracting portions having larger spatial differential values than a predetermined value as features.
6. A system according to claim 1, wherein said background feature detecting means includes means for detecting the background features belonging to a background region from the features of the plurality of image frames.
7. A system according to claim 1, wherein said background feature detecting means includes means for displaying an image, means for designating the background region in the image displayed by said image displaying means in accordance with an operator's operation, and means for detecting the features belonging to the background region designated by said designating means as the background features.
8. A system according to claim 1, wherein said prediction parameter calculating means includes means for calculating the prediction parameter on the basis of a plurality of image frames at the start of movement.
9. A system according to claim 1, wherein said prediction parameter calculating means includes means for determining whether features are correlated to each other on the basis of a similarity between image pat-terns near the given feature.
10. A system according to claim 1, wherein said prediction parameter calculating means includes means for displaying an image, means for designating corre-lated features in a plurality of image frames displayed by said image displaying means, and means for detecting the features of the plurality of image frames designated by said designating means as the correlated features.
11. A system according to claim 1, wherein said region estimating means includes means for estimating the region as a line region.
12. A system according to claim 1, wherein said moving object determining means includes means for determining that features for which correlated features are not present in the region are defined as new features.
13. A system according to claim 12, wherein said moving object determining means includes means for determining whether features belong to the moving object on the basis of the correlation of the new features between a plurality of image frames.
14. A method of detecting a moving object, comprising:
the image acquiring step of causing an imaging system to sequentially obtain image frames and acquiring the image frames;
the feature extracting step of extracting features of each acquired image frame;
the background feature detecting step of detecting background features belonging to a background region of the image frame from the extracted features;
the prediction parameter calculating step of obtaining prediction parameters for predicting a motion of the background region in the image frame upon move-ment of said imaging system in accordance with a posi-tional relationship between the correlated background features between a plurality of image frames on the basis of the background features;
the region estimating step of estimating a region of a given feature corresponding to the feature of an image frame of an immediately preceding frame by using the prediction parameters when the given feature detected from the image frame obtained by said imaging system is assumed as a feature belonging to a background; and the moving object determining step of determining whether a feature corresponding to the given feature is present in the estimation region.
the image acquiring step of causing an imaging system to sequentially obtain image frames and acquiring the image frames;
the feature extracting step of extracting features of each acquired image frame;
the background feature detecting step of detecting background features belonging to a background region of the image frame from the extracted features;
the prediction parameter calculating step of obtaining prediction parameters for predicting a motion of the background region in the image frame upon move-ment of said imaging system in accordance with a posi-tional relationship between the correlated background features between a plurality of image frames on the basis of the background features;
the region estimating step of estimating a region of a given feature corresponding to the feature of an image frame of an immediately preceding frame by using the prediction parameters when the given feature detected from the image frame obtained by said imaging system is assumed as a feature belonging to a background; and the moving object determining step of determining whether a feature corresponding to the given feature is present in the estimation region.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP1249071A JP2953712B2 (en) | 1989-09-27 | 1989-09-27 | Moving object detection device |
| JP1-249071 | 1989-09-27 |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CA2026290A1 CA2026290A1 (en) | 1991-03-28 |
| CA2026290C true CA2026290C (en) | 1993-12-14 |
Family
ID=17187574
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA002026290A Expired - Fee Related CA2026290C (en) | 1989-09-27 | 1990-09-26 | Moving object detecting system |
Country Status (5)
| Country | Link |
|---|---|
| US (1) | US5103305A (en) |
| EP (1) | EP0420657B1 (en) |
| JP (1) | JP2953712B2 (en) |
| CA (1) | CA2026290C (en) |
| DE (1) | DE69032146T2 (en) |
Families Citing this family (74)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6856873B2 (en) | 1995-06-07 | 2005-02-15 | Automotive Technologies International, Inc. | Vehicular monitoring systems using image processing |
| US6772057B2 (en) | 1995-06-07 | 2004-08-03 | Automotive Technologies International, Inc. | Vehicular monitoring systems using image processing |
| US6442465B2 (en) * | 1992-05-05 | 2002-08-27 | Automotive Technologies International, Inc. | Vehicular component control systems and methods |
| US5663879A (en) * | 1987-11-20 | 1997-09-02 | North American Philips Corporation | Method and apparatus for smooth control of a vehicle with automatic recovery for interference |
| JP2921936B2 (en) * | 1990-07-13 | 1999-07-19 | 株式会社東芝 | Image monitoring device |
| DE69124777T2 (en) * | 1990-11-30 | 1997-06-26 | Canon Kk | Device for the detection of the motion vector |
| US5590217A (en) * | 1991-04-08 | 1996-12-31 | Matsushita Electric Industrial Co., Ltd. | Vehicle activity measuring apparatus |
| FR2680931B1 (en) * | 1991-08-27 | 1997-08-14 | Thomson Csf | METHOD FOR DETECTING AND TRACKING MOVING OBJECTS BY ANALYSIS OF IMAGE SEQUENCES. |
| JPH06101018B2 (en) * | 1991-08-29 | 1994-12-12 | インターナショナル・ビジネス・マシーンズ・コーポレイション | Search of moving image database |
| EP0532052B1 (en) * | 1991-09-12 | 2008-02-13 | FUJIFILM Corporation | Method for extracting object images and method for detecting movements thereof |
| JP2873338B2 (en) * | 1991-09-17 | 1999-03-24 | 富士通株式会社 | Moving object recognition device |
| EP0557007A2 (en) * | 1992-02-15 | 1993-08-25 | Sony Corporation | Picture processing apparatus |
| US5706417A (en) * | 1992-05-27 | 1998-01-06 | Massachusetts Institute Of Technology | Layered representation for image coding |
| DE69322470T2 (en) * | 1992-08-12 | 1999-07-15 | Ibm | System and method for localizing video segment transitions |
| USRE38420E1 (en) * | 1992-08-12 | 2004-02-10 | British Broadcasting Corporation | Derivation of studio camera position and motion from the camera image |
| GB2270436A (en) * | 1992-09-05 | 1994-03-09 | Ibm | Target tracking system |
| DE4311972A1 (en) * | 1993-04-10 | 1994-10-13 | Bosch Gmbh Robert | Process for the detection of changes in moving images |
| KR950002458A (en) * | 1993-06-02 | 1995-01-04 | 이헌조 | Video signal compression / extension device |
| US5687249A (en) * | 1993-09-06 | 1997-11-11 | Nippon Telephone And Telegraph | Method and apparatus for extracting features of moving objects |
| US5434927A (en) * | 1993-12-08 | 1995-07-18 | Minnesota Mining And Manufacturing Company | Method and apparatus for machine vision classification and tracking |
| JP3450449B2 (en) * | 1994-07-18 | 2003-09-22 | キヤノン株式会社 | Imaging device and imaging method thereof |
| US5889878A (en) * | 1994-07-29 | 1999-03-30 | Kabushiki Kaisha Toshiba | Moving object image pickup apparatus |
| KR100287211B1 (en) * | 1994-08-30 | 2001-04-16 | 윤종용 | Bidirectional motion estimation method and system |
| JP3569992B2 (en) * | 1995-02-17 | 2004-09-29 | 株式会社日立製作所 | Mobile object detection / extraction device, mobile object detection / extraction method, and mobile object monitoring system |
| US6493031B1 (en) * | 1995-03-31 | 2002-12-10 | Canon Kabushiki Kaisha | Visual information processing method and apparatus for extracting feature quantities from a two-dimensional image signal |
| US5654771A (en) * | 1995-05-23 | 1997-08-05 | The University Of Rochester | Video compression system using a dense motion vector field and a triangular patch mesh overlay model |
| US5854856A (en) * | 1995-07-19 | 1998-12-29 | Carnegie Mellon University | Content based video compression system |
| US5959673A (en) * | 1995-10-05 | 1999-09-28 | Microsoft Corporation | Transform coding of dense motion vector fields for frame and object based video coding applications |
| US5774591A (en) * | 1995-12-15 | 1998-06-30 | Xerox Corporation | Apparatus and method for recognizing facial expressions and facial gestures in a sequence of images |
| FR2742901B1 (en) * | 1995-12-22 | 1998-02-13 | Thomson Multimedia Sa | CORRECTION METHOD OF MOTION ESTIMATION IN IMAGES WITH PERIODIC STRUCTURES |
| DE19601005A1 (en) * | 1996-01-15 | 1997-07-17 | Bosch Gmbh Robert | Process for the detection of moving objects in successive images |
| US6037988A (en) * | 1996-03-22 | 2000-03-14 | Microsoft Corp | Method for generating sprites for object-based coding sytems using masks and rounding average |
| US5767922A (en) * | 1996-04-05 | 1998-06-16 | Cornell Research Foundation, Inc. | Apparatus and process for detecting scene breaks in a sequence of video frames |
| US6075875A (en) * | 1996-09-30 | 2000-06-13 | Microsoft Corporation | Segmentation of image features using hierarchical analysis of multi-valued image data and weighted averaging of segmentation results |
| US6351565B1 (en) * | 1997-02-07 | 2002-02-26 | Matsushita Electric Industrial Co, Ltd. | Data structure for image transmission, method of image transmission, image decoding apparatus, and data recording media |
| JP3384314B2 (en) * | 1997-12-02 | 2003-03-10 | ヤマハ株式会社 | Tone response image generation system, method, apparatus, and recording medium therefor |
| US6400831B2 (en) | 1998-04-02 | 2002-06-04 | Microsoft Corporation | Semantic video object segmentation and tracking |
| US6711278B1 (en) * | 1998-09-10 | 2004-03-23 | Microsoft Corporation | Tracking semantic objects in vector image sequences |
| US6466205B2 (en) * | 1998-11-19 | 2002-10-15 | Push Entertainment, Inc. | System and method for creating 3D models from 2D sequential image data |
| JP4146955B2 (en) * | 1999-02-15 | 2008-09-10 | キヤノン株式会社 | Image processing method and image processing apparatus |
| US6754367B1 (en) * | 1999-09-30 | 2004-06-22 | Hitachi Denshi Kabushiki Kaisha | Method and apparatus for automatically detecting intrusion object into view of image pickup device |
| US6826292B1 (en) * | 2000-06-23 | 2004-11-30 | Sarnoff Corporation | Method and apparatus for tracking moving objects in a sequence of two-dimensional images using a dynamic layered representation |
| US7386170B2 (en) * | 2000-06-30 | 2008-06-10 | Texas Instruments Incorporated | Image object ranking |
| US6778705B2 (en) * | 2001-02-27 | 2004-08-17 | Koninklijke Philips Electronics N.V. | Classification of objects through model ensembles |
| JP4187448B2 (en) | 2002-03-07 | 2008-11-26 | 富士通マイクロエレクトロニクス株式会社 | Method and apparatus for tracking moving object in image |
| US6983020B2 (en) * | 2002-03-25 | 2006-01-03 | Citrix Online Llc | Method and apparatus for fast block motion detection |
| JP4337738B2 (en) | 2002-07-30 | 2009-09-30 | ソニー株式会社 | Image signal processing apparatus and image signal processing method |
| CN1565124B (en) | 2002-07-30 | 2012-12-12 | 索尼株式会社 | Memory device, signal processing apparatus, image signal processing apparatus and methods of the same |
| AU2003270386A1 (en) * | 2002-09-06 | 2004-03-29 | Rytec Corporation | Signal intensity range transformation apparatus and method |
| US8230096B2 (en) | 2005-01-14 | 2012-07-24 | Citrix Systems, Inc. | Methods and systems for generating playback instructions for playback of a recorded computer session |
| US8296441B2 (en) | 2005-01-14 | 2012-10-23 | Citrix Systems, Inc. | Methods and systems for joining a real-time session of presentation layer protocol data |
| US20060159432A1 (en) * | 2005-01-14 | 2006-07-20 | Citrix Systems, Inc. | System and methods for automatic time-warped playback in rendering a recorded computer session |
| US8200828B2 (en) * | 2005-01-14 | 2012-06-12 | Citrix Systems, Inc. | Systems and methods for single stack shadowing |
| US8935316B2 (en) | 2005-01-14 | 2015-01-13 | Citrix Systems, Inc. | Methods and systems for in-session playback on a local machine of remotely-stored and real time presentation layer protocol data |
| JP4177826B2 (en) * | 2005-03-23 | 2008-11-05 | 株式会社東芝 | Image processing apparatus and image processing method |
| JP5061444B2 (en) * | 2005-09-20 | 2012-10-31 | ソニー株式会社 | Imaging apparatus and imaging method |
| CN101137012B (en) * | 2006-07-25 | 2010-09-08 | 富士胶片株式会社 | Screening device and method |
| US7725547B2 (en) * | 2006-09-06 | 2010-05-25 | International Business Machines Corporation | Informing a user of gestures made by others out of the user's line of sight |
| JP4885690B2 (en) * | 2006-11-28 | 2012-02-29 | 株式会社エヌ・ティ・ティ・ドコモ | Image adjustment amount determination device, image adjustment amount determination method, image adjustment amount determination program, and image processing device |
| US7801332B2 (en) * | 2007-01-12 | 2010-09-21 | International Business Machines Corporation | Controlling a system based on user behavioral signals detected from a 3D captured image stream |
| US8295542B2 (en) * | 2007-01-12 | 2012-10-23 | International Business Machines Corporation | Adjusting a consumer experience based on a 3D captured image stream of a consumer response |
| US7971156B2 (en) * | 2007-01-12 | 2011-06-28 | International Business Machines Corporation | Controlling resource access based on user gesturing in a 3D captured image stream of the user |
| US8588464B2 (en) * | 2007-01-12 | 2013-11-19 | International Business Machines Corporation | Assisting a vision-impaired user with navigation based on a 3D captured image stream |
| US7840031B2 (en) * | 2007-01-12 | 2010-11-23 | International Business Machines Corporation | Tracking a range of body movement based on 3D captured image streams of a user |
| US7792328B2 (en) * | 2007-01-12 | 2010-09-07 | International Business Machines Corporation | Warning a vehicle operator of unsafe operation behavior based on a 3D captured image stream |
| US7877706B2 (en) * | 2007-01-12 | 2011-01-25 | International Business Machines Corporation | Controlling a document based on user behavioral signals detected from a 3D captured image stream |
| US8269834B2 (en) | 2007-01-12 | 2012-09-18 | International Business Machines Corporation | Warning a user about adverse behaviors of others within an environment based on a 3D captured image stream |
| CN101908214B (en) * | 2010-08-10 | 2012-05-23 | 长安大学 | Moving object detection method with background reconstruction based on neighborhood correlation |
| US8620026B2 (en) * | 2011-04-13 | 2013-12-31 | International Business Machines Corporation | Video-based detection of multiple object types under varying poses |
| US8615159B2 (en) | 2011-09-20 | 2013-12-24 | Citrix Systems, Inc. | Methods and systems for cataloging text in a recorded session |
| US10037467B2 (en) * | 2013-09-26 | 2018-07-31 | Nec Corporation | Information processing system |
| US9589179B2 (en) * | 2013-12-19 | 2017-03-07 | Microsoft Technology Licensing, Llc | Object detection techniques |
| JP6986721B2 (en) * | 2014-03-18 | 2021-12-22 | パナソニックＩｐマネジメント株式会社 | Decoding device and coding device |
| US20150271514A1 (en) * | 2014-03-18 | 2015-09-24 | Panasonic Intellectual Property Management Co., Ltd. | Prediction image generation method, image coding method, image decoding method, and prediction image generation apparatus |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPS6084610A (en) * | 1983-10-17 | 1985-05-14 | Hitachi Ltd | Guiding device |
| EP0330455A3 (en) * | 1988-02-22 | 1990-07-04 | Kabushiki Kaisha Toshiba | Image encoding apparatus |
-
1989
- 1989-09-27 JP JP1249071A patent/JP2953712B2/en not_active Expired - Fee Related
-
1990
- 1990-09-26 CA CA002026290A patent/CA2026290C/en not_active Expired - Fee Related
- 1990-09-26 US US07/589,782 patent/US5103305A/en not_active Expired - Fee Related
- 1990-09-27 DE DE69032146T patent/DE69032146T2/en not_active Expired - Fee Related
- 1990-09-27 EP EP90310621A patent/EP0420657B1/en not_active Expired - Lifetime
Also Published As
| Publication number | Publication date |
|---|---|
| US5103305A (en) | 1992-04-07 |
| JP2953712B2 (en) | 1999-09-27 |
| JPH03111792A (en) | 1991-05-13 |
| DE69032146D1 (en) | 1998-04-23 |
| EP0420657A2 (en) | 1991-04-03 |
| EP0420657A3 (en) | 1993-12-22 |
| DE69032146T2 (en) | 1998-08-20 |
| EP0420657B1 (en) | 1998-03-18 |
| CA2026290A1 (en) | 1991-03-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CA2026290C (en) | Moving object detecting system | |
| Harville et al. | Foreground segmentation using adaptive mixture models in color and depth | |
| KR100519781B1 (en) | Object tracking method and apparatus | |
| JP2978406B2 (en) | Apparatus and method for generating motion vector field by eliminating local anomalies | |
| EP0181009B1 (en) | Image processing device and method for controlling the same | |
| US20080253617A1 (en) | Method and Apparatus for Determining the Shot Type of an Image | |
| CN110176028B (en) | Asynchronous corner detection method based on event camera | |
| JPH09261646A (en) | Motion detector for image | |
| JPH0220988A (en) | Moving vector detection system for animation encoding device | |
| JPH05236452A (en) | Method and device for detection motion vector | |
| CN111524086B (en) | Moving object detection device, moving object detection method, and storage medium | |
| JP2000175202A (en) | Motion vector detector | |
| US5682438A (en) | Method and apparatus for determining a motion vector | |
| JP2001307104A (en) | Object extraction device for moving image | |
| JP3237815B2 (en) | Motion vector search method and apparatus | |
| JP2506500B2 (en) | Image motion detector | |
| Awasthy | Automatic obstacle detection for a star algorithm using digital image processing | |
| Abidi et al. | Automatic target acquisition and tracking with cooperative fixed and PTZ video cameras | |
| Al-Khalidy et al. | Neuro-Fuzzy SVD Technique for Image Recognition and Safety in Construction Site | |
| JPH02131690A (en) | Movement compensation circuit | |
| Weis | Processing of optical sensor data for tool monitoring with neural networks | |
| Persoon et al. | Industrial image processing by means of an image recognition integrated system | |
| Kim et al. | Moving object discrimination and tracking for unmanned surveillance system | |
| Kim et al. | Optimal representative blocks for the efficient tracking of a moving object | |
| CN117876970A (en) | Workshop intelligent management method and system based on image processing and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| EEER | Examination request | ||
| MKLA | Lapsed | ||
| MKLA | Lapsed |
Effective date: 19990927 |