CA2132515C - An object monitoring system - Google Patents

Abstract

An object monitoring system includes a camera node (2) for monitoring movement of an object (18) to determine as acquisition time when an image of the object (18) is to be acquired and acquiring the image at the predetermined time. The system includes a camera (6) which is able to monitor moving objects (18), and image processing circuitry (10), responsive to the camera (6), which is able to detect a predetermined moving object (18) from other moving and static objects. From the image acquired, information identifying the object (18) can be automatically extracted. The system is particularly suited to monitoring and discriminating large vehicles (18) from other vehicles over a multi-lane roadway, and acquiring high resolution images of the large vehicles (18) at a predetermined acquisition point (22). Image data acquired by a plurality of camera nodes (2) can be sent over a digital telecommunications network (45) to a central image processing system (42) which can exact extract vehicle identifying data, such as licence plate details, and obtain information on vehicle travel between nodes (2).

Description

WO 93/19441_ _ _ ~ ~ ~ ~ ~ ~ ~ p~'/Al,'93/0p1 i5 -I-Arr oatECr MarrcroRarrc sY~rr~.rr i The prrsenc invention relates to as object monitoring system sad, its particular, to a system for monitoring vehicles.
Authorities responsible for traffic maaagcment and the iayus which govern the use of vehicles require systems which can monitor t:affc continuously and detect breach of the law, withour requiting the eapease of having personnel present at rise scene of cye iaftiagemeac. syscecas which are able to monitor a large number of locations, detect in5ria~meats and issue iatrissgement sodas are particularly advantageous as they relieve personnel; such as police, from the task of traffic management and allow there co pursue ocher tasks. By coatiauously moaiaoring a location the systems also as as a deterrent to is in5iagers and may asai:t in redtcciag acadeats which cause toad fatalities sad casualties.
It would also be advto be able to monitor road wage is order to make decisions on toad damage by heavy veiudes. .
A number of traffic management systems atr preaaatly in use, such as speed 30 caraecas sad red light amerss for road t:'all~c. The known sy:rcems employ cameras which else triggered wl~n sa ia~tlagement is detected, optical :enaors placed oa the side of the toad, due aea:ora placed underneath the road sad radar signals reflected from vehicles are wed to detect the p:eseace of a vehicle and deta~mine ia>xio~g~neat. The semoa sad ruler dgoals art used to generate a trigger signal to a4tivace a camera to take r5 a picture of vehicle whidb include details isom which the vehicle can be idled, such as a cat licence plate. Use of toad based kneels is disadvantageous as they require the road to be altered ~ excavated for iascallation or, when placed o~ the side of du road, can be easily det~ed and daaa~aged. Also elearieal cabling aceds to be installed and conaecteil between the aensots and the camera. The use of dtic aigasts which 30 are transmitted to sad reaeaed from a vehicle. suds as aadu signal:, to decea p and ia6~iagamaat is also d>sadwntageous as these signals can be~detaaed by deteai~
r units planed is a vehids to glen the driver as to their .
i,.

H'0 93/14441 PC?/Ah93l0pt t c It is advantageous therefore to provide a system which can detect vehicle presence anti in$ingement without transmitting any electromagnetic signals or using road based sensOn.
'Ihe cameras pteseathy is use also use photographic film which has the disadvantage that it needs to be ooatiauaily replaced at the I location of the camera.
Accordiapty, a number of red light cameras is metropolitan areas do not always include film and do not continuously monitor the corresponding intersection.
I
t0 Speed detection systems which ux only cameras ate bed is a number of publications. T'he systems are able to monitor tragic flow and dared instaataaeous speed iafriagemeats but the systems an relatively limited with respect to the iaformatioa they can obtain on a vehicle whilst it is being monitored, and the systems are also tenable to selectively acquire inforrrnation on specified vehicle types.
is .
T3e pintoes or image: aoqnired by the caaxra also norm~lty need to be examined by personnel to eucract the informacioa to identify the vehicle slid de:ermiaa elx persaa responsible for it, which is a time ~ming process. If the 'ale could be prae~ed within a relatively short time of acquisition rhea it could be task as a bats for alerting 20 authorities is the region to seek tad hold the vehicle, for example, if the infonoaation identifies it a: being uolea. A~ooordiagly, it would be advantaget~'ous to provide s system which cat process images is real time to obtain detailed iafon~atioa on a vehicle and ' i issue alert iaFocmacion tad ia>yiagament notices without reqtdti~ human internatioa.
:S Whoa trsvallins a long distance, vehicle user's, is partia~r truck drivers, read to traasgrass apoed timiu so as ro shorten the lima in travelling to t»
deaiaation and bring the journey their speed may vary from a range which is the limit to one which extxeds the limit. 'the known systems for deteaiag speed ia~ringement coaoentrate oa derocxiag the inataataaeous speed of a vehicle at a pastiarlar ytoeation tad thesefoee 30 don the location a which the deletion unit is pLaoed, it may not detect user's who infri~e sporadically over a long diaraace.. Also tntek and bus drivers wlm exceed a reoammended time of travel by avoiding rest slaps tad iasaurately oomptae log books f~
I

WO 93/ l9aa 1 pGT/A L'93/00114

2~j~~~~ ' _3_ I
may not be detected. Heave, it would be advantageous to provide a syseem which can detect the average speed of a vehicle over a relatively long distaece. It is also advantageous to provide a system which can a~tonitor vehicles-in more than one lane of a mufti-lane carriageway. j i s .
The present invention provides an object monitoring system comprising camen means for monitoring movement of as object to determine an acquisition tame when as imsge of said object is to lx acquired and acquiring said image at said predetermaaed time.
~
The pseseat invention also provides as object monitoring systeaa oonmp:~ing camera means for monitoring moving objects, sad image processing means, responsive to said camera mesas, for detetxiag a predetermined moving object from other moving and static objects.
~s ~ 1 The present inveatian further provides an object monitoring system comprising camera rnea~ for sacking sad acquiring as image of a mo~!ing object from which information idattifying :aid object can be automatically extracted.
30 Preferably said system includes mesas for transmitting said image over a digital telocommunicatiaas network.
The pre:aut invention also provicks a vehicle moaitorlag system, comprising camaca n~ for coatinuaa:ty detactiag sad tnckiag moving velrielea over a mufti-lane 25 carlageway, sad ~quiilaa images of predetermined vehicles at, m aalttisition area oa said catr6a~way from wlriich idrntifyiag information on said vehicles can be exttaaed.
The present iavencion fortf~a provides a vehicle monitoring system comprising a plurality of casters means for trackins sad acquiring images of predetarminod moving 30 vehicles for a reapoctive ales, sad means for processing the image data obuined from said areas ro i~ntify acid vdycle: and obtain iafo:matian oa the travd of said vehicles beeween said areas. i The pr~aent invention also provides a vehicle monitoring system comprising camera means for monitoring; moving vehicles to determine if' said vehicle is of a predetermined type and, in response thereto, capturing respective images of vehicles of said predetermined type.
The present inventi,m further provides a vehicle monitoring system comprising camera means for monitoring a vehicle to detect a law infringement and determine a predetermined time to acquire an image c~f said vehicle, and for capturing an image of said vehicle at said predetermined time in response to detecting said i o infringement.
The present invention also provides a. vehicle monitoring system comprising camera means for monitoring vehicles on a roadway, discriminating between large vehicles, such as trucks and buses, and small vehicles, such as cars, on said roadway 15 so as to acquire images of only the large vehicles from which vehicle information can be obtained.
In accordance with one aspect of the present invention an object monitoring system comprising camera means characterised in that the camera means is adapted to 2o monitor movement of an object to predetermine, based on the monitored movement of the object an acquisition limy at which an image can be acquired at a predetermined position of said object relative to said camera means, and to acquire an image at the predetermined acquisition time and the predetermined position.
25 In accordance with a~zother aspect of the present invention an object monitoring system comprising:
camera means for generating images of an area and for acquiring an image of a prc;determined object, and image procc;s;sing means including:
3o means for subtracting a background image of said area from said images of said area to generate difference images representative of moving objects in said area;

_ L~a_ segmentation means for processing said difference images to generate region images representative of regions corresponding to parts of said moving objects in said area;
classification means for processing and classifying said region images, said classification means including:
means for analyzing the shape of said regions and, on the basis of the analysis, determining valid regions and invalid regions, clustering means for rejecting said invalid regions and generating, on the basis of the geometry of raid valid regions, clusters corresponding to respective ones of said rr~oving object , and means for classifying said clusters by comparing at least one characteristic of said clusters to classification data of said system to determine if one of said clusters corresponds to said predetermined object; and tracking means for tracking said one of said clusters corresponding to t5 said predetermined object to trigger said camera means to acquire said image of said predetermined object.
In accordance with a further aspect of the present invention an object monitoring system compri~~ing:
2o camera means for generating images of an area and for acquiring an image of a predetermined object;
image processing means including:
means for subtracting a background image of said area from said images of said area to gencyrate difference irr~ages representative of moving objects in 25 said area, segmentation means for processing said difference images to generate region image; representative of regions corresponding to parts of said moving objects in said area, classificatic:>n means for processing said region images, said 3o classification means inclu~:Iing:
means for analyzing the shape of said regions and, on the basis of the analysis, determining valii:l regions and invalid regions, -4b-clustering means for rejecting said invalid regions and generating, on the basis of the: geometry of said valid regions, clusters corresponding to respective ones of said moving objects, and means for classifying said clusters by comparing at least one characteristic of said clusters to classification data of said system to determine if one of said clusters correspond:; t~ said predetermined object, and tracking means for tracking said one of said clusters corresponding to said predetermined object to trigger said camera means to acquire said image of said predetermined object; and t o extraction means for processing said image of said predetermined object to extract information identifying said predetermined object.
In accordance with one aspect of the present invention a vehicle monitoring system comprising:
t 5 camera means for generating images of a carriageway and for acquiring imal;es o1' predetermined vehicles, and image processing means including:
means for s~,ibtracting a background image of said carriageway from said images of said carriageway to generate difference images representative of moving vehicles on said carriageway;
2o segmentation rr~eans for processing said difference images to generate region images representati~s;e of regions corresponding to parts of said moving vehicles on sand carriagew;ay;;
classification means for processing said region images, said classification means including:
25 means for analyzing the shape of said regions and, on the basis of the analysis, determining valid regions and invalid regions, clustering means for rejecting said invalid regions and generating, on the basis of the geometry of said valid regions, clusters corresponding to respective ones of said moving vehicles, and 3o means for cuassifying said clusters by comparing at least one characteristic of said clusters to classification data of said system to determine if said clusters correspond to said predetermined veihicles; and -~C-tracking means for tracking said clusters corresponding to said predetermined vehicles to trigger said camera means to acquire said images of said predetermined vehicles.
In accordance with another aspect of the present invention a vehicle monitoring system comprising:
a plurality c~f c;amera means for generating images of respective areas and for acquiring images o l-' predetermined vf:hicles, said areas being remote with respect to one another; and a plurality of image processing means including:
means fox sxabtracting background images of said areas from said images of said areas to generate difference images representative of moving vehicles in said areas;
segmentation means for processing said difference images to generate region images representative of regions corresponding to parts of said moving vehicles in said area;
classification means for processing said region images, said classification rneans including means for analyzing the shape of said regions and, on the basis of the: analysis, determining valid regions and invalid regions, clustering means for rejecting said insealid regions and generating, on the basis of the geometry of said valid regions, clusters corresponding to respective ones of said moving vehicles, and means for cla;ssiif;ying said clusters by comparing at least one characteristic of said clusters to classification data of said system to determine if said clusters corresponds to said predetermined vehicles;
tracking means for tracking said clusters corresponding to said predetermined vehicles to trigger said camera means to acquire said images of said predetermined vehicles; and recognition means for processing said images of said predetermined vehicles to obtain information identifying said predetermined vehicles' In accordance with ,:mother aspect of the present invention a vehicle monitoring system comprising:

-4~d-camera means for generating images of an area and for acquiring an image of a vehicle associated with a law infringement, and image processing means including:
means for subtracting a background image of said area from said images of said area to gencvrate difference images representative of moving vehicles in said area;
segmentati<m means for processing said difference images to generate region images representative of regions corresponding to parts of said moving vehicles in said area;
1 o classificati<an means for processing said region images, said classification means inclucling:
means for analyzing the shape of said regions and, on the basis of the analysis, determining valic:l regions and invalid regions, clustering means for rejecting said invalid regions and generating, on 15 the basis of the geometry <~f said valid regions, clusters corresponding to respective ones of said moving vehicles, and means for detecting said law infringement by comparing at least one characteristic of said clusters, t:o classification data of said system to determine if one of said clusters corresponds i:o said vehicle; and 2o tracking me:ar~s for tracking said one of said clusters corresponding to said vehicle to trigger said camera means to acquire said image of said vehicle.
In accordance with a further aspect of the present invention a vehicle monitoring system comprising camera means for generating images of a carriageway 25 and for acquiring high resc:~lution images of large vehicles, such as trucks and buses, and image processing means including:
means for subtracting a background image of said carriageway from said images cof said carriageway to generate difference images representative of moving vehicles of said carriiageway;
3o segmentation means for processing said difference images to generate region images. representative of regions corresponding to parts of said moving vehicles on said carriageway;
classificatic:m means for processing said region images, said classification means including:

~.e~
means for analyzing the shape of said regions and, on the basis of the analysis, determining valid regions and invalid regions, clustering means for rejecting said invalid regions and generating, on the basis of thc: geometry of said valid regions, clusters corresponding to respective ones of said moving vehicles, and means for classifying said clusters by comparing of at least one characteristic of said clusters to classification. data of said system to determine if said clusters correspond to said large vehicles; and tracking means for tracking said clusters corresponding to said large 1 o vehicles to trigger said camera means to acquire said high resolution images of said large vehicles.
In accordance with one aspect of the present invention an object monitoring system comprising:
I 5 video camera means for generating images of an area to monitor moving objects in said area:, image capture camera means for acquiring a high resolution image of a predetermined object; and image processing means including:
2o means for subtracting a background image of said area from said images of said area to generate difference images representative of said moving objects in said area;
segmentatio~u means for processing said difference images to generate region images representative of regions corresponding to parts of said moving objects 25 in said area;
classification. means for processing said region images, said classification means including:
means for analyzing the shape of said regions and, on the basis of the analysis, determining valid regions and invalid regions, 3o clustering means for rejecting said invalid regions and generating, on the basis of the geometry oi~ said valid regions, clusters corresponding to respective ones of said moving objects, and - 4f~
means for classifying said clusters by comparing at least one characteristic of said clust~:rs to classification data of said system to determine if one of said clusters corresponds to said predetermined object; and tracking means for tracking said one of said clusters corresponding to said predetermined object I:o trigger said image capture means to acquire said high resolution image of said predetermined ob~eca.
In accordance with another aspect of the present invention an object monitoring system comprisin.g.:
1 o video camera :means for generating images of an area to monitor moving objects in said area;
image capture camera means for acquiring a high resolution image of a predetermined object; and image processing means including:
15 means for subtracting a background image of said area from said images of said area to generate difference images representative of said moving objects in said area;
segmentation means for processing said difference images to generate region images representative of regions corresponding to parts of said moving objects 2o in said area;
classification means for processing said region images, said classification means including means for analyzing the shape of said regions and, on the basis of the analysis, determining valid regions and invalid regions, 25 clustering means for rejecting said invalid regions and generating, on the basis of the geometry of said valid regions, clusters corresponding to respective ones of said moving objects, and means for clr:~ssifying said clusters by comparing at least one characteristic of said clusters to classification data of said system to determine if one 30 of said clusters corresponds to said predetermined object;
tracking means for tracking said one of'said clusters corresponding to said predeternai.ned object to trigger said image capture camera means to acquire said high resolution image of said predetermined abject; and _~.g_ extraction means for processing said image of said predetermined object to extract information identifying said predetermined object.
In accordance with a further aspect of the present invention a vehicle monitoring system comprising:
video camera means for generating images of a carriageway to monitor moving vehicles in said carriageway;
image capture camera means fox acquiring a high resolution image of a predetermined vehicle; anct 1o image processing means including:
means for sg;~btracting a background image of said carriageway from said images of'said caxriagcway to generate difference images representative of said moving vehicles on said carriageway;
segmentation means for processing said difference images to generate 15 region images representative of regions corresponding to parts o:f said moving vehicles on said carriageway;
classification means for processing said region images, said classification means including;:
means for analyzing the shape of said regions and, on the basis of the 2o analysis, determining valid regions and invalid regions, clustering means far rejecting said invalid regions and generating, on the basis of the: geometry ol's,aid valid regions, clusters corresponding to respective ones of said moving vehiclw°,s, and means for classifying said clusters by comparing at least one 25 characteristic of said clusters to classification data of said system to determine if said clusters correspond to said r~ra:determined vehicle; and tracking mesan;> for tracking said clusters corresponding to said predetermined vehicle to trigger said image capture camera means to acquire said high resolution image of said predetermined vehicle.
In accordance with ono aspect of the present invention a vehicle monitoring system comprising:

-4h-a plurality of 'video camera means for generating images of respective areas to monitor moving vehicles in said area, said areas being remote with respect to one another;
a plurality c~f image capture camera means for acquiring a high resolution image of one or more predetermined vehicles; and a plurality c;~f iimage processing means including:
means for subtracting background images of said areas from said images of said. areas to generate difference images representative of said moving vehicles in said areas;
segmentation means for processing said difference images to generate region images representative of regions corresponding to parts of said moving vehicles in said areas;
classification uneans for processing said region images, said classification means including:
15 means for analyzing the shapes of said regions and, on the basis of the analysis, determining valid regions and invalid regions, clustering means for rejecting said invalid regions and generating, on the basis ofthc; geometry of said valid regions, clusters corresponding to respective ones of said moving vehicles., and 20 means for classifying said clusters by comparing at least one characteristic of said clusters to classification data of said system to determine if said clusters correspond to said predetermined vehicle;
tracking mean's for tracking said clusters corresponding to said predetermined vehicle to trigl;er said camera means to acquiring said image of said 25 predetermined vehicle; and recognition means for processing said images of said predetermined vehicle to obtain intormatiom identifying said predetermined vehicle.
In accordance with another aspect of the present invention a vehicle 3o monitoring system comprisi.n;~:
video camera means for generating images of a carriageway to monitor moving vehiclc;s in said area;
image captua°e camera means for acquiring a high resolution image of a large vehicle, such as a truck and a bus; and - ~r ~
image proces sing means including:
means for subtracting a background image of said carnageway from said images ojE said carriageway to generate difference images representative of said moving vehicl'aes on said carriageway;
segmentaticyn means for processing said difference images to generate region images representative of regions corresponding to parts of said moving vehicles on said carriagew;:~y;
classification means for processing said region images, said classification means including:
means for analyzing the shape of said regions and, on the basis of the analysis, determining valid relgions and invalid regions, clustering nnerrns for rejecting said invalid regions and generating, on the basis of the geometry of said valid regions, clusters corresponding to respective ones of said moving vehicles, and t5 means for cl.as;sifying said clusters by comparing at least one characteristic of said clusters to classification data of said system to determine if said clusters correspond to said large vehicle; and tracking me;:~ns for tracking said clusters corresponding to said large vehicle to trigl;er said image c<rpture camera means to acquire said high resolution 2o image of said large vehicle.
In accordance with ,;t further aspect a vehicle monitoring system comprising:
video camera means for generating images of an area to monitor moving vehicles in said area;
2s image capture camera means for acquiring a high resolution image of a vehicle associated with a law infringement; and image processing means including:
means for subtracting a background image of said area from said images of said area to generate difference images representative of said moving 3o vehicles in said area;
segmentation means for processing said difference images to generate region images representative of regions corresponding to parts of said moving vehicles in said area:

_ ra.l _ classification means for processing said region images, said classification :means including means for analyzing the shape of said regions and, on the basis of the analysis, determining valid regions and invalid regions, clustering means for rejecting said invalid regions and generating, on the basis of the geometry of ;>aid valid regions, clusters corresponding to respective ones of said moving vehicies, and means fur detecting said law infringement by comparing at least one characteristic of said clustf~rs to classification data of said system to determine if one to of said clusters corresponds to said vehicle; and tracking means for tracking said one of said clusters corresponding to said vehicle to trigger said image capture camera means to acquire said high resolution image of said vehicle.
t 5 A prefE;rred embodiment of the present invention is hereinafter described, by way of example only, with reference to the accompanying drawings wherein:
Figures 1 to 3 are side views illustrating use of a preferred system for monitoring vehicles;
Figurca 4 is a front perspective view illustrating use of a preferred system for 2o monitoring vehicles;
Figure 5 is a block diagram of a preferred embodiment of the vehicle monitoring system;
Figure 6 is a block diagram of connection across a digital telecommunications network of two nodes and a central server of the vehicle monitoring system;
25 Figure '7 is a view illustrating connection of a large number of nodes of the vehicle monitoring system;
Figure ~ is a block diagram of vehicle detection and image capture circuitry of the vehicle monitoring system;
Figure'3 is a digitised image produced by the vehicle detection circuitry from 3o an image generated by a detection camera of the system;

W9 93119441 ~ ~ ~j ~ ~ ~ ~ I Pf'T/AL:93/0p11:
_s_ Figure IO is a block dial of the control of the circuit boards of the vehicle detection circuitry to perform a se~entation process;
Figure 11 is a static baek~ound image stored in the vehicle detection circuitry:
Figure 12 is a di~etence image generated by the vehicle detection circuitry;
Figure 13 is an image illustrating regions of shadow which are faltered from the image obtained by the detection cataeta:
I"sgs::~-~~ is w ~ae~lxd ::'.»~ de:i~~o~ b~ ;.L.a :~as~..°~k.
,ioexe:o:~-.-.soia:;r~;_ ...
Figure is is a histogram of pixel grey levels;
Figure 16 is a real time status display generated by the system;
Figure 17 is a flow diagram illustrating flow between the software tasks of the system;
Figure 1B is a diagram of the formation of "black triangles" in a processing window of the system;
Figure I9 is a diagram illustrating meaaureiment of cawerage of blob regions produced by the system;
Figure 20 is a diagram itlusttating vertical exte:xsion of ,'blob Legions to perform Blusters;
Figure 21 is a graph of extension amounts which are stored in a look-up table of the system;
30 Figure Z2 is a disgtam illusttitiag extaasioa based on blob region width;
Figure 23 is a diagtsam of overlap detection for clusters produced by the system;
Figure 24 is a diagram illustrating a labelling method performed by the systear:
Figure 25 is a daagtam of the roadway coordinates used,'by the system;
Figure 26 is a gnpls of the trajoaory of clusters; ;:
Figure 27 is a graph of the trajectory of clusters transformed to the roadway coordinates;
Figure 28 is a diag:am of data values obtained by tr~jecxory software of the System; I
Figuae 29 is a block diagram of a timing control board bf the system;
Figure 30 is a graph of tl3e operating chaaaaecistics of the aequiaition camera and infrared flash of tire vehicle monitoring system;
Figures 31 sad 32 are image: acquired by the system;

WO 93/ l9d~t t ~ ~ ~ ~ ~ ~ ~ PCT/A 1:93/0011:

Figure 33 is a block diagram of components of the acquisition ratnera, and intorface components for the camera of the image capture circuitry;
Figure 34 is a block diagram of comtnuttications components of nodes of the system, and the eotnponents of an acquisition imago processing system of the system S eotmected over the digital telecommunications network;
Figure 35 is a diagram of the memory layout for a buffer board of the image capture cuaaitry;
Figure 36 is a flow diagram illusreatiag software modules of the acquisition image processing system and communications modules of the ;
Figure 37 is a block diagram of a lice. plate reco~ieian system of the vehicle monitoring system;
Figure 38 is a flow diagram of an image a~uisition p:oc~tte of the liceaex plate recognition system;
Figure 39 is a flow diagram of the software module of the licence plate recognition system;
Figure 40 is a flow diagram of a locate piste module of the liccaa piste recognition system; and Figure 41 is a flow diagram of an optical cbaracter recognition module of the license plate recognition system.
i A vebicie monito~iag system, as shown is Fisures 1 to 7; includes a camera node 2 which is mounted on a bridge or pylon 4 above vehicle trsffi~, as shown in Figures 1 to 3. 'The ca~eru node 2 includes a vehicle detection camera i6, as image ae quisiaoa camera 8 and a tads ooatrol unit 10. Both canuras 6 and 8~ are moaocitrome Ct'D
?3 cameras, with the vehicle detection camera 6 being a wide angle video camera of medium resolution, and the image acquisition camera being a high raso~udon camera.
The detection camera 6 has a wide; field of view 1~ of part of a vehicle cartiageway 16 whic3t is to be monitored by the node 2. '~'he detaxitm camera b 30 monitors vehicles in the fleid of view 12 sad the coat:oi unit 10 pthe images acquired by the detection camera 10 to detect and disa~imisiate vehicles from other objects in the field of view 12. As a vehicle lg eattxa the 5e1~ of view 12 and moves i i WO 93/ 1944a ~ ~ ~ ~ ~ ~ ~ , P~'f / A l'93/40 t 13 _?
towards the node 2, the node 2 analyses the images produced by the detection careers 6 to first detect the vehicle 18 as bring a moving object, which is different from other moving objects or the still background in the view 12, and determines whether the vehicle 18 constieutes an object for which a high resolution image thereof should be obtained by the image acquisition camera 8. The image acquisition cavtaera 8 is mounted on the bridge or pylon 4 so as to have a limited ~cld of view 20 which will include the front of a vehicle 18 when it rcach~ a predetermined Iocaiion 22 oa a carriageway 16. TThe location 22 and the field of view 20 are chosen to bs near the point where movinE
vehicles will leave the 5eld of view 12 of the detection camera 6, as shown in Figure 3.
On determining that the vehicle 18 reprexats as object for which an image is to be acquired, the node 2 estimates the time when the vehicle 8 will enter the 5eld of view of the acquisition camera 8, on the basis of the movement of the vehicle which has been monitored by the detection camera 6. The nod 2 provides trigger ia~ormation to control circuitry associated with the aoquisitioa camera 8 so as to trigger the camera 8 15 at the estimated time. A high resolution image of the $oat of the vehicle I8 is obtained from which considerable identifying information can be derived, such as vehicle type sad licence plate details, by subxquent digital elecxronic processia~~of the image.
i In addition to identifying the vehicle 18 and estimating the time for triggering the 20 acquisition camera 8 the node 2 is able to ux the iafrom ~ the detection camera 6 to discriminate between vehicles on a number of charaete 'rrktica, such as size, to determine those for which high resolution images are t~ be acqtt~red. For example, the system is able to distinguish between large vehicles such as hurk~ sad coaches, and other moving objects within the field of view 12, sub as cars and motor bicycles.
'The soda : is also able to determine from the images obtains by the ~deteccion camera 6 the current speed of tho vehicle 18 sad whether the dtwer is oommi~t~ say crafi;ic or other offences, such as tailgating or illegal lane cheagi~ag. The system can also be used to detect stolen vehicles.
'1"he detection camera 6 sad the control wait 10 are ab6e to monitor all of the moving vehicles 18 and Z2 within the held of view 12 whilst aoquiriag the images of selected vehicles at the location 22. For a mufti-lane carriageway 21, as shown is Figure wp 93i19~4t ~ ~ ~ ~ ~ ~ ~ p~ T/At;~93/ppt 1:
_8_ 4, the field of view 12 of the detection caatacra 6 extends over all of the lanes ~3 and 25 of the carziageway and an image acquisition camera 8 is provided for each Lane ~3 and 35. The node 2 is therefore able to monitor the moving vehicle 18 to determine in which lane it will be when it roaches the image caprute location 22 and activates.
as required.
the acquisition camera 8 corresponding to that lane 23 or 23. .
i The control unit I0, as shown in Figure S, includes vehicle detection circuitry 30 for processing the images generated by the detetxion camera 6 so as to provide nigger signals oa a bus 32 to the image acquisition camera 8. ~r sele~d camera 8 is triggered to acquire an image in aaordaner. with the timing iafotmation determined by the detection circuitry 30, and the camera 8 provides a trigger sigmil on a line 36 to a flash ttiggtring circuit 38, of a corresponding infrared flash 40 mouatod adjacent the scteaed acquisition camera 8. The image obtained by the trigger acquisition camera 8 is received by as image acquisition circuit 34. The detection circuit 3fl deteriniaes the light intensity within the field of view 12 of the detection camera b so as to da~ermiae the correct revel of exposure for the acquisition emcee 8, and is turn the correct level of erJergy to be discharged by the flash 40 to achieve the desired level of expostue. The use of as flt flash is advantageous as activation is difficult to detect visuall~r. Visible wavelengths produced by the flash are removal by 1R band pass filters.
?0 The vehicle monitoring system includes as acquisition '~mage processing system 42 connected to the control unit 10 for receiving and processing'the images acquired by the caaaara $ to extract vehicle information therefrom. 'Zbe aequ~sition image processing system 42 may form pt~rt of the node 2 of be pocitionad remote from the node and r5 connected to the control unit by a telecammuttications iiae 44 from the acquisition circuit 34. The system 42 comprises a processing station 43 eoaf'tgused ~o automatically extract the required information $om the image. such as licence plats Is 50.
'The acquisition intagc processing system ~2 when impleedentod at a remote cxntrai 30 site, a: shown in Figure 6, include: somrauaicatio~ ouan~ol3ers 55 ~eaed to a public digital telecommunications network 45, and a 1 oomputa server 47 which serves a Local area aetverork (l.c~d) connecting computer3 which implement as acquisition image ACT/ A 1.93/001 ! 5 W~93/t94~1 ~~~~515 _g_ database 49, a iicence piste recognition system 51 and a remote site user interface 53.
T'he communications controllers 55 are provided for each ~e ? which sends itmages to the processing system 42. T'he nodes ~. each ineiude an image buyer and communications controller 57 for storing images ob~ined by the acquisition circuit and communicating with the communications coneroilers 55 of the central image processing system 42 to send the images over the integrated services digital networlt (ISDN) 45 to the central server 47. The eonataunications controller 55 manage the high speed image transfers over the ISDN 45, and handle houxkeeping, error detection grad correction for image transfers between the nodes 2 arid the cxntral ses~ier 4'~. The central server 47 commuaicatcs with the controllers 55 sa the nodes 2 as asextensions of the LAN
maintained by the server 47. Image processing can also lx performed at each of the nodes r, for example, the nodes : may each include a liceasme~ plats recognition system S1 which performs optical character recognition (~) on tile acquired images to e~ctract vehicle information, such as licence plate details.
The vehicle monitoring system, as shown in Figure 7,' comprises a plurality of camera n~s 2 mounted at a number o$ locations 52 to 60 oa dehicie caraiageways. The nodes 2 may be connected by telecommunications lines of the tSDN 45 to communicate with another as~idlor connected to a central coatroi station. 62, so as to compare information collected at each of the avdas :. The control; station 62 includes the acquisition image processing system 42. ?he nodes 2 sad the coactol station 62 are able to monitor a vehicle's progress along the carriageways 16, 64 usi~rg information collected by the nodes 2, which iarludes, in addition to vehicle identifying information, the ante, time and iocation at which as image is acquired. This ix paaticvlarly advantageous as the 35 information eau be used to determine the avetage speed at which a vehicle has travelled between two nodes 2. If the average speed indicate that the vehicle has exceeded the speed limit fn travelling between the codes, then authorities cacn be coata~ed so as to intercept the vehicle. Alternativeiy, the centre! station b2 iss>ses as iafrittgemeat notice to the 'registered owner of the vehicle. 'The station 62 aad/dr the nodes 2 may also captain information on stole$ vehicles sad the authorities are ~wbea a stoics vehicle is detected. Vehicle drivels negotiating long distaa~ would be relucxast to instantaneously exceed the speed limit at chosen iocatiotas, if they are aware that they will i ~1'O 93/ 1944 i r P~T/AL~93~0011 ~
~~j~~l~ ' -10- ' be intercepted or issued with an infringement notice by tsavellirsg between two locations 5. and 54 of two nodes, too quickly. The distance bctwecn the nodes would be relatively large and an allowable time for travel between the nods would be established corresponding to a permitted average speed. The ability to rnoaitor average speeds by the system represents a significant development which can be used to deter excessive speeding by large vehicles, such as tructcs and busts, on major loads, and further scabies deaection of drivers who fail to take scheduled rest stops.
The detection camera 6 produces video herds of 312 and X13 horizontal scan liaaes respectively which are each duplicated to producx a complete 6~5 lice vido frame:. The fields are converted into S12 x 512 pixel 8 bit quaattised digitall images which oecttr st a video field period of 20 ms. The vertical resolution of the ~tection camera 6 is dependent on the ve~ical field line resolution which is appzo~imately 300 elements, digitixd i~oto 512 piurels, for a maximum. distance which the i~amera 6 can view on a horiaontal roadway. The maximum distance D is gives by:
h - titan I,~b? ~ ~1 i t1) where D = distance along road covered by camera view h = height of sera above road D,r = distance of closest position of camera vii along roadway 4f = lenr field of view eagle The !lord of view across the roadway is given by:
~V =~~~
where W ~ held of view across the roadway w = width of the shot ' f ~ lens focal length L ~ object distance from camera The camera 6 includes a 12 mm lane aced as 8.8 mm x 6.6 mm BCD sensor to WO 93119441- ~ ~ J ~ ~ ~ ~ P~'T/r1193/001 t=
1~ _ optimise vehicle image size and maintain a four lane c~verage, 3.5 metres per lane, at the image acquisition points 2Z. An antiblooming and antismear sensor is included to prevent blooming or smearing of an image by vehicle tights. The in~~d filter of the camera permits a infrared wavelengths up to 450 ram, which allows the detection camera 6 to receive the infrared component of vehicle lights, thereby providing more image information to detect and monitor vehicles. The detection camera 6 bas a +~0 dB gain range, and the exposure time is iced at the field period, 20 tns.
The exposurt eontrol of the detention camera 6 controls the intensity of light falling on the camera sensor so as to maintain consistent video s~gZtal quality and obtain a predictable repre:eatation of a vehicle. Acceptable ~cposu~e of the sensor can be maintained through the appropriate match of sensor sensitivity arid control of the intensity or power of the elec~tmtnagaetic wavele:agth failing oa tht ~seasor, as shows with F
retetence to equation 3.
E ac (HA)T (3) 1 s where E = exposure of light on sensor H R isadent e.m.r. power per cma (uradiance) A = area of pixel site in cm=
T s time in seconds that light or e.m.r. falls on tensor The time T light falls on the trigger camera is held const~at at the video field rate of 20 ms. Thin is autficieatiy short to "freeze" tix motion of tha~wehicle in the nlativcly large field of view 12 of a mufti-lane carriageway. A shuttei is not included is the deteaioa camera 6 as elscaonic abutters or short duration ex~trre control produced adveme affects from either image smear or blooming from sunlight reflections or vehicle ZS headlights, as exposure times ware sho:<ened. The incident lighf iaradiaace, H, required to provide su~ciaat expvattre of a sensor pixel is dependent' oa the sensitivity to a particular wavelength of light. Sensor pixels also have a miaia~um light sensitivity to produce a satisfactory signal to noise ratio in the video signal, and a maximum light level before the senSdr pixels bet:4me saturated. The range of tight in~adis~e that can be imaged in a single exposure for the sensor is approximately 100:1. The range of light t WO 93! i 94A a i'CT/ ~ 1.93/001 l ~
w _ ~. e) ~ ~ ~ c~
irradianse which can be presented to the camera 6 duavag a 2~ hour period c~a be varied by as much as 10s:1. Accordingly, the exposure control system litai~ l~
sufficiently to maizltain is within the dynamic taa~,e of the sensor to prevent setzsor saturation from the illumination levels typically ptese:rt durita~ a 24 hour period. 'The exposure control is a f1.8 to f1000 auto iris lens systtm which is designed to provide exposure adjusttraeat based on leas aperture and progressive neutral density fllterin$ of light as the tens aperture decreases. The rate of change of the exposure control, or the rate that H
changes. is restricted as atoviag vehiefes are located by di~ereneiatg images obtained by the camera 6 tom a slowly changing background image, as d~cxibed hereinafter.
'The rate of change is restricted to ensure chat>ga in exposure of the ~ season are sot mistakes for changes in the background image, which would adver~ly affect detection and nvnitoring of vehicles. The auto iris reaction time is set to t~atc~ the ratio at which background images are subtracted from the current ice. 'f~e~~iow rate of cbaage also prevents the leis responding too fast to transient c6aages in lig~, for example, reflected off roofs of vehicles as they peas close to the camera 6. The rate of change is restricted to 10 seconds for a halvir>s or doubling of light irradiance H.
i, The exposure comrol system e»taaes that traasirnt e~cr~sly bri~6t reflections yr headlights do not saturate the sensor pixels by limiting the exposiue on the season to keep 30 is below the sensor's saturation level for the peavk intensity of light received in the field of view lr. The peak video level obtained fmm the camera 6 is; monitored. as discussed hereinafter, sad usexi as a basis for controlling the setting of the diaphragm ~f the iris.
i The sensor sensitivity is selected in order to psodttce video sills which allow 35 the subtraction of the background for vehicles not using headlights during dusk and dawn illumination levels. The sensor is also respot~ive to near 6afra-red light to maximise the sisal from large vehicle side and perimeter lights, yet the respr~ase must be still below a threshold where blooming may occur from vehiclt headlights. ~ 'I~e lei of the camera 6 can be controlled fully to provide sufficient exposure for the ae~sor for vehicles without 30 headlights during the dav~ta sad dusk periods. The maximum lenns aperture is held at f4 for a lurais>aace value of about 10 r~llmi reflecting ~ the ~ , y. ice the c~tiageway luminance level fall below ap iy ~~'o' off this level, vehicle t~.T/Al'93/00. s~
w0 93/ t 944 z ~ 3. ~ ~ ~ ~, 5 segmentation, as discussed hereinafter, is based on vehicle headla~,hts.
Cotnrol si8nals representative of the illumination levels are derived from an illumination hisiogrartt of video signet levels for the pixels, described herei»after.
The control unit 10 of a camera nods 2, as shown .in Figure 8, includes a Motorola 68030 CPU 64 and a detection and trigger sub-system b6 connected to receive images frown the detection camera b, and as acquisition sub-system b8 eoctaseted to receive images from the acquisition camera 8. The sub-systems 66 and 68 include a number of Dataeube pipelined pixel rate video processing circuit boards which are controlled by the C'PU 64. The boards and the CPU 64 are mounted on and i~atcrluoiked by a VM>r (Veaa Module Europe) bus. The CPU 64 and the boards of the sub-systeaas 66 and 68 run a software operrrttaag system knowta as VxWor~s, which is a real time mufti-tasking system. The detection sub-system 66, the fPU 64 and controlling software form the detection circuit 30, and the acquisition sub-~ysttm 68, the and the controlling software form the acquisition cir~it 34. ~ The image buffer and communications controller 57 caw be connected to the acquisition circuit to provide access to she ISD~I 4s.
~i i The detection sub-system 66 the 512 x 512 piatel images of each video field obtained by the detection camera 6 and is dcsigaed to achi~,we low latency between change: in the field of view 12, by using pipelined processing off' the image data with rto intermediate storage. The data rate through tlse video data paths of the pipeline, known as MA7~US, is 10 million pixels per second. 'Processing the vadeo fields individtsaliy, as two consecutive frtmes of half vertical resolution, achieves a ply rate of 50 HZ and :.5 eliminates the deiaterIacing latency required for full frame pro i essing.
The deeectioa sub-sysseas b6 includes a video digitiser ,beard 74 which reoeivts the Eeids output via the detection cannery 6 and converts them into the 512 x 512 pixel representation. The digitiser board 74 is a Dataaxtbe Digima~ board wad produus a greyscale image representation with each pixel having a value within the 2's complement positive range of 0 to 127~ 0 representing black and 127 ra:presentisg white.
'I~ac 313 x 512 pixels era able to produce a live image display as shohvn in Figure 9. The ~'O 93/t9d4;, ~ ~~ ~ ~ ~ ~ ~ I PCC/AL'93/0011;:
_ 1~ _ image produced by the digitiser board 74 is input t~ a background diffezeneer board 76 which, as shown in Figurc 10, subtras;ts a background image, as shown io Figure 11, from the cutreut of rive image to produce a pretiminary diffcrencc raga, shown in Figure 12.
'il~e difference image contprises a grey lever of representation of the moving objects within the ejeld of view 12. ~y virtue of the imaage subtraction the pixci image raage for the difference image extends from -128 to 127. The background differences board 76 is a Datacube MaxSP board.
The background image represents the static backgaound 'viewed by the d~te~ion camera b and is stored in one of two framGStores 71 of a background image score board 70, being a Datacube Fra~mestore board. The ~~ ~~ is coutisualty updated by a background update board 72. which is aaotber l~atacube ISP beard that ensuecs i one of the framestores 71 holds an image correctly representative. of the static d within the ~sld of view 12 of the deteexion camera 6. 'The updatb board 72 then receaves the curnnt background image from one of the faamestorec 71b ~amd is combifled with a filtered form of the preliminary difference image to produce a; new ad image which is outputted by the update board 72 t~ the other framestriae 7ia. The cxmtrotling software thcat switches to the other framestore 71a for submission of the bac~eour~d image to the differettcer board 76, ~ ettsuses the next updated '~tnage is submitted to the ?0 first framestore 71b. 'The background update board i:,lters thtprelim6nary difference image is accordance with a filter characteristic 73, as shown in Figure I0, which is brad in RAI~t aad perfoans a limiting function vn the grey level p3xe9g of the pnlimiaary difference image so as to restrict them between a programmable range, for example -3 and +2 pixel ran;e. The timitins functicm te~accs the ion made to the current 35 background image when' it is combined with the dif~e~ 'u, after having been subject to a delay 74 to allow for the time taken to apply the liittititag filter function 73.
The limiting fuaetion ensue the correction made to the baokgr~und image per frame ~
only slight so that traastent dtffere:xes, such as those produ~d~ by moving ~b~eocsy are not allowed to signafirantly alter the stored background inaageheld in the image store 30 board 7Q. "f be shape of the altar function 73 that greet level differences added to the background image are to a level t for all ' ~ levels ~t and -t for all difference levels <-t, where t is s low tjtreshold such as 2< The state of the ii PCT/ A 1,'93/0011 - L~ -bacleground update board 72 can also be changed to disable update of t$te background image. The rate of change in the background image is xt so as to be faster theta the rate of change of scenic exposure due to variation in the lens aperture of the detection camera 6. The rate change governed by the limiting function is impoata>9t because if tht rate is too slow fighting changes tin produce incorrect difference images, and if the rate is too fast then moving objects may appear in the background image as a blur.
The preliminary difference image produced by the backgtouad differencer board 76 is outpusted to a third Datacube MaxSP board, a shadow eliaxination board 77. The shadows produced by vehielas which appear in the di~ereace ieaage, shown in Figure ~2, pose a significant problem for the images processed to determiated the type of vehicle.
T7te shadows can mistakenly represent the vehicle as being larger,; than its actual site, and if a discrimination is being made between the large vehicles, such as trucks and buses, and small vehicles, such as cars and motorcycles, then the shadow cast by a cat cast lead to it being classified as a large vehicle. Therefore the shadow elimination board 77 is employed to eliminate all grey levels is the difference imaged which caould represent shadows. This is done by defining a grey level window range 79 is RAM, a shown in Figure 10, wheml~y ~ preliminary difference image is proceed so as to set to zero all pixels having a gaey level within the window 79. The result is; then u~d to mask the preliminary difference image so that the elimiaatioa board 77 outputs a shadow 5ltered difference image having ail of the pixels with grey levels withi~a the window range 79 removed. Figure I3 illustrates a Iive image with all of the pixels having a grey level within the range of the window 79 shown as given. T'he range defined by the window 79 is adjusted depending on the light conditions within the ~ald of view 12 of the ~5 detection camera 6, as discussed hereinafter.
The shadow filtered difference image is inputted to a threshold and mediaa 5lter board 78, which is a Dacacube Snap board. The f iter board 78 ~petforms b inary image processing on the difference image so as to convert the grey level representation of the moving objes~ to a binary repraentadoa, which oorrcsponds' . to white or black, for further pr~oeessing by the deteaioa sub-system 66. 3be 5lter board 78 tries a threshold value to convert all of the pixels, with grey level values within t~ range -i28 to *1Z7.

w~ ~3W a4t_ pCWA~,~93ioot t:

to pixels having values of either 0 or ?55. Accordingly, the faatal difference image produced by the filter board 78, when viewed by a real time display, shows the m~vi~g objects within the field of view 12 as a collection of white pixel blobs, as illustrated in Figure 14. The blobs may correspond to parts of moving vehicles which reflect sunlight grad, at night, may correspond to light produced by a vehicle's external lights. Noise regions of one or more pixels in sin ate eliminated by the board 78 which performs binary median filtering on 3 by 3 pixel neighbout5.
The light conditions within the field of view 12 of the detection eamera 6 are determined with reference to a histogram 150, as shown is Figural 15, of pixel grey levels produced by the CPU 64. Tbc CPU 64 processes a window o~ the stored background image which is appeo~dmately 300 x 400 pixels every 10 seconds. The CPU 64 calculates the number of pixels its the window having each grey level and sabulate5 the results as the histogram 150, with the number of pixels on the vertical axis 152 and the grey level values on the horizontal axis 154. The lusto150 can be displayed to provide a real time representation of the light within the field of view 12.
From the grey level value which represents the position of the median 136; one of three lighting conditions, day, dusk, or ttigbt, can be instantaneously detetmin~ed. Dawn is considered to be the same lighting a9ndition as dusk. The positiooa of thd peak i55, median 156 and the minimum 15g art used to determine the range of the 79 used its the shadow elimination board 77. For daytime conditions, the ,shadow window 79 is determined as being from the values a.peak to (peak + mediea)~, where a is typically 0.5. For dusk conditions, the shadow window 79 is from minimu~a to (peak +
~edian)/2.
Shadow pixels of a~urse, do not need to be eliminated duri~ night conditions.
r5 Estimation of the shadow pixel range is as approximate techeiqu~ which is aided if areas of Permanent shadow are in the i9eld of view 12, such as cast fby trees or an overpass bridge.
't i The segmented im~ss produced by the fitter board 78 ale submitted to ~ Area Perimeter Aareletator (APA) board 80, which is an APA 512 boaitd produced by Atlantek Micsosystetns, of Adelaide Australia, d~i~ed to acxaelermte ' the pressing of axes parameters of objects in a video scene, The hosed 80 v~ith concrollin=
software ~'O 93/ 19~d1 ~CT/A L'93/pOt t o ~~.Jj~~S

to perform analysis of the white piacel blobs within a 3~ x 4p0 pixel window corresponding to the window on which the histogram 150 is produced. The APA
board Rti and the software perfor>?t a classification and feature extraction process iat teal tints on the blobs so as to facSlitate the fomaation of clustezs of blobs which correspond to a moving vehicle. The APA board 8t? computes features of the white pixel blobs and the ftatures are used by the clustering software to determine, on the basis of rules aztd classification code, whether the blobs can be combined to form a cluster. Unce formed, the size of a cluster indicates whether it corresponds to a large vehicle, such as a truck or bug, Or a small vehacl~, such as a csr. Labelling software is used t0 monitor movement of clustezs over successive fields so as to detetaaine ~srhich clusters are to be assigned a unique label and which clusters are to share a label, a~ they are considered to relate to the same vehicle.
Different considerations apply in respect to whether the c~iageway 15 is being viewed by the detection camera 6 at night or during the day, and the :ales and cia$sifications used are adjusted, on the basis of the data provided by the histogram 150, to account for night conditions, rain and inctement weather, w~sioh result in a moving vehicle producing different corresponding pixel blobs. For rxsanple, the :ales and classification code needs to be adjusted to account for refleetidn produced by vehicle '?0 lights on the road during night conditions.
Once a cluster his been formed, its movement is mot~tored to detern~ine its insiaataneous speed and its position with rcspeci to a point on th~ edge of the road using KaAmaa filter techniques. Corrections are made for per~ctive as the cluster moves 35 towards the cameras b and 8, The information obtained from mibnitoring the movement of the cluster is used by this C'CpU 64 to predict when the cluster will cater the field of view 20 of the acquisition camera 8, aid in particular when a vehicle scathes a position '-d which an image of the vehicle is t0 acquired. The prrrdieted liras estimate is updated for overt' field generated by the detection samara 6, 50 tunes peg second, The predicted 30 time is continually corrected as the Cpt3 b4 ~onito~ m~verne~t of a cluster until it is satisfied the cluster will enter the 5eid off view within 10 to 20 ~uts. A CPU
64 predicts the time by specifying the number of scan li~aes wlti~ net t~ ~~ sped by the camera ~O 93/19441 PCT/At,'9310011c ~13~~1J

6 before the clusters within the field of ~~iow ~0.
Performance of the control utsit 10 can be monitored and'controiled by peripheral devices, such as a printer 94 for error and event lagging, a real came seaeus display 98, and a control workstation 100, which may all be cormected to the CPU 64 and the boards of the control unit 10 directly or by a local area network 102. A display of the rest time I
status display 98 is illustrated in Figtue 16 w~eJt is the live imago produced by the digitiser board y4 superimposed with cluster markings and other data. The histogr~ 150 is displayed at the left of the screen and the box around the vehicles are cl,~t~ which have been formed. The label number for each duster is sbowm me the lower right hand i comer of oath cluster, and the estimated speed of the vehicle, obt~iaed by monitoring the cluster, is displayed directly below the label cumber. TI~ large box around the vehicles represents the processing window, on which the clustering, ~ labelling and freckles sofsware operate, in addition to the Iaistogram software. The I'>~ across the window is an acquisition line which cxlTesponds to the position 22 at which,! high resolution images are to be acquired by the acquisition camera 8. A diagnostic glSaphics botttd 82, which is a ~atacubc Maxgraph board, is used to queue sad configure graphic irna=es for the real i time status display 98.
i The image processing performed by the ~7 64 and the APA hosed 80 for vehicle classification is baadled by feature extraction, clustatan~. labelling and erackirrg software. The operation of the software a largely controlled Iby parameter variables, which msy bo altered via an interactive shell of the software or byroarrote procedure calls ~ a graphical interactive command tool runaiag under Xv~hado~rs oa the control ZS workshtion 140.
The AF'A hood 80 roduads the binary image pixels ~to a stream of feature vectors representing the blobs, or regions, in the imates. Only; a small sub-set of the features which can be computed by the APA are requared, being! the area, perimeter and 30 bounding box for each blob, or region. A region is tepresermd ,by raw data of ib bytes and for a field of view I2 which includes 20 r~egio~, ehe dam ate a I6 kbytesls which is less then 0.2~r of the data rate for binary images, and is ~bte for software i WO 93Jt9441- ~ ~ ~ ~ ~ ~ ~ p~lpD.'93/00115 P~~sin~ by the CPti 64.
The raw seed parameters are read from the ,~pe~ h~~,are by the A,Qt~Task 170.
as shown in Figure 17. A timC stamp is givers to each blob, nerd some initial screening is performed, why regions such as "black triangles" described hereinafter, are located and removed. Time stamping, inter alia, allows any latency in the system to be mcasyred and compensated for. The seeds which ~~~nd to wee blobs within certain area constraints are passed via a VxVWorks message pipe to the aeedTaSk 17:. The setdTask unpacks the raw seed parameters, or structures, and perftnms classification of regions based on each regions height to width ratio, "circularity", (urea sad "coverage". as described hereinafter. Umvanted regions such as headlight end road reflections are renooved and then each classified region is passed via aaot6er message pipe to the clusterTask 174. i i 1s The clustering task is divided fnco five ~bs~tions 1'16, region classification.
region extension, clustering, region unextenaion and cluster el~tsification.
One the regions have been clustered into clusters wlueb have been classified as corrtspondia~; to separate vehicles, the coordinates of the eluste~ are gassed onto a label task 178, once again by a message pipe. The label cask monitors each ch~stdr over a given period of 30 time and if a cluster appears in rnugltly the same place as did ~ cluster from a previous video flame, then the label task considers them to be the same ~,~~ter. In this case, the new cluster inherits t,'be label from ehe previous cluster.
ache if no match can be made, the new cittster is given a new label. The elustea's label, l: they panned via .m dies, along with its allege pipe to a trajotxory 180. The trajectory task I80 determines the tiara to tr9Bger the acquisition c~aaexa 8 for cluster of a selected cFass, such as large vehicles. The put cluster box task 182, move cluster box task 184, put label task 186, remove label task 188 and the histogna~ t~k 190 are tai used to generate graphics overlaid on the video image, as shown in ir'iguure 16, for diagnostic purposes, ?he blob shape analysis performed by t~ ,~sATssk 174 sad seedlask I72 is not extensive during daytime s=ensation, as all blobs are ooa~idered valid.
However, i ~'O 93119491 ~° ~, ~ ,~ ~ ~ ~ 1 ~~I r~ lr'93/p~p 1 f ~
-r0-during dusk and night time segate>ztation, blobs tine occur due to vehicle headlight reflection, and if these blobs are clustered in with tnae vehicle blobs, then the front-of-vehicle coordinates, which are taken from the bottom of the cluster box, wilt bs incorrect. Itt order to correctly locate each cluster box at the front of each vehicle, blobs which are recognised as being due to headlight reflections are identified and removed fxfore blobs are clustered. ether problem blobs are those which cc,~gsp~d to road lane markets. These appear when the mount for the detection ca~anera 6 shakes.
During camera shake, the iacon~ing video israage ao longer precisely coarespoatds to the stored static back~ound image, and therefore the result frorai the backgcntand image t0 subtraction is that the road tnukers appear to have moved. Ate, the blobs that result from camera shake aae identi~cd snd filtered out before cdusteri~ commes. A
further problem is "black triangles". The APA board 80 posse3ses a hardware fault which causes the polarity of blobs to be specified iacorzectly. If a black region finishes at the right head side of the pixel processing window, it can be ~ ~ tly labelled as a white IS region by the APA board 80. These white regions can rhea' become eandidates for clustering unless filtered out by the seedl"ask 19Z. Typically, ,~whm a lane marker 190 appears on the right side of the pixel processing window 19Z, ~ s~~ Figure 18, it pmduees a black triangular blob 194, a "bt~~e", which is iaadvtrtently represented by white pixels, in the top right heard corner. 'I9~ triangular blob 1~4 is ?0 identified and removed. A canvenient side effect of the polarity fault is that the toad lane line~matker 1!~0, which usually mast be identified and tetnoved by other shape characteristics, is la~'belled by the APA beard gp ~ blue, ~ ~ therefore automatically filtered nut. ~ .
Regions are classified into one of the following types;
(i) Headlight refleaioas;
(iij Road artefacts; such as road lane markers, which ~e to carnets i shake, (iiij Laghts; attd (iv) Other; dutilag daytime segraeatatioa staost of ~he regions that are not classified as road artefacts are classified "other".

''~'O 93/19441 ~ .~ e9 :v ;) ~ e7 PCTI~1,~3/0011s I
During day and dusk conditions, illumiaaated headlig' do not appear segmented from other segmented parts of a moving vehicle, and so ~. :. ~ts are trot classified. At night, however, cfrcular regions are elassified as cithcr "headli~t" or "stnallalight".
depending on the area and position within the field of view lm: Distant headlight pairs which are typically segmented tom the baekgtound image as a'siagle joined region, are classified as "joined headlights". To obtain coma initial ~ Blusters, distant joined headlights need to be distinguishod horn the small perimeter lights of large vehicles.
The main shape measure that is used duaing dusk and bight time processing is "circularity". This is a measure which co~iskrs bow dose each blob is to the shape of the circle by comparing the blob's area to its perimeter. Ia tht case of a circle:
i ~a = errs (4D
= 2Rr (~
The radius team eaa be elimiaatexi, since it is only relevant for circles, by squaring the perimeter equation sad taking the quotient of the two terms. For a drcle, this produces a constant:
I
><r= = 1 (paimeca~~ (2aa~ 4rt (6) To make a circularity measurement equal to 1 for a cirCls, equatioa 6 is simply multiplied by the iaverse of the constant. Tlvs provides a circularity measure which can be sppiiexi to blobs whereby a cirarlar blob will have a m ~ea~urement value of 1, as followvs:
4~ ' i.o ~ c~>
i For a square blob of unit area, Area = 1. Perimeter = 4, ~ the circularity measurers ZO is as follows:

WO93/194~t1 I P(:T/A~.'93/mpll~
Circularity = 4~ _ 'e = p.7g5 (g) (4)=
i-For an c~uilatcral triangle with sides of unit length, Atca = X314, lyeaimeter s 3.
the circuiariey treasures is as follows;
= 0.6 (9) A further measurement employed, that is particularly u~ful in detecting road laad/line markings, is "coverage". Coverase is the measured ratio between the arcs of s a blob to the area of ice bounding box. The bounding box 200, as shown is Figure I9, is aligned with the ApA board coordinate axes, which arc the sates of the APA
processing window. The APA axes ate not ne~ssariiy aligned with the anajor axis of the blob itself.
For inssaace, a rectangular blob X02 which is aligned with she APA coordinate axes would have a high coverage value, whereat a rectangul,~ blob ZOt which is not aligned with the axes nsay have a medium coverage value. A concave ape 20b would produced a medium coverage value, sad a line 208, diagonal to the A19A eaoordinate axis r01 would produce a low coverage value. Road lace markings can be simply detected betwause they have a low coverage value. If the lane markings are sat diagqnrtl, but vertical, then the measure is insufficient and is such cases a measure of the ratio of the blob's major axis length to it's minor axis length can be used instead. ' During night time segmentation the coagulated blobs ~ of joined headlights are identified by their height to width ratio as they Mead to be twice the expected area of one headlight. Joined headlights need to be detected sd that a headlight count maintained for 30 each cluster is correct. i Headlight reflections appear as large elongated blob:, acid are detected initially on the basis of their size and chara~cristie shape. The blobs art; fed as relating td headlight reflections by extending the blobs ver~ric~lly to deter~ine whether try extend r5 from a headlight region.
As the vehicle moaatoriag system is capable of ~riairous automatic operative.
I

~'O 93/19441 ~ ~ ~ ~ ~ ~ ~ PCT/AC,'93/OOt 1~
- r3 clustering of regions takes into account different lighting conditions. 'the technique of static background subtraction. described previously, segments moving objects froth the .
~~ideo image obtained by the detection oarncra 6, but the regions that result from the scgrmentation process depend on the ambient lighting conditions at the time of day.
Dining daytime scgmeniatiaa> large regions typically result, whereas during night time only headlights and the smaller sidelights on trucks are segaieated. lauring dusk, lit headlights do not appear segmented from the other visible parts of moving vehicles, however, reflections upon the surface of the road caused by the headlights need to be removed, as discussed above.
The clustering process operates on the segmented regions or blobs sad each vehicle is typically segmented into several separate regions, as ~hovYn is Figure 12. For instance, a car will often appear split by its wi»dscxeen into a roof-region and a bonnet-region. Large vehicles typically segment into more regions. The cluster task groups these regions into "logical vehicles" so that they can be backed.
Distant vehicles tend to be segmented together into one region due to vehicle, oociusioa at the image horizon. Whale the segtaeneed regions at this distance car ba~ tracked, they cannot be reliably clustered into separate vshiclcs. Emphasis is planed on cbrrcctly clustering lower regions that art: closer to the acquisition line 22, and con:equeatly the clustering process scans from lower regions to higher regions in each image.
i I~ two vehicles are sagrneated into the,same region, tha~ they will be clustered together. The cluster task does not separate vehicles that have been segmented together into a single region. The coordinates of each cluster are seat to label task 178 which ?5 matches tend separates clusters over consecutive video fields. The cluster task and the label task classify clusters on the basis of classi~cstioa data. The coordinates passed to the trajectory task 180 Correspond to as estimation as to the fr ,~t of the vehicle, at the road surface level. Cluster information oa all vehicles is provid~d to the trajectory task, which tracks the clusters and selects far which vehicles itaagcsue to be acquired.
clustering is achieved as a middle paint batw~a "over Clustering,. sad "under clustering". dot the over clustering extreme, all rcgioat sae clustered into one ~'O 93/19441 ~ ~ ~ ~' ~ ~ ~ PCT~Atr'9310~11:~
single cluster and then only the lowest vehicle in the cluster is tracked.
'I°his is because the lowest point of each cluster is passed by the label task t~ the trajectory task. The classification of the cluster, which is based on its height and width will be iacotrect. At the under ciusteria~~ extreme, if no regions are clustered together, that is each region obtains its own unique cluster and label, then the trajectory task is over-burdened in an attempt to track every region, vehieie classification will fail is a taun~ber of instances, and images will be inadvertently acquired sand missed. For the purposes of vehicle image acquisition, it is better to mistake a vehicle-roof for a vehieie-fi~nt and begin to track it than it is to mistake a vehicle-front for vehicle-roof and so, by addiacg it to the beak of another cluster, not track the vehicle-front, Therefore the cluster task has been written to use an optimal middle point which ties on the side of under clustering rather than over clustering. i The cluster task performs clustering essentially by extending the boundary of each segmented region by a certain ataouttt, and then joining asiy region$ that overlap.
Regions that overlap are "clustered". The cluster task, howeverl determines correctly the amount of cutension which should be applied to each region. I?uring daytime segmentation, very little region extension is required, yet ~duting night time, the segmentation process produces small sparse region that require large amounts of ?0 extension in order to achieve overlap.
i An important aspect is the construction of a cluster is these the bottom region of each cluster should be the front of a vehicle. Invalid rcgxons~ such as regions due to headlight reflections, must not tx clustered, and are thus not extended. After every valid ~5 vehicle region is the image is extead~ by a oertaan aaaount. the clustering process begins with the lowest region is the image, The lowest is considered' first which is the region most likely to cause triggering of the acquisition camera g. ~' The coordinates of the lowest region are used to initaalix a Bluster strueture.
30 'rhea al! exteacdcd regions above the initial region are tied fdr overlap.
If any region does not overlap with the coordinates of the clctster, then t~ cluster coosdinatcs are updated to include the region and the region is a~arkod as sh~tertd.
V6~hsnever a new wc~ ~~~,~~ot ~ ~. ~ ~' S 1 ~ , ~cr~,~LV3~oozt:
~~s~
region is added to a cluster, all remaining unclustered regions bccorne possible cluster candidates again. Thus the list of regions is traversed agasa fiom the bottotit of the image. Although the regions in the list which have already beets snacked as cltastered cats be skipped, this is considered sub«-optimal. t?nce the entire rtst of regions have beet, traversed without any overlap detected, the next cluster is begun with the lowtst remaining region. The clustering pcontinues in this saaaner until no regions are left unciustcred. The list of clttstets are then unextended and passed to the label task.
1rt perfvrrning region extension, regions are extended by a variable aa~otsat in the vertical direction, but extended by a standard amount in the horizontal direction, with reference to the APA coordinate axis. Horizontal extessioti is unnecessary during daytime segmentation, as a vehicle blobs tend to be coasecte~ aaoss the fuQ
width of the vehicle. It is in the vertical disectioa that blobs due to ~tlae same vehicle appear disc~nected. For example, two blobs that typis~liy re t ~ car might be due to its bonnet and its roof. 'Ihcss two blobs wall streteh over the full dvidth of the vehicle, and appear one above the other. Furthermore, so long as ono blob ~or each vehicle st~ches the full width, the cluster croordiaates will be wide enough to '~corpocate stay blobs that might otherwise need horizontal extension to be clustered tog,at~ter. The full width blob provides the extension. with reference to the example iltusuated in Figure 20, tl~ region 30 r10 becomes added to the region 212 on the right, from which;the cluster 214 is begun, only because the full width region 21b above was added to tb~ region '13 to form the cluster 214. It the repon list wag not researched from t ~be beginning of the list I
continuously, the overlap of she previously tested region 2I0 wo!uid oot have been found.
It is for this reason that the clustering task, as di~ussrd above, r9es~oasiders all uaclustered '?5 aegioas after addia~g a region.
The cluster task is able to perform one of ~o eeatttens~ioa methods. 'I~e fist method takes the vertical or Y coordinate of the region as as input to a loak~up table that speeiSes the amount of extension to be applied. 'IVs amount oi~tbe extension, and hence 30 the degree of cltastering, is they modified according to Gghtiag itions.
.~s the outside light level de~asss, sad regions reduce is sizes, the anaouat of extension applied to regions can be gradually inasascd. Fut2hermote, the ~'ve is the image can be I

WO 93119441 PCT/A193/0011:
~~j~~$~
_ 26 -compensated for by adjusting the values stored in the iook~up t2~ble accordiztgly, i.e.
distant regions high in the canasta irz~age earl be extended less than near regions which are low in the image. An example of the extension values stored in the look-up table is illustrated by the graph 2I8 shown in Figure 21 of extensioa~ar~ount v. Y
coordinates.
S All extension amounts arc pxovidcd in pixel numbers. The ixcond extensioat method extends each region by an amount proportional to its width. 'IVs method is largely based on an observation of the shapes of legions obtained during day;ime segmentation. Small regions, which arc typically far away, are n~inimaily extended, large vehicle body regions.
which are typically close, square and occur one per vehicle, axe~rninimally extended, and wide short regions, which are often vehicle fxon~, ace greatly txtended.
Essendaliy, as illustrated in Figure 22, this results is every region bai~ary 220 and 2',w', being approximately square. In Figute 22, the boe~ies 220 and 22$ of both regions 224 and 2~6 have been extended vetticaily to equal at least their width. ~'Thcrefore the wide short region 224 has been extended a great deal more ihaa the large s~uara region 226. Region IS 224 would tx a vehicle front portion disposed under the vehicle body region '. a6.
Therefore, the two regions 224 and 226 can be matched without too much extension. If i the large region 226 is ovtr extended, then it may overlap with a succeeding vehicle fi~ant. In tht preferred embodiment, this atethod is only t ~ ploycd during daytime segmentation as nighi time processing requires a large amount of region extension, 30 although it is envisaged the extetASion factor used in the extension calculation can be enlarged foe night time use.
During night time clustering all of the regions to be elus~ered arc essentially small circles, asd s truck cluster, for example, is aonsaucted by eoa~idering the possibility of 25 whether each light could feasibly fit into a stored wck template. For the first region in a cluster, to fit within the template, there is a maximum distance of light separation which cannot be cxaeeded. :i Overlap of regions is detected by competing the coordinates of regions and 30 clusters, wherein the top_left (xt,Y') and bottom-right (xyY~ coordinates for both regions and clusters are known. For the image pleas coordinates, x ' from left to right and y increases from top to bottom. Considering first the boaixontal, x coordinate.
i w0 93/t944t PCT/,~L'93/OOt 1~
overlap for the regions R" Ra, R" R" Rs and R6 illustrated is Figure ?3 the test for overlap with the cluster C~ i~:
R~(xt) < C~(x~ (10) Cat'tt) ' Ra(~ (11) If both of the two equations a~tte true, then there is overlap is the horiaontal direction. Therefore, horizontal overlap is tnae for R~, ltr, R, and ~ but region R= fails 5 the test as equation 10 is not true and region R6 fails the scat a equation 8 is not true.
A similar test is performed is the vertical direciioa as follows:
I~(yz) a C~(yi) (i2) There is no need to perform the complimsatary test for R,(y~ because the regions are outputted from the APA board 80 is order tom top to bottorm and as the cluster task processes all regions in a list fiom the bottom up. the complimentary test, C,(Y~' R,(Y~), is unnecessary as it will always be true.
Clustering during day ti~htir~' conditions is based oa the overlap test discussed above, yet during dusk and night oonditioas clusterir~ involves soasideratioa of additional soles, primarily due to she increased ambiguity and greater separation between regions of the same vehicle. Certain regions should also txvet be clustered, such as headlight reflections and aoiae from baeitgmund image areas due to vibration of the detection camera 6 discussed previously. Clustetzag therefore also involves consideration of a series of rules based oa the various re&ioa cl~sificxtiona 'dzs~sed previously. The rules include: ;
30 (i) An e~cteaded region must spatially overlap a cluster t~ be added to that cluster.
(ii) If a region overlaps more that one cluster, then it' is added to the lowest cluster.
(iii) A region to be clustered caaaat already .have bees added to the cluster.
(iv) A "joined headlights" rewoaa ratumt be added to an existing cluster.
Retiomms of this type cps only initiate a cluster.

F°C1'I,~L'93/001 IS

_2g_ (v) f?nly a predetermined number of "headlight" regions can be added to a cluster, the predetermined number being a system parameter which can ~
adjusted froth the user interface.
(vi) As many "other" and "sasall light" regions as is spatially allowed teas be added to a clustet.
(v ii) A region which touches or includes part of the cop of the processing window can initiate a cluster but cannot be added to a cluster.
(viii) A further "headlight" region to be added to a cluster must be horiaontally aligned with another "headlight" region in that cluster, which is determined an the basis of the difference between the regions lower y ordinates.
(ix) "Reflection" and "toad artefact" rtgioas are not added to any cluster.
For monitoring a roadway, clusters are classi~od into one of three claosses:
car, ute (a small fiat-bed utility truck) or truck. Therefore all large vehicles, such as buses and artiarlated vehicles, are classified as a crock. Cluster ciass~fi~tion is based on the height and width of each cluster box, sand the number of lights within the cluster dur9ag night conditions. The height sad width data for each classification is modified via procedure calls to the histogram tart 190 a~ the lighting conditions change faom day to dusk and eo night, ate. The cluster width is as important as th i cluster height because, ?0 for example, a large four wheel drive vehicle towing a trailer might product a cluster which exceeds ttu truck height throshoid but is unlikely to be atwide as a uvck or bus.
A histogram of cluster heights and widths of motor vehicles iachides distinct peaks which correspond to vanious vehicle class, and . is used to set the stored classification i thresholds automatically, The height and width histogram is in the display of ~5 Ffgure 16, For example, a cluster is classified as a truck ~f' one of the following co:Dditions is true:
(i) The cluster height sad width excxed the truck threshold.
(ii) The lighting condition is night and the cluster .exceeds the truck width threshold.
~0 (iii) The lighting condition is night sad the number of fail lights in the cluster exceeds the small light thick threshold.
(iv) The chsster height is within a predc r~oge of the ttu~k height i r W~ 93/19441 /A4'93/ti011c - ?9 r threshold and the number of small light regions in the ~luster~ exceeds the truck small light threshold.
~,s ambient lighting drops, the six of the track c ~ stars are reduced, and S consequently the height and width thresholds decxease, dtpgndin~ on the lighting conditions, as determined by the histogram task 190. The classification for eaeh Bluster is scored in a clustered data strut~turc, together with the cluster's coordinates and time stamp. The clustered data is then passed to the label task 1713.
The label task 1'18 assigns a label to each unique clusreri and tracks clusters over time by matching an array of previously seen clustcas to each subsequent video field of clusters. If a cluster appears in roughly the same place as a cluster from a previotes field, then the label task 178 consideas them to be the sane cltasmr.~ Where a mateb cats be made, the new cluster inherits the ttaique label of the previously s~en duster. If a cluster cannot be matched, then a txw label is created for that olusser. I Clusters may disappear for a few fields, and it is an objective of the label task 178 to ~iierntine whether a cluster is indeed new or whether It has just appeared again after a parsed of absence.
i The matching of clusters is bases: on location. C,'h~te~, size can be used as as emra match parameter but the current location heuristic has been found suffieacat. It oar be assumed the clusters will not move very far from their position in the previous held, and if a cluster moves so far that its boundary coordinates is the present frame do not overlap with its boundary ooordinatea from the previous frame, then the previous label will not lx traasferted. Cttuters can split and join, both vereie~liy and horizontally, as ~5 they an tracked from held to field. Two labelling methods heave been developed, with the second being the preferred method which is presently used.
i The first labelling method involves two seoipxo~l tests whiob are used to determine whether a new cluster should inherit an old clusters label.
°t'hha fiast test is to determine whether the carne of a ~vv cluster 230 lien witlltin the boundary of any clusters 232 and 234, as shows m 1 igurc 24, on a list of pf~evidusly aeon elttstets, oall~
the label list. For the ehrster 230, the tsst fails, but for the ~v citasters ~6 and ?3$

~'~'O 93/ 194A 1 p~ T/ A L' 93~ 0011 ~1j~~1~
_3p_ their centres fall within the older cluster 240 so the lowest new cluster 238 inherits the tabet of the old cluster 240, and the upper new cluster ?36 is assi~ed a new label. The second test, whieh is executed when the fizst test fails, determines whether the centres of any of the clusters on the label List lie within the boundaries ~f the clusters from the S current video field. Therefore as the centres of the old clusters 232 and 234 fall within the boundaries of the new cluster 230, a match is detected, sled the new cluster 230 inherits the label of the lower old Bluster 234. Applying the .second cast to the new clusters 236 and 238 results in failure as the centre of the old cluster 240 does not Iie within any of the sew ciusters 236 and 238, and therefore applying this test to these clusters would result in the new clusters 236 and 238 both being assigened sew labels.
The second labelling method is based ors the dustetilsg overlap technique described previously. Essentially, the bounding box of tech duster from the current field is tested for an overlap with dustem in the cluster list. The duster list is search from bosom to top, in a similar mauaer to the search method described for detecting overlapping regions. In this way, if two clusters merje into a single duster, then the 5rsa overlap found will be an overlap with the lower duster. Once a match is found, the search is terminated, and the label which is tasnsferred is marked as applied to a new cluster. Therefore a label censor be transferred twice within one search of a new video frame. 'ihe second method is preferred as it requires half the nu~nbcr of tests as the first method, and a cluster can move further between successive frrlmes yet still inherit its label, Ia the fsnt method, where ceatres are matched to the boundaries, the maxilmum displseament allowed between fields is half the width (or height) ~of the clusters, wherta~s in the second method, where boundaries are checked for overlap, the maximum displacement is the entire width (or height) of the diner. '17>e~fore the second method allows a cluster to move ewice the distance of the first method:
As clusters travel successive fields in tune, they tend to split or join, and if a cluster splits, then the lalxl is taaasferrred eo the lower of the two dusters, and the upper cluster, which would typically be another vehide behiad t~ idwcr duster, is provided with a new label. Alternatively, if two dustet3 join, then the old lower duster's label is transfersed to the sew combiaed duster, and the other duster's label is allowed to expire.

~~ 93119441, ~ ~ ~ ~ ~ ~ ~ ~~'T/Al.'931p01 t i _31_ The label of the lower of two clusteas is uansfcrred after a split or join because the lowest cluster is most likely to include the front of a vehicle, an~ is therefore gi~~en priority with regard to maintaining t:luster labels.
A record of the bounding box coordinates is maintained for each cluster in the cluster list, together with its label, the labels age, and when the cluster was last soon.
whenever a lalx! is inherited, its age increases, aed its last scene value is reset. If a label is not transferred in the couur~e of one bald, its last scene value is incremented. A
label is removed from the cluster list if its last scene value exceeds a label tenure lU chacshold. Cluster labels, coordinates and classifications are passed to the trajectory task 1 gl0. i The trajectory task 180 uses the received cluster data to track the elustets over successive video ~clds. Tine coordinates used for tracking ~ cluster box are the coordinates of the cenm of the base of the box, and the eoo~dinatc system for the roadway 16 which is adopted is illustrated in Figure 25. 'I"ltc datum 300 of the roadway ~ .
coordinate system is an arbitrary poina on the roadway, which ~as been chorea as the centre of the left hand fog line underneath the edge of as overpass brid=e holding the cameras b and 8. Vehicles 302 travel in the positive Y axis direction on the roadway lb, 30 staving at a negative value is the distance. The trajectory of a cluster box in image plane coordinates (xi, y~, as shown in the graph of Figure 26 is not linear with time due to the effect of perspective. Therefore a camera transformation is e~plied so as to covert image plane coordinates to real world 3-D coordinates. In m~atri~c forge, the coverall camera transformation is as follows:

~ 0 J~ 0 ~ ar ~0 0 0 I 0 0 a,I, ~y (13) a 0 0 -l~f 1 0 0 1 ~ ~ 0 0 I I I
2~ where ax X-axis scaling factor is pixelslmm (iatr~asic) c~, Y-axis scalang factor in pixelslmm (in~ias'sc) ' WO 93/19441 PCT/AL'93/Opl 1:~
~~j~~l~
_ Xa image place offset in pixels (intrinsic) Ya image plane offset is pixels (intrinsic) focal length (intrinsic) °f~~, detection camera 6 position in world coordinates-(extrinsic) 'I~e intrinsic parameters are incite characteristics of the sera and sensor, while ttte extrinsic parameters are characteristics only of the position and orientation of the camera. The principle point of the image plane is the intsrststioa of the optical axis and that plane, at coordinates (Xo,Yo). Equation 13 can be writtenas:
x x yt o ~ i x; 1 where C is the camera calibration matrix, a 3 x 4 homogeneous transform which performs scaling, translation and perspective corteaion. The image pl~e coordinates are then cxgressed in terms of homogeneous coordinate= as:
X~ s si ~' iii s y i ~ .'~~ ~ , !1~
z The general petspectlve transform maps a ray in three ~ dimensional space to a point on the image plane. For vehicle coordinates in the imaue plane as seen by the detection camera 6, a unique three dimensional location of the vehicle cannot be detesmiaed so the bottorg of a cluster box received ft~ the label task is considered to v be on the roadway, i,e. z * 0, and therefore the box can be with reference to the i roadway x and y coordinates. Tire equations 14, 15 and lh,; given the image pleas coordinates and z, can be solved simultaneously for the roadway co~d~ates x and y to specify the position of a vehicle. The equations have been s0~ved asiag the computer algebra package ARAPLE, and the solution, is C notation, ~ a~s~.follows:
den * (-Xi'C31'C22+Xi'4'3.'.'C~lt(Yi'1-X21)°C12+(-Y~'C32+C~)'Cll);

~'O 93/ 19~d4 d PCT/de L'93/00 t !:

y = -(-Xi'C31°C"24+~Ci°(:34'C21+(Yi°C?1-C'~1)°Cl~
b (Xi'C33°L'? 1-Xi'C31 °CZ3)°z+(Yi°C31-C'1 ~'z'C: ;ø
{-Yi'C34+~4+(-xi°C33+C,~3)sz)'Cll ) / dcn;
~ x = (-C'r4°Xi°C;3.',+C'=''Xl°C34+(Yi°C3'?-C~')'C14.+
{CZ3'Xi'C33-C23°Xi'C3Z)°z+
(Yi°C32-C'2Z)'z°CI3+{-'Yi~C~4+C'24+(-Yi°C33+G"23)°z)°C1s ) I den; _ .
'The solution explicitly includes height above the rosdw~y, z, which can be set at zero for daytime operation or some marginal distance above the roadway, whereas at ttigbt, the bottom of the cluster box geaeraily corresponds to the height of the headlights above the road, and therefore z is set to a notional headlight heist. Figure M7 illustrates a graph of the same vehicle trajeetory as in Fib 2b, after the trajectory bas been mapped to the roadway cootdinatea x and y. 'tee trajectory illuStratcs the vehicle is moving at a constant speed, and is the left hand lane.
The time at which the vehicle 302 wilt reach the acq:iisition line Zo, and the future location of the vehicle 302, need to be predicted, due tb latency is ehe systeia.
Considerable latency exists between a taigger request and 'ea~age aaquisitlon via the r0 acquisition camera 8, and additional latency is caused by pixel transfer, image processing pipeline delay and software processing delay. The iaformatiaa obtaaned off the basis of the iazagea required by the detection eamera 6 provide a delayed representation of the actual vehicle position, and therefore it is neerxsary estimate the ~utuze position and speed of the vehicle 302.
The position estimates of a vehicle obtained by the inverse perspective discussed above are quite noisy due to quantisation effects, gartieularly when vehicles are in the distance, therefore simple di~ersncing c~onot be used to ate velocity of a vehicle and therefore the software uses a Kahn filter to reco~ruot Vibe vehicle's lateral and 30 longitudinal position and velocity states, based oa flea noisy observations of the vehicle position. °1'he vehi~a s~ for each of the loagicudiaal a~ lateral axes comprises position m and speed w of the vehicle, represented as follows;

t~~ 93/19441 ~ PC."'f~~193/QQI1;
~~.~ ~1~ i K=t~~lr (1~
Ire space state for8n, assuming constant va6ocity motion, ~h~e vehicle dynamics arc I-K = d~K ~ (li) 3C ° Chi ( 19) I
where Y is the observable output of the system, bei~~ the vebacle's lateral or longitudinal position, ~ is the state-transition mataix, and C ~ the observation matrix.
For constant velocity motion the asatriees ate as follows:
~ $ 1 °t' ' (20) 0 1~
C m (I ~1 !Zl) where T is the sa~pliag interval, beta' equal to the video field interval which is ms. The Kalman filter e~uativns for one axis are I
IS K m ~PCr(~PC'r + I~"~ i (Z~) $ = dd$ + K(a -1C~ i (x3) r .
P ~ ~P~ + ItyI= - K' (Z~) ?.4 The filter is predictive, and ~ is the predictive valve of the vehicle state for the next sample interval. K is a gala, P is the error co-variance' ~a~ix, sad Ia is a 3 x 3 identity matrix. R, and Its are iapest and output co-variance etes, and a~ee used to adjust the dynamics of the filter.
' i The Kalmaa filter aquatio~ 22. 23 and 24 are c~plex acd time ping to execute in matriu form, and the computer algebra ' was iced to redoes i ~'~ 93119441 ~ ~ ~ ~ ~_ ~ ~ II p~/,4 L'93/pp t 1=

the equations to ss~lar form, as follows, in C notation:
;' compute the filter gain "/ ' data = kp-apl l + 'R2; _ kI = (kp-apil + T ' kp-aplZ) I den k°' = kp-apl2 / den:
I' update the sate vector 'I
xl = kp->xi + T " kp-~~x2 +ki " (°y - kp-axl);
i0 x2 = kp->x2 + k2 ' ('y - kp->xl);
kp-axl = xI;
kP-a~ _ ~;
r /" update the eovar matrix (symmetric so keep only 3 eletaen:t) "/
pll = 'Rl + kp-apl l + 2.0 ' T " kp->p12 + T ' T ' kp->p22 -kl °' kp-apl l - kl ' kp-apl2 " T;
p12 = kp->pI2 + T ° kp->p22 - kl ' kp-apl2;
p2 : ~ °Rl + kp-ap~"2 - I~c2 ~ kp-apl2;
!
kp~->pil = pli;
kp->p1Z = p12; ' kP'>PZ2 = P22 The estimated values for the state of the vehicle ammd error wariaace for the her are calculated using the equations and are stored iu a data a ~kp. Optis~al values for Rt and R2 are detetmi~ed empirically. Figure 2~ ill gasphs which can be plotted from the estimated values for one axis, being the t~l position and estimated speed of the vehicle, aad rhea estimated error ed'wlth the camera alter calculations, as tech vidso ~eid as received. The estimate acq~isitio~ rims is calculated by using the estimated vehicle state data. ~ the potion ~ at which acquisition co ocs~r is , the estimated acqui;zitiott time is calculated ~y taking the difference a H'O 93! 19~! 1 Pt_°t'/ A t.' 93/00 t t ~

bctweea the estimated position and the acquisition position, and dividing the result by the estimated velocity of the vehicle. 'den the estimated acquisition tune falls below a vaiue whip indicates acquisition is to occur within the time of the next vide~
field then the estimated time information is provided to a trigger board g4: -The estimated c~ehicle state coordinate fez the x direction indicates which camera g of a mufti-lane cartaageway is to be triggered.
The scaling matrix C of equation 14 muy be calibtat~ usiap road markers or preferabiy telescopic stakes which are placid at predeter~'ua~d positions along the roadway 16. The stakes are surveyed with respect to the r0aduv~y datum 300 to obtain the x, y sad z coordiaates for di~eteat positions o3o the stapes, sad rhea removed.
Equation 14 can be expanded as follows:
Csix * C~ * C~Z * C~ - CayX'x - C~C'y - Cx,X'Z - Cue' a p (Zgj C= x * C *
mY ~ - Ct~ - C-s~Y'x ° ~~'Y - ~Y'~ - Ca.y' ~ 0 t26) i which relate as image pleas eoordiaate (X'.Y°) to a real world coordinate (x,y,z).
For n observations this can be expressed in math form as folldws:
xt yt zt 1 0 0 0 0 -X'txt -X'tyt -X'~zt ~ x'I
0 0 0 0 xt yt zt 1 -Y'txt -Y'tyt ..lntzt ~ Ctt Y't . . . . . . . . . . , I
~ . . . . . . . a . . ; .
xa ya Zs E6 ~ 0 0 ~ -X''~x~ -X'~s -X'ata ~ ~a 0 0 0 0 x' y~ z' a -~'.~ -Y'~'. -Y'az. ~ Z"s The nations are hem I g e9 ogeaeous and therefore the overall sin of the C matrix is simply chorea so that Ca, ~ 1, sad this parameter is not idettified.
Equation 27 has 11 unknowns sad for a solution reqstires at least 5.5 ob~r,ratiooos, being pairs of (X','~'°) ~ (x~Y~)~ '~ system of equation: is generally ova deter~iaed, and a least square solution is obtained using s singular value d~poiitioa tee~aique. For solution the H'O 93/19441 ~ ~ ~ ~ ~ ~ ~ P~/A~.'93/m(111a --3?~
calibration points x,y.z must not lie in a common plane. The real world coordinates are obtained from the survey results, and the image plane coordiasaces (X'.Y°) are obtained from a display of the detection camera image of the survey stakes using a cursor plotting software package.
To achieve correct triggering of the acquisition camera g, the timing of the systetn needs to cake into account the follawin=:
{i) The system timing: the system must have suffiCimt temporal resolution to facilitate accurate image capture, i.e. the system ~ttst have a sufficient '~ehicle aoquisltion rate, such ,as two vehicles gex second, to avoid omi~ion of vehicles on the roadway.
(ii) Prediction: determining the time of w$ich an image of a vehicle is to ix acquired, and thus initiate image acquisition. ' (iii) Acquisition data flow: timing to perform the physical ineetfaciag between 13 the acquisition camera and the acquisition sub-system 6g res~oasibte for each image capture and storage.
The system timing is resolved at two levels, a coarse leveg,considered to start from periods greater than 24 hours, and a high resolution, one level. xhe coarse level timing 34 is maintained by a real time master clock 354 of the trigger board 84, as shown in Figtue ?9. 'The geometry of the acquisition camera 8 is chorea ~to limit tl:e effects of perspective, limit image blur and take into account other constraints imposed by limitations in the depdt of field available, and for an overpass bridge mounting, the image acquisition point 22 is between 17 and 20 metres fraza the camera 8, and tbs camera is '?5 at as eagle greater rhea 15° and approximately 24° t~ the roadv~ay. A target vehicle traverses the acquisition point 22 within the field of view 20 in approRimately 40 ms, being the acquisition window, at a nominal vehicle speed of 1~0 . 'I~e real time clock 354 provides timing down to 20 ms intervals. Due to the uncertsinties in the position of.tarset vehicle accumulated dutaag the segmentation, ,rlustett°ing and trajectory 30 tasks, one or more timing events duria: t~ a~t~isition window are not sufficient to allow reliable image capture, therefore the high resolution tiaoiag is resolved to hori$ontal video line scan times, being apprr»tamatcly G4,sd.

~'O 93/19~AI
PC°T~ A L' 83/0011 _ 38 -The CPtJ 64, as described above. is able to classify v~hieles during the region analysis and clusterirsg procedures and, in particular, is able to distinguish large vehicles and small vehicles on the basis of the size of a cluster, if theLCPU 64 determines that a cluster represents a vehicle for which an image is to be acquired, the final estimated acduisltion time determined by the trajectory cask is supplied t~ the erir board 86, as shown in Figure 27 via the VIA bras interface 350. The CP1J 64 supplies the estimated time as a 3I bit value, which regeesents the number of the hori~o~atal scan line during the next field, which when reached indicates the acquisition camera 8 is to be triggered. The vI~IE bus can be used to writs the number of tire scatsning lin~ at which uisitioa is ZO ao occur into an acquisition line register 352 of the board X84, For a mufti-la>se carriageway, the CPL 6a also provides data to indicate the cor~ct acquisition cataera 8 to be activated, as determined by the horizontal position data of the vehicle, yn addition eo the acquisition line register 352, and the master clock 354, the tsi~ts board 84 ~includas a comparator 336, and a scanning line counter 38 which also includes a count reg5ster to store the value of the line s~unt. The master clock bas a battery back-up 360 and is synchronised to the horizontal sync of the detection camera 6 so ~ to atxurately keep traaDc of video fields, reset the line counter 38 at the end of each ~aeld and be used as a basis on which timE scamp infot~ation can be generated 9oe allocation to the raw seed parameters processed by the APA b~uard 80. After the mtmber of the acquisition line 30 has been read into the acquisition line register 35:, the line ~ counter 358 counts the horizontal scan iiaas on the basis of pulses provided from the digitiser board 74. T'hs fins coast of the countex 338 dad the number held in the acquisiti~n line register 352 are compared by the ccmoparator dad when the two numbers are tt~e same, abe coaaparator issues as aoquisitioa pulse on the lint: 32 for the acquisition damera 8.
Providing the 35 trigger board 84 to trigger the acquisition camera 8 as also more acataate than relying on software eorataol as the CPU 64 is open co intemcpts and theref ~ose be relied on co accurately control c~ signals of real time events. I
The image acquisition caanera 8 bas been developed to acquired detailed electronic 30 stills or images of vehicle: travelling head-oa to"rards the cato~esa 8 at speeds up t~ if0 km/h. 'tee ~amc rate is at least two pitaures per using a inoa-interlaced mode. Stand~d camera architeccums suffertd limitation of insuf~tdent resolution. image ~'O 93/ 19441 pCfIAL'93/001 I:
~~~~~~a smear and unwanted effects caused by blooming of the image sensor whey vehicle driving lights or sun reflections shone into the camera 8. Hiooming is considered an unacceptable image anomaly, particularly if it appears in the licence plate region of an image which can severely restrict the automatic or even manual reading of the licence piste characters. Another unacceptable image anomaly is image smear, which tended to occur for standard camera architectures in areas of high contrast, which may include the licence plate region as most large ve>ucles have bght~ meted in the vicinity of the licence plate. The effect of image smear tended to iae as sensor exposure decreases, sad for standard camera architectures, image smear was uaptably detrimental at < xstrre times of 1 ms.
The image acquisition camera ~ is a high resolution. t~.illumivated full-a camera architecture having a 1280 x 1024 pixel ~o'M~ moatnatic silicon charge coupled device (CCD) serssor. To prevent smeaaag or bloomang across as image, the IS earners 8 includes a lateral overflow drain sensor architecture which provides 10008 antiblooming characteristics. The architecture provides a site to drain exce$s electrons for each pixel, and Eastman Kodak Co. has developed one suchv:emsor incor~ratiag this architecture. 'This combined with extended infrared aensitiv~ty to 1.I
micrometres, enables near infrared imaging of vehicles and reduces bioomia~ to as accxptable image without degrading the clarity of the vehicle licence plate in the images.
The pixels of the camera 8 arc I6 miarometres equate w~th a 7096 frli factor and have a qwntum efficiency of 0.25 e-lpboton at the image exposure wavelengeb of 800-g00 am. This makes the camera suitable to operation st ;exposure times of 1 ms, d5 which is required to fete the moving vehicle:. The seasoe has low light i~o0aging capability at 1 millisecond exposure time, but is pFacxice the i~a$r:red flash 40 is required to provide SII-in illurrniaatioa as duaiag m~ openti>ag condition extreane lighting ratios wore experienced. This occurs, for example, when shadows iaipiage on the vehicle oar when imaging is performed at eight. ~Synrluonous shuttering of the CCD sensor is achieved with a mechanical shutter, a camera butter made bye Robot, Germany, which is elsetronicslly triggered for I millisaecond. The shutter also pr~vi~s a basis for sYr~~Onaaataon of the olectronic flash 40, ate desexibsd below.

w'O 93/l9aai ~~T/mtr'93/OOt 1~
~~~~~~
i The analogue output from the image sensor is directly cot9verted to digital data by the camera 8, and the digital irrtage data is capable of being outputted in either an 8 bit grsy level format or in a compressed format, using stastda3d ll~IrG image compression.
The flash 40 has a flash head which includes an six-cooed Xenon short-duration (~500 acs) flash rube anounted behind a mirror reflector. The aniaror reflector products a narrow beam width for the illumination of one lane 35. 'The power pack for the flash consists of an air-cooled 100 to 15~ Joule variable outpaat povubr capacitor pack which has a cycle time of two flashes per second. The flash 40 bss aj wavelength range of 695 nm to 1300 nrn. An infrared band pass filter is placed on tb~ front of the flash tube which trangrnits electroma~etic wavelengths primarily outside the human visible range, thereby preventing "flash dazzle" of oncoming drivers ~ pm~ly eiaating delectability of the flash 40, is The wavelettgtb at which the 5lter allows transmission is selected so as to balancx elimination of driver "flash dazzle" and still obtain an .ptabie contrast ran;e for retro- 'reflective licence plates. Licence plates with both the cl~a~cters staid background having rctxo-.reflective properties are relatively ~di~cult to image, and the sefecte~d balance between the CCIy adasor spectral sensitivity, the flat and pass filter and the ~0 lens filter for the camera 8 is illustrated in the graph of Figure 3t9.
,fin expt~ure control circuit is connceted to the Robot shutter and the iris aperture rdec~nist» of the lens of the camera 8. The C;irwit controls the aperture p~ition in accordance with the level of asnbiont light sensed by t~ circuit. T'he circuit provides a k sigh ~ line 36 to control the power and triggering of the infrared flash ~0. As tie a~uisition camera 8 'S aperture closes with imxeased ambient ildumiaarioa, the flasf t power is increased to maintain an optimum balatnae between ambient light and flash "fill-in"
illumination. T'he circuit also includes a delay element io maintain the average gash power dosing large transient fluetuation of light received that can be why white tsucks pass of sunlight is directly reflected faom vehicle windscreens onto the carntra 8.
The circuit is 30 based ort standard expo:ore control circasits, and, in ~dicio~ to the delay element, includes an infraaed sensor to measum the ambient light. The ~~ power is opntrolled by adjusting the capacitanot= of the power pack for the flash 40.

evp 93/d944r d'C'T/AL'93/ppd d:
~1~~~~~ ' _41_ 'I~a infrared flash 40 is mountctt at an angle of 9.5° With respect to the optical axis of the acquisition camera 8, ancJ at an angle of grCater Chars 15' to the roadway I6.
The field of view 39 of the flash 40 is sitniIar to the field of vaew ?0 of the acquisition camera 8. The geometry of the flash 40 is important so as to reduce any retro-reflective effect from the exposed vehicle, in particular its licence plato. The rctro-reflective progenies of the paint used on licence plates is such that the maxinsum reflected light is back along the axis of the flash illuminating beans. The angle of illumination and the illumination energy is seieaed to rafts into at:count the diverse Mange of retro-reflective and non-retroreflective paint colours and formulations used on Licence plates.
Examples IO of the images which can be obtained by the acquisition ca~era 8 of the vehicle monitoring system are illustrated in Figures 31 and 32. ' The acquisition camera 8 is connected to the deteetie~f sub-system 66 aid acquisition sub-system 58 by an interface board 359, as sho own in Figure 33.
'T>ae I3 interface board 359 provides power to the camera 8, can issud data iatertupts for the ptncessor 360 of the camera 8, and, is connected to an image buffer 361 and trigger interfaces 363 of the r.~era 8 by optical isolators 365. The iateifaoe board 359 provides communications to the control unit 10 via differential RS422 communications interfaces 367 which are coaneaed by communications cables 369. The tr~gg~r signal is provided 2t) from the trigger board 84 to the trigger interfacx 363 of the sa~deta 8 by the RS4.°'.'-interconneet. Image data produced by the CCD sensor 371 i~ available in the image bu~'er 361 of the can;rera 8 approximately 300 ms after the r~me~a 8 receives the trigger signal. Ac that time a dace interrupt signal is , sent fiom the cx>ntroi unit 10 to request traasfsr of the image data from the camera 8. ?be image data' is read from the image ?5 buyer 361 as 16 bit vvords at a rate of 1 I~Iword/s, where each word represents two 8 bit pixel values. A strobe clock edge is also included in each ~16 bit word for tirauag purposes. The 16 bit data stream is coeverted to 8 bit data st ~ logic levels by the CPU 64, and the async)uonous imaas data is rhea ,~ by a frame grabber 86 of the acquisition sub-system 68, which is a Datacube A~axsauu~ board. 'T'he image data 30 is then clocked into as acquisition image buffer board 88 where et is held until transferred by a bus d~epeater 89 to the image buffer and eommunfrations e~natroller 57 or a lioeaa~s plate recognition system 51, as shovrn in Figure 6.
i w0 93!19441 PCT/A,~.'93/0~1 t' The images captured by the acquisition carnets 8 possess the following eharactetistics:
(i) A full lane width of 3.~ metres is imaged.
(ii) The pixel tesoiutiotts of each licence piste character. for character sets of 40 x 80 mm, were at least 10 x 20 pixels for W, Z and 9, and a nqiaimum of four pixels for a eltaracter stroke, such as the letters I, L etc. Pixel resole;io8s of up to 15 x 30 were achieved on characters for a full lane field of view 20.
i (ill) The average grey level of a character sttolee;is at least .'.0 grey levels higher thaw the background of the grey level of Bhe licence pate.
(iv) Both the ficxnce plate region and the vehicle; body work are imaged adequately to enable identification and verification of vehicle ~ype.
(v) The quality of the licencx plate image is nlati~ely constane throughout a 24 hour period for all vehicle aad lidettct plate types.
13 The image buffer and communications controi57 include a Silicon Caraphics Personal IRIS 4DI355 msshane as a buffer box 381 for handling intermediate storage of images on disk 383, a CISC4 Internet Psotoso! (IP) roster 385 and a Summit Technologies 52000 ISDId bandwidth manager 387, as shown in iFigure 34. The remaining description relates to image transfer between the repel ter 89 and the buffer box 30 381, but the description also applies to amaac transfer between the repeater 89 and a licence plate recog~ution system 51 located at the node 2, as shown in Figure 6.
The data tsruosfer by the bus repeater.89 to the buffer; box 38i is made by a

3 M~/s digital line. The repeater 89, which is a ~tT3 Model 413!x-bus repe:..er, with '$ D1~1A capability, ettabies the buffer box 381 to copy data due~tly from the buffer 88 in the acquisition subsystem 68. To coordinate i~ga transfer between the buffer box 381 and the system 68, an image header scruauro is established fob storage of the images in the buffer 88, sad messages are allowed to be pasted back and forth between the buffer box 381 sad the system 68 via incenqtpts in a mail box location; 'The memory layout for 30 the image buffer 88 is shown in Figure 35 and the highca armory locations are used to store acquired images in bu~ea segments 370 with a ~72 for each image buffer being stored in the lower mesaory locations, The ' header 372 includes date of I

w093/19441. I PCT/AL'93/pOtt>
21j~J15 image acquisition, a base address far the image in the buffer 88, a busy flag to indicate whether the image is presently being read, and information on the size of the image. A
memory header 374 at the lowest location in the buffer is shared'with the buffer box 381.
and inciudes the following 5elds:
1. ha-hostintr: used by the buffer box 381 to specify which type of intenvpts they are sending.
I
2. ha-imagenum: used to tell the buffer box 381 which image to read after an image available irnermpt is sent.
i 3. ha-numbufs: the number of image buffers allocated in ithe buffer 88.

4. ha-height, ha-width: the orgaafsation of the image within the buffer 88.
is S. ha-bufsize: the siu of the buffer, which is s multiple of ~6 bytes.
i i The architecture of tht software modules usod by the buffer box and the acquisition image proceaaittg system 42 is illustrated in Figura l3fi. '1"he buffer box runs '_0 a capture module 401 which is responsible for communication ,between the acquisition sub-system 68 vla the BIT3 interface board 4$9. Ti:e modtt~e poles the acquisition sub-ayacem 68 for images, stores them in a memory buffer, aid rhea stores them in a dimctory GprureQ as a 51e with a unique name. 'The came is trade up of the 5rst 5ve cbaraaers of the name of the buffer brnc 381 eu~d a ten digit number. A CRC
essor :5 chec~ng value is generated and image data is stored in a header of the 51e, including the name of the remote site or nook 2, the tiara the image was capet~ied by the camera 8, the image header length, the CRC value and the image width and height. The CaptureQ is able to score S00 images, the data of which each occupy approximately 1.5 Mbytes. 1~
the CaptuseQ overflows, the images are discarded, and the file names of the lost image 30 is recorded in an error fife together with the time the images ueidiaasded.
Overflow of the CaptureQ may ocau if the acquisition sub-system 68 acquits imps at a high rate for a long perm of time, or the lick thsvugh the ISDN 45 to the,' ventral server 47 is cue ~O 93/19441 PCT~AI'93/Opl 1?
of servict for as cxcended period of cimc. The communications link to the eentsai server 47 fzom each remote site : is prov ided by the scoter 385 connected to the buffer box 381.
and a ~G? 1 protoeol little 389 between the muter 385 afld the bandwidth manager 387.
which prov ides a Macrolink'" 391 to the ISDN 45. The central server 47 in turn is also connected to the ISDN 45 for each remote site 2 by a CISCO IP scoter 385, a Summit Technologies S2000 bandwidth manager 387, an X21 link 389 between the manager and the muter 385, aad a Macroliak 391 to the iSDN 45. The bandwidth manager and the muter 385 form the communications controller 42 of the acquisition image proceaain~ system 42. The X21 links 389 are scaadard H-ISI~N communications Iink governed by CCITT standards. The Macroliaks 391 are Primary Rate Access Iitzlcs provided by the second applicant aad are based on the CCITT s for Primary Rate Access in the B-ISDN. The X21 Iirrks operate at 768 KRJs ~d the Macroliaks provide two virtual links operaciag at 384 KHJs. The bandwidth manage 381 is esaetatially a multiplexes which uses a data aggregation protocol sad provide aocxss to the ISDN 45.
t5 The remote sites 2 are each represented at the central server 47, which is a Silicoa i Gtaphia Crimson machine, by a retrieve module 403, as shown in Figure 36, which makes a socket connection to the respective remote site 2 and polls for an image from the remote site 2. The FTP protocol, which is a Uaix fate traa~fer protocol, is used to retrieve images, including their associated dsts, from the xemotc;site 2 and when received the ima~,e is checked for integrity of the imago data on the basis of the CRC
value, aad stored oa a Ret:;avaliQ directory 405 of the carver 47 which has a c~aeity of 720 images.
'Ihe images arc atost~d on the RetrievalQ 405 with the time w~ea the image was first r~equeued and the rites when the image was finsliy received. Aa SDistributor module 35 40'1 is responsible for distributing the image 81e names to store modules 409 of the central server 47. The store modules 409 retrieve images fmmi the R~etrievalQ
405 sad archive them in respective imags store: 4i1 which have the ~pacity to store images acquired over a week from each site 2. 'Ihe image store 4i1 ate Exabyte IOI
tape storage systems which can each hold up to ten tapes that each hive a capacity set at 3000 images. The store module 409 cotamunicates with a tape drivtf for each fore 411 which based on a tape driver developed by Crene Daonsk of Vulcan 'haboratories; U.S.
The driver controls laadiag sad ualoadiag of a tape from a stare 4~ 1 by a robot arm. The W093/19441 ~ ~ r9 ~ ~ ~ ~ ~Pt=1'lAl.'93/0411~
I

driver on initialisation dctcrmincs the number of tapes in the store 411, and for a Bold start formats each tape and loads the 5rst tape, For a warm start the drivel simply xlects the tape last used. When a tape reaches its 3000 image capacity it is returned to its storage bay and the next tape is selected.
The SDistributor module 407 has a list of the names of the Sles in the RetrievalQ
405 called store list and another list of the names of files which are in the process of being stored on an istage store 411 rxlled InlProgress list. When a store module 409 requests a file name, the SDistributot module 407 returns a file name from the store list and moves that same to the InProgress list. If a f1e names is not available, the module 407 a«xsses names fmm the RetrievalQ, adds them to the store Kist and then returns file aaaus. When the module 407 receives a storage ackaowledgmevt from the store module 409, rhea the file name is removed from the IaProg:esa list. T'ht s:are module 409 poles the SDistributor module 407 for a 5ie name, sad on recxiviag the file name retrieves the corresponding 51e from the RetrievalQ and copies it onto the imaiSe store. The same files is also copied onto a directory of the server 47, IPQ 413 whi~h can hold 750 ieaages.
It IPQ 413 is thll, the file is discarded and the hander of the 51e is copied onto a further directory DatabaseQ. Act acknowledgment message is rhea seat to the SD~stributor module 407. A dare staaop is plsnd on all files indicating whoa the file is archived.
'0 An IPDfatributor module 417 distributes images to a licence plate rccogoitinn aystam 51 connected to the Ethernef LAN 419 of the ocntral ter~er 47. The module 417 maintain: a ties of 51e names, called Image list, which its the 5les paid is IPQ
413. Whoa the liceacx plate ceoogaition system S1 poles for a file name, the module 417 ?5 returns a file name from image liar sad moves that Sle acme td another list, IP~'rogzess list. When the sy:tem 51 acknowledges that it bas received the oocreaponding 5ie, then the file name is deleted from the IPI?rogzess list, together with the file froze IPQ 413.
If file namts ace not available is Image list, the names are obtrihed from the IPQ by the module 417, sad added to the liar. The module 4I7 rommuni~ates with the system 30 via a socket coattaxion. Licentx plate dataits cxRraaed by the >iecogaitioa system 51 art stored on a DatabaseQ 415 of the aervcr 47 together with othsr image data details, such as image acquisition time, and ins:aaaaaeous speed of vehicle which have already been I .

W093119~4t ' 'y ~' PC"t/A14310011~
~1~~5~
provided with the image from the remote sites 3. A database module 419 poles foe files placed on DatabaseQ 415, and then stores the fibs on an imag i database 421.
The licence plats recognition system 51 has been implemented using a Silicon ~ Graphics workstation 400 which is connected to the LAN 419, as shown in Figure 37.
but can also be connected directly to the repeater board 89 at a remote site 3. A Pixar II
image computer 402 is connected to the workstation 400 and acts as an image co-processor. The system 51 also includes a monitor 404, keyboud 406, disc storage of 600 MB 408 and optfcal disc storage of 1.2 GB 410 oonnecte~ to the worksution 400.
The workstation 4p0 usrs, inter alia, VIVID (Vehicle Identification by Video Image Detection) software owned by the State of Victoeia~ which is abio~to locate a m;mberplate its a vehicle image, and then perform optical character recogtution (OCR) on the located numberplate to extract the liaace plate cha:acters. 'Ihs parameter settings of the VIVID
software have been adjusted to handle the images provided by theiscquisitioa sub-system 68, according to the size and contrast of the images. To accept tie images at a peak rate of ? per second, a real time image handling procedure 412, as shown in Figure 38 is used. The procedure begin at step 414 by requesting as image 51e name from the IP
distributor model 417. If a name is not received at step 416, the IP
distributor module 417 is polled again, ot6ervvi:e the received namo is used to sooe~ the If'Q
414 and store the image 51e o,n the disk 408, at step 418.
Images aro saxsaed from the disk 408 and procr~scd b~y four separate software modules of the workstation 400, a locate plate ~ moduie 420, a glyph extraction module 422, and OCR motiule 424 ttnd s plate recognition module 426, ac shown is Figure 39.
23 The iocate plate module 420, as shown in Figure 40, begirt at step 430 by preparing the 1280 x 1024 pixel image for processing as a number of pusl windows for the Pixar co-processor 402. At step 432, the system 51 attempts to dote~t an edge of a ehafacter sire object, and when deceetcd the object's location is determined at step 434. An object assembler is used at stop 436 to group adjsoent objects together, and the groups ate processed by a plate clauifier 438 to determine whethca the objecx groups could constitute a licence plate. If an object group is classed as a ølate according to a plate template, a bounding box is formed, sad its coordinates to the gtypb extraction ~~~~~1~
WO 93/ 19441 pCT/ A L:93/001 !
_4'7-module 423. The glyph extraction rnoG~:.e 4?3 processes each bounding box to binarise and extract individual characters in a pounding box and then pass the "glyphs", i.e.
licer:cc plate letters and numbers, to the OCR module 424. 'Fhe OCR module 424, as shown in Figure 41 begins at step 4ø8 by building a typological graphical representation of a glyph from the glyph bitmap provided by the glyph extraction module 42'.', for each glyph. The graphical representation is analysed at step 440 so as to detect any characteristic features, such as holes, arcs and vertical and hotizontai lines. From the results of step 440 an 81 bit string representing the charasxeristic features of the glyph is created at step 442. A bayesian statistical analysis is then performed at step 444 on the feature string to try and match the features against a set of prcviouely determined features characteristic of known ASCII characias. The ASCII value of the snatch with the highest probably of being correrx is retutaad to the plate recognition module 426.
The plate recognition module 426 detesmiaes whethest the glyphs in a bounding box constitute a valid lictace plate. The module 42b effe~ively coatmls the other image processing modules as it has the ability to override the results bf the OCR
module 424 of to force the glyph extraction module 422 to use a bounding box othex thaw that found by the locate module 420. The majority of vehicle !leaner plates in Australia have six characters and fall into one of two classes, Federal plates or non=Federal plates. Federal ..0 plates comprise two alphabetic characters, two digits and two alphabetic cha:aacra.
whereas non-Feder<t place comprise three alphabetic cbaraccoa aad are followed by three digits. The plate recognition module 426 is able to determine whether a valid licencx plate has been found oa the basis of this information, aad.~other inf0n~atioa, such as the spacing of characters and the specific charaetcristic alphanumeric sequetxxs used .'.5 by the non-Federal plates. The OCR module, for example, may not be able to distinguish between capital B and 8, sad fps many plate foots; there: is no difference between a 0 sad O or a 1 sad as i. Therefore: the plate caoog~itiob module 426 tray need to override the results obtained by the OCdt raodula 424 The plate re~gnitiota module 426 is also able to ithe glyph extraction module 424 to procGa6 an altered 30 bounding box if the module 426 de;terrmines that there tray be ~ additional glyph to the left or right of an original bounding box returned by the locate module 420.
The: licence plate details obtained by the plate reso~ition metduie 426 art; shed oa l~atabaaeQ 415 ~'~'O 93/ 19441 PCT/ A ir'93/o0 i i i _48~
of the server 47, arad archived on the optical disk 410. The opticisl disk 410 alSO archives image files which the system 51 is nnablc to process when received.
The database on the optical disc 410 stores for each processed image as does DatabaseQ 415, data concerning the position, size and characters of the aumberplate located in the image, and other details such as time and date df aequisitiota.
It is also structured with data pointers whieh facilitate access to the stored data by the workstation 400. The workstation 400 iz:cludes graphical ttxr interface software which enables as operator to review the results of the procedures 412 and 414, anti perform further optical character recognition or mmbe~late regions. as xlected. Any further dCR
proc~3iingg performed on a plate region xlecced by the operator of the woritatatiota 400 is normally used to aaalyse the performaece Of the proctrdures 412 and 4I4 abd not to alter cht integrity of the data held im the optical disc 410.
The image data stared on database 421 is procaud by aiatchiag software which looks for matches amongst the Licence plate details fields of the image data so as to locate ocxurrences of detection of the sasae iiatxe plate at different remote sites or nodes 2. Once a match has been located, the acquisition time field: era be used to determine whether speed or time violatiom~ have oaurred in travel betvrie,~n trmdte sites 2, as distance between the; sites 2 is known. The m~atchiaig aofflvare is run 4a a Sun Miarosyatems worknstion 450 cotusecced to the 1 AN 4I9, or aloernstively, the matching software is run on a systr~m of a road traffic authority, with the image data being sent by the cemral carver 47 over the iSDN 45 to the road traffic auttiority. The road traff'ae authotfty is able to c~~mmttaicate with the central server 47 via the ISDN 45 to obtain :S archived images, s~ required. ' To avoid sending all images to the central server 47, a lame cumber of which may not be of interest, images can be archived at the nods 2, aid licetaoe plate details extracted at the remote nodes 2 by respective licence plate seca~itioa systems conaeaed directly co the BITS npsatrr 89 of a ttode'a aoquisitioa stab-sy~em 6B.
The server 47 then only recsives the ext:saed licence plotte details, and other data on the image, such as acquisition time, the remote site, and insnmta:sestss spetd, sad not the WO 93/39441 ~ ~ j ro ~ ~ ~ p~,"i'/e~L'93/pOtle image itself. Images archived at the remote sites 2 can be retrieved by the central stwct 47 when required.
Control of the remote nodes ? is performed by the remote site user interface which tuns on the Sua workstation 450 connected to the LAN 419 of the central server 47. The interface 53 includes a user tool which communicates vvitb a super task of each eemote site 2 using a Sun Miaosystetns Remote Procedure CaII (RPC'~
communications protocol. The super cask provides a set of paocedural functions wlrieh caa be called by the user tool using the RPC protocol, regardless of the locations of the workstation 450.
The RPC protocol handlca data type conversions and atignmoat. 'ihe procedures provided by the super task perform various actions which together allow ramplete control of the software of a node 2. For exempla, a parameter Tilt maintains a fiat of all vatiablas u;s~
by the sof:'vare of the node= 2, to;cthrer with their initial values. ~ The form of the values indicates the variable type, which may be a decimal integer, ~ bexadaimal integer, a floating point value, a character string or booleaa value. The vsr$ables can be aiteted by adjustins tae parameter file, sad location of the variables Ustod ~ in the parameter file is done vis a VxWorka system: table which contains adl global symbols. The user tool, in addition to changing system parameters, can accxss the super task to obtain status and configuratiota information on each node 2.
The super task accepu ItPC transaction via both tba~ Transnaissios Control Protocol (TCP) and the User Datagram Protocol (UDP), both of which use the Interest protocol (IP) for otansmission of data~raa~ betvvcan composer systems. UDP fs eaaneaioale:s proton>l which prlnuuily involve: multiplexing of datagtams, wheres~s TCP
:5 is a connection orientated protocol which seeks to rnsure dats~integsity is maintained.
The user tool pnsandy uses TCP/iP which, together with the RPC protocol, is provided with Sun Miaosystam's SunOs oreration system sad the VxWorkt rea! time operating system. To proterx against different central stations accessing a remote node sad making conil9cting changes to system parameters, tho user tool ptoviber infoamation on the current state of the node software before any alteration can be .
The master clocks 354 of the remote sites 2 era synchto>aiaed to the clock of the Vo~~ 93/19a4i pCT/A1,93~0~11~
_so_ central ser~~ez 47, and the systems S 1 and 450 connected to the L~tl~1491 using a network time protocol (wv°l'P), which is a staazdard UNIX utility norsnally used to synchronise the clocks of stations on a LAN. Tht: N'IT polls the remote sites 3 and on the basis of information received from the sites 2 eoncstning r~uaunisatidns between the sites and the server 47, the hTP applies offsets to the remote sites 2 so as to synchronise the sites 2 and accourn for network propagation delays, including transient network problems such as 6ink congestion.
The vehicle monitoring system is particularly advantageous as it is able to detect and discrimiztate moving vehicles from other objects, and enquire an image of selected vehicles from which they can be identified, using only elactroaie cbmeras and processing circuitry and software housed at a relents sits 2. 1"ha system eatbles automatic e~ttraction of licence plate details and does not serlttin road based equipment or markings, the emission of electromagnetic sigaala or the aeplaament of filet at the node 2.
The system is able to simultaneously track a number of vehicles on mufti-lane carriageways sad classify them by vehicle type. A high rasoludon image of a vehicle teat be obtained over a full traffic lane, the resolution sad clarity of the invention being sufficient to enable eatrxtion of the licence plate detail:. "L'l:re system can operate ~0 continuously in all ccmditions where visibility is greater than 100 metres.
using in»ared imaging techniyue~t. The high maolucion camera iaco:porata~ antibloomiag txbnOlogy to prevent pixel saturation due to vehicle haadJights, and the in~ted flash used is oonflgured so as to t;~e substantially undetectabia and inhibit flash dale.
i YS The system Gas also be controlled and initialised frond a remote ctataal station.
with images and dew being traastnitted ova a digital oommuaicatioats network.

I
The system can further be usod for a somber of puepioets. such as moaitoritng tailgating offences, road toll collertioa, sad transit lane monitoring. It ran also be 30 adapted for red light intersation monitoring.

WO 93/19441 ~ ~ ~ ~' ~ ~ ~ PC'T/At'9310~1I~
'Phe system caa also be adapted to monitor aaad acquire images of other moving objects, such as the movement of shipping containers within transport depots, and the mov emeat of objects oa as assembly tine, s

Claims (141)

CLAIMS:

1.~An object monitoring system comprising camera means characterised in that the camera means is adapted to monitor movement of an object to predetermine, based on the monitored movement of the object an acquisition time at which an image can be acquired at a predetermined position of said object relative to said camera means, and to acquire an image at the predetermined acquisition time and the predetermined position.

2. ~An object monitoring system as claimed in claim 1. wherein said camera means is adapted to detect said object and discriminate said object from static objects and other moving objects.

3. ~An object monitoring system as claimed in claim 2, wherein the camera means includes video camera means for monitoring a respective area in which objects move, and image processing means for subtracting a background image of said area from images of said area generated by said video camera means so as to produce difference images representative of moving objects in said area.

4. ~An object monitoring system as claimed in claim 3, wherein said image processing means includes classification means for forming clusters from parts of said difference images which correspond to the same moving object.

5. ~An object monitoring system as claimed in claim 4, wherein said image processing means processes each cluster to determine if it corresponds to said object and determines said acquisition time.

6. ~An object monitoring system as claimed in claim 5, wherein said image processing means filters said difference images to disregard pixels within a predetermined level range.

7. ~An object monitoring system as claimed in claim 6, wherein said image processing means includes segmentation means for processing said difference images to generate segmented images which include regions corresponding to parts of moving objects in said area that produce at least a predetermined light level at said camera means.

8. ~An object monitoring system is claimed in claim 7, wherein said classification means analyses and generates measurements of the shape of said regions and on the basis of the analysis and measurements determines valid regions and invalid regions to be disregarded.

9. ~An object monitoring system as claimed in claim 8, wherein said classification means includes clustering means for generating said clusters, each cluster comprising the valid regions which are considered to correspond to an object, said regions being extended to determine if regions overlap with and have to be combined with another to form a cluster.

10. An object monitoring system as claimed in claim 9, wherein said classification means includes labelling means for assigning a label to each cluster for each image to identify respective clusters and for matching and separating clusters over consecutive images to determine if labels are to be inherited or new labels assigned.

11. An object monitoring system as claimed in claim 10, wherein said classification means is adapted to classify said clusters as corresponding to predetermined objects by comparing characteristics of said clusters with classification data of said system, such that said classification means is adapted to identify a cluster corresponding to said object.

12. ~An object monitoring system as claimed in claim 11, including means for maintaining a histogram of said characteristics for objects monitored by said camera means and adjusting said classification data on the basis of said histogram.

13. ~An object monitoring system as claimed in claim 12, including light intensity means for monitoring a lighting level of said area, and wherein said predetermined level range, said analysis of said regions, the extension applied to said regions by said clustering means, and said classification data are adjusted depending on said lighting level.

14. ~An object monitoring system as claimed in claim 13, wherein said image processing means includes means for tracking the cluster corresponding to said object over consecutive images, comprising transformation means for transforming coordinates of said cluster to compensate for a perspective view of said camera means, and means for predicting the speed and position of said cluster for each succeeding image.

15. ~An object monitoring system as claimed in claim 14, wherein said tracking means determines said acquisition time an the basis of said predetermined position and the predicted speed and position of said cluster.

16. ~An object monitoring system as claimed in claim 15, wherein said camera-means includes image capture camera means to acquire said image of said object at said acquisition time.

17. ~An object monitoring system as claimed in claim 16, wherein the image capture: camera means is adapted to acquire a high resolution image of said object.

18. ~An object monitoring system as claimed in claim 17, wherein said video camera means has a wide field view relative to said image capture camera means, which has a limited field of view.

19. ~An object monitoring system as claimed in claim 18, including an infrared flash which is synchronised with said image capture camera means, the energy level of said flash being dependent on said lighting level.

20. ~An object monitoring system as claimed in claim 19, wherein said image capture camera means includes image sensor means and exposure control means for inhibiting saturation of said image sensor means in response to said lighting level.

21. ~An object monitoring system as claimed in claim 20, wherein said flash includes means for inhibiting the emission of visible light therefrom.

22. ~An object monitoring system as claimed in claim 21, wherein said extension applied by said clustering means is increased when said lighting level corresponds to a night condition.

23. ~An object monitoring system as claimed in claim 22, wherein said extension is less for regions corresponding to objects distant from said camera means.

24. ~An object monitoring system as claimed in claim 23, wherein said labelling means performs said matching and separating on the basis of comparing boundaries or centres of said clusters for consecutive images.

25. ~An object monitoring system as claimed in claim 24, wherein said difference images are filtered and used to update said background image.

26. ~An object monitoring system as claimed in claim 25, including means for triggering said image capture camera means at said acquisition time, comprising means for receiving and storing a number of a scan line corresponding to said acquisition time from said tracking means, means for counting scan lines of said images, and means for generating a trigger signal for said image capture camera means when said count reaches said number.

27. ~An abject monitoring system as claimed in claim 26, wherein said light intensity means generates a histogram of pixel grey levels for said images generated by said video camera means, and determines a day, night or twilight light condition on the basis of the median of said histogram.

28. ~An object monitoring system as claimed in claim 27, wherein said predetermined level range is determined on the basis of the minimum, median and peak of said histogram.

29. ~An object monitoring system as claimed in claim 28, wherein said measurements comprise circularity and coverage of said regions.

30. ~An object monitoring system as claimed in any one of claims 1 to 29, including recognition means for processing the acquired image to obtain information identifying said object.

31. ~An object monitoring system as claimed in claim 30, including a plurality of said camera means for monitoring respective areas and adapted to communicate with one another so as to transfer information on said object.

32. ~An object monitoring system as claimed in claim 30 or 31, including a plurality of said camera means for monitoring respective areas, and adapted to communicate with a central station so as to transfer information on said object.

33. ~An object monitoring system as claimed in claim 32, wherein said information on said object is acquired by at least two of said camera means and said information can be used to determine the time which said object took to travel between at least two of said areas.

34. ~An object monitoring system as claimed in claim 33, wherein said central station includes remote control means for controlling said camera means from said central station.

35. ~An object monitoring system as claimed in claim 34, wherein said central station and said plurality of camera means include respective telecommunications controllers and communicate using a digital telecommunications network.

36. ~An object monitoring system as claimed in claim 35, including means for archiving said information and allowing subsequent access thereto.

37. ~An object monitoring system as claimed in claim 36, wherein said information includes acquired images of said object and the times of acquisition, and said central station includes said recognition means.

38. ~An object monitoring system as claimed in claim 36, wherein said information includes said identifying information and the times of acquisition of acquired images of said object, and a plurality of said recognition means are connected to said plurality of camera means, respectively, at the sites of said plurality of camera means.

39. ~An object monitoring system as claimed in claim 37 or 38, wherein said recognition means is adapted to process said acquired image to locate pixels representative of characteristic pixels identifying said object.

40. ~An object monitoring system as claimed in any one of claims 1 to 39, wherein said object is a vehicle.

41. ~An object monitoring system as claimed in claim 40 when dependent on claim 30, wherein said recognition means comprises means for locating a licence plate in said image and means for determining the characters of said licence plate, said characters comprising said identifying information.

42. ~An object monitoring system as claimed in claim 41 when dependent on claim 6 wherein said predetermined level range covers pixel levels produced by shadows of said vehicle.

43. ~An object monitoring system as claimed in claim 42 when dependent on claim 8, wherein said invalid regions correspond to headlight reflections produced by said vehicles or road lane markings within said area.

44. An object monitoring system as claimed in any one of claims 40 to 43, wherein said vehicle is a large vehicle, such as a bus or truck.

45. A monitoring system as claimed in any of claims 1 to 44, wherein said object is a vehicle and wherein said camera means is adapted to monitor said vehicle to detect a law infringement acid captures an image of said vehicle at said predetermined time in response to detecting said infringement.

46. A monitoring system as claimed in claim 45, including recognition means for processing said images to obtain information identifying said vehicle.

47. A monitoring system according to claim 1, wherein the camera means generates images of an area and acquires an image of a predetermined object, including image processing means having:
means for subtracting a background image of said area from said images of said area to generate difference images representative of moving objects in said area;
segmentation means for processing said difference images to generate region images representative of regions corresponding to parts of said moving objects in said area;
classification means for processing said region images, said classification means including means for analysing the shape of said regions and, on the basis of the analysis, determining valid regions and invalid regions, clustering means for generating, on the basis of said valid regions, clusters corresponding to respective ones of said moving objects, and means for classifying said clusters. by comparing at least one characteristic of said clusters to classification data of said system to determine if one of said clusters corresponds to said predetermined object;
and tracking means for tracking said one of said clusters corresponding to said predetermined object to determine an image acquisition time for acquiring said image of said predetermined object.

48. ~An object monitoring system as claimed in claim 47, wherein said image processing means filters said difference images to disregard pixels within a predetermined intensity range.

49. ~An object monitoring system as claimed in claim 48, wherein said parts of moving objects correspond to at least a predetermined light level received at said camera means.

50. ~An object monitoring system as claimed in claim 49, wherein said classification means extends said valid regions to determine if said regions have to be combined to form said clusters.

51. ~An object monitoring system comprising:
camera means for generating images of an area and for acquiring an image of a predetermined object, and image processing means including:
means for subtracting a background image of said area from said images of said area to generate difference images representative of moving objects in said area;
segmentation means for processing said difference images to generate region images representative of regions corresponding to parts of said moving objects in said area;

classification means for processing and classifying said region images, said classification means including:
means for analyzing the shape of said regions and, on the basis of the analysis, determining valid regions and invalid regions, clustering means for rejecting said invalid regions and generating, on the basis of the geometry of said valid regions, clusters corresponding to respective ones of said moving objects, and means for classifying said clusters by comparing at least one characteristic of said clusters to classification data of said system to determine if one of said clusters corresponds to said predetermined object; and tracking means for tracking said one of'said clusters corresponding to said predetermined object to trigger said camera means to acquire said image of said predetermined object.

52. ~An object monitoring system as claimed in claim 51, wherein said image processing means filters said difference images to disregard pixels within a predetermined intensity range.

53. ~An object monitoring system as claimed in claim 52, wherein said parts of moving objects correspond to at least a predetermined light level received at said camera means.

54. ~An object monitoring system as claimed in claim 53, wherein said classification means extends said valid regions to determine if said valid regions have to be combined to form said clusters.

55. ~An object monitoring system as claimed in claim 54, wherein said classification means includes labeling means for assigning labels to clusters, respectively, for each one of said images of said area to identify said clusters, and for matching clusters over consecutive ones of said images of said area to determine if labels are to be inherited or new labels assigned.

56. ~~An object monitoring system as claimed in claim 55, including means for maintaining a histogram of said at least one characteristic of said clusters, and adjusting said classification data on the basis of said histogram.

57. ~~An object monitoring system as claimed in claim 56, including light intensity means for determining a lighting level of said area, and wherein said predetermined intensity range, said analysis of said regions, extension applied to said valid regions by said classification means, and said classification data are adjusted depending on said lighting level.

58. ~~An object monitoring system as claimed in claim 57, wherein said tracking means includes transformation means for transforming coordinates of said one of said clusters to compensate for a perspective view of said camera means, and means for predicting the speed and position of said one of said clusters for each succeeding image of said images of said area.

59. ~~An object monitoring system as claimed in claim 57, wherein said tracking means determines an image acquisition time on the basis of an image capture position and the position of said one of said clusters.

60. ~~An object monitoring system as claimed in claim 59, wherein said camera means includes video camera means for monitoring said moving objects and image capture camera means to acquire said image of said predetermined object at said acquisition time, said image being a high resolution image of said predetermined object.

61. ~~An object monitoring system as claimed in claim 60, wherein said video camera means has a wide field view relative to said image capture camera means, which has a limited field of view.

62. ~~An object monitoring system as claimed in claim 61, including an infrared flash which is synchronized with said image capture camera means, the energy level of said flash being dependent on said lighting level.

63. ~~An object monitoring system as claimed in claim 62, wherein said image capture camera means includes image sensor means and exposure control means for inhibiting saturation of said image sensor means in response to said lighting level.

64. ~~An object monitoring system as claimed in claim 63, wherein said flash includes means for attenuating the emission of visible light therefrom.

65. ~~An object monitoring system as claimed in claim 64, wherein said extension applied by said valid regions is increased when said lighting level corresponds to a night condition.

66. ~~An object monitoring system as claimed in claim 65, wherein said extersion is less for valid regions corresponding to said moving objects distant from said camera means.

67. ~~An object monitoring system as claimed in claim 66, wherein said labeling means performs said matching on the basis of comparing boundaries of said clusters for said consecutive ones of said images of said area.

68. ~~An object monitoring system as claimed in claim 67, wherein said difference images are filtered and used to update said background image.

69. ~~An object monitoring system as claimed in claim 68, including means for triggering said image capture camera means at said acquisition time, comprising means for receiving and storing a number of scan lines corresponding to said acquisition time from said tracking means, means for counting scan lines of said images of said area, and means for generating a trigger signal for said image capture camera means when said count reaches said number.

70. ~~An object monitoring system as claimed in claim 68, wherein said light intensity means generates a histogram of pixel grey levels for said images generated by said video camera means, and determines a day, night or twilight light condition on the basis of the median of said histogram.

71. ~~An object monitoring system as claimed in claim 70, wherein said predetermined. intensity range is determined on the basis of the minimum, median and peak of said histogram.

72. ~~An object monitoring system as claimed in claim 71, wherein said analysis includes determining circularity and coverage of said valid and invalid regions.

73. ~~An object monitoring system as claimed in claim 66, wherein said labeling means performs said matching on the basis of comparing centres of said clusters for said consecutive: one of said images of said area.

74. ~~An object monitoring system as claimed in claim 57, wherein said camera means includes exposure control means for adjusting camera exposure on the basis of said lighting level.

75. ~~An object monitoring system as claimed in claim 51, including recognition means for processing said image of said predetermined object to obtain information identifying said predetermined object.

76. ~~An object monitoring system as claimed in claim 7.5, including a plurality of said camera means and image processing means for monitoring a plurality of areas, respectively, and being adapted to communicate with one another so as to transfer information on said predetermined object, said areas being remote with respect to one another.

77. ~~An object monitoring system as claimed in claim 75, including a plurality of said camera means and image processing means for monitoring a plurality of areas, respectively, and being adapted to communicate with a central station so as to transfer information on said predetermined object, said areas being remote with respect to one another.

78. ~~An object monitoring system as claimed in claim 77, wherein said information on said predetermined abject is acquired by at least two of said camera means and image processing means and said information can be used to determine the time which said predetermined object took to travel between at least two of said areas.

79. ~~An object monitoring system as claimed in claim 78, wherein said central station includes remote control means for controlling said plurality of said camera means and image processing means from said central station.

80. ~An abject monitoring system as claimed in claim 79, wherein said central station and said plurality of camera means and image processing means include respective telecommunications controllers and communicate using a digital telecommunications network.

81. ~An object monitoring system as claimed in claim 80, including means for archiving said information and allowing subsequent access thereto.

82. ~An object monitoring system as claimed in claim 81, wherein said information includes acquired images of said predetermined object and the times of acquisition, and said central station includes said recognition means.

83. ~An object monitoring system as claimed in claim 82, wherein said recognition means is adapted to process said image of said predetermined object to locate pixels representative of'characteristic pixels identifying said object.

84. ~An object monitoring system as claimed in claim 81, wherein said information includes said identifying information and the times of acquisition of acquired images of said predetermined abject, and a plurality of said recognition means are connected to said plurality of camera means and said image processing means, respectively, at the sites of said plurality of camera means and image processing means.

85. ~An object monitoring system as claimed in claim 78, wherein the objects are vehicles.

86. ~An object monitoring system as claimed in claim 85, wherein said recognition means comprises means for locating a licence plate in said image of said predetermined object and means for determining the characters of said licence plate, said characters comprising said identifying information.

87. ~~An object monitoring system as claimed in claim 86, wherein said predetermined intensity range covers pixel intensities produced by shadows of said vehicles.

88. ~~An object monitoring system as claimed in claim 87, wherein said invalid regions correspond to headlight reflections produced by said vehicles.

89. ~~An object monitoring system as claimed in claim 88, wherein said predetermined object is a large vehicle, such as a bus or truck.

90. ~~An object monitoring system as claimed in claim 89, wherein said central station includes remote control means for controlling said plurality of camera means and image processing means from said central station.

91. ~~An object monitoring system as claimed in claim 90, wherein said central station and said plurality of camera means and image processing means include respective telecommunications controllers and communicate using a digital telecommunications network.

92. ~~An object monitoring system as claimed in claim 91, including means for archiving said information and allowing subsequent access thereto.

93. ~~An object monitoring system as claimed in claim 92, wherein said information includes said high resolution image and the time of acquisition, and said central station includes said recognition means.

94. ~~An object monitoring system as claimed in claim 92, wherein said information includes said identifying information and the time of acquisition of said high resolution image, and a plurality of said recognition means are connected to said plurality of camera means and image processing means, respectively, at the sites of said plurality of camera means and image processing means.

95. ~~An object monitoring system as claimed in claim 93, wherein said recognition means is adapted to process said high resolution image to locate pixels representative of characteristic pixels identifying said predetermined object.

96. ~~An object monitoring system as claimed in claim 87, wherein said invalid regions correspond to road lane markings within said area.

97. ~~An object monitoring system as claimed in claim 51, wherein said classification means generates and operates on the basis of region feature vectors representative of said regions and cluster feature vectors representative of said clusters.

98. ~~An object monitoring system comprising:
camera means for generating images of an area and for acquiring an image of a predetermined object;
image processing means including:
means for subtracting a background image of said area from said images of said area to generate difference images representative of moving objects in said area, segmentation means for processing said difference images to generate region images representative of regions corresponding to parts of said moving objects in said area, classification means for processing said region images, said classification means including:
means for analyzing the shape of said regions and, on the basis of the analysis, determining valid regions and invalid regions, clustering means for rejecting said invalid regions and generating, on the basis of the geometry of said valid regions, clusters corresponding to respective ones of said moving object, and means for classifying said clusters by comparing at least one characteristic of said clusters to classification data of said system to determine if one of said clusters corresponds to said predetermined object, and tracking means for tracking said one of said clusters corresponding to said predetermined object to trigger said camera means to acquire said image of said predetermined object; and extraction means for processing said image of said predetermined object to extract information identifying said predetermined object.

99. ~~An object monitoring system as claimed in claim 98, including means for transmitting said image of said predetermined object over a digital telecommunications network.

100. ~An object monitoring system as claimed in claim 98, including a plurality of said camera means and image processing means for monitoring a plurality of areas, respectively, said areas being remote with respect to one another, and means for comparing said information respectively obtained at said areas.

101. ~An object monitoring system as claimed in claim 98, wherein the objects are vehicles.

102. ~An object monitoring system as claimed in claim 98, wherein said classification means generates and operates on the basis of region feature vectors representative of said regions and cluster feature vectors representative of said clusters.

103. ~A vehicle monitoring system comprising:
camera means for generating images of a carriageway and for acquiring images of predetermined vehicles, and image processing means including:
means for subtracting a background image of said carriageway from said images of said carriageway to generate difference images representative of moving vehicles on said carriageway;
segmentation means for processing said difference images to generate region images representative of regions corresponding to parts of said moving vehicles on said carriageway;
classification means for processing said region images, said classification means including:
means for analysing the shape of said regions and, on the basis of the analysis, determining valid regions and invalid regions, clustering means for rejecting said invalid regions and generating, on the basis of the geometry of said valid regions, clusters corresponding to respective ones of said moving vehicles, and means for classifying said clusters by comparing at least one characteristic. of said clusters to classification data of said system to determine if said clusters correspond to said predetermined vehicles; and tracking means for tracking said clusters corresponding to said predetermined vehicles to trigger said camera means to acquire said images of said predetermined vehicles.

104. ~~A vehicle monitoring system as claimed in claim 103, wherein said camera means includes video camera means for monitoring said carriageway and a plurality of image capture camera means for acquiring said images of said predetermined vehicle for respective lanes of said carriageway.

105. A vehicle monitoring system as claimed in claim 104, wherein said images of said predetermined vehicles are high resolution images covering the width of a lane of said carriageway and enable optical character recognition means to extract licence plate characters of said predetermined vehicles.

106. A vehicle monitoring system as claimed in claim 103, including optical character recognition means for processing said images of said predetermined vehicles to extract licence plate characters identifying said predetermined vehicles.

107. A vehicle monitoring system as claimed in claim 103, wherein said classification means generates and operates on the basis of region feature vectors representative of said regions and cluster feature vectors representative of said clusters.

108. A vehicle monitoring system as claimed in claim 103, wherein said images of said predetermined vehicles are high resolution images covering the width of a lane of said carriageway and enable optical character recognition means to extract licence plate characters of said predetermined vehicles.

109. A vehicle monitoring system comprising:
a plurality of camera means for generating images of respective areas and for acquiring images of predetermined vehicles, said areas being remote with respect to one another; and a plurality of image processing means including:

means for subtracting background images of said areas from said images of said areas to generate difference images representative of moving vehicles in said areas;
segmentation means for processing said difference images to generate region images representative of regions corresponding to parts of said moving vehicles in said area;
classification means for processing said region images, said classification means including means for analyzing the shape of said regions and, on the basis of the analysis, determining valid regions and invalid regions, clustering means for rejecting said invalid regions and generating, on the basis of the geometry of said valid regions, clusters corresponding to respective ones of said moving;
vehicles, and means for classifying said clusters by comparing at least one characteristic of said clusters to classification data of said system to determine if said clusters corresponds to said predetermined vehicles;
tracking means for tracking said clusters corresponding to said predetermined vehicles to trigger said camera means to acquire said images of said predetermined vehicles; and recognition means for processing said images of said predetermined vehicles to obtain information identifying said predetermined vehicles.

110. A vehicle monitoring system as claimed in claim 109, including means for comparing said information obtained to determine the average speed between at least two of said areas of at least one of said predetermined vehicles.

111. A vehicle monitoring system as claimed in claim 109, wherein said classification means generates and operates on the basis of region feature vectors representative of said regions and cluster feature vectors representative of said clusters.

112. A vehicle monitoring system comprising:
camera means for generating images of an area and for acquiring an image of a vehicle associated with a law infringement, and image processing means including:
means for subtracting a background image of said area from said images of said area to generate difference images representative of moving vehicles in said area;
segmentation means for processing said difference images to generate region images representative of regions corresponding to parts of said moving vehicles in said area;
classification means for processing said region images, said classification means including:
means for analyzing the shape of said regions and, on the basis of the analysis, determining valid regions and invalid regions, clustering means for rejecting said invalid regions and generating, on the basis of the geometry of said valid regions, clusters corresponding to respective ones of said moving vehicles, and means for detecting said law infringement by comparing at least one characteristic of said clusters to classification data of said system to determine if one of said clusters corresponds to said vehicle; and tracking means for tracking said one of said clusters corresponding to said vehicle to trigger said camera means to acquire said image of said vehicle.

113. A vehicle monitoring system as claimed in claim 112, including recognition means for processing said image of said vehicle to obtain information identifying said vehicle.

114. A vehicle monitoring system as claimed in claim 112, wherein said classification means generates and operates on the basis of region feature vectors representative of said regions and cluster feature vectors representative of said clusters.

115. A vehicle monitoring system comprising camera means for generating images of a carriageway and for acquiring high resolution images of large vehicles, such as trucks and buses, and image processing means including:
means for subtracting a background image of said carriageway from said images of said carriageway to generate difference images representative of moving vehicles of said carriageway;
segmentation means for processing said difference images to generate region images representative of regions corresponding to parts of said moving vehicles on said carriageway;
classification means for processing said region images, said classification means including:
means for analyzing the shape of said regions and, on the basis of the analysis, determining valid regions and invalid regions, clustering means for rejecting said invalid regions and generating, on the basis of the geometry oil said valid regions, clusters corresponding to respective ones of said moving vehicles, and means for classifying said clusters by comparing of at least one;
characteristic of said clusters to classification data of said system to determine if said clusters correspond to said large vehicles; and tracking means for tracking said clusters corresponding to said large vehicles to trigger said camera means to acquire said high resolution images of said large vehicles.

116. A vehicle monitoring system as claimed in claim 115, including recognition means for automatically extracting information on said large vehicles, such as licence plate characters, from said high resolution images.

117. A vehicle monitoring system as claimed in claim 115, wherein said classification means generates and operates on the basis of region feature vectors representative of said regions and cluster feature vectors representative of said clusters.

118. An object monitoring system comprising:
video camera means for generating images of an area to monitor moving objects in said area;
image capture camera means for acquiring a high resolution image of a predetermined object; and image processing means including:
means for subtracting a background image of said area from said images of said area to generate difference images representative of said moving objects in said area;
segmentation means for processing said difference images to generate region images representative of regions corresponding to parts of said moving objects in said area;
classification means for processing said region images, said classification means including:
means for analyzing the shape of said regions and, on the basis of the analysis, determining valid regions and invalid regions, clustering means for rejecting said invalid regions and generating, on the basis of the geometry of said valid regions, clusters corresponding to respective ones of said moving objects, and means for classifying said clusters by comparing at least one characteristic of said clusters to classification data of said system to determine if one of said clusters corresponds to said predetermined object; and tracking means for tracking said one of said clusters corresponding to said predetermined object to trigger said image capture means to acquire said high resolution image of said predetermined object.

119. The object monitoring system of claim 118 further including a plurality of said video camera means and image processing means for monitoring a plurality of said areas, respectively, said areas being remote with respect to one another, and means for comparing said information respectively obtained at said areas.

120. The object monitoring system of claim 118, wherein the predetermined object is a vehicle and said area is a carriageway.

121. The object monitoring system of claim 120 further including a plurality of image capture camera means for acquiring said image of said vehicle in respective lanes of said carriageway.

122. The object monitoring system of claim 118 further including recognition means for processing said image of said predetermined object to obtain information identifying said object.

123. An object monitoring system comprising:
video camera means for generating images of an area to monitor moving objects in said area;
image capture camera means for acquiring a high resolution image of a predetermined object; and image processing means including:
means for subtracting a background image of said area from said images of said area to generate difference images representative of said moving objects in said area;
segmentation means for processing said difference images to generate region images representative of regions corresponding to parts of said moving objects in said area;

classification means for processing said region images, said classification means including means for analyzing the shape of said regions and, on the basis of the analysis, determining valid regions and invalid regions, clustering means for rejecting said invalid regions and generating, on the basis of the geometry of said valid regions, clusters corresponding to respective ones of said moving objects, and means for classifying said clusters by comparing at least one characteristic of said clusters to classification data of said system to determine; if one of said clusters corresponds to said predetermined object;
tracking means for tracking said one of said clusters corresponding to said predetermined object to trigger said image capture camera means to acquire said high resolution image of said predetermined object; and extraction means for processing said image of said predetermined object to extract information identifying said predetermined object.

124. The object monitoring system of claim 123, including a plurality of said video camera means and image processing means for monitoring a plurality of areas, respectively, said areas being remote with respect to one another, and means for comparing said information respectively obtained at said areas.

125. The object monitoring system of claim 123, wherein the predetermined object is a vehicle and said area is a carriageway.

126. The object monitoring system of claim 125 further including a plurality of image capture camera means for acquiring said image of said vehicle in respective lanes of said carriageway.

127. The object monitoring system of claim 123 further including recognition means for processing said image of said predetermined object to obtain information identifying said object.

128. A vehicle monitoring system comprising:
video camera means for generating images of a carriageway to monitor moving vehicles in said carriageway:
image capture camera means for acquiring a high resolution image of a predetermined vehicle; and image processing means including:
means for subtracting a background image of said carriageway from said images of said carriageway to generate difference images representative of said moving vehicles on said carriageway;
segmentation means for processing said difference images to generate region images representative of regions corresponding to parts of said moving;
vehicles on said carriageway;
classification means for processing said region images, said classification means including:
means for analyzing the shape of said regions and, on the basis of the analysis, determining valid regions and invalid regions, clustering means for rejecting said invalid regions and generating, on the basis of the geometry of said valid regions, clusters corresponding to respective ones of said moving vehicles, and means for classifying said clusters by comparing at least one characteristic of said clusters to classification data of said system to determine if said clusters correspond to said predetermined vehicle; and tracking means for tracking said clusters corresponding to said predetermined vehicle to trigger said image capture camera means to acquire said high resolution image of said predetermined vehicle.

129. The vehicle monitoring system of claim 128, including a plurality of said video camera means and image processing means for monitoring a plurality of carriageways, respectively, said carriageways being remote with respect to one another, and means for comparing said information respectively obtained at said carriageways.

130. The vehicle monitoring system of claim 128 further including a plurality of image capture camera means for acquiring said image of said predetermined vehicle in respective lanes of said carriageway.

131. The vehicle monitoring system of claim 128 further including recognition means for processing said image of said predetermined vehicle to obtain information identifying said vehicle.

132. A vehicle monitoring system comprising:
a plurality of video camera means for generating images of respective areas to monitor moving vehicles in said area, said areas being remote with respect to one another;
a plurality of image capture camera means for acquiring a high resolution image of one or more predetermined vehicles; and a plurality of image processing means including:
means for subtracting background images of said areas from said images of said areas to generate difference images representative of said moving vehicles in said areas;
segmentation means for processing said difference images to generate region images representative of regions corresponding to parts of said moving vehicles in said areas;
classification means for processing said region images, said classification means including:

means for analyzing the shape of said regions and, on the basis of the analysis, determining valid regions and invalid regions, clustering means for rejecting said invalid regions and generating, on the basis of the geometry of said valid regions, clusters corresponding to respective ones of said moving vehicles, and means for classifying said clusters by comparing at least one characteristic of said clusters to classification data of said system to determine if said clusters correspond to said predetermined vehicle;
tracking means for tracking said clusters corresponding to said predetermined vehicle to trigger said camera means to acquiring said image of said predetermined vehicle; and recognition means for processing said images of said predetermined vehicle to obtain information identifying said predetermined vehicle.

133. The vehicle monitoring system of claim 132, wherein said area is a carriageway, and said image capture camera means acquire said images of said predetermined vehicle in respective lanes of said carriageway.

134. A vehicle monitoring system comprising:
video camera means for generating images of an area to monitor moving vehicles in said area;

image capture camera means for acquiring a high resolution image of a vehicle associated with a law infringement; and image processing means including:
means for subtracting a background image of said area from said images of said area to generate difference images representative of said moving vehicles in said area;
segmentation means for processing said difference images to generate region images representative of regions corresponding to parts of said moving vehicles in said area;

classification means for processing said region images, said classification means including means for analyzing the shape of said regions and, on the basis of the analysis, determining valid regions and invalid regions, clustering means for rejecting said invalid regions and generating, on the basis of the geometry of said valid regions, clusters corresponding to respective ones of said moving vehicles, and means for detecting said law infringement by comparing at least one characteristic of said clusters to classification data of said system to determine if one of said clusters corresponds to said vehicle; and tracking means for tracking said one of said clusters corresponding to said vehicle to trigger said image capture camera means to acquire said high resolution image of said vehicle.

135. The vehicle monitoring system of claim 134, including a plurality of said video camera means and image processing means for monitoring a plurality of areas, respectively, said areas being remote with respect to one another, and means for comparing said information respectively obtained at said areas.

136. The vehicle monitoring system of claim 134 further including a plurality of image capture camera means for acquiring said image of said vehicle in respective lanes of said carriageway.

137. The vehicle monitoring system of claim 134 further including recognition means for processing said image of said vehicle associated with said law infringement to obtain information identifying said vehicle.

138. A vehicle monitoring system comprising:

video camera means for generating images of a carriageway to monitor moving vehicles in said area;
image capture camera means for acquiring a high resolution image of a large vehicle, such as a truck and a bus; and image processing means including:
means for subtracting a background image of said carriageway from said images of said carriageway to generate difference images representative of said moving vehicles on said carriageway;
segmentation means for processing said difference images to generate region images representative of regions corresponding to parts of said moving vehicles on said carriageway;
classification means for processing said region images, said classification means including:
means for analyzing the shape of said regions and, on the basis of the analysis, determining valid regions and invalid regions, clustering means for rejecting said invalid regions and generating, on the basis of the geometry of said valid regions, clusters corresponding to respective ones of said moving vehicles, and means for classifying said clusters by comparing at least one characteristic of said clusters to classification data of said system to determine if said clusters correspond to said large vehicle; and tracking means for tracking said clusters corresponding to said large vehicle to trigger said image capture camera means to acquire said high resolution image of said large vehicle.

139. The vehicle monitoring system of claim 138, including a plurality of said video camera means and image processing means for monitoring a plurality of carriageways, respectively, said carriageways being remote with respect to one another, and means for comparing said information respectively obtained at said carriageways.

140. The vehicle monitoring system of claim 138 further including a plurality of image capture camera means for acquiring said image of said large vehicle in respective lanes of said carriageway.

141. The vehicle monitoring system of claim 138 further including recognition means for processing said image of said large vehicle to obtain information identifying said vehicle.