WO2002041255A1 - Improved two-image cropping system and method and the use thereof in the digital printing and surveillance field - Google Patents

Improved two-image cropping system and method and the use thereof in the digital printing and surveillance field Download PDF

Info

Publication number
WO2002041255A1
WO2002041255A1 PCT/EP2000/011508 EP0011508W WO0241255A1 WO 2002041255 A1 WO2002041255 A1 WO 2002041255A1 EP 0011508 W EP0011508 W EP 0011508W WO 0241255 A1 WO0241255 A1 WO 0241255A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
video
camera
background
images
Prior art date
Application number
PCT/EP2000/011508
Other languages
French (fr)
Inventor
Ico Thieme
Original Assignee
Orlean Holding N.V.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Orlean Holding N.V. filed Critical Orlean Holding N.V.
Priority to AU2001215223A priority Critical patent/AU2001215223A1/en
Priority to PCT/EP2000/011508 priority patent/WO2002041255A1/en
Priority to US09/834,920 priority patent/US20010055414A1/en
Publication of WO2002041255A1 publication Critical patent/WO2002041255A1/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/002D [Two Dimensional] image generation
    • G06T11/60Editing figures and text; Combining figures or text
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/20Analysis of motion

Definitions

  • the present invention relates to improvements in and/or related to a two-image cropping system and method and the application thereof to the digital printing and surveillance field, accordmg to the preambles of claims 1, 2 and 7,8, respectively.
  • the novel two-image cropping technique disclosed thereby has been used for digitally printing a composite image, for example a card, with the face of a user and/or other alternative printed products incorporated therein, such as m visiting/business cards or bills with photographs, greeting cards with photographs, stickers and so on.
  • the inventor has found, after a lot of tests under very different conditions, that such a two-image cropping system and method, and the related software, could be, on one side, improved, and that, on the other side, after the application of said improvements, the two-image cropping technique could be used in fields different from the digital printing field, for example in the spatial surveillance or safety system field.
  • the inventor has found that, by arranging the system in crowded places, upon a continuous movement of persons inside and outside of the video camera surveillance field, the presence sensor, operating based on the microwave technology, sensed the continuous displacements of the persons, even if they were outside of the video camera shooting field, thereby preventing a "clean" reference background from being taken.
  • the difference of the microwave technology used in the presence sensor, and which is based on the presence of a mass, such as the physical body of a person, and the "visual perception" of the system, based on the detection of the images by the video camera, as for the human vision, cold affect the reliability of the two-image cropping system in the mentioned system installation condition, i.e. in crowded spaces continuously traversed by persons passing through the video camera shooting field and/or the adjoining regions.
  • the outside presence sensor which constitutes per se a physical component, or a hardware component of the system, must substantially meet two requirements, and more specifically: 1) it must respect and functionally occupy, as far as possible, the video camera overshooting field, and 2) it must discriminate the same situations seen by the video camera.
  • a parallax problem related to the video camera optical overshooting field occurred, with a consequent impossibility of "focalizing" the sensitivity lobe of the microwave sensor with respect to the taking field of the video camera.
  • a first object of the present invention is to provide a novel technique or method for detecting a presence of intruding persons or bodies, without using prior presence microwave sensors, which novel detecting technique or method can be carried out by a program module or software and by using the system disclosed in the previous application.
  • a further object of the invention is to simultaneously use the software program of said technique or method, to also simplify and speed up the prior managing software, by eliminating therefrom the prior provided Backlni.exe and BackBuilkd.exe programs.
  • Another object of the present invention is to suggest firther application fields for said software and tlie system for the accomplishment thereof, optionally hi an advantageously simplified fonn.
  • Yet another object of the present invention is to allow the improved software to be also used for broadenmg tlie printed article types to be obtained by both the system accordmg to the previous application and the system according to tlie present invention.
  • the optical detection of intruder presence by software which can be autonomously performed by the system, allows to fully eliminate both the prior detector, per se, i.e. made as a hardware component, and the drawbacks related to the operation thereof.
  • Another important advantage is that the provision of the above mentioned novel algorithm has allowed an indirect and immediate development of the software in fields different from the digital printing of a composite image, for example in the spatial surveillance and safety field.
  • the BackGenerator.exe program module according to the present invention fully replaces the two Backlni.exe and BackBuild.exe modules, since the functions carried out by these two programs have been embedded or incorporated in the novel BackGenerator.exe software module.
  • Figure 1 illustrates a typical sensitivity lobe of a microwave sensor
  • Figure 2 illustrates the system video camera overshooting field, as tending to infinite
  • Figure 3 illustrates the parallax phenomenon related to the use of the mixed detection technique provided in PCT/EP 00/ 03389;
  • Figure 4 illustrates the background updating or refreshment at the moment of the BackGenerator, and the building of a "virtual reference background" (16B1);
  • Figure 5 illustrates as a detail the composition of the "virtual background" (16B1)
  • Figure 6 illustrates the new cycle for eliminating the backgrounds (16B1)
  • Figure 7 is a schematic general block diagram of a simplified surveillance and safety or security system according to the present invention
  • Figure 8 illustrates the inside of a store being surveilled or monitored by the surveillance and safety or security system according to the present invention
  • Figure 9 illustrates a monitoring and surveilling room of the store shown in figure 8, according to the prior art
  • Figure 10 illustrates a "sample image" which can be stored in the system as a “reference background” or as a “first image”;
  • Figure 11 illustrates an image as cyclically provided by the video-camera, or "second image” and which is automatically compared with the reference image or "first image” of figure 10;
  • Figure 12 illustrates a result of an analysis between the "second image” and "first image", which, after having performed the cropping, shows the presence of remaining areas indicating a presence of an intruder;
  • Figure 13 shows a color changing of the surveillance monitor screen and the displaying thereon of the remaining or residual areas after the cropping, or of the intruder;
  • Figure 14 shows a flow chart illustrating the operation mode of the surveillance and safety system and method according to the present invention.
  • Figure 15 is a schematic general block diagram of the system, shown by a dash and double-dots frame and including a first electronic component assembly, known per se, shown by a dash and single-point frame, and an additional electronic component assembly, shown by a dashed frame;
  • Figures 16A to 16E schematically illustrate, by way of an example, a "panorama" and tlie operating steps for making a composite card by the system and method according to PCT/EP 00/03389 reference;
  • Figure 17 conceptually illustrates the files provided for forming the time "queue” of the scratchpad at the starting of the system, corresponding to the "overshoot background";
  • Figure 18 schematically illustrates tl e background back slipping principle to carry out the self-updating step of the "reference background"
  • Figure 19 illustrates, by way of an example, the background interpolation function principle as applied to a "twin" image, of the "background with subject” image, to update the “reference background” as the "background with subject” image is taken, as well as to eliminate any transient noises from the "overshoot backgrounds”.
  • the BackGenerator.exe program operates as follows: For verifying that no disturbing persons or elements are arranged before the apparatus or system, two overshootings or images are remotely taken, for example, at 1 second from one another, by using the same video camera 18 of the system.
  • the two images are chromatically compared with respect to their pixels, i.e. each individual pixel of the first overshooting or image is measured and compared with the pixel at the same position of the second overshooting or image. If the chromatic difference would be less than a preset given tolerance, then said pixel would be judged as the same, otlierwise said pixel being marked as different.
  • the different pixels are less than a given tolerance (for example 200, with reference to a total pixel number of over 442,000 of the whole image), then it will be judged that no variations of the two images have occurred and that, accordingly, before the video camera no person is arranged (who could not remain absolutely static), and that any disturbing elements, such as casted shadows (i.e. persons who are directly arranged hi tlie visual field of the video camera optics system but provide light interferences), or light reflections either of a direct or of a mirror or polished element reflected type are present.
  • a given tolerance for example 200, with reference to a total pixel number of over 442,000 of the whole image
  • the system will switch the illm ⁇ inath g system on and will take two other overshootings, spaced by 1 second from one another. These two images too are analyzed by tlie same technique to verify that, in the meanwhile, no disturbing element has entered the visual field of the video camera lens.
  • the system will continue to take overshootings or images, at a distance of 1 second from one another, while performing a comparing thereof so as to found a pair of overshootings or images, without a difference greater than the provided tolerance (for example 200).
  • the system will provide a signal, such as an acoustic signal or warning signal, and open a window on the monitor including a short message asking the persons near the video-camera to move away, while informing said persons that their moving away would be necessary to perform a periodic self maintenance operation, or allow the system to properly operate.
  • a signal such as an acoustic signal or warning signal
  • the virtual reference background, figure 16B1 is now caused to backward slip or slide by two positions (figure 6), together with all the old backgrounds, with the exception of the Back5 background, which is now affected only by the BackGenerator.exe module, and accordingly by updated images, figure 6, which operation occurred, in the shown example, at 16.00 hours.
  • a subject overshooting operation for forming a card is performed.
  • the image taken by the video camera is stored in the BackO background, and the background interpolation 0 function is started for summarily eliminating the subject areas, and then replacing them by those areas arranged at the same position, coming from the background.
  • a reference virtual background, figure 16B1 is formed, in which the image portion not covered by the subject is updated at the overshooting time, whereas the portion "masked" by the subject must be recovered from a previous information (Backl ⁇ Back4).
  • This virtual reference background constitutes the image which will be used by the cropping algorithm in order to discriminate the "subject" areas from the "background” areas.
  • Two are the innovations introduced into the software module representing the interface to the client, i.e. into the TheMask.exe module.
  • the first allows to take decisions related to the "card” and “greetings bill” products, as novel printed forms/patterns.
  • the cards were printed by the so called “live printing” method, in which the image occupied tl e overall surface on the card.
  • the user is now provided with the possibility of choosing the end product according to three patterns, for example: 1) with the prior live printed pattern or 2) with a frame shaped perimetrical edge or 3) with a frame shaped perimetrical edge and a caption at the bottom portion of the card, greetings bill or the like.
  • the second innovation is related to the so-called "stickers” and "visiting bills or cards”, in which, according to the invention, it is now possible to select if tlie photo pagination must be vertical (a commercial form) or horizontal, thereby admitting the presence of two persons simultaneously, for example a husband-wife pair, a friend set and so on.
  • the two-image cropping technique has as the main principle of performing a comparative analysis of two images in order to establish their differences.
  • figure 16C By the above disclosed operation set, it is possible to identify different areas within an image being analyzed, figure 16C, with respect to a reference image, fig. 16B.
  • this identifying mechanism related to the analyzed image variations can be used in principle, according to tlie invention, in all the fields in which it would be necessary to perform an image automatic analysis for different purposes.
  • an area 40 such as the inside area of a goods store, monitored by one or more video-cameras, such as four video-cameras, not shown, figure 8, in which the case or box 42 number herein provided and the video-camera number, and related monitors 43, figures 9, make difficult for a monitoring operator 44 to safely control the overall area 40, will be hereinafter disclosed.
  • a possible intruder could not be easily dete ⁇ tqek . as he/she moves through the large boxes 42 by concealing therebehind. If the surveillance operator does not observe the related monitor at the intruder movement instant, then the surveillance operator will not be able of detecting the presence of the intruder who could operate in a rather free manner.
  • the surveillance and safety system accordmg to the invention is simplified in comparison to the system of the previous application and is adapted to analyze the image supplied by one or more video cameras and detect the moving intruder bodies, independently from the image complexity or the presence of objects through the area being monitored, such as frirniture pieces, vehicles and the like.
  • the simplified surveillance and safety system comprises a PC 13, a video acquisition board 16, a monitor 17 and one or more video camera 18, for example of the type described in the previous application.
  • a “sample image” is at first overshoot or “captured”, for example at the safety system energizing moment, and this "sample image” is stored in the system as a "reference background", figure 10, for example as shown for a safe box 45 in a room 46.
  • control monitor 43 (a single monitor being advantageously provided) of the video camera/video cameras, will provide the normal image taken through the environment 46, figure
  • the image is stored, figure 12, together with the event hour and its place, for example the room access door area.
  • the surveilling operator 44 can immediately display, in the control room, the image of the intruder 47.
  • the time for taking or "capturing" the reference image having the best safety characteristics is the alarm actuating or energizing time.
  • an image storing cycle would be provided, to operate as a "reference image" for the cropped pattern, figure 10, with a typical time which can vary, for example, from 30 to 600 seconds, depending on the environment variation degree, and the area variation analysis will be performed on a "reference background", figure 10, related to few minutes before the video- camera overshooting or taking time.
  • a typical time which can vary, for example, from 30 to 600 seconds, depending on the environment variation degree, and the area variation analysis will be performed on a "reference background", figure 10, related to few minutes before the video- camera overshooting or taking time.

Abstract

An improved two-image cropping system and a method for the digital printing field, for example for making and digitally editing a composite image, for example a picture card, with a face of a user incorporated therein in which a presence sensor is used which is in the form of a software operating through the video-camera (18) of the system. The optical software operated presence sensor also allows the use thereof in a simplified two-image cropping system for the surveillance and safety field.

Description

IMPROVED TWO-LMAGE CROPPING SYSTEM AND METHOD AND THE USE THEREOF IN THE DIGITAL PRINTING AND SURVEILLANCE FIELD
Background of the invention
The present invention relates to improvements in and/or related to a two-image cropping system and method and the application thereof to the digital printing and surveillance field, accordmg to the preambles of claims 1, 2 and 7,8, respectively.
The principle of the two-image cropping has been disclosed in the International Patent Application PCT/EP 00/03389, filed on 14.04.2000 in the name of the same applicant, which is fully herein included for reference and which will be hereinafter called the "previous application". Several figures of this previous Application have been reproduced in the drawings of this application with their original reference numbers and marks, and reference to said figures for facilitating the understanding of the invention and preventing unnecessary repetitions will be made.
In the PCT/EP 00/03389, at present not published, the novel two-image cropping technique disclosed thereby has been used for digitally printing a composite image, for example a card, with the face of a user and/or other alternative printed products incorporated therein, such as m visiting/business cards or bills with photographs, greeting cards with photographs, stickers and so on.
Recently, the inventor has found, after a lot of tests under very different conditions, that such a two-image cropping system and method, and the related software, could be, on one side, improved, and that, on the other side, after the application of said improvements, the two-image cropping technique could be used in fields different from the digital printing field, for example in the spatial surveillance or safety system field.
With respect to the system, in particular, the inventor has found that, by arranging the system in crowded places, upon a continuous movement of persons inside and outside of the video camera surveillance field, the presence sensor, operating based on the microwave technology, sensed the continuous displacements of the persons, even if they were outside of the video camera shooting field, thereby preventing a "clean" reference background from being taken.
Moreover, other types of noises, typically the light reflections or person shadows, were not detected since devoid of mass.
Actually, the difference of the microwave technology used in the presence sensor, and which is based on the presence of a mass, such as the physical body of a person, and the "visual perception" of the system, based on the detection of the images by the video camera, as for the human vision, cold affect the reliability of the two-image cropping system in the mentioned system installation condition, i.e. in crowded spaces continuously traversed by persons passing through the video camera shooting field and/or the adjoining regions.
With respect to the method and program or software modules for managing the system, it has been found, on the oilier hand, that the background slipping mode in the background- interpolation function could be in turn improved due to following reasons. In fact, as it should be clear from figure 18, the backgrounds from BackO to Back5, useful for building the reference background are caused to "slip" to provide a "time history" of the shooting conditions. In this connection, it should be pointed out that, at the shooting time, in the BackO background a "virtual reference background" is built-in, as shown in figure 16B1, with elements taken both from the "background with subject" image, figure 16C, and from the "reference background" image, figure 16B, i.e. without the subject. This, "virtual reference background", fig. 16B1, accordingly, will be held in the reference background sequence, and will slip therewith.
At the time in which, in the two image cropping method shown by the previous Application, a Backlni is carried out, see figure 17, the BackO to Back5 reference backgrounds are "updated" by new and more actual images, and, accordingly, the "virtual" background/backgrounds, figure 16B1, as well as the old background/backgrounds is/are eliminated from the image chain required for building novel virtual reference backgrounds.
However, if the system is installed in a place where a continuous displacements of persons would hinder a regular BackBuild-Backlni cycle, then the sequence of the BackO _ Back5 reference backgrounds could hold herein only old "virtual reference backgrounds", i.e. without any prima facie or current information related to the real environment or outside word. For each operation performed by the user, a new "virtual reference background", figure 16B1, would be generated with the danger that it could be consequently "built-in" on "old" already used virtual backgrounds instead of "updated" taken backgrounds, and this because of the above shown and hereinafter disclosed operation of the presence sensor.
Operations to be performed by the outside presence sensor
The outside presence sensor, which constitutes per se a physical component, or a hardware component of the system, must substantially meet two requirements, and more specifically: 1) it must respect and functionally occupy, as far as possible, the video camera overshooting field, and 2) it must discriminate the same situations seen by the video camera. In the system disclosed in the previous application, in addition to the detection of different objects or articles, i.e. objects or articles either having or not a mass, a parallax problem related to the video camera optical overshooting field occurred, with a consequent impossibility of "focalizing" the sensitivity lobe of the microwave sensor with respect to the taking field of the video camera. This parallax problem, due to a mixed use of the two technologies of the system and method disclosed in the previous application, is shown in figures 1, 2 and 3 respectively illustrating a typical sensitivity lobe 35 of an outside presence microwave sensor 26, in figure 1, the taking or overshooting field 36 of the video-camera 18 of the system in figure 2, as well as the parallax effects deriving from the use of the mixed detecting technique of figure 3, in which ZCC shows a correct coverage zone, ZAN an unjustified alarm zone and ZNR a variation not- detecting zone.
Finally, with respect to the digitally printed products, such as composite cards, they were printed exclusively by a single so- called "live" mode, i.e. with the image occupying the overall surface of the card.
Summary of the invention
A first object of the present invention is to provide a novel technique or method for detecting a presence of intruding persons or bodies, without using prior presence microwave sensors, which novel detecting technique or method can be carried out by a program module or software and by using the system disclosed in the previous application.
A further object of the invention is to simultaneously use the software program of said technique or method, to also simplify and speed up the prior managing software, by eliminating therefrom the prior provided Backlni.exe and BackBuilkd.exe programs. Another object of the present invention is to suggest firther application fields for said software and tlie system for the accomplishment thereof, optionally hi an advantageously simplified fonn.
Yet another object of the present invention is to allow the improved software to be also used for broadenmg tlie printed article types to be obtained by both the system accordmg to the previous application and the system according to tlie present invention.
The above mentioned objects are achieved, accordmg to the invention, by the teachings of claims 1, 2, and 7 and 8, respectively.
Further characteristics of the invention are defined in the dependent claims.
The invention provides several advantages. At first, the optical detection of intruder presence by software, which can be autonomously performed by the system, allows to fully eliminate both the prior detector, per se, i.e. made as a hardware component, and the drawbacks related to the operation thereof.
In fact, a reduced number of components to be assembled and wired, is required, thereby providing a greater operation reliability, less jamming or idling of the system, as well as a less cost thereof.
Another important advantage is that the provision of the above mentioned novel algorithm has allowed an indirect and immediate development of the software in fields different from the digital printing of a composite image, for example in the spatial surveillance and safety field.
The use, as a sensor, of the video camera of the system itself, allows to shnplify tlie control of the overall system by a program or software module, called "BackGenerator.exe", which is able, through the system or video camera overshooting field, to accurately discriminate the most important features related to the two-image cropping algorithm disclosed in the previous application, which algorithm has been held unchanged.
The BackGenerator.exe program module according to the present invention fully replaces the two Backlni.exe and BackBuild.exe modules, since the functions carried out by these two programs have been embedded or incorporated in the novel BackGenerator.exe software module.
Brief description of the drawings
Further characteristics and advantages of the present invention will become more apparent hereinafter from the following disclosure of several embodiments of the invention, which are illustrated, by way of an indicative example, in the accompanying drawings, some of which show some aspects of the two-image cropping technique disclosed in. the previous application PCT/EP 00/03389.
Figure 1 illustrates a typical sensitivity lobe of a microwave sensor;
Figure 2 illustrates the system video camera overshooting field, as tending to infinite;
Figure 3 illustrates the parallax phenomenon related to the use of the mixed detection technique provided in PCT/EP 00/ 03389;
Figure 4 illustrates the background updating or refreshment at the moment of the BackGenerator, and the building of a "virtual reference background" (16B1);
Figure 5 illustrates as a detail the composition of the "virtual background" (16B1);
Figure 6 illustrates the new cycle for eliminating the backgrounds (16B1); Figure 7 is a schematic general block diagram of a simplified surveillance and safety or security system according to the present invention;
Figure 8 illustrates the inside of a store being surveilled or monitored by the surveillance and safety or security system according to the present invention;
Figure 9 illustrates a monitoring and surveilling room of the store shown in figure 8, according to the prior art;
Figure 10 illustrates a "sample image" which can be stored in the system as a "reference background" or as a "first image";
Figure 11 illustrates an image as cyclically provided by the video-camera, or "second image" and which is automatically compared with the reference image or "first image" of figure 10;
Figure 12 illustrates a result of an analysis between the "second image" and "first image", which, after having performed the cropping, shows the presence of remaining areas indicating a presence of an intruder;
Figure 13 shows a color changing of the surveillance monitor screen and the displaying thereon of the remaining or residual areas after the cropping, or of the intruder; and
Figure 14 shows a flow chart illustrating the operation mode of the surveillance and safety system and method according to the present invention.
hi the drawings, related to the figures taken from PCT/EP 00/03389:
Figure 15 is a schematic general block diagram of the system, shown by a dash and double-dots frame and including a first electronic component assembly, known per se, shown by a dash and single-point frame, and an additional electronic component assembly, shown by a dashed frame; Figures 16A to 16E schematically illustrate, by way of an example, a "panorama" and tlie operating steps for making a composite card by the system and method according to PCT/EP 00/03389 reference;
Figure 17 conceptually illustrates the files provided for forming the time "queue" of the scratchpad at the starting of the system, corresponding to the "overshoot background";
Figure 18 schematically illustrates tl e background back slipping principle to carry out the self-updating step of the "reference background";
Figure 19 illustrates, by way of an example, the background interpolation function principle as applied to a "twin" image, of the "background with subject" image, to update the "reference background" as the "background with subject" image is taken, as well as to eliminate any transient noises from the "overshoot backgrounds".
Description of the preferred embodiments
With reference to the above mentioned figures, and to the contents of PCT/EPOO/03389 for a more detailed understanding of the two-image cropping technique, the several aspects and objects of the present invention will be disclosed hereinafter.
Reference is made at first to the system diagram of fig. 15, which comprises a PC 13, a video acquisition board 16, a monitor 17, a video-camera 18, a printer 19, a bankonote or money read-out device 21, an optional lighting device 22, an optional loudspeaker 23, an outer PLC 24, a presence sensor 26, an optional LED 27, and optional communication means 28 and 29. Reference will now be made to the BackGenerator.exe program or software module according to the present invention, allowing to elhrrh ate tlie prior outside presence sensor 26 and to provide an optical software assisted presence detection.
Operation of tlie BackGenerator.exe program module
The BackGenerator.exe program operates as follows: For verifying that no disturbing persons or elements are arranged before the apparatus or system, two overshootings or images are remotely taken, for example, at 1 second from one another, by using the same video camera 18 of the system.
The two images are chromatically compared with respect to their pixels, i.e. each individual pixel of the first overshooting or image is measured and compared with the pixel at the same position of the second overshooting or image. If the chromatic difference would be less than a preset given tolerance, then said pixel would be judged as the same, otlierwise said pixel being marked as different.
If, within the second image, the different pixels are less than a given tolerance (for example 200, with reference to a total pixel number of over 442,000 of the whole image), then it will be judged that no variations of the two images have occurred and that, accordingly, before the video camera no person is arranged (who could not remain absolutely static), and that any disturbing elements, such as casted shadows (i.e. persons who are directly arranged hi tlie visual field of the video camera optics system but provide light interferences), or light reflections either of a direct or of a mirror or polished element reflected type are present.
If the condition is favorable, i.e. no disturbing element is arranged before the video camera, the system will switch the illmτιinath g system on and will take two other overshootings, spaced by 1 second from one another. These two images too are analyzed by tlie same technique to verify that, in the meanwhile, no disturbing element has entered the visual field of the video camera lens.
In affirmative positive case, i.e. in the absence of disturbing elements, the system will switch the illuminating system off and store the second overshooting in the BackO to Back5 files, figure 17, i.e. the sequence of the reference files used for building the "virtual reference background" as shown in figure 16B1.
It should be apparent that by the previous system, as shown in the previous Application, during the updating of the BackO~Back5 backgrounds, it was necessary to carry out a background-interpolating operation, to etarnfnate possible defects of the overshooting or taken image. Now, on the contrary, this is no longer necessary, and represents an important advantage since it is the system itself that "filters" the defects before taking the overshooting.
If the two last overshootings have been found as different, then the system will continue to take overshootings or images, at a distance of 1 second from one another, while performing a comparing thereof so as to found a pair of overshootings or images, without a difference greater than the provided tolerance (for example 200).
If, after a number of attempts, no "reference background" is found, then the system will provide a signal, such as an acoustic signal or warning signal, and open a window on the monitor including a short message asking the persons near the video-camera to move away, while informing said persons that their moving away would be necessary to perform a periodic self maintenance operation, or allow the system to properly operate.
As a like overshooting or image pair is detected, then the lights are switches off, and the video-screen will display a greetings message, thereby allowing the system to complete the last overshooting storing operations.
Optimization of the BackO ~ Back5 (Background Interpolation )) sequence:
As shown in the introductory of the disclosure, there are conditions under which the background interpolation mechanism, for building a "virtual reference background", figure 16B1, could be unreliable. In order to overcome such a situation, according to the invention, the logics for managing the BackO to Back5 reference backgrounds has been slightly modified, as it will become more apparent hereinafter.
According to the previous application, each time a background interpolation operations was performed, the overall "time history" was caused to backward slip, by eliminating the last reference background (Back5.bmp), with the risk of loosing all the directly taken information, and only the "already used" reference backgrounds were processed, figure 19.
On the contrary, according to the invention, after having performed the cropping, the virtual reference background, figure 16B1, is now caused to backward slip or slide by two positions (figure 6), together with all the old backgrounds, with the exception of the Back5 background, which is now affected only by the BackGenerator.exe module, and accordingly by updated images, figure 6, which operation occurred, in the shown example, at 16.00 hours.
Later, for example at 16.15 hours, a subject overshooting operation for forming a card is performed. The image taken by the video camera is stored in the BackO background, and the background interpolation 0 function is started for summarily eliminating the subject areas, and then replacing them by those areas arranged at the same position, coming from the background. Thus, a reference virtual background, figure 16B1, is formed, in which the image portion not covered by the subject is updated at the overshooting time, whereas the portion "masked" by the subject must be recovered from a previous information (Backl~Back4).
This virtual reference background, figure 16B1, constitutes the image which will be used by the cropping algorithm in order to discriminate the "subject" areas from the "background" areas.
At the end of the cropping operation, figure 5, the Back4 image is eliminated, the Back3.bmp image is displaced into the Back4 image, the Back2 image is displaced into the Back3 image, and the reference virtual background, figure 16B1, (BackO), is displaced into the Back2 file.
As a last operation, the image present in Back5 is copied into Backl, thereby providing an updated information for the next inte olating-background operation, figure 6.
Broadening of Hie formed product range
Two are the innovations introduced into the software module representing the interface to the client, i.e. into the TheMask.exe module.
The first allows to take decisions related to the "card" and "greetings bill" products, as novel printed forms/patterns. According to the previous Application, the cards were printed by the so called "live printing" method, in which the image occupied tl e overall surface on the card. According to the invention, the user is now provided with the possibility of choosing the end product according to three patterns, for example: 1) with the prior live printed pattern or 2) with a frame shaped perimetrical edge or 3) with a frame shaped perimetrical edge and a caption at the bottom portion of the card, greetings bill or the like. The second innovation is related to the so-called "stickers" and "visiting bills or cards", in which, according to the invention, it is now possible to select if tlie photo pagination must be vertical (a commercial form) or horizontal, thereby admitting the presence of two persons simultaneously, for example a husband-wife pair, a friend set and so on.
Surveillance or safety application:
As shown in great details in the previous application, the two-image cropping technique has as the main principle of performing a comparative analysis of two images in order to establish their differences. By the above disclosed operation set, it is possible to identify different areas within an image being analyzed, figure 16C, with respect to a reference image, fig. 16B.
In actual practice, this identifying mechanism related to the analyzed image variations can be used in principle, according to tlie invention, in all the fields in which it would be necessary to perform an image automatic analysis for different purposes.
By way of an example, an application of the two image cropping technique to the surveillance and safety field, susceptible to be easily fitted to dangerous area control embodiments, as well as access monitoring and so on embodiments, will be hereinafter disclosed.
The prior art status
As an example, an area 40 such as the inside area of a goods store, monitored by one or more video-cameras, such as four video-cameras, not shown, figure 8, in which the case or box 42 number herein provided and the video-camera number, and related monitors 43, figures 9, make difficult for a monitoring operator 44 to safely control the overall area 40, will be hereinafter disclosed. In such conditions, a possible intruder could not be easily deteαtqek. as he/she moves through the large boxes 42 by concealing therebehind. If the surveillance operator does not observe the related monitor at the intruder movement instant, then the surveillance operator will not be able of detecting the presence of the intruder who could operate in a rather free manner.
Improvement according to the invention
On the contrary, the surveillance and safety system accordmg to the invention is simplified in comparison to the system of the previous application and is adapted to analyze the image supplied by one or more video cameras and detect the moving intruder bodies, independently from the image complexity or the presence of objects through the area being monitored, such as frirniture pieces, vehicles and the like.
The simplified surveillance and safety system comprises a PC 13, a video acquisition board 16, a monitor 17 and one or more video camera 18, for example of the type described in the previous application.
To achieve the desired end, a "sample image" is at first overshoot or "captured", for example at the safety system energizing moment, and this "sample image" is stored in the system as a "reference background", figure 10, for example as shown for a safe box 45 in a room 46.
It should be pointed out that, differently from the provisions for making cards, in the considered use it is not necessary to provide a "virtual reference background" by interpolating the previous images, since the final end is not that of providing a perfectly cropped image, but that of "capturing" with a maximum safety each possible variations related to a given time or image, for example at the system energizing time.
Under not-alarm conditions, i.e. in an intruder lacking condition; figure 10, the control monitor 43 (a single monitor being advantageously provided) of the video camera/video cameras, will provide the normal image taken through the environment 46, figure
10. With a cyclic frequency, for example of 3 seconds, the image supplied by the video camera, figure 11 is automatically compared with the reference image, figure 10.
During the analyzing operation, all the shared and accordingly like areas are eliminated from the image, and it is controlled if are present remaining agglomerated pixels in more or less homogeneous areas, figure 12, i.e. areas which may represent a moving intruder person or body, not pertaining to the surveilled environment, figure 12. In an affirmative case, i.e. if an intruder person or body is present, the background of the control monitor 43 will assume a contrasting color pattern, for example a red color, and on the monitor the areas different or extraneous from the reference image, i.e., in the considered case, the presence of an intruder will be stored, figure 12.
Simultaneously, the image is stored, figure 12, together with the event hour and its place, for example the room access door area. In this case, the surveilling operator 44 can immediately display, in the control room, the image of the intruder 47.
Considerations on the selection of the reference image actuating time (figure 10)
Ideally, the time for taking or "capturing" the reference image having the best safety characteristics, is the alarm actuating or energizing time.
However, cases can occur in which the taken image, figure
11, is not a static image, such as, for example, in an outside area case, in which the sun rise could trigger false alarms due to the formations of like-shadow zones and the displacements thereof. In order to overcome this excessive sensitivity of the warning mechanism, it would be sufficient to use, accordi g tø\jbE invention, a reference image programmed updating, as shown in figure 10.
To that end an image storing cycle would be provided, to operate as a "reference image" for the cropped pattern, figure 10, with a typical time which can vary, for example, from 30 to 600 seconds, depending on the environment variation degree, and the area variation analysis will be performed on a "reference background", figure 10, related to few minutes before the video- camera overshooting or taking time. Thus, the variations related to a time period in which the natural events, such as a light variation or the like, would not be sufficient to generate an alarm, but in which the presence of an intruder moving person or body could be detected and safely signaled, will be monitored.
A detailed operation provided for such a safety application is shown in the flow chart of figure 14.
From this figure it should be easily apparent that, in a case of a fixed reference image, it would not be necessary to program the re-updating type whereas, in the case of a variable reference image, the re-updating time, will be selected depending on the environment conditions of the video-camera 18. For example, in a windowed store, i.e. a solar light receiving store, it would be required a re-updating time of, for example, 3 minutes, whereas a bank caveau would not generally require any re-updating of the reference image since it would not be subjected to solar light impinging radiations but exclusively to a constant artificial light or dark condition.
From the above structural and functional-operation disclosure of the inventive systems and methods, it should be apparent that they fully achieve tlie above mentioned objects and aims, as well as the mentioned advantages.
It should be apparent that one skilled in the electronic field could put in actual practice the teachings of the invention also by
RECTIFIED SHEET (RULE-91) ISA/EP modifymg hi different manners tlie software and hardware portion, without departing from the invention scope and spirit as defined hi the accompanying clahns.

Claims

1. An unproved two-image cropping system for the digital prmting field, for example for making and digitally editing a composite image, for example a picture card, with a face of a user incorporated tiierein, comprising substantially, arranged hi a housing casing:
- a central computer (13),
- a video acquisition board (16),
- a monitor (17),
- a video-camera (18),
- a banknote reading device (21),
- printing means (19),
- a lighting device (22),
- a loudspeaker (23). which can be power supplied by electric power and which operatively interact by operating sequences which can be controlled by software programs or modules, wherein tlie video- camera (18) takes images with a free taking field, or with "multichromatic" and "dynamic" outer backgrounds, said system further comprising:
- a presence sensor adapted to detect the presence of persons or objects movable through the taking field of the video-camera (18), and
- an outer PLC (24) operatively coupled to said central computer (13), banknote reading device (21), lighting device (22) and presence sensor (26), characterized in that said presence sensor is an optical presence sensor in the form of a software operating through tlie video- camera ( 18) of the system.
2. A system according to Claim 1, characterized in that said system further comprises a functional-operating architecture comprising the following operating software modules or programs cooperating with one another and controlling the associated components of the system (13, 16, 17, 18, 19, 20, 21, 22, 23, 24, 27) as follows:
- a Module A, for example (TheMask.exe), or a user-system interface, displaying on said screen (17) different options to be selected by the user, communicating to the system the selections performed by the user and supplying corresponding graphics animations;
- a Module B, e.g. (Core.exe), which, through said video acquisition panel (16), captures the images generated by the video- camera (18), and converts the input video signal by transforming it into an ordered sequence of pixel constituting the mathematics expression of all the geometric patterns present in the considered image, said software Module B extrapolating the image of the "subject" (6) from the "background-subject assembly" (Figure 16C) and locating said image on the "view" (4) selected by the user, said extrapolation being performed by different analyses of the different chromatic equivalent area existing between a "first image", constituting a "reference background" (Figure 16B1) and a "second image" formed by the "background-subject assembly" (Figure 16C) as taken by the video-camera (18) with a free taking field;
- a Module D, for example (Mailer.exe), which sends all the messages to the different components of the system, and, more specifically, between the user interface, Module A, "TheMask.exe", and the module B, "Core.exe", during the acquisition by the video-camera (18), and with the outer PLC (24) for controlling the lighting device (22) and the operations of tlie banknote readmg device (21) and with tlie printer (19), thereby controlling a proper printing process, all tl e message exchange between the Module D, "Mailer.exe" and the Module B, "Core.exe" occurring through the Registry of tlie computer (13), the message flow being a bidirectional message flow,
- a Module E, for example (Golem.bin), which is arranged in the outer PLC (24) and turns said lighting device (22) on as said "subject" is taken, and communicates to the computer (13) an amount introduced into the banknote reading device (21), wherein the control of tlie "timers" and the presence sensor which actuate and allow the taking of the "taken backgrounds" is preferably provided inside tlie modul "Mailer.exe", and wherein the loudspeaker (23) is operated by the central computer
(13) characterized hi that it further comprises
- a Module C, for example (BackGenerator.exe) which on one hand replaces both modules (Cl and C2) Backlni.exe and BackBuild.exe of tlie previous application and on the other hand through the video-camera (18) is able to accurately discriminate tl e most important features related to the unchanged two-image cropping algorithm of the previous application which is therefore a presence sensor without any physical reality.
3. An unproved two-image cropping method for tl e digital field, for example for making and digitally editing a composite image, for example a picture card with a face of a user incorporated tiierein, as disclosed in the PCT/EP 00/ 03389, characterized hi that for verifying if disturbing persons or bodies are arranged before tlie system an optical presence sensor operating through the system video-camera and a dedicated software is used, and in that it comprises the following steps: a) two overshootings or images are remotely taken, for example, at 1 second from one another, by using the same video- camera of the system, b) said two images are chromatically compared with respect to their pixels, i.e. each individual pixel of the first overshooting or image is measured and compared with the pixel at the same position of the second image, c) if the chromatic difference is less than a preset given tolerance, then said pixel is judged as the same, otherwise said pixel being marked as different, d) if, within the second image, the different pixels are less than a given tolerance (for example 200, with reference to a total pixel number of over 442,000 of the whole image), then it is judged that no variations of the two images have occurred and that, accordingly, before the video camera no persons or disturbing bodies or elements, such as casted shadows or light reflections are present, e) if no disturbing person or element is arranged before the video camera, the system will switch the illuminating system on and will take two further images, spaced by 1 second from one another, f) said two further images are also analyzed like as provided for in steps b), c) and d) to verify if, in the meanwhile, disturbing person or elements have entered the visual field of the video-amera, g) in the case that a first and a second images are found equal, i.e. in the absence of disturbing persons or elements, the system switches the illuminating system off and stores the second images in the BackO to Back5 files (figure 17), i.e. tl e sequence of the reference files which will be used for building the "virtual reference background" (figure 16B1), whereas h) in the case that a first and a second images are found different, i.e. in the presence of disturbing persons or elements, then the system will continue to take overshootings or images, at a distance of 1 second from one another, while performing a comparing thereof as provided for in steps b), c) and d) untill a pair of or images is found without a difference greater than the provided tolerance (for example 200), i) if, after a number of attempts, no "reference background" can be built, then the system will provide a signal, such as an acoustic signal or warning signal, and preferably open a window on the monitor showing a short message asking the persons near the video-camera to move away for allowing the system to properly operate, j) as a subsequent pair of equal first and second images is detected, the system will switch off the lights, and the video-screen will preferably display a greetings message, thereby allowing the system to complete the last image storing operations.
4. Method according to claim 3, characterized in that it comprises the following steps k) after having perfonned tlie cropping, the virtual reference background (figure 16B1), is now caused to backward slip or slide by two positions (figure 6), together with all the old backgrounds, with the exception of the Back5 background, which is now affected only by the BackGenerator.exe module, and accordingly by updated images, which operation occurred at a certain time, for example at 16.00 hours (figure 4),
1) later, for example at 16.15 hours, a subject overshooting operation for forming a card is performed and the image taken by the video camera is stored in the BackO background, and the background interpolation () function is started which will summarily eliminate the subject areas, and then replace them by those areas arranged at the same position, coming from the Backl background, whereby a reference virtual background (figure 16B1) is formed, in which the image portion not covered by the subject is updated at the overshooting time, whereas the portion "masked" by the subject must be recovered from a previous information (Backl~Back4), whereby said virtual reference background (figure 16B1) constitutes the image which will be used by the cropping algorithm in order to discriminate the "subject" areas from the "background" areas, m) at the end of the cropping operation (figure 4), the Back4 image is ehminated, the Back3.bmp image is displaced into the Back4 image, the Back2 image is displaced into the Back3 image, and the reference virtual background, figure 16B1, (BackO), is displaced into the Back2 file, whereby n) as a last operation, the image present in Back5 is copied into Backl, thereby providing an updated information for the next interpolating-backgroimd operation (figure 6).
5. Method according to claim 3 and 4, characterized in that into the TheMask.exe module is provided for the client the possibility to take decisions related to the "card" and "greetings bill" products as novel printed format, that is of choosing for the end product among three patterns, for example among 1) an end product having the traditional so called "live printing" format, 2) an end product with a frame shaped perimetrical edge, or 3) an end product with a frame shaped perimetrical edge and a caption at the bottom portion of the card, greetings bill or the like.
6 Method according to claim 3 and 4, characterized in that into the TheMask.exe module is provided for the client the possibility as to the so-called "stickers" and "visiting bills or cards" to select if the photo pagination must be vertical (the commercial form) or horizontal.
7. A simplified two-image cropping system of the type disclosed in claim 1 and 2, characterized in that it is used in the surveillance and safety field and comprises, arranged in a housing casing:
- a central computer (13),
- a video acquisition board (16),
- a monitor (17),
- a video-camera (18), which can be power supplied by electric power and which operatively interact by operating sequences which can be controlled by software programs or modules of the type disclosed in claim 2, wherein the video-camera (18) takes images with a free taking field, or with "multichromatic" and "dynamic" outer backgrounds, wherein said system further comprises an optical presence sensor operating through the system video-camera (18) and software.
8. An simplified two-image cropping method of the type disclosed in claims 3 and 4, characterized in that it is used in the surveillance and security field and comprises the following steps
A) a "sample image" is at first overshoot or "captured", for example at the safety system energizing moment,
B) said "sample image" is stored in the system as a "reference background" (figure 10),
C) under not-alarm conditions, i.e. in an intruder lacking condition, figure 10, the control monitor 43 (a single monitor being advantageously provided) of the video-camera/video-cameras, will provide the normal image taken through the environment, (figure 10), whereby with a cyclic frequency, for example of 3 seconds, the image supplied by the video-camera, figure 11 is automatically compared with the reference image, figure 10,
D) during the analyzing operation, all the shared and accordingly like areas are eliminated from the image, and it is controlled if are present remaining agglomerated pixels in more or less homogeneous areas, figure 12, i.e. areas which may represent a moving intruder person or body, not pertaining to the surveilled environment,
E) in affirmative case, i.e. if an intruder person or body is present, the background of tlie control monitor will preferably assume a contrasting color pattern, for example a red color, and on the monitor the areas different or extraneous from the reference image, for example the image of an intruder, will be stored, and
F) simultaneously, the image is stored together with the event hour and its place, for example the room access door area, so that the surveilling operator can immediately display the image of the intruder.
9. An improved two-image cropping system according to claim 1, characterized in that tlie PLC is provided as an inner component.
10. Composite cards, greeting cards, photo-cards and stickers, visiting cards and the like, characterized in that they are made and printed by a systems and method according to one or more of the claims 1 to 6.
PCT/EP2000/011508 2000-04-14 2000-11-20 Improved two-image cropping system and method and the use thereof in the digital printing and surveillance field WO2002041255A1 (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
AU2001215223A AU2001215223A1 (en) 2000-11-20 2000-11-20 Improved two-image cropping system and method and the use thereof in the digitalprinting and surveillance field
PCT/EP2000/011508 WO2002041255A1 (en) 2000-11-20 2000-11-20 Improved two-image cropping system and method and the use thereof in the digital printing and surveillance field
US09/834,920 US20010055414A1 (en) 2000-04-14 2001-04-16 System and method for digitally editing a composite image, e.g. a card with the face of a user inserted therein and for surveillance purposes

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/EP2000/011508 WO2002041255A1 (en) 2000-11-20 2000-11-20 Improved two-image cropping system and method and the use thereof in the digital printing and surveillance field

Publications (1)

Publication Number Publication Date
WO2002041255A1 true WO2002041255A1 (en) 2002-05-23

Family

ID=8164166

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2000/011508 WO2002041255A1 (en) 2000-04-14 2000-11-20 Improved two-image cropping system and method and the use thereof in the digital printing and surveillance field

Country Status (2)

Country Link
AU (1) AU2001215223A1 (en)
WO (1) WO2002041255A1 (en)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4891660A (en) * 1988-11-29 1990-01-02 Pvi, Inc. Automatic photographic system and frame dispenser
GB2235347A (en) * 1989-08-21 1991-02-27 Pamela L Barber Making electronically-produced postcards
EP0635983A2 (en) * 1993-07-19 1995-01-25 AT&T Corp. Method and means for detecting people in image sequences
US5668605A (en) * 1994-10-25 1997-09-16 R. T. Set Object keying in video images based on distance from camera
EP0805405A2 (en) * 1996-02-05 1997-11-05 Texas Instruments Incorporated Motion event detection for video indexing
US5764306A (en) * 1997-03-18 1998-06-09 The Metaphor Group Real-time method of digitally altering a video data stream to remove portions of the original image and substitute elements to create a new image
WO1999055995A1 (en) * 1998-04-29 1999-11-04 Malcolm William Thomas An access control system

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4891660A (en) * 1988-11-29 1990-01-02 Pvi, Inc. Automatic photographic system and frame dispenser
GB2235347A (en) * 1989-08-21 1991-02-27 Pamela L Barber Making electronically-produced postcards
EP0635983A2 (en) * 1993-07-19 1995-01-25 AT&T Corp. Method and means for detecting people in image sequences
US5668605A (en) * 1994-10-25 1997-09-16 R. T. Set Object keying in video images based on distance from camera
EP0805405A2 (en) * 1996-02-05 1997-11-05 Texas Instruments Incorporated Motion event detection for video indexing
US5764306A (en) * 1997-03-18 1998-06-09 The Metaphor Group Real-time method of digitally altering a video data stream to remove portions of the original image and substitute elements to create a new image
WO1999055995A1 (en) * 1998-04-29 1999-11-04 Malcolm William Thomas An access control system

Also Published As

Publication number Publication date
AU2001215223A1 (en) 2002-05-27

Similar Documents

Publication Publication Date Title
US20010055414A1 (en) System and method for digitally editing a composite image, e.g. a card with the face of a user inserted therein and for surveillance purposes
US6937743B2 (en) Process and device for detecting fires based on image analysis
US20190244504A1 (en) Fire monitoring system
JP3221883B2 (en) Apparatus for combining multiple data sources on a printed document
US4999614A (en) Monitoring system using infrared image processing
AU2005241466B2 (en) ATM security system
US6469734B1 (en) Video safety detector with shadow elimination
US8004575B2 (en) Methods and systems for detecting bright objects and/or adaptively modifying video camera exposure
US9159203B2 (en) Automated teller machine comprising at least one camera that produces image data to detect manipulation attempts
CN107664705A (en) The speed detection system and its speed detection method of passenger conveyor
US6819353B2 (en) Multiple backgrounds
KR101625538B1 (en) Car Number Recognition system
JP5042177B2 (en) Image sensor
CN101764999A (en) Sub-camera video capture device
JP2010182021A (en) Image monitor system
JP5393086B2 (en) Image sensor
US5757431A (en) Apparatus for coupling multiple data sources onto a printed document
KR20170012874A (en) Method and program for processing image signal in display screen
WO2002041255A1 (en) Improved two-image cropping system and method and the use thereof in the digital printing and surveillance field
JP2008042652A (en) Monitoring camera device
JP2847597B2 (en) Infrared monitoring system
CN1044524C (en) Fire alarm with image distinguish function
JP2010118039A (en) Mobile object detector
JP7328778B2 (en) Image processing device and image processing program
CN106034209B (en) Obtain the method and device of video

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 09834920

Country of ref document: US

AK Designated states

Kind code of ref document: A1

Designated state(s): AE AL AU BA BG BR CA CN CZ DZ EE HR HU IL IN IS JP KR LT LV MA MD MK MX NO NZ PL RO RU SG SI SK TR UA US VN YU ZA ZW

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE TR

121 Ep: the epo has been informed by wipo that ep was designated in this application
122 Ep: pct application non-entry in european phase