DE19848490B4 - Image information transmission method and apparatus - Google Patents

Abstract

Image information transmission method, in particular for data channels of low bandwidth and terminals with low capacity, for the transmission of image data of a stationary image recording setting, which is set up for the detection of directionally moving objects with a small image portion in front of a background scene, with the following steps: Providing an image recording device with the image recording setting, the image recording device being designed in this way is that it can detect the background scene and directionally moving objects in a distinguishable manner in terms of image data; Detecting whether at least one directionally moving object is present and, if so, extracting the corresponding image data by the image recording device; and transmitting the corresponding extracted image data or corresponding modified image data obtained from the extracted image data from the image pickup device to an image data processing device; Assigning at least one classifiable feature to the extracted image data or to the corresponding modified image data obtained from the extracted image data; and storing the transmitted image data together with the associated value of the classifiable feature in a storage device; characterized by the step of providing text modules and assigning the respective ...

Description

STATE OF THE ART

The present invention relates to an image information transmission method, particularly for low-bandwidth data channels and low-capacity terminals, for transmitting image data of a stationary image pickup set up for detecting directed low-content moving objects in front of a background scene, comprising the steps of: providing an image pickup device having the image pickup setting; wherein the image pickup device is configured to detectably distinguishably detect the background scene and directionally moved objects image-wise; Detecting whether there is at least one directionally moved object and, if so, extracting the corresponding image data by the image capture device; Transmitting the respective extracted image data or corresponding modified image data obtained from the extracted image data from the image pickup device to an image data processing device; Assigning at least one classifiable feature to the extracted image data or to the corresponding modified image data obtained from the extracted image data; and storing the transmitted image data together with the associated value of the classifiable feature in a storage device.

Likewise, the present invention relates to a corresponding image transmission device.

There are a large number of image information transmission methods and standards in the prior art for medium and lower bandwidth transmission media, such as e.g. LAN (100 Mb) with H.263, ISDN (64 kb) with H.320, POTS (14.4 kb) and GSN (9.6 kb).

For transmission media, which find use in buildings, such. B. in the building control with LON (Powerline), LSN (safety network, Robert Bosch GmbH) or EIB (European Installation Bus) and in factories with fieldbus systems, these image codes are not suitable because they require a high transmission cost.

From the GB 2 183 878 A1 For example, a monitoring system is known which comprises means for monitoring a zone and means for image processing, wherein an input image is compared with a stored reference image by means of the means for image processing in order to detect irregularities.

The WO 96/038 39 A1 discloses a video surveillance system for monitoring and recording objects, as well as for recording management information into the video signal.

From the DE 43 11 972 A1 A method for detecting changes in moving pictures is known. In particular, this method is carried out by means of a computing unit, wherein the image signals are present in pixels and are stored in a memory and by means of the arithmetic unit, the change of a picture detail is determined with at least one image pixel compared to the same image plane in the image plane at least one preceding image, the Change is compared by the arithmetic unit with a predetermined threshold and for each image pixel of the image in the memory stored memory addresses depending on whether the change is greater or less than the threshold, different values are assigned.

From the arithmetic unit, an observation window in the image / period is selected from the stored image signals, which contains the image section to be compared and at least one further image section already compared with a threshold. The threshold is changed by the arithmetic unit depending on whether the change of the already compared image section was smaller or larger than its threshold. The result of the comparison between the change of the image detail to be compared and the determined threshold is assigned to at least one of the addresses stored in the memory of an image pixel of the image detail.

The DE 1 96 01 005 A1 also discloses a method for detecting moving objects in temporally successive images.

The idea underlying the known transmission method defined at the outset is that the image recording scene recorded by the image recording device is first deposited on the image reproduction apparatus like a pictorial site plan of the scene or transmitted there as a video image from the image recording scene. If a transmission takes place (wirelessly or by cable or similar), then expediently the image of the background scene in an initialization phase according to a general coding method, such. B. JPEG, transmitted via the narrow-band transmission channel once from the transmitter (camera) to the receiver (image display device). This process typically takes 100 seconds for a channel capacity of 1 kb.

If changes in the monitored area now occur, they are analyzed by the image acquisition device (intelligent video sensor). These changes are doing by the image pickup device in disturbance variables, such. B. lighting changes, and movements of interest directed moving objects such. B. Person movements, classified.

The detected directionally moved objects are detected as a pictorial object (complete pixel image) and additionally, for example, with a pixel-precise object mask (silhouette of the silhouette) and coded separately from the background scene. Thus, background image, pictorial object and associated object mask are available separately for a detected scene.

Then, only the corresponding extracted image data or corresponding modified image data of the detected directionally moved object obtained from the extracted image data is transferred from the first image pickup device to an image data processing device. The background scene is not retransmitted or transmitted much less frequently than the object image data.

Since in normal scenes of the video sensor, the true objects appear small compared to the overall scene (as opposed to Bildtelephonanwendungen with cameras in portrait setting), take these objects, ie people or vehicles, etc., only a fraction of the overall picture. Typical size ratios are a few per thousand to percent. This corresponds to a typical picture size of approx. 200 bytes.

Therefore, the known method provides a significant reduction in the transmission overhead, because it is limited to the transmission of the filtered-out essential information and provides a simple and fast transmission capability without the requirement of high bandwidth or capacity. Further advantages of the known method include the possibility of using extremely narrow-band channels and thus the utilization of infrastructures that may be present but have not yet appeared sufficient for use in such methods (eg pagers or the like). It can be the movements of objects, eg. As persons and vehicles, for a large area and simply record. The information obtained about the object locations can be provided to building information systems, etc.

According to a preferred variant, the corresponding image data of the directionally moved object are continuously updated and the modified image data obtained from the updated image data are transferred from the first image acquisition device to an image data processing device. In other words, the viewer is kept constantly informed of the movements of the directionally moving object with minimal transmission effort.

Conveniently, the image data processing apparatus comprises an image data reproducing apparatus on which reproduction of the transmitted image data or modified image data is performed.

The modified image data of the detected directionally moved object are usually selected from the following group: positional frame of the object, locus of the silhouette of the object, size of the object, position of a particular point (eg, base point, center point, ...) of the object Path of a certain point of the object, motion vector of a certain point of the object. Which image data are transmitted in the specific case depends on the bandwidth of the transmission channel and the presentation capacity of the receiver.

In order to make the transmitted data concerning detected directionally moving objects accessible to a statistical database system and to make easily extractable statements about the behavior of the detected directionally moved objects, video sequences with a feature index or classification characteristics are recorded, allowing the user to search for the event sought with the associated images in one or more classifiable characteristics. By means of object-detecting methods, such image-related additional information can be obtained and assigned to image sequences.

A position determination with the image information transmission method according to the invention can, for. For example, help locate an object that was last in a particular area. The scene sections are not stored only in chronological order, but classified according to scenes or object-related characteristics.

The pure digital video recording is thus linked to the possibilities of video sensor technology in order to extract image-related data or features from the scene itself and to use it as a classification criterion. A big advantage of this system is that with existing video sensors, z. As for the nightly burglary surveillance, without much effort during the day the video database with statistical functions for obtaining information can be activated. Intrusion detection system and control system are independent in time.

It is possible to organize the stored image data according to the classifiable characteristic (s) in the form of a database. This makes it possible to access the desired video sequence to an incident or an event in the fastest way possible. Access to the history and the current situation is possible for each feature. The dataset of the scene features which reflect events can be evaluated in a statistic as a whole. The statistical quantities describe frequencies, averages, variances and deviations of the measured values. They can be very well displayed in color in the scene at the corresponding coordinate.

Furthermore, derivation of control data for a control device for actuating an actuating device can take place in accordance with the evaluation of the database. The features currently obtained from the scene compared to predetermined thresholds are suitable for obtaining control data. If a measured value or feature vector exceeds a predetermined threshold, a control signal can be issued, e.g. B. if the number of objects exceeds a maximum value or if the object flow is too low, so that a queue or a queue arises. Such control data can be forwarded to higher-level systems, in particular control computers and the like. The output of the control data can be caused visually or acoustically via a signal generator directly, in particular by controlling displays, preferably traffic lights, billboards, monitors, playback of audio data, controlling actuators, in particular doors and barriers.

The present invention underlying technical problem is that the classifiable teristics male, z. For example, size, position, speed, direction of movement, object path, dwell time in scene or position, gray value / color, number of objects, entry / exit point of the object are initially purely numerical values that appear difficult to understand without the pictorial context. These values are therefore only implicitly visualized in the displayed image.

ADVANTAGES OF THE INVENTION

The image information transmission method according to the invention with the features of claim 1 and the corresponding image information transmission device according to claim 8 have over the known approaches to the solution on the advantage that complicated issues from the field of video sensors are of course linguistically implemented. This offers particular advantages in terminals with very limited possibilities of data transmission or the representation of the transmitted data, such. B. pagers, pure audio channels or phones. It is the usual infrastructure intelligently coupled with modern video sensors, so that the optimal use of existing staff is guaranteed. Central security offices are made leaner and staff in the contract sector can be used and managed more efficiently.

The idea underlying the present invention is that an image representation is usefully translated into natural language and this language is reproduced either visually or acoustically together with the image representation or without the image representation in the terminal.

In the respective subclaims there are advantageous developments and improvements of the image information transmission method specified in claim 1.

According to a preferred embodiment according to another preferred embodiment

DRAWINGS

Embodiments of the invention are illustrated in the drawings and explained in more detail in the following description.

1 shows the representation of a detected directionally moving object in the form of a person through a pictorial representation and a movement trajectory in front of a static stored background image according to an embodiment of the present invention.

DESCRIPTION OF THE EMBODIMENTS

Although usable in principle for any outdoor and indoor image information transmission applications, the present invention will be described with respect to the surveillance of an outdoor area, namely a parking lot.

When applied to the interior, one could expect a nearly constant environment. The lighting changes, vegetation changes and weather influences which one is exposed in the outdoor area, namely, are not found indoors.

However, there are interference-free imaging devices, especially for outdoor areas with lighting changes and movements of the background, eg. B. movements of trees o. Ä., Are suitable. These devices use intelligent video sensors adapting to the respective background scene, which can distinguish between such disturbances and directionally moved objects which are the objects actually to be detected.

Specifically, first, the provision of a first image pickup device in the form of a video sensor with the image pickup attitude directed to a parking lot, wherein the first image pickup device is designed such that it can detect the background scene and directionally moving objects image data distinguishable.

Then, the image data of the background scene is transferred to the image data reproducing apparatus once in an initialization phase.

In operation, it is constantly detected whether there is at least one directionally moved object, and, if so, the corresponding image data extracted by the first image pickup device.

Then, the corresponding extracted image data or corresponding modified image data obtained from the extracted image data is transferred from the first image pickup device to the image data processing device having the image data reproducing device.

Further, there is a constant updating of the respective image data of the directionally moving object and the transmission of the corresponding updated image data or corresponding modified image data obtained from the updated image data from the first image pickup device to the image data processing device.

Which image data or modified image data are transmitted and how the corresponding representations on the image data reproduction device, for. B. in the form of a monitor will be explained in more detail below.

In 1 denotes reference numeral 1 a background scene in the form of a parking lot and 2 a detected directionally directed object in the form of the silhouette of a person. The figure of the person is completely dissolved and their movement trajectory 3 as well as her velocity vector 4 appears.

• – die Standorte des bewegten Objekts als Positionsrahmen in der bildhaften Kameraszene und/oder
• – die sich zeitlich verändernde pixelgenaue Maske in der bildhaften Kameraszene und/oder
• – das bildhafte Objekt in der bildhaften Kameraszene

einmalig als Alarmbild und/oder der gesamte Alarmsequenzablauf übertragen und dargestellt werden. Die Ansprüche an die Kapazität bzw. Bandbreite sind dabei von der ersten bis zur letzten der angegebenen Konstellationen ansteigend.Depending on the provided transmission bandwidth or reception capacity, the image information transmission method according to the invention can be used selectively or in combination:

• The locations of the moving object as position frames in the pictorial camera scene and / or
• - The time-varying pixel-precise mask in the pictorial camera scene and / or
• - the pictorial object in the pictorial camera scene

be transmitted and displayed once as an alarm image and / or the entire alarm sequence sequence. The demands on the capacity or bandwidth are increasing from the first to the last of the specified constellations.

For a medium frame rate of 10 frames / second processed by the video sensor, the following capacities per second are typically required: Location plan Locations 80 bytes Production picture with picture mask about 0.5 kb Scene image with pictorial object 1 kb Scenery with pictorial object sequence about 2.5 kb

The pictorial representation of an alarm scene on the receiver side in the manner described is completely sufficient for instructed users. An evaluation of alarms in unwanted messages and true alarms can be performed. Distinctions are possible for the user via the following recognition patterns: Location Locations exact position, size and temporal movement pattern (object trajectory in the camera scene) Production picture with object mask Shape of the object through the morphological appearance of the mask in time and space Scene image with pictorial object pictorial appearance of the individual alarm object Scenery with pictorial object sequence pictorial appearance of the object of the entire alarm sequence

For a trained user, it is possible to classify objects without detailing a camera scene. It is possible for it to recognize objects without using the graphic appearance of the object or the entire scene. Only with the help of morphological representations (rectangle, contour, movement sequence, trajectory) can the alarm in the video sensor, the unwanted message such. B. lighting changes, small animals, insects, shadows, ... of true object alarms, such. B. personal movements are distinguished by burglars and vehicles.

Next, a recording of the scene statistics will be described. The video image is needed here only for analysis purposes, it is sufficient a pure momentary feature recording of the scene. Optionally, the video image can be continuously recorded. The video image can z. B. serve to illustrate the extracted statistical quantities from the pictorial scene. It is also possible to draw the measured variables at the corresponding coordinate in a graduated color. The principle is used by a Videosor in such a way that the moving object is marked in color with a frame and the distance covered is drawn.

In the exemplary embodiment, provision is made of text modules and an assignment device for assigning respective text modules to specific values of the classifiable feature or classifiable features, which will be explained in more detail below.

A language is basically defined by a grammar that combines individual words into whole sentences. A simple sentence consists, for example, of the sentence parts subject, predicate, object, attribute, adverbial determination of the location, time, etc. This is explained in more detail in Table I below with reference to the method according to the invention. Table I for sentence formation subject predicate attribute 1. Adv. Best. (Local) 2. Adv. Best. (Local) A person going at 5 km / h at the entrance A in the direction of northeast (NO) A bicycle moves with 10 km / h at the entrance B towards the south (S) A car moves at 50 km / h in area C towards southwest SW A truck moves with 30 km / h at the gate D towards east (O) A motorcycle moves with more than 50 km / h at the fence E towards west (W)

Table I shows example sentences which are to be formed from the classifiable features and stored text blocks by the allocation device. The assignment takes place, for example, via an allocation algorithm or an allocation table. This is illustrated in Tables II to VI. Table II for subject Classifiable parameter glossary Silhouette type A person Silhouette type B bicycle Silhouette type C car Silhouette type D truck Silhouette type E motorcycle

In other words, z. B. associated image data, for example, the subject text module truck if the extracted image data or the modified image data belonging to the class Silhouette type D, which has been previously determined and stored for comparison. Table III for predicate Classifiable parameter glossary Silhouette type A running Silhouette type B moves Silhouette type C moves Silhouette type D moves Silhouette type E running

In other words, z. B. the relevant image data, for example, the predicate text module runs assigned, if the extracted image data or the modified image data belonging to the class Silhouette type A, which has been previously determined and stored for comparison. Table IV for attribute Classifiable parameter glossary Speed vector up to 5 km / h at 5 km / h Speed vector 5 to 20 km / h with 10 km / h Speed vector 40 to 50 km / h at 50 km / h Speed vector 20 to 30 km / h with 30 km / h Speed vector 50 to 100 km / h with more than 50 km / h

In other words, z. B. the image data concerned, for example, the attribute text module associated with 5 km / h, if the extracted image data or the modified image data belonging to the class velocity vector to 5 km / h, which has been previously determined and stored for comparison. Table V for 1. Adv. Best. (Local) Classifiable parameter glossary Coordinate vector (a1, b1) at the entrance A Coordinate vector (a2, b2) at the entrance B Coordinate vector (a3, b3) in area C Coordinate vector (a4, b4) at the gate D Coordinate vector (a5, b5) at the fence E

In other words, z. For example, if the extracted image data or the modified image data belong to the class coordinate vector (a5, b5) which has been previously defined and stored for comparison , Table VI for 2. Adv. Best. (Local) Classifiable parameter glossary Motion vector (x1, y1) in the direction of northeast (NO) Motion vector (x2, y2) in the direction of Sud (S) Motion vector (x3, y3) towards southwest SW Motion vector (x4, y4) towards east (O) Motion vector (x5, y5) towards west (W)

In other words, z. For example, if the extracted image data or the modified image data belong to the class of motion vectors (x5, y5) previously determined and for comparison, for example, the respective image data is assigned the 1st adv. Best has been saved.

• a) zusammen mit den Bilddaten als eingeblendeter Text;
• b) zusammen mit den Bilddaten als eingeblendeter Text und Sprachausgabe;
• c) zusammen mit den Bilddaten als Sprachausgabe;
• d) ohne die Bilddaten als eingeblendeter Text;
• e) ohne die Bilddaten als eingeblendeter Text und Sprachausgabe;
• f) ohne die Bilddaten als Sprachausgabe usw.

When all assignments have been made, the output of the text modules takes place together with the respective classifiable characteristic. Depending on the device, this can be done as follows:

• a) together with the image data as displayed text;
• b) together with the image data as displayed text and speech output;
• c) together with the image data as speech output;
• d) without the image data as overlayed text;
• e) without the image data as displayed text and speech output;
• f) without the image data as a voice output, etc.

The spatial arrangement may be such that a camera is provided as an image data recording device, and the transmission of the corresponding extracted image data or corresponding modified image data obtained from the extracted image data from the camera to a computer with monitor as the image data processing device is performed including mapping and storage expires.

Also, image processing and classification may already take place in the camera and transmission to a pager may occur.

Also, image processing and classification may take place in an intermediate device or in the terminal. This depends on the respective task, eg. As monitoring of an area, monitoring of multiple areas, monitoring by a guard, monitoring by several guards, etc. from.

With the aid of the method according to the invention for the linguistic description of the change of image contents by corresponding transmission and evaluation of image data information, it is possible to couple text and video databases in context. Scripts and / or audits can be created for individual camera scenes or superordinate surveillance areas by individual text descriptions, which reflect all processes. Through appropriate indexes, the video data can be coupled.

In the case of a reportable event, a message can be sent that describes the incident purely textually, eg. As an email or a pager message, a fax, etc. The message can also mitenthalten corresponding image data. The transmission can take place over radio, over lines of an Internet or Intranet, over the telephone network etc. With the help of a voice output unit, the description of the scene situation can be realized via audio playback.

The sentence formation can contain any number of sentence parts, and the assignment can be made via discrete values, ranges, adaptation algorithms, tables, templates, etc.

Claims (8)

An image information transmission method, in particular for low-bandwidth data channels and low-capacity terminals, for transmitting image data of a stationary image pick-up set up for detecting directed moving objects of low image content in front of a background scene, comprising the following steps: Providing an image pickup device with the image pickup setting, the image pickup device being configured to detectably distinguishably detect the background scene and directionally moved objects image-wise; Detecting whether there is at least one directionally moved object and, if so, extracting the corresponding image data by the image capture device; and Transmitting the respective extracted image data or corresponding modified image data obtained from the extracted image data from the image pickup device to an image data processing device; Assigning at least one classifiable feature to the extracted image data or to the corresponding modified image data obtained from the extracted image data; and Storing the transmitted image data together with the associated value of the classifiable feature in a storage device; characterized by the step Providing text modules and associating respective stored text modules with specific values of the classifiable feature; and Output of the text modules as a sentence with subject and predicate. Image information transmission method according to claim 1, characterized in that the output is acoustic and / or optical. An image information transmission method according to claim 1 or 2, characterized in that the outputting is performed together with the transmitted image data. An image information transmission method according to claim 1 or 2, characterized in that the output is performed without the transmitted image data. Image information transmission method according to one of the preceding claims, characterized in that the classifiable feature is selected from the following group: size, position, velocity, direction of movement, object path, dwell time in scene or position, gray value / color, number of objects, entry / exit point of the object. Image information transmission method according to one of the preceding claims, characterized by the step of organizing the stored image data according to the classifiable characteristics and the text modules in the form of a database. An image information transmission method according to any one of the preceding claims, characterized by the step of evaluating the database according to at least one of the following criteria: number of objects in a certain position, frequenting a given path, total / average dwell time in a certain position, length of the covered path, time duration for certain Track sections, number of passing objects, number of objects moving in or out, in particular persons, vehicles, workpieces, entry / exit points, mean or current object flow. Apparatus for carrying out the image information transmission method according to at least one of claims 1 to 7.

Images