WO2000040022A2

WO2000040022A2 - Method of and device for video image capturing, storage and processing

Info

Publication number: WO2000040022A2
Application number: PCT/ZA2000/000005
Authority: WO
Inventors: Barend Hendrik Coetzer
Original assignee: Barend Hendrik Coetzer
Priority date: 1998-12-24
Filing date: 2000-01-10
Publication date: 2000-07-06
Also published as: WO2000040022A3; AU2638900A

Abstract

A method of preparing video images for storing including the steps of compressing (18) a digital video image to reduce the number of bytes of storage required to store the digital video image; combining or multiplexing (22) a plurality of the compressed digital video images into a data block or data stream in a manner which allows the individual digital video images to be extracted from the data block or stream, when stored; and optionally, storing (26) the data block or data stream onto a single medium. The method may include the prior step of digitizing (14) analogue video images from an analogue video source (12) in, for example, CCIR, PAL, NTSC, EIA, or the like, formats as well as the step of storing the digitized digital video images in a frame buffer or similar memory prior to compression thereof.

Description

METHOD OF AND DEVICE FOR VIDEO IMAGE CAPTURING, STORAGE

AND PROCESSING

This invention relates to video image capturing, storage and processing.

The inventor is aware of present video image storage systems in which video images are stored onto magnetic media such as analogue video tape using recorders such as video cassette recorders (VCR's). These systems have the drawback that the image quality is compromised, the tape material of the video tape wears out, only single images can be recorded, and the recording time is relatively short due to the recording time being limited by the length of the tape material which can be wound into a video tape casing. Using present technology, some of these drawbacks are overcome by time-lapse recorders which extend the recording time, and multiplexers which combine multiple images onto a single video tape.

Digital video technology requires large amounts of data to be stored and processed as the amount of data in a video image is very large. Attempts have been made to combat this drawback by storing images at reduced resolution, using compression techniques, or dropping the rate of recording of images. Thus a need exists for a system which can store images from multiple image sources onto a single digital storage system without significantly sacrificing image quality and/or recording rate of images.

Standard video cameras sample the video scene typically at a frame rate of 25 frames per second, split up into two fields each. If two such fields are considered as separate images the typical field rate is 50 fields per second. When observing high speed objects it is often required that the trajectory of these objects be sampled more often than at this rate. Such cameras are available but are extremely expensive.

The applicant believes that the above can be achieved by multiplexing a number of standard cameras, combined and synchronised in such a way that a much higher sampling rate can be achieved, in spite of the slow sample rate per camera. Such a device, it is believed, would be less expensive than conventional high speed cameras.

It is believed that is possible to determine three-dimensional trajectories of movement by analysing the slight offset between different camera line of sight. Such a system also allows multi-angle views for 3-D viewing. The inventor believes that the invention is useful in many applications where high speed recording or processing is needed. One example is in aiding a third umpire in cricket where the trajectory of the ball could be captured and because of the many samples of the ball the trajectory can easily be determined and predicted to assist with LBW, runout and stumping decisions. Other applications could possibly include speed monitoring of vehicles or other objects. As such it will be useful in measuring equipment.

According to the invention, there is provided a method of preparing video

images for storing, the method including the steps of:

compressing a digital video image to reduce the number of bytes of

storage required to store the digital video image; combining or multiplexing a plurality of the compressed digital video images into a data block or data stream in a manner which allows the individual digital video images to be extracted from the data block or stream, when stored; and optionally, storing the data block or data stream onto a single medium.

The method may include the prior step of digitising analogue video images

from an analogue video source in, for example, CCIR, PAL, NTSC, EIA, or

the like, formats.

The method may include the step of storing the digitised digital video images in a frame buffer or similar memory prior to compression thereof.

The compression step may be achieved by JPEG, Wavelet, H261 , MPEG-X

(X= 1, 2, 3 , or higher), or any other digital compression technique. The method may include the step of extracting individual video images from the combined or multiplexed video images by de-multiplexing. The demultiplexing may be achieved by identifying the bytes of the individual video images in the multiplexed data block, thereby to permit viewing of video images from a specific source from the multiplexed data block containing the video images from several sources.

The invention extends to a method of storing video images, including the steps of the preparing method described above and further including the additional step of storing the multiplexed data stream or data block on a writeable data storage medium, for example, magnetic tape, CD-ROM, magnetic hard disk, DVD-ROM, or any suitable storage medium capable of supporting the data transfer rate required by the system.

In one embodiment, the storing of multiplexed data is on readable and writeable data storage medium, to permit simultaneous, or substantially simultaneous, reading and writing to and from the medium, of data images, for example, to permit simultaneous viewing and recording of video images.

In the case of a randomly accessible high speed storage medium, such as a

computer hard disk, it is possible to store multiplexed data, and during off- times in between frames, previously stored video images can be read from the storage medium. According to a second aspect of the invention, there is provided a system for high speed transmission of multiple video images or signals, the method including: multiplexing the video images substantially as described above; and transmitting the thus multiplexed data to a remote location via a data transmission system.

The system may include the de-multiplexing of the multiplexed video images at the remote location, substantially as described above.

According to a further aspect of the invention, there is provided a security system capable of recording images from several conventional surveillance cameras onto a data storage medium, the system including: one or more surveillance cameras; a data multiplexing system using the method for preparing video images for storage, substantially as described above; and a data storage medium for storing the multiplexed data while permitting

viewing of individual surveillance camera video images in real time.

It will be clear to those skilled in the art that a system using the video image storage method or preparation for storage method in accordance with the invention has many applications in addition to those disclosed in this specification. According to yet a further aspect of the invention there is provided a method of using a plurality of cameras for high speed camera work, the method including the steps of: linking a plurality of cameras together in parallel to focus on a common point or scene; dividing a frame period between the plurality of cameras; and multiplexing the images from the plurality of cameras.

The method may include the step of digitising the image or images prior to the multiplexing thereof.

The method may further include the step of storing the camera outputs into one or more buffer memories. Typically each camera would be associated with a buffer memory.

The dividing of the frame period may be performed by a phasing circuit, in which the cameras are slaved to one or more master clock.

The phasing circuit provides phase offset for driving the cameras at a phase offset to one another thus permitting a scene to be sampled many times in between normal sampling periods.

According to yet a further aspect of the invention, there is provided a high speed camera system, the system including: a plurality of cameras configured to be focusable on a common point or scene; a phasing circuit including a master clock, to which each of the plurality of cameras are connected for phase offsetting the cameras from each other; and a multiplexing circuit for multiplexing the phase offset input from the plurality of cameras into a data block for storage.

The system may include one or more buffer memory for intermediate storage of the camera outputs at any stage prior to multiplexing.

The system may also include analogue to digital conversion (A-D) means for converting an analogue camera output to digital format, for storage in the buffer memory. This analogue digital conversion means may convert an analogue video signal, such as CCIR, PAL, NTSC or EIA, into a digital form.

A high speed camera system in accordance with the invention, would typically consist of a number of video cameras, rigidly combined together, aimed at the same object or area. These cameras would all be linked to an external phasing signal which is created such that the frame period is equally divided between a number of cameras. Thus camera one will start its capture at the

beginning of the master phase, T₀, camera two will start at Ti where Ti = T₀ + Ti /N, camera three at T₂ where T₂ = T₀ + 2Tι/N, and so on. (In the above formula T is the frame period, and N is the number of cameras). Each of these camera's outputs (analogue or digital) would be connected to a digital memory storage area either directly or via an analogue to digital converter. Optionally each of these signals could be compressed digitally, by conventional means or in accordance with this invention, before being stored to conserve space and to improve digital transmission rates. The memory system is such that the data can be transferred out of this memory into a larger memory. This transfer mechanism happens sequentially, first from one memory, then the next and so on. In this way a combined video stream is created that contains one frame from the first camera, then one from the next, then one from the next, and so on.

The combined digital stream can now be recorded onto high speed digital disk for later retrieval.

The invention will now be described, for illustration purposes only, with reference to the accompanying schematic drawings and flow diagrams. In the drawings

Figure 1 shows, in flow diagram representation, a system for storage of multiple video images, broadly in accordance with the invention; Figure 2 shows, in schematic representation, the reading and writing of digitised video images to and from a data storage medium, broadly in accordance with the invention; Figure 3 shows, in flow diagram representation, high speed camera system broadly in accordance with the invention; and

Figure 4 shows, in schematic representation, the multiplexing portion of the

system of figure 3.

In Figures 1 and 2, a multiple image storage system 10, broadly in accordance with the invention, is shown consisting of a number of inputs or

cameras 12, whether analogue or digital. Depending on the video signal being analogue or digital the signal is converted into a digital format by an analogue to digital converter 14 which may or may not include a frame buffer memory (not shown).

The resultant digital stream 16 is then compressed using a suitable

compression technique by the compression module 18. The output 20 is a

digital stream at a much lower bit rate. This stream is captured as a block of

data and combined in a multiplexer 22 with similar blocks of data 20 from other compression modules 18 to form a single, continuous stream of data

24, now at a much faster bit rate. The technique of combining is crucial in that it must be possible to uniquely extract any specific block of data from the combined stream. Some buffer memory may be used to perform the bit rate

conversion required for final storage.

This, combined digital stream is now transferred to a data storage device 26

as indicated. Some control functions are added to perform storage device dependent functions such as head seek, cylinder switch, tape rewind, etc. This is done by the control CPU 28 which could also be implemented in a dedicated hardware circuitry.

The user interface 30 can consist of a remote control software suite executing on a remote computer or could include a front panel with switches to perform the necessary functions.

To view a resultant image the combined digital stream 24 is sent, through decompression circuit 32, if required, and then converted from digital to analogue in a digital to analogue converter 34, in order to be displayed on a monitor 36.

The timing diagram, Figure 2, illustrates the real time read and write function. In this diagram the combined digital stream 24 (of Figure 1 ) is shown as it is recorded to the medium 40. During spare times the medium head is moved to the retrieval position 42, and the selected frame or frames, is read from the medium. The head is then moved back into the position 44 to continue with its sequential writing process.

In Figures 3 and 4, the block diagram 50, illustrates the same arrangement as before with respect to the cameras 12 and analogue to digital converters 14. Whilst compression 18 (of Figure 1) can be introduced at this point, the crux of the design is about increasing the effective sampling rate of a camera. In this case it is assumed that the cameras are all observing the same scene. By moving the data of one camera from a memory 52 to the main storage 26(54) at a much faster rate with than that at which it is entered into the memory 52, there is sufficient time to move results from other cameras to the main storage within one frame period. This task of sequentially moving data from all cameras to the main storage is performed by the high speed multiplexer 56.

This process on its own would not necessarily achieve an effective faster sampling rate unless the images captured by any individual camera is sampled at a different time. This function is achieved by the camera synchronisation system 58 which generates a master clock 60, from which suitably delayed synchronisation phases are derived 62, 64, 66. These phases signals 62, 64, 66 are used to force the cameras 12 into sampling at specific, well-timed intervals, independently from one another.

The control system 68 is aimed at providing internal timing control as well as

user interface capability.

To view a resultant image the combined digital stream is sent, through a decompression circuit 70, if required, and then converted from digital to analogue in a digital to analogue converter 72, in order to be displayed on a monitor 74. The timing diagram, Figure 4, illustrates the combinatorial effect of sampling the same image with three independent systems and then achieving the high speed sequence by combining them in the described way.

Claims

1. A method of preparing video images for storing, the method including the steps of: - compressing a digital video image to reduce the number of bytes of storage required to store the digital video image; combining or multiplexing a plurality of the compressed digital video images into a data block or data stream in a manner which allows the individual digital video images to be extracted from the data block or stream, when stored; and optionally, storing the data block or data stream onto a single medium.

2. A method as claimed in claim 1 which includes a first step of digitising analogue video images from an analogue video source.

3. A method as claimed in claim 2 wherein the analogue video source includes any one of the group comprising CCIR, PAL, NTSC, EIA, or the like, formats.

4. A method as claimed in any one of the preceding claims which includes the step of storing the digitised digital video images in a frame buffer

or similar memory prior to compression thereof.

5. A method as claimed in any one of the preceding claims wherein the the compression step is achieved by any one of the group comprising JPEG, Wavelet, H261 , MPEG-X (X= 1, 2, 3 , or higher), or any other suitable digital compression technique.

6. A method as claimed in any one of the preceding claims which includes the step of extracting individual video images from the combined or multiplexed video images by de-multiplexing.

7. A method as claimed in claim 6 wherein the de-multiplexing is achieved by identifying the bytes of the individual video images in the multiplexed data block, thereby to permit viewing of video images from a specific source from the multiplexed data block containing the video images from several sources.

8. A method of storing video images including the steps of the preparing method as claimed in any one of the preceding claims and further including the additional step of storing the multiplexed data stream or data block on a writeable data storage medium, for example, magnetic tape, CD-ROM,

magnetic hard disk, DVD-ROM, or any suitable storage medium capable of supporting the data transfer rate required by the system.

9. A method as claimed in claim 8 wherein the storing of multiplexed data is on readable and writeable data storage medium, to permit simultaneous, or substantially simultaneous, reading and writing to and from the medium, of data images, for example, to permit simultaneous viewing and recording of video images.

10. A method as claimed in claim 8 or 9 wherein multiplexed data is stored on a randomly accessible high speed storage medium, such as a computer hard disk, to permit during off-times in between frames, the reading of previously stored video images from the storage medium.

11. A system for high speed transmission of multiple video images or signals, the system including: - multiplexing the video images according to a method as claimed in any one of the preceding claims; and transmitting the thus multiplexed data to a remote location via a data

transmission system.

12. A system as claimed in claim 11 which includes de-multiplexing of the multiplexed video images at the remote location.

13. A security system capable of recording images from several conventional surveillance cameras onto a data storage medium, the system including:

one or more surveillance cameras; a data multiplexing system using a method for preparing video

images for storing, as claimed in any one of claims 1 to 10; and - a data storage medium for storing the multiplexed data while permitting

viewing of individual surveillance camera video images in real time.

14. A method of using a plurality of cameras for high speed camera work, the method including the steps of: - linking a plurality of cameras together in parallel to focus on a common point or scene; dividing a frame period between the plurality of cameras; and

multiplexing the images from the plurality of cameras.

15. A method as claimed in claim 14 which includes the step of digitising the image or images prior to the multiplexing thereof.

16. A method as claimed in claim 14 or 15 which includes the step of

storing the camera outputs into one or more buffer memories.

17. A method as claimed in any one of claims 14 to 16 wherein each

camera is associated with a buffer memory.

18. A method as claimed in any one of claims 14 to 17 wherein the

dividing of the frame period is performed by a phasing circuit, in which the

cameras are slaved to one or more master clock.

19. A method as claimed in claim 18 wherein the phasing circuit provides phase offset for driving the cameras at a phase offset to one another thus

permitting a scene to be sampled many times in between normal sampling

periods.

20. A high speed camera system, the system including: a plurality of cameras configured to be focusable on a common point or scene; a phasing circuit including a master clock, to which each of the plurality of cameras are connected for phase offsetting the cameras

from each other; and a multiplexing circuit for multiplexing the phase offset input from the

plurality of cameras into a data block for storage.

21. A system as claimed in claim 20 which includes one or more buffer

memory for intermediate storage of the camera outputs at any stage prior to

multiplexing.

22. A system as claimed in claim 20 or 21 which includes analogue to digital conversion (A-D) means for converting an analogue camera output to digital format, for storage in the buffer memory.

23. A system as claimed in claim 22 wherein the analogue digital conversion means is adapted to convert an analogue video signal, selectable from the group including CCIR, PAL, NTSC OR EIA, into a digital form.

24. A system as claimed in any one of claims 20 to 24 wherein the video cameras are rigidly combined together and aimed at the same object or area.

25. A system as claimed in any one of claims 20 to 24 wherein the cameras are all linked to an external phasing signal which is created such that the frame period is equally divided between a number of cameras thereby to permit camera one to start its capture at the beginning of the master phase, T₀, camera two to start at Ti where Ti = T₀ + Ti /N, camera three at T₂ where T₂ = T₀ + 2Tι/N, and so on, T representing the frame period, and N representing the number of cameras.

26. A system as claimed in any one of claims 2o to 25 wherein each of the camera's outputs (analogue or digital) is connected to a digital memory

storage area either directly or via an analogue to digital converter, optionally each of these signals being compressed digitally or by conventional means before being stored to conserve space and to improve digital transmission rates.

27. A system as claimed in claim 26 wherein data can be transferred out of the memory area into a larger memory, the transfer occurring sequentially, first from one memory, then the next and so on in order that a combined video stream is created that contains one frame from the first camera, then one from the next, then one from the next, and so on.

28. A system as claimed in claim 27 wherein the combined digital stream can be recorded onto a high speed digital disk for later retrieval.

29. A method of preparing video images for storing substantially as hereinbefore described and/or as illustrated.

30. A method of storing video images substantially as hereinbefore described and/or as illustrated.

31. A system for high speed transmission of multiple video images or signals substantially as hereinbefore described and/or as illustrated.

32. A security system capable of recording images from several conventional surveillance cameras onto a data storage medium substantially as hereinbefore described and/or as illustrated.

33. A high speed camera system substantially as hereinbefore described

and/or as illustrated.

34. A method of using a plurality of cameras for high speed camera work, substantially as hereinbefore described and/or as illustrated.