WO1994003013A1 - Image processing system - Google Patents

Image processing system Download PDF

Info

Publication number
WO1994003013A1
WO1994003013A1 PCT/AU1993/000368 AU9300368W WO9403013A1 WO 1994003013 A1 WO1994003013 A1 WO 1994003013A1 AU 9300368 W AU9300368 W AU 9300368W WO 9403013 A1 WO9403013 A1 WO 9403013A1
Authority
WO
WIPO (PCT)
Prior art keywords
pixels
video image
pixelε
image
depleted
Prior art date
Application number
PCT/AU1993/000368
Other languages
French (fr)
Inventor
Amedeo Filiberto Sala
Original Assignee
Dr. Sala And Associates Pty. Ltd.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Dr. Sala And Associates Pty. Ltd. filed Critical Dr. Sala And Associates Pty. Ltd.
Priority to AU45500/93A priority Critical patent/AU4550093A/en
Priority to JP6504031A priority patent/JPH07509111A/en
Priority to EP93915546A priority patent/EP0651932A4/en
Priority to KR1019950700287A priority patent/KR950703256A/en
Publication of WO1994003013A1 publication Critical patent/WO1994003013A1/en

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/12—Systems in which the television signal is transmitted via one channel or a plurality of parallel channels, the bandwidth of each channel being less than the bandwidth of the television signal
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/12—Systems in which the television signal is transmitted via one channel or a plurality of parallel channels, the bandwidth of each channel being less than the bandwidth of the television signal
    • H04N7/122—Systems in which the television signal is transmitted via one channel or a plurality of parallel channels, the bandwidth of each channel being less than the bandwidth of the television signal involving expansion and subsequent compression of a signal segment, e.g. a frame, a line
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/587—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/59—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/90—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals

Definitions

  • the present invention relates to an image processing system particularly, although not exclusively, envisaged for use in reducing the bandwidth requirements for transmission of video signals (i.e. for high definition television(HDTV) , TV, video and video-phones), increasing the resolution of a video picture for a given bandwidth (HDTV) and in digitising ' signals for storage onto video cassette tapes.
  • video signals i.e. for high definition television(HDTV) , TV, video and video-phones
  • HDTV high definition television
  • the present invention is able to do this by taking into consideration the psychophysical attributes of the human visual perception system, namely BETA APPARENT MOVEMENT and SUPER EDGING (the Julesz stereopsis experiment) .
  • Synchronisation achieved by sequentially scanning video pictures starting at the top left of the frame and progressing to the bottom right on a line by line basis, and repeating the process for subsequent frames.
  • the BETA effect was u ⁇ ed in Au ⁇ tralian Patent No. 493435 where, instead of relying on large number ⁇ of pixel ⁇ in order to di ⁇ play graphical images of high resolution, the BETA effect was simulated and the human visual system was tricked into perceiving a resolution much higher than was actually being displayed.
  • the channel would need to be able to handle a rate of 1,562,500 x 8, or approximately 12.5 megabit ⁇ , 25 times each second! This is equivalent to 12.5 x 25 i.e. 312.5 mega bit/sec.
  • the challenge is to reduce this data rate to the capabilities of ISDN network communication speeds of the order of 64 kb/sec.
  • the first step in handling ⁇ uch a huge task i ⁇ to implement a ⁇ y ⁇ tem which can compress the picture, free it of any redundant information, and further reduce transmission times by sending only the field and block differences rather than the whole field information.
  • This is the intention of the proposed CCITT H261 JPEG/MPEG document covering the Discrete Cosine Transform (DCT), Motion Estimation Prediction (MEP) and ancillary algorithms.
  • the BETA APPARENT MOVEMENT effect (the BETA effect) ;
  • Two dimensional interlacing
  • the present invention relies on these techniques to enable considerable depletion of an image without causing significant los ⁇ of visual intelligibility of the image.
  • the BETA effect is used in a scanning and raster technique which work ⁇ equally well whether the video image is stationary or moving and is referred to as a DEPLETION OPTIMISATION TECHNIQUE raster (DOT ra ⁇ ter) .
  • the DOT ra ⁇ ter adopt ⁇ the BETA effect described in Australian patent 493435, except that, instead of requiring the video image to move with respect to a plurality of spaced apart stationary column ⁇ of video element ⁇ , a ra ⁇ ter of column ⁇ are moved backward ⁇ and forward ⁇ with re ⁇ pect to a fixed or moving video image. Hence, the pixel ⁇ are moved backwards and forwards to create apparent movement - even in still video image ⁇ .
  • Thi ⁇ i ⁇ equivalent to taking the picket fence analogy of Au ⁇ tralian patent 493435 and moving the picket fence with respect to the video image instead of moving the video image with respect to the stationary picket fence.
  • the effect can be simulated by clo ⁇ ing one eye, ⁇ preading your finger ⁇ slightly apart and waving them in your field of view.
  • a row ra ⁇ ter can ⁇ imultaneou ⁇ ly be used to provide a horizontal depletion - to create a two dimensional interlace raster, as shown in Figure la.
  • the raster movement of the pixel ⁇ doe ⁇ not have to be limited to linear movement in rows and column ⁇ , but can be random in 2 dimen ⁇ ions, as shown in Figure lb.
  • the DOT raster produces highly viewable video images even where the fields are reduced from 625 to 64 picture columns, each with only 128 out of 625 vertically arranged picture element ⁇ (pixels). This represent ⁇ a horizontal depletion of 10:1 and a vertical depletion of 4:1, giving an overall depletion of 40:1. That i ⁇ , each field is depleted by 40:1, but a time separated interlace of a plurality of the fields which build up a full video image having a resolution of 625 x 625 pixels.
  • the horizontal resolution which can be achieved is not dependent upon the number of pixels in a single field, but is dependent on the number of unique positions that the DOT raster can acquire over time.
  • the columns can be broken up and rearranged into a two dimensional interlaced array (a checker board pattern a ⁇ ⁇ hown in Figure ⁇ 2a, 2b and 2c).
  • Thi ⁇ ha ⁇ the effect of allowing the entire image to build up over a number of field ⁇ and to reduce the tendency for the viewer to lock onto the moving checker board.
  • the DOT raster does not preclude the use of other data compression techniques such as Discrete Cosine Transform (DCT) and motion prediction algorithms. Hence, even greater ⁇ aving ⁇ in bandwidth are possible.
  • DCT Discrete Cosine Transform
  • the increase in sharpness can be explained by a consideration of the SUPER EDGING effect (see Figures 3a and 3b), where any random structure improves the sharpness of an image, this is the property of the viewer to "see" non ⁇ existent details between certain point ⁇ .
  • the edge ⁇ of the shapes shown in Figure ⁇ 3a and 3b appear extremely sharp even though the edges are formed from a random arrangement of pixels. The sharpne ⁇ s of the edges in the ⁇ e examples would be unobtainable by ⁇ imple connecting lines.
  • an object of the pre ⁇ ent invention to provide an image proce ⁇ ing ⁇ y ⁇ tem relying upon the BETA APPARENT MOVEMENT effect to enhance the perceived re ⁇ olution of a video image.
  • an image proce ⁇ sing sy ⁇ tem having: interlocking means for interlacing the pixel ⁇ of subsequent fields of pixels horizontally; whereby, the interlacing causes the pixels to move backwards and forwards at a rate substantially imperceptible to a human viewer for creating a perceived resolution of a video image formed by a plurality of the fields so interleaved, wherein the perceived resolution is greater than the actual resolution.
  • an image processing system for compressing a video image referred to as an oniginal video image
  • the image processing system comprising: a digitiser mean ⁇ for digitising the original video image, the original video image being formed of a plurality of original video fields each having M rows of pixel ⁇ and N column ⁇ of pixel ⁇ ; and, a process control means for proces ⁇ ing the digitised original fields, the proces ⁇ control mean ⁇ ⁇ electing one pixel out of every d pixels and for deleting the remainder of the d-l pixels for generating depleted field ⁇ in a depleted video image, the depleted field having m row ⁇ of pixel ⁇ and n column ⁇ of pixel ⁇ where m is less than M and n is les ⁇ than N; whereby, a receiver means can receive the depleted video image and can generate d-l pixel ⁇ from each selected pixel and can display each selected pixel and its a ⁇ sociated d-l generated pixel ⁇ in a manner to
  • a method for compre ⁇ sing a video image referred to as an oniginal video image the method compri ⁇ ing the steps of: digitising a field of the original video image into a plurality of data bytes referred to a ⁇ pixel ⁇ , each original video field having M row ⁇ of pixel ⁇ and N column ⁇ of pixel ⁇ ; selecting one pixel out of every d pixel ⁇ ; deleting the remainder of the d-l pixel ⁇ ; and, generating a depleted field of pixels in a depleted video image, the depleted field having m row ⁇ of pixel ⁇ and n column ⁇ of pixel ⁇ where m i ⁇ le ⁇ than M and n i ⁇ less than N; whereby, a receiver means can receive the depleted video image, generate d-l pixels from each ⁇ elected pixel and di ⁇ play each ⁇ elected pixel and its associated d-l generated pixels on a di ⁇ play mean ⁇ in a manner to simulate movement
  • an image proce ⁇ ing ⁇ y ⁇ tem for decompressing a video image referred to a ⁇ a depleted video image
  • the image processing system comprising: a digitiser means for digitising the depleted video image, the depleted video image being formed of a plurality of depleted video fields each having m rows of pixels and n columns of pixel ⁇ ; and, a proce ⁇ control mean ⁇ for proce ⁇ ing the digitised depleted fields, the proce ⁇ control mean ⁇ generating d-l pixel ⁇ from each selected pixel and displaying each selected pixel and its as ⁇ ociated d-l generated pixel ⁇ over a period of time imperceptible to a viewer for ⁇ imulating movement of the pixels on a display to regenerate an original video image having M rows of pixels and N column ⁇ of pixels; wherein, the process controller relies upon the BETA APPARENT MOVEMENT effect in regenerating the original video image.
  • a method for decompressing a video image referred to as a depleted video image the method compri ⁇ ing the ⁇ teps of: digitising the depleted video image, the depleted video image being formed of a plurality of depleted video fields each having m rows of pixel ⁇ and n columns of pixels; selecting a pixel from the depleted video image; generating d-l pixel ⁇ from each selected pixel; displaying each selected pixel and it ⁇ associated d-l generated pixels over a period of time imperceptible to a viewer for ⁇ imulating movement of the pixels on a display means for reconstructing an original video image with the selected pixels and the generated pixels, the reconstructed video image having M rows of pixels and N columns of pixels, where M is greater than m and N in greater than n; wherein, the simulated movement relies upon the BETA APPARENT MOVEMENT effect in reconstructing the original video image.
  • luminance determining means is provided to determine the luminance of the adjacent pixels and to set the luminance component of the pseudo pixel to a value proximate the actual value but different enough so a ⁇ to induce an observer' ⁇ perception sy ⁇ tem to ⁇ elect vi ⁇ ually the mo ⁇ t probable value.
  • Figure la shows a two dimensional interlace of pixel ⁇ , being in an ODD field "0" and an EVEN field "E";
  • Figure lb shows a two dimensional random raster movement of pixels for a modulo-3 raster ⁇ cheme
  • Figures 2a to 2c show the break-up of a plurality of columns of video (Figure 2a) into a single depleted field (Figure 2b) and into two interlaced depleted fields (Figure 2c);
  • Figures 3a and 3b are diagrams showing the effect known a ⁇ SUPER EDGING in relation to a ⁇ quare and a triangle;
  • Figures 4a and 4b are diagra ⁇ ⁇ howing the effect known a ⁇ FILLING-IN in relation to a triangle;
  • Figure 5 i ⁇ a block diagram of an image proce ⁇ ing ⁇ y ⁇ tem in accordance with the pre ⁇ ent invention
  • Figure 6 i ⁇ a block diagram of a receiver of the image proce ⁇ ing sy ⁇ tem shown in Figure 5;
  • Figure 7 is a block diagram showing a transmitter of the image proces ⁇ ing ⁇ y ⁇ tem shown in Figure 5;
  • Figure 8 is a graphical representation of a two interlaced regenerated fields U and G of the receiver shown in Figure 6; and,
  • Figure 9 shows a modulo-4 raster ⁇ cheme.
  • the image proce ⁇ ing system 9 comprises a transmitter 10 and a receiver 12, shown in Figure ⁇ 6 and 7 re ⁇ pectively.
  • a CODEC 14 is connected to an output of the transmitter 10 and another CODEC 16 is connected to an input of the receiver 12.
  • a video ⁇ ource 18 is connected to an input of the tran ⁇ mitter 10 and a video monitor 20 i ⁇ connected to an output of the receiver 12.
  • the receiver 12 ha ⁇ a video ⁇ ignal conditioner 32a and a sync separator 32b connected to a video input 33 which is connected to the CODEC 16.
  • the sync ⁇ eparator 32b i ⁇ connected to a clock circuit 32c and a timing controller 32d which controls a clock timer 32e.
  • the clock circuit 32c typically operates at a frequency of 12 megahertz and is hereinafter referred to as the "pixel clock" 32c.
  • the video conditioner 32a is connected to an analogue digital converter 34a and thereby to a field store 34b.
  • the output of the field store 34b is typically an 8-bit databus which is shown as a thick line in Figure 6.
  • the field store 34b i ⁇ connected to a delay circuit 34c and thereby to an 8-bit latch 34d.
  • the delay of the delay circuit 34c i ⁇ typically about 1 micro ⁇ econd ⁇ o a ⁇ to enable correct ⁇ yncing.
  • the 8- bit latch 34d typically ha ⁇ a refresh rate of about 5 million times per second.
  • the sync and colour subcarrier components are extracted from the video signal at the video input 33 by a colour extractor 32 connected to the video input 33.
  • the colour extractor 32 i ⁇ connected to a colour processor 34 which typically includes a control circuit ⁇ ubstantially the same a ⁇ the remainder of the receiver 12.
  • the colour extractor 34 operate ⁇ on ⁇ tandard B-Y and R-Y ⁇ ignal ⁇ ⁇ ent in the video signal from the transmitter 10.
  • the B-Y and R-Y ⁇ ignal ⁇ are at a reduced re ⁇ olution, ⁇ uch a ⁇ half the re ⁇ olution of the video ⁇ ignal proce ⁇ ed by the pixel processor 26 (the compres ⁇ ed Y ⁇ ignal).
  • the analogue to digital converter 34a and the field store 34b digitise the video luminance signal from the video conditioner 32a.
  • the analogue to digital converter 34a generates digital values corre ⁇ ponding to the luminance of each pixel (hereinafter referred to as the "pixels") "on the fly” as the luminance ⁇ ignal is p-ocessed by the video conditioner 32a.
  • the pixel ⁇ are then ⁇ tored in the appropriate memory address of the field store 34b.
  • the luminance for each pixel is determined to be, for example, one of 256 luminance levels per pixel depending on the instantaneous voltage of the video signal (i.e. one in 2 8 ) .
  • the field store 34b collect ⁇ one full field of the luminance ⁇ ignal and ⁇ tore ⁇ it for further processing. That is, consider that the receiver 12 operates one field behind the incoming video signals at the video input 33.
  • the clock circuit 32c controls the timing of the analogue to digital converter 34a so a ⁇ to digitise the signal received from the video conditioner 34a at the appropriate time to correspond to each pixel location of the video source 18.
  • the ⁇ ync ⁇ eparator 32b re-aligns the clock circuit 32c at the beginning of each line of the video signal by use of the horizontal sync signal contained therein.
  • the outputs of the field ⁇ tore 34b and the 8-bit latch 34d are connected to a pixel proce ⁇ or 36.
  • the pixel proce ⁇ or 36 ha ⁇ a proce ⁇ controller 36a connected to latche ⁇ and buffer ⁇ 36c, 36d and 36e.
  • the latch and buffer 36c is connected to an ALU unit 38c and thereby to a latch and buffer 38d.
  • the clock timer 32e is connected to the 8- bit latch 34d, the latches and buffers 36c, 36d, 36e and 38d.
  • the clock timer 32e combines the clock and sync signal ⁇ from the ⁇ ync ⁇ eparator 32b and the clock circuit 32c for controlling the timing of the pixel proce ⁇ or 36.
  • the proce ⁇ controller 36a reads the pixels from the field store 34b and allows them to proceed to the latch and buffer 36d. Also, the proce ⁇ controller 36a can allow pa ⁇ age of the pixel ⁇ to the latch and buffers 36c and 36e.
  • the pixels which pa ⁇ to the latch and buffer 36d correspond to those un-depleted pixels which were tran ⁇ mitted to the receiver 12.
  • a pseudo noise generator 38a is connected to the
  • the pseudo noise generator 38a allow ⁇ injection of noise pixels into the video image di ⁇ played on the monitor 20.
  • the p ⁇ eudo noi ⁇ e generator 38a typically generates pixels with luminance values which have a random value between the values of the adjacent pixels.
  • the outputs of the latches and buffer ⁇ 36d, 36e and 38d are connected to a digital to analogue converter 40 which i ⁇ connected to a video output 42 and hence to the monitor 20.
  • the pixel from the latches and buffers 36d, 36e and 38d are recombined with the colour information at the digital to analogue converter 40 via the colour proces ⁇ or 34 and with the ⁇ ync information from the ⁇ ync ⁇ eparator 32b hence forming a video signal corresponding to a regenerated video field for display on the monitor 20.
  • the proce ⁇ controller 36a reads pixels from the field store 34b one at a time in sequential order, corre ⁇ ponding to row ⁇ of the video signal from the video source 18 i.e. Ull, U13,...Ul,n (where n is the number of columns) a ⁇ ⁇ hown in Figure 8. However, once the pixel ⁇ Ull, U13...Ul,n have been read the proce ⁇ controller 36a generate ⁇ pixel ⁇ G12, G14,...Gln-l and sends them to the latch and buffer 36d before their each of as ⁇ ociated pixels Ull, U13,...Uln are ⁇ ent to the latch and buffer 36d via the 8-bit latch 34d.
  • the proce ⁇ controller 36a generates the pixels between the un-depleted pixels from the video signal as it reads the un-depleted pixel ⁇ and ⁇ end ⁇ each generated pixel for di ⁇ play before it ⁇ end ⁇ the la ⁇ t pixel read from the field store 34b to the latch and buffer 36d.
  • the pixel ⁇ are referred to a ⁇ "un-depleted pixel ⁇ " because they are the pixel ⁇ which remain after the depletion proce ⁇ of the transmitter 10, described hereinafter.
  • the control processor 36a generates the generated pixel ⁇ G by con ⁇ idering the luminance value ⁇ (between and 0 and 255) of the last two pixels read from the field store 34b and makes the generated pixel luminance value ⁇ tatistically dependent thereon.
  • the value of the generated pixel could be a polynomial interpolation of the values of the two un-depleted pixels.
  • the value could be a random value with the values of the two depleted pixels as its upper and lower limits.
  • the process controller passes control to the latch and buffer 36c and the pixel is generated by the pseudo noise generator 38a and sent to the DAC 40 via the latch and buffer 38d.
  • the process controller 36a then reads the pixels U31, U33,...U3,n of row three of the video image. These are the next pixels in the field store 34b.
  • the control proce ⁇ or 36a al ⁇ o generate ⁇ the pixel ⁇ G32, G34, ...G3n+1.
  • the control proce ⁇ or 36a generate ⁇ the pixels G21, G22,...G2n for the line of pixels between the first and third lines of pixels read from the field store 34b.
  • each depleted field commences with only 312 out of 625 pixels acro ⁇ and 312 out of 625 pixel ⁇ down in each column but re ⁇ ult ⁇ in a full 625 line video ⁇ ignal.
  • each depleted field had only one quarter of the pixels of the original field and the receiver 12 regenerates the other three quarters of the pixels, as shown graphically in Figure 8.
  • the above de ⁇ cription relate ⁇ to a modulo-2 depletion of the pixels of the original video signals. It amounts to a horizontal and vertical interlace of two fields, namely Un,m/Gn,m and un,m/gn,m.
  • the horizontal interlace has the effect of moving the image back and forth behind "a picket fence" and hence induces the BETA effect to give the illu ⁇ ion of higher than actual re ⁇ olution to the viewer.
  • the vertical interlace give ⁇ higher vertical re ⁇ olution.
  • the transmitter 10 ha ⁇ a tran ⁇ mi ⁇ ion ⁇ ignal conditioner 22a and a ⁇ ince ⁇ eparator 22b connected to a video input 23 which i ⁇ driven by the video ⁇ ource 18 (such as a video camera or HDTV program or VCR or the like).
  • the ⁇ ync separator 22b is connected to a clock circuit 22c and a timing controller 22d which controls a clock timer 22e.
  • the clock circuitry 22c typically operates at a frequency of 12 megahertz and is hereinafter referred to as the pixel clock 22c.
  • the video conditioner 22a is connected to an analogue digital converter 24a and thereby to a field store 24b.
  • the output of the field store 24b is typically an 8-bit databus which i ⁇ ⁇ hown a ⁇ a thick line in Figure 7.
  • the field ⁇ tore 24b is connected to a delay circuit 24c and thereby to an 8- bit latch 24d.
  • the delay of the delay circuit 24c is typically about 1 microsecond ⁇ o a ⁇ to enable correct syncing.
  • the 8-bit latch 24d typically has a refresh rate of about 5 million times per ⁇ econd.
  • the signal conditioner 22a extracts the luminance component from the video signal.
  • a colour extractor 22 extracts the colour component from the video signal received from the video source 18.
  • the colour extractor 22 i ⁇ connected to a colour proce ⁇ or 27 which typically includes a control circuit substantially the same as the remainder of the transmitter 10.
  • the colour extractor 27 operates on standard B-Y and R-Y signals sent in the video signal from the tran ⁇ mitter 10.
  • the B-Y and R-Y ⁇ ignal ⁇ are at a reduced re ⁇ olution, such a ⁇ half the re ⁇ olution of the video ⁇ ignal processed by the pixel processor 26 (the compres ⁇ ed Y signal) .
  • the analogue to digital converter 24a and the field store 24b digitise each field received from the video source 18 and store it for digital proces ⁇ ing.
  • the analogue to digital converter 24a generate ⁇ digital values of the luminance of each pixel , (hereinafter referred to as "pixels") "on the fly" as the luminance signal is processed by the video conditioner 22a.
  • the luminance is determined to be, for example, one of 256 luminance levels per pixel (i.e. one in 2 8 ) .
  • the clock circuit 22c controls the timing of the analogue to digital converter 24a so a ⁇ to digitise the signal received from the video conditioner 24a at the appropriate time to correspond to each pixel location of the video source 18.
  • the sync separator 22b re-aligns the clock circuit 22c at the beginning of each line of the video signal by use of the horizontal sync ⁇ ignal contained therein.
  • the pixel processor 26 has a proces ⁇ controller 26a connected to latches and buffers 26c and 26d.
  • the latch and buffer 26c is connected to an ALU unit 28a and thereby to a latch and buffer 28b.
  • the clock timer 22e is connected to the 8-bit latch 24d and the latches and buffer ⁇ 26c, 26d and 28b.
  • the clock timer 22e combines the clock and sync signal ⁇ from the ⁇ ync ⁇ eparator 22b and the clock circuit 22c for controlling the timing of the pixel processor 26.
  • the proces ⁇ controller 26a read ⁇ luminance value ⁇ from the field ⁇ tore 24b one at a time in ⁇ equential order, corresponding to rows of the video signal from the video source 18.
  • the field store 24b collects one full field of the luminance ⁇ ignal and ⁇ tore ⁇ it for further proce ⁇ ing. That i ⁇ , consider that the transmitter 10 operates one field behind the incoming video signal ⁇ at the video input 23.
  • the proce ⁇ controller 26a ⁇ elect ⁇ , for example, the pixel ⁇ corre ⁇ ponding to the ODD rows and the ODD columns, i.e. Un,m for all values of n and . These pixels are referred to as the "un-depleted pixels" and are allowed to proceed to the latch and buffer 26d for transmission to the receiver 12. Also, the proces ⁇ controller 26a allow ⁇ pa ⁇ age of other one ⁇ of the pixel ⁇ the latch and buffer 26c.
  • the pixel ⁇ which pass to the latch and buffer 26c are used to be displayed on a monitor 15 by the ALU unit 28a, the latch and buffer 28b and a digital to analogue converter 31.
  • the digital to analogue converter 31 is al ⁇ o connected to an output of the latch and buffer 26d and therefore, the monitor 15 can ⁇ how a video ⁇ ignal similar to that which will be shown on the monitor 20 at the receiver 12.
  • a pseudo noi ⁇ e generator 28c i ⁇ connected to the ALU unit 28a via a latch 28d.
  • the p ⁇ eudo noi ⁇ e generator 28c allow ⁇ injection of noi ⁇ e pixel ⁇ into the video image displayed on the monitor 15.
  • the process controller 26a ha ⁇ the effect of, for example, selecting ODD pixels from ODD video fields and EVEN pixels from EVEN video fields.
  • the combined effect i ⁇ a field having two fields which are interlaced both horizontally and vertically a ⁇ ⁇ hown in Figure 8. That i ⁇ , every ⁇ econd column and every ⁇ econd row from the original video ⁇ ignal ha ⁇ been deleted.
  • the re ⁇ ultant video ⁇ ignal for tran ⁇ mission i ⁇ referred to a ⁇ "depleted" .
  • each field i ⁇ depleted from 625 down to 312 pixels across and from 625 down to 321 pixels down in each column.
  • each depleted field has one quarter of the information of the original field. This correspond ⁇ to a modulo-2 depletion of the pixel ⁇ of the original video ⁇ ignal ⁇ .
  • the luminance of the depleted pixels is recombined with the sync and colour information at the digital to analogue converter 30 via the colour proce ⁇ or 27 and the sync separator 22b respectively.
  • a ⁇ with the receiver 12 other forms of depletion of the video signal could be u ⁇ ed, ⁇ uch as, for example, modulo 3, 4, 5 etc. or even a relatively random form of depletion as shown in Figure lb in which over sub ⁇ equent fields each pixel describes the path ⁇ hown by line ⁇ 1 to 9.
  • the displayed picture looks similar to Figure 2a where part of a 64 column pattern is depicted;
  • the image proce ⁇ ing ⁇ y ⁇ tem of the present invention allows for considerable compres ⁇ ion of a video signal by relying on the psychophy ⁇ ical attribute ⁇ of the human perception ⁇ ystem to undertake appropriate interpolation of the video image to provide a perceived resolution which is substantially the same a ⁇ that of the unco pre ⁇ sed signal - even though the actual resolution has been ⁇ everely depleted.
  • the DOT ⁇ y ⁇ tem relie ⁇ on the BETA APPARENT MOTION effect to achieve the high compre ⁇ ion by deletion of all information from the video ⁇ ignal which is not necessary for the human perception sy ⁇ tem.
  • the viewability of the interlaced field ⁇ i ⁇ enhanced due to the very high ⁇ tatistical correlation - between the pixels of sub ⁇ equent fields (by stepping the pixels backward ⁇ and forward ⁇ ).
  • the VET then add ⁇ random noi ⁇ e to the ⁇ ignal, once received, in order to make the re ⁇ ultant video image sharper - by relying on the phenomenon of SUPER EDGING.
  • the system of the present invention preferably operates in a digital format and so avoids analogue artefacts (i.e. overshoot and smear) which are very difficult to remove. Statistical manipulation of the video signal can allow for even greater compres ⁇ ion.
  • the video ⁇ ignal, a ⁇ compre ⁇ ed by the DOT ra ⁇ ter could be proce ⁇ ed by a DCT to give a further compression of 40:1, thus giving a total compression of 1600:1.
  • a video signal can be compres ⁇ ed and tran ⁇ mitted via a telephone line having a bandwidth of 64kHz - thu ⁇ being applicable to video telephone without requiring ⁇ pecial or multiple telephone line ⁇ .
  • the DOT can be used to enhance the resolution of a ⁇ tandard video signal to achieve a high definition video result. And, such high definition can be achieved ⁇ imply by proce ⁇ es at the receiver.
  • the DOT could be used to digitise video signals for video recorders and video camera ⁇ .

Abstract

An image processing system (9) having a transmitter (10) capable of selecting a proportion of pixels (Umn) and deleting a proportion of pixels (Gmn, umn, gmn) of a video field for achieving data compression. A receiver (12) is provided for regenerating the deleted pixels (Gmn, umn, gmn) for regenerating the video field. The receiver (12) having a processor control (26a) for horizontally interlacing the regenerated pixels over a period of time substantially imperceptable to a viewer for relying upon the BETA APPARENT MOVEMENT effect for providing an apparent resolution which is greater than the actual resolution of the regenerated video field. Enhancement of the resolution of the video field is also provided by adding random noise pixels into the regenerated video field.

Description

TITLE IMAGE PROCESSING SYSTEM
FIELD OF THE INVENTION
The present invention relates to an image processing system particularly, although not exclusively, envisaged for use in reducing the bandwidth requirements for transmission of video signals (i.e. for high definition television(HDTV) , TV, video and video-phones), increasing the resolution of a video picture for a given bandwidth (HDTV) and in digitising 'signals for storage onto video cassette tapes.
The present invention is able to do this by taking into consideration the psychophysical attributes of the human visual perception system, namely BETA APPARENT MOVEMENT and SUPER EDGING (the Julesz stereopsis experiment) .
BACKGROUND TO THE INVENTION The design of the modern television has been influenced by previous discoveries in the cinema industry and are based on three basic design parameters:
1. Frame Format - in cinema it was discovered that a frame rate of 16 frames per second was needed to sustain smooth apparent motion. This was increased to 24 frames per second in order to reproduce sound. Then the rate was increased to half of the mains frequency (25 Hz in Europe and 30Hz in USA) so as to avoid mains interference.
2. Synchronisation - achieved by sequentially scanning video pictures starting at the top left of the frame and progressing to the bottom right on a line by line basis, and repeating the process for subsequent frames.
3. Flicker - which has important effects on selection of the frame frequency. If the frame frequency iε too low (say 16 Hz or less) flicker results; at medium frequencies (say 25 Hz) stroboεcopic effects are created (i.e. wheels appearing to rotate counter to their actual movement; whereas at too high a frequency (50 Hz) the bandwidth required is too large and costs become prohibitive. In relation to the latter, video compression techniques have been developed to reduce the bandwidth requirements. However, most of these solutions are based on algorithms which are mainly designed to detect and predict changes such as those described in the proposed CCITT standard H261.
Conventional display techniques including television and electronic displays work on the principle that for an image to be seen by a viewer, the complete image has to be represented by the illumination of the appropriate picture elements (pixels) in the form of the image. When these pixels are lit in their appropriate positions, the human eye can reconstruct the complete image, preserving the positional relationships of the pixels.
The biggest drawback of this method of display is that for images that require a higher level of resolution to be properly displayed (i.e. graphical images, including graphical company logos like Coca-Cola), a display consisting of a higher number of pixels iε required. Similarly, for displays that scroll across the display screen to appear to be moving smoothly, large numbers of columnε of these pixels are required, so that the resolution of the image, as it moves across the screen, can be preserved.
However, it was discovered that for moving objects, the human visual system iε able to function perfectly on significantly less information than previously was assumed required. An example of this is an effect known as the BETA APPARENT MOVEMENT or picket fence effect, where an observer views a moving object through a picket fence that reveals no more than 10% of the object at any instant, yet the viewer is able to εee the object perfectly. It is the movement of the object behind the picket fence which is fundamental to the viewer being able to εee the entire object.
Most drivers may notice that when moving (even at low speeds), the rails of bridges, low denεity vegetation, and the common picket fence become effectively transparent.
Similarly, objects become quite visible through the spokes of rotating wheels, or the blades of rotating propellers.
In these cases, up to 90% of an object can be obscured by a moving obstacle, and still the full details of the object can be resolved. The converse also applies; a moving object can be shielded by a fixed a barrier which is up to 90% solid and the full details of he object can still be seen.
This is not due to the "persiεtence of vision", which in the past was ascribed to retinal retention. It is in fact due to the "BETA APPARENT MOVEMENT" effect. This effect is the correct reason for humans being able to interpret and connect different images into a sensation of motion. Conversely, the integration of partially presented images, scrolled over time, is also made possible by the same principle.
The BETA effect was uεed in Auεtralian Patent No. 493435 where, instead of relying on large numberε of pixelε in order to diεplay graphical images of high resolution, the BETA effect was simulated and the human visual system was tricked into perceiving a resolution much higher than was actually being displayed.
In Australian Patent No. 493435 instead of using a display fully covered with pixels, a relatively small number of columns of pixels, spaced relatively far apart, were used (like the gaps in the fence that are the sole source of information). If the image is then moved acrosε the display, either left to right or right to left, at a suitable rate, it was discovered that the human visual system would "fill in the gaps", giving the viewer the perception that the viewed image was being displayed at full reεolution. The viewer thuε would not notice that the image at any given inεtant contained only a small fraction of the complete image. The proceεε of deleting a significant portion of the image and regenerating it over time for recombination by the human visual system is herein referred to as "IMAGE DEPLETION" - the image was DEPLETED.
There are two main advantages to uεing image depletion. Firεt, a display that only requires a column where every tenth column should be results in significant cost savingε in the production and operation of εuch a display. Secondly, if thiε display is fed with information through an existing information network, significantly less information has to be tranεmitted for each "field" of the display to represent an image. This in turn has a two-fold advantage of either transmitting at a higher field rate than would be posεible for the full non-depleted image, or transmitting on a network of narrower bandwidth than would otherwise be required for the non-depleted image.
The main problem with pictures, when stored and transmitted, is that they have a lot of data. Typically, an average television picture needs, when digitised, buffers running into millions of bytes. A PAL standard colour televiεion picture needs three monochrome components, each of some 520,833 bytes. This image is assumed to have 625 x 625 = 390,625 pixels each with an aspect ratio of 4 x 3, add up to 520,833 bytes. The total, for a colour picture, equals three times this number, or approximately 1,562,500 byteε.
To εend such a picture on a TV channel, the channel would need to be able to handle a rate of 1,562,500 x 8, or approximately 12.5 megabitε, 25 times each second! This is equivalent to 12.5 x 25 i.e. 312.5 mega bit/sec.
The challenge is to reduce this data rate to the capabilities of ISDN network communication speeds of the order of 64 kb/sec. The first step in handling εuch a huge task iε to implement a εyεtem which can compress the picture, free it of any redundant information, and further reduce transmission times by sending only the field and block differences rather than the whole field information. This is the intention of the proposed CCITT H261 JPEG/MPEG document covering the Discrete Cosine Transform (DCT), Motion Estimation Prediction (MEP) and ancillary algorithms.
Some of the reduction in transmission bandwidth requirement can be handled by these algorithms, but they have finite limits. We have diεcovered, however, that enhancement to picture quality, as well as reductions in transmisεion bandwidth requirement, can be achieved by improving the machine-to-human interface by the use of
Psychophysics.
Prior art video systems (with the exception of
Australian Patents 493435 and 573024) have been designed on the basis of a machine-to-machine interface and have not taken the special needε of the human perception εyεtem into consideration.
THE ROLE OF PSYCHOPHYSICS In order to compresε a video image for low bandwidth, we use the potential of the visual perception syεtem of the obεerver to fill-in missing details. This requires a series of techniques which rely upon certain unique features in the manner in which the human brain functionε. Hence, we can produce a considerable difference between actual and perceived resolution and so enhance the machine-to-human interface. These techniques are chosen from:-
1. The BETA APPARENT MOVEMENT effect (the BETA effect) ; 2. Two dimensional interlacing;
3. Statistical fill-in; and,
4. Super Edging.
The latter two of which constitute a VISUAL ENHANCEMENT TECHNIQUE (VET). The present invention relies on these techniques to enable considerable depletion of an image without causing significant losε of visual intelligibility of the image. The BETA effect is used in a scanning and raster technique which workε equally well whether the video image is stationary or moving and is referred to as a DEPLETION OPTIMISATION TECHNIQUE raster (DOT raεter) . The DOT raεter adoptε the BETA effect described in Australian patent 493435, except that, instead of requiring the video image to move with respect to a plurality of spaced apart stationary columnε of video elementε, a raεter of columnε are moved backwardε and forwardε with reεpect to a fixed or moving video image. Hence, the pixelε are moved backwards and forwards to create apparent movement - even in still video imageε. Thiε iε equivalent to taking the picket fence analogy of Auεtralian patent 493435 and moving the picket fence with respect to the video image instead of moving the video image with respect to the stationary picket fence. The effect can be simulated by cloεing one eye, εpreading your fingerε slightly apart and waving them in your field of view.
It is the movement of the column raster which enables the present invention to apply the BETA effect to a stationary video image. Also, a row raεter can εimultaneouεly be used to provide a horizontal depletion - to create a two dimensional interlace raster, as shown in Figure la. In fact the raster movement of the pixelε doeε not have to be limited to linear movement in rows and columnε, but can be random in 2 dimenεions, as shown in Figure lb.
The DOT raster produces highly viewable video images even where the fields are reduced from 625 to 64 picture columns, each with only 128 out of 625 vertically arranged picture elementε (pixels). This representε a horizontal depletion of 10:1 and a vertical depletion of 4:1, giving an overall depletion of 40:1. That iε, each field is depleted by 40:1, but a time separated interlace of a plurality of the fields which build up a full video image having a resolution of 625 x 625 pixels.
Hence, the horizontal resolution which can be achieved is not dependent upon the number of pixels in a single field, but is dependent on the number of unique positions that the DOT raster can acquire over time.
The columns can be broken up and rearranged into a two dimensional interlaced array (a checker board pattern aε εhown in Figureε 2a, 2b and 2c). Thiε haε the effect of allowing the entire image to build up over a number of fieldε and to reduce the tendency for the viewer to lock onto the moving checker board. Also, the DOT raster does not preclude the use of other data compression techniques such as Discrete Cosine Transform (DCT) and motion prediction algorithms. Hence, even greater εavingε in bandwidth are possible.
Further, statistical fill-in methods can be used to reduce any artefacts introduced into the video image by the DOT raster. This is achieved with a VISUAL ENHANCEMENT TECHNIQUE (VET) which relies on the injection of band limited noise into the video signal, once decoded, and manifeεtε itself as pixels appearing between the other pixels of the decoded checker board pattern. The luminance value of the noise approximates that of the intensity of neighbouring pixelε. Thiε haε the very surprising effect that the addition of noiεe increaεeε the sharpness of the video image. Also, the injected noise creates pseudo pixelε and hence can double the apparent reεolution. Aε the noiεe doeε not have a fixed poεition within the DOT raster but is overlapped by true pixels, by the action of the Beta effect, the viewer perceives the average of the noise and active image. The increase in sharpness can be explained by a consideration of the SUPER EDGING effect (see Figures 3a and 3b), where any random structure improves the sharpness of an image, this is the property of the viewer to "see" non¬ existent details between certain pointε. The edgeε of the shapes shown in Figureε 3a and 3b appear extremely sharp even though the edges are formed from a random arrangement of pixels. The sharpneεs of the edges in theεe examples would be unobtainable by εimple connecting lines. It is the psychophysics of the human perception εyεtem which interpolateε the random pixelε and createε the perception of an extremely sharp edge - hence SUPER EDGING. The ability of the viewer's visual perception system is such that, lacking full information', the missing information is "created" in dramatic detail. It is thiε ability which enableε a viewer to appreciate "impreεεionistic" paintings. Further exampleε of the viewer "creating" the miεεing information to achieve a more complete image iε shown in Figures 4a and 4b. In each case the viewer can perceive a triangle, yet if the spotε are taken individually (by covering the other two) the viewer "sees" only what is present. SUMMARY OF THE INVENTION
Therefore, it is an object of the preεent invention to provide an image proceεεing εyεtem relying upon the BETA APPARENT MOVEMENT effect to enhance the perceived reεolution of a video image. In accordance with one aεpect of the invention there iε provided an image proceεsing syεtem having: interlocking means for interlacing the pixelε of subsequent fields of pixels horizontally; whereby, the interlacing causes the pixels to move backwards and forwards at a rate substantially imperceptible to a human viewer for creating a perceived resolution of a video image formed by a plurality of the fields so interleaved, wherein the perceived resolution is greater than the actual resolution. In accordance with another aspect of the invention there is provided an image processing system for compressing a video image referred to as an oniginal video image, the image processing system comprising: a digitiser meanε for digitising the original video image, the original video image being formed of a plurality of original video fields each having M rows of pixelε and N columnε of pixelε; and, a process control means for procesεing the digitised original fields, the procesε control meanε εelecting one pixel out of every d pixels and for deleting the remainder of the d-l pixels for generating depleted fieldε in a depleted video image, the depleted field having m rowε of pixelε and n columnε of pixelε where m is less than M and n is lesε than N; whereby, a receiver means can receive the depleted video image and can generate d-l pixelε from each selected pixel and can display each selected pixel and its aεsociated d-l generated pixelε in a manner to εimulate movement of the pixelε on the diεplay to substantially reconstruct the original video image by relying on the BETA APPARENT MOVEMENT effect. In accordance with another aspect of the invention there iε provided a method for compreεsing a video image referred to as an oniginal video image, the method compriεing the steps of: digitising a field of the original video image into a plurality of data bytes referred to aε pixelε, each original video field having M rowε of pixelε and N columnε of pixelε; selecting one pixel out of every d pixelε; deleting the remainder of the d-l pixelε; and, generating a depleted field of pixels in a depleted video image, the depleted field having m rowε of pixelε and n columnε of pixelε where m iε leεε than M and n iε less than N; whereby, a receiver means can receive the depleted video image, generate d-l pixels from each εelected pixel and diεplay each εelected pixel and its associated d-l generated pixels on a diεplay meanε in a manner to simulate movement of the pixels to εubstantially reconstruct the original video image by relying on the BETA APPARENT MOVEMENT effect. In accordance with another aεpect of the invention there iε provided an image proceεεing εyεtem for decompressing a video image referred to aε a depleted video image, the image processing system comprising: a digitiser means for digitising the depleted video image, the depleted video image being formed of a plurality of depleted video fields each having m rows of pixels and n columns of pixelε; and, a proceεε control meanε for proceεεing the digitised depleted fields, the proceεε control meanε generating d-l pixelε from each selected pixel and displaying each selected pixel and its asεociated d-l generated pixelε over a period of time imperceptible to a viewer for εimulating movement of the pixels on a display to regenerate an original video image having M rows of pixels and N columnε of pixels; wherein, the process controller relies upon the BETA APPARENT MOVEMENT effect in regenerating the original video image.
In accordance with another aεpect of the invention there iε provided a method for decompressing a video image referred to as a depleted video image, the method compriεing the εteps of: digitising the depleted video image, the depleted video image being formed of a plurality of depleted video fields each having m rows of pixelε and n columns of pixels; selecting a pixel from the depleted video image; generating d-l pixelε from each selected pixel; displaying each selected pixel and itε associated d-l generated pixels over a period of time imperceptible to a viewer for εimulating movement of the pixels on a display means for reconstructing an original video image with the selected pixels and the generated pixels, the reconstructed video image having M rows of pixels and N columns of pixels, where M is greater than m and N in greater than n; wherein, the simulated movement relies upon the BETA APPARENT MOVEMENT effect in reconstructing the original video image. Preferably, the luminance component of the injected noise is congruent with the luminance of adjacent pixels. Also, the injected noise is preferably band limited .
Preferably, luminance determining means is provided to determine the luminance of the adjacent pixels and to set the luminance component of the pseudo pixel to a value proximate the actual value but different enough so aε to induce an observer'ε perception syεtem to εelect viεually the moεt probable value.
BRIEF INTRODUCTION OF THE DRAWINGS One embodiment, being an example only, of the preεent invention will now be described with reference to the accompanying drawings, in which:-
Figure la shows a two dimensional interlace of pixelε, being in an ODD field "0" and an EVEN field "E";
Figure lb shows a two dimensional random raster movement of pixels for a modulo-3 raster εcheme;
Figures 2a to 2c show the break-up of a plurality of columns of video (Figure 2a) into a single depleted field (Figure 2b) and into two interlaced depleted fields (Figure 2c); Figures 3a and 3b are diagrams showing the effect known aε SUPER EDGING in relation to a εquare and a triangle;
Figures 4a and 4b are diagra ε εhowing the effect known aε FILLING-IN in relation to a triangle;
Figure 5 iε a block diagram of an image proceεεing εyεtem in accordance with the preεent invention;
Figure 6 iε a block diagram of a receiver of the image proceεεing syεtem shown in Figure 5;
Figure 7 is a block diagram showing a transmitter of the image procesεing εyεtem shown in Figure 5; Figure 8 is a graphical representation of a two interlaced regenerated fields U and G of the receiver shown in Figure 6; and,
Figure 9 shows a modulo-4 raster εcheme.
DESCRIPTION OF THE PREFERRED EMBODIMENT The following relateε to an embodiment of the image proceεεing εyεtem 9 capable of providing a video depletion of about 40:1. The image proceεεing system 9 comprises a transmitter 10 and a receiver 12, shown in Figureε 6 and 7 reεpectively. A CODEC 14 is connected to an output of the transmitter 10 and another CODEC 16 is connected to an input of the receiver 12. A video εource 18 is connected to an input of the tranεmitter 10 and a video monitor 20 iε connected to an output of the receiver 12. Optionally, a monitor 21 iε connected to the tranεmitter 10.
RECEIVER The receiver 12 haε a video εignal conditioner 32a and a sync separator 32b connected to a video input 33 which is connected to the CODEC 16. The sync εeparator 32b iε connected to a clock circuit 32c and a timing controller 32d which controls a clock timer 32e. The clock circuit 32c typically operates at a frequency of 12 megahertz and is hereinafter referred to as the "pixel clock" 32c. The video conditioner 32a is connected to an analogue digital converter 34a and thereby to a field store 34b. The output of the field store 34b is typically an 8-bit databus which is shown as a thick line in Figure 6. The field store 34b iε connected to a delay circuit 34c and thereby to an 8-bit latch 34d. The delay of the delay circuit 34c iε typically about 1 microεecond εo aε to enable correct εyncing. The 8- bit latch 34d typically haε a refresh rate of about 5 million times per second. The sync and colour subcarrier components are extracted from the video signal at the video input 33 by a colour extractor 32 connected to the video input 33. The colour extractor 32 iε connected to a colour processor 34 which typically includes a control circuit εubstantially the same aε the remainder of the receiver 12. The colour extractor 34 operateε on εtandard B-Y and R-Y εignalε εent in the video signal from the transmitter 10. Typically, the B-Y and R-Y εignalε are at a reduced reεolution, εuch aε half the reεolution of the video εignal proceεεed by the pixel processor 26 (the compresεed Y εignal).
The analogue to digital converter 34a and the field store 34b digitise the video luminance signal from the video conditioner 32a. The analogue to digital converter 34a generates digital values correεponding to the luminance of each pixel (hereinafter referred to as the "pixels") "on the fly" as the luminance εignal is p-ocessed by the video conditioner 32a. The pixelε are then εtored in the appropriate memory address of the field store 34b. Typically, the luminance for each pixel is determined to be, for example, one of 256 luminance levels per pixel depending on the instantaneous voltage of the video signal (i.e. one in 28) .
For ease of understanding the conception of the receiver consider that the field store 34b collectε one full field of the luminance εignal and εtoreε it for further processing. That is, consider that the receiver 12 operates one field behind the incoming video signals at the video input 33.
The clock circuit 32c controls the timing of the analogue to digital converter 34a so aε to digitise the signal received from the video conditioner 34a at the appropriate time to correspond to each pixel location of the video source 18. The εync εeparator 32b re-aligns the clock circuit 32c at the beginning of each line of the video signal by use of the horizontal sync signal contained therein. The outputs of the field εtore 34b and the 8-bit latch 34d are connected to a pixel proceεεor 36. The pixel proceεεor 36 haε a proceεε controller 36a connected to latcheε and bufferε 36c, 36d and 36e. The latch and buffer 36c is connected to an ALU unit 38c and thereby to a latch and buffer 38d. The clock timer 32e is connected to the 8- bit latch 34d, the latches and buffers 36c, 36d, 36e and 38d. The clock timer 32e combines the clock and sync signalε from the εync εeparator 32b and the clock circuit 32c for controlling the timing of the pixel proceεεor 36. The proceεε controller 36a reads the pixels from the field store 34b and allows them to proceed to the latch and buffer 36d. Also, the proceεε controller 36a can allow paεεage of the pixelε to the latch and buffers 36c and 36e. The pixels which paεε to the latch and buffer 36d correspond to those un-depleted pixels which were tranεmitted to the receiver 12. A pseudo noise generator 38a is connected to the
ALU unit 38c via a latch 38b. The pseudo noise generator 38a allowε injection of noise pixels into the video image diεplayed on the monitor 20. The pεeudo noiεe generator 38a typically generates pixels with luminance values which have a random value between the values of the adjacent pixels.
The outputs of the latches and bufferε 36d, 36e and 38d are connected to a digital to analogue converter 40 which iε connected to a video output 42 and hence to the monitor 20. The pixel from the latches and buffers 36d, 36e and 38d are recombined with the colour information at the digital to analogue converter 40 via the colour procesεor 34 and with the εync information from the εync εeparator 32b hence forming a video signal corresponding to a regenerated video field for display on the monitor 20.
The proceεε controller 36a reads pixels from the field store 34b one at a time in sequential order, correεponding to rowε of the video signal from the video source 18 i.e. Ull, U13,...Ul,n (where n is the number of columns) aε εhown in Figure 8. However, once the pixelε Ull, U13...Ul,n have been read the proceεε controller 36a generateε pixelε G12, G14,...Gln-l and sends them to the latch and buffer 36d before their each of asεociated pixels Ull, U13,...Uln are εent to the latch and buffer 36d via the 8-bit latch 34d. That iε, the generated pixel G12 iε εent to the latch and buffer 36d before the depleted pixel U13 and εo on. Hence, the proceεε controller 36a generates the pixels between the un-depleted pixels from the video signal as it reads the un-depleted pixelε and εendε each generated pixel for diεplay before it εendε the laεt pixel read from the field store 34b to the latch and buffer 36d. The pixelε are referred to aε "un-depleted pixelε" because they are the pixelε which remain after the depletion proceεε of the transmitter 10, described hereinafter.
The control processor 36a generates the generated pixelε G by conεidering the luminance valueε (between and 0 and 255) of the last two pixels read from the field store 34b and makes the generated pixel luminance value εtatistically dependent thereon. For example, the value of the generated pixel could be a polynomial interpolation of the values of the two un-depleted pixels. Alternatively, the value could be a random value with the values of the two depleted pixels as its upper and lower limits. In this case the process controller passes control to the latch and buffer 36c and the pixel is generated by the pseudo noise generator 38a and sent to the DAC 40 via the latch and buffer 38d.
The process controller 36a then reads the pixels U31, U33,...U3,n of row three of the video image. These are the next pixels in the field store 34b. The control proceεεor 36a alεo generateε the pixelε G32, G34, ...G3n+1. Then the control proceεεor 36a generateε the pixels G21, G22,...G2n for the line of pixels between the first and third lines of pixels read from the field store 34b.
The effect of the above is that each depleted field commences with only 312 out of 625 pixels acroεε and 312 out of 625 pixelε down in each column but reεultε in a full 625 line video εignal. Hence, each depleted field had only one quarter of the pixels of the original field and the receiver 12 regenerates the other three quarters of the pixels, as shown graphically in Figure 8. The above deεcription relateε to a modulo-2 depletion of the pixels of the original video signals. It amounts to a horizontal and vertical interlace of two fields, namely Un,m/Gn,m and un,m/gn,m.
The horizontal interlace has the effect of moving the image back and forth behind "a picket fence" and hence induces the BETA effect to give the illuεion of higher than actual reεolution to the viewer. The vertical interlace giveε higher vertical reεolution.
In Figure 9 there iε εhown a graphical representation of a modulo-4 raster scheme. In modulo-4 four fields are effectively interlaced horizontally and step through positions marked "1" through "7". In this scheme the receiver 12 generates pixels G and g for three out of every four columns. Vertical interlacing could be included in this raεter scheme. Further, as εhown in Figure lb the raster could be random (a random 2 dimensional modulo-3) in which over subεequent fieldε each pixel deεcribed the path shown by lines enumerated 1 to 9.
TRANSMITTER The transmitter 10 haε a tranεmiεεion εignal conditioner 22a and a εince εeparator 22b connected to a video input 23 which iε driven by the video εource 18 (such as a video camera or HDTV program or VCR or the like). The εync separator 22b is connected to a clock circuit 22c and a timing controller 22d which controls a clock timer 22e. The clock circuitry 22c typically operates at a frequency of 12 megahertz and is hereinafter referred to as the pixel clock 22c. The video conditioner 22a is connected to an analogue digital converter 24a and thereby to a field store 24b. The output of the field store 24b is typically an 8-bit databus which iε εhown aε a thick line in Figure 7. The field εtore 24b is connected to a delay circuit 24c and thereby to an 8- bit latch 24d. The delay of the delay circuit 24c is typically about 1 microsecond εo aε to enable correct syncing. The 8-bit latch 24d typically has a refresh rate of about 5 million times per εecond. The signal conditioner 22a extracts the luminance component from the video signal. A colour extractor 22 extracts the colour component from the video signal received from the video source 18. The colour extractor 22 iε connected to a colour proceεεor 27 which typically includes a control circuit substantially the same as the remainder of the transmitter 10. The colour extractor 27 operates on standard B-Y and R-Y signals sent in the video signal from the tranεmitter 10. Typically, the B-Y and R-Y εignalε are at a reduced reεolution, such aε half the reεolution of the video εignal processed by the pixel processor 26 (the compresεed Y signal) . The analogue to digital converter 24a and the field store 24b digitise each field received from the video source 18 and store it for digital procesεing. The analogue to digital converter 24a generateε digital values of the luminance of each pixel , (hereinafter referred to as "pixels") "on the fly" as the luminance signal is processed by the video conditioner 22a. Typically the luminance is determined to be, for example, one of 256 luminance levels per pixel (i.e. one in 28) .
The clock circuit 22c controls the timing of the analogue to digital converter 24a so aε to digitise the signal received from the video conditioner 24a at the appropriate time to correspond to each pixel location of the video source 18. The sync separator 22b re-aligns the clock circuit 22c at the beginning of each line of the video signal by use of the horizontal sync εignal contained therein.
The output of the field εtore 24b and the 8-bit latch 24d iε connected to a pixel proceεsor 26. The pixel processor 26 has a procesε controller 26a connected to latches and buffers 26c and 26d. The latch and buffer 26c is connected to an ALU unit 28a and thereby to a latch and buffer 28b. The clock timer 22e is connected to the 8-bit latch 24d and the latches and bufferε 26c, 26d and 28b. The clock timer 22e combines the clock and sync signalε from the εync εeparator 22b and the clock circuit 22c for controlling the timing of the pixel processor 26.
The procesε controller 26a readε luminance valueε from the field εtore 24b one at a time in εequential order, corresponding to rows of the video signal from the video source 18.
For ease of understanding the conception of the transmitter consider that the field store 24b collects one full field of the luminance εignal and εtoreε it for further proceεεing. That iε, consider that the transmitter 10 operates one field behind the incoming video signalε at the video input 23. Referring to Figure 8 the proceεε controller 26a εelectε, for example, the pixelε correεponding to the ODD rows and the ODD columns, i.e. Un,m for all values of n and . These pixels are referred to as the "un-depleted pixels" and are allowed to proceed to the latch and buffer 26d for transmission to the receiver 12. Also, the procesε controller 26a allowε paεεage of other oneε of the pixelε the latch and buffer 26c. These pixels are referred to as "depleted pixelε". The pixelε which pass to the latch and buffer 26c are used to be displayed on a monitor 15 by the ALU unit 28a, the latch and buffer 28b and a digital to analogue converter 31. The digital to analogue converter 31 is alεo connected to an output of the latch and buffer 26d and therefore, the monitor 15 can εhow a video εignal similar to that which will be shown on the monitor 20 at the receiver 12.
A pseudo noiεe generator 28c iε connected to the ALU unit 28a via a latch 28d. The pεeudo noiεe generator 28c allowε injection of noiεe pixelε into the video image displayed on the monitor 15. The process controller 26a haε the effect of, for example, selecting ODD pixels from ODD video fields and EVEN pixels from EVEN video fields. The combined effect iε a field having two fields which are interlaced both horizontally and vertically aε εhown in Figure 8. That iε, every εecond column and every εecond row from the original video εignal haε been deleted. Hence, the reεultant video εignal for tranεmission iε referred to aε "depleted" .
The output of the latch and buffer 26d iε connected to a digital to analogue converter 30 which is connected to the CODEC 14 for tranεmiεεion via, for example, an antenna. The sync separator 22b is connected to the DAC 30 to control its timing. The effect of the above iε that each field iε depleted from 625 down to 312 pixels across and from 625 down to 321 pixels down in each column. Hence, each depleted field has one quarter of the information of the original field. This correspondε to a modulo-2 depletion of the pixelε of the original video εignalε.
The luminance of the depleted pixels is recombined with the sync and colour information at the digital to analogue converter 30 via the colour proceεεor 27 and the sync separator 22b respectively.
Aε with the receiver 12 other forms of depletion of the video signal could be uεed, εuch as, for example, modulo 3, 4, 5 etc. or even a relatively random form of depletion as shown in Figure lb in which over subεequent fields each pixel describes the path εhown by lineε 1 to 9. Thiε iε a two dimenεional verεion of modulo-3.
In modulo-4, aε εhown in Figure 9, each field image iε digitised and saved in the field store 24b in the following manner: - during field 1, εtarting from the first pixel, only 1 pixel in 8 is saved on each line. The displayed picture looks similar to Figure 2a where part of a 64 column pattern is depicted;
- during field 2, starting from pixel 3, only the three-modulo-8 elementε are saved;
- during fields 3 and 4 the same proceεε is used for pixel 5 and 7 respectively.
The combined effect is a frame of 4 fields that looks similar to Figure 2b, in which every second line haε been deleted - both horizontally and vertically.
The image proceεεing εyεtem of the present invention allows for considerable compresεion of a video signal by relying on the psychophyεical attributeε of the human perception εystem to undertake appropriate interpolation of the video image to provide a perceived resolution which is substantially the same aε that of the unco preεsed signal - even though the actual resolution has been εeverely depleted. The DOT εyεtem relieε on the BETA APPARENT MOTION effect to achieve the high compreεεion by deletion of all information from the video εignal which is not necessary for the human perception syεtem. The BETA effect iε achieved in εtill video imageε by moving the pixels, such as, backwards and forwardε. Alεo, the viewability of the interlaced fieldε iε enhanced due to the very high εtatistical correlation - between the pixels of subεequent fields (by stepping the pixels backwardε and forwardε). The VET then addε random noiεe to the εignal, once received, in order to make the reεultant video image sharper - by relying on the phenomenon of SUPER EDGING. Also, the system of the present invention preferably operates in a digital format and so avoids analogue artefacts (i.e. overshoot and smear) which are very difficult to remove. Statistical manipulation of the video signal can allow for even greater compresεion. For example, the video εignal, aε compreεεed by the DOT raεter, could be proceεεed by a DCT to give a further compression of 40:1, thus giving a total compression of 1600:1. Hence, a video signal can be compresεed and tranεmitted via a telephone line having a bandwidth of 64kHz - thuε being applicable to video telephone without requiring εpecial or multiple telephone lineε. Also, the DOT can be used to enhance the resolution of a εtandard video signal to achieve a high definition video result. And, such high definition can be achieved εimply by proceεεes at the receiver. Still further, the DOT could be used to digitise video signals for video recorders and video cameraε.
Modification and variationε εuch aε would be apparent to a skilled addressee are considered within the εcope of the preεent invention. For example, other εyεtems of movement of the pixels could be uεed to take advantage of the BETA effect e.g. circular or random movement of pixelε.

Claims

1. An image procesεing εyεtem having: interlocking meanε for interlacing the pixelε of subsequent fields of pixels horizontally; whereby, the interlacing causes the pixels to move backwards and forwards at a rate subεtantially imperceptible to a human viewer for creating a perceived reεolution of a video image formed by a plurality of the fieldε so interleaved, wherein the perceived resolution is greater than the actual resolution.
2. An image procesεing εyεtem according to claim 1, in which the interlacing meanε interlaceε pixelε horizontally and vertically.
3. An image proceεεing εyεtem according to claim 1, in which the interlacing meanε interlaceε the pixels in a random manner.
4. An image procesεing εyεtem according to any one of the preceding claimε, alεo compriεing a noise generation means for generating pixelε of random luminance for interεperεing between the pixelε for relying upon the εuper edging effect for enhancing the perceived reεolution of the video image.
5. An image proceεεing εyεtem for compressing a video image referred to as an original video image, the image processing εyεtem compriεing: a digitiεer means for digitising the original video image, the original video image being formed of a plurality of original video fields each having M rows of pixels and N columns of pixels; and, a process control means for proceεεing the digitised original fields, the procesε control meanε selecting one pixel out of every d pixels and for deleting the remainder of the d-l pixels for generating depleted fields in a depleted video image, the depleted field having m rowε of pixelε and n columns of pixels where m iε leεε than M and n iε leεs than N; whereby, a receiver means can receive the depleted video image and can generate d-l pixelε from each εelected pixel and can diεplay each εelected pixel and itε aεsociated d-l generated pixelε in a manner to simulate movement of the pixels on the diεplay to εubεtantially reconεtruct the original video image by relying on the BETA APPARENT MOVEMENT effect.
6. An image proceεεing εyεtem according to claim 5, in which the proceεε controller interlaces the pixels in a random manner.
7. A image procesεing εyεtem according to claim 5, alεo compriεing a noiεe generation means for generating pixels of random luminance for interspersing between the pixels for relying upon the εuper edging effect for enhancing the perceived reεolution of the video image.
8. A method for compreεεing a video image referred to as an original video image, the method comprising the steps of: digitising a field of the original video image into a plurality of data byteε referred to aε pixelε, each original video field having M rowε of pixelε and N columnε of pixels; εelecting one pixel out of every d pixelε; deleting the remainder of the d-l pixelε; and, generating a depleted field of pixelε in a depleted video image, the depleted field having m rowε of pixelε and n columnε of pixels where m is less than M and n is less than N; whereby, a receiver meanε can receive the depleted video image, generate d-l pixelε from each εelected pixel and display each selected pixel and its asεociated d-l generated pixels on a display means in a manner to simulate movement of the pixels to substantially reconstruct the original video image by relying on the BETA APPARENT MOVEMENT effect.
9. An image processing system for decompressing a video image referred to as a depleted video image, the image procesεing εystem comprising: a digitiεer means for digitising the depleted video image, the depleted video image being formed of a plurality of depleted video fields each having m rows of pixels and n columns of pixels; and, a process control means for procesεing the digitiεed depleted fieldε, the proceεε control means generating d-l pixels from each selected pixel and displaying each selected pixel and its associated d-l generated pixels over a period of time imperceptible to a viewer for simulating movement of the pixelε on a display to regenerate an original video image having M rows of pixels and N columns of pixels; wherein, the process controller relies upon the BETA APPARENT MOVEMENT effect in regenerating the original video image.
10. An image procesεing system according to claim 9, also compriεing a noiεe generation means for generating pixels of random luminance for intersperεing between the pixelε for relying upon the super edging effect for enhancing the perceived resolution of the video image.
11. A method for decompresεing a video image referred to aε a depleted video image, the method comprising the steps of: digitising the depleted video image, the depleted video image being formed of a plurality of depleted video fields each having m rows of pixels and n columns of pixelε; εelecting.a pixel from the depleted video image; generating d-l pixelε from each εelected pixel; displaying each selected pixel and its associated d-l generated pixelε over a period of time imperceptible to a viewer for εimulating movement of the pixelε on a display means for reconεtructing an original video image with the εelected pixelε and the generated pixelε, the reconεtructed video image having M rowε of pixels and N columns of pixels, where M is greater than m and N in greater than n; wherein, the simulated movement relies upon the BETA APPARENT MOVEMENT effect in reconstructing the original video image.
12. An image procesεing system according to claim 11, also comprising a noiεe generation meanε for generating pixelε of random luminance for interεperεing between the pixelε for relying upon the εuper edging effect for enhancing the perceived resolution of the video image.
PCT/AU1993/000368 1992-07-21 1993-07-21 Image processing system WO1994003013A1 (en)

Priority Applications (4)

Application Number Priority Date Filing Date Title
AU45500/93A AU4550093A (en) 1992-07-21 1993-07-21 Image processing system
JP6504031A JPH07509111A (en) 1992-07-21 1993-07-21 Image processing device
EP93915546A EP0651932A4 (en) 1992-07-21 1993-07-21 Image processing system.
KR1019950700287A KR950703256A (en) 1992-07-21 1993-07-21 IMAGE PROCESSING SYSTEM

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
AUPL372692 1992-07-21
AUPL3726 1992-07-21

Publications (1)

Publication Number Publication Date
WO1994003013A1 true WO1994003013A1 (en) 1994-02-03

Family

ID=3776314

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/AU1993/000368 WO1994003013A1 (en) 1992-07-21 1993-07-21 Image processing system

Country Status (5)

Country Link
EP (1) EP0651932A4 (en)
JP (1) JPH07509111A (en)
KR (1) KR950703256A (en)
CA (1) CA2140893A1 (en)
WO (1) WO1994003013A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2392793A (en) * 2002-07-31 2004-03-10 Blip X Ltd Video compression by removing complementary sets of pixels

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US2479880A (en) * 1936-07-04 1949-08-23 Toulon Pierre Marie Gabriel Discontinuous interlaced scanning system
US3136847A (en) * 1962-07-20 1964-06-09 Bell Telephone Labor Inc Narrow band television with interlace conversion
US3566023A (en) * 1967-08-03 1971-02-23 Itt Sequential dot, digitally encoded television system
GB1232108A (en) * 1967-08-03 1971-05-19
US3586775A (en) * 1968-02-28 1971-06-22 Itt Pseudo-random dot interlace television system
US4068265A (en) * 1974-11-25 1978-01-10 Eli S. Jacobs Method and apparatus for sampling and reproducing television information
EP0146713A1 (en) * 1983-10-19 1985-07-03 Nippon Hoso Kyokai Multiplex subsampling transmission system for a high definition color television picture signal
EP0174056A2 (en) * 1984-09-04 1986-03-12 Nagoya University Bandwidth compression system

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO1987001493A1 (en) * 1985-08-29 1987-03-12 Ran Data Pty. Ltd. Graphic display systems

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US2479880A (en) * 1936-07-04 1949-08-23 Toulon Pierre Marie Gabriel Discontinuous interlaced scanning system
US3136847A (en) * 1962-07-20 1964-06-09 Bell Telephone Labor Inc Narrow band television with interlace conversion
US3566023A (en) * 1967-08-03 1971-02-23 Itt Sequential dot, digitally encoded television system
GB1232108A (en) * 1967-08-03 1971-05-19
GB1236749A (en) * 1967-08-03 1971-06-23 Int Standard Electric Corp Television system
US3586775A (en) * 1968-02-28 1971-06-22 Itt Pseudo-random dot interlace television system
GB1250226A (en) * 1968-02-28 1971-10-20
US4068265A (en) * 1974-11-25 1978-01-10 Eli S. Jacobs Method and apparatus for sampling and reproducing television information
EP0146713A1 (en) * 1983-10-19 1985-07-03 Nippon Hoso Kyokai Multiplex subsampling transmission system for a high definition color television picture signal
EP0174056A2 (en) * 1984-09-04 1986-03-12 Nagoya University Bandwidth compression system

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP0651932A4 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2392793A (en) * 2002-07-31 2004-03-10 Blip X Ltd Video compression by removing complementary sets of pixels

Also Published As

Publication number Publication date
EP0651932A1 (en) 1995-05-10
CA2140893A1 (en) 1994-02-03
KR950703256A (en) 1995-08-23
EP0651932A4 (en) 1995-08-23
JPH07509111A (en) 1995-10-05

Similar Documents

Publication Publication Date Title
US5193000A (en) Multiplexing technique for stereoscopic video system
US4605952A (en) Compatible HDTV system employing nonlinear edge compression/expansion for aspect ratio control
US4628344A (en) Method and apparatus for encoding and decoding video
EP2211557B1 (en) Stereoscopic video sequences coding system and method
CN1083661C (en) Seamless switched converter for automatic imagery scan format
EP0806110B1 (en) Flicker reduction and size adjustment for video controller with interlaced video output
US4794447A (en) Method and apparatus for transmitting and receiving a high definition NTSC compatible television signal over a single DBS channel in time division multiplex form
US3982063A (en) Methods and apparatus for reducing the bandwidth of a video signal
JP2000270326A (en) Method for transmitting high definition television through channel of narrow band width
US4908697A (en) Two-line mac high definition television system
DE4300043A1 (en) Wide-screen TV picture-within-picture horizontal deflection system - transfers succession of lines of smaller picture from memory with delay variable for selected horizontal position.
US4979041A (en) High definition television system
US4679084A (en) Method and apparatus for freezing a television picture
US4704629A (en) Television system and data generator and receiver suitable therefor
EP0187406B1 (en) High resolution television transmission system
CN1127055A (en) Method and apparatus for providing scaleable compressed video signal
US6259479B1 (en) Letterbox filter apparatus and method
AU5720698A (en) System and method for synthesizing three-dimensional video from a two-dimensio nal video source
US5097332A (en) Processing method for wide-aspect-ratio television signal for standards-compatible transmission and display
US6023262A (en) Method and apparatus in a computer system to generate a downscaled video image for display on a television system
DE4424702A1 (en) Device for generating a three-dimensional video signal and a three-dimensional video display device
EP0781493B1 (en) Packed yuv9 format for interleaved storage and efficient processing of digital video data
US4550336A (en) Progressive scan speed-up processor
EP0103488B1 (en) Method and apparatus for encoding and decoding video
US5113242A (en) Two-line MAC high definition television system

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AT AU BB BG BR BY CA CH CZ DE DK ES FI GB HU JP KP KR KZ LK LU MG MN MW NL NO NZ PL PT RO RU SD SE SK UA US VN

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): AT BE CH DE DK ES FR GB GR IE IT LU MC NL PT SE BF BJ CF CG CI CM GA GN ML MR NE SN TD TG

DFPE Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101)
121 Ep: the epo has been informed by wipo that ep was designated in this application
WWE Wipo information: entry into national phase

Ref document number: 1993915546

Country of ref document: EP

WWE Wipo information: entry into national phase

Ref document number: 2140893

Country of ref document: CA

WWP Wipo information: published in national office

Ref document number: 1993915546

Country of ref document: EP

REG Reference to national code

Ref country code: DE

Ref legal event code: 8642

WWW Wipo information: withdrawn in national office

Ref document number: 1993915546

Country of ref document: EP