WO1991020159A1

WO1991020159A1 - Evaluation of detail in video images, and applications thereof

Info

Publication number: WO1991020159A1
Application number: PCT/GB1991/000954
Authority: WO
Inventors: Alois Martin Bock
Original assignee: National Transcommunications Limited
Priority date: 1990-06-13
Filing date: 1991-06-13
Publication date: 1991-12-26
Also published as: AU7961691A; GB9013217D0

Abstract

The amount of high frequency variation, horizontally and/or vertically, which is in a video image is evaluated by processing a video signal representing the image so as to produce a lower definition version thereof, reconstructing the video signal from the lower definition version, subtracting the reconstructed signal from the original signal to produce an error signal and low pass filtering the modulus of the error signal. The resultant signal is indicative of the amount of detail in the video image and, for example, may be used, in a system where a high definition image signal is conveyed by the combination of a low definition signal (compatible with conventional receivers) and an error signal, to decide whether or not to transmit the error signal in respect of a particular part of the video image.

Description

Evaluation of detail in video images, and applications thereof

The present invention relates to evaluation of the amount of spatial detail (high frequency variation horizontally and/or vertically) present in video images and to applications of the result of the evaluation. In particular, the invention relates to the making and the use of such an evaluation in relation to video signals comprising an image signal and an enhancement signal.

In the field of high definition television, proposals have been made for the transmission of the high definition image information in the form of a basic image signal accompanied by an enhancement signal. Such an arrangement is designed to be compatible with existing television receivers which would produce a picture using the basic image signal alone.

When dealing with video signals of a type comprising an image signal and an enhancement signal (which may be used in combination to produce an overall image) it can be useful, for a variety of applications, to know in which areas of the picture there is detail, ie high frequency spatial variation.

The present invention provides an evaluation of the high frequency spatial variation in an image represented by a video signal by processing an enhancement signal forming part of the video signal. According to a preferred embodiment of the invention the processing of the enhancement signal comprises providing a signal indicative of the magnitude of the enhancement signal and passing this magnitude signal through a spatial low pass filter. The resultant signal, which is at scanning rate, is indicative of the instantaneous amount of high-frequency detail in the image.

One area in which the evaluation technique of the present invention finds application is in the control of the data rate of a transmitted enhancement signal, for example in a PAL-compatible system with digital enhancement signals where the bit-rate of the enhancement signal needs to match a given, constant channel capacity.

The picture quality obtained using conventional bit-rate reduction techniques, such as quantisation or Discrete Cosine-Transformation (DCT), on an enhancement signal generally deteriorates for high compression ratios. For bit-rate reduction of digital enhancement signals this is not acceptable because application of the enhancement signal would worsen picture quality, not improve it. In general it is often not feasible to vary the compression ratio of a given coding algorithm to such an extent that the bit-rate of the enhancement signal is constant for any given source material.

The present invention makes it possible at the source to spatially reduce the amount of enhancement so that the image compression ratio can be kept constant. This is achieved by using the signal indicative of high frequency detail in the image (produced by the evaluation technique outlined above) as a basis for deciding which areas of the picture have the biggest impact on picture quality and so are more important to enhance; the enhancement signal is modified so as to include full enhancement information only in relation to these more important areas.

Further features and advantages of the present invention will become clear from the following detailed description of embodiments thereof, given by way of example, with reference to the accompanying drawings in which:

Figure 1 shows in block diagrammatic form a transmitter of a system for selective enhancement of PAL signals using the evaluation technique according to the invention; and

Figure 2 shows in block diagrammatic form a receiver compatible with the transmitter of Figure 1.

The following description, given in relation to figures 1 and 2, relates to the application of the evaluation technique of the present invention in a system for selective enhancement of PAL television signals. As mentioned above, the aim of the system is to fully enhance only those parts of the picture which tend to have the biggest impact as far as picture quality is concerned, but not others. The difficulty is to decide which parts of the picture are important and which are not. It is considered important, for example, to enhance an entire object if it contains a significant amount of detail whereas picture areas with low detail density need not be enhanced at all. Furthermore, once the enhancement signal is applied to a certain object, this object should continue to be enhanced even if it starts to move. Thus the algorithm described below produces clusters of enhancement, with areas in between which are not enhanced.

Figure 1 shows a block diagram of a transmitter in a PAL-compatible high definition television system according to an embodiment of the present invention. The high definition source signal, S, contains twice the usual number of lines per field.

In order for the transmission to be compatible with conventional PAL receivers a PAL signal usable by those receivers is produced by downconverting the source signal, in a downconvertor 1, to the usual line rate and then passing the downconverted signal to a PAL coder 2.

In this type of system the transmitted PAL signal would be used in a high definition receiver to reconstruct the broad features of the original high definition image (by decoding the received signal and upconverting it to the source line rate). The resultant "reconstructed" signal will differ from the original high definition source signal, S, because of the limitations of the conversion process and of the PAL coder/decoder. Accordingly in order for the high definition receiver to be able to produce a high definition image it is necessary for an enhancement signal to be transmitted which carries information on the differences between the high definition source signal and the "reconstructed" signal.

Conventionally the enhancement signal is produced at the transmitter by additionally feeding the provided PAL signal to a PAL decoder 3 and upconvertor 4 (so as to produce a signal, r, corresponding to the "reconstructed" signal at a high definition receiver) and subtracting the resulting signal, r, from the high definition source signal, S, to produce the enhancement signal. A compensatory delay (not shown) is provided in the path of signal, S, from the source to the subtractor. This enhancement signal is coded to reduce the bit rate and then transmitted.

In the Fig. 1 embodiment of transmitter an initial enhancement signal, e-] , is produced in the above-described manner but this signal is modified, before coding and transmission, so as to include full enhancement information only- in relation to those parts of the image which have greatest high frequency spatial^' variation.

The evaluation of the degree of high frequency detail in the image is made in the circuitry 5 marked with dashed lines in figure 1. First the magnitude of the initial enhancement signal, e-* , is calculated in the device 6 to give an output, e2, where: e₂ = |e-^*|

Subsequently the signal e₂ is spatially low- pass filtered to produce a signal e3. This signal is indicative of the instantaneous high frequency detail in the image and is used in determining how the initial enhancement signal, e-* , is modified in a spatial reduction unit 9.

The size of the spatial low-pass filter 7 (ie the number of lines and pixels it covers) is more important than the detailed shape of the filter, (which is intended to provide a smooth transition between areas of little or no enhancement and neighbouring areas of full enhancement). Although other shapes are possible, for the selective enhancement system illustrated in Fig. 1 it is preferred to use a filter having a triangular shape, with a slope of about 10% per pixel (or vertically 10% per line) but truncated to 5 pixels (3 lines) .

In order to ensure that the selection signal, e3, indicative of high frequency detail in the image, is used by the spatial reduction unit 9 in relation to the appropriate portion of the enhancement signal, e-* , a compensatory delay (not shown) may be included in the path of the enhancement signal e*- to the spatial reduction unit 9.

Since it is desired to transmit full enhancement data only for the most visually significant areas of the picture it would be possible for the spatial reduction unit 9 to output the input initial enhancement signal e-* only for picture areas where there is great high frequency detail, ie where the selection signal, 63, is greater than-a threshold level, and to output zero for other picture areas. In practice, however, it is preferable to have a transition region between no enhancement and full enhancement in order to reduce the visibility of the boundaries. A linear transition may be obtained using a lower threshold level q-] and a higher threshold level q₂ , as follows: e₄ = 0 for e3 < q-* ; ^e4 = e-3 ~ Q1 ^e1 f°^r qi e₃ < q₂; 2 ^~ <31 e = e-i for e₃ > q₂ where e₄ = output from spatial reduction unit 9.

The spatially reduced enhancement signal, e₄, is then passed to the bit-rate reduction unit 10 and then to the transmission buffer 11 whose output is used as the enhancement signal to be transmitted with the PAL signal to form the PAL-compatible high definition video signal.

The reduction of the amount of information required to be carried in the enhancement signal can be exploited by the bit rate reduction algorithm. After the spatial reduction process the resultant signal, e₄, is likely to contain a large number of zeros and the bit rate reduction algorithm may substantially reduce the number of output bits representing those zeros by, for example, run-length coding, or block activity signalling (eg if the enhancement signals for a block of pixels contain only "zeros" then the enhancement signals for the block are not transmitted but are replaced by a single bit, or a code, indicating "zero" block activity) .

Preferably, as shown in Fig. 1 , the two threshold levels, q-* and q₂ used by the spatial reduction unit 9 are related to the fill level of the output (transmission) buffer 11.

Because the reduction of the enhancement signal is carried out at the transmitter, the high definition receiver complexity can be kept to a minimum, as shown in Figure 2.

In the simple high definition receiver of Figure 2 the PAL signal and enhancement signal portions are extracted from the received signal. The PAL signal is fed to a conventional PAL decoder 20 and the line rate of the decoded output is increased by using an upconvertor 21 (which may be of conventional type). The enhancement signal is loaded into a receiver buffer 23 and then into a bit rate expansion unit 24. The processed PAL and enhancement signals are combined in an adder 25 to produce an enhanced signal for display.

Application of the spatial reduction of the enhancement signal means that a simpler bit-rate compression algorithm can be used for the same level of enhancement. As an example, simulations have shown that spatial reduction followed by simple quantisation can . achieve a picture quality which is at least as good as intra-field DCT on its own. Implementation of the selective enhancement circuit in the transmitter can, therefore, lead to a significant reduction of the complexity of receivers.

The method described above selectively enhances only certain parts of a picture. Simulations have shown that for realistic values of the two thresholds q*- and q₂ the method can be used in conjunction with conventional image compression techniques to reduce the bit-rate of digital enhancement signals considerably while, at the same time making sure that the output bit-rate after buffering is constant. By spatially reducing the enhancement signal before coding it is guaranteed that the bit-rate compression will operate in a region where the enhancement signal can faithfully be reproduced.

If the spatial reduction is brought to an extreme (because there is only very low channel capacity available for the enhancement signal), then only small areas of the picture are enhanced rendering the enhancement system without much effect on picture quality. In such a case it is preferable to operate at a lower sampling rate at the source. It is prefered to use a lower source sampling rate where otherwise the enhancement signal would require compression to less than about 0.05 bits/pixel.

Generation of the signal e3 by the above-given method can also be of benefit for the control of other parameters in video systems.

It is well known that the visibility of noise is greatly reduced in areas of high picture detail. The instantaneous level of auxiliary signals which introduce some noise to the main video signal (such as a signal in quadrature with the vision carrier) may therefore be controlled using the signal e3 with the aim of keeping the visibility of the noise at a low but constant level.

At the receiver the level of the auxiliary signal may be re-adjusted in a number of different ways. For example: an error signal similar to that produced at the transmitter may be generated at the receiver and used to control the readjustment of the auxiliary signal's level; the auxiliary signal itself may be related to the error signal and may, indirectly or directly, convey information about its level (eg one field in advance); or, for a relatively small variation in the level of the auxiliary signal, a form of automatic gain control may be used at the receiver (if the auxiliary signal conveys digital information then the effect of the resulting variation in its signal-to- noise ratio may be reduced, if necessary, using forward error correction).

In another embodiment, an error signal according to the invention may be produced to control the application of a non-linear signal enhancement process at the receiver. Again the error signal is used to reduce the enhancement process in picture areas with little or no detail (where the application of such an enhancement process would only produce "artefacts" or increase noise).

Claims

CLAIMS :

1. A method for evaluating the amount of high frequency variation horizontally and/or vertically there is in a video image represented by a video signal (S), the method comprising the steps of: processing said video signal (S) so as to produce a basic video signal of lower definition than the original video signal; processing said basic video signal so as to substantially reconstruct the original video signal; comparing the reconstructed signal (r) with the original video signal (S) so as to produce an error signal indicative of the difference therebetween; and low pass filtering the error signal whereby to produce a signal indicative of the amount of high frequency variation horizontally and/or vertically there is in.the image represented by the original video signal.

2. A method according to claim 1 , wherein the comparing step comprises taking the modulus of the difference between the original video signal (S) and the reconstructed signal (r).

3. A method according to claim 1 or 2, wherein the filtering step comprises passing the error signal through a low pass filter operative over five pixels of the video image.

4. A method according to claim 1, 2 or 3, wherein the filtering step comprises passing the error signal through a low pass filter operative over three lines of the video image.

5. A method according to any one of claims 1 to , wherein the filtering step comprises passing the error signal through a low pass filter having a frequency response which is triangular in shape.

6. A method according to claim 5, wherein the slope of the frequency response of the filter is of the order of 10% per pixel and/or, vertically, 10% per line, of the video image.

7. A method for processing a video signal, the method comprising the steps of: evaluating, by the method according to any of claims 1 to 6 , the amount of high frequency variation horizontally and/or vertically there is in a video image represented by the video signal; outputting the basic video signal; and comparing the filtered error signal with a first threshold level, q-j , outputting a signal related to the difference between the original video signal and the reconstructed signal when the magnitude of the filtered error signal exceeds said first threshold level, and outputting a zero signal when the filtered error signal is lower than the first threshold level,

<31 ^•

8. A method according to claim 7, wherein the comparing step comprises comparing the filtered error signal with a second threshold level, q₂, outputting a signal which is a fraction of the difference between the original video signal and the reconstructed signal when the magnitude of the filtered error signal is between the first and second threshold levels, the size of the fraction being related to the ratio of the difference between the filtered error signal and the first threshold level, q-* , to the difference between the first, qi , and second, q , threshold levels, and outputting a signal representative of the difference between the original video signal and the reconstructed signal when the magnitude of the filtered error signal exceeds said second threshold level, q .

9. A method according to claim 7 or 8, and further comprising the steps of performing a bit rate reduction on the signal output in the comparing step, monitoring the bit rate of the signal resulting from the bit rate reduction step, and altering the threshold level or levels used in the comparing step dependent upon the magnitude of the monitored bit rate.

10. A method for processing a video signal intended to be transmitted in the form of a main signal and an auxiliary signal modulated in quadrature with said main signal, the method comprising the steps of: evaluating, by the method according to any one of claims 1 to 6, the amount of high frequency horizontal and/or vertical variation there is a video image represented by the video signal; and controlling the level of the auxiliary signal such that the level of the auxiliary signal is relatively high when the magnitude of the filtered error signal is relatively high and the level of the auxiliary signal is relatively low when the magnitude of the filtered error signal is relatively low.

11. Apparatus for evaluating the amount of high frequency variation horizontally and/or vertically there is in a video image represented by a video signal (S) the apparatus comprising: means (1,2) for processing said video signal (S) so as to produce a basic video signal of lower definition than the original video signal; means (3,4) for processing said basic video signal so as to substantially reconstruct the original video signal; means for comparing the reconstructed signal (r) with the original video signal (s) so as to produce an error signal indicative of the difference therebetween; and low pass filtering means (7) for filtering the error signal, whereby to produce a signal indicative of the amount of high frequency variation horizontally and/or vertically there is in the image represented by the original video signal.

12. Apparatus according to claim 11, wherein the comparing means comprises means (6) for taking the modulus of the difference between the original video signal (S) and the reconstructed signal (r).

13. Apparatus according to claim 11 or 12, wherein the filtering means (7) is adapted to operate over 5 pixels of the video image.

14. Apparatus according to claim 11, 12 or 13, wherein the filtering means is adapted to operate over 3 lines of the video image.

15. Apparatus according to any one of claims 11 to 14, wherein the filtering means (7) has a frequency response which is triangular in shape.

16. Apparatus according to claim 15, wherein the slope of the frequency response of the filtering means (7) is of the order of 10% per pixel and/or vertically 10% per line, of the video image.

17. Apparatus according to any one of claims 11 to 16, wherein the processing means (1,2) for producing a basic video signal comprises a downconverter (1) and the processing means (3,4) for producing the reconstructed video signal comprises an upconverter (4).

18. Apparatus for processing a video signal, comprising: apparatus according to any one of claims 11 to 17 for evaluating the amount of high frequency variation horizontally and/or vertically there is in a video image represented by the video signal; means for outputting the basic video signal; and means for comparing the filtered error signal with a first threshold level, qi , outputting a signal related to the difference between the original video signal and the reconstructed signal when the magnitude of the filtered error signal exceeds said first threshold level, and outputting a zero signal when the filtered error signal is lower than the first threshold level, qi .

19. Apparatus according to claim 18, wherein the comparing means comprises means for comparing the filtered error signal with a second threshold level, q , outputting a signal which is a fraction of the difference between the original video signal and the reconstructed signal when the magnitude of the filtered error signal is between the first and second threshold levels, the size of the fraction being related to the ratio of the difference between the filtered error signal and the first threshold level, qi , to the difference between the first, q , and second, q₂, threshold levels, and outputting a signal representative of the difference between the original video signal and the reconstructed signal when the magnitude of the filtered error signal exceeds the second threshold level, q .

20. Apparatus according to claims 18 or 19, and further comprising means (10) for performing a bit rate reduction on the signal output by the comparing means, means for monitoring the bit rate of the signal output from the bit rate reduction means (10), and means for altering the threshold level or levels used by the comparing means dependent upon the magnitude of the

21. Apparatus for processing a video signal intended to be transmitted in the form of a main signal and an auxiliary signal modulated in quadrature with said main signal, the apparatus comprising: apparatus according to any on of claims 11 to 17 for evaluating the amount of high frequency horizontal and/or vertical variation there is in a video image represented by the video signal; and means for controlling the level of the auxiliary signal such that the level of the auxiliary signal is relatively high when the magnitude of the filtered error signal is relatively high, and the level of the auxiliary signal is relatively low when the magnitude of the filtered error signal is relatively low.