WO1999013646A2

WO1999013646A2 - Image signal processing method and apparatus

Info

Publication number: WO1999013646A2
Application number: PCT/SE1998/001597
Authority: WO
Inventors: Lars Farm
Original assignee: Limt Technology Ab
Priority date: 1997-09-08
Filing date: 1998-09-08
Publication date: 1999-03-18
Also published as: SE9703233D0; EP1013095A2; AU9101598A; WO1999013646A3; SE9703233L; SE512832C2

Abstract

A processing method and apparatus for use in an image signal coding system for coding an input image signal into a compressed output image signal, the image signal comprising digitized image data in a plurality of pixels. Image data preferably being in the frequency domain, is selectively filtered such that image quality of said compressed output signal is prioritised by retaining pixel coefficients contributing significantly to image quality and filtering out the rest of the pixel coefficients under first conditions. Under other conditions, the image data is filtered such that a high achievable compression factor for said set of pixel coefficients is prioritised over image quality of said compressed output signal by filtering out also pixel coefficients contributing significantly to image quality.

Description

Image Signal Processing Method and Apparatus

Field of the Invention

The present invention relates generally to a processing method and apparatus for image signal compression, and more particularly to an adaptive filter for use in image signal compression, for example in an image signal encoding system.

Background of the Invention

The development of digital technology has entailed new possibilities in the use and processing of images, such as television, film production and image telecommunication. In such applications there is often a need to store or transmit an image signal in digitized form. In order to obtain an acceptable image quality, digitized images, and in particular digitized signals of moving images have to be represented by large amounts of digital data. However, the storage space and processing capacity of image processing systems as well as the available frequency bandwidth of transmission systems are usually limited for technical or economical reasons. Therefore, in order to e.g. store, process or transmit an image signal it is often necessary to compress the large amount of data in order to achieve an acceptable data rate or bit rate of the image signal. The technical background of compression and different state of the art compression techniques are for example described in John Watkinson, Compression in Video & Audio. Focal Press 1995, ISBN 0 240 51394, which is hereby incorporated by reference. A moving image signal or a video signal involves data in four dimensions, viz. the magnitude of a sample, the horizontal and vertical spatial positions and the time. Compression can be undertaken in any combination of the four dimensions, and in fact hybrid-coding techniques combining spatial and temporal compression are known to be the most efficient ones. In this connection, the so called compression factor, defined as the ratio between the source data rate and the output data rate, is a commonly used measure of the efficiency of a performed compression. The underlying aim of image compression is to remove redundancy from an image signal in order to represent the image signal with a minimum of data. Processing of an image signal, such as filtering and signal transforming, is normally carried out in order to express the image information in the best way for identifying and removing redundant data in the actual compression stage. Examples of compression methods known in state of the art are the standard compression schemes ISO JPEG (Joint Photographic Experts Group) and MPEG (Moving Picture Experts Group). JPEG and MPEG are examples of methods that comprise what is called block based image compression, where the image signal is processed block by block. A block is in this context a part of an image consisting of e.g. 8x8 pixels. In the processing, the blocks are transformed from the spatial domain to a frequency domain, in most cases by means of discrete cosine transform (DCT), but other frequency domain transforms are also conceivable. One of the reasons for transforming is that redundancy is easier to detect in the frequency domain than in the spatial domain. In connection with the transform, frequency components are usually quantized and scaled to small numbers, and coefficients are then stored or transmitted as small numbers together with a separate scaling factor. This kind of encoding technique further employ what is known as run-length encoding. In run-length encoding, information about the number of subsequent coefficients having the same value is stored or transmitted instead of the coefficient values themselves. After transformation to the frequency domain, typical image material often give rise to a number of signal transform coefficients having a zero amplitude, and therefore subsequent or trailing zeroes are particularly favourable for these compression schemes.

The decoder, receiving as an input the compressed image signal in the form of transformed and quantized blocks, applies the scaling factor to the small transform coefficients and reproduces the indicated number of zero coefficients, and then transforms each block back to the spatial domain in order to recreate the non- compressed image signal.

The scaling of image signal transform coefficients, e.g. DCT-coefficients as in the exemplifying JPEG and MPEG compression schemes, is one of the major contributors to data compression in this kind of methods. In fact, the quantization scale factor is a very important instrument for controlling the compression factor, which in particular is required in for instance fixed bit rate coding schemes. The scale factor can also be varied in order to adapt to the actual compression need, whereby lowering the scale factor improves the image quality and can be allowed when there is sufficient space for data. Raising the scale factor entails a better compression gain, however, to the price of decreased image quality. When driven too far, the scaling of transform coefficients can cause visible artefacts, for instance in the form of quantization noise such as blocking. The latter appearing when the edges of neighbouring transform blocks do not match and the transform block boundaries become visible as a mosaic structure. The result is a recreated image signal representing images of unsatisfactory poor quality.

The US Patent No. 5,309,231 to Takahiro Hamada et al shows an example of a method for adaptive quantization of a block of coefficients for use in a moving or still- picture encoder. This quantization method operates on two levels, on one hand on picture level and on the other hand on a block level. In summary, the adaptive algorithm contains the following measures: A. For each picture L: al- pre-quantizing by conventional means; a2- computing a complexity measure for the picture L, a provisional code length M' per block and an average provisional code length L' per block; a3- determining a pre-designed number of coefficients to transmit per block dependent on L' as a function of M' such that L' is small when M' is large and vice versa. B. For each block bl- carrying out adaptive quantization by b 1.1 - counting the number N' of significant coefficients such that the luminance threshold is considered significant, the threshold matrix being pre-computed and fixed for all blocks; bl .2- selecting a quantizer by computing a step size or weight Wij = Wij (L^*, N) for the block using L' and N', high N' masking quantization noise and allowing a coarser quantization; bl.3- applying selected quantizer; b2- carrying out adaptive base-band selection and excessive bits elimination by b2.1- selecting scan pattern, i.e. the order if transmitting coefficients, choosing the scan pattern that maximizes the number of significant coefficients; b2.2- trimming the base band from end. dependent on the provisional code length M' and bit rate requirements; b3- transmitting (outputting) quantized coefficients and control signals for effective base-band, scan order, L' and N'. It can be noted that this is a complete adaptive quantization method that adapts to two complexity measures, i.e. the average provisional code length, L', and the local

(blockwise) number of significant coefficients. The threshold is fixed, not adaptive, and is devised for computing a complexity measure by counting the number of significant coefficients. It is, furthermore, a fairly complex method with implementation and processing requirements, which are more severe than e.g. MPEG. The method also requires much general control data to be transmitted for each block or macro-block. This gives inefficient compression and excludes the method for use in MPEG and similar schemes.

The US patent No. 5,301,242, to Gonzales and Viscito shows in its turn a method for adaptive scaling of quantized coefficients in an MPEG or similar moving or still- picture encoders. The scaling is controlled by a scale factor that is part of the MPEG standard, which requires scaling or quantized coefficients without prescribing or even describing how to compute such a scale factor. The adaptive quantization according to this piece of prior art can be summed up in the following steps; A. Choosing two quantizer matrices that can be used through the entire stream in accordance with MPEG-rules; B. For each macro-block, i.e. group of, for example, four blocks in each picture, carrying out adaptive quantization by: bl- pre-quantizing blocks using either of two quantization matrices, but not the actual quantization scale factor; b2- computing dependent on an output buffer status and pre-quantization, the quantize scale factor and for each block of the macro-block finding the "maximum-energy-coefficient"; b3- scaling down all pre-quantized coefficients using the computed scale factor; b4- transmitting the quantized values in accordance with the MPEG specification. In short, this piece of prior art is directed to adaptive scaling of quantized data. The scaling adapts to the "energy" of a block and the filling rate of an output buffer. It is also suitable for MPEG, however, some of the previously mentioned problems related to quantization remain.

The above described problems are examples of the general problem of balancing the need for low bit rate, i.e. by means of a high compression factor, with a requirement for the best possible image quality. In fact, in image compression technology there is a dualism between compression factor and image quality, and these aspects are not always easily reconcilable. None of the known prior art has been capable to solve these problems in an adequate manner.

Summary of the Invention

The object of the invention and the problem to be solved is thus to provide a processing method and an apparatus for use in an image signal coding system capable of improving the balance between required compression factor and image quality. A particular aspect of the problem to be solved is the capability of reducing or eliminating the detrimental effects of quantization and scaling of image signal transform coefficients normally found in compression processes involving image data represented in a frequency domain. A further aspect of the problem to be solved, is to increase the compression factor with maintained subjective image quality or, conversely, to improve the image quality for a given compression factor in other words, to increase image quality per coded bit ratio. Yet other aspects of the problems to be solved is to achieve the objects in existing decoders or encoders without significant changes to these existing devices, in particular without increased running time.

In accordance with the invention, a method and an apparatus is provided which in essence is an adaptive filer operating to pre-process an image signal in order to enhance the effect of a subsequent coding step by increasing the number of zeroes in pixel values or length of zero clusters without impairing image quality.

The adaptive filter in accordance with the invention is based on the occurrence of different conditions or cases during compression of a stream of image data, and measures are taken in order to utilize or meet the requirements of the specific conditions in each case. So, in a first predetermined condition priority may be given to letting through pixel coefficients contributing to image quality, whereas in a second pre-determined condition priority instead may be given to filter out pixel coefficients giving a significant contribution to the output bit rate.

Therefore, the current condition with respect to selected criteria is first evaluated in order to select a subsequent filtering step. A first filtering step is selected in a first case, i.e. in a first pre-determined condition, wherein the compression factor and the resulting image quality is smoothly balanced or the image quality even given priority, preferably when there is a moderate requirement on the instantaneous or the overall compression factor. Normally, an extra margin is thereby created for compression needs later in the image information signal. A second filtering step is selected in a second case, i.e. in a second predetermined condition, wherein the image quality is momentarily sacrificed or disregarded, for example when the image quality has gone below a predetermined minimum quality level due to a strained bit budget or when image details are undetectable by the human vision system. By momentarily sacrificing image quality, preferably when it already has deteriorated, the bit budget balance is recovered faster than with prior art bit budget control and image quality can thereafter be given priority again. When performing the second step in response to e.g. undetectable image detail or the like, corresponding unnecessary image information is sorted out and again an extra margin can be created for later compression needs. In reality, the different cases are often not so distinct and therefore the invention is realised as a dynamically variable adaptive threshold of threshold coefficients Tij devised such that input pixel coefficients Vij are replaced by output pixel coefficients Vij according to: vi_j = uij if |uij| > τij

wherein larger (i,j) corresponds to higher spatial frequencies and higher threshold coefficients T (i, j).

In accordance with the invention, the threshold coefficients may depend on a selection of, for example the following criteria: - output buffer status, i.e. currently available buffer space desired output bit rate; and image properties such as: local image complexity motion - brightness colour edges inter/intra block coding, etc. The inventive processing is carried out by selectively filtering the image signal before it is introduced into the actual compression or coding stage and the normal bit rate control mechanisms of an image signal coding system. The mentioned steps can be carried out separately, i.e. in different cases, or in sequence, depending on the predetermined criteria. Different filtering control parameters are used in different embodiments, and for example the resulting image quality can be indirectly controlled by means of a compression control parameter upon which image quality depends. In accordance with one aspect of the invention, the invention seeks to make it possible to avoid unnecessary quantization and scaling by selectively filtering a frequency domain representation of an image block in order to reduce the amount of information needed to code or compress. A coding system may at most produce a predetermined maximum allowed bit rate for an output compressed image signal and the system has to work to stay within a given bit rate budget. Furthermore, image processing coding systems has to take account not only of the possible compression rate but also of the resulting image quality and, as explained above, there is generally a trade off between image quality and the bit rate of a compressed image signal. The invention is thus based on the inventors realisation that the image quality can be given priority under certain conditions, e.g. when there is a sufficient margin in the bit rate budget, and conversely temporarily disregarded under other conditions, e.g. when the bit rate budget is strained. In block based coding systems, occasional blocks of poor quality are hard to detect due to the fact that they are spatially and temporally limited. The image quality of occasional blocks can therefore be sacrificed in order to balance the bit rate budget.

An exemplifying embodiment of the invention provides a processing method for use in an image signal coding system, wherein an image comprises digitized image data in a plurality of pixels, wherein the image data is transformed into a frequency domain and wherein the system optionally includes quantization means for quantizing image data in accordance with a predetermined required compression factor. A set of pixels, commonly called a block, which constitutes a part of an image, is received as an input. The image data of the pixels are preferably in the form of coefficients in a frequency domain, where the pixels for example may be transformed by means of a discrete cosine transform. In one embodiment, the bit rate budget is controlled by means of a bit rate control parameter, preferably the quantization scale factor. In fact, the current quantization scale factor is an indirect measure of the compression rate of a current block, of the compression rate of precedent blocks, of deviation from a desired compression rate and image quality of a current block. Many compression apparatuses can in fact be looked upon as feedback control loops, where the bit rate is controlled e.g. as in state of the art by varying the quantization scale factor and the resulting bit rate is fed back to the compression apparatus itself. In one embodiment, the invention is applied in the bit rate control loop and operates to reduce the degree of quantization needed in subsequent blocks, whereby quantizing in subsequent blocks can be carried out with a smaller quantizing step size and hence subsequent data can be coded with better precision.

The block of pixels, or rather the frequency transform coefficients of the pixels are selectively filtered in order to reduce the amount of information to be coded. In accordance with an embodiment of the invention, the filtering is carried out in the following manner.

In a first step, when e.g. the instantaneous or current bit rate or another selected condition or parameter is within a predetermined deviation range from a predetermined set point, the coefficients are filtered in accordance with a first coefficient threshold function. The first coefficient threshold function is preferably dependent on coefficient frequency and the state of the encoder. In one embodiment, the coefficient frequency dependency is for the first threshold function devised such that coefficients contributing significantly to image quality are prioritised to have an intact amplitude value and the rest of the coefficients are replaced with a predetermined run-length amplitude value, preferably amounting to zero. In principle, this is carried out such that transform coefficients having a low amplitude are removed and replaced by e.g. zeroes and thereby lengthening the number of consecutive zeroes. This in turn increases the compression factor without sacrificing perceived image quality. Instead, the compression factor is increased because of an increased number of zeroes in the coefficients, a correspondingly decreased number of non-zero coefficients and the fact that adjacent zeroes will be run-length encoded in this embodiment. Large transform coefficients, i.e. coefficients of any frequency having large amplitude, are preferably kept intact since they contribute most significantly to the image quality. Preferably, also low frequency transform coefficients are kept intact since their absence contribute significantly to the visibility of certain coding artefacts, particularly the phenomenon known as blocking. The first step can be used preventively to create a margin for the bit rate budget without compromising with image quality, and is mostly applied when the bit rate is balanced. The above mentioned fact that the first step entails remaining data to be coded with better precision, in its turn often even results in an improved image quality. In a second step, when on the other hand the condition parameter, e.g. the instantaneous or current bit rate is outside said predetermined deviation range, the coefficients are instead filtered in accordance with a coefficient threshold function dependent e.g. on the quantization scale factor or on another parameter or criterion such that the bit rate deviation is decreased. This is in one embodiment in principle carried out by replacing a larger number of coefficients with the predetermined run- length amplitude value. In general, this will entail that more coefficients are filtered out and consequently the quality of the resulting image will be suffering. Preferably, the coefficient threshold function is in this case dynamically adapted to the state of the encoder and to the characteristics of each block or group of blocks. The second step is primarily carried out as a reactive measure in cases when image quality can be relinquished, for example when image quality already has deteriorated due to far driven compression. This second step can also be performed preventively, i.e. to prepare for future compression needs, for example when image quality cannot be appreciated by the human vision system. In one embodiment, the above described conditional steps are performed such that the first step is always carried out as a minimum measure for each block regardless of the condition parameter, e.g. the current bit rate. The second step is thereafter carried out if the condition for that said second step is fulfilled. In another embodiment, the status of the condition parameter, e.g. the current bit rate or a parameter reflecting or indicating the image quality, is first checked and thereafter the appropriate step is carried out. A third embodiment is devised such that the first step or the second steps, or a sequence of said first and second steps are performed dependent on selected condition parameters.

Other criteria or conditions for selecting the one or the other of the filtering steps and/or for designing the coefficient threshold function are for example found in the following image characteristics. The luminance value can be used to trig the prioritising of compression over image quality, since very dark and very bright image areas conceal details and colour to the human eye. Motion within the block also conceals detail and such blocks can also be further compressed. When temporal or spatial masking is used, image quality for masked blocks may be less critical. The characteristics of the coefficient threshold function may also be dependent on whether a processed block is an intra-coded block having coefficients in absolute values, or a non-intra-coded or predicted block involving motion estimation and therefore having coefficients expressing a difference between blocks. To sum-up, a mild filtering is applied e.g. to all blocks or when the bit rate budget is well balanced. The mild filtering allows comparatively much information to be coded. A harsher filtering is applied e.g. for blocks where the bit rate budget is strained or exceeded, and/or for blocks where the harsher treatment is judged to pass unnoticed regarding the image quality. The harsher treatment of coefficients can be applied preventively, when the treatment is expected to be harmless to image quality, and/or reactively when the compression has been found to be insufficient and image is already lost and therefore momentarily unimportant. The inventive procedure is advantageously applied in addition to and before quantization in encoding processes involving a frequency domain transform and quantization. A particular advantage is that a decoder needs no information about the inventive processing.

Further advantages are: simplicity, i.e. easy to implement and having low processive requirements; flexibility, i.e. the inventive filter is easy to adapt to new filtering requirements; independence, i.e. it does not depend on used quantization or coding method, it does not introduce any dependencies on its environment, it does not need to transmit extra information to a subsequent decoder, it does not need to transmit information to any other part of an encoder; it is suitable for MPEG and similar standard encoding schemes; locality, i.e. filtering decisions primarily depend on a current block and the current status of the buffer; increased compression with MPEG and similar schemes due to better utilization of the special treatment of zero coefficients; increased compression with other compression schemes or any other control signals need to be transmitted from the adaptive filter. In experimental evaluations of the invention, the following effects have been observed. With the invention applied in a coding system operating with fixed bit rate, the average quantization scale factor has typically been 20-30 % lower than without the invention. In coding systems with a fixed quantization scale factor, the invention applied entails a bit rate reduction from 15 % up to 65 % in extreme cases. However, typically about 20-30 % lower bit rates is achievable with reasonable effects in the image quality. Low fixed quantization parameters give higher bit rate effect since the invention substitutes for a dynamically varying quantization scale factor. In both cases image quality appears not to suffer, even if increased compression is used to reduce bit rate. With a constant bit rate, the image is improved when the invention is applied. It should be noted that these results are examples found in experimental evaluations of the invention under certain conditions, and are highly dependent on the selected coefficient threshold functions.

Brief Description of the Drawing The present invention will be further explained in the following description of embodiments taken in conjunction with the accompanying drawings, in which:

Fig 1 shows a block diagram of an inventive processing apparatus employed in an image signal encoding system;

Fig 2 shows a more detailed block diagram of an embodiment of the processing apparatus according to the invention employed in one variety of an image signal encoding system; and

FIG 3 shows an example of a coefficient threshold function used in the invention.

Detailed Description of Embodiments Referring to FIG. 1, there is shown an embodiment of an apparatus according to the invention in the shape of a selective filtering means 2 integrated in an image signal coding system 1, comprising an image signal transforming means 4 and an output coding stage 6. An input of the filtering means 2 is communicatively coupled to the transformer 4, which is devised for transforming an image signal from one domain, e.g. a spatial domain, to a frequency domain. An output of said filtering means 2 is coupled to the output coding stage 6 where the actual coding is carried out. The transformer takes as an input an image signal 3a comprising digitized image data represented as pixel values and outputs a frequency domain image signal 8 wherein the pixel values are represented in the form of coefficients in the current frequency domain. The output coding stage 6 produces and outputs a coded or compressed image signal 12a, which thereafter e.g. can be stored in compressed form or transmitted as a compressed bit stream in a data transmission channel (not shown).

The selective filtering means preferably takes as an input one or a number of control parameters or condition parameters, in FIG. 1 exemplified by a feedback coded image signal 12b, a coder status signal 14 and/or the input spatial domain image signal 3b. The control parameters are used to select filtering rules dependent on the state of the encoder, the frequency content of the transformed image signal 8 and perhaps also the image data content of the input image signal 3b or other selected parameters. The filtering rules are for example expressed as one or more coefficient threshold functions.

Fig 2 shows a more detailed embodiment of the invention applied in a per se known image signal coding system, for example an MPEG encoder. An input image signal 3 a is received by a coding system input stage 14, which may comprise means for motion estimation, noise reduction means and the like. The image signal 3a is transformed into the frequency domain by means of a discrete cosine transformer (DCT) 16 and the resulting frequency domain image signal 22 is thereafter input into an adaptable filter 34 comprised in the selective filtering means 2. A filtered signal 24 is then transmitted to a quantizer 18 for quantizing or scaling before the proper output coding stage 20, wherein the coding is performed. The filter applies selectable filtering rules, e.g. in the form of a threshold function, under the control of a filtering rule selecting means 32, here in the shape of threshold function selector 32. The filtering rule selector 32 is in a preferred embodiment devised to select a stored threshold function or to dynamically compute an appropriate threshold function dependent on predetermined parameters. The filtering rule selector is in its turn controlled by means of an evaluating means 26 devised to evaluate filtering conditions with the aid of one or more condition parameters. In the embodiment of FIG. 2, a condition parameter indicating the coder system status, e.g. the quantization scale factor, is taken from the quantizer 18 through line 28a to an input of the evaluator 26. A preferred status parameter to monitor is the output buffer status, i.e. the currently available space in the buffer which instead may be taken directly from the output coding stage or output buffer through line 28b. However, the quantization scale factor is a conveniently available parameter giving an indirect measure of the buffer status. Optionally, a DCT signal 30, the input image signal 3b and/or a control or status signal 5 from the input stage may as well be communicated to the evaluator 26. The filtering rule may also be selected by means of a control signal or command input actuated by a user and received by the evaluation means or the filtering rule selecting means.

In the block diagrams of the drawings, functional blocks refer to hardware means or computer program functions, dependent on the chosen implementation. Equally, the lines and the numerals referring to signals also, dependent on the form of implementation of the invention, refer to signal conductors, communication channels, data structures or the like carrying the corresponding information.

One embodiment of the inventive method as employed in the apparatus of FIG 2 comprises the steps of: -Receiving as input a set of pixels preferably constitutes a part of an image, the image data of the pixels being in the form of coefficients in a frequency domain. In a common case the set of pixels is a block of 8x8 pixels.

-Evaluating a quantization scale factor taken from a quantizer or a buffer status parameter directly from an output buffer, and possibly comparing the scale factor to a predeteimined scale factor threshold. -Selecting a filtering rule in the shape of a coefficient threshold value or a coefficient threshold function, dependent on the quantization scale factor. -Replacing amplitude values being lower than said coefficient threshold value with a predetermined run-length amplitude value, said run-length amplitude value preferably being zero.

-Outputting a selectively filtered frequency domain image signal, preferably to a subsequent quantizer.

The normal quantization and coding procedure comprising bit rate control etc. is then performed on the filtered image signal. The manner of determining suitable criteria for the selection of threshold surfaces as well as determining actual threshold values is investigative and experimental. Today, there is no known way to optimize the selection criteria nor the threshold values in a computational manner. The man skilled in the art implementing the invention instead has to evaluate subjectively the consequences of different criteria and threshold values on a trial and error basis. The important framework of the invention, however, is that subjectively experienced image quality in certain situations may be sacrificed in order to avoid overflow, and conversely a high degree of available output buffer space may be utilized for perhaps bulky, but image quality contributing coefficients. When realising the invention, heuristic rules about an output buffer status and heuristic rules primarily related to subjective visual image content have been used to adapt a threshold matrix. The adaptation as described below is set out in terms of level and slope of a threshold surface defined by the threshold matrix. The individual values of the threshold matrix can also be altered dynamically in accordance with selected rules. Image content properties such as smoothness, texture, edges, brightness, motion, colour, etc are detected by other means than the filter. For example, an MPEG encoder performs motion estimation and thereby detects the motion of a block, it detects the luminance and the chrominance of a block and yet other properties can be deducted from visual inspection of the raw image. The deductible procedure is not a central feature of the invention itself, however the results of it are used in order to set criteria and threshold values.

The following table is a numerical example of a threshold matrix suitable for appliance in intrablock coding in a stable condition. The threshold level can be varied by means of a parameter devised for the whole matrix.

Level = 3 0 8.5 9.4 10.4

1 8.2 9.1 10.1 11.1

2 7.8 8.8 9.8 10.7 11.7

3 7.5 8.5 9.4 10.4 11.4 12.4

4 7.2 8.2 9.1 10.1 11.1 12.0 13.0

5 6.9 7.8 8.8 9.8 10.7 11.7 12.7 13.6

6 7.5 8.5 9.4 10.4 11.4 12.4 13.3 14.3

7 8.2 9.1 10.1 11.1 12.0 13.0 14.0 14.9

Level = 6 0 16.9 18.9 20.8

1 16.3 18.2 20.2 22.1

2 15.7 17.6 19.5 21.5 23.4

3 15.0 16.9 18.9 20.8 22.8 24.7

4 14.4 16.3 18.2 20.2 22.1 24.1 26.0

5 13.7 15.7 17.6 19.5 21.5 23.4 25.4 27.3

6 15.0 16.9 18.9 20.8 22.8 24.7 26.7 28.6

7 16.3 18.2 20.2 22.1 24.1 26.0 27.9 29.9

Level = 9 0 25.4 28.3 31.2

1 24.5 27.4 30.3 33.2

2 23.5 26.4 29.3 32.2 35.1

3 22.5 25.4 28.3 31.2 34.2 37.1

4 21.5 24.5 27.4 30.3 33.2 36.1 39.0

5 20.6 23.5 26.4 29.3 32.2 35.1 38.0 40.9

6 22.5 25.4 28.3 31.2 34.2 37.1 40.0 42.9

7 24.5 27.5 30.3 33.2 36.1 39.0 41.9 44.8

Level = 12 0 33.9 37.8 41.7

1 32.6 36.5 40.4 44.2

2 31.3 35.2 39.1 43.0 46.8

3 30.0 33.9 37.8 41.7 45.5 49.4

4 28.7 32.6 36.5 40.4 44.2 48.1 52.0

5 27.4 31.3 35.2 39.1 43.0 46.8 50.7 54.6

6 30.0 33.9 37.8 41.7 45.5 49.4 53.3 57.2

7 32.6 36.5 40.4 44.2 48.1 52.0 55.9 59.8 An embodiment of the algorithm for the adaptive filter according to the invention can be described as follows, with a selection of the listed criteria. The filter threshold is perceived as a surface having threshold coefficients Tij. For each block A. Determine threshold coefficients Tij al- select base threshold surface Tij from a set of basic pre-computed threshold surfaces; a2- if block is very dark, e.g. the average luminance of the block in the range of 3-6% from extreme => raise level and slope of threshold surface; a3- if block is very bright, e.g. the average luminance of the block in the range of 3-6% from extreme => raise level and slope of threshold surface; a4- if block contains rapid motions, detectable through inspection of motion vectors in macro-block ___> raise level and slope of threshold surface; a5- if buffer well filled or close to overflow, i.e. an undesirable situation to pass quickly =_> raise level and slope of threshold surface relatively much a6- if image smooth, i.e. no great detail needed in image ___> raise slope of threshold surface; a7- if image has coarse texture ___> raise level and slope of threshold surface; a8- if image has edge, i.e. great detail needed in image ___> lower threshold surface; a9- if buffer status stable with margin => select medium threshold level; B. Apply filter bl- for each source coefficient Uij set output Vij if abs (Vij) > Tij then Vij = Uij else Vij = 0. In the above example a number of optionally selectable criteria are listed.

Criteria a2, a3, a4 and a7 are examples of image properties entailing barely detectable image details, where a harsher filtering may be applied.

FIG 3 shows in a 3D-perspective an example of a coefficient threshold function for a block of 8x8 pixels in the frequency domain. The coefficient positions 0-7 in the block are indicated in the horizontal area, the absolute value of the coefficient amplitude is indicated along the vertical axis and the coefficient threshold value is indicated as a function of the coefficient frequency and coefficient amplitude. The lowest frequency is situated in the 0-0 position, whereas the highest frequency is situated in the 7-7 position. As is shown in FIG 3, the coefficient threshold function is preferably a three-dimensional surface and coefficients having an amplitude below the threshold surface are filtered out in accordance with the invention. The level characteristics of the threshold surface are devised in accordance with selected criteria as discussed above.

The inventive method can be realised by means of hardware as well as by means of a computer program executed on a computer comprising a processor, storage means and input/output devices. Any realisation comprises functional means devised to carry out the different steps and functions of the invention as described herein. A computer program can further be embodied in a computer program product comprising a recording medium, and means, recorded on the recorded medium, for directing a computer to perform the functions and steps of the invention.

While the present invention has been shown and described with respect to particular embodiments, it will apparent for those skilled in the art that different modifications and combinations of features may be made without departing from the scope of the invention as described in the appended claims.

Claims

Claims 1. A processing method for use in an image signal coding system for coding an input image signal into a compressed output image signal, the image signal comprising digitized image data in a plurality of pixels; the method comprising the steps of:

-receiving as an input a set of pixel coefficients constituting a part of a non-compressed image;

- evaluating a processing condition which is dependent on the characteristics of said input image data and the status of an output stage of said image signal coding system; - if a first predeteimined condition is detected, filtering in a first filtering step, the set of pixel coefficients such that image quality of said compressed output signal is given priority by retaining pixel coefficients contributing significantly to image quality and filtering out the rest of the pixel coefficients;

- if a second predetermined condition is detected filtering in a second filtering step, the set of pixel coefficients such that a high achievable compression factor for said set of pixel coefficients is given priority over image quality of said compressed output signal; -outputting a selectively filtered set of pixel coefficients to a subsequent stage of the image signal coding system.

2. The method of claim 1, wherein said first filtering step is carried out for every received set of pixels, or when a coding system status parameter, e.g. output bit rate or currently available output buffer space, is within a predetermined deviation range from a predetermined setpoint.

3. The method of claim 1, wherein said second filtering step is carried out when image quality of said compressed output signal is worse than a predetermined minimum image quality or when image quality detail is invisible to the human vision system.

4. The method of claim 3, wherein image quality is evaluated by means of an encoding system parameter, e.g. a quantization scale factor, being a direct or indirect measure of the image quality of said compressed output signal.

5. The method of claim 4, wherein the filtering of the first and the second filtering steps are carried out in accordance with a first and a second predetermined coefficient threshold function, respectively, each coefficient threshold function being dependent on the state of the image signal coding system and predetermined coefficient characteristics and/or possibly the image data content of the input image signal and/or possibly the state of the output compressed image signal.

6. The method of any of the precedent claims, wherein the image data of said received set of pixels are in the form of coefficients in the frequency domain, the coefficients being characterised by their amplitude and frequency.

7. The method of claim 5, wherein the first coefficient threshold function is predetermined in accordance with requirement criteria, such as required image quality, required bit rate/set of pixels or user commands.

8. The method of claim 5, wherein the second coefficient threshold function is determined dynamically for each individual received set of pixel coefficients and possibly also for each individual coefficient of said set of pixel coefficients.

9. A processing method for use in an image signal coding system for coding an input image signal into a compressed output image signal, the image signal comprising digitized image data in a plurality of pixels and the image signal coding system having quantization means for quantizing image data in accordance with a predetermined required compression factor to achieve a predetermined maximum bit rate used in said system; the method comprising the steps of:

-receiving as an input a set of pixel coefficients constituting a part of an image, the image data of said pixels being in the form of coefficients in a frequency domain; -evaluating the current status of the image signal coding system by evaluating an output buffer status with respect to available buffer space e.g. by means of a quantization scale factor of said quantization means; -selecting a predetermined coefficient threshold function dependent on the quantization scale factor and on the characteristics of said set of pixels; -filtering the coefficients in accordance with said selected coefficient threshold function by replacing coefficients having an amplitude below said coefficient threshold function with a predetermined run-length amplitude value, preferably zero; -outputting a selectively filtered set of pixel coefficients to said quantizing means.

10. The method of claim 9, wherein a coefficient threshold function is devised such that frequency coefficients below a predetermined frequency level are preserved, coefficients having an amplitude above a predetermined amplitude level are preserved and coefficients having an amplitude below said amplitude level are filtered out in said filtering step.

11. The method of claim 9, wherein

-a first coefficient threshold function is devised with a frequency and amplitude level dependency such that coefficients contributing significantly to image quality in the compressed image signal are above said coefficient threshold function; and wherein -a second coefficient threshold function is devised with a frequency and amplitude level dependency such that coefficients restricting a currently achievable compression rate are below said second threshold function.

12. The method of claim 11, wherein the first coefficient threshold functions is selected for all received set of pixel coefficients.

13. The method of claim 11, wherein the first coefficient threshold function is selected when the quantization scale factor is below a predetermined scale factor threshold corresponding to a moderate compression rate.

14. The method of claim 11,12 or 13, wherein the second coefficient threshold function is selected when the quantization scale factor is above a predetermined scale factor threshold corresponding to a compression rate having a significant negative influence on the image quality of the compressed output image signal.

15. The method of claim 11 , further comprising the step of evaluating the image data content of the input image signal and selecting the second coefficient threshold function when image quality details are invisible to the human vision system.

16. An apparatus (2) for processing an image signal, for use in an image signal coding system (1) for coding an input image signal (3a) into a compressed output image signal

(12,12a), the image signal comprising digitized image data in a plurality of pixels, the apparatus comprising: -an input for receiving a set of pixel coefficients (8,30) constituting a part of an image of said image signal, the image data of said pixels being in the form of coefficients in a frequency domain;

-an input for receiving a coding system status parameter (12b, 14,28);

-evaluating means (26) for evaluating said coding system status parameter (12b, 14,28) and/or the characteristics of said image signal;

-selecting means (32) for selecting a predetermined coefficient threshold function dependent on the coding system status parameter and/or on the characteristics of said image signal;

-filtering means (2,34) for filtering the coefficients in accordance with said selected coefficient threshold function by replacing coefficients having an amplitude below said coefficient threshold function with a predetermined run-length amplitude value, preferably zero;

-an output for a selectively filtered set of pixel coefficient (24) to a subsequent stage of said image signal coding system.

17. The apparatus of claim 16, comprising means for storing or calculating a coefficient threshold function devised such that frequency coefficients below a predetermined frequency level are preserved, coefficients having an amplitude above a predetermined amplitude level are preserved and coefficients having an amplitude below said amplitude level are filtered out by said filtering means.

18. The apparatus of claim 17, wherein

-a first coefficient threshold function is devised with a frequency and amplitude level dependency such that coefficients contributing significantly to image quality in the compressed image signal are above said coefficient threshold function; and wherein -a second coefficient threshold function is devised with a frequency and amplitude level dependency such that coefficients contributing to a limitation of a currently achievable compression rate are below said second threshold function.

19. The apparatus of claim 18, wherein the first coefficient threshold function is selected for all received set of pixel coefficients.

20. The apparatus of claim 18, wherein the first coefficient threshold function is selected when the quantization scale factor is below a predetermined scale factor threshold corresponding to a moderate compression rate.

21. The apparatus of claim 18, 19 or 20, wherein the second coefficient threshold function is selected when the quantization scale factor is above a predetermined scale factor threshold corresponding to a compression rate having a significant negative influence on the image quality of the compressed output image signal.

22. The apparatus of claim 18, further comprising means for evaluating the image data content of the input image signal and means for selecting the second coefficient threshold function when image quality details are invisible to or hard to detect by the human vision system.

23. The apparatus of any of the preceding claims, wherein said image signal coding system comprises an MPEG encoder.

24. A computer program product, for use with a computer system in an image signal coding system for coding an input image signal into a compressed output image signal, the image signal comprising digitized image data in a plurality of pixels; the computer program product comprising: -a recording medium; -means, recorded on the recording medium, for directing the computer system to receive as an input a set of pixel coefficients constituting a part of a non-compressed image;

-means, recorded on the recording medium, for directing the computer system to filter in a first filtering step, in response to a first predetermined condition, the set of pixel coefficients such that image quality of said compressed output signal is prioritised by retaining pixel coefficients contributing significantly to image quality and filtering out the rest of the pixel coefficients;

-means, recorded on the recording medium, for directing the computer system to filter in a second filtering step, in response to a second predetermined condition; the set of pixel coefficients such that a high achievable compression factor for said set of pixel coefficients is prioritised over image quality of said compressed output signal by filtering out also pixel coefficients contributing significantly to image quality;

-means, recorded on the recording medium, for directing the computer system to output a selectively filtered set of pixel coefficients to a subsequent stage of the image signal coding system.

25. The computer program product of claim 24, wherein said first filtering step is carried out for every received set of pixels, or when a coding system status parameter, e.g. output bit rate, is within a predetermined deviation range from a predetermined setpoint.

26. The computer program product of claim 24, wherein said second filtering step is carried out when image quality of said compressed output signal is worse than a predetermined minimum image quality or when image quality detail is invisible to or hard to detect by the human vision system.

27. The computer program product of claim 26, wherein image quality is evaluated by means of an encoding system parameter, e.g. a quantization scale factor, being a direct or indirect measure of the image quality of said compressed output signal.

28. The computer program product of claim 27, wherein the filtering of the first and the second filtering steps are carried out in accordance with a first and a second predetermined coefficient threshold function, respectively, each coefficient threshold function being dependent on the state of the image signal coding system and predetermined coefficient characteristics and/or possibly the image data content of the input image signal and/or possibly the state of the output compressed image signal..

29. The computer program product of any of the precedent claims, wherein the image data of said received set of pixels are in the form of coefficients in the frequency domain, the coefficients being characterised by their amplitude and frequency.

30. The computer program product of claim 28, wherein the first coefficient threshold function is predetermined in accordance with requirement criteria, such as required image quality, required bit rate/set of pixels or user commands.

31. The computer program product of claim 28, wherein the second coefficient threshold function is determined dynamically for each individual received set of pixel coefficients and possibly also for each individual coefficient of said set of pixel coefficients.

32. A computer program product, for use with a computer system in an image signal coding system for coding an input image signal into a compressed output image signal, the image signal comprising digitized image data in a plurality of pixels and the image signal coding system having quantization means for quantizing image data in accordance with a predetermined required compression factor to achieve a predetermined maximum bit rate used in said system; the computer program product comprising: -a recording medium; -means, recorded on the recording medium, for directing the computer system to receive as an input a set of pixel coefficients constituting a part of an image, the image data of said pixels being in the form of coefficients in a frequency domain; -means, recorded on the recording medium, for directing the computer system to evaluate the current status of the image signal coding system by evaluating said quantization scale factor; -means, recorded on the recording medium, for directing the computer system to select a predetermined coefficient threshold function dependent on the quantization scale factor and on the coefficient frequency characteristics of said set of pixels; -means, recorded on the recording medium, for directing the computer system to filter the coefficients in accordance with said selected coefficient threshold function by replacing coefficients having an amplitude below said coefficient threshold function with a predetermined run-length amplitude value, preferably zero; -means, recorded on the recording medium, for directing the computer system to output a selectively filtered set of pixel coefficients to said quantizing means.

33. The computer program product of claim 32, wherein a coefficient threshold function is devised such that frequency coefficients below a predetermined frequency level are preserved, coefficients having an amplitude above a predetermined amplitude level are preserved and coefficients having an amplitude above said amplitude level are filtered out in said filtering step.

34. The computer program product of claim 32, wherein

35. The method of claim 34, wherein the first coefficient threshold functions is selected for all received set of pixel coefficients.

36. The method of claim 34, wherein the first coefficient threshold function is selected when the quantization scale factor is below a predetermined scale factor threshold corresponding to a moderate compression rate.

37. The method of claim 34, 35 or 36, wherein the second coefficient threshold function is selected when the quantization scale factor is above a predetermined scale factor threshold corresponding to a compression rate having a significant negative influence on the image quality of the compressed output image signal.

38. The method of claim 34, further comprising means, recorded on the recording medium, for directing the computer system to evaluate the image data content of the input image signal and to select the second coefficient threshold function when image quality details are invisible to the human vision system.