WO1999008449A1 - Signal coding and decoding - Google Patents

Abstract

A method of signal coding comprising the step of decomposing the signal into transform coefficients, further comprising the step of generating mask data indicating for which, of said transform coefficients, transform data will not be sent. The mask data may indicate the depth (in bit-planes) at which data will start to be sent for each coefficient.

Description

SIGNAL CODING AND DECODING

This invention relates to methods and apparatus for signal coding and decoding, and particularly but not exclusively to such methods and apparatus for encoding and decoding image signals such as still pictures or video.

It is known to encode image signals to effect compression of the volume of image data. Such coding processes may make use of redundancy reduction to perform "lossless" coding in which the original image is perfectly reproduced after decoding, or may additional or alternatively perform "lossy" coding in which part of the image data deemed less significant on some criteria is omitted during the coding process.

Examples of prior image coding processes are the well known joint picture expert group (JPEG) algorithm for still picture encoding and decoding, and the moving picture expert group (MPEG) algorithms for encoding and decoding video. These, together with many other encoding and decoding systems, make use of spatial transforms such as the discrete cosine transforms (DCT) to convert the original image data into a transformed form for encoding.

Much of the image information in many types of image is concentrated at lower spatial frequencies, and correspondingly the values of higher order transform coefficients are often relatively low, so that compression may be effected by eliminating zero valued coefficients, or coefficients below a given threshold, or by coarsely quantising such coefficients.

One known image type of image transformation is the discrete wavelet transform, described in Antonioni, M et al, "Image coding using wavelet transform", IEEE Trans. Image Processing, Vol. 1 , No. 2, pp205-220, April 1992. Some coding schemes organise the coefficients into a single sequence; for example, in the JPEG and MPEG schemes, the transform coefficients are organised into a linear array in order of sequency. Runs of coefficients are then encoded, so as to encode continuous sequences of zero valued coefficients efficiently. Other schemes order the coefficients into a tree structure, rather than a linear sequence, and then make use of the tree structure to indicate groups of coefficients to be encoded in some manner rather than runs of coefficients as in the above described JPEG and MPEG schemes. For example several such schemes are described in "Image coding using optimised significance tree quantisation", Davis and Chawla, Proc. IEEE Data Compression Conference, Snowbird, Utah, 1997, pages 387-396.

Other such schemes are described in "A new, fast, and efficient image codec based on set partitioning in hierarchical trees", Said and Pearlman, IEEE Trans. Circuits and Systems for Video Technology 6(3): 243-250, June 1996, and "Embedded image coding using zero trees of wavelet coefficients" Shapiro, IEEE Trans, on signal processing, 41 (12): 3445-3462 Dec. 1993 (further described in US 5321776 and US 531 5670). The present invention is applicable, but not limited, to encoding schemes using the discrete wavelet transform in which the coefficients may be considered to be tree-structured. It will be understood also to be applicable to other spatial transforms.

For example, it is proven in Monro and Dickson, "Zerotree coding of DCT coefficients", IEEE International Conference on Image Processing ICIP 1997, that the Shapiro zerotree method may be applied to the DCT, and in Monro, "Embedded band masked coder", ISO/IEC JTC1 /SC19/ING1 N651 , October 1997 that the present embodiments can be used with the Lapped Orthogonal Transform (LOT).

Accordingly, the present invention provides, in one aspect, a method of signal coding comprising the step of decomposing the signal into transform coefficients, further comprising the step of generating mask data representing a significance mask indicating for which, of said transform coefficients, transform data will not be sent.

The present invention may be employed in encoders and decoders coupled to transmission channels such as RF or telephone channels, or in encoders and decoders employed in image storage, but is not limited to these applications.

In one embodiment, the transform coefficients are ordered into layers of significance (e.g. bitpianes), and at each bitplane, coefficient data is sent before an indication of those coefficients for which, in following less significant layers, data will be sent ( i.e. "mask" data). This leads to a more embedded coding scheme - i.e. one in which, if transmission is terminated early, a relatively good image may be recovered at the receiver. Further, in a variant of this embodiment, coefficient data relating to lower significance layers may be transmitted prior to mask data for a given layer.

These embodiments depend upon the principle that, for each successive unit of coefficient data transmitted, the power contribution to the reconstructed image should be maximised, relative to the number of bits required, to achieve embedded coding in which the most significant data is sent first.

Accordingly, the order of transmission is determined taking into account the power contribution of each unit of data, relative to the bits required to represent it.

This incremental power contribution depends, for each coefficient, upon the magnitude of coefficient data already sent. This principle could also be used independently of the use of mask data, as described above.

Embodiments of the invention will now be described in greater detail, with reference to the accompanying drawings in which:

Figure 1 a is a block diagram showing the structure of an image transmission encoder incorporating an embodiment of the present invention; and

Figure 1 b is a block diagram of a corresponding image transmission decoder;

Figure 2a is a block diagram of the transmission encoder within the embodiment of Figure 1 a; and Figure 2b is block diagram of the corresponding decoder within Figure 1 b;

Figure 3 is a block diagram indicating the elements of a transform coder forming part of Figure 2a and of a transform decoder forming part of Figure 2b;

Figure 4 is an illustrative diagram showing stages of a one dimensional wavelet transform and reverse transform applied by the structures of Figure 3; Figure 5 is an illustrative diagram showing the stages of a two dimensional wavelet transform;

Figures 6a and 6b are illustrative diagrams illustrating the relationship between groups of coefficients resulting from a two dimensional wavelet transform; Figure 7 is a flow diagram showing the process of applying a wavelet transform in the embodiment of Figures 1 to 3;

Figure 8 is a diagram illustrating a hierarchical arrangement of coefficients according to embodiments of the invention; Figure 9 is a flow diagram showing schematically the operation of an encoder according to a first embodiment of the invention;

Figure 10 is a flow diagram showing schematically the operation of a corresponding decoder; Figure 1 1 is a diagram showing schematically the arrangement of data transmitted according to the first embodiment;

Figure 12 is a diagram illustrating the bit planes used in embodiments of the invention;

Figure 13 (comprising Figures 13a-13c) is a flow diagram showing the operation of an encoder according to a third embodiment of the invention;

Figure 14 is a flow diagram showing the operation of the corresponding decoder;

Figure 15 is a flow diagram showing the operation of an encoding according to a fourth embodiment of the invention; and Figure 16 is a flow diagram showing the operation of the corresponding decoder;

Figure 17 is a flow diagram showing the operation of an encoding according to an eighth embodiment of the invention; and

Figure 18 is a flow diagram showing the operation of the corresponding decoder.

First Embodiment

Referring to Figure 1 a, an image transmission system with which the present invention may be used comprises a camera 1 2 (or other image source); an image coder 14 according to the present embodiment; a channel coder 16; and a communications interface 18. The image signal scanned by the camera 12 is encoded by the image coder as will be described further below, and the encoded image data is supplied to the channel coder 1 6 which applies error protection encoding (e.g. block or convolutionai encoding). The transmission-coded image signal is then supplied to an interface 18 (such as an modem or an RF transmission circuit) from which it is supplied to a transmission path such as a radio path (either terrestrially or via a satellite); a telephone line; or a virtual circuit such as a data transmission channel carried over a network using, for example, Internet protocols (i.e. "the Internet"). Referring to Figure 1 b, at a receiver, the signal is received at an interface 28 such as a modem or RF interface, corresponding to the functional inverse of the interface 18; the received signal is applied to a channel decoder 26 which decodes the transmission coding applied by the transmission coder 16; and the image signal thus decoded is supplied to an image decoder 24 in accordance with the present embodiment, which decodes the image to an image format suitable for display on a display terminal (i.e. VDU 22). The system thus described in relation to Figures 1A and 1 B may form one half of a videotelephone or videoconference system, for example, at which each terminal comprises both the apparatus shown in Figure 1 a and the apparatus shown in Figure 1 b so as to enable a bi-directional image communications session to be carried out.

Referring to Figure 2a, the image coder 14 will now be disclosed in greater detail. The analogue image signal from the camera 1 2 is digitised by an analogue to digital converter 32 and the resulting stream of (for example, 8 bit) samples are formatted into image rows by a formatter 34 and stored within a frame store 36, so that the frame store 36 receives digitised pixels corresponding to the entire image.

A transform coder 38 applies a spatial transform (in this case, a discrete wavelet transform) to the image within the frame store 36, to generate a set of coefficients which are stored within a coefficient store 40. Thus far, the components of the coder correspond to those of a conventional signal coder.

According to this embodiment, there is also provided a coefficient coder 42 and a mask buffer 44. The mask buffer stores a significance mask, giving an indication of which coefficients are to be masked (and their values correspondingly not sent), as will be described in greater detail below. The coefficient coder accesses the coefficient store, reads values of the coefficients stored therein, and controls the contents of the mask buffer in dependence thereon. It also accesses the coefficient store for those coefficients which are not masked and supplies their coefficient values, together with data indicating the contents of the mask buffer 44, for output as the coded image signal.

Referring to Figure 2b, in the corresponding image decoder 24, the coded image signals are received by a coefficient decoder 62, which receives the data representing the contents of the mask buffer 44 and sets corresponding data within a decoder mask buffer 64. In accordance with the contents of the mask buffer 64, the coefficient decoder 62 then receives transform coefficients contained within the coded image data for those coefficients which are not indicated to be masked by the contents of the mask buffer 64, and writes the coefficients into corresponding locations within a receiver coefficient store 60. Thus, after operation of the coefficient decoder 62 on the entire image, the coefficient store 60 contains the values of those coefficients which have been transmitted, the remaining coefficients being set to some predetermined value such as zero. The contents of the coefficient store 60 are supplied to a transform decoder 58, which performs the reverse spatial transform to the transform coder 38, and supplies output image values to the frame store 56, equivalent to the frame store 36 at the encoder, from which the values are read out, formatted into lines by a formatter 54 arranged to perform the reverse operation to the formatter 34, and supplied to a digital to analogue converter 52 the output of which comprises an analogue image signal. The components 60 to 52 correspond to those of a conventional transform decoder, and mirror the operation of the corresponding components of the encoder shown in Figure 2A.

Conveniently, the formatter 54, transform decoder 58 and coefficient decoder 62 may all be providing by a single programmed processor such as a microprocessor, microcontroller or digital signal processor (DSP) device, and the frame store, coefficient store and mask buffer may all be provided by partitioned areas of a common random access memory (RAM) or other read/ write memory such as a disk drive; the same is true of the corresponding components of the encoder of Figure 2a. Wavelet Transform

Although the operation of the transform encoder 38 and the transform decoder 58 are known of themselves, and therefore need not be described in detail, it will be helpful to the following description to give at this point a brief disclosure of their operation with reference to Figures 3 to 6.

Figure 3 shows the structure of the transform coder 38 and transform decoder 58. The transform coder 38 comprises a cascade of analysis filters, each followed by a down-conversion (i.e. sub-sampling) stage correspondingly, the transform decoder 58 comprises a cascade of synthesising filters, each preceded by an up-sampler (i.e. interpolator) stage. In more detail, the image signal is received by a pair of mirror filters comprising a high pass filter 102 and a low pass filter 104, the crossover frequency of the two filters lying at around half the maximum spatial frequency present (i.e. half the data rate of the image signal).

The output of the high pass filter 102 is then sub-sampled by a factor of two, as is the output of the low pass filter 104, by down samplers 106a, 106b, by ignoring every other sample, or averaging adjacent pairs of samples, for example. Since the output of each filter now only contains half the bandwidth of the input, the sub sampling does not result in loss of data, and the overall volume of the data (ignoring differences in precision and hence number of bytes between the input and the output) is the same.

The output of the low pass filter is then passed to a further pair of mirror filters 108, 1 10 which likewise have their transitional crossover frequency at the centre of the maximum spatial frequency of the output of the sub sampler 106b. Again, the outputs of the filters 108, 1 10 are down sampled to half the rate of the input by a pair of sub-samplers 1 1 2a, 1 12b. Thus, again, there is no increase in the number of filtered values over the number of input image values. The outputs of the low pass filter 1 10 for the pair is again split and sub-sampled by a pair of filters 1 14, 1 16 and sub samplers 1 18a, 1 18b, and the process is repeated a number of times (for example five) to result in a set of parallel streams of data, at differing data rates.

The operation of the transform decoder is essentially the reverse of that of the encoder 38 as will be apparently from Figure 3; prior to filtering, each signal stream is up-sampled by a factor of two, for example by interpolating between each pair of adjacent signal values a new signal value corresponding to the average of its neighbours, or by repeating each signal value.

The filters may be finite impulse response filters with high regularity, as described in Villasenor et al, "Wavelet filter evaluation for Image Compression", IEEE Trans. On Image Processing, Vol. 4, No. 8, pp1053-1060, August 1995.

Information on the design of mirror filter pairs may be found in, for example, US

4896356. Referring to Figure 4, and considering the processing of a single line of the image, as described above, the original signal is split into a pair of low and high pass signal streams, (labelled "L" and "H") by the first filter pair, and the low pass signal part is then further split into two signal components which replace it (labelled "LL" and "LH"). The lowest frequency component (LL) is then once more split into a pair of signals (labelled "LLL" and LLH") by the third stage of filtering. On reconstruction, at each level of synthesis filtering, the lowest frequency pair of sub bands are recombined until the original signal is recovered. Figure 4 thus corresponds to a one dimensional wavelet transform. For a two dimensional image, the corresponding process is illustrated in

Figure 5. In this case, and according to the present embodiment, the processor which will be described with reference to Figures 5 to 7 is performed.

In step 702 of Figure 7, a line of the image signal is read from the frame store 36 and the first filter pair 102, 104 is applied to generate two corresponding filtered lines, each containing half the number of data of the original line after subsampling by the downsamplers 106a, 106b. In step 704, the original line is replaced with the two filtered bands, shown as occupying the space left by the original line.

In step 706, it is determined whether the last line in the frame store 36 has been reached and, if not, the next line is selected in step 708, and the process recommences at step 702.

When the last line is reached (step 706) in the first pass thus far described, the frame store will therefore contain a low frequency line image and a high frequency line image, as shown at stage 1 of Figure 5. However, although each line has been filtered, the columns still contain full bandwidth vertical image information.

In step 712, the first column of the image held in the frame store 36 is filtered by the first pair of filters 102, 104 and down sampled. In step 714, the data previously stored in the columns (i.e. the line filtered image) is replaced with the high frequency and low frequency filtered signals, each of which comprises half as many data as the original column.

In step 716 it is determined whether the last column in the frame store 36 has been reached and if not, in step 718 the next column is selected. Once it has been determined that the last column has been processed, the contents of the frame store will therefore correspond to the product of stage 2 shown in Figure 5; one quarter of the frame store will be occupied by data which corresponds to an image which has been low pass filtered both vertically and horizontally (and is therefore a lower spatial resolution version of the original image labelled LL); one quarter will be occupied by an image which has been high pass filtered both vertically and horizontally (labelled HH); and the remaining two quarters of the frame store will be occupied by an image which has been low pass filtered horizontally and high pass filtered vertically (labelled LH) and an image which has been high pass filtered horizontally and low pass filtered vertically (labelled HL).

In step 720, it is determined whether the last level of filter has been reached and, in step 722 the next level of filtering is selected (corresponding in this case to the pair of filters (108, 1 10) and in step 724, the lowest frequency band (i.e. of the four filtered sets of data just previously produced, that which has been low pass filtered both horizontally and vertically) is selected. The process just described in relation to steps 702 through 720 is then repeated on just this low spatial frequency component, to split it into four spectral frequency components each represented by one quarter the volume of data, and shown after step 3 of Figure 5 as LLLL, LLHL, LLLH and LLHH.

The process is then repeated in the same manner until the last level is reached in step 720 (which may, for example, be the fifth level as shown in Figure 5).

In passing, it is noted that of the four components into which an image is split at each stage, the "LL" components contain, as noted, a lower resolution version of the original image; the "LH" components retain high frequency vertical information and therefore retain vertical edges; the "HL" bands retain high frequency horizontal information and therefore represent a horizontal edge image; and the "HH" components maintain diagonal edge information, noise and texture. As indicated in Figure 6, there is a correspondence between coefficients

(which are effectively pixel values) in each band of the transform, since these relate to the same areas of the original image; the top left hand coefficient shown in Figure 6 represents the lowest horizontal and vertical frequency components into which the image has been resolved, and each pixel has four corresponding pixels resulting from the same stage of filtering, in the LH, HL and LH bands.

It also maps onto a group of four pixels at the corresponding positions in the next level up, and a group of sixteen pixels in each of the three bands in the level beyond that, and so on. Each of these groups of pixels from each level corresponds to the same portion of the original image.

Figure 6b illustrates the order in which the subbands or subsets of Figure 6a are scanned according to embodiments of the present invention. The first coefficients scanned are those of the lowest frequency subblock, labelled 1. The subbands 2, 3, 4 are then scanned, in numerical order; these three bands (rightwards and below the first) thus form a layer of coefficients next higher in frequency than the lowest.

The subbands 5, 6, 7 are then scanned; these three subbands may be considered to form the next layer. Likewise, subbands 8, 9 and 10 form the next layer and subbands 1 1 , 12 and 13 the last layer.

Conveniently, within each subband, the coefficients are scanned in a raster sequence although other sequences can be employed. It will be seen that this ordering of coefficients results in coefficients having similar frequency content being scanned in adjust subbands (i.e. in the same layer) and coefficients relating to adjacent image areas being adjacent within each scan within a subband. Where spatial and spectral correlation exists, this scan order therefore permits runs of similar mask values (as discussed below) to be encoded efficiently; more so than, for example, the zigzag scan method employed in JPEG or MPEG transform coding methods. Hierarchical arrangement

It is known (e.g. Woods and O'Neill, "Sub-band coding of images", IEEE Trans. Acoustics, Speech and Signal Processing (ICASSP) Vol. 39, No. 5, pp1278- 1288, 1986) to decompose subband data into a pyramidal hierarchy, and embodiments of the invention utilise this principle. In greater detail, then, each pixel for every sub-band other than the lowest low frequency sub-band (the top left hand corner of Figure 6) is, in this embodiment, regarded as being hierarchically related to a pixel in a lower frequency sub-band (the "controlling" or "parent" pixel). In the top left hand corner of Figure 6, resulting from the last stage of filtering, are four sets of filtered coefficients. Each pixel in the LL band (i.e. low frequency vertical/low frequency horizontal band) controls three pixels; one from each corresponding position in the HL, HH and LH bands. This relationship within the lowest frequency set of four bands (resulting from the last stage of filter) therefore constitutes the first level of the hierarchy.

In turn, each pixel in the HL band controls a group of four pixels at a corresponding position in the next level of filtering up from the lowest, as shown in Figure 6; thus, a pixel in the HH band controls a group of four pixels in the HH band of the filter level below (shown as the block diagonally below and to the right in Figure 6); each pixel in the HL band controls a group of four pixels in the block to the right of it in Figure 6; and each pixel in the LH band controls the group of four in the block below it in Figure 6. Thus, the second, and each successive, layer of the hierarchy provides a four to one mapping. This hierarchy is therefore as schematically illustrated in Figure 8 (in which many of the coefficients are omitted for clarity). All pixels in all bands relating to a common area of the image are hierarchically grouped by direct or indirect reference to a controlling pixel in the lowest frequency LL band.

Each controlling pixel in the lowest band (the top left hand band in Figure 6) therefore defines a tree of data linked to the same spatial position in the image as that pixel. Encoding

The process of encoding will now be described in greater detail with reference to Figure 9. In a first step 802, a threshold pixel magnitude is set by scanning all the stored values in the coefficient store 40, locating the maximum magnitude (positive or negative), incrementing that magnitude by one and dividing by two. This first threshold therefore represents the mid value within the scale of values actually present in the coefficients within the coefficient store 40. Also the first subset or frequency layer of coefficients is selected (i.e. the pixels within the lowest frequency subband).

For each coefficient, in step 806 it is then determined whether a mask value had already been set in the mask buffer 44; if not (as will be the case for the first threshold), the value of the coefficient is compared with the threshold in step 808.

In step 810, the value in the mask buffer 44 is set depending on the results of the comparison; if the magnitude of the coefficient (positive or negative) exceeds the threshold, the mask value is set ON, otherwise it remains set OFF.

As the same time, the polarity of the coefficient value is saved; in other words, it is noted whether the coefficient lies above the positive threshold or below the negative threshold.

In step 812, it is determined whether the last coefficient in the subset has been thus processed and, if not, in step 814, the next coefficient is selected and the process returns to step 806.

After the last coefficient in the layer has been processed, in step 816 the mask values are coded and sent by the use of four codes (which may be represented by four different two bit digits); a first code (ZERO) indicating that the mask for the relevant coefficient should remain OFF (because the coefficient value did not exceed the threshold); a second value (PSIG) indicating that the relevant coefficient exceeded the threshold and had a positive value; a third coefficient (NSIG) indicating that the coefficient exceed the threshold and had a negative polarity; or a fourth value (STOP) indicating that the coefficient is the last in the layer which is not OFF.

Mask symbols are only sent in respect of those coefficients which were not previously found to be set ON in the mask buffer. For the first threshold pass just described above, using the mask coding scheme described above, this will mean that mask data is sent in respect of all coefficients in order until the STOP value is sent, but it will be seen that it is not the case for subsequent thresholds as described below.

Where, within the layer, any coefficients already had mask values set in the mask buffer 44 (as determined in step 806), the one bit data values for such coefficients are then sent in step 817. The order of transmission of data is therefore as shown in Figure 1 1 .

In step 818, it is determined whether the layer was the last layer present in the image (the least filtered layer, comprising the blocks shown on the rightmost and lowermost sides of Figure 6). If not, then in step 820 the next layer is selected and the process returns to step 806. After the last layer has been processed, the entire image will have been coded and transmitted at a low level of accuracy and can, as described hereafter, be reconstructed in an approximate form. The coefficients will typically be encoded to eight or twelve bits of precision, and accordingly a corresponding number (e.g. eight) of thresholds will be employed. In step 822, it is determined whether the last (e.g. the eighth) threshold, corresponding to the least significant bit of image information, has yet been employed and, if not, in step 824, the next threshold is calculated by dividing the existing threshold by two. Each successive threshold therefore represents one further bit of precision in the encoding of the coefficients. Figure 12 illustrates the bit pianes; each plane corresponds to one bit of each of the transform coefficients of the transformed image. The process then returns to step 804.

It will be seen that in the second, and subsequent, threshold passes (i.e. executions of steps 804 on) certain masks in respect of corresponding coefficients will already have been set in the mask buffer 44 by the preceding operations of the process. In other words, for successive levels of precision, it will already have been determined that the most significant bits of certain coefficients have a significant value. In the case of such coefficients, it is therefore always necessary to send the less significant bits and unnecessary to decide whether to do so or not.

Accordingly, as noted above, where a mask for the coefficient has already been set ON within the buffer 44, it is unnecessary to determine whether to do so again, and instead, the corresponding data bit value (one or zero) is sent. Since the receiver has a corresponding mask buffer 64, it is able to determine for which coefficients masks have already been set ON and to read received data sequentially as relating to those coefficients.

As already noted above, much of the energy of the image signal is concentrated at lower frequencies and correspondingly, the coefficients at higher frequencies therefore have relatively small values. Thus, for the application of the first few thresholds, the STOP symbol occurs relatively early in each layer, so that relatively few mask values need be sent. As successively lower thresholds are applied, more and more coefficients will exceed the threshold and so more mask symbols will be sent. Each remains ON and therefore does not need to be sent for successive thresholds, in which data values for that coefficient are sent instead. Thus, by employing mask symbols to indicate which data values are being transmitted, and by ordering the transmission of the mask symbols in successive frequency layers of a transformed image, the present invention reduces the volume of data which is to be sent, achieving high compression, by not sending higher order bits of low valued coefficients. The grouping of the data into subsets or subbands produces an efficient coding of the masks (for example using run length coding or a STOP code as described above) since adjacent coefficients are of similar frequencies and regions of the image.

The above described scheme is progressive in two respects. Firstly, since data for high significance bit planes is sent before that for lower significance bit planes, a coarsely quantised set of all transform coefficients is received early and, if the receiver terminates reception early, it can be used to reconstruct a coarsely quantised image. Secondly, since the coefficients are sent in layers as described above in relation to Figure 6b, even if within one bit plane only the first few layers are received, a low spatial resolution image can nonetheless be reconstructed. Referring to Figure 10, the corresponding reception process performed at the receiver of Figures 1 a and 2b will now be described in general terms.

Initially, in step 902, the contents of the mask buffer 64 are set to "OFF" and the contents of the coefficient store 40 are set to zero (or, in general, some predetermined value). In a step 904, the mask symbols are received from the coefficient decoder

62, and in a step 906, the values of data (for any coefficients the masks for which have previously been transmitted - which will not have been the case in the first bit plane threshold pass) are received.

In a step 908, the coefficient decoder 62 writes the values of any such received data bits into the corresponding bit positions of coefficients within the coefficient store 60. The addresses to which the coefficients are written are determined by, for each received datum, locating the address of the next mask within the mask buffer 64 which is set to "ON" and using the corresponding address within the coefficient store 60. Thus, the mask buffer 64 serves to map the received bits of data onto the corresponding addresses within the coefficient store 40.

In step 910, the received mask symbols are used to set new addresses within the mask buffer 64 to "ON", and in step 912, the values of the received mask symbols (i.e. whether they have positive significance or negative significance) are used to set the values of the coefficients within the coefficient store 40 correspondingly. The pass (bit plane, or level of thresholding) within which the mask symbol is received is used to select which bit of the coefficient is set (all higher order bits being left at zero) and the value (i.e. positive significance or negative significance) of the mask symbol is used to set a sign bit of the coefficient, and to interpret the meaning of the following bits (depending upon the convention used to represent negative numbers).

In step 914, the coefficient decoder 62 determines whether the last subset of data has been received and, if not, selects the next subband (step 916) and proceeds to step 904.

In step 918, the coefficient decoder 62 determines whether the last bit plane (i.e. last threshold) has been employed and, if not, in step 920 the next threshold level is selected (i.e. the next lowest significance bit plane is selected) and the process proceeds to step 904 to receive image data for this next bit plane.

Once all data have been received, corresponding values will have been written to any coefficients which have a significant value, within the coefficient store 60, and the transform decoder 58 is thereafter operable to reconstruct the transformed image into the frame store 56, from which an analogue image may be generated as described above.

Figure 1 1 illustrates the format in which a frame of data is sent, comprising, for each bit plane in turn, the mask symbols and associated data values for those coefficients for which mask symbols have already been set previously. The mask symbols described above may conveniently be encoded using run length coding for runs of zero value (i.e. "OFF" mask symbols), rather than explicitly sending a zero value for each. Such run length coding may be implemented in ways familiar to one of skill in the art; for example, entropy coding or other types of variable length coding may be used to represent runs of commonly occurring lengths.

Other minor variations may easily be envisaged. For example, it may be unnecessary to transmit mask values in respect of the last plane, and both the encoder and decoder may instead be arranged simply to transmit bit values for the last plane.

Although, in the above embodiment, masks are encoded using an indicator of positive or negative significance, it will be apparent that it would equally be possible simply to represent each mask by one of two symbols; for example, a "1 " indicating that the mask should be switching ON and a "0" (or a code indicating a run of O's) to indicate that the mask should stay OFF. In this case, the transmitted data would, in the above embodiment, relate not only to coefficients which had previously had their masks set ON, but also to those which were set ON in the same transmission. The necessary modifications to the processes described above will be immediately apparent. Second Embodiment

In this embodiment, rather than transmitting a single or two bit symbol to indicate the point at which a mask should be turned on, during the bit plane at which that mask is turned on, a multibit symbol is sent for each mask indicating at which bit plane that mask should be turned on. For example, if eight bit planes (i.e. threshold levels) are used, and a given coefficient becomes non zero only at the fifth bit, a mask value of 5 is transmitted for that coefficient.

The receiver accordingly expects no data for that coefficient until the sixth bit plane is reached, and leaves the mask switched off for the first five bit planes. If variable length coding is not employed, then for, for example, eight bit planes, a three bit mask symbol may be used for each coefficient to indicate one of eight possible bit plane values. For all coefficients which turn ON later than the third bit, this embodiment therefore provides additional lossless coding gain.

Thus, in this embodiment, the processes described in relation to the first embodiment are varied in that all stages of thresholding are performed before any mask or image data is sent. Then, a block of mask data is sent indicating, for each coefficient, the bit plane at which the mask for that coefficient should be turned on. Finally, the data for the coefficients are sent, one bit plane at a time, starting, for each coefficient, at the first bit plane at which a non zero bit is encountered.

At the receiver, for each bit plane, masks are turned ON for those coefficients where the mask symbol indicates the corresponding bit plane.

In this embodiment, the mask symbols (hereafter referred to as "mask depth" symbols) may be entropy encoded, using variable length codes, to give the most commonly occurring mask depths the shortest code words. A special code indicating that the mask never turns on may be sent.

Again, since bit planes are sent one plane at time, it is possible for the receiver to terminate reception prior to reception of later bit planes, and still reconstruct a coarsely quantised image. In such cases, it will not have been necessary to send any image data for those coefficients for which the depth value is greater than the last bit plane received (although mask data will have been sent in respect of those coefficients).

Alternatively, mask and symbol data could be combined rather than represented completely separately, for example by sending a first "pass" in which depth values of "always off", positive significance (i.e. on above a first threshold number of levels), negative significance (i.e. on below that threshold number of levels) and/or a STOP code to indicate all following coefficients are always off, could be sent. Third Embodiment

In this embodiment, further use is made of the hierarchical relationship between coefficients which is illustrated in Figure 8. Briefly, for each depth plane the encoder sends only the depth mask values for those coefficients which have a parent coefficient in the hierarchy which has already been turned ON. Additionally, the depth value of such offspring coefficients are specified relative to the depth value specified for their parent coefficients. The encoding process will now be described with reference to Figure 13.

In a step 1002, the first threshold is calculated as in the first embodiment.

A first tree of coefficients is selected comprising one of the coefficients of the lowest frequency band (shown at the top left hand block of coefficients in Figure

8) together with all other coefficients directly or indirectly hierarchically related to it as shown in Figure 8, and therefore relating to the same area of the original image. In a step 1004, each coefficient is compared with the first threshold as in the first embodiment, and all subsequent, reduced thresholds, to generate a quantised transform coefficient.

In a step 1006, the value of each quantised coefficient is examined, and the position at which the first significant (i.e. non zero) bit occurs is noted, and buffered as a depth mask value. Alternatively, the depth mask values may be directly calculated, as the logarithm (to base 2) of the value of the coefficient (or the integer part thereof).

As a general observation, the lower frequency coefficients generally have higher amplitudes, and therefore shorter runs of leading zeros, than the higher frequency coefficients.

In a step 1008, it is determined whether the last coefficient in the tree has been reached and, if not, in a step 1010 the next coefficient is selected.

In a step 1012, starting from the highest frequency (i.e. lowest layer of hierarchy) coefficients, it is determined for each coefficient whether an offspring coefficient (i.e. one lower in the hierarchy of Figure 8) has a shorter run of leading zeros (in other words becomes significant earlier) than the coefficient in question. If not, the existing depth mask value (equal to the number of leading zeros plus one, i.e. the position of the first significant bit within the coefficient) is retained for that coefficient within the mask buffer 44. If so, the depth value set within the mask buffer is set equal to that of the highest valued offspring coefficient in step 1014. Additionally, a flag is set indicating that the data values for that coefficient should be adjusted, as described below.

In step 1016 it is determined whether the last coefficient has been processed and if not, in step 1018, the next coefficient is selected, until it is determined in step 1016 that all coefficients have finally been processed (ending with the lowest frequency).

Then, in step 1020, starting with the lowest frequency coefficients (i.e. those within the first layer of Figure 8) for each coefficient, the depth mask value within the mask buffer 44 is re-written as a difference from the depth value of the parent coefficient in the layer above. Thus, if a coefficient in one layer has three leading zeros (and hence a depth mask value of four) and that in the layer below has six leading zeros, the latter is re-written as two in the mask buffer 44. In step 1022 it is determined whether the last coefficient (i.e. the highest frequency coefficient has been reached and, if not, in step 1024 the next coefficient is selected, until ail coefficients have thus had differential depth mask values written into the mask buffer 44. Having thus constructed the differential depth mask within the buffer 44 the encoding of the depth mask and image data will now be described.

In step 1030 the process is initialised with the first bit plane, and the first subset or layer of data as in the first embodiment.

In step 1032, it is determined whether the bit plane is yet that one for which the parent coefficient (if any) of the current coefficient turns "ON" (i.e. is significant). For example, if the parent of the current coefficient indicates a depth mask value of two, and the bit plane is the second plane, then the parent will already be "ON".

If so, then the depth mask value for the current coefficient is queued for sending in step 1034. If not, the process proceeds to step 1036, in which it is determined whether the last coefficient of the current layer has been reached. If not, then in step 1038, the next coefficient is selected and the process returns to step 1032.

When all coefficients of the layer have been thus processed (step 1036), then in step 1040 the mask and image data is sent. The image data is represented by three symbols; two representing 0 or 1 (or + or -) and a third, rarely used, symbol indicating that the data is not significant. This last symbol is used where, as described above in relation to step 1014, the depth of a parent symbol is reset to that of a child symbol; for bit planes between that in which the parent is actually turned on due to the child and that at which it should have turned on (i.e. significant data first occurs) this symbol is sent for the parent coefficient, and the decoder relates it to the parent coefficient and ignores its value.

In step 1042, it is determined whether the subset is the last subset and, if not, the next subset is then selected in step 1044. When the last subset has been processed, it is determined (step 1046) whether the last plane has been reached and, if not, the next plane is selected (step 1048) and the process resumes at step 1032. It will thus be seen that, in this embodiment, the hierarchical arrangement is utilised in two ways. Firstly, the length of depth mask values of coefficients lower in the hierarchy (which generally have lower magnitudes than those higher in the hierarchy) are defined relative to the depth masks of those higher in the hierarchy and can therefore generally be represented by lower numbers of bits. Either shorter symbols (i.e. comprising fewer bits) can be allocated for coding them, or they can be represented using variable length codes in which the more frequently occurring offset lengths are represented by shorter codes. In either case, the volume of coding data necessary to represent the depth masks is generally reduced.

Secondly, instead of sending all the depth masks initially, it is made possible only to send depth masks for offspring coefficients after those for parent coefficients have been transmitted. Thus, this embodiment is a "progressive" coder; if the operator of the receiver wishes only to reproduce a low resolution image making use of the first or a few subsequent bit planes, he never needs to receive the depth masks for those coefficients which only become significant subsequently and which are, in this embodiment, only transmitted after their parent becomes significant.

Thus, the sending of some depth masks is deferred until after the sending of their parents, which generally have higher magnitudes and therefore are switched on at an earlier bit plane.

It will be apparent that these two separate uses of the hierarchical structure of data could be employed separately; for example, depth mask values could be defined differentially as described above but all transmitted at the beginning of the transmission prior to any image data where it is not required to implement a progressive coding scheme, or likewise all depth masks could be defined in absolute values rather than differentially, but a progressive scheme could be employed.

It will be apparent that the process of adjusting parent depth mask values to take account of offspring depth mask values described above in relation to steps

1012-1018 is performed to avoid coding errors, since whilst offspring coefficients generally have lower magnitudes than their parents this is not invariably the case.

However, these steps could be omitted if a certain amount of error were to be acceptable, with some slight increase in compression and corresponding loss of accuracy; or alternative means for subsequently transmitting a correction if necessary could be used.

Referring to Figure 14, the corresponding process carried out at the decoder will now be described.

In a step 1 162, the values in the mask buffer 64 are set to the maximum possible lengths, and the coefficient values in the coefficient store 60 are set to zero (or some predetermined value).

In step 1 162, depth mask data are received and, in step 1 164, image data are received.

In step 1 166, it is determined which coefficients are now, having regard to the current bit plane being received, to be turned on, (i.e., those for which the current bit plane number being received matches the depth value stored in the mask buffer 64). By reference to these, it is determined which offspring depth mask values are now being received in step 1 1 62, and these values are then written to the depth mask buffer 64, overwriting the previously stored maximum values therein, in step 1 1 68.

In step 1 170, the values currently stored in the depth mask buffer 64 are reviewed to determine which are currently on (including those just written in the preceding step), and by reference to these, the data received in step 1 164 are allocated to the relevant plane of the relevant coefficient addresses in the coefficient store 60. As noted above, where a special symbol is received indicating that it is insignificant, but relates to a coefficient which is ON because of an offspring coefficient, that data is allocated to the correct coefficient but ignored.

In step 1 172, it is determined whether the current subset is the last subset of the image, and if not, then in step 1 174 the next subset is selected and the process resumes at step 1 162.

After processing the last subset, it is determined in step 1 176 whether the current image plane is the last image plane in the image and, if not, in step 1 178 the next bit plane is selected and the process resumes at step 1 162.

The decoder may be equipped with a control circuit (not shown) arranged to terminate reception at some point, either because enough image data has been received to reconstruct a recognisable image on the VDU 22 or because of external time constraints. It will be seen that, at each cycle passing through step 1 180, an entire bit plane of the image has been received and written to corresponding bit positions within the coefficient store 60 and, accordingly, upon such early termination an image can be reconstructed by the transform decoder 58. Additionally, within a bit plane, low resolution coefficients are received first, allowing the image to be partially reconstructed even where only part of the image data is received. This may be described as an "embedded" coding scheme.

As noted above, it may be preferred to arrange that the depth values are treated by the receiver in such a way as to expect image data for all transform coefficients of the last plane, since most of such coefficient may be "ON" for the least significant bit, thus avoiding the need to send separate mask data and image data.

The depths could be encoded using run-length coding, rather than fixed- length symbols, or other convenient encoding schemes as desired. Fourth Embodiment

Referring to Figures 1 5 and 16, a fourth embodiment of the invention will now be described. The fourth embodiment utilises the hierarchical arrangement of the third embodiment but retains the simple mask symbols of the first. In a step 1 202, steps 902-916 of Figure 13 are performed, as described above, to create a depth mask within the mask buffer 44, in which each coefficient is represented by a mask value which does not turn ON at bit plane later than that of any of its offspring coefficients.

Next, in step 1204, the process is initialised at the first subset of coefficients and the first bit plane.

In step 1206, for the current coefficient, it is determined whether the bit plane is such that the parent coefficient is ON yet in the mask buffer 44, as described above in relation to the second embodiment. If so, in a step 1208, it is determined whether the bit plane is such that the current coefficient is yet ON, and in dependence on this, either a one (if the coefficient should be ON) and a zero (if not) is queued for sending in step 1 208.

In step 1210, it is determined whether the current coefficient is the last of the current subband and, if not, the next coefficient is selected in step 1212 and the process resumes at step 1206. If the current coefficient is the last of the subset, then in step 1214, the mask symbols are transmitted together with bits of image data for all coefficients which are now ON in the mask buffer 44 (i.e. have been turned on in this and all previous bit plane passes). As in the previous embodiments, where a coefficient has been turned on "early" because an offspring coefficient is of greater magnitude, a special data symbol is sent for that coefficient indicating that it does not have a significant value for that bit plane.

In step 1 216, it is determined whether the last subset has been processed and, if not, in step 1218, the next subset is selected and the process returns to step 1206.

When the last subset has been processed, it is determined in step 1220 whether the last bit plane has been processed and, if not, in step 1 222 the next bit plane is selected, the first subset is selected and the process returns to step 1206.

Thus, in this embodiment, mask data for each plane is sent only after the parent coefficient is ON, as in the second embodiment. As with the second embodiment, this enables early termination of the reception process, which will now be described with reference to Figure 16.

In a first step 1302, as in the first embodiment, the contents of the mask buffer 64 are set to OFF and the contents of the coefficient store 60 are set to zero. It is noted that the mask buffer 64 of the receiver in this instances merely retains an ON/OFF flag, regardless of whether that at the transmitter stores depth information or ON/OFF flags.

In a step 1304, mask data for a transmitted subband is received and in a step 1306, image data corresponding to that subband is received. In a step 1308, the masks which are already ON within the mask buffer

64 are located, and the mask data received at step 1304 is interpreted as including offspring mask symbols (either OFF or ON) for each of the offspring.

In step 1310, the new mask values are set in the mask buffer 64, and in step 1312, the image data bit values received in step 1306 are allocated to bit positions corresponding to the current bit plane of the coefficients within the coefficient store 60 for which the masks are set ON within the mask buffer 64 (those symbols indicating non-significant values, if any, being allocated but not written to the coefficient store). In step 1314, it is determined whether the current subband is the last subband of the image and, if not, in step 1316 the next subset is selected, and the process resumes at step 1304.

If the last subset has been reached, in step 1318 it is determined whether the current bit plane is the last bit plane and, if not, in step 1320 the next bit plane is selected and the process resumes at step 1304. Fifth Embodiment

In the preceding third and fourth embodiments, a hierarchical scheme was employed in which parent coefficients were reset to the depth of any offspring coefficient of greater magnitude (this being relatively rare). In those embodiments, the resulting issue of allocating data which was therefore expected by the decoder for that coefficient was solved by providing a special data symbol associated with no significance, and allocated but ignored by the decoder.

This may be well adapted to some forms of encoding of the image data (for example, arithmetic coding), but where it is desired to code the image data using only binary values (and to be able to use entropy or variable length coding such as Huffman coding) an alternative is needed.

Accordingly, in this embodiment, a special mask data symbol is used indicating that the mask is to be set ON but no significant data is to be expected (because it is set ON due to an offspring coefficient). When the third embodiment is modified to utilise this principle, the set of symbols representing the mask data includes not only "normal" depth symbols as described in relation to that embodiment, but also symbols which indicate (for a coefficient with a higher- valued offspring) not only the true depth of the coefficient (as opposed to the adjusted depth as in the third embodiment) but also the fact that the coefficient has offspring which turn ON earlier. The transmitted image data is then binary, and runs of binary values may be Huffman encoded.

At the decoder, on receipt of such a symbol, the decoder expects mask data for one or more of the offspring coefficients earlier than the image data for the parent.

Again, in this embodiment, a special symbol indicating that a given coefficient never turns on may be sent. Likewise, a symbol is provided for the situation where such a coefficient has a higher valued offspring, to indicate that data and a mask value should be expected at the decoder for the offspring of such a symbol.

Sixth Embodiment

In this embodiment, in addition to sending a depth value at which a coefficient is turned ON at the receiver, as in the second or third embodiments, the receiver also has information enabling it automatically to turn a coefficient OFF again, thus truncating its value by omitting lower order bits.

This may be arranged by providing that the receiver automatically expects no data for the last bit planes for the highest frequency coefficients (which need not always be represented to high accuracy).

Alternatively, the depth values for some or all coefficients may extend only over a limited number of bit planes, so that the receiver automatically resets the mask for a given coefficient OFF, say, three bits after it is turned ON. In this case, where it is desired to send further image data, a further depth symbol turning the coefficient ON again can also be sent.

Equally, for all or just high-frequency coefficients, it is possible to send a further set of depth mask values additional to those described in the second, third or fifth embodiments, to set the bit planes at which each coefficient is switched OFF. Seventh Embodiment

In the preceding embodiments, systems have been described in which the new mask data, for newly significant coefficients, is transmitted before the signal data for existing coefficients. The present inventor has, however, realised that a more embedded result than this is possible, by sending coefficients for existing data first.

In this embodiment, for a given bit plane, the data is sent in the following order; new coefficient data bits for existing "ON" coefficients; then, either mask symbols which combine the mask and data values (as in the first embodiment) or mask bits followed by the data bits for them (as described in the second embodiment).

It will be seen that, for existing ON coefficients in a given bit plane, only one bit is necessary to represent the bit of the coefficient. On the other hand, for coefficients which have just been turned ON in that plane (i.e. become significant in that plane), two bits are necessary - one to turn the mask ON and one to represent the data, or two to make up a combined mask/significance symbol.

Thus, if reception is terminated during a given bit plane, it is a more efficient use of available bits prior to termination to use as many as necessary to represent coefficient data for existing ON coefficients rather than for turning on new coefficients.

The necessary modifications to the above-described embodiments for implementation of this embodiment will be apparent to the skilled person.

Eighth Embodiment

This embodiment extends more generally the principle expressed above in the seventh embodiment, to allow coefficient data of a given bit plane to be sent for coefficients which are already ON before sending mask data for other coefficients, even where they become significant at a higher order bit plane.

The object of an embedded coding scheme is to send the highest significance information first, in the fewest bits - in other words, to ensure that each item of data is sent in as fine a resolution as possible, and contributes . This can be approximated by sending data carrying the highest image power first - in other words, at each stage of transmission, maximising 5P/B where dP is the change in power of the received signal and SB is the number of bits to transmit that change.

The amplitude of a coefficient is the sum of amplitudes of its bits. Each zero-valued bit contributes nothing to the amplitude, and all other bits contribute in proportion to 2', where i indicates the bit plane (starting at zero for the least significant bit, and in order of significance).

The power of a coefficient is proportional to the square of its amplitude, in other words to (∑b'.2' )² where b is the bit value. Thus, for a coefficient containing just two non-zero terms b' and b^J, the power is

(2¹ + 2j)² = 2²ⁱ + 2.2ⁱ⁺ⁱ + 2²ⁱ Thus, in addition to the sum of the amplitudes of the bits, each bit makes a contribution to the overall power of the coefficient which is a cross-product of its amplitude and that of other (non-zero) bits.

In progressive coding schemes, higher order bits will already have been transmitted for some coefficients. Lower order bits may not be transmitted at all. In the above equation, one of the terms (2²' ) will thus already be available at the receiver, since one of the bits will already have been sent before the other. The value of the other two terms will be zero if the bit value b' is zero, and 2²ⁱ plus the cross product term otherwise.

The 2²ⁱ term will, of course, have the same value for all non-zero bits of that bit plane, but the value of the cross-product term will vary, and will be greater for coefficients which are higher-valued. In schemes, such as those described above, in which mask bits are sent, then the cross-product term will always be zero for coefficients which are not yet ON.

Thus, the impact (i.e. the power change at the receiver) of transmitting a given bit can be determined by evaluating the cross-product terms of the amplitude represented by that bit and those of the bits of the same coefficient which have gone before it.

Accordingly, in this embodiment, when the coefficients have been determined and stored in the coefficient store 60 at the encoder, rather than ordering and transmitting them as described in relation to Figure 9 above, the following process illustrated in Figure 17 is performed.

Starting in the coefficient order described in the first embodiment, each coefficient or mask symbol is transmitted, in turn, for the first bit plane in step 2002. In step 2004, for each coefficient in turn, the power increment per bit for sending a non-zero bit on the next bit plane is calculated.

For coefficients which are ON, this is achieved by calculating the power represented by the bits so far sent (i.e. the square of their amplitude) and the power which would be obtained by adding a non-zero bit on the next bit-plane (rather than the actual value of the next bit) and taking the difference between the two. For coefficients which are OFF, since an extra bit is required to turn the coefficient ON first, the result thus calculated is divided by two.

The next bit or symbol of the coefficient having the highest of these power increments is then sent (step 2006). Where multiple coefficients have the same power increment value, the order of the previous embodiments (i.e. within each bitplane, coefficient order within each subband) is preserved.

Referring to Figure 18, at the receiver, likewise, the first bit-plane is received (step 2012) and the power increment for each coefficient is calculated (step 2014), in exactly the same way, from the same data as at the transmitter.

The next bit to be received in step 201 5 is then allocated to the coefficient with the highest power increment. Since this calculation is based on the same data as at the transmitter, the coefficient selected will be correct.

After this, at both the transmitter and the receiver (steps 2008 and 2018 respectively) the incremental power is recalculated for that coefficient, by examining the next bitplane for that coefficient.

The process at the transmitter then returns to step 2006, and that at the receiver to step 2106. Now, each coefficient incremental power is re-examined at the transmitter. The highest valued is again transmitted. At this stage, this may be a bit of a next coefficient, of the same plane as that before it, or a mask symbol of a next coefficient in the same plane. Alternatively, it may be the next bit of the coefficient which has just been transmitted, depending upon whether the power increment for the next bitplane b_j'^{+ 1} of that coefficient is greater than that b for the same bitplane of the next coefficient.

Thus, at the transmitter and at the receiver, bits are transmitted, received and allocated in order of the incremental power which would have been received by a non-zero bit sent next for each coefficient. The actual (zero or non-zero or mask) value is, in each case, sent.

The incremental power values of all mask symbols will be the same, and lower than the incremental power values of data bits, for the same plane, of each ON coefficient as noted above. Thus, after transmitting data bits for the coefficients of a plane (and some for the plane below), all the mask values for the plane will be sent. To assist the receiver, a demarcation flag code indicating the start of a block of mask symbols may be sent.

The order of transmission of bits therefore lies between transmission of entire coefficients, in order of magnitude, and transmission in the order described in the preceding embodiments.

In a further modification of this embodiment, the process of transmission is carried out as follows:

FOR EACH IMAGE (OR PART THEREOF)

FOR EACH MASK PLANE PS (MSB TO LSB) dS = 5POWER/5BITS TO SWITCH

FOR EACH COEFF ALREADY ON (HIGHEST MAGNITUDE TO LOWEST) FOR EACH REFINEMENT PLANE PR (PS TO LSB)

IF NOT ALREADY REFINED

5R = 5POWER/5BITS FOR REFINING IF 5R >5S REFINE IT END

END END END SWITCH ON NEW COEFFS FOR PLANE PS END

END 2 PS

In calculating 9S, 5P0WER is taken as 2 In calculating 9R, δPOWER is taken as a² - (a + 2^PR)² = a^{2PR+ 1} + 2^2PR

Estimating dBITS is more difficult - the actual number of bits used to encode depends upon the encoding scheme and where variable length coding (e.g. Huffman or run length coding) is used, the necessary number of bits is variable. In the above, it may be taken as 2. In other embodiments, the actual number of bits which would be required could be calculated by taking the encoding scheme into account.

Thus, in this process, at each bit plane ("switch plane"), prior to sending the mask data for newly significant coefficients, coefficient data is sent for that plane of each currently ON coefficient (as in the seventh embodiment).

Additionally, for each coefficient, "refinement" bits from less significant planes are continually sent until they cease to contribute more power than sending bits from the switch plane of the next coefficient.

Ranking the coefficients into magnitude order assists in making the coding embedded, since the cross-product power terms will be higher for larger-magnitude coefficients which are sent first. The decoder is able to perform the reverse process to allocate the bits, as it has the same information as at the encoder.

Although this embodiment has been described as a variant of the first, it is apparent that it could use the depth mask principles of the second and third embodiments.

It will be understood that the principle underlying this embodiment could be applied to other types of codec using other scanning orders (such as zigzag scanning), even those not using masks (for example where all coefficients are always sent).

Finally, it is noted that a ranking in order of incremental power may be obtained by the following variant of this embodiment:

• At the transmitter, converting each coefficient amplitude value to a power value (i.e. squaring each coefficient); • Ordering the coefficients by magnitude;

• Sending the bitplanes of the power values, but in the mariner disclosed above in relation to amplitude (together with sign information on the original coefficients, where these are bipolar);

• At the receiver, receiving the power bits;

• Converting the received power bits into amplitude bits.

This has the disadvantage of increasing the range, and hence number of bitplanes, used but the advantage of simplifying the ordering process. Other Embodiment and Modifications It will be understood that the present invention is not limited to the above- described embodiments. In particular, the features of each above-described embodiment can be combined, except where they are obviously incompatible, with any other; in particular, each may have separate or combined mask and image data streams, and all hierarchical embodiments may either indicate parent coefficient insignificance in the image data or mask data streams. In each case, mask and image data may be encoded using variable length, run length, Huffman, or other encoding schemes to further reduce redundancy.

Other variants are also possible. For example, although the Discrete Wavelet Transform has been disclosed, we have found that the present invention is also applicable to other transforms. Also, Discrete Wavelet Transforms in which high frequency bands are also further filtered are known; these are referred to as "Wavelet Packets" in Strong & Nguyen, "Wavelets and Filter banks", Wellesley, 1996.

For example, the Fourier, Discrete Fourier or Fast Fourier transforms may be used, as may the Discrete Cosine Transform (DCT), the Walsh transform, the Lapped Orthogonal Transform (LOT), the Karhunen-Loeve transform (KLT) and others.

In such cases, it is preferable that the coefficients should be ordered into subbands which are analogous to those produced in the Discrete Wavelet Transform described above, and that the subbands should be organised into layers as described above. The partitioning which may be employed for Discrete Cosine Transforms is described in "A DCT-based embedded image coder", Xiong et al, IEEE signal processing letters Vol. 3 No. 1 1 November 1996; other partitioning methods applicable to wavelet transforms are described in Davis & Chawla, referred to above.

Although two-dimensional image transforms are disclosed above, the application of the invention to one-dimensional transforms on, for example, speech data, or to higher dimensional transforms, is not excluded.

Although the above described order of scanning and transmitting data is preferred for the advantages given, it will be understood that where it is desired to dispense with these advantages other scanning orders may be employed.

Although the above description discloses the use of bit planes, it will be understood that ternary or other symbols could be employed rather than binary bits, or more general arrangements of thresholds could be employed. Although the above described embodiments illustrate a transmission system, it will be understood that the present invention does not require the presence of peripherals such as cameras, displays and analogue to digital converters, but could instead receive image data already in digital form (or even in transformed form) and process the data directly. For example, the present invention may be employed to record data onto a record carrier such as a CD ROM or other record carrier. In such cases, the invention requires only a digital processor and associated memory for either the decoding or encoding process, or both.

Many other modifications and variants will be apparent to the skilled person, and the present invention is accordingly not to be limited to the above described embodiments.

Claims

1 . A method of signal coding comprising the step of decomposing the signal into transform coefficients, further comprising the step of generating mask data indicating for which, of said transform coefficients, transform data will not be sent.

2. A method according to claim 1 , further comprising ordering said mask data for sending before said transform data.

3. A method according to claim 1 , further comprising the step of decomposing each of said transform coefficients into a sequence of symbols of decreasing significance defining multiple symbol planes, and sending high significance symbols of some coefficients before low significance symbols of at least some others.

4. A method according to claim 3, in which said symbols are binary bits.

5. A method according to claim 3, in which mask data for coefficients is sent prior to one or more symbol planes to which it relates.

6. A method according to claim 1 , in which said mask data consist of symbols indicating whether transform data will or will not be sent for each coefficient.

7. A method according to claim 1 , in which said mask data consist of symbols indicating either that transform data will not be sent for a coefficient, or the most significant symbol of the coefficient.

8. A method according to any preceding claim, in which the mask data comprises one or more codes relating to a number of successive coefficients for which transform data will not be sent.

9. A method according to any preceding claim, in which said coefficients are hierarchically ordered, and in which said mask data for each offspring coefficient may be sent only if the mask symbol for the parent coefficient indicates that transform data will be sent for the parent coefficient.

10. A method according to claim 9, in which said mask data includes symbol indicating, for a coefficient, that transform data will not be sent for that coefficient, and that mask data will be sent for offspring coefficients thereof.

1 1 . A method according to claim 3 or any of claims 4 to 10 appended thereto, in which the mask data comprise depth symbols indicating, for each coefficient, at which of said symbol planes transform data for said coefficient will first be sent.

12. A method according to claim 1 1 , in which said coefficients are hierarchically ordered, and in which said depth symbols comprise symbols which indicate the number of symbol planes for which transform data will not be transmitted for said coefficient, the number for each offspring coefficient being defined relative to the number for its parent within the hierarchical order.

13. A method according to claim 12 or claim 1 1 appended to claim 9, in which said depth symbols for each parent coefficient are set to indicate the shortest of the lengths at which said sequences of symbols for that parent coefficient or any offspring coefficient thereof first becomes non-zero.

14. A method according to any of claims 1 1 to 13 in which said depth signals comprise variable length signals, such as run length symbols.

15. A method according to any preceding claim, in which said step of decomposing into transform coefficients comprises applying a two-dimensional spatial transform.

16. A method according to claim 1 5, in which said transform is performed by repeated analysis filtering/subsampiing stages.

17. A method according to claim 15, in which the transform is a wavelet transform.

18. A method according to claim 15, in which the transform is an image transform operating on an image.

19. A method according to claim 18, in which the transform is arranged to order said coefficients into a plurality of layers composed of blocks of coefficients, each successively lower layer having more coefficients than the layer above.

20. A method according to claim 19, in which said successively lower layers relate to successively higher frequencies than the layers above.

21 . A method according to any preceding claim, further comprising the step of grouping coefficients into sub-bands.

22. A method according to claim 19 or claim 20, further comprising coding the mask data and transform data for sending in a progressive sequence of said layers.

23. A method according to any preceding claim, further comprising the step of sending the mask data and transform data for recording.

24. A method according to any preceding claim, further comprising the step of sending the mask data and transform data for transmission.

25. A method according to claim 3, in which mask data for coefficients is sent after data for one or more symbol planes, to which it relates, of other coefficients.

26. Encoding apparatus for performing the method according to any preceding claim.

27. A method of signal decoding comprising the steps of receiving mask data indicating for which, of a plurality of transform coefficients, transform data will not be received; allocating received transform data to transform coefficients in accordance with said mask data; and performing an inverse transform on said received transform data to recover said signal.

28. A method according to claim 27, in which said transform data comprises a sequence of symbols of decreasing significance, further comprising the step of receiving high significance symbols of some coefficients before low significance symbols of at least some others.

29. A method according to claim 28, in which said mask data comprises depth symbols indicating, for each coefficient, at which of said sequence of symbols transform data for said coefficient will first be received, in which said step of allocating takes place for a coefficient on arrival of the symbols indicated by said depth symbols.

30. A method of decoding a signal encoded by the method of any of claims 1 to 25.

31 . Signal decoding apparatus for performing the method according to any of claims 27 to 30.

32 A method of signal encoding comprising the steps of: decomposing the signal into coefficients; decomposing the coefficients into a sequence of symbols of decreasing significance defining multiple symbol planes; and determining a symbol transmission sequence for data representing the symbols, from higher significance symbols to lower significance symbols; characterised in that the sequence is determined by taking into account the values of symbols previously entered in the sequence for each coefficient, in a manner which can be reproduced by a decoder.

33 A method according to claim 32, in which said order permits transmission of a first symbol of a first plane and at least one further symbol of progressively decreasing significance, of one coefficient, prior to transmission of at least one symbol of said first plane of a further coefficient.

34. A method of decoding a signal comprising: receiving a sequence of symbols; allocating said symbols to a set of signal coefficients, in order of decreasing significance for each coefficient; and reconstructing the signal from the coefficients; characterised in that the step of allocating is performed by taking into account the value of the symbols previously received for each coefficient.

35 A method of signal coding comprising the steps of : decomposing the signal into coefficients; decomposing the coefficients into a sequence of symbols of decreasing significance defining multiple symbol planes; starting with the highest significance symbol plane, determining a symbol transmission sequence; characterised by determining the order of the coefficients and symbols thereof taking into account the signal power which would be generated by the transmission of each symbol of each coefficient.

36 A method of coding a plurality of signal coefficients comprising deriving a measure of the power of each, using the measure to order data representing each coefficient within symbol planes thereof, and coding the data thus ordered.

37 A method according to claim 36, in which said measure is derived by squaring the coefficients to order the data and to provide said measure.