WO2007044333A2 - Encoding and decoding of a video signal - Google Patents

Encoding and decoding of a video signal Download PDF

Info

Publication number
WO2007044333A2
WO2007044333A2 PCT/US2006/038624 US2006038624W WO2007044333A2 WO 2007044333 A2 WO2007044333 A2 WO 2007044333A2 US 2006038624 W US2006038624 W US 2006038624W WO 2007044333 A2 WO2007044333 A2 WO 2007044333A2
Authority
WO
WIPO (PCT)
Prior art keywords
adaptive
image area
motion
wavelet transform
frames
Prior art date
Application number
PCT/US2006/038624
Other languages
French (fr)
Other versions
WO2007044333A3 (en
Inventor
Ebroul Izquierdo
Marta Mrak
Nikola Sprljan
Original Assignee
Motorola Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Motorola Inc. filed Critical Motorola Inc.
Priority to EP06816113A priority Critical patent/EP1938598A4/en
Publication of WO2007044333A2 publication Critical patent/WO2007044333A2/en
Publication of WO2007044333A3 publication Critical patent/WO2007044333A3/en

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/167Position within a video image, e.g. region of interest [ROI]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/129Scanning of coding units, e.g. zig-zag scan of transform coefficients or flexible macroblock ordering [FMO]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/12Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
    • H04N19/122Selection of transform size, e.g. 8x8 or 2x4x8 DCT; Selection of sub-band transforms of varying structure or type
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136Incoming video signal characteristics or properties
    • H04N19/137Motion inside a coding unit, e.g. average field, frame or block difference
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136Incoming video signal characteristics or properties
    • H04N19/137Motion inside a coding unit, e.g. average field, frame or block difference
    • H04N19/139Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/157Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
    • H04N19/159Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/61Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/61Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
    • H04N19/619Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding the transform being operated outside the prediction loop
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/63Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding using sub-band based transform, e.g. wavelets
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/63Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding using sub-band based transform, e.g. wavelets
    • H04N19/635Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding using sub-band based transform, e.g. wavelets characterised by filter definition or implementation details

Definitions

  • the invention relates to encoding and/or decoding of a video signal and in particular to encoding and/or decoding using a wavelet transform.
  • the redundancy of information between consecutive frames is often used to reduce the coding rate.
  • adjacent frames in a video sequence comprise similar images and it is therefore possible to represent information of a frame using data from already encoded frames.
  • motion compensation techniques are used to remove temporal redundancy to produce so-called temporal frames.
  • Motion models (such as well known block-based models) are used in this context as parameters that drive motion compensation.
  • an improved system for video encoding would be advantageous and in particular a system allowing increased flexibility, improved wavelet transform encoding/decoding, improved adaptation, facilitated implementation, reduced computational requirements, reduced data rate and/or improved performance would be advantageous.
  • the Invention seeks to preferably mitigate, alleviate or eliminate one or more of the above mentioned disadvantages singly or in any combination.
  • an encoder for encoding a video signal comprising: means for generating motion compensated video frames for the video signal; wavelet means for applying an adaptive spatial wavelet transform to the motion compensated video frames to generate spatially transformed frames; means for generating encoded video data for the video signal in response to the spatially transformed frames; and adaptive means for determining adaptive parameters for the adaptive spatial wavelet transform in response to motion data for the motion compensated video frames.
  • the invention may allow an improved video encoding.
  • An improved wavelet transformation may be used which more closely reflects the characteristics of the motion compensated video frames thereby allowing a more efficient encoding resulting in improved quality to data rate ratio.
  • the invention may allow adaptation of an adaptive spatial wavelet transform at the encoder (and decoder) without resulting in the necessity for explicit data for the adaptation to be distributed with the encoded video data. Thus, a reduced data rate may be achieved. Also, the adaptation may be based on processing which has already been performed for other purposes resulting in a facilitated and reduced complexity implementation.
  • the invention may provide a better wavelet domain representation taking into consideration both spatial and temporal characteristics of the video signal.
  • the improved wavelet domain representation may allow a more efficient compression of the video signal.
  • the adaptive parameters may comprise one or more adaptive parameters which are used to modify the operation of the adaptive spatial wavelet transform.
  • the motion data may comprise ' identifications of one or more motion compensated image areas in one or more of the motion compensated video frames.
  • the adaptive parameters comprise wavelet transform coefficients .
  • the adaptive means is arranged to determine a first image area and a second image area of a first frame of the motion compensated video frames in response to the motion data and the adaptive parameters comprise wavelet transform coefficients for a boundary between the first image area and the ' second image area.
  • the wavelet transform may be adapted to provide a more efficient wavelet representation of higher frequency components around image area transitions which tend to have in increased concentration of such components.
  • the feature may allow a better encoding of detail around an image area transition.
  • the adaptive means is arranged to determine the first image area and the second image area as image areas having different motion characteristics.
  • This may provide efficient encoding and/or facilitated/ reduced complexity encoding.
  • different motion characteristics tend to indicate that the image areas belong to different objects in the image and thus to the existence of a sharp image transition (e.g. an object edge) comprising high frequency components and high levels of detail.
  • the first image area is a motion compensated image area and the second image area is a non-motion compensated image area.
  • the adaptation of the wavelet transform may be achieved without necessitating that additional adaptation data is included in the encoded video signal.
  • the first image area is an inter-coded image area and the second image area is an intra-coded image area.
  • An inter-coded image area is an image area encoded relative to an image area in another frame.
  • An intra-coded image area is an image area which is not encoded relative to an image area in any other frame.
  • the feature may provide efficient encoding and/or facilitated/ reduced complexity encoding.
  • the adaptation of the wavelet transform may be achieved without necessitating that additional adaptation data is included in the encoded video signal.
  • the adaptive parameters comprise wavelet transform coefficients for picture elements of the first frame adjacent to the boundary.
  • This may provide efficient encoding and/or facilitated/ reduced complexity encoding. It may allow a particularly efficient and/or facilitated adaptation of the adaptive spatial wavelet transform.
  • the adaptive parameters do not comprise wavelet transform coefficients for picture elements of the first frame not adjacent to the boundary.
  • This may provide efficient encoding and/or facilitated/ reduced complexity encoding. It may allow a particularly efficient and/or facilitated adaptation of the adaptive spatial wavelet transform.
  • the wavelet means is arranged to apply the adaptive spatial wavelet transform by applying an adaptive wavelet lifting operation.
  • the adaptive parameters comprise coefficient values for multiple levels of the adaptive wavelet lifting operation.
  • the boundary may be the same for each of the multiple levels.
  • the adaptive parameters comprise different coefficient values for different levels of the multiple levels.
  • the wavelet means is arranged to apply the adaptive spatial wavelet transform to transformed spatial subband data.
  • the encoder further comprises means for generating an encoded data signal for the video signal comprising the encoded video data and the motion data.
  • the encoded data signal is a self consistent signal comprising all the data which is required to decode the video signal.
  • the encoded data signal may thus be transmitted, broadcast or distributed to one or more decoders .
  • the encoder is arranged to not include data of the adaptive parameters in the encoded data signal.
  • This may provide for a more efficient encoded data signal and may in particular reduce the data rate for a given encoded video quality.
  • a decoder for decoding an encoded video signal, the decoder comprising: means for receiving the encoded video signal; means for generating spatially transformed frames from the encoded video signal; wavelet means for applying an adaptive inverse spatial wavelet transform to the spatially transformed frames to generate motion compensated video frames; means for generating motion data for the motion compensated video frames; and adaptive means for determining adaptive parameters for the adaptive inverse spatial wavelet transform in response to the motion data.
  • the adaptive means is arranged to determine a first image area and a second image area of a first frame of the motion compensated video frames in response to the motion data and the adaptive parameters comprise wavelet transform coefficients for a boundary between the first image area and the second image area.
  • the adaptive means is arranged to determine the first and second image areas as image areas having different motion characteristics .
  • a method of encoding a video signal comprising: generating motion compensated video frames for the video signal; applying an adaptive spatial wavelet transform to the motion compensated video frames to generate spatially transformed frames; generating encoded video data for the video signal in response to the spatially transformed frames; and determining adaptive parameters for the adaptive spatial wavelet transform in response to motion data for the motion compensated video frames.
  • a method of decoding an encoded video signal comprising: receiving the encoded video signal; generating spatially transformed frames from the encoded video signal; applying an adaptive inverse spatial wavelet transform to the spatially transformed frames to generate motion compensated video frames; generating motion data for the motion compensated video frames; and determining adaptive parameters for the adaptive inverse spatial wavelet transform in response to the motion data.
  • a computer program product enabling the carrying out of a method as described above.
  • FIG. 1 is an illustration of an encoder for encoding a video signal in accordance with some embodiments of the invention
  • FIG. 2 illustrates an example of a lifting operation in accordance with some embodiments of the invention.
  • FIG. 3 illustrates a decoder in accordance with some embodiments of the invention.
  • FIG. 1 is an illustration of an encoder for encoding a video signal in accordance with some embodiments of the invention.
  • the encoder 100 comprises a video frame source 101 which generates video frames from a video signal to be encoded.
  • the video frame source 101 can be a receiver for receiving a digitised video signal already comprising suitable video frames from an external source, or that it can for example be an analog video signal which is then digitised and structured in suitable frames as will be well known to the person skilled in the art.
  • the video frame source 101 is coupled to a motion estimation processor 103 which performs motion estimation and compensation on the video frames to generate motion compensated video frames.
  • a motion estimation processor 103 which performs motion estimation and compensation on the video frames to generate motion compensated video frames.
  • Various methods and algorithms for motion estimation and compensation of video frames is well known to the person skilled in the art and is for brevity not described in detail herein.
  • the motion estimation processor 103 simply determines the movement of motion estimation blocks of a fixed size in different video frames.
  • the block size can be 16 x 16 pixels (picture elements) . If the video frames comprise images with a moving object, a given block of one image can frequently be found in a subsequent image at an (often slightly) different position. An efficient video encoding can then be obtained by merely encoding the displacement of the original block rather than the actual image data of the block in the subsequent frame. This displacement is known as the motion vector. Improved quality can be obtained by determining the differential between the blocks of the different frames and encoding this residual or error signal. It will be appreciated that more advanced motion compensation is 5 possible using for example further degrees of freedom in the movement (such as rotation) or differently shaped motion compensation blocks (possibly adaptive) . Indeed, advanced two or three dimensional motion estimation models for different objects can be used.
  • a simple approach comprises a brute force search of subsequent frame (s) to detect any blocks that closely correspond to a block of a previous frame in order
  • motion estimation comprises evaluating a simple or complex object motion model to determine locations in subsequent frames.
  • the motion estimation processor 103 is coupled to a 20 wavelet processor 105.
  • the wavelet processor 105 implements an adaptive spatial wavelet transform which is applied the compensated video frames to generate spatially transformed frames.
  • wavelet processing may in many situations 25 provide a more efficient representation than conventional Fourier transforms.
  • the motion estimation processor 103 provides temporal frames data to the wavelet processor 30 105, on which the adaptive spatial wavelet transform is applied.
  • the output of the motion estimation processor 103 is the spatio-temporal subbands of the source video frames .
  • the wavelet processor 105 is coupled to an encoding processor 107 which is fed the motion compensated video frames (in the form of spatio-temporal subband data) .
  • the encoding processor 107 is arranged to generate encoded video data for the transformed video frames. Specifically, the encoding processor 107 performs quantisation, scaling and encoding of the data values of the transformed video frames as will be known to the person skilled in the art.
  • the incoming processor 107 can proceed to generate a complete encoded video signal which comprises all the information required to decode the video signal.
  • the encoding processor 107 includes the motion data used by the motion estimation processor 103 for the motion compensation.
  • the encoding processor 107 is coupled to a transmit interface 109 which is fed the encoded video signal and which is arranged to transmit the signal to one and more decoders .
  • the transmit interface 109 for example can be an interface to the Internet or e.g. a radio transceiver, such as cellular radio transceiver, arranged to transmit the encoded video signal over an air interface of a wireless communication system.
  • the applied wavelet transform is an adaptive wavelet transform. Specifically, some or all of the coefficients of the wavelet transform can be adapted to the specific characteristics of the individual video frames.
  • the adaptation of the wavelet transform is performed in response to motion data generated by the motion estimation processor 103 and used for the motion compensation of the motion compensated video frames.
  • the encoder 100 comprises an adaptation processor 111 which is arranged to determine adaptive parameters for the adaptive spatial wavelet transform in response to motion data for the motion compensated video frames .
  • the adaptation processor 111 and the wavelet processor 105 is applied to one frame at a time. In the following, the processing of one frame will be described in more detail.
  • the adaptation processor 111 comprises an image area processor 113 which is arranged to identify different image areas in the video frame in response to the motion data. Specifically, the image area processor 113 divides the video frame into image areas for which motion compensation has been performed and image areas for which no motion compensation has been performed. The different image areas can easily be determined by the image area processor 113 as the motion data generated by the motion estimation processor 103 explicitly defines areas that are motion compensated. This information is already required by a decoder in order to apply the corresponding motion compensation at the decoding end. In the example, motion compensation is performed relative to video data in other video frames. Thus, a motion compensated area will be an inter-coded image area, meaning that it is encoded relative to video data in other frames. At the same time, non motion compensated image areas will be intra-coded image areas meaning that they will be encoded without reference to video data in other video frames.
  • the adaptation may be based only on inter/intra image areas, only on motion compensated/non-motion compensated image areas or on both inter/intra and motion compensated/non-motion compensated image areas .
  • the image area processor 113 thus divides the video frame into inter and intra areas. This creates a boundary between the intra areas and the inter areas and the wavelet transformation is adapted for these boundaries .
  • the adaptation processor 111 further comprises a parameter processor 115 which determines suitable adaptive parameters (may in some embodiments be a single adaptive parameter) for the wavelet transformation. Specifically, the parameter processor 115 receives information from the image area processor 113 identifying the inter and intra image areas (for example by an identification of the boundary between these) . The parameter processor 115 proceeds to determine adaptive parameters which are suited for the specific image area characteristics of the video frame .
  • the parameter processor 115 determines coefficients for the wavelet transformation taking into account the identified different areas of the video frame.
  • the parameter processor 115 of FIG. 1 selects a fixed set of wavelet coefficients for wavelet operations on pixels within each image area. However, for pixels adjacent to the boundaries between intra and inter image areas, different coefficients are applied.
  • the adaptation coefficients are selected such that the wavelet coefficients for pixels adjacent to a boundary are better suited for sharp transitions or edges in the image. Thus, the coefficients are selected such that the prediction and correlation for pixels of different image areas is substantially reduced.
  • image areas having different motion compensation characteristics are transformed as independent (or at least less dependent) objects.
  • motion information to determine the adaptation parameters for the wavelet transformation of a temporal frame
  • a spatially transformed frame which is much better suited for further coding is achieved.
  • the adaptation is based on information which is already- required for decoding, no additional data is required to be transmitted thus allowing a reduced data rate for a given encoded video quality.
  • the application of an adaptive spatial wavelet transform results in the energy introduced into low pass subbands by wavelet filtering over edges between intra and inter areas to be reduced. Specifically, intra-coded areas in high pass frames possess different properties than the other areas of motion-compensated frames.
  • the non-adaptive spatial energy compaction fails to concentrate most of the energy into low pass frames, leaving high-amplitude "edge" coefficients in high-frequency subbands.
  • the described approach thus allows the adaptation of the wavelet transform to be based on local properties of a high-pass temporal frame.
  • the wavelet processor 105 specifically implements the wavelet transform by a lifting algorithm.
  • the adaptation processor 111 is arranged to reduce the effect of the linear prediction of the lifting implementation on the adaptation boundaries between the inter- and intra-coded image areas of a frame. As a consequence of reducing this prediction between intra and inter coded areas, intra classified areas are transformed as (more) independent objects.
  • FIG. 2 illustrates an example of a lifting operation in accordance with some embodiments of the invention.
  • a 1-dimensional wavelet transform lifting algorithm is illustrated.
  • a non-adaptive lifting algorithm (resulting in a fixed transform) is applied to pixels (or input coefficients) that are not next to any adaptation boundary (or adaptation border) .
  • the wavelet transform within each area is the same whether the area is an inter area or an intra area and the same lifting coefficients are used.
  • the coefficients are changed to adapt the lifting algorithm.
  • the detected adaptation boundaries are used for choosing the suitable lifting algorithm coefficients during the transform of each frame pixel .
  • a series of lifting steps are applied to the pixels using adapted lifting coefficients or weights between pixels that belong to different image areas. Specifically, the absolute values of the coefficients are reduced between pixels of different image areas relative to coefficients applied to pixels within the same image area.
  • the lifting step that is performed on a pixel in one area, the neighbouring pixel from a different area is taken with the zero weight, while the other pixel from the same area is taken with the weight of 2.
  • the coupling between pixels from areas of different types is reduced or completely cancelled (illustrated by dashed lines in FIG. 2) .
  • the wavelet processor 105 uses the same adaptation boundary for all levels (or steps) of the lifting operation. Similarly, the same coefficients may be used at different levels of the lifting operation, or in some embodiments the coefficients may be varied between different levels.
  • the wavelet processor 105 of the encoder 100 of FIG. 1 applies the adaptive transform to the original high-pass frame and to each low-pass subband produced by one level of spatial transform.
  • the adaptive spatial wavelet transform can be applied separately to each colour component (e.g. Y, U, V), as typically the same motion information is used for all components.
  • the adaptation of the adaptive spatial wavelet transform of the encoder 100 is based on motion data which is already included in the encoded video signal in order to allow a decoder to perform correct decoding of the motion compensated data. Accordingly, the encoding processor 107 includes only the motion data in the encoded video signal and does not include any other side information related to the adaptation of the adaptive spatial wavelet transform.
  • FIG. 3 illustrates a decoder 300 in accordance with some embodiments of the invention.
  • the decoder 300 is operable to decode the signal received from the encoder 100 of FIG. 1. Specifically, the decoder 300 is arranged perform the ⁇ inverse operations of the encoder 100 and in particular it is arranged perform an inverse adaptive spatial wavelet transform in response to motion data.
  • the decoder 300 comprises a video data receiver 301 which receives the encoded video signal from the encoder 100.
  • the video data receiver 301 is coupled to a motion data extractor 303 which extracts motion data from the encoded video signal.
  • the motion data extractor 303 furthermore generates spatially transformed frames from the encoded video signal.
  • the motion data extractor 303 is fed to an inverse wavelet processor 305 which applies an adaptive inverse spatial wavelet transform to the spatially transformed frames to generate motion compensated video frames.
  • the motion compensated video frames are fed to a decoding processor 307 coupled to the inverse wavelet processor 305.
  • the decoding processor 307 is arranged to generate a decoded video signal from the motion compensated video frames.
  • the decoder 300 furthermore comprises a decoder adaptation processor 309 which is coupled to the motion data extractor 303 and the inverse wavelet processor 305.
  • the decoder adaptation processor 309 is arranged to determine adaptive parameters for the adaptive inverse spatial wavelet transform in response to the motion data.
  • the decoder adaptation processor 309 comprises a decode image area processor 311 which divides a given video frame into motion compensated and non-motion compensated image areas (corresponding to inter/intra image areas) .
  • the decoder adaptation processor 309 furthermore comprises a decode parameter processor 313 which determines the adaptive parameters in response to the identified inter/intra image areas.
  • the decode parameter processor 313 operates similarly to the parameter processor 115 of the encoder 100 and determines adaptive wavelet coefficients for pixels adjacent a boundary between the intra/intra image areas. These coefficients are fed to the inverse wavelet processor 305 and are used for the inverse adaptive spatial wavelet transform.
  • the operation of the adaptive spatial wavelet transform of the encoder 100 is effectively inversed using only motion data already included in the encoded signal for other purposes.
  • motion information drives the inverse spatial transform in the same way as at the encoder 100.
  • the temporal subbands are synthesised using the same adaptation boundaries that define the applied adaptive lifting. Therefore invertibility of the adaptive transform is ensured.
  • the proposed adaptive transform can be used in scalable video coding scenarios for all combinations of spatial, temporal and quality scalabilities .
  • the invention can be implemented in any suitable form including hardware, software, firmware or any combination of these.
  • the invention may optionally be implemented at least partly as computer software running on one or more data processors and/or digital signal processors.
  • the elements and components of an embodiment of the invention may be physically, functionally and logically implemented in any suitable way. Indeed the functionality may be implemented in a single unit, in a plurality of units or as part of other functional units. As such, the invention may be implemented in a single unit or may be physically and functionally distributed between different units and processors .

Abstract

An encoder (100) comprises a motion estimation processor (103) which generates motion compensated video frames for a video signal. The motion compensated video frames are fed to a wavelet processor (105) which applies an adaptive spatial wavelet transform to the frames to generate spatially transformed frames. An encoding processor (107) generates encoded video data for the video signal in response to the spatially transformed frames. The encoder (100) further comprises an adaptation processor (111) which determines adaptive parameters for the adaptive spatial wavelet transform in response to motion data for the motion compensated video frames. The adaptive parameters may be wavelet coefficients for the adaptive spatial wavelet transform. The decoder (300) performs the inverse operation and determines adaptive parameters for an inverse adaptive spatial wavelet transform based on received motion data. The use of motion information for adaptation of the adaptive spatial wavelet transforms allows improved video quality to data rate ratio without necessitating additional side data.

Description

ENCODING AND DECODING OF A VIDEO SIGNAL
Field of the invention
The invention relates to encoding and/or decoding of a video signal and in particular to encoding and/or decoding using a wavelet transform.
Background of the Invention
As the distribution of digital content becomes increasingly popular the importance of efficient encoding of content signals becomes increasingly significant. For example, a number of advanced techniques and standards for encoding and decoding of video signals have been developed including the well known MPEG (Motion Picture Expert Group) standards. Furthermore, research is continuously being undertaken to develop further improved video encoding techniques. In recent years, there has been significant research into the application of wavelet transforms to video signals. In wavelet video encoding, wavelets were initially used as a replacement for the two dimensional transform, and later were extended to the temporal axis, thus yielding full three dimensional (spatio-temporal) wavelet coding schemes. The wavelet transform in the temporal direction can be performed either without or with motion compensated prediction, the latter case leading to a higher compression ratio. The key techniques for efficient video coding are based on temporal and spatial decorrelation of video frames. The application of a wavelet transform in the temporal and spatial domains provides high coding efficiency and allows an embedded and scalable representation of a video bitstream.
In video coding, the redundancy of information between consecutive frames is often used to reduce the coding rate. Typically, adjacent frames in a video sequence comprise similar images and it is therefore possible to represent information of a frame using data from already encoded frames. Specifically, motion compensation techniques are used to remove temporal redundancy to produce so-called temporal frames. Motion models (such as well known block-based models) are used in this context as parameters that drive motion compensation.
It has been proposed to use adaptive wavelet transforms for encoding of images rather than to use a fixed set of basis functions. This can achieve high energy compaction and thus an efficient encoding. However, there are also a number of disadvantages associated with currently known adaptive wavelet transform encoding techniques.
Specifically, current techniques tend to focus on individual images thereby not fully exploiting information of temporal characteristics of a video signal. Furthermore, most known techniques using adaptive wavelet transforms are based on an explicit identification of different areas within an image, adapting the wavelet transform to these areas and including data identifying the areas in the encoding data. Although, this may allow an adaptive wavelet transform encoding, it adds additional side data which must be included in the data stream thereby increasing the data rate. Also, known techniques tend to lead to suboptimal encoding performance in many situations and in particular the adaptation of the wavelet transform to the characteristics of the images tend to be suboptimal .
Hence, an improved system for video encoding would be advantageous and in particular a system allowing increased flexibility, improved wavelet transform encoding/decoding, improved adaptation, facilitated implementation, reduced computational requirements, reduced data rate and/or improved performance would be advantageous.
Summary of the Invention
Accordingly, the Invention seeks to preferably mitigate, alleviate or eliminate one or more of the above mentioned disadvantages singly or in any combination.
According to a first aspect of the invention there is provided an encoder for encoding a video signal, the encoder comprising: means for generating motion compensated video frames for the video signal; wavelet means for applying an adaptive spatial wavelet transform to the motion compensated video frames to generate spatially transformed frames; means for generating encoded video data for the video signal in response to the spatially transformed frames; and adaptive means for determining adaptive parameters for the adaptive spatial wavelet transform in response to motion data for the motion compensated video frames.
The invention may allow an improved video encoding. An improved wavelet transformation may be used which more closely reflects the characteristics of the motion compensated video frames thereby allowing a more efficient encoding resulting in improved quality to data rate ratio. The invention may allow adaptation of an adaptive spatial wavelet transform at the encoder (and decoder) without resulting in the necessity for explicit data for the adaptation to be distributed with the encoded video data. Thus, a reduced data rate may be achieved. Also, the adaptation may be based on processing which has already been performed for other purposes resulting in a facilitated and reduced complexity implementation.
The invention may provide a better wavelet domain representation taking into consideration both spatial and temporal characteristics of the video signal. The improved wavelet domain representation may allow a more efficient compression of the video signal.
The adaptive parameters may comprise one or more adaptive parameters which are used to modify the operation of the adaptive spatial wavelet transform. The motion data may comprise' identifications of one or more motion compensated image areas in one or more of the motion compensated video frames. According to an optional feature of the invention, the adaptive parameters comprise wavelet transform coefficients .
This may allow a practical, easy to implement and/or efficient adaptation of the adaptive spatial wavelet transform.
According to an optional feature of the invention, the adaptive means is arranged to determine a first image area and a second image area of a first frame of the motion compensated video frames in response to the motion data and the adaptive parameters comprise wavelet transform coefficients for a boundary between the first image area and the' second image area.
This may allow improved video encoding and may in particular provide an improved quality to data rate ratio. For example, the wavelet transform may be adapted to provide a more efficient wavelet representation of higher frequency components around image area transitions which tend to have in increased concentration of such components. Specifically, the feature may allow a better encoding of detail around an image area transition.
According to an optional feature of the invention, the adaptive means is arranged to determine the first image area and the second image area as image areas having different motion characteristics.
This may provide efficient encoding and/or facilitated/ reduced complexity encoding. In particular, different motion characteristics tend to indicate that the image areas belong to different objects in the image and thus to the existence of a sharp image transition (e.g. an object edge) comprising high frequency components and high levels of detail.
According to an optional feature of the invention, the first image area is a motion compensated image area and the second image area is a non-motion compensated image area.
This may provide efficient encoding and/or facilitated/ reduced complexity encoding. In particular, as information identifying the motion compensated areas is typically included in the encoded video signal, the adaptation of the wavelet transform may be achieved without necessitating that additional adaptation data is included in the encoded video signal.
According to an optional feature of the invention, the first image area is an inter-coded image area and the second image area is an intra-coded image area.
An inter-coded image area is an image area encoded relative to an image area in another frame. An intra-coded image area is an image area which is not encoded relative to an image area in any other frame. The feature may provide efficient encoding and/or facilitated/ reduced complexity encoding. In particular, as information of which areas are inter- or intra-coded areas are typically included in the encoded video signal, the adaptation of the wavelet transform may be achieved without necessitating that additional adaptation data is included in the encoded video signal.
According to an optional feature of the invention, the adaptive parameters comprise wavelet transform coefficients for picture elements of the first frame adjacent to the boundary.
This may provide efficient encoding and/or facilitated/ reduced complexity encoding. It may allow a particularly efficient and/or facilitated adaptation of the adaptive spatial wavelet transform.
According to an optional feature of the invention, the adaptive parameters do not comprise wavelet transform coefficients for picture elements of the first frame not adjacent to the boundary.
This may provide efficient encoding and/or facilitated/ reduced complexity encoding. It may allow a particularly efficient and/or facilitated adaptation of the adaptive spatial wavelet transform.
According to an optional feature of the invention, the wavelet means is arranged to apply the adaptive spatial wavelet transform by applying an adaptive wavelet lifting operation.
This may allow a particularly efficient implementation which may easily be adapted in response to the motion data. According to an optional feature of the invention, the adaptive parameters comprise coefficient values for multiple levels of the adaptive wavelet lifting operation.
This may allow particularly efficient adaptation which is easy to implement. The boundary may be the same for each of the multiple levels.
According to an optional feature of the invention, the adaptive parameters comprise different coefficient values for different levels of the multiple levels.
This may allow an improved and more flexible adaptation and thus improved encoding.
According to an optional feature of the invention, the wavelet means is arranged to apply the adaptive spatial wavelet transform to transformed spatial subband data.
This may provide efficient encoding and/or facilitated/ reduced complexity encoding.
According to an optional feature of the invention, the encoder further comprises means for generating an encoded data signal for the video signal comprising the encoded video data and the motion data.
The encoded data signal is a self consistent signal comprising all the data which is required to decode the video signal. The encoded data signal may thus be transmitted, broadcast or distributed to one or more decoders . According to an optional feature of the invention, the encoder is arranged to not include data of the adaptive parameters in the encoded data signal.
This may provide for a more efficient encoded data signal and may in particular reduce the data rate for a given encoded video quality.
According to another aspect of the invention, there is provided a decoder for decoding an encoded video signal, the decoder comprising: means for receiving the encoded video signal; means for generating spatially transformed frames from the encoded video signal; wavelet means for applying an adaptive inverse spatial wavelet transform to the spatially transformed frames to generate motion compensated video frames; means for generating motion data for the motion compensated video frames; and adaptive means for determining adaptive parameters for the adaptive inverse spatial wavelet transform in response to the motion data.
It will be appreciated that the comments and advantages provided for the encoder tend to apply equally to the decoder.
According to an optional feature of the invention, the adaptive means is arranged to determine a first image area and a second image area of a first frame of the motion compensated video frames in response to the motion data and the adaptive parameters comprise wavelet transform coefficients for a boundary between the first image area and the second image area.
According to an optional feature of the invention, the adaptive means is arranged to determine the first and second image areas as image areas having different motion characteristics .
According to another aspect of the invention, there is provided a method of encoding a video signal comprising: generating motion compensated video frames for the video signal; applying an adaptive spatial wavelet transform to the motion compensated video frames to generate spatially transformed frames; generating encoded video data for the video signal in response to the spatially transformed frames; and determining adaptive parameters for the adaptive spatial wavelet transform in response to motion data for the motion compensated video frames.
According to another aspect of the invention, there is provided a method of decoding an encoded video signal, the method comprising: receiving the encoded video signal; generating spatially transformed frames from the encoded video signal; applying an adaptive inverse spatial wavelet transform to the spatially transformed frames to generate motion compensated video frames; generating motion data for the motion compensated video frames; and determining adaptive parameters for the adaptive inverse spatial wavelet transform in response to the motion data. According to another aspect of the invention, there is provided a computer program product enabling the carrying out of a method as described above.
These and other aspects, features and advantages of the invention will be apparent from and elucidated with reference to the embodiment (s) described hereinafter.
Brief Description of the Drawings
Embodiments of the invention will be described, by way of example only, with reference to the drawings, in which
FIG. 1 is an illustration of an encoder for encoding a video signal in accordance with some embodiments of the invention;
FIG. 2 illustrates an example of a lifting operation in accordance with some embodiments of the invention; and
FIG. 3 illustrates a decoder in accordance with some embodiments of the invention.
Detailed Description of Some Embodiments of the Invention
FIG. 1 is an illustration of an encoder for encoding a video signal in accordance with some embodiments of the invention. In the example, the encoder 100 comprises a video frame source 101 which generates video frames from a video signal to be encoded. It will be appreciated that the video frame source 101 can be a receiver for receiving a digitised video signal already comprising suitable video frames from an external source, or that it can for example be an analog video signal which is then digitised and structured in suitable frames as will be well known to the person skilled in the art.
The video frame source 101 is coupled to a motion estimation processor 103 which performs motion estimation and compensation on the video frames to generate motion compensated video frames. Various methods and algorithms for motion estimation and compensation of video frames is well known to the person skilled in the art and is for brevity not described in detail herein.
In the specific example, the motion estimation processor 103 simply determines the movement of motion estimation blocks of a fixed size in different video frames. For example, the block size can be 16 x 16 pixels (picture elements) . If the video frames comprise images with a moving object, a given block of one image can frequently be found in a subsequent image at an (often slightly) different position. An efficient video encoding can then be obtained by merely encoding the displacement of the original block rather than the actual image data of the block in the subsequent frame. This displacement is known as the motion vector. Improved quality can be obtained by determining the differential between the blocks of the different frames and encoding this residual or error signal. It will be appreciated that more advanced motion compensation is 5 possible using for example further degrees of freedom in the movement (such as rotation) or differently shaped motion compensation blocks (possibly adaptive) . Indeed, advanced two or three dimensional motion estimation models for different objects can be used.
10
There are many different approaches known for motion estimation. A simple approach comprises a brute force search of subsequent frame (s) to detect any blocks that closely correspond to a block of a previous frame in order
15 to allow motion compensation. Typically, motion estimation comprises evaluating a simple or complex object motion model to determine locations in subsequent frames.
The motion estimation processor 103 is coupled to a 20 wavelet processor 105. The wavelet processor 105 implements an adaptive spatial wavelet transform which is applied the compensated video frames to generate spatially transformed frames. As will be known to the person skilled in the art, wavelet processing may in many situations 25 provide a more efficient representation than conventional Fourier transforms.
In the example of FIG. 1, the motion estimation processor 103 provides temporal frames data to the wavelet processor 30 105, on which the adaptive spatial wavelet transform is applied. The output of the motion estimation processor 103 is the spatio-temporal subbands of the source video frames .
The wavelet processor 105 is coupled to an encoding processor 107 which is fed the motion compensated video frames (in the form of spatio-temporal subband data) . The encoding processor 107 is arranged to generate encoded video data for the transformed video frames. Specifically, the encoding processor 107 performs quantisation, scaling and encoding of the data values of the transformed video frames as will be known to the person skilled in the art. The incoming processor 107 can proceed to generate a complete encoded video signal which comprises all the information required to decode the video signal. Specifically the encoding processor 107 includes the motion data used by the motion estimation processor 103 for the motion compensation.
The encoding processor 107 is coupled to a transmit interface 109 which is fed the encoded video signal and which is arranged to transmit the signal to one and more decoders . It will be appreciated that the transmit interface 109 for example can be an interface to the Internet or e.g. a radio transceiver, such as cellular radio transceiver, arranged to transmit the encoded video signal over an air interface of a wireless communication system.
In the encoder 100 of FIG. 1, the applied wavelet transform is an adaptive wavelet transform. Specifically, some or all of the coefficients of the wavelet transform can be adapted to the specific characteristics of the individual video frames. In the encoder 100 the adaptation of the wavelet transform is performed in response to motion data generated by the motion estimation processor 103 and used for the motion compensation of the motion compensated video frames.
Accordingly, the encoder 100 comprises an adaptation processor 111 which is arranged to determine adaptive parameters for the adaptive spatial wavelet transform in response to motion data for the motion compensated video frames .
In the example, the adaptation processor 111 and the wavelet processor 105 is applied to one frame at a time. In the following, the processing of one frame will be described in more detail.
The adaptation processor 111 comprises an image area processor 113 which is arranged to identify different image areas in the video frame in response to the motion data. Specifically, the image area processor 113 divides the video frame into image areas for which motion compensation has been performed and image areas for which no motion compensation has been performed. The different image areas can easily be determined by the image area processor 113 as the motion data generated by the motion estimation processor 103 explicitly defines areas that are motion compensated. This information is already required by a decoder in order to apply the corresponding motion compensation at the decoding end. In the example, motion compensation is performed relative to video data in other video frames. Thus, a motion compensated area will be an inter-coded image area, meaning that it is encoded relative to video data in other frames. At the same time, non motion compensated image areas will be intra-coded image areas meaning that they will be encoded without reference to video data in other video frames.
It will be appreciated that in some embodiments there may not be a one-to-one correspondence between inter/intra image areas and motion compensated/non-motion compensated image areas. In such embodiments, the adaptation may be based only on inter/intra image areas, only on motion compensated/non-motion compensated image areas or on both inter/intra and motion compensated/non-motion compensated image areas .
In a specific example, the image area processor 113 thus divides the video frame into inter and intra areas. This creates a boundary between the intra areas and the inter areas and the wavelet transformation is adapted for these boundaries .
The adaptation processor 111 further comprises a parameter processor 115 which determines suitable adaptive parameters (may in some embodiments be a single adaptive parameter) for the wavelet transformation. Specifically, the parameter processor 115 receives information from the image area processor 113 identifying the inter and intra image areas (for example by an identification of the boundary between these) . The parameter processor 115 proceeds to determine adaptive parameters which are suited for the specific image area characteristics of the video frame .
In the example, the parameter processor 115 determines coefficients for the wavelet transformation taking into account the identified different areas of the video frame. The parameter processor 115 of FIG. 1 selects a fixed set of wavelet coefficients for wavelet operations on pixels within each image area. However, for pixels adjacent to the boundaries between intra and inter image areas, different coefficients are applied.
The adaptation coefficients are selected such that the wavelet coefficients for pixels adjacent to a boundary are better suited for sharp transitions or edges in the image. Thus, the coefficients are selected such that the prediction and correlation for pixels of different image areas is substantially reduced.
Thus, in the encoder 100, image areas having different motion compensation characteristics are transformed as independent (or at least less dependent) objects. By using motion information to determine the adaptation parameters for the wavelet transformation of a temporal frame, a spatially transformed frame which is much better suited for further coding is achieved. Furthermore, as the adaptation is based on information which is already- required for decoding, no additional data is required to be transmitted thus allowing a reduced data rate for a given encoded video quality. The application of an adaptive spatial wavelet transform results in the energy introduced into low pass subbands by wavelet filtering over edges between intra and inter areas to be reduced. Specifically, intra-coded areas in high pass frames possess different properties than the other areas of motion-compensated frames. Since the boundaries between intra and inter areas can typically be considered sharp edges, the non-adaptive spatial energy compaction fails to concentrate most of the energy into low pass frames, leaving high-amplitude "edge" coefficients in high-frequency subbands. The described approach thus allows the adaptation of the wavelet transform to be based on local properties of a high-pass temporal frame.
In the encoder 100, the wavelet processor 105 specifically implements the wavelet transform by a lifting algorithm. The adaptation processor 111 is arranged to reduce the effect of the linear prediction of the lifting implementation on the adaptation boundaries between the inter- and intra-coded image areas of a frame. As a consequence of reducing this prediction between intra and inter coded areas, intra classified areas are transformed as (more) independent objects.
FIG. 2 illustrates an example of a lifting operation in accordance with some embodiments of the invention. In the example, a 1-dimensional wavelet transform lifting algorithm is illustrated.
In the lifting operation of FIG. 2, a non-adaptive lifting algorithm (resulting in a fixed transform) is applied to pixels (or input coefficients) that are not next to any adaptation boundary (or adaptation border) . Thus, the wavelet transform within each area is the same whether the area is an inter area or an intra area and the same lifting coefficients are used. However, around the adaptation boundaries, the coefficients are changed to adapt the lifting algorithm. The detected adaptation boundaries are used for choosing the suitable lifting algorithm coefficients during the transform of each frame pixel .
As illustrated in FIG. 2, a series of lifting steps (prediction and update) are applied to the pixels using adapted lifting coefficients or weights between pixels that belong to different image areas. Specifically, the absolute values of the coefficients are reduced between pixels of different image areas relative to coefficients applied to pixels within the same image area. In some embodiments, for an intra/inter transition, the lifting step that is performed on a pixel in one area, the neighbouring pixel from a different area is taken with the zero weight, while the other pixel from the same area is taken with the weight of 2. As a result of adaptation, the coupling between pixels from areas of different types is reduced or completely cancelled (illustrated by dashed lines in FIG. 2) .
As illustrated in FIG. 2, the wavelet processor 105 uses the same adaptation boundary for all levels (or steps) of the lifting operation. Similarly, the same coefficients may be used at different levels of the lifting operation, or in some embodiments the coefficients may be varied between different levels. The wavelet processor 105 of the encoder 100 of FIG. 1 applies the adaptive transform to the original high-pass frame and to each low-pass subband produced by one level of spatial transform.
Furthermore, for colour video signals, the adaptive spatial wavelet transform can be applied separately to each colour component (e.g. Y, U, V), as typically the same motion information is used for all components.
It will be appreciated that the adaptation of the adaptive spatial wavelet transform of the encoder 100 is based on motion data which is already included in the encoded video signal in order to allow a decoder to perform correct decoding of the motion compensated data. Accordingly, the encoding processor 107 includes only the motion data in the encoded video signal and does not include any other side information related to the adaptation of the adaptive spatial wavelet transform.
FIG. 3 illustrates a decoder 300 in accordance with some embodiments of the invention. The decoder 300 is operable to decode the signal received from the encoder 100 of FIG. 1. Specifically, the decoder 300 is arranged perform the ^inverse operations of the encoder 100 and in particular it is arranged perform an inverse adaptive spatial wavelet transform in response to motion data.
The decoder 300 comprises a video data receiver 301 which receives the encoded video signal from the encoder 100. The video data receiver 301 is coupled to a motion data extractor 303 which extracts motion data from the encoded video signal. The motion data extractor 303 furthermore generates spatially transformed frames from the encoded video signal.
The motion data extractor 303 is fed to an inverse wavelet processor 305 which applies an adaptive inverse spatial wavelet transform to the spatially transformed frames to generate motion compensated video frames.
The motion compensated video frames are fed to a decoding processor 307 coupled to the inverse wavelet processor 305. The decoding processor 307 is arranged to generate a decoded video signal from the motion compensated video frames.
The decoder 300 furthermore comprises a decoder adaptation processor 309 which is coupled to the motion data extractor 303 and the inverse wavelet processor 305. The decoder adaptation processor 309 is arranged to determine adaptive parameters for the adaptive inverse spatial wavelet transform in response to the motion data.
Specifically, the decoder adaptation processor 309 comprises a decode image area processor 311 which divides a given video frame into motion compensated and non-motion compensated image areas (corresponding to inter/intra image areas) .
The decoder adaptation processor 309 furthermore comprises a decode parameter processor 313 which determines the adaptive parameters in response to the identified inter/intra image areas. Specifically, the decode parameter processor 313 operates similarly to the parameter processor 115 of the encoder 100 and determines adaptive wavelet coefficients for pixels adjacent a boundary between the intra/intra image areas. These coefficients are fed to the inverse wavelet processor 305 and are used for the inverse adaptive spatial wavelet transform.
Thus, the operation of the adaptive spatial wavelet transform of the encoder 100 is effectively inversed using only motion data already included in the encoded signal for other purposes.
Hence, at the decoder 300, motion information drives the inverse spatial transform in the same way as at the encoder 100. The temporal subbands are synthesised using the same adaptation boundaries that define the applied adaptive lifting. Therefore invertibility of the adaptive transform is ensured. Moreover, the proposed adaptive transform can be used in scalable video coding scenarios for all combinations of spatial, temporal and quality scalabilities .
It will be appreciated that the above description for clarity has described embodiments of the invention with reference to different functional units and processors. However, it will be apparent that any suitable distribution of functionality between different functional units or processors may be used without detracting from the invention. For example, functionality illustrated to be performed by separate processors or controllers may be performed by the same processor or controllers. Hence, references to specific functional units are only to be seen as references to suitable means for providing the described functionality rather than indicative of a strict logical or physical structure or organization.
The invention can be implemented in any suitable form including hardware, software, firmware or any combination of these. The invention may optionally be implemented at least partly as computer software running on one or more data processors and/or digital signal processors. The elements and components of an embodiment of the invention may be physically, functionally and logically implemented in any suitable way. Indeed the functionality may be implemented in a single unit, in a plurality of units or as part of other functional units. As such, the invention may be implemented in a single unit or may be physically and functionally distributed between different units and processors .
Although the present invention has been described in connection with some embodiments, it is not intended to be limited to the specific form set forth herein. Rather, the scope of the present invention is limited only by the accompanying claims. Additionally, although a feature may appear to be described in connection with particular embodiments, one skilled in the art would recognize that various features of the described embodiments may be combined in accordance with the invention. In the claims, the term comprising does not exclude the presence of other elements or steps. Furthermore, although individually listed, a plurality of means, elements or method steps may be implemented by e.g. a single unit or processor. Additionally, although individual features may be included in different claims, these may possibly be advantageously combined, and the inclusion in different claims does not imply that a combination of features is not feasible and/or advantageous. Also the inclusion of a feature in one category of claims does not imply a limitation to this category but rather indicates that the feature is equally applicable to other claim categories as appropriate. Furthermore, the order of features in the claims does not imply any specific order in which the features must be worked and in particular the order of individual steps in a method claim does not imply that the steps must be performed in this order. Rather, the steps may be performed in any suitable order.

Claims

1. An encoder for encoding a video signal, the encoder comprising: means for generating motion compensated video frames for the video signal; wavelet means for applying an adaptive spatial wavelet transform to the motion compensated video frames to generate spatially transformed frames; means for generating encoded video data for the video signal in response to the spatially transformed frames; and adaptive means for determining adaptive parameters for the adaptive spatial wavelet transform in response to motion data for the motion compensated video frames.
2. The encoder of claim 1 wherein the adaptive parameters comprise wavelet transform coefficients.
3. The encoder of claim 2 wherein the adaptive means is arranged to determine a first image area and a second image area of a first frame of the motion compensated video frames in response to the motion data and the adaptive parameters comprise wavelet transform coefficients for a boundary between the first image area and the second image area.
4. The encoder of claim 3 wherein the adaptive means is arranged to determine the first image area and the second image area as image areas having different motion characteristics .
5. The encoder of claim 4 wherein the first image area is a motion compensated image area and the second image area is a non-motion compensated image area.
6. The encoder of the claims 3 wherein the first image area is an inter-coded image area and the second image area is an intra-coded image area.
7. The encoder of claims 3 wherein the adaptive parameters comprise wavelet transform coefficients for picture elements of the first frame adjacent to the boundary.
8. The encoder of claim 3 wherein the wavelet means is arranged to apply the adaptive spatial wavelet transform by aPPlying an adaptive wavelet lifting operation.
9. The encoder of claim 8 wherein the adaptive parameters comprise coefficient values for multiple levels of the adaptive wavelet lifting operation.
10. The encoder of claim 9 wherein the adaptive parameters comprise different coefficient values for different levels of the multiple levels.
11. A decoder for decoding an encoded video signal, the decoder comprising: means for receiving the encoded video signal; means for generating spatially transformed frames from the encoded video signal; wavelet means for applying an adaptive inverse spatial wavelet transform to the spatially transformed frames to generate motion compensated video frames; means for generating motion data for the motion compensated video frames; and adaptive means for determining adaptive parameters for the adaptive inverse spatial wavelet transform in response to the motion data.
12. The decoder of claim 11 wherein the adaptive means is arranged to determine a first image area and a second image area of a first frame of the motion compensated video frames in response to the motion data and the adaptive parameters comprise wavelet transform coefficients for a boundary between the first image area and the second image area.
PCT/US2006/038624 2005-10-10 2006-10-03 Encoding and decoding of a video signal WO2007044333A2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP06816113A EP1938598A4 (en) 2005-10-10 2006-10-03 Encoding and decoding of a video signal

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
GB0520557A GB2431064A (en) 2005-10-10 2005-10-10 Encoding a video signal using a motion-dependent adaptive spatial wavelet transform
GB0520557.0 2005-10-10

Publications (2)

Publication Number Publication Date
WO2007044333A2 true WO2007044333A2 (en) 2007-04-19
WO2007044333A3 WO2007044333A3 (en) 2007-12-21

Family

ID=35430100

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2006/038624 WO2007044333A2 (en) 2005-10-10 2006-10-03 Encoding and decoding of a video signal

Country Status (4)

Country Link
EP (1) EP1938598A4 (en)
CN (1) CN101322404A (en)
GB (1) GB2431064A (en)
WO (1) WO2007044333A2 (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112616058A (en) * 2019-10-03 2021-04-06 腾讯美国有限责任公司 Video encoding or decoding method, apparatus, computer device and storage medium

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5495292A (en) * 1993-09-03 1996-02-27 Gte Laboratories Incorporated Inter-frame wavelet transform coder for color video compression
US6084908A (en) * 1995-10-25 2000-07-04 Sarnoff Corporation Apparatus and method for quadtree based variable block size motion estimation
JP4254017B2 (en) * 2000-03-10 2009-04-15 ソニー株式会社 Image coding apparatus and method
US7023922B1 (en) * 2000-06-21 2006-04-04 Microsoft Corporation Video coding system and method using 3-D discrete wavelet transform and entropy coding with motion information
US7991051B2 (en) * 2003-11-21 2011-08-02 Electronics And Telecommunications Research Institute Interframe wavelet coding apparatus and method capable of adjusting computational complexity
US7580461B2 (en) * 2004-02-27 2009-08-25 Microsoft Corporation Barbell lifting for wavelet coding

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
RUSERT ET AL.: "Signal Processing, Image Communication", vol. 19, 1 August 2004, ELSEVIER SCIENCE PUBLISHERS, article "Enhanced Interframe Wavelet Video Coding Considering the Interrelation of Spatio-Temporal Transform and Motion Compensation"
See also references of EP1938598A4

Also Published As

Publication number Publication date
GB0520557D0 (en) 2005-11-16
CN101322404A (en) 2008-12-10
EP1938598A2 (en) 2008-07-02
GB2431064A (en) 2007-04-11
EP1938598A4 (en) 2011-06-01
WO2007044333A3 (en) 2007-12-21

Similar Documents

Publication Publication Date Title
KR100802203B1 (en) Bitstream controlled post-processing filtering
US7120197B2 (en) Motion compensation loop with filtering
CN100446560C (en) Decoder apparatus and method for smoothing artifacts created during error concealment
EP1404136B1 (en) Image encoder, image decoder, image encoding method, and image decoding method
KR100957322B1 (en) Encoder assisted frame rate up conversion using various motion models
CN100364338C (en) Method of and apparatus for estimating noise of input image, and method and recording media of eliminating noise
US20150350667A1 (en) Methods and systems for motion vector derivation at a video decoder
KR100578432B1 (en) Signaling for fading compensation
KR100578433B1 (en) Fading estimation/compensation
US8290058B2 (en) Method and device for coding and decoding
US20040076231A1 (en) Method and apparatus for efficient video processing
WO2012033971A1 (en) Recovering a pruned version of a picture in a video sequence for example - based data pruning using intra- frame patch similarity
CN110199521A (en) Low complex degree hybrid domain for damaging Video coding cooperates with in-loop filter
EP2105030A2 (en) Spatial sparsity induced temporal prediction for video compression
KR20030086905A (en) Parameterization for fading compensation
WO2014117048A1 (en) Spatially adaptive video coding
JPH06292173A (en) Video signal coding/decoding method
WO2006046834A1 (en) Method for coding and decoding moving image
JP5746193B2 (en) Efficient adaptive filtering method and apparatus for video encoding and decoding
JP4943586B2 (en) Method and apparatus for efficient video processing
JPH01228384A (en) Moving image coding system using area division
EP1938598A2 (en) Encoding and decoding of a video signal
KR100801155B1 (en) A spatial error concealment method with low complexity in h. 264/avc video coding
KR100801532B1 (en) Temporal error concealment based on optical flow in h. 264/avc video coding
CN117941352A (en) Inter prediction method, encoder, decoder, and storage medium

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 200680045459.7

Country of ref document: CN

121 Ep: the epo has been informed by wipo that ep was designated in this application
WWE Wipo information: entry into national phase

Ref document number: 2006816113

Country of ref document: EP

NENP Non-entry into the national phase

Ref country code: DE