CN101371571B - Processing multiview video - Google Patents

Abstract

Decoding a multiview video signal comprises: receiving a bitstream comprising encodings of multiple views of the multiview video signal, each view comprising multiple pictures segmented into multiple segments; obtaining an offset value for illumination compensation of a first segment with respect to a reference picture, wherein the offset value is predicted using an offset value for illumination compensation of at least one neighboring segment determined based on characteristics associated with the neighboring segment; and decoding the bitstream using illumination compensation for the first segment including forming a sum that includes a predictor for pixels of the first segment obtained from the reference picture, a residual for pixels of the first segment, and a corresponding offset value for illumination compensation.

Description

Process multi-view video

Technical field

The present invention relates to process multi-view video.

Background technology

Multi-view video coding (MVC) relates to the video sequence (for example, image or " image " sequence) that compression is obtained by corresponding camera usually.Video sequence or " view " can be according to standard mpeg encodeds for example.Image in video sequence can represent the field of full frame of video or frame of video.Band (slice) is the absolute coding part of image, and band comprises some or all macro blocks in image, and macro block comprises the piece of pictorial element (perhaps " pixel ").

Video sequence can be encoded into the multi-view video sequence according to codec technology H.264/AVC, and a lot of developer adapts to the multi-view video sequence to revision research of standard.

Three shelves (profile) of specific function have been stipulated to be used for supporting in current H.264 standard.Term " shelves " indication is used for the standardization of the skill element of encoding and decoding of video algorithm.In other words, shelves are one group of skill elements stipulating for the bit stream of decoding compressed sequence, and can be considered to substandard.Above-mentioned three shelves are baseline shelves, main shelves and expansion shelves.H.264 defining the various functions that are used for encoder in standard, thereby encoder can be compatible with baseline shelves, main shelves and expansion shelves respectively.

According to for the treatment of the Video coding layer (VCL) of moving image encoding (being sequential coding) and with can send/network abstraction layer (NAL) that the subsystem of memory encoding information is associated, be configured to the H.264/AVC bit stream of standard.The output data of cataloged procedure are the VCL data, and are mapped to the NAL unit before it is sent out or stores.Each NAL unit comprises the raw byte sequence payload (RBSP) corresponding to compressed video data or header information.

The NAL unit comprises NAL head and RBSP.The NAL head comprise flag information (for example, nal_ref_idc) and the sign (ID) information (for example, nal_unit_type).Whether flag information " nal_ref_idc " indication exists as the band of NAL elements reference image.The shelves type of id information " nal_unit_type " indication NAL unit.Initial data after the RBSP store compressed.RBSP end potential energy enough is added to the decline of RBSP, thereby the length of RBSP can be represented by the multiple of 8 bits.

There is various NAL unit, for example, instantaneous decoding refresh (IDR) image, sequence parameter set (SPS), picture parameter set (PPS) and supplemental enhancement information (SEI) etc.

This standard is used various shelves and level objective definition product usually, thereby can be with suitable cost realize target product.Decoder satisfies the predetermined constraints at the shelves of correspondence and grade place.

Shelves and level can be indicated function or the parameter of decoder, thereby which compressed image their indication decoders can process.Indicate which in a plurality of shelves can utilize a grade id information to identify corresponding to the customizing messages of bit stream.Shelves id informations " profile_idc " are provided for identifying the sign of the shelves that are associated with bit stream.H.264/AVC standard comprises three shelves identifiers (IDs).If shelves id informations " profile_idc " are set as " 66 ", bit stream is based on the baseline shelves.If shelves id informations " profile_idc " are set as " 77 ", bit stream is based on main shelves.If shelves id informations " profile_idc " are set as " 88 ", bit stream extension-based shelves.Above-mentioned " profile_idc " information can for example be comprised in SPS (sequence parameter set).

Summary of the invention

In one aspect, usually, a kind of method for decoding multi-view video signal comprises: the bit stream that receives the coding of a plurality of views that comprise the multi-view video signal, each view comprises and (for example is segmented into a plurality of sections, the image block section of single or macro block for example, the perhaps section of image band for example) a plurality of images; From the flag information that part be associated of bitstream extraction with the multi-view video signal, this flag information indicates whether and can carry out luminance compensation to the section in the described part of multi-view video signal; And for can carry out the part of luminance compensation according to the flag information that extracts, the value that is associated with section this part from bitstream extraction, and determine whether to carry out luminance compensation to this section according to the value of described extraction.

Aspect of the present invention can comprise one or more in following feature.

Section comprises image block.

The method also comprises, for carrying out first that the value of luminance compensation is associated with indication, and use the deviant that is used at least one adjacent block that is adjacent to first is carried out luminance compensation, obtains the predicted value of carrying out luminance compensation for to first.

By formation comprise for the predicted value of the luminance compensation of adjacent block and residual value and, obtain to be used for the deviant of the luminance compensation of adjacent block.

Obtain to comprise for the predicted value of carrying out luminance compensation to first with the deviant that is used for being adjacent at least one adjacent block execution luminance compensation of first: select this at least one adjacent block at adjacent block according to predesigned order.

Select this at least one adjacent block to comprise according to predesigned order: to determine according to the order of following one or more diagonal angle neighbor after one or more vertical or horizontal neighbor whether adjacent block satisfies one or more condition.

Flag information makes it possible to the one or more luminance compensations that carry out to the sequence that comprises this piece, view, a group of image, image and band.

This flag information makes it possible to the band that contains described is carried out luminance compensation.

Institute's extraction value comprise be used to contain described macro block flag information or be used for the flag information of described.

Institute's extraction value comprises be used to the flag information that contains the macro block of described.

On the other hand, usually, a kind of method for decoding multi-view video signal comprises: receive the bit stream of the coding of a plurality of views that comprise the multi-view video signal, each view comprises and is segmented into a plurality of images of a plurality of sections; And use the deviant that is used at least one adjacent segment that is adjacent to first paragraph is carried out luminance compensation, obtain to be used for first paragraph is carried out the predicted value of luminance compensation, be included in adjacent segment and select this at least one adjacent segment according to predesigned order.

Aspect of the present invention can comprise one or more in following feature.

First paragraph and at least one adjacent segment comprise image block.

By formation comprise for the predicted value of the luminance compensation of adjacent block and residual value and, obtain to be used for the deviant of the luminance compensation of adjacent block.

Select this at least one adjacent block to comprise according to predesigned order: to determine according to the order of following one or more diagonal angle neighbor after one or more vertical or horizontal neighbor whether adjacent block satisfies one or more condition.

Select this at least one adjacent block to comprise in the following order determining whether satisfy one or more condition for adjacent block according to predesigned order: left adjacent block, then upper adjacent block, be the upper right adjacent block subsequently subsequently, is the upper left adjacent block subsequently.

Select this at least one adjacent block to comprise in the following order determining whether satisfy one or more condition for adjacent block according to predesigned order: upper adjacent block is left adjacent block subsequently, is the upper right adjacent block subsequently, is the upper left adjacent block subsequently.

Determine that whether satisfying one or more condition for adjacent block comprises: from the value that bitstream extraction is associated with adjacent block, this value indicates whether to carry out the luminance compensation of adjacent block.

Institute's extraction value comprise be used to contain described macro block flag information or be used for the flag information of described.

Obtaining described predicted value comprises: the deviant that definite use is used for the luminance compensation of single adjacent block is still used a plurality of deviants for the luminance compensation of corresponding adjacent block.

The method also comprises, when using a plurality of deviant, obtains for the predicted value of carrying out the luminance compensation of first by making up a plurality of deviants.

Make up these a plurality of deviants and comprise mean value or the intermediate value that adopts deviant.

On the other hand, usually, a kind of method for decoding multi-view video signal comprises: receive the bit stream of the coding of a plurality of views that comprise the multi-view video signal, each view comprises and is segmented into a plurality of images of a plurality of sections; Obtain to be used for respect to reference picture, first paragraph being carried out the deviant of luminance compensation, wherein, use is used for predicting described deviant based on the deviant of the luminance compensation of determined at least one adjacent segment of characteristic that is associated with adjacent segment; And with the described bit stream of decoding for the luminance compensation of first paragraph, comprise and form the summation comprise following value: the predicted value that is used for the pixel of the first paragraph that obtains from reference picture, the residual value that is used for the pixel of first paragraph, and the deviant that is used for the correspondence of luminance compensation.

Aspect of the present invention can comprise one or more in following feature.

First paragraph and at least one adjacent segment comprise image block.

By formation comprise for the predicted value of the luminance compensation of adjacent block and residual value and, obtain to be used for the deviant of the luminance compensation of adjacent block.

The method also comprises: based on whether satisfying one or more condition according to the order of following one or more diagonal angle neighbor after one or more vertical or horizontal neighbor for adjacent block, select at least one adjacent block.

Select at least one adjacent block to comprise in the following order determining whether satisfy one or more condition for adjacent block: left adjacent block is upper adjacent block subsequently, is the upper right adjacent block subsequently, is the upper left adjacent block subsequently.

Determine that whether satisfying one or more condition for adjacent block comprises: from the value that bitstream extraction is associated with adjacent block, this value indicates whether to carry out the luminance compensation of adjacent block.

Institute's extraction value comprise be used to contain described macro block flag information or be used for the flag information of described.

Select at least one adjacent block to comprise: to determine to use the deviant of the luminance compensation that is used for single adjacent block, or use a plurality of deviants of the luminance compensation that is used for corresponding adjacent block.

The method also comprises, when using a plurality of deviant, obtains for the predicted value of carrying out the luminance compensation of first by making up a plurality of deviants.

Make up these a plurality of deviants and comprise mean value or the intermediate value that adopts deviant.

On the other hand, usually, a kind of method for decoding multi-view video signal comprises: receive the bit stream of the coding of a plurality of views that comprise the multi-view video signal, each view comprises and is segmented into a plurality of images of a plurality of sections; Obtain to be used for respect to reference picture, first paragraph being carried out the predicted value of luminance compensation; The deviant that is identified for the luminance compensation of first paragraph comprise form comprise predicted value and residual value and; And with the luminance compensation that is used for first paragraph decode described bit stream comprise form comprise following value and: the predicted value that is used for the pixel of the first paragraph that obtains from reference picture, the residual value that is used for the pixel of first paragraph, and the deviant that is used for the correspondence of luminance compensation.

Aspect of the present invention can comprise one or more in following feature.

Described section comprises image block.

Use luminance compensation to comprise for first paragraph: by formation comprise for the predicted value of the luminance compensation of adjacent block and residual value and, obtain to be used for the deviant of the luminance compensation of adjacent block.

The method also comprises: based on whether satisfying one or more condition according to the order of following one or more diagonal angle neighbor after one or more vertical or horizontal neighbor for adjacent block, select at least one adjacent block.

Select at least one adjacent block to comprise in the following order determining whether satisfy one or more condition for adjacent block: left adjacent block is upper adjacent block subsequently, is the upper right adjacent block subsequently, is the upper left adjacent block subsequently.

Determine that whether satisfying one or more condition for adjacent block comprises: from the value that bitstream extraction is associated with adjacent block, this value indicates whether to carry out the luminance compensation of adjacent block.

Institute's extraction value comprise be used to contain described macro block flag information or be used for the flag information of described.

Select at least one adjacent block to comprise: to determine to use the deviant of the luminance compensation that is used for single adjacent block, or use a plurality of deviants of the luminance compensation that is used for corresponding adjacent block.

The method also comprises, in the time will using a plurality of deviant, obtains for the predicted value of carrying out the luminance compensation of first by making up a plurality of deviants.

Make up these a plurality of deviants and comprise mean value or the intermediate value that adopts deviant.

On the other hand, usually, a kind of method for decoding multi-view video signal comprises: receive the bit stream of the coding of a plurality of views that comprise the multi-view video signal, each view comprises and is segmented into a plurality of images of a plurality of sections; And whether identical with the reference picture that is associated with adjacent segment according to the reference picture that is associated with first paragraph, use the deviant that is used at least one adjacent segment that is adjacent to first paragraph is carried out luminance compensation, obtain with respect to reference picture, first paragraph to be carried out the predicted value of luminance compensation.

Aspect of the present invention can comprise one or more in following feature.

Described section comprises image block.

Use luminance compensation to comprise for first paragraph: by formation comprise for the predicted value of the luminance compensation of adjacent block and residual value and, obtain to be used for the deviant of the luminance compensation of adjacent block.

The method also comprises: based on whether satisfying one or more condition according to the order of following one or more diagonal angle neighbor after one or more vertical or horizontal neighbor for adjacent block, select at least one adjacent block.

Select at least one adjacent block to comprise and judge in the following order for adjacent block whether satisfy one or more condition: left adjacent block is upper adjacent block subsequently, is the upper right adjacent block subsequently, is the upper left adjacent block subsequently.

Determine that whether satisfying one or more condition for adjacent block comprises: from the value that bitstream extraction is associated with adjacent block, this value indicates whether to carry out the luminance compensation of adjacent block.

Institute's extraction value comprise be used to contain described macro block flag information or be used for the flag information of described.

Select at least one adjacent block to comprise: to determine to use the deviant of the luminance compensation that is used for single adjacent block, or use a plurality of deviants of the luminance compensation that is used for corresponding adjacent block.

The method also comprises, in the time will using a plurality of deviant, obtains for the predicted value of carrying out the luminance compensation of first by making up a plurality of deviants.

Make up these a plurality of deviants and comprise mean value or the intermediate value that adopts deviant.

On the other hand, usually, for each corresponding coding/decoding method, a kind of method for encoded video signal comprises: generation can utilize this kind corresponding decoding method to be decoded into the bit stream of vision signal.

For example, on the other hand, usually, a kind of method for coded bit stream comprises: form the bit stream of the coding of a plurality of views that comprise the multi-view video signal, each view comprises and is segmented into a plurality of images of a plurality of sections; The flag information that is associated with the part of multi-view video signal is provided in bit stream, and this flag information indicates whether to make it possible to the section in the described part of multi-view video signal is carried out luminance compensation; And for according to the extraction flag information in the part that can carry out luminance compensation, provide in bit stream with this part in the value that is associated of section, and be worth to determine whether according to described extraction will be to this section execution luminance compensation.

On the other hand, usually, a kind of method for coded bit stream comprises: form the bit stream of the coding of a plurality of views that comprise the multi-view video signal, each view comprises and is segmented into a plurality of images of a plurality of sections; And use the deviant that is used at least one adjacent segment that is adjacent to first paragraph is carried out luminance compensation, and be provided for first paragraph is carried out the predicted value of luminance compensation, be included in adjacent segment and select this at least one adjacent segment according to predesigned order.

On the other hand, usually, a kind of method for coded bit stream comprises: form the bit stream of the coding of a plurality of views that comprise the multi-view video signal, each view comprises and is segmented into a plurality of images of a plurality of sections; Be provided for respect to reference picture, first paragraph being carried out the deviant of luminance compensation, wherein, can predict described deviant with the deviant that is used for based on the luminance compensation of determined at least one adjacent segment of characteristic that is associated with adjacent segment; And based on the information that comprises following value and be provided for first paragraph is carried out luminance compensation: be used for the predicted value of the pixel of the first paragraph that obtains from reference picture, be used for the residual value of the pixel of first paragraph, and the deviant that is used for the correspondence of luminance compensation.

On the other hand, usually, a kind of method for coded bit stream comprises: form the bit stream of the coding of a plurality of views that comprise the multi-view video signal, each view comprises and is segmented into a plurality of images of a plurality of sections; Be provided for respect to reference picture, first paragraph being carried out the predicted value of luminance compensation; Based on comprise predicted value and residual value and, be provided for first paragraph is carried out the deviant of luminance compensation; And based on the information that comprises following value and be provided for first paragraph is carried out luminance compensation: be used for the predicted value of the pixel of the first paragraph that obtains from reference picture, be used for the residual value of the pixel of first paragraph, and the deviant that is used for the correspondence of luminance compensation.

On the other hand, usually, a kind of method for coded bit stream comprises: form the bit stream of the coding of a plurality of views that comprise the multi-view video signal, each view comprises and is segmented into a plurality of images of a plurality of sections; And whether identical with the reference picture that is associated with adjacent segment according to the reference picture that is associated with first paragraph, the deviant of luminance compensation is carried out in use at least one adjacent segment that is adjacent to first paragraph, be provided for respect to reference picture, first paragraph being carried out the predicted value of luminance compensation.

On the other hand, usually, for each corresponding coding/decoding method, the computer program of storing on computer-readable medium comprises be used to causing computer to carry out the instruction of this corresponding coding/decoding method.

On the other hand, usually, for each corresponding coding/decoding method, the view data of implementing on machine-readable information carrier can utilize this corresponding coding/decoding method to be decoded into vision signal.

On the other hand, usually, for each corresponding coding/decoding method, decoder comprises for the device of carrying out corresponding coding/decoding method.

On the other hand, usually, for each corresponding coding/decoding method, encoder comprises for generation of utilizing corresponding coding/decoding method to be decoded into the device of the bit stream of vision signal.

On the other hand, usually, a kind of method for encoded video sequence comprises: the average pixel value that a) obtains at least one piece from the reference block of the adjacent block of current block and another view; B) from the average pixel value of resulting at least one piece, the prediction average pixel value of derivation current block; And c) obtain difference between the average pixel value of the prediction average pixel value of current block and current block.

On the other hand, usually, provide a kind of method for decoding video sequence, comprising: 1) obtaining can be from the difference of the average pixel value of vision signal reconstruct current block; M) from the prediction average pixel value of the reference block derivation current block of another view; And n) come the average pixel value of reconstruct current block based on prediction average pixel value and difference.

Aspect another, usually, provide a kind of equipment for encoded video sequence, comprising: the average pixel value of average pixel value that is used for obtaining the reference block of the adjacent block of current block and another view obtains the unit; Be used for from the derive average pixel value predicting unit of prediction average pixel value of current block of the average pixel value that obtains; And be used for obtaining in the prediction average pixel value of current block and the differential coding unit of the difference between average pixel value.

Aspect another, usually, provide a kind of equipment for decoding video sequence, comprising: the difference decoding unit that is used for obtaining from the bit stream that receives difference; Be used for from the average pixel value predicting unit of the prediction average pixel value of the reference block derivation current block of another view; With the luminance compensation unit that is used for coming based on prediction average pixel value and difference the average pixel value of reconstruct current block.

Aspect another, usually, a kind of method for decoded video signal comprises: use the deviant of at least one adjacent block that is adjacent to current block, obtain the predicted value for the luminance compensation of carrying out current block; And the deviant of using this predicted value reconstruct current block, wherein, the reference key that whether equals adjacent block by the reference key of determining current block is determined described predicted value.

Aspect another, usually, provide a kind of method for decoded video signal, comprising: reconstruct current block deviant, this current block deviant is indicated the difference between the average pixel value of the average pixel value of current block and at least one reference block; And if utilize two or more reference blocks to carry out predictive coding to current block, use this deviant to obtain respectively the deviant of the reference block of current block.

Aspect another, usually, a kind of method for decoded video signal comprises: the flag information that obtains to indicate whether current block is carried out luminance compensation; And if carry out luminance compensation according to this flag information, the deviant of the difference of reconstruct indication between the average pixel value of the average pixel value of current block and reference block.

Aspect another, usually, provide a kind of method for decoded video signal, comprising: the flag information that a) obtains to be used for allowing a specific order of vision signal is carried out luminance compensation; And b) a specific order that utilizes flag information to carry out the vision signal of luminance compensation is decoded, wherein, a specific order of this vision signal is corresponding to any one in sequence-level, view level, GOP (image sets) level, image level, slice level, macro-block level and piece level.

Aspect another, usually, provide a kind of method for encoded video signal, comprising: the deviant that obtains the current block of the difference of indication between the average pixel value of current block and reference block; And the reference block that uses this deviant search and current block Optimum Matching; And obtain motion vector from the reference block of coupling, and this motion vector of encoding.

Aspect of the present invention can have one or more following advantage.

The method or the equipment that are used for coding/decoding video sequence are predicted the mean value of the current block that will be encoded based on peripheral piece, and are sent in the difference between current block and peripheral piece, reduce thus the amount of information that will be sent out for luminance compensation.

The method is effectively carried out the luminance compensation of the multi-view video sequence that needs mass data, increases thus code rate.The method uses the correlation between piece or view to realize a kind of efficient coding/decode system.

Caught the view sequence of multi-view video data by different cameras, thereby because there are luminance difference in camera inside or external factor.In order to address this problem, the method is predicted the deviant of current block with the information of adjacent block, only be sent in the residual value between current block and adjacent block, thereby it can reduce the amount of information that is sent out for luminance compensation.In the situation of the deviant of predicting current block, the method judges whether the reference key of current block equals the reference key of adjacent block, and result has realized Accurate Prediction.

The method prediction indicates whether current block is carried out the flag information of luminance compensation, and only is sent in the residual value between flag information, reduces thus the amount of information that is sent out.The method judges whether the reference key of current block equals the reference key of adjacent block, and result has realized Accurate Prediction.The method is used the correlation between piece or view, and result has realized the efficient coding process.

Caught the view sequence of multi-view video data by different cameras, thereby because there are luminance difference in camera inside or external factor.In order to address the above problem, the method is predicted the deviant of current block with the information of adjacent block, only be sent in the residual value between current block and adjacent block, thereby it can reduce the amount of information that is sent out be used to carrying out luminance compensation.The method prediction indicates whether current block is carried out the flag information of luminance compensation, and only sends residual value, makes thus the amount of information of transmission minimum.

If carry out the predictive coding process with two or more reference blocks, the method uses at least a method to adopt deviant and flag information, and result has realized the efficient coding process.Distribute each zone that indicates whether vision signal to carry out the flag bit of luminance compensation, thereby can effectively use the luminance compensation technology.The method assesses the cost by the luminance difference that is reflected in motion estimation process, and result has realized correct predictive coding.

Require to know further feature and advantage from following explanation and accessory rights.

Brief Description Of Drawings

Fig. 1 is exemplary decoding device.

Fig. 2 is the flow chart that illustrates for the method for encoded video sequence.

Fig. 3 is the block diagram that illustrates for from the process of the prediction average pixel value of the reference block derivation current block of other view.

Fig. 4 is the detailed diagram that illustrates for from the process of the prediction average pixel value of the reference block derivation current block of other view.

Fig. 5 is the diagram that 16 * 16 macro blocks are shown.

Fig. 6 A-6B is the diagram that 16 * 8 macro blocks are shown.

Fig. 7 A-7B is the diagram that 8 * 16 macro blocks are shown.

Fig. 8 A-8B is the diagram of indication 8 * 8 macro blocks.

Fig. 9 is the diagram that illustrates for the process of the deviant that obtains current block.

Figure 10 is the flow chart that illustrates for the process of the luminance compensation of carrying out current block.

Figure 11 illustrates the flow chart that obtains the method for predicted value for the reference key that whether equals adjacent block by the reference key of determining current block.

Figure 12 illustrates the flow chart of carrying out the method for luminance compensation based on the type of prediction of current block.

Figure 13 illustrates for carry out the flow chart of the method for luminance compensation with the flag information of the luminance compensation that indicates whether execution block.

Figure 14 illustrates the flow chart of method of predicting the flag information of current block for the reference key that whether equals adjacent block by the reference key of determining current block.

Figure 15 illustrates the flow chart of carrying out the method for luminance compensation when utilizing two or more reference blocks to come predictive coding when current block.

Figure 16 illustrates for not only with the sign of the luminance compensation that indicates whether to carry out current block but also carry out the flow chart of the method for luminance compensation with the deviant of current block.

Figure 17 A-17B is the diagram that illustrates be used to the method for using the sign that is associated with the piece of P and B band and deviant to carry out luminance compensation.

Figure 18 illustrates the flow chart of carrying out the method for luminance compensation when utilizing two or more reference block predictive codings when current block.

Figure 19 illustrates for carry out the flow chart of the method for luminance compensation with the sign of the luminance compensation that indicates whether to carry out current block.

Figure 20 A-20C is the diagram of scope that the flag information of the luminance compensation that indicates whether to carry out current block is shown.

Figure 21 illustrates the flow chart that obtains the method for motion vector be used to the deviant of considering current block.

Embodiment

In order effectively to process many view sequence, incoming bit stream comprises and allows decoding device to determine the whether information relevant with many views shelves of incoming bit stream.In definite incoming bit stream related to the situation of many views shelves, the side information that is associated with many view sequence was added to bit stream and sends to decoder according to grammer.For example, many views shelves ID can indicate for process the shelves pattern of multi-view video data according to the revision of standard H.264/AVC.

MVC (multi-view video coding) technology is the revision technique of standard H.264/AVC.That is, information is that the MVC pattern is added specific syntax as a supplement.Be used for supporting that this revision of MVC technology can be more effective than the alternative of using unconditional grammer.For example, if many views of shelves identifier indication shelves of AVC technology add many view sequence information and can improve code efficiency.

H.264/AVC the sequence parameter set of bit stream (SPS) indication comprises the header information of the information (for example shelves and level) that is associated with the complete sequence coding.

Full compression moving image (being sequence) can begin with sequence header, thereby arrives decoder corresponding to the sequence parameter set (SPS) of header information early than the data by the parameter set reference.As a result, sequence parameter set RBSP at import S1 place as the header information (Fig. 2) of the packed data of moving image.If receive bit stream, shelves id informations " profile_idc " identify in several grades which shelves corresponding to the bit stream that receives.

Shelves id informations " profile_idc " can for example be made as " MULTI_VIEW_PROFILE ", thereby the grammer that comprises grade id information can determine whether the bit stream that receives relates to many views shelves.When the bit stream that receives relates to many views shelves, can add following configuration information.

Fig. 1 illustrates the block diagram of exemplary decoding device (perhaps " decoder ") of multi-view video system that contains the vision signal of multi-view video sequence for decoding.The multi-view video system comprises corresponding encoding device (perhaps " encoder ") providing the multi-view video sequence as bit stream, and this bit stream is included in the upper coded image data of implementing of machine-readable information carrier (for example machinable medium or the machine readable energy signal propagated) between transmitter and receiver.

With reference to figure 1, decoding device comprises resolution unit 10, entropy decoding unit 11, inverse quantization/inverse transformation unit 12, inter prediction unit 13, intraprediction unit 14, deblocking filter 15 and decoding image buffer 16.

Inter prediction unit 13 comprises motion compensation units 17, luminance compensation unit 18 and luminance compensation skew predicting unit 19.

The video sequence that 10 pairs of resolution unit receive in the NAL unit is carried out and is resolved the video sequence that is received with decoding.Usually, before to band head and strip data decoding, one or more sequence parameter set and picture parameter set are sent to decoder.In this case, the extended area of NAL head or NAL head can comprise various configuration informations, for example, and time domain level information, view level information, anchor picture id information and view id information etc.

Term " time stage information " expression is used for providing from vision signal the hierarchy information of time domain scalability, thus can via above-mentioned time domain level information with the various times district sequence offer the user.

Term " view level information " expression is used for providing from vision signal the hierarchy information of view scalability.The multi-view video sequence can limit time domain level and view level, thereby can various time domain sequences and view sequence be offered the user according to the time domain level that limits and view level.

In this way, if restricted class information as mentioned above, the user can adopt time domain scalability and view scalability.Therefore, the user can check the sequence corresponding to expected time and view, perhaps can check the sequence corresponding to another restriction.Also can set up in every way above-mentioned level information according to reference conditions.For example, can change level information according to position of camera, and also can change a grade information according to the camera layout types.In addition, also can not add any specific reference ground information of foundation level arbitrarily.

Term " anchor picture " represents that wherein all bands only refer to when the band in front view but not the coded image of the band in other view.Random access between view can be based on anchor picture to carry out many view sequence decodings.

The anchor picture id information can be used to carry out random access procedure does not need with the data of access particular figure the mass data of decoding.

Term " view id information " expression is used for difference when the customizing messages of the image of the image of front view and another view.When the video sequence signal is encoded, for an image is differentiated from other image, can use image sequence number (POC) and frame number information (frame_num).

If current sequence is confirmed as the multi-view video sequence, can predict between execution view.Identifier is used to distinguish when the image of front view and the image of another view.

Can define view identifier with the view of indicating image.Decoding device can obtain to be different from above-mentioned view identifier the information of the image in the view of view of present image, thereby it can come decoded video signal with the information of image.Above-mentioned view identifier can be applied to whole coding/decoding processes of vision signal.And, consider that the above-mentioned view identifier of view also can be applied to using the multi-view video coding process of frame number information " frame_num ".

Usually, many view sequence have mass data, and the hierarchical coding function of each view (being also referred to as " view scalability ") can be used to process mass data.For execution view scalability function, can define the predict of the view of considering many view sequence.

Can predict that the direction of order or several view sequence defines above-mentioned predict by structuring.For example, if given several view sequence that will be encoded, all basic view is located in the center of layout, thereby the view sequence that will be encoded can be selected by layering.All end or the other parts of layout can be set as basic view.

If the number of camera view represents by the exponential depth of " 2 ", the said circumstances of the camera view that can represent based on the exponential depth by " 2 " forms the hierarchical prediction structure between several view sequence.In other situation, if the number of camera view be can't help the exponential depth of " 2 " and represented, can and can form predict based on virtual view with virtual view.If the camera layout refers to two dimensional topology, can replace (by turns) foundation prediction order along level or vertical direction.

Resolved bit stream carries out the entropy decoding by entropy decoding unit 11, and the coefficient of data such as each macro block, motion vector etc. are extracted.Inverse quantization/inverse transformation unit 12 use predetermined constant go to take advantage of the quantized value that receives obtaining the coefficient value after conversion, and the coefficient value that obtains is carried out inverse transformation, thus its reconstructed pixel value.The pixel value of inter prediction unit 13 use reconstruct is carried out the inter prediction function by the decoded samples of present image.

Simultaneously, deblocking filter 15 is applied to each decoded macroblock to reduce the piece distortion level.Deblocking filter 15 execution block edges level and smooth, thus its improves the picture quality of decoded frame.The selective dependency of filtering is in boundary intensity with in the gradient of the image pattern of boundary vicinity layout.Filtered image is stored in decoded picture buffer device 16, thereby they can be output or be used as reference picture.

16 storages of decoded picture buffer device or output precoding image are to carry out the inter prediction function.In this case, the frame number information " frame_num " of image and POC (image sequence number) information is used to storage or output precoding image.In the situation of MVC technology, the image of other view may reside in above-mentioned pre-encoded video.Therefore, in order to use above-mentioned image as the reference image, can not only use as required " frame_num " and POC information, and use the view identifier of indicating image view.

The reference picture that inter prediction unit 13 use are stored in decoded picture buffer device 16 is carried out inter prediction.Inter-coded macroblocks can be divided into the macro block part.Each macro block part can be utilized one or two reference picture predictions.

The information that motion compensation units 17 use receive from entropy decoding unit 11 compensates the motion of current block.Motion compensation units 17 is extracted the motion vector of the adjacent block of current block from vision signal, and obtains the motion vector predictor of current block.The motion that the motion vector that motion compensation units 17 use are extracted from vision signal and the motion vector predictor that obtains and the difference between predicted value compensate current block.Above-mentioned motion compensation can utilize only reference picture execution, perhaps also can utilize a plurality of reference pictures to carry out.

Therefore, if above-mentioned reference picture is confirmed as being different from the image when other view of front view, can carry out motion compensation according to the view identifier of other view of indication.

Direct Model represents for based on predicted the coding mode of the movable information of current block by the movable information of the piece of complete decoding.Above-mentioned Direct Model can reduce for the needed figure place of coded motion information, and result has improved compression efficiency.

For example, time domain direct mode is predicted the movable information of current block with the correlation of the movable information of time domain direction.Be similar to time domain direct mode, decoder can be predicted with the correlation of the movable information of view direction the movable information of current block.

If the bit stream that receives corresponding to many view sequence, can be caught view sequence by different cameras respectively, thereby because luminance difference can occur inside or the external factor of camera.In order to reduce the potential inefficiency problem that is associated with luminance difference, luminance compensation unit 18 is carried out the luminance compensation function.

In the situation of carrying out the luminance compensation function, flag information can be used to indicate the luminance compensation of whether carrying out at a specific order place of vision signal.For example, luminance compensation unit 18 can be carried out the luminance compensation function with the flag information of the luminance compensation that indicates whether to carry out corresponding band or macro block.And the said method that is used for using above-mentioned flag information to carry out luminance compensation can be applied to various macro block (mb) types (for example, interframe 16 * 16 patterns, B skip (B-skip) pattern, Direct Model etc.).

For the reconstruct current block, can use the information of adjacent block or be different from the information of the piece in the view of view of current block when carrying out luminance compensation, and also can use the deviant of current block.

In this case, the deviant of current block indication is at the average pixel value of current block and corresponding to the difference between the average pixel value of the reference block of current block.Example as using above-mentioned deviant can obtain by the adjacent block with current block the predicted value of current block deviant, and can use the residual value between deviant and predicted value.Therefore, decoder can come with residual value and predicted value the deviant of reconstruct current block.

In order to obtain the predicted value of current block, can use as required the information of adjacent block.

For example, can be by predict the deviant of current block with the deviant of adjacent block.Before prediction current block deviant, determine whether the reference key of current block equals the reference key of adjacent block.According to definite result, which adjacent block is luminance compensation unit 18 can determine to use or use which value.

Luminance compensation unit 18 can be carried out luminance compensation with the type of prediction of current block.If utilize two reference blocks to the current block predictive coding, luminance compensation unit 18 can obtain with the deviant of current block the deviant corresponding to each reference block.

As mentioned above, select according to predictive mode by inter prediction image or infra-frame prediction image that luminance compensation and motion compensation are obtained, and the reconstruct present image.

Various examples for the coding/decoding method of reconstruct present image will be described in the document afterwards.

Fig. 2 is the flow chart that illustrates for the method for encoded video sequence.

With reference to figure 2, at step S131, the average pixel value of at least one piece in the adjacent block of the example of video sequence coding method acquisition current block and the reference block of another view.When receiving the value that obtains, the video sequence coding method is used the prediction average pixel value of at least a pattern derivation current block in several patterns at step S132.The video sequence coding method obtains in the prediction average pixel value of current block and the difference between the actual average pixel value at step S133.The video sequence coding method is measured each code efficiency of above-mentioned several patterns at step S134, and selects optimization model from these several patterns.Can select in every way above-mentioned optimization model, for example, can use for selecting the method for minimal difference from the acquisition difference, and be used for using the method etc. of equation of the relation of indication rate distortion (rate-distortion, RD).

In this case, above-mentioned RD equation is not only recognized the coding figure place that produces but also is recognized the distortion value of indicating the difference that is associated with real image during the coding of corresponding blocks, thereby it assesses the cost with coding figure place and distortion value.In further detail, the video sequence coding method uses the Lagrange multiplier of being determined by quantization parameter to go to take advantage of figure place, and adds distortion value to result of product, thereby it assesses the cost.Sign (ID) information of the selected pattern of indication if selection optimization model, video sequence coding method can be encoded, and send coding result.Alternately, if select optimization model, can not only the encode id information of the selected pattern of indication but also the difference that coding utilizes selected pattern to obtain of video sequence coding method, and send coding result at step S135.

Fig. 3 is the block diagram that illustrates for from the process of the prediction average pixel value of the reference block derivation current block of another view.

With reference to figure 3, suppose B _cThe average pixel value of piece is m _c, B _{R, 1}The average pixel value of piece is m _{R, 1}, and the average pixel value of all the other pieces is by above-mentioned symbolic representation.Have for the information according to one or more adjacent block and predict m _cThe whole bag of tricks of information.For convenience of description, suppose at coding B _cUse reference frame #1 as candidate's reference frame in the situation of piece.

Be used for the information prediction m according to one or more adjacent block _cThe first method of information is for predicting m based on the average pixel value corresponding to the reference block of another view of current block _cThe first mode method (Mode1) of information.In further detail, first mode method (Model) indication is used for using the B of reference frame #1 _{R, 1}The average pixel value of piece is predicted m _cThe method of information.Difference can be represented by following equation 1:

[equation 1]

e＝m _c-m _r，1

Be used for prediction and at the average pixel value of current block with corresponding to the second method of the difference between the average pixel value of the reference block of another view of current block be for come second mode method (Mode2) of prediction difference based on the difference between the average pixel value of each adjacent block of current block and reference block.In further detail, the second mode method (Mode2) uses at adjacent block Between the difference of average pixel value predict at the average pixel value of current block and the B of reference frame #1 _{R, 1}Difference between the average pixel value of piece.

This difference can be represented by following equation 2:

[equation 2]

$e=(m_c-m_{r,1})-(m_c^1-m_{r,1}^1)$

Be used for prediction and at the average pixel value of current block with corresponding to the third method of the difference between the average pixel value of the reference block of another view of current block be used to using difference between the average pixel value of the average pixel value of the adjacent block of current block and reference block to come the three-mode method (Mode3) of prediction difference.In further detail, three-mode method (Mode3) is based at adjacent block

Average pixel value and the B of reference frame #1 _{R, 1}Difference between the average pixel value of piece is predicted m _cInformation.In this case, difference can be represented by following equation 3:

[equation 3]

$e=(m_c-m_{r,1})-(m_c^1-m_{r,1})=m_c-m_c^1$

In the situation of the adjacent block by the current block of encoding with the adjacent block of the reference block of another view, have for the prediction average pixel value based on the adjacent block of current block and predict m _cThe four-mode method (Mode4) of information.In other words, if pass through with reference to reference frame #2's

Piece and right The piece precoding is at current block (B _c) and corresponding to the reference block (B of current block _{R, 1}) average pixel value between difference can utilize adjacent block at current block Average pixel value and the adjacent block of another view reference block

Average pixel value between difference predict.

In this case, difference can be represented by following equation 4:

[equation 4]

$e=(m_c-m_{r,1})-(m_c^1-m_{r,2}^1)$

Stating in the use Mode2, Mode3 and Mode4 method uses in the situation of adjacent block information, although above-mentioned Mode2, Mode3 and Mode4 method be an open information only exemplarily using piece on next, should be noted that around the combination of the information of several adjacent blocks of current block and also can be used as example.

Fig. 4 is the detailed diagram that illustrates for the process of the prediction average pixel value of the current block of deriving from the reference block of other view.

In further detail, Fig. 4 illustrates other piece of the pre-encoded blocks of current block, each and current block Border and each and reference block Border.In this case, Mode2 method equation, Mode3 method equation and Mode4 method equation can be represented by following equation 5:

[equation 5]

$\mathrm{Mode}2:e=(m_c-m_{r,1})-\frac{{\mathrm{\Σ}}_i{w}_i(m_c^i-m_{r,1}^i)}{{\mathrm{\Σ}}_i{w}_i}$

$\mathrm{Mode}3:e=(m_c-m_{r,1})-\frac{{\mathrm{\Σ}}_i{w}_i(m_c^i-m_{r,1})}{{\mathrm{\Σ}}_i{w}_i}{=m}_c-\frac{{\mathrm{\Σ}}_i{w}_i{m}_c^i}{{\mathrm{\Σ}}_i{w}_i}$

$\mathrm{Mode}4:e=(m_c-m_{r,1})-\frac{{\mathrm{\Σ}}_i{w}_i(m_c^i-m_{r,k}^i)}{{\mathrm{\Σ}}_i{w}_i}$

In above-mentioned Mode4 equation,

Indication exists

The reference block of piece is positioned at the average pixel value of this reference block under the condition at reference frame #k place.

In equation 5, w _iThe indication weight coefficient.The adjacent block that is used for prediction is not limited to the piece of Border, and also can comprise as required other piece that is adjacent to above-mentioned adjacent block.In other situation, above-mentioned adjacent block also can adopt the only some parts of other piece.The scope of above-mentioned adjacent block can be by w _iAdjust.In this way, difference (e) is quantized and is coded by entropy, thereby the entropy coded message is sent to decoding unit.

Calculating several steps after the actual bit stream stage, consider speed and distortion factor, the reference frame of above-mentioned Mode1, Mode2, Mode3 and Mode4 method is confirmed as optimal frames.Have for the whole bag of tricks of selecting optimization model, for example, for the method for the AD HOC of selecting minimal difference from the acquisition difference with for the method for using the RD relation.

The actual bit stream of each pattern is calculated in above-mentioned RD-relational approach, and considers speed and distortion (rate and the distortion) and select optimization model.In the situation of computing block residual value, the average pixel value of each piece is inferred in above-mentioned RD relational approach from current block, infer the average pixel value of each piece from reference block, and calculate the difference between institute's inferred results of current block and reference block, as represented by following equation 6:

[equation 6]

$\underset{i}{\mathrm{\Σ}}\underset{j}{\mathrm{\Σ}}|I_c(i,j)-{\tilde{m}}_c-(I_r(i+\mathrm{\Δx},j+\mathrm{\Δy})-m_r|$

In equation 6, Δ x Δ y represents difference vector, and I is pixel value.If utilize value and the difference of the information prediction of adjacent block to be quantized, and the quantized result value of predicted value and difference is reconstructed, and the end value of reconstruct is added, and adds result by equation 6

Expression.In this case,

Value be suitable for obtaining identical value from coding unit and decoding unit.m _rThe average pixel value of indication reference block.In the situation of decoded picture, coding unit has the m identical with decoding unit _rIn fact, searching for reference piece in time-domain, and in the space-time territory the optimum piece of search.Therefore, indicate whether and to use the id information of luminance compensation to be made as explicitly " 0 " or " 1 " with each frame and piece, and resulting id information is coded by entropy.

If the selection optimization model, the only selected pattern of can encoding, thus the coding result of selected pattern can be sent to decoding unit.Except the coding result of selected pattern, the difference of utilizing selected pattern to obtain also can be encoded and send.Selected pattern information is represented by index type, and also can utilize adjacent pattern information to predict.In addition, the difference between the index of the index of current preference pattern and predictive mode also can be encoded and send.

As required, can consider all above-mentioned patterns, can select some in above-mentioned pattern, perhaps also can only select in above-mentioned pattern.In the situation of the single method in using all methods availalbes, need not coding mode index respectively.

Obtain average pixel value and the situation of the prediction average pixel value of deriving in, the pre decoding pixel value can be applied to the reference frame that will be encoded and the current block of target frame.

Basically, the predecode value of the predecode value of left pixel and upside pixel is used to predict the average pixel value of current block.In the situation of coding actual video sequence, come encoded video sequence based on macro block.16 * 16 macro blocks are divided into 16 * 8,8 * 16 and 8 * 8, and then decoded.8 * 8 also can be divided into 8 * 4,4 * 8 and 4 * 4.Have for predict the whole bag of tricks of the average pixel value of sub-block based on single macro block.

Fig. 5 illustrates for explaining that situation at the average pixel value of derivation current block and prediction average pixel value is positioned at the concept diagram of 16 * 16 macro blocks of use of the pre decoding pixel value on the left side of whole and top.

Can use all pixel values on left and top with reference to figure 5,16 * 16 macro blocks.Therefore, in the situation of the average pixel value of predicting current block, calculate top pixels (h1～h16) and the left part pixel (average pixel value of v1～v16), and utilize the pixel calculate (v1～v16, h1～～average pixel value h16) predicts the average pixel value of current block.The average pixel value of 16 * 16 in this case, (by " B16 * 16 " expressions) can be represented by following equation 7:

[equation 7]

$\frac{\underset{i=1}{\overset{16}{\mathrm{\Σ}}}\mathrm{hi}+\underset{i=1}{\overset{16}{\mathrm{\Σ}}}\mathrm{vi}}{32}$

Fig. 6 A is the concept diagram that 16 * 8 macro blocks are shown, and is used for explanation and surrounds the use of all pixels of divided block in the situation of the average pixel value of derivation divided block and prediction average pixel value.Fig. 6 B is the concept diagram that 16 * 8 macro blocks are shown, and is used for explanation and only surrounds the use of the pixel of divided block in the situation of the average pixel value of derivation divided block and prediction average pixel value.

In Fig. 6 A, in the situation of using all pixels of surrounding divided block, the mean value of B16 * 8_0 piece and B16 * 8_1 piece can be represented by following equation 8:

[equation 8]

$\frac{\underset{i=1}{\overset{16}{\mathrm{\Σ}}}\mathrm{hi}+\underset{i=1}{\overset{16}{\mathrm{\Σ}}}\mathrm{vi}}{32}$

In Fig. 6 B, in the situation of using all pixels of surrounding divided block, the mean value of B16 * 8_0 piece can be represented by following equation 9, and the mean value of B16 * 8_1 piece can be represented by following equation 10:

[equation 9]

$\frac{\underset{i=1}{\overset{16}{\mathrm{\Σ}}}\mathrm{hi}+\underset{i=1}{\overset{8}{\mathrm{\Σ}}}\mathrm{vi}}{24}$

[equation 10]

$\frac{\underset{i=1}{\overset{16}{\mathrm{\Σ}}}\mathrm{gi}+\underset{i=9}{\overset{16}{\mathrm{\Σ}}}\mathrm{vi}}{24}$

In the said circumstances of Fig. 6 A-6B, the value h0 that is positioned at the place, bight of macro block also can be added to result of calculation as required.In this case, the average pixel value of the B16 of Fig. 6 A * 8_0 piece can be represented by following equation 11, and the average pixel value of B16 * 8_0 of Fig. 6 B can be represented by following equation 12:

[equation 11]

$\frac{\underset{i=0}{\overset{16}{\mathrm{\Σ}}}\mathrm{hi}+\underset{i=1}{\overset{16}{\mathrm{\Σ}}}\mathrm{vi}}{33}$

[equation 12]

$\frac{\underset{i=0}{\overset{16}{\mathrm{\Σ}}}\mathrm{hi}+\underset{i=1}{\overset{8}{\mathrm{\Σ}}}\mathrm{vi}}{25}$

In the said circumstances of Fig. 6 A-6B, the value h0 and the v8 that are positioned at the place, bight of macro block also can be added to result of calculation as required.In this case, the average pixel value of the B16 of Fig. 6 A * 8_1 piece can be represented by following equation 13, and the average pixel value of B16 * 8_1 of Fig. 6 B can be represented by following equation 14:

[equation 13]

$\frac{\underset{i=0}{\overset{16}{\mathrm{\Σ}}}\mathrm{hi}+\underset{i=1}{\overset{16}{\mathrm{\Σ}}}\mathrm{vi}}{33}$

[equation 14]

$\frac{\underset{i=0}{\overset{16}{\mathrm{\Σ}}}\mathrm{gi}+\underset{i=8}{\overset{16}{\mathrm{\Σ}}}\mathrm{vi}}{25}$

Fig. 7 A is the concept diagram that 8 * 16 macro blocks are shown, and is used for explanation and surrounds the use of all pixels of divided block in the situation of the average pixel value of derivation divided block and prediction average pixel value.Fig. 7 B is the concept diagram that 8 * 16 macro blocks are shown, and is used for explanation and only surrounds the use of the pixel of divided block in the situation of the average pixel value of derivation divided block and prediction average pixel value.Method and Fig. 6 A-6B identical that is used for the average pixel value of derivation divided block.

Fig. 8 A is the concept diagram that 8 * 8 macro blocks are shown, and is used for explanation and surrounds the use of all pixels of divided block in the situation of the average pixel value of derivation divided block and prediction average pixel value.Fig. 8 B is the concept diagram that 8 * 8 macro blocks are shown, and is used for explanation and only surrounds the use of the pixel of divided block in the situation of the average pixel value of derivation divided block and prediction average pixel value.

Method and Fig. 6 A-6B identical that is used for the average pixel value of derivation divided block.

These 8 * 8 can be divided into a plurality of sub-blocks.

The average pixel value of the corresponding blocks of the current block of the present frame that prediction will be encoded, thus the prediction average pixel value is set as

The average pixel value of the corresponding blocks of prediction reference frame, thus the average pixel value of prediction is set as

Infer each prediction average pixel value from all pixels of each piece, and can by following equation 15, calculate the difference between the pixel value of the pixel value that uses the reference block prediction and current block:

[equation 15]

$\underset{i}{\mathrm{\Σ}}\underset{j}{\mathrm{\Σ}}|I_c(i,j)-{\hat{m}}_c-(I_r(i+\mathrm{\Δx},j+\mathrm{\Δy})-{\hat{m}}_r)|$

In equation 15, (Δ x, Δ y) indicates difference vector, and I is pixel value.Reference block with smallest blocks residual value is selected as the optimum piece of luminance compensation.In this case, difference vector is represented by (Δ x, Δ y).In fact, the above-mentioned luminance compensation situation of systematic comparison and another situation of not compensated of brightness wherein, and select in these two kinds of situations one preferably.

Revise example for one as such scheme, the average pixel value of reference block does not utilize the pixel value prediction of adjacent block, but utilizes the average pixel value of all pixels that comprise in actual block directly to calculate.

As another modification example of such scheme, can increase the number of left and top pixels.In further detail, can use the pixel of two of current layer or more heterogeneous adjacent bed to replace the only pixel of a layer of contiguous current layer.

Decoding unit determines whether to carry out the luminance compensation of corresponding blocks with id information.If the execution luminance compensation, the decode value of decoding unit calculated difference (e), and obtain predicted value according to above-mentioned Forecasting Methodology.The decode value of difference (e) is added to predicted value, thus value

Can be decoded.Infer from reference block

Value, described reference block is the prediction piece, be exactly so-called predicted value for current block, and institute's inferred results is added to the decode value of residual block, thereby can obtains at last the value of current block.Current block can followingly be reconstructed:

Wherein B is the value of current block, and reference block is the predicted value for current block,

Be the forecasted variances of average pixel value, namely be used for the prediction drift value of the luminance compensation of current block, and e be difference.Decoding unit obtains poor between the deviant of the luminance compensation of current block and predicted difference, and can come with acquisition residual block value and predicted difference the deviant of the luminance compensation of reconstruct current block.

Fig. 9 is the diagram that illustrates for the process of the deviant that obtains current block.

Can carry out luminance compensation during estimation.When it compares current block and reference block, consider the luminance difference between two pieces.New estimation and new motion compensation are used to compensate for brightness difference.New SAD (summation of poor absolute value) can be represented by following equation 16 and 17:

[equation 16]

$\mathrm{SAD}=\underset{x=m}{\overset{M+m-1}{\mathrm{\Σ}}}\underset{y=n}{\overset{N+n-1}{\mathrm{\Σ}}}|(I_c(x,y)-M_c)-(I_r(x+\mathrm{\Δx},y+\mathrm{\Δy})-M_r)|$

$=\underset{x=m}{\overset{M+m-1}{\mathrm{\Σ}}}\underset{y=n}{\overset{N+n-1}{\mathrm{\Σ}}}|{(I}_c(x,y)-I_r(x+\mathrm{\Δx},y+\mathrm{\Δy}\left)\right)-(M_c-M_r)|$

[equation 17]

$M_c=\underset{x=m}{\overset{M+m-1}{\mathrm{\Σ}}}\underset{y=n}{\overset{N+n-1}{\mathrm{\Σ}}}{I}_c(x,y)$

$M_r=\underset{x=m}{\overset{M+m-1}{\mathrm{\Σ}}}\underset{y=n}{\overset{N+n-1}{\mathrm{\Σ}}}{I}_r(x+\mathrm{\Δx},y+\mathrm{\Δy})$

With reference to equation 16 and 17, M _cThe average pixel value of expression current block, and M _rThe average pixel value of expression reference block.I _c(x, y) is illustrated in the pixel value that the preferred coordinates (x, y) of current block are located, and I _r(x+ Δ x, y+ Δ y) is illustrated in the pixel value that the motion vector (Δ x, Δ y) of reference block is located.Carry out estimation based on the new SAD by equation 16 expression, thereby can obtain the difference between the average pixel value of the average pixel value of current block and reference block.The difference of the average pixel value between current block and reference block is called as deviant (IC_offset).

If carry out the estimation that is applied to luminance compensation, obtain deviant and motion vector.Can carry out luminance compensation by the equation 18 below deviant and motion vector utilization:

[equation 18]

R(x，y)＝I _c(x，y)-I _r(x+Δx，y+Δy)-(M _c-M _r)

With reference to equation 18, R (x, y) expression luminance compensation residual value.

Deviant (IC_offset=M _c-M _r) be sent to decoding unit.Can utilize following equation 19 to carry out the luminance compensation of decoding unit:

[equation 19]

I′ _c(x，y)＝I _r(x+Δx，y+Δy)+R′(x，y)+(M _c-M _r)

With reference to equation 19, the residual value of R ' (x, y) expression reconstruct and luminance compensation, and I ' _cThe pixel value of (x, y) expression current block.

For the reconstruct current block, deviant is sent to decoding unit, and can utilize the data of adjacent block to come the prediction drift value.In order further to reduce the figure place that is used for the code offset value, the difference (R between current block deviant (IC_offset) and adjacent block deviant (IC_offset_pred) _{IC_offset}) can be sent to decoding unit 50, as represented by following equation 20:

[equation 20]

R _{IC_offset}＝IC_offset-IC_offset_pred

Figure 10 is the flow chart that illustrates for the process of the luminance compensation of carrying out current block.

With reference to Figure 10, if the luminance compensation sign of current block is set as " 0 ", do not carry out the luminance compensation of current block.Otherwise, if the luminance compensation sign of current block is set as " 1 ", carry out the process of the deviant that is used for the reconstruct current block.In the situation of the predicted value that obtains current block, can adopt the information of adjacent block.Judge at step S210 whether the reference key of current block equals the reference key of adjacent block.Obtain predicted value for the luminance compensation of carrying out current block at step S211 based on this result of determination.At step S212 by using the predicted value that obtains to come the deviant of reconstruct current block.In this case, with hereinafter with reference to Figure 11 describe to be used for determine the reference key of current block whether equal adjacent block reference key step S210 and be used for determining that based on this result obtains the step S211 of predicted value.

Figure 11 illustrates the flow chart that obtains the method for predicted value for the reference key that whether equals adjacent block by the reference key of determining current block.

With reference to Figure 11, in order to carry out luminance compensation, decoding unit extracts various information from vision signal, for example, the flag information of the adjacent block of current block and deviant, and the reference key of the reference block of current and adjacent block, thereby decoding unit can obtain with the information of extracting the predicted value of current block.Decoding unit obtains in the deviant of current block and the residual value between predicted value, and can come with the residual value that obtains and predicted value the deviant of reconstruct current block.

In the situation of the predicted value that obtains current block, can adopt the information of adjacent block.For example, can utilize the deviant of adjacent block to predict the deviant of current block.Before the deviant of prediction current block, can determine whether the reference key of current block equals the reference key of adjacent block, thereby can determine that result determines to use which value or which adjacent block by reference.And, determine whether the flag information of adjacent block is set as "True", thereby can determine that result determines whether to use adjacent block by reference.

According to the first example, determine whether to exist the adjacent block with reference key identical with current block at step S220.If determine to have only adjacent block with reference key identical with current block, at step S221, the deviant that will have the adjacent block of this same reference index is distributed to the predicted value of current block.If determine to exist each to have two adjacent blocks of the reference key identical with current block at step S220, at step S222, the mean value of the deviant of these two adjacent blocks is distributed to the predicted value of current block.If determine to exist each to have three adjacent blocks of the reference key identical with current block at step S220, at step S223, the intermediate value of the deviant of these three adjacent blocks is distributed to the predicted value of current block.If according to determining not have the adjacent block with reference key identical with current block in definite result of step S220, at step S224, the predicted value of current block is made as " 0 ".The step S220 whether reference key that if necessary, be used for to determine current block equals the reference key of adjacent block can also comprise another step that whether is set as " 1 " for the sign of determining adjacent block.

According to the second example, determine whether adjacent block has the reference key identical with current block, and whether the sign of definite adjacent block is set as " 1 ".If determine that adjacent block has the reference key identical with current block, and have the sign of " 1 ", the deviant of adjacent block can be set as the predicted value of current block.In this case, can according to left adjacent block-＞upper adjacent block-＞the upper right adjacent block-＞a plurality of adjacent blocks of sequential search of upper left adjacent block.If necessary, also can according to upper adjacent block-＞left adjacent block-＞the upper right adjacent block-＞the sequential search adjacent block of upper left adjacent block.If there is no to satisfy the adjacent block of these two conditions, and the sign of these three adjacent blocks (being left adjacent block, upper adjacent block and upper right (perhaps upper left) adjacent block) is set as " 1 " respectively, and the intermediate value of the deviant of these three pieces is set as predicted value.Otherwise the predicted value of current block can be set as " 0 ".

Figure 12 illustrates for carry out the flow chart of the method for luminance compensation based on the type of prediction of current block.

With reference to Figure 12, can change according to the type of prediction of current block the adjacent block as reference block.For example, if current block has the shape identical with adjacent block, utilize the intermediate value of adjacent block to predict current block.Otherwise, if the shape of current block is different from the shape of adjacent block, will adopt another kind of method.

For example, if the piece that is positioned on the left of current block is divided into several sub-blocks, the sub-block that goes up most in sub-block is used to prediction.And, be divided into several sub-blocks if be positioned at the piece on current block top, the most left sub-block is used to prediction.In this case, can change predicted value according to the type of prediction of current block.Therefore, the example of Figure 12 is determined adjacent block at step S231 according to the type of prediction of current block.Determine at step S232 whether the reference key of determined adjacent block equals the reference key of current block.The step S232 that whether equals the reference key of current block for the reference key of determining adjacent block also can comprise another step that whether is set as " 1 " for the sign of determining adjacent block.Can be at step S233 based on determining that result obtains the predicted value for the luminance compensation of carrying out current block.Utilize the deviant of the predicted value reconstruct current block that obtains, thereby can carry out luminance compensation at step S234.In this case, will be described in more detail below for coming the process of execution in step S233 by the result of refer step S232, and its detailed description will be similar to Figure 11.

For example, if the indication of the type of prediction of current block is positioned at the adjacent block execution prediction in current block left side by use, determine whether the reference key of left side adjacent block equals the reference key of current block.If the reference key of current block equals the reference key of left side adjacent block, the deviant of left side adjacent block is assigned to the predicted value of current block.And, if the indication of the type of prediction of current block is carried out prediction by the left adjacent and upper adjacent block of reference current block, if perhaps by carrying out prediction with reference to three adjacent blocks (being left adjacent block, upper adjacent block and upper right adjacent block), will use similarly each situation as the method for Figure 11.

Figure 13 illustrates for carry out the flow chart of the method for luminance compensation with the flag information of the luminance compensation that indicates whether execution block.

With reference to Figure 13, the flag information (IC_flag) that indicates whether to carry out the luminance compensation of current block also can be used to the deviant of reconstruct current block.In addition, also can obtain predicted value with the method that is used for checking the method for reference key and being used for prediction indication information of Figure 11.At first, determine whether to exist the adjacent block with reference key identical with current block at step S241.Utilize this to determine that result obtains the predicted value for the luminance compensation of carrying out current block at step S242.Whether sign that in this case, be used for to determine adjacent block is that the process of " 1 " also can be included in step S242.At step S243, determine that based on this result predicts the flag information of current block.By the deviant of using the predicted value that obtains and prediction indication information to come the reconstruct current block, thereby can carry out luminance compensation at step S244.In this case, can be as the method for Figure 11 applying step S242 similarly, and with reference to Figure 14, step S243 is described hereinafter.

Figure 14 illustrates the flow chart of method of predicting the flag information of current block for the reference key that whether equals adjacent block by the reference key of determining current block.

With reference to Figure 14, determine whether to exist the adjacent block with reference key identical with current block at step S250.If determine only to exist an adjacent block with reference key identical with current block, utilize the flag information of the adjacent block with this same reference index to predict the flag information of current block at step S251.If determine to exist each to have two adjacent blocks of the reference key identical with current block at step S250, utilize any one in the flag information of these two adjacent blocks with this same reference index to predict the flag information of current block at step S252.

If determine to exist each to have three adjacent blocks of the reference key identical with current block at step S250, utilize the intermediate value of the flag information of these three adjacent blocks to predict the flag information of current block at step S253.In addition, if do not have the adjacent block with reference key identical with current block according to definite result of step S250, do not predict the flag information of current block at step S254.

Figure 15 is the flow chart that the method that is used for carrying out luminance compensation when utilizing two or more reference blocks to carry out predictive coding to current block is shown.

With reference to Figure 15, during carrying out luminance compensation, if by coming the predictive coding current block with two reference blocks, decoding unit can not Direct Recognition goes out the deviant corresponding to each reference block, because it uses the average pixel value of these two reference blocks when obtaining the deviant of current block.Therefore, in an example, obtain the deviant corresponding to each reference block, result has realized correct Prediction.At step S261, by the deviant of coming the reconstruct current block with predicted value and the residual value of current block.If by coming the predictive coding current block with two reference blocks, utilize this deviant acquisition corresponding to the deviant of each reference block, as represented by following equation 21 at step S262:

[equation 21]

IC_offset＝m _c-w ₁×m _r，1-w ₂×m _r，2

IC_offsetL0＝m _c-m _r，1＝IC_offset+(w ₁-1)×m _r，1+w ₂×m _r，2

IC_offsetL1＝m _c-m _r，2＝IC_offset+w ₁×m _r，1+(w ₂-1)×m _r，2

In equation 21, m _cIt is the average pixel value of current block.m _{R, 1}And m _{R, 2}Indicate respectively the average pixel value of reference block.w ₁And w ₂Indication is used for the weight coefficient of two predictive coding processes respectively.

In an example of luminance compensation method, system obtains the deviant accurately corresponding to each reference block independently, thereby it can correctly carry out the predictive coding process.In the situation of the deviant of step S262 reconstruct current block, system is with residual value and the addition of predicted value value of reconstruct, thereby its obtains deviant.In this case, obtain respectively the predicted value of reference picture of List0 and List1 reference picture predicted value and these predicted values are made up, thereby this system can obtain the predicted value for the deviant of reconstruct current block.

According to another example, this system also can be applied to skip macroblock (skip-marcoblock).In this case, carry out prediction to obtain to be used for the information of luminance compensation.The value of utilizing adjacent block to predict is used as indicating whether to carry out the flag information of luminance compensation.The deviant of utilizing adjacent block to predict can be used as the deviant of current block.For example, if flag information is set as "True", deviant is added to reference block.In using P-and skipping the situation of macro block of (P-skip) pattern, carry out prediction by sign and the deviant of using left adjacent block and upper adjacent block, thereby can obtain sign and the deviant of macro block.The sign that has " 1 " iff a piece, the sign of current block and deviant can be set as sign and the deviant of this piece respectively.If two pieces have the sign of " 1 ", the sign of current block is set as " 1 ", and the deviant of current block is set as the mean deviation value of these two adjacent blocks.

According to another example, this system also can be applied to Direct Model, and for example, time domain direct mode, B-skip (B-skip) pattern etc.In this case, carry out prediction to obtain the information of luminance compensation.The variable method that is used for prediction indication and skew by use can obtain each predicted value.This predicted value can be set as actual tag and the actual shifts value of current block.If each piece has a pair of sign and offset information, can obtain the predicted value of each piece.In this case, if having two reference blocks and check the reference key of these two reference blocks, determine whether the reference key of current block equals the reference key of adjacent block.In addition, if each reference block comprises unique deviant, can obtain the first prediction indication information, the first prediction drift value, the second prediction indication information and the second prediction drift value.In this case, utilize the value of adjacent block prediction can be used as flag information.The deviant of these two reference blocks can be used as the first prediction drift value and the second prediction drift value respectively.In this case, the deviant of current block can be set as the mean deviation value of each reference block.

In Direct Model or skip macroblock (skip macroblock) pattern, this system can indicate Direct Model or skip macroblock pattern whether to be applied to the flag information of current block by coding/decoding.In further detail, add or do not add deviant according to value of statistical indicant.Residual value between deviant and prediction drift value also can be encoded/decode.In this case, more correctly reconstruct expected data, and can consider that RD (rate distortion) relation selects optimization model.If reference picture can not be used to forecasting process, that is, if the reference picture number less than " 1 ", flag information or prediction indication information can be set as " vacation ", and deviant or prediction drift value also can be set as " 0 ".

According to another example, this system also can be applied to the entropy cataloged procedure.With flag information explicitly, can use three context models (context model) according to the value of statistical indicant of adjacent block (for example being positioned at the piece on a left side and the top of current block).

If determine that value of statistical indicant is set as "True", generation value " 1 ".If determine that value of statistical indicant is set as " vacation ", generation value " 0 ".If two values of this of these two kinds of situations " 1 " and " 0 " addition can obtain three kinds of situations.By coming the coding/decoding flag information with three context models.The coding method of conversion coefficient level can be used to the prediction residual value of deviant.In other words, utilize the binarization of UEG0 executing data, single context model can be applied to a bin value, and another context model is applied to all the other bin values of monobasic prefix part (unary prefix part).Utilize bypass (bypass) pattern-coding/decoding symbols position.According to another example of flag information, can consider two contexts according to the prediction indication value, thereby can carry out the coding/decoding process.

Figure 16 illustrates for not only with the flag information of the luminance compensation that indicates whether to carry out current block but also carry out the flow chart of the method for luminance compensation with the deviant of current block.

With reference to Figure 16, in order to carry out luminance compensation, decoding unit extracts various information from vision signal, for example, the flag information of adjacent block current block and current block and deviant, and the index information of the reference block of current and adjacent block, thereby decoding unit can obtain with above extracted information the predicted value of current block.Decoding unit 50 obtains in the deviant of current block and the residual value between predicted value, and can come with the residual value that obtains and predicted value the deviant of reconstruct current block.In the situation of the deviant of reconstruct current block, can use the flag information (IC_flag) of the luminance compensation that indicates whether to carry out current block.

Decoding unit obtains to indicate whether to carry out the flag information of the luminance compensation of current block at step S271.If carry out luminance compensation according to above-mentioned flag information (IC_flag), step S272 can the average pixel value of reconstruct indication between current block and reference block the deviant of current block of difference.In this way, the difference of the average pixel value of above-mentioned luminance compensation technology for encoding between the piece of different images.If comprise corresponding piece in the P band when the sign that indicates whether to carry out luminance compensation is applied to each piece, single flag information and single deviant are encoded/decode.Yet, if comprise corresponding piece in the B band, can make in all sorts of ways, and hereinafter its detailed description is described with reference to Figure 17 A`17B.

Figure 17 A-17B is the diagram that illustrates be used to the method for using the flag information that is associated with the piece of P and B band and deviant to carry out luminance compensation.

With reference to figure 17A, " C " represents current block, the adjacent block of " N " expression current block (C), the reference block of " R " expression current block (C), the reference block of the adjacent block (N) of " S " expression current block (C), and " m _c" expression current block (C) average pixel value, " m _r" average pixel value of reference block of expression current block (C).If the deviant of current block (C) is by " IC_offset " expression, " IC_offset " information can be by " IC_offset=m _c-m _r" expression.

In this way, if the deviant of adjacent block (S) by " IC_offset_pred " expression, coding unit can be with the residual value (R between the deviant (IC_offset_pred) of the deviant (IC_offset) of current block and adjacent block _{IC_offset}) send to decoding unit, thus the deviant " IC_offset " that it can reconstruct current block (C).In this case, " R _{IC_offset}" information also can be by above-mentioned equation 20 expressions.

In the situation of the predicted value that flag information or deviant based on adjacent block produce current block, can make in all sorts of ways.For example, can adopt the only information of an adjacent block, perhaps also can adopt the information of two or more adjacent blocks.In the situation of the information that adopts two or more adjacent blocks, can adopt mean value or intermediate value.In this way, if utilize single reference block to come the predictive coding current block, can carry out luminance compensation with single deviant and single flag information.

Yet, if comprise corresponding piece in the B band, that is, if utilize two or more reference blocks to come the predictive coding current block, can make in all sorts of ways.

For example, as shown in Figure 17 B, suppose " C " expression current block, the adjacent block of " N " expression current block (C), " R0 " expression is positioned at the reference block of being located by the reference picture (1) of the List0 of current block reference, " S0 " expression is positioned at the reference block of being located by the reference picture (1) of the List0 of adjacent block reference, " R1 " expression is positioned at the reference block of being located by the reference picture (3) of the List1 of current block reference, and " S1 " expression is positioned at the reference block of being located by the reference picture (3) of the List1 of adjacent block reference.In this case, the flag information of current block and deviant are associated with each reference block, thereby each reference block comprises two values.Therefore, can adopt respectively at least one in flag information and deviant.

According to the first example, can obtain via motion compensation the predicted value of current block by the information that makes up two reference blocks.In this case, single flag information indicates whether to carry out the luminance compensation of current block.If flag information is confirmed as "True", obtains single deviant from current block and predicted value, thereby can carry out the coding/decoding process.

According to the second example, in movement compensation process, determine whether luminance compensation is applied to each of two reference blocks.Flag information is assigned to each of two reference blocks, and can be encoded or decode by the single deviant of using above-mentioned flag information to obtain.In this case, should be noted that, can use two flag informations based on reference block, and can use single deviant based on current block.

According to the 3rd example, single flag information can indicate luminance compensation whether to be applied to corresponding piece based on current block.Can be for two each deviants of reference block coding/decoding.If luminance compensation is not applied to any one in reference block during cataloged procedure, corresponding deviant is set as " 0 ".In this case, can use single flag information based on current block, and can use two deviants based on reference block.

According to the 4th example, can be for each reference block coding/decoding flag information and deviant.In this case, can use two signs and two deviants based on reference block.

According to above-mentioned the first to the 4th example, deviant is by without any change ground coding (not encoded without any change), but utilizes the deviant of adjacent block to predict, thereby its residual value is encoded.

Figure 18 illustrates for carry out the flow chart of the method for luminance compensation when utilizing two or more reference blocks to come the predictive coding current block.

With reference to Figure 18, in order to carry out luminance compensation under the condition that comprises current block in the B band, extract flag information and the deviant of the adjacent block of current block from vision signal, and extract the index information of reference block of the correspondence of current and adjacent block, thereby can obtain the predicted value of current block by using the information of extracting.Decoding unit obtains in the deviant of current block and the residual value between predicted value, and can come with the residual value that obtains and predicted value the deviant of reconstruct current block.In the situation of the deviant of reconstruct current block, can use as required the flag information (IC_flag) of the luminance compensation that indicates whether to carry out current block.

Decoding unit obtains to indicate whether to carry out the flag information of the luminance compensation of current block at step S291.If carry out luminance compensation according to above-mentioned flag information (IC_flag), can be in the deviant of the current block of the difference of the average pixel value of step S292 reconstruct indication between current block and reference block.

Yet if utilize two reference blocks to come the predictive coding current block, decoder can not Direct Recognition goes out the deviant corresponding to each reference block, because it uses the average pixel value of two reference blocks when obtaining the deviant of current block.Therefore, according to the first example, obtain the deviant corresponding to each reference block, result has realized correct Prediction.Therefore, if utilize two reference blocks to come the predictive coding current block, step S293 can by obtain with above-mentioned deviant corresponding to each reference block deviant, as represented by following equation 22:

[equation 22]

IC_offset＝m _c-w ₁×m _r，1-w ₂×m _r，2

IC_offsetL0＝m _c-m _r，1＝IC_offset+(w ₁-1)×m _r，1+w ₂×m _r，2

IC_offsetL1＝m _c-m _r，2＝IC_offset+w ₁×m _r，1+(w ₂-1)×m _r，2

In equation 22, m _cIt is the average pixel value of current block.m _{R, 1}And m _{R, 2}The average pixel value that represents respectively reference block, w ₁And w ₂Expression is used for the weight coefficient of two predictive coding processes respectively.

The method of stating is in the use carried out in the situation of luminance compensation, and this system obtains the accurate deviant corresponding to each reference block independently, thereby it can carry out the predictive coding process more exactly.In the situation of the deviant of reconstruct current block, system is with residual value and the addition of predicted value value of reconstruct, thereby its obtains deviant.In this case, the predicted value of List0 and the predicted value of List1 is obtained and the combination, thereby system can obtain the predicted value value for the deviant of reconstruct current block.

Figure 19 illustrates the flow chart that the flag information that indicates whether to carry out the luminance compensation of current block for use is carried out the method for luminance compensation.

The luminance compensation technology is applicable to compensate for brightness difference or color distortion.If the scope of expansion luminance compensation technology also can be in the luminance compensation technology of the application extension between sequence that obtains that is captured by same camera.The luminance compensation technology can prevent brightness or color distortion appreciable impact estimation.Yet in fact, cataloged procedure adopts the flag information that indicates whether to carry out luminance compensation.The range of application of luminance compensation can be extended to sequence, view, GOP (image sets), image, band, macro block and sub-block etc.

If the luminance compensation technology is applied to the small size zone, also can control regional area, yet, should be noted that, consume the many figure places that are used for flag information.May not need the luminance compensation technology.Therefore, indicate whether that the flag bit of carrying out luminance compensation is assigned to regional, thereby this system can use the luminance compensation technology effectively.This system obtains can allow a specific order of vision signal is carried out the flag information of luminance compensation at step S201.

For example, following flag information can be assigned to regional." seq_IC_flag " information is assigned to sequence-level, " view_IC_flag " information is assigned to the view level, " GOP_IC_flag " information is assigned to the GOP level, " pic_IC_flag " information is assigned to image level, " slice_IC_flag " information is assigned to slice level, " mb_IC_flag " information is assigned to macro-block level, and " blk_IC_flag " information is assigned to the piece level.With reference to Figure 20 A-20C, the detailed description of above-mentioned flag information is described.Can decode to a specific order that utilizes flag information to carry out the vision signal of luminance compensation at step S302.

Figure 20 A-20C is the concept diagram that the scope of flag information is shown, and this flag information indicates whether to carry out the luminance compensation of current block.

With reference to figure 20A-20C, indicate whether that the flag information of carrying out luminance compensation can be classified with layered mode.For example, as from Figure 20 A-20C as seen, " seq_IC_flag " information 311 is assigned to sequence-level, " view_IC_flag " information 312 is assigned to the view level, " GOP_IC_flag " information 313 is assigned to the GOP level, and " pic_IC_flag " information 314 is assigned to image level, and " slice_IC_flag " information 315 is assigned to slice level, " mb_IC_flag " information 316 is assigned to macro-block level, and " blk_IC_flag " information 317 is assigned to the piece level.

In this case, each sign consists of by 1 position.The number of above-mentioned sign can be set as at least one.Above-mentioned sequence/view/image/slice level sign can be positioned at corresponding parameter set or head place, perhaps also can be positioned at another parameter set place.For example, " seq_IC_flag " information 311 can be positioned at the sequence parameter set place, " view_IC_flag " information 312 can be positioned at the view parameter set place, " pic_IC_flag " information 314 can be positioned at the picture parameter set place, and " slice_IC_flag " information 315 can be positioned at band head place.

If have two or more signs, the customizing messages that indicates whether to carry out higher level's luminance compensation can be controlled the luminance compensation of whether carrying out subordinate.In other words, if each sign place value all is set as " 1 ", the luminance compensation technology can be applied to subordinate.

For example, if " pic_IC_flag " information is set as " 1 ", " slice_IC_flag " information of each band that comprises in the image of correspondence can be set as " 1 " or " 0 ", " mb_IC_flag " information of each macro block can be set as " 1 " or " 0 ", and perhaps " blk_IC_flag " information of each piece can be set as " 1 " or " 0 ".If under the condition of view parameter set, " seq_IC_flag " information is set as " 1 " existing, " view_IC_flag " value of each view can be set as " 1 " or " 0 ".Otherwise if " view_IC_flag " information is set as " 1 ", the sign place value of GOP, image, band, macro block or the piece of corresponding view can be set as " 1 " or " 0 ", as shown in Figure 20 A.Obviously, as required, the above-mentioned sign place value of GOP, image, band, macro block or the piece of corresponding view can not be set as " 1 " or " 0 ".If the above-mentioned sign place value of GOP, image, band, macro block or the piece of corresponding view can not be set as " 1 " or " 0 ", this represents GOP sign, logos, band sign, macro block sign or block mark be can't help the view flag information and are controlled, as shown in Figure 20 B.

If the sign place value of upper scope is set as " 0 ", the sign place value of lower scope is made as " 0 " automatically.For example, if " seq_IC_flag " information is set as " 0 ", this indication luminance compensation technology is not applied to corresponding sequence.Therefore, " view_IC_flag " information is set as " 0 ", " GOP_IC_flag " information is set as " 0 ", " pic_IC_flag " information is set as " 0 ", " slice_IC_flag " information is set as " 0 ", " mb_IC_flag " information is set as " 0 ", and " blk_IC_flag " information is set as " 0 ".If necessary, according to the specific implementation method of luminance compensation technology, only can adopt " mb_IC_flag " information or " blk_IC_flag " information only.If necessary, when view parameter set newly is applied to multi-view video coding, can adopt " view_IC_flag " information.According to as the macro block of lowermost level unit or the sign place value of sub-block, the deviant of current block can be by coding/decoding additionally.

As from Figure 20 C as seen, indicate the sign of IC technology application also can be applied to slice level and macro-block level.For example, if " slice_IC_flag " information is set as " 0 ", this indication IC technology is not applied to corresponding band.If " slice_IC_flag " information is set as " 1 ", this indication IC technology is applied to corresponding band.In this case, if " mb_IC_flag " information is set as " 1 ", " IC_offset " information of corresponding macro block is reconstructed.If " mb_IC_flag " information is set as " 0 ", this indication IC technology is not applied to corresponding macro block.

According to another example, if be confirmed as "True" higher than the higher level's of macro-block level flag information, this system can obtain to indicate the deviant of current block of the difference of the average pixel value between current block and reference block.In this case, as required, the flag information of the flag information of macro-block level or piece level can not be used.The luminance compensation technology can service marking information indicate whether each piece is carried out luminance compensation.The luminance compensation technology also can with particular value for example motion vector indicate whether each piece is carried out luminance compensation.Above-mentioned example also can be applied to the various application of luminance compensation technology.With upper scope (being sequence, view, GOP and image) explicitly, above-mentioned example can service marking information indicates whether lower scope is carried out luminance compensation.Can use deviant effectively to indicate whether to carry out luminance compensation and service marking position not as the macro block of lowest range or piece level.Be similar to the method for using motion vector, can carry out the predictive coding process.For example, if the predictive coding process is applied to current block, the deviant of adjacent block is assigned to the deviant of current block.If the predictive coding scheme is confirmed as two predictive coding schemes, obtain the deviant of each reference block by calculating the reference block that detects from List0 and List1.Therefore, in the situation of deviant of coding current block, do not utilize the deviant of adjacent block to the deviant direct coding of each reference block, and residual value is encoded/decode.The method that is used for the prediction drift value can be confirmed as above-mentioned skew Forecasting Methodology or be used for the method that acquisition is used for the intermediate value of motion vectors.In the situation of bi-directional predicted Direct Model, do not have the method identical with motion vector of use to carry out coding/decoding to side information, and can utilize predetermined information to obtain deviant.

According to another example, use decoding unit (for example based on H.264 decoding unit) to substitute the MVC decoding unit.Should utilize decode view sequence with traditional decoding unit compatibility of traditional decoding unit, thereby " view_IC_flag " information is set as " vacation " or " 0 ".In this case, need basis for interpretation view concept.Should be noted that may need and the single view sequence of decoder compatibility H.264/AVC.Therefore, at least one view that can independently be decoded is defined and is called basic view.Reference-view in this basis view table registration view (being many views).Utilize ordinary video encoding scheme (such as MPEG-2, MPEG-4, H.263 and H.264 wait) that the sequence corresponding to basic view in the MVC scheme is encoded, thereby it produce with the form of individual bit stream.Above-mentioned basic view sequence can be compatible with scheme H.264/AVC, perhaps can not be compatible with it.Yet, can always be set as basic view with the view sequence of scheme compatibility H.264/AVC.

Figure 21 illustrates the flow chart that obtains the method for motion vector be used to the deviant of considering current block.

With reference to Figure 21, this system can obtain at step S321 the deviant of current block.This system uses the reference block of this deviant search and current block optimum Match at step S322.This system obtains motion vector at step S323 from reference block, and encoding motion vector.For carrying out luminance compensation, during estimation, various factors is taken in.For example, in the situation that is used for coming by the average pixel value that is offset first and second first of comparison and second method, infer the average pixel value of these two pieces from the pixel value of each piece during estimation, thereby can calculate the similitude between these two pieces.In this case, the deviant between these two pieces is by absolute coding, thereby reflects the cost for absolute coding in motion estimation process.Can calculate traditional cost by following equation 23:

[equation 23]

COST＝SAD+λ _MOTION·GenBit

In the situation of using luminance compensation, SAD (summation of poor absolute value) can be represented by following equation 24:

[equation 24]

$\mathrm{SAD}=\underset{\mathrm{ij}}{\mathrm{\Σ}}|(I_c(m,n)-M_c)-(I_r(m,n)-M_r)|$

In equation 24, I _cThe pixel value of expression current block, and I _rThe pixel value of expression reference block.M _cThe average pixel value of expression current block, and M _rThe average pixel value of expression reference block.Shift into and originally can be included in above-mentioned SAD computational process, as represented by following equation 25 and 26:

[equation 25]

COS _IC＝SAD _IC+λ _MOTION·GenBit

[equation 26]

SAD _IC＝α|offset-offset_pred|+∑|(I _c(m，n)-M _c)-(I _r(m，n)-M _r)|

Represent weight coefficient with reference to equation 25 and 26, α.If the value of α is set as " 1 ", reflect the absolute value of deviant.For the another kind of method that is used for reflection luminance compensation cost, have and a kind ofly be used for by prediction the method that the needed figure place of code offset value reflects the luminance compensation cost.Following equation 27 a kind of methods for the prediction drift bits of coded of expression.In this case, can with the predictive coding position, ground that is in proportion of skew residual value.

[equation 27]

GenBit _IC＝GenBit+Bit _IC

In this case, can calculate new cost by following equation 28:

[equation 28]

Cost＝SAD+λ _MOTION·GenBit _IC

Claims (3)

1. one kind is used for method that the multi-view video signal is decoded, comprising:

Reception comprises the bit stream of a plurality of views of described multi-view video signal;

Extract the first flag information that is associated with the part of described multi-view video signal from described bit stream, this first flag information indicates whether can be to the described part execution luminance compensation of described multi-view video signal, and wherein said part comprises one or more bands; And

For carrying out the part of luminance compensation according to described the first flag information, from the second flag information that described bitstream extraction is associated with macro block in this part, described the second flag information indication is to described macro block execution luminance compensation; And

For will carry out with indication the macro block that described second flag information of luminance compensation is associated, the deviant that at least one adjacent block that is adjacent to described macro block is carried out luminance compensation is obtained for the predicted value to the luminance compensation of described macro block with being used for,

The residual value that is used for the luminance compensation of described macro block from described bitstream extraction, and

Comprise for the summation of the residual value of the luminance compensation of the predicted value of the luminance compensation of described macro block and macro block by formation, obtain the deviant for the luminance compensation of macro block,

The deviant of wherein said luminance compensation for macro block is the difference between the average pixel value of the average pixel value of described macro block and reference block, and described reference block is quoted by described macro block.

2. method according to claim 1 is wherein according to selecting described adjacent block adjacent to the predesigned order in the described adjacent block of described macro block.

3. whether method according to claim 2, wherein, satisfy one or more condition and select described adjacent block based on adjacent block in the following sequence, and upper adjacent block is left adjacent block subsequently, is the upper right adjacent block subsequently, is the upper left adjacent block subsequently.

Images