USRE44680E1 - Processing multiview video - Google Patents

Processing multiview video Download PDF

Info

Publication number
USRE44680E1
USRE44680E1 US13/720,479 US201213720479A USRE44680E US RE44680 E1 USRE44680 E1 US RE44680E1 US 201213720479 A US201213720479 A US 201213720479A US RE44680 E USRE44680 E US RE44680E
Authority
US
United States
Prior art keywords
view
inter
video data
information
prediction
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active, expires
Application number
US13/720,479
Inventor
Jeong Hyu Yang
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
LG Electronics Inc
Original Assignee
LG Electronics Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from KR1020060037773A external-priority patent/KR20070076356A/en
Priority claimed from KR1020060110337A external-priority patent/KR20070076391A/en
Priority claimed from KR1020060110338A external-priority patent/KR20070076392A/en
Priority claimed from US11/622,592 external-priority patent/US7856148B2/en
Application filed by LG Electronics Inc filed Critical LG Electronics Inc
Priority to US13/720,479 priority Critical patent/USRE44680E1/en
Assigned to LG ELECTRONICS INC. reassignment LG ELECTRONICS INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: YANG, JEONG HYU
Application granted granted Critical
Publication of USRE44680E1 publication Critical patent/USRE44680E1/en
Active legal-status Critical Current
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/187Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a scalable video layer
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103Selection of coding mode or of prediction mode
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103Selection of coding mode or of prediction mode
    • H04N19/105Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136Incoming video signal characteristics or properties
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/189Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding
    • H04N19/196Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding being specially adapted for the computation of encoding parameters, e.g. by averaging previously computed encoding parameters
    • H04N19/197Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding being specially adapted for the computation of encoding parameters, e.g. by averaging previously computed encoding parameters including determination of the initial value of an encoding parameter
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/46Embedding additional information in the video signal during the compression process
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/597Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/61Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding

Definitions

  • the invention relates to processing multiview video.
  • Multiview Video Coding relates to compression of video sequences (e.g., a sequence of images or “pictures”) that are typically acquired by respective cameras.
  • the video sequences or “views” can be encoded according to a standard such as MPEG.
  • a picture in a video sequence can represent a full video frame or a field of a video frame.
  • a slice is an independently coded portion of a picture that includes some or all of the macroblocks in the picture, and a macroblock includes blocks of picture elements (or “pixels”).
  • the video sequences can be encoded as a multiview video sequence according to the H.264/AVC codec technology, and many developers are conducting research into amendment of standards to accomodate multiview video sequences.
  • the term “profile” indicates the standardization of technical components for use in the video encoding/decoding algorithms.
  • the profile is the set of technical components prescribed for decoding a bitstream of a compressed sequence, and may be considered to be a sub-standard.
  • the above-mentioned three profiles are a baseline profile, a main profile, and an extended profile.
  • a variety of functions for the encoder and the decoder have been defined in the H.264 standard, such that the encoder and the decoder can be compatible with the baseline profile, the main profile, and the extended profile respectively.
  • the bitstream for the H.264/AVC standard is structured according to a Video Coding Layer (VCL) for processing the moving-image coding (i.e., the sequence coding), and a Network Abstraction Layer (NAL) associated with a subsystem capable of transmitting/storing encoded information.
  • VCL Video Coding Layer
  • NAL Network Abstraction Layer
  • the output data of the encoding process is VCL data, and is mapped into NAL units before it is transmitted or stored.
  • Each NAL unit includes a Raw Byte Sequence Payload (RBSP) corresponding to either compressed video data or header information.
  • RBSP Raw Byte Sequence Payload
  • the NAL unit includes a NAL header and a RBSP.
  • the NAL header includes flag information (e.g., nal_ref_idc) and identification (ID) information (e.g., nal_unit_type).
  • the flag information “nal_ref_idc” indicates the presence or absence of a slice used as a reference picture of the NAL unit.
  • the ID information “nal_unit_type” indicates the type of the NAL unit.
  • the RBSP stores compressed original data. An RBSP trailing bit can be added to the last part of the RBSP, such that the length of the RBSP can be represented by a multiple of 8 bits.
  • NAL units for example, an Instantaneous Decoding Refresh (IDR) picture, a Sequence Parameter Set (SPS), a Picture Parameter Set (PPS), and Supplemental Enhancement Information (SEI), etc.
  • IDR Instantaneous Decoding Refresh
  • SPS Sequence Parameter Set
  • PPS Picture Parameter Set
  • SEI Supplemental Enhancement Information
  • the standard has generally defined a target product using various profiles and levels, such that the target product can be implemented with appropriate costs.
  • the decoder satisfies a predetermined constraint at a corresponding profile and level.
  • the profile and the level are able to indicate a function or parameter of the decoder, such that they indicate which compressed images can be handled by the decoder.
  • Specific information indicating which one of multiple profiles corresponds to the bitstream can be identified by profile ID information.
  • the profile ID information “profile_idc” provides a flag for identifying a profile associated with the bitstream.
  • the H.264/AVC standard includes three profile identifiers (IDs). If the profile ID information “profile_idc” is set to “66”, the bitstream is based on the baseline profile. If the profile ID information “profile_idc” is set to “77”, the bitstream is based on the main profile. If the profile ID information “profile_idc” is set to “88”, the bitstream is based on the extended profile.
  • the above-mentioned “profile_idc” information may be contained in the SPS (Sequence Parameter Set), for example.
  • a method for decoding a video signal comprises: receiving a bitstream comprising the video signal encoded according to a first profile that represents a selection from a set of multiple profiles that includes at least one profile for a multiview video signal, and profile information that identifies the first profile; extracting the profile information from the bitstream; and decoding the video signal according to the determined profile using illumination compensation between segments of pictures in respective views when the determined profile corresponds to a multiview video signal with each of multiple views comprising multiple pictures segmented into multiple segments (e.g., an image block segment such as a single block or a macroblock, or a segment such as a slice of an image).
  • aspects can include one or more of the following features.
  • the method further comprises extracting from the bitstream configuration information associated with multiple views when the determined profile corresponds to a multiview video signal, wherein the configuration information comprises at least one of view-dependency information representing dependency relationships between respective views, view identification information indicating a reference view, view-number information indicating the number of views, view level information for providing view scalability, and view-arrangement information indicating a camera arrangement.
  • the profile information is located in a header of the bitstream.
  • the view level information corresponds to one of a plurality of levels associated with a hierachical view prediction structure among the views of the multiview video signal.
  • the view-dependency information represents the dependency relationships in a two-dimensional data structure.
  • the two-dimensional data structure comprises a matrix.
  • the segments comprise image blocks.
  • Using illumination compensation for a first segment comprises obtaining an offset value for illumination compensation of a neighboring block by forming a sum that includes a predictor for illumination compensation of the neighboring block and a residual value.
  • the method further comprises selecting at least one neighboring block based on whether one or more conditions are satisfied for a neighboring block in an order in which one or more vertical or horizontal neighbors are followed by one or more diagonal neighbors.
  • Selecting at least one neighboring block comprises determining whether one or more conditions are satisfied for a neighboring block in the order of: a left neighboring block, followed by an upper neighboring block, followed by a right-upper neighboring block, followed by a left-upper neighboring block.
  • Determining whether one or more conditions are satisfied for a neighboring block comprises extracting a value associated with the neighboring block from the bitstream indicating whether illumination compensation of the neighboring block is to be performed.
  • Selecting at least one neighboring block comprises determining whether to use an offset value for illumination compensation of a single neighboring block or multiple offset values for illumination compensation of respective neighboring blocks.
  • a method for decoding a multiview video signal comprises: receiving a bitstream comprising the multiview video signal encoded according to dependency relationships between respective views, and view-dependency data representing the dependency relationships; extracting the view-dependency data and determining the dependency relationships from the extracted data; and decoding the multiview video signal according to the determined dependency relationships using illumination compensation between segments of pictures in respective views, where the multiview video signal includes multiple views each comprising multiple pictures segmented into multiple segments.
  • aspects can include one or more of the following features.
  • the view-dependency data represents the dependency relationships in a two-dimensional data structure.
  • the view-dependency data comprises a matrix.
  • the method further comprises extracting from the bit-stream configuration information comprising at least one of view identification information indicating a reference view, view-number information indicating the number of views, view level information for providing view scalability, and view-arrangement information indicating a camera arrangement.
  • the segments comprise image blocks.
  • Using illumination compensation for a first segment comprises obtaining an offset value for illumination compensation of a neighboring block by forming a sum that includes a predictor for illumination compensation of the neighboring block and a residual value.
  • the method further comprises selecting at least one neighboring block based on whether one or more conditions are satisfied for a neighboring block in an order in which one or more vertical or horizontal neighbors are followed by one or more diagonal neighbors.
  • Selecting at least one neighboring block comprises determining whether one or more conditions are satisfied for a neighboring block in the order of: a left neighboring block, followed by an upper neighboring block, followed by a right-upper neighboring block, followed by a left-upper neighboring block.
  • Determining whether one or more conditions are satisfied for a neighboring block comprises extracting a value associated with the neighboring block from the bitstream indicating whether illumination compensation of the neighboring block is to be performed.
  • Selecting at least one neighboring block comprises determining whether to use an offset value for illumination compensation of a single neighboring block or multiple offset values for illumination compensation of respective neighboring blocks.
  • the method further comprises, when multiple offset values are to be used, obtaining the predictor for performing illumination compensation of the first block by combining the multiple offset values.
  • Combining the multiple offset values comprises taking an average or median of the offset values.
  • a method for encoding a video signal comprises generating a bitstream capable of being decoded into the video signal by the respective decoding method.
  • a method for encoding a bitstream comprises: forming the bitstream according to a first profile that represents a selection from a set of multiple profiles that includes at least one profile for a multiview video signal, and profile information that identifies the first profile; and providing information for illumination compensation between segments of pictures in respective views when the determined profile corresponds to a multiview video signal with each of multiple views comprising multiple pictures segmented into multiple segments.
  • a method for encoding a bitstream comprises: forming the bitstream according to dependency relationships between respective views, and view-dependency data representing the dependency relationships; and providing information for illumination compensation between segments of pictures in respective views when the determined profile corresponds to a multiview video signal with each of multiple views comprising multiple pictures segmented into multiple segments.
  • a computer program stored on a computer-readable medium, comprises instructions for causing a computer to perform the respective decoding method.
  • image data embodied on a machine-readable information carrier is capable of being decoded into a video signal by the respective decoding method.
  • a decoder for each respective decoding method, comprises means for performing the respective decoding method.
  • an encoder comprises means for generating a bitstream capable of being decoded into a video signal by the respective decoding method.
  • FIG. 1 is an exemplary decoding apparatus.
  • an input bitstream includes information that allows a decoding apparatus to determine whether the input bitstream relates to a multiview profile.
  • supplementary information associated with the multiview sequence is added according to a syntax to the bitstream and transmitted to the decoder.
  • the multiview profile ID can indicate a profile mode for handling multiview video data as according to an amendment of the H.264/AVC standard.
  • the MVC (Multiview Video Coding) technology is an amendment technology of the H.264/AVC standards. That is, a specific syntax is added as supplementary information for an MVC mode. Such amendment to support MVC technology can be more effective than an alternative in which an unconditional syntax is used. For example, if the profile identifier of the AVC technology is indicative of a multiview profile, the addition of multiview sequence information may increase a coding efficiency.
  • the sequence parameter set (SPS) of an H.264/AVC bitstream is indicative of header information including information (e.g., a profile, and a level) associated with the entire-sequence encoding.
  • the entire compressed moving images (i.e., a sequence) can begin at a sequence header, such that a sequence parameter set (SPS) corresponding to the header information arrives at the decoder earlier than data referred to by the parameter set.
  • SPS sequence parameter set
  • the sequence parameter set RBSP acts as header information of a compressed data of moving images at entry S 1 ( FIG. 1 ). If the bitstream is received, the profile ID information “profile_idc” identifies which one of profiles from among several profiles corresponds to the received bitstream.
  • the profile ID information “profile_idc” can be set, for example, to “MULTI_VIEW_PROFILE)”, so that the syntax including the profile ID information can determine whether the received bitstream relates to a multiview profile.
  • the following configuration information can be added when the received bitstream relates to the multiview profile.
  • FIG. 1 is a block diagram illustrating an exemplary decoding apparatus (or “decoder”) of a multiview video system for decoding a video signal containing a multiview video sequence.
  • the multiview video system includes a corresponding encoding apparatus (or “encoder”) to provide the multiview video sequence as a bitstream that includes encoded image data embodied on a machine-readable information carrier (e.g., a machine-readable storage medium, or a machine-readable energy signal propagating between a transmitter and receiver.)
  • a machine-readable information carrier e.g., a machine-readable storage medium, or a machine-readable energy signal propagating between a transmitter and receiver.
  • the decoding apparatus includes a parsing unit 10 , an entropy decoding unit 11 , an Inverse Quantization/Inverse Transform unit 12 , an inter-prediction unit 13 , an intra-prediction unit 14 , a deblocking filter 15 , and a decoded-picture buffer 16 .
  • the inter-prediction unit 13 includes a motion compensation unit 17 , an illumination compensation unit 18 , and an illumination-compensation offset prediction unit 19 .
  • the parsing unit 10 performs a parsing of the received video sequence in NAL units to decode the received video sequence.
  • one or more sequence parameter sets and picture parameter sets are transmitted to a decoder before a slice header and slice data are decoded.
  • the NAL header or an extended area of the NAL header may include a variety of configuration information, for example, temporal level information, view level information, anchor picture ID information, and view ID information, etc.
  • time level information is indicative of hierarchical-structure information for providing temporal scalability from a video signal, such that sequences of a variety of time zones can be provided to a user via the above-mentioned temporal level information.
  • view level information is indicative of hierarchical-structure information for providing view scalability from the video signal.
  • the multiview video sequence can define the temporal level and view level, such that a variety of temporal sequences and view sequences can be provided to the user according to the defined temporal level and view level.
  • the user may employ the temporal scalability and the view scalability. Therefore, the user can view a sequence corresponding to a desired time and view, or can view a sequence corresponding to another limitation.
  • the above-mentioned level information may also be established in various ways according to reference conditions. For example, the level information may be changed according to a camera location, and may also be changed according to a camera arrangement type. In addition, the level information may also be arbitrarily established without a special reference.
  • anchor picture is indicative of an encoded picture in which all slices refer to only slices in a current view and not slices in other views. A random access between views can be used for multiview-sequence decoding.
  • Anchor picture ID information can be used to perform the random access process to access data of a specific view without requiring a large amount of data to be decoded.
  • view ID information is indicative of specific information for discriminating between a picture of a current view and a picture of another view.
  • a Picture Order Count (POC) and frame number information (frame_num) can be used.
  • inter-view prediction can be performed.
  • An identifier is used to discriminate a picture of the current view from a picture of another view.
  • a view identifier can be defined to indicate a picture's view.
  • the decoding apparatus can obtain information of a picture in a view different from a view of the current picture using the above-mentioned view identifier, such that it can decode the video signal using the information of the picture.
  • the above-mentioned view identifier can be applied to the overall encoding/decoding process of the video signal. Also, the above-mentioned view identifier can also be applied to the multiview video coding process using the frame number information “frame_num” considering a view.
  • the multiview sequence has a large amount of data, and a hierarchical encoding function of each view (also called a “view scalability”) can be used for processing the large amount of data.
  • a prediction structure considering views of the multiview sequence may be defined.
  • the above-mentioned prediction structure may be defined by structuralizing the prediction order or direction of several view sequences. For example, if several view sequences to be encoded are given, a center location of the overall arrangement is set to a base view, such that view sequences to be encoded can be hierarchically selected. The end of the overall arrangement or other parts may be set to the base view.
  • the number of camera views is denoted by an exponential power of “2”
  • a hierarchical prediction structure between several view sequences may be formed on the basis of the above-mentioned case of the camera views denoted by the exponential power of “2”. Otherwise, if the number of camera views is not denoted by the exponential power of “2”, virtual views can be used, and the prediction structure may be formed on the basis of the virtual views. If the camera arrangement is indicative of a two-dimensional arrangement, the prediction order may be established by turns in a horizontal or vertical direction.
  • a parsed bitstream is entropy-decoded by an entropy decoding unit 11 , and data such as a coefficient of each macroblock, a motion vector, etc., are extracted.
  • the inverse quantization/inverse transform unit 12 multiplies a received quantization value by a predetermined constant to acquire a transformed coefficient value, and performs an inverse transform of the acquired coefficient value, such that it reconstructs a pixel value.
  • the inter-prediction unit 13 performs an inter-prediction function from decoded samples of the current picture using the reconstructed pixel value.
  • the deblocking filter 15 is applied to each decoded macroblock to reduce the degree of block distortion.
  • the deblocking filter 15 performs a smoothing of the block edge, such that it improves an image quality of the decoded frame.
  • the selection of a filtering process is dependent on a boundary strength and a gradient of image samples arranged in the vicinity of the boundary.
  • the filtered pictures are stored in the decoded picture buffer 16 , such that they can be outputted or be used as reference pictures.
  • the decoded picture buffer 16 stores or outputs pre-coded pictures to perform the inter-prediction function.
  • frame number information “frame_num” and POC (Picture Order Count) information of the pictures are used to store or output the pre-coded pictures.
  • Pictures of other view may exist in the above-mentioned pre-coded pictures in the case of the MVC technology. Therefore, in order to use the above-mentioned pictures as reference pictures, not only the “frame_num” and POC information, but also view identifier indicating a picture view may be used as necessary.
  • the inter-prediction unit 13 performs the inter-prediction using the reference pictures stored in the decoded picture buffer 16 .
  • the inter-coded macroblock may be divided into macroblock partitions. Each macroblock partition can be predicted by one or two reference pictures.
  • the motion compensation unit 17 compensates for a motion of the current block using the information received from the entropy decoding unit 11 .
  • the motion compensation unit 17 extracts motion vectors of neighboring blocks of the current block from the video signal, and obtains a motion-vector predictor of the current block.
  • the motion compensation unit 17 compensates for the motion of the current block using a difference value between the motion vector and a predictor extracted from the video signal and the obtained motion-vector predictor.
  • the above-mentioned motion compensation may be performed by only one reference picture, or may also be performed by a plurality of reference pictures.
  • the motion compensation may be performed according to a view identifier indicating the other views.
  • a direct mode is indicative of a coding mode for predicting motion information of the current block on the basis of the motion information of a block which is completely decoded.
  • the above-mentioned direct mode can reduce the number of bits required for encoding the motion information, resulting in the increased compression efficiency.
  • a temporal direct mode predicts motion information of the current block using a correlation of motion information of a temporal direction. Similar to the temporal direct mode, the decoder can predict the motion information of the current block using a correlation of motion information of a view direction.
  • view sequences may be captured by different cameras respectively, such that a difference in illumination may occur due to internal or external factors of the cameras.
  • an illumination compensation unit 18 performs an illumination compensation function.
  • flag information may be used to indicate whether an illumination compensation at a specific level of a video signal is performed.
  • the illumination compensation unit 18 may perform the illumination compensation function using flag information indicating whether the illumination compensation of a corresponding slice or macroblock is performed.
  • the above-mentioned method for performing the illumination compensation using the above-mentioned flag information may be applied to a variety of macroblock types (e.g., an inter 16 ⁇ 16 mode, a B-skip mode, a direct mode, etc.)
  • information of a neighboring block or information of a block in views different from a view of the current block may be used, and an offset value of the current block may also be used.
  • the offset value of the current block is indicative of a difference value between an average pixel value of the current block and an average pixel value of a reference block corresponding to the current block.
  • a predictor of the current-block offset value may be obtained by using the neighboring blocks of the current block, and a residual value between the offset value and the predictor may be used. Therefore, the decoder can reconstruct the offset value of the current block using the residual value and the predictor.
  • the offset value of the current block can be predicted by using the offset value of a neighboring block. Prior to predicting the current-block offset value, it is determined whether the reference index of the current block is equal to a reference index of the neighboring blocks. According to the determined result, the illumination compensation unit 18 can determine which one of neighboring blocks will be used or which value will be used.
  • the illumination compensation unit 18 may perform the illumination compensation using a prediction type of the current block. If the current block is predictively encoded by two reference blocks, the illumination compensation unit 18 may obtain an offset value corresponding to each reference block using the offset value of the current block.
  • the inter-predicted pictures or intra-predicted pictures acquired by the illumination compensation and motion compensation are selected according to a prediction mode, and reconstructs the current picture.

Abstract

Decoding a video signal comprises receiving a bitstream comprising the multiview video signal encoded according to dependency relationships between respective views, and view-dependency data representing the dependency relationships; extracting the view-dependency data and determining the dependency relationships from the extracted data; and decoding the multiview video signal according to the determined dependency relationships using illumination compensation between segments of pictures in respective views, where the multiview video signal includes multiple views each comprising multiple pictures segmented into multiple segments.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS
This application claims the benefit of U.S. Application Ser. No. 60/758,234 filed on Jan. 12, 2006, U.S. Application Ser. No. 60/759,620 filed on Jan. 18, 2006, U.S. Application Ser. No. 60/762,534 filed on Jan. 27, 2006, U.S. Application Ser. No. 60/787,193 filed on Mar. 30, 2006, U.S. Application Ser. No. 60/818,274 filed on Jul. 5, 2006, U.S. Application Ser. No. 60/830,087 filed on Jul. 12, 2006, U.S. Application Ser. No. 60/830,328 filed on Jul. 13, 2006, Korean Application No. 10-2006-0004956 filed on Jan. 17, 2006, Korean Application No. 10-2006-0027100 filed on Mar. 24, 2006, Korean Application No. 10-2006-0037773 filed on Apr. 26, 2006, Korean Application No. 10-2006-0110337 filed on Nov. 9, 2006, and Korean Application No. 10-2006-0110338 filed on Nov. 9, 2006, each of which is incorporated herein by reference.
This application is related to U.S. application Ser. No. 11/622,591 titled “PROCESSING MULTIVIEW VIDEO”, U.S. application Ser. No. 11/622,611 titled “PROCESSING MULTIVIEW VIDEO”, U.S. application Ser. No. 11/622,618 titled “PROCESSING MULTIVIEW VIDEO”, U.S. application Ser. No. 11/622,709 titled “PROCESSING MULTIVIEW VIDEO”, U.S. application Ser. No. 11/622,675 titled “PROCESSING MULTIVIEW VIDEO”, U.S. application Ser. No. 11/622,803 titled “PROCESSING MULTIVIEW VIDEO”, and U.S. application Ser. No. 11/622,681 titled “PROCESSING MULTIVIEW VIDEO”, each of which is being filed concurrently with the present application, and each of which is also incorporated herein by reference.
BACKGROUND
The invention relates to processing multiview video.
Multiview Video Coding (MVC) relates to compression of video sequences (e.g., a sequence of images or “pictures”) that are typically acquired by respective cameras. The video sequences or “views” can be encoded according to a standard such as MPEG. A picture in a video sequence can represent a full video frame or a field of a video frame. A slice is an independently coded portion of a picture that includes some or all of the macroblocks in the picture, and a macroblock includes blocks of picture elements (or “pixels”).
The video sequences can be encoded as a multiview video sequence according to the H.264/AVC codec technology, and many developers are conducting research into amendment of standards to accomodate multiview video sequences.
Three profiles for supporting specific functions are prescribed in the current H.264 standard. The term “profile” indicates the standardization of technical components for use in the video encoding/decoding algorithms. In other words, the profile is the set of technical components prescribed for decoding a bitstream of a compressed sequence, and may be considered to be a sub-standard. The above-mentioned three profiles are a baseline profile, a main profile, and an extended profile. A variety of functions for the encoder and the decoder have been defined in the H.264 standard, such that the encoder and the decoder can be compatible with the baseline profile, the main profile, and the extended profile respectively.
The bitstream for the H.264/AVC standard is structured according to a Video Coding Layer (VCL) for processing the moving-image coding (i.e., the sequence coding), and a Network Abstraction Layer (NAL) associated with a subsystem capable of transmitting/storing encoded information. The output data of the encoding process is VCL data, and is mapped into NAL units before it is transmitted or stored. Each NAL unit includes a Raw Byte Sequence Payload (RBSP) corresponding to either compressed video data or header information.
The NAL unit includes a NAL header and a RBSP. The NAL header includes flag information (e.g., nal_ref_idc) and identification (ID) information (e.g., nal_unit_type). The flag information “nal_ref_idc” indicates the presence or absence of a slice used as a reference picture of the NAL unit. The ID information “nal_unit_type” indicates the type of the NAL unit. The RBSP stores compressed original data. An RBSP trailing bit can be added to the last part of the RBSP, such that the length of the RBSP can be represented by a multiple of 8 bits.
There are a variety of the NAL units, for example, an Instantaneous Decoding Refresh (IDR) picture, a Sequence Parameter Set (SPS), a Picture Parameter Set (PPS), and Supplemental Enhancement Information (SEI), etc.
The standard has generally defined a target product using various profiles and levels, such that the target product can be implemented with appropriate costs. The decoder satisfies a predetermined constraint at a corresponding profile and level.
The profile and the level are able to indicate a function or parameter of the decoder, such that they indicate which compressed images can be handled by the decoder. Specific information indicating which one of multiple profiles corresponds to the bitstream can be identified by profile ID information. The profile ID information “profile_idc” provides a flag for identifying a profile associated with the bitstream. The H.264/AVC standard includes three profile identifiers (IDs). If the profile ID information “profile_idc” is set to “66”, the bitstream is based on the baseline profile. If the profile ID information “profile_idc” is set to “77”, the bitstream is based on the main profile. If the profile ID information “profile_idc” is set to “88”, the bitstream is based on the extended profile. The above-mentioned “profile_idc” information may be contained in the SPS (Sequence Parameter Set), for example.
SUMMARY
In one aspect, in general, a method for decoding a video signal comprises: receiving a bitstream comprising the video signal encoded according to a first profile that represents a selection from a set of multiple profiles that includes at least one profile for a multiview video signal, and profile information that identifies the first profile; extracting the profile information from the bitstream; and decoding the video signal according to the determined profile using illumination compensation between segments of pictures in respective views when the determined profile corresponds to a multiview video signal with each of multiple views comprising multiple pictures segmented into multiple segments (e.g., an image block segment such as a single block or a macroblock, or a segment such as a slice of an image).
Aspects can include one or more of the following features.
The method further comprises extracting from the bitstream configuration information associated with multiple views when the determined profile corresponds to a multiview video signal, wherein the configuration information comprises at least one of view-dependency information representing dependency relationships between respective views, view identification information indicating a reference view, view-number information indicating the number of views, view level information for providing view scalability, and view-arrangement information indicating a camera arrangement.
The profile information is located in a header of the bitstream.
The view level information corresponds to one of a plurality of levels associated with a hierachical view prediction structure among the views of the multiview video signal.
The view-dependency information represents the dependency relationships in a two-dimensional data structure.
The two-dimensional data structure comprises a matrix.
The segments comprise image blocks.
Using illumination compensation for a first segment comprises obtaining an offset value for illumination compensation of a neighboring block by forming a sum that includes a predictor for illumination compensation of the neighboring block and a residual value.
The method further comprises selecting at least one neighboring block based on whether one or more conditions are satisfied for a neighboring block in an order in which one or more vertical or horizontal neighbors are followed by one or more diagonal neighbors.
Selecting at least one neighboring block comprises determining whether one or more conditions are satisfied for a neighboring block in the order of: a left neighboring block, followed by an upper neighboring block, followed by a right-upper neighboring block, followed by a left-upper neighboring block.
Determining whether one or more conditions are satisfied for a neighboring block comprises extracting a value associated with the neighboring block from the bitstream indicating whether illumination compensation of the neighboring block is to be performed.
Selecting at least one neighboring block comprises determining whether to use an offset value for illumination compensation of a single neighboring block or multiple offset values for illumination compensation of respective neighboring blocks.
In another aspect, in general, a method for decoding a multiview video signal comprises: receiving a bitstream comprising the multiview video signal encoded according to dependency relationships between respective views, and view-dependency data representing the dependency relationships; extracting the view-dependency data and determining the dependency relationships from the extracted data; and decoding the multiview video signal according to the determined dependency relationships using illumination compensation between segments of pictures in respective views, where the multiview video signal includes multiple views each comprising multiple pictures segmented into multiple segments.
Aspects can include one or more of the following features.
The view-dependency data represents the dependency relationships in a two-dimensional data structure.
The view-dependency data comprises a matrix.
The method further comprises extracting from the bit-stream configuration information comprising at least one of view identification information indicating a reference view, view-number information indicating the number of views, view level information for providing view scalability, and view-arrangement information indicating a camera arrangement.
The segments comprise image blocks.
Using illumination compensation for a first segment comprises obtaining an offset value for illumination compensation of a neighboring block by forming a sum that includes a predictor for illumination compensation of the neighboring block and a residual value.
The method further comprises selecting at least one neighboring block based on whether one or more conditions are satisfied for a neighboring block in an order in which one or more vertical or horizontal neighbors are followed by one or more diagonal neighbors.
Selecting at least one neighboring block comprises determining whether one or more conditions are satisfied for a neighboring block in the order of: a left neighboring block, followed by an upper neighboring block, followed by a right-upper neighboring block, followed by a left-upper neighboring block.
Determining whether one or more conditions are satisfied for a neighboring block comprises extracting a value associated with the neighboring block from the bitstream indicating whether illumination compensation of the neighboring block is to be performed.
Selecting at least one neighboring block comprises determining whether to use an offset value for illumination compensation of a single neighboring block or multiple offset values for illumination compensation of respective neighboring blocks.
The method further comprises, when multiple offset values are to be used, obtaining the predictor for performing illumination compensation of the first block by combining the multiple offset values.
Combining the multiple offset values comprises taking an average or median of the offset values.
In another aspect, in general, for each respective decoding method, a method for encoding a video signal comprises generating a bitstream capable of being decoded into the video signal by the respective decoding method. For example, in another aspect, in general, a method for encoding a bitstream comprises: forming the bitstream according to a first profile that represents a selection from a set of multiple profiles that includes at least one profile for a multiview video signal, and profile information that identifies the first profile; and providing information for illumination compensation between segments of pictures in respective views when the determined profile corresponds to a multiview video signal with each of multiple views comprising multiple pictures segmented into multiple segments. In another aspect, in general, a method for encoding a bitstream comprises: forming the bitstream according to dependency relationships between respective views, and view-dependency data representing the dependency relationships; and providing information for illumination compensation between segments of pictures in respective views when the determined profile corresponds to a multiview video signal with each of multiple views comprising multiple pictures segmented into multiple segments.
In another aspect, in general, for each respective decoding method, a computer program, stored on a computer-readable medium, comprises instructions for causing a computer to perform the respective decoding method.
In another aspect, in general, for each respective decoding method, image data embodied on a machine-readable information carrier is capable of being decoded into a video signal by the respective decoding method.
In another aspect, in general, for each respective decoding method, a decoder comprises means for performing the respective decoding method.
In another aspect, in general, for each respective decoding method, an encoder comprises means for generating a bitstream capable of being decoded into a video signal by the respective decoding method.
Other features and advantages will become apparent from the following description, and from the claims.
DESCRIPTION OF DRAWINGS
FIG. 1 is an exemplary decoding apparatus.
DESCRIPTION
In order to effectively handle a multiview sequence, an input bitstream includes information that allows a decoding apparatus to determine whether the input bitstream relates to a multiview profile. In cases that it is determined that the input bitstream relates to the multiview profile, supplementary information associated with the multiview sequence is added according to a syntax to the bitstream and transmitted to the decoder. For example, the multiview profile ID can indicate a profile mode for handling multiview video data as according to an amendment of the H.264/AVC standard.
The MVC (Multiview Video Coding) technology is an amendment technology of the H.264/AVC standards. That is, a specific syntax is added as supplementary information for an MVC mode. Such amendment to support MVC technology can be more effective than an alternative in which an unconditional syntax is used. For example, if the profile identifier of the AVC technology is indicative of a multiview profile, the addition of multiview sequence information may increase a coding efficiency.
The sequence parameter set (SPS) of an H.264/AVC bitstream is indicative of header information including information (e.g., a profile, and a level) associated with the entire-sequence encoding.
The entire compressed moving images (i.e., a sequence) can begin at a sequence header, such that a sequence parameter set (SPS) corresponding to the header information arrives at the decoder earlier than data referred to by the parameter set. As a result, the sequence parameter set RBSP acts as header information of a compressed data of moving images at entry S1 (FIG. 1). If the bitstream is received, the profile ID information “profile_idc” identifies which one of profiles from among several profiles corresponds to the received bitstream.
The profile ID information “profile_idc” can be set, for example, to “MULTI_VIEW_PROFILE)”, so that the syntax including the profile ID information can determine whether the received bitstream relates to a multiview profile. The following configuration information can be added when the received bitstream relates to the multiview profile.
FIG. 1 is a block diagram illustrating an exemplary decoding apparatus (or “decoder”) of a multiview video system for decoding a video signal containing a multiview video sequence. The multiview video system includes a corresponding encoding apparatus (or “encoder”) to provide the multiview video sequence as a bitstream that includes encoded image data embodied on a machine-readable information carrier (e.g., a machine-readable storage medium, or a machine-readable energy signal propagating between a transmitter and receiver.)
Referring to FIG. 1, the decoding apparatus includes a parsing unit 10, an entropy decoding unit 11, an Inverse Quantization/Inverse Transform unit 12, an inter-prediction unit 13, an intra-prediction unit 14, a deblocking filter 15, and a decoded-picture buffer 16.
The inter-prediction unit 13 includes a motion compensation unit 17, an illumination compensation unit 18, and an illumination-compensation offset prediction unit 19.
The parsing unit 10 performs a parsing of the received video sequence in NAL units to decode the received video sequence. Typically, one or more sequence parameter sets and picture parameter sets are transmitted to a decoder before a slice header and slice data are decoded. In this case, the NAL header or an extended area of the NAL header may include a variety of configuration information, for example, temporal level information, view level information, anchor picture ID information, and view ID information, etc.
The term “time level information” is indicative of hierarchical-structure information for providing temporal scalability from a video signal, such that sequences of a variety of time zones can be provided to a user via the above-mentioned temporal level information.
The term “view level information” is indicative of hierarchical-structure information for providing view scalability from the video signal. The multiview video sequence can define the temporal level and view level, such that a variety of temporal sequences and view sequences can be provided to the user according to the defined temporal level and view level.
In this way, if the level information is defined as described above, the user may employ the temporal scalability and the view scalability. Therefore, the user can view a sequence corresponding to a desired time and view, or can view a sequence corresponding to another limitation. The above-mentioned level information may also be established in various ways according to reference conditions. For example, the level information may be changed according to a camera location, and may also be changed according to a camera arrangement type. In addition, the level information may also be arbitrarily established without a special reference.
The term “anchor picture” is indicative of an encoded picture in which all slices refer to only slices in a current view and not slices in other views. A random access between views can be used for multiview-sequence decoding.
Anchor picture ID information can be used to perform the random access process to access data of a specific view without requiring a large amount of data to be decoded.
The term “view ID information” is indicative of specific information for discriminating between a picture of a current view and a picture of another view. In order to discriminate one picture from other pictures when the video sequence signal is encoded, a Picture Order Count (POC) and frame number information (frame_num) can be used.
If a current sequence is determined to be a multiview video sequence, inter-view prediction can be performed. An identifier is used to discriminate a picture of the current view from a picture of another view.
A view identifier can be defined to indicate a picture's view. The decoding apparatus can obtain information of a picture in a view different from a view of the current picture using the above-mentioned view identifier, such that it can decode the video signal using the information of the picture. The above-mentioned view identifier can be applied to the overall encoding/decoding process of the video signal. Also, the above-mentioned view identifier can also be applied to the multiview video coding process using the frame number information “frame_num” considering a view.
Typically, the multiview sequence has a large amount of data, and a hierarchical encoding function of each view (also called a “view scalability”) can be used for processing the large amount of data. In order to perform the view scalability function, a prediction structure considering views of the multiview sequence may be defined.
The above-mentioned prediction structure may be defined by structuralizing the prediction order or direction of several view sequences. For example, if several view sequences to be encoded are given, a center location of the overall arrangement is set to a base view, such that view sequences to be encoded can be hierarchically selected. The end of the overall arrangement or other parts may be set to the base view.
If the number of camera views is denoted by an exponential power of “2”, a hierarchical prediction structure between several view sequences may be formed on the basis of the above-mentioned case of the camera views denoted by the exponential power of “2”. Otherwise, if the number of camera views is not denoted by the exponential power of “2”, virtual views can be used, and the prediction structure may be formed on the basis of the virtual views. If the camera arrangement is indicative of a two-dimensional arrangement, the prediction order may be established by turns in a horizontal or vertical direction.
A parsed bitstream is entropy-decoded by an entropy decoding unit 11, and data such as a coefficient of each macroblock, a motion vector, etc., are extracted. The inverse quantization/inverse transform unit 12 multiplies a received quantization value by a predetermined constant to acquire a transformed coefficient value, and performs an inverse transform of the acquired coefficient value, such that it reconstructs a pixel value. The inter-prediction unit 13 performs an inter-prediction function from decoded samples of the current picture using the reconstructed pixel value.
At the same time, the deblocking filter 15 is applied to each decoded macroblock to reduce the degree of block distortion. The deblocking filter 15 performs a smoothing of the block edge, such that it improves an image quality of the decoded frame. The selection of a filtering process is dependent on a boundary strength and a gradient of image samples arranged in the vicinity of the boundary. The filtered pictures are stored in the decoded picture buffer 16, such that they can be outputted or be used as reference pictures.
The decoded picture buffer 16 stores or outputs pre-coded pictures to perform the inter-prediction function. In this case, frame number information “frame_num” and POC (Picture Order Count) information of the pictures are used to store or output the pre-coded pictures. Pictures of other view may exist in the above-mentioned pre-coded pictures in the case of the MVC technology. Therefore, in order to use the above-mentioned pictures as reference pictures, not only the “frame_num” and POC information, but also view identifier indicating a picture view may be used as necessary.
The inter-prediction unit 13 performs the inter-prediction using the reference pictures stored in the decoded picture buffer 16. The inter-coded macroblock may be divided into macroblock partitions. Each macroblock partition can be predicted by one or two reference pictures.
The motion compensation unit 17 compensates for a motion of the current block using the information received from the entropy decoding unit 11. The motion compensation unit 17 extracts motion vectors of neighboring blocks of the current block from the video signal, and obtains a motion-vector predictor of the current block. The motion compensation unit 17 compensates for the motion of the current block using a difference value between the motion vector and a predictor extracted from the video signal and the obtained motion-vector predictor. The above-mentioned motion compensation may be performed by only one reference picture, or may also be performed by a plurality of reference pictures.
Therefore, if the above-mentioned reference pictures are determined to be pictures of other views different from the current view, the motion compensation may be performed according to a view identifier indicating the other views.
A direct mode is indicative of a coding mode for predicting motion information of the current block on the basis of the motion information of a block which is completely decoded. The above-mentioned direct mode can reduce the number of bits required for encoding the motion information, resulting in the increased compression efficiency.
For example, a temporal direct mode predicts motion information of the current block using a correlation of motion information of a temporal direction. Similar to the temporal direct mode, the decoder can predict the motion information of the current block using a correlation of motion information of a view direction.
If the received bitstream corresponds to a multiview sequence, view sequences may be captured by different cameras respectively, such that a difference in illumination may occur due to internal or external factors of the cameras. In order to reduce potential inefficiency associated with the difference in illumination, an illumination compensation unit 18 performs an illumination compensation function.
In the case of performing illumination compensation function, flag information may be used to indicate whether an illumination compensation at a specific level of a video signal is performed. For example, the illumination compensation unit 18 may perform the illumination compensation function using flag information indicating whether the illumination compensation of a corresponding slice or macroblock is performed. Also, the above-mentioned method for performing the illumination compensation using the above-mentioned flag information may be applied to a variety of macroblock types (e.g., an inter 16×16 mode, a B-skip mode, a direct mode, etc.)
In order to reconstruct the current block when performing the illumination compensation, information of a neighboring block or information of a block in views different from a view of the current block may be used, and an offset value of the current block may also be used.
In this case, the offset value of the current block is indicative of a difference value between an average pixel value of the current block and an average pixel value of a reference block corresponding to the current block. As an example for using the above-mentioned offset value, a predictor of the current-block offset value may be obtained by using the neighboring blocks of the current block, and a residual value between the offset value and the predictor may be used. Therefore, the decoder can reconstruct the offset value of the current block using the residual value and the predictor.
In order to obtain the predictor of the current block, information of the neighboring blocks may be used as necessary.
For example, the offset value of the current block can be predicted by using the offset value of a neighboring block. Prior to predicting the current-block offset value, it is determined whether the reference index of the current block is equal to a reference index of the neighboring blocks. According to the determined result, the illumination compensation unit 18 can determine which one of neighboring blocks will be used or which value will be used.
The illumination compensation unit 18 may perform the illumination compensation using a prediction type of the current block. If the current block is predictively encoded by two reference blocks, the illumination compensation unit 18 may obtain an offset value corresponding to each reference block using the offset value of the current block.
As described above, the inter-predicted pictures or intra-predicted pictures acquired by the illumination compensation and motion compensation are selected according to a prediction mode, and reconstructs the current picture.
Examples of various aspects and features of the system are described in more detail in concurrently filed applications: U.S. application Ser. No. 11/622,611 titled “PROCESSING MULTIVIEW VIDEO”, and U.S. application Ser. No. 11/622,709 titled “PROCESSING MULTIVIEW VIDEO”, each of which is incorporated herein by reference.

Claims (24)

What is claimed is:
1. A method for decoding multi-view video data in a multi-view video data stream, with a decoder, the method comprising:
obtaining, with a Network Abstraction Layer parsing unit, identification information representing the multi-view video data stream including inter-view prediction structure information of a random access picture, all slices in the random access picture referring only to slices having a same temporal position and being in a different view of the multi-view video data;
obtaining, with the Network Abstraction Layer parsing unit, inter-view prediction structure information of the random access picture from the multi-view video data stream based on the identification information, the inter-view prediction structure information indicating a reference relation between inter-view reference pictures;
determining, with a decoded picture buffer unit, a reference picture list of a current slice for inter-view prediction using the inter-view prediction structure information of the random access picture;
determining, with an inter-prediction unit, a prediction value of a macroblock in the current slice based on the determined reference picture list for inter-view prediction; and
decoding the macroblock in the current slice using the prediction value,
wherein the multi-view video data includes video data of a base view and an ancillary view, the base view indicating a view being decoded independently of other views without using inter-view prediction, and the ancillary view being a view other than the base view,
wherein the inter-view reference pictures are identified by decoding order information between pictures, output order information between pictures, and view information identifying a view of each picture,
wherein the decoder includes the Network Abstraction Layer parsing unit, the decoded picture buffer unit, and the inter-prediction unit.
2. The method of claim 1, wherein the inter-view prediction structure information includes number information and view identification information, the number information indicating a total number of views in the multi-view video data, and the view identification information providing a view identifier of each reference view in the multi-view video data.
3. The method of claim 1, wherein the inter-view prediction structure information of the random access picture is obtained by considering a predictive direction.
4. The method of claim 3, wherein the predictive direction represents a forward direction or a backward direction in an output order of pictures.
5. The method of claim 1, wherein the ancillary view is decoded by referring to the base view.
6. The method of claim 1, wherein the inter-view prediction structure information is obtained from sequence parameter set information of a multi-view video.
7. An apparatus for decoding multi-view video data in a multi-view video data stream, comprising:
a Network Abstraction Layer parsing unit obtaining identification information representing the multi-view video data stream including inter-view prediction structure information of a random access picture, all slices in the random access picture referring only to slices having a same temporal position and being in a different view of the multi-view video data, and obtaining inter-view prediction structure information of the random access picture from the multi-view video data stream based on the identification information, the inter-view prediction structure information indicating a reference relation between inter-view reference pictures;
a decoded picture buffer unit determining a reference picture list of a current slice for inter-view prediction using the inter-view prediction structure information of the random access picture; and
an inter-prediction unit determining a prediction value of a macroblock in the current slice based on the determined reference picture list for inter-view prediction, and decoding the macroblock in the current slice using the prediction value,
wherein the multi-view video data includes video data of a base view and an ancillary view, the base view indicating a view being decoded independently of other views without using inter-view prediction, and the ancillary view being a view other than the base view,
wherein the inter-view reference pictures are identified by decoding order information between pictures, output order information between pictures and view information identifying a view of each picture.
8. The apparatus of claim 7, wherein the inter-view prediction structure information includes number information and view identification information, the number information indicating a total number of views in the multi-view video data, and the view identification information providing a view identifier of each reference view in the multi-view video data.
9. The apparatus of claim 7, wherein the inter-view prediction structure information of the random access picture is obtained by considering a predictive direction.
10. The apparatus of claim 9, wherein the predictive direction represents a forward direction or a backward direction in an output order of pictures.
11. The apparatus of claim 7, wherein the ancillary view is decoded by referring to the base view.
12. The apparatus of claim 7, wherein the inter-view prediction structure information is obtained from sequence parameter set information of a multi-view video.
13. A method for decoding multi-view video data in a multi-view video data stream, with a decoder, the method comprising:
obtaining, with a Network Abstraction Layer parsing unit, identification information representing the multi-view video data stream including inter-view prediction structure information of a random access picture, all slices in the random access picture referring only to slices having a same temporal position and being in a different view of the multi-view video data;
obtaining, with the Network Abstraction Layer parsing unit, inter-view prediction structure information of the random access picture from the multi-view video data stream based on the identification information, the inter-view prediction structure information indicating a reference relation between inter-view pictures;
determining, with a decoded picture buffer unit, a reference picture list of a current slice for inter-view prediction using the inter-view prediction structure information of the random access picture;
determining, with an inter-prediction unit, a prediction value of a macroblock in the current slice based on the determined reference picture list for inter-view prediction; and
decoding the macroblock in the current slice using the prediction value,
wherein the multi-view video data includes video data of a base view and an ancillary view, the base view indicating a view being decoded independently of other views without using inter-view prediction, and the ancillary view being a view other than the base view,
wherein the inter-view pictures are identified by view information identifying a view of each picture,
wherein the decoder includes the Network Abstraction Layer parsing unit, the decoded picture buffer unit, and the inter-prediction unit.
14. The method of claim 13, wherein the inter-view prediction structure information includes number information and view identification information, the number information indicating a total number of views in the multi-view video data, and the view identification information providing a view identifier of each reference view in the multi-view video data.
15. The method of claim 13, wherein the inter-view prediction structure information of the random access picture is obtained by considering a predictive direction.
16. The method of claim 15, wherein the predictive direction represents a forward direction or a backward direction in an output order of pictures.
17. The method of claim 13, wherein the ancillary view is decoded by referring to the base view.
18. The method of claim 13, wherein the inter-view prediction structure information is obtained from sequence parameter set information of a multi-view video.
19. An apparatus for decoding multi-view video data in a multi-view video data stream, comprising:
a Network Abstraction Layer parsing unit obtaining identification information representing the multi-view video data stream including inter-view prediction structure information of a random access picture, all slices in the random access picture referring only to slices having a same temporal position and being in a different view of the multi-view video data, and obtaining inter-view prediction structure information of the random access picture from the multi-view video data stream based on the identification information, the inter-view prediction structure information indicating a reference relation between inter-view pictures;
a decoded picture buffer unit determining a reference picture list of a current slice for inter-view prediction using the inter-view prediction structure information of the random access picture; and
an inter-prediction unit determining a prediction value of a macroblock in the current slice based on the determined reference picture list for inter-view prediction, and decoding the macroblock in the current slice using the prediction value,
wherein the multi-view video data includes video data of a base view and an ancillary view, the base view indicating a view being decoded independently of other views without using inter-view prediction, and the ancillary view being a view other than the base view,
wherein the inter-view pictures are identified by view information identifying a view of each picture.
20. The apparatus of claim 19, wherein the inter-view prediction structure information includes number information and view identification information, the number information indicating a total number of views in the multi-view video data, and the view identification information providing a view identifier of each reference view in the multi-view video data.
21. The apparatus of claim 19, wherein the inter-view prediction structure information of the random access picture is obtained by considering a predictive direction.
22. The apparatus of claim 21, wherein the predictive direction represents a forward direction or a backward direction in an output order of pictures.
23. The apparatus of claim 19, wherein the ancillary view is decoded by referring to the base view.
24. The apparatus of claim 19, wherein the inter-view prediction structure information is obtained from sequence parameter set information of a multi-view video.
US13/720,479 2006-01-12 2012-12-19 Processing multiview video Active 2029-04-03 USRE44680E1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US13/720,479 USRE44680E1 (en) 2006-01-12 2012-12-19 Processing multiview video

Applications Claiming Priority (19)

Application Number Priority Date Filing Date Title
US75823406P 2006-01-12 2006-01-12
KR20060004956 2006-01-17
KR10-2006-0004956 2006-01-17
US75962006P 2006-01-18 2006-01-18
US76253406P 2006-01-27 2006-01-27
KR20060027100 2006-03-24
KR10-2006-0027100 2006-03-24
US78719306P 2006-03-30 2006-03-30
KR10-2006-0037773 2006-04-26
KR1020060037773A KR20070076356A (en) 2006-01-18 2006-04-26 Method and apparatus for coding and decoding of video sequence
US81827406P 2006-07-05 2006-07-05
US83008706P 2006-07-12 2006-07-12
US83032806P 2006-07-13 2006-07-13
KR1020060110337A KR20070076391A (en) 2006-01-18 2006-11-09 A method and apparatus for decoding/encoding a video signal
KR1020060110338A KR20070076392A (en) 2006-01-18 2006-11-09 A method and apparatus for decoding/encoding a video signal
KR10-2006-0110337 2006-11-09
KR10-2006-0110338 2006-11-09
US11/622,592 US7856148B2 (en) 2006-01-12 2007-01-12 Processing multiview video
US13/720,479 USRE44680E1 (en) 2006-01-12 2012-12-19 Processing multiview video

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
US11/622,592 Reissue US7856148B2 (en) 2006-01-12 2007-01-12 Processing multiview video

Publications (1)

Publication Number Publication Date
USRE44680E1 true USRE44680E1 (en) 2013-12-31

Family

ID=39731061

Family Applications (1)

Application Number Title Priority Date Filing Date
US13/720,479 Active 2029-04-03 USRE44680E1 (en) 2006-01-12 2012-12-19 Processing multiview video

Country Status (2)

Country Link
US (1) USRE44680E1 (en)
KR (1) KR101276847B1 (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10034015B2 (en) 2012-02-03 2018-07-24 Sun Patent Trust Image coding method and image coding apparatus
US10129563B2 (en) 2012-01-20 2018-11-13 Sun Patent Trust Methods and apparatuses for encoding and decoding video using temporal motion vector prediction
US10212447B2 (en) 2012-03-06 2019-02-19 Sun Patent Trust Moving picture coding method, moving picture decoding method, moving picture coding apparatus, moving picture decoding apparatus, and moving picture coding and decoding apparatus

Citations (41)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0698312A (en) 1992-09-16 1994-04-08 Fujitsu Ltd High efficiency picture coding system
CN1242907A (en) 1997-01-13 2000-01-26 皇家菲利浦电子有限公司 Emboding supplemental data in digital video signal
US6055012A (en) 1995-12-29 2000-04-25 Lucent Technologies Inc. Digital multi-view video compression with complexity and compatibility constraints
KR20020032954A (en) 2000-10-28 2002-05-04 김춘호 3D Stereosc opic Multiview Video System and Manufacturing Method
US6438169B1 (en) 1998-03-05 2002-08-20 Sony Corporation Video signal compressing method and apparatus, and compressed data multiplexing method and apparatus
JP2003517743A (en) 1998-08-27 2003-05-27 ジェネラル・インスツルメント・コーポレイション Dynamic bit allocation for statistical multiplexing of compressed and uncompressed digital video signals
US20030202592A1 (en) 2002-04-20 2003-10-30 Sohn Kwang Hoon Apparatus for encoding a multi-view moving picture
KR20040013540A (en) 2002-08-07 2004-02-14 한국전자통신연구원 The multiplexing method and its device according to user's request for multi-view 3D video
CN1482802A (en) 2003-08-18 2004-03-17 北京工业大学 Video coding method based on time-space domain correlation quick movement estimate
EP1418762A1 (en) 2002-05-22 2004-05-12 Matsushita Electric Industrial Co., Ltd. Moving image encoding method, moving image decoding method, and data recording medium
CN1515550A (en) 1994-03-04 2004-07-28 Thrombosis resisting preparation
CN1545808A (en) 2002-06-20 2004-11-10 ������������ʽ���� Decoding device and decoding method
US20040247159A1 (en) 2003-06-07 2004-12-09 Niranjan Damera-Venkata Motion estimation for compression of calibrated multi-view image sequences
WO2005001772A1 (en) 2003-06-30 2005-01-06 Koninklijke Philips Electronics, N.V. System and method for video processing using overcomplete wavelet coding and circular prediction mapping
WO2005018217A2 (en) 2003-08-07 2005-02-24 Sony Electronics, Inc. Semantics-based motion estimation for multi-view video coding
EP1515550A1 (en) 2002-06-20 2005-03-16 Sony Corporation Decoding apparatus and decoding method
JP2005510187A (en) 2001-11-21 2005-04-14 エレクトロニクス アンド テレコミュニケーションズ リサーチ インスチチュート Binocular / multi-view 3D moving image processing system and method
WO2005069630A1 (en) 2004-01-20 2005-07-28 Daeyang Foundation Method, medium, and apparatus for 3-dimensional encoding and/or decoding of video
KR20050122717A (en) 2004-06-25 2005-12-29 학교법인연세대학교 Method for coding/decoding for multiview sequence where view selection is possible
WO2006014057A1 (en) 2004-08-03 2006-02-09 Daeyang Foundation Method, medium, and apparatus predicting direct mode motion of a multi-angle moving picture
WO2006062377A1 (en) 2004-12-10 2006-06-15 Electronics And Telecommunications Research Institute Apparatus for universal coding for multi-view video
US20060133501A1 (en) 2004-11-30 2006-06-22 Yung-Lyul Lee Motion estimation and compensation method and device adaptive to change in illumination
US20060133493A1 (en) 2002-12-27 2006-06-22 Suk-Hee Cho Method and apparatus for encoding and decoding stereoscopic video
US20060132610A1 (en) 2004-12-17 2006-06-22 Jun Xin Multiview video decomposition and encoding
US20060146143A1 (en) 2004-12-17 2006-07-06 Jun Xin Method and system for managing reference pictures in multiview videos
US20060146141A1 (en) 2004-12-17 2006-07-06 Jun Xin Method for randomly accessing multiview videos
US20070064800A1 (en) 2005-09-22 2007-03-22 Samsung Electronics Co., Ltd. Method of estimating disparity vector, and method and apparatus for encoding and decoding multi-view moving picture using the disparity vector estimation method
US20070071107A1 (en) 2005-09-29 2007-03-29 Samsung Electronics Co., Ltd. Method of estimating disparity vector using camera parameters, apparatus for encoding and decoding multi-view picture using the disparity vector estimation method, and computer-readable recording medium storing a program for executing the method
US7444664B2 (en) 2004-07-27 2008-10-28 Microsoft Corp. Multi-view video format
CN101375594A (en) 2006-01-12 2009-02-25 Lg电子株式会社 Processing multiview video
US20090168874A1 (en) 2006-01-09 2009-07-02 Yeping Su Methods and Apparatus for Multi-View Video Coding
US20090237516A1 (en) 2008-02-20 2009-09-24 Aricent Inc. Method and system for intelligent and efficient camera motion estimation for video stabilization
US20090257669A1 (en) 2006-10-18 2009-10-15 Jae Hoon Kim Local illumination and color compensation without explicit signaling
US7613344B2 (en) 2003-12-08 2009-11-03 Electronics And Telecommunications Research Institute System and method for encoding and decoding an image using bitstream map and recording medium thereof
US7671893B2 (en) 2004-07-27 2010-03-02 Microsoft Corp. System and method for interactive multi-view video
US20100118942A1 (en) 2007-06-28 2010-05-13 Thomson Licensing Methods and apparatus at an encoder and decoder for supporting single loop decoding of multi-view coded video
US7728878B2 (en) 2004-12-17 2010-06-01 Mitsubishi Electric Research Labortories, Inc. Method and system for processing multiview videos for view synthesis using side information
US20100165077A1 (en) 2005-10-19 2010-07-01 Peng Yin Multi-View Video Coding Using Scalable Video Coding
US7782948B2 (en) 2006-03-30 2010-08-24 Lg Electronics, Inc. Method and apparatus for decoding/encoding a video signal
US7817866B2 (en) 2006-01-12 2010-10-19 Lg Electronics Inc. Processing multiview video
US20100329347A1 (en) 2008-01-29 2010-12-30 Dong Hyung Kim Method and apparatus for encoding and decoding video signal using motion compensation based on affine transformation

Patent Citations (62)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0698312A (en) 1992-09-16 1994-04-08 Fujitsu Ltd High efficiency picture coding system
CN1515550A (en) 1994-03-04 2004-07-28 Thrombosis resisting preparation
US6055012A (en) 1995-12-29 2000-04-25 Lucent Technologies Inc. Digital multi-view video compression with complexity and compatibility constraints
CN1242907A (en) 1997-01-13 2000-01-26 皇家菲利浦电子有限公司 Emboding supplemental data in digital video signal
US6438169B1 (en) 1998-03-05 2002-08-20 Sony Corporation Video signal compressing method and apparatus, and compressed data multiplexing method and apparatus
JP2003517743A (en) 1998-08-27 2003-05-27 ジェネラル・インスツルメント・コーポレイション Dynamic bit allocation for statistical multiplexing of compressed and uncompressed digital video signals
KR20020032954A (en) 2000-10-28 2002-05-04 김춘호 3D Stereosc opic Multiview Video System and Manufacturing Method
KR100375708B1 (en) 2000-10-28 2003-03-15 전자부품연구원 3D Stereosc opic Multiview Video System and Manufacturing Method
JP2005510187A (en) 2001-11-21 2005-04-14 エレクトロニクス アンド テレコミュニケーションズ リサーチ インスチチュート Binocular / multi-view 3D moving image processing system and method
US20030202592A1 (en) 2002-04-20 2003-10-30 Sohn Kwang Hoon Apparatus for encoding a multi-view moving picture
US6999513B2 (en) 2002-04-20 2006-02-14 Korea Electronics Technology Institute Apparatus for encoding a multi-view moving picture
EP1418762A1 (en) 2002-05-22 2004-05-12 Matsushita Electric Industrial Co., Ltd. Moving image encoding method, moving image decoding method, and data recording medium
EP1515550A1 (en) 2002-06-20 2005-03-16 Sony Corporation Decoding apparatus and decoding method
CN1545808A (en) 2002-06-20 2004-11-10 ������������ʽ���� Decoding device and decoding method
KR20040013540A (en) 2002-08-07 2004-02-14 한국전자통신연구원 The multiplexing method and its device according to user's request for multi-view 3D video
US20060133493A1 (en) 2002-12-27 2006-06-22 Suk-Hee Cho Method and apparatus for encoding and decoding stereoscopic video
US20040247159A1 (en) 2003-06-07 2004-12-09 Niranjan Damera-Venkata Motion estimation for compression of calibrated multi-view image sequences
US7286689B2 (en) 2003-06-07 2007-10-23 Hewlett-Packard Development Company, L.P. Motion estimation for compression of calibrated multi-view image sequences
WO2005001772A1 (en) 2003-06-30 2005-01-06 Koninklijke Philips Electronics, N.V. System and method for video processing using overcomplete wavelet coding and circular prediction mapping
WO2005018217A2 (en) 2003-08-07 2005-02-24 Sony Electronics, Inc. Semantics-based motion estimation for multi-view video coding
CN1482802A (en) 2003-08-18 2004-03-17 北京工业大学 Video coding method based on time-space domain correlation quick movement estimate
US7613344B2 (en) 2003-12-08 2009-11-03 Electronics And Telecommunications Research Institute System and method for encoding and decoding an image using bitstream map and recording medium thereof
WO2005069630A1 (en) 2004-01-20 2005-07-28 Daeyang Foundation Method, medium, and apparatus for 3-dimensional encoding and/or decoding of video
KR20050122717A (en) 2004-06-25 2005-12-29 학교법인연세대학교 Method for coding/decoding for multiview sequence where view selection is possible
WO2006001653A1 (en) 2004-06-25 2006-01-05 Lg Electronics Inc. Method of coding and decoding multiview sequence and method of displaying thereof
KR100679740B1 (en) 2004-06-25 2007-02-07 학교법인연세대학교 Method for Coding/Decoding for Multiview Sequence where View Selection is Possible
US7671893B2 (en) 2004-07-27 2010-03-02 Microsoft Corp. System and method for interactive multi-view video
US7444664B2 (en) 2004-07-27 2008-10-28 Microsoft Corp. Multi-view video format
US20060029137A1 (en) 2004-08-03 2006-02-09 Daeyang Foundation Method, medium, and apparatus predicting direct mode motion of a multi-angle moving picture
WO2006014057A1 (en) 2004-08-03 2006-02-09 Daeyang Foundation Method, medium, and apparatus predicting direct mode motion of a multi-angle moving picture
US20060133501A1 (en) 2004-11-30 2006-06-22 Yung-Lyul Lee Motion estimation and compensation method and device adaptive to change in illumination
WO2006062377A1 (en) 2004-12-10 2006-06-15 Electronics And Telecommunications Research Institute Apparatus for universal coding for multi-view video
US20060146141A1 (en) 2004-12-17 2006-07-06 Jun Xin Method for randomly accessing multiview videos
US7710462B2 (en) 2004-12-17 2010-05-04 Mitsubishi Electric Research Laboratories, Inc. Method for randomly accessing multiview videos
US7728878B2 (en) 2004-12-17 2010-06-01 Mitsubishi Electric Research Labortories, Inc. Method and system for processing multiview videos for view synthesis using side information
US20060146143A1 (en) 2004-12-17 2006-07-06 Jun Xin Method and system for managing reference pictures in multiview videos
US20060132610A1 (en) 2004-12-17 2006-06-22 Jun Xin Multiview video decomposition and encoding
US20070064800A1 (en) 2005-09-22 2007-03-22 Samsung Electronics Co., Ltd. Method of estimating disparity vector, and method and apparatus for encoding and decoding multi-view moving picture using the disparity vector estimation method
US20070071107A1 (en) 2005-09-29 2007-03-29 Samsung Electronics Co., Ltd. Method of estimating disparity vector using camera parameters, apparatus for encoding and decoding multi-view picture using the disparity vector estimation method, and computer-readable recording medium storing a program for executing the method
US20100165077A1 (en) 2005-10-19 2010-07-01 Peng Yin Multi-View Video Coding Using Scalable Video Coding
JP2007159113A (en) 2005-11-30 2007-06-21 Mitsubishi Electric Research Laboratories Inc Method and system for randomly accessing multiview video
US20090168874A1 (en) 2006-01-09 2009-07-02 Yeping Su Methods and Apparatus for Multi-View Video Coding
CN101375594A (en) 2006-01-12 2009-02-25 Lg电子株式会社 Processing multiview video
US7831102B2 (en) 2006-01-12 2010-11-09 Lg Electronics Inc. Processing multiview video
US7856148B2 (en) 2006-01-12 2010-12-21 Lg Electronics Inc. Processing multiview video
US7817865B2 (en) 2006-01-12 2010-10-19 Lg Electronics Inc. Processing multiview video
US7970221B2 (en) 2006-01-12 2011-06-28 Lg Electronics Inc. Processing multiview video
US7817866B2 (en) 2006-01-12 2010-10-19 Lg Electronics Inc. Processing multiview video
US7782948B2 (en) 2006-03-30 2010-08-24 Lg Electronics, Inc. Method and apparatus for decoding/encoding a video signal
US20100215100A1 (en) 2006-03-30 2010-08-26 Byeong Moon Jeon Method and Apparatus for Decoding/Encoding a Video Signal
US7782943B2 (en) 2006-03-30 2010-08-24 Lg Electronics, Inc. Method and apparatus for decoding/encoding a video signal
US7782947B2 (en) 2006-03-30 2010-08-24 Lg Electronics, Inc. Method and apparatus for decoding/encoding a video signal
US7782949B2 (en) 2006-03-30 2010-08-24 Lg Electronics, Inc. Method and apparatus for decoding/encoding a video signal
US7782946B2 (en) 2006-03-30 2010-08-24 Lg Electronics, Inc. Method and apparatus for decoding/encoding a video signal
US7782950B2 (en) 2006-03-30 2010-08-24 Lg Electronics, Inc. Method and apparatus for decoding/encoding a video signal
US7782944B2 (en) 2006-03-30 2010-08-24 Lg Electronics, Inc. Method and apparatus for decoding/encoding a video signal
US7782945B2 (en) 2006-03-30 2010-08-24 Lg Electronics, Inc. Method and apparatus for decoding/encoding a video signal
US20090257669A1 (en) 2006-10-18 2009-10-15 Jae Hoon Kim Local illumination and color compensation without explicit signaling
US20100135388A1 (en) 2007-06-28 2010-06-03 Thomson Licensing A Corporation SINGLE LOOP DECODING OF MULTI-VIEW CODED VIDEO ( amended
US20100118942A1 (en) 2007-06-28 2010-05-13 Thomson Licensing Methods and apparatus at an encoder and decoder for supporting single loop decoding of multi-view coded video
US20100329347A1 (en) 2008-01-29 2010-12-30 Dong Hyung Kim Method and apparatus for encoding and decoding video signal using motion compensation based on affine transformation
US20090237516A1 (en) 2008-02-20 2009-09-24 Aricent Inc. Method and system for intelligent and efficient camera motion estimation for video stabilization

Non-Patent Citations (54)

* Cited by examiner, † Cited by third party
Title
"Advanced video coding for generic audiovisual services; H.264 (May 2003)," ITU-T Standard Superseded(s), International Telecommunication Union, Geneva, CH, No. H.264 (May 2003), May 30, 2003, pp. 110-123.
"Description of Core Experiments in MVC." International Organisation for Standardisation, ISO/IEC JTC1/SC29/WG11, No. MPEG2006/W8019, Montreux, Switzerland, Apr. 2006, 38 pages.
A. Smolic, K. Müller, P. Merkie, C. Fehn, P. Kauff, P. Eisert, and T. Wiegand, "3D Video and Free Viewpoint Video-Technologies, Applications and MPEG Standards". In Proceedings of International Conference on Multimedia & Expo, pp. 2161-2164, Jul. 2006.
Examination Report, European Patent Office, EP Application No. 07 768 721.8, dated Jan. 20, 2011, 7 pages.
Hangzhou: "wftp3.itu.int-/av-arch/jvt-site/2006-10-Hangzhou/" Intenet Citation, pp. 1-2, XP007916683, Retrieved from the Internet: URL: http://wftp3.itu.int/av-arch/jvtsite/2006-10-Hangzhou/ [retrieved on Jan. 11, 2011].
Hideaki Kimara, Masaki Kitabara, Kazuto Kamikura, and Yoshiyaki Yashima, "Free-viewpoint Video Communication Using Multi-view Video Coding", NTT Technical Review Online, Aug. 2004 vol. 2 No. 8 3-D Display and Information Technologies.
ISO/IEC JTC1/SC29/WG11, "Survey of Algorithms used for Multi-view Video Coding (MVC)", Doc. N6909, Hong Kong, China, Jan. 2005.
Joaquin Lopez, Jae Hoon Kim, Antonio Ortega, and George Chen, "Block-based Illlumination Compensation and Search Techniques for Multiview Video Coding," Picture Coding Symposium, San Francisco, CA, Dec. 2004.
Kim, Jae Hoon et al., "Dependent Bit Allocation in Multiview Video Coding." IEEE International Conference on Genova, Italy, Sep. 11-14, 2005, Piscataway, NJ, USA, vol. 2, Sep. 11, 2005, pp. 293-296.
Kim, Sang Hyun et al., "Fast local motion-compensation algorithm for video sequences with brightness variations", IEEE Transactions on Circuits and Systems for Video Technology , Publication Date: Apr. 2003, vol. 13, Issue: 4, pp. 289-299.
Kim, Yongtae et al., "Fast Disparity and Motion Estimation for Multi-view Video Coding." IEEE Transactions on Consumer Electronics, vol. 53, No. 2, May 2007, pp. 712-719.
Kimata, H. Kitahara, M. Kamikura, K. Yashima, Y., "Hierarchical reference picture selection method for temporal scalability beyond H.264" In Proceedings of Internatioanl Conference on Multimedia & Expo, pp. 181-184, Jun. 2004.
Kimata, Hideaki et al., "Free-viewpoint Video Communication Using Multi-view Video Coding", NTT Technical Review Online, Aug. 2004 vol. 2 No. 8, 3-D Display and Information Technologies.
Koo, Han-Suh et al., "AHG Report: MVC motion/disparity vector coding." Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG (ISO/IEC JTC1/SC29/WG11 and ITU-T SG16 Q.6), 23rd Meeting: San Jose, California, USA, Apr. 21-27, 2007, Document: JVT-W012, 4 pages.
Koo, Han-Suh et al., "CE11: MVC Motion Skip Mode." Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T SG16 Q.6), 22nd Meeting: Marrakech, Morocco, Jan. 13-19, 2007, Document: JVT-V069.
Koo, Han-Suh et al., "Core Experiment on Disparity and Motion Vector Coding (CE11)." Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG (ISO/IEC JTC1/SC29/WG11 and ITU-T SG16 Q.6), 21st Meeting: Hangzhou, China, Oct. 20-27, 2006, Document: JVT-U311, 3 pages.
Koo, Han-Suh et al., "Motion Skip Mode for MVC." Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG (ISO/IEC JTC1/SC29/WG11 and ITU-T SG16 Q.6), 21st Meeting: Hangzhou, China, Oct. 23-27, 2006, Document: JVT-U091-L, 7 pages.
Koo, Han-Suh et al., "MVC Motion Skip Mode." Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG (ISO/IEC JTC1/SC29/WG11 and ITU-T SG16 Q.6), 23rd Meeting: San Jose, California, USA, Apr. 21-27, 2007, Document: JVT-W081, 13 pages.
Lee, Sang-Heon et al., "Inter-view motion information prediction method in MVC," Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG (ISO/IEC JTC1/SC29/WG11 and ITU-T SG16 Q.6) 20th Meeting: Klagenfurt, Austria, Jul. 15-21, 2006, Document: JVT-T135, Filename: JVT-T135.doc, 13 pages.
Lee, Yung-Lyul et al., "Multi-view Video Coding Using Illumination Change-Adaptive Motion Estimation and 2-D Direct Mode." PCM 2005, Part I, LNCS 3767, Jan. 1, 2005, Springer-Verlag Berlin Heidelberg, Germany, 2005, pp. 396-407.
Lee, Yung-Lyul et al., "Result of CE2 on Multi-view Video Coding." International Organization for Standardization, ISO/IEC JTC1/SC29/WG11, MPEG2006/M13143, Jan. 2006, Switzerland, Montreux, pp. 1-12.
Lee, Yung-Lyul et al., "Result of CE2 on Multi-view Video Coding." International Organization for Standardization, ISO/IEC JTC1/SC29/WG11, MPEG2006/M13498, Jul. 2006, Klagenfurt, Austria, pp. 1-23.
Li, Shiping et al., "Approaches to H.264-Based Stereoscopic Coding." Proceedings of the Third International Conference on Image and Graphics (ICIG'04), Dec. 18-20, 2004, Dec. 18, 2004, pp. 365-368.
Lopez, Joaquin et al., "Block-based Illumination Compensation and Search Techniques for Multiview Video Coding," Picture Coding Symposium, San Francisco, CA, Dec. 2004.
Merkle, P, Muller, K. Smolic, A. Wiegand, T., "Efficient Compression of Multi-View Video Exploiting Inter-View Dependencies Based on H.264/MPEG4-AVC", In Proceedings of International Conference on Multimedia & Expo, pp. 2161-2164, Jul. 2006.
Merkle, P. et al., "Efficient Compression of Multi-View Video Exploiting Inter-View Dependencies Based on H.264/MPEG4-AVC", In Proceedings of International Conference on Multimedia & Expo, pp. 2161-2164, Jul. 2006.
Non-final Office Action dated Jul. 17, 2012 from U.S. Appl. No. 13/752,055, 11 pages.
Non-final Office Action in U.S. Appl. No. 11/622,803, dated Oct. 21, 2010, 24 pages.
Non-final Office Action issued in U.S. Appl. No. 11/622,675 dated Oct. 13, 2011, 18 pages.
Non-final Office Action issued in U.S. Appl. No. 11/622,675, mailed May 25, 2011, 9 pages.
Non-final Office Action issued in U.S. Appl. No. 11/622,681, mailed Jun. 20, 2011, 9 pages.
Non-final Office Action issued in U.S. Appl. No. 11/622,709, mailed Jul. 11, 2011, 9 pages.
Notice of Allowance issued by the Chinese Patent Office for Application No. 200780003083.8 dated Mar. 5, 2013 (with English translation) 6 pages.
Notice of Allowance issued in U.S. Appl. No. 11/622,611, dated Apr. 30, 2010, 8 pages.
Notice of Allowance issued in U.S. Appl. No. 11/622,709 dated Dec. 5, 2011, 8 pages.
Office Action cited in U.S. Appl. No. 12/309,225 dated Mar. 9, 2012; 11 pages.
Ohm, Jens-Rainer, "Stereo/Multiview Video Encoding Using the MPEG Family of Standards." Part of the IS&T/SPIE Conference on Stereoscopic Displays and Applications X, San Jose, California, Jan. 1998, SPIE vol. 3639, pp. 242-253.
P. Kauff. A. Smolie, P. Eisert, C. Fehn, K. Muller, and R. Schäfer "Data Format and Coding for Free Viewpoint Video," Proc. International Broadcast Convention IBC 2005, Amsterdam, Netherlands, pp., Sep. 2005.
Sang Hyun Kim and Rae-Hong Park, "Fast local motion-compensation algorithm for video sequences with brightness variations", IEEE Transactions on Circuits and Systems for Video Technology , Publication Date: Apr. 2003, vol. 13, Issue: 4 pp. 289-299.
Search Report issued in EP application No. 07 768 721.8, dated Sep. 3, 2010, 5 pages.
Senoh, Taka et al., "Disparity Vector Prediction CE Plan for MVC/CE4." International Organisation for Standardisation, ISO/IEC JTC1/SC29/WG11, No. M13166, Montreux, Switzerland, Apr. 2006, 6 pages.
Smolic, A. and Kauff, P., "Interactive 3-D video representation and coding technologies" Proceedings of the IEEE, Publication Date: Jan. 2005, vol. 93, Issue: 1, pp. 98-110.
Smolic, A. et al., "3D Video and Free Viewpoint Video-Technologies, Applications and MPEG Standards", In Proceedings of International Conference on Multimedia & Expo, pp. 2161-2164, Jul. 2006.
Smolic, A. et al., "Development of MPEG Standards for 3D and Free Viewpoint Video", SPIE Conference Optics East 2005: Communications, Multimedia & Display Technologies, vol. 6014, pp. 262-273, Nov. 2005.
Smolic, A. et al., "Interactive 3-D video representation and coding technologies" Proceedings of the IEEE, Publication Date: Jan 2005, vol. 93, Issue: 1, pp. 98-110.
Smolic, A.; Kimata, H.; Vetro, A., "Development of MPEG Standards for 3D and Free Viewpoint Video", SPIE Conference Optics East 2005:Communications, Multimedia & Display Technologies, vol. 6014, pp. 262-273, Nov. 2005.
Song, Hak-Sup et al., "Macroblock Information Skip for MVC." Joint Video Team (JVT) of 73 ISO/IEC MPEG & ITU-T VCEG (ISO/IEC JTC1ISC29/WG11 and ITU-T SG16 Q.6), 22nd Meeting: Marrakech, Morocco, Jan. 13-19, 2007, Document: NT-V052rl, 7 pages.
Supplementary European Search Report issued in application No. EP07700952, dated May 18, 2010, 9 pages.
Supplementary European Search Report issued in application No. EP07700955, dated May 18, 2010, 10 pages.
Supplementary European Search Report issued in European Application No. EP 07768721, mailed Feb. 2, 2010, 3 pages.
Taiwanese Search Report, Taiwan Advance Patent & Trademark Office, issued in application No. 096125507, dated Nov. 1, 2010, 2 pages.
Wenxian Yang; Peng Wu; Yan Lu; Jianfei Cai; King Ngi Ngan Shipeng Li, "Scalable multiview video coding using wavelet" Nanyang Technol. Univ., Singapore: IEEE International Symposium on Circuits and Systems, May 2005.
Yang, Wenxian et al., "Scalable multiview video coding using wavelet" Nanyang Technol. Univ., Singapore; IEEE International Symposium on Circuits and Systems, May 2005.
Zhu, Gang et al., "Inter-view Direct Mode in MVC." International Organisation for Standardisation, ISO/IEC JTC1/SC29/WG11, No. MPEG2006/m13177, Montreux, Switzerland, Apr. 2006, 5 pages.

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10616601B2 (en) 2012-01-20 2020-04-07 Sun Patent Trust Methods and apparatuses for encoding and decoding video using temporal motion vector prediction
US10129563B2 (en) 2012-01-20 2018-11-13 Sun Patent Trust Methods and apparatuses for encoding and decoding video using temporal motion vector prediction
US10623762B2 (en) 2012-02-03 2020-04-14 Sun Patent Trust Image coding method and image coding apparatus
US10334268B2 (en) 2012-02-03 2019-06-25 Sun Patent Trust Image coding method and image coding apparatus
US10034015B2 (en) 2012-02-03 2018-07-24 Sun Patent Trust Image coding method and image coding apparatus
US10904554B2 (en) 2012-02-03 2021-01-26 Sun Patent Trust Image coding method and image coding apparatus
US11451815B2 (en) 2012-02-03 2022-09-20 Sun Patent Trust Image coding method and image coding apparatus
US11812048B2 (en) 2012-02-03 2023-11-07 Sun Patent Trust Image coding method and image coding apparatus
US10560716B2 (en) 2012-03-06 2020-02-11 Sun Patent Trust Moving picture coding method, moving picture decoding method, moving picture coding apparatus, moving picture decoding apparatus, and moving picture coding and decoding apparatus
US10212447B2 (en) 2012-03-06 2019-02-19 Sun Patent Trust Moving picture coding method, moving picture decoding method, moving picture coding apparatus, moving picture decoding apparatus, and moving picture coding and decoding apparatus
US10880572B2 (en) 2012-03-06 2020-12-29 Sun Patent Trust Moving picture coding method, moving picture decoding method, moving picture coding apparatus, moving picture decoding apparatus, and moving picture coding and decoding apparatus
US11595682B2 (en) 2012-03-06 2023-02-28 Sun Patent Trust Moving picture coding method, moving picture decoding method, moving picture coding apparatus, moving picture decoding apparatus, and moving picture coding and decoding apparatus
US11949907B2 (en) 2012-03-06 2024-04-02 Sun Patent Trust Moving picture coding method, moving picture decoding method, moving picture coding apparatus, moving picture decoding apparatus, and moving picture coding and decoding apparatus

Also Published As

Publication number Publication date
KR101276847B1 (en) 2013-06-18
KR20080094048A (en) 2008-10-22

Similar Documents

Publication Publication Date Title
US7856148B2 (en) Processing multiview video
KR101619451B1 (en) Method and apparatus for processing a multiview video signal
US20070177671A1 (en) Processing multiview video
EP2786573B1 (en) Sequence level information for multiview video coding (mvc) compatible three-dimensional video coding (3dvc)
US7782948B2 (en) Method and apparatus for decoding/encoding a video signal
US11909960B2 (en) Method and apparatus for processing video signal
JP2010525724A (en) Method and apparatus for decoding / encoding a video signal
EP2700228A1 (en) Motion vector prediction in video coding
JP2009543508A (en) Signal processing method and apparatus
KR20080007086A (en) A method and apparatus for decoding/encoding a video signal
USRE44680E1 (en) Processing multiview video
KR101366289B1 (en) A method and apparatus for decoding/encoding a video signal
WO2023202956A1 (en) Video coding and decoding
KR20080023210A (en) A method and apparatus for decoding a video signal

Legal Events

Date Code Title Description
AS Assignment

Owner name: LG ELECTRONICS INC., KOREA, REPUBLIC OF

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:YANG, JEONG HYU;REEL/FRAME:030610/0513

Effective date: 20070405

FEPP Fee payment procedure

Free format text: PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

FPAY Fee payment

Year of fee payment: 4

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 8TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1552)

Year of fee payment: 8

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 12TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1553); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Year of fee payment: 12