US20120076208A1 - Memory reduced h264/mpeg-4 avc codec - Google Patents

Memory reduced h264/mpeg-4 avc codec Download PDF

Info

Publication number
US20120076208A1
US20120076208A1 US13/311,972 US201113311972A US2012076208A1 US 20120076208 A1 US20120076208 A1 US 20120076208A1 US 201113311972 A US201113311972 A US 201113311972A US 2012076208 A1 US2012076208 A1 US 2012076208A1
Authority
US
United States
Prior art keywords
picture
upsampling
circuit
downsampling
memory
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US13/311,972
Inventor
Lowell L. Winger
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Avago Technologies International Sales Pte Ltd
Original Assignee
Winger Lowell L
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Winger Lowell L filed Critical Winger Lowell L
Priority to US13/311,972 priority Critical patent/US20120076208A1/en
Publication of US20120076208A1 publication Critical patent/US20120076208A1/en
Assigned to DEUTSCHE BANK AG NEW YORK BRANCH, AS COLLATERAL AGENT reassignment DEUTSCHE BANK AG NEW YORK BRANCH, AS COLLATERAL AGENT PATENT SECURITY AGREEMENT Assignors: AGERE SYSTEMS LLC, LSI CORPORATION
Assigned to AVAGO TECHNOLOGIES GENERAL IP (SINGAPORE) PTE. LTD. reassignment AVAGO TECHNOLOGIES GENERAL IP (SINGAPORE) PTE. LTD. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: LSI CORPORATION
Assigned to LSI CORPORATION, AGERE SYSTEMS LLC reassignment LSI CORPORATION TERMINATION AND RELEASE OF SECURITY INTEREST IN PATENT RIGHTS (RELEASES RF 032856-0031) Assignors: DEUTSCHE BANK AG NEW YORK BRANCH, AS COLLATERAL AGENT
Abandoned legal-status Critical Current

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/59Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
    • H04N19/423Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation characterised by memory arrangements
    • H04N19/426Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation characterised by memory arrangements using memory downsizing methods
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/186Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a colour or a chrominance component
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51Motion estimation or motion compensation
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51Motion estimation or motion compensation
    • H04N19/573Motion compensation with multiple frame prediction using two or more reference frames in a given prediction direction
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/61Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding

Definitions

  • the present invention relates to video coding/decoding/transcoding (codecs) generally and, more particularly, to a memory reduced H.264/MPEG-4 AVC codec.
  • H.264 An International Telecommunication Union-Telecommunication Standardization Sector (ITU-T) recommendation H.264 and an International Organization for Standardization/International Electrotechnical Commission (ISO/IEC) recommendation MPEG-4 Part 10 for Advanced Video Coding (MPEG-4/AVC) concern digital video codecs.
  • MPEG-4/AVC documents (hereafter simply referred to as H.264) specify both decoder operation (semantics) and compressed video representation (bitstream syntax). Due to efficient syntax and many new predictive options, a conventional H.264 video encoder produces bitstreams that provide MPEG-2 quality with a 50% lower bitrate.
  • video distribution channels such as high definition digital optical media formats (Blu-ray DiscTM, HD DVD), cable (High Definition Video On Demand), satellite (DirectTV, DISH NetworkTM), Internet Protocol Television (IPTV), terrestrial high definition television (TV), pay TV (France, England) and mobile systems (3GPP) are deploying H.264 capable equipment.
  • Blue-Ray is a trademark of the Blu-ray Disc Association, Tokyo, Japan.
  • DISH Network is a registered trademark of EchoStar Satellite L.L.C., Englewood, Colo.
  • Inter-frame predictions and intra-frame predictions are used to remove redundancy, such that improved compression is possible.
  • Inter-frame predictions use previously, sometimes motion compensated, decoded video frames or fields for prediction of current blocks.
  • Predictive P-blocks use only one block from a previous frame or field to predict.
  • Bi-predictive B-blocks use a (potentially weighted) average of predictions from two previously decoded blocks.
  • intra-frame predictions use previously decoded adjacent blocks within the current field or frame.
  • I-frames Key-frames that exclusively use intra-frame predictions (i.e., I-frames) may be used as access points into a compressed bitstream for channel changes or error recovery. Intra-predictions have a significant effect upon how mismatches from approximations or errors accumulate in video and, therefore, upon the effectiveness of different approximation based memory reduction techniques.
  • FIG. 1 a diagram of a conventional H.264 multi-frame inter-prediction is shown.
  • FIG. 2 a diagram of a conventional H.264 hierarchical group of pictures (GOP) with reference B-frames is shown.
  • Picture storage memory is the most expensive element of a video decoder. Memory storage increasingly dominates decoder costs.
  • Application memory criteria for H.264 are typically specified to be higher than for other commonly specified codecs, such as ISO MPEG-2 (ITU-T H.262) or Society of Motion Picture and Television Engineers (SMPTE) VC-1 (WMV-9, Microsoft Windows Media 9).
  • the additional memory is used to support H.264 inter-frame predictive coding tools, such as multiple reference frames, hierarchical frames and reference B-frames, as shown in FIG. 1 and FIG. 2 .
  • Picture-storage memory can be reduced by downsampling (i.e., reducing resolution horizontally and/or vertically), as described for MPEG-2.
  • downsampling may be merged with the final codec block/picture reconstruction stage.
  • the IDCT (inverse discrete cosine transform) stage may be used and for (ii) H.264 and VC-1, the in-loop (i.e., de-blocking filters) may be used.
  • an efficient implementation may merge the upsample with the sub-pel motion compensated inter-prediction load.
  • the present invention concerns a method for reducing memory utilization in a digital video codec.
  • the method generally comprises the steps of (A) generating a second reference picture by downsampling a first reference picture using a pattern, wherein the pattern (i) comprises a two-dimensional grid and (ii) is unachievable by performing a vertical downsampling and separately performing a horizontal downsampling, (B) generating a third reference picture by upsampling the second reference picture and (C) processing an image in a video signal using the third reference picture.
  • the objects, features and advantages of the present invention include providing a digital video codec that may (i) reduce memory utilization in an H.264
  • FIG. 1 is a diagram of a conventional H.264 multi-frame inter-prediction
  • FIG. 2 is a diagram of a conventional H.264 hierarchical group of pictures with reference B-frames
  • FIG. 3 is a block diagram of a circuit in accordance with a preferred embodiment of the present invention.
  • FIG. 4 is a diagram of H.264 multiple intra-prediction modes
  • FIG. 5 is a diagram of a quincunx sampling pattern
  • FIG. 6 is a flow diagram of an example method for codec memory reduction
  • FIG. 7 is a detailed block diagram of an example architecture of the system.
  • FIG. 8 is a histogram of frame peak signal to noise ratios in an Ice Age sequence.
  • the present invention generally concerns an H.264
  • the MPEG-4/AVC document may be defined by the Moving Pictures Expert Group, International Organization for Standards, Geneva, Switzerland.
  • the H.264 document may be defined by the International Telecommunication Union-Telecommunication Standardization Sector, Geneva, Switzerland.
  • Products with H.264 decoders, transcoders and/or encoders achieving several different price and performance points may be produced using the described techniques.
  • a Blu-ray DiscTM HDTV player using the methods of the present invention instead of the picture storage reduction techniques used successfully for MPEG-2, may achieve significantly superior subjective quality, while reducing a total system memory (e.g., from 256 megabytes (MB) to 128 MB).
  • the circuit (or system) 100 may be implemented as a media processor circuit.
  • the circuit 100 generally comprises a circuit (or module) 102 , a bus 103 and a circuit (or module) 104 .
  • An input signal (e.g., IN) may be received by the circuit 102 .
  • An output signal (e.g., OUT) may be generated and presented by the circuit 102 .
  • the circuit 102 and the circuit 104 may be in communication with each other via the bus 103 to exchange data.
  • the circuit 100 may implement a digital video encoder, a digital video decoder or a digital video transcoder.
  • the signal IN may be (i) a compressed video signal (e.g., for decoders and transcoders), generally referred to as a bitstream, or (ii) an uncompressed video signal (e.g., for encoders).
  • the signal IN may comprise a sequence of progressive-format frames and/or interlace-format fields.
  • the signal IN may be compliant with the H.264 recommendations.
  • the signal OUT may be (i) one or more analog video signals (e.g., decoder), (ii) one or more digital video signals (e.g., for decoders) or (iii) a compressed bitstream (e.g., for encoders and transcoders).
  • the signal OUT generally comprises a sequence of progressive-format frames and/or interlace-format fields.
  • the signal OUT may include synchronization signals suitable for synchronizing a display with the video information.
  • the signal OUT may be generated in analog form as, but is not limited to, an RGB (Red, Green, Blue) signal, an EIA-770 (e.g., YCrCb) signal, an S-video signal and/or a Composite Video Baseband Signal (CVBS).
  • the signal OUT may be generated as, but is not limited to, a High Definition Multimedia Interface (HDMI) signal, a Digital Video Interface (DVI) signal and/or a BT.656 signal.
  • the signal OUT may be formatted as a standard definition signal or a high definition signal.
  • the signal OUT may be compliant with the H.264 recommendations.
  • the circuit 102 may be implemented as a video digital signal processor (VDSP) configured as a codec circuit.
  • the circuit 102 may be operational to decode, encode or transcode the video information received in the signal IN to generate the signal OUT.
  • the circuit 102 may be fabricated on a first chip (or die).
  • the bus 103 may be implemented as a memory bus.
  • the circuit 104 may be implemented as a memory circuit.
  • the circuit 104 may be operational to temporarily buffer frames/fields (pictures or images) and other information used in decoding/encoding/transcoding and formatting the digital video and audio data.
  • a buffering region within the circuit 104 used to buffer decoded pictures/images may be referred to as a decoded picture buffer (DPB).
  • DPB decoded picture buffer
  • the circuit 104 may be fabricated on a second chip (or die) independently of the first chip and external to the circuit 102 .
  • MPEG-4 Part 2 Advanced Simple Profile ASP
  • VC-1 H.264
  • an 8 ⁇ 8 intra-coded block of luma (brightness) samples (pixels) may pass through a discrete cosine transform (DCT) to be coded as a de-correlated set of horizontal, vertical and mixed frequencies.
  • DCT discrete cosine transform
  • a DCT-based de-correlation generally aids compression, as does predictive coding.
  • the DC (e.g., average block value) DCT coefficient may be predictively coded with respect to the DC value of the previously coded block to reduce average bits representing coded blocks.
  • the prediction modes may be available for intra-coded blocks: (i) block DC, (ii) horizontal DCT coefficients and (iii) vertical DCT coefficients.
  • intra-predictions are generally not formed using reconstructed pixels.
  • coded coefficients as predictors
  • inter-prediction error introduced into a frame from sub-sampled stored reference frames may not be propagate through intra prediction into adjacent intra-coded blocks. Therefore, with the above codecs, a major quality effect due to reference frame sub-sampling may be a loss of sharpness that accumulates over successive inter-predicted frames.
  • Such a prediction “drift” effect is generally limited by periodic I-frames.
  • an in-loop de-blocking filter of VC-1 is a potential source of additional error propagation that may cause the simple sub-sampling methods used for MPEG-2 to be less effective.
  • FIG. 4 a diagram of H.264 multiple intra-prediction modes is shown.
  • intra prediction from adjacent pixels of previously coded blocks generally precede transformation.
  • Nine 4 ⁇ 4 luminance (luma), nine 8 ⁇ 8 luma, four 16 ⁇ 16 luma and four 8 ⁇ 8 chrominance (chroma) directional intra prediction modes may be available, as shown in FIG. 4 .
  • adjacent pixels used for intra prediction differ significantly from the specified values, a resulting error may propagate in a direction of the intra prediction across the block.
  • a large area of intra-blocks lies adjacent to an inter-predicted block approximated with reference-frame sub-sampling, the approximation error may propagate across the entire area.
  • the subsequent reference P-frames and reference B-frames may continue to propagate the error.
  • the loop filters in H.264 and VC-1 may cause a similar, although lesser, effect by cumulatively (recursively) propagating pixel values spatially. Since the VC-1 loop filter is generally weaker than the H.264 loop filter (e.g., potentially affects fewer pixels) the in-frame error propagation effect may be further reduced for VC-1.
  • H.264 reference pictures significantly more accurate sub-sampling and interpolation is generally specified for H.264 reference pictures and, to a lesser extent, the VC-1 reference pictures.
  • dynamic buffer management techniques in H.264 may be applied to limit the error.
  • MPEG-2 For MPEG-2, downsampling and upsampling with the simple methods discussed above may be sufficient. In practice, a solution that yields high quality MPEG-2 generally stores reference pictures at half-horizontal resolution by decimating and interpolating only horizontally. For H.264, a few additional options exist for stored luma picture sub-sampling:
  • HHR half-horizontal with averaging downsampling and duplication upsampling.
  • the HHRe method generally includes 8 ⁇ 1 DC (column) differences between each two 8 ⁇ 1 blocks embedded in the seven least significant bits (LSBs) of the 8 ⁇ 1 averaged pixels and the last LSB being the block average LSB. After upsampling, the 8 ⁇ 1 block DCs may be restored and the LSBs loaded with the average LSB.
  • 8 ⁇ 1 DC columnar least significant bits
  • QHR quincunx downsampling may include unfiltered downsampling and horizontal averaging upsampling.
  • QDR quincunx downsampling may have four-neighbor horizontal and vertical mean upsampling in each frame.
  • QMR quincunx downsampling generally includes four-neighbor four-point median upsampling in each frame.
  • QMRa quincunx downsampling with four-neighbor four-point median upsampling, followed by three-point median with the two horizontal neighbors (to improve vertical detail retention).
  • the QMRa method may be applied to frames or fields for progressive and interlaced sources, respectively.
  • the quincunx pattern may be similar to a “5” spot on a dice/card/domino. Each square generally represents a sample location. Each circle may represent a sample retained by the downsampling. The non-circled positions may be eliminated.
  • a quincunx downsampling generally cannot be created by using a horizontal downsampling and a separate vertical downsampling.
  • Downsampling of stored pictures with MPEG-2, MPEG-4 Part 2 ASP and/or VC-1 may be relatively straightforward as the stored frames may be reference frames. Furthermore, all stored reference frames are generally of similar significance for forward propagation of errors between frames or fields. For example, excepting proximity to an I-frame, approximation in any stored frame contributes similarly to temporal drift that propagates into future frames.
  • H.264 generally behaves differently than MPEG-2, MPEG-4 Part 2 ASP and VC-1.
  • the decoded picture buffer may be used for both frame-reordering and for storage of reference frames.
  • some reference frames may only propagate errors forward for a very small number of frames, while other reference frames may influence a much larger number. Limited propagation is generally, although not always, a property of the reference B-frames.
  • Dynamic management of the decoded picture buffer may limit a “worst-case” influence from reducing memory in an H.264 decode.
  • An example set of prioritized rules that may be applied in order until sufficient memory space is freed to store the next decoded field/frame generally follows:
  • Rule 1 Reduce non-reference pictures first since the non-reference pictures generally cause no temporal on-going drift.
  • Rule 2 Reduce reference pictures with short (e.g., 1 or 2 pictures in either direction) temporal potential indirect influences on prediction/reconstruction to reduce the temporal duration of drift due to mismatches.
  • Rule 3 Reduce the “oldest” reference pictures, or the reference pictures that may be temporally furthest from succeeding decoding pictures on the principle that reference pictures that are further away may be used less frequently (in general) for prediction/reconstruction.
  • Rule 4 Take into account a memory bandwidth and a processing cost of downsizing pictures already present in the memory. If the memory bandwidth and/or processing cycles are scarce, try to reduce pictures that are currently being decoded so that (i) the reconstructed information may be stored at reduced resolution once, rather than (ii) stored at full resolution, read back, reduced in resolution and then written back to the memory. Rule 4 may take precedence over rules 1-3 according to the system bottlenecks. If a memory storage is a main system bottleneck, generally apply rules 1, 2, 3 in order. If one or both of the processing cycles and/or memory bandwidth are the main system bottlenecks, rule 4 may be considered until cycles/bandwidth are within budgets. Other management rules may be implemented to meet the criteria of a particular application. For example, the rules may discard the oldest reference frames, rather than sub-sampling the oldest reference frames.
  • the method 120 may implement a reference frame storage process.
  • the method 120 generally comprises a step (or block) 122 , a step (or block) 124 and a step (or block) 126 .
  • the method 120 may be performed by the circuit 100 .
  • reference frames received through the signal IN may be downsampled to save memory.
  • the downsampling may be in accordance with the above four rules.
  • the downsampling may comprise the QMRa quincunx downsampling (method 8). Other quincunx downsampling options may be implemented to meet the criteria of a particular application.
  • the step 122 may comprise a step (or block) 130 and an optional step (or block) 132 .
  • luminance (luma) channels of the reference frames may be downsampled.
  • the step 132 may be implemented to downsample both chrominance (chroma) channels of each of the reference frames. The downsampling may be part of a decoding loop.
  • the particular reference frame may be read from the memory 104 and upsampled to restore an original resolution.
  • the upsampling may be implemented as one or more operations.
  • the upsampling may be implemented as a linear (e.g., an averaging) interpolation in a step (or block) 134 .
  • the upsampling may be implemented as a nonlinear interpolation per a step (or block) 136 .
  • the nonlinear interpolation may include, but is not limited to, a four-neighbor four-point median upsampling and a two-neighbor three-point median upsampling.
  • the upsampling may be implemented as a two-stage interpolation in a step (or block) 138 .
  • the step 138 generally comprises a step (or block) 140 and a step (or block) 142 forming a first stage and a second stage, respectively.
  • the step 140 may perform the four-neighbor four-point median upsampling to fill a missing sample location.
  • the three-point median upsampling using two horizontal neighbors may be performed to complete the interpolation.
  • the step 124 processing of additional pictures received in the signal IN may be performed.
  • the specific processing generally depends on the system being implemented.
  • the step 124 may have a decoding step (or block) 144 for a decoder system wherein the signal OUT is a non-compressed video signal.
  • the step 124 may have an encoding step (or block) 146 .
  • the step 146 generally generates the signal OUT as an encoded bitstream.
  • the step 124 may have a transcoding step (or block) 148 in a transcoder system.
  • the step 148 may generate the signal OUT as an encoded bitstream.
  • a software H.264 decoder was applied to each of the eight memory downsampling/upsampling methods above for every frame of video luminance in the signal IN.
  • the signal IN was generated as an H.264 video bitstream having with the 4:2:0 chroma format. (The 4:2:0 format generally has chroma samples at half the vertical and horizontal resolution of the luma samples.) No dynamic picture buffer management was used in order not to incur additional bandwidth costs. Instead, all pictures were sub-sampled as the pictures were being decoded.
  • Sequence 3 Aeon Flux, 1280 ⁇ 720 pixels, 60 frames/second.
  • Sequence 4 Soccer, 704 ⁇ 480i 60 fields/second, was encoded with a half-second GOP of IBBP, a constant QP of 32 and no 8 ⁇ 8 intra mode.
  • PSNR peak signal to noise ratio
  • dB decibels
  • RMSE root mean squared error
  • test results are generally shown in TABLE I below.
  • Data within the TABLE I generally indicates a mean sequence PSNR (dB) for 50% reduction.
  • the mean PSNR may reflect an average quality of an entire sequence.
  • the visual severity and number of severely impaired frames may indicate a visual assessment of the quality.
  • HHR jagged diagonals, quite blurry, significant and frequent blocking artifacts, particularly in background/flat regions.
  • HHRe all aspects may be somewhat improved, most noticeably blocking. A significant increased memory bandwidth was used for motion compensated prediction. All 8 ⁇ 1 blocks containing appropriate data were accessed to reconstruct the full-resolution data.
  • QHR all quincunx sampling methods may be significantly better than the method HHR in all respects.
  • the method QHR generally has unique jagged weaving artifacts that may be visually unpleasant.
  • QDR generally less sharp and also has more artifacts than the method QMR.
  • QMR a good option for progressive sequences.
  • the method QMR may be worse than the method QMRi on interlaced sequences.
  • QMRi a good option for most interlaced content.
  • the method QMRi may not be as good as the method QMR on progressive sequences.
  • the method QMRi may have a disadvantage, relative to QHR, that thin horizontal lines may be broken or attenuated.
  • QMRa the best practical option for all content. Despite similar PSNR to the other preceding methods on interlaced sequences, the method QMRa usually appears superior due to fewer gross artifacts at the expense of some fine detail distortion/loss. An exception may be that the method HHRe (and the method QHR to a lesser extent) generally appears superior to the method QMRa on detailed long GOP sequences without frequent 1-blocks (such as simple pans in Mobile and Plateau). While the method HHRe may introduce more block artifacts and vertical streaking (from the 1 ⁇ 8 block operations), the method HHRe generally retains more detail. Retaining high vertical detail in panning interlaced sequences may be particularly important for preventing blurring.
  • the method QMRa was generally conceived to improve preservation of high frequency vertical details.
  • the method QMRa may be implemented as a two-step cascaded median with the second stage three-point median re-introducing the horizontal neighbors.
  • a marked improvement was usually noted for the method QMRa relative to the method QMR and the method QMRi.
  • an overall benefit was generally noted on all tested sequences.
  • the circuit 102 generally comprises a circuit (or module) 160 , a circuit (or module) 162 and a circuit (or module) 164 .
  • the circuit 102 may further comprises an optional block (or function) 166 and an optional block (or function) 168 .
  • the block 162 may comprise an optional block (or function) 170 and an optional block (or function) 172 .
  • the circuit 104 generally comprises a circuit (or module) 174 and a circuit (or module) 176 .
  • the circuit 174 may comprise an optional block (or function) 178 and an optional block (or function) 180 .
  • the circuit 160 may be implemented as a cache memory.
  • the circuit 160 is generally operational to buffer picture data. Depending on the storage capacity of the circuit 160 , the data may range from a few macroblocks to one or more macroblock rows, one or more fields or one or more frames.
  • the circuit 162 may be implemented as a memory controller.
  • the circuit 162 generally controls reading and writing data to and from the circuit 104 via the bus 103 .
  • the circuit 162 may be in communication with the circuit 160 to exchange data between the circuit 160 and the circuit 104 .
  • the circuit 164 may be implemented as a processing circuit.
  • the processing may include, but is not limited to, display processing (e.g., a video output module), motion compensation processing, encoding, decoding, transcoding and the like.
  • the circuit 164 may be in communication with the circuit 160 to receive cached data.
  • the circuit 164 may be in communication with the circuit 162 to receive buffered data read from the circuit 104 .
  • the circuit 174 may be implemented as a bus interface circuit.
  • the circuit 174 is generally operational to communicate on the bus 103 and transfer data between the bus 103 and the circuit 176 .
  • the circuit 176 may be implemented as a dynamic random access memory (DRAM).
  • DRAM dynamic random access memory
  • the circuit 176 generally performs the actual storage of the data received via the bus 103 .
  • the circuit 176 may be implemented as a single data rate (SDR) dynamic random access memory (DRAM) or a double data rate (DDR) DRAM.
  • SDR single data rate
  • DRAM dynamic random access memory
  • DDR double data rate
  • Other memory technologies may be implemented to meet the criteria of a particular application.
  • Quincunx downsampling is generally performed during a store operation of the picture data to memory by the circuit 102 .
  • the store operation may be implemented as a store-strip command and/or a write-to-RAM command.
  • the uncompressed picture may be a reference field/frame or a non-reference field/frame.
  • the downsampling may be applied to all fields/frames stored for either display order re-ordering and/or for use as reference fields/frames.
  • the downsampling may be performed in one or more locations within the circuit 100 prior to completion of the store operation, based on a particular application. For example, data may be downsampled by the block 164 prior to caching in the circuit 160 . In another example, downsampling may be performed in the circuit 162 shortly before the data is transferred across the bus 103 , as illustrated by the block 170 . In still another example, data received at the circuit 174 from the bus 103 may be downsampled by the block 178 prior to buffering in the circuit 176 .
  • the store operation may be for one or more uncompressed macroblocks at a time, one or more macroblock rows at a time, an entire field or an entire frame.
  • the data is generally written to the circuit 160 first and subsequently transferred to the circuit 176 .
  • the data may be written just to the circuit 160 , without the transfer to the circuit 176 .
  • a row of macroblocks may be cached in the circuit 160 before copying to the circuit 176 .
  • Non-linear upsampling is generally performed during a read operation of a picture from memory.
  • the read operation may be implemented as a read-strip command, a load-motion-compensated-block command and/or a read-from-RAM command.
  • the upsampling may be performed in one or more locations in the circuit 100 . For best efficiency, the upsampling is usually performed within the circuit 102 .
  • data read from the circuit 160 may be upsampled by the block 168 .
  • data just received from the bus 103 may be upsampled by the block 172 .
  • the circuit 174 may also implement upsampling.
  • the upsampling may be performed by the block 180 prior to placing the data on the bus 103 .
  • the upsampling may be performed during read operations of reference fields/frames prior to any motion compensated interpolation used for decoding/recontructing other blocks/macroblocks.
  • upsampling may be performed as part of generating a displayable-format video signal.
  • Other upsampling conditions may be implemented to meet the criteria of a particular application.
  • data read from memory e.g., circuit 160 and/or circuit 176
  • the method HHR may result in approximately 10% extremely poor quality frames ( ⁇ 32 dB).
  • the methods HHR, HHRe and QHR may also have approximately 10% poor frames (32-36 dB). All options, except the method QMR, generally have approximately 10% or more somewhat impaired frames (36-40 dB). Other frames (>40 dB) are generally visually acceptable. In particular, the method QMR may yield visually acceptable high definition decode quality on the Ice Age clip.
  • Quincunx pixel downsampling (sub-sampling) along with median upsampling may reduce in half a storage criteria of luma samples for H.264 decoding.
  • the quincunx pixel downsampling generally yields quality consistently superior (up to nearly 10 dB PSNR) to traditional horizontal and/or vertical downsampling and interpolation.
  • a two-stage cascaded median upsampling may further improve quality, particularly for highly detailed panning interlaced sequences that benefit from improved vertical detail.
  • memory bandwidth is reduced and memory access is not significantly complicated with simple sub-sampling methods such as the quincunx.
  • FIGS. 3 , 6 and 7 may be implemented using a conventional general purpose digital computer programmed according to the teachings of the present specification, as will be apparent to those skilled in the relevant art(s).
  • Appropriate software coding can readily be prepared by skilled programmers based on the teachings of the present disclosure, as will also be apparent to those skilled in the relevant art(s).
  • the present invention may also be implemented by the preparation of ASICs, FPGAs, or by interconnecting an appropriate network of conventional component circuits, as is described herein, modifications of which will be readily apparent to those skilled in the art(s).
  • the present invention thus may also include a computer product which may be a storage medium including instructions which can be used to program a computer to perform a process in accordance with the present invention.
  • the storage medium can include, but is not limited to, any type of disk including floppy disk, optical disk, CD-ROM, magneto-optical disks, ROMs, RAMS, EPROMs, EEPROMs, Flash memory, magnetic or optical cards, or any type of media suitable for storing electronic instructions.

Abstract

A method for reducing memory utilization in a digital video codec. The method generally includes the steps of (A) generating a second reference picture by downsampling a first reference picture using a pattern, wherein the pattern (i) comprises a two-dimensional grid and (ii) is unachievable by performing a vertical downsampling and separately performing a horizontal downsampling, (B) generating a third reference picture by upsampling the second reference picture and (C) processing an image in a video signal using the third reference picture.

Description

  • This is a continuation of U.S. Ser. No. 11/606,541, filed Nov. 30, 2006, which is incorporated by reference.
  • FIELD OF THE INVENTION
  • The present invention relates to video coding/decoding/transcoding (codecs) generally and, more particularly, to a memory reduced H.264/MPEG-4 AVC codec.
  • BACKGROUND OF THE INVENTION
  • An International Telecommunication Union-Telecommunication Standardization Sector (ITU-T) recommendation H.264 and an International Organization for Standardization/International Electrotechnical Commission (ISO/IEC) recommendation MPEG-4 Part 10 for Advanced Video Coding (MPEG-4/AVC) concern digital video codecs. The H.264|MPEG-4/AVC documents (hereafter simply referred to as H.264) specify both decoder operation (semantics) and compressed video representation (bitstream syntax). Due to efficient syntax and many new predictive options, a conventional H.264 video encoder produces bitstreams that provide MPEG-2 quality with a 50% lower bitrate. As such, video distribution channels such as high definition digital optical media formats (Blu-ray Disc™, HD DVD), cable (High Definition Video On Demand), satellite (DirectTV, DISH Network™), Internet Protocol Television (IPTV), terrestrial high definition television (TV), pay TV (France, England) and mobile systems (3GPP) are deploying H.264 capable equipment. Blue-Ray is a trademark of the Blu-ray Disc Association, Tokyo, Japan. DISH Network is a registered trademark of EchoStar Satellite L.L.C., Englewood, Colo.
  • Two classes of predictions are used in ISO/IEC and ITU block-based hybrid predictions and transform video codecs. Inter-frame predictions and intra-frame predictions are used to remove redundancy, such that improved compression is possible. Inter-frame predictions use previously, sometimes motion compensated, decoded video frames or fields for prediction of current blocks. (Predictive) P-blocks use only one block from a previous frame or field to predict. (Bi-predictive) B-blocks use a (potentially weighted) average of predictions from two previously decoded blocks. In contrast, intra-frame predictions use previously decoded adjacent blocks within the current field or frame. Key-frames that exclusively use intra-frame predictions (i.e., I-frames) may be used as access points into a compressed bitstream for channel changes or error recovery. Intra-predictions have a significant effect upon how mismatches from approximations or errors accumulate in video and, therefore, upon the effectiveness of different approximation based memory reduction techniques.
  • Referring to FIG. 1, a diagram of a conventional H.264 multi-frame inter-prediction is shown. Referring to FIG. 2, a diagram of a conventional H.264 hierarchical group of pictures (GOP) with reference B-frames is shown. Picture storage memory is the most expensive element of a video decoder. Memory storage increasingly dominates decoder costs. Application memory criteria for H.264 are typically specified to be higher than for other commonly specified codecs, such as ISO MPEG-2 (ITU-T H.262) or Society of Motion Picture and Television Engineers (SMPTE) VC-1 (WMV-9, Microsoft Windows Media 9). The additional memory is used to support H.264 inter-frame predictive coding tools, such as multiple reference frames, hierarchical frames and reference B-frames, as shown in FIG. 1 and FIG. 2.
  • Picture-storage memory can be reduced by downsampling (i.e., reducing resolution horizontally and/or vertically), as described for MPEG-2. In practice, downsampling may be merged with the final codec block/picture reconstruction stage. For (i) MPEG-2, the IDCT (inverse discrete cosine transform) stage may be used and for (ii) H.264 and VC-1, the in-loop (i.e., de-blocking filters) may be used. Similarly, an efficient implementation may merge the upsample with the sub-pel motion compensated inter-prediction load. For MPEG-2, unlike H.264|MPEG-4/AVC, horizontal sub-sampling by a factor of two with either of the following two simple methods yields good quality: (i) downsample without filtering (i.e., drop alternate columns) and upsample with bilinear interpolation and (ii) downsample by averaging (i.e., adjacent columns) and upsample without filtering (i.e., duplication, sample and hold).
  • SUMMARY OF THE INVENTION
  • The present invention concerns a method for reducing memory utilization in a digital video codec. The method generally comprises the steps of (A) generating a second reference picture by downsampling a first reference picture using a pattern, wherein the pattern (i) comprises a two-dimensional grid and (ii) is unachievable by performing a vertical downsampling and separately performing a horizontal downsampling, (B) generating a third reference picture by upsampling the second reference picture and (C) processing an image in a video signal using the third reference picture.
  • The objects, features and advantages of the present invention include providing a digital video codec that may (i) reduce memory utilization in an H.264|MPEG-4/AVC method/system, (ii) provide better performance compared with conventional techniques, (iii) maintain relatively simple memory access with no bandwidth penalty, (iv) use quincunx patterns for downsampling pictures, (v) use nonlinear upsampling of pictures and/or (vi) use two-stage median upsampling of pictures.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • These and other objects, features and advantages of the present invention will be apparent from the following detailed description and the appended claims and drawings in which:
  • FIG. 1 is a diagram of a conventional H.264 multi-frame inter-prediction;
  • FIG. 2 is a diagram of a conventional H.264 hierarchical group of pictures with reference B-frames;
  • FIG. 3 is a block diagram of a circuit in accordance with a preferred embodiment of the present invention;
  • FIG. 4 is a diagram of H.264 multiple intra-prediction modes;
  • FIG. 5 is a diagram of a quincunx sampling pattern;
  • FIG. 6 is a flow diagram of an example method for codec memory reduction;
  • FIG. 7 is a detailed block diagram of an example architecture of the system; and
  • FIG. 8 is a histogram of frame peak signal to noise ratios in an Ice Age sequence.
  • DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
  • The present invention generally concerns an H.264|MPEG4-AVC codec implementation with a significantly reduced memory size. The MPEG-4/AVC document may be defined by the Moving Pictures Expert Group, International Organization for Standards, Geneva, Switzerland. The H.264 document may be defined by the International Telecommunication Union-Telecommunication Standardization Sector, Geneva, Switzerland.
  • Products with H.264 decoders, transcoders and/or encoders achieving several different price and performance points may be produced using the described techniques. For example, a Blu-ray Disc™ HDTV player using the methods of the present invention, instead of the picture storage reduction techniques used successfully for MPEG-2, may achieve significantly superior subjective quality, while reducing a total system memory (e.g., from 256 megabytes (MB) to 128 MB).
  • Referring to FIG. 3, a block diagram of a circuit 100 is shown in accordance with a preferred embodiment of the present invention. The circuit (or system) 100 may be implemented as a media processor circuit. The circuit 100 generally comprises a circuit (or module) 102, a bus 103 and a circuit (or module) 104. An input signal (e.g., IN) may be received by the circuit 102. An output signal (e.g., OUT) may be generated and presented by the circuit 102. The circuit 102 and the circuit 104 may be in communication with each other via the bus 103 to exchange data. The circuit 100 may implement a digital video encoder, a digital video decoder or a digital video transcoder.
  • The signal IN may be (i) a compressed video signal (e.g., for decoders and transcoders), generally referred to as a bitstream, or (ii) an uncompressed video signal (e.g., for encoders). The signal IN may comprise a sequence of progressive-format frames and/or interlace-format fields. For decoders and transcoders, the signal IN may be compliant with the H.264 recommendations.
  • The signal OUT may be (i) one or more analog video signals (e.g., decoder), (ii) one or more digital video signals (e.g., for decoders) or (iii) a compressed bitstream (e.g., for encoders and transcoders). The signal OUT generally comprises a sequence of progressive-format frames and/or interlace-format fields.
  • Regarding decoders, the signal OUT may include synchronization signals suitable for synchronizing a display with the video information. The signal OUT may be generated in analog form as, but is not limited to, an RGB (Red, Green, Blue) signal, an EIA-770 (e.g., YCrCb) signal, an S-video signal and/or a Composite Video Baseband Signal (CVBS). In digital form, the signal OUT may be generated as, but is not limited to, a High Definition Multimedia Interface (HDMI) signal, a Digital Video Interface (DVI) signal and/or a BT.656 signal. The signal OUT may be formatted as a standard definition signal or a high definition signal. Regarding encoders and transcoders, the signal OUT may be compliant with the H.264 recommendations.
  • The circuit 102 may be implemented as a video digital signal processor (VDSP) configured as a codec circuit. The circuit 102 may be operational to decode, encode or transcode the video information received in the signal IN to generate the signal OUT. The circuit 102 may be fabricated on a first chip (or die). The bus 103 may be implemented as a memory bus.
  • The circuit 104 may be implemented as a memory circuit. The circuit 104 may be operational to temporarily buffer frames/fields (pictures or images) and other information used in decoding/encoding/transcoding and formatting the digital video and audio data. A buffering region within the circuit 104 used to buffer decoded pictures/images may be referred to as a decoded picture buffer (DPB). The circuit 104 may be fabricated on a second chip (or die) independently of the first chip and external to the circuit 102.
  • Sub-sampling drift with MPEG-2, MPEG-4 Part 2 Advanced Simple Profile (ASP), VC-1 and H.264 have been considered. In MPEG-2, an 8×8 intra-coded block of luma (brightness) samples (pixels) may pass through a discrete cosine transform (DCT) to be coded as a de-correlated set of horizontal, vertical and mixed frequencies. A DCT-based de-correlation generally aids compression, as does predictive coding. In MPEG-2, the DC (e.g., average block value) DCT coefficient may be predictively coded with respect to the DC value of the previously coded block to reduce average bits representing coded blocks.
  • Similarly, with MPEG-4 Part 2 ASP and VC-1, the prediction modes may be available for intra-coded blocks: (i) block DC, (ii) horizontal DCT coefficients and (iii) vertical DCT coefficients. As with MPEG-2, intra-predictions are generally not formed using reconstructed pixels. By using coded coefficients as predictors, inter-prediction error introduced into a frame from sub-sampled stored reference frames may not be propagate through intra prediction into adjacent intra-coded blocks. Therefore, with the above codecs, a major quality effect due to reference frame sub-sampling may be a loss of sharpness that accumulates over successive inter-predicted frames. Such a prediction “drift” effect is generally limited by periodic I-frames. However, an in-loop de-blocking filter of VC-1 is a potential source of additional error propagation that may cause the simple sub-sampling methods used for MPEG-2 to be less effective.
  • Referring to FIG. 4, a diagram of H.264 multiple intra-prediction modes is shown. Under H.264, intra prediction from adjacent pixels of previously coded blocks generally precede transformation. Nine 4×4 luminance (luma), nine 8×8 luma, four 16×16 luma and four 8×8 chrominance (chroma) directional intra prediction modes may be available, as shown in FIG. 4. If adjacent pixels used for intra prediction differ significantly from the specified values, a resulting error may propagate in a direction of the intra prediction across the block. If a large area of intra-blocks lies adjacent to an inter-predicted block approximated with reference-frame sub-sampling, the approximation error may propagate across the entire area. If the current frame is a reference frame for inter-prediction, the subsequent reference P-frames and reference B-frames may continue to propagate the error. The loop filters in H.264 and VC-1 may cause a similar, although lesser, effect by cumulatively (recursively) propagating pixel values spatially. Since the VC-1 loop filter is generally weaker than the H.264 loop filter (e.g., potentially affects fewer pixels) the in-frame error propagation effect may be further reduced for VC-1.
  • To mitigate a severity of in-frame propagation of prediction error, significantly more accurate sub-sampling and interpolation is generally specified for H.264 reference pictures and, to a lesser extent, the VC-1 reference pictures. In addition, dynamic buffer management techniques in H.264 may be applied to limit the error.
  • For MPEG-2, downsampling and upsampling with the simple methods discussed above may be sufficient. In practice, a solution that yields high quality MPEG-2 generally stores reference pictures at half-horizontal resolution by decimating and interpolating only horizontally. For H.264, a few additional options exist for stored luma picture sub-sampling:
  • Method 1. HHR: half-horizontal with averaging downsampling and duplication upsampling.
  • Method 2. HHRe: enhanced half-horizontal averaging/duplication. The HHRe method generally includes 8×1 DC (column) differences between each two 8×1 blocks embedded in the seven least significant bits (LSBs) of the 8×1 averaged pixels and the last LSB being the block average LSB. After upsampling, the 8×1 block DCs may be restored and the LSBs loaded with the average LSB.
  • Method 3. QHR: quincunx downsampling may include unfiltered downsampling and horizontal averaging upsampling.
  • Method 4. QVR: quincunx downsampling and vertical averaging upsampling in each field.
  • Method 5. QDR: quincunx downsampling may have four-neighbor horizontal and vertical mean upsampling in each frame.
  • Method 6. QMR: quincunx downsampling generally includes four-neighbor four-point median upsampling in each frame.
  • Method 7. QMRi: quincunx downsampling including 4-neighbor four-point median upsampling in each field.
  • Method 8. QMRa: quincunx downsampling with four-neighbor four-point median upsampling, followed by three-point median with the two horizontal neighbors (to improve vertical detail retention). The QMRa method may be applied to frames or fields for progressive and interlaced sources, respectively.
  • Referring to FIG. 5, a diagram of a quincunx sampling pattern is shown. The quincunx pattern may be similar to a “5” spot on a dice/card/domino. Each square generally represents a sample location. Each circle may represent a sample retained by the downsampling. The non-circled positions may be eliminated. A quincunx downsampling generally cannot be created by using a horizontal downsampling and a separate vertical downsampling.
  • Downsampling of stored pictures with MPEG-2, MPEG-4 Part 2 ASP and/or VC-1 may be relatively straightforward as the stored frames may be reference frames. Furthermore, all stored reference frames are generally of similar significance for forward propagation of errors between frames or fields. For example, excepting proximity to an I-frame, approximation in any stored frame contributes similarly to temporal drift that propagates into future frames.
  • H.264 generally behaves differently than MPEG-2, MPEG-4 Part 2 ASP and VC-1. With H.264, the decoded picture buffer may be used for both frame-reordering and for storage of reference frames. Furthermore, some reference frames may only propagate errors forward for a very small number of frames, while other reference frames may influence a much larger number. Limited propagation is generally, although not always, a property of the reference B-frames.
  • Dynamic management of the decoded picture buffer may limit a “worst-case” influence from reducing memory in an H.264 decode. An example set of prioritized rules that may be applied in order until sufficient memory space is freed to store the next decoded field/frame generally follows:
  • Rule 1: Reduce non-reference pictures first since the non-reference pictures generally cause no temporal on-going drift.
  • Rule 2: Reduce reference pictures with short (e.g., 1 or 2 pictures in either direction) temporal potential indirect influences on prediction/reconstruction to reduce the temporal duration of drift due to mismatches.
  • Rule 3: Reduce the “oldest” reference pictures, or the reference pictures that may be temporally furthest from succeeding decoding pictures on the principle that reference pictures that are further away may be used less frequently (in general) for prediction/reconstruction.
  • Rule 4: Take into account a memory bandwidth and a processing cost of downsizing pictures already present in the memory. If the memory bandwidth and/or processing cycles are scarce, try to reduce pictures that are currently being decoded so that (i) the reconstructed information may be stored at reduced resolution once, rather than (ii) stored at full resolution, read back, reduced in resolution and then written back to the memory. Rule 4 may take precedence over rules 1-3 according to the system bottlenecks. If a memory storage is a main system bottleneck, generally apply rules 1, 2, 3 in order. If one or both of the processing cycles and/or memory bandwidth are the main system bottlenecks, rule 4 may be considered until cycles/bandwidth are within budgets. Other management rules may be implemented to meet the criteria of a particular application. For example, the rules may discard the oldest reference frames, rather than sub-sampling the oldest reference frames.
  • Referring to FIG. 6, a flow diagram of an example method 120 for codec memory reduction is shown. The method (or process) 120 may implement a reference frame storage process. The method 120 generally comprises a step (or block) 122, a step (or block) 124 and a step (or block) 126. The method 120 may be performed by the circuit 100.
  • In the step 122, reference frames received through the signal IN may be downsampled to save memory. The downsampling may be in accordance with the above four rules. The downsampling may comprise the QMRa quincunx downsampling (method 8). Other quincunx downsampling options may be implemented to meet the criteria of a particular application.
  • Generally, the step 122 may comprise a step (or block) 130 and an optional step (or block) 132. In the step 130, luminance (luma) channels of the reference frames may be downsampled. In one or more embodiments, the step 132 may be implemented to downsample both chrominance (chroma) channels of each of the reference frames. The downsampling may be part of a decoding loop.
  • When a particular reference frame is appropriate for further processing, the particular reference frame may be read from the memory 104 and upsampled to restore an original resolution. The upsampling may be implemented as one or more operations. For example, the upsampling may be implemented as a linear (e.g., an averaging) interpolation in a step (or block) 134. In another example, the upsampling may be implemented as a nonlinear interpolation per a step (or block) 136. The nonlinear interpolation may include, but is not limited to, a four-neighbor four-point median upsampling and a two-neighbor three-point median upsampling. In still another example, the upsampling may be implemented as a two-stage interpolation in a step (or block) 138.
  • The step 138 generally comprises a step (or block) 140 and a step (or block) 142 forming a first stage and a second stage, respectively. The step 140 may perform the four-neighbor four-point median upsampling to fill a missing sample location. In the step 142, the three-point median upsampling using two horizontal neighbors may be performed to complete the interpolation.
  • In the step 124, processing of additional pictures received in the signal IN may be performed. The specific processing generally depends on the system being implemented. For example, the step 124 may have a decoding step (or block) 144 for a decoder system wherein the signal OUT is a non-compressed video signal. In an encoder system, the step 124 may have an encoding step (or block) 146. The step 146 generally generates the signal OUT as an encoded bitstream. Furthermore, the step 124 may have a transcoding step (or block) 148 in a transcoder system. The step 148 may generate the signal OUT as an encoded bitstream.
  • A software H.264 decoder was applied to each of the eight memory downsampling/upsampling methods above for every frame of video luminance in the signal IN. The signal IN was generated as an H.264 video bitstream having with the 4:2:0 chroma format. (The 4:2:0 format generally has chroma samples at half the vertical and horizontal resolution of the luma samples.) No dynamic picture buffer management was used in order not to incur additional bandwidth costs. Instead, all pictures were sub-sampled as the pictures were being decoded. Since block DC artifacts may be much more visually annoying in chroma than in luma and sub-sampling the luma generally saves twice as much memory as sub-sampling the chroma (for the 4:2:0 format), only the luminance component of the reference frames were downsampled. The following publicly available coded H.264 sequences were used for simulation (encoded with a one second group of pictures (GOP) of IBP, no 8×8 Intra/Transform and a QP ˜[12,26]):
  • Sequence 1. Ice Age, 1904×1056 pixels, 60 frames/second.
  • Sequence 2. Warren Miller, 1920×1080 pixels, 60 frames/second.
  • Sequence 3. Aeon Flux, 1280×720 pixels, 60 frames/second.
  • The following non-publicly available video sequences were encoded with a JM 10.2 H.264 reference encoder:
  • Sequence 4. Soccer, 704×480i 60 fields/second, was encoded with a half-second GOP of IBBP, a constant QP of 32 and no 8×8 intra mode.
  • Sequence 5. Mobile Teranex, Parkrun, and Stockholm, 1920×1080i 60 fields/second, was encoded with a five second GOP of IBBP and a constant QP of 32 with all intra modes including 8×8 intra.
  • Sequence 6. Plateau, 1920×1080i 60 fields/second, was encoded with a one second GOP of IPPP and a constant QP of 32 with all intra modes including 8×8 intra.
  • The results of each method applied to each sequence were compared with a peak signal to noise ratio (PSNR) clipped to a maximum of 100 decibels (dB), relative to a full reference decode. (PSNR in dB may be computed as 20*log 10(255/RMSE), where RMSE may be a root mean squared error.) While the results correspond to a visual ranking, the results generally do not indicate a magnitude of quality differences.
  • The test results are generally shown in TABLE I below. Data within the TABLE I generally indicates a mean sequence PSNR (dB) for 50% reduction. The mean PSNR may reflect an average quality of an entire sequence. The visual severity and number of severely impaired frames may indicate a visual assessment of the quality.
  • TABLE I
    HHR HHRe QHR QVR QDR QMR QMRi QMRa
    Ice Age 45.5 46.0 49.0 49.5 51.4 53.3 51.0 53.5
    Warren 36.9 40.7 38.9 42.2 41.2 43.9 41.6 43.9
    Miller
    Aeon 49.1 45.9 50.9 46.9 48.6 54.6 52.1 54.6
    Flux
    Soccer 26.8 36.5 34.2 NA NA NA 34.1 35.2
    Mobile 23.6 26.2 24.4 25.4 24.9 27.6 26.3 26.8
    Plateau 31.3 35.4 33.8 NA NA NA 33.8 34.3
    Parkrun 21.0 23.3 25.5 NA NA NA 23.8 24.8
    Stock- 23.8 28.5 28.0 NA NA NA 24.8 26.4
    holm
  • Several visual assessments of the simulations generally follow:
  • Assessment 1. HHR: jagged diagonals, quite blurry, significant and frequent blocking artifacts, particularly in background/flat regions.
  • Assessment 2. HHRe: all aspects may be somewhat improved, most noticeably blocking. A significant increased memory bandwidth was used for motion compensated prediction. All 8×1 blocks containing appropriate data were accessed to reconstruct the full-resolution data.
  • Assessment 3. QHR: all quincunx sampling methods may be significantly better than the method HHR in all respects. The method QHR generally has unique jagged weaving artifacts that may be visually unpleasant.
  • Assessment 4. QVR: somewhat better than the method QHR, but has similar (although vertical) weaving artifacts.
  • Assessment 5. QDR: generally less sharp and also has more artifacts than the method QMR.
  • Assessment 6. QMR: a good option for progressive sequences. The method QMR may be worse than the method QMRi on interlaced sequences.
  • Assessment 7. QMRi: a good option for most interlaced content. However, the method QMRi may not be as good as the method QMR on progressive sequences. The method QMRi may have a disadvantage, relative to QHR, that thin horizontal lines may be broken or attenuated.
  • Assessment 8. QMRa: the best practical option for all content. Despite similar PSNR to the other preceding methods on interlaced sequences, the method QMRa usually appears superior due to fewer gross artifacts at the expense of some fine detail distortion/loss. An exception may be that the method HHRe (and the method QHR to a lesser extent) generally appears superior to the method QMRa on detailed long GOP sequences without frequent 1-blocks (such as simple pans in Mobile and Plateau). While the method HHRe may introduce more block artifacts and vertical streaking (from the 1×8 block operations), the method HHRe generally retains more detail. Retaining high vertical detail in panning interlaced sequences may be particularly important for preventing blurring. The method QMRa was generally conceived to improve preservation of high frequency vertical details. The method QMRa may be implemented as a two-step cascaded median with the second stage three-point median re-introducing the horizontal neighbors. For the sequences that improved with the method HHRe, a marked improvement was usually noted for the method QMRa relative to the method QMR and the method QMRi. Furthermore, an overall benefit was generally noted on all tested sequences.
  • Referring to FIG. 7, a detailed block diagram of an example architecture of the circuit 100 is shown. The circuit 102 generally comprises a circuit (or module) 160, a circuit (or module) 162 and a circuit (or module) 164. The circuit 102 may further comprises an optional block (or function) 166 and an optional block (or function) 168. The block 162 may comprise an optional block (or function) 170 and an optional block (or function) 172. The circuit 104 generally comprises a circuit (or module) 174 and a circuit (or module) 176. The circuit 174 may comprise an optional block (or function) 178 and an optional block (or function) 180.
  • The circuit 160 may be implemented as a cache memory. The circuit 160 is generally operational to buffer picture data. Depending on the storage capacity of the circuit 160, the data may range from a few macroblocks to one or more macroblock rows, one or more fields or one or more frames.
  • The circuit 162 may be implemented as a memory controller. The circuit 162 generally controls reading and writing data to and from the circuit 104 via the bus 103. The circuit 162 may be in communication with the circuit 160 to exchange data between the circuit 160 and the circuit 104.
  • The circuit 164 may be implemented as a processing circuit. The processing may include, but is not limited to, display processing (e.g., a video output module), motion compensation processing, encoding, decoding, transcoding and the like. The circuit 164 may be in communication with the circuit 160 to receive cached data. The circuit 164 may be in communication with the circuit 162 to receive buffered data read from the circuit 104.
  • The circuit 174 may be implemented as a bus interface circuit. The circuit 174 is generally operational to communicate on the bus 103 and transfer data between the bus 103 and the circuit 176.
  • The circuit 176 may be implemented as a dynamic random access memory (DRAM). The circuit 176 generally performs the actual storage of the data received via the bus 103. The circuit 176 may be implemented as a single data rate (SDR) dynamic random access memory (DRAM) or a double data rate (DDR) DRAM. Other memory technologies may be implemented to meet the criteria of a particular application.
  • Quincunx downsampling is generally performed during a store operation of the picture data to memory by the circuit 102. The store operation may be implemented as a store-strip command and/or a write-to-RAM command. The uncompressed picture may be a reference field/frame or a non-reference field/frame. Generally, the downsampling may be applied to all fields/frames stored for either display order re-ordering and/or for use as reference fields/frames.
  • The downsampling may be performed in one or more locations within the circuit 100 prior to completion of the store operation, based on a particular application. For example, data may be downsampled by the block 164 prior to caching in the circuit 160. In another example, downsampling may be performed in the circuit 162 shortly before the data is transferred across the bus 103, as illustrated by the block 170. In still another example, data received at the circuit 174 from the bus 103 may be downsampled by the block 178 prior to buffering in the circuit 176.
  • Typically, the store operation may be for one or more uncompressed macroblocks at a time, one or more macroblock rows at a time, an entire field or an entire frame. The data is generally written to the circuit 160 first and subsequently transferred to the circuit 176. Depending on the storage capacity of the circuit 160, the data may be written just to the circuit 160, without the transfer to the circuit 176. If the circuit 160 is of an intermediate in storage capacity, a row of macroblocks may be cached in the circuit 160 before copying to the circuit 176.
  • Non-linear upsampling is generally performed during a read operation of a picture from memory. The read operation may be implemented as a read-strip command, a load-motion-compensated-block command and/or a read-from-RAM command. The upsampling may be performed in one or more locations in the circuit 100. For best efficiency, the upsampling is usually performed within the circuit 102. For example, data read from the circuit 160 may be upsampled by the block 168. In another example, data just received from the bus 103 may be upsampled by the block 172. The circuit 174 may also implement upsampling. For example, the upsampling may be performed by the block 180 prior to placing the data on the bus 103.
  • Several conditions generally utilize the upsampling operation. In a first condition, the upsampling may be performed during read operations of reference fields/frames prior to any motion compensated interpolation used for decoding/recontructing other blocks/macroblocks. In a second condition, upsampling may be performed as part of generating a displayable-format video signal. Other upsampling conditions may be implemented to meet the criteria of a particular application. Generally, data read from memory (e.g., circuit 160 and/or circuit 176) may be upsampled prior to processing by the circuit 164.
  • Referring to FIG. 8, a histogram of frame peak signal to noise ratios in the Ice Age sequence is shown. From TABLE I, the method HHR may result in approximately 10% extremely poor quality frames (<32 dB). The methods HHR, HHRe and QHR may also have approximately 10% poor frames (32-36 dB). All options, except the method QMR, generally have approximately 10% or more somewhat impaired frames (36-40 dB). Other frames (>40 dB) are generally visually acceptable. In particular, the method QMR may yield visually acceptable high definition decode quality on the Ice Age clip.
  • Quincunx pixel downsampling (sub-sampling) along with median upsampling may reduce in half a storage criteria of luma samples for H.264 decoding. The quincunx pixel downsampling generally yields quality consistently superior (up to nearly 10 dB PSNR) to traditional horizontal and/or vertical downsampling and interpolation. A two-stage cascaded median upsampling may further improve quality, particularly for highly detailed panning interlaced sequences that benefit from improved vertical detail. Additionally, in contrast to block or segment-based memory compression methods, memory bandwidth is reduced and memory access is not significantly complicated with simple sub-sampling methods such as the quincunx.
  • The function performed by the diagrams of FIGS. 3, 6 and 7 may be implemented using a conventional general purpose digital computer programmed according to the teachings of the present specification, as will be apparent to those skilled in the relevant art(s). Appropriate software coding can readily be prepared by skilled programmers based on the teachings of the present disclosure, as will also be apparent to those skilled in the relevant art(s).
  • The present invention may also be implemented by the preparation of ASICs, FPGAs, or by interconnecting an appropriate network of conventional component circuits, as is described herein, modifications of which will be readily apparent to those skilled in the art(s).
  • The present invention thus may also include a computer product which may be a storage medium including instructions which can be used to program a computer to perform a process in accordance with the present invention. The storage medium can include, but is not limited to, any type of disk including floppy disk, optical disk, CD-ROM, magneto-optical disks, ROMs, RAMS, EPROMs, EEPROMs, Flash memory, magnetic or optical cards, or any type of media suitable for storing electronic instructions.
  • While the invention has been particularly shown and described with reference to the preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made without departing from the scope of the invention.

Claims (20)

1. A method for reducing memory utilization in a digital video codec, comprising the steps of
(A) generating a second picture by downsampling a first picture using a pattern, wherein said pattern is unachievable by performing a vertical downsampling and separately performing a horizontal downsampling;
(B) generating a third picture by upsampling said second picture; and
(C) decoding a fourth picture by inter-predicting from said third picture.
2. The method according to claim 1, wherein said pattern comprises a quincunx pattern.
3. The method according to claim 1, wherein said upsampling comprises (i) a first stage a four-neighbor, four-point median upsampling and (ii) a second stage two-neighbor, three-point median upsampling.
4. The method according to claim 1, wherein said upsampling comprises a nonlinear interpolation.
5. The method according to claim 1, wherein said digital video codec is compliant with at least one of (i) an H.264 recommendation and (ii) an MPEG-4/Advanced Video Coding recommendation.
6. The method according to claim 1, wherein said downsampling is applied to a luminance channel and two chrominance channels of said first picture.
7. A method for reducing memory utilization in a digital video codec, comprising the steps of:
(A) generating a second picture by downsampling a first picture;
(B) generating a third picture by upsampling said second picture, wherein said upsampling comprises a nonlinear interpolation; and
(C) decoding a fourth picture by inter-predicting from said third picture.
8. The method according to claim 7, wherein said nonlinear interpolation comprises a two-stage interpolation.
9. The method according to claim 7, wherein said nonlinear interpolation comprises (i) a first stage a four-neighbor, four-point median upsampling and (ii) a second stage two-neighbor, three-point median upsampling.
10. The method according to claim 7, wherein said downsampling comprises a quincunx downsampling.
11. A system comprising:
a circuit configured to (i) generate a second picture by downsampling a first picture using a pattern, wherein said pattern is unachievable by performing a vertical downsampling and separately performing a horizontal downsampling, (ii) generate a third picture by upsampling said second picture and (iii) decode a fourth picture by inter-predicting from said third picture; and
a memory configured to buffer said second picture.
12. The system according to claim 11, wherein (i) said pattern comprises a quincunx pattern and (ii) said upsampling comprises a nonlinear interpolation.
13. The system according to claim 11, wherein said circuit comprises a memory controller configured to (i) downsample a particular picture in a quincunx pattern and (ii) transfer said particular picture to said memory.
14. The system according to claim 11, wherein said circuit comprises a memory controller configured to (i) receive a particular picture from said memory and (ii) upsample said particular picture with a nonlinear interpolation.
15. The system according to claim 11, wherein said upsampling comprises (i) a first stage a four-neighbor, four-point median upsampling and (ii) a second stage two-neighbor, three-point median upsampling.
16. A system comprising:
a circuit configured to (i) generate a second picture by downsampling a first picture using a quincunx pattern, (ii) generate a third picture by upsampling said second picture using a nonlinear interpolation and (iii) decode a fourth picture by inter-prediction from said third picture; and
a first memory configured to buffer said second picture.
17. The system according to claim 16, wherein said circuit is further configured to control buffering of a plurality of fifth pictures in a second memory.
18. The system according to claim 17, wherein (i) said circuit and said first memory are formed on a first chip and (ii) said second memory is formed on a second chip.
19. The system according to claim 17, wherein said second memory is configured to (i) downsample said fifth pictures received from said circuit using said quincunx pattern and (ii) upsample said fifth pictures prior to transmission to said circuit using said nonlinear interpolation.
20. The system according to claim 16, wherein said nonlinear interpolation comprises (i) a first stage a four-neighbor, four-point median upsampling and (ii) a second stage two-neighbor, three-point median upsampling.
US13/311,972 2006-11-30 2011-12-06 Memory reduced h264/mpeg-4 avc codec Abandoned US20120076208A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US13/311,972 US20120076208A1 (en) 2006-11-30 2011-12-06 Memory reduced h264/mpeg-4 avc codec

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US11/606,541 US8121195B2 (en) 2006-11-30 2006-11-30 Memory reduced H264/MPEG-4 AVC codec
US13/311,972 US20120076208A1 (en) 2006-11-30 2011-12-06 Memory reduced h264/mpeg-4 avc codec

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
US11/606,541 Continuation US8121195B2 (en) 2006-11-30 2006-11-30 Memory reduced H264/MPEG-4 AVC codec

Publications (1)

Publication Number Publication Date
US20120076208A1 true US20120076208A1 (en) 2012-03-29

Family

ID=39468212

Family Applications (2)

Application Number Title Priority Date Filing Date
US11/606,541 Active 2030-12-22 US8121195B2 (en) 2006-11-30 2006-11-30 Memory reduced H264/MPEG-4 AVC codec
US13/311,972 Abandoned US20120076208A1 (en) 2006-11-30 2011-12-06 Memory reduced h264/mpeg-4 avc codec

Family Applications Before (1)

Application Number Title Priority Date Filing Date
US11/606,541 Active 2030-12-22 US8121195B2 (en) 2006-11-30 2006-11-30 Memory reduced H264/MPEG-4 AVC codec

Country Status (7)

Country Link
US (2) US8121195B2 (en)
EP (1) EP2100449A4 (en)
JP (1) JP5022446B2 (en)
KR (1) KR101182343B1 (en)
CN (1) CN101543069B (en)
TW (1) TWI382769B (en)
WO (1) WO2008066601A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10171811B2 (en) 2012-04-16 2019-01-01 Samsung Electronics Co., Ltd. Method and apparatus for determining reference picture set of image

Families Citing this family (24)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8121195B2 (en) * 2006-11-30 2012-02-21 Lsi Corporation Memory reduced H264/MPEG-4 AVC codec
US8121189B2 (en) * 2007-09-20 2012-02-21 Microsoft Corporation Video decoding using created reference pictures
KR101446771B1 (en) * 2008-01-30 2014-10-06 삼성전자주식회사 Apparatus of encoding image and apparatus of decoding image
US9848209B2 (en) * 2008-04-02 2017-12-19 Microsoft Technology Licensing, Llc Adaptive error detection for MPEG-2 error concealment
US9924184B2 (en) 2008-06-30 2018-03-20 Microsoft Technology Licensing, Llc Error detection, protection and recovery for video decoding
US9788018B2 (en) * 2008-06-30 2017-10-10 Microsoft Technology Licensing, Llc Error concealment techniques in video decoding
TWI410136B (en) * 2008-08-08 2013-09-21 Innolux Corp Data compression method and video processing system and display using thereof
US9131241B2 (en) * 2008-11-25 2015-09-08 Microsoft Technology Licensing, Llc Adjusting hardware acceleration for video playback based on error detection
WO2010092740A1 (en) * 2009-02-10 2010-08-19 パナソニック株式会社 Image processing apparatus, image processing method, program and integrated circuit
US8340510B2 (en) * 2009-07-17 2012-12-25 Microsoft Corporation Implementing channel start and file seek for decoder
KR20110011361A (en) * 2009-07-28 2011-02-08 삼성전자주식회사 Apparatus and method for image data encoding/decoding using sampling
WO2012092841A1 (en) * 2011-01-03 2012-07-12 Mediatek Inc. Method of filter-unit based in-loop filtering
CN103119943B (en) * 2011-06-03 2017-03-08 太阳专利托管公司 Method for encoding images and picture decoding method
US9001895B2 (en) * 2011-06-08 2015-04-07 Panasonic Intellectual Property Management Co., Ltd. Image display device and image processing device
US9531990B1 (en) * 2012-01-21 2016-12-27 Google Inc. Compound prediction using multiple sources or prediction modes
US8737824B1 (en) 2012-03-09 2014-05-27 Google Inc. Adaptively encoding a media stream with compound prediction
GB2501535A (en) * 2012-04-26 2013-10-30 Sony Corp Chrominance Processing in High Efficiency Video Codecs
US9185414B1 (en) 2012-06-29 2015-11-10 Google Inc. Video encoding using variance
US10375405B2 (en) * 2012-10-05 2019-08-06 Qualcomm Incorporated Motion field upsampling for scalable coding based on high efficiency video coding
US9628790B1 (en) 2013-01-03 2017-04-18 Google Inc. Adaptive composite intra prediction for image and video compression
EP2983360A4 (en) * 2013-04-05 2016-11-16 Mitsubishi Electric Corp Color image encoding apparatus, color image decoding apparatus, color image encoding method, and color image decoding method
US9374578B1 (en) 2013-05-23 2016-06-21 Google Inc. Video coding using combined inter and intra predictors
US9609343B1 (en) 2013-12-20 2017-03-28 Google Inc. Video coding using compound prediction
CN110503928B (en) * 2018-05-18 2021-09-07 晶典有限公司 Generating a five-dot shaped video stream for a light modulating backplane having configurable multi-electrode pixels

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5923815A (en) * 1996-02-13 1999-07-13 Samsung Electronics Co., Ltd. Apparatus and method for decoding MPEG video data
US6061402A (en) * 1994-10-11 2000-05-09 Hitachi America, Ltd. Methods and apparatus for efficiently decoding bi-directionally coded image data
US20070041448A1 (en) * 2005-08-17 2007-02-22 Miller Casey L Artifact and noise reduction in MPEG video
US20070058722A1 (en) * 2001-10-17 2007-03-15 Shinya Kadono Moving picture coding method and moving picture decoding method
US7430238B2 (en) * 2004-12-10 2008-09-30 Micronas Usa, Inc. Shared pipeline architecture for motion vector prediction and residual decoding
US8121195B2 (en) * 2006-11-30 2012-02-21 Lsi Corporation Memory reduced H264/MPEG-4 AVC codec

Family Cites Families (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE69129871T2 (en) * 1990-03-21 1999-03-04 Philips Electronics Nv Method and device for signal processing
FI88091C (en) * 1990-09-25 1993-03-25 Salora Oy FOERFARANDE OCH ANORDNING FOER DIGITAL INTERPOLERING AV EN PIXEL
FI91472C (en) * 1991-10-10 1994-06-27 Salora Oy A method for processing an incoming HDTV signal in sample mode and the apparatus used in the method
US5644361A (en) * 1994-11-30 1997-07-01 National Semiconductor Corporation Subsampled frame storage technique for reduced memory size
JP3575508B2 (en) * 1996-03-04 2004-10-13 Kddi株式会社 Encoded video playback device
EP0817498B1 (en) * 1996-06-28 2001-08-22 STMicroelectronics S.r.l. MPEG-2 decoding with a reduced RAM requisite by ADPCM recompression before storing MPEG-2 decompressed data optionally after a subsampling algorithm
US6018753A (en) * 1997-07-29 2000-01-25 Lucent Technologies Inc. Interpolating filter banks in arbitrary dimensions
JP3384727B2 (en) * 1997-11-05 2003-03-10 三洋電機株式会社 Image decoding device
US6782132B1 (en) * 1998-08-12 2004-08-24 Pixonics, Inc. Video coding and reconstruction apparatus and methods
US6233277B1 (en) * 1999-04-02 2001-05-15 Sony Corporation Reduced-memory video decoder for compressed high-definition video data
JP2000333166A (en) * 1999-05-19 2000-11-30 Hitachi Ltd Means for decoding high efficiency coded image and image display device having the means
US6714258B2 (en) * 2000-03-15 2004-03-30 Koninklijke Philips Electronics N.V. Video-apparatus with noise reduction
US6983017B2 (en) * 2001-08-20 2006-01-03 Broadcom Corporation Method and apparatus for implementing reduced memory mode for high-definition television
JP2003289544A (en) * 2002-03-27 2003-10-10 Sony Corp Equipment and method for coding image information, equipment and method for decoding image information, and program
US7120305B2 (en) * 2002-04-16 2006-10-10 Ricoh, Co., Ltd. Adaptive nonlinear image enlargement using wavelet transform coefficients
EP1578134A1 (en) * 2004-03-18 2005-09-21 STMicroelectronics S.r.l. Methods and systems for encoding/decoding signals, and computer program product therefor
WO2005104560A1 (en) * 2004-04-27 2005-11-03 Koninklijke Philips Electronics N.V. Method of processing decoded pictures.
CN100340118C (en) * 2004-07-29 2007-09-26 Thomson宽带研发(北京)有限公司 Super large scale integrated circuit system structure of moving estimation and data buffer storage method
US8290057B2 (en) * 2005-02-18 2012-10-16 Mediatek Incorporation Method of decoding a digital video sequence and related apparatus
US8391368B2 (en) * 2005-04-08 2013-03-05 Sri International Macro-block based mixed resolution video compression system
US20060233258A1 (en) * 2005-04-15 2006-10-19 Microsoft Corporation Scalable motion estimation
CN100353758C (en) * 2005-05-16 2007-12-05 杭州海康威视数字技术有限公司 Coding method for monitoring image resolution and its apparatus

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6061402A (en) * 1994-10-11 2000-05-09 Hitachi America, Ltd. Methods and apparatus for efficiently decoding bi-directionally coded image data
US5923815A (en) * 1996-02-13 1999-07-13 Samsung Electronics Co., Ltd. Apparatus and method for decoding MPEG video data
US20070058722A1 (en) * 2001-10-17 2007-03-15 Shinya Kadono Moving picture coding method and moving picture decoding method
US7430238B2 (en) * 2004-12-10 2008-09-30 Micronas Usa, Inc. Shared pipeline architecture for motion vector prediction and residual decoding
US20070041448A1 (en) * 2005-08-17 2007-02-22 Miller Casey L Artifact and noise reduction in MPEG video
US8121195B2 (en) * 2006-11-30 2012-02-21 Lsi Corporation Memory reduced H264/MPEG-4 AVC codec

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10171811B2 (en) 2012-04-16 2019-01-01 Samsung Electronics Co., Ltd. Method and apparatus for determining reference picture set of image
US10609378B2 (en) 2012-04-16 2020-03-31 Samsung Electronics Co., Ltd. Method and apparatus for determining reference picture set of image
US11006120B2 (en) 2012-04-16 2021-05-11 Samsung Electronics Co., Ltd. Method and apparatus for determining reference picture set of image
US11490091B2 (en) 2012-04-16 2022-11-01 Samsung Electronics Co., Ltd. Method and apparatus for determining reference picture set of image
US11856201B2 (en) 2012-04-16 2023-12-26 Samsung Electronics Co., Ltd. Method and apparatus for determining reference picture set of image

Also Published As

Publication number Publication date
US8121195B2 (en) 2012-02-21
US20080130754A1 (en) 2008-06-05
TW200833124A (en) 2008-08-01
EP2100449A1 (en) 2009-09-16
CN101543069B (en) 2012-07-04
KR101182343B1 (en) 2012-09-20
CN101543069A (en) 2009-09-23
JP2010512046A (en) 2010-04-15
TWI382769B (en) 2013-01-11
WO2008066601A1 (en) 2008-06-05
EP2100449A4 (en) 2015-02-25
KR20090094345A (en) 2009-09-04
JP5022446B2 (en) 2012-09-12

Similar Documents

Publication Publication Date Title
US8121195B2 (en) Memory reduced H264/MPEG-4 AVC codec
US7324595B2 (en) Method and/or apparatus for reducing the complexity of non-reference frame encoding using selective reconstruction
JP4384130B2 (en) Video decoding method and apparatus
US7813429B2 (en) System and method for segmentation of macroblocks
RU2409005C2 (en) Method of scalable coding and decoding of video signal
US8170107B2 (en) Flexible reduced bandwidth compressed video decoder
US6301304B1 (en) Architecture and method for inverse quantization of discrete cosine transform coefficients in MPEG decoders
US7787541B2 (en) Dynamic pre-filter control with subjective noise detector for video compression
JP2003032682A (en) Method and apparatus for transcoding group of macroblock of partially coded input bitstream
US20100226437A1 (en) Reduced-resolution decoding of avc bit streams for transcoding or display at lower resolution
EP1461959A2 (en) Sharpness enhancement in post-processing of digital video signals using coding information and local spatial features
US9386310B2 (en) Image reproducing method, image reproducing device, image reproducing program, imaging system, and reproducing system
KR100364748B1 (en) Apparatus for transcoding video
US8204122B2 (en) Compressed non-reference picture reconstruction from post-processed reference pictures
JP4339784B2 (en) A method for decoding compressed bitstream encoded video encoded as a plurality of blocks.
WO2000040033A1 (en) Video memory management for mpeg video decode and display system
JP3265290B2 (en) Apparatus and method for block decoding of interlaced scanned digital video signal
US8233709B2 (en) Color effects for compressed digital video
KR20080048262A (en) Video decoder and decoding method
JP3809118B2 (en) Coding method for interlaced scanning digital video signal
JP3809117B2 (en) Decoding method for interlaced scanning digital video signal
JP3809116B2 (en) Coding method for interlaced scanning digital video signal
JP3809121B2 (en) Decoding method for interlaced scanning digital video signal
JP3809119B2 (en) Block decoding apparatus and method for interlaced scanning digital video signal
JP3809115B2 (en) Coding method for interlaced scanning digital video signal

Legal Events

Date Code Title Description
AS Assignment

Owner name: DEUTSCHE BANK AG NEW YORK BRANCH, AS COLLATERAL AG

Free format text: PATENT SECURITY AGREEMENT;ASSIGNORS:LSI CORPORATION;AGERE SYSTEMS LLC;REEL/FRAME:032856/0031

Effective date: 20140506

AS Assignment

Owner name: AVAGO TECHNOLOGIES GENERAL IP (SINGAPORE) PTE. LTD

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:LSI CORPORATION;REEL/FRAME:035390/0388

Effective date: 20140814

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO PAY ISSUE FEE

AS Assignment

Owner name: LSI CORPORATION, CALIFORNIA

Free format text: TERMINATION AND RELEASE OF SECURITY INTEREST IN PATENT RIGHTS (RELEASES RF 032856-0031);ASSIGNOR:DEUTSCHE BANK AG NEW YORK BRANCH, AS COLLATERAL AGENT;REEL/FRAME:037684/0039

Effective date: 20160201

Owner name: AGERE SYSTEMS LLC, PENNSYLVANIA

Free format text: TERMINATION AND RELEASE OF SECURITY INTEREST IN PATENT RIGHTS (RELEASES RF 032856-0031);ASSIGNOR:DEUTSCHE BANK AG NEW YORK BRANCH, AS COLLATERAL AGENT;REEL/FRAME:037684/0039

Effective date: 20160201