US20020172288A1 - Device and method for performing half-pixel accuracy fast search in video coding - Google Patents

Device and method for performing half-pixel accuracy fast search in video coding Download PDF

Info

Publication number
US20020172288A1
US20020172288A1 US09/801,584 US80158401A US2002172288A1 US 20020172288 A1 US20020172288 A1 US 20020172288A1 US 80158401 A US80158401 A US 80158401A US 2002172288 A1 US2002172288 A1 US 2002172288A1
Authority
US
United States
Prior art keywords
pixel
value
sub
mad
lowest
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US09/801,584
Inventor
Nyeongku Kwon
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Lambert Everest Ltd
Original Assignee
AVT AUDIO VISUAL TELECOMMUNICATIONS Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by AVT AUDIO VISUAL TELECOMMUNICATIONS Corp filed Critical AVT AUDIO VISUAL TELECOMMUNICATIONS Corp
Priority to US09/801,584 priority Critical patent/US20020172288A1/en
Assigned to AVT AUDIO VISUAL TELECOMMUNICATIONS CORPORATION reassignment AVT AUDIO VISUAL TELECOMMUNICATIONS CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: KWON, NYEONGKU
Priority to AU2002240754A priority patent/AU2002240754A1/en
Priority to JP2002570523A priority patent/JP4739651B2/en
Priority to PCT/CA2002/000324 priority patent/WO2002071741A2/en
Priority to US10/471,085 priority patent/US7792191B2/en
Publication of US20020172288A1 publication Critical patent/US20020172288A1/en
Assigned to LAMBERT EVEREST LTD. reassignment LAMBERT EVEREST LTD. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: AVT AUDIO VISUAL TELECOMMUNICATIONS CORPORATION
Priority to JP2008244850A priority patent/JP2009027744A/en
Priority to JP2009230638A priority patent/JP2010045816A/en
Priority to US12/877,109 priority patent/US20110064138A1/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51Motion estimation or motion compensation
    • H04N19/523Motion estimation or motion compensation with sub-pixel accuracy

Definitions

  • the present invention relates to motion estimation algorithms for video compression and, in particular, to a device and method for performing a half-pixel accuracy fast search algorithm in video coding.
  • HDTV digital and high definition television
  • video conferencing video conferencing
  • computer imaging computer imaging
  • high quality video tape recorders Uncompressed digital video signals constitute a huge amount of data and therefore require a large amount of bandwidth and memory to store and transmit.
  • one of the formats defined for HDTV broadcasting within the United States is 1920 pixels horizontally by 1080 lines vertically, at 30 frames per second. If these numbers are all multiplied together, along with eight bits for each of the three primary colors, the total data rate required will be approximately 1.5 Gb/sec.
  • each channel only supports a data rate of 19.2 Mb/sec, which is further reduced to 18 Mb/sec because the channel must also support audio, transport, and ancillary data information.
  • the original signal must be compressed by a factor of approximately 83:1.
  • Digital compression devices are commonly referred to “encoders”, while devices that perform decompression are referred to as “decoders”. Devices that perform both encoding and decoding are often referred to as “codecs”.
  • MPEG moving picture experts group
  • Motion picture video sequences consist of a series of still pictures or “frames” that are sequentially displayed to provide the illusion of continuous motion.
  • Each frame may be described as a two-dimensional array of picture elements or “pixels”.
  • Each pixel describes a particular point in the picture in terms of brightness and hue.
  • Methods have been devised to reduce the amount of transmission data required to represent each frame. The reduction of transmission data is referred to as data compression.
  • compression methods Rather than transmitting large amounts of information for each pixel location's color and brightness, compression methods divide each frame into a predetermined number of “macroblocks”.
  • the macroblocks are typically defined as a 16 ⁇ 16 array of pixels. Since there is some similarity within and between successive frames, it is more efficient to transmit only the differences between the frames.
  • the motion vectors are determined by comparing each pixel location in the current macroblock with each pixel location in a successive reference frame.
  • the integer location which differs the least between the two macroblocks is used to generate the motion vector.
  • the process of searching every pixel location is referred to as a full or exhaustive search and the process of searching less than every pixel location is referred to as a fast search.
  • a fast search algorithm for integer pixel locations is described in U. S. Pat. No. 6,128,047 entitled “Motion Estimation Process And System Using Sparse Search Block-Matching And Integral Projection”, to Chang et al.
  • the current macro block does not always shift an integer number of pixels in a given direction in real world video encoding.
  • the macro block may be displaced by a fractional portion of a pixel.
  • Several methods exist which obtain the integer pixel location previously determined by a Full or Fast search integer pixel search algorithm and perform a second search on all 8 surrounding half-pixel locations to more accurately define the motion vector. While this yields a more accurate motion vector, there is an increase in the number of computations necessary to locate the correct half-pixel location.
  • 1 ⁇ 2 pixel search algorithms that perform a fast search, i.e., not all 8 1 ⁇ 2 pixel locations are searched.
  • f(i, j) represents a block of 16 ⁇ 16 pixels (macroblock) from the current frame
  • g(i, j) represents the same macroblock but from a reference frame (either previous or future in time)
  • the reference macroblock is displaced by a vector (dx, dy), representing the search location.
  • the MAD function calculates which integer pixel location in a succeeding frame contains a minimum difference. This pixel value becomes the integer pixel displacement value for a motion vector.
  • the Lee article describes how the 1 ⁇ 2 pixel points surrounding the previously determined integer pixel location are tested.
  • the surrounding half-pixel integer locations are separated into a horizontal and vertical pair.
  • Horizontal half-pixel points 2 and 7 and vertical half-pixel points 4 and 5 are interpolated based upon the surrounding integer pixel values and the MADs of location 2 as compared with location 7 , and location 4 as compared with location 5 to determine a minimum MAD pair.
  • the interpolated pixels are determined using MPEG approved bi-linear interpolation techniques.
  • the block with the lowest MAD value is then compared with the blocks of the opposing pairs to determine a minimum pair.
  • the minimum MAD pair is to be 2 and 5 and point 4 is additionally considered for half-pixel motion prediction.
  • the half-pixel accuracy motion vector is determined by comparing the MADs of the centered integer pixel block, the minimum candidate block, and the most recently considered point.
  • a half-pixel accuracy fast search algorithm in video coding that performs a hierarchical search method for motion estimation which initially searches for an integer accuracy motion vector and then continues its search having a sub-pixel accuracy over surrounding, reconstructed conjugate sub-pixels of the integer motion vector.
  • FIG. 1 is a graphical depiction of integer pixel and half-pixel locations according to the prior art
  • FIG. 2 is a graph depicting integer and half integer locations for interpolation
  • FIG. 3 is a graphical depiction of the surrounding half-pixel locations around an integer pixel location
  • FIG. 4 is a graphical representation of the half-pixel accuracy fast search algorithm according to the present invention.
  • FIG. 5 is a flow chart illustrating the half-pixel accuracy fast search algorithm in video coding according to the present invention
  • FIG. 6 is a diagram of a system consistent with the present invention.
  • FIGS. 7 - 10 are graphical representations of the peak signal to noise ratio (PSNR) performance of the present invention with test video stream inputs.
  • PSNR peak signal to noise ratio
  • Motion picture video sequences consist of a series of still pictures or “frames” that are sequentially displayed to provide the illusion of continuous motion.
  • Each frame may be described as a two-dimensional array of picture elements, or “pixels”.
  • Each pixel describes a particular point in the picture in terms of brightness and hue.
  • Pixel information can be represented in digital form and displayed on a monitor such as a High Definition television, or encoded, and broadcast.
  • the novel component is called entropy and it is the true information in the signal.
  • the remainder is called redundancy because it is not essential and may be recreated from other frames. Redundancy may be spatial, as it is in large plain areas of the picture where adjacent pixels have almost the same value. Redundancy can also be temporal, as it is where similarities between successive pictures are used. Compression systems work by separating the entropy from the redundancy in the encoder. The entropy is recorded or transmitted along with motion vectors which describe redundant data. The decoder uses the entropy data and redundancy data to generate complete frames to display.
  • An ideal encoder transmits the entropy of a sequence of frames and recreates the additional redundant information using previous or subsequent frames of redundant pixel information without any loss of image quality.
  • the higher the compression ratio the more prone the signal transmission is to digital artifacts or errors, thus reducing signal quality. Since ideal encoders do not presently exist, there is a balancing between compression ratio and signal quality.
  • Intracoding is a technique that exploits spatial redundancy, or redundancy within the picture while interceding is a technique that exploits temporal redundancy, or redundancy between successive pictures.
  • Intracoding is a technique that exploits spatial redundancy, or redundancy within the picture
  • interceding is a technique that exploits temporal redundancy, or redundancy between successive pictures.
  • Two types of motion estimation methods are typically used to estimate the motion vectors; pixel-recursive algorithms and block-matching algorithms.
  • Pixel-recursive techniques predict the displacement of each pixel from corresponding pixels in neighboring frames.
  • Block matching algorithms estimate the displacement between frames on a block-by-block basis and choose displacement vectors that minimize the difference. The entire macroblock is then displaced by the motion vector.
  • the current image to be encoded is divided into equally sized blocks of pixel information. These “macro blocks” typically consist of a 16 ⁇ 16 sample array of luminance samples together with 18 ⁇ 8 block of samples for each of the two chrominance components.
  • Block matching motion estimation algorithms are generally categorized as either a full search or a fast search algorithm based on its search strategy.
  • Full search also known as the exhaustive search, computes the error between successive macro blocks at all possible candidate integer pixel locations in order to find the motion vector of the macro block. While it is simple in complexity to implement, it carries an extensive computational burden to search the entire area of each macro block and successive macro block locations. Hence, many fast search algorithms have been proposed that reduce the amount of integer pixel locations to search for a minimum error between successive macro blocks.
  • the present invention proposes a hierarchical approach having sub-pixel accuracy.
  • the term hierarchical connotes that the algorithm first calculates a minimum integer pixel location and then performs a sub-integer pixel search.
  • the present invention obtains the integer pixel value with the lowest MAD value and then interpolates conjugate half-pixel values in a first direction.
  • the algorithm determines which half-pixel value yields the lowest MAD value and then interpolates conjugate half-pixel values in a second direction from the previously calculated minimum MAD half-pixel location from the first direction.
  • This conjugate search method described reduces the calculations performed by the conventional half-pixel search methods from eight to four half-pixel locations.
  • a two-dimensional reconstructed image using interpolation is described in a mathematical expression as follows, where two-dimensional convolution is conducted between sampled pixel image and interpolation function.
  • B(.) is the reconstructed interpolation image and ⁇ overscore (B) ⁇ (.) and h(.) are sampled pixel image and interpolation filter function respectively.
  • FIG. 2 is a graph depicting integer and half integer locations for use with bilinear interpolation.
  • Bilinear interpolation is a process by which half-pixel values are determined and has been adopted by MPEG as the standard for calculating half-pixel values.
  • half-pixel values a, b, c, d and e are calculated as follows:
  • A, B, C, and D are integer-pixels and a, b, c, d, and e are surrounding half-pixels.
  • the half pixel values are used by the present invention in determining a half pixel accuracy motion vector.
  • FIG. 3 is a graphical representation of the surrounding half-pixel locations around an integer pixel location.
  • half-pixel locations 1 through 8 which surround an integer pixel location zero are calculated using, for example, the formulas previously described.
  • FIG. 4 shows a graphical representation of an integer location zero surrounded by multiple half-pixel locations 1 through 8 respectively.
  • an example of possible search locations for the half-pixel accuracy fast search algorithm in video coding according to the present invention are described with reference to I Left , I Center , I Right , and II Center , II Top , and II Bottom .
  • the fast half-pixel search first calculates an integer pixel location in a macroblock which has a minimum MAD value using a known integer pixel full or fast search algorithm. Once that integer pixel location is calculated, the conjugate half-pixel values in a first direction I Left and I Right are determined. For example, assume the first direction is horizontal and parallel to an X-axis of a 2 dimensional grid. The MADs of I Left and I Right are calculated and the half-pixel location with the lowest MAD value is stored. Conjugate half-pixel values in a second direction are also calculated starting from the previously stored pixel location having the lowest MAD value. Based upon the above, presume the second direction is in a vertical direction and parallel to a y-axis of a grid.
  • the first and second directions are perpendicular to each other.
  • II Top and II Bottom are determined. Note that I Right I Center or I Left will be equal in value to II Center when the first and second directions are perpendicular to each other.
  • MAD values for those two locations are determined.
  • the half-pixel location having the lowest MAD value is also stored and used as an offset for the integer pixel value previously determined. Accordingly, a new motion vector combining integer pixel location l center plus the half-pixel location having the minimum MAD value are used to generate a final motion vector for displacing the macroblock.
  • the motion vector MV(x, y) is defined as:
  • MV ( x,y ) MV integer ( x,y )+ d h ( x,y ) (2)
  • MV integer (x, y) is the integer accuracy motion vector
  • d h (x, y) is the half-pixel displacement
  • the method may also compare the integer location with the conjugate sub-pixel locations to determine the location having the lowest MAD value.
  • a half-pixel motion estimation algorithm operating according to the present invention yields more accurate motion displacement information than an integer pixel motion vector and reduces the computations required in a full search from 8 to 4 , thereby reducing the computations by 50%.
  • search order could be performed in a vertical direction followed by a horizontal direction, the particular order being a matter of design choice.
  • first direction could be diagonally from top right to bottom left and the second direction could be from bottom right to top left. The conjugate values being determined with respect to the diagonals.
  • FIG. 5 is a flow chart illustrating a half-pixel accuracy fast search in a video coding example according to the present invention.
  • First perform an initial integer search using a previously known integer search algorithm and determine the motion vector MV integer (x, y).
  • Step 500 performs a sub-pixel conjugate interpolation in a first direction.
  • Step 505 calculate MAD values for each of the conjugate sub-pixel locations in the first direction D(x+1, y), and D(x ⁇ 1, y), where the second direction variable y is fixed.
  • Step 510 Next determine a minimum MAD value in the first direction.
  • Step 515 determines a minimum MAD value in the first direction.
  • Step 520 perform sub-pixel conjugate interpolation in a second direction centered on the sub-pixel location having the minimum MAD value in the first direction.
  • Step 520 calculate MAD values for the conjugate sub-pixel locations in the second direction D(x, y+1) and D(x, y ⁇ 1) where the x variable is fixed.
  • Step 525 determine the minimum MAD value in the second direction.
  • Step 530 Calculate a new motion vector based upon the sub-pixel location having the minimum MAD value.
  • Step 535 set the motion vector for the macroblock equal to the sum of the motion vector for the integer pixel plus the motion vector for the sub-pixel location.
  • Step 540 set the motion vector for the macroblock equal to the sum of the motion vector for the integer pixel plus the motion vector for the sub-pixel location.
  • FIG. 6 is a diagram of a system 600 consistent with the present invention.
  • System 600 is comprised of a central processing unit (“CPU”) 605 , a random access memory (“RAM”) 610 , a read only memory (“ROM”) 620 and a computer readable medium 625 interconnected via a system bus 670 .
  • Computer readable medium 625 contains computer readable instructions necessary to perform the half-pixel accuracy fast search according to the present invention and may be embodied as, but not limited to, any media that provides instructions according to the present invention to CPU 605 for execution. Examples of computer readable medium may be RAM, ROM, Electrically Erasable Programmable ROM (“EEPROM”), CD-ROM, or transmission medium such as Network connections, Radio Waves, or other wireless information transmission means.
  • EEPROM Electrically Erasable Programmable ROM
  • An input/output (I/O) signal 660 is connected to system 600 and may provide an input stream of Audio/Video data to compress or decompress.
  • CPU 605 accesses I/O signal 660 along with instructions contained in medium 625 to determine the correct half-pixel accuracy motion vectors as described above.
  • CPU 605 may receive image data from an input device 650 or display I/O signal 660 on a display 630 . Once the image data is received via input device 650 or I/O signal 660 and processed via CPU 605 , the image data can be either stored on a storage device 675 or transmitted via a transmitter and antenna 680 .
  • System 600 as well as medium 625 may be incorporated into many devices including, but not limited to, Motion video recorders, Digital Video Disk recorders (“DVDs”), Satellite and television transmitters and receivers, Video Display cards for personal computers and other MPEG compliant devices, to reduce the amount of information required to transmit digital video information and reduce the amount of calculations required to obtain accurate motion vectors.
  • DVDs Digital Video Disk recorders
  • Satellite and television transmitters and receivers Video Display cards for personal computers and other MPEG compliant devices
  • PSNR peak signal to noise ratio
  • N is the number of pixels
  • O 1 and R 1 are the amplitudes of the original and the reconstructed pixels.
  • Both half-pixel search methods show better performance with the difference of up to 2dB in average PSNR compared to the integer-pixel search method. In comparison of both half-pixel search methods, those methods, fast and full search, do not show much difference in terms of PSNR with all of four video sequences, while the required number of computation as described in the present invention was reduced to half of the full search method.
  • Table 2 shows the computer simulation data of the present invention with comparison to integer-pixel full search, fast search, and half-pixel full search in terms of the number of computations.
  • the processing speed improvement of the present invention is clearly shown in the simulation data, especially when it is involved in integer-pixel fast search algorithm.
  • the portion of total half-pixel search points is approximately 1% of the total number of points searched during an integer-pixel full search.
  • the number of computations performed during the conventional half-pixel full search becomes even larger than the one of integer-pixel fast search.
  • Table 1 indicates that the present invention improved the processing speed by reducing the number of required.
  • the simulation data shows about 30% reduction in the overall number of computations required and a 50% reduction compared to the conventional half-pixel full search.
  • FIGS. 7 - 10 present the PSNR data of each picture frame from 100 frames with four test video sequences.
  • the full half pixel search, the fast half pixel search according to the present invention and the integer pixel search are compared in each of the graphs.
  • the full half pixel search and the fast half pixel search according to the present invention are almost overlapping each other showing similar performance with respect to the PSNR, however, the integer pixel search, remains below the other two fast search data with up to 2dB PSNR difference. Therefore, the present invention reduces the number of computations required to generate a half-pixel accurate motion vector by 50% while experiencing only a negligible amount of noise as compared to the full half pixel search.

Abstract

A half-pixel accuracy fast search algorithm in video coding performs a hierarchical search method for motion, which initially searches integer accuracy motion vector and then continues its search sub-pixel accuracy over surrounding reconstructed sub-pixels of the integer motion vector. The term hierarchical connotes that the algorithm first calculates a minimum integer pixel location and then performs a sub-integer pixel search. The present invention obtains the integer pixel value with the lowest MAD value and then interpolates conjugate half-pixel values in a first direction. The algorithm determines which half-pixel value yields the lowest MAD value and then interpolates conjugate half-pixel values in a second direction from the previously calculated minimum MAD half-pixel location from the first direction.

Description

    FIELD OF THE INVENTION
  • The present invention relates to motion estimation algorithms for video compression and, in particular, to a device and method for performing a half-pixel accuracy fast search algorithm in video coding. [0001]
  • BACKGROUND OF THE INVENTION
  • Advancements in digital technology have produced a number of digital video applications. Digital video is currently used in digital and high definition television (HDTV), video conferencing, computer imaging, and high quality video tape recorders. Uncompressed digital video signals constitute a huge amount of data and therefore require a large amount of bandwidth and memory to store and transmit. For example, one of the formats defined for HDTV broadcasting within the United States is 1920 pixels horizontally by 1080 lines vertically, at 30 frames per second. If these numbers are all multiplied together, along with eight bits for each of the three primary colors, the total data rate required will be approximately 1.5 Gb/sec. Because of the 6 MHz channel bandwidth limitation imposed by the FCC, each channel only supports a data rate of 19.2 Mb/sec, which is further reduced to 18 Mb/sec because the channel must also support audio, transport, and ancillary data information. Given the transmission data rate restriction, the original signal must be compressed by a factor of approximately 83:1. Many digital video systems, therefore, reduce the amount of digital video data by employing data compression techniques that are optimized for particular applications. Digital compression devices are commonly referred to “encoders”, while devices that perform decompression are referred to as “decoders”. Devices that perform both encoding and decoding are often referred to as “codecs”. [0002]
  • In the interests of standardizing methods for motion picture video compression, the moving picture experts group (MPEG) issued a number of standards for digital video processing. [0003]
  • Motion picture video sequences consist of a series of still pictures or “frames” that are sequentially displayed to provide the illusion of continuous motion. Each frame may be described as a two-dimensional array of picture elements or “pixels”. Each pixel describes a particular point in the picture in terms of brightness and hue. Methods have been devised to reduce the amount of transmission data required to represent each frame. The reduction of transmission data is referred to as data compression. Rather than transmitting large amounts of information for each pixel location's color and brightness, compression methods divide each frame into a predetermined number of “macroblocks”. The macroblocks are typically defined as a 16×16 array of pixels. Since there is some similarity within and between successive frames, it is more efficient to transmit only the differences between the frames. If the current macroblock is found in a successive or preceding frame, it is more efficient to transmit a motion vector that details where to move the current macroblock. The motion vectors are determined by comparing each pixel location in the current macroblock with each pixel location in a successive reference frame. The integer location which differs the least between the two macroblocks is used to generate the motion vector. The process of searching every pixel location is referred to as a full or exhaustive search and the process of searching less than every pixel location is referred to as a fast search. A fast search algorithm for integer pixel locations is described in U. S. Pat. No. 6,128,047 entitled “Motion Estimation Process And System Using Sparse Search Block-Matching And Integral Projection”, to Chang et al. [0004]
  • The current macro block does not always shift an integer number of pixels in a given direction in real world video encoding. In reality, the macro block may be displaced by a fractional portion of a pixel. Several methods exist which obtain the integer pixel location previously determined by a Full or Fast search integer pixel search algorithm and perform a second search on all 8 surrounding half-pixel locations to more accurately define the motion vector. While this yields a more accurate motion vector, there is an increase in the number of computations necessary to locate the correct half-pixel location. There exist ½ pixel search algorithms that perform a fast search, i.e., not all 8 ½ pixel locations are searched. One such fast search algorithm is described in “Fast Two-Step Half-Pixel Accuracy Motion Vector Prediction”, [0005] K. H. Lee, J. H. Choi, B. K. Lee and D. G. Kim, Electronics Letters, 30th Mar., 2000 Vol. 36, no. 7. The Lee article performs a fast two-step half-pixel accuracy motion vector prediction algorithm by first determining which integer location of a macro block contains the lowest Mean of Absolute Difference (MAD). The MAD function is shown in equation (1): MAD ( dx , dy ) = 1 256 i = 0 15 j = 0 15 f ( i , j ) - g ( i - dx , j - dy ) ( 1 )
    Figure US20020172288A1-20021121-M00001
  • where, f(i, j) represents a block of 16×16 pixels (macroblock) from the current frame, g(i, j) represents the same macroblock but from a reference frame (either previous or future in time), and the reference macroblock is displaced by a vector (dx, dy), representing the search location. The MAD function calculates which integer pixel location in a succeeding frame contains a minimum difference. This pixel value becomes the integer pixel displacement value for a motion vector. [0006]
  • To find the best matching block producing the minimum MAD value, we need to calculate the MAD at several locations in the search range. As mentioned previously, a full or exhaustive search calculates the MAD at all of the locations, while a fast or partial search selects predefined values. [0007]
  • Referring to FIG. 1, the Lee article describes how the ½ pixel points surrounding the previously determined integer pixel location are tested. First, the surrounding half-pixel integer locations are separated into a horizontal and vertical pair. Horizontal half-[0008] pixel points 2 and 7 and vertical half- pixel points 4 and 5 are interpolated based upon the surrounding integer pixel values and the MADs of location 2 as compared with location 7, and location 4 as compared with location 5 to determine a minimum MAD pair. The interpolated pixels are determined using MPEG approved bi-linear interpolation techniques. The block with the lowest MAD value is then compared with the blocks of the opposing pairs to determine a minimum pair. For example, assume that point 2 has the minimum MAD and 5 has a smaller MAD than 4, the minimum MAD pair is to be 2 and 5 and point 4 is additionally considered for half-pixel motion prediction. Finally, the half-pixel accuracy motion vector is determined by comparing the MADs of the centered integer pixel block, the minimum candidate block, and the most recently considered point.
  • While the Lee article describes a fast two step half-pixel motion vector prediction algorithm, the algorithm performs [0009] 5 calculations and only reduces the computational burden by 37.5% compared with that of the conventional full half-pixel search method. Accordingly, there remains a need for a fast half-pixel motion estimation algorithm which further reduces the computational burden of determining a half-pixel accuracy motion vector.
  • SUMMARY OF THE INVENTION
  • It is therefore an aspect of the present invention to provide a half-pixel accuracy fast search algorithm in video coding that reduces the computational burden of determining a motion vector having a half-pixel accuracy as compared with conventional methods. [0010]
  • It is an additional aspect of the present invention to provide the half-pixel accuracy fast search algorithm in video coding that is simple to implement and execute. [0011]
  • To achieve the above and other aspects of the present invention, there is provided a half-pixel accuracy fast search algorithm in video coding that performs a hierarchical search method for motion estimation which initially searches for an integer accuracy motion vector and then continues its search having a sub-pixel accuracy over surrounding, reconstructed conjugate sub-pixels of the integer motion vector. [0012]
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • Other aspects and features of the present invention will become more apparent from the following detailed description considered in connection with the accompanying drawings which disclose an embodiment of the present invention. It should be understood, however, that the drawings are designed for the purpose of illustration only and not as a definition of the limits of the invention. In the drawings, wherein similar reference characters denote similar elements throughout the several views: [0013]
  • FIG. 1 is a graphical depiction of integer pixel and half-pixel locations according to the prior art; [0014]
  • FIG. 2 is a graph depicting integer and half integer locations for interpolation; [0015]
  • FIG. 3 is a graphical depiction of the surrounding half-pixel locations around an integer pixel location; [0016]
  • FIG. 4 is a graphical representation of the half-pixel accuracy fast search algorithm according to the present invention; [0017]
  • FIG. 5 is a flow chart illustrating the half-pixel accuracy fast search algorithm in video coding according to the present invention; [0018]
  • FIG. 6 is a diagram of a system consistent with the present invention; and [0019]
  • FIGS. [0020] 7-10 are graphical representations of the peak signal to noise ratio (PSNR) performance of the present invention with test video stream inputs.
  • DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
  • Motion picture video sequences consist of a series of still pictures or “frames” that are sequentially displayed to provide the illusion of continuous motion. Each frame may be described as a two-dimensional array of picture elements, or “pixels”. Each pixel describes a particular point in the picture in terms of brightness and hue. Pixel information can be represented in digital form and displayed on a monitor such as a High Definition television, or encoded, and broadcast. [0021]
  • There are two components of a broadcast signal: 1) those which can not be anticipated (i.e. they are novel and unpredictable), and 2) those which can be anticipated. The novel component is called entropy and it is the true information in the signal. The remainder is called redundancy because it is not essential and may be recreated from other frames. Redundancy may be spatial, as it is in large plain areas of the picture where adjacent pixels have almost the same value. Redundancy can also be temporal, as it is where similarities between successive pictures are used. Compression systems work by separating the entropy from the redundancy in the encoder. The entropy is recorded or transmitted along with motion vectors which describe redundant data. The decoder uses the entropy data and redundancy data to generate complete frames to display. [0022]
  • An ideal encoder transmits the entropy of a sequence of frames and recreates the additional redundant information using previous or subsequent frames of redundant pixel information without any loss of image quality. Typically, the higher the compression ratio, the more prone the signal transmission is to digital artifacts or errors, thus reducing signal quality. Since ideal encoders do not presently exist, there is a balancing between compression ratio and signal quality. [0023]
  • For video compression, the encoder makes use of both “intracoding” and “interceding” techniques. Intracoding is a technique that exploits spatial redundancy, or redundancy within the picture while interceding is a technique that exploits temporal redundancy, or redundancy between successive pictures. If a given picture is available at the decoder, the next picture may be created by sending only the picture differences. The picture differences will be increased when objects are moved, but the differences can be offset by using motion compensation, since a moving object does not generally change its appearance very much from one picture to the next. If the motion can be measured, a closer approximation to the current picture can be created by shifting part of the previous picture to a new location. The shifting processes are controlled by a motion vector that is transmitted to the decoder. The motion vector transmission requires less data than sending the picture difference data or the complete frame. [0024]
  • Two types of motion estimation methods are typically used to estimate the motion vectors; pixel-recursive algorithms and block-matching algorithms. Pixel-recursive techniques predict the displacement of each pixel from corresponding pixels in neighboring frames. Block matching algorithms, on the other hand, estimate the displacement between frames on a block-by-block basis and choose displacement vectors that minimize the difference. The entire macroblock is then displaced by the motion vector. Typically, the current image to be encoded is divided into equally sized blocks of pixel information. These “macro blocks” typically consist of a 16×16 sample array of luminance samples together with 18×8 block of samples for each of the two chrominance components. [0025]
  • Block matching motion estimation algorithms are generally categorized as either a full search or a fast search algorithm based on its search strategy. Full search, also known as the exhaustive search, computes the error between successive macro blocks at all possible candidate integer pixel locations in order to find the motion vector of the macro block. While it is simple in complexity to implement, it carries an extensive computational burden to search the entire area of each macro block and successive macro block locations. Hence, many fast search algorithms have been proposed that reduce the amount of integer pixel locations to search for a minimum error between successive macro blocks. In order to improve the accuracy of integer pixel motion vectors, the present invention proposes a hierarchical approach having sub-pixel accuracy. [0026]
  • The term hierarchical connotes that the algorithm first calculates a minimum integer pixel location and then performs a sub-integer pixel search. The present invention obtains the integer pixel value with the lowest MAD value and then interpolates conjugate half-pixel values in a first direction. The algorithm determines which half-pixel value yields the lowest MAD value and then interpolates conjugate half-pixel values in a second direction from the previously calculated minimum MAD half-pixel location from the first direction. This conjugate search method described reduces the calculations performed by the conventional half-pixel search methods from eight to four half-pixel locations. [0027]
  • A two-dimensional reconstructed image using interpolation is described in a mathematical expression as follows, where two-dimensional convolution is conducted between sampled pixel image and interpolation function.[0028]
  • B(x,y)=Σ{overscore (B)}(x k ,y k)h(x−x k ,y−y k)
  • where B(.) is the reconstructed interpolation image and {overscore (B)}(.) and h(.) are sampled pixel image and interpolation filter function respectively. Two-dimensional liner interpolation function is given by [0029] h ( x , y ) = ( 1 - x X ) · ( 1 - y Y ) , for both x X and y Y 1
    Figure US20020172288A1-20021121-M00002
  • where X and Y are the horizontal and vertical spacing in the image frame. [0030]
  • Turning now in detail to the drawings, FIG. 2 is a graph depicting integer and half integer locations for use with bilinear interpolation. Bilinear interpolation is a process by which half-pixel values are determined and has been adopted by MPEG as the standard for calculating half-pixel values. Using the above bilinear interpolation equation on integer-pixel locations of an image, half-pixel values a, b, c, d and e are calculated as follows: [0031]
  • a=(A+B)/2 [0032]
  • b=(A+C)/2 [0033]
  • c=(A+B+C+D)/4 [0034]
  • d=(B+D)/2 [0035]
  • e=(C+D)/2 [0036]
  • where A, B, C, and D are integer-pixels and a, b, c, d, and e are surrounding half-pixels. The half pixel values are used by the present invention in determining a half pixel accuracy motion vector. [0037]
  • FIG. 3 is a graphical representation of the surrounding half-pixel locations around an integer pixel location. In accordance with the present invention, half-[0038] pixel locations 1 through 8 which surround an integer pixel location zero are calculated using, for example, the formulas previously described.
  • FIG. 4 shows a graphical representation of an integer location zero surrounded by multiple half-[0039] pixel locations 1 through 8 respectively. In addition, an example of possible search locations for the half-pixel accuracy fast search algorithm in video coding according to the present invention are described with reference to ILeft, ICenter, IRight, and IICenter, IITop, and IIBottom.
  • The fast half-pixel search according to the present invention first calculates an integer pixel location in a macroblock which has a minimum MAD value using a known integer pixel full or fast search algorithm. Once that integer pixel location is calculated, the conjugate half-pixel values in a first direction I[0040] Left and IRight are determined. For example, assume the first direction is horizontal and parallel to an X-axis of a 2 dimensional grid. The MADs of ILeft and IRight are calculated and the half-pixel location with the lowest MAD value is stored. Conjugate half-pixel values in a second direction are also calculated starting from the previously stored pixel location having the lowest MAD value. Based upon the above, presume the second direction is in a vertical direction and parallel to a y-axis of a grid. Thus, although not required, in this case, the first and second directions are perpendicular to each other. Using bilinear interpolation, IITop and IIBottom are determined. Note that IRight ICenter or ILeft will be equal in value to IICenter when the first and second directions are perpendicular to each other. Once IITop and IIBottom have been calculated, MAD values for those two locations are determined. The half-pixel location having the lowest MAD value is also stored and used as an offset for the integer pixel value previously determined. Accordingly, a new motion vector combining integer pixel location lcenter plus the half-pixel location having the minimum MAD value are used to generate a final motion vector for displacing the macroblock. Using equation (2) the motion vector MV(x, y) is defined as:
  • MV(x,y)=MV integer(x,y)+d h(x,y)  (2)
  • where MV[0041] integer (x, y) is the integer accuracy motion vector and dh (x, y) is the half-pixel displacement.
  • In an alternative approach, the method may also compare the integer location with the conjugate sub-pixel locations to determine the location having the lowest MAD value. [0042]
  • Advantageously, a half-pixel motion estimation algorithm operating according to the present invention yields more accurate motion displacement information than an integer pixel motion vector and reduces the computations required in a full search from [0043] 8 to 4, thereby reducing the computations by 50%.
  • By way of example, referring to FIG. 4, suppose the integer pixel location having the minimum MAD value is located at I[0044] Center. The present invention interpolates half-pixel locations ILeft and IRight, and determines which of those two points yields a minimum MAD value. Assuming IRight yields the minimum MAD value, half-pixel values IITop and IIBottom, are calculated in a second direction. Assuming IITop contains a minimum MAD value, the resulting motion vector for this macroblock would be equal to the motion vector of the integer plus the half-pixel IITop motion vector. It should be understood, that the search order employed in the example above is for illustration purposes only. Any other suitable search order could be used as an alternative, for example, the search could be performed in a vertical direction followed by a horizontal direction, the particular order being a matter of design choice. In another example, the first direction could be diagonally from top right to bottom left and the second direction could be from bottom right to top left. The conjugate values being determined with respect to the diagonals.
  • FIG. 5 is a flow chart illustrating a half-pixel accuracy fast search in a video coding example according to the present invention. First, perform an initial integer search using a previously known integer search algorithm and determine the motion vector MV integer (x, y). (Step [0045] 500). Following, perform a sub-pixel conjugate interpolation in a first direction. (Step 505). Then, calculate MAD values for each of the conjugate sub-pixel locations in the first direction D(x+1, y), and D(x−1, y), where the second direction variable y is fixed. (Step 510). Next determine a minimum MAD value in the first direction. (Step 515). Following, perform sub-pixel conjugate interpolation in a second direction centered on the sub-pixel location having the minimum MAD value in the first direction. (Step 520). Next, calculate MAD values for the conjugate sub-pixel locations in the second direction D(x, y+1) and D(x, y−1) where the x variable is fixed. (Step 525) Next, determine the minimum MAD value in the second direction. (Step 530) Calculate a new motion vector based upon the sub-pixel location having the minimum MAD value. (Step 535) Then, set the motion vector for the macroblock equal to the sum of the motion vector for the integer pixel plus the motion vector for the sub-pixel location. (Step 540).
  • FIG. 6 is a diagram of a [0046] system 600 consistent with the present invention. System 600 is comprised of a central processing unit (“CPU”) 605, a random access memory (“RAM”) 610, a read only memory (“ROM”) 620 and a computer readable medium 625 interconnected via a system bus 670. Computer readable medium 625 contains computer readable instructions necessary to perform the half-pixel accuracy fast search according to the present invention and may be embodied as, but not limited to, any media that provides instructions according to the present invention to CPU 605 for execution. Examples of computer readable medium may be RAM, ROM, Electrically Erasable Programmable ROM (“EEPROM”), CD-ROM, or transmission medium such as Network connections, Radio Waves, or other wireless information transmission means.
  • An input/output (I/O) signal [0047] 660 is connected to system 600 and may provide an input stream of Audio/Video data to compress or decompress. CPU 605 accesses I/O signal 660 along with instructions contained in medium 625 to determine the correct half-pixel accuracy motion vectors as described above. In addition, CPU 605 may receive image data from an input device 650 or display I/O signal 660 on a display 630. Once the image data is received via input device 650 or I/O signal 660 and processed via CPU 605, the image data can be either stored on a storage device 675 or transmitted via a transmitter and antenna 680.
  • [0048] System 600 as well as medium 625 may be incorporated into many devices including, but not limited to, Motion video recorders, Digital Video Disk recorders (“DVDs”), Satellite and television transmitters and receivers, Video Display cards for personal computers and other MPEG compliant devices, to reduce the amount of information required to transmit digital video information and reduce the amount of calculations required to obtain accurate motion vectors.
  • The performance of the present invention is described in terms of peak signal to noise ratio (PSNR) using four test video sequences such as “Claire”, “Carphone”, “Miss-America”, and “Salesman”, and compared to both the conventional half-pixel search and integer-pixel search method. Equation (3) details the calculation of the PSNR while Table 1 shows the computer simulation data and its comparison to other search algorithm in terms of PSNR. The simulation was performed in a H.263 framework and PSNR data was obtained by averaging over 100 picture frame data, PSNR was calculated as follows. [0049] PSNR = 10 log 255 2 1 N i = 1 N ( O i - R i ) 2 ( 3 )
    Figure US20020172288A1-20021121-M00003
  • where N is the number of pixels, and O[0050] 1 and R1 are the amplitudes of the original and the reconstructed pixels. Both half-pixel search methods show better performance with the difference of up to 2dB in average PSNR compared to the integer-pixel search method. In comparison of both half-pixel search methods, those methods, fast and full search, do not show much difference in terms of PSNR with all of four video sequences, while the required number of computation as described in the present invention was reduced to half of the full search method.
  • Table 2 shows the computer simulation data of the present invention with comparison to integer-pixel full search, fast search, and half-pixel full search in terms of the number of computations. The processing speed improvement of the present invention is clearly shown in the simulation data, especially when it is involved in integer-pixel fast search algorithm. The portion of total half-pixel search points is approximately 1% of the total number of points searched during an integer-pixel full search. The number of computations performed during the conventional half-pixel full search becomes even larger than the one of integer-pixel fast search. Hence, from the point of processing speed improvement, it is critical to reduce the number of computations in the half-pixel search. Table 1 indicates that the present invention improved the processing speed by reducing the number of required. The simulation data shows about 30% reduction in the overall number of computations required and a 50% reduction compared to the conventional half-pixel full search. [0051]
    TABLE 1
    Performance comparison in PSNR
    Integer Pixel Half-pixel Full Half-pixel Fast
    Search (dB) Search (dB) Search (dB)
    Claire 35.486 36.201 36.199
    Carphone 31.933 33.255 33.245
    Miss-America 36.012 37.159 37.171
    Sales-man 31.289 31.708 31.716
  • [0052]
    TABLE 2
    Performance comparison in number of computations
    Integer- Integer-
    pixel Full pixel Fast Half-pixel Half-pixel
    Search Search Full Search Fast Search
    Claire 77439 461 792 396
    Carphone 77439 534 791 396
    Miss-America 77439 472 792 396
    Sales-man 77439 464 792 396
  • FIGS. [0053] 7-10 present the PSNR data of each picture frame from 100 frames with four test video sequences. The full half pixel search, the fast half pixel search according to the present invention and the integer pixel search are compared in each of the graphs. The full half pixel search and the fast half pixel search according to the present invention are almost overlapping each other showing similar performance with respect to the PSNR, however, the integer pixel search, remains below the other two fast search data with up to 2dB PSNR difference. Therefore, the present invention reduces the number of computations required to generate a half-pixel accurate motion vector by 50% while experiencing only a negligible amount of noise as compared to the full half pixel search.
  • While an embodiment of the present invention has been shown and described, it is to be understood that many changes and modifications may be made thereunto without departing from the spirit and scope of the present invention as defined in the appended claims. [0054]

Claims (26)

What is claimed is:
1. A method of obtaining a motion vector between a first macroblock contained in a first frame and a second macroblock contained in a second frame of video image data comprising:
at an integer pixel location having a lowest mean of absolute difference (“MAD”) value in the second frame relative to the first frame,
identifying a first conjugate sub-pixel value in a first direction having a lowest first direction MAD value;
identifying a second conjugate sub-pixel value in a second direction having a lowest second direction MAD value; and
determining a motion vector representing the distance between the first macroblock and the second macroblock using the integer pixel location and a sub-integer pixel location, wherein the sub-pixel location is obtained from the first conjugate sub-pixel value and the second conjugate sub-pixel value having the lowest MAD value.
2. The method according to claim 1, wherein identifying a first conjugate sub-pixel value further comprises:
performing bi-linear interpolation to determine a first and a second half-pixel value in the first direction centered on the integer pixel location, the first and the second half-pixel values being conjugally related;
determining a MAD value at each of the first and second half-pixel locations; and
comparing the MAD values to determine the half-pixel location having the lowest MAD value in the first direction.
3. The method according to claim 1 wherein identifying a second conjugate sub-pixel value further comprises:
performing bi-linear interpolation to determine a first and a second half-pixel value in the second direction centered on the integer pixel location, the first and the second half-pixel values being conjugally related;
determining a MAD value at each of the first and the second half-pixel locations; and
comparing the MAD values to determine the half-pixel location having the lowest MAD value in the second direction.
4. The method according to claim 1 further comprising:
performing bi-linear interpolation to determine a first and a second half-pixel value in the second direction centered on the first conjugate sub-pixel value in the first direction having the lowest first direction MAD value, the first and the second half-pixel values in the second direction being conjugally related;
determining a MAD value at each of the first and the second half-pixel locations in the second direction; and
comparing the MAD values to determine the half-pixel location having the lowest MAD value in the second direction.
5. The method according to claim 1 wherein determining the motion vector further comprises:
generating an integer motion vector using the integer pixel location;
generating a sub-pixel motion vector using the sub-pixel location; and
generating a final motion vector by adding the integer motion vector to the sub-pixel motion vector.
6. The method according to claim 2 further comprising:
performing bi-linear interpolation to determine a first and a second half-pixel value in the second direction centered on the half-pixel location having the lowest MAD value in the first direction, the first and the second half-pixel values in the second direction being conjugally related;
determining a MAD value at each of the first and the second half-pixel locations in the second direction; and
comparing the MAD values to determine the half-pixel location having the lowest MAD value in the second direction.
7. The method according to claim 1 wherein the first direction is horizontal and parallel to an x axis and the second direction is vertical and parallel to a y-axis.
8. The method according to claim 1 wherein the first direction is perpendicular to the second direction.
9. An apparatus for obtaining a motion vector between a first and a second frame of video image data having sub-pixel accuracy comprising:
a memory having a program and a processor, the program configured, when executed, to perform:
identifying a first conjugate sub-pixel value in a first direction having a lowest first direction MAD value;
identifying a second conjugate sub-pixel value in a second direction having a lowest second direction MAD value; and
determining a motion vector representing the distance between the first macroblock and the second macroblock using the integer pixel location and a sub-integer pixel location, wherein the sub-pixel location is obtained from the first conjugate sub-pixel value and the second conjugate sub-pixel value having the lowest MAD value.
10. The apparatus according to claim 9 wherein program is further configured to perform:
performing bi-linear interpolation to determine a first and a second half-pixel value in the first direction centered on the integer pixel location, the first and the second half-pixel values being conjugally related;
determining the MAD value at each of the first and second half-pixel locations; and
comparing the MAD values to determine the half-pixel location having the lowest MAD value in the first direction.
11. The apparatus according to claim 10 wherein program is further configured to perform:
performing bi-linear interpolation to determine a first and a second half-pixel value in the second direction centered on the half-pixel location having the lowest MAD value in the first direction, the first and the second half-pixel values in the second direction being conjugally related;
determining a MAD value at each of the first and the second half-pixel locations in the second direction; and
comparing the MAD values to determine the half-pixel location having the lowest MAD value in the second direction.
12. A computer program comprising:
a computer-usable medium having program components embodied therein, when executed by a processor, for obtaining a motion vector having half-pixel accuracy between a first and a second frame of video image data, the computer usable medium comprising:
at an integer pixel location having a lowest mean of absolute difference (“MAD”) value in the second frame relative to the first frame,
a component of the program configured to identify a first conjugate sub-pixel value in a first direction having a lowest first direction MAD value;
a component of the program configured to identify a second conjugate sub-pixel value in a second direction having a lowest second direction MAD value; and
a component of the program configured to determine a motion vector representing the distance between the first macroblock and the second macroblock using the integer pixel location and a sub-integer pixel location, wherein the sub-pixel location is obtained from the first conjugate sub-pixel value and the second conjugate sub-pixel value having the lowest MAD value.
13. The computer program according to claim 12, wherein the component of the program configured to identify the first conjugate sub pixel value further comprises:
a component of the program configured to perform bi-linear interpolation to determine a first and a second half-pixel value in the first direction centered on the integer pixel location, the first and the second half-pixel values being conjugally related;
a component of the program configured to determine a MAD value at each of the first and second half-pixel locations; and
a component of the program configured to compare the MAD values to determine the half-pixel location having the lowest MAD value in the first direction.
14. The computer program according to claim 13, wherein the component of the program configured to identify the second conjugate sub pixel value further comprises:
a component of the program configured to perform bi-linear interpolation to determine a first and a second half-pixel value in the second direction centered on the half-pixel location having the lowest MAD value in the first direction, the first and the second half-pixel values in the second direction being conjugally related;
a component of the program configured to determine a MAD value at each of the first and the second half-pixel locations in the second direction; and
a component of the program configured to compare the MAD values to determine the half-pixel location having the lowest MAD value in the second direction.
15. A device for obtaining a motion vector between a first macroblock contained in a first frame and a second macroblock contained in a second frame of video image data comprising:
a memory device that stores executable program codes;
a processor that accesses the memory device and executes the instructions contained therein, the processor comprising:
a component configured to identify a first conjugate sub-pixel value in a first direction having a lowest first direction MAD value;
a component configured to identify a second conjugate sub-pixel value in a second direction having a lowest second direction MAD value; and
a component configured to determine a motion vector representing the distance between the first macroblock and the second macroblock using the integer pixel location and a sub-integer pixel location, wherein the sub-pixel location is obtained from the first conjugate sub-pixel value and the second conjugate sub-pixel value having the lowest MAD value.
16. The device according to claim 15 wherein the component configured to identify the first conjugate sub pixel value further comprises:
a component configured to perform bi-linear interpolation to determine a first and a second half-pixel value in the first direction centered on the integer pixel location, the first and the second half-pixel values being conjugally related;
a component configured to determine a MAD value at each of the first and second half-pixel locations; and
a component configured to compare the MAD values to determine the half-pixel location having the lowest MAD value in the first direction.
17. The device according to claim 16 wherein the component configured to identify the second conjugate sub-pixel value further comprises:
a component configured to perform bi-linear interpolation to determine a first and a second half-pixel value in the second direction centered on the half-pixel location having the lowest MAD value in the first direction, the first and the second half-pixel values in the second direction being conjugally related;
a component configured to determine a MAD value at each of the first and the second half-pixel locations in the second direction; and
a component configured to compare the MAD values to determine the half-pixel location having the lowest MAD value in the second direction.
18. A fast search sub-pixel accuracy method for obtaining a motion vector between successive frames of video image data, the method comprising the steps of:
determining a first macroblock of a first frame comprising a plurality of pixels arranged in a two-dimensional array;
selecting a second macroblock of a second frame comprising a plurality of pixels arranged in a two-dimensional array having the lowest mean of absolute differences (MAD) value between the first and the second macroblocks;
determining a minimum pixel location having the lowest MAD value in the second macroblock;
calculating conjugate sub-pixel values in a first direction centered on the minimum pixel location;
determining the conjugate sub-pixel value in the first direction having the lowest MAD value;
calculating conjugate sub-pixel values in a second direction centered on the sub-pixel having the lowest MAD value in the first direction;
determining the conjugate sub-pixel value in the second direction having the lowest MAD value; and
determining a motion vector for the macroblock representing the distance between the first macroblock and the second macroblock using the integer pixel location and the sub integer pixel location having the lowest MAD values.
19. The fast search sub-pixel method according to claim 18, wherein the step of determining the motion vector further comprises the steps of:
generating an integer motion vector using the integer pixel location;
generating a sub-pixel motion vector using the sub-pixel location; and
generating a motion vector by adding the integer motion vector to the sub-pixel motion vector.
20. The fast search sub-pixel method according to claim 18, wherein the first direction is in a horizontal direction parallel to an x axis and the second direction is in a vertical direction parallel to a y axis of a grid.
21. The fast search sub-pixel method according to claim 18, wherein the step of calculating conjugate sub-pixel values further comprises the step of performing bilinear interpolation to determine a half-pixel value.
22. The fast search sub-pixel method according to claim 18, wherein the step of determining the motion vector for the macroblock further comprises the step of:
determining an integer pixel motion vector for the minimum pixel location;
determining a sub-pixel motion vector for the conjugate sub-pixel in the second direction having the lowest MAD value; and
adding the integer pixel motion vector together with the sub-pixel motion vector.
23. The fast search sub-pixel method according to claim 18, wherein the steps of determining the conjugate sub-pixel value having the lowest MAD value further comprises the step of comparing the conjugate sub-pixel values with the integer pixel value to determine the lowest MAD value.
24. A method of obtaining a motion vector between a first macroblock contained in a first frame and a second macroblock contained in a second frame of video image data comprising:
at an integer pixel location having a lowest mean of absolute difference (“MAD”) value in the second frame relative to the first frame, determining a first sub-pixel value in a first direction having a lowest first direction MAD value;
determining a second sub-pixel value in a second direction centered on the first sub-pixel value, the second sub-pixel value having a lowest second direction MAD value; and
determining a motion vector representing the distance between the first macroblock and the second macroblock using the integer pixel location and the second sub-integer pixel location.
25. A method of obtaining a motion vector between a first macroblock contained in a first frame and a second macroblock contained in a second frame of video image data comprising:
at an integer pixel location having a lowest mean of absolute difference (“MAD”) value in the second frame relative to the first frame, determining a first sub-pixel value in a first diagonal direction having a lowest first direction MAD value;
determining a second sub-pixel value in a second diagonal direction centered on the first sub-pixel value, the second sub-pixel value having a lowest second direction MAD value; and
determining a motion vector representing the distance between the first macroblock and the second macroblock using the integer pixel location and the second sub-integer pixel location.
26. A method of obtaining a motion vector between a first macroblock contained in a first frame and a second macroblock contained in a second frame of video image data in a video recorder, the method comprising:
at an integer pixel location having a lowest mean of absolute difference (“MAD”) value in the second frame relative to the first frame, determining a first sub-pixel value in a first direction having a lowest first direction MAD value;
determining a second sub-pixel value in a second direction centered on the first sub-pixel value, the second sub-pixel value having a lowest second direction MAD value;
determining a motion vector representing the distance between the first macroblock and the second macroblock using the integer pixel location and the second sub-integer pixel location; and
storing the motion vector on a storage device.
US09/801,584 2001-03-08 2001-03-08 Device and method for performing half-pixel accuracy fast search in video coding Abandoned US20020172288A1 (en)

Priority Applications (8)

Application Number Priority Date Filing Date Title
US09/801,584 US20020172288A1 (en) 2001-03-08 2001-03-08 Device and method for performing half-pixel accuracy fast search in video coding
AU2002240754A AU2002240754A1 (en) 2001-03-08 2002-03-08 A device and method for performing half-pixel accuracy fast search in video coding
JP2002570523A JP4739651B2 (en) 2001-03-08 2002-03-08 Apparatus and method for realizing high-precision intermediate pixel high-speed search in video encoding
PCT/CA2002/000324 WO2002071741A2 (en) 2001-03-08 2002-03-08 A device and method for performing half-pixel accuracy fast search in video coding
US10/471,085 US7792191B2 (en) 2001-03-08 2002-03-08 Device and method for performing half-pixel accuracy fast search in video coding
JP2008244850A JP2009027744A (en) 2001-03-08 2008-09-24 Device and method for performing half-pixel accuracy fast search in video coding
JP2009230638A JP2010045816A (en) 2001-03-08 2009-10-02 Device and method for performing half-pixel accuracy fast search in video coding
US12/877,109 US20110064138A1 (en) 2001-03-08 2010-09-08 Device and method for performing half pixel accuracy fast search in video coding

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US09/801,584 US20020172288A1 (en) 2001-03-08 2001-03-08 Device and method for performing half-pixel accuracy fast search in video coding

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US10471085 Continuation-In-Part 2002-03-08

Publications (1)

Publication Number Publication Date
US20020172288A1 true US20020172288A1 (en) 2002-11-21

Family

ID=25181521

Family Applications (3)

Application Number Title Priority Date Filing Date
US09/801,584 Abandoned US20020172288A1 (en) 2001-03-08 2001-03-08 Device and method for performing half-pixel accuracy fast search in video coding
US10/471,085 Expired - Fee Related US7792191B2 (en) 2001-03-08 2002-03-08 Device and method for performing half-pixel accuracy fast search in video coding
US12/877,109 Abandoned US20110064138A1 (en) 2001-03-08 2010-09-08 Device and method for performing half pixel accuracy fast search in video coding

Family Applications After (2)

Application Number Title Priority Date Filing Date
US10/471,085 Expired - Fee Related US7792191B2 (en) 2001-03-08 2002-03-08 Device and method for performing half-pixel accuracy fast search in video coding
US12/877,109 Abandoned US20110064138A1 (en) 2001-03-08 2010-09-08 Device and method for performing half pixel accuracy fast search in video coding

Country Status (4)

Country Link
US (3) US20020172288A1 (en)
JP (3) JP4739651B2 (en)
AU (1) AU2002240754A1 (en)
WO (1) WO2002071741A2 (en)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060215058A1 (en) * 2005-03-28 2006-09-28 Tiehan Lu Gradient adaptive video de-interlacing
WO2007030716A2 (en) * 2005-09-09 2007-03-15 Broadcast International, Inc. Bit-rate reduction of multimedia data streams
US9131202B1 (en) * 2014-05-30 2015-09-08 Paofit Holdings Pte. Ltd. Systems and methods for motion-vector-aided video interpolation using real-time smooth video playback speed variation
US10810798B2 (en) 2015-06-23 2020-10-20 Nautilus, Inc. Systems and methods for generating 360 degree mixed reality environments
US10828570B2 (en) 2011-09-08 2020-11-10 Nautilus, Inc. System and method for visualizing synthetic objects within real-world video clip
US20210216774A1 (en) * 2016-02-19 2021-07-15 Carrier Corporation Cloud based active commissioning system for video analytics
USRE48845E1 (en) 2002-04-01 2021-12-07 Broadcom Corporation Video decoding system supporting multiple standards

Families Citing this family (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040146108A1 (en) * 2003-01-23 2004-07-29 Shih-Chang Hsia MPEG-II video encoder chip design
NO319629B1 (en) * 2003-11-28 2005-09-05 Tandberg Telecom As Procedure for correcting interpolated pixel values
CN100377599C (en) * 2004-09-03 2008-03-26 北京航空航天大学 A fast sub-picture element movement estimating method
US7751478B2 (en) 2005-01-21 2010-07-06 Seiko Epson Corporation Prediction intra-mode selection in an encoder
US7830961B2 (en) 2005-06-21 2010-11-09 Seiko Epson Corporation Motion estimation and inter-mode prediction
US7843995B2 (en) 2005-12-19 2010-11-30 Seiko Epson Corporation Temporal and spatial analysis of a video macroblock
US8170102B2 (en) 2005-12-19 2012-05-01 Seiko Epson Corporation Macroblock homogeneity analysis and inter mode prediction
US9307122B2 (en) * 2006-09-27 2016-04-05 Core Wireless Licensing S.A.R.L. Method, apparatus, and computer program product for providing motion estimation for video encoding
KR101107254B1 (en) * 2007-01-25 2012-01-20 삼성전자주식회사 Method for estimating motion vector using motion vector of near block and apparatus therefor
US8363727B2 (en) 2008-09-30 2013-01-29 Microsoft Corporation Techniques to perform fast motion estimation
CN101867815B (en) * 2010-04-30 2011-09-14 西北工业大学 Rapid fractional pixel hierarchical searching method
EP2424243B1 (en) * 2010-08-31 2017-04-05 OCT Circuit Technologies International Limited Motion estimation using integral projection
US10045046B2 (en) 2010-12-10 2018-08-07 Qualcomm Incorporated Adaptive support for interpolating values of sub-pixels for video coding
US20120163460A1 (en) * 2010-12-23 2012-06-28 Qualcomm Incorporated Sub-pixel interpolation for video coding
US8483489B2 (en) 2011-09-02 2013-07-09 Sharp Laboratories Of America, Inc. Edge based template matching
EP3023938A1 (en) 2014-11-21 2016-05-25 Thomson Licensing Method and apparatus for tracking the motion of image content in a video frames sequence using sub-pixel resolution motion estimation
CN106658024B (en) * 2016-10-20 2019-07-16 杭州当虹科技股份有限公司 A kind of quick method for video coding

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5936672A (en) * 1996-03-22 1999-08-10 Daewoo Electronics Co., Ltd. Half pixel motion estimator
US6104439A (en) * 1992-02-08 2000-08-15 Samsung Electronics Co., Ltd. Method and apparatus for motion estimation
US6118901A (en) * 1997-10-31 2000-09-12 National Science Council Array architecture with data-rings for 3-step hierarchical search block matching algorithm
US6175593B1 (en) * 1997-07-30 2001-01-16 Lg Electronics Inc. Method for estimating motion vector in moving picture
US6332002B1 (en) * 1997-11-01 2001-12-18 Lg Electronics Inc. Motion prediction apparatus and method
US6584155B2 (en) * 1999-12-27 2003-06-24 Kabushiki Kaisha Toshiba Method and system for estimating motion vector

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH06105970B2 (en) 1988-03-14 1994-12-21 松下電工株式会社 Motion compensation method
KR950014862B1 (en) * 1992-02-08 1995-12-16 삼성전자주식회사 Motion estimation method and apparatus
US6101313A (en) * 1992-06-29 2000-08-08 Sony Corporation High efficiency encoding and decoding of picture signals and recording medium containing same
JP2636674B2 (en) * 1993-05-25 1997-07-30 日本電気株式会社 Video motion vector detection device
DE4342305A1 (en) * 1993-12-11 1995-06-29 Thomson Brandt Gmbh Method for hierarchical motion estimation in a television signal
JPH08265770A (en) * 1995-03-20 1996-10-11 Sony Corp High efficiency encoding method, high efficiency encoder, recording and reproducing device and information transmission system
JP3539835B2 (en) 1997-03-04 2004-07-07 三菱電機株式会社 Block matching search method and apparatus using the method
US6011870A (en) * 1997-07-18 2000-01-04 Jeng; Fure-Ching Multiple stage and low-complexity motion estimation for interframe video coding
JP2000253407A (en) 1999-02-26 2000-09-14 Matsushita Electric Ind Co Ltd Method for detecting motion vector and method for encoding image
JP2000308064A (en) * 1999-04-22 2000-11-02 Mitsubishi Electric Corp Motion vector detecting device
US6757330B1 (en) * 2000-06-01 2004-06-29 Hewlett-Packard Development Company, L.P. Efficient implementation of half-pixel motion prediction

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6104439A (en) * 1992-02-08 2000-08-15 Samsung Electronics Co., Ltd. Method and apparatus for motion estimation
US5936672A (en) * 1996-03-22 1999-08-10 Daewoo Electronics Co., Ltd. Half pixel motion estimator
US6175593B1 (en) * 1997-07-30 2001-01-16 Lg Electronics Inc. Method for estimating motion vector in moving picture
US6118901A (en) * 1997-10-31 2000-09-12 National Science Council Array architecture with data-rings for 3-step hierarchical search block matching algorithm
US6332002B1 (en) * 1997-11-01 2001-12-18 Lg Electronics Inc. Motion prediction apparatus and method
US6584155B2 (en) * 1999-12-27 2003-06-24 Kabushiki Kaisha Toshiba Method and system for estimating motion vector

Cited By (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
USRE48845E1 (en) 2002-04-01 2021-12-07 Broadcom Corporation Video decoding system supporting multiple standards
US20090322942A1 (en) * 2005-03-28 2009-12-31 Tiehan Lu Video de-interlacing with motion estimation
US7907210B2 (en) 2005-03-28 2011-03-15 Intel Corporation Video de-interlacing with motion estimation
US20060215058A1 (en) * 2005-03-28 2006-09-28 Tiehan Lu Gradient adaptive video de-interlacing
US7567294B2 (en) * 2005-03-28 2009-07-28 Intel Corporation Gradient adaptive video de-interlacing
WO2007030716A3 (en) * 2005-09-09 2007-10-25 Broadcast International Inc Bit-rate reduction of multimedia data streams
US20070058716A1 (en) * 2005-09-09 2007-03-15 Broadcast International, Inc. Bit-rate reduction for multimedia data streams
US8160160B2 (en) 2005-09-09 2012-04-17 Broadcast International, Inc. Bit-rate reduction for multimedia data streams
WO2007030716A2 (en) * 2005-09-09 2007-03-15 Broadcast International, Inc. Bit-rate reduction of multimedia data streams
US10828570B2 (en) 2011-09-08 2020-11-10 Nautilus, Inc. System and method for visualizing synthetic objects within real-world video clip
US9131202B1 (en) * 2014-05-30 2015-09-08 Paofit Holdings Pte. Ltd. Systems and methods for motion-vector-aided video interpolation using real-time smooth video playback speed variation
US9659596B2 (en) 2014-05-30 2017-05-23 Paofit Holdings Pte. Ltd. Systems and methods for motion-vector-aided video interpolation using real-time smooth video playback speed variation
US10810798B2 (en) 2015-06-23 2020-10-20 Nautilus, Inc. Systems and methods for generating 360 degree mixed reality environments
US20210216774A1 (en) * 2016-02-19 2021-07-15 Carrier Corporation Cloud based active commissioning system for video analytics
US11721099B2 (en) * 2016-02-19 2023-08-08 Carrier Corporation Cloud based active commissioning system for video analytics

Also Published As

Publication number Publication date
WO2002071741A2 (en) 2002-09-12
JP2010045816A (en) 2010-02-25
WO2002071741A8 (en) 2003-10-30
US20040196909A1 (en) 2004-10-07
AU2002240754A1 (en) 2002-09-19
US20110064138A1 (en) 2011-03-17
JP2009027744A (en) 2009-02-05
JP2004526363A (en) 2004-08-26
US7792191B2 (en) 2010-09-07
WO2002071741A3 (en) 2002-10-31
JP4739651B2 (en) 2011-08-03

Similar Documents

Publication Publication Date Title
US20110064138A1 (en) Device and method for performing half pixel accuracy fast search in video coding
US6483876B1 (en) Methods and apparatus for reduction of prediction modes in motion estimation
US6128047A (en) Motion estimation process and system using sparse search block-matching and integral projection
EP1262073B1 (en) Methods and apparatus for motion estimation using neighboring macroblocks
US6130912A (en) Hierarchical motion estimation process and system using block-matching and integral projection
US6414997B1 (en) Hierarchical recursive motion estimator for video images encoder
US6483928B1 (en) Spatio-temporal recursive motion estimation with 1/2 macroblock and 1/4 pixel undersampling
EP0762776B1 (en) A method and apparatus for compressing video information using motion dependent prediction
US20030063673A1 (en) Motion estimation and/or compensation
US8428135B2 (en) Device and method for fast sub sample block-matching motion estimation in video encoders
US6690728B1 (en) Methods and apparatus for motion estimation in compressed domain
KR20010071705A (en) Motion estimation for digital video
EP1413144A2 (en) Methods and apparatus for sub-pixel motion estimation
US20030161400A1 (en) Method and system for improved diamond motion search
US9008450B1 (en) Directional cross hair search system and method for determining a preferred motion vector
US6996180B2 (en) Fast half-pixel motion estimation using steepest descent
US6480629B1 (en) Motion estimation method using orthogonal-sum block matching
US6912296B2 (en) Motion estimation method
EP1420595B1 (en) Motion vector selection in a video motion estimator based on a preferred reference point
KR0152014B1 (en) Method and apparauts for moving estimation in image data compression
US6463164B1 (en) Motion vector estimation based on statistical features of an image frame
KR100197371B1 (en) Estimating motion vectors of image in sub-band encoding system

Legal Events

Date Code Title Description
AS Assignment

Owner name: AVT AUDIO VISUAL TELECOMMUNICATIONS CORPORATION, C

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:KWON, NYEONGKU;REEL/FRAME:011877/0585

Effective date: 20010507

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION

AS Assignment

Owner name: LAMBERT EVEREST LTD., VIRGIN ISLANDS, BRITISH

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:AVT AUDIO VISUAL TELECOMMUNICATIONS CORPORATION;REEL/FRAME:018877/0206

Effective date: 20070124