WO2004063992A1 - Video frame correlation for motion estimation - Google Patents

Video frame correlation for motion estimation Download PDF

Info

Publication number
WO2004063992A1
WO2004063992A1 PCT/CA2004/000093 CA2004000093W WO2004063992A1 WO 2004063992 A1 WO2004063992 A1 WO 2004063992A1 CA 2004000093 W CA2004000093 W CA 2004000093W WO 2004063992 A1 WO2004063992 A1 WO 2004063992A1
Authority
WO
WIPO (PCT)
Prior art keywords
correlation
search window
pixels
window
frame
Prior art date
Application number
PCT/CA2004/000093
Other languages
French (fr)
Inventor
Patrick Rault
Zhihua Zeng
Original Assignee
Vixs Systems Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Vixs Systems Inc. filed Critical Vixs Systems Inc.
Priority to AT04701876T priority Critical patent/ATE555457T1/en
Priority to JP2006500441A priority patent/JP2006517069A/en
Priority to EP04701876A priority patent/EP1584069B1/en
Publication of WO2004063992A1 publication Critical patent/WO2004063992A1/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion

Definitions

  • the present disclosure relates generally to video processing, and more particularly to a method and system of encoding video data.
  • Encoding of digital video data is generally performed in order to compress the amount of data needed to represent a video stream. It is known in the art to encode a frame one set of pixels at a time. For example, each macroblock of a frame can be encoded by comparing the macroblock's pixels to the pixels of other macroblocks of a frame that has been previously displayed, or decoded. Once a set of pixels in the previous frame that corresponds closely to the set of pixels being encoded is found, a motion vector pointing to that set of pixels is identified. Once a motion vector to the set of pixels is identified, any information difference between the two sets of pixels can be quantified and efficiently compressed.
  • Known methods of encoding will correlate the set of pixels being encoded to sets of pixels in the previous frame or a portion of the previous frame until a correlation threshold is met. Once the correlation threshold is met, it is known that a pixel set in a previous frame that corresponds sufficiently close to the pixel set being encoded has been found. Due to the large amount of data associated with video streams, and video frames, the encoding process can be a very time-consuming process. Therefore, a system or method capable of more efficiently encoding pixels sets would be useful.
  • Figure 1 illustrates, in block diagram form, the graphical representation of a reference frame of video data in accordance with the present disclosure
  • Figure 2 illustrates, in flow diagram form, a method in accordance with the present disclosure
  • FIGURE 3 illustrates, in block diagram form, a system for implementing specific embodiments of the present disclosure.
  • a first window is selected in a reference frame of video data.
  • the first window contains one or more pixel sets, each that are to be correlated with a pixel set being encoded to determine a correlation factor.
  • a first threshold value is determined for the first search window.
  • the first threshold value is the value to which correlation factors are compared. For example, if a correlation factor between a specific pixel set of the first search window and the pixel set being encoded meets the first threshold value, a successful match between the two pixel sets has been found, and a corresponding motion vector can be assigned to the pixel set being encoded.
  • a second search window within the first frame is selected.
  • a second threshold value different than the first threshold value is determined for the second search window.
  • the second threshold value will be less stringent than the first threshold value.
  • the correlation factors for the pixel sets of the second search window that are correlated to the pixel set being encoded are compared to the second threshold value.
  • Figure 1 illustrates frames Fl 102 and F2 104.
  • Frame Fl 102 represents a reference frame with respect to the frame F2 104 that is being encoded.
  • Frame F 1 102 is a reference frame in that the pixel data associated with frame F 1 will be available, and used, at the time that pixel sets of frame F2 104 are being encoded or decoded. Therefore, during the encoding process each pixel set, such as a macroblock or a block, of frame F2 104 will be compared against a portion of Frame Fl 102 to determine whether a substantially similar set of pixels resides within Frame Fl 102.
  • a specific embodiment of the present disclosure will be further discussed with reference to the method of Figure 2 and frame data of FIG. 1.
  • a first search window size and location is determined.
  • the search window 111 is defined.
  • the search window 111 represents an area of frame F 1 102 that has one or more pixel sets that will be correlated to macroblock F2/52.
  • F2/52 refers to the macroblock at macroblock location 52 in frame F2 104, which is the macroblock being encoded.
  • the searches are not constrained by specific pixel sets, such as macroblock boundaries. In other words, the pixel sets identified within a search window can reside across encoding pixel set boundaries.
  • the first search window 111 contains a single pixel set to be searched. In other embodiments, multiple pixel sets are contained with the first search window.
  • the location of the first search window is generally be based upon a predicted motion vector. It will be appreciated that there are many method of predicting motion vectors, and how the initial location of the first search window is determined can be determined by a variety of methods.
  • a correlation threshold is set for the first window.
  • a correlation factor is determined. This correlation factor is compared to the correlation threshold of the first window to determine when a successful match between the pixel sets has been found.
  • the correlation threshold for the first window would be set relatively high, compared to other correlation thresholds, because the number of pixels sets within the first window is relatively small. Because of the relatively small number pixel sets anticipated within the first window, a somewhat higher correlation factor can be used without affecting performance.
  • a search window pixel set is determined. With respect to Figure 1 a single search window pixel set may be present. Regardless, one pixel set within the search window will be identified.
  • a correlation factor is determined between the pixel set being encoded and the search window pixel set.
  • the correlation factor can be determined using any of a variety of correlation techniques. Such techniques can be as simple as subtracting one pixel set from the other to determine a difference between the pixel sets. Other techniques can be more complicated. For example, where the pixel set information represents spatial data, the correlation technique can perform a mathematical transform to convert the data to non-spatial pixel set data. For example, sub- sampling techniques and/or techniques that operate on frequency domain data can be used. Furthermore, it will be appreciated, that each application of the step 204, in the loop formed by steps 204-205-211-210, can apply the same correlation technique or different correlation techniques. For example, the first search window location can use a different correlation technique than a subsequent search window.
  • step 205 a determination is made whether or not the correlation factor meets the correlation threshold identified in step 202. If the correlation factor does meet the correlation threshold the flow proceeds to step 206 where a motion vector is used corresponding to the search window pixel set. If at step 205 the correlation factor does not meet the correlation threshold, the flow proceeds to step 211. Note that in an embodiment where the current window is to be the last window searched, the correlation threshold can be set so that it will never be met. This would allow the pixel set with the best correlation factor to be selected at step 221.
  • step 211 a determination is made whether or not more search window pixel sets exist in the current search window. If so, the flow proceeds to step 210, otherwise the flow proceeds to step 209.
  • a next search window pixel set is determined within the current window. Once the next search window pixel set is determined the flow proceeds to step 204; A loop including steps 204, 205, 211, and 210 continues until all of the search window pixel sets within the current window have been correlated to the pixel set being encoded, or until a successful correlation has occurred.
  • step 211 When no more search window pixel sets exist in the current search window the flow proceeds from step 211 to step 209. At step 209, a determination is made whether or not more search windows are to be identified. If not, the flow proceeds from step 209 to step 221 where the method of FIG. 2 selects the best pixel set location and/or terminates without a successful correlation being found. However, if additional search windows are to be identified, the flow proceeds to step 208.
  • a next search window size and location is determined.
  • a search window 112 is the next search window identified.
  • Step 207 a correlation threshold is set for the next search window 112.
  • a new correlation threshold value would be set for the window size 112. Because the window size 112 contains more pixel sets than the search window Ili a less stringent correlation factor will be tolerated. In other words, to avoid a long encode time, a tradeoff in the quality of correlation is allowed.
  • a new search window pixel set is identified. It will be appreciated that where the first search window 111 is a subset of the new search window 112, the correlation factors associated with the pixel sets of the first search window can be maintained, without re-correlating, and compared to the new threshold value. By saving the correlation factors for the first search window only the search window pixel sets that are unique to the second search window 112 would have to be correlated, thereby saving processing time.
  • the steps 204, 205, 211, and 210 will be repeated until either all of the search window pixel sets have been correlated against the pixel set being encoded, or a successful pixel set correlation has occurred.
  • search windows 111, 112, and 113 can each be searched for pixels sets meeting different correlation threshold values.
  • search windows 111, 112, and 113 can each be searched for pixels sets meeting different correlation threshold values.
  • tradeoffs between picture quality and the amount of processing time expected to be potentially spent can be dynamically maintained. It will be appreciated, that this is an advantage over the prior art, which would identify a single window and a single threshold value.
  • Figure 3 illustrates a system in accordance with a specific embodiment of the present disclosure.
  • Figure 3 illustrates a system 300 having a data processor 310, and a memory 320.
  • the data processor 310 accesses the memory 300 to execute program instructions 322 and to manipulate video data 324.
  • the video data 324 would generally include the video frame data of frames Fl 202 and F2 204 of Figure 1.
  • the video processor 310 would generally comprise an instruction execution unit for implementing the instructions.
  • the data processor 310 can include co-processors 312, which can include specific hardware accelerators and/or microcode engines, capable of implementing some or all of the encoding process described herein.
  • the information processor 300 of Figure 3 can be part of a general purpose computer, special purpose computer, or integrated as a portion of a larger system.

Abstract

A first search window within a reference frame of video data is identified along with a first correlation threshold value for the first window. The first correlation threshold value is a value to which correlation factors between a pixel set being encoded and pixels sets of the first search window are compared. For example, if a correlation factor between a specific pixel set of the first search window and a pixel set being encoded meets the first threshold value, a successful match between the two pixel sets has been found, and a corresponding motion vector can be assigned to the pixel set being encoded. If none of the pixel sets within the first window meet the first threshold value, a second search window within the first frame is selected along with a second correlation threshold value. The correlation factors for pixel sets in the second window are compared to the second correlation threshold value.

Description

VIDEO FRAME CORRELATION FOR MOTION ESTIMATION
FIELD OF THE DISCLOSURE
The present disclosure relates generally to video processing, and more particularly to a method and system of encoding video data.
BACKGROUND
Encoding of digital video data is generally performed in order to compress the amount of data needed to represent a video stream. It is known in the art to encode a frame one set of pixels at a time. For example, each macroblock of a frame can be encoded by comparing the macroblock's pixels to the pixels of other macroblocks of a frame that has been previously displayed, or decoded. Once a set of pixels in the previous frame that corresponds closely to the set of pixels being encoded is found, a motion vector pointing to that set of pixels is identified. Once a motion vector to the set of pixels is identified, any information difference between the two sets of pixels can be quantified and efficiently compressed.
Known methods of encoding will correlate the set of pixels being encoded to sets of pixels in the previous frame or a portion of the previous frame until a correlation threshold is met. Once the correlation threshold is met, it is known that a pixel set in a previous frame that corresponds sufficiently close to the pixel set being encoded has been found. Due to the large amount of data associated with video streams, and video frames, the encoding process can be a very time-consuming process. Therefore, a system or method capable of more efficiently encoding pixels sets would be useful.
BRIEF DESCRIPTION OF THE DRAWINGS
Figure 1 illustrates, in block diagram form, the graphical representation of a reference frame of video data in accordance with the present disclosure;
Figure 2 illustrates, in flow diagram form, a method in accordance with the present disclosure; and
Figure 3 illustrates, in block diagram form, a system for implementing specific embodiments of the present disclosure. DETAILED DESCRIPTION OF THE FIGURES
In accordance with the specific embodiment of the present disclosure, a first window is selected in a reference frame of video data. The first window contains one or more pixel sets, each that are to be correlated with a pixel set being encoded to determine a correlation factor. In addition, a first threshold value is determined for the first search window. The first threshold value is the value to which correlation factors are compared. For example, if a correlation factor between a specific pixel set of the first search window and the pixel set being encoded meets the first threshold value, a successful match between the two pixel sets has been found, and a corresponding motion vector can be assigned to the pixel set being encoded.
If none of the pixel sets within the first window meet the first threshold value, a second search window within the first frame is selected. A second threshold value different than the first threshold value, is determined for the second search window. Typically, the second threshold value will be less stringent than the first threshold value. The correlation factors for the pixel sets of the second search window that are correlated to the pixel set being encoded are compared to the second threshold value. By using multiple search windows with varying threshold values, a more efficient encoding process can be obtained. For example, by setting the first search window to be relatively small, a faster match can be expected while maintaining a high correlation level. This provides a decoder system a mechanism to set a trade off between processing power of an encode process, and the desired quality of the encode process. Figures 1 through 4 further illustrate specific embodiments of the present disclosure.
Figure 1 illustrates frames Fl 102 and F2 104. Frame Fl 102 represents a reference frame with respect to the frame F2 104 that is being encoded. Frame F 1 102 is a reference frame in that the pixel data associated with frame F 1 will be available, and used, at the time that pixel sets of frame F2 104 are being encoded or decoded. Therefore, during the encoding process each pixel set, such as a macroblock or a block, of frame F2 104 will be compared against a portion of Frame Fl 102 to determine whether a substantially similar set of pixels resides within Frame Fl 102. A specific embodiment of the present disclosure will be further discussed with reference to the method of Figure 2 and frame data of FIG. 1.
At step 201 a first search window size and location is determined. For example, referring to Figure 1 , the search window 111 is defined. The search window 111 represents an area of frame F 1 102 that has one or more pixel sets that will be correlated to macroblock F2/52. Note that F2/52 refers to the macroblock at macroblock location 52 in frame F2 104, which is the macroblock being encoded. It will be further appreciated, that within specific search windows, the searches are not constrained by specific pixel sets, such as macroblock boundaries. In other words, the pixel sets identified within a search window can reside across encoding pixel set boundaries. In one embodiment, the first search window 111 contains a single pixel set to be searched. In other embodiments, multiple pixel sets are contained with the first search window.
It will be appreciated that the location of the first search window is generally be based upon a predicted motion vector. It will be appreciated that there are many method of predicting motion vectors, and how the initial location of the first search window is determined can be determined by a variety of methods.
At step 202 of Figure 2, a correlation threshold is set for the first window. As the pixel set being encoded is correlated to the pixel sets of the first window a correlation factor is determined. This correlation factor is compared to the correlation threshold of the first window to determine when a successful match between the pixel sets has been found. Generally, the correlation threshold for the first window would be set relatively high, compared to other correlation thresholds, because the number of pixels sets within the first window is relatively small. Because of the relatively small number pixel sets anticipated within the first window, a somewhat higher correlation factor can be used without affecting performance.
At step 203, a search window pixel set is determined. With respect to Figure 1 a single search window pixel set may be present. Regardless, one pixel set within the search window will be identified.
At step 204, a correlation factor is determined between the pixel set being encoded and the search window pixel set. The correlation factor can be determined using any of a variety of correlation techniques. Such techniques can be as simple as subtracting one pixel set from the other to determine a difference between the pixel sets. Other techniques can be more complicated. For example, where the pixel set information represents spatial data, the correlation technique can perform a mathematical transform to convert the data to non-spatial pixel set data. For example, sub- sampling techniques and/or techniques that operate on frequency domain data can be used. Furthermore, it will be appreciated, that each application of the step 204, in the loop formed by steps 204-205-211-210, can apply the same correlation technique or different correlation techniques. For example, the first search window location can use a different correlation technique than a subsequent search window.
At step 205, a determination is made whether or not the correlation factor meets the correlation threshold identified in step 202. If the correlation factor does meet the correlation threshold the flow proceeds to step 206 where a motion vector is used corresponding to the search window pixel set. If at step 205 the correlation factor does not meet the correlation threshold, the flow proceeds to step 211. Note that in an embodiment where the current window is to be the last window searched, the correlation threshold can be set so that it will never be met. This would allow the pixel set with the best correlation factor to be selected at step 221.
At step 211 a determination is made whether or not more search window pixel sets exist in the current search window. If so, the flow proceeds to step 210, otherwise the flow proceeds to step 209.
At step 210, a next search window pixel set is determined within the current window. Once the next search window pixel set is determined the flow proceeds to step 204; A loop including steps 204, 205, 211, and 210 continues until all of the search window pixel sets within the current window have been correlated to the pixel set being encoded, or until a successful correlation has occurred.
When no more search window pixel sets exist in the current search window the flow proceeds from step 211 to step 209. At step 209, a determination is made whether or not more search windows are to be identified. If not, the flow proceeds from step 209 to step 221 where the method of FIG. 2 selects the best pixel set location and/or terminates without a successful correlation being found. However, if additional search windows are to be identified, the flow proceeds to step 208.
At step 208, a next search window size and location is determined. Referring to Figure 1, a search window 112 is the next search window identified.
Step 207, a correlation threshold is set for the next search window 112. Again, with reference to Figure 1, a new correlation threshold value would be set for the window size 112. Because the window size 112 contains more pixel sets than the search window Ili a less stringent correlation factor will be tolerated. In other words, to avoid a long encode time, a tradeoff in the quality of correlation is allowed. At step 203, a new search window pixel set is identified. It will be appreciated that where the first search window 111 is a subset of the new search window 112, the correlation factors associated with the pixel sets of the first search window can be maintained, without re-correlating, and compared to the new threshold value. By saving the correlation factors for the first search window only the search window pixel sets that are unique to the second search window 112 would have to be correlated, thereby saving processing time.
Again in the manner previously described, the steps 204, 205, 211, and 210 will be repeated until either all of the search window pixel sets have been correlated against the pixel set being encoded, or a successful pixel set correlation has occurred.
In the manner described, a plurality of search windows, i.e., search windows 111, 112, and 113, can each be searched for pixels sets meeting different correlation threshold values. In this manner, tradeoffs between picture quality and the amount of processing time expected to be potentially spent can be dynamically maintained. It will be appreciated, that this is an advantage over the prior art, which would identify a single window and a single threshold value.
Figure 3 illustrates a system in accordance with a specific embodiment of the present disclosure. Specifically, Figure 3 illustrates a system 300 having a data processor 310, and a memory 320. In operation, the data processor 310 accesses the memory 300 to execute program instructions 322 and to manipulate video data 324. For example, the video data 324 would generally include the video frame data of frames Fl 202 and F2 204 of Figure 1. Likewise, the video processor 310 would generally comprise an instruction execution unit for implementing the instructions. In addition, the data processor 310 can include co-processors 312, which can include specific hardware accelerators and/or microcode engines, capable of implementing some or all of the encoding process described herein. In will be further appreciated, that the information processor 300 of Figure 3 can be part of a general purpose computer, special purpose computer, or integrated as a portion of a larger system.
In the preceding detailed description of the embodiments, reference has been made to the accompanying drawings which for a part thereof, and in which is shown by way of illustration specific embodiments in which the disclosure may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the disclosure, and it is to be understood that other embodiments may be utilized and logical, mechanical and electrical changes maybe made without departing from the spirit or scope of the present disclosure. To avoid detail not necessary to enable those skilled in the art to practice the disclosure, the description may omit certain information known to those skilled in the art. Furthermore, many other varied embodiments that incorporate the teaching of the disclosure may be easily constructed by those skilled in the art. Accordingly, the present disclosure is not intended to be limited to the specific form set forth herein, but on the contrary, it is intended to cover such alternatives, modifications, and equivalents, as can be reasonably included within the spirit and scope of the disclosure. The preceding detailed description is, therefore, not to be taken in a limiting sense, and the scope of the present disclosure is defined only by the appended claims.

Claims

WHAT IS CLAIMED IS:
1. A method of processing a video signal, the method comprising: selecting a first search window of a first frame of video data, the first search window having a first window size; determining a first correlation factor between a first set of pixels within the first search window with respect to a second set of pixels in a second frame of video data; comparing the first correlation factor to a first threshold value; selecting a second search window of the first frame, the second search window having a second window size, the second window size greater than the first window size; determining a second correlation factor between a third set of pixels within the second search window with respect to the second set of pixels; and comparing the second correlation factor to a second threshold value, the second threshold value being different than the first threshold value.
2. The method of claim 1 , wherein selecting the second search window comprises the first set of pixels being within the second search window.
3. The method of claim 1 , further comprising selecting a third search window of the first frame, the third search window having a third window size, the third window size greater than the second window size.
4. The method of claim 3, further comprising determining a third correlation factor between a third fourth of pixels within the third search window with respect to the second set of pixels.
5. The method of claim 4, further comprising comparing the third correlation factor to a third threshold having a third correlation threshold value, the third correlation threshold value being different than the second correlation threshold value and different than the first correlation threshold value.
6. The method of claim 1 , further comprising determining a motion vector based on at least one of the pixels within the third set of pixels within the third search window.
7. The method of claim 1 further comprising: saving the first correlation factor; and comparing the first correlation factor to the second threshold value.
8. The method of claim 7 wherein comparing the first correlation factor to the second threshold value occurs before comparing the second correlation factor to the second threshold value.
9. The method of claim 1 , wherein determining the first correlation factor is determined based on a first correlation technique and determining the second correlation factor is determined based on a second correlation technique different than the first correlation technique.
10. The method of claim 2, wherein determining the first correlation factor is determined based on a first correlation technique and determining the second correlation factor is determined based on the first correlation technique.
11. The method of claim 1 further comprising: selecting a third search window of the first frame, the third search window having a third window size, the third window size greater than the first window size; determining a third correlation factor between the fourth set of pixels within the third search window with respect to the second set of pixels; when the fourth set of pixels has a best correlation factor of sets of pixels in the third search window, identifying the fourth set of pixels as a set of pixels with the best correlation factor in the third window.
12. A method comprising: selecting a first search window of a first frame of video data, the first search window having a first window size; comparing a first set of pixels within the first search window to a set of pixels in a second frame of video data to determine a first correlation; selecting a second search window of the first frame of video data, the second search window having a second window size, the second window size greater than the first window size; and comparing a second set of pixels within the second search window to the set of pixels in a second frame of video data to determine a second correlation.
13. The method of claim 12, further comprising evaluating the first correlation using a first threshold and evaluating the second correlation using a second threshold, the first threshold value being more stringent than the second threshold value.
14. The method of claim 12, further comprising selecting a third search window of the first frame of video data, the third search window having a third window size, the third window size greater than the second window size.
15. The method of claim 14, further comprising comparing a third set of pixels within the third search window to the set of pixels in the second frame of video data to determine a third correlation.
16. The method of claim 12, wherein the first window size is a equal to a size of the second set of pixels.
17. The method of claim 1 , wherein comparing the first set of pixels includes the first correlation being determined based on a first correlation technique and comparing the second set of pixels includes the second correlation being determined based on a second correlation technique different than the first correlation technique.
18. The method of claim 1 , wherein comparing the first set of pixels includes the first correlation being determined based on a first correlation technique and comparing the second set of pixels includes the second correlation being determined based on the first correlation technique.
19. The method of claim 12, wherein selecting the second search window comprises the first set of pixels being within the second search window.
20. A system comprising: a video data processing element; a memory coupled to the video data processing element, the memory comprising: a video data storage region to store a first frame of video data and a second frame of video data; and a program storage region to store program instructions, the program instructions to facilitate selecting a first search window of a first frame of video data, the first search window having a first window size; comparing a first set of pixels within the first search window to a set of pixels in a second frame of video data to determine a first correlation; selecting a second search window of the first frame of video data, the second search window having a second window size, the second window size greater than the first window size; and comparing a second set of pixels within the second search window to the set of pixels in a second frame of video data to determine a second correlation.
21. The system of claim 20, wherein the program instructions further comprise facilitating evaluating the first correlation using a first threshold; evaluating the second correlation using a second threshold, the first threshold being different than the second threshold.
22. The system of claim 20, wherein the program instructions further comprise facilitating selecting a third search window of the first frame of video data, the third search window having a third window size, wherein the third window size is greater than the second window size.
23. The system of claim 20, wherein the program instructions selecting the first window size comprise the first window size being a single pixel and the second window size includes a plurality of pixels.
24. A method of determining a motion vector for a pixel set being encoded comprising: selecting a first search window of a first frame of video data, the first search window having a first window size;
• determining if any pixel set associated with the first search window meets a first correlation requirement with respect to the pixel set being encoded; determining a motion vector for the pixel set being encoded when any pixel set associated the first search window meets the first correlation requirement; and when no pixel set associated the first search window meets the first correlation requirement: selecting a second search window of the first frame of video data, the second search window having a first window size larger than the first window size; determining if any pixel set associated with the first search window meets a second correlation requirement with respect to the pixel set being encoded, the second correlation requirement being different than the first correlation requirement; determining a motion vector for the pixel set being encoded when any pixel set associated the second search window meets the second correlation requirement.
25. The method of claim 24, wherein the step of determining a motion vector for the pixel set being encoded comprises the second correlation requirement being less stringent than the first correlation requirement.
PCT/CA2004/000093 2003-01-16 2004-01-14 Video frame correlation for motion estimation WO2004063992A1 (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
AT04701876T ATE555457T1 (en) 2003-01-16 2004-01-14 VIDEO SINGLE IMAGE CORRELATION FOR MOTION ESTIMATION
JP2006500441A JP2006517069A (en) 2003-01-16 2004-01-14 Motion vector prediction method and system
EP04701876A EP1584069B1 (en) 2003-01-16 2004-01-14 Video frame correlation for motion estimation

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US10/345,847 2003-01-16
US10/345,847 US7408989B2 (en) 2003-01-16 2003-01-16 Method of video encoding using windows and system thereof

Publications (1)

Publication Number Publication Date
WO2004063992A1 true WO2004063992A1 (en) 2004-07-29

Family

ID=32712009

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CA2004/000093 WO2004063992A1 (en) 2003-01-16 2004-01-14 Video frame correlation for motion estimation

Country Status (6)

Country Link
US (1) US7408989B2 (en)
EP (1) EP1584069B1 (en)
JP (1) JP2006517069A (en)
CN (1) CN100359535C (en)
AT (1) ATE555457T1 (en)
WO (1) WO2004063992A1 (en)

Families Citing this family (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN100370808C (en) * 2005-06-13 2008-02-20 北京中星微电子有限公司 Sports detecting method
US8296662B2 (en) * 2007-02-05 2012-10-23 Brother Kogyo Kabushiki Kaisha Image display device
WO2008137432A2 (en) * 2007-05-01 2008-11-13 Dyyno Sharing of information and formatting information for transmission over a communication network
WO2009026803A1 (en) 2007-08-27 2009-03-05 Yuvad Technologies Co., Ltd. A method for identifying motion video content
US8611701B2 (en) 2008-05-21 2013-12-17 Yuvad Technologies Co., Ltd. System for facilitating the search of video content
US8370382B2 (en) 2008-05-21 2013-02-05 Ji Zhang Method for facilitating the search of video content
US8488835B2 (en) 2008-05-21 2013-07-16 Yuvad Technologies Co., Ltd. System for extracting a fingerprint data from video/audio signals
WO2009140824A1 (en) 2008-05-22 2009-11-26 Yuvad Technologies Co., Ltd. A system for identifying motion video/audio content
US8548192B2 (en) 2008-05-22 2013-10-01 Yuvad Technologies Co., Ltd. Method for extracting a fingerprint data from video/audio signals
WO2009140823A1 (en) * 2008-05-22 2009-11-26 Yuvad Technologies Co., Ltd. A method for identifying motion video/audio content
KR101559754B1 (en) * 2008-11-04 2015-10-13 삼성전자주식회사 Noise determining device and method thereof
CN101996398B (en) * 2009-08-12 2012-07-04 睿励科学仪器（上海）有限公司 Image matching method and equipment for wafer alignment
CN102378002B (en) * 2010-08-25 2016-05-04 无锡中感微电子股份有限公司 Dynamically adjust method and device, block matching method and the device of search window
TW201409994A (en) 2012-08-30 2014-03-01 Novatek Microelectronics Corp Encoding method and encoding device for 3D video
WO2014204830A2 (en) * 2013-06-17 2014-12-24 Immedia Semiconductor, Inc. Image processing and motion compensation

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0896300A2 (en) * 1997-08-07 1999-02-10 Matsushita Electric Industrial Co., Ltd. Device and method for motion vector detection

Family Cites Families (79)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4866395A (en) 1988-11-14 1989-09-12 Gte Government Systems Corporation Universal carrier recovery and data detection for digital communication systems
US5115812A (en) 1988-11-30 1992-05-26 Hitachi, Ltd. Magnetic resonance imaging method for moving object
GB2231227B (en) 1989-04-27 1993-09-29 Sony Corp Motion dependent video signal processing
GB2248361B (en) * 1990-09-28 1994-06-01 Sony Broadcast & Communication Motion dependent video signal processing
US5093847A (en) 1990-12-21 1992-03-03 Silicon Systems, Inc. Adaptive phase lock loop
US5696531A (en) 1991-02-05 1997-12-09 Minolta Camera Kabushiki Kaisha Image display apparatus capable of combining image displayed with high resolution and image displayed with low resolution
SE469866B (en) * 1991-04-12 1993-09-27 Dv Sweden Ab Method for estimating motion content in video signals
FR2680619B1 (en) 1991-08-21 1993-12-24 Sgs Thomson Microelectronics Sa IMAGE PREDICTOR.
US5253056A (en) 1992-07-02 1993-10-12 At&T Bell Laboratories Spatial/frequency hybrid video coding facilitating the derivatives of variable-resolution images
US5614952A (en) 1994-10-11 1997-03-25 Hitachi America, Ltd. Digital video decoder for decoding digital high definition and/or digital standard definition television signals
JP3332443B2 (en) 1993-01-18 2002-10-07 キヤノン株式会社 Information processing apparatus and information processing method
JP3486427B2 (en) 1993-01-18 2004-01-13 キヤノン株式会社 Control device and control method
PT2334083E (en) 1993-03-24 2013-09-30 Sony Corp Method of coding and decoding motion vector and apparatus thereof, and method of coding and decoding picture signal and apparatus thereof
KR970009302B1 (en) 1993-08-17 1997-06-10 Lg Electronics Inc Block effect reducing apparatus for hdtv
JPH07107484A (en) * 1993-10-01 1995-04-21 Mitsubishi Electric Corp Picture transmitter
US5764698A (en) 1993-12-30 1998-06-09 International Business Machines Corporation Method and apparatus for efficient compression of high quality digital audio
JPH07210670A (en) 1994-01-21 1995-08-11 Fuji Xerox Co Ltd Image processor
US5732391A (en) 1994-03-09 1998-03-24 Motorola, Inc. Method and apparatus of reducing processing steps in an audio compression system using psychoacoustic parameters
US5940130A (en) 1994-04-21 1999-08-17 British Telecommunications Public Limited Company Video transcoder with by-pass transfer of extracted motion compensation data
DE4416967A1 (en) 1994-05-13 1995-11-16 Thomson Brandt Gmbh Method and device for transcoding bit streams with video data
EP0687112B1 (en) 1994-06-08 2006-09-20 Matsushita Electric Industrial Co., Ltd. Image conversion apparatus
KR0148154B1 (en) * 1994-06-15 1998-09-15 김광호 Coding method and apparatus with motion dimensions
US5602589A (en) 1994-08-19 1997-02-11 Xerox Corporation Video image compression using weighted wavelet hierarchical vector quantization
KR0178231B1 (en) * 1995-08-10 1999-05-01 배순훈 Method and apparatus for detecting motion vectors based on hierarchical motion estimation
US5644361A (en) 1994-11-30 1997-07-01 National Semiconductor Corporation Subsampled frame storage technique for reduced memory size
US5652749A (en) 1995-02-03 1997-07-29 International Business Machines Corporation Apparatus and method for segmentation and time synchronization of the transmission of a multiple program multimedia data stream
JPH08275160A (en) 1995-03-27 1996-10-18 Internatl Business Mach Corp <Ibm> Discrete cosine conversion method
US5559889A (en) 1995-03-31 1996-09-24 International Business Machines Corporation System and methods for data encryption using public key cryptography
EP0739138A3 (en) 1995-04-19 1997-11-05 AT&T IPM Corp. Method and apparatus for matching compressed video signals to a communications channel
JP3994445B2 (en) * 1995-12-05 2007-10-17 ソニー株式会社 Motion vector detection apparatus and motion vector detection method
DE19615490A1 (en) * 1996-04-19 1997-10-23 Philips Patentverwaltung Method for fractal image coding and arrangement for carrying out the method
GB9608271D0 (en) 1996-04-22 1996-06-26 Electrocraft Lab Video compession
JP3423835B2 (en) 1996-05-01 2003-07-07 沖電気工業株式会社 Compression encoding device with scramble and decompression reproduction device thereof
US6141693A (en) 1996-06-03 2000-10-31 Webtv Networks, Inc. Method and apparatus for extracting digital data from a video stream and using the digital data to configure the video stream for display on a television set
US6222886B1 (en) 1996-06-24 2001-04-24 Kabushiki Kaisha Toshiba Compression based reduced memory video decoder
US6215821B1 (en) 1996-08-07 2001-04-10 Lucent Technologies, Inc. Communication system using an intersource coding technique
US5850443A (en) 1996-08-15 1998-12-15 Entrust Technologies, Ltd. Key management system for mixed-trust environments
FR2752655B1 (en) 1996-08-20 1998-09-18 France Telecom METHOD AND EQUIPMENT FOR ALLOCATING A COMPLEMENTARY CONDITIONAL ACCESS TO A TELEVISION PROGRAM ALREADY WITH CONDITIONAL ACCESS
US6366614B1 (en) 1996-10-11 2002-04-02 Qualcomm Inc. Adaptive rate control for digital video compression
SE515535C2 (en) 1996-10-25 2001-08-27 Ericsson Telefon Ab L M A transcoder
US6480541B1 (en) 1996-11-27 2002-11-12 Realnetworks, Inc. Method and apparatus for providing scalable pre-compressed digital video with reduced quantization based artifacts
US6005624A (en) 1996-12-20 1999-12-21 Lsi Logic Corporation System and method for performing motion compensation using a skewed tile storage format for improved efficiency
JP3328532B2 (en) 1997-01-22 2002-09-24 シャープ株式会社 Digital data encoding method
US6182203B1 (en) 1997-01-24 2001-01-30 Texas Instruments Incorporated Microprocessor
EP0901285A4 (en) 1997-02-26 2002-05-29 Mitsubishi Electric Corp Device, system, and method for distributing video data
KR100234264B1 (en) * 1997-04-15 1999-12-15 윤종용 Block matching method using moving target window
US6014694A (en) 1997-06-26 2000-01-11 Citrix Systems, Inc. System for adaptive video/audio transport over a network
US6144402A (en) 1997-07-08 2000-11-07 Microtune, Inc. Internet transaction acceleration
CN1236527A (en) 1997-07-22 1999-11-24 皇家菲利浦电子有限公司 Method for switching between video sequences and corresponding device
JP4016227B2 (en) * 1998-01-07 2007-12-05 ソニー株式会社 Image processing apparatus and method, and recording medium
US6310919B1 (en) 1998-05-07 2001-10-30 Sarnoff Corporation Method and apparatus for adaptively scaling motion vector information in an information stream decoder
US6385248B1 (en) 1998-05-12 2002-05-07 Hitachi America Ltd. Methods and apparatus for processing luminance and chrominance image data
KR100548891B1 (en) 1998-06-15 2006-02-02 마츠시타 덴끼 산교 가부시키가이샤 Audio coding apparatus and method
US6584509B2 (en) 1998-06-23 2003-06-24 Intel Corporation Recognizing audio and video streams over PPP links in the absence of an announcement protocol
IL141104A0 (en) 1998-07-27 2002-02-10 Webtv Networks Inc Remote computer access
US6167084A (en) 1998-08-27 2000-12-26 Motorola, Inc. Dynamic bit allocation for statistical multiplexing of compressed and uncompressed digital video signals
US6219358B1 (en) 1998-09-11 2001-04-17 Scientific-Atlanta, Inc. Adaptive rate control for insertion of data into arbitrary bit rate data streams
US6259741B1 (en) 1999-02-18 2001-07-10 General Instrument Corporation Method of architecture for converting MPEG-2 4:2:2-profile bitstreams into main-profile bitstreams
US6625211B1 (en) 1999-02-25 2003-09-23 Matsushita Electric Industrial Co., Ltd. Method and apparatus for transforming moving picture coding system
JP3677165B2 (en) * 1999-04-23 2005-07-27 株式会社東芝 Moving picture coding apparatus and moving picture coding method
US6263022B1 (en) 1999-07-06 2001-07-17 Philips Electronics North America Corp. System and method for fine granular scalable video with selective quality enhancement
DE19946267C2 (en) 1999-09-27 2002-09-26 Harman Becker Automotive Sys Digital transcoding system
FR2800222B1 (en) 1999-10-26 2001-11-23 Mitsubishi Electric Inf Tech METHOD FOR COMPLIANCE WITH A TRAFFIC CONTRACT OF A PACKET STREAM OF A PACKET TRANSPORT NETWORK WITH VARIABLE LENGTH
US6714202B2 (en) 1999-12-02 2004-03-30 Canon Kabushiki Kaisha Method for encoding animation in an image file
US6300973B1 (en) 2000-01-13 2001-10-09 Meir Feder Method and system for multimedia communication control
US6647061B1 (en) 2000-06-09 2003-11-11 General Instrument Corporation Video size conversion and transcoding from MPEG-2 to MPEG-4
US6438168B2 (en) 2000-06-27 2002-08-20 Bamboo Media Casting, Inc. Bandwidth scaling of a compressed video stream
JP2002051340A (en) * 2000-08-01 2002-02-15 Matsushita Electric Ind Co Ltd Moving picture compression apparatus and method of the same
FR2813742A1 (en) 2000-09-05 2002-03-08 Koninkl Philips Electronics Nv BINARY FLOW CONVERSION METHOD
JP2002135784A (en) * 2000-10-19 2002-05-10 Matsushita Electric Ind Co Ltd Video coding method
US6748020B1 (en) 2000-10-25 2004-06-08 General Instrument Corporation Transcoder-multiplexer (transmux) software architecture
US6608792B2 (en) 2000-11-09 2003-08-19 Texas Instruments Incorporated Method and apparatus for storing data in an integrated circuit
JP4517495B2 (en) 2000-11-10 2010-08-04 ソニー株式会社 Image information conversion apparatus, image information conversion method, encoding apparatus, and encoding method
KR100433516B1 (en) 2000-12-08 2004-05-31 삼성전자주식회사 Transcoding method
CN1156168C (en) * 2001-01-12 2004-06-30 北京航空航天大学 Quick video motion estimating method
US6549561B2 (en) 2001-02-21 2003-04-15 Magis Networks, Inc. OFDM pilot tone tracking for wireless LAN
US8107524B2 (en) 2001-03-30 2012-01-31 Vixs Systems, Inc. Adaptive bandwidth footprint matching for multiple compressed video streams in a fixed bandwidth network
US6993647B2 (en) 2001-08-10 2006-01-31 Hewlett-Packard Development Company, L.P. Method and apparatus for booting an electronic device using a plurality of agent records and agent codes
US7403564B2 (en) 2001-11-21 2008-07-22 Vixs Systems, Inc. System and method for multiple channel video transcoding

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0896300A2 (en) * 1997-08-07 1999-02-10 Matsushita Electric Industrial Co., Ltd. Device and method for motion vector detection

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
LIANG-WEI LEE: "DYNAMIC SEARCH-WINDOW ADJUSTMENT AND INTERLACED SEARCH FOR BLOCK- MATCHING ALGORITHM", IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, IEEE INC. NEW YORK, US, vol. 3, no. 1, 1 February 1993 (1993-02-01), pages 85 - 87, XP000334581, ISSN: 1051-8215 *
OH H-S ET AL.: "Real-Time Imaging", vol. 6, October 2000, ACADEMIC PRESS LIMITED, article "Block matching algorithm based on an adaptive reduction of the search area for motion estimation", pages: 407 - 414
OH H-S ET AL: "Block-matching algorithm based on an adaptive reduction of the search area for motion estimation", REAL-TIME IMAGING, ACADEMIC PRESS LIMITED, GB, vol. 6, no. 5, October 2000 (2000-10-01), pages 407 - 414, XP004419498, ISSN: 1077-2014 *

Also Published As

Publication number Publication date
ATE555457T1 (en) 2012-05-15
EP1584069A1 (en) 2005-10-12
US7408989B2 (en) 2008-08-05
EP1584069B1 (en) 2012-04-25
CN100359535C (en) 2008-01-02
JP2006517069A (en) 2006-07-13
CN1739121A (en) 2006-02-22
US20040141556A1 (en) 2004-07-22

Similar Documents

Publication Publication Date Title
US7408989B2 (en) Method of video encoding using windows and system thereof
US20040001544A1 (en) Motion estimation/compensation for screen capture video
US8660191B2 (en) Software video decoder display buffer underflow prediction and recovery
JP2001285712A (en) Dissolve/fade detection method in mpeg compression video environment
US7068720B2 (en) Coding of digital video with high motion content
Zhang et al. Low-complexity and high-efficiency background modeling for surveillance video coding
JP2007336590A (en) Method for performing motion estimation in video encoding, video encoding system, and video encoding device
JP6985915B2 (en) Decoding error detection and estimation device, video decoding device, and their programs
US8208556B2 (en) Video coding using spatio-temporal texture synthesis
US20040141555A1 (en) Method of motion vector prediction and system thereof
JP2001251627A (en) Coder, coding method and recording medium recorded with program
US7397856B2 (en) Method and apparatus for storing and processing picture information of temporally successive pictures
TW201403489A (en) Motion compensation image processing apparatus and image processing method
US10999582B1 (en) Semantically segmented video image compression
CN113810692A (en) Method for framing changes and movements, image processing apparatus and program product
CN105847822A (en) Video decoding method and device
JP2008508787A (en) Error concealment technology for inter-coded sequences
KR20190067577A (en) Apparatus and method for encoding and decoding of data
JP2005516501A (en) Video image encoding in PB frame mode
JP2006014183A (en) Image encoding device and method, and program therefor
US20230209064A1 (en) Identifying long term reference frame using scene detection and perceptual hashing
JP2008072608A (en) Apparatus and method for encoding image
TWI324482B (en) Algorithm of video coding
Xuemei et al. Video residual reconstruction with sparsity estimation
CN112788340A (en) Method and apparatus for adaptively determining frame number of picture group for encoding

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BW BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE EG ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NA NI NO NZ OM PG PH PL PT RO RU SC SD SE SG SK SL SY TJ TM TN TR TT TZ UA UG US UZ VC VN YU ZA ZM ZW

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): BW GH GM KE LS MW MZ SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LU MC NL PT RO SE SI SK TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
WWE Wipo information: entry into national phase

Ref document number: 2004701876

Country of ref document: EP

Ref document number: 01330/KOLNP/2005

Country of ref document: IN

Ref document number: 1330/KOLNP/2005

Country of ref document: IN

WWE Wipo information: entry into national phase

Ref document number: 2006500441

Country of ref document: JP

Ref document number: 20048022611

Country of ref document: CN

WWP Wipo information: published in national office

Ref document number: 2004701876

Country of ref document: EP