US20070147517A1

US20070147517A1 - Video processing system capable of error resilience and video processing method for same

Info

Publication number: US20070147517A1
Application number: US11/464,845
Authority: US
Inventors: Shih-Chang Hu
Original assignee: MediaTek Inc
Current assignee: MediaTek Inc
Priority date: 2005-12-27
Filing date: 2006-08-16
Publication date: 2007-06-28
Also published as: TW200729965A; CN1992897A; US20070150650A1; TWI342162B; US7738325B2; CN100539697C

Abstract

A video processing method for AV synchronization resilience is provided. A first frame is decoded and displayed while a second frame is decoded for a predetermined period of time. It is determined whether the decoding time for the second frame exceeds a defined duration. If so, the first frame is continuously displayed during the decoding time for the second frame. The second frame is displayed while a third frame is decoded for a predetermined period of time.

Description

CROSS REFERENCE TO RELATED APPILCATIONS

This application claims the full benefit and priority of provisional U.S. patent application Ser. No. 60/754,142, filed Dec. 27, 2005, entitled “Video Processing System Capable of Error Resilience and Video Processing Method for Same”, and incorporates the entire contents of the application herein.

BACKGROUND OF THE INVENTION

1. Field of the Invention
The present invention relates to video processing, and in particular to a video processing system capable of error resilience and a video processing method for same.
2. Description of the Related Art
MPEG-4 is an ISO/IEC standard developed by the MPEG (Moving Picture Experts Group), the committee that also developed the Emmy Award winning standards MPEG-1 and MPEG-2. MPEG-4 is the successor to MPEG-1 and MPEG-2 video standards. The ISO standards committee implemented this standard in 1998. Instead of the current frame-based video technology, MPEG-4, has adopted the object-oriented concept. The object-oriented concept integrates existing multimedia technologies, such as 2D and 3D graphics, animation, video codecs, multimedia streaming, interactivity, and programmatic environments into a single architecture.
Video processing based on MPEG-4 format images is described in the following.
A video stream is a sequence of video frames. Each frame is a still image. A video player displays individual frames sequentially, typically at a rate close to 30 frames per second. Frames are divided into 16×16 pixel macroblocks (not shown). Each macroblock consists of four 8×8 luminance blocks and two 8×8 chrominance blocks (1 U and 1 V) (not shown). Macroblocks are units for measuring motion-compensated compression. Blocks are used for DCT compression.
Video data complying with MPEG-4 format files is composed of three different types of frames, comprising intra-frames (I-frames), forward predicted frames (P-frames), and bidirectional predicted frames (B-frames). An I-frame is encoded as a single image, with no reference to any past or future frames, referring to the fact that the various lossless and lossy compression techniques are performed on information contained only within a current frame, and not relative to any other frame in the video sequence. In other words, no temporal processing is performed outside the current picture or frame. A P-frame is encoded relative to the past reference frame. A reference frame is a P- or an I-frame. The past reference frame is the closest preceding reference frame. Each macroblock in a P-frame can be encoded either as an Intra-macroblock or as an Inter-macroblock. In an I-frame, encoded with no reference, intra-macroblocks are encoded in the same manner as macroblocks. A B-frame is encoded relative to the past reference frame, the future reference frame, or both frames. The future reference frame is the closest following reference frame (I or P). The encoding for B-frames is similar to P-frames, except that motion vectors may refer to areas in the future reference frames. For macroblocks that use both past and future reference frames, the two 16×16 areas are averaged.
As previously described, a video stream is a sequence of video frames. An input encoded sequence of video frames, for example, is represented as “I(0) P(3) B(1) B(2) P(6) B(4) B(5) I(9) B(7) B(8) P(12) B(11) P(12)” (the bracketed number is the display order of each frame). The output decoded sequence of the video frames is then represented as “I(0) B(1) B(2) P(3) B(4) B(5) P(6) B(7) B(8) I(9) B(10) B(11) P(12)”.
Video data can be further composed of combined frames, each comprising both a P-frame and a B-frame or an I-frame and a B-frame, and empty frames (Pe). Referring to FIG. 1, which shows a schematic diagram of a combined frame 100 (regarded as one frame to be introduced in the video data), comprising a P-frame and a B-frame, encoding processes of each frame are identical to independent ones. Pe frames indicate predicted frames not comprising video data.
The foregoing descriptions explain the types of frames of video data and input and output sequences in a video stream. Video data translated into playable video files must be encoded as MPEG-1/2/4, or other video formats due to the considerable file size to be played in a video system. A video system for playing video and audio (AV) files is described in the following. FIG. 2 is a schematic view of a video system. Video system 100 comprises storage medium 110, a file system 120, a file parser 130, a video decoder 140, a first Post-processing unit 145, an audio decoder 150, a second Post-processing unit 155, an AV synchronization unit 160, and a display system 170. Display system 170 is comprises a multiple display function. AV files are first stored in storage medium 110 and file system 120 accesses AV files therefrom. Next, file parser 130 parses the AV files to retrieve video and audio signals and pass the signals to video decoder 140 and audio decoder 150 respectively. Video decoder 140 and audio decoder 150 decode the video and audio signals and Post-processing units 145 and 155 implement post-processing operations, such as deblocking or deringing, to adjust the quality of video and audio signals respectively. The adjusted signals are transmitted to AV synchronization unit 160 for signal synchronization and are output to display system 170 to display the AV files using first display device 171 or both first display device 171 and second display device 173.
Playback quality is limited in that video files may not play smoothly in a video system comprising restricted memory resources, central processing unit (CPU) speed, or system architecture. Resource limitations are described in the following. High bit rate files (bit per second (bps)) must be accessed by file systems or storage media of adequate operating speed. Higher frame rate or a fast forward function for video decoding requires relatively better hardware performance. Post-processing, such as deblocking and deringing methods, can improve video quality but employs more system resources. Additionally, greater memory bandwidth in liquid crystal displays and TVs is required if larger frame size is requested. As described, higher frame rate, larger frame size, higher bit rate, increased post-processing, or more display elements require high-grade systems for high performance. Limitations of embedded systems included blurry video images and failure of audio signal playback.
Thus, a video processing method for a video processing system capable of error resilience is desirable.

BRIEF SUMMARY OF THE INVENTION

A video processing method for AV synchronization resilience is provided. A first frame is decoded and displayed while a second frame is decoded for a predetermined period of time. It is determined whether the decoding time for the second frame exceeds a defined duration. If so, the first frame is continuously displayed during the decoding time for the second frame, and the second frame is displayed while a third frame is decoded for a predetermined period of time.
A video processing method for dynamic frame dropping is provided. A first frame is decoded and displayed while a second frame is decoded. It is determined whether the display time for the first frame exceeds a defined duration. If the display time for the first frame exceeds the defined duration, the second frame is dropped, and the first frame is continuously displayed while a third frame is decoded for a predetermined period of time.
A video processing method for dynamic frame dropping is provided. A plurality of frames is provided. It is determined whether a portion of the frames is selectively ignored, and, if so, the frames are displayed at a predetermined number of frame intervals.
A video processing method for auto pause/resume, applied to a file system/storage medium, is provided. A first frame is decoded and displayed for a predetermined period of time while a second frame is decoded. It is determined whether the access speed of the file system/storage medium is not adequate enough is adequate for processing the second frame. If the access speed of the file system/storage medium is not adequate enough to process the second frame, an auto pause function is enabled for a predetermined period of time and audio signals are disabled while the second frame is ignored and a third frame is decoded. The auto pause function is disabled, and an auto resume function and the audio signals are enabled.
A video processing method capable of reducing system resources utilized for fast forward is provided. It is determined whether AV files are played with N multiples. If the AV files are played with N multiples, audio signals of the AV files are disabled. Video signals of the AV files are played with the N multiples. Post-processing is disabled and negligible frames of the AV files are not displayed by a display system.
A video processing method for frame skipping for fast forward is provided. Post-processing for frames of AV files is disabled. A first frame is decoded and displayed while a second frame is decoded for a predetermined period of time. It is determined whether decoding time for the second frame exceeds a defined duration. If the decoding time for the second frame exceeds the defined duration, the first frame is continuously displayed during the decoding time for the second frame, and the second frame is not displayed while a third frame is decoded for a predetermined period of time.
A detailed description is given in the following embodiments with reference to the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention can be more fully understood by reading the subsequent detailed description and examples with references made to the accompanying drawings, wherein:

FIG. 1 is a schematic view of a combined frame comprising a P-frame and a B-frame;

FIG. 2 is a schematic view of a video system;

FIG. 3 is a workflow of an embodiment of a video processing method for AV synchronization resilience;

FIG. 4 is a flowchart of an embodiment of a video processing method for AV synchronization resilience;

FIG. 5 is a workflow of an embodiment of a video processing method for dynamic frame dropping for a single display device;

FIG. 6 is a workflow of an embodiment of a video processing method for dynamic frame dropping for at least one first display device and one second display device;

FIG. 7 is a flowchart of an embodiment of a video processing method for dynamic frame dropping;

FIG. 8 is a workflow of an embodiment of a video processing method for an auto pause/resume function having insufficient file system/storage media access speed;

FIG. 9 is a flowchart of an embodiment of a video processing method for an auto pause/resume function having insufficient file system/storage media access speed having insufficient file system/storage media access speed;

FIG. 10 is a workflow of an embodiment of a video processing method for an auto pause/resume function with insufficient decoding capability;

FIG. 11 is a flowchart of an embodiment of a video processing method for an auto pause/resume function having insufficient file system/storage media access speed;

FIG. 12 is a workflow of an embodiment of a video processing method for reducing system resources utilized for fast forward;

FIG. 13 is a flowchart of an embodiment of a video processing method for reducing system resources utilized for fast forward function; and

FIG. 14 is a flowchart of an embodiment of a video processing method for frame skipping based on the fast forward shown in FIG. 13.

DETAILED DESCRIPTION OF THE INVENTION

Several exemplary embodiments of the invention are described with reference to FIGS. 3 through 14, which generally relate to video processing. It is to be understood that the following disclosure provides many different embodiments as examples, for implementing different features of the invention. Specific examples of components and arrangements are described below to simplify the present disclosure. These are, of course, merely examples and are not intended to be limiting. In addition, the present disclosure may repeat reference numerals and/or letters in the various examples. This repetition is for the purpose of simplicity and clarity and does not in itself dictate a relationship between the various embodiments and/or configurations discussed.
The invention discloses a video processing system capable of error resilience and a video processing method for same. The video processing method comprises features of AV synchronization resilience, adaptive Post-processing, dynamic frame dropping, auto pause/resume, and fast forward function. Individual examples are described in the following.

AV Synchronization Resilience

FIG. 3 is a workflow of an embodiment of a video processing method for AV synchronization resilience. AV synchronization resilience is frequently required, especially for high frame rate display. As shown in FIG. 3, Section 210 represents ideal video display, where the ideal duration for each frame equals 33.33 millisecond (ms) where video files are encoded at a rate of 30 frames per second and the decoding time for each frame is less than 33 ms. Here, the display duration for each frame is not considered.
In reality, Section 220 represents asynchronization solution for audio and video signals. As shown in FIG. 3, Frame 1 is decoded while Frame 0 is decoded and displayed. The decoding of Frame 1 requires 43 ms, which is longer than the frame duration of 33.33 ms. After the decoding, the first asynchronization of audio and video signals is then detected during a predetermined time period (a first asynchronization time period) shown by the first star symbol. At the first asynchronization time period, display of Frame 0 continues until Frame 1 is completely decoded while audio signals corresponding to Frame 1 are output. Next, after the decoding of Frame 1 is completed, Frame 1 is displayed and Frame 2 is decoded in the following 32 ms. The second asynchronization of audio and video signals is detected due to the excessive decoding time of Frame 2 during a second asynchronization time period, as shown by the second star symbol. During the second asynchronization time period, Frame 1 is displayed until Frame 2 is completely decoded while audio signals corresponding to Frame 2 are output.
Next, after the decoding of Frame 2 is completed, Frame 2 is displayed and Frame 3 is decoded in 20 ms. Herein, completely decoded Frame 3 can not be immediately displayed since the ideal duration for a frame has not passed, such that display of Frame 2 continues. Next, when the ideal duration for a frame expires, Frame 3 is displayed and Frame 4 is decoded in 25 ms. Similarly, completely decoded Frame 4 can not be displayed since the ideal duration for a frame has not passed, such that display of Frame 3 continues. Next, Frame 4 is displayed and Frame 5 is decoded, and the process repeats for each frame until completion. In this embodiment, AV asynchroinzation will be frequently detected and audio and video signals of AV files can be eventually synchronized due to discrepant decoding time.
FIG. 4 is a flowchart of an embodiment of a video processing method for AV synchronization resilience. Frame N+i (i=0) (for example, Frame 0) is decoded and displayed while frame N+i (i=i+1) (for example, Frame 1) is decoded for a predetermined period of time (step S11), where N=0 and i=0˜n. It is determined whether the decoding time for frame N+i (i=i+1) (for example, Frame 1) exceeds a defined duration (33.33 ms, for example) (step S12). If so, the display of frame N+i (Frame 0) continues for the duration of the decoding for frame N+i (i=i+1) (for example, Frame 1) and frame N+i (i=i+1) (for example, Frame 1) is then displayed while frame N+i (i=i+2) (for example, Frame 2) is decoded for a predetermined period of time (step S13), i=i+1 (step S15), and the process proceeds to step S11. If not, frame N+i (i=i+1) (for example, Frame 1) is then displayed while frame N+i (i=i+2) (for example, Frame 2) is decoded for a predetermined period of time (step S14), i=i+1 (step S15), and the process proceeds to step S12.

Dynamic Frame Dropping

Frames may be dropped when they can not be timely displayed. The following situations may result in frame dropping, comprising higher frame rate, multiple display devices, excessive post processes, and larger display frame size. With respect to higher frame rate, some frames are dropped to reduce frame rate. With respect to multiple display devices, insignificant frames displayed at a secondary display device can be dropped to reduce frame rate or frames are not output to a secondary display device. Since frames, for example, are output to TV and LCD, frame rate relating to frames output to the LCD can be reduced to save system resources. Additionally, frame defects may appear if Post-processing is disabled, such that frame dropping is implemented to solve the problem.
FIG. 5 is a workflow of an embodiment of a video processing method for dynamic frame dropping with single display device. As shown in FIG. 5, Section 310 represents ideal audio play along with ideal video display with ideal duration for displaying each frame equals 33.33 ms, where video files are encoded at 30 frames per second and a decoding time for each frame is less than 33.33 ms. Here, a display duration for each frame is not considered. In Section 310, ideally, each frame can be timely decoded and displayed within each time interval of 33.33 ms.
In reality, frames however may not be able to completely display within each preset time interval. Section 320 represents the method of dynamic frame dropping with single display device of the present invention. As shown in Section 320, between each time point ‘0’, ‘1’, ‘2’, ‘3’ and ‘4’, Frame 1, Frame 2, Frame 3, Frame 4 and Frame 5 are sequentially decoded within each time interval of 33.33 ms. The time points ‘A’, ‘B’, ‘C’, ‘D’ and ‘E’, on the other hand shows the actual display duration of the frames. Some of the frames may take more than 33.33 ms to display.
As shown in FIG. 5, Frame 0 is displayed at the time between the time point A and B for 60 ms, and Frame 1 is decoded between the time point ‘0’ and the time point ‘1’ for less than 33.33 ms. Herein, display of frame 0 takes longer than the ideal duration (33.33 ms). In the second time interval between the time point ‘1’ and the time point ‘2’, since Frame 0 has not finished its display, Frame 1 of the present invention will be dropped and will not be displayed. Between the time point ‘2’ and the time point ‘3’ where the display of Frame 0 is complete, Frame 2 will be sequentially displayed. That is, in this embodiment, as long as the previous frame is still being displayed after one preset time interval (for example, 33.33 ms), at least one next frame will be dropped.
FIG. 6 is a workflow of an embodiment of a video processing method for dynamic frame dropping for multiple display devices. As shown in FIG. 6, Section 410 represents ideal audio play along with ideal video display with ideal duration for displaying each frame equals 33.33 ms, where video files are encoded at 30 frames per second and a decoding time for each frame is less than 33.33 ms. Here, a display duration of each frame is not considered. Different from the embodiment shown in FIG. 5, this embodiment provides a first display device and a second display device. The first display device is used for displaying important data source whereas the second display device is used for displaying less important data source. In this embodiment, for important data source, it is restricted not to drop any frames even its previous frame takes more than one time intervals to display. But for less important data source, for efficiency, some frames may be dropped.
In this embodiment, Section 420 represents AV file display of the first display device and section 430 represents AV file display of the second display device which is allowed for dynamic frame dropping. As shown in FIG. 6, Frame 0 is displayed between time point ‘0’ and the time point ‘3’ in the second display device where Frames 1 and 2 are decoded but not displayed. After Frame 0 finishes its display, the second display device will continue to display Frame 3. The process repeats for each frame until completion.
FIG. 7 is a flowchart of an embodiment of a video processing method for dynamic frame dropping. Frame N+i (i=0) (Frame 0) is decoded and displayed for a predetermined period of time while frame N+i (i=i+1) (Frame 1) is decoded (step S21), where N=0 and i=0˜n. It is determined whether the display time for frame N+i (Frame 0) exceeds a defined duration (33.33 ms, for example) (step S22). If so, frame N+i (i=i+1) (Frame 1) is dropped and the display of frame N+i (Frame 0) continues while frame N+i (i=i+2) (Frame 2) is decoded for a predetermined period of time (step S23), i=i+2 (step S24), and the process proceeds to step S21. If not, frame N+i (i=i+1) (Frame 1) is then displayed while N+i (i=i+2) (Frame 2) is decoded for a predetermined period of time (step S25), i=i+1 (step S26), and the process proceeds to step S21.

Post-Processing

Post-processing, such as deblocking and deringing methods, can further improve video quality but it may utilize more system resources. Thus, for speed concern, the present invention may turn off the post-processing operations to save system resources. Cooperating with AV synchronization resilience, Post-processing, for example, may be disabled when video signals of a current frame are asynchronous with audio signals.

Auto/Pause Function

In some embodiment, when system resources are insufficient, an auto/pause function may be automatically implemented. If file system and CPU access speed, for example, is too slow to process high-bit-rate files or large AV asynchronization is detected, the auto pause/resume function is automatically implemented to pause file processing for a period of time and resume video and audio signal synchronization.
FIG. 8 is a workflow of an embodiment of a video processing method for an auto pause/resume function having insufficient file system/storage media access speed. As shown in FIG. 8, Section 510 represents ideal video display (30 frames per second), where each frame is completely decoded in 33.33 ms. Section 520 represents an auto pause/resume process for insufficient access speed. Here, Frame 1 is decoded while Frame 0 is decoded and displayed, and Frame 1 is displayed while Frame 2 is decoded. In this embodiment, file system/storage media access speed for processing AV files comprising high frame rate is inadequate, hence, Frames 0, 1 and 2 are decoded and displayed but the file system/storage medium cannot prepare enough bitstream data (Assume FS speed is low) to decode frame 3. Thus, an auto pause mode is enabled as shown by the start symbol, in which audio signals are stopped, and start to re-prepare the bitstream data to bistream buffer from Frame 3 through the file system/storage medium until the bistream buffer is full of many frame's bitstream data (Assume the duration is 50 ms). Then resume this video from frame 3 to be decoded/display. Thus, AV synchronization is enabled from Frame 3 (Frame 2 is display during preparing bitstream buffer. In this example, it is about 50 ms) and Frame 3 is displayed and Frame 4 is decoded, and Frame 4 is displayed and Frame 5 is decoded, and the process repeats for each frame until completion.
FIG. 9 is a flowchart of an embodiment of a video processing method for auto pause/resume having insufficient file system/storage media access speed. A first frame (Frame 1, for example) is decoded and displayed for a predetermined period of time while a second frame (Frame 2, for example) is decoded (step S31). It is determined whether the access speed of a file system/storage medium is not adequate enough (Bitstream data is not in Buffer) to process the third frame (Frame 3, for example) (step S32). If so, an auto pause function is enabled, audio is disabled (step S33), and re-prepare the bitsteram data until bitstream buffer is full, and a third frame (Frame 3, for example) is decoded (step S34), the auto pause function is disabled (step S35), and an auto resume function and audio are enabled (step S36). If not, the second frame (Frame 2, for example) is displayed and the third frame (Frame 3, for example) is decoded (step S37), and the process repeats for each frame until completion.
FIG. 10 is a workflow of an embodiment of a video processing method for auto pause/resume with insufficient decoding capability. As shown in FIG. 10, Section 610 represents ideal video display (30 frames per second), where each frame is completely decoded in 33.33 ms. Section 620 represents an auto pause/resume process for inadequate decoding capability, in which decoding and display time for each frame is not considered and an acceptable non-synchronization time is less than 30 ms. Frame 1 is decoded in 59 ms while Frame 0 is decoded and is displayed, resulting in AV asynchronization, in which, compared to the duration of Frames 0 and 1 shown in section 610, the asynchronization time (about 25.67 ms) less than 30 ms is acceptable, such that Frame 1 is then displayed and Frame 2 is decoded in 46 ms. Another AV asynchronization is detected due to the long decoding time over the duration of Frames 0 and 1 shown in Section 610, in which, compared to the duration of Frames 1 and 2 shown in Section 610, the asynchronization time (about 38.34 ms) greater than 30 ms is unacceptable, such that an auto pause mode is enabled as shown by the start symbol, in which audio signals are stopped, and the file system/storage medium decodes and stores bit stream data from Frame 3 to a bit stream buffer. The pause mode continues until bitstream buffer is full (50 ms, for example) and the pause mode is disabled and an auto resume mode is enabled. Thus, AV synchronization is enabled from Frame 3 such that Frame 2 is ignored and Frame 3 is displayed and Frame 4 is decoded, and Frame 4 is displayed and Frame 5 is decoded, and the process repeats for each frame until completion.
FIG. 11 is a flowchart of an embodiment of a video processing method for auto pause/resume having insufficient file system/storage media access speed. A first frame (Frame 0, for example) is decoded and displayed for a predetermined period of time while a second frame (Frame 1, for example) is decoded (step S41). It is determined whether AV asynchronization is detected and if an asynchronization time is greater than a threshold value (step S42). If so, an auto pause function is enabled for a predetermined period of time and audio is disabled (step S43) while the second frame is ignored and a third frame (Frame 2, for example) is decoded (step S44), the auto pause function is disabled (step S45), and an auto resume function and audio are enabled (step S46). If not, the second frame (Frame 1, for example) is displayed and the third frame (Frame 2, for example) is decoded (step S47), and the process repeats for each frame until completion.

Fast Forward Function

FIG. 12 is a workflow of an embodiment of a video processing method for reducing system resources utilized by a fast forward function. In this embodiment, audio signals are disabled while a fast forward process is implemented to prevent AV asynchronization. While fast forwarding a video file, it is assumed that users are not concerned about audio output. Therefore, audio signals are disabled to allow more video frames to be decoded and displayed. Additionally, selectively frame dropping for a display device is enabled to reduce bus load, access and post-processing functions are completely or partially disabled to accelerate processing speed. Deblocking or deranging for example, occupy more system resources and may be disabled.
As shown in FIG. 12, compared to standard processing speed, Section 710 represents quadruple processing speed (4×) for fast forward, where each frame is displayed in 16.66 ms, video files are encoded at 15 frames per second, a display duration for each frame is not considered, and a decoding time is for each frame less than 16 ms. Section 720 represents a fast forward process with Nx (N>1) quadruplicate processing speed of the invention, in which a display time for all frames equals 10 ms and Post-processing functions for all decoded frames are disabled. Frame 1 is decoded while Frame 0 is decoded and is displayed. Frame 1 is not displayed due to the decoding time of 20 ms over an ideal duration (16.66 ms) while display of Frame 0 should terminate with 16.66 ms passing through but continues for another duration time (16.66 ms) since Frame 2 is decoded in 8 ms. Next, Frame 2 is displayed in 10 ms and Frame 3 is decoded in 10 ms, and Frame 3 is normally displayed and Frame 4 is decoded in 28 ms. Frame 4 is not displayed due to the decoding time of 28 ms over an ideal duration (16.66 ms) while display of Frame 3 should terminate with 16.66 ms passing through but continues for another two time intervals (16.66*2 ms) since Frame 5 is decoded in 26 ms. Next, Frame 5 is displayed in 10 ms and Frame 6 is decoded in 10 ms, and Frame 6 is displayed and Frame 7 is decoded in 8 ms, and the process repeats for each frame until completion. Note that a recount duration frame starts from frame 5.
In this embodiment, a frame is skipped when decoding is slow. A frame is skipped when a previous frame cannot be displayed in time. If a previous frame is not displayed, at least one of the subsequent frames must be displayed. Additionally, frames can be periodically dropped. A frame, for example, is dropped at every interval of three frames.
FIG. 13 is a flowchart of an embodiment of a video processing method for reducing system resources utilized by a fast forward function. It is determined whether AV files are played with N multiples (Nx speed, N!=1) (step S51). If so, audio is disabled, video files of the AV files are played with Nx speed (step S52), Post-processing is disabled (step S53), and negligible frames are skipped (step S54) for display by a display system (step S58). If not, audio and video signals are respectively enabled (step S55) decoded and implemented by post-processing operations (step S56). The processed audio and video files are synchronized (step S57) display by a display system (step S58).
FIG. 14 is a flowchart of an embodiment of a video processing method for frame skipping for fast forward shown in FIG. 13. Post-processing for all frames is first disabled (step S61). Frame N (N+i, i=0) (Frame 0) is decoded and displayed while frame N+1 (N+i, i=1) (Frame 1) is decoded for a predetermined period of time (step S62), where N=0 and i=0˜n. It is determined whether the decoding time for frame N+1 (N+i, i=1) (Frame 1) exceeds a defined duration (33.33 ms, for example) (step S63). If so, the display of frame N (N+i, i=0) (Frame 0) continues for the duration of the decoding of frame N+1 (N+i, i=1) (Frame 1) and frame N+1 (N+i, i=1) (Frame 1) is not displayed while frame N+2 (N+i, i=2) (Frame 2) is decoded for a predetermined period of time (step S64), i=i+2 (step S65), and the process proceeds to step S62. If not, frame N+1 (N+i, i=1) (Frame 1) is then displayed while frame N+2 (N+i, i=2) (Frame 2) is decoded for a predetermined period of time (step S66), i=i+1 (step S67), and the process proceeds to step S62.
Note that the described methods shown in FIGS. 4, 7, 9, 11, 13, and 14 can also be implemented in various storage media.
A video processing method capable of error resilience of the invention is capable of achieving AV synchronization resilience, adaptive post-processing, dynamic frame dropping for display, and auto pause/resume for system resources utilized by a fast forward function.
While the invention has been described by way of example and in terms of the preferred embodiments, it is to be understood that the invention is not limited to the disclosed embodiments. To the contrary, it is intended to cover various modifications and similar arrangements (as would be apparent to those skilled in the art). Therefore, the scope of the appended claims should be accorded the broadest interpretation so as to encompass all such modifications and similar arrangements.

Claims

1. A video processing method for AV synchronization resilience, comprising:

decoding and displaying a first frame while a second frame is decoded for a predetermined period of time;

determining whether the decoding time for the second frame exceeds a defined duration;

if the decoding time for the second frame exceeds the defined duration, continuously displaying the first frame by the decoding time for the second frame; and

displaying the second frame while a third frame is decoded for a predetermined period of time.

2. The video processing method as claimed in claim 1, further comprising, if the decoding time for the second frame is equal to the defined duration, displaying the second frame while the third frame is decoded for the predetermined time when the display of the first frame is complete.

3. The video processing method as claimed in claim 2, further comprising, if the decoding time for the second frame is less than the defined duration, continuously displaying the first frame and, when an ideal duration for a frame expires, displaying the second frame while the third frame is decoded for the predetermined time.

4. A video processing method for dynamic frame dropping, comprising:

decoding and displaying a first frame while a second frame is decoded;

determining whether the display time for the first frame exceeds a defined duration;

if the display time for the first frame exceeds the defined duration, dropping the second frame; and

continuously displaying the first frame while a third frame is decoded for a predetermined period of time.

5. The video processing method as claimed in claim 4, further comprising, if the display duration for the first frame is equal to the defined duration, displaying the second frame while the third frame is decoded for the predetermined time.

6. The video processing method as claimed in claim 5, further comprising, if the display time for the first frame is less than the defined duration, continuously displaying the first frame by the defined duration.

7. A video processing method for dynamic frame dropping, comprising:

providing a plurality of frames;

determining whether a portion of the frames is selectively ignored; and

if so, displaying the frames at intervals of a predetermined number of frames.

8. A video processing method for auto pause/resume, applied to a file system/storage medium, comprising:

decoding and displaying a first frame for a predetermined period of time while a second frame is decoded;

determining whether access speed of the file system/storage medium is not adequate enough adequate enough to process the second frame;

if the access speed of the file system/storage medium is not adequate enough to process the second frame, enabling an auto pause function for a predetermined period of time and disabling audio signals while the second frame is ignored and a third frame is decoded; and

disabling the auto pause function; and

enabling an auto resume function and the audio signals.

9. The video processing method as claimed in claim 8, further comprising, if the access speed of the file system/storage medium is adequate enough to process the second frame, displaying the second frame and decoding the third frame.

10. The video processing method as claimed in claim 8, further comprising:

determining whether decoding ability of the file system/storage medium is not adequate enough to process the second frame to result in AV asynchronization;

if the decoding ability of the file system/storage medium is not adequate enough to process the second frame to result in AV asynchronization, determining whether asynchronization time is acceptable;

if the asynchronization time is unacceptable, enabling an auto pause mode and disabling the audio signals;

decoding and storing bitstream data relating to the third frame in a bitstream buffer;

disabling the pause mode and enabling an auto resume mode, thereby ignoring the second frame and displaying the third frame while a fourth frame is decoded.

11. The video processing method as claimed in claim 10, further comprising, if the asynchronization time is acceptable, displaying the second frame and decoding the third frame.

12. A video processing method for reducing system resources utilized by a fast forward function, comprising:

determining whether AV files are played with N multiples;

if the AV files are played with N multiples, disabling audio signals of the AV files;

playing video signals of the AV files with the N multiples;

disabling Post-processing; and

skipping negligible frames of the AV files to be displayed by a display system.

13. The video processing method as claimed in claim 12, further comprising:

if the AV files are not played with the N multiples, enabling and decoding the audio and video signals;

implementing Post-processing to the audio and video signals respectively;

synchronizing the processed audio and video signals to be displayed by the display system.

14. A video processing method for frame skipping for fast forward, comprising:

disabling Post-processing for frames of AV files;

determining whether decoding time for the second frame exceeds a defined duration;

not displaying the second frame while a third frame is decoded for a predetermined period of time.

15. The video processing method as claimed in claim 14, further comprising, if the decoding time for the second frame does not exceed the defined duration, displaying the second frame while the third frame is decoded for the predetermined time.

16. A computer-readable storage medium storing a computer program providing a video processing method for AV synchronization resilience, comprising using a computer to perform the steps of:

if the decoding time for the second frame exceeds the defined duration, continuously displaying the first frame by the decoding time for the second frame until second frame is decoded complete; and

17. The computer-readable storage medium as claimed in claim 16, further comprising, if the decoding time for the second frame is equal to the defined duration, displaying the second frame while the third frame is decoded for the predetermined time when the display of the first frame is complete.

18. The computer-readable storage medium as claimed in claim 17, further comprising, if the decoding time for the second frame is less than the defined duration, continuously displaying the first frame and, when an ideal duration for a frame expires, displaying the second frame while the third frame is decoded for the predetermined time.

19. A computer-readable storage medium storing a computer program providing a video processing method for dynamic frame dropping, comprising using a computer to perform the steps of:

decoding and displaying a first frame while a second frame is decoded;

20. The computer-readable storage medium as claimed in claim 19, further comprising, if the display duration for the first frame is equal to the defined duration, displaying the second frame while the third frame is decoded for the predetermined time.

21. The computer-readable storage medium as claimed in claim 20, further comprising, if the display time for the first frame is less than the defined duration, continuously displaying the first frame by the defined duration.

22. A computer-readable storage medium storing a computer program providing a video processing method for dynamic frame dropping, comprising using a computer to perform the steps of:

providing a plurality of frames;

determining whether a portion of the frames is selectively ignored; and

if so, displaying the frames at a predetermined number of frame intervals.

23. A computer-readable storage medium storing a computer program providing a video processing method for auto pause/resume, comprising using a computer to perform the steps of:

determining whether access speed of a file system/storage medium is not adequate enough to process the second frame;

disabling the auto pause function; and

enabling an auto resume function and the audio signals.

24. The computer-readable storage medium as claimed in claim 23, further comprising, if the access speed of the file system/storage medium is adequate enough to process the second frame, displaying the second frame and decoding the third frame.

25. The computer-readable storage medium as claimed in claim 23, further comprising:

26. The computer-readable storage medium as claimed in claim 25, further comprising, if the asynchronization time is acceptable, displaying the second frame and decoding the third frame.

27. A computer-readable storage medium storing a computer program providing a video processing method for reducing system resources utilized by a fast forward function, comprising using a computer to perform the steps of:

determining whether AV files are played with N multiples;

playing video signals of the AV files with the N multiples;

disabling Post-processing; and

skipping negligible frames of the AV files to be displayed by a display system.

28. The computer-readable storage medium as claimed in claim 27, further comprising:

implementing Post-processing to the audio and video signals respectively;

29. A computer-readable storage medium storing a computer program providing a video processing method for frame skipping for fast forward, comprising using a computer to perform the steps of:

disabling Post-processing for frames of AV files;

30. The computer-readable storage medium as claimed in claim 29, further comprising, if the decoding time for the second frame does not exceed the defined duration, displaying the second frame while the third frame is decoded for the predetermined time.