US20120275502A1

US20120275502A1 - Apparatus for dynamically adjusting video decoding complexity, and associated method

Info

Publication number: US20120275502A1
Application number: US13/094,787
Authority: US
Inventors: Fang-Yi Hsieh; Jian-Liang Lin
Original assignee: MediaTek Inc
Current assignee: Xueshan Technologies Inc
Priority date: 2011-04-26
Filing date: 2011-04-26
Publication date: 2012-11-01
Also published as: CN102761739A; US20170006307A1; TW201244485A; US9930361B2; TWI549483B; CN102761739B

Abstract

An apparatus for dynamically adjusting video decoding complexity includes a decoding resolution control circuit and an adaptive spatial resolution decoder. The decoding resolution control circuit is arranged to dynamically determine whether at least one portion of a plurality of frames should be decoded in accordance with a specific resolution differing from an original resolution of the plurality of frames. In addition, the adaptive spatial resolution decoder is arranged to decode the plurality of frames according to whether the at least one portion of the plurality of frames should be decoded in accordance with the specific resolution. In particular, the apparatus further includes a system capability analyzing circuit arranged to analyze system capability of at least a portion of the apparatus, in order to generate analyzing results for being sent to the decoding resolution control circuit. An associated method is also provided.

Description

BACKGROUND

The present invention relates to video decoding, and more particularly, to an apparatus for dynamically adjusting video decoding complexity, and to an associated method.
When an end user is viewing a video program utilizing a home theater system at home, encoded data of the video program is decoded for being displayed on a display panel of the home theater system, and the decoded results of the encoded data can be scaled up or scaled down to meet the size and resolution of the display panel. Typically, a decoding device of the home theater system can decode the encoded data with ease, no matter how complicated the algorithm for decoding the encoded data is, and no matter whether the resolution of the video program is high or not.
It would be very convenient for the end user to own a portable electronic device (e.g., a mobile phone or a personal digital assistant (PDA)) that can be utilized for viewing the same video program anywhere. However, when a research and development (R&D) team of a manufacturer is designing such a portable electronic device, some problems may arise. For example, the decoding capability of the portable electronic device may be insufficient in a situation where the algorithm for decoding the encoded data is too complicated and/or the resolution of the video program is high. In another example, the power consumption may be too high when one or more processing circuits within the portable electronic device operate at the highest operation frequency available. Thus, there is a need for highly efficient and cost effective video decoders, in order to implement the portable electronic device mentioned above.

SUMMARY

It is therefore an objective of the claimed invention to provide an apparatus for dynamically adjusting video decoding complexity, and to provide an associated method, in order to solve the above-mentioned problems.
It is another objective of the claimed invention to provide an apparatus for dynamically adjusting video decoding complexity, and to provide an associated method, in order to reduce the calculation load and the power consumption within a portable electronic device such as that mentioned above.
It is another objective of the claimed invention to provide an apparatus for dynamically adjusting video decoding complexity, and to provide an associated method, in order to reduce complexity of decoding operations by dynamically adjusting decoding complexity of each of a plurality of frames. More particularly, the apparatus can selectively make at least a portion of components/modules/units therein operate in accordance with a resolution differing from that of original frames, in order to dynamically adjust decoding complexity frame by frame.
An exemplary embodiment of an apparatus for dynamically adjusting video decoding complexity comprises a decoding resolution control circuit and an adaptive spatial resolution decoder. The decoding resolution control circuit is arranged to dynamically determine whether at least one portion of a plurality of frames should be decoded in accordance with a specific resolution differing from an original resolution of the plurality of frames. In addition, the adaptive spatial resolution decoder is arranged to decode the plurality of frames according to whether the at least one portion of the plurality of frames should be decoded in accordance with the specific resolution. For example, the adaptive spatial resolution decoder decodes at least one frame of the plurality of frames by controlling the adaptive spatial resolution decoder to operate in accordance with the specific resolution.
An exemplary embodiment of a method for dynamically adjusting video decoding complexity comprises: dynamically determining whether at least one portion of a plurality of frames should be decoded in accordance with a specific resolution differing from an original resolution of the plurality of frames; and decoding the plurality of frames according to whether the at least one portion of the plurality of frames should be decoded in accordance with the specific resolution. For example, the step of decoding the plurality of frames further comprises decoding at least one frame of the plurality of frames by controlling an adaptive spatial resolution decoder to operate in accordance with the specific resolution.
These and other objectives of the present invention will no doubt become obvious to those of ordinary skill in the art after reading the following detailed description of the preferred embodiment that is illustrated in the various figures and drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A illustrates an apparatus for dynamically adjusting video decoding complexity according to a first embodiment of the present invention.

FIG. 1B illustrates some implementation details of the adaptive spatial resolution decoder shown in FIG. 1A according to an embodiment of the present invention.

FIG. 2 illustrates a flowchart of a method for dynamically adjusting video decoding complexity according to an embodiment of the present invention.

FIGS. 3A-3H respectively illustrate some video decoding schemes involved with the method shown in FIG. 2 according to different embodiments of the present invention.

DETAILED DESCRIPTION

Certain terms are used throughout the following description and claims, which refer to particular components. As one skilled in the art will appreciate, electronic equipment manufacturers may refer to a component by different names. This document does not intend to distinguish between components that differ in name but not in function. In the following description and in the claims, the terms “include” and “comprise” are used in an open-ended fashion, and thus should be interpreted to mean “include, but not limited to . . . ”. Also, the term “couple” is intended to mean either an indirect or direct electrical connection. Accordingly, if one device is coupled to another device, that connection may be through a direct electrical connection, or through an indirect electrical connection via other devices and connections.
Please refer to FIG. 1A, which illustrates an apparatus 100 for dynamically adjusting video decoding complexity according to a first embodiment of the present invention. The apparatus 100 comprises a decoding resolution control circuit 102, an adaptive spatial resolution decoder 104, and a system capability analyzing circuit 106, where the numeral 108 is utilized for representing an input bit stream. According to some embodiments, such as the first embodiment or some variations thereof, the apparatus 100 may represent at least a portion (e.g. a portion or all) of a portable electronic device (e.g. a mobile phone or a personal digital assistant (PDA)). For example, the apparatus 100 may represent the whole of the portable electronic device. In another example, the apparatus 100 may represent a processing circuit of the portable electronic device.
According to the first embodiment, the decoding resolution control circuit 102 is arranged to dynamically determine whether at least one portion of a plurality of frames (and more particularly, at least one frame within the frames carried by the input bit stream 108) should be decoded in accordance with a specific resolution differing from an original resolution of the plurality of frames, in order to dynamically adjust decoding complexity frame by frame. In addition, the adaptive spatial resolution decoder 104 is arranged to decode the plurality of frames (more particularly, the frames carried by the input bit stream 108, which is input into the adaptive spatial resolution decoder 104) according to whether the aforementioned at least one portion of the plurality of frames should be decoded in accordance with the specific resolution. In a situation where the decoding resolution control circuit 102 determines that a frame under consideration should be decoded in accordance with the specific resolution, the adaptive spatial resolution decoder 104 decodes the frame under consideration in accordance with the specific resolution. On the contrary, in a situation where the decoding resolution control circuit 102 determines that a frame under consideration should not be decoded in accordance with the specific resolution, the adaptive spatial resolution decoder 104 decodes the frame under consideration in accordance with the original resolution. Additionally, the system capability analyzing circuit 106 is arranged to analyze system capability of at least a portion of the apparatus 100 (e.g. the adaptive spatial resolution decoder 104 and/or other components/modules within the apparatus 100), in order to generate analyzing results for being sent to the decoding resolution control circuit 102, where the analyzing results are utilized for performing decoding resolution control.
Please note that, in this embodiment, the original resolution mentioned above corresponds to (W*H) pixels per frame, and the specific resolution mentioned above corresponds to (M*N) pixels per frame, where the specific resolution is typically lower than the original resolution. This is for illustrative purposes only, and is not meant to be a limitation of the present invention. According to some variations of this embodiment, the specific resolution can be higher than or equivalent to the original resolution.
FIG. 1B illustrates some implementation details of the adaptive spatial resolution decoder 104 shown in FIG. 1A according to an embodiment of the present invention. As shown in FIG. 1B, the adaptive spatial resolution decoder 104 comprises a variable dimension/resolution spatial prediction module 110 (labeled “Variable Dim Spatial Prediction”), a variable dimension temporal prediction module 120 (labeled “Variable Dim Temporal Prediction”), a variable length decoding (VLD) and inverse quantization (IQ) module 132 (labeled “VLD+IQ”), a variable dimension/resolution inverse transform (IT) unit 134 (labeled “Variable Dim IT”), an arithmetic unit such as an adder 136 (labeled “+”), a frame storage 140 (labeled “Frame Store”) such as a frame buffer, a variable dimension/resolution reconstructed frame output unit 150 (labeled “Variable Dim Reconstructed Frame Out”) comprising a de-blocking filter such as an in-loop filter 152, and a reference frame extraction unit 160 (labeled “Reference Frame”). In addition, the variable dimension/resolution spatial prediction module 110 comprises an intra prediction initial processing unit 112 (labeled “Intra Prediction Init”) and an intra prediction unit 114 (labeled “Intra Prediction”), and the variable dimension temporal prediction module 120 comprises a motion vector prediction unit 122 (labeled “Motion Vector Prediction”) and a motion compensation unit 124 (labeled “Motion Compensation”). This is for illustrative purposes only, and is not meant to be a limitation of the present invention. According to some variations of this embodiment, the de-blocking filter can be implemented outside the variable dimension/resolution reconstructed frame output unit 150, and more particularly, can be an out-loop filter.
In the embodiment shown in FIG. 1B, the variable dimension/resolution spatial prediction module 110 is arranged to perform spatial prediction according to the input bit stream 108 and/or according to frame data temporarily stored in the frame storage 140, where the intra prediction initial processing unit 112 performs intra prediction initial processing, and the intra prediction unit 114 performs intra prediction operations. The variable dimension temporal prediction module 120 is arranged to perform temporal prediction according to the input bit stream 108 and according to one or more reference frames extracted by the reference frame extraction unit 160, where the motion vector prediction unit 122 performs motion vector prediction, and the motion compensation unit 124 performs motion compensation. In addition, the VLD and IQ module 132 is arranged to perform VLD and IQ operations on the input bit stream 108 to generate inverse quantization results, and the variable dimension/resolution IT unit 134 is arranged to perform IT operations on the inverse quantization results to generate inverse transform results. As shown in FIG. 1B, the aforementioned arithmetic unit such as the adder 136 (labeled “+”) is arranged to sum up outputs of the intra prediction unit 114, the motion compensation unit 124, and the variable dimension/resolution IT unit 134 to generate frame data, which can be temporarily stored in the frame storage 140. Additionally, the reconstructed frame output unit 150 is arranged to generate a plurality of reconstructed frames according to frame data temporarily stored in the frame storage 140, where the aforementioned de-blocking filter (e.g. the in-loop filter 152) is arranged to perform de-blocking filtering operations.
According to some variations of this embodiment, at least a portion of the adaptive spatial resolution decoder 104 can omit partial information being processed by the portion of the adaptive spatial resolution decoder 104, in order to reduce complexity of decoding the input bit stream 108.
According to various embodiments, such as the embodiment shown in FIG. 1B and some variations thereof, the decoding resolution control circuit 102 can selectively control at least a portion of components/modules/units within the adaptive spatial resolution decoder 104 to operate in accordance with a resolution differing from the original resolution of any frame carried by the input bit stream 108, in order to dynamically adjust decoding complexity frame by frame. Examples of the components/modules/units may comprise the variable dimension/resolution spatial prediction module 110, the variable dimension temporal prediction module 120, the variable dimension/resolution IT unit 134, and the variable dimension/resolution reconstructed frame output unit 150. As a result, the decoding resolution control circuit 102 can reduce complexity of decoding operations by dynamically adjusting decoding complexity frame by frame.
FIG. 2 illustrates a flowchart of a method 200 for dynamically adjusting video decoding complexity according to an embodiment of the present invention. The method shown in FIG. 2 can be applied to the apparatus 100 shown in FIG. 1A. The method is described as follows.
In Step 210, the decoding resolution control circuit 102 dynamically determines whether at least one portion of a plurality of frames (and more particularly, at least one frame within the aforementioned frames carried by the input bit stream 108) should be decoded in accordance with the specific resolution differing from the original resolution of the plurality of frames, in order to dynamically adjust decoding complexity frame by frame.
In Step 220, the adaptive spatial resolution decoder 104 decodes the plurality of frames according to whether the aforementioned at least one portion of the plurality of frames should be decoded in accordance with the specific resolution. For example, the adaptive spatial resolution decoder 104 decodes at least one frame of the plurality of frames by controlling at least a portion of components/modules/units within the adaptive spatial resolution decoder 104 to operate in accordance with the specific resolution.
In practice, the operations of Step 210 and Step 220 can be performed repeatedly. In addition, at least a portion of the operations of Step 210 and at least a portion of the operations of Step 220 can be performed at the same time. For example, before the decoding resolution control circuit 102 completes the operations of determining whether a current frame should be decoded in accordance with the specific resolution, the adaptive spatial resolution decoder 104 may still be decoding a previous frame. In another example, when the adaptive spatial resolution decoder 104 is decoding the current frame, the decoding resolution control circuit 102 may start the operations of determining whether a next frame should be decoded in accordance with the specific resolution.
According to the embodiment shown in FIG. 2, the system capability analyzing circuit 106 analyzes the system capability of at least a portion of the apparatus 100, in order to generate analyzing results such as those mentioned above, for use of decoding resolution control. Based on the analyzing results, the decoding resolution control circuit 102 dynamically determines whether the aforementioned at least one portion of the plurality of frames should be decoded in accordance with the specific resolution. In some embodiments, such as the embodiment shown in FIG. 2 and some variations thereof, the analyzing results may correspond to power consumption of the apparatus 100, the number of frames in a display buffer within/outside the apparatus 100, the system loading of the apparatus 100, and/or the decoding time of at least one frame (e.g. the average decoding time of some frames, or the overall decoding time of some frames). That is, in these embodiments, the analyzing results may correspond to one or a combination of a portion/all of the power consumption of the apparatus 100, the number of frames in the display buffer within/outside the apparatus 100, the system loading of the apparatus 100, and the decoding time of the aforementioned at least one frame.
According to some embodiments, such as the embodiment shown in FIG. 2 and some variations thereof, the decoding resolution control circuit 102 adaptively controls decoding resolution(s) (and more particularly, the decoding resolution(s) of the aforementioned at least one portion of the plurality of frames) based on the display resolution of a display module (e.g., a liquid crystal display module (LCM)), the system capability of the apparatus 100, the input bit stream 108 carrying the plurality of frames mentioned above, the target frame rate of the frames, the frame type(s) of the frames, and/or the structure of group of picture (GOP), where the display module mentioned above may be positioned within or outside the apparatus 100. That is, in these embodiments, the decoding resolution control circuit 102 adaptively controls the decoding resolution(s) (and more particularly, the decoding resolution(s) of the aforementioned at least one portion of the plurality of frames) based on one or a combination of a portion/all of the display resolution of the display module (e.g., the LCM), the system capability of the apparatus 100, the input bit stream 108 carrying the plurality of frames mentioned above, the target frame rate of the frames, the frame type(s) of the frames, and the structure of GOP.
For example, in a situation where the aforementioned at least one portion of the plurality of frames comprises a non-reference frame, the non-reference frame can be a bi-directional predicted frame (B-frame). In another example, in a situation where the aforementioned at least one portion of the plurality of frames comprises a reference frame, the reference frame can be an intra frame (I-frame), a predicted frame (P-frame), or a reference B-frame.
FIGS. 3A-3H respectively illustrate some video decoding schemes involved with the method 200 shown in FIG. 2 according to different embodiments of the present invention. In at least a portion of these embodiments, the complexity of decoding operations can be reduced by dynamically adjusting decoding complexity of each of the plurality of frames. For example, the decoding resolution control circuit 102 can control some decoding operations, such as those of temporal prediction, spatial prediction, VLD and IQ, IT, motion compensation, reconstruction addition, and de-blocking, to be performed in accordance with the specific resolution mentioned in Step 210. More particularly, for the aforementioned at least one portion of the plurality of frames (e.g. at least one frame within the aforementioned frames carried by the input bit stream 108), the adaptive spatial resolution decoder 104 allows motion compensation reference in an original domain or a down-sampling domain, and allows adaptive adjustment of the decode complexity in accordance with system capability, and further allows decoding complexity change (or change of the reference domain), where down-scaled decoding operations may be applied to I-frames, P-frames, and B-frames, while up-scaled decoding operations may be applied to I-frames, P-frames, and B-frames. In addition, some of the frames may be re-sampled in accordance with a display size (or a display resolution) of a displayer (e.g. the aforementioned display module such as the LCM in some of the embodiments shown in FIGS. 3A-3H) or resized by a resizer within the apparatus 100 (and more particularly, within the adaptive spatial resolution decoder 104).
Referring to FIG. 3A, under control of the decoding resolution control circuit 102, the adaptive spatial resolution decoder 104 decodes an I-frame, three B-frames, and a P-frame (respectively labeled “I”, “B”, “B”, “B”, and “P” in FIG. 3A, in the display order) in accordance with the original resolution mentioned in Step 210. Thus, these frames are decoded in accordance with the same resolution. In this embodiment, the resizer mentioned above resizes all of the frames shown in FIG. 3A in accordance with the display size (or the display resolution) of the LCM, and more particularly, performs down sampling operations on the I-frame, the B-frames, and the P-frame, where the video decoding scheme shown in FIG. 3A corresponds to a standard decoder mode of the adaptive spatial resolution decoder 104.
Referring to FIG. 3B, under control of the decoding resolution control circuit 102, the adaptive spatial resolution decoder 104 decodes an I-frame (labeled “I” in FIG. 3B) in accordance with the original resolution mentioned in Step 210, and decodes three B-frames (respectively labeled “B”, “B”, and “B” in FIG. 3B, in the display order) in accordance with the specific resolution mentioned in Step 210, and decodes a P-frame (labeled “P” in FIG. 3B) in accordance with the original resolution mentioned in Step 210, where the specific resolution of this embodiment is lower than the display resolution of the LCM. According to this embodiment, the three B-frames are decoded with low complexity, and therefore, can be referred to as low complexity B-frames. For example, the low complexity B-frames can be lossy. In another example, the low complexity B-frames can be lossless. In this embodiment, the resizer mentioned above resizes all of the frames shown in FIG. 3B in accordance with the display size (or the display resolution) of the LCM, and more particularly, performs down sampling operations on the I-frame and the P-frame and performs up sampling operations on the B-frames, where the video decoding scheme shown in FIG. 3B corresponds to a low complexity B-frame mode of the adaptive spatial resolution decoder 104.
Referring to FIG. 3C, under control of the decoding resolution control circuit 102, the adaptive spatial resolution decoder 104 decodes an I-frame (labeled “I” in FIG. 3C) in accordance with the original resolution mentioned in Step 210, and decodes three B-frames (respectively labeled “B”, “B”, and “B” in FIG. 3C, in the display order) in accordance with the specific resolution mentioned in Step 210, and decodes a P-frame (labeled “P” in FIG. 3C) in accordance with the original resolution mentioned in Step 210, where the specific resolution of this embodiment is equivalent to the display resolution of the LCM. According to this embodiment, the three B-frames are decoded with low complexity, and therefore, can be referred to as low complexity B-frames. For example, the low complexity B-frames can be lossy. In another example, the low complexity B-frames can be lossless. In this embodiment, the resizer mentioned above resizes some of the frames shown in FIG. 3C in accordance with the display size (or the display resolution) of the LCM, and more particularly, performs down sampling operations on the I-frame and the P-frame, where the video decoding scheme shown in FIG. 3C corresponds to a low complexity B-frame mode of the adaptive spatial resolution decoder 104.
Regarding the low complexity B-frame mode of any of the embodiments respectively shown in FIG. 3B and FIG. 3C, for a non-reference frame such as any B-frame mentioned in these embodiments (i.e. any of the low complexity B-frames), the decoding resolution control circuit 102 adaptively controls the decoding resolution(s) based on the display resolution of the display module mentioned above (e.g. the LCM in any of the embodiments shown in FIGS. 3B-3C), the system capability of the apparatus 100, the input bit stream 108 carrying the plurality of frames mentioned above, the target frame rate of these frames, the frame type(s) of these frames, and/or the structure of GOP. For example, the decoding resolution control circuit 102 parses the input bit stream 108 and controls the adaptive spatial resolution decoder 104 to decode and reconstruct partial images, in order to generate the low complexity B-frames. In a situation where the specific resolution of this embodiment is equivalent to the display resolution of the LCM, the low complexity B-frames can be displayed directly.
Referring to FIG. 3D, under control of the decoding resolution control circuit 102, the adaptive spatial resolution decoder 104 decodes an I-frame (labeled “I” in FIG. 3D) in accordance with the original resolution mentioned in Step 210, and decodes three B-frames and a P-frame (respectively labeled “B”, “B”, “B” and “P” in FIG. 3D, in the display order) in accordance with the specific resolution mentioned in Step 210, where the specific resolution of this embodiment is lower than the display resolution of the LCM. According to this embodiment, the three B-frames and the P-frame are decoded with low complexity, and therefore, can be referred to as low complexity B/P-frames, respectively. For example, the low complexity B/P-frames can be lossy. In another example, the low complexity B/P-frames can be lossless. In this embodiment, the resizer mentioned above resizes all of the frames shown in FIG. 3D in accordance with the display size (or the display resolution) of the LCM, and more particularly, performs a down sampling operation on the I-frame and performs up sampling operations on the B-frames and the P-frame, where the video decoding scheme shown in FIG. 3D corresponds to a low complexity P/B-frame mode of the adaptive spatial resolution decoder 104.
Referring to FIG. 3E, under control of the decoding resolution control circuit 102, the adaptive spatial resolution decoder 104 decodes an I-frame, three B-frames, and a P-frame (respectively labeled “I”, “B”, “B”, “B” and “P” in FIG. 3E, in the display order) in accordance with the specific resolution mentioned in Step 210, where the original resolution such as that of the original I-frame (labeled “I₀”, for comparison purposes only) is higher than the display resolution of the LCM, and the specific resolution of this embodiment is lower than the display resolution of the LCM. According to this embodiment, the I-frame (labeled “I”), the three B-frames, and the P-frame are all decoded with low complexity, and therefore, can be referred to as low complexity I/B/P-frames, respectively. For example, the low complexity B/P-frames can be lossy. In another example, the low complexity B/P-frames can be lossless. In this embodiment, the resizer mentioned above resizes the frames (e.g. those respectively labeled “I”, “B”, and “P” in FIG. 3E) in accordance with the display size (or the display resolution) of the LCM, and more particularly, performs up sampling operations on the I-frame (labeled “I”), the B-frames, and the P-frame, where the video decoding scheme shown in FIG. 3E corresponds to a low complexity flow mode of the adaptive spatial resolution decoder 104.
Referring to FIG. 3F, under control of the decoding resolution control circuit 102, the adaptive spatial resolution decoder 104 decodes an I-frame (labeled “I” in FIG. 3F) in accordance with the original resolution mentioned in Step 210, and decodes three B-frames and a P-frame (respectively labeled “B”, “B”, “B” and “P” in FIG. 3F, in the display order) in accordance with the specific resolution mentioned in Step 210, where the specific resolution of this embodiment is equivalent to the display resolution of the LCM. According to this embodiment, the three B-frames and the P-frame are decoded with low complexity, and therefore, can be referred to as low complexity B/P-frames, respectively. For example, the low complexity B/P-frames can be lossy. In another example, the low complexity B/P-frames can be lossless. In this embodiment, the resizer mentioned above resizes at least one frame of the frames shown in FIG. 3F in accordance with the display size (or the display resolution) of the LCM, and more particularly, performs a down sampling operation on the I-frame, where the video decoding scheme shown in FIG. 3F corresponds to a low complexity P/B-frame mode of the adaptive spatial resolution decoder 104.
Referring to FIG. 3G, under control of the decoding resolution control circuit 102, the adaptive spatial resolution decoder 104 decodes an I-frame, three B-frames, and a P-frame (respectively labeled “I”, “B”, “B”, “B” and “P” in FIG. 3G, in the display order) in accordance with the specific resolution mentioned in Step 210, where the original resolution such as that of the original I-frame (labeled “I₀”, for comparison purposes only) is higher than the display resolution of the LCM, and the specific resolution of this embodiment is equivalent to the display resolution of the LCM. According to this embodiment, the I-frame (labeled “I”), the three B-frames, and the P-frame are all decoded with low complexity, and therefore, can be referred to as low complexity I/B/P-frames, respectively. For example, the low complexity B/P-frames can be lossy. In another example, the low complexity B/P-frames can be lossless. In this embodiment, it is unnecessary to resize the frames (e.g. those respectively labeled “I”, “B”, and “P” in FIG. 3G) in accordance with the display size (or the display resolution) of the LCM, and therefore, performing up/down sampling operations is not required, where the video decoding scheme shown in FIG. 3G corresponds to a low complexity flow mode of the adaptive spatial resolution decoder 104.
Referring to FIG. 3H, under control of the decoding resolution control circuit 102, the adaptive spatial resolution decoder 104 decodes an I-frame, three B-frames, and a P-frame (respectively labeled “I”, “B”, “B”, “B” and “P” in FIG. 3H, in the display order) in accordance with the specific resolution mentioned in Step 210, where the original resolution such as that of the original I-frame (labeled “I₀”, for comparison purposes only) is higher than the display resolution of the LCM, and the specific resolution of this embodiment is higher than the display resolution of the LCM. According to this embodiment, the I-frame (labeled “I”), the three B-frames, and the P-frame are all decoded with low complexity, and therefore, can be referred to as low complexity I/B/P-frames, respectively. For example, the low complexity B/P-frames can be lossy. In another example, the low complexity B/P-frames can be lossless. In this embodiment, the resizer mentioned above resizes the frames (e.g. those respectively labeled “I”, “B”, and “P” in FIG. 3H) in accordance with the display size (or the display resolution) of the LCM, and more particularly, performs down sampling operations on the I-frame (labeled “I”), the B-frames, and the P-frame, where the video decoding scheme shown in FIG. 3H corresponds to a low complexity flow mode of the adaptive spatial resolution decoder 104.
Regarding the low complexity P/B-frame mode of any of the embodiments respectively shown in FIG. 3D and FIG. 3F and the low complexity flow mode of any of the embodiments respectively shown in FIG. 3E, FIG. 3G, and FIG. 3H, for a reference frame such as any I-frame (labeled “I”) mentioned in the embodiments shown in FIG. 3E, FIG. 3G, and FIG. 3H, any reference B-frame within the B-frames mentioned in the embodiments shown in FIGS. 3D-3H (i.e. any reference B-frame within the low complexity B-frames), and any P-frame mentioned in the embodiments shown in FIGS. 3D-3H (i.e. any of the low complexity P-frames), the decoding resolution control circuit 102 adaptively controls the decoding resolution(s) based on the display resolution of the display module mentioned above (e.g. the LCM in any of the embodiments shown in FIGS. 3D-3H), the system capability of the apparatus 100, the input bit stream 108 carrying the plurality of frames mentioned above, the target frame rate of these frames, the frame type(s) of these frames, and/or the structure of GOP. For example, the decoding resolution control circuit 102 parses the input bit stream 108 and controls the adaptive spatial resolution decoder 104 to decode and reconstruct partial images, in order to generate the low complexity I/P/B-frames. More particularly, for some following frames (based on the decoding order), the adaptive spatial resolution decoder 104 performs motion compensation based on the reconstructed partial images in accordance with the specific resolution, rather than the original resolution. In a situation where the specific resolution of this embodiment is equivalent to the display resolution of the LCM, the low complexity I/P/B-frames can be displayed directly.
In practice, when controlling the adaptive spatial resolution decoder 104 to partially reconstruct a frame, the decoding resolution control circuit 102 can adaptively controls at least a portion of components/modules/units within the adaptive spatial resolution decoder 104 to operate in accordance with a resolution differing from that of original frames, in order to reduce the complexity and to reduce memory usage and power consumption. For example, the variable dimension/resolution spatial prediction module 110 can generate a partial image during intra prediction, the variable dimension temporal prediction module 120 can generate a partial image in motion compensation during inter prediction, an IQ and IT module (which may comprise at least a portion of the VLD and IQ module 132 and comprise the variable dimension/resolution IT unit 134) can output a partial reconstructed residual image during IQ and IT operations, the frame storage 140 can store a partial image, and the aforementioned de-blocking filter (e.g. the in-loop filter 152) can filter a partial image, where partial information may be decoded during entropy decoding.
It is an advantage of the present invention that the present invention method and apparatus can reduce complexity of decoding operations by dynamically adjusting decoding complexity of at least a portion of frames. As a result, the calculation load and the power consumption within a portable electronic device such as that mentioned above can be reduced, where the related art problems (e.g. insufficient decoding capability and high power consumption) will no longer be an issue.
Those skilled in the art will readily observe that numerous modifications and alterations of the device and method may be made while retaining the teachings of the invention. Accordingly, the above disclosure should be construed as limited only by the metes and bounds of the appended claims.

Claims

1. An apparatus for dynamically adjusting video decoding complexity, the apparatus comprising:

a decoding resolution control circuit arranged to dynamically determine whether at least one portion of a plurality of frames should be decoded in accordance with a specific resolution differing from an original resolution of the plurality of frames; and

an adaptive spatial resolution decoder arranged to decode the plurality of frames according to whether the at least one portion of the plurality of frames should be decoded in accordance with the specific resolution.

2. The apparatus of claim 1, wherein the adaptive spatial resolution decoder decodes at least one frame of the plurality of frames by controlling the adaptive spatial resolution decoder to operate in accordance with the specific resolution.

3. The apparatus of claim 1, further comprising:

a system capability analyzing circuit arranged to analyze system capability of at least a portion of the apparatus, in order to generate analyzing results for being sent to the decoding resolution control circuit.

4. The apparatus of claim 3, wherein based on the analyzing results, the decoding resolution control circuit dynamically determines whether the at least one portion of the plurality of frames should be decoded in accordance with the specific resolution.

5. The apparatus of claim 3, wherein the analyzing results correspond to one or a combination of a portion/all of power consumption, a number of frames in a display buffer, system loading, and a decoding time.

6. The apparatus of claim 1, wherein the decoding resolution control circuit adaptively controls decoding resolution(s) based on one or a combination of a portion/all of a display resolution of a display module, system capability, an input bit stream carrying the frames, a target frame rate of the frames, frame type(s) of the frames, and structure of group of picture (GOP).

7. The apparatus of claim 6, wherein the at least one portion of the plurality of frames comprises a non-reference frame.

8. The apparatus of claim 7, wherein the non-reference frame is a bi-directional predicted frame (B-frame).

9. The apparatus of claim 6, wherein the at least one portion of the plurality of frames comprises a reference frame.

10. The apparatus of claim 9, wherein the reference frame is an intra frame (I-frame), a predicted frame (P-frame), or a reference bi-directional predicted frame (B-frame).

11. A method for dynamically adjusting video decoding complexity, the method comprising:

dynamically determining whether at least one portion of a plurality of frames should be decoded in accordance with a specific resolution differing from an original resolution of the plurality of frames; and

decoding the plurality of frames according to whether the at least one portion of the plurality of frames should be decoded in accordance with the specific resolution.

12. The method of claim 11, wherein the step of decoding the plurality of frames further comprises:

decoding at least one frame of the plurality of frames by controlling an adaptive spatial resolution decoder to operate in accordance with the specific resolution.

13. The method of claim 11, further comprising:

analyzing system capability, in order to generate analyzing results, wherein the analyzing results are utilized for performing decoding resolution control.

14. The method of claim 13, wherein the step of dynamically determining whether the at least one portion of the plurality of frames should be decoded in accordance with the specific resolution further comprises:

based on the analyzing results, dynamically determining whether the at least one portion of the plurality of frames should be decoded in accordance with the specific resolution.

15. The method of claim 13, wherein the analyzing results correspond to one or a combination of a portion/all of power consumption, a number of frames in a display buffer, system loading, and a decoding time.

16. The method of claim 11, wherein the step of dynamically determining whether the at least one portion of the plurality of frames should be decoded in accordance with the specific resolution further comprises:

adaptively controlling decoding resolution(s) based on one or a combination of a portion/all of a display resolution of a display module, system capability, an input bit stream carrying the frames, a target frame rate of the frames, frame type(s) of the frames, and structure of group of picture (GOP).

17. The method of claim 16, wherein the at least one portion of the plurality of frames comprises a non-reference frame.

18. The method of claim 17, wherein the non-reference frame is a bi-directional predicted frame (B-frame).

19. The method of claim 16, wherein the at least one portion of the plurality of frames comprises a reference frame.

20. The method of claim 19, wherein the reference frame is an intra frame (I-frame), a predicted frame (P-frame), or a reference bi-directional predicted frame (B-frame).