US9271008B2 - Video encoding apparatus and method, video encoding program, and storage medium which stores the program - Google Patents

Video encoding apparatus and method, video encoding program, and storage medium which stores the program Download PDF

Info

Publication number
US9271008B2
US9271008B2 US12/667,650 US66765008A US9271008B2 US 9271008 B2 US9271008 B2 US 9271008B2 US 66765008 A US66765008 A US 66765008A US 9271008 B2 US9271008 B2 US 9271008B2
Authority
US
United States
Prior art keywords
prediction error
step size
amount
upper limit
quantization step
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active, expires
Application number
US12/667,650
Other versions
US20110013693A1 (en
Inventor
Atsushi Shimizu
Yasuyuki Nakajima
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nippon Telegraph and Telephone Corp
Original Assignee
Nippon Telegraph and Telephone Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nippon Telegraph and Telephone Corp filed Critical Nippon Telegraph and Telephone Corp
Assigned to NIPPON TELEGRAPH AND TELEPHONE CORPORATION reassignment NIPPON TELEGRAPH AND TELEPHONE CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: NAKAJIMA, YASUYUKI, SHIMIZU, ATSUSHI
Publication of US20110013693A1 publication Critical patent/US20110013693A1/en
Application granted granted Critical
Publication of US9271008B2 publication Critical patent/US9271008B2/en
Active legal-status Critical Current
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
    • H04N19/436—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation using parallelised computational arrangements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/124—Quantisation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136—Incoming video signal characteristics or properties
    • H04N19/137—Motion inside a coding unit, e.g. average field, frame or block difference
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146—Data rate or code amount at the encoder output
    • H04N19/149—Data rate or code amount at the encoder output by estimating the code amount by means of a model, e.g. mathematical model or statistical model
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146—Data rate or code amount at the encoder output
    • H04N19/15—Data rate or code amount at the encoder output by monitoring actual compressed data size at the memory before deciding storage at the transmission buffer
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146—Data rate or code amount at the encoder output
    • H04N19/152—Data rate or code amount at the encoder output by measuring the fullness of the transmission buffer
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding

Definitions

  • the present invention relates to a video encoding apparatus and a corresponding method for applying orthogonal transformation to a prediction error signal between a video signal of an encoding target area and a predicted signal thereof, and quantizing an obtained orthogonal transformation coefficient by using a quantization step size so as to encode the coefficient, and also relates to a video encoding program used for implementing the video encoding apparatus and a storage medium which stores the program.
  • the present invention relates to a video encoding apparatus and a corresponding method, which do not require re-encoding or encoding which handles two or more encoding modes and implement encoding which generates codes less than an upper limit amount of code, a video encoding program used for implementing the video encoding apparatus, and a storage medium which stores the program.
  • the upper limit amount of code for one macroblock is determined (see, for example, Non-Patent Document 1).
  • a video encoding apparatus based on H.264 should perform encoding in a manner such that the amount of generated code generated for one macroblock does not exceed the above upper limit amount.
  • the amount of generated code is measured after encoding, and if the measured amount exceeds the upper limit, encoding should be again performed with revised encoding conditions.
  • encoding processes (orthogonal transformation, quantization, information source encoding, and the like) corresponding to two or more encoding modes to which different encoding conditions are assigned are simultaneously executed, and one which produces an encoding result whose amount of generated code does not exceed the relevant upper limit is selected.
  • H.264 employs a pulse code modulation (PCM) mode in which the pixel value is directly transmitted without compression (i.e., without quantization).
  • PCM pulse code modulation
  • encoding is executed after determining the encoding mode, and the amount of code generated in the encoding is measured. If the measured value exceeds an upper limit, re-encoding is performed in the PCM mode.
  • an arithmetic coding method employed in H.264 has a feature such that the amount of code cannot be instantaneously measured.
  • Non-Patent Document 2 relating to hardware devices for performing pipeline operation for macroblocks as units, when there is a macroblock whose number of bits exceeds an upper limit, not the input image of the macroblock but a local decoded image thereof in the relevant encoder is re-encoded in the PCM mode.
  • encoding should be performed in a manner such that the amount of code generated for one macroblock is within a specific upper limit.
  • the amount of generated code is measured after an encoding process, and if the amount of generated code exceeds a specific upper limit, re-encoding may be performed with revised encoding conditions.
  • encoding is performed after determining the encoding mode, and the amount of code generated in the encoding is measured. If the measured value exceeds an upper limit, re-encoding is performed in the PCM mode.
  • the arithmetic coding method employed in H.264 has a feature such that the amount of code cannot be instantaneously measured, and thus a processing delay occurs in a hardware device which executes a pipeline operation.
  • an object of the present invention is to provide a novel image encoding technique which does not require re-encoding or encoding corresponding to two or more encoding modes, and implements an encoding whose amount of generated code does not exceed an upper limit without awaiting a measured result of the amount of generated code.
  • the present invention provides a video encoding apparatus for applying orthogonal transformation to a prediction error signal between a video signal of an encoding target area and a predicted signal for the video signal, and quantizing an obtained orthogonal transformation coefficient by using a preset quantization step size so as to encode the coefficient.
  • the apparatus comprises:
  • processing devices can also be implemented by a computer program.
  • a computer program may be provided by storing it in an appropriate computer-readable storage medium, or by means of a network, and can be installed and operate on a control device such as a CPU so as to implement the present invention.
  • an approximate amount of generated code can be estimated.
  • a prediction error power which is a power of the prediction error signal (as an encoding target) is computed. Based on the computed prediction error power and the quantization step size to be used in the encoding, an amount of code generated when performing the quantization using the quantization step size to be used in the encoding is estimated. The estimated value is compared with the relevant upper limit of the amount of generated code, so that it can be determined whether or not the amount of code generated when performing quantization using the quantization step size to be used in the encoding exceeds the upper limit.
  • the amount of generated code is directly estimated.
  • the determination process is equivalent to a process for determining whether or not the prediction error power is within a permissive power range defined based on the upper limit of the amount of generated code.
  • a permissive power for the prediction error power is computed based on the upper limit of the amount of generated code and the quantization step size to be used in the encoding, and the permissive power is compared with the computed prediction error power so as to determine whether or not the amount of code generated when performing the quantization using the quantization step size to be used in the encoding exceeds the upper limit.
  • the estimated value for the amount of generated code or the permissive power for the prediction error power, which is used in the determination process, can be easily computed by means of a function or a table.
  • the permissive power for the prediction error power by setting variables of a function, which are the upper limit of the amount of generated code and the quantization step size, to the values of the upper limit and the quantization step size, where the value of the function is the permissive power for the prediction error power. It is also possible to compute the permissive power for the prediction error power by referring to a table in which a relationship between data values of the upper limit of the amount of generated code, the quantization step size, and the permissive power for the prediction error power is defined.
  • a quantized value of the orthogonal transformation coefficient may not be encoded, but the video signal may be encoded without quantizing the video signal; and (ii) in a second example, a quantization step size may be obtained, which is computed based on the prediction error power and the upper limit of the amount of generated code and implements generation of the amount of code which does not exceed the upper limit, and the quantization step size may be switched from the quantization step size to be used in the encoding to the obtained quantization step size.
  • the computation of the quantization step size used in the above switching operation is implemented using an inverse function of the function which is used for the above-described estimation of the amount of generated code. Therefore, also in this case, the relevant quantization step size can be easily computed using a function or a table.
  • the present invention can be applied to an apparatus for applying orthogonal transformation to a prediction error signal between a video signal of an encoding target area and a predicted signal for the video signal, and quantizing an obtained orthogonal transformation coefficient by using a quantization step size so as to encode the coefficient.
  • the present invention can implement encoding which generates codes less than an upper limit amount of code, without measuring the amount of generated code. Therefore, the present invention does not require re-encoding or encoding which handles two or more encoding modes and can implement the encoding which generates codes less than the upper limit amount of code.
  • the present invention can implement an encoding whose amount of generated code does not exceed an upper limit without awaiting a measured result of the amount of generated code, no processing delay occurs in a hardware device which executes a pipeline operation.
  • FIG. 1 is a diagram showing a video encoding apparatus as an embodiment of the present invention.
  • FIG. 2 is a diagram showing an example structure of the code amount estimator in the embodiment.
  • FIG. 3 is a diagram showing an example structure of the quantization step size computation unit in the embodiment.
  • FIG. 4 is a flowchart executed by the video encoding apparatus of the embodiment.
  • FIG. 5A is also a flowchart executed by the video encoding apparatus.
  • FIG. 5B is also a flowchart executed by the video encoding apparatus.
  • FIG. 6 is a diagram showing another example structure of the code amount estimator in the embodiment.
  • FIG. 7 is a flowchart executed by the video encoding apparatus employing the structure in FIG. 6 .
  • FIG. 8A is also a flowchart executed by the video encoding apparatus.
  • FIG. 8B is also a flowchart executed by the video encoding apparatus.
  • FIG. 9 is a diagram showing a video encoding apparatus as another embodiment of the present invention.
  • FIG. 10 is a flowchart executed by the video encoding apparatus of the embodiment.
  • FIG. 11 is another flowchart executed by the video encoding apparatus of the embodiment.
  • FIG. 12 is a diagram showing an example structure of the generated code amount estimator.
  • FIG. 13 is a diagram showing another example structure of the generated code amount estimator.
  • FIG. 14 is a diagram showing an example structure of the permissive prediction error power computation unit.
  • FIG. 15 is a diagram showing another example structure of the permissive prediction error power computation unit.
  • FIG. 16 is a diagram showing an example structure of the quantization step size computation unit.
  • FIG. 17 is a diagram showing another example structure of the quantization step size computation unit.
  • FIG. 18 is a flowchart explaining a conventional technique.
  • FIG. 1 shows a video encoding apparatus as an embodiment of the present invention.
  • reference numeral 10 indicates a structural part (surrounded by a dotted line) as an H.264 video encoding apparatus based on H.264
  • reference numeral 20 indicates a code amount estimator provided for implementing the present invention
  • reference numeral 21 indicates a quantization step size computation unit provided for implementing the present invention
  • reference numeral 22 indicates a selector switch for implementing the present invention.
  • the part 10 as the H.264 video encoding apparatus includes a motion detector 100 , a motion compensator 101 , a frame memory 102 , an interframe prediction mode determination unit 103 , an intraframe prediction mode determination unit 104 , a selector switch 105 , a subtractor 106 , an orthogonal transformer 107 , a quantizer 108 , a quantization controller 109 , an inverse quantizer 110 , an inverse orthogonal transformer 111 , an adder 112 , a loop filter 113 , and an information source encoder 114 .
  • the orthogonal transformer 107 subjects the generated prediction error signal to orthogonal transformation.
  • the quantizer 108 quantizes orthogonal transformation coefficients obtained by the orthogonal transformation.
  • the information source encoder 114 subjects the quantized values to entropy encoding so as to encode the video signal.
  • FIG. 2 shows an example structure of the code amount estimator 20 .
  • the code amount estimator 20 receives an upper limit value of the amount of code generated for the relevant macroblock (i.e., an upper limit amount of code), the prediction error signal generated by the subtractor 106 , and the quantization step size set by the quantization controller 109 , and includes a prediction error (electric) power computation unit 200 , a generated code amount estimator 201 , and a code amount comparator 202 .
  • the prediction error power computation unit 200 computes a prediction error power, which is a power of the prediction error signal generated by the subtractor 106 .
  • the generated code amount estimator 201 estimates an amount of code generated when quantizing the encoding target macroblock by the relevant quantization step size.
  • the code amount comparator 202 compares the estimated amount of generated code obtained by the generated code amount estimator 201 with the upper limit (defined in H.264) for the amount of code generated for the macroblock. If the estimated amount of generated code obtained by the generated code amount estimator 201 is greater than the upper limit for the amount of code generated for the macroblock, the code amount comparator 202 directs the selector switch 22 to switch the quantization step size supplied to the quantizer 108 from the quantization step size set by the quantization controller 109 to a quantization step size computed by the quantization step size computation unit 21 .
  • the code amount comparator 202 directs the selector switch 22 to switch the quantization step size supplied to the quantizer 108 from the quantization step size set by the quantization controller 109 to a quantization step size computed by the quantization step size computation unit 21 .
  • the code amount comparator 202 directs the selector switch 22 to directly use the quantization step size set by the quantization controller 109 as the quantization step size supplied to the quantizer 108 .
  • FIG. 3 shows an example structure of the quantization step size computation unit 21 .
  • the quantization step size computation unit 21 receives the upper limit value of the amount of code generated for the relevant macroblock (i.e., the upper limit amount of code) and the prediction error signal generated by the subtractor 106 , and includes a prediction error (electric) power computation unit 210 and a minimum quantization step size computation unit 211 .
  • the prediction error power computation unit 210 computes a prediction error power, which is a power of the prediction error signal generated by the subtractor 106 .
  • the minimum quantization step size computation unit 211 computes a quantization step size for implementing code amount generation which does not exceed the upper limit (i.e., a minimum quantization step size).
  • FIGS. 4 to 5B show flowcharts executed by the video encoding apparatus of the present embodiment.
  • the encoding mode is determined in the first step S 10 .
  • the amount of code generated when performing the encoding by using the currently-set quantization step size is estimated.
  • next step S 12 it is determined whether or not the estimated amount of code generated for the relevant macroblock is greater than the upper limit defined therefor. If it is determined that the estimated amount is greater than the upper limit, the operation proceeds to step S 13 , where the quantization step size is changed. In the next step S 14 , encoding is performed using the newly-set quantization step size.
  • step S 12 If it is determined in the determination of step S 12 that the estimated amount is smaller than or equal to the upper limit, the operation directly proceeds to step S 14 by skipping step S 13 , and encoding is performed using the currently-set quantization step size.
  • FIG. 5A is a flowchart of the process of estimating the amount of generated code in step S 11
  • FIG. 5B is a flowchart of the process of changing the quantization step size in step S 13 .
  • step S 11 the prediction error signal is received so as to compute the prediction error power (see first step S 110 ), and in the next step S 111 , the currently-set quantization step size is received.
  • the next step S 112 based on the computed prediction error power and the received quantization step size, the amount of code generated when performing the encoding by using the currently-set quantization step size is estimated.
  • step S 13 the upper limit value of the amount of code generated for the relevant macroblock (i.e., the upper limit amount of code) is received in the first step S 130 , and in the next step S 131 , the prediction error signal is received so as to compute the prediction error power.
  • the next step S 132 based on the received upper limit of the amount of generated code and the computed prediction error power, a quantization step size for implementing the code amount generation which does not exceed the upper limit is computed.
  • the quantization step size used in the relevant encoding is changed.
  • the video encoding apparatus shown in FIG. 1 estimates the amount of code generated when performing the encoding using the currently-set quantization step size, based on a relationship between the amount of generated code and the quantization step size. If the estimated value is greater than the upper limit for the amount of code generated for the relevant macroblock, a quantization step size for implementing the code amount generation which does not exceed the upper limit is computed, and the quantization step size used in the encoding is changed to the computed value.
  • re-encoding or encoding corresponding to two or more encoding modes is unnecessary, and an encoding whose amount of generated code does not exceed an upper limit can be implemented without awaiting a measured result of the amount of generated code.
  • FIG. 6 shows another example structure of the code amount estimator 20 .
  • the code amount estimator 20 receives an upper limit value of the amount of code generated for the relevant macroblock (i.e., an upper limit amount of code), the prediction error signal generated by the subtractor 106 , and the quantization step size set by the quantization controller 109 , and includes a prediction error power computation unit 200 , a permissive prediction error power computation unit 203 , and a prediction error comparator 204 .
  • the prediction error power computation unit 200 computes a prediction error power, which is a power of the prediction error signal generated by the subtractor 106 .
  • the permissive prediction error power computation unit 203 computes a permissive power of the prediction error power (i.e., permissive prediction error power) based on the upper limit of the amount of code generated for the macroblock and the quantization step size set by the quantization controller 109 .
  • the prediction error comparator 204 compares the prediction error power computed by the prediction error power computation unit 200 with the permissive prediction error power computed by the permissive prediction error power computation unit 203 . If the prediction error power computed by the prediction error power computation unit 200 is larger than the permissive prediction error power computed by the permissive prediction error power computation unit 203 , the prediction error comparator 204 directs the selector switch 22 to switch the quantization step size supplied to the quantizer 108 from the quantization step size set by the quantization controller 109 to a quantization step size computed by the quantization step size computation unit 21 .
  • the prediction error comparator 204 directs the selector switch 22 to directly use the quantization step size set by the quantization controller 109 as the quantization step size supplied to the quantizer 108 .
  • FIGS. 7 to 8B show flowcharts executed by the video encoding apparatus of the present embodiment when the code amount estimator 20 has the structure as shown in FIG. 6 .
  • the video encoding apparatus of the present embodiment determines the encoding mode in the first step S 20 .
  • the permissive prediction error power as a permissive power of the prediction error power is computed.
  • next step S 22 it is determined whether or not the prediction error power is larger than the permissive prediction error power. If it is determined that the prediction error power is larger than the permissive power, the operation proceeds to step S 23 , where the quantization step size is changed. In the next step S 24 , encoding is performed using the newly-set quantization step size.
  • step S 21 when computing the permissive prediction error power in step S 21 , the prediction error power used in step S 22 is also computed.
  • step S 22 If it is determined in the determination of step S 22 that the prediction error power is smaller than or equal to the permissive prediction error power, the operation directly proceeds to step S 24 by skipping step S 23 , and encoding is performed using the currently-set quantization step size.
  • FIG. 8A is a flowchart of the process of computing the permissive prediction error power in step S 21
  • FIG. 8B is a flowchart of the process of changing the quantization step size in step S 23 .
  • step S 21 the prediction error signal is received so as to compute the prediction error power (see first step S 210 ), and in the next step S 211 , the upper limit of the amount of code generated for the macroblock (i.e., the upper limit amount of code) is received.
  • the next step S 212 the currently-set quantization step size is received, and in the next step S 213 , the permissive prediction error power is computed based on the received upper limit of the amount of generated code and the received quantization step size.
  • step S 23 the upper limit value of the amount of code generated for the macroblock (i.e., the upper limit amount of code) is received in the first step S 230 , and in the next step S 231 , the prediction error signal is received so as to compute the prediction error power.
  • the next step S 232 based on the received upper limit of the amount of generated code and the computed prediction error power, a quantization step size for implementing the code amount generation which does not exceed the upper limit is computed.
  • the quantization step size used in the relevant encoding is changed.
  • the video encoding apparatus shown in FIG. 1 determines based on the permissive power of the prediction error power, which is derived by the upper limit of the amount of generated code and the quantization step size, whether or not the amount of code generated when performing the encoding by using the currently-set quantization step size exceeds the upper limit. If the relevant amount of generated code is greater than the upper limit, a quantization step size for implementing the code amount generation which does not exceed the upper limit is computed, and the quantization step size used in the encoding is changed to the computed value.
  • re-encoding or encoding corresponding to two or more encoding modes is unnecessary, and an encoding whose amount of generated code does not exceed an upper limit can be implemented without awaiting a measured result of the amount of generated code.
  • FIG. 9 shows another embodiment of the video encoding apparatus in accordance with the present invention.
  • parts identical to those explained in FIG. 1 are given identical reference numerals.
  • reference numeral 10 indicates a structural part (surrounded by a dotted line) as an H.264 video encoding apparatus based on H.264
  • reference numeral 30 indicates a code amount estimator provided for implementing the present invention
  • reference numeral 31 indicates a PCM encoder provided for implementing the present invention
  • reference numeral 32 indicates a selector switch for implementing the present invention.
  • the PCM encoder 31 subjects the relevant video signal as an encoding target to PCM encoding, without performing quantization, and outputs the encoded data via the selector switch 32 to the information source encoder 114 .
  • the code amount estimator 30 has a basic structure identical to that of the code amount estimator 20 in the embodiment shown in FIG. 1 , and may have a structure shown in FIG. 2 .
  • the code amount estimator 30 directs the selector switch 32 to supply the signal output from the PCM encoder 31 to the information source encoder 114 .
  • the code amount estimator 30 directs the selector switch 32 to supply the signal output from the quantizer 108 to the information source encoder 114 .
  • the video encoding apparatus of the present embodiment first determines the encoding mode (see the first step S 30 ).
  • the next step S 31 the amount of code generated when performing the encoding using the currently set quantization step size is estimated.
  • step S 32 it is determined whether or not the estimated amount of generated code is greater than the upper limit for the relevant amount of generated code. If it is determined that the estimated amount is greater than the upper limit, the operation proceeds to step S 33 , where the PCM encoding is executed. If it is determined that the estimated amount is smaller than or equal to the upper limit, the operation proceeds to step S 34 , where the ordinary encoding is executed.
  • the code amount estimator 30 may have the structure shown in FIG. 6 . In such a case, if the prediction error power is higher than a permissive prediction error power, the code amount estimator 30 directs the selector switch 32 to supply a signal output from the PCM encoder 31 to the information source encoder 114 . On the contrary, if the prediction error power is lower than or equal to the permissive prediction error power, the code amount estimator 30 directs the selector switch 32 to supply a signal output from the quantizer 108 to the information source encoder 114 .
  • the video encoding apparatus of the present embodiment first determines the encoding mode (see the first step S 40 ).
  • a permissive prediction error power as the permissive power for the prediction error power, is computed.
  • the video encoding apparatus of the present embodiment determines whether or not the amount of code generated when encoding is executed using the currently set quantization step size exceeds an upper limit defined therefore. If the amount of generated code exceeds the upper limit, PCM encoding is executed without performing quantization.
  • the code amount estimator 201 shown in FIG. 2 the permissive prediction error power computation unit 203 shown in FIG. 6 , and the minimum quantization step size computation unit 211 shown in FIG. 3 each can be implemented using a function or a look-up table.
  • the generated code amount estimator 201 in FIG. 2 has a plurality of functions (function 1 to function N) corresponding to a plurality of prediction modes, and one of the functions which is suitable for the prediction mode of the encoding target macroblock is selected and used.
  • the generated code amount estimator 201 in FIG. 2 has a plurality of look-up tables (LT 1 to LTN) corresponding to a plurality of prediction modes, and one of the look-up tables which is suitable for the prediction mode of the encoding target macroblock is selected and used.
  • the permissive prediction error power computation unit 203 in FIG. 6 has a plurality of functions (function 1 to function N) corresponding to a plurality of prediction modes, and one of the functions which is suitable for the prediction mode of the encoding target macroblock is selected and used.
  • the permissive prediction error power computation unit 203 in FIG. 6 has a plurality of look-up tables (LT 1 to LTN) corresponding to a plurality of prediction modes, and one of the look-up tables which is suitable for the prediction mode of the encoding target macroblock is selected and used.
  • the minimum quantization step size computation unit 211 shown in FIG. 3 has a plurality of functions (function 1 to function N) corresponding to a plurality of prediction modes, and one of the functions which is suitable for the prediction mode of the encoding target macroblock is selected and used.
  • the minimum quantization step size computation unit 211 shown in FIG. 3 has a plurality of look-up tables (LT 1 to LTN) corresponding to a plurality of prediction modes, and one of the look-up tables which is suitable for the prediction mode of the encoding target macroblock is selected and used.
  • the present invention can be applied to a video encoding apparatus for applying orthogonal transformation to a prediction error signal between a video signal of an encoding target area and a predicted signal for the video signal, and quantizing an obtained orthogonal transformation coefficient by using a quantization step size so as to encode the coefficient.
  • the present invention does not require re-encoding or encoding which handles two or more encoding modes and can implement encoding which generates codes less than an upper limit amount of code, without awaiting a measured result of the amount of generated code.

Abstract

A video encoding apparatus and a corresponding method for applying orthogonal transformation to a prediction error signal between a video signal of an encoding target area and a predicted signal for the video signal, and quantizing an obtained orthogonal transformation coefficient by using a preset quantization step size so as to encode the coefficient. A prediction error power which is a power of the prediction error signal is computed. For input information such as the computed prediction error power, the preset quantization step size, and an upper limit of an amount of code generated for the encoding target area, it is determined whether or not an amount of code generated when performing quantization using the preset quantization step size exceeds the upper limit. An encoding process is changed based on a result of the determination.

Description

TECHNICAL FIELD
The present invention relates to a video encoding apparatus and a corresponding method for applying orthogonal transformation to a prediction error signal between a video signal of an encoding target area and a predicted signal thereof, and quantizing an obtained orthogonal transformation coefficient by using a quantization step size so as to encode the coefficient, and also relates to a video encoding program used for implementing the video encoding apparatus and a storage medium which stores the program. In particular, the present invention relates to a video encoding apparatus and a corresponding method, which do not require re-encoding or encoding which handles two or more encoding modes and implement encoding which generates codes less than an upper limit amount of code, a video encoding program used for implementing the video encoding apparatus, and a storage medium which stores the program.
Priority is claimed on Japanese Patent Application No. 2007-185374, filed Jul. 17, 2007, the contents of which are incorporated herein by reference.
BACKGROUND ART
In H.264 as an international coding standard, the upper limit amount of code for one macroblock is determined (see, for example, Non-Patent Document 1).
Therefore, a video encoding apparatus based on H.264 should perform encoding in a manner such that the amount of generated code generated for one macroblock does not exceed the above upper limit amount.
In order to implement the above condition, the amount of generated code is measured after encoding, and if the measured amount exceeds the upper limit, encoding should be again performed with revised encoding conditions.
However, in such a method, the amount of computation or the processing time increases due to re-encoding with revised encoding conditions.
In a proposed method for solving the above problem, encoding processes (orthogonal transformation, quantization, information source encoding, and the like) corresponding to two or more encoding modes to which different encoding conditions are assigned are simultaneously executed, and one which produces an encoding result whose amount of generated code does not exceed the relevant upper limit is selected.
However, in such a method, encoding processes corresponding to two or more encoding modes having different encoding conditions should be simultaneously executed, and an encoding result whose amount of generated code does not exceed the upper limit is not always obtained.
Therefore, in order to reliably encode each macroblock of any input image with a number of bits less than an upper limit, H.264 employs a pulse code modulation (PCM) mode in which the pixel value is directly transmitted without compression (i.e., without quantization).
In a conventional technique using the above, as shown in FIG. 18, encoding is executed after determining the encoding mode, and the amount of code generated in the encoding is measured. If the measured value exceeds an upper limit, re-encoding is performed in the PCM mode.
On the other hand, in comparison with a conventional encoding method using a coding table, an arithmetic coding method employed in H.264 has a feature such that the amount of code cannot be instantaneously measured.
Therefore, an excess over the upper limit number of bits may be detected after the processing of the next macroblock is started. In such a situation, a problem occurs in that there is a delay in a pipeline operation (i.e., parallel execution).
Accordingly, in a hardware device for performing a pipeline operation for macroblocks (as units), if an input image of a macroblock whose number of bits exceeds an upper limit is re-encoded in the above-described PCM mode, an additional memory is necessary for storing the input image until the encoding reaches the final stage.
Therefore, in a currently-proposed technique (see, for example, Non-Patent Document 2) relating to hardware devices for performing pipeline operation for macroblocks as units, when there is a macroblock whose number of bits exceeds an upper limit, not the input image of the macroblock but a local decoded image thereof in the relevant encoder is re-encoded in the PCM mode.
  • Non-Patent Document 1: ITU-T Rec.H.264, “Advanced video coding for generic audio visual services”, pp. 249-256, 2003.
  • Non-Patent Document 2: Keiichi Chono, Yuzo Senda, Yoshihiro Miyamoto, “A PCM coding method using decoded images for obeying the upper limit on the number of bits of MB in H.264 encoding”, pp. 119-120, PCSJ2006.
DISCLOSURE OF INVENTION Problem to be Solved by the Invention
As described above, in a video encoding apparatus based on H.264, encoding should be performed in a manner such that the amount of code generated for one macroblock is within a specific upper limit. In order to implement this condition, the amount of generated code is measured after an encoding process, and if the amount of generated code exceeds a specific upper limit, re-encoding may be performed with revised encoding conditions.
However, in such a method, the amount of computation or the processing time increases due to re-encoding with revised encoding conditions.
In a proposed method for solving the above problem, encoding processes corresponding to two or more encoding modes to which different encoding conditions are assigned are simultaneously executed, and the one which produces an encoding result whose amount of generated code does not exceed the relevant upper limit is selected.
However, in such a method, encoding processes corresponding to two or more encoding modes having different encoding conditions should be simultaneously executed, and an encoding result whose amount of generated code does not exceed the upper limit is not always obtained.
Therefore, in the conventional technique as shown in the above-referenced FIG. 18, encoding is performed after determining the encoding mode, and the amount of code generated in the encoding is measured. If the measured value exceeds an upper limit, re-encoding is performed in the PCM mode.
However, in the above conventional technique, even when the amount of generated code can be reduced in comparison with the re-encoding in the PCM mode, such a possibility is disregarded.
Furthermore, the arithmetic coding method employed in H.264 has a feature such that the amount of code cannot be instantaneously measured, and thus a processing delay occurs in a hardware device which executes a pipeline operation.
In light of the above circumstances, an object of the present invention is to provide a novel image encoding technique which does not require re-encoding or encoding corresponding to two or more encoding modes, and implements an encoding whose amount of generated code does not exceed an upper limit without awaiting a measured result of the amount of generated code.
Means for Solving the Problem
In order to achieve the object, the present invention provides a video encoding apparatus for applying orthogonal transformation to a prediction error signal between a video signal of an encoding target area and a predicted signal for the video signal, and quantizing an obtained orthogonal transformation coefficient by using a preset quantization step size so as to encode the coefficient. The apparatus comprises:
  • (1) a computation device that computes a prediction error power which is a power of the prediction error signal;
  • (2) a determination device that receives the prediction error power computed by the computation device, the preset quantization step size to be used in the relevant encoding, and an upper limit of an amount of code generated for the encoding target area, and determines whether or not an amount of code generated when performing quantization using the quantization step size to be used in the encoding exceeds the upper limit; and
  • (3) a change device that changes an encoding process based on a result of the determination by the determination device,
    wherein the determination device computes a permissive power for the prediction error power computed by the computation device, based on the upper limit of the amount of generated code and the quantization step size to be used in the encoding, and compares the permissive power with the prediction error power computed by the computation device so as to determine whether or not the amount of code generated when performing the quantization step size to be used in the encoding exceeds the upper limit.
The above-described processing devices can also be implemented by a computer program. Such a computer program may be provided by storing it in an appropriate computer-readable storage medium, or by means of a network, and can be installed and operate on a control device such as a CPU so as to implement the present invention.
Generally, amount G of generated code and quantization step size Q have the following relationship:
G=X/Q
where X is a value depending on the input signal.
In addition, for the same quantization step size Q, there is a correlation between the amount G of generated code and power D of the input signal. Therefore, in the selection of the prediction mode used in the encoding, a mode for minimizing the prediction error power is selected.
In accordance with the above relationships, an approximate amount of generated code can be estimated.
In consideration of the above, a prediction error power which is a power of the prediction error signal (as an encoding target) is computed. Based on the computed prediction error power and the quantization step size to be used in the encoding, an amount of code generated when performing the quantization using the quantization step size to be used in the encoding is estimated. The estimated value is compared with the relevant upper limit of the amount of generated code, so that it can be determined whether or not the amount of code generated when performing quantization using the quantization step size to be used in the encoding exceeds the upper limit.
In the above determination process, the amount of generated code is directly estimated. However, the determination process is equivalent to a process for determining whether or not the prediction error power is within a permissive power range defined based on the upper limit of the amount of generated code.
Therefore, in the video encoding apparatus of the present invention, a permissive power for the prediction error power is computed based on the upper limit of the amount of generated code and the quantization step size to be used in the encoding, and the permissive power is compared with the computed prediction error power so as to determine whether or not the amount of code generated when performing the quantization using the quantization step size to be used in the encoding exceeds the upper limit.
The estimated value for the amount of generated code or the permissive power for the prediction error power, which is used in the determination process, can be easily computed by means of a function or a table.
That is, it is possible to estimate the amount of generated code by setting variables of a function, which are the prediction error power and the quantization step size, to the values of the prediction error power and the quantization step size, where the value of the function is the relevant amount of generated code. It is also possible to estimate the amount of generated code by referring to a table in which a relationship between data values of the prediction error power, the quantization step size, and the relevant amount of generated code is defined.
It is also possible to compute the permissive power for the prediction error power by setting variables of a function, which are the upper limit of the amount of generated code and the quantization step size, to the values of the upper limit and the quantization step size, where the value of the function is the permissive power for the prediction error power. It is also possible to compute the permissive power for the prediction error power by referring to a table in which a relationship between data values of the upper limit of the amount of generated code, the quantization step size, and the permissive power for the prediction error power is defined.
Strictly speaking, different encoding modes (prediction modes) have different overhead amounts of code or the like, and such a function or look-up table depends on the encoding mode. Therefore, it is preferable that such a function or look-up table is provided for each encoding mode, and one suitable for the encoding mode of the encoding target area is selected and used.
If it is determined by the above determination process that the amount of code generated when performing quantization using the quantization step size to be used in the encoding exceeds the upper limit, then (i) in a first example, a quantized value of the orthogonal transformation coefficient may not be encoded, but the video signal may be encoded without quantizing the video signal; and (ii) in a second example, a quantization step size may be obtained, which is computed based on the prediction error power and the upper limit of the amount of generated code and implements generation of the amount of code which does not exceed the upper limit, and the quantization step size may be switched from the quantization step size to be used in the encoding to the obtained quantization step size.
The computation of the quantization step size used in the above switching operation is implemented using an inverse function of the function which is used for the above-described estimation of the amount of generated code. Therefore, also in this case, the relevant quantization step size can be easily computed using a function or a table.
That is, it is possible to compute the quantization step size which implements generation of the amount of code which does not exceed the upper limit, by setting variables of a function, which are the prediction error power and the upper limit of the amount of generated code, to the values of the prediction error power and the upper limit, where the value of the function is the quantization step size which implements the generation of the amount of code which does not exceed the upper limit.
It is also possible to compute the quantization step size which implements generation of the amount of code which does not exceed the upper limit of the amount of generated code, by referring to a table in which a relationship between data values of the prediction error power, the upper limit, and the quantization step size which implements the generation of the amount of code which does not exceed the upper limit is defined.
Strictly speaking, different encoding modes (prediction modes) have different overhead amounts of code or the like, and such a function or look-up table depends on the encoding mode. Therefore, it is preferable that such a function or look-up table is provided for each encoding mode, and one suitable for the encoding mode of the encoding target area is selected and used.
EFFECT OF THE INVENTION
As described above, the present invention can be applied to an apparatus for applying orthogonal transformation to a prediction error signal between a video signal of an encoding target area and a predicted signal for the video signal, and quantizing an obtained orthogonal transformation coefficient by using a quantization step size so as to encode the coefficient. The present invention can implement encoding which generates codes less than an upper limit amount of code, without measuring the amount of generated code. Therefore, the present invention does not require re-encoding or encoding which handles two or more encoding modes and can implement the encoding which generates codes less than the upper limit amount of code.
Additionally, As the present invention can implement an encoding whose amount of generated code does not exceed an upper limit without awaiting a measured result of the amount of generated code, no processing delay occurs in a hardware device which executes a pipeline operation.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 is a diagram showing a video encoding apparatus as an embodiment of the present invention.
FIG. 2 is a diagram showing an example structure of the code amount estimator in the embodiment.
FIG. 3 is a diagram showing an example structure of the quantization step size computation unit in the embodiment.
FIG. 4 is a flowchart executed by the video encoding apparatus of the embodiment.
FIG. 5A is also a flowchart executed by the video encoding apparatus.
FIG. 5B is also a flowchart executed by the video encoding apparatus.
FIG. 6 is a diagram showing another example structure of the code amount estimator in the embodiment.
FIG. 7 is a flowchart executed by the video encoding apparatus employing the structure in FIG. 6.
FIG. 8A is also a flowchart executed by the video encoding apparatus.
FIG. 8B is also a flowchart executed by the video encoding apparatus.
FIG. 9 is a diagram showing a video encoding apparatus as another embodiment of the present invention.
FIG. 10 is a flowchart executed by the video encoding apparatus of the embodiment.
FIG. 11 is another flowchart executed by the video encoding apparatus of the embodiment.
FIG. 12 is a diagram showing an example structure of the generated code amount estimator.
FIG. 13 is a diagram showing another example structure of the generated code amount estimator.
FIG. 14 is a diagram showing an example structure of the permissive prediction error power computation unit.
FIG. 15 is a diagram showing another example structure of the permissive prediction error power computation unit.
FIG. 16 is a diagram showing an example structure of the quantization step size computation unit.
FIG. 17 is a diagram showing another example structure of the quantization step size computation unit.
FIG. 18 is a flowchart explaining a conventional technique.
 REFERENCE SYMBOLS
  • 10 structural part as an H.264 video encoding apparatus based on H.264
  • 20 code amount estimator
  • 21 quantization step size computation unit
  • 22 selector switch
  • 200 prediction error power computation unit
  • 201 generated code amount estimator
  • 202 code amount comparator
  • 210 prediction error power computation unit
  • 211 minimum quantization step size computation unit
BEST MODE FOR CARRYING OUT THE INVENTION
Below, the present invention will be explained in detail in accordance with embodiments thereof.
FIG. 1 shows a video encoding apparatus as an embodiment of the present invention.
In FIG. 1, reference numeral 10 indicates a structural part (surrounded by a dotted line) as an H.264 video encoding apparatus based on H.264, reference numeral 20 indicates a code amount estimator provided for implementing the present invention, reference numeral 21 indicates a quantization step size computation unit provided for implementing the present invention, and reference numeral 22 indicates a selector switch for implementing the present invention.
Similar to conventional video encoding apparatuses based on H.264, the part 10 as the H.264 video encoding apparatus includes a motion detector 100, a motion compensator 101, a frame memory 102, an interframe prediction mode determination unit 103, an intraframe prediction mode determination unit 104, a selector switch 105, a subtractor 106, an orthogonal transformer 107, a quantizer 108, a quantization controller 109, an inverse quantizer 110, an inverse orthogonal transformer 111, an adder 112, a loop filter 113, and an information source encoder 114. After the subtracter 106 generates a prediction error signal between a video signal of an encoding target macroblock and a predicted signal thereof, the orthogonal transformer 107 subjects the generated prediction error signal to orthogonal transformation. In accordance with the quantization step size set by the quantization controller 109, the quantizer 108 quantizes orthogonal transformation coefficients obtained by the orthogonal transformation. The information source encoder 114 subjects the quantized values to entropy encoding so as to encode the video signal.
FIG. 2 shows an example structure of the code amount estimator 20.
The code amount estimator 20 receives an upper limit value of the amount of code generated for the relevant macroblock (i.e., an upper limit amount of code), the prediction error signal generated by the subtractor 106, and the quantization step size set by the quantization controller 109, and includes a prediction error (electric) power computation unit 200, a generated code amount estimator 201, and a code amount comparator 202.
The prediction error power computation unit 200 computes a prediction error power, which is a power of the prediction error signal generated by the subtractor 106.
Based on the prediction error power computed by the prediction error power computation unit 200 and the quantization step size set by the quantization controller 109, the generated code amount estimator 201 estimates an amount of code generated when quantizing the encoding target macroblock by the relevant quantization step size.
The code amount comparator 202 compares the estimated amount of generated code obtained by the generated code amount estimator 201 with the upper limit (defined in H.264) for the amount of code generated for the macroblock. If the estimated amount of generated code obtained by the generated code amount estimator 201 is greater than the upper limit for the amount of code generated for the macroblock, the code amount comparator 202 directs the selector switch 22 to switch the quantization step size supplied to the quantizer 108 from the quantization step size set by the quantization controller 109 to a quantization step size computed by the quantization step size computation unit 21. In contrast, if the estimated amount of generated code obtained by the generated code amount estimator 201 is smaller than or equal to the upper limit for the amount of code generated for the macroblock, the code amount comparator 202 directs the selector switch 22 to directly use the quantization step size set by the quantization controller 109 as the quantization step size supplied to the quantizer 108.
FIG. 3 shows an example structure of the quantization step size computation unit 21.
The quantization step size computation unit 21 receives the upper limit value of the amount of code generated for the relevant macroblock (i.e., the upper limit amount of code) and the prediction error signal generated by the subtractor 106, and includes a prediction error (electric) power computation unit 210 and a minimum quantization step size computation unit 211.
The prediction error power computation unit 210 computes a prediction error power, which is a power of the prediction error signal generated by the subtractor 106.
Based on the prediction error power computed by the prediction error power computation unit 210 and the upper limit of the amount of code generated for the macroblock, the minimum quantization step size computation unit 211 computes a quantization step size for implementing code amount generation which does not exceed the upper limit (i.e., a minimum quantization step size).
FIGS. 4 to 5B show flowcharts executed by the video encoding apparatus of the present embodiment.
Based on the flowcharts, the operation of the video encoding apparatus in the present embodiment will be explained in detail.
As shown in the flowchart of FIG. 4, in the video encoding apparatus of the present embodiment, the encoding mode is determined in the first step S10. In the next step S11, the amount of code generated when performing the encoding by using the currently-set quantization step size is estimated.
In the next step S12, it is determined whether or not the estimated amount of code generated for the relevant macroblock is greater than the upper limit defined therefor. If it is determined that the estimated amount is greater than the upper limit, the operation proceeds to step S13, where the quantization step size is changed. In the next step S14, encoding is performed using the newly-set quantization step size.
If it is determined in the determination of step S12 that the estimated amount is smaller than or equal to the upper limit, the operation directly proceeds to step S14 by skipping step S13, and encoding is performed using the currently-set quantization step size.
FIG. 5A is a flowchart of the process of estimating the amount of generated code in step S11, and FIG. 5B is a flowchart of the process of changing the quantization step size in step S13.
As shown in the flowchart of FIG. 5A, in step S11, the prediction error signal is received so as to compute the prediction error power (see first step S110), and in the next step S111, the currently-set quantization step size is received. In the next step S112, based on the computed prediction error power and the received quantization step size, the amount of code generated when performing the encoding by using the currently-set quantization step size is estimated.
In the above-explained step S13, as shown in the flowchart of FIG. 5B, the upper limit value of the amount of code generated for the relevant macroblock (i.e., the upper limit amount of code) is received in the first step S130, and in the next step S131, the prediction error signal is received so as to compute the prediction error power. In the next step S132, based on the received upper limit of the amount of generated code and the computed prediction error power, a quantization step size for implementing the code amount generation which does not exceed the upper limit is computed. In the next step S133, in accordance with the computed quantization step size, the quantization step size used in the relevant encoding is changed.
Accordingly, the video encoding apparatus shown in FIG. 1 estimates the amount of code generated when performing the encoding using the currently-set quantization step size, based on a relationship between the amount of generated code and the quantization step size. If the estimated value is greater than the upper limit for the amount of code generated for the relevant macroblock, a quantization step size for implementing the code amount generation which does not exceed the upper limit is computed, and the quantization step size used in the encoding is changed to the computed value.
Therefore, in accordance with the video encoding apparatus of the present embodiment, re-encoding or encoding corresponding to two or more encoding modes is unnecessary, and an encoding whose amount of generated code does not exceed an upper limit can be implemented without awaiting a measured result of the amount of generated code.
FIG. 6 shows another example structure of the code amount estimator 20.
When employing the shown structure, the code amount estimator 20 receives an upper limit value of the amount of code generated for the relevant macroblock (i.e., an upper limit amount of code), the prediction error signal generated by the subtractor 106, and the quantization step size set by the quantization controller 109, and includes a prediction error power computation unit 200, a permissive prediction error power computation unit 203, and a prediction error comparator 204.
The prediction error power computation unit 200 computes a prediction error power, which is a power of the prediction error signal generated by the subtractor 106.
The permissive prediction error power computation unit 203 computes a permissive power of the prediction error power (i.e., permissive prediction error power) based on the upper limit of the amount of code generated for the macroblock and the quantization step size set by the quantization controller 109.
The prediction error comparator 204 compares the prediction error power computed by the prediction error power computation unit 200 with the permissive prediction error power computed by the permissive prediction error power computation unit 203. If the prediction error power computed by the prediction error power computation unit 200 is larger than the permissive prediction error power computed by the permissive prediction error power computation unit 203, the prediction error comparator 204 directs the selector switch 22 to switch the quantization step size supplied to the quantizer 108 from the quantization step size set by the quantization controller 109 to a quantization step size computed by the quantization step size computation unit 21. In contrast, if the prediction error power computed by the prediction error power computation unit 200 is smaller than or equal to the permissive prediction error power computed by the permissive prediction error power computation unit 203, the prediction error comparator 204 directs the selector switch 22 to directly use the quantization step size set by the quantization controller 109 as the quantization step size supplied to the quantizer 108.
FIGS. 7 to 8B show flowcharts executed by the video encoding apparatus of the present embodiment when the code amount estimator 20 has the structure as shown in FIG. 6.
Based on the flowcharts, the operation of the video encoding apparatus in this case will be explained in detail.
As shown in the flowchart of FIG. 7, when the code amount estimator 20 has the structure as shown in FIG. 6, the video encoding apparatus of the present embodiment determines the encoding mode in the first step S20. In the next step S21, the permissive prediction error power as a permissive power of the prediction error power is computed.
In the next step S22, it is determined whether or not the prediction error power is larger than the permissive prediction error power. If it is determined that the prediction error power is larger than the permissive power, the operation proceeds to step S23, where the quantization step size is changed. In the next step S24, encoding is performed using the newly-set quantization step size.
Although it is not shown in the flowchart of FIG. 7, when computing the permissive prediction error power in step S21, the prediction error power used in step S22 is also computed.
If it is determined in the determination of step S22 that the prediction error power is smaller than or equal to the permissive prediction error power, the operation directly proceeds to step S24 by skipping step S23, and encoding is performed using the currently-set quantization step size.
FIG. 8A is a flowchart of the process of computing the permissive prediction error power in step S21, and FIG. 8B is a flowchart of the process of changing the quantization step size in step S23.
As shown in the flowchart of FIG. 8A, in step S21, the prediction error signal is received so as to compute the prediction error power (see first step S210), and in the next step S211, the upper limit of the amount of code generated for the macroblock (i.e., the upper limit amount of code) is received. In the next step S212, the currently-set quantization step size is received, and in the next step S213, the permissive prediction error power is computed based on the received upper limit of the amount of generated code and the received quantization step size.
In the above-explained step S23, as shown in the flowchart of FIG. 8B, the upper limit value of the amount of code generated for the macroblock (i.e., the upper limit amount of code) is received in the first step S230, and in the next step S231, the prediction error signal is received so as to compute the prediction error power. In the next step S232, based on the received upper limit of the amount of generated code and the computed prediction error power, a quantization step size for implementing the code amount generation which does not exceed the upper limit is computed. In the next step S233, in accordance with the computed quantization step size, the quantization step size used in the relevant encoding is changed.
Accordingly, when the code amount estimator 20 has the structure shown in FIG. 6, the video encoding apparatus shown in FIG. 1 determines based on the permissive power of the prediction error power, which is derived by the upper limit of the amount of generated code and the quantization step size, whether or not the amount of code generated when performing the encoding by using the currently-set quantization step size exceeds the upper limit. If the relevant amount of generated code is greater than the upper limit, a quantization step size for implementing the code amount generation which does not exceed the upper limit is computed, and the quantization step size used in the encoding is changed to the computed value.
Therefore, in accordance with the video encoding apparatus of the present embodiment, re-encoding or encoding corresponding to two or more encoding modes is unnecessary, and an encoding whose amount of generated code does not exceed an upper limit can be implemented without awaiting a measured result of the amount of generated code.
FIG. 9 shows another embodiment of the video encoding apparatus in accordance with the present invention. In FIG. 9, parts identical to those explained in FIG. 1 are given identical reference numerals.
In FIG. 9, reference numeral 10 indicates a structural part (surrounded by a dotted line) as an H.264 video encoding apparatus based on H.264, reference numeral 30 indicates a code amount estimator provided for implementing the present invention, reference numeral 31 indicates a PCM encoder provided for implementing the present invention, and reference numeral 32 indicates a selector switch for implementing the present invention.
The PCM encoder 31 subjects the relevant video signal as an encoding target to PCM encoding, without performing quantization, and outputs the encoded data via the selector switch 32 to the information source encoder 114.
The code amount estimator 30 has a basic structure identical to that of the code amount estimator 20 in the embodiment shown in FIG. 1, and may have a structure shown in FIG. 2. When having the structure shown in FIG. 2, if the estimated value of the amount of generated code is greater than the upper limit of the amount of generated code, the code amount estimator 30 directs the selector switch 32 to supply the signal output from the PCM encoder 31 to the information source encoder 114. In contrast, if the estimated value of the amount of generated code is smaller than or equal to the upper limit thereof, the code amount estimator 30 directs the selector switch 32 to supply the signal output from the quantizer 108 to the information source encoder 114.
That is, as shown in a flowchart of FIG. 10, when the code amount estimator 30 has the structure shown in FIG. 2, the video encoding apparatus of the present embodiment first determines the encoding mode (see the first step S30). In the next step S31, the amount of code generated when performing the encoding using the currently set quantization step size is estimated. In the next step S32, it is determined whether or not the estimated amount of generated code is greater than the upper limit for the relevant amount of generated code. If it is determined that the estimated amount is greater than the upper limit, the operation proceeds to step S33, where the PCM encoding is executed. If it is determined that the estimated amount is smaller than or equal to the upper limit, the operation proceeds to step S34, where the ordinary encoding is executed.
The code amount estimator 30 may have the structure shown in FIG. 6. In such a case, if the prediction error power is higher than a permissive prediction error power, the code amount estimator 30 directs the selector switch 32 to supply a signal output from the PCM encoder 31 to the information source encoder 114. On the contrary, if the prediction error power is lower than or equal to the permissive prediction error power, the code amount estimator 30 directs the selector switch 32 to supply a signal output from the quantizer 108 to the information source encoder 114.
That is, as shown in a flowchart of FIG. 11, when the code amount estimator 30 has the structure shown in FIG. 6, the video encoding apparatus of the present embodiment first determines the encoding mode (see the first step S40). In the next step S41, a permissive prediction error power, as the permissive power for the prediction error power, is computed. In the next step S42, it is determined whether or not the prediction error power is higher than the permissive prediction error power. If it is deter mined that the prediction error power is higher than the permissive power, the operation proceeds to step S43, where the PCM encoding is executed. If it is determined that the prediction error power is lower than or equal to the permissive power, the operation proceeds to step S44, where the ordinary encoding is executed.
As described above, the video encoding apparatus of the present embodiment (see FIG. 9) determines whether or not the amount of code generated when encoding is executed using the currently set quantization step size exceeds an upper limit defined therefore. If the amount of generated code exceeds the upper limit, PCM encoding is executed without performing quantization.
Specifically, the code amount estimator 201 shown in FIG. 2, the permissive prediction error power computation unit 203 shown in FIG. 6, and the minimum quantization step size computation unit 211 shown in FIG. 3 each can be implemented using a function or a look-up table.
Strictly speaking, different prediction modes (encoding modes) have different overhead amounts of code or the like, and such a function or look-up table depends on the prediction mode. Therefore, it is preferable that such a function or look-up table is provided for each prediction mode, and one suitable for the prediction mode of the encoding target macroblock is selected and used.
That is, in a preferable example shown in FIG. 12, the generated code amount estimator 201 in FIG. 2 has a plurality of functions (function 1 to function N) corresponding to a plurality of prediction modes, and one of the functions which is suitable for the prediction mode of the encoding target macroblock is selected and used.
In another preferable example shown in FIG. 13, the generated code amount estimator 201 in FIG. 2 has a plurality of look-up tables (LT1 to LTN) corresponding to a plurality of prediction modes, and one of the look-up tables which is suitable for the prediction mode of the encoding target macroblock is selected and used.
In another preferable example shown in FIG. 14, the permissive prediction error power computation unit 203 in FIG. 6 has a plurality of functions (function 1 to function N) corresponding to a plurality of prediction modes, and one of the functions which is suitable for the prediction mode of the encoding target macroblock is selected and used.
In another preferable example shown in FIG. 15, the permissive prediction error power computation unit 203 in FIG. 6 has a plurality of look-up tables (LT1 to LTN) corresponding to a plurality of prediction modes, and one of the look-up tables which is suitable for the prediction mode of the encoding target macroblock is selected and used.
In another preferable example shown in FIG. 16, the minimum quantization step size computation unit 211 shown in FIG. 3 has a plurality of functions (function 1 to function N) corresponding to a plurality of prediction modes, and one of the functions which is suitable for the prediction mode of the encoding target macroblock is selected and used.
In another preferable example shown in FIG. 17, the minimum quantization step size computation unit 211 shown in FIG. 3 has a plurality of look-up tables (LT1 to LTN) corresponding to a plurality of prediction modes, and one of the look-up tables which is suitable for the prediction mode of the encoding target macroblock is selected and used.
INDUSTRIAL APPLICABILITY
The present invention can be applied to a video encoding apparatus for applying orthogonal transformation to a prediction error signal between a video signal of an encoding target area and a predicted signal for the video signal, and quantizing an obtained orthogonal transformation coefficient by using a quantization step size so as to encode the coefficient. The present invention does not require re-encoding or encoding which handles two or more encoding modes and can implement encoding which generates codes less than an upper limit amount of code, without awaiting a measured result of the amount of generated code.

Claims (5)

The invention claimed is:
1. A video encoding apparatus for applying orthogonal transformation to a prediction error signal between a video signal of an encoding target area and a predicted signal for the video signal, and quantizing an obtained orthogonal transformation coefficient by using a preset quantization step size so as to encode the coefficient, the apparatus comprising:
a determination circuit that receives information indicative of a prediction error power, the preset quantization step size, and an upper limit of an amount of code generated for the encoding target area, and determines whether or not an amount of code generated when performing quantization using the preset quantization step size exceeds the upper limit; and
a change circuit that changes an encoding process based on a result of the determination by the determination circuit,
wherein the determination circuit applies a permissive power for the prediction error power based on the upper limit and the preset quantization step size, and compares the permissive power with the prediction error power so as to determine whether or not the amount of code generated when performing the quantization using the preset quantization step size exceeds the upper limit.
2. The video encoding apparatus in accordance with claim 1, wherein:
the determination circuit applies the permissive power for the prediction error power by setting variables of a function, which are the upper limit and the quantization step size, to the values of the upper limit and the quantization step size, where the value of the function is the permissive power.
3. The video encoding apparatus in accordance with claim 1, wherein:
the determination circuit applies the permissive power for the prediction error power by referring to a table in which a relationship between data values of the upper limit, the quantization step size, and the permissive power is defined.
4. A non-transitory computer-readable storage medium which stores a video encoding program by which a computer executes an operation for implementing the video encoding apparatus in accordance with claim 1.
5. A video encoding method for applying orthogonal transformation to a prediction error signal between a video signal of an encoding target area and a predicted signal for the video signal, and quantizing an obtained orthogonal transformation coefficient by using a preset quantization step size so as to encode the coefficient, the method comprising:
a determination step that receives information indicative of a prediction error power, the preset quantization step size, and an upper limit of an amount of code generated for the encoding target area, and determines whether or not an amount of code generated when performing quantization using the preset quantization step size exceeds the upper limit; and
a change step that changes an encoding process based on a result of the determination,
wherein the determination step applies a permissive power for the prediction error power based on the upper limit and the preset quantization step size, and compares the permissive power with the prediction error power so as to determine whether or not the amount of code generated when performing the quantization using the preset quantization step size exceeds the upper limit.
US12/667,650 2007-07-17 2008-07-10 Video encoding apparatus and method, video encoding program, and storage medium which stores the program Active 2032-05-07 US9271008B2 (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
JP2007-185374 2007-07-17
JP2007185374 2007-07-17
PCT/JP2008/062484 WO2009011279A1 (en) 2007-07-17 2008-07-10 Video encoding device and method, video encoding program, and recording medium containing the program

Publications (2)

Publication Number Publication Date
US20110013693A1 US20110013693A1 (en) 2011-01-20
US9271008B2 true US9271008B2 (en) 2016-02-23

Family

ID=40259615

Family Applications (1)

Application Number Title Priority Date Filing Date
US12/667,650 Active 2032-05-07 US9271008B2 (en) 2007-07-17 2008-07-10 Video encoding apparatus and method, video encoding program, and storage medium which stores the program

Country Status (10)

Country Link
US (1) US9271008B2 (en)
EP (1) EP2169962A4 (en)
JP (1) JP4987080B2 (en)
KR (1) KR101103698B1 (en)
CN (1) CN101743754B (en)
BR (1) BRPI0813491B1 (en)
CA (1) CA2692782C (en)
RU (1) RU2444857C2 (en)
TW (1) TWI386065B (en)
WO (1) WO2009011279A1 (en)

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3567852B1 (en) * 2009-03-23 2022-12-21 Ntt Docomo, Inc. Image predictive decoding device and image predictive decoding method
CN101867799B (en) * 2009-04-17 2011-11-16 北京大学 Video frame processing method and video encoder
JP5850214B2 (en) 2011-01-11 2016-02-03 ソニー株式会社 Image processing apparatus and method, program, and recording medium
UA109312C2 (en) 2011-03-04 2015-08-10 PULSE-CODE MODULATION WITH QUANTITATION FOR CODING VIDEO INFORMATION
JP5873290B2 (en) * 2011-10-26 2016-03-01 キヤノン株式会社 Encoder
US8923388B2 (en) * 2011-11-21 2014-12-30 Texas Instruments Incorporated Early stage slice cap decision in video coding
JP5850272B2 (en) * 2014-01-10 2016-02-03 ソニー株式会社 Image processing apparatus and method, program, and recording medium
JP6332502B2 (en) * 2017-03-03 2018-05-30 ソニー株式会社 Image processing apparatus and method, program, and recording medium
JP6299901B2 (en) * 2017-03-03 2018-03-28 ソニー株式会社 Image processing apparatus and method, program, and recording medium
TWI754206B (en) * 2020-01-10 2022-02-01 祥碩科技股份有限公司 Data storage system, data storage device and management method thereof
US20230078190A1 (en) * 2021-09-15 2023-03-16 Synaptics Incorporated Image compression method and apparatus

Citations (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0998427A (en) 1995-09-29 1997-04-08 Toshiba Corp Dynamic image encoding device
RU2123769C1 (en) 1993-04-09 1998-12-20 Сони Корпорейшн Method and device for encoding images and information medium for storing images
JPH11331850A (en) 1998-03-16 1999-11-30 Mitsubishi Electric Corp Dynamic image coding system
US6404933B1 (en) * 1997-06-02 2002-06-11 Nec Corporation Image encoding method and apparatus thereof
US20020136297A1 (en) 1998-03-16 2002-09-26 Toshiaki Shimada Moving picture encoding system
RU2217882C2 (en) 1997-08-12 2003-11-27 Томсон Конзьюмер Электроникс, Инк. Device for compressing pixel blocks in image processing system
CA2491522A1 (en) 2002-07-01 2004-01-08 E G Technology Inc. Efficient compression and transport of video over a network
US20050036698A1 (en) 2003-08-12 2005-02-17 Samsung Electronics Co., Ltd. Method of controlling encoding rate, method of transmitting video data, encoding rate controller for video encoder, and video data transmission system using the encoding rate controller
JP2005086249A (en) 2003-09-04 2005-03-31 Toshiba Corp Dynamic image coding method and dynamic image coding apparatus
US6961375B1 (en) * 1997-02-06 2005-11-01 Sony Corporation Picture coding device and method, picture transmitting device and method and recording medium
US6963608B1 (en) 1998-10-02 2005-11-08 General Instrument Corporation Method and apparatus for providing rate control in a video encoder
US20070133892A1 (en) 2005-12-09 2007-06-14 Takuma Chiba Image coding device, method and computer program
JP2007158430A (en) 2005-11-30 2007-06-21 Matsushita Electric Ind Co Ltd Image information encoder
KR20070075585A (en) 2006-01-13 2007-07-24 부산대학교 산학협력단 Method and apparatus for approximating distortion in video compression
RU2322770C2 (en) 2002-04-23 2008-04-20 Нокиа Корпорейшн Method and device for indication of quantizer parameters in video encoding system

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3416649B2 (en) * 1998-01-21 2003-06-16 松下電器産業株式会社 Variable length coding device
CN1194544C (en) * 2003-04-25 2005-03-23 北京工业大学 Video encoding method based on prediction time and space domain conerent movement vectors

Patent Citations (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
RU2123769C1 (en) 1993-04-09 1998-12-20 Сони Корпорейшн Method and device for encoding images and information medium for storing images
JPH0998427A (en) 1995-09-29 1997-04-08 Toshiba Corp Dynamic image encoding device
US6961375B1 (en) * 1997-02-06 2005-11-01 Sony Corporation Picture coding device and method, picture transmitting device and method and recording medium
US6404933B1 (en) * 1997-06-02 2002-06-11 Nec Corporation Image encoding method and apparatus thereof
RU2217882C2 (en) 1997-08-12 2003-11-27 Томсон Конзьюмер Электроникс, Инк. Device for compressing pixel blocks in image processing system
JPH11331850A (en) 1998-03-16 1999-11-30 Mitsubishi Electric Corp Dynamic image coding system
US20020136297A1 (en) 1998-03-16 2002-09-26 Toshiaki Shimada Moving picture encoding system
US6963608B1 (en) 1998-10-02 2005-11-08 General Instrument Corporation Method and apparatus for providing rate control in a video encoder
RU2322770C2 (en) 2002-04-23 2008-04-20 Нокиа Корпорейшн Method and device for indication of quantizer parameters in video encoding system
CA2491522A1 (en) 2002-07-01 2004-01-08 E G Technology Inc. Efficient compression and transport of video over a network
US20050036698A1 (en) 2003-08-12 2005-02-17 Samsung Electronics Co., Ltd. Method of controlling encoding rate, method of transmitting video data, encoding rate controller for video encoder, and video data transmission system using the encoding rate controller
JP2005086249A (en) 2003-09-04 2005-03-31 Toshiba Corp Dynamic image coding method and dynamic image coding apparatus
JP2007158430A (en) 2005-11-30 2007-06-21 Matsushita Electric Ind Co Ltd Image information encoder
JP2007166039A (en) 2005-12-09 2007-06-28 Matsushita Electric Ind Co Ltd Image coding equipment
US20070133892A1 (en) 2005-12-09 2007-06-14 Takuma Chiba Image coding device, method and computer program
KR20070075585A (en) 2006-01-13 2007-07-24 부산대학교 산학협력단 Method and apparatus for approximating distortion in video compression

Non-Patent Citations (7)

* Cited by examiner, † Cited by third party
Title
Hoang, Dzung T., et al., "Efficient Cost Measures for Motion Estimation at Low Bit Rates," IEEE Transactions on Circuits and Systems for Video Technology, vol. 8, No. 4, Aug. 1998, pp. 488-500.
ITU-T Rec.H.264, "Advanced video coding for generic audio visual services", pp. 249-256, 2005.
Keesman, Gertjan, et al., "Bit-rate control for MPEG encoders," Signal Processing: Image Communication, vol. 6, No. 6, Feb. 1995, pp. 545-560.
Keiichi Chono, Yuzo Senda, Yoshihiro Miyamoto, "A PCM Coding Method using Decoded Images for obeying the upper limit on the number of bits of MB in H.264 encoding", pp. 119-120, The Proceedings of the 21th Picture Coding Symposium of Japan, 2006.
Office Action, European Patent Application No. 08 791 042.8, Jan. 11, 2013.
Sullivan, Gary, "Seven Steps Toward a More Robust Codec Design," Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG (ISO/IEC JTC1/SC29/WG11 and ITU-T SG16 Q.6), Doc. JVT-C117, 3rd Meeting: Fairfax, Virginia, May 6-10, 2002.
Summons to attend oral proceedings pursuant to Rule 115(1) EPC, European Patent Application No. 08 791 042.8, Apr. 10, 2015.

Also Published As

Publication number Publication date
CA2692782C (en) 2013-09-24
JPWO2009011279A1 (en) 2010-09-24
EP2169962A1 (en) 2010-03-31
JP4987080B2 (en) 2012-07-25
TW200908752A (en) 2009-02-16
RU2009148766A (en) 2011-07-10
BRPI0813491B1 (en) 2020-10-06
EP2169962A4 (en) 2012-04-11
US20110013693A1 (en) 2011-01-20
RU2444857C2 (en) 2012-03-10
WO2009011279A1 (en) 2009-01-22
KR101103698B1 (en) 2012-01-11
BRPI0813491A2 (en) 2015-08-18
CN101743754B (en) 2012-04-18
CA2692782A1 (en) 2009-01-22
CN101743754A (en) 2010-06-16
TWI386065B (en) 2013-02-11
KR20100018033A (en) 2010-02-16

Similar Documents

Publication Publication Date Title
US9271008B2 (en) Video encoding apparatus and method, video encoding program, and storage medium which stores the program
US20060098733A1 (en) Variable-length coding device and method of the same
KR101362590B1 (en) Image processing device and method
US20070081589A1 (en) Adaptive quantization controller and methods thereof
JP2006157881A (en) Variable-length coding device and method of same
JP2004527960A (en) Dynamic complexity prediction and adjustment of MPEG2 decoding process in media processor
JP2006180298A (en) Image encoding control method and its device
KR100708182B1 (en) Rate control apparatus and method in video encoder
JP2011091772A (en) Image encoder
KR20040007818A (en) Method for controlling DCT computational quantity for encoding motion image and apparatus thereof
JP4253276B2 (en) Image coding method
JP4452733B2 (en) Motion evaluation apparatus, method and program for moving picture coding, and recording medium therefor
US20060274832A1 (en) Device for encoding a video data stream
US20060008004A1 (en) Video encoder
KR20050105550A (en) H.263/mpeg video encoder for controlling using average histogram difference formula and its control method
JP2005260576A (en) Image encoding method and device
US20070297517A1 (en) Entropy encoding and decoding apparatuses, and entropy encoding and decoding methods
KR100555750B1 (en) Very low bit rate image coding apparatus and method
KR100778473B1 (en) Bit rate control method
KR100480698B1 (en) MPEG encoding method
KR100286220B1 (en) apparatus and method for adaptative quantizing in image coder
JP2009296328A (en) Encoding picture type determination method, device, program, and recording medium thereof
JP4465529B2 (en) Image coding apparatus and method
JP2004349855A (en) Coder
JP4798652B2 (en) Video rate converter

Legal Events

Date Code Title Description
AS Assignment

Owner name: NIPPON TELEGRAPH AND TELEPHONE CORPORATION, JAPAN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:SHIMIZU, ATSUSHI;NAKAJIMA, YASUYUKI;REEL/FRAME:023729/0736

Effective date: 20091214

STCF Information on status: patent grant

Free format text: PATENTED CASE

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Year of fee payment: 4

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 8TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1552); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Year of fee payment: 8