CA2648125A1 - Method and system for real-time adaptive quantization control - Google Patents
Method and system for real-time adaptive quantization control Download PDFInfo
- Publication number
- CA2648125A1 CA2648125A1 CA002648125A CA2648125A CA2648125A1 CA 2648125 A1 CA2648125 A1 CA 2648125A1 CA 002648125 A CA002648125 A CA 002648125A CA 2648125 A CA2648125 A CA 2648125A CA 2648125 A1 CA2648125 A1 CA 2648125A1
- Authority
- CA
- Canada
- Prior art keywords
- data block
- quantizer scale
- samples
- luminance
- quantizer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/189—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding
- H04N19/192—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding the adaptation method, adaptation tool or adaptation type being iterative or recursive
- H04N19/194—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding the adaptation method, adaptation tool or adaptation type being iterative or recursive involving only two passes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/124—Quantisation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/136—Incoming video signal characteristics or properties
- H04N19/137—Motion inside a coding unit, e.g. average field, frame or block difference
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/136—Incoming video signal characteristics or properties
- H04N19/137—Motion inside a coding unit, e.g. average field, frame or block difference
- H04N19/139—Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/182—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a pixel
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/189—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding
- H04N19/196—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding being specially adapted for the computation of encoding parameters, e.g. by averaging previously computed encoding parameters
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/189—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding
- H04N19/196—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding being specially adapted for the computation of encoding parameters, e.g. by averaging previously computed encoding parameters
- H04N19/197—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding being specially adapted for the computation of encoding parameters, e.g. by averaging previously computed encoding parameters including determination of the initial value of an encoding parameter
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
- H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
Abstract
A system and method for controlling video compression quantization comprising generating a quantizer scale offset based on diagonal frequencies of luminance components of a data block samples, luminance intensity of the samples and motion activity of the data block, adjusting a first quantizer scale using the quantizer scale offset to receive a second quantizer scale and quantizing the data block using the second quantizer scale.
Description
Attomcy Docket No.: P-9927-CA
METHOD AND SYSTEM FOR REAIrTIME ADApTivE QUANTIZATION
CONTROL
BACKGROUND
[001] Digital video compression may be used in a variety of implementations, such as broadcasting, streaming and storage. Some video compressiort algorithms are considered to br: loosy algorithms due to the fact that a ceitain amount of dflta may be lost during the compression process, for e7cample, during the process of quantization. When using such algorithms the bit rate of the processed data may be reduced variably, depending on the entropy of the encoded block.
METHOD AND SYSTEM FOR REAIrTIME ADApTivE QUANTIZATION
CONTROL
BACKGROUND
[001] Digital video compression may be used in a variety of implementations, such as broadcasting, streaming and storage. Some video compressiort algorithms are considered to br: loosy algorithms due to the fact that a ceitain amount of dflta may be lost during the compression process, for e7cample, during the process of quantization. When using such algorithms the bit rate of the processed data may be reduced variably, depending on the entropy of the encoded block.
[002] Quantization is usually a major cause of information loss during video compression process. Control of the quantization process in video encoding may significantly affect the quality of image_ A real-time solution of an adaptive quailtization control in order to enhance the subjective quality of the picture is highly required.
Attorney Docket No.: P-992 7-CA
BR[EF DFSCRIPTION OF THE DRA'VVINGS
Attorney Docket No.: P-992 7-CA
BR[EF DFSCRIPTION OF THE DRA'VVINGS
[003] The subject matter regarded as the invention is particularly pointed out and distinctly claimed in the concluding portion of the speci#ication. The invention, however, both as to or6anization and method of operation, together with objects, features, and advantages thereof, may best be understood by reference to the following detailed description when read with the accompanying drawings in which:
[004] Fig. 1 is a block diagram of an exemplary video encoder according to embodiments ot the present invention;
[005] Figs. 2A and 2B are sub sampling maps of an exeniplary sub-sampled set of macro blocks used in accordance with embodiments of the present invention; and [006] Fig. 3 is a flowchart of a method for performing adaptive quantization control according to enibodiments of the present invention.
[007) It will be appreciated that for simplicity and clarity of illustration, elements shown in the figures have not necessarily been drawn to scale. For example, the dimensions of some of the elements niay be exaggerated relative to other elements for clarity. Further, where considered appropriate, reference numerals may be repeated axnong the figures to indicate corresponding or analogous elements.
Attorney Docket No.: P-9927=CA
DETAILED DESCRIPTION OF DEMONSTRATIVE EMBODIMETS OF THE
PRESENT INVENTION
Attorney Docket No.: P-9927=CA
DETAILED DESCRIPTION OF DEMONSTRATIVE EMBODIMETS OF THE
PRESENT INVENTION
[008] In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the invention. However, it will be understood by those skilled in the art that the present invention may be practiced without these specific details. In other instances, well-known metliods, procedures, and components have not been described in detail so as not to obscure the present invention.
[009] Although embodiments of the invention are not limited in this regard, discussions utilizing terms such as, for exantple, "processing," "computing,"
"calculating,"
"determining," "establishing", "analyzing", "checking", or the likc, may refer to operation(s) and/or process(es) of a computer, a comptiring platform, a computing system, or other electronic computing device, that manipulate and/or transform data represented as physical (e,g,, electronic) quantities within the computer's registers and/or meniories into other data similarly represented as physical quantities within the computer's registers and/or memories or other information storage medium that may store instructions to perfonn operations and/or proeesses.
"calculating,"
"determining," "establishing", "analyzing", "checking", or the likc, may refer to operation(s) and/or process(es) of a computer, a comptiring platform, a computing system, or other electronic computing device, that manipulate and/or transform data represented as physical (e,g,, electronic) quantities within the computer's registers and/or meniories into other data similarly represented as physical quantities within the computer's registers and/or memories or other information storage medium that may store instructions to perfonn operations and/or proeesses.
[0010] Although embodiments of the invention are not limited in this regard, the terms "plurality" and "a plurality" as used berein may include, for example, "rnultiple" or "two or more". The tern-is "plurality" or "a plurality" may be used througliout the specification to describe two or more companents, devices, elements, units, paramcters, or the like.
[0011] Reference is now made to Fig. 1, which is a block diagram of an exemplary video encoder according to embodiments of the present invention. A video encoder 100 may include a frame mr.~mory 101 which may store an original image or franie to be encoded.
Thc image or frame may be divided into macro blocks (MD), each MB may include a group of pixels, for example, by 16x16 pixels. A subtractor 102 may calculate the difference between the pixels of a current frame, stored in franie ntemory 101 and the pixels of a reference frame, storr;d in a reference frame znemory 109- The difference tnay be trattsfornied to the ftequency domain by a Discrete Cosine Transform (DCT) 103 and the transforrned output niay be an array of coefficients representing the energy of the input data by horizontal, vertical and diagonal spatial frequencies. Video encoder 100 may include a quantizer 104 which may quantize the transformed output of DCT 103 and may Attomey Docliet No,: P-9927-CA
be dynamically controlled by a quantization controller 105 as described in detail below according to embadin-ients of the present invention.
[0012J The quantized output signal may be inverse-quantized at an inversr:
quantizer 106 prior to underl;oing an inverse DCT (TDCT) conversion at IDCT block 107. The output from 1DCT 107 is added to the output of a motion estimation/compensation block 110 by an adder 108 and the calculated sum may be stored in reference fraine memory 109. The data stored in referc,~nce frame memory 109 as well as the original image stored in frame memory 101 may both be used as inputs to subtractor 102 and to inotion estimation/compensation block 110_ [0013] Although in the exemplary illustration of Fig. 1, DCT and IDCT are being used, it should be understood to a person skilled in art that the invention is not limited in this respect and according to embodiments of the present invention any other mathematical transform which may map a function from the time domain to the frequency domain may be used.
1'0014'1 The output of quantizer 104 may also be a subject to cntropy coding at entropy coding block 112 and the coded output signal from entropy coding block 112 inay be output through bit buffer 113. Encoder 100 may include a bit rate controller 111 which may receive an input from bit buffer 113 indicating the fullness of bit buffer 113 and may transfer control signals and/or parameters to quantization controller 105.
l,00] 5] According to embodiments of the present invention, quantization controller 105 may receive from bit xate controller 111 at least one parameter, also referred to herein as "quantizer scale" 114 wliich may be used for the quantization of the DCT
transfonned coefficients in quantizer 104_ Quantization controller 105 may control the quantization of the DCT transformed coefficients in quantizer 104 by adapting or changing quantizer scale 1 14 received from bit rate controller 111 according to the coinplexity of the current MB
and providing a new quantizer scale 115 to quantizer 104 as is described in detail below.
[00161 It is k.nown that the human eye is less sensitive to quantization noise at "noisy"
poi'tions of the image, e.g., detailed surfaces, dark portions of a picture and fast moving objects. Accordingly, quantization controller 105 may control the quantization of such portions of a picture to be more coarse and may use saved bits for other portions, for exainple, where noise is very obvious e.g., smooth portions of image, faces.
Quantization Attorney Docket No.: P-9927-CA
controller 105 nray controi the quantization of quanti.zer 104 to be coarse for high spatial activity and less coarse for lower spatial activity by changing the quantizer scale in order to 4nhance the subjective quality of the picture.
[0017] According to embodiments of the present invention, quantization controller 105 may provide a real-time adaptive quantization control to improve image quality.
Quantization controller 105 may include a quantizer enhancement block 120, a diagonal frequency detector (T)FD) 122, a luma level detector 124 and a motion level detector 126.
The frequency detector (DFD) 122, the luma level detector 124 and the motion level dctector 126 may all be coupled to the quantizer enliancement block 120.
Quantizer enhancement block 120 may generate a new quantizer scale 115 based on inputs received 1.'rom DFD 122, luminance (luma) level detector 124 and motion level detector 126.
[0018] According to embodiments of the present invention, DFD 122 may receive MB
data of the cun:ent picture and may apply a diagonal frequency detection algorithm which may classif'y the smoothness level of the MB. The algorithni or function implemented by DFD 122 may include a low complexity caleulation which may accurately detect high amplitudes at the diagonal transformed coefficients, which points block n~oisiness_ The DFD algorithm may be applied on the luminance component (Y) saniples of a eurrent image, stored in faame mcmory 1.01, before performing the actual calculation of the DCT
transform.
[0019] Accordinb to embodiments of the present invention, luma level detector 124 may reeeive M13 data of the cunent picture and may calculate the lumnlance intensity of the input MB, Luma level detector 124 may furtlier map the calculated value to predetermined levels using a conversion table.
[0020] According to embodiments of the present invention, Motion level detector 126 may receive one or more motion vectors from motion estimationlcompensation block 110 and tnay detect the level of motion the current pictures by suiruning the received vectors components.
[0021] Quantizer enhancement 120 may use a weighted function of the diagonal frequency detection algorithm results, received from DFD 120, luminance level caleulations received froni luma level detector 124 and motion level information received from motion level Attonicy Docket No.: P-9927-CA
detector 126 to produce quantizer afflets, and to use these offsets to detennine a new quantizer scale 115.
[0022] Although the present invention is not limited in this respect, the function detecting diagoiual frequencies, implemented by DFD 120 and the function detecting luma intensity, implemented by luma level analyzer 124 may be applied on all the pixels of a MB or ot7 a sub-sampied set of pixels of a MB. One example of a sub-sainpled set of pixels of a MB is shown in figures 2A and 2B. It should be understood to a person skilled in the art that the calculations and functions applied to the sub-sampled set of 1vIB pixels may be applied to any other sub-smnpled set of M8 pixels or to the entire MB, namely, to all the pixels of a MB.
[0023] Reference is now made to Figs. 2A and 213,-which are maps of an exeniplary sub-sampled set of MB pixels used in accordance with embodiments of the present invention.
[0024] Map 200 illustrates an exernplary MB of 16x16 pixels, where each square of the map represent a single pixel. It should be understood to a person skilled in the art that any other size of MB may be used. Embodiments of the present invention may use all pixels of niap 200, or any sub set of pixels of map 200, the sub set of the pixels may be chosen according to the encoder implementation or speed, desired accuracy of the outputs of DFD 122 and luma level detector 124 or according to any other parameter.
For example, encoders which pertonn parallel 4 bytes operation which are well optimized for, for example, DSP implementations may use the sub set of pixels of map 200 indicated by x's.
Any other set of pixels may be used.
[0025] According to embodiments of the present invention, DFD 122 may subtract pairs of pixels to determine horizontai and vertical gradients as follows;
Df'17 = Y-I{Y(i j)-Y(i,j+1)}- {Y(i+1j)-Y(i+lj+l)}l [i]
for- i = 0 to horizontal MB size j= 0: vertical MB size and wliere Y represents the luminance component of the pixel.
[0026] The low complexity calculation of equation [1] may accurately detect high amplitudes at the diagonal transformed coefficients, which may points block noisiness, Attoiiicy Docket No.: P-9927-CA
e.g,, a high diagonal frequencies detection (DFD) result may imply that the MB
may contain some high diagonal frequencies. The result of equation [1] may be transferred to the quantizer i;nhancement block 120 for determining the offset of the quantizer scale.
[0027] It should be understood to one skilled in the art that equation [1 ]
may be applicable in embodiments which use all the pixels of a MB, and that equation [1] may be adapted to suit any other choice of sub set of pixels, for example as described in equations [2]-[10]
that relate to DFD calculation of the sub set of pixels shown in map 250_ [002F] The pixels included in the Dl~'l7 calculation of map 250 are pixels indicated by Latin letters (including x). Equation [1] may be adapted to subtract pairs of pixels of the marked pixels and may be defined as follows:
Y1 = Ya - Ye [2]
Y2=Yb-Yf [3]
Y3=Yc-Yg [4]
Y4=Yd-Yh [5]
2 1 = abs(Y1-Y2) + abs(Y2-Y3) + abs(Y3-Y4) [6]
where: Ya to Yh represent the lurninance component of the pixels marked with a to h.
[0029] Equations [2]-[4] may be used for every group of the 20 sub groups (260-279 ) of the MB each containing 8 pixels, for example, groups 260, 261, 262 and 279.
After equations 2-4 are performed for each group of groups 260-279, the final DFD
value may be calculated as follows:
DFD = T- Zi [7]
for; i I to 20.
[0030] Although in the exeniplaiy illustration of map 250 of Fig. 2B, 20 groups, each containing 8 pixels are shown, it should be understood to a person sldlled in art that the invention is not limite:d in this respect and according to einbodiments of the present iiivention, any suitable groups of a MB containing any number of pixels may be used by Attorney Docket No.: P-9927-C'A
ppp 122 for the DF calculations and equation [1) may be adapted to fit any selection of pixels of a frame.
[0031] According to embodvnents of the present invention, luma level detector 124 may calculate the intensity of the luminance component of the chosen pixels of a MB as follows;
Luma Level = Y_Y(i,j) [8]
for i= 0 to horizontal MB size; and j= 0 to vertical MB size [0032] The result of equation [8] may be transferred ta the quantizer enhancement block 120 for determining the offset of the quantizer scale.
[0033] It should be understood to one skilled in the art tl-iat equation [8]
may be applicable in embodiments which use all the pixels of a MB, and that equation [8] may be adapted to suit any other choice of sub-set of pixels, for example as described in equations [9]-[10]
which relate to luminance level calculation of the sub-set of pixels showla in map 250.
[0034] The pix4ls included in the luminance level calculatiou of map 250 are the pixels marked with Latin letters (including x). Equation [8] may be adapted to calculate thc iiZtensity of the marked pixels by summing the luminance component of the marked pixels and nlay be described as follows:
Ll = (a+U+ c+d)+(e+f+g+h) [9]
where Ll represent the luminance level of group 260, [00351 Equation [9] may be used for every group of the 20 sub groups 260-279, each contaiiiing 8 pixels, for example, sub-group 261 illustrated by the pixels marked i to p.
After equation [9] is perfor-rned for all 20 groups 260-279, the final luminance level value may be calculated as follows:
L.CJMA_LEVEL- ELi [10]
fior_i=1to2Ø
Attomey Docket No.: P-9927-CA
[0036] Although in the exemplary illustration of map 250, groups 260-279, each containing 8 pixels are shown, it should be understood to a person skilled in art that the invention is not limited in this respect and according to embodiments of the present invention, any suitable groups of a MB containing any nurnber of pixels may be used by luma level det:ector 124 for the luminance level calculations and equation [9]-[10] may be adapted to fit any selection of pixels of a fframe.
[0037] According to embodiments of the present invention, motion level detector 126 may receive a plurality of motion vectors from motion estimation/cotnpensation block 110.
Each vector may be represented by an horizontal component (Vx) and a vertical component (Vy). Motion level detector niay sum the absolute values of the motion vectois components of inter pictures as follows:
Motion = 2:jV ixj + Y_IV iy~ [11 ]
for i^ 0 to number of received vectors, [0038] The result of equation [11] may be transferred to quantizer enliancement block 120 for detemlining the offset of the quantizer scale.
[0039:1 According to embodiments of the present invention, quantizer enhancement block 120 may receive the calculated DFD from DFD 122, the luminance level from luma level detector 124 and the absolute motion of a MB froin motion level detector 126.
In addition quantizer enhancement block 120 may receive the original quantizer scale, also refetred to herein as "first" qua.ntizer scale frotn bit rate controller 111. Qtiantizer enhaneentent block 120 may process one or more of tlie inputs and may use a weighting function of flle calculated DpD, luminance level and absolute motion to determine an offset and to usc this offset to generate a new quantizer scale, also referred to herein as "second"
quantizer scale as desciibed below.
(0040] Although the presr.vnt invention is not limited in this respect, the calculated DFD
received from DFD 122 may be further processed by quantizer enhancement block 120, for example, the DFD value mav be converted to another value by performing any mathematical operation r.equircd. The niathematical operation or conversiort may be 9 f Attomoy Docket No.: P-9927-CA
perforrned by, for example, a shifter, an adder, a subtractor or by any other component or block which may convert the received DFD value. The mathematical operation may be deterniined based on parameters related to encoder's operation, such as the required output bit iate, parameters related to encoder implementation, such as for example, DSP, ASIC, FPGA or software impiementation, parameters related to the video compression standard used by the encoder, such as MPEG-2, MPEG-4, H-264, or based on parameters related to Any other encoder's characteristics.
[0041] Although in the description above the mathematical operation on the DFD
value is described as perfornzed by quantizer enhancement block 120, it should be understood to a person skilled in art that the invention is not limited in this respect and according to embodiments of the present invention, any such mathematical operation may be performed in DFD 122 prior to the operation of qua.ntizer enhancement block 120. For example, a shifter to shift the DFD value may be implemented in either DFp 122, quantizer enliancement block 120 or both DFD 122 and quantizer enhanceinent block 120.
(0042] Although the present invention is not limited in this respect, the input received from luma level detector 124 may be further processed by quantizer enhancement block 120, for exaniple, the luminance level value received which indicate the intensity oF'a MB
may be mapped, converted or scaled using one or more conversion tables and/or any other inathematical operation, such as shift operation in order to fit into a desired weighting function. The conversion table and/or mathematical operation txiay be predefned and may be based on parameters related to encoder operation, such as the required output bit rate, parameters related to encoder implementation, such as for eycample, DSP, ASIC.
FPGA or software iniplementation, parameters related to the video compression standard used by the encoder, such as MPBG-2, MPEG-4, H-264, or based on parameters related to any other encoder's characteristics.
[0043] Although in dhe description above rhe conversion table applied on the luma level values is described as a part quantizer enhancement block 120, it should bc understood to a person skilled in art that the invention is not limited in this respeCt and according to embodiinents of the present invention, any conversion table and/or other mathematieal operation niay be peiformed in luminance level detector 124 prior to the operation of quantizer enhancement block120. For example, a predefined conversion table may be lo Attorney Qocket No.: P-9927-CA
included ita either lurna level detector 124, quantizer enhancement block 120 or both luirra level detector 124 and quantizer enhancem.ent block 120.
[0044] Although the present invelition is not limited in tlus respect, the received input from motion level detector 126 may be further processed by quantizer enhancement block 120, for example, the motion level value received may be mapped, converted or scaled using one or more conversion tables andlor any other inathematical operation, such as shift operation in order to fit into a desired weighting fitnction. The conversion table and/or mathematical operation may be predefined and may be based on parameters related to encoder's operation, such as the required output bit rate, parameters related to encoder implenientation, such as for exaniple, DSP, ASIC, FPGA or software implenientation, parameters related to the video compression standard used by the encoder, such as MPBG-2, MPEG-4, H-264, or based on parameters related to any other encoder's characteristics.
[0045j Although in the description above the conversion table applied on the motion level values is described as a part quantizer enhancement block 120, it should be understood to a person skilled in art that the invention is not limited in this respect and according to embodiments of the present invention, any conversion table and/or other mathematical operation may be performed in motion level detector 126 prior to the operation of quantizer enhancement 120. For example, a predefined conversion table may be included in either motion level detector 126, quantizer enhancement block 120 or both motion level detector 126 and quantizer enilancement block 120_ [0046] According to embodiments of the present invention, Quantizer enhancement block 120 inay calculate a quantizer scale offset by a pre-defined weighted tunction as follows;
Quantizer scale offset = (DFD)xW0 + (.Luma_Level)xW1 + (Motion)xW2 [12]
[0047] Equation [12] rnay describe a general form of a weighted srun of products. WO, W 1 and W2 may represent weights detemiined by a plurality of parameters, sucli as for example, parameters related to encoder operation, such as the required output bit rate, paranleters related to encoder iniplementation, such as for example, DSP, ASIC, FPGA or softwar-e implementation, parameters related to the video compression standard used by the encoder, such as MPEG-2, MPEG-4, H-264 or any other parameter. The above parameters ll Attorney Docket No,; P-9927-CA
and/or any other parameter or characteristic related to the encoder may referred to herein as "encoder characteristies". According to other embodiments of the invention, the weighted sum of products may be an adaptive weighted funetion which may be dynamically cliange based on parameters related to characteristies of a video stream to be encoded.
[0048] The general form of equation [12] may relate to the DFD, Luma_i.,eve1 and Motion values as the required values after processing, other embodirnents may use a different weighted sum, for example, such as equation 13 which inay include some or all of the pre-processing or conversions of some or all of the three inputs weighted of general equation 12 for example;
Quantizer scale offset - (DFD >> aO)xWO +(tablel [Lurna Level>>al ])xW 1+
(tablo;2[Motion >> a2])xW2 [13]
where: aO, al, a2 are constant variables determined by the encoder characteristics;
WO, W1, W2 are weights determined by encoder characteristics;
Tablel is a predefined table used for conversion. of luminance level values;
Table2 -is a predefined table used for conversion of motion level values.
">>" represents a right shift;
"table [y] represent the value at an entry "y" in a table, [0049] According to embodiments of the present invention, the quantizer scale offset calculated by equation [12], equation [13] or any other, weighted functior of the DFD, luminance level and motion level inputs, may be applied to the original quantizer scale, received from bit rate controller 111. For example, it may be added, subtracted, shifted or inultiplied to4rom original quantizer scale to produce the new quantizer scale 115. The new or second quantizer scale may be transferred from qua.ntization control block 105 to quantization block 104 in order to dynamically change the quantization process and to provide a real-time adaptive quantization control.
[0050] Experimental measurentents of Peak To Signal Ratio (PSNR) of a H.264 encoder for video streams with frame resolution of 720x480 pixels showed improvement in PSNR
when quantization control according to embodiments of the invention was applied as indicated in the following table:
Attomey Docket No.: P-9927-CA
PSNR W+th PSNR Noa quantization quantization ideo Stream Bitrate enhancement enhancement number Kb s dB dB
1 2000 25.66 25.66 2 2000 28.64 28.34 3 2000 30.25 29.8 4 2000 32.14 31.91 1 1500 24.71 24.71 3 1500 28.7 28.21 4 1500 30.9 30.73 3 1000 26.64 26.27 4 1000 29.25 29.18 Table [1]
[0051] The values of table [1] were measured while using the following paraineters and conversion tables:
a0=6, a1=15360 , a2= 3 W0= 0.6, W 1=0.2, VV2=0.2 tablel [ ] = {-2, -1, l, 3}
table2 {0,0, 1,2,4}
(0052] Reference is now made to Fig. 3, whiah is a flowchart of a met1iod for performing adaptive quantization control according to embodiments of the present invention.
Operations of the metlzod may be implemented, for example, by encoder 100 of Fig. 1, and/or by other suitable units, devices, and/or systems.
[0053] As indicated at box 310, the xnethod may include receiving inputs whidh may be used for controlling the quantization process. The inputs may include MB data of a current pieture to be enooded, one or more motion vectors and a current used.
quantizer seale. All the inputs may be received by a quantization control block, for example, quantization control block 105 (of figure 1).
[0054] As indicated at box 320, the method may include detecting, obtaining or calculating the luminance level of the received MB data. laetecting the lurninanee level niay include summing the intensities af the luminance components of a chosen set of pixels of the received MB data, Detecting the luminance level may be performed by a Auomey Docket Na, P-9927-CA
dedicated unit or block, for example, by luma level detector 124 (of figure 1). It should be understood, however to a person skilled in the art that any other mathematical operation inay be perfonned in order to detect or calculate the luminance level and any other suitable unit or block may be used.
[0055] As indicated at box 330, the niethod may include detecting, obtaining or calculating the diagonal frequencies (DF) of the reaeived MB data_ Detecting the DF may accurately detect high amplitudes at the diagonal transformed coefficients of the received MB data, which may points block noisiness. Detecting the DF may include subtracting luminance eoinponents of riairs of pixels and summing the absolute values of the differences to determine horizontal and vertical gradients of the received MB
data_ Detecting the DF may be pertonned by a dedicated unit or block, for exanzple, by DFD 122 (of figure 1). It shonld be understood, however to a person skilled in the art that any other mathematical operation may be performed in order to detect or calculate the DF
and any other suitable unit or block may be used.
[00561 As indicated at box 340, the method may include detecting, obtaining or calculating the motion level between two or niore frames. Detecting the motion level may include by summing the absolute values of vertical and horizontal components of the received motiota vectors. Det4ctyng the niotion level may be performed by a dedicated unit or block, fbr example, by motion level detector 126 (of figure 1). It should be understood, however to a person skilled in the art that any other inathematical operation may be performed in order to detect or calculate the motion level and any other suitable unit or block may be used.
[0057] As indicated at box 350, the method may include generating a quantizer scale offset based on the values d.eteCted at boxes 320, 330 and 340. The generation, production or calculation of a quanti2er scale offset may include sununing products of weights determined by encoder characteristics multiplied by the values calculated at boxes 320, 330 and 340. The generation of the quantizer scale offset inay be perfonned after further processing of the values calculated at boxes 320, 330 and 340. Generating the quantizer scale offset may be performed by a dedicated unit or block, for example, by quantizer enha-ncement block 120 (of tigure 1). It should be understood, however to a person skilled Attorncy Doaket No,; P-9927-CA
in the art that any other mathematical operation may be perfonned in order to geneiat+e or calculate the quantizer seale offset and any other suitable unit or block may be used.
[0058] As indicated at box 360, the method may include changing a first quantizer scale to receive a second quantizer scale in order to enhance t'he quantization process of an encoder, for example, encoder 100 (of figure 1), Changing of the first quantizer scale which may be received as an input at box 310, may be perforcned by applying the quantizer scale offset generated at box 350 by for example, by adding, subtracting, shiRing, multiplying to/from the first quantizer scale. Any other mathematical operation may be performed on the first quat7tizer scale and quantizer scale offset to receive the second quantizer scale. Changing the quantizer scale rnay be performed by a dedicated uiiit or block, for example, by quantizer enhancenient block 120 (of figure 1). It should be understood, however to a person skilled in the ait that any otber mathematical operation may be perfonned in order to generate or calculate the second quantizer scale and any other suitable unit or block may be used.
[0059] While certain features of the invention have been illustrated and described herein, many modifications, substitutions, changes, and equivalents vvill now occur to those of ordinary skill in the art. It is, therefore, to be understood that the appended claims are intended to covcr all suclr niodifications and changes as fall within the true spirit of the invention.
Thc image or frame may be divided into macro blocks (MD), each MB may include a group of pixels, for example, by 16x16 pixels. A subtractor 102 may calculate the difference between the pixels of a current frame, stored in franie ntemory 101 and the pixels of a reference frame, storr;d in a reference frame znemory 109- The difference tnay be trattsfornied to the ftequency domain by a Discrete Cosine Transform (DCT) 103 and the transforrned output niay be an array of coefficients representing the energy of the input data by horizontal, vertical and diagonal spatial frequencies. Video encoder 100 may include a quantizer 104 which may quantize the transformed output of DCT 103 and may Attomey Docliet No,: P-9927-CA
be dynamically controlled by a quantization controller 105 as described in detail below according to embadin-ients of the present invention.
[0012J The quantized output signal may be inverse-quantized at an inversr:
quantizer 106 prior to underl;oing an inverse DCT (TDCT) conversion at IDCT block 107. The output from 1DCT 107 is added to the output of a motion estimation/compensation block 110 by an adder 108 and the calculated sum may be stored in reference fraine memory 109. The data stored in referc,~nce frame memory 109 as well as the original image stored in frame memory 101 may both be used as inputs to subtractor 102 and to inotion estimation/compensation block 110_ [0013] Although in the exemplary illustration of Fig. 1, DCT and IDCT are being used, it should be understood to a person skilled in art that the invention is not limited in this respect and according to embodiments of the present invention any other mathematical transform which may map a function from the time domain to the frequency domain may be used.
1'0014'1 The output of quantizer 104 may also be a subject to cntropy coding at entropy coding block 112 and the coded output signal from entropy coding block 112 inay be output through bit buffer 113. Encoder 100 may include a bit rate controller 111 which may receive an input from bit buffer 113 indicating the fullness of bit buffer 113 and may transfer control signals and/or parameters to quantization controller 105.
l,00] 5] According to embodiments of the present invention, quantization controller 105 may receive from bit xate controller 111 at least one parameter, also referred to herein as "quantizer scale" 114 wliich may be used for the quantization of the DCT
transfonned coefficients in quantizer 104_ Quantization controller 105 may control the quantization of the DCT transformed coefficients in quantizer 104 by adapting or changing quantizer scale 1 14 received from bit rate controller 111 according to the coinplexity of the current MB
and providing a new quantizer scale 115 to quantizer 104 as is described in detail below.
[00161 It is k.nown that the human eye is less sensitive to quantization noise at "noisy"
poi'tions of the image, e.g., detailed surfaces, dark portions of a picture and fast moving objects. Accordingly, quantization controller 105 may control the quantization of such portions of a picture to be more coarse and may use saved bits for other portions, for exainple, where noise is very obvious e.g., smooth portions of image, faces.
Quantization Attorney Docket No.: P-9927-CA
controller 105 nray controi the quantization of quanti.zer 104 to be coarse for high spatial activity and less coarse for lower spatial activity by changing the quantizer scale in order to 4nhance the subjective quality of the picture.
[0017] According to embodiments of the present invention, quantization controller 105 may provide a real-time adaptive quantization control to improve image quality.
Quantization controller 105 may include a quantizer enhancement block 120, a diagonal frequency detector (T)FD) 122, a luma level detector 124 and a motion level detector 126.
The frequency detector (DFD) 122, the luma level detector 124 and the motion level dctector 126 may all be coupled to the quantizer enliancement block 120.
Quantizer enhancement block 120 may generate a new quantizer scale 115 based on inputs received 1.'rom DFD 122, luminance (luma) level detector 124 and motion level detector 126.
[0018] According to embodiments of the present invention, DFD 122 may receive MB
data of the cun:ent picture and may apply a diagonal frequency detection algorithm which may classif'y the smoothness level of the MB. The algorithni or function implemented by DFD 122 may include a low complexity caleulation which may accurately detect high amplitudes at the diagonal transformed coefficients, which points block n~oisiness_ The DFD algorithm may be applied on the luminance component (Y) saniples of a eurrent image, stored in faame mcmory 1.01, before performing the actual calculation of the DCT
transform.
[0019] Accordinb to embodiments of the present invention, luma level detector 124 may reeeive M13 data of the cunent picture and may calculate the lumnlance intensity of the input MB, Luma level detector 124 may furtlier map the calculated value to predetermined levels using a conversion table.
[0020] According to embodiments of the present invention, Motion level detector 126 may receive one or more motion vectors from motion estimationlcompensation block 110 and tnay detect the level of motion the current pictures by suiruning the received vectors components.
[0021] Quantizer enhancement 120 may use a weighted function of the diagonal frequency detection algorithm results, received from DFD 120, luminance level caleulations received froni luma level detector 124 and motion level information received from motion level Attonicy Docket No.: P-9927-CA
detector 126 to produce quantizer afflets, and to use these offsets to detennine a new quantizer scale 115.
[0022] Although the present invention is not limited in this respect, the function detecting diagoiual frequencies, implemented by DFD 120 and the function detecting luma intensity, implemented by luma level analyzer 124 may be applied on all the pixels of a MB or ot7 a sub-sampied set of pixels of a MB. One example of a sub-sainpled set of pixels of a MB is shown in figures 2A and 2B. It should be understood to a person skilled in the art that the calculations and functions applied to the sub-sampled set of 1vIB pixels may be applied to any other sub-smnpled set of M8 pixels or to the entire MB, namely, to all the pixels of a MB.
[0023] Reference is now made to Figs. 2A and 213,-which are maps of an exeniplary sub-sampled set of MB pixels used in accordance with embodiments of the present invention.
[0024] Map 200 illustrates an exernplary MB of 16x16 pixels, where each square of the map represent a single pixel. It should be understood to a person skilled in the art that any other size of MB may be used. Embodiments of the present invention may use all pixels of niap 200, or any sub set of pixels of map 200, the sub set of the pixels may be chosen according to the encoder implementation or speed, desired accuracy of the outputs of DFD 122 and luma level detector 124 or according to any other parameter.
For example, encoders which pertonn parallel 4 bytes operation which are well optimized for, for example, DSP implementations may use the sub set of pixels of map 200 indicated by x's.
Any other set of pixels may be used.
[0025] According to embodiments of the present invention, DFD 122 may subtract pairs of pixels to determine horizontai and vertical gradients as follows;
Df'17 = Y-I{Y(i j)-Y(i,j+1)}- {Y(i+1j)-Y(i+lj+l)}l [i]
for- i = 0 to horizontal MB size j= 0: vertical MB size and wliere Y represents the luminance component of the pixel.
[0026] The low complexity calculation of equation [1] may accurately detect high amplitudes at the diagonal transformed coefficients, which may points block noisiness, Attoiiicy Docket No.: P-9927-CA
e.g,, a high diagonal frequencies detection (DFD) result may imply that the MB
may contain some high diagonal frequencies. The result of equation [1] may be transferred to the quantizer i;nhancement block 120 for determining the offset of the quantizer scale.
[0027] It should be understood to one skilled in the art that equation [1 ]
may be applicable in embodiments which use all the pixels of a MB, and that equation [1] may be adapted to suit any other choice of sub set of pixels, for example as described in equations [2]-[10]
that relate to DFD calculation of the sub set of pixels shown in map 250_ [002F] The pixels included in the Dl~'l7 calculation of map 250 are pixels indicated by Latin letters (including x). Equation [1] may be adapted to subtract pairs of pixels of the marked pixels and may be defined as follows:
Y1 = Ya - Ye [2]
Y2=Yb-Yf [3]
Y3=Yc-Yg [4]
Y4=Yd-Yh [5]
2 1 = abs(Y1-Y2) + abs(Y2-Y3) + abs(Y3-Y4) [6]
where: Ya to Yh represent the lurninance component of the pixels marked with a to h.
[0029] Equations [2]-[4] may be used for every group of the 20 sub groups (260-279 ) of the MB each containing 8 pixels, for example, groups 260, 261, 262 and 279.
After equations 2-4 are performed for each group of groups 260-279, the final DFD
value may be calculated as follows:
DFD = T- Zi [7]
for; i I to 20.
[0030] Although in the exeniplaiy illustration of map 250 of Fig. 2B, 20 groups, each containing 8 pixels are shown, it should be understood to a person sldlled in art that the invention is not limite:d in this respect and according to einbodiments of the present iiivention, any suitable groups of a MB containing any number of pixels may be used by Attorney Docket No.: P-9927-C'A
ppp 122 for the DF calculations and equation [1) may be adapted to fit any selection of pixels of a frame.
[0031] According to embodvnents of the present invention, luma level detector 124 may calculate the intensity of the luminance component of the chosen pixels of a MB as follows;
Luma Level = Y_Y(i,j) [8]
for i= 0 to horizontal MB size; and j= 0 to vertical MB size [0032] The result of equation [8] may be transferred ta the quantizer enhancement block 120 for determining the offset of the quantizer scale.
[0033] It should be understood to one skilled in the art tl-iat equation [8]
may be applicable in embodiments which use all the pixels of a MB, and that equation [8] may be adapted to suit any other choice of sub-set of pixels, for example as described in equations [9]-[10]
which relate to luminance level calculation of the sub-set of pixels showla in map 250.
[0034] The pix4ls included in the luminance level calculatiou of map 250 are the pixels marked with Latin letters (including x). Equation [8] may be adapted to calculate thc iiZtensity of the marked pixels by summing the luminance component of the marked pixels and nlay be described as follows:
Ll = (a+U+ c+d)+(e+f+g+h) [9]
where Ll represent the luminance level of group 260, [00351 Equation [9] may be used for every group of the 20 sub groups 260-279, each contaiiiing 8 pixels, for example, sub-group 261 illustrated by the pixels marked i to p.
After equation [9] is perfor-rned for all 20 groups 260-279, the final luminance level value may be calculated as follows:
L.CJMA_LEVEL- ELi [10]
fior_i=1to2Ø
Attomey Docket No.: P-9927-CA
[0036] Although in the exemplary illustration of map 250, groups 260-279, each containing 8 pixels are shown, it should be understood to a person skilled in art that the invention is not limited in this respect and according to embodiments of the present invention, any suitable groups of a MB containing any nurnber of pixels may be used by luma level det:ector 124 for the luminance level calculations and equation [9]-[10] may be adapted to fit any selection of pixels of a fframe.
[0037] According to embodiments of the present invention, motion level detector 126 may receive a plurality of motion vectors from motion estimation/cotnpensation block 110.
Each vector may be represented by an horizontal component (Vx) and a vertical component (Vy). Motion level detector niay sum the absolute values of the motion vectois components of inter pictures as follows:
Motion = 2:jV ixj + Y_IV iy~ [11 ]
for i^ 0 to number of received vectors, [0038] The result of equation [11] may be transferred to quantizer enliancement block 120 for detemlining the offset of the quantizer scale.
[0039:1 According to embodiments of the present invention, quantizer enhancement block 120 may receive the calculated DFD from DFD 122, the luminance level from luma level detector 124 and the absolute motion of a MB froin motion level detector 126.
In addition quantizer enhancement block 120 may receive the original quantizer scale, also refetred to herein as "first" qua.ntizer scale frotn bit rate controller 111. Qtiantizer enhaneentent block 120 may process one or more of tlie inputs and may use a weighting function of flle calculated DpD, luminance level and absolute motion to determine an offset and to usc this offset to generate a new quantizer scale, also referred to herein as "second"
quantizer scale as desciibed below.
(0040] Although the presr.vnt invention is not limited in this respect, the calculated DFD
received from DFD 122 may be further processed by quantizer enhancement block 120, for example, the DFD value mav be converted to another value by performing any mathematical operation r.equircd. The niathematical operation or conversiort may be 9 f Attomoy Docket No.: P-9927-CA
perforrned by, for example, a shifter, an adder, a subtractor or by any other component or block which may convert the received DFD value. The mathematical operation may be deterniined based on parameters related to encoder's operation, such as the required output bit iate, parameters related to encoder implementation, such as for example, DSP, ASIC, FPGA or software impiementation, parameters related to the video compression standard used by the encoder, such as MPEG-2, MPEG-4, H-264, or based on parameters related to Any other encoder's characteristics.
[0041] Although in the description above the mathematical operation on the DFD
value is described as perfornzed by quantizer enhancement block 120, it should be understood to a person skilled in art that the invention is not limited in this respect and according to embodiments of the present invention, any such mathematical operation may be performed in DFD 122 prior to the operation of qua.ntizer enhancement block 120. For example, a shifter to shift the DFD value may be implemented in either DFp 122, quantizer enliancement block 120 or both DFD 122 and quantizer enhanceinent block 120.
(0042] Although the present invention is not limited in this respect, the input received from luma level detector 124 may be further processed by quantizer enhancement block 120, for exaniple, the luminance level value received which indicate the intensity oF'a MB
may be mapped, converted or scaled using one or more conversion tables and/or any other inathematical operation, such as shift operation in order to fit into a desired weighting function. The conversion table and/or mathematical operation txiay be predefned and may be based on parameters related to encoder operation, such as the required output bit rate, parameters related to encoder implementation, such as for eycample, DSP, ASIC.
FPGA or software iniplementation, parameters related to the video compression standard used by the encoder, such as MPBG-2, MPEG-4, H-264, or based on parameters related to any other encoder's characteristics.
[0043] Although in dhe description above rhe conversion table applied on the luma level values is described as a part quantizer enhancement block 120, it should bc understood to a person skilled in art that the invention is not limited in this respeCt and according to embodiinents of the present invention, any conversion table and/or other mathematieal operation niay be peiformed in luminance level detector 124 prior to the operation of quantizer enhancement block120. For example, a predefined conversion table may be lo Attorney Qocket No.: P-9927-CA
included ita either lurna level detector 124, quantizer enhancement block 120 or both luirra level detector 124 and quantizer enhancem.ent block 120.
[0044] Although the present invelition is not limited in tlus respect, the received input from motion level detector 126 may be further processed by quantizer enhancement block 120, for example, the motion level value received may be mapped, converted or scaled using one or more conversion tables andlor any other inathematical operation, such as shift operation in order to fit into a desired weighting fitnction. The conversion table and/or mathematical operation may be predefined and may be based on parameters related to encoder's operation, such as the required output bit rate, parameters related to encoder implenientation, such as for exaniple, DSP, ASIC, FPGA or software implenientation, parameters related to the video compression standard used by the encoder, such as MPBG-2, MPEG-4, H-264, or based on parameters related to any other encoder's characteristics.
[0045j Although in the description above the conversion table applied on the motion level values is described as a part quantizer enhancement block 120, it should be understood to a person skilled in art that the invention is not limited in this respect and according to embodiments of the present invention, any conversion table and/or other mathematical operation may be performed in motion level detector 126 prior to the operation of quantizer enhancement 120. For example, a predefined conversion table may be included in either motion level detector 126, quantizer enhancement block 120 or both motion level detector 126 and quantizer enilancement block 120_ [0046] According to embodiments of the present invention, Quantizer enhancement block 120 inay calculate a quantizer scale offset by a pre-defined weighted tunction as follows;
Quantizer scale offset = (DFD)xW0 + (.Luma_Level)xW1 + (Motion)xW2 [12]
[0047] Equation [12] rnay describe a general form of a weighted srun of products. WO, W 1 and W2 may represent weights detemiined by a plurality of parameters, sucli as for example, parameters related to encoder operation, such as the required output bit rate, paranleters related to encoder iniplementation, such as for example, DSP, ASIC, FPGA or softwar-e implementation, parameters related to the video compression standard used by the encoder, such as MPEG-2, MPEG-4, H-264 or any other parameter. The above parameters ll Attorney Docket No,; P-9927-CA
and/or any other parameter or characteristic related to the encoder may referred to herein as "encoder characteristies". According to other embodiments of the invention, the weighted sum of products may be an adaptive weighted funetion which may be dynamically cliange based on parameters related to characteristies of a video stream to be encoded.
[0048] The general form of equation [12] may relate to the DFD, Luma_i.,eve1 and Motion values as the required values after processing, other embodirnents may use a different weighted sum, for example, such as equation 13 which inay include some or all of the pre-processing or conversions of some or all of the three inputs weighted of general equation 12 for example;
Quantizer scale offset - (DFD >> aO)xWO +(tablel [Lurna Level>>al ])xW 1+
(tablo;2[Motion >> a2])xW2 [13]
where: aO, al, a2 are constant variables determined by the encoder characteristics;
WO, W1, W2 are weights determined by encoder characteristics;
Tablel is a predefined table used for conversion. of luminance level values;
Table2 -is a predefined table used for conversion of motion level values.
">>" represents a right shift;
"table [y] represent the value at an entry "y" in a table, [0049] According to embodiments of the present invention, the quantizer scale offset calculated by equation [12], equation [13] or any other, weighted functior of the DFD, luminance level and motion level inputs, may be applied to the original quantizer scale, received from bit rate controller 111. For example, it may be added, subtracted, shifted or inultiplied to4rom original quantizer scale to produce the new quantizer scale 115. The new or second quantizer scale may be transferred from qua.ntization control block 105 to quantization block 104 in order to dynamically change the quantization process and to provide a real-time adaptive quantization control.
[0050] Experimental measurentents of Peak To Signal Ratio (PSNR) of a H.264 encoder for video streams with frame resolution of 720x480 pixels showed improvement in PSNR
when quantization control according to embodiments of the invention was applied as indicated in the following table:
Attomey Docket No.: P-9927-CA
PSNR W+th PSNR Noa quantization quantization ideo Stream Bitrate enhancement enhancement number Kb s dB dB
1 2000 25.66 25.66 2 2000 28.64 28.34 3 2000 30.25 29.8 4 2000 32.14 31.91 1 1500 24.71 24.71 3 1500 28.7 28.21 4 1500 30.9 30.73 3 1000 26.64 26.27 4 1000 29.25 29.18 Table [1]
[0051] The values of table [1] were measured while using the following paraineters and conversion tables:
a0=6, a1=15360 , a2= 3 W0= 0.6, W 1=0.2, VV2=0.2 tablel [ ] = {-2, -1, l, 3}
table2 {0,0, 1,2,4}
(0052] Reference is now made to Fig. 3, whiah is a flowchart of a met1iod for performing adaptive quantization control according to embodiments of the present invention.
Operations of the metlzod may be implemented, for example, by encoder 100 of Fig. 1, and/or by other suitable units, devices, and/or systems.
[0053] As indicated at box 310, the xnethod may include receiving inputs whidh may be used for controlling the quantization process. The inputs may include MB data of a current pieture to be enooded, one or more motion vectors and a current used.
quantizer seale. All the inputs may be received by a quantization control block, for example, quantization control block 105 (of figure 1).
[0054] As indicated at box 320, the method may include detecting, obtaining or calculating the luminance level of the received MB data. laetecting the lurninanee level niay include summing the intensities af the luminance components of a chosen set of pixels of the received MB data, Detecting the luminance level may be performed by a Auomey Docket Na, P-9927-CA
dedicated unit or block, for example, by luma level detector 124 (of figure 1). It should be understood, however to a person skilled in the art that any other mathematical operation inay be perfonned in order to detect or calculate the luminance level and any other suitable unit or block may be used.
[0055] As indicated at box 330, the niethod may include detecting, obtaining or calculating the diagonal frequencies (DF) of the reaeived MB data_ Detecting the DF may accurately detect high amplitudes at the diagonal transformed coefficients of the received MB data, which may points block noisiness. Detecting the DF may include subtracting luminance eoinponents of riairs of pixels and summing the absolute values of the differences to determine horizontal and vertical gradients of the received MB
data_ Detecting the DF may be pertonned by a dedicated unit or block, for exanzple, by DFD 122 (of figure 1). It shonld be understood, however to a person skilled in the art that any other mathematical operation may be performed in order to detect or calculate the DF
and any other suitable unit or block may be used.
[00561 As indicated at box 340, the method may include detecting, obtaining or calculating the motion level between two or niore frames. Detecting the motion level may include by summing the absolute values of vertical and horizontal components of the received motiota vectors. Det4ctyng the niotion level may be performed by a dedicated unit or block, fbr example, by motion level detector 126 (of figure 1). It should be understood, however to a person skilled in the art that any other inathematical operation may be performed in order to detect or calculate the motion level and any other suitable unit or block may be used.
[0057] As indicated at box 350, the method may include generating a quantizer scale offset based on the values d.eteCted at boxes 320, 330 and 340. The generation, production or calculation of a quanti2er scale offset may include sununing products of weights determined by encoder characteristics multiplied by the values calculated at boxes 320, 330 and 340. The generation of the quantizer scale offset inay be perfonned after further processing of the values calculated at boxes 320, 330 and 340. Generating the quantizer scale offset may be performed by a dedicated unit or block, for example, by quantizer enha-ncement block 120 (of tigure 1). It should be understood, however to a person skilled Attorncy Doaket No,; P-9927-CA
in the art that any other mathematical operation may be perfonned in order to geneiat+e or calculate the quantizer seale offset and any other suitable unit or block may be used.
[0058] As indicated at box 360, the method may include changing a first quantizer scale to receive a second quantizer scale in order to enhance t'he quantization process of an encoder, for example, encoder 100 (of figure 1), Changing of the first quantizer scale which may be received as an input at box 310, may be perforcned by applying the quantizer scale offset generated at box 350 by for example, by adding, subtracting, shiRing, multiplying to/from the first quantizer scale. Any other mathematical operation may be performed on the first quat7tizer scale and quantizer scale offset to receive the second quantizer scale. Changing the quantizer scale rnay be performed by a dedicated uiiit or block, for example, by quantizer enhancenient block 120 (of figure 1). It should be understood, however to a person skilled in the ait that any otber mathematical operation may be perfonned in order to generate or calculate the second quantizer scale and any other suitable unit or block may be used.
[0059] While certain features of the invention have been illustrated and described herein, many modifications, substitutions, changes, and equivalents vvill now occur to those of ordinary skill in the art. It is, therefore, to be understood that the appended claims are intended to covcr all suclr niodifications and changes as fall within the true spirit of the invention.
Claims (23)
1. A method for controlling video compression quantization comprising:
generating a quantizer scale offset based on diagonal frequencies of luminance components of data block samples, luminance intensity of the samples and motion activity of the data block;
adjusting a first quantizer scale using the quantizer scale offset to receive a second quantizer scale; and quantizing the data block using the second quantizer scale.
generating a quantizer scale offset based on diagonal frequencies of luminance components of data block samples, luminance intensity of the samples and motion activity of the data block;
adjusting a first quantizer scale using the quantizer scale offset to receive a second quantizer scale; and quantizing the data block using the second quantizer scale.
2. The method of claim 1, wherein generating the quantizer scale offset comprises applying a weighted function on the diagonal frequencies of the luminance components, the luminance intensity of the samples and the motion activity of the data block.
3. The method of claim 1, wherein generating the quantizer scale offset comprises:
detecting the diagonal frequencies of the luminance components of the data block samples;
calculating the luminance intensity of the samples; and detecting the absolute motion level of the data block.
detecting the diagonal frequencies of the luminance components of the data block samples;
calculating the luminance intensity of the samples; and detecting the absolute motion level of the data block.
4. The method of claim 2, wherein applying the weighted function comprises processing of at least one of the diagonal frequencies of the luminance components, the luminance intensity of the samples and the motion activity of the data block, based on parameters related to characteristics of a video compression encoder.
5. The method of claim 3, wherein detecting the diagonal frequencies and calculating the luminance intensity are performed using all samples of the data block.
6. The method of claim 3, wherein detecting the diagonal frequencies and calculating the luminance intensity are performed using a sub-set of samples of the data block.
7. The method of claim 2, wherein the weighted function includes a pre-defined weighted sum of products based on parameters related to characteristics of a video compression encoder.
8. The method of claim 2, wherein the weighted function includes an adaptive weighted sum of products based on parameters related to characteristics of a video stream to be encoded.
9. The method of claim 1, wherein quantizing comprises dynamically quantizing the data block.
10. The method of claim 9, wherein dynamically quantizing comprises updating the quantizer scale offset and the second quantizer scale in real-time.
11. A system for controlling video compression quantization comprising:
a bit rate controller to generate a first quantizer scale;
a quantization controller to generate a quantizer scale offset based on diagonal frequencies of luminance components of data block samples, luminance intensity of the samples and motion activity of the data block and to adjust the first quantizer scale using the quantizer scale offset to receive a second quantizer scale; and a quantizer to quantize the data block using the second quantizer scale.
a bit rate controller to generate a first quantizer scale;
a quantization controller to generate a quantizer scale offset based on diagonal frequencies of luminance components of data block samples, luminance intensity of the samples and motion activity of the data block and to adjust the first quantizer scale using the quantizer scale offset to receive a second quantizer scale; and a quantizer to quantize the data block using the second quantizer scale.
12. The system of claim 11, wherein the quantization controller is to generate the quantizer scale offset by applying a weighted function on the diagonal frequencies of the luminance components, the luminance intensity of the samples and the motion activity of the data block.
13. The system of claim 11, wherein the quantization controller comprises:
a diagonal frequencies detector to detect the diagonal frequencies of the luminance components of the data block samples;
a luminance level detector to calculate the luminance intensity of the samples, and a motion level detector to detect the absolute motion level of the data block.
a diagonal frequencies detector to detect the diagonal frequencies of the luminance components of the data block samples;
a luminance level detector to calculate the luminance intensity of the samples, and a motion level detector to detect the absolute motion level of the data block.
14. The system of claim 12, wherein applying the weighted function comprises processing of at least one of the diagonal frequencies of the luminance components, the luminance intensity of the samples and the motion activity of the data block, based on parameters related to characteristics of a video compression encoder.
15. The system of claim 13, wherein all samples of the data block are being used by the diagonal frequencies detector to detect the diagonal frequencies and by the luminance level detector to calculate the luminance intensity.
16. The system of claim 13, wherein a sub-set of samples of the data block are being used by the diagonal frequencies detector to detect the diagonal frequencies and by the luminance level detector to calculate the luminance intensity.
17. The system of claim 12, wherein the weighted function includes a pre-defined weighted sum of products based on parameters related to characteristics of a video compression encoder.
18. The system of claim 12, wherein the weighted function includes an adaptive weighted sum of products based on parameters related to characteristics of a video stream to be encoded.
19. The system of claim 11, wherein the quantizer is to dynamically quantize the data block.
20. The system of claim 11, wherein the quantization controller is to update the quantizer scale offset and the second quantizer scale in real-time.
21. A computer readable-medium, having stored thereon instructions, that when executed on a computing platform, result in:
generating a quantizer scale offset based on diagonal frequencies of luminance components of data block samples, luminance intensity of the samples and motion activity of the data block;
adjusting a first quantizer scale using the quantizer scale offset to receive a second quantizer scale; and quantizing the data block using the second quantizer scale.
generating a quantizer scale offset based on diagonal frequencies of luminance components of data block samples, luminance intensity of the samples and motion activity of the data block;
adjusting a first quantizer scale using the quantizer scale offset to receive a second quantizer scale; and quantizing the data block using the second quantizer scale.
22. The medium of claim 21, wherein the instructions results in generating the quantizer scale offset comprises applying a weighted function on the diagonal frequencies of the luminance components, the luminance intensity of the samples and the motion activity of the data block.
23. The medium of claim 21, wherein the instructions results in generating the quantizer scale offset comprises:
detecting the diagonal frequencies of the luminance components of the data block samples;
calculating the luminance intensity of the samples; and detecting the absolute motion level of the data block.
detecting the diagonal frequencies of the luminance components of the data block samples;
calculating the luminance intensity of the samples; and detecting the absolute motion level of the data block.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US11/967,288 US8213502B2 (en) | 2007-12-31 | 2007-12-31 | Method and system for real-time adaptive quantization control |
| US11/967,288 | 2007-12-31 |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CA2648125A1 true CA2648125A1 (en) | 2009-06-30 |
| CA2648125C CA2648125C (en) | 2014-09-23 |
Family
ID=40592056
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA2648125A Active CA2648125C (en) | 2007-12-31 | 2008-12-24 | Method and system for real-time adaptive quantization control |
Country Status (4)
| Country | Link |
|---|---|
| US (1) | US8213502B2 (en) |
| EP (1) | EP2076045B1 (en) |
| JP (1) | JP5015129B2 (en) |
| CA (1) | CA2648125C (en) |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4833309B2 (en) * | 2009-03-06 | 2011-12-07 | 株式会社東芝 | Video compression encoding device |
| KR101885258B1 (en) * | 2010-05-14 | 2018-08-06 | 삼성전자주식회사 | Method and apparatus for video encoding, and method and apparatus for video decoding |
| US9172960B1 (en) * | 2010-09-23 | 2015-10-27 | Qualcomm Technologies, Inc. | Quantization based on statistics and threshold of luminanceand chrominance |
| US8885706B2 (en) | 2011-09-16 | 2014-11-11 | Google Inc. | Apparatus and methodology for a video codec system with noise reduction capability |
| US9131073B1 (en) | 2012-03-02 | 2015-09-08 | Google Inc. | Motion estimation aided noise reduction |
| US9344729B1 (en) | 2012-07-11 | 2016-05-17 | Google Inc. | Selective prediction signal filtering |
| GB2513112B (en) * | 2013-04-08 | 2020-01-08 | Snell Advanced Media Ltd | Video sequence processing |
| US10102613B2 (en) | 2014-09-25 | 2018-10-16 | Google Llc | Frequency-domain denoising |
Family Cites Families (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5073821A (en) * | 1989-01-30 | 1991-12-17 | Matsushita Electric Industrial Co., Ltd. | Orthogonal transform coding apparatus for reducing the amount of coded signals to be processed and transmitted |
| JP2839339B2 (en) * | 1990-08-06 | 1998-12-16 | 松下電器産業株式会社 | Orthogonal transform coding apparatus and orthogonal transform coding method |
| DE69213511T2 (en) * | 1991-03-07 | 1997-04-17 | Mitsubishi Electric Corp | Encoding device for encoding a digital image signal |
| JP2646920B2 (en) * | 1991-03-07 | 1997-08-27 | 三菱電機株式会社 | High efficiency coding device |
| US5479211A (en) * | 1992-04-30 | 1995-12-26 | Olympus Optical Co., Ltd. | Image-signal decoding apparatus |
| KR100192270B1 (en) * | 1996-02-03 | 1999-06-15 | 구자홍 | The video decoding circuit in hdtv |
| KR100243430B1 (en) * | 1997-12-31 | 2000-02-01 | 구자홍 | Method of adaptive quantization control |
| US6658157B1 (en) * | 1999-06-29 | 2003-12-02 | Sony Corporation | Method and apparatus for converting image information |
| US6782135B1 (en) * | 2000-02-18 | 2004-08-24 | Conexant Systems, Inc. | Apparatus and methods for adaptive digital video quantization |
| JP4218920B2 (en) * | 2000-04-07 | 2009-02-04 | キヤノン株式会社 | Image processing apparatus, image processing method, and storage medium |
| US6741758B2 (en) * | 2000-04-07 | 2004-05-25 | Canon Kabushiki Kaisha | Image processor and image processing method |
| JP4130780B2 (en) * | 2002-04-15 | 2008-08-06 | 松下電器産業株式会社 | Image encoding method and image decoding method |
| JP4196726B2 (en) * | 2003-05-14 | 2008-12-17 | ソニー株式会社 | Image processing apparatus, image processing method, recording medium, and program |
| JP2005005862A (en) * | 2003-06-10 | 2005-01-06 | Victor Co Of Japan Ltd | Picture coding device |
| JP4617644B2 (en) * | 2003-07-18 | 2011-01-26 | ソニー株式会社 | Encoding apparatus and method |
| US7639884B2 (en) * | 2005-04-26 | 2009-12-29 | Nippon Hoso Kyokai | Prefilter, compressive coding pre-processing apparatus and decompressive decoding post-processing apparatus, and compressive coding apparatus and decompressive decoding apparatus |
| JP2007081474A (en) * | 2005-09-09 | 2007-03-29 | Toshiba Corp | Image coding apparatus and method |
| JP2007104342A (en) * | 2005-10-04 | 2007-04-19 | Canon Inc | Device and method for image encoding |
| AU2006338425B2 (en) * | 2006-02-13 | 2010-12-09 | Kabushiki Kaisha Toshiba | Moving image encoding/decoding method and device and program |
| US20080294446A1 (en) * | 2007-05-22 | 2008-11-27 | Linfeng Guo | Layer based scalable multimedia datastream compression |
-
2007
- 2007-12-31 US US11/967,288 patent/US8213502B2/en active Active
-
2008
- 2008-12-24 CA CA2648125A patent/CA2648125C/en active Active
- 2008-12-24 EP EP08172915.4A patent/EP2076045B1/en active Active
- 2008-12-25 JP JP2008329930A patent/JP5015129B2/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| US8213502B2 (en) | 2012-07-03 |
| US20090168869A1 (en) | 2009-07-02 |
| EP2076045B1 (en) | 2018-11-28 |
| JP2009165124A (en) | 2009-07-23 |
| EP2076045A1 (en) | 2009-07-01 |
| JP5015129B2 (en) | 2012-08-29 |
| CA2648125C (en) | 2014-09-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CA2648125A1 (en) | Method and system for real-time adaptive quantization control | |
| US7860160B2 (en) | Video encoding device | |
| CN105379269B (en) | The Video coding of interest region perception | |
| CN1253008C (en) | Spatial scalable compression | |
| CA3011659C (en) | Intra prediction of a processing block using a predicted value which is proportional to the amount of change in the horizontal direction of the signal value of a pixel adjacent tothe left of the processing block | |
| CN104285433B (en) | The speed control that motion sensor for Video coding aids in | |
| US9270993B2 (en) | Video deblocking filter strength derivation | |
| KR100389891B1 (en) | Method for processing perceptual error and image coding apparatus using the same | |
| JPH11112992A (en) | Motion compensating picture encoder | |
| US20080107176A1 (en) | Method and Apparatus for Detecting All Zero Coefficients | |
| WO2020139414A1 (en) | Video stream adaptive filtering for bitrate reduction | |
| Koz et al. | Optimized tone mapping with perceptually uniform luminance values for backward-compatible high dynamic range video compression | |
| JP2009188826A (en) | Moving image encoder | |
| Lai et al. | Low latency directional filtering for inter-layer prediction in scalable video coding using HEVC | |
| CN111886868B (en) | Method and apparatus for adaptive temporal filtering for substitute reference frame rendering | |
| Wige et al. | Analysis of in-loop denoising in lossy transform coding | |
| KR101247024B1 (en) | Method of motion estimation and compensation using in-loop preprocessing filtering | |
| JPH0410788A (en) | Method for controlling coding variable of image signal | |
| EP1511323A1 (en) | Device and method for the compression or decompression of digital images | |
| CN112154667B (en) | Encoding and decoding of video | |
| US20120230425A1 (en) | Encoding apparatus and method of conversion block for increasing video compression efficiency | |
| JPS63121374A (en) | Motion compensation interframe coding system | |
| Jo et al. | Perceptual intra-frame coding for HEVC still picture profile based on invisible signal suppression | |
| JP2701274B2 (en) | High efficiency coding device | |
| JP5353560B2 (en) | Image processing circuit and image encoding apparatus |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| EEER | Examination request |
Effective date: 20131212 |