CA2407143A1 - Texture replacement in video sequences and images - Google Patents

Texture replacement in video sequences and images Download PDF

Info

Publication number
CA2407143A1
CA2407143A1 CA002407143A CA2407143A CA2407143A1 CA 2407143 A1 CA2407143 A1 CA 2407143A1 CA 002407143 A CA002407143 A CA 002407143A CA 2407143 A CA2407143 A CA 2407143A CA 2407143 A1 CA2407143 A1 CA 2407143A1
Authority
CA
Canada
Prior art keywords
texture
video sequence
synthesized
frames
bit rate
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CA002407143A
Other languages
French (fr)
Other versions
CA2407143C (en
Inventor
Adriana Dumitras
Barin Geoffry Haskell
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
AT&T Corp
Original Assignee
AT&T Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by AT&T Corp filed Critical AT&T Corp
Publication of CA2407143A1 publication Critical patent/CA2407143A1/en
Application granted granted Critical
Publication of CA2407143C publication Critical patent/CA2407143C/en
Anticipated expiration legal-status Critical
Expired - Fee Related legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/002D [Two Dimensional] image generation
    • G06T11/001Texturing; Colouring; Generation of texture or colour
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/40Analysis of texture
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/40Analysis of texture
    • G06T7/41Analysis of texture based on statistical description of texture
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00Image coding
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00Image coding
    • G06T9/001Model-based coding, e.g. wire frame
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136Incoming video signal characteristics or properties
    • H04N19/14Coding unit complexity, e.g. amount of activity or edge presence estimation
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/167Position within a video image, e.g. region of interest [ROI]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/186Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a colour or a chrominance component
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/20Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video object coding
    • H04N19/23Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video object coding with coding of regions that are present throughout a whole video segment, e.g. sprites, background or mosaic
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/20Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video object coding
    • H04N19/27Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video object coding involving both synthetic and natural picture components, e.g. synthetic natural hybrid coding [SNHC]

Abstract

Systems and methods for reducing bit rates by replacing original texture in a video sequence with synthesized texture. Reducing the bit rate of the video sequence begins by identifying and removing selected texture from frames in a video sequence. The removed texture is analyzed to generate texture parameters. New texture is synthesized using the texture parameters in combination with a set of constraints. Then, the newly synthesized texture is mapped back into the frames of the video sequence from which the original texture was removed. The resulting frames are then encoded. The bit rate of the video sequence with the synthesized texture is less than the bit rate of the video sequence with the original texture. Also, the ability of a decoder to decode the new video sequence is not compromised because no assumptions are made about the texture synthesis capabilities of the decoder.
CA002407143A 2001-10-11 2002-10-04 Texture replacement in video sequences and images Expired - Fee Related CA2407143C (en)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US32862701P 2001-10-11 2001-10-11
US60/328,627 2001-10-11
US10/237,489 US6977659B2 (en) 2001-10-11 2002-09-09 Texture replacement in video sequences and images
US10/237,489 2002-09-09

Publications (2)

Publication Number Publication Date
CA2407143A1 true CA2407143A1 (en) 2003-04-11
CA2407143C CA2407143C (en) 2006-08-29

Family

ID=26930735

Family Applications (1)

Application Number Title Priority Date Filing Date
CA002407143A Expired - Fee Related CA2407143C (en) 2001-10-11 2002-10-04 Texture replacement in video sequences and images

Country Status (2)

Country Link
US (7) US6977659B2 (en)
CA (1) CA2407143C (en)

Families Citing this family (52)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7023447B2 (en) * 2001-05-02 2006-04-04 Eastman Kodak Company Block sampling based method and apparatus for texture synthesis
US6977659B2 (en) 2001-10-11 2005-12-20 At & T Corp. Texture replacement in video sequences and images
US7606435B1 (en) * 2002-02-21 2009-10-20 At&T Intellectual Property Ii, L.P. System and method for encoding and decoding using texture replacement
JP4305071B2 (en) * 2003-06-30 2009-07-29 株式会社ニコン Signal correction method
US20070140335A1 (en) * 2003-10-31 2007-06-21 Koninklijke Phillips Electronics N.V. Method of encoding video signals
US7567715B1 (en) * 2004-05-12 2009-07-28 The Regents Of The University Of California System and method for representing and encoding images
US8422546B2 (en) 2005-05-25 2013-04-16 Microsoft Corporation Adaptive video encoding using a perceptual model
US7605821B1 (en) * 2005-09-29 2009-10-20 Adobe Systems Incorporated Poisson image-editing technique that matches texture contrast
US7995649B2 (en) 2006-04-07 2011-08-09 Microsoft Corporation Quantization adjustment based on texture level
US8059721B2 (en) 2006-04-07 2011-11-15 Microsoft Corporation Estimating sample-domain distortion in the transform domain with rounding compensation
US8503536B2 (en) 2006-04-07 2013-08-06 Microsoft Corporation Quantization adjustments for DC shift artifacts
US8711925B2 (en) 2006-05-05 2014-04-29 Microsoft Corporation Flexible quantization
EP1926321A1 (en) * 2006-11-27 2008-05-28 Matsushita Electric Industrial Co., Ltd. Hybrid texture representation
KR101381600B1 (en) * 2006-12-20 2014-04-04 삼성전자주식회사 Method and apparatus for encoding and decoding using texture synthesis
US8238424B2 (en) 2007-02-09 2012-08-07 Microsoft Corporation Complexity-based adaptive preprocessing for multiple-pass video compression
US8498335B2 (en) 2007-03-26 2013-07-30 Microsoft Corporation Adaptive deadzone size adjustment in quantization
US20080240257A1 (en) * 2007-03-26 2008-10-02 Microsoft Corporation Using quantization bias that accounts for relations between transform bins and quantization bins
US8243797B2 (en) 2007-03-30 2012-08-14 Microsoft Corporation Regions of interest for quality adjustments
US8442337B2 (en) 2007-04-18 2013-05-14 Microsoft Corporation Encoding adjustments for animation content
US8331438B2 (en) 2007-06-05 2012-12-11 Microsoft Corporation Adaptive selection of picture-level quantization parameters for predicted video pictures
US8208556B2 (en) * 2007-06-26 2012-06-26 Microsoft Corporation Video coding using spatio-temporal texture synthesis
US20090033674A1 (en) * 2007-08-02 2009-02-05 Disney Enterprises, Inc. Method and apparatus for graphically defining surface normal maps
DE102007036215B4 (en) * 2007-08-02 2009-09-10 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Detection of defects in pictures / videos
JP4659005B2 (en) * 2007-08-17 2011-03-30 日本電信電話株式会社 Moving picture encoding method, decoding method, encoding apparatus, decoding apparatus based on texture synthesis, program thereof, and recording medium thereof
US8013862B2 (en) * 2007-11-16 2011-09-06 Microsoft Corporation Texture codec
US8189933B2 (en) * 2008-03-31 2012-05-29 Microsoft Corporation Classifying and controlling encoding quality for textured, dark smooth and smooth video content
WO2009155089A1 (en) * 2008-05-29 2009-12-23 Telcordia Technologies, Inc. Method and system for generating and presenting mobile content summarization
US8584048B2 (en) * 2008-05-29 2013-11-12 Telcordia Technologies, Inc. Method and system for multi-touch-based browsing of media summarizations on a handheld device
US8897359B2 (en) 2008-06-03 2014-11-25 Microsoft Corporation Adaptive quantization for enhancement layer video coding
JP5872896B2 (en) * 2008-06-27 2016-03-01 トムソン ライセンシングThomson Licensing Method and apparatus for texture compression using patch-based sampling texture synthesis
JP5680283B2 (en) * 2008-09-19 2015-03-04 株式会社Nttドコモ Moving picture encoding apparatus, moving picture decoding apparatus, moving picture encoding method, moving picture decoding method, moving picture encoding program, and moving picture decoding program
US8296675B2 (en) * 2009-03-09 2012-10-23 Telcordia Technologies, Inc. System and method for capturing, aggregating and presenting attention hotspots in shared media
EP2583460A1 (en) * 2010-06-15 2013-04-24 Thomson Licensing Method for coding and decoding a video picture
JP5739531B2 (en) 2010-07-27 2015-06-24 テルコーディア テクノロジーズ インコーポレイテッド Interactive projection and playback of related media segments on 3D facets
US9712847B2 (en) * 2011-09-20 2017-07-18 Microsoft Technology Licensing, Llc Low-complexity remote presentation session encoder using subsampling in color conversion space
US9324133B2 (en) * 2012-01-04 2016-04-26 Sharp Laboratories Of America, Inc. Image content enhancement using a dictionary technique
GB2522012B (en) * 2014-01-06 2017-04-19 Sony Interactive Entertainment Inc Apparatus and method of texture encoding and decoding
CN105261001A (en) * 2014-07-14 2016-01-20 王科 Image processing method and device
US9753625B2 (en) * 2015-03-17 2017-09-05 Adobe Systems Incorporated Image selection control
DE102015009981A1 (en) * 2015-07-31 2017-02-02 Eberhard Karls Universität Tübingen Method and apparatus for image synthesis
US11409791B2 (en) 2016-06-10 2022-08-09 Disney Enterprises, Inc. Joint heterogeneous language-vision embeddings for video tagging and search
CN106682424A (en) * 2016-12-28 2017-05-17 上海联影医疗科技有限公司 Medical image adjusting method and medical image adjusting system
US10311326B2 (en) * 2017-03-31 2019-06-04 Qualcomm Incorporated Systems and methods for improved image textures
EP3662670A1 (en) * 2017-12-08 2020-06-10 Huawei Technologies Co., Ltd. Polynomial fitting for motion compensation and luminance reconstruction in texture synthesis
WO2019110124A1 (en) 2017-12-08 2019-06-13 Huawei Technologies Co., Ltd. Frequency adjustment for texture synthesis in video coding
CN111434115B (en) * 2017-12-08 2021-09-14 华为技术有限公司 Method and related device for coding and decoding video image comprising pixel points
US10791343B2 (en) * 2018-03-13 2020-09-29 Google Llc Mixed noise and fine texture synthesis in lossy image compression
US10558761B2 (en) * 2018-07-05 2020-02-11 Disney Enterprises, Inc. Alignment of video and textual sequences for metadata analysis
US11315256B2 (en) * 2018-12-06 2022-04-26 Microsoft Technology Licensing, Llc Detecting motion in video using motion vectors
US10810782B1 (en) 2019-04-01 2020-10-20 Snap Inc. Semantic texture mapping system
US10951902B2 (en) * 2019-06-12 2021-03-16 Rovi Guides, Inc. Systems and methods for multiple bit rate content encoding
US11551385B1 (en) * 2021-06-23 2023-01-10 Black Sesame Technologies Inc. Texture replacement system in a multimedia

Family Cites Families (44)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5097427A (en) 1988-07-06 1992-03-17 Hewlett-Packard Company Texture mapping for computer graphics display controller system
US5097437A (en) 1988-07-17 1992-03-17 Larson Ronald J Controller with clocking device controlling first and second state machine controller which generate different control signals for different set of devices
DE3930147A1 (en) 1989-09-09 1991-03-21 Danfoss As HYDRAULIC STEERING UNIT
US5204920A (en) 1990-01-12 1993-04-20 U.S. Philips Corporation Method and apparatus for region and texture coding
EP0547696B1 (en) 1991-12-18 1999-04-21 Laboratoires D'electronique Philips S.A.S. System for the transmission and/or storage of signals corresponding to textured images
FR2695743A1 (en) 1992-09-16 1994-03-18 Philips Electronique Lab Textured image processing system, texture analyzer and texture synthesizer.
JP3030485B2 (en) * 1994-03-17 2000-04-10 富士通株式会社 Three-dimensional shape extraction method and apparatus
US5926576A (en) * 1994-03-30 1999-07-20 Newton; Dale C. Imaging method and system concatenating image data values to form an integer, partition the integer, and arithmetically encode bit position counts of the integer
DE69636695T2 (en) * 1995-02-02 2007-03-01 Matsushita Electric Industrial Co., Ltd., Kadoma Image processing device
US5872867A (en) 1995-08-04 1999-02-16 Sarnoff Corporation Method and apparatus for generating image textures
JPH09135447A (en) * 1995-11-07 1997-05-20 Tsushin Hoso Kiko Intelligent encoding/decoding method, feature point display method and interactive intelligent encoding supporting device
US5764814A (en) 1996-03-22 1998-06-09 Microsoft Corporation Representation and encoding of general arbitrary shapes
JP3510733B2 (en) 1996-04-03 2004-03-29 ペンタックス株式会社 Video signal processing device connectable to electronic endoscope
US6381364B1 (en) * 1996-12-31 2002-04-30 Intel Corporation Content texture sensitive picture/video encoder/decoder
US6850249B1 (en) * 1998-04-03 2005-02-01 Da Vinci Systems, Inc. Automatic region of interest tracking for a color correction system
US6215526B1 (en) * 1998-11-06 2001-04-10 Tivo, Inc. Analog video tagging and encoding system
KR100444776B1 (en) * 1999-02-05 2004-08-16 삼성전자주식회사 Image texture retrieving method and apparatus thereof
JP3325239B2 (en) 1999-06-09 2002-09-17 日本テレビ放送網株式会社 Caption material creation system, caption material creation method and recording medium storing caption material creation program
US6647132B1 (en) 1999-08-06 2003-11-11 Cognex Technology And Investment Corporation Methods and apparatuses for identifying regions of similar texture in an image
US6594391B1 (en) * 1999-09-03 2003-07-15 Lucent Technologies Inc. Method and apparatus for texture analysis and replicability determination
US6512846B1 (en) * 1999-11-29 2003-01-28 Eastman Kodak Company Determining orientation of images containing blue sky
US6421386B1 (en) 1999-12-29 2002-07-16 Hyundai Electronics Industries Co., Ltd. Method for coding digital moving video including gray scale shape information
US6642940B1 (en) 2000-03-03 2003-11-04 Massachusetts Institute Of Technology Management of properties for hyperlinked video
GB2362533A (en) 2000-05-15 2001-11-21 Nokia Mobile Phones Ltd Encoding a video signal with an indicator of the type of error concealment used
KR100450793B1 (en) 2001-01-20 2004-10-01 삼성전자주식회사 Apparatus for object extraction based on the feature matching of region in the segmented images and method therefor
JP3876227B2 (en) 2001-02-06 2007-01-31 コーニンクレッカ フィリップス エレクトロニクス エヌ ヴィ A preprocessing method applied to the texture of objects of arbitrary shape
US6919903B2 (en) * 2001-03-02 2005-07-19 Mitsubishi Electric Research Laboratories, Inc. Texture synthesis and transfer for pixel images
US7023447B2 (en) * 2001-05-02 2006-04-04 Eastman Kodak Company Block sampling based method and apparatus for texture synthesis
US7136072B2 (en) * 2001-08-03 2006-11-14 Hewlett-Packard Development Company, L.P. System and method for performing texture synthesis
US6977659B2 (en) * 2001-10-11 2005-12-20 At & T Corp. Texture replacement in video sequences and images
US7606435B1 (en) 2002-02-21 2009-10-20 At&T Intellectual Property Ii, L.P. System and method for encoding and decoding using texture replacement
EP1602242A2 (en) 2003-03-03 2005-12-07 Koninklijke Philips Electronics N.V. Video encoding
US7426285B2 (en) 2004-09-21 2008-09-16 Euclid Discoveries, Llc Apparatus and method for processing video data
US7308443B1 (en) 2004-12-23 2007-12-11 Ricoh Company, Ltd. Techniques for video retrieval based on HMM similarity
EP1958453B1 (en) 2005-11-30 2017-08-09 Koninklijke Philips N.V. Encoding method and apparatus applying coefficient reordering
CA2654960A1 (en) 2006-04-10 2008-12-24 Avaworks Incorporated Do-it-yourself photo realistic talking head creation system and method
US8207989B2 (en) 2008-12-12 2012-06-26 Microsoft Corporation Multi-video synthesis
US8320666B2 (en) 2009-08-14 2012-11-27 Genesis Group Inc. Real-time image and video matting
JP5634111B2 (en) 2010-04-28 2014-12-03 キヤノン株式会社 Video editing apparatus, video editing method and program
US9067320B2 (en) 2010-06-09 2015-06-30 Disney Enterprises, Inc. Robotic texture
WO2012090181A1 (en) 2010-12-29 2012-07-05 Nokia Corporation Depth map coding
US8712122B2 (en) 2011-03-31 2014-04-29 International Business Machines Corporation Shape based similarity of continuous wave doppler images
US20130176390A1 (en) 2012-01-06 2013-07-11 Qualcomm Incorporated Multi-hypothesis disparity vector construction in 3d video coding with depth
US20130329987A1 (en) 2012-06-11 2013-12-12 Genesis Group Inc. Video segmentation method

Also Published As

Publication number Publication date
US20030076334A1 (en) 2003-04-24
CA2407143C (en) 2006-08-29
US20080055332A1 (en) 2008-03-06
US7545989B1 (en) 2009-06-09
US7995072B2 (en) 2011-08-09
US7307639B1 (en) 2007-12-11
US20110292993A1 (en) 2011-12-01
US10282866B2 (en) 2019-05-07
US20070268301A1 (en) 2007-11-22
US6977659B2 (en) 2005-12-20
US20050243099A1 (en) 2005-11-03
US8896616B2 (en) 2014-11-25
US7564465B2 (en) 2009-07-21
US20150077427A1 (en) 2015-03-19

Similar Documents

Publication Publication Date Title
CA2407143A1 (en) Texture replacement in video sequences and images
AU2003241110A1 (en) Fgs decoder based on quality estimated at the decoder
MY182370A (en) Improved interpolation of video compression frames
HK1089030A1 (en) A method and associated device for filtering digital video images
EP2544451A3 (en) Method and system for digital decoding 3D stereoscopic video images
CA2245172A1 (en) Temporal and resolution layering in advanced television
EP2209319A3 (en) Video encoder and method of encoding video
EP2268036A3 (en) Video signal encoding and decoding method
ATE245329T1 (en) CODING AND DECODING OF GRAPHIC SYMBOLS
CA2574047A1 (en) Video encoding method and apparatus and video decoding method and apparatus
MY161138A (en) Parameter set and picture header in video coding
EP1250012A3 (en) Adaptive quantization based on bit rate prediction and prediction error energy
TW200708107A (en) Adaptive frame skipping techniques for rate controlled video encoding
CA2201516A1 (en) Method and system for a user to manually alter the quality of previously encoded video frames
AU2003206051A8 (en) Fgst coding method employing higher quality reference frames
EP2535871A3 (en) Video-information decoding apparatus and method
EP1437896A3 (en) Method and apparatus for improving MPEG picture compression
EP1408696A4 (en) Motion vector deriving method, dynamic image encoding method, and dynamic image decoding method
MY139318A (en) Moving picture coding method and moving picture decoding method
EP1261212A3 (en) Method and apparatus for coding and decoding image data
TW200603636A (en) A video encoder and method of video encoding
EP1182877A3 (en) Sending progressive video sequences
EP1146747A3 (en) A method for encoding and decoding moving picture signals
CA2311770A1 (en) Video image coding device and video image decoding device
CA2440348A1 (en) Testing loops for channel codecs

Legal Events

Date Code Title Description
EEER Examination request
MKLA Lapsed

Effective date: 20151005