WO2004086751A2 - Procede d'evaluation de visibilite et d'exposition de logotypes dans une video - Google Patents

Procede d'evaluation de visibilite et d'exposition de logotypes dans une video Download PDF

Info

Publication number
WO2004086751A2
WO2004086751A2 PCT/CH2004/000182 CH2004000182W WO2004086751A2 WO 2004086751 A2 WO2004086751 A2 WO 2004086751A2 CH 2004000182 W CH2004000182 W CH 2004000182W WO 2004086751 A2 WO2004086751 A2 WO 2004086751A2
Authority
WO
WIPO (PCT)
Prior art keywords
tuples
logo
pattern
points
invariants
Prior art date
Application number
PCT/CH2004/000182
Other languages
English (en)
Other versions
WO2004086751A3 (fr
Inventor
Sergei Startchik
Original Assignee
Sergei Startchik
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sergei Startchik filed Critical Sergei Startchik
Priority to EP20040722778 priority Critical patent/EP1611738A2/fr
Publication of WO2004086751A2 publication Critical patent/WO2004086751A2/fr
Publication of WO2004086751A3 publication Critical patent/WO2004086751A3/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
    • G06V20/635—Overlay text, e.g. embedded captions in a TV program
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
    • G06V10/757—Matching configurations of points or features

Definitions

  • Keywords logo visibility, Multimedia, Advertisement, Video sequence, geometric invariants, chromatic invariants, point tuples.
  • the visibility and exposure of logos in video is of important commercial concern for advertisement on television.
  • a logo placed on the flat or round surface appears from different viewpoints, partially visible or occluded.
  • the logo visibility time, its size (relative to the screen), position (relative to the terrain) and percent of the non-occluded part allows to directly compute its advertising impact.
  • This patent discloses a method for automatic computation of the visible part of the given logo in the video sequence. For each frame of the video sequence the method computes four parameters that describe the visibility of a given logo.
  • First parameter is the percentage of the visible part with respect to the whole logo.
  • Second and third are the position and size of the logo with respect to the image.
  • the present invention belongs to the domain of registration (or matching) methods in computer vision.
  • methods of registration of two-dimensional (planar) patterns like logos under pro- jective transformation (modelling the image acquisition by a camera) are addressed.
  • the registration of patterns is based on registration of features of those patterns. Therefore, registration methods are classified with respect to features that are used.
  • some methods use template matching where an example of the logo original is matched to the image by applying all possible transformations to the original and evaluate the match. The best found match is the position of the logo.
  • This method is very sensitive to occlusion, size changes and blur.
  • Another family of methods use local features of the pattern. In particular, if the shape is composed of several curves and edges, comparison of their features can be used as registration. How- ever, the limitation of this method is the requirement for the pattern/logo to have special features th ⁇ t can not be guaranteed in practice. The algorithm is unable to locate logos that do not have such feature s .
  • the advantages of the new method is its capacity to work with real conditions including high occlusion of the logo, illumination changes and blur. These latter conditions are very frequent in sport events (see Fig. 2) thus making the method very useful for this application.
  • the method for registration of visible part of the logo is based on the dense and massive matching of point tuples.
  • Each point tuple is matched using the comparison of its properties described by values that are invariant to geometric transformation produced by a camera and chromatic transformations produced by illumination changes.
  • an example image is used to specify a user-defined zone.
  • the obtained representation is used to search for learned logo in a given video sequence by analysing each individual frame , .
  • the stage of computing visibility of a particular logo in 1 frame consist of registering numerous learned tuples corresponding to example image and verifying their consistency.
  • the stage of registration of individual tuple is explained in section III.1.
  • the stage for computing the registration of a particular logo from registered tuples is described in III.4.
  • the main advantage of the invention is that a top-down approach is used to look for point tuples and that no support region is used to find points. Therefore, high resolution matching can be performed independently of any occlusion.
  • Registering a tuple of several points P , in two images is done by comparing joint geometric and chromatic properties of those points. Depending on the complexity of transformation, involved in image formation a various number of points and their properties is required to perform the registration robustly.
  • the geometric transformation from the plane with the logo in the scene to the sensor of the camera can be modelled by 2D affine, 2D projective or more complex transformation. Some additional transformation is required to account for lens distortion, but in the case of television cameras the distance to the object reduces the distortion effect. If 2D affine transformation is used to model camera, four points are necessary to compute two values that are independent on that transformation and thus characterizes the tuple independently of that information. If a 2D projective transformation is used, already five points are necessary to compute two different geometric invariant values.
  • the known geometric invariant to projective transformation is the two cross-ratios of five points defined by:
  • Another property of the tuple of points that can be used as a stable characteristic is the value based on chromatic values of individual points and that is invariant to illumination changes. If the changing illumination (sun, clouds, reflections) is modelled by the scaling transformation in rgb domain, two points are needed to compute one invariant value. The invariant to this chromatic transformation is the ratio of several rgb values. Therefore, for five points selected above four invariant values can be computed.
  • the search should be done without relying on the pixel grid of the image being searched. Therefore, a sub- pixeling computations should be done during search. This is achieved by selecting instead of individual pixels the triples of neighbour pixels whose intensity (or color) values establish an interval containing the value that is searched for as shown in Fig. 7.
  • the algorithms with subpixel precision is described in the Fig. 6.
  • triangles of pixels (P Q t , R t as shown in Fig. 7) are selected that have values defining an interval containing the value. For example, if the searched intensity value is 128 in graylevel intensity, the triangle of pixels with values 140, 124, 127 does contain the 128 value somewhere between them and thus can be selected.
  • each point S t position can vary inside the triangle P,, Q t , R, , its coordinates are expressed in barycentric form with respect to the three points defining the triangle.
  • P t [x ( (w,, v f ), >>,( «,, v ( )] .
  • four points will be defined with eight parameters.
  • an interpolated intensity value can be computed c u t , v ( ) .
  • the geometric constraint is satisfied by checking whether the corresponding comers of triangles P v P 2 , P ⁇ , P , P 5 satisfy the invariant values (withing certain tolerance).
  • the chromatic value is computed by interpolation and the final expression for c 5 will depend on eight parameters c 5 ⁇ u ⁇ , v,, w 2 , v 2 , M 3 , V 3 , U 4 , V 4 ) .
  • c 5 ⁇ u ⁇ , v,, w 2 , v 2 , M 3 , V 3 , U 4 , V 4 ) .
  • the algorithm for learning a suitable representation of a given pattern is outlined in Fig. 4 and illustrated in Fig. 1.
  • an area Learning zone (10) that contains an unoccluded pattern Logo (6) is specified by the user as in Fig. 1. Then, this area is searched for tuples of points that are representative in this whole area. In other words, tuples need to be found whose properties (geometric and chromatic invariant values) are sufficiently different from properties of all other point tuples in the learning zone. Tuples, satisfying such unicity conditions will not have similar tuples in the whole learning area and could, thus, be easily found at the search stage without mismatches.
  • the algorithm of search for such unique tuples is described in Fig. 4.
  • the Fig. 1 illustrates the operation of representative tuple research.
  • the Candidate tuple (14) is selected so that all points have different color or intensity values to increase discrimination between points.
  • a fixed neighbourhood Search zone (9) is defined around each of the points of the tuple in the Image frame (8).
  • Potential neighbour tuple (13) from those search zones its geometric and chromatic invariants are computed and their values are compared with invariants of the Candidate tuple (14) that is being tested for unicity.
  • the Search zone (9) is equivalent to the whole Learning zone (10) to obtain full unicity.
  • Tuples that satisfy this unicity criterion are stored as part of logo representation.
  • the use of those pixels in other tuples is reduced (they, however, will be used in neighbor tuples for comparison).
  • the redundancy of representation is important, the participation of each pixel in a fixed number of tuples (for example ten). One avoids the use of one pixel in too many tuples to avoid extreme dependence on this point.
  • the learning process will gradually use all the pixels in the Learning zone (10) (c.f. Fig. 1 ). If each pixel will be used several times in one or more Learned tuple (11) this area will be covered with several "layers" of tuples in a dense manner. With more tuples using a pixel, the whole representation will gain in reliability. At the end the whole representation will correspond to the large number of tuples characterised by their absolute intensity values, geometric invariant values and relative chromatic. When such dense representation is constructed, it can then be used for locating this logo in a new image.
  • the operation of locating a logo in a new image corresponds to the search of learned tuples in this image.
  • the whole image will be analysed for the presence of the learned tuples one by one.
  • the density of representation with tuples allows to deal with almost random occlusion since every point is virtually related to (at least) four other points at various parts of the image.
  • the Logo (6) can be occluded in any way by an Occluding object (7) as shown in Fig. 2, there will always be visible points not hidden by this object and that are related by tuples.
  • the position of the logo frame is estimated. Every learned tuple contain information about its position relatively to the Learning zone (10). This information (which is in fact a transformation) could be inversed and the position of the Learning zone (10) (or reference frame) obtained from the position of the tuple. There will be false matches between tuples, therefore taking the frarne position confirmed by the majority of found tuples allows to find the right frame. This frame is also used to compute the position of the logo on the screen as well as its size relative to the screen.
  • the visible part This is estimated as the number of pixels inside the frame covered by found tuples to the total number of pixels in the frar ⁇ e. This is done by computing the ratio of visually found points to the total number of points associated with particular logo. A different measure can be applied since not all points are visible all the time and something relative to pixels can be applied.
  • Another operation that can be performed once all the point tuples were identified, is the modification of the visible part of the logo in order to improve its visual quality or provide a neat image of the logo. If the number of the identified points is sufficient, one can add many other points that can improve the resolution of the logo if it is viewed as a remote object.
  • the last operation that can be performed is the replacement of the logo by visual information that is perceptually different from the logo appearance. This operation might be useful for hiding the logo, if its visibility in this video sequence is not desired.
  • This transformation can be modelled with several approximating transformations like for example scaling of each of rgb channels, linear transformation in rgb space, etc.
  • the scaling transformation corresponds to the scaling of the every chromatic channel by an independent scaling factor:
  • these invariant values for every pair of pixels in the tuple. Then, these values will play a role of additional constraint for selection of tuples. This selection will thus be independent of the illumination changes, all pairs that have that value are retrieved. In the search image, these values can be computed only by combining several (N-tuples). Even if the number of those corfc. bination is high we are obliged to compute them since no other information is available.
  • This st ⁇ p is an optional step in the search algorithm that is outlined in
  • Fig. 1 An example image of a logo and tuple being learned
  • Fig. 5 Algorithm for finding learned 5-tuple in the new image (pixel-based algorithm).
  • Fig. 6 Algorithm for finding learned 5-tuple in the new image (sub-pixel algorithm).
  • the algorithms of learning tuples and searching them can be implemented as an image processing software modules. This software can be run on a DSP within an embedded system or on a stardard computer that is connected to a camera.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • Databases & Information Systems (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)
  • Illuminated Signs And Luminous Advertising (AREA)

Abstract

La visibilité et l'exposition de logotypes dans une vidéo constitue une préoccupation commerciale importante pour la publicité à la télévision. Au cours d'un match de tennis par exemple, un logotype placé sur une surface plane ou arrondie apparaît à partir de points de vue différents, partiellement visible ou occultée. Le temps de visibilité du logotype, sa taille (relativement à l'écran), sa position (relativement au terrain) et le pourcentage de la partie non occultée permettent de calculer directement son impact publicitaire. La présente invention concerne un procédé de calcul automatique de la partie visible du logotype donné dans la séquence vidéo. Pour chaque trame de la séquence vidéo, le procédé permet de calculer quatre paramètres qui décrivent la visibilité d'un logotype donné. Premièrement, on trouve le pourcentage de la partie visible par rapport au logotype dans son ensemble. Deuxièmement et troisièmement viennent la position et la taille du logotype par rapport à l'image. Quatrièmement, on trouve la qualité (flou, mouvement) de la partie visible. Les avantages du nouveau procédé sont la capacité à travailler avec des séquences vidéo réelles comprenant une occultation, des changements d'éclairage et un flou élevés. Ces dernières conditions sont réalistes, rendant ainsi le procédé très utile et robuste pour ladite application. Les informations obtenues peuvent être utilisées pour modifier ou remplacer un logotype ainsi que pour améliorer sa qualité.
PCT/CH2004/000182 2003-03-27 2004-03-24 Procede d'evaluation de visibilite et d'exposition de logotypes dans une video WO2004086751A2 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP20040722778 EP1611738A2 (fr) 2003-03-27 2004-03-24 Procede d'evaluation de visibilite et d'exposition de logotypes dans une video

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CHPCT03/00199 2003-03-27
CH0300199 2003-03-27

Publications (2)

Publication Number Publication Date
WO2004086751A2 true WO2004086751A2 (fr) 2004-10-07
WO2004086751A3 WO2004086751A3 (fr) 2005-02-03

Family

ID=33035093

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CH2004/000182 WO2004086751A2 (fr) 2003-03-27 2004-03-24 Procede d'evaluation de visibilite et d'exposition de logotypes dans une video

Country Status (2)

Country Link
EP (1) EP1611738A2 (fr)
WO (1) WO2004086751A2 (fr)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2008109608A1 (fr) * 2007-03-05 2008-09-12 Sportvision, Inc. Mesure automatique de l'efficacité de publicités
US7783130B2 (en) 2005-01-24 2010-08-24 The United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration Spatial standard observer
CN106792153A (zh) * 2016-12-01 2017-05-31 腾讯科技（深圳）有限公司 一种视频标识处理方法及装置
CN107153809A (zh) * 2016-03-04 2017-09-12 无锡天脉聚源传媒科技有限公司 一种确认电视台图标的方法及装置
CN113469216A (zh) * 2021-05-31 2021-10-01 浙江中烟工业有限责任公司 零售终端海报识别与完整性判断方法、系统及存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4817166A (en) 1986-05-05 1989-03-28 Perceptics Corporation Apparatus for reading a license plate
WO2000007367A2 (fr) 1998-07-28 2000-02-10 Koninklijke Philips Electronics N.V. Appareil et procede de localisation d'une pub se trouvant dans un flux de donnees video
TW434520B (en) 1998-06-30 2001-05-16 Sony Corp Two-dimensional code recognition processing method, device therefor and medium
WO2001052547A1 (fr) 2000-01-14 2001-07-19 Koninklijke Philips Electronics N.V. Insertion simplifiee d'un logo dans un signal code
US20030016921A1 (en) 2001-07-23 2003-01-23 Seung-Kyu Paek Video reproducing/recording system to change a logo image/sound, and method of changing the logo image/sound

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4817166A (en) 1986-05-05 1989-03-28 Perceptics Corporation Apparatus for reading a license plate
TW434520B (en) 1998-06-30 2001-05-16 Sony Corp Two-dimensional code recognition processing method, device therefor and medium
WO2000007367A2 (fr) 1998-07-28 2000-02-10 Koninklijke Philips Electronics N.V. Appareil et procede de localisation d'une pub se trouvant dans un flux de donnees video
US6100941A (en) 1998-07-28 2000-08-08 U.S. Philips Corporation Apparatus and method for locating a commercial disposed within a video data stream
WO2001052547A1 (fr) 2000-01-14 2001-07-19 Koninklijke Philips Electronics N.V. Insertion simplifiee d'un logo dans un signal code
US20030016921A1 (en) 2001-07-23 2003-01-23 Seung-Kyu Paek Video reproducing/recording system to change a logo image/sound, and method of changing the logo image/sound

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
DOERMAN D. S. ET AL.: "Logo recognition using geometric invariants", 2ND INT. CONF. ON DOCUMENT ANALYSIS AND RECOGNITION, October 1993 (1993-10-01), pages 894 - 897
G. MEDIONI; G.GUY; H. ROM; A. FRANÇOIS: "Real-Time Billboard Substitution in a Video Stream", PROC. 10TH TYRRHENIAN INTERNAT. WORKSHOP ON DIGITAL COMMUNICATION, 1998, pages 71 - 84

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7783130B2 (en) 2005-01-24 2010-08-24 The United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration Spatial standard observer
US8139892B2 (en) 2005-01-24 2012-03-20 The United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration (Nasa) Spatial standard observer
WO2008109608A1 (fr) * 2007-03-05 2008-09-12 Sportvision, Inc. Mesure automatique de l'efficacité de publicités
CN107153809A (zh) * 2016-03-04 2017-09-12 无锡天脉聚源传媒科技有限公司 一种确认电视台图标的方法及装置
CN107153809B (zh) * 2016-03-04 2020-10-09 无锡天脉聚源传媒科技有限公司 一种确认电视台图标的方法及装置
CN106792153A (zh) * 2016-12-01 2017-05-31 腾讯科技（深圳）有限公司 一种视频标识处理方法及装置
CN106792153B (zh) * 2016-12-01 2020-07-28 腾讯科技（深圳）有限公司 一种视频标识处理方法、装置和计算机可读存储介质
CN113469216A (zh) * 2021-05-31 2021-10-01 浙江中烟工业有限责任公司 零售终端海报识别与完整性判断方法、系统及存储介质
CN113469216B (zh) * 2021-05-31 2024-02-23 浙江中烟工业有限责任公司 零售终端海报识别与完整性判断方法、系统及存储介质

Also Published As

Publication number Publication date
EP1611738A2 (fr) 2006-01-04
WO2004086751A3 (fr) 2005-02-03

Similar Documents

Publication Publication Date Title
Hu et al. Revisiting single image depth estimation: Toward higher resolution maps with accurate object boundaries
JP3740065B2 (ja) 領域分割された映像の領域特徴値整合に基づいた客体抽出装置およびその方法
Revaud et al. Epicflow: Edge-preserving interpolation of correspondences for optical flow
CN110210276A (zh) 一种移动轨迹获取方法及其设备、存储介质、终端
US7162102B2 (en) Method and system for compositing images to produce a cropped image
US7379583B2 (en) Color segmentation-based stereo 3D reconstruction system and process employing overlapping images of a scene captured from viewpoints forming either a line or a grid
Koschan et al. Color active shape models for tracking non-rigid objects
US20040252886A1 (en) Automatic video object extraction
Venkatesh et al. Efficient object-based video inpainting
US9982995B2 (en) 3D scanner using structured lighting
CN101512601A (zh) 从图像中确定深度图的方法以及确定深度图的设备
CN111445389A (zh) 一种高分辨率图像的宽视角快速拼接方法
CN107154017A (zh) 一种基于sift特征点匹配的图像拼接方法
CN110866882A (zh) 基于深度置信度的分层联合双边滤波深度图修复算法
Stentoumis et al. A local adaptive approach for dense stereo matching in architectural scene reconstruction
Matas Colour-based object recognition
CN111402429B (zh) 一种尺度还原、三维重建方法、系统、存储介质及设备
CN113128433A (zh) 颜色迁移匹配特征的视频监控影像增强方法
EP1611738A2 (fr) Procede d'evaluation de visibilite et d'exposition de logotypes dans une video
Torres-Méndez et al. Reconstruction of 3d models from intensity images and partial depth
CN112733624B (zh) 室内密集场景的人流密度检测方法、系统存储介质及终端
JPH11510351A (ja) オブジェクト追跡の装置および方法
Patricio et al. A similarity-based adaptive neighborhood method for correlation-based stereo matching
Ross et al. Learning static object segmentation from motion segmentation
Palma et al. Enhanced visualization of detected 3d geometric differences

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A2

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BW BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE EG ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NA NI NO NZ OM PG PH PL PT RO RU SC SD SE SG SK SL SY TJ TM TN TR TT TZ UA UG US UZ VC VN YU ZA ZM ZW

AL Designated countries for regional patents

Kind code of ref document: A2

Designated state(s): BW GH GM KE LS MW MZ SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LU MC NL PL PT RO SE SI SK TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
WWE Wipo information: entry into national phase

Ref document number: 2004722778

Country of ref document: EP

WWP Wipo information: published in national office

Ref document number: 2004722778

Country of ref document: EP