US20020018070A1 - Video superposition system and method - Google Patents
Video superposition system and method Download PDFInfo
- Publication number
- US20020018070A1 US20020018070A1 US08/716,597 US71659796A US2002018070A1 US 20020018070 A1 US20020018070 A1 US 20020018070A1 US 71659796 A US71659796 A US 71659796A US 2002018070 A1 US2002018070 A1 US 2002018070A1
- Authority
- US
- United States
- Prior art keywords
- image
- graphic image
- foreground object
- graphic
- video
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
- G06T15/10—Geometric effects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/272—Means for inserting a foreground image in a background image, i.e. inlay, outlay
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/14—Systems for two-way working
- H04N7/141—Systems for two-way working between two video terminals, e.g. videophone
- H04N7/147—Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
- G06F3/1454—Digital output to display device ; Cooperation and interconnection of the display device with other functional units involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
- G06T2207/30201—Face
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G2340/00—Aspects of display data processing
- G09G2340/12—Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels
- G09G2340/125—Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels wherein one of the images is motion video
Definitions
- the present invention relates to the field of video superposition devices, and more particularly to multiple image source windowed display generation systems.
- a known video superposition system known as “chroma keying” employs a foreground image which is separated from an actual background by detection of a background screen chrominance value.
- a background screen chrominance value For example, a person is presented in front of a blue screen.
- a video processing circuit detects the chrominance level, producing a signal when the key color is detected.
- This color is generally a deep blue, for two reasons First, this color is generally uncommon in natural foreground scenes, so that artifacts are minimized. Second, this color represents an extreme, so that a single ended comparator may be used to produce the key signal.
- a video source switches a synchronized (genlocked) background video signal to the output.
- the foreground is output, while where the key color is detected, the background signal is output.
- U.S. Pat. No. 4,200,890 and 4,409,618 relate to digital video effects systems employing a chroma key tracking technique.
- U.S. Pat. No. 4,319,266 relates to a chroma keying system.
- U.S. Pat. No. 5,251,016 relates to a chroma keyer with secondary hue selector for reduced artifacts.
- 5,313,275 relates to a chroma processor including a look-up table or memory, permitting chroma key operation.
- U.S. Pat. No. 5 , 398 , 075 relates to the use of analog chroma key technology in a computer graphics environment.
- U.S. Pat. No. 5,469,536 relates to an image editing system including masking capability, which employs a computerized hue analysis of the image to separate a foreground object from the background.
- a number of spatial position sensor types are known. These include electromagnetic, acoustic, infrared, optical, gyroscopic, accelerometer, electromechanical, and other types.
- systems are available from Polhemus and Ascension which accurately measure position and orientation over large areas, using electromagnetic fields.
- Rangefinder systems are known, which allow the determination of a distance to an object.
- Known systems include optical focus zone, optical parallax, infrared, and acoustic methods.
- non-contact depth mapping systems which determine a depth profile of an object without physical contact with a surface of the object.
- U.S. Pat. No. 5,521,373 relates to a position tracking system having a position sensitive radiation detector.
- U.S. Pat. No. 4,988,981 relates to a glove-type computer input device.
- U.S. Pat. No. 5,227,985 relates to a computer vision system for position monitoring in three dimensions using non-coplanar light sources attached to a monitored object.
- U.S. Pat. No. 5,423,554 relates to a virtual reality game method and apparatus employing image chroma analysis for tracking a colored glove as an input to a computer system.
- U.S. Pat. No. 5,502,482 relates to a system for deriving a studio camera position and motion from the camera image by image analysis.
- U.S. Pat. No. 5,513,129 relates to a method and system for controlling a computer-generated virtual environment with audio signals.
- the present invention employs a live video source, a background image source, a mask region generator and an overlay device which merges the foreground with the background image based on the output of the mask region generator.
- Two classes of mask region generators are provided; first, an “in-band” system is provided which acquires the necessary mask region boundaries based on the foreground image acquisition system, and second an “out-of-band” system which provides a separate sensory input to determine the mask region boundary.
- a preferred embodiment of the “in-band” system is a rangefinder system which operates through the video camera system, to distinguish the foreground object in the live video source from its native background based on differences in distance from the camera lens.
- this preferred embodiment of the system defines the boundary of the object through its focal plane or parallax.
- a preferred embodiment of the “out-of-band” system includes an absolute position and orientation sensor physically associated with the foreground object with a predetermined relationship of the sensor to the desired portion of the foreground object.
- the sensor may be an electromagnetic position sensor mounted centrally on top of the head with the mask region defined by an oval boundary below and in front of the position and orientation sensor.
- the foreground image is a portrait of a person
- the background image is a computer generated image of a figure.
- a position sensor tracks a head position in the portrait, which is used to estimate a facial area.
- the image of the facial area is then merged in an anatomically appropriate fashion with the background figure.
- the background image is, for example, an animated “character”, with a masked facial portion.
- the live video signal in this case includes, as the foreground image, a face, with the face generally having a defined spatial relation to the position sensor.
- the masked region of the character is generated, based on the output of the position sensor in an appropriate position, so that the face may be superimposed within the masked region.
- the live image of the face is presented within a mask of an animated character, presenting a suitable foundation for a consumer entertainment system.
- the mask may obscure portions of the face, as desired.
- Manual inputs or secondary position sensors for the arms or legs of the individual may be used as further control inputs, allowing the user to both control the computer generated animation and to become a part of the resultant image.
- This system may therefore be incorporated into larger virtual reality systems to allow an increased level of interaction, while minimizing the need for specialized environments.
- the region around the face is preferably provided with an image which appears as a mask.
- the background image may appear as a masked character, with the foreground image as a video image of a face within the mask region.
- the mask region may be independent of the video image data, or developed based on an image processing algorithm of the video image data.
- the composite output may be initially provided as a video image data independent mask which is modified over time, when the image is relatively static, for greater correspondence with the actual image.
- the preferred embodiment provides a background image which is tolerant of misalignments and misadjustments of the video image with the background image.
- this tolerance includes providing an edge portion of the mask which merges appropriately with a variety of types of facial images, e.g., men, women, children, and possibly pet animals.
- the system is not limited to a chroma key superposition system
- the information from the position sensor and the video camera allow simple extraction of the image of an individual's face in a more generalized computer graphic image, based on an estimate of its position.
- multiple individuals may be presented in a single graphic image, each interacting with his or her environment or with each other. While these individuals may be present in the same environment, for example, within the field of view of a single video camera, this ability to build complex images from multiple inputs allow individuals at remote locations within a computer network to interact while viewing each other's faces. Therefore, personalized multiplayer “video” games become possible.
- This same technology my also have uses outside the fields of entertainment, including communications and video conferencing.
- This personalized representation separated from its native background also forms the basis for a new style of multi-user graphic interface system.
- the position estimation system preferably acts as an input to a computer generated animation as the background image figure.
- the generation of the resulting combined image is performed through a chroma key system. Therefore, in such systems, the background figure image is provided with a key color in a facial region or desired superposed video image region(s) of the figure.
- the chroma key appears in the presented background, with the live video image overlayed in the chroma key region.
- chroma key technology is not the only method for combining the various image information, and in fact the process may be performed digitally in a computerized system.
- the position sensor defines a predefined window, which is translated around the video space.
- the orientation and distance of the foreground object from the video camera may be compensated.
- the shape of the window may be a regular shape or an outline of a foreground image.
- the image may be initially processed to determine the shape.
- the shape may be translated, resized or otherwise transformed as the window.
- the shape of the window may be periodically redetermined, but need not be recalculated in real time.
- the live video image is an image of a person having a face, with a position sensor mounted on top of a set of headphones.
- An oval mask region is defined with respect to the position sensor, so that the position of the face within the video image is predicted to be within the oval mask region.
- the position sensor also serves as an input to a computer animation graphic image generator, which generates an animated body in appropriate position and orientation for combining with the face. Further position sensors may be provided on the arms of the person, as further inputs to the computer animation graphic image generator, allowing further control over the resulting image.
- the computer animation graphic image includes a chroma key portion in a region intended for the facial image. The live video image is then merged with the computer animation graphic image and presented as a composite.
- the position tracking system is, for example, an Ascension position tracking system mounted centrally on a bridging portion on a set of headphones, worn by a person.
- the person is present within the image of a video camera, and the system calibrated to locate the position tracking system unit within the field of view of the video camera.
- the face of the person is estimated to be within an oval area approximately 10 inches down and 8 inches wide below the position tracking system sensor, when the person is facing the camera. Since the preferred position tracking sensor senses movement in six degrees of freedom, the window is altered to correspond to the expected area of presentation of the face in the image.
- the border between the live video image of the face and the animated character need not be presented as a simple oval region, and may include images which overlay the face, as well as complex boundaries.
- the preferred embodiment avoids the need for sophisticated image analysis, thereby allowing relatively simple and available components.
- known types of position sensors also provide orientation information which may be useful for providing control inputs into the background image generation system and also to control the position and shape of the mask region to compensate for skew, profile, tilting and other degrees of freedom of the object.
- the computer generated animated image responds to the position tracking sensor as an input, allowing the animation to track the movements of the person.
- While one preferred embodiment employs an Ascension tracking system, which, while of high quality, is also expensive, another preferred embodiment employs an acoustic sensor to determine location in three or more dimensions. This system is similar to the known “power glove” accessory for video games. Other types of position sensors may also be used.
- the present invention avoids the need for a defined background for a foreground image when electronically superimposing video images by providing a position sensor to provide information for determining a location of the desired foreground image in a foreground video stream.
- the position sensor thus minimizes the need for analysis of the foreground image stream, allowing relatively simple merging of the video streams.
- the use of the position sensor to define a mask region in the video image substantially reduces a computational complexity required to extract a facial portion from a video image, especially as compared to a typical digital image processing system.
- the margin of the face need not be determined with high precision in many instances, and therefore the background image which is generated to surround the facial image may be provided to include a degree of tolerance to this imprecision, such as a wide edge margin and avoidance of structures which should be precisely aligned with facial features.
- the use of the present system allows transmission of the masked portion of the image only, reducing the amount of information which must be transmitted and thus compressing the image data.
- a known paradigm for user interaction with computers is known as an “Avatar”, a computer generated representation of a user, which is generally wholly animated.
- These Avatars may be transmitted through a computer network system, for example the Internet, to allow a user to interact with a graphical environment of a system.
- these Avatars need not be completely computer generated, and may therefore include a real time video image of a face. This system therefore allows, with reduced computational requirements and limited bandwidth requirements, the personalization of Avatars.
- the present invention provides a new type of graphical user interface in which a user is represented as an actual image within a computer graphic space. Multiple users may therefore interact while viewing actual images of each other, even where the users are separated over nodes of a computer network.
- an in-band mask region determining system may operate based on the foreground video input device.
- the position sensing system need not include physically separate hardware.
- the video signal superposition system need not be an external chroma key superposition system, and may be integrated with the animation generation system.
- an outline of a major foreground object is determined, and the outline used to define a mask.
- the foreground object is irradiated with an unobtrusive radiation, e.g., infrared, which is detected by a special video camera or optical sensors.
- an optical transmitter e.g., one or more LEDs, preferably including a blue LED, is mounted on the headphones, visible to the video camera. The presence of an illuminated spot is detected, and a mask defined in relation to the position of the spot.
- a plurality of LEDs may be mounted, in a configuration sufficient to allow estimation of position and orientation.
- the position detecting system may operate through the video feed without requiring rigorous image analysis, which often cannot be performed in or near real time.
- the resulting image of the method according to the present system and method may be presented on a video monitor, transmitted over a video network for rendering at a remote site, or stored on a video storage medium, such as video tape.
- a video storage medium such as video tape.
- the opportunities for complex background generation become apparent.
- the image is not simply transient, a higher level of detail in the background image may be preferred, because the stored image may be reviewed a number of times.
- the background since the background is computer generated, it need not be constant.
- the foreground image and control signals e.g., position and orientation signals, may be stored on a CD-ROM, with the background image generated in real time on reproduction of the images on a computer system. Since the video image and the control parameters are stored, the reproducted image sequence need not be fixed, and may therefore vary based on a set of background parameters.
- An alternative set of embodiments provide different processing systems to capture the facial image for presentation. For example, the location of the image of the face may be identified, with the facial image texture mapped onto a computer generated image. Thus, the boundary between the foreground image and background image need not be a discrete edge, and the present invention therefore allows a more subtle merging of the images.
- the location of the foreground image need not be determined with a typical position sensor, and other systems may be used.
- the focal plane of the foreground object e.g., person
- the boundary of the foreground object may be determined by detecting a focal plane boundary.
- a rangefinder system may be used to obtain a depth map of a face in real-time, with the resulting data used as control parameters for a computer generated character's face.
- This rangefinder information allows use of facial expression as a control input, while reducing the need for strict image analysis.
- This depth information may also be employed to assist in texture mapping the video image information on the background.
- other objects or images may be tracked and used as control inputs.
- the background image need not be computer generated.
- the background image may also represent a video feed signal.
- the background image is a video image of a robot or computer automated mechanical structure, which, e.g., responds to the position and orientation inputs from the foreground input to provide coordination.
- the merging or foreground and background images in this case may be through the use of typical chroma key technology.
- a graphic image system comprising a source of a first signal representing a first image including a moving human subject having a bead with a face; an image position estimating system for estimating the instantaneous position of said head of said human subject; a source of a second signal representing a second image including a character having a head with a mask outline; and means, responsive to said position estimating system and to said first and second signals, for dynamically defining an estimated boundary of said face of said human subject in said first image and for merging the face in said first image, as limited by said estimated boundary, with the second image within the mask outline.
- It is also an object of the invention to provide a video system comprising a video input, receiving a video signal representing an image having a movable foreground object; a position tracking system for tracking a position of said movable foreground object; and means, responsive to said position tracking system, for dynamically defining an estimated boundary of said moveable foreground object in said image.
- the background video image preferably comprises a computer generated animated image stream.
- the position tracking system may be a radio frequency field sensor, an electro-acoustic transducer or an optical position sensing system.
- the position tracking system may have various degrees of freedom, for example two, three or six.
- the position tracking system may include a physical transducer mounted on the foreground object.
- the mask or estimated boundary may be geometric in shape, for example oval, round or having multiple discontinuous geometric segments.
- the position tracking system produces a position, and optionally orientation of the foreground object within the field of view of the video camera.
- the normal relation of foreground and background signals in a chroma key video superposition unit are reversed, with the “background” image having a region of chroma activating the keying circuit, to allow a foreground object to be presented.
- the location of the region is determined by the position tracking system.
- It is another object according to the present invention to provide a video system comprising, in combination: a video camera producing a first video signal defining a first image including a foreground object and a background; a position tracking system for identifying a position with respect to said foreground object, said foreground object having features in constant physical relation to said position; and a computer, responsive to said position tracking system, for defining a mask region separating said foreground object from said background.
- the computer preferably generates a second video signal including a portion corresponding to said mask region, wherein an image in the second video signal is responsive to said position tracking system.
- the mask region differentiates image regions which are processed differently.
- the mask region preferably is employed in a chroma key system to form a composite image of the first and second images, although an out-of-band signal may be used to define the mask region.
- the position tracking sensor determines a position and orientation of the foreground object, and is used to control a size, shape and position of the mask region.
- FIG. 1 shows a perspective view of a first embodiment of the invention, having an electromagnetic position tracking device
- FIG. 2 shows a perspective view of a second embodiment of the invention, having an acoustic position tracking device
- FIG. 3 shows a block diagram of a system according to the present invention
- FIG. 4 shows a flow chart of a method according to the present invention.
- FIG. 5 shows a multi-user networked embodiment according to the present invention.
- a video camera 2 is positioned near a video display 4 , which may be, for example, a rear projection video monitor.
- An electromagnetic transmitting antenna 6 is positioned within the environment, producing a time-varying magnetic field which is detected by a sensor 8 .
- the antenna 6 produces a field which has a characteristic spatial variation, so that the sensor 8 allows detection of the position and orientation of the sensor 8 with respect to the antenna 6 .
- the sensor 8 is mounted at the apex of the bridging portion of a set of headphones 10 . The position and orientation of the sensor 8 is calibrated.
- a computer 12 receives the position sensor 8 output, and is used to generate an animated graphic image 14 on the monitor 4 .
- the animation responds to the position sensor 8 as an input, for example performing corresponding changes in orientation and position.
- the animated figure of the graphic image 14 has a facial area 16 which is rendered in a deep blue color, having a border 18 which may be mask-like.
- the video image is optionally justified based on the output of the position sensor.
- a window 20 is generated in the video graphic image 14 at the expected location of the face 22 based on the position sensor 8 position and orientation.
- the window 20 with the face 22 image is then displayed, using a chroma key circuit keyed to the deep blue color, in the position of the mask
- the senor 8 may be used to control animation with a portion of a live video image superimposed on a computer generated graphic image 14 , responding to a motion of an object in the live video image.
- the sensor 8 may be used to control animation with a portion of a live video image superimposed on a computer generated graphic image 14 , responding to a motion of an object in the live video image.
- FIG. 2 a simpler type of image position sensing system is employed in the system generally according to the description of FIG. 1.
- the position sensor 24 , 26 is sensitive to position, but not orientation, and further has a limited range.
- the computer system 28 presumes that the live video image 30 includes an image of a person 32 looking directly at the video camera 2 , so that the position of the mask 34 in the video image 30 is altered. While this presents reduced opportunity for animation control, variations in position may be used, within the range of the display 4 . Thus, the strategy of use of the position sensor may differ.
- the system according to the second embodiment has another type of application.
- a distracting background is present, which further increases a required bandwidth to transmit a compressed image. Therefore, by determining the position of the head of a videoconference participant, the head may be selectively processed. While headphones are not required in this application, in situations with distracting backgrounds, often there is also stray noise, so that the headphones may be desirable.
- the acoustic position sensing system 24 , 26 operates on a triangulation technique.
- the position sensor element 26 mounted on the headphones 10 in this instance is an ultrasonic transmitter. By detecting a differential amplitude, as well as phase delay, in a fixed receiver 24 location, a position of the transmitter 26 in space may be determined.
- FIG. 3 shows a configuration of the electronics hardware according to one embodiment according to the present invention.
- the system animation 40 receives a position input signal 42 from a position tracking system 54 which is used to determine a mask position within the live video feed 48 of a foreground object 56 and also as a control parameter for a computer generated animation. Other control inputs may also be received, such as arm or wrist position, joysticks, or other known input devices.
- the live video 48 mask is then overlayed in a chroma key unit 46 over a computer animated image 50 and displayed on a monitor 52 . It is noted that completely digital processing of the image may also be used. However, the chroma key technology requires a reduced computational load and may potentially be implemented with lower cost. Since the mask location, as determined by a position transducer 58 , may change within the live video 48 image, the mask image may optionally be translated to a desired location within the composite image.
- FIG. 4 shows a flow diagram of a method according to the present invention.
- the position of the position sensor is calibrated with respect to the live video image 68 .
- a person wears the headphones.
- a mask location is defined 62 in the live video image by determining the position of the position sensor 60 .
- the output of the position sensor is also used to control animation 64 , generally to produce a character who moves corresponding to the movements of the position sensor.
- the face of the animated character is rendered in a chroma key color 66 .
- a separate chroma key unit receives the animated image and the live video image, with the live video corresponding to the mask portion overlayed where the chroma key is active 70 .
- the superposed image is then output 72 .
- FIG. 5 An embodiment of the invention is provided as shown in FIG. 5.
- a plurality of users 81 , 82 interact with a computer network system through various nodes.
- the communication protocol is, for example, TCP/IP over Ethernet 10BaseT.
- a typical user station includes a personal computer (PC) 93 , 105 running a graphical user interface, e.g., Windows NT or Solaris.
- the personal computer 93 , 105 includes a number of adapter cards or functional interfaces.
- Each computer generally includes a video display adapter 90 , 108 , a frame grabber 91 , 106 , a control for a video camera 92 , 107 , and optionally a hardware MPEG encoder/decoder system 94 , 109 .
- the video compression and decompression may also be implemented on the host personal computer 93 , 105 .
- the personal computers communicate with each other over Ethernet 10BaseT networks 96 , 102 , using TCP/IP protocol. Therefore, each personal computer 93 , 105 includes a TCP/IP stack 95 , 103 , which is generally a software construct.
- the computer network includes, for example, LAN hubs 97 , 101 and WAN routers or bridges 99 , 100 .
- Each user has a video camera 86 , 112 mounted centrally over a video monitor 83 , 115 .
- the video camera 86 , 112 control 92 , 107 allows control over video camera parameters, including lens 113 , 85 focus and zoom.
- the video camera 86 , 112 intermittently scans through the various focal planes of objects within its field of view, with an edge contrast determination algorithm operative to determine optimum focus for any region.
- This edge contrast determination algorithm may be a predetermined program within the personal computer 93 , 105 .
- the optimum focus edge serves as a mask region 118 for the user's 81 head, and mask region 119 for user's 82 head, situated in front of the monitor 93 , 115 .
- the area within the mask regions 118 , 119 are retained, while outside the mask regions 118 , 119 are masked.
- the outline and position of the mask regions 118 , 119 serve as control parameters for an Avatar 81 ′, 82 ′, which may be transmitted as necessary through the computer network, along with a compressed representation of the image within the mask regions 118 , 119 .
- the Avatar 81 ′, 82 ′ need not be presented identically on each display, and therefore may have distinct interaction at each local site.
- Suitable compression for this image may include JPEG, MPEG, H.320, H.324, H.261, GIF, wavelet and fractal image compression, as well as other known compression formats.
- MPEG encoder/decoder system 94 , 109 is available, MPEG is preferred because of its bandwidth efficiency.
- the Avatars 81 ′, 82 ′ then form a part of a network graphic user interface, in which users 81 , 82 act and interact within the computer environment space 84 , 114 through graphic representations having personalized and real time varying facial expressions. This personalized interface system may reduce stress for novice users and provides an efficient and innate means of communication for people within the network.
- the personalized Avatar 81 ′, 82 ′ generation is simplified and the data transmission requirements are reduced.
Abstract
Description
- The present invention relates to the field of video superposition devices, and more particularly to multiple image source windowed display generation systems.
- A known video superposition system known as “chroma keying” employs a foreground image which is separated from an actual background by detection of a background screen chrominance value. Thus, for example, a person is presented in front of a blue screen. A video processing circuit detects the chrominance level, producing a signal when the key color is detected. This color is generally a deep blue, for two reasons First, this color is generally uncommon in natural foreground scenes, so that artifacts are minimized. Second, this color represents an extreme, so that a single ended comparator may be used to produce the key signal.
- When the key signal occurs, a video source switches a synchronized (genlocked) background video signal to the output. Thus, where the key level in the foreground is not detected, the foreground is output, while where the key color is detected, the background signal is output. This technology is well established, and many variations and modifications exist. U.S. Pat. No. 4,200,890 and 4,409,618 relate to digital video effects systems employing a chroma key tracking technique. U.S. Pat. No. 4,319,266 relates to a chroma keying system. U.S. Pat. No. 5,251,016 relates to a chroma keyer with secondary hue selector for reduced artifacts. U.S. Pat. No. 5,313,275 relates to a chroma processor including a look-up table or memory, permitting chroma key operation. U.S. Pat. No.5,398,075 relates to the use of analog chroma key technology in a computer graphics environment. U.S. Pat. No. 5,469,536 relates to an image editing system including masking capability, which employs a computerized hue analysis of the image to separate a foreground object from the background.
- Computer generated graphics are well known, as are live video windows within computer graphics screens. U.S. Pat. No. 3,899,848 relates to the use of a chroma key system for generating animated graphics. U.S. Pat. No. 5.384,912 relates to a computer animated graphics system employing a chroma key superposition technique. U.S. Pat. No. 5,345,313 relates to an image editing system for taking a background and inserting part of an image therein, relying on image analysis of the foreground image. U.S. Pat. No. 5,394,517 relates to a virtual reality, integrated real and virtual environment display system employing chroma key technology to merge the two environments.
- A number of spatial position sensor types are known. These include electromagnetic, acoustic, infrared, optical, gyroscopic, accelerometer, electromechanical, and other types. In particular, systems are available from Polhemus and Ascension which accurately measure position and orientation over large areas, using electromagnetic fields.
- Rangefinder systems are known, which allow the determination of a distance to an object. Known systems include optical focus zone, optical parallax, infrared, and acoustic methods. Also known are non-contact depth mapping systems which determine a depth profile of an object without physical contact with a surface of the object.
- U.S. Pat. No. 5,521,373 relates to a position tracking system having a position sensitive radiation detector. U.S. Pat. No. 4,988,981 relates to a glove-type computer input device. U.S. Pat. No. 5,227,985 relates to a computer vision system for position monitoring in three dimensions using non-coplanar light sources attached to a monitored object. U.S. Pat. No. 5,423,554 relates to a virtual reality game method and apparatus employing image chroma analysis for tracking a colored glove as an input to a computer system.
- U.S. Pat. No. 5,502,482 relates to a system for deriving a studio camera position and motion from the camera image by image analysis. U.S. Pat. No. 5,513,129 relates to a method and system for controlling a computer-generated virtual environment with audio signals.
- The present invention employs a live video source, a background image source, a mask region generator and an overlay device which merges the foreground with the background image based on the output of the mask region generator. Two classes of mask region generators are provided; first, an “in-band” system is provided which acquires the necessary mask region boundaries based on the foreground image acquisition system, and second an “out-of-band” system which provides a separate sensory input to determine the mask region boundary.
- A preferred embodiment of the “in-band” system is a rangefinder system which operates through the video camera system, to distinguish the foreground object in the live video source from its native background based on differences in distance from the camera lens. Thus, rather than relying on an analysis of the image per se to extract the foreground object, this preferred embodiment of the system defines the boundary of the object through its focal plane or parallax.
- A preferred embodiment of the “out-of-band” system includes an absolute position and orientation sensor physically associated with the foreground object with a predetermined relationship of the sensor to the desired portion of the foreground object. Thus, where the foreground object is a person, the sensor may be an electromagnetic position sensor mounted centrally on top of the head with the mask region defined by an oval boundary below and in front of the position and orientation sensor.
- In a preferred embodiment, the foreground image is a portrait of a person, while the background image is a computer generated image of a figure. A position sensor tracks a head position in the portrait, which is used to estimate a facial area. The image of the facial area is then merged in an anatomically appropriate fashion with the background figure.
- The background image is, for example, an animated “character”, with a masked facial portion. The live video signal in this case includes, as the foreground image, a face, with the face generally having a defined spatial relation to the position sensor. The masked region of the character is generated, based on the output of the position sensor in an appropriate position, so that the face may be superimposed within the masked region. As seen in the resulting composite video image, the live image of the face is presented within a mask of an animated character, presenting a suitable foundation for a consumer entertainment system. The mask may obscure portions of the face, as desired. Manual inputs or secondary position sensors for the arms or legs of the individual may be used as further control inputs, allowing the user to both control the computer generated animation and to become a part of the resultant image. This system may therefore be incorporated into larger virtual reality systems to allow an increased level of interaction, while minimizing the need for specialized environments.
- In practice, it is generally desired to mask a margin of the face so that no portion of the background appears in a composite image. Thus, the actual video background is completely obscured and irrelevant. In order to produce an aesthetically pleasing and natural appearing result, the region around the face is preferably provided with an image which appears as a mask. Thus, the background image may appear as a masked character, with the foreground image as a video image of a face within the mask region. The mask region may be independent of the video image data, or developed based on an image processing algorithm of the video image data. In the later case, where processing latencies are substantial, the composite output may be initially provided as a video image data independent mask which is modified over time, when the image is relatively static, for greater correspondence with the actual image. Thus, such a progressive rendering system will allow operation on platforms having various available processing power for image processing, while yielding acceptable results on systems having a low amount of available processing power.
- It is not always possible to adjust the size and placement of an image mask for each user of the system. Thus, the preferred embodiment provides a background image which is tolerant of misalignments and misadjustments of the video image with the background image. In the case of a masked character background image, this tolerance includes providing an edge portion of the mask which merges appropriately with a variety of types of facial images, e.g., men, women, children, and possibly pet animals.
- Because the system is not limited to a chroma key superposition system, the information from the position sensor and the video camera allow simple extraction of the image of an individual's face in a more generalized computer graphic image, based on an estimate of its position. Thus, multiple individuals may be presented in a single graphic image, each interacting with his or her environment or with each other. While these individuals may be present in the same environment, for example, within the field of view of a single video camera, this ability to build complex images from multiple inputs allow individuals at remote locations within a computer network to interact while viewing each other's faces. Therefore, personalized multiplayer “video” games become possible. This same technology my also have uses outside the fields of entertainment, including communications and video conferencing. This personalized representation separated from its native background also forms the basis for a new style of multi-user graphic interface system.
- In implementation, the position estimation system preferably acts as an input to a computer generated animation as the background image figure. In one set of embodiments, the generation of the resulting combined image is performed through a chroma key system. Therefore, in such systems, the background figure image is provided with a key color in a facial region or desired superposed video image region(s) of the figure. In contrast to typical applications of chroma key technology, the chroma key appears in the presented background, with the live video image overlayed in the chroma key region. Of course, chroma key technology is not the only method for combining the various image information, and in fact the process may be performed digitally in a computerized system.
- In one embodiment, the position sensor defines a predefined window, which is translated around the video space. Where further refinement is desired, the orientation and distance of the foreground object from the video camera may be compensated. The shape of the window may be a regular shape or an outline of a foreground image. Thus, with an image of a person as the foreground image, the image may be initially processed to determine the shape. Thereafter, the shape may be translated, resized or otherwise transformed as the window. In this latter case, the shape of the window may be periodically redetermined, but need not be recalculated in real time.
- In a particularly preferred embodiment, the live video image is an image of a person having a face, with a position sensor mounted on top of a set of headphones. An oval mask region is defined with respect to the position sensor, so that the position of the face within the video image is predicted to be within the oval mask region. The position sensor also serves as an input to a computer animation graphic image generator, which generates an animated body in appropriate position and orientation for combining with the face. Further position sensors may be provided on the arms of the person, as further inputs to the computer animation graphic image generator, allowing further control over the resulting image. The computer animation graphic image includes a chroma key portion in a region intended for the facial image. The live video image is then merged with the computer animation graphic image and presented as a composite.
- The position tracking system is, for example, an Ascension position tracking system mounted centrally on a bridging portion on a set of headphones, worn by a person. The person is present within the image of a video camera, and the system calibrated to locate the position tracking system unit within the field of view of the video camera. The face of the person is estimated to be within an oval area approximately 10 inches down and 8 inches wide below the position tracking system sensor, when the person is facing the camera. Since the preferred position tracking sensor senses movement in six degrees of freedom, the window is altered to correspond to the expected area of presentation of the face in the image. The border between the live video image of the face and the animated character need not be presented as a simple oval region, and may include images which overlay the face, as well as complex boundaries.
- By employing a separate position tracking sensor, the preferred embodiment avoids the need for sophisticated image analysis, thereby allowing relatively simple and available components. Further, known types of position sensors also provide orientation information which may be useful for providing control inputs into the background image generation system and also to control the position and shape of the mask region to compensate for skew, profile, tilting and other degrees of freedom of the object. The computer generated animated image responds to the position tracking sensor as an input, allowing the animation to track the movements of the person.
- While one preferred embodiment employs an Ascension tracking system, which, while of high quality, is also expensive, another preferred embodiment employs an acoustic sensor to determine location in three or more dimensions. This system is similar to the known “power glove” accessory for video games. Other types of position sensors may also be used.
- Thus, the present invention avoids the need for a defined background for a foreground image when electronically superimposing video images by providing a position sensor to provide information for determining a location of the desired foreground image in a foreground video stream. The position sensor thus minimizes the need for analysis of the foreground image stream, allowing relatively simple merging of the video streams.
- In systems where the facial image is captured and electronically processed, rather than genlocked and superimposed, the use of the position sensor to define a mask region in the video image substantially reduces a computational complexity required to extract a facial portion from a video image, especially as compared to a typical digital image processing system. As noted above, the margin of the face need not be determined with high precision in many instances, and therefore the background image which is generated to surround the facial image may be provided to include a degree of tolerance to this imprecision, such as a wide edge margin and avoidance of structures which should be precisely aligned with facial features. Where the image is to be transmitted over a computer image, and where the facial portion of the image is the most important component of the image, the use of the present system allows transmission of the masked portion of the image only, reducing the amount of information which must be transmitted and thus compressing the image data.
- A known paradigm for user interaction with computers is known as an “Avatar”, a computer generated representation of a user, which is generally wholly animated. These Avatars may be transmitted through a computer network system, for example the Internet, to allow a user to interact with a graphical environment of a system. According to the present invention, these Avatars need not be completely computer generated, and may therefore include a real time video image of a face. This system therefore allows, with reduced computational requirements and limited bandwidth requirements, the personalization of Avatars. Thus, the present invention provides a new type of graphical user interface in which a user is represented as an actual image within a computer graphic space. Multiple users may therefore interact while viewing actual images of each other, even where the users are separated over nodes of a computer network.
- As stated above, an in-band mask region determining system may operate based on the foreground video input device. Thus, the position sensing system need not include physically separate hardware. Likewise, the video signal superposition system need not be an external chroma key superposition system, and may be integrated with the animation generation system.
- In a first image analysis embodiment, an outline of a major foreground object is determined, and the outline used to define a mask. In a second image analysis embodiment, the foreground object is irradiated with an unobtrusive radiation, e.g., infrared, which is detected by a special video camera or optical sensors. Thus, the infrared contrast of the foreground image defines the foreground object, and a corresponding mask provided. In a third embodiment, an optical transmitter, e.g., one or more LEDs, preferably including a blue LED, is mounted on the headphones, visible to the video camera. The presence of an illuminated spot is detected, and a mask defined in relation to the position of the spot. If distance and orientation information are desired, a plurality of LEDs may be mounted, in a configuration sufficient to allow estimation of position and orientation. Thus, it can be seen that the position detecting system may operate through the video feed without requiring rigorous image analysis, which often cannot be performed in or near real time.
- The resulting image of the method according to the present system and method may be presented on a video monitor, transmitted over a video network for rendering at a remote site, or stored on a video storage medium, such as video tape. In the latter case, the opportunities for complex background generation become apparent. Where the image is not simply transient, a higher level of detail in the background image may be preferred, because the stored image may be reviewed a number of times. Further, since the background is computer generated, it need not be constant. Thus, for example, the foreground image and control signals, e.g., position and orientation signals, may be stored on a CD-ROM, with the background image generated in real time on reproduction of the images on a computer system. Since the video image and the control parameters are stored, the reproducted image sequence need not be fixed, and may therefore vary based on a set of background parameters.
- An alternative set of embodiments provide different processing systems to capture the facial image for presentation. For example, the location of the image of the face may be identified, with the facial image texture mapped onto a computer generated image. Thus, the boundary between the foreground image and background image need not be a discrete edge, and the present invention therefore allows a more subtle merging of the images.
- The location of the foreground image need not be determined with a typical position sensor, and other systems may be used. Advantageously, the focal plane of the foreground object, e.g., person, differs from the background. In this case, the boundary of the foreground object may be determined by detecting a focal plane boundary. This technique offers two advantages. First, it allows redundant use of the focus control system found in many video camera systems, eliminating the need for a separate position sensing system. Second, it allows imaging of irregular-shaped objects, e.g., a person wearing a hat, without being limited by a predefined mask shape.
- A rangefinder system may be used to obtain a depth map of a face in real-time, with the resulting data used as control parameters for a computer generated character's face. This rangefinder information allows use of facial expression as a control input, while reducing the need for strict image analysis. This depth information may also be employed to assist in texture mapping the video image information on the background. Likewise, other objects or images may be tracked and used as control inputs.
- It is noted that, while many embodiments according to the present invention employ a computer generated graphic image, the background image need not be computer generated. Thus, the background image may also represent a video feed signal. In one embodiment, the background image is a video image of a robot or computer automated mechanical structure, which, e.g., responds to the position and orientation inputs from the foreground input to provide coordination. The merging or foreground and background images in this case may be through the use of typical chroma key technology.
- It is thus an object of the present invention to provide a graphic image system comprising a source of a first signal representing a first image including a moving human subject having a bead with a face; an image position estimating system for estimating the instantaneous position of said head of said human subject; a source of a second signal representing a second image including a character having a head with a mask outline; and means, responsive to said position estimating system and to said first and second signals, for dynamically defining an estimated boundary of said face of said human subject in said first image and for merging the face in said first image, as limited by said estimated boundary, with the second image within the mask outline.
- It is also an object of the invention to provide a video system comprising a video input, receiving a video signal representing an image having a movable foreground object; a position tracking system for tracking a position of said movable foreground object; and means, responsive to said position tracking system, for dynamically defining an estimated boundary of said moveable foreground object in said image.
- It is a further object to provide a video system having a source of background video image and a video superposition control for superposing the foreground object of said image within said estimated boundary on said background video image.
- It is a still further object of the invention to provide a video system wherein said background video image is responsive to said position tracking system.
- It is another object of the invention to provide a video superposition control having a chroma key video superposition unit. The background video image preferably comprises a computer generated animated image stream.
- According to various objects of the invention, the position tracking system may be a radio frequency field sensor, an electro-acoustic transducer or an optical position sensing system. The position tracking system may have various degrees of freedom, for example two, three or six. The position tracking system may include a physical transducer mounted on the foreground object.
- According to the present invention, the mask or estimated boundary may be geometric in shape, for example oval, round or having multiple discontinuous geometric segments.
- The position tracking system produces a position, and optionally orientation of the foreground object within the field of view of the video camera.
- According to another object of the invention, the normal relation of foreground and background signals in a chroma key video superposition unit are reversed, with the “background” image having a region of chroma activating the keying circuit, to allow a foreground object to be presented. The location of the region is determined by the position tracking system.
- It is another object according to the present invention to provide a video system comprising, in combination: a video camera producing a first video signal defining a first image including a foreground object and a background; a position tracking system for identifying a position with respect to said foreground object, said foreground object having features in constant physical relation to said position; and a computer, responsive to said position tracking system, for defining a mask region separating said foreground object from said background. The computer preferably generates a second video signal including a portion corresponding to said mask region, wherein an image in the second video signal is responsive to said position tracking system. The mask region differentiates image regions which are processed differently. The mask region preferably is employed in a chroma key system to form a composite image of the first and second images, although an out-of-band signal may be used to define the mask region.
- According to another object of the invention, the position tracking sensor determines a position and orientation of the foreground object, and is used to control a size, shape and position of the mask region.
- These and other objects and features of the present invention will become more fully apparent from the following description and appended claims taken in conjunction with the accompanying drawings, in which like numerals refer to like parts.
- FIG. 1 shows a perspective view of a first embodiment of the invention, having an electromagnetic position tracking device;
- FIG. 2 shows a perspective view of a second embodiment of the invention, having an acoustic position tracking device;
- FIG. 3 shows a block diagram of a system according to the present invention;
- FIG. 4 shows a flow chart of a method according to the present invention; and
- FIG. 5 shows a multi-user networked embodiment according to the present invention.
- The invention will now be described by way of the drawings, in which corresponding reference numerals indicate corresponding structures in the figures.
- As shown in FIG. 1, a video camera2 is positioned near a video display 4, which may be, for example, a rear projection video monitor. An electromagnetic transmitting antenna 6 is positioned within the environment, producing a time-varying magnetic field which is detected by a
sensor 8. The antenna 6 produces a field which has a characteristic spatial variation, so that thesensor 8 allows detection of the position and orientation of thesensor 8 with respect to the antenna 6. Thesensor 8 is mounted at the apex of the bridging portion of a set ofheadphones 10. The position and orientation of thesensor 8 is calibrated. Acomputer 12 receives theposition sensor 8 output, and is used to generate an animatedgraphic image 14 on the monitor 4. The animation responds to theposition sensor 8 as an input, for example performing corresponding changes in orientation and position. The animated figure of thegraphic image 14 has afacial area 16 which is rendered in a deep blue color, having a border 18 which may be mask-like. The video image is optionally justified based on the output of the position sensor. A window 20 is generated in the videographic image 14 at the expected location of theface 22 based on theposition sensor 8 position and orientation. The window 20 with theface 22 image is then displayed, using a chroma key circuit keyed to the deep blue color, in the position of the mask - Thus, with a fixed relation of the antenna6 with respect to the video camera 2, the
sensor 8 may be used to control animation with a portion of a live video image superimposed on a computer generatedgraphic image 14, responding to a motion of an object in the live video image. By adaptively masking the live video image based on a determined location within the imaging space of the video camera, without requiring a special background type, and without intrusion of stray objects in the video image into the resulting composite image. - In a second embodiment, as shown in FIG. 2, a simpler type of image position sensing system is employed in the system generally according to the description of FIG. 1. In this case, the
position sensor 24, 26 is sensitive to position, but not orientation, and further has a limited range. In this case, thecomputer system 28 presumes that thelive video image 30 includes an image of aperson 32 looking directly at the video camera 2, so that the position of themask 34 in thevideo image 30 is altered. While this presents reduced opportunity for animation control, variations in position may be used, within the range of the display 4. Thus, the strategy of use of the position sensor may differ. - The system according to the second embodiment has another type of application. In a video conferencing system, often a distracting background is present, which further increases a required bandwidth to transmit a compressed image. Therefore, by determining the position of the head of a videoconference participant, the head may be selectively processed. While headphones are not required in this application, in situations with distracting backgrounds, often there is also stray noise, so that the headphones may be desirable. The acoustic
position sensing system 24, 26 operates on a triangulation technique. The position sensor element 26 mounted on theheadphones 10 in this instance is an ultrasonic transmitter. By detecting a differential amplitude, as well as phase delay, in a fixedreceiver 24 location, a position of the transmitter 26 in space may be determined. - FIG. 3 shows a configuration of the electronics hardware according to one embodiment according to the present invention. The
system animation 40 receives aposition input signal 42 from aposition tracking system 54 which is used to determine a mask position within the live video feed 48 of aforeground object 56 and also as a control parameter for a computer generated animation. Other control inputs may also be received, such as arm or wrist position, joysticks, or other known input devices. Thelive video 48 mask is then overlayed in a chromakey unit 46 over a computer animated image 50 and displayed on amonitor 52. It is noted that completely digital processing of the image may also be used. However, the chroma key technology requires a reduced computational load and may potentially be implemented with lower cost. Since the mask location, as determined by aposition transducer 58, may change within thelive video 48 image, the mask image may optionally be translated to a desired location within the composite image. - FIG. 4 shows a flow diagram of a method according to the present invention. The position of the position sensor is calibrated with respect to the
live video image 68. A person wears the headphones. A mask location is defined 62 in the live video image by determining the position of the position sensor 60. The output of the position sensor is also used to controlanimation 64, generally to produce a character who moves corresponding to the movements of the position sensor. The face of the animated character is rendered in a chroma key color 66. A separate chroma key unit receives the animated image and the live video image, with the live video corresponding to the mask portion overlayed where the chroma key is active 70. The superposed image is then output 72. - An embodiment of the invention is provided as shown in FIG. 5. A plurality of users81, 82 interact with a computer network system through various nodes. The communication protocol is, for example, TCP/IP over Ethernet 10BaseT. Thus, a typical user station includes a personal computer (PC) 93, 105 running a graphical user interface, e.g., Windows NT or Solaris. The
personal computer 93, 105 includes a number of adapter cards or functional interfaces. Each computer generally includes a video display adapter 90, 108, aframe grabber video camera 92, 107, and optionally a hardware MPEG encoder/decoder system personal computer 93, 105. The personal computers communicate with each other over Ethernet 10BaseT networks 96, 102, using TCP/IP protocol. Therefore, eachpersonal computer 93, 105 includes a TCP/IP stack 95, 103, which is generally a software construct. The computer network includes, for example, LAN hubs 97, 101 and WAN routers or bridges 99, 100. - Each user has a video camera86, 112 mounted centrally over a
video monitor 83, 115. The video camera 86, 112control 92, 107 allows control over video camera parameters, including lens 113, 85 focus and zoom. In this case, the video camera 86, 112 intermittently scans through the various focal planes of objects within its field of view, with an edge contrast determination algorithm operative to determine optimum focus for any region. This edge contrast determination algorithm may be a predetermined program within thepersonal computer 93, 105. The optimum focus edge serves as a mask region 118 for the user's 81 head, and mask region 119 for user's 82 head, situated in front of themonitor 93, 115. The area within the mask regions 118, 119 are retained, while outside the mask regions 118, 119 are masked. The outline and position of the mask regions 118, 119 serve as control parameters for an Avatar 81′, 82′, which may be transmitted as necessary through the computer network, along with a compressed representation of the image within the mask regions 118, 119. The Avatar 81′, 82′ need not be presented identically on each display, and therefore may have distinct interaction at each local site. - Suitable compression for this image may include JPEG, MPEG, H.320, H.324, H.261, GIF, wavelet and fractal image compression, as well as other known compression formats. Where suitable MPEG encoder/
decoder system computer environment space 84, 114 through graphic representations having personalized and real time varying facial expressions. This personalized interface system may reduce stress for novice users and provides an efficient and innate means of communication for people within the network. - By providing
background 117, 116 masking for the foreground image, the personalized Avatar 81′, 82′ generation is simplified and the data transmission requirements are reduced. - While the above detailed description has shown, described and pointed out the fundamental novel features of the invention as applied to various embodiments, it will be understood that various omissions and substitutions and changes in the form and details of the device illustrated may be made by those skilled in the art, without departing from the spirit of the invention. Consequently, the full scope of the invention should be ascertained by the appended claims.
Claims (49)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US08/716,597 US6400374B2 (en) | 1996-09-18 | 1996-09-18 | Video superposition system and method |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US08/716,597 US6400374B2 (en) | 1996-09-18 | 1996-09-18 | Video superposition system and method |
Publications (2)
Publication Number | Publication Date |
---|---|
US20020018070A1 true US20020018070A1 (en) | 2002-02-14 |
US6400374B2 US6400374B2 (en) | 2002-06-04 |
Family
ID=24878671
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US08/716,597 Expired - Lifetime US6400374B2 (en) | 1996-09-18 | 1996-09-18 | Video superposition system and method |
Country Status (1)
Country | Link |
---|---|
US (1) | US6400374B2 (en) |
Cited By (46)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20010039520A1 (en) * | 2000-03-28 | 2001-11-08 | Motoki Nakade | Communication service method and communication apparatus thereof |
US20030003991A1 (en) * | 2001-06-29 | 2003-01-02 | Konami Corporation | Game device, game controlling method and program |
US20040001074A1 (en) * | 2002-05-29 | 2004-01-01 | Hideki Oyaizu | Image display apparatus and method, transmitting apparatus and method, image display system, recording medium, and program |
US20040113757A1 (en) * | 2002-12-10 | 2004-06-17 | White Melvin Joseph | Power line communication system and method of operating the same |
US20040179034A1 (en) * | 2003-02-26 | 2004-09-16 | Burritt David Ray | Apparatus and method for displaying background images (themes) for computer and handheld computer devices |
US20050076134A1 (en) * | 2001-05-17 | 2005-04-07 | Gil Bialik | Apparatus and method for multiple rich media formats video broadcasting |
US20050084835A1 (en) * | 2003-10-16 | 2005-04-21 | The Singing Machine Company, Inc. | Karaoke system with built-in camera |
US20050204287A1 (en) * | 2004-02-06 | 2005-09-15 | Imagetech Co., Ltd | Method and system for producing real-time interactive video and audio |
EP1582065A2 (en) * | 2002-12-30 | 2005-10-05 | Motorola, Inc. | A method, system and apparatus for telepresence communications |
US20050222712A1 (en) * | 2004-03-31 | 2005-10-06 | Honda Motor Co., Ltd. | Salesperson robot system |
US20050220871A1 (en) * | 2000-03-28 | 2005-10-06 | Schwarz Franz X | Taste masking granules |
WO2005117435A1 (en) * | 2004-05-31 | 2005-12-08 | Telecom Italia S.P.A | System for and method of personalising videocommunications, and computer program product therefor |
US20060140508A1 (en) * | 2002-10-23 | 2006-06-29 | Kiyoshi Ohgishi | Image combining portable terminal and image combining method used therefor |
EP1779328A1 (en) * | 2004-08-03 | 2007-05-02 | William J. Benman | System and method for providing a functional virtual environment with real time extracted and transplanted images |
US20070190506A1 (en) * | 2005-12-26 | 2007-08-16 | Industrial Technology Research Institute | Online interactive multimedia system and the transmission method thereof |
US20070200925A1 (en) * | 2006-02-07 | 2007-08-30 | Lg Electronics Inc. | Video conference system and method in a communication network |
US7298381B1 (en) * | 1997-10-31 | 2007-11-20 | Thomson Licensing | Arrangement for mixing and/or processing video signals |
US20080019576A1 (en) * | 2005-09-16 | 2008-01-24 | Blake Senftner | Personalizing a Video |
EP1976291A1 (en) * | 2007-03-02 | 2008-10-01 | Deutsche Telekom AG | Method and video communication system for gesture-based real-time control of an avatar |
US20080266324A1 (en) * | 2007-04-30 | 2008-10-30 | Navteq North America, Llc | Street level video simulation display system and method |
WO2008139251A2 (en) * | 2006-04-14 | 2008-11-20 | Patrick Levy Rosenthal | Virtual video camera device with three-dimensional tracking and virtual object insertion |
US7472910B1 (en) * | 1999-03-30 | 2009-01-06 | Canon Kabushiki Kaisha | Animation display apparatus, arcade game machine, control method and apparatus thereof, and storage medium |
US20090033737A1 (en) * | 2007-08-02 | 2009-02-05 | Stuart Goose | Method and System for Video Conferencing in a Virtual Environment |
US20090097754A1 (en) * | 2007-10-11 | 2009-04-16 | Hon Hai Precision Industry Co., Ltd. | Video communication device and image processing system and method of the same |
US20090199078A1 (en) * | 2008-02-04 | 2009-08-06 | Siemens Communications, Inc. | Method and apparatus for enhanced video mixing |
JP2009536406A (en) * | 2006-05-07 | 2009-10-08 | 株式会社ソニー・コンピュータエンタテインメント | How to give emotional features to computer-generated avatars during gameplay |
US20100033502A1 (en) * | 2006-10-13 | 2010-02-11 | Freescale Semiconductor, Inc. | Image processing apparatus for superimposing windows displaying video data having different frame rates |
FR2958487A1 (en) * | 2010-04-06 | 2011-10-07 | Alcatel Lucent | A METHOD OF REAL TIME DISTORTION OF A REAL ENTITY RECORDED IN A VIDEO SEQUENCE |
WO2012040115A1 (en) * | 2010-09-20 | 2012-03-29 | Qualcomm Incorporated | Virtual video capture device |
JP2012253551A (en) * | 2011-06-02 | 2012-12-20 | Sony Corp | Display control apparatus, display control method, and program |
US20130009980A1 (en) * | 2011-07-07 | 2013-01-10 | Ati Technologies Ulc | Viewing-focus oriented image processing |
US20130044254A1 (en) * | 2011-08-18 | 2013-02-21 | Meir Tzur | Image capture for later refocusing or focus-manipulation |
US20130257877A1 (en) * | 2012-03-30 | 2013-10-03 | Videx, Inc. | Systems and Methods for Generating an Interactive Avatar Model |
US20140002677A1 (en) * | 2011-12-09 | 2014-01-02 | Robert Schinker | Methods and apparatus for enhanced reality messaging |
CN103581728A (en) * | 2012-07-20 | 2014-02-12 | 英特尔公司 | Selective post-processing of decoded video frames based on focus point determination |
US8753203B1 (en) * | 2008-04-10 | 2014-06-17 | Acme Embedded Solutions, Inc. | Compositing device for combining visual content |
EP2786561A1 (en) * | 2011-12-01 | 2014-10-08 | Tangome Inc. | Augmenting a video conference |
US20150228247A1 (en) * | 2012-07-20 | 2015-08-13 | Feng-Chia Wu | Image generating method |
US9565363B1 (en) * | 2015-08-10 | 2017-02-07 | X Development Llc | Stabilization of captured images for teleoperated walking biped robots |
US20170287097A1 (en) * | 2016-03-29 | 2017-10-05 | Ati Technologies Ulc | Hybrid client-server rendering in a virtual reality system |
US10078909B1 (en) * | 2017-05-16 | 2018-09-18 | Facebook, Inc. | Video stream customization using graphics |
US10237528B2 (en) | 2013-03-14 | 2019-03-19 | Qualcomm Incorporated | System and method for real time 2D to 3D conversion of a video in a digital camera |
US10276210B2 (en) * | 2015-11-18 | 2019-04-30 | International Business Machines Corporation | Video enhancement |
US10835828B1 (en) * | 2018-07-25 | 2020-11-17 | Facebook, Inc. | Augmented-reality game overlays in video communications |
CN114501140A (en) * | 2021-12-28 | 2022-05-13 | 安徽清新物联科技有限公司 | Video character superposition processing method, system and storage medium |
US11749020B2 (en) * | 2018-07-27 | 2023-09-05 | Beijing Microlive Vision Technology Co., Ltd | Method and apparatus for multi-face tracking of a face effect, and electronic device |
Families Citing this family (169)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6430997B1 (en) | 1995-11-06 | 2002-08-13 | Trazer Technologies, Inc. | System and method for tracking and assessing movement skills in multidimensional space |
GB9712724D0 (en) * | 1997-06-18 | 1997-08-20 | Holmes Steven | Method and apparatus for interaction with broadcast television content |
AU1099899A (en) * | 1997-10-15 | 1999-05-03 | Electric Planet, Inc. | Method and apparatus for performing a clean background subtraction |
JP3840898B2 (en) * | 1998-02-03 | 2006-11-01 | セイコーエプソン株式会社 | Projection display device, display method therefor, and image display device |
WO1999060522A1 (en) * | 1998-05-19 | 1999-11-25 | Sony Computer Entertainment Inc. | Image processing apparatus and method, and providing medium |
RU2157054C2 (en) * | 1998-09-04 | 2000-09-27 | Латыпов Нурахмед Нурисламович | Method for production of video programs and device which implements said method |
US7184100B1 (en) * | 1999-03-24 | 2007-02-27 | Mate - Media Access Technologies Ltd. | Method of selecting key-frames from a video sequence |
US7839399B2 (en) * | 1999-07-29 | 2010-11-23 | Benman William J | System and method for volumetric display of video images extracted from arbitrary background environments |
US7136525B1 (en) * | 1999-09-20 | 2006-11-14 | Microsoft Corporation | System and method for background maintenance of an image sequence |
US6856329B1 (en) * | 1999-11-12 | 2005-02-15 | Creative Technology Ltd. | Automated acquisition of video textures acquired from a digital camera for mapping to audio-driven deformable objects |
JP2001230972A (en) * | 1999-12-09 | 2001-08-24 | Canon Inc | Image pickup device, image compositing method, image processor and image processing method |
JP2001197366A (en) * | 2000-01-12 | 2001-07-19 | Hitachi Ltd | Picture synthesis method and recording medium recording picture synthesis program |
US6774912B1 (en) * | 2000-03-16 | 2004-08-10 | Matrox Graphics Inc. | Multiple display device display controller with video overlay and full screen video outputs |
US6954193B1 (en) * | 2000-09-08 | 2005-10-11 | Apple Computer, Inc. | Method and apparatus for correcting pixel level intensity variation |
US6680739B1 (en) * | 2000-11-17 | 2004-01-20 | Hewlett-Packard Development Company, L.P. | Systems and methods for compositing graphical data |
US6985162B1 (en) * | 2000-11-17 | 2006-01-10 | Hewlett-Packard Development Company, L.P. | Systems and methods for rendering active stereo graphical data as passive stereo |
US6621500B1 (en) * | 2000-11-17 | 2003-09-16 | Hewlett-Packard Development Company, L.P. | Systems and methods for rendering graphical data |
US7961201B1 (en) * | 2000-12-21 | 2011-06-14 | Cognex Corporation | Method and apparatus for producing graphical machine vision content for distribution via a network |
KR100416991B1 (en) * | 2001-01-11 | 2004-02-05 | 삼성전자주식회사 | Video terminal apparatus having displaying virtual background and implementing method thereof |
US20020131643A1 (en) * | 2001-03-13 | 2002-09-19 | Fels Sol Sidney | Local positioning system |
US7392287B2 (en) | 2001-03-27 | 2008-06-24 | Hemisphere Ii Investment Lp | Method and apparatus for sharing information using a handheld device |
US7259747B2 (en) * | 2001-06-05 | 2007-08-21 | Reactrix Systems, Inc. | Interactive video display system |
US8035612B2 (en) * | 2002-05-28 | 2011-10-11 | Intellectual Ventures Holding 67 Llc | Self-contained interactive video display system |
US8300042B2 (en) * | 2001-06-05 | 2012-10-30 | Microsoft Corporation | Interactive video display system using strobed light |
US7079706B2 (en) * | 2001-06-20 | 2006-07-18 | Paul Peterson | Methods and apparatus for generating a multiple composite image |
US20020198724A1 (en) * | 2001-06-20 | 2002-12-26 | Paul Peterson | Methods and apparatus for producing a lenticular novelty item interactively via the internet |
US7079279B2 (en) * | 2001-06-20 | 2006-07-18 | Paul Peterson | Methods and apparatus for producing a lenticular novelty item at a point of purchase |
DE10159610B4 (en) * | 2001-12-05 | 2004-02-26 | Siemens Ag | System and method for creating documentation of work processes, especially in the area of production, assembly, service or maintenance |
US20030156113A1 (en) * | 2002-02-19 | 2003-08-21 | Darryl Freedman | Methods of combining animation and live video using looped animation and related systems |
JP3948986B2 (en) * | 2002-03-14 | 2007-07-25 | 三洋電機株式会社 | Captured image display device and captured image display method |
US20050177463A1 (en) * | 2004-02-10 | 2005-08-11 | Crutchfield William G.Jr. | Virtual showroom for interactive electronic shopping |
US7710391B2 (en) * | 2002-05-28 | 2010-05-04 | Matthew Bell | Processing an image utilizing a spatially varying pattern |
US7348963B2 (en) * | 2002-05-28 | 2008-03-25 | Reactrix Systems, Inc. | Interactive video display system |
US6801642B2 (en) * | 2002-06-26 | 2004-10-05 | Motorola, Inc. | Method and apparatus for limiting storage or transmission of visual information |
US7227976B1 (en) * | 2002-07-08 | 2007-06-05 | Videomining Corporation | Method and system for real-time facial image enhancement |
US7161579B2 (en) | 2002-07-18 | 2007-01-09 | Sony Computer Entertainment Inc. | Hand-held computer interactive device |
US7102615B2 (en) * | 2002-07-27 | 2006-09-05 | Sony Computer Entertainment Inc. | Man-machine interface using a deformable device |
US7623115B2 (en) * | 2002-07-27 | 2009-11-24 | Sony Computer Entertainment Inc. | Method and apparatus for light input device |
US8797260B2 (en) * | 2002-07-27 | 2014-08-05 | Sony Computer Entertainment Inc. | Inertially trackable hand-held controller |
US7883415B2 (en) * | 2003-09-15 | 2011-02-08 | Sony Computer Entertainment Inc. | Method and apparatus for adjusting a view of a scene being displayed according to tracked head motion |
US7646372B2 (en) * | 2003-09-15 | 2010-01-12 | Sony Computer Entertainment Inc. | Methods and systems for enabling direction detection when interfacing with a computer program |
US8947347B2 (en) * | 2003-08-27 | 2015-02-03 | Sony Computer Entertainment Inc. | Controlling actions in a video game unit |
US7627139B2 (en) * | 2002-07-27 | 2009-12-01 | Sony Computer Entertainment Inc. | Computer image and audio processing of intensity and input devices for interfacing with a computer program |
US7850526B2 (en) * | 2002-07-27 | 2010-12-14 | Sony Computer Entertainment America Inc. | System for tracking user manipulations within an environment |
US9393487B2 (en) | 2002-07-27 | 2016-07-19 | Sony Interactive Entertainment Inc. | Method for mapping movements of a hand-held controller to game commands |
US7918733B2 (en) * | 2002-07-27 | 2011-04-05 | Sony Computer Entertainment America Inc. | Multi-input game control mixer |
US8313380B2 (en) | 2002-07-27 | 2012-11-20 | Sony Computer Entertainment America Llc | Scheme for translating movements of a hand-held controller into inputs for a system |
US7352359B2 (en) * | 2002-07-27 | 2008-04-01 | Sony Computer Entertainment America Inc. | Method and system for applying gearing effects to inertial tracking |
US8570378B2 (en) | 2002-07-27 | 2013-10-29 | Sony Computer Entertainment Inc. | Method and apparatus for tracking three-dimensional movements of an object using a depth sensing camera |
US10086282B2 (en) * | 2002-07-27 | 2018-10-02 | Sony Interactive Entertainment Inc. | Tracking device for use in obtaining information for controlling game program execution |
US9474968B2 (en) | 2002-07-27 | 2016-10-25 | Sony Interactive Entertainment America Llc | Method and system for applying gearing effects to visual tracking |
US7854655B2 (en) | 2002-07-27 | 2010-12-21 | Sony Computer Entertainment America Inc. | Obtaining input for controlling execution of a game program |
US8139793B2 (en) * | 2003-08-27 | 2012-03-20 | Sony Computer Entertainment Inc. | Methods and apparatus for capturing audio signals based on a visual image |
US8160269B2 (en) | 2003-08-27 | 2012-04-17 | Sony Computer Entertainment Inc. | Methods and apparatuses for adjusting a listening area for capturing sounds |
US7352358B2 (en) * | 2002-07-27 | 2008-04-01 | Sony Computer Entertainment America Inc. | Method and system for applying gearing effects to acoustical tracking |
US8233642B2 (en) * | 2003-08-27 | 2012-07-31 | Sony Computer Entertainment Inc. | Methods and apparatuses for capturing an audio signal based on a location of the signal |
US9174119B2 (en) | 2002-07-27 | 2015-11-03 | Sony Computer Entertainement America, LLC | Controller for providing inputs to control execution of a program when inputs are combined |
US7760248B2 (en) | 2002-07-27 | 2010-07-20 | Sony Computer Entertainment Inc. | Selective sound source listening in conjunction with computer interactive processing |
US7391409B2 (en) * | 2002-07-27 | 2008-06-24 | Sony Computer Entertainment America Inc. | Method and system for applying gearing effects to multi-channel mixed input |
US8686939B2 (en) * | 2002-07-27 | 2014-04-01 | Sony Computer Entertainment Inc. | System, method, and apparatus for three-dimensional input control |
US7803050B2 (en) * | 2002-07-27 | 2010-09-28 | Sony Computer Entertainment Inc. | Tracking device with sound emitter for use in obtaining information for controlling game program execution |
WO2007130793A2 (en) * | 2006-05-04 | 2007-11-15 | Sony Computer Entertainment America Inc. | Obtaining input for controlling execution of a game program |
US7053915B1 (en) * | 2002-07-30 | 2006-05-30 | Advanced Interfaces, Inc | Method and system for enhancing virtual stage experience |
US9682319B2 (en) | 2002-07-31 | 2017-06-20 | Sony Interactive Entertainment Inc. | Combiner method for altering game gearing |
US20040136870A1 (en) * | 2002-10-25 | 2004-07-15 | Kochy Thomas E. | Automatic analysis apparatus |
AU2003301043A1 (en) | 2002-12-13 | 2004-07-09 | Reactrix Systems | Interactive directed light/sound system |
US7576727B2 (en) * | 2002-12-13 | 2009-08-18 | Matthew Bell | Interactive directed light/sound system |
US6990211B2 (en) * | 2003-02-11 | 2006-01-24 | Hewlett-Packard Development Company, L.P. | Audio system and method |
US9177387B2 (en) | 2003-02-11 | 2015-11-03 | Sony Computer Entertainment Inc. | Method and apparatus for real time motion capture |
US7505862B2 (en) * | 2003-03-07 | 2009-03-17 | Salmon Technologies, Llc | Apparatus and method for testing electronic systems |
EP1621010A1 (en) * | 2003-05-02 | 2006-02-01 | Allan Robert Staker | Interactive system and method for video compositing |
US8072470B2 (en) | 2003-05-29 | 2011-12-06 | Sony Computer Entertainment Inc. | System and method for providing a real-time three-dimensional interactive environment |
US20070118812A1 (en) * | 2003-07-15 | 2007-05-24 | Kaleidescope, Inc. | Masking for presenting differing display formats for media streams |
US10279254B2 (en) | 2005-10-26 | 2019-05-07 | Sony Interactive Entertainment Inc. | Controller having visually trackable object for interfacing with a gaming system |
US8323106B2 (en) | 2008-05-30 | 2012-12-04 | Sony Computer Entertainment America Llc | Determination of controller three-dimensional location using image analysis and ultrasonic communication |
US9573056B2 (en) | 2005-10-26 | 2017-02-21 | Sony Interactive Entertainment Inc. | Expandable control device via hardware attachment |
US7874917B2 (en) | 2003-09-15 | 2011-01-25 | Sony Computer Entertainment Inc. | Methods and systems for enabling depth and direction detection when interfacing with a computer program |
US8287373B2 (en) | 2008-12-05 | 2012-10-16 | Sony Computer Entertainment Inc. | Control device for communicating visual information |
WO2005041579A2 (en) | 2003-10-24 | 2005-05-06 | Reactrix Systems, Inc. | Method and system for processing captured image information in an interactive video display system |
CN102034197A (en) | 2003-10-24 | 2011-04-27 | 瑞克楚斯系统公司 | Method and system for managing an interactive video display system |
US7663689B2 (en) | 2004-01-16 | 2010-02-16 | Sony Computer Entertainment Inc. | Method and apparatus for optimizing capture device settings through depth information |
WO2005076210A1 (en) * | 2004-02-05 | 2005-08-18 | Vodafone K.K. | Image processing method, image processing apparatus, and mobile communication terminal apparatus |
US7388580B2 (en) * | 2004-05-07 | 2008-06-17 | Valve Corporation | Generating eyes for a character in a virtual environment |
JP4227561B2 (en) * | 2004-06-03 | 2009-02-18 | キヤノン株式会社 | Image processing method and image processing apparatus |
US7865834B1 (en) | 2004-06-25 | 2011-01-04 | Apple Inc. | Multi-way video conferencing user interface |
US8547401B2 (en) | 2004-08-19 | 2013-10-01 | Sony Computer Entertainment Inc. | Portable augmented reality device and method |
US8631342B2 (en) * | 2004-12-22 | 2014-01-14 | Hewlett-Packard Development Company, L.P. | Computer display control system and method |
US20060168537A1 (en) * | 2004-12-22 | 2006-07-27 | Hochmuth Roland M | Computer display control system and method |
US20060187193A1 (en) * | 2005-02-22 | 2006-08-24 | Esquilin Eileen C | System and method for generating a virtual reality presentation |
US7852353B1 (en) | 2005-03-31 | 2010-12-14 | Apple Inc. | Encoding a transparency (alpha) channel in a video bitstream |
US7353034B2 (en) | 2005-04-04 | 2008-04-01 | X One, Inc. | Location sharing and tracking using mobile phones or other wireless devices |
US9128519B1 (en) | 2005-04-15 | 2015-09-08 | Intellectual Ventures Holding 67 Llc | Method and system for state-based control of objects |
US8081822B1 (en) | 2005-05-31 | 2011-12-20 | Intellectual Ventures Holding 67 Llc | System and method for sensing a feature of an object in an interactive video display |
US8606950B2 (en) * | 2005-06-08 | 2013-12-10 | Logitech Europe S.A. | System and method for transparently processing multimedia data |
DE602005022779D1 (en) * | 2005-06-08 | 2010-09-16 | Thomson Licensing | METHOD AND DEVICE FOR ALTERNATING IMAGE VIDEO INSERT |
US7663691B2 (en) | 2005-10-11 | 2010-02-16 | Apple Inc. | Image capture using display device as light source |
US8085318B2 (en) | 2005-10-11 | 2011-12-27 | Apple Inc. | Real-time image capture and manipulation based on streaming data |
US20060284895A1 (en) * | 2005-06-15 | 2006-12-21 | Marcu Gabriel G | Dynamic gamma correction |
US9230601B2 (en) | 2005-07-01 | 2016-01-05 | Invention Science Fund I, Llc | Media markup system for content alteration in derivative works |
US8203609B2 (en) | 2007-01-31 | 2012-06-19 | The Invention Science Fund I, Llc | Anonymization pursuant to a broadcasted policy |
US9065979B2 (en) | 2005-07-01 | 2015-06-23 | The Invention Science Fund I, Llc | Promotional placement in media works |
US9426387B2 (en) | 2005-07-01 | 2016-08-23 | Invention Science Fund I, Llc | Image anonymization |
US20070005651A1 (en) * | 2005-07-01 | 2007-01-04 | Searete Llc, A Limited Liability Corporation Of The State Of Delaware | Restoring modified assets |
US8126938B2 (en) * | 2005-07-01 | 2012-02-28 | The Invention Science Fund I, Llc | Group content substitution in media works |
US9092928B2 (en) | 2005-07-01 | 2015-07-28 | The Invention Science Fund I, Llc | Implementing group content substitution in media works |
US8732087B2 (en) | 2005-07-01 | 2014-05-20 | The Invention Science Fund I, Llc | Authorization for media content alteration |
US9583141B2 (en) | 2005-07-01 | 2017-02-28 | Invention Science Fund I, Llc | Implementing audio substitution options in media works |
US9492750B2 (en) * | 2005-07-29 | 2016-11-15 | Pamela Leslie Barber | Digital imaging method and apparatus |
US8625845B2 (en) * | 2005-08-06 | 2014-01-07 | Quantum Signal, Llc | Overlaying virtual content onto video stream of people within venue based on analysis of the people within the video stream |
JP4726577B2 (en) * | 2005-08-25 | 2011-07-20 | 富士フイルム株式会社 | Slide show generating apparatus, slide show data generating apparatus, control method therefor, and program for controlling them |
US20070097106A1 (en) * | 2005-11-03 | 2007-05-03 | Honeywell International Inc. | Constant point in space lateral map lighting |
US8098277B1 (en) | 2005-12-02 | 2012-01-17 | Intellectual Ventures Holding 67 Llc | Systems and methods for communication between a reactive video system and a mobile communication device |
US20070133940A1 (en) * | 2005-12-10 | 2007-06-14 | Freeman Andrew P | System and method for generating and documenting personalized stories |
JP3962803B2 (en) * | 2005-12-16 | 2007-08-22 | インターナショナル・ビジネス・マシーンズ・コーポレーション | Head detection device, head detection method, and head detection program |
EP2013865A4 (en) * | 2006-05-04 | 2010-11-03 | Sony Comp Entertainment Us | Methods and apparatus for applying gearing effects to input based on one or more of visual, acoustic, inertial, and mixed data |
US20110014981A1 (en) * | 2006-05-08 | 2011-01-20 | Sony Computer Entertainment Inc. | Tracking device with sound emitter for use in obtaining information for controlling game program execution |
US9019300B2 (en) | 2006-08-04 | 2015-04-28 | Apple Inc. | Framework for graphics animation and compositing operations |
US8130226B2 (en) * | 2006-08-04 | 2012-03-06 | Apple Inc. | Framework for graphics animation and compositing operations |
WO2008021091A2 (en) * | 2006-08-11 | 2008-02-21 | Packetvideo Corp. | 'system and method for delivering interactive audiovisual experiences to portable devices' |
US8781151B2 (en) | 2006-09-28 | 2014-07-15 | Sony Computer Entertainment Inc. | Object detection using video input combined with tilt angle information |
USRE48417E1 (en) | 2006-09-28 | 2021-02-02 | Sony Interactive Entertainment Inc. | Object direction using video input combined with tilt angle information |
US8310656B2 (en) | 2006-09-28 | 2012-11-13 | Sony Computer Entertainment America Llc | Mapping movements of a hand-held controller to the two-dimensional image plane of a display screen |
US20080147488A1 (en) * | 2006-10-20 | 2008-06-19 | Tunick James A | System and method for monitoring viewer attention with respect to a display and determining associated charges |
US8234392B2 (en) | 2006-11-17 | 2012-07-31 | Apple Inc. | Methods and apparatuses for providing a hardware accelerated web engine |
US20090017910A1 (en) * | 2007-06-22 | 2009-01-15 | Broadcom Corporation | Position and motion tracking of an object |
US9215512B2 (en) | 2007-04-27 | 2015-12-15 | Invention Science Fund I, Llc | Implementation of media content alteration |
US20080303949A1 (en) * | 2007-06-08 | 2008-12-11 | Apple Inc. | Manipulating video streams |
US8122378B2 (en) * | 2007-06-08 | 2012-02-21 | Apple Inc. | Image capture and manipulation |
US9189875B2 (en) * | 2007-08-06 | 2015-11-17 | Apple Inc. | Advanced import/export panel notifications using a presentation application |
US8482635B2 (en) * | 2007-09-12 | 2013-07-09 | Popnoggins, Llc | System, apparatus, software and process for integrating video images |
JP5430572B2 (en) | 2007-09-14 | 2014-03-05 | インテレクチュアル ベンチャーズ ホールディング 67 エルエルシー | Gesture-based user interaction processing |
US8159682B2 (en) * | 2007-11-12 | 2012-04-17 | Intellectual Ventures Holding 67 Llc | Lens system |
SG152952A1 (en) * | 2007-12-05 | 2009-06-29 | Gemini Info Pte Ltd | Method for automatically producing video cartoon with superimposed faces from cartoon template |
US8542907B2 (en) | 2007-12-17 | 2013-09-24 | Sony Computer Entertainment America Llc | Dynamic three-dimensional object mapping for user-defined control device |
US20090172557A1 (en) * | 2008-01-02 | 2009-07-02 | International Business Machines Corporation | Gui screen sharing between real pcs in the real world and virtual pcs in the virtual world |
CN102016877B (en) | 2008-02-27 | 2014-12-10 | 索尼计算机娱乐美国有限责任公司 | Methods for capturing depth data of a scene and applying computer actions |
US8259163B2 (en) | 2008-03-07 | 2012-09-04 | Intellectual Ventures Holding 67 Llc | Display with built in 3D sensing |
US8368753B2 (en) | 2008-03-17 | 2013-02-05 | Sony Computer Entertainment America Llc | Controller with an integrated depth camera |
US20090241039A1 (en) * | 2008-03-19 | 2009-09-24 | Leonardo William Estevez | System and method for avatar viewing |
US8446426B2 (en) * | 2008-04-28 | 2013-05-21 | Apple Inc. | Technique for visually compositing a group of graphical objects |
US8595218B2 (en) | 2008-06-12 | 2013-11-26 | Intellectual Ventures Holding 67 Llc | Interactive display management systems and methods |
US9901829B2 (en) * | 2008-07-15 | 2018-02-27 | Pamela Barber | Digital imaging method and apparatus |
US8961313B2 (en) | 2009-05-29 | 2015-02-24 | Sony Computer Entertainment America Llc | Multi-positional three-dimensional controller |
US8527657B2 (en) | 2009-03-20 | 2013-09-03 | Sony Computer Entertainment America Llc | Methods and systems for dynamically adjusting update rates in multi-player network gaming |
US8342963B2 (en) | 2009-04-10 | 2013-01-01 | Sony Computer Entertainment America Inc. | Methods and systems for enabling control of artificial intelligence game characters |
US8393964B2 (en) | 2009-05-08 | 2013-03-12 | Sony Computer Entertainment America Llc | Base station for position location |
US8142288B2 (en) | 2009-05-08 | 2012-03-27 | Sony Computer Entertainment America Llc | Base station movement detection and compensation |
KR101593573B1 (en) * | 2009-06-19 | 2016-02-12 | 삼성전자주식회사 | Method of creating contents using camera in terminal and apparatus thereof |
US8413218B1 (en) * | 2009-07-02 | 2013-04-02 | United Services Automobile Association (Usaa) | Systems and methods for videophone identity cloaking |
US8189964B2 (en) | 2009-12-07 | 2012-05-29 | Google Inc. | Matching an approximately located query image against a reference image set |
US8238671B1 (en) | 2009-12-07 | 2012-08-07 | Google Inc. | Scene classification for place recognition |
US8774527B1 (en) | 2009-12-07 | 2014-07-08 | Google Inc. | Matching an approximately located query image against a reference image set using cellular base station and wireless access point information |
US8488011B2 (en) | 2011-02-08 | 2013-07-16 | Longsand Limited | System to augment a visual data stream based on a combination of geographical and visual information |
US8509525B1 (en) | 2011-04-06 | 2013-08-13 | Google Inc. | Clustering of forms from large-scale scanned-document collection |
US8493353B2 (en) | 2011-04-13 | 2013-07-23 | Longsand Limited | Methods and systems for generating and joining shared experience |
US9064326B1 (en) | 2012-05-10 | 2015-06-23 | Longsand Limited | Local cache of augmented reality content in a mobile computing device |
US9430876B1 (en) | 2012-05-10 | 2016-08-30 | Aurasma Limited | Intelligent method of determining trigger items in augmented reality environments |
CN104219197A (en) * | 2013-05-30 | 2014-12-17 | 腾讯科技(深圳)有限公司 | Video conversation method, video conversation terminal, and video conversation system |
US20160217699A1 (en) * | 2013-09-02 | 2016-07-28 | Suresh T. Thankavel | Ar-book |
US10182187B2 (en) * | 2014-06-16 | 2019-01-15 | Playvuu, Inc. | Composing real-time processed video content with a mobile device |
CN106575446B (en) * | 2014-09-24 | 2020-04-21 | 英特尔公司 | Facial motion driven animation communication system |
JP6520831B2 (en) * | 2016-06-07 | 2019-05-29 | オムロン株式会社 | Display control apparatus, display control system, display control method, display control program, recording medium |
TWI655447B (en) * | 2016-12-26 | 2019-04-01 | 宏達國際電子股份有限公司 | Tracking system and tracking method |
US11486697B1 (en) | 2017-12-29 | 2022-11-01 | II John Tyson | Optical structural health monitoring |
US11132479B1 (en) | 2017-12-29 | 2021-09-28 | II John Tyson | Augmented reality system for component assembly and archival baseline clone |
US11232617B2 (en) * | 2018-01-11 | 2022-01-25 | Pamela L. Barber | Digital imaging method and apparatus |
US10636218B2 (en) | 2018-09-24 | 2020-04-28 | Universal City Studios Llc | Augmented reality for an amusement ride |
US11217345B2 (en) * | 2019-05-22 | 2022-01-04 | Mocxa Health Private Limited | Anonymization of audio-visual medical data |
WO2022001635A1 (en) * | 2020-07-03 | 2022-01-06 | 海信视像科技股份有限公司 | Display device and display method |
Family Cites Families (25)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US3899848A (en) | 1975-01-20 | 1975-08-19 | Aniforms Inc | Animated cartoon character and method |
US4200890A (en) | 1977-07-11 | 1980-04-29 | Nippon Electric Company, Ltd. | Digital video effects system employing a chroma-key tracking technique |
US4319266A (en) | 1979-09-04 | 1982-03-09 | The Grass Valley Group, Inc. | Chroma keying system |
US4872056A (en) * | 1987-02-05 | 1989-10-03 | Video Graphic Styling, Inc. | Method for displaying selected hairstyles in video form |
US4988981B1 (en) | 1987-03-17 | 1999-05-18 | Vpl Newco Inc | Computer data entry and manipulation apparatus and method |
US5384912A (en) | 1987-10-30 | 1995-01-24 | New Microtime Inc. | Real time video image processing system |
US5060171A (en) * | 1989-07-27 | 1991-10-22 | Clearpoint Research Corporation | A system and method for superimposing images |
US5521373A (en) | 1990-08-02 | 1996-05-28 | Vpl Research, Inc. | Position tracking system using a radiation director which directs radiation from a radiation source onto a radiation sensor, depending on the position of the radiation source |
GB9019538D0 (en) | 1990-09-07 | 1990-10-24 | Philips Electronic Associated | Tracking a moving object |
US5534917A (en) | 1991-05-09 | 1996-07-09 | Very Vivid, Inc. | Video image based control system |
US5227985A (en) | 1991-08-19 | 1993-07-13 | University Of Maryland | Computer vision system for position monitoring in three dimensions using non-coplanar light sources attached to a monitored object |
GB9119964D0 (en) * | 1991-09-18 | 1991-10-30 | Sarnoff David Res Center | Pattern-key video insertion |
GB9121707D0 (en) | 1991-10-12 | 1991-11-27 | British Aerospace | Improvements in computer-generated imagery |
US5469536A (en) | 1992-02-25 | 1995-11-21 | Imageware Software, Inc. | Image editing system including masking capability |
US5345313A (en) | 1992-02-25 | 1994-09-06 | Imageware Software, Inc | Image editing system for taking a background and inserting part of an image therein |
US5251016A (en) | 1992-05-11 | 1993-10-05 | The Grass Valley Group, Inc. | Chroma keyer with secondary hue selector |
GB9217098D0 (en) | 1992-08-12 | 1992-09-23 | British Broadcasting Corp | Derivation of studio camera position and motion from the camera image |
US5313275A (en) | 1992-09-30 | 1994-05-17 | Colorgraphics Systems, Inc. | Chroma processor including a look-up table or memory |
US5526022A (en) * | 1993-01-06 | 1996-06-11 | Virtual I/O, Inc. | Sourceless orientation sensor |
US5513129A (en) | 1993-07-14 | 1996-04-30 | Fakespace, Inc. | Method and system for controlling computer-generated virtual environment in response to audio signals |
US5423554A (en) | 1993-09-24 | 1995-06-13 | Metamedia Ventures, Inc. | Virtual reality game method and apparatus |
US5623587A (en) * | 1993-10-15 | 1997-04-22 | Kideo Productions, Inc. | Method and apparatus for producing an electronic image |
US5398075A (en) | 1993-11-19 | 1995-03-14 | Intel Corporation | Analog chroma keying on color data |
US5563988A (en) * | 1994-08-01 | 1996-10-08 | Massachusetts Institute Of Technology | Method and system for facilitating wireless, full-body, real-time user interaction with a digitally represented visual environment |
US5880731A (en) * | 1995-12-14 | 1999-03-09 | Microsoft Corporation | Use of avatars with automatic gesturing and bounded interaction in on-line chat session |
-
1996
- 1996-09-18 US US08/716,597 patent/US6400374B2/en not_active Expired - Lifetime
Cited By (80)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7298381B1 (en) * | 1997-10-31 | 2007-11-20 | Thomson Licensing | Arrangement for mixing and/or processing video signals |
US7472910B1 (en) * | 1999-03-30 | 2009-01-06 | Canon Kabushiki Kaisha | Animation display apparatus, arcade game machine, control method and apparatus thereof, and storage medium |
US10671153B2 (en) | 2000-03-28 | 2020-06-02 | Sony Corporation | Communication service method and communication apparatus thereof |
US20010039520A1 (en) * | 2000-03-28 | 2001-11-08 | Motoki Nakade | Communication service method and communication apparatus thereof |
US8621039B2 (en) | 2000-03-28 | 2013-12-31 | Sony Corporation | Communication service method and communication apparatus thereof |
US10142703B2 (en) * | 2000-03-28 | 2018-11-27 | Sony Corporation | Communication service method and communication apparatus thereof |
US20160171563A1 (en) * | 2000-03-28 | 2016-06-16 | Sony Corporation | Communication service method and communication apparatus thereof |
US20080095855A1 (en) * | 2000-03-28 | 2008-04-24 | Schwarz Franz X | Taste Masking Granules |
US8156014B2 (en) * | 2000-03-28 | 2012-04-10 | Sony Corporation | Communication service method and communication apparatus thereof for transmitting advertisements in a private communication environment |
US9430780B2 (en) | 2000-03-28 | 2016-08-30 | Sony Corporation | Communication service method and communication apparatus thereof |
US20050220871A1 (en) * | 2000-03-28 | 2005-10-06 | Schwarz Franz X | Taste masking granules |
US20050076134A1 (en) * | 2001-05-17 | 2005-04-07 | Gil Bialik | Apparatus and method for multiple rich media formats video broadcasting |
US20030003991A1 (en) * | 2001-06-29 | 2003-01-02 | Konami Corporation | Game device, game controlling method and program |
US7452275B2 (en) * | 2001-06-29 | 2008-11-18 | Konami Digital Entertainment Co., Ltd. | Game device, game controlling method and program |
US20040001074A1 (en) * | 2002-05-29 | 2004-01-01 | Hideki Oyaizu | Image display apparatus and method, transmitting apparatus and method, image display system, recording medium, and program |
US20060140508A1 (en) * | 2002-10-23 | 2006-06-29 | Kiyoshi Ohgishi | Image combining portable terminal and image combining method used therefor |
US20040113757A1 (en) * | 2002-12-10 | 2004-06-17 | White Melvin Joseph | Power line communication system and method of operating the same |
EP1582065A2 (en) * | 2002-12-30 | 2005-10-05 | Motorola, Inc. | A method, system and apparatus for telepresence communications |
US20060210045A1 (en) * | 2002-12-30 | 2006-09-21 | Motorola, Inc. | A method system and apparatus for telepresence communications utilizing video avatars |
US8072479B2 (en) | 2002-12-30 | 2011-12-06 | Motorola Mobility, Inc. | Method system and apparatus for telepresence communications utilizing video avatars |
EP1582065A4 (en) * | 2002-12-30 | 2009-03-11 | Motorola Inc | A method, system and apparatus for telepresence communications |
US20040179034A1 (en) * | 2003-02-26 | 2004-09-16 | Burritt David Ray | Apparatus and method for displaying background images (themes) for computer and handheld computer devices |
US20050084835A1 (en) * | 2003-10-16 | 2005-04-21 | The Singing Machine Company, Inc. | Karaoke system with built-in camera |
US20050204287A1 (en) * | 2004-02-06 | 2005-09-15 | Imagetech Co., Ltd | Method and system for producing real-time interactive video and audio |
US20050222712A1 (en) * | 2004-03-31 | 2005-10-06 | Honda Motor Co., Ltd. | Salesperson robot system |
WO2005117435A1 (en) * | 2004-05-31 | 2005-12-08 | Telecom Italia S.P.A | System for and method of personalising videocommunications, and computer program product therefor |
EP1779328A1 (en) * | 2004-08-03 | 2007-05-02 | William J. Benman | System and method for providing a functional virtual environment with real time extracted and transplanted images |
EP1779328A4 (en) * | 2004-08-03 | 2009-08-26 | William J Benman | System and method for providing a functional virtual environment with real time extracted and transplanted images |
US20080019576A1 (en) * | 2005-09-16 | 2008-01-24 | Blake Senftner | Personalizing a Video |
US20100202750A1 (en) * | 2005-09-16 | 2010-08-12 | Flixor, Inc., A California Corporation | Personalizing a Video |
US20090080855A1 (en) * | 2005-09-16 | 2009-03-26 | Flixor, Inc., A California Corporation | Personalizing a Video |
US7974493B2 (en) | 2005-09-16 | 2011-07-05 | Flixor, Inc. | Personalizing a video |
US20070190506A1 (en) * | 2005-12-26 | 2007-08-16 | Industrial Technology Research Institute | Online interactive multimedia system and the transmission method thereof |
US20070200925A1 (en) * | 2006-02-07 | 2007-08-30 | Lg Electronics Inc. | Video conference system and method in a communication network |
US8111280B2 (en) | 2006-02-07 | 2012-02-07 | Lg Electronics Inc. | Video conference system and method in a communication network |
EP1841226A3 (en) * | 2006-02-07 | 2008-01-23 | LG Electronics Inc. | Video conference system and method in a communication network |
WO2008139251A3 (en) * | 2006-04-14 | 2009-03-12 | Rosenthal Patrick Levy | Virtual video camera device with three-dimensional tracking and virtual object insertion |
WO2008139251A2 (en) * | 2006-04-14 | 2008-11-20 | Patrick Levy Rosenthal | Virtual video camera device with three-dimensional tracking and virtual object insertion |
JP2009536406A (en) * | 2006-05-07 | 2009-10-08 | 株式会社ソニー・コンピュータエンタテインメント | How to give emotional features to computer-generated avatars during gameplay |
EP2082393B1 (en) * | 2006-10-13 | 2015-08-26 | Freescale Semiconductor, Inc. | Image processing apparatus for superimposing windows displaying video data having different frame rates |
US20100033502A1 (en) * | 2006-10-13 | 2010-02-11 | Freescale Semiconductor, Inc. | Image processing apparatus for superimposing windows displaying video data having different frame rates |
EP1976291A1 (en) * | 2007-03-02 | 2008-10-01 | Deutsche Telekom AG | Method and video communication system for gesture-based real-time control of an avatar |
US9240029B2 (en) * | 2007-04-30 | 2016-01-19 | Here Global B.V. | Street level video simulation display system and method |
US20080266324A1 (en) * | 2007-04-30 | 2008-10-30 | Navteq North America, Llc | Street level video simulation display system and method |
US20090033737A1 (en) * | 2007-08-02 | 2009-02-05 | Stuart Goose | Method and System for Video Conferencing in a Virtual Environment |
US8279254B2 (en) | 2007-08-02 | 2012-10-02 | Siemens Enterprise Communications Gmbh & Co. Kg | Method and system for video conferencing in a virtual environment |
US7991203B2 (en) * | 2007-10-11 | 2011-08-02 | Hon Hai Precision Industry Co., Ltd. | Video communication device and image processing system and method of the same |
US20090097754A1 (en) * | 2007-10-11 | 2009-04-16 | Hon Hai Precision Industry Co., Ltd. | Video communication device and image processing system and method of the same |
US9813671B2 (en) * | 2008-02-04 | 2017-11-07 | Unify Gmbh & Co. Kg | Method and apparatus for enhanced video mixing |
US20090199078A1 (en) * | 2008-02-04 | 2009-08-06 | Siemens Communications, Inc. | Method and apparatus for enhanced video mixing |
US8753203B1 (en) * | 2008-04-10 | 2014-06-17 | Acme Embedded Solutions, Inc. | Compositing device for combining visual content |
WO2011124830A1 (en) * | 2010-04-06 | 2011-10-13 | Alcatel Lucent | A method of real-time cropping of a real entity recorded in a video sequence |
FR2958487A1 (en) * | 2010-04-06 | 2011-10-07 | Alcatel Lucent | A METHOD OF REAL TIME DISTORTION OF A REAL ENTITY RECORDED IN A VIDEO SEQUENCE |
WO2012040115A1 (en) * | 2010-09-20 | 2012-03-29 | Qualcomm Incorporated | Virtual video capture device |
JP2012253551A (en) * | 2011-06-02 | 2012-12-20 | Sony Corp | Display control apparatus, display control method, and program |
US20130009980A1 (en) * | 2011-07-07 | 2013-01-10 | Ati Technologies Ulc | Viewing-focus oriented image processing |
US20130044254A1 (en) * | 2011-08-18 | 2013-02-21 | Meir Tzur | Image capture for later refocusing or focus-manipulation |
US9501834B2 (en) * | 2011-08-18 | 2016-11-22 | Qualcomm Technologies, Inc. | Image capture for later refocusing or focus-manipulation |
EP2786561A1 (en) * | 2011-12-01 | 2014-10-08 | Tangome Inc. | Augmenting a video conference |
EP2786561A4 (en) * | 2011-12-01 | 2015-04-15 | Tangome Inc | Augmenting a video conference |
US8823807B2 (en) * | 2011-12-09 | 2014-09-02 | Robert Schinker | Methods and apparatus for enhanced reality messaging |
US20140002677A1 (en) * | 2011-12-09 | 2014-01-02 | Robert Schinker | Methods and apparatus for enhanced reality messaging |
US20130257877A1 (en) * | 2012-03-30 | 2013-10-03 | Videx, Inc. | Systems and Methods for Generating an Interactive Avatar Model |
US20150228247A1 (en) * | 2012-07-20 | 2015-08-13 | Feng-Chia Wu | Image generating method |
CN103581728A (en) * | 2012-07-20 | 2014-02-12 | 英特尔公司 | Selective post-processing of decoded video frames based on focus point determination |
TWI619071B (en) * | 2012-07-20 | 2018-03-21 | 英特爾公司 | Selective post-processing of decoded video frames based on focus point determination |
US10237528B2 (en) | 2013-03-14 | 2019-03-19 | Qualcomm Incorporated | System and method for real time 2D to 3D conversion of a video in a digital camera |
US9565363B1 (en) * | 2015-08-10 | 2017-02-07 | X Development Llc | Stabilization of captured images for teleoperated walking biped robots |
US9749535B1 (en) * | 2015-08-10 | 2017-08-29 | X Development Llc | Stabilization of captured images for a robot |
US11894023B2 (en) | 2015-11-18 | 2024-02-06 | International Business Machines Corporation | Video enhancement |
US10276210B2 (en) * | 2015-11-18 | 2019-04-30 | International Business Machines Corporation | Video enhancement |
US20170287097A1 (en) * | 2016-03-29 | 2017-10-05 | Ati Technologies Ulc | Hybrid client-server rendering in a virtual reality system |
US10529109B1 (en) | 2017-05-16 | 2020-01-07 | Facebook, Inc. | Video stream customization using graphics |
US10311619B1 (en) | 2017-05-16 | 2019-06-04 | Facebook, Inc. | Video stream customization using graphics |
US10078909B1 (en) * | 2017-05-16 | 2018-09-18 | Facebook, Inc. | Video stream customization using graphics |
US10835828B1 (en) * | 2018-07-25 | 2020-11-17 | Facebook, Inc. | Augmented-reality game overlays in video communications |
US11628367B2 (en) * | 2018-07-25 | 2023-04-18 | Meta Platforms, Inc. | Augmented-reality game overlays in video communications |
US20230249086A1 (en) * | 2018-07-25 | 2023-08-10 | Meta Platforms, Inc. | Augmented-Reality Game Overlays in Video Communications |
US11749020B2 (en) * | 2018-07-27 | 2023-09-05 | Beijing Microlive Vision Technology Co., Ltd | Method and apparatus for multi-face tracking of a face effect, and electronic device |
CN114501140A (en) * | 2021-12-28 | 2022-05-13 | 安徽清新物联科技有限公司 | Video character superposition processing method, system and storage medium |
Also Published As
Publication number | Publication date |
---|---|
US6400374B2 (en) | 2002-06-04 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US6400374B2 (en) | Video superposition system and method | |
US11887234B2 (en) | Avatar display device, avatar generating device, and program | |
US5563988A (en) | Method and system for facilitating wireless, full-body, real-time user interaction with a digitally represented visual environment | |
US7825948B2 (en) | 3D video conferencing | |
CN109952759B (en) | Improved method and system for video conferencing with HMD | |
US9225973B2 (en) | Image processing apparatus, image processing method, and image communication system | |
AU2017232507B2 (en) | Wide baseline stereo for low-latency rendering | |
US20140232749A1 (en) | Vision-based augmented reality system using invisible marker | |
Theobalt et al. | Combining 2D feature tracking and volume reconstruction for online video-based human motion capture | |
US20120200667A1 (en) | Systems and methods to facilitate interactions with virtual content | |
US11204495B2 (en) | Device and method for generating a model of an object with superposition image data in a virtual environment | |
JPH08237629A (en) | System and method for video conference that provides parallax correction and feeling of presence | |
JP7317024B2 (en) | Image generation device and image generation method | |
US11652970B2 (en) | Apparatus and method for representing a spatial image of an object in a virtual environment | |
WO2003063086A1 (en) | Image processing system, image processing apparatus, and display apparatus | |
US7023454B1 (en) | Method and apparatus for creating a virtual video of an object | |
WO2022020058A1 (en) | 3d conversations in an artificial reality environment | |
US20220114784A1 (en) | Device and method for generating a model of an object with superposition image data in a virtual environment | |
JPH1196366A (en) | Method and device for synthesizing facial image of person wearing head mount display | |
CN112236740A (en) | Heat map display device and heat map display program | |
EP4113982A1 (en) | Method for sensing and communicating visual focus of attention in a video conference | |
JP7403256B2 (en) | Video presentation device and program | |
CN113409469A (en) | Reality fusion interaction system and method | |
Ohya | Virtual metamorphosis systems | |
WO2012010920A1 (en) | Method for visualizing a user of a virtual environment |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: EYEMATIC INTERFACES, INC., CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:LANIER, JARON;REEL/FRAME:010687/0949 Effective date: 20000303 |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
AS | Assignment |
Owner name: EYEMATIC INTERFACES, INC., CALIFORNIA Free format text: SECURITY AGREEMENT;ASSIGNOR:NEVENGINEERING, INC.;REEL/FRAME:014532/0427 Effective date: 20030702 |
|
AS | Assignment |
Owner name: NEVENGINEERING, INC., CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:EYEMATIC INTERFACES, INC.;REEL/FRAME:015008/0717 Effective date: 20031003 |
|
FPAY | Fee payment |
Year of fee payment: 4 |
|
AS | Assignment |
Owner name: GOOGLE INC.,CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:NEVENGINEERING, INC.;REEL/FRAME:018616/0814 Effective date: 20061206 Owner name: GOOGLE INC., CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:NEVENGINEERING, INC.;REEL/FRAME:018616/0814 Effective date: 20061206 |
|
FPAY | Fee payment |
Year of fee payment: 8 |
|
FPAY | Fee payment |
Year of fee payment: 12 |
|
AS | Assignment |
Owner name: GOOGLE LLC, CALIFORNIA Free format text: CHANGE OF NAME;ASSIGNOR:GOOGLE INC.;REEL/FRAME:044127/0735 Effective date: 20170929 |