WO2007061678A2 - Screen mounted video teleconferencing camera apparatus - Google Patents

Screen mounted video teleconferencing camera apparatus Download PDF

Info

Publication number
WO2007061678A2
WO2007061678A2 PCT/US2006/044096 US2006044096W WO2007061678A2 WO 2007061678 A2 WO2007061678 A2 WO 2007061678A2 US 2006044096 W US2006044096 W US 2006044096W WO 2007061678 A2 WO2007061678 A2 WO 2007061678A2
Authority
WO
WIPO (PCT)
Prior art keywords
video
screen
camera
video screen
teleconferencing
Prior art date
Application number
PCT/US2006/044096
Other languages
French (fr)
Other versions
WO2007061678A3 (en
Inventor
Roy Sandberg
Dan Sandberg
Original Assignee
Roy Sandberg
Dan Sandberg
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Roy Sandberg, Dan Sandberg filed Critical Roy Sandberg
Publication of WO2007061678A2 publication Critical patent/WO2007061678A2/en
Publication of WO2007061678A3 publication Critical patent/WO2007061678A3/en

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00Television systems
    • H04N7/14Systems for two-way working
    • H04N7/141Systems for two-way working between two video terminals, e.g. videophone
    • H04N7/142Constructional details of the terminal equipment, e.g. arrangements of the camera and the display
    • H04N7/144Constructional details of the terminal equipment, e.g. arrangements of the camera and the display camera and display on the same optical axis, e.g. optically multiplexing the camera and display for eye to eye contact

Definitions

  • the present invention is related to the field of video teleconferencing, more specifically, the invention is an apparatus for reducing gaze-perception discrepancies while video teleconferencing.
  • Gaze preservation concerns the problem of appearing to maintain eye contact with a remote user when video teleconferencing. This is advantageous for maintaining the psychological illusion of face-to-face conversation.
  • Standard video teleconferencing systems involve a screen and a camera, where the camera is placed some distance away from the screen. Because a user typically looks at the screen and not the camera, a remote viewer does not perceive the user to be looking into the remote viewer's eyes. This creates a sense of psychological distance between the user and the viewer, and to some degree prevents the video teleconferencing experience from being as emotionally rich as it might be in person.
  • the present invention is a new and improved method and apparatus to preserve gaze across a video teleconferencing link.
  • the invention is a low-cost solution to gaze preservation consisting of mounting a camera directly on the display screen, and preferentially along the same axis as the line of sight between the local user and the remote user's eyes as displayed on the display screen.
  • the frontal area of the camera By keeping the frontal area of the camera small, the camera only minimally disrupts the local user's view of the remote. user.
  • the wire leading away from the camera is designed to be as a narrow as reasonably possible to avoid blocking screen real estate.
  • a suction cup or other means of attaching the camera in front of the screen should be provided.
  • the camera is held in front of the screen using a rigid support, the rigid support in turn connecting to an off-screen location.
  • An optional quick release feature for the suction cup allows a user to press a button on the camera and easily detach the camera from the screen by re-pressurizing the area between the suction cup and the screen.
  • the device is operated by placing it as near as possible to the image of eyes of the person being video teleconferenced with on the display screen.
  • the camera is placed on the surface of the display screen between the eyes of the person displayed on the screen, that person being the person who is being video teleconferenced with.
  • the camera is located at the horizontal center point of the display surface and about 4 inches below the top edge of the display surface.
  • the wires that supply the camera power and data are routed from the camera to the edge of the screen.
  • the device is then used in the same manner as existing video cameras or web cams.
  • a "web cam” style camera is built out of a lens, lens holder, and CCD or CMOS sensor.
  • other electronic components are often required to make a functioning CCD or CMOS camera.
  • integrated circuit packages may be used that convert the output of the CCD or CMOS camera into a format suitable for streaming over a Universal Serial Bus (“USB”) or other serial interface.
  • USB Universal Serial Bus
  • these additional electronic packages are situated behind the CCD or CMOS chip, and thus do not add to frontal area of the camera, thereby enabling an overall form factor with a smaller footprint on the LCD screen.
  • Techniques know in the art of electronic printed circuit board manufacturing such as double sided printed circuit boards, headers, and daughter boards may be used to implement this embodiment.
  • the camera and LCD form an integrated system.
  • the camera is attached to a slide or pivot mechanism that allows it to move from a non-overlapping position to a position that overlaps and obscures a portion of the LCD screen. This is useful for applications where the user does not always want a camera overlapping the screen. By sliding or pivoting the camera on or off the screen, the user can configure the system to best suit his immediate needs.
  • the slide or pivot assembly is composed of a transparent material such as polycarbonate or acrylic, and thin or transparent wires are routing along the slide or pivot assembly to supply power and read data from the camera. This minimizes the degree to which the screen is obstructed by the camera and the slide or pivot assembly.
  • the camera is attached to a slide or pivot mechanism, and the slide or pivot mechanism connects to a hanger section.
  • the hanger section allows the invention to hang over the top of a monitor or LCD panel
  • the slide or pivot mechanism allows the camera to be moved from a non- overlapping position to a position that overlaps and obscures a portion of the monitor or LCD screen.
  • the slide or pivot assembly is composed of a transparent material such as polycarbonate or acrylic, and thin or transparent wires are routing along the slide or pivot assembly to supply power and read data from the camera. This reduces the degree to which the screen is obstructed by the camera and the slide or pivot assembly.
  • the invention provides a low cost solution to the problem of gaze preservation, and is usable with all forms of video teleconferencing utilizing a glass or plastic display screen. While some portion of the display screen is blocked, that portion, (ideally the portion near or at the image of the bridge of the other user's nose) is not critical to gathering visual cues about the other party.
  • An alternative solution consists of mounted a small prism or mirror to the display surface, and reflecting the image of the local user as positioned in front of the display surface from the prism or mirror to a camera mounted near or along the edge of the computer screen.
  • the mirror or prism is oriented such that the scene in front of the display surface is reflected into the camera.
  • the mirror or prism is manufactured with a concave curvature that directs light from at least a 20 degree diagonal field of view in front of the display surface onto the camera's lens element.
  • the prism or mirror is mounted to the screen using a suction cup or other means of positioning the prism or mirror in from of the screen.
  • An optional quick release feature for the suction cup allows a user to press a button on the prism or mirror assembly and easily detach the assembly from the screen by re-pressurizing the area between the suction cup and the screen.
  • the device is operated by placing the mirror or prism assembly as near as possible to the image of the person being video teleconferenced with on the display screen.
  • the camera is placed between the eyes of the person being video teleconferenced with.
  • a camera is then mounted coaxially with the reflected image of the local user that has been redirected by the prism or mirror. This typically requires that the camera is mounted along the edge of the screen and attached with glue, hook-and-loop cloth, tape, clamps, or other means of attaching the camera known in the art.
  • a means of aligning the camera with the prism or mirror assembly is necessitated for proper operation of the system. This may involve adjustment screws on the camera or prism, or alignment guides that are removed after proper alignment has occurred.
  • the device is then used in the same manner as existing web cameras.
  • the preferred embodiment of this invention uses digital image stabilization techniques to maintain the video image of the remote user in a constant position oit the video screen. This assists in maintaining the video image of the remote user's face in a constant position on the local video screen, thereby assisting in maintaining the video camera's position in its preferred location, typically between the eyes of the remote user as displayed on the local video screen.
  • Techniques similar to image stabilization such as face-tracking, and jitter-reduction algorithms may also be used. In all cases, the techniques may be implemented as software executing on a processor, or using dedicated hardware, or a combination of both techniques.
  • the image stabilization may be done at the remote location or at the local location. In the preferred embodiment the image stabilization would be done at the remote location, as this would have the added benefit of reducing the bandwidth of the transmitted video data due a reduction in frame-to-frame movement when using temporal video compression techniques.
  • a hole may also be made through the opaque layers of an LCD panel during the LCD manufacturing process.
  • a CMOS or CCD imaging camera may then be located behind this hole, and via this means, the LCD screen appears to be unmodified on casual inspection.
  • the imaging sensor and the LCD may also both be produced on the same semiconductor substrate, thereby saving the extra step of integrating the imaging sensor and LCD.
  • metal wires or other conductive materials are used to transfer power and data to and from the CCD. Conductors can then be routed along the LCD screen or along a support structure without being visible to the viewer.
  • the invention provides a low cost solution to the problem of gaze preservation, and Js usable with all forms of video teleconferencing utilizing a glass or plastic display screen DETABLED DESCRIPTION OF THE DRAWINGS
  • FIG. 1 is an exemplary embodiment of the camera on the screen apparatus.
  • FIG. 2 illustrates the orientation of the local user and the remote user during a video conferencing session when using the invention.
  • FIG. 3 illustrates the slide mechanism embodiment of the invention.
  • the present invention is a new and improved video teleconferencing method and apparatus to preserve gaze across a video teleconferencing link by directly attaching a camera, or reflective device such as a mirror or prism to the active video display surface.
  • FIG. 1 is an exemplary embodiment of the camera on the screen apparatus.
  • the monitor casing 101 is shown in a portrait orientation, but in an alternative embodiment, the landscape orientation may also be used.
  • the monitor active display surface 102 is a conventional glass screen, although other display surfaces, such as rigid plastic or flexible plastic may also be used.
  • the monitor active display surface 102 is shown as being flat, but a curved display surface may be used in alternative embodiments.
  • the data and power cable 103 is shown as a thin wire that has been glued onto the active display surface and routes into the monitor casing 101. In alternative embodiments, the data and power cable may be freely hanging, and need not enter the monitor case. Attachment points for the data and power cable on the monitor casing may be used in alternative embodiments.
  • the video camera 104 is shown as a small tubular chassis with a cylindrical lens that faces away from the display surface. Other video camera configurations and sizes may also be used. In the preferred embodiment a video.camera based on charge coupled device technology is used, but other video camera technologies known in the art may also be used. Larger or small video cameras may also be used, but in the preferred embodiment, a video camera with a small surface area as measured on the plane formed by the active display will be used.
  • the video camera to active display surface interface 105 is attached with an epoxy adhesive. Other adhesives known in the art, such as polyurethane adhesives, silicon-based adhesives, anaerobic adhesives, or acrylic adhesives may also be used.
  • a thin steel rod may extend from the edge of the bezel and substantially parallel to the display surface. The far end of the thin steel rod attaches mechanically or by other means known in the art to the video camera.
  • a glass or transparent plastic rod, sheet, or plate may be used to position the video camera at the appropriate location on the active portion of the video display.
  • a mirror is placed on the screen in lieu of the camera.
  • the video camera is oriented so that it faces the mirror assembly. Optic properties of the mirror and the camera enlarge the apparent size of the mirror's image such that substantially all of the field of view of the video camera consists of the view as seen by the mirror.
  • the mirror's orientation allows imagery located substantially in front of the display screen to be reflected into the lens of the video camera. By this design, a person located in front of the display will appear to be located in front of the video camera. Because reflection through a mirror reflects the orientation of the image as seen in the video camera, the image may need to be digitally manipulated so that it is reflected back to its proper orientation * Techniques to accomplish the manipulation are well known in the art.
  • a concave reflecting surface is required to form an image with a sufficiently large field of view on the image sensor.
  • Digital imaging techniques known in the art may be used to correct for any optical aberrations due to the curvature of the reflecting surface.
  • the mirror assembly may be bonded to the display surface with an epoxy adhesive.
  • Other adhesives known in the art such as polyurethane adhesives, silicon-based adhesives, anaerobic adhesives, or acrylic adhesives may also be used.
  • Other techniques used to join materials to glass or plastic such as diffusion welding or vacuum-based bonding, for example, suction cups, may also be used.
  • Mechanical attachment to the monitor bezel and an offsetting mechanism may also be used. For example, an thin steel rod may extend from the edge of the bezel and substantially parallel to the display surface.
  • the far end of the thin steel rod may attach mechanically or by other means known in the art to the mirror assembly.
  • a glass or transparent plastic rod, sheet, or plate may be used to position the mirror assembly at the appropriate location on the active portion of the video display.
  • an alternative method of optical reflection such as a prism may also be used.
  • both sides of a video teleconferencing call will use the apparatus described herein. If both sides are using the device, both sides will feel that the other user is looking them straight in the eye. This is preferable to when only side A of a conversation has the apparatus, which results in side B feeling like she is being looked at directly while side A will feel that side B is looking elsewhere.
  • a visual cue can be provided to facilitate this.
  • the user is easily able to move his head ( or the camera if the camera is movable ) so that his eyes are at the correct position for the remote user.
  • FIG. 2 illustrates the orientation of the local user and the remote user during a video conferencing session when using the invention.
  • the monitor casing 201 is shown in a landscape orientation, but in an alternative embodiment, the portrait orientation may also be used.
  • the removable video camera 203 is shown as a small tubular chassis with a cylindrical lens that faces away from the monitor active display surface 202.
  • the video camera 203 is attached to the monitor active display surface 202 with a suction cup concealed within the tubular chassis 2023
  • a vacuum release button 204 is used to create and release vacuum within the suction cup, facilitating removal of the invention when not being used.
  • the video camera power and data wire 205 is shown stretched over the top of the monitor.
  • the wire continues until it connects to a personal computer or other computational device capable of accepting a video camera input, not shown.
  • the local user 206 is shown with his gaze directed at the video camera. Due to the small angular distance between the video camera 203 and the remote user's eyes 207 depicted on the monitor active display surface 203 , the remote user perceives the local user 206 as looking substantially into his eyes.
  • a hole is made through the opaque layers of an LCD panel during the LCD manufacturing process.
  • a CCD or CMOS imaging camera is located behind this hole, and via this means, the LCD screen appears to be unmodified.
  • This also has the benefit of blocking less LCD screen "real-estate" as the hole in the LCD panel need only be small enough to provide enough light for the aperture of the imaging device's lens.
  • the aperture surface area is, of course, smaller than the surface area required to house an entire video camera.
  • the imaging sensor and the LCD are both produced on the same glass screen substrate, thereby saving the extra step of integrating the imaging sensor and LCD.
  • thin metal wires or other transparent conductive materials that are invisible to the naked eye are used to transfer power and data to and from the imaging device. These wires can be routed along the LCD screen without being visible to the viewer. In an alternative embodiment, the wires can be routed along a transparent support structure that holds the camera assembly in a fixed location on the active portion of the monitor.
  • conductive materials such as Poly(3,4- ethylenedioxythiophene)/ ⁇ oly(styrenesulfonate), which is available under the tradename Baytron® P, may also be used. Carbon nanotubes may also be used to form a transparent conductive electrical conduit.
  • HG. 3 illustrates the slide mechanism embodiment of the invention.
  • the monitor casing 301 is shown in a landscape orientation, but in an alternative embodiment, the portrait orientation may also be used.
  • the video camera 302 is shown as a small tubular chassis with a cylindrical lens that faces away from the monitor active display surface 303.
  • the video camera 302 is attached to a slide mechanism 304.
  • the slide mechanism 304 slides vertically through a slide mechanism retainer 305.
  • the slide mechanism retainer is permanently attached to the monitor casing using fabrication techniques known in the art. In the preferred embodiment, the slide mechanism retainer can be removed from the monitor casing.
  • Various removable means of attachment such as vacuum bonding (for example, suction cups), hook-and-loop fastener, ledge and ledge grabber, or a friction interface may be used to removably attach the slide mechanism retainer.
  • a user can position the video camera 302 either on-screen or off-screen by sliding the slide mechanism 304 up and down within the slide mechanism retainer 305.
  • the video camera 302 is shown on-screen, covering the video image of a remote user's forehead 306. Because of the proximity of the video camera 302 to the image of the remote user's eyes 307, the remote user will perceive the local user's gaze as being directed at him to a greater degree than if the camera 302 is located offscreen.
  • the preferred embodiment of this invention uses digital image stabilization techniques to maintain the video image of the remote user in a constant position on the video screen. This allows the image of the remote user to be stable in its preferred position, where the eyes of the remote user, when displayed on the local video screen, straddle the on-screen video camera.
  • Techniques known in the art similar to image stabilization, such as face-tracking, and jitter-reduction algorithms may also be used. In all cases, the techniques may be implemented as software executing on a processor, or using dedicated hardware, or a combination of both techniques.
  • the image stabilization may be done at the remote location or at the local location.
  • the image stabilization would be done at the remote location, as this would have the added benefit of reducing the bandwidth of the transmitted video data due a reduction in frame-to-frame movement when using temporal video compression techniques
  • a face tracking algorithm known in the art, such as the algorithm presented by RJ. Qian, MJ. Sezan, and K.E. Matthews in their paper, A robust real-time face tracking algorithm, (ISBN: 0-8186- 8821-1, 1998) is used to determine the center of the location of a user's face in a video frame. This location is representing as an (x,y) offset from the upper left corner of the video frame.
  • a preferred camera position on the local users face (typically at the bridge of the nose or slightly higher) is represented as an offset (u,v) from the center location of a user's face.
  • a video frame is displayed on the remote side, it is translated by (x-u, y-v) relative to the center of the video frame. This maintains the local user's face in a constant position, with the camera located in its preferred location as viewed by the remote user.

Abstract

An apparatus for reducing gaze-perception discrepencies while video teleconferencing, consisting of a camera (203) mounted directly on display screen (202), and preferentially along the same axis as line of sight between the local user (206) and the remote user's eyes (207) as displayed on the display screen.

Description

Screen Mounted Video Teleconferencing Camera
~ Apparatus
by Roy Benjamin Sandberg, residing at 1515 Alabama St., Apt. #C, Huntington Beach, CA
and
Dan Ron Safidberg, residing at 163 Faiπnount St., San Francisco, CA
BACKGROUND OF THE IN VENΗON
(1) Field of Invention
The present invention, is related to the field of video teleconferencing, more specifically, the invention is an apparatus for reducing gaze-perception discrepancies while video teleconferencing.
(2) Related Art
Gaze preservation concerns the problem of appearing to maintain eye contact with a remote user when video teleconferencing. This is advantageous for maintaining the psychological illusion of face-to-face conversation. Standard video teleconferencing systems involve a screen and a camera, where the camera is placed some distance away from the screen. Because a user typically looks at the screen and not the camera, a remote viewer does not perceive the user to be looking into the remote viewer's eyes. This creates a sense of psychological distance between the user and the viewer, and to some degree prevents the video teleconferencing experience from being as emotionally rich as it might be in person.
Other means of gaze preservation across a video teleconferencing link are known, and include: U.S. Patent number 6,889,120 (Jouppi). Complex systems involving software transformations of the camera image or partial mirrors placed in front of the screen also exist. However, these methods require complex and expensive methods and apparatuses and/or do not perfectly preserve proper gaze.
SUMMARY OF THE INVENTION
The present invention is a new and improved method and apparatus to preserve gaze across a video teleconferencing link. The invention is a low-cost solution to gaze preservation consisting of mounting a camera directly on the display screen, and preferentially along the same axis as the line of sight between the local user and the remote user's eyes as displayed on the display screen. By keeping the frontal area of the camera small, the camera only minimally disrupts the local user's view of the remote. user. Additionally, the wire leading away from the camera is designed to be as a narrow as reasonably possible to avoid blocking screen real estate. A suction cup or other means of attaching the camera in front of the screen should be provided. In alternative embodiments, the camera is held in front of the screen using a rigid support, the rigid support in turn connecting to an off-screen location. An optional quick release feature for the suction cup allows a user to press a button on the camera and easily detach the camera from the screen by re-pressurizing the area between the suction cup and the screen.
The device is operated by placing it as near as possible to the image of eyes of the person being video teleconferenced with on the display screen. IQ the preferred embodiment, the camera is placed on the surface of the display screen between the eyes of the person displayed on the screen, that person being the person who is being video teleconferenced with. By way of example, in an typical embodiment using a 17 inch monitor with full-screen video conferencing, the camera is located at the horizontal center point of the display surface and about 4 inches below the top edge of the display surface. The wires that supply the camera power and data are routed from the camera to the edge of the screen. The device is then used in the same manner as existing video cameras or web cams.
A "web cam" style camera is built out of a lens, lens holder, and CCD or CMOS sensor. In addition to the sensor, other electronic components are often required to make a functioning CCD or CMOS camera. For example, integrated circuit packages may be used that convert the output of the CCD or CMOS camera into a format suitable for streaming over a Universal Serial Bus ("USB") or other serial interface. In the preferred embodiment of the invention, these additional electronic packages are situated behind the CCD or CMOS chip, and thus do not add to frontal area of the camera, thereby enabling an overall form factor with a smaller footprint on the LCD screen. Techniques know in the art of electronic printed circuit board manufacturing such as double sided printed circuit boards, headers, and daughter boards may be used to implement this embodiment.
In another embodiment of the invention, the camera and LCD form an integrated system. In the preferred embodiment, the camera is attached to a slide or pivot mechanism that allows it to move from a non-overlapping position to a position that overlaps and obscures a portion of the LCD screen. This is useful for applications where the user does not always want a camera overlapping the screen. By sliding or pivoting the camera on or off the screen, the user can configure the system to best suit his immediate needs. In the preferred embodiment, the slide or pivot assembly is composed of a transparent material such as polycarbonate or acrylic, and thin or transparent wires are routing along the slide or pivot assembly to supply power and read data from the camera. This minimizes the degree to which the screen is obstructed by the camera and the slide or pivot assembly.
In another embodiment of the invention, the camera is attached to a slide or pivot mechanism, and the slide or pivot mechanism connects to a hanger section. The hanger section allows the invention to hang over the top of a monitor or LCD panel, and the slide or pivot mechanism allows the camera to be moved from a non- overlapping position to a position that overlaps and obscures a portion of the monitor or LCD screen. This allows the invention to be used with a standard monitor or LCD panel while being easily moved between on-screen and off-screen positions. In the preferred embodiment, the slide or pivot assembly is composed of a transparent material such as polycarbonate or acrylic, and thin or transparent wires are routing along the slide or pivot assembly to supply power and read data from the camera. This reduces the degree to which the screen is obstructed by the camera and the slide or pivot assembly.
The invention provides a low cost solution to the problem of gaze preservation, and is usable with all forms of video teleconferencing utilizing a glass or plastic display screen. While some portion of the display screen is blocked, that portion, (ideally the portion near or at the image of the bridge of the other user's nose) is not critical to gathering visual cues about the other party.
An alternative solution consists of mounted a small prism or mirror to the display surface, and reflecting the image of the local user as positioned in front of the display surface from the prism or mirror to a camera mounted near or along the edge of the computer screen. The mirror or prism is oriented such that the scene in front of the display surface is reflected into the camera. Furthermore, the mirror or prism is manufactured with a concave curvature that directs light from at least a 20 degree diagonal field of view in front of the display surface onto the camera's lens element. This solution eliminates the requirement for a wire running along the screen, and allows the screen real estate blocked by the prism or mirror to be reduced relative to that blocked by the previous version of the invention. The prism or mirror is mounted to the screen using a suction cup or other means of positioning the prism or mirror in from of the screen. An optional quick release feature for the suction cup allows a user to press a button on the prism or mirror assembly and easily detach the assembly from the screen by re-pressurizing the area between the suction cup and the screen.
The device is operated by placing the mirror or prism assembly as near as possible to the image of the person being video teleconferenced with on the display screen. In the preferred embodiment, the camera is placed between the eyes of the person being video teleconferenced with. A camera is then mounted coaxially with the reflected image of the local user that has been redirected by the prism or mirror. This typically requires that the camera is mounted along the edge of the screen and attached with glue, hook-and-loop cloth, tape, clamps, or other means of attaching the camera known in the art. A means of aligning the camera with the prism or mirror assembly is necessitated for proper operation of the system. This may involve adjustment screws on the camera or prism, or alignment guides that are removed after proper alignment has occurred. The device is then used in the same manner as existing web cameras.
The preferred embodiment of this invention uses digital image stabilization techniques to maintain the video image of the remote user in a constant position oit the video screen. This assists in maintaining the video image of the remote user's face in a constant position on the local video screen, thereby assisting in maintaining the video camera's position in its preferred location, typically between the eyes of the remote user as displayed on the local video screen. Techniques similar to image stabilization, such as face-tracking, and jitter-reduction algorithms may also be used. In all cases, the techniques may be implemented as software executing on a processor, or using dedicated hardware, or a combination of both techniques. The image stabilization may be done at the remote location or at the local location. In the preferred embodiment the image stabilization would be done at the remote location, as this would have the added benefit of reducing the bandwidth of the transmitted video data due a reduction in frame-to-frame movement when using temporal video compression techniques.
In lieu of placing the video camera on the active display surface, a hole may also be made through the opaque layers of an LCD panel during the LCD manufacturing process. A CMOS or CCD imaging camera may then be located behind this hole, and via this means, the LCD screen appears to be unmodified on casual inspection.
The imaging sensor and the LCD may also both be produced on the same semiconductor substrate, thereby saving the extra step of integrating the imaging sensor and LCD.
Finally, metal wires or other conductive materials are used to transfer power and data to and from the CCD. Conductors can then be routed along the LCD screen or along a support structure without being visible to the viewer.
The invention provides a low cost solution to the problem of gaze preservation, and Js usable with all forms of video teleconferencing utilizing a glass or plastic display screen DETABLED DESCRIPTION OF THE DRAWINGS
FIG. 1 is an exemplary embodiment of the camera on the screen apparatus. FIG. 2 illustrates the orientation of the local user and the remote user during a video conferencing session when using the invention. FIG. 3 illustrates the slide mechanism embodiment of the invention.
DETAg-ED DESCRIPTION OF THE INVENTION
The present invention is a new and improved video teleconferencing method and apparatus to preserve gaze across a video teleconferencing link by directly attaching a camera, or reflective device such as a mirror or prism to the active video display surface.
FIG. 1 is an exemplary embodiment of the camera on the screen apparatus. The monitor casing 101 is shown in a portrait orientation, but in an alternative embodiment, the landscape orientation may also be used. The monitor active display surface 102 is a conventional glass screen, although other display surfaces, such as rigid plastic or flexible plastic may also be used. The monitor active display surface 102 is shown as being flat, but a curved display surface may be used in alternative embodiments. The data and power cable 103 is shown as a thin wire that has been glued onto the active display surface and routes into the monitor casing 101. In alternative embodiments, the data and power cable may be freely hanging, and need not enter the monitor case. Attachment points for the data and power cable on the monitor casing may be used in alternative embodiments. The video camera 104, is shown as a small tubular chassis with a cylindrical lens that faces away from the display surface. Other video camera configurations and sizes may also be used. In the preferred embodiment a video.camera based on charge coupled device technology is used, but other video camera technologies known in the art may also be used. Larger or small video cameras may also be used, but in the preferred embodiment, a video camera with a small surface area as measured on the plane formed by the active display will be used. The video camera to active display surface interface 105 is attached with an epoxy adhesive. Other adhesives known in the art, such as polyurethane adhesives, silicon-based adhesives, anaerobic adhesives, or acrylic adhesives may also be used. Other techniques used to join materials to glass or plastic, such as diffusion welding or vacuum-based bonding, for example, suction cups, may also be used. Mechanical attachment to the monitor bezel and an offsetting mechanism may also be used. For example, a thin steel rod may extend from the edge of the bezel and substantially parallel to the display surface. The far end of the thin steel rod attaches mechanically or by other means known in the art to the video camera. Alternatively, a glass or transparent plastic rod, sheet, or plate may be used to position the video camera at the appropriate location on the active portion of the video display. These methods alleviates the requirement that a direct bond between the video camera and the display surface exist.
In an alternative embodiment, a mirror is placed on the screen in lieu of the camera. The video camera is oriented so that it faces the mirror assembly. Optic properties of the mirror and the camera enlarge the apparent size of the mirror's image such that substantially all of the field of view of the video camera consists of the view as seen by the mirror. The mirror's orientation allows imagery located substantially in front of the display screen to be reflected into the lens of the video camera. By this design, a person located in front of the display will appear to be located in front of the video camera. Because reflection through a mirror reflects the orientation of the image as seen in the video camera, the image may need to be digitally manipulated so that it is reflected back to its proper orientation* Techniques to accomplish the manipulation are well known in the art. Furthermore, a concave reflecting surface is required to form an image with a sufficiently large field of view on the image sensor. Digital imaging techniques known in the art may be used to correct for any optical aberrations due to the curvature of the reflecting surface. The mirror assembly may be bonded to the display surface with an epoxy adhesive. Other adhesives known in the art, such as polyurethane adhesives, silicon-based adhesives, anaerobic adhesives, or acrylic adhesives may also be used. Other techniques used to join materials to glass or plastic, such as diffusion welding or vacuum-based bonding, for example, suction cups, may also be used. Mechanical attachment to the monitor bezel and an offsetting mechanism may also be used. For example, an thin steel rod may extend from the edge of the bezel and substantially parallel to the display surface. The far end of the thin steel rod may attach mechanically or by other means known in the art to the mirror assembly. Alternatively, a glass or transparent plastic rod, sheet, or plate may be used to position the mirror assembly at the appropriate location on the active portion of the video display. These methods alleviates the requirement that a direct bond between the mirror assembly and the display surface exist.
In lieu of the mirror, an alternative method of optical reflection such as a prism may also be used.
This patent application incorporates by reference copending application 11/223675 (Sandberg). Matter essential to the understanding of the present application is contained therein. Specifically, by mounting a teleconferencing apparatus of the sort described in this invention on a remotely controlled platform as per pending patent 11/223675 (Sandberg) a user is able to project his presence to a remote location in a more realistic manner.
By being able to maintain direct eye contact with users in the apparatus' environment, the user's sense of presence, as well as the perception of presence by those near the apparatus is heightened. The apparatus described herein, combined with electro-mechanical actuators allowing the orientation and position of the apparatus to be changed (as described in pending patent 11/223675) can thus be used to greatly enhance the perception of presence in a remote telepresence device.
Ideally, both sides of a video teleconferencing call will use the apparatus described herein. If both sides are using the device, both sides will feel that the other user is looking them straight in the eye. This is preferable to when only side A of a conversation has the apparatus, which results in side B feeling like she is being looked at directly while side A will feel that side B is looking elsewhere.
When one side of a video teleconferencing call is aware that the other side is using the apparatus described herein he should make an effort to align his own head so that his eyes will be displayed near the camera on the remote user's apparatus. A visual cue can be provided to facilitate this. By showing the user his own image with a visual cue superimposed on top of it showing where his eyes should be, the user is easily able to move his head ( or the camera if the camera is movable ) so that his eyes are at the correct position for the remote user.
FIG. 2 illustrates the orientation of the local user and the remote user during a video conferencing session when using the invention. The monitor casing 201 is shown in a landscape orientation, but in an alternative embodiment, the portrait orientation may also be used. The removable video camera 203, is shown as a small tubular chassis with a cylindrical lens that faces away from the monitor active display surface 202. The video camera 203 is attached to the monitor active display surface 202 with a suction cup concealed within the tubular chassis 2023 A vacuum release button 204 is used to create and release vacuum within the suction cup, facilitating removal of the invention when not being used. The video camera power and data wire 205 is shown stretched over the top of the monitor. The wire continues until it connects to a personal computer or other computational device capable of accepting a video camera input, not shown. The local user 206 is shown with his gaze directed at the video camera. Due to the small angular distance between the video camera 203 and the remote user's eyes 207 depicted on the monitor active display surface 203 , the remote user perceives the local user 206 as looking substantially into his eyes.
In an alternative embodiment, a hole is made through the opaque layers of an LCD panel during the LCD manufacturing process. A CCD or CMOS imaging camera is located behind this hole, and via this means, the LCD screen appears to be unmodified. This also has the benefit of blocking less LCD screen "real-estate" as the hole in the LCD panel need only be small enough to provide enough light for the aperture of the imaging device's lens. The aperture surface area is, of course, smaller than the surface area required to house an entire video camera.
In yet another alternative embodiment, the imaging sensor and the LCD are both produced on the same glass screen substrate, thereby saving the extra step of integrating the imaging sensor and LCD.
In yet another alternative embodiment, thin metal wires or other transparent conductive materials that are invisible to the naked eye are used to transfer power and data to and from the imaging device. These wires can be routed along the LCD screen without being visible to the viewer. In an alternative embodiment, the wires can be routed along a transparent support structure that holds the camera assembly in a fixed location on the active portion of the monitor. In addition to extremely thin wires, conductive materials such as Poly(3,4- ethylenedioxythiophene)/ρoly(styrenesulfonate), which is available under the tradename Baytron® P, may also be used. Carbon nanotubes may also be used to form a transparent conductive electrical conduit.
HG. 3 illustrates the slide mechanism embodiment of the invention. The monitor casing 301 is shown in a landscape orientation, but in an alternative embodiment, the portrait orientation may also be used. The video camera 302, is shown as a small tubular chassis with a cylindrical lens that faces away from the monitor active display surface 303. The video camera 302 is attached to a slide mechanism 304. The slide mechanism 304 slides vertically through a slide mechanism retainer 305. In one embodiment of the invention, the slide mechanism retainer is permanently attached to the monitor casing using fabrication techniques known in the art. In the preferred embodiment, the slide mechanism retainer can be removed from the monitor casing. Various removable means of attachment such as vacuum bonding (for example, suction cups), hook-and-loop fastener, ledge and ledge grabber, or a friction interface may be used to removably attach the slide mechanism retainer. A user can position the video camera 302 either on-screen or off-screen by sliding the slide mechanism 304 up and down within the slide mechanism retainer 305. In the figure, the video camera 302 is shown on-screen, covering the video image of a remote user's forehead 306. Because of the proximity of the video camera 302 to the image of the remote user's eyes 307, the remote user will perceive the local user's gaze as being directed at him to a greater degree than if the camera 302 is located offscreen.
The preferred embodiment of this invention uses digital image stabilization techniques to maintain the video image of the remote user in a constant position on the video screen. This allows the image of the remote user to be stable in its preferred position, where the eyes of the remote user, when displayed on the local video screen, straddle the on-screen video camera. Techniques known in the art, similar to image stabilization, such as face-tracking, and jitter-reduction algorithms may also be used. In all cases, the techniques may be implemented as software executing on a processor, or using dedicated hardware, or a combination of both techniques. The image stabilization may be done at the remote location or at the local location. In the preferred embodiment the image stabilization would be done at the remote location, as this would have the added benefit of reducing the bandwidth of the transmitted video data due a reduction in frame-to-frame movement when using temporal video compression techniques, In the preferred embodiment, a face tracking algorithm known in the art, such as the algorithm presented by RJ. Qian, MJ. Sezan, and K.E. Matthews in their paper, A robust real-time face tracking algorithm, (ISBN: 0-8186- 8821-1, 1998) is used to determine the center of the location of a user's face in a video frame. This location is representing as an (x,y) offset from the upper left corner of the video frame. A preferred camera position on the local users face (typically at the bridge of the nose or slightly higher) is represented as an offset (u,v) from the center location of a user's face. When a video frame is displayed on the remote side, it is translated by (x-u, y-v) relative to the center of the video frame. This maintains the local user's face in a constant position, with the camera located in its preferred location as viewed by the remote user.
Advantages
What has been described is a new and improved video teleconferencing method and apparatus to preserve gaze across a video teleconferencing link by attaching a camera to the active video display surface.
While certain exemplary embodiments have been described in detail and shown in the accompanying drawings, it is to be understood that such embodiments are merely illustrative of and not restrictive on the broad invention, and that this invention is not to be limited to the specific arrangements and constructions shown and described, since various other modifications may occur to those with ordinary skill in the art.

Claims

What is claimed:
1. An apparatus for preserving gaze across a video teleconferencing link consisting of:
A video teleconferencing device consisting of a video screen, video camera, and video teleconferencing processing element, wherein the video camera is mounted on an active portion of the video screen.
2. The apparatus of claim 1, further comprising: a video camera data wire for receiving data from the video camera, the video camera data wire being substantially invisible to the unaided human eye.
3. The apparatus of claim 1, wherein the video camera is mounted on a location of the video screen that is displaying the bridge of the nose of a remote video teleconferencing user.
4. The apparatus of claim 1, wherein image stabilization algorithms are used to stabilize a remote video teleconferencing user's position on the video screen.
5. The apparatus of claim 1, wherein the remote user can maintain eye contact with anyone in the apparatus' space by controlling the location or orientation of the apparatus through the use of electro-mechanical actuators.
6. The apparatus of claim 1, further comprising:
An eye level alignment visual cue presented to the remote user whereby the remote user may position his head such that the image of his eyes on the remote screen are coincident with the camera on the apparatus.
7. An apparatus for preserving gaze across a video teleconferencing link consisting of:
A video teleconferencing device consisting of a video screen, video camera, and video teleconferencing processing element; and
A reflective device mounted on the active portion of the video screen, wherein the reflective device reflects an image that is substantially orthogonal to the video screen surface into the lens of the video camera.
8. The apparatus of claim 7, wherein the reflective device is a mirror.
9. The apparatus of claim 7, wherein the reflective device is a prism.
10. The apparatus of claim 7, wherein the reflective device is mounted on a location of the video screen that is displaying the bridge of the nose of a remote video teleconferencing user.
11. The apparatus of claim 7, wherein image stabilization algorithms are used to stabilize a remote video teleconferencing user's position on the video screen.
12. The apparatus of claim 7, wherein the remote user can maintain eye contact with anyone in the apparatus' space by controlling the location or orientation of the apparatus through the use of electro-mechanical actuators.
13. An image display and capture device for preserving gaze across a video teleconferencing link, consisting of: a video screen, the video screen composed of multiple layer of materials, wherein all of the opaque layers of material include a small hole sufficient to allow visible light frequencies to pass through to the back of the video screen; and a video camera mounted behind the video screen, such that the aperture of the video camera is substantially aligned with the small hole on the video screen, whereby an image in front of the video screen may be detected through the small hole on1 the video screen by the video camera;
14. The apparatus of claim 13, wherein the remote user can maintain eye contact with anyone in the apparatus' space by controlling the location or orientation of the apparatus through the use of electro-mechanical actuators.
15. An video camera apparatus for preserving gaze across a video teleconferencing link consisting of : a lens; a lens holder holding the lens; an imaging sensor integrated circuit capturing light from the lens; a video screen attachment mechanism attaching the video camera to an active display surface; and a translating mechanism permitting movement of the lens to two positions, the first position obscuring a users view of a portion of the video screen, and the second position not obscuring the users view of the video screen.
16. The apparatus of claim 15, wherein the video screen attachment mechanism removably attaches the video camera to an active display surface.
17. The apparatus of claim 15, wherein the remote user can maintain eye contact with anyone in the apparatus' space by controlling the location or orientation of the apparatus through the use of electro-mechanical actuators.
PCT/US2006/044096 2005-11-18 2006-11-14 Screen mounted video teleconferencing camera apparatus WO2007061678A2 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US73824705P 2005-11-18 2005-11-18
US60/738,247 2005-11-18

Publications (2)

Publication Number Publication Date
WO2007061678A2 true WO2007061678A2 (en) 2007-05-31
WO2007061678A3 WO2007061678A3 (en) 2007-11-29

Family

ID=38067725

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2006/044096 WO2007061678A2 (en) 2005-11-18 2006-11-14 Screen mounted video teleconferencing camera apparatus

Country Status (1)

Country Link
WO (1) WO2007061678A2 (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102012216615A1 (en) * 2012-09-18 2014-03-20 LivePoint GmbH Camera system e.g. webcam system, for use with computer system for e.g. online video conference, has camera fixed at display screen system and set within imaging area before image-radiating device of display screen system
WO2013140359A3 (en) * 2012-03-21 2015-08-13 Svip Llc, Series 9 Apparatus and system for imaging in video calls
CN108696688A (en) * 2018-07-13 2018-10-23 维沃移动通信有限公司 A kind of camera, terminal and image pickup method
WO2022123223A1 (en) * 2020-12-07 2022-06-16 Baker-Collins Communications Limited Web camera apparatus
US11388324B2 (en) * 2020-06-15 2022-07-12 Intel Corporation Camera device, base unit, computation device, and corresponding methods and computer programs
DE102021212977A1 (en) 2021-11-18 2023-05-25 Robert Bosch Gesellschaft mit beschränkter Haftung Method for operating an image display device and image display device

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6323892B1 (en) * 1998-08-04 2001-11-27 Olympus Optical Co., Ltd. Display and camera device for videophone and videophone apparatus
US6373516B1 (en) * 1999-11-15 2002-04-16 Ericsson, Inc. Picture position indicator for picture phone
US20040155956A1 (en) * 2003-02-03 2004-08-12 Libbey Kent Alexander System for maintaining eye contract during videoconferencing
US20040174438A1 (en) * 2003-03-07 2004-09-09 Samsung Electronics Co., Ltd. Video communication terminal for displaying user's face at the center of its own display screen and method for controlling the same
US6798457B2 (en) * 2001-09-26 2004-09-28 Digeo, Inc. Camera positioning system and method for eye-to-eye communication
US20040189794A1 (en) * 2003-03-26 2004-09-30 Darwin Rambo Integral eye-path alignment on telephony and computer video devices using a pinhole image sensing device
US20050007445A1 (en) * 2003-07-11 2005-01-13 Foote Jonathan T. Telepresence system and method for video teleconferencing
US6943843B2 (en) * 2001-09-27 2005-09-13 Digeo, Inc. Camera positioning system and method for eye-to eye communication

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6323892B1 (en) * 1998-08-04 2001-11-27 Olympus Optical Co., Ltd. Display and camera device for videophone and videophone apparatus
US6373516B1 (en) * 1999-11-15 2002-04-16 Ericsson, Inc. Picture position indicator for picture phone
US6798457B2 (en) * 2001-09-26 2004-09-28 Digeo, Inc. Camera positioning system and method for eye-to-eye communication
US6943843B2 (en) * 2001-09-27 2005-09-13 Digeo, Inc. Camera positioning system and method for eye-to eye communication
US20040155956A1 (en) * 2003-02-03 2004-08-12 Libbey Kent Alexander System for maintaining eye contract during videoconferencing
US20040174438A1 (en) * 2003-03-07 2004-09-09 Samsung Electronics Co., Ltd. Video communication terminal for displaying user's face at the center of its own display screen and method for controlling the same
US20040189794A1 (en) * 2003-03-26 2004-09-30 Darwin Rambo Integral eye-path alignment on telephony and computer video devices using a pinhole image sensing device
US20050007445A1 (en) * 2003-07-11 2005-01-13 Foote Jonathan T. Telepresence system and method for video teleconferencing

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2013140359A3 (en) * 2012-03-21 2015-08-13 Svip Llc, Series 9 Apparatus and system for imaging in video calls
DE102012216615A1 (en) * 2012-09-18 2014-03-20 LivePoint GmbH Camera system e.g. webcam system, for use with computer system for e.g. online video conference, has camera fixed at display screen system and set within imaging area before image-radiating device of display screen system
CN108696688A (en) * 2018-07-13 2018-10-23 维沃移动通信有限公司 A kind of camera, terminal and image pickup method
US11388324B2 (en) * 2020-06-15 2022-07-12 Intel Corporation Camera device, base unit, computation device, and corresponding methods and computer programs
WO2022123223A1 (en) * 2020-12-07 2022-06-16 Baker-Collins Communications Limited Web camera apparatus
DE102021212977A1 (en) 2021-11-18 2023-05-25 Robert Bosch Gesellschaft mit beschränkter Haftung Method for operating an image display device and image display device

Also Published As

Publication number Publication date
WO2007061678A3 (en) 2007-11-29

Similar Documents

Publication Publication Date Title
US11258930B2 (en) Optical adapters for mobile devices with a camera
TWI223561B (en) Data processing device, data processing system and method for displaying conversation parties
US9154677B2 (en) Camera accessory for angled camera viewing
FI111892B (en) Multifunction messaging device
US20080239061A1 (en) First portable communication device
WO2007061678A2 (en) Screen mounted video teleconferencing camera apparatus
CN110392144A (en) Double-screen intelligent phone and portable device with full display screen
CN110636276B (en) Video shooting method and device, storage medium and electronic equipment
US20040155956A1 (en) System for maintaining eye contract during videoconferencing
CN108667964A (en) Electronic device and its display control method
US20060055773A1 (en) Device and method for stereoscopic reproduction of picture information on a screen
JP2002541743A (en) Interface mainly using mirrors for computer viewing
US20090195645A1 (en) System of displaying and capturing images using non-planar mirrors
US20230240445A1 (en) Method and apparatus for dynamically mounting a camera in front of a display screen
JP3205552B2 (en) 3D image pickup device
JP3480562B2 (en) Image input / output device
CN108696698B (en) Image pickup device, electronic equipment and image acquisition method
TWI288561B (en) Digital camera module with solid-imaging
JP2005323010A (en) Image sharing system
KR200321200Y1 (en) Eye-contact web camera
CN208609081U (en) A kind of filming image and communication device
JP3567249B2 (en) Remote face-to-face communication device capable of combining and transmitting a participant's image and other images
KR200367767Y1 (en) Imaging device
JP2005064681A (en) Image pick-up/display device, image pick-up/display system, video image forming method, program of the method and recording medium having the program recorded thereon
WO2013140359A2 (en) Apparatus and system for imaging in video calls

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application
NENP Non-entry into the national phase in:

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 06837501

Country of ref document: EP

Kind code of ref document: A2