CA2056296C

CA2056296C - Video display system

Info

Publication number: CA2056296C
Application number: CA002056296A
Authority: CA
Inventors: Richard D. Gitlin; John F. Ribera; Clark Woodworth
Original assignee: American Telephone and Telegraph Co Inc
Current assignee: AT&T Corp
Priority date: 1990-12-31
Filing date: 1991-11-27
Publication date: 1995-04-11
Anticipated expiration: 2011-11-27
Also published as: EP0493893A2; EP0493893A3; CA2056296A1; US5159445A; JPH04317287A

Abstract

Direct eye-to-eye contact and compactness are achieved in a video display system including a camera positioned behind, or opposite the viewing side of, a display screen having first and second modes of operation. As a result, the camera and the person or persons viewing the display are on opposite sides of the display screen. The display screen is controlled to switch from the first or image display mode to the second or substantially transparent mode. When the display screen is in the substantially transparent mode, the camera is controlled to record images appearing on the viewing side of the screen.

Description

IMPROVED VIDEO DISPLAY ~Y~

Technical Field This invention relates to displays and, more particularly, to displays adapted for teleconf~e.lcing applic~tion~ and the like wherein images of the calling 5 and called persons viewing the displays are recorded and displayed on the called and calling displays, respectively.

Back~round of the L.v~..li~..
Teleconre~ cillg was introduced clec~cles ago in a simplified form with picture telephollcs wherein bidirectional video and audio links were established10 bel~.~n calling and called parties. With the advent of personal, desktop COlllpuLcl~, teleconferencing has ?~sllme~ a more complex form. Digital images such as text and graphics are displayed on each conrclccs display tçnnin~l while video images of the COl~l~S are also displayed in a portion of the display. The latter display is made possible by po~itioning a camera to one side (top, bottom, left, right) of the display 15 screen for recording images of the particular conrcl~,c viewing the display terminal.
Since the conferee focusses attention on the screen and because the camera is po~ition~ off to one side of the screen, eye contact is lacking between the conferee and the associated camera. This is known as a problem of parallax --- a problem which arises whellevel a camera must view one or more persons watching a display20 screen. Parallax is also a problem in the broadcasting industry where text pl~ Lillg devices are employed.
From the perspective of the camera suffering a parallax problem, the conferee appears to be dozing when the camera is above the screen, gazing to the left or right when the camera is right or left of the screen, or looking at the ceiling when 25 the camera is below the screen. As the conf~'s scrutiny of the screen displaybecolllcs more close, the problem of parallax becomcs more noticeable. Eye contact with the camera establishes eye-to-eye contact with each of the confel~,cs shown on the display screen thereby creating a feeling of interest among the conferees.
Similarly, a lack of direct eye contact with the camera causes a lack of eye-to-eye 30 contact with each of the conferees shown on the display screen which, in turn, creates a ~el-;eption of disint~;,L or preoccupation.

2o56296 While some teleconferencing display terminals continue to be produced with an inherent parallax problem, conventional solutions have been proposed. One such solution involves the combination of a CRT display with a side-mounted camera focused on the conferee through a properly angled beam splitter. Each conferee can concentrate on the display screen while maintaining eye contact with the camera. Although such a display terminal conquers the parallax problem, it cannot be overlooked that the cost of success is quite high.
This display terminal is extremely bulky covering an area several feet square. This is a significant percentage of a standard desk surface. Bulkiness is an inherentproblem caused by the introduction of a beam splitter.

Summary of the Invention Direct eye-to-eye contact and compactness are achieved in a video display system including a camera positioned behind, or opposite the viewing side of, a display screen having first and second modes of operation. As a result, the camera and the person or persons viewing the display are on opposite sides of the display screen. The display screen is controlled to switch from the first or image display mode to the second or substantially transparent mode. When the display screen is in the substantially transparent mode, the camera is controlled to record images appearing on the viewing side of the screen.
Compactness results from the use of a flat panel display such as liquid crystal or plasma. Further size reductions are possible by employing charge-coupled device (CCD) image sensors in the camera.
For display screens requiring backlighting for viewing the displayed image, backlight illumination is provided together with the necessary control toextinguish or sufficiently lower the illumination level when the display screen is in the substantially transparent mode of operation. In another embodiment, a shutter provided between the camera lens and the display screen is controllably switched from a closed or reflective position to an open or transmissive position when the display screen enters the second mode of operation.

,~

- 2a -In accordance with one aspect oE the invention there is provided a display system for providing video images to a viewer and for collecting images of the viewer, the system comprising, display means having first and second controllable modes of operation, the display means having a front and back 5 surfaces, the front surface for viewing the video images, image sensing means for generating a video representation of the viewer image, the image sensing means being located adjacent to the back surface of the display means, and synchronizing means connected to said display means for controlling the display means to switch to the first controllable mode of operation during a first time interval so that the 10 video image is displayed by the display means, said synchronizing means for controlling the display means to switch to the second controllable mode of operation during a second time interval so that viewer images are recorded by the image sensing means through the display means.
In accordance with another aspect of the invention there is provided 15 a display system for providing video images to a viewer and for collecting images of the viewer, the system comprising, display means having first and second modes of operation, the display means having a front and back surfaces, the front surface for viewing the video images, image sensing means for generating a video representation of the viewer image, the image sensing means being located 20 adjacent to the back surface of the display means, and synchronizing means connected to said display means for controlling the display means to switch to the first mode of operation during a first time interval so that the video image is displayed by the display means, said display means automatically reverting to the second mode of operation during a second time interval so that viewer images are25 recorded by the image sensing means through the display means.

Brief Description of the Drawin~
A more complete understanding of the invention may be obtained by reading the following description of specific illustrative embodiments of theinvention in conjunction with the appended drawing in which:

-3~ 2056296 FIG. 1 is a simplifiçcl block diagram of a bidirectional video display system; and FIGs. 2 and 3 show exemplary timing diagrams for the video display system in FIG. 1.

S Detailed Description FIG. 1 shows an exemplary embodilllellt of the video display system realized in accordance with the principles of the invention. The display system comprises display 1, b~cklight~ 2, camera or image sensor 3, shutter 4, display processor 5, control circuit 6, and reflector 12.
Display 1 compri~es a substantially flat panel display element which has a trans~ ,nt mode of operation and an "on" mode of operation. During the "on"
mode of operation, a person can view images depicted on the display through front surface 13. During the transparent mode of operation, display 1 is sufficiently clear to allow light to pass through the display to camera 3 thereby allowing camera 3 to 15 record an image of the person or persons viewing the display.
Exemplary flat panel displays useful for re~li7ing display 1 include liquid crystal displays, plasma displays and ferro-electric displays. In general, the display is arranged as an active matrix of rows and columns in~ rling an active device for providing high contrast (ON/OFF) at each cross point. The cross point or 20 inle.~eclion bet~cen a particular row lead and particular column lead corresponds to a pixel which is activated by energizing the particular row and particular column leads. Active devices co.. nly used at the cross points include thin film transistors ( 1~1), p-i-n diodes, MIM diodes and plasma devices. The latter device is understood by those skilled in the art to be dirrt;lenl from the plasma display 25 m~ntir~ne~l above. Since liquid crystal displays are not light emitting types of devices, the display would normally require some form of backlighting as shown, for example, by b~klight~ 2. It is cnnte...~ ted that display 1 can provide monocl~ le, gray scale, or color display of the im~g~s In an example from e~c~ ,tal practice, display 1 is realized as a TFT active matrix liquid crystal 30 display such as the con~lllGl-;ially available Sharp display model LQ14P01.
Display 1 receives video images for display together with control signals via a set of leads shown simply as lead 7 from display processor 5. In some applications, it may be desirable to permit the display to be written in one part of the frame with a video image and subsequently to decay in the rem~ining portion of the 35 frame to its transpal~llt mode. In other applications, it may be desirable to write an image rapidly on display 1 and then, at a later time within the frame period, switch the display to the transparent mode. Each pixel in display 1 is written or activated once per frame period. For the latter application above, a video image can be accnmlll~ted in a frame buffer (not shown) of display processor 5 so that the video 5 image can be written to the pixels of display 1 within a small fraction of the frame period.
As shown in FIG. 1, camera 3 is mounted behind display 1. It is shown adjacent to, and spaced apart from, back surface 14 of display 1. The ~ t~nce from the camera to the back surface of the display is delGlll~h~ed by the type of image 10 sensor employed and any auxiliary focussing apparatus employed. When display 1 is switched into the transparent mode by a signal on lead 7 from display processor 5, camera 3 receives a start signal on lead 11 from control circuit 6 to begin recording images of the person or persons viewing the display. The video output signal of the viewer images recorded by camera 3 is shown on the lead labeled VIDEO OUT.
In general, camera 3 is mounted in a snfficiently centralized position behind the display to view persons watchillg the display thereby elimin~ting parallax by maint~ining direct eye contact between the camera and the viewers. Camera 3 must have sufficient lux sensitivity to m~int~in acceptable video output of viewers sit~l~ted at a normal viewing distance from front surface 13 for display 1 when 20 lGcorling images in low levels of the available light. The transparency of display 1 and shutter4 can be optimized to provide very little loss of available light.
Exemplary embodiments of the video display system have been realized and op~,laled in the transparent mode with col~ lcially available video cameras such as Sony models XC-711 and XC-007 (low lux rating). Additionally, standard CCD
25 sensor arrays may be employed together with al)pl~liate lens or focussing appa~lus. For a 2/3 inch CCD image sensing array, a 10-lSmm focal length lens appala~us provides adequate viewing quality of the person reading text from the display screen.
Synchr ~ni7~tion b~,lweGn camera 3 and display 1 is m~int~ined by a 30 synchronizing circuit comprising display processor5 and control circuit6. Thesynchronizing circuit provides rapid time multiplexing between the transparent and "on" modes of display 1 to allow viewers to see the video images on display directly while ~lllliuillg the camera to record images of the viewers. The synchronizing circuit receives video images destined for display 1 on a lead labeled VIDEO IN.35 Control circuit 6 develops filnd~mPntal synchronization from an externally supplied signal on the SYNC lead. Control circuit6 utilizes standard circuits to manage -S- 2056~96 operation of the camera and shutter. Display plucessor 5 converts the input video images to a form co...p~l;ble with displayl. Since display processorS controls operation of display 1, it is convellient to have it also control operation of ba ~ ht~ 2.
A synchroni7~tion signal is externally supplied on the SYNC lead from either camera 3 or display processorS or another device external to the display system. This synchronization signal provides a reference for establishing timingintervals based upon either the beginning of a recording period by camera 3 or the beginning of a video image display period by display 1, for example. Coordination 10 be~weell control circuit 6 and display processor 5 is established by signals on lead 9.
This coorflin~tion is needed to insure that (1) the shutter and camera operations are synchronized with the trans~ t mode for the display and (2) the illumin~tion occurs during the image display mode or "on" mode of operation for display 1.
Other aspects of the synchronizing circuit are described below in relation to 15 ~ lition~l ele~ in the video display system.
Duty cycles are defin~, for camera 3, as the ratio between the time period for the transparent mode and the total frame period and, for display 1, as the time period for the "on" mode of display 1 and the total frame period. The duty cycle dele....i-les the relative light levels received by camera3 and by a person 20 viewing the display. In order to achieve acceptable contrast and brightness of displayed video images for a viewer, display 1 is ,..~inl~ined in the "on" mode for a sufficiently long period of time. Similarly, the display is be m~in~inecl in thetransparent mode for a snffici~nt period of time to achieve proper light levels for the camera. Although a duty cycle of 50% is used in the examples described herein, it is 25 contemplated that duty cycles other than 50% can be utili7~d The frequency ofmode switching for the display is dete....ined by the display technology and theframe rate of the display. It is i~ t to note that mode switching is generally synchronized with the display frame, camera operation depending on the persistence of the image sensor, oper~tion of b~r~ hts~ and operation of the shutter. Generally, 30 the frame rate for the display is chosen to be 1/30 second and, in a 50% duty cycle enviho~ ,ent, the display is in the "on" mode for 1/15 second and in the transparent mode for 1/15 second. The frame rate is defined above for non-interlaced frames on display 1. Where intPrl~ced frames are desired, the rate of interest is the field rate whGlcill an odd and an even field comprise the frame.

Shutter 4 is an optional element shown interposed between the image sensor (camera 3) and back surface 14 of display 1. Shutter 4, when employed, isuseful for obscuring display images which could reach the image sensor in the "on"
mode and for obscuring the camera lens ape.~ulG in reflector 12. In the latter case, 5 this provides a ur,irolm b~ckli~hting surface becallse, in the absence of a shutter, an image of the open a~el~ul~ could reach the viewer when display 1 is in the "on"
mode. Shutter 4 is controlled to open and close by a signal on lead 10 from control circuit 6 of the synchronizing circuit. Shutter 4 remains open at least for the time period that camera 3 is recording images of the viewer through transparent display 1.
10 Shutter 4 closes in l.,i,ponse to a signal on lead 10 coincident with display 1 entering the "on" mode, thereby providing a uniroll~ backlighting surface with reflector 12.
Typical shutters for use as shutter4 include mech~ni~l, optical and electrooptic effect devices. Mechanical shutters which permit electronic controlprovide sllfflciPnt bloc_ing of the video images from the display to the image sensor 15 but may lack the necess~ speed of response in various applic~tion~. Optical and electrooptic effect devices suitable for use as shutter 4 include liquid crystal devices acting as polarizers and Kerr effect devices. These devices respond to an applied signal to switch to an optically non-tr~n~mi~ive state wLereas the devices may be normally tr~n~mi~sive or tr~n~p~rent. The speed of response for each of these types 20 of devices is well suited to the video display system described herein. Both latter devices are well known to those skilled in the art.
While the shutter is simple to implement, it may be advantageous to avoid the use of a shutter and run the image sensor at a speed twice the frame rate, that is, 1/15 second, where an exemplary standard frame rate is 1/30 second. In this 25 type of operation, the image sensor gcnc.iltes a video output signal having the viewer images are interleaved with display im~ges. By selectively writing only the viewer images into an output frame buffer (not shown), it is possible to read the images out from the output frame buffer at the standard frame rate. CCD image sensing arrays are collllll~ ;ially available and can be combined with applopliate well known 30 circuits to p~,lrullll the higher speed image sensing. Output frame buffering with dirrel1,lt writing and reading speeds is also well known in the art.
R~ckli~ht~ 2 and reflective enclosure 12 provide ilhlmin~tinn of display 1 so that video images on the display can be viewed with ap~lupliate brightness levels and contrast. B~cklight~ 2 are optional elements. Moreover, it is 35 contemplated that the backlights may be kept on during operation of the videodisplay system without intclreling with the operation of the image sensor for -20~6296 camera 3. Operation of backlights 2 is controlled to be either on or switched on and off by a signal on lead 8 from display processor 5 of the synchronizing circuit. For switched operation, cold cathode lamps are suitable for use as bacl~light~ 2 located generally about the pelil)he~r of the display. Reflective enclosure 12 reflects the 5 majority of light back toward the back surface of the display. The reflective enclosure incl~ldes a small ape,Lul~, through which the camera and optional shutter view the display and viewers.
When backlight ill~lmin~ti~n of the display is employed, it may be desirable to reduce reflections from the back surface of display 1 toward camera 3.
10 To this end, a standard antirçflection coating or antireflection film can be applied to back surface 14.
FIG. 2 shows a simplified timing diagram for exemplary operation of the video display system shown in FIG. 1. Operation of only the display 1, imagesensor or camera 3 and shutter 4 are concidered in FIG. 2. B~cklight~ 2 are assumed 15 to be on. It should be noted that the displays are not interlaced in this example. The video image for frame i is sent to display 1 beginning at time to. The final pixel of the video image is written to the display at time t 1 . Each pixel is allowed to decay during the time interval riesign~t~d frame i+l from time tl to time t3. Since the camera is generally positioned along a central axis substantially normal to and 20 centered on the display, it is necess~ry for the center portion of the display to become transparent before the camera begins operation. This occurs at time t2 when the camera receives a start signal. Recording of the viewer image continues until time t4, at which time the center pixels are again activated with another video display image for frame i+2. From time t2 until time t4, the shutter remains open.
25 At time t3, the next video display image is written to display 1. A similar sequence of events occurs in each successivc frame period.
In this example, the video frame rate from the camera is twice the frame rate to the display because the images sensed by the camera are output while theshutter is open and while the shutter is closed. It is expected that this technique may 30 produce unacceplable levels of flickering on the display.
For displays where the decay time to a transparent mode for individual pixels is one-half of a frame period or less, the display can be operated at a normal frame rate. That is, display 1 can display every video frame i, frame i+l, frame i+2, etc. In order to accomplish this, it is necessaly to write the video image to the 35 display from a device such as frame buffer. This permits the display to begindisplaying the image almost instantaneously at the beginning of the frame period.

By the middle of the frame period, the pixels begin to revert to a transparent mode.
Recording of the viewer image commences three-quarters of the way through a frame period and continues through the first quarter of the next frame period.
FIG. 3 shows a timing diagram for exemplary operation of the system in 5 FIG. 1 wherein display 1 has the capability for being cleared to a transparent mode or whe~cill display 1 has the capability to for individual pixels to decay rapidly into the transparent mode. The displays described above can have integral memory or separate frame buffers. In this example, the video display image is written to display 1 in the time period to to tl. From time tl to t2, the display is actively 10 displaying the image and the back illnmin~tjon is also on. Also at t3, the min~tinn is turned off. From time t2 to t3, the display is allowed to decay to a~alls~J~e. t mode. Alt~rn~tively, the display receives a signal from the synchroni7~tion circuit c~ in~ all pixels to switch to a tran~ ellt mode until frame i+l is written to the display at time t4. Once the display is in the transparent mode 15 at time t3, the camera is activated by the start signal and the shutter is opened to allow recording of the viewer im~s A similar sequence of events occurs in each s-lccessive frame period. By IllAin~ g the writing period (to to tl) and the clç~ring period (t2 to t3) as small as possible, light loss is minimi7~1 For the example shown in FIG. 3, ferroelectric displays are attractive 20 because they can be written rapidly and they retain the video image after writing.
Switching this type of display to the transparent state is accomplished by rewriting the entire display or by cl~ring the display with the proper control signal.

Claims

1. A display system for providing video images to a viewer and for collecting images of the viewer, the system comprising, display means having first and second controllable modes of operation, the display means having a front and back surfaces, the front surface for viewing the video images, image sensing means for generating a video represention of the viewer image, the image sensing means being located adjacent to the back surface of thedisplay means, and synchronizing means connected to said display means for controlling the display means to switch to the first controllable mode of operation during a first time interval so that the video image is displayed by the display means, said synchronizing means for controlling the display means to switch to the second controllable mode of operation during a second time interval so that viewer images are recorded by the image sensing means through the display means.

2. The display system as defined in claim 1 wherein the display means comprises a substantially flat panel display.

3. The display system as defined in claim 2 wherein the substantially flat panel display is of the type selected from the group consisting of liquid crystal, plasma, and ferro-electric displays.

4. The display system as defined in claim 1 wherein the image sensor means includes a video camera.

5. The display system as defined in claim 1 wherein the image sensor means includes a charge-coupled device array.

6. The display system as defined in claim 1 wherein the display system further includes a frame buffer for storing a next video image the frame buffer responsive to the synchronizing means for transferring the stored next video image to the display means substantially at the beginning of the first time interval.

7. The display system as defined in claim 1 further comprising, shutter means interposed between the image sensing means and the back surface of the display means and having first and second modes of operation, the first mode of operation for scattering light impinging thereon and the second mode of operation for transmitting light therethrough so that images are transmitted to the image sensing means, and the synchronizing means connected to the shutter means for controlling the shutter means to switch to the first mode of operation during the first timeinterval, and the synchronizing means for controlling the shutter means to switch to the second mode of operation during the second time interval.

8. The display system as defined in claim 1 further comprising, illumination means positioned adjacent to the back surface of the display means for backlighting the display means.

9. The display system as defined in claim 8 wherein the illumination means is switchably controllable to an "ON" state and to and "OFF" state, and the synchronizing means is connected to the illumination means for controlling the illumination means to switch to the "ON" state during substantially the first time interval, the synchronizing means for controlling the illumination means to switch to the "OFF" state during substantially the second time interval.

10. The display system as defined in claim 7 further comprising, illumination means positioned adjacent to the back surface of the display means for backlighting the display means.

11. The display system as defined in claim 10 wherein the illumination means is switchably controllable to an "ON" state and to and "OFF" state, and the synchronizing means is connected to the illumination means for controlling the illumination means to switch to the "ON" state during substantially the first time interval, the synchronizing means for controlling the illumination means to switch to the "OFF" state during substantially the second time interval.

12. The display system as defined in claim 7 wherein the shutter means includes an electrooptic device.

13. The display system as defined in claim 12 wherein the electrooptic device is a Kerr cell.

14. The display device as defined in claim 7 wherein the shutter means includes a liquid crystal cell.

15. A display system for providing video images to a viewer and for collecting images of the viewer, the system comprising, display means having first and second modes of operation, the display means having a front and back surfaces, the front surface for viewing the video images, image sensing means for generating a video representation of the viewer image, the image sensing means being located adjacent to the back surface of thedisplay means, and synchronizing means connected to said display means for controlling the display means to switch to the first mode of operation during a first time interval so that the video image is displayed by the display means, said display means automatically reverting to the second mode of operation during a second time interval so that viewer images are recorded by the image sensing means through the display means.

16. The display system as defined in claim 15 wherein the display means comprises a substantially flat panel display.

17. The display system as defined in claim 16 wherein the substantially flat panel display is of the type selected from the group consisting of liquid crystal, plasma, and ferro-electric displays.

18. The display system as defined in claim 15 wherein the image sensor means includes a video camera.

19. The display system as defined in claim 15 wherein the image sensor means includes a charge-coupled device array.

20. The display system as defined in claim 15 wherein the display system further includes a frame buffer for storing a next video image the frame buffer responsive to the synchronizing means for transferring the stored next video image to the display means substantially at the beginning of the first time interval.

21. The display system as defined in claim 15 further comprising, shutter means interposed between the image sensing means and the back surface of the display means and having first and second modes of operation, the first mode of operation for scattering light impinging thereon and the second mode of operation for transmitting light therethrough so that images are transmitted to the image sensing means, and the synchronizing means connected to the shutter means for controlling the shutter means to switch to the first mode of operation during the first timeinterval, and the synchronizing means means for controlling the shutter means to switch to the second mode of operation during the second time interval.

22. The display system as defined in claim 21 wherein the shutter means includes an electrooptic device.

23. The display system as defined in claim 22 wherein the electrooptic device is a Kerr cell.

24. The display device as defined in claim 21 wherein the shutter means includes a liquid crystal cell.

25. The display system as defined in claim 15 further comprising, illumination means positioned adjacent to the back surface of the display means for backlighting the display means.

26. The display system as defined in claim 25 wherein the illumination means is switchably controllable to an "ON" state and to and "OFF" state, and the synchronizing means is connected to the illumination means for controlling the illumination means to switch to the "ON" state during substantially the first time interval, the synchronizing means for controlling the illumination means to switch to the "OFF" state during substantially the second time interval.

27. The display system as defined in claim 21 further comprising, illumination means positioned adjacent to the back surface of the display means for backlighting the display means.

28. The display system as defined in claim 27 wherein the illumination means is switchably controllable to an "ON" state and to and "OFF" state, and the synchronizing means is connected to the illumination means for controlling the illumination means to switch to the "ON" state during substantially the first time interval, the synchronizing means for controlling the illumination means to switch to the "OFF" state during substantially the second time interval.