US20050104802A1 - Infrared audio/video interface for head-mounted display - Google Patents
Infrared audio/video interface for head-mounted display Download PDFInfo
- Publication number
- US20050104802A1 US20050104802A1 US09/756,648 US75664801A US2005104802A1 US 20050104802 A1 US20050104802 A1 US 20050104802A1 US 75664801 A US75664801 A US 75664801A US 2005104802 A1 US2005104802 A1 US 2005104802A1
- Authority
- US
- United States
- Prior art keywords
- video
- collecting lens
- receiver
- video signal
- signal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000003252 repetitive effect Effects 0.000 claims abstract description 4
- 230000005236 sound signal Effects 0.000 claims description 27
- 239000006185 dispersion Substances 0.000 claims description 25
- 230000003287 optical effect Effects 0.000 claims description 22
- 238000000034 method Methods 0.000 claims description 18
- 238000012545 processing Methods 0.000 claims description 17
- 230000005672 electromagnetic field Effects 0.000 claims description 8
- 238000005286 illumination Methods 0.000 claims description 7
- 239000000463 material Substances 0.000 claims description 7
- 230000005540 biological transmission Effects 0.000 abstract description 13
- 238000013461 design Methods 0.000 abstract description 6
- 230000000712 assembly Effects 0.000 abstract 1
- 238000000429 assembly Methods 0.000 abstract 1
- 238000010586 diagram Methods 0.000 description 22
- 230000006870 function Effects 0.000 description 15
- 230000005855 radiation Effects 0.000 description 14
- 239000007787 solid Substances 0.000 description 8
- 210000003128 head Anatomy 0.000 description 7
- 238000006243 chemical reaction Methods 0.000 description 6
- 238000003384 imaging method Methods 0.000 description 6
- 239000004417 polycarbonate Substances 0.000 description 6
- 229920000515 polycarbonate Polymers 0.000 description 6
- 210000001747 pupil Anatomy 0.000 description 6
- 239000000872 buffer Substances 0.000 description 5
- 238000012937 correction Methods 0.000 description 5
- 230000008901 benefit Effects 0.000 description 4
- 239000002131 composite material Substances 0.000 description 4
- 230000009977 dual effect Effects 0.000 description 4
- 230000004438 eyesight Effects 0.000 description 4
- 230000005043 peripheral vision Effects 0.000 description 4
- 230000035945 sensitivity Effects 0.000 description 4
- 229920004738 ULTEM® Polymers 0.000 description 3
- 238000001514 detection method Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 239000013307 optical fiber Substances 0.000 description 3
- 230000004044 response Effects 0.000 description 3
- 238000000926 separation method Methods 0.000 description 3
- 238000012546 transfer Methods 0.000 description 3
- OAICVXFJPJFONN-UHFFFAOYSA-N Phosphorus Chemical compound [P] OAICVXFJPJFONN-UHFFFAOYSA-N 0.000 description 2
- XUIMIQQOPSSXEZ-UHFFFAOYSA-N Silicon Chemical compound [Si] XUIMIQQOPSSXEZ-UHFFFAOYSA-N 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000010168 coupling process Methods 0.000 description 2
- 238000005859 coupling reaction Methods 0.000 description 2
- 230000003247 decreasing effect Effects 0.000 description 2
- 230000003111 delayed effect Effects 0.000 description 2
- 238000001839 endoscopy Methods 0.000 description 2
- 238000002329 infrared spectrum Methods 0.000 description 2
- 238000003780 insertion Methods 0.000 description 2
- 230000037431 insertion Effects 0.000 description 2
- 230000013011 mating Effects 0.000 description 2
- 230000008447 perception Effects 0.000 description 2
- 230000002688 persistence Effects 0.000 description 2
- 229920003229 poly(methyl methacrylate) Polymers 0.000 description 2
- 239000004926 polymethyl methacrylate Substances 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 229910052710 silicon Inorganic materials 0.000 description 2
- 239000010703 silicon Substances 0.000 description 2
- 230000004308 accommodation Effects 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 230000002411 adverse Effects 0.000 description 1
- XAGFODPZIPBFFR-UHFFFAOYSA-N aluminium Chemical compound [Al] XAGFODPZIPBFFR-UHFFFAOYSA-N 0.000 description 1
- 229910052782 aluminium Inorganic materials 0.000 description 1
- 230000033228 biological regulation Effects 0.000 description 1
- 230000000747 cardiac effect Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 230000001143 conditioned effect Effects 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 210000004351 coronary vessel Anatomy 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000005670 electromagnetic radiation Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 231100000040 eye damage Toxicity 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 230000000670 limiting effect Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000002324 minimally invasive surgery Methods 0.000 description 1
- 230000005693 optoelectronics Effects 0.000 description 1
- 230000000399 orthopedic effect Effects 0.000 description 1
- 239000004033 plastic Substances 0.000 description 1
- 229920003023 plastic Polymers 0.000 description 1
- 230000002829 reductive effect Effects 0.000 description 1
- 230000001105 regulatory effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
- 238000001356 surgical procedure Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
- G02B27/0172—Head mounted characterised by optical features
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
- A61B90/36—Image-producing devices or illumination devices not otherwise provided for
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
- G02B2027/0178—Eyeglass type
Definitions
- This invention relates generally to image display systems and more particularly to infrared video and audio interfaces for head-mounted displays, principally eyeglass-mounted displays.
- High quality medical remote imaging has gained increasing importance. This is particularly true of imaging during surgical procedures, most importantly minimally invasive procedures in which direct viewing of the surgical field is difficult.
- a method for performing coronary artery bypass relies on viewing the cardiac region through a thoracoscope or other viewing scope (see for example Sterman et al. U.S. Pat. No. 5,452,733 and Gifford, III et al. U.S. Pat. No. 5,695,504).
- a surgeon may perform a delicate vascular- or neuro-microsurgical reconstruction through a minimal incision under remote viewing.
- Remote imaging is now common in orthopedics, ophthalmology, urology, gynecology, anesthesiology, and other medical specifications.
- remote imaging is accomplished by attaching a video camera to an endoscope or other minimally invasive instrument and transmitting the video image via cable to a conventional CRT video monitor.
- This is often cumbersome in a crowded, brightly lighted operating room, where surgical team members are frequently moving around and the surgeon's view of the image screen is obstructed.
- the CRT monitor is incapable of providing the surgeon with critical depth perception, since it is not stereographic.
- Head-mounted displays potentially offer a method to overcome viewing obstructions typical in a surgical environment. While head-mounted displays have been designed, developed and deployed in military applications for many years, such displays are generally bulky, expensive, application-specific devices poorly suited to commercial or surgical applications. Additionally, users of head-mounted displays are frequently restricted in their range of motion by cumbersome interface cabling.
- a compact HMD system requires a very small display device, such as those found in modern camcorder viewfinders, but with significantly higher resolution.
- a number of such devices are now becoming available, including transmissive and reflective liquid-crystal microdisplay devices and micro-mirror devices having resolutions at or in excess of VGA quality (640 pixels by 480 pixels) with pixel sizes on the order of 15 microns or less.
- VGA quality 640 pixels by 480 pixels
- a medical stereographic HMD system having dual display devices is described in Heacock et al. “Viewing Ocular Tissues with A Stereoscopic Endoscope Coupled to a Head Mounted Display (HMD),” http://www.hitl.washington.edu/publications/heacock/, Feb. 17, 1998. Kaiser Electro-Optics (2752 Loker Avenue West, Carlsbad, Calif. 92008 manufactures the “CardioView,” “Series 8000 ,” and “StereoSite” HMD display systems for Vista Medical Technologies. These systems are bulky, heavy, and expensive, and require two LCD display devices. For peripheral vision correction they require the user to wear the HMD over conventional corrective eyeglasses, aggravating user inconvenience and discomfort. Meyerhofer et al. U.S. Pat. No. 5,619,373, issued Apr. 8, 1997, describes a single display device involving beamsplitters for non-stereographic, biocular viewing.
- the scan formats of video source devices are not directly compatible with typical solid state display devices.
- frame rates conforming with NTSC or PAL standards are too slow, and produce undesirable perceived flicker in solid state displays, which do not have the luminous persistence of phosphor screen displays, for example conventional TV displays. Therefore scan format and frame rate conversion are needed.
- a compact, high resolution, high contrast microdisplay system particularly for surgical viewing, that is suitable for head-mounted display use without requiring undue complexity or expense and that preferably supports biocular and/or truly stereographic viewing.
- the system should incorporate format and frame rate conversion to provide compatibility between solid state display devices and conventional video input sources.
- the system should provide good color fidelity and should incorporate ergonomic design for comfort and efficiency, including peripheral vision accommodation and minimal cabling.
- Apparatus includes a video or audio/video interface linking a base station with a remote video display.
- Video processing circuitry at the base station converts an input color video signal conforming to NTSC (525 lines) or PAL (625 lines) formats from a conventional source, e.g., a video camera, into a modulated video signal having a format appropriate to drive a solid state video display, e.g., a sequential color LCD display.
- the modulated video signal has a data structure containing a repetitive sequence of uniform frame times. Each frame time consists of substantially equal consecutive field times for each of three component color fields. Image information for each color field is encoded as a burst of pixel luminance data occupying roughly one-half of each field time.
- the data structure also typically contains embedded scan and frame control signals and an embedded pulse-amplitude audio channel.
- a frame rate of the order of 80 frames per second, or 240 color fields per second is required. This in turn requires a field time of roughly 4 msec.
- To handle this video data throughput typically requires a bandwidth of at least 100 MHz.
- a few such displays can operate at 60 frames per second without objectional flicker. These displays can then utilize roughly 5.3 msec. per field with at least 75 MHz. bandwidth.
- the modulated video signal is transmitted to a remote receiver, located adjacent to a remote video display.
- the remote video display can be a mobile display, for example mounted in a headset worn by a user.
- a remote circuit interconnected between the receiver and the display demodulates the modulated video signal and controls the image data loading and color illumination of the display device.
- the display device is loaded with pixel luminance data for a single color field.
- the display device is illuminated by a color LED corresponding to the color field just loaded. The process is repeated sequentially for the three color fields of the image frame, such that bursts of pixel luminance data alternate synchronously with illumination by an appropriate LED.
- two separate video display devices can be driven in parallel for a simplified binocular optical configuration.
- two separate video display devices can be driven alternately. Each display is illuminated while the other display is loaded with video data.
- the two alternating burst datastreams are derived from a single time-multiplexed modulated video signal.
- the video interface can transmit the modulated video signal through a conductive coaxial cable from the base station to the remote receiver, cables are cumbersome for mobile receivers, for example head-mounted displays, where they restrict the motion of a user.
- the video interface can transmit the modulated video signal on a modulated beam of infrared or other electromagnetic energy.
- an infrared (IR) wavelength can be selected outside the visible region in the range of approximately 700 nm to approximately 1100 nm, where good conventional photodetectors, LEDs, and laser diodes are available.
- the IR beam can be guided through an optical fiber connecting the base station with the remote receiver, but this method has the same drawbacks as the coaxial cable.
- the IR beam is generated by an array of IR LEDs and transmitted through a free atmospheric path. All the LEDs in the array emit an identical optical signal.
- the IR beam is projected onto a diffuse reflecting surface (for example the ceiling or an efficient scattering panel adjacent to the ceiling) by an array of conventional infrared LEDs with integral collimating lenses (typically for a ⁇ 10 degree radiation pattern from each LED) connected to the base station.
- the lensed LEDs typically create a 2 ft diameter on a scattering surface 6 feet away.
- the scattering surface is positioned and oriented such that the scattered IR signal is concentrated in a volume including the probable location of a remote receiver.
- the receiver includes a collecting lens assembly consisting of three coaxial components: a refractive lens with a conic sidewall and planar exit pupil, an inner cone with a diffusely reflective sidewall mating to the lens' conic sidewall, and an outer smooth reflective cone extended above the lens.
- the photodetector is optically cemented to the lens' planar exit pupil to couple some energy into the photodetector that would otherwise be lost to total internal reflection within the lens. This combination of elements dramatically enhances the photodetector's effective aperture over a wide angular field of view, thereby permitting the use of a smaller, wider bandwidth, less expensive photodetector to collect sufficient levels of infrared signal.
- An optional prismatic dispersion plate overlying the outer conic cavity widens the collecting angle of the assembly.
- An optional optical bandpass filter is positioned over the assembly.
- one or more optical elements of the collecting lens assembly are aspheric.
- Transmissive optical elements are typically made from transparent polymeric material, for example polymethyl methacrylate, polycarbonate, and ULTEM® grade polycarbonate manufactured by General Electric Company.
- the interface includes a return audio link that provides return audio communication from the receiver location to the base station.
- the return audio can modulate a LED, which emits an audio modulated IR signal through the atmosphere.
- the audio modulated IR signal is then detected by a separate receiver connected to the base station, where the audio data is processed and restored.
- the audio receiver has the same three-component coaxial structure as the remote receiver.
- FIG. 1A is a schematic view of a surgical environment including an infrared video interface for a head-mounted display, in accordance with the present invention
- FIG. 1B is a schematic view of an alternative surgical environment to that of FIG. 1A ;
- FIG. 1C is a graphic representation of a typical burst mode data structure for a solid state video display, in accordance with an embodiment of the present invention
- FIG. 1D is graphic representation of a typical horizontal line within the data burst structure of FIG. 1C , illustrated on an expanded time scale;
- FIG. 2A is a functional block diagram of a video processing circuit located, for example, in a base station;
- FIG. 2B is a functional block diagram illustrating the operation of a scan rate converter
- FIG. 2C is a block diagram of a transceiver module
- FIGS. 2D and 2E are more detailed transceiver schematic block diagram, in accordance with further embodiments of the present invention.
- FIG. 3A is a schematic front view of an IR module incorporated in a headset, containing components of the IR video interface;
- FIG. 3B is a top schematic view of an embodiment of a headset including an IR module
- FIG. 3C is a side schematic view of a user wearing an embodiment of a headset including an IR module
- FIG. 3D is a functional block diagram of a remote electronics circuit, located for example at a headset;
- FIG. 3E is a detailed functional block diagram of a headset timing generator
- FIG. 4 is a schematic diagram of an array of IR LEDs
- FIG. 5A is a cross-sectional schematic diagram of a collecting lens assembly, in accordance with an embodiment of the present invention.
- FIGS. 5B-5D are cross-sectional schematic diagrams illustrating the transmission and capture of IR radiation incident from various angles onto a collecting lens assembly
- FIGS. 5E and 5F are cross-sectional schematic diagrams of a collecting lens assembly with a prismatic dispersion plate
- FIG. 5G is a plane view of a prismatic dispersion plate
- FIG. 5H is a cross-sectional schematic diagram illustrating the transmission and capture of IR radiation incident from various angles onto a prismatic dispersion plate.
- FIG. 5I is a graphic representation of the calculated radiative capture by various combinations of elements of the assembly of FIG. 5A , relative to the radiative capture by an unaided photodetector.
- FIG. 1A is a schematic view of a surgical environment including an infrared video interface 100 for a head-mounted display, in accordance with the present invention.
- a user 102 e.g., a surgeon or assistant, wears a headset 104 , containing a remote video display device 140 and a remote electronic circuit 142 , including ancillary optical, audio, and electronic apparatus, described in more detail below.
- all of the receiving, processing, audio, and display functions relating to the head-mounted display are performed within headset 104 .
- some of these functions are performed within an optional utility module 105 a attached, for example, to the clothing or belt of user 102 , and connected to headset 104 by a utility cable 105 b .
- Batteries 144 configured to power the respective head-mounted display functions can be mounted at headset 104 or optionally at utility module 105 a .
- a remote mobile video bandwidth receiver 146 located, e.g., at headset 104 receives a diffusely reflected infrared signal 118 carrying video and/or audio data on a modulated beam of electromagnetic energy.
- the lensed LEDs typically create a 2 ft diameter on a diffusely reflective scattering surface 6 feet away.
- transceiver module 110 is integral with base station 112 .
- the LED array 130 is significantly less expensive than a single laser diode of equivalent power and offers a significant reliability advantage over the laser diode.
- the spatially distributed nature of the LED array 130 avoids the high power density of the laser diode and minimizes the possibility of total blockage of the signal path to the headset.
- the lower power density of the LED array 130 avoids the design considerations and concerns related to potential eye damage and associated regulatory controls, such as FDA regulations.
- lensed LED array 108 project modulated IR signal 106 through the atmosphere onto a diffusely reflective target area of the ceiling 116 or (not shown) mounted adjacent ceiling 116 .
- Infrared signal 106 is scattered through the atmosphere from the diffuse target area as diffusely reflected IR signal 118 , a portion of which illuminates headset 104 .
- the diffuse target area e.g., ceiling 116
- the diffuse target area has a lenticular or other well known surface structure, providing a directionally preferred scattering pattern of scattered infrared signal 118 .
- headset 104 provides a return audio signal back to base station 112 .
- a return IR fan 120 carries the modulated audio signal at least in part through an atmospheric transmission path, generally retracing the transmission paths of diffusely reflected IR signal 118 and projected infrared signal 106 .
- FIG. 1A Although a surgical environment is depicted in FIG. 1A , in other embodiments similar configurations including an infrared video interface 100 for a head-mounted display, in accordance with the present invention, are applied to a variety of environments.
- FIG. 1B is a schematic view of an alternative surgical environment to that of FIG. 1A .
- Transceiver module 110 is attached above the working space of users 102 , e.g., suspended from ceiling 116 or other elevated support, and is connected with base station 112 by coaxial cable 182 .
- An array 130 of IR LEDs is mounted in transceiver module 110 and is configured to direct an IR beam 132 within the volume including the probable location of respective users 102 .
- infrared video interface 100 derive from the requirements of head-mounted video display 140 .
- these requirements are described in Hebert, U.S. patent application Ser. No. 09/056,934, cited above.
- headset 104 does not require a frame memory.
- the display device is sequentially illuminated with red, green, and blue light sources, for example LEDs.
- red, green, and blue light sources for example LEDs.
- a user's eye merges the sequence of colors and creates a perception of full color.
- infrared video interface 100 carries each sequential field of red, green, or blue video information as a burst of data. The sequence is repeated at a data rate, such that full motion and full color images are simulated.
- a solid state display is preferably illuminated at approximately an 80 Hz frame rate (a 240 Hz field rate representing a three-color sequence for each frame) to minimize the amount of flicker perceived by the eye. This is equivalent to one color field each 4.16 msec.
- the NTSC (National Television Standards Committee) video format provides a 60 Hz frame rate
- PAL (phase Alternating Line) video format provides a 50 Hz frame rate. Both of these frame rates are too slow to prevent perceived flicker in the solid state display.
- conventional video displays e.g. TV screens, are more tolerant of the slower frame rates. Therefore, frame rate conversion is performed in base station 112 .
- the method adopted to increase the effective frame rate is cyclical repetition of one or two of the sequential red, green, or blue fields.
- the specific scheme applied depends on whether the input source conforms with an NTSC or PAL format.
- conventional video standards and sources see for example K. Jack, “Video Demystified, A Handbook for the Digital Engineer,” Second Edition, 1996, published by HighText Interactive, Inc., San Diego.
- RGB indicates a new input three-color field sequence (red, green, blue)
- rgb indicates repeated color fields (red, green, blue) stored in the memory of base station 112
- the following sequence can be generated: RGBrGBRgBRGb RGBrGBRgBRGb.
- the insertion of one repeated output color field for each consecutive set of three input color fields increases the perceived frame rate and thereby reduces perceived flicker.
- the repeated color selection and the consecutive input color field set are rotated cyclically, thereby preserving the original color sequence and retaining color fidelity.
- RGB indicates a new input three-color field sequence (red, green, blue)
- rgb indicates repeated color fields (red, green, blue) stored in the memory of base station 112 .
- the insertion of two repeated output color fields for each consecutive set of three input color fields increases the perceived frame rate and thereby reduces perceived flicker.
- the repeated color selections and the consecutive input color field set are rotated cyclically, thereby preserving the original color sequence and retaining color fidelity.
- the total available field time (either 4.16 msec for NTSC or 4.0 msec for PAL) is partitioned into several phases, including: (1) erase display, (2) load display, and (3) illuminate display with an appropriate red, green, or blue LED.
- the display loading time is selected to be approximately 2 msec for each color field of information.
- each of the 800 ⁇ 600 pixels in the display is loaded with luminance data, namely, some gradation between black and white.
- infrared video interface 100 transmits the equivalent of 240 million pixels per second, roughly a pixel every 4 nsec.
- the IR interface data transport scheme can be described as “burst mode amplitude modulation”. Amplitude modulation is preferred to simplify the receiver design.
- the encoding scheme can alternatively utilize pulse modulation without changing the fundamental structure of IR video interface 100 . However, receiver circuits would need to convert the pulse modulation back into amplitude modulation to be compatible with a display of the present embodiment.
- FIG. 1C is a graphic representation of a typical burst mode data structure for a solid state video display, in accordance with an embodiment of the present invention.
- signal amplitude 136 is shown as a function of time 138 .
- Sequential input color fields are labeled R, G, and B, followed by a repeated color field stored in the memory of base station 112 labeled r.
- a repetitive sequence of horizontal sync pulses 152 provides basic synchronization for the burst mode structure.
- An individual color field 137 has a time duration t 137 , or approximately 4 msec.
- the pixel luminance data 139 within each color field is transmitted as a data burst and has a time duration t 139 , or approximately 2 msec.
- FIG. 1D is graphic representation of a typical horizontal line within data burst 139 of FIG. 1C , illustrated on an expanded time scale.
- a typical horizontal sync pulse 152 has a substantially rectangular waveform with an amplitude that defines a grey scale between black 154 and white 156 .
- Between consecutive horizontal sync pulses 152 is one horizontal line of analog pixel data 158 , e.g. 800 pixels to a horizontal line.
- the full width t 152 of a horizontal sync pulse 152 provides a clock basis for data burst timing and has a duration typically equal to that of 32 pixels, which corresponds to approximately 125 nsec.
- the time duration t 158 between consecutive horizontal sync pulses is approximately 3.25 ⁇ sec in the example shown.
- IR video interface 100 is adaptable for driving one or two video/audio channels.
- half of each color field time is used for data transfer, and the other half is used for illumination.
- one half of each color field time is used for data transfer to a first channel, and the other half of the time is used for data transfer to the second channel.
- First and second channels' displays are alternately illuminated, such that the illumination occurs for one display while data is transferred to the other display, as described below in more detail.
- Two data channels can be used for presentation of true stereo imagery or, alternatively, a different view (i.e., inverted view or different video source) sent to different observers.
- IR video interface 100 If IR video interface 100 is required to transmit data for two display channels, then each channel can be alternately loaded and illuminated. Thus the interface carries burst mode image data for a first channel during 2 msec, while a second channel is illuminated. Likewise, during the next approximately 2 msec interval, the first channel is illuminated while the interface transmits image data to the second channel.
- This scheme can be described as time division multiplexing (or time-duplexing) with burst mode modulation.
- bandwidth manifests itself as the ability to resolve fine spatial details, such as the sharp edges of objects. More specifically, the measured bandwidth of a high quality endoscopy video camera is approximately 5.33 MHz for each individual RGB channel.
- An example of such a video camera is the Telecam SL NTSC, sold by Karl Storz Endoscopy, 91 Carpenter Hill Road, Charlton, Mass. 01507.
- the Storz camera bandwidth represents approximately 275 horizontal cycles (black-white transitions). This is based on a 60 Hz ( 525 line) system using a standard 52 ⁇ sec horizontal line time, i.e. (5.33 MHz/52 ⁇ sec) approximately 275 cycles per line.
- the IR video bandwidth required to deliver 275 cycles in one display line time, namely t 158 seconds as shown in FIG. 1D is about 85 MHz, i.e. (275 cycles/3.25 ⁇ sec). Bandwidth calculations for more cycles (higher resolutions) yield about 100 MHz for VGA (320 horizontal cycles) or 125 MHz for SVGA (400 horizontal cycles). In light of these considerations, the infrared interface must support a bandwidth of about 85 MHz to display high quality images from traditional NTSC or PAL sources, and greater than 100 MHz for VGA or SVGA computer generated images.
- Transmitting sequential color fields across IR video interface 100 increases the bandwidth requirement, but reduces the complexity of receiver and color decoder circuits, described below in more detail. Additionally, repeating selected color fields in the sequence enables an increase in the rate of updating a display frame, thereby reducing perceived flicker.
- S/N signal to noise ratio
- dB decibels
- a 40 dB S/N ratio represents one part of noise in 100 parts of signal, i.e., one percent noise.
- This S/N ratio equates to the EIA standard for “fine quality” broadcast television, which is seen under nearly ideal reception conditions.
- IR video interface 100 is designed to meet a higher S/N standard than 40 dB.
- FIG. 1B illustrates transceiver module 110 positioned above users 102 and connected to base station 112 by coaxial cable 182 .
- the IR signal 132 from transceiver module 110 is generated by an LED array 130 , described below in more detail.
- transceiver module 110 is located closer to base station 112 , and infrared signal 106 is partially collimated by lensed LED array 108 and aimed at a diffuse surface above users 102 .
- This configuration is less efficient than that of FIG. 1B , since IR signal 106 undergoes an extra diffuse reflection.
- FIG. 2A is a functional block diagram of a video processing circuit 160 located, for example, in base station 112 .
- An input video signal 161 enters at the upper left hand corner and is applied to a video decoder 162 .
- Input signal sources include conventional video sources, such as NTSC, PAL, or SECAM encoded composite sources, e.g., cameras, and 525/625 line component sources such as YUV or S-Video.
- Video decoder 162 consists substantially of a conventional Digital Multistandard Color Decoder IC, for example Philips Semiconductors, 811 East Arques Avenue, Sunnyvale, Calif. 94088, Model SAA 7110 Decoder.
- Video decoder 162 the NTSC/PAL/SECAM analog video signals are demodulated and converted to digital format luminance and chroma (color) for processing by subsequent circuit modules. Video decoder 162 also provides scaling and offset of various video components such as hue, brightness, and saturation for user preferences.
- Digitally formatted luminance and chroma signals from video decoder 162 are applied to a de-interlacer 164 , which converts the signals into digital RGB format and then combines the odd and even line fields into a sequential frame format.
- De-interlacer 164 consists substantially of a conventional interlaced to sequential frame converter, for example, Genesis Microchip, Inc., 2071 Landings Drive, Mountain View, Calif. 94043, Model gmVLD8 De-Interlacer, with an external pixel buffer RAM memory.
- de-interlacer 164 interpolates between the two fields to minimize tearing of vertical lines when the image is rapidly panned.
- the resulting RBG color pixel data are applied to a format scalar 166 , which interpolates to a higher pixel resolution as required for the display.
- VGA format data 640 ⁇ 480 pixels
- SVGA format data 800 ⁇ 600 pixels
- Format scaler 166 consists substantially of a conventional image scaling and anti-aliasing IC, for example, Genesis Microchip, Inc., 2071 Landings Drive, Mountain View, Calif. 94043, Model gmZ1 Scaler. Scaling algorithms familiar in the art are used to minimize video artifacts created during the scaling process.
- the RGB pixel data are applied to a scan rate converter 168 , which converts the video input frame rates of 50/60 Hz to 83.3/80 Hz to minimize the potential for visible flicker in the display.
- Scan rate converter 168 then outputs the frame data in separate red, green, and blue fields.
- Scan rate converter 168 is implemented using a Field Programmable Gate Array (FPGA), for example Xilinx, Inc., 2100 Logic Drive, San Jose, Calif., Model 4010 PQ160 FPGA, and a synchronous high speed SDRAM.
- FPGA Field Programmable Gate Array
- FIG. 2B is a functional block diagram illustrating the operation of scan rate converter 168 .
- the digital RGB color pixel data 190 from format scaler 166 enters scan rate converter 168 from the left and is stored in one of two memory banks 192 , 194 .
- Each memory bank 192 , 194 is segmented into red, green, and blue field storage labeled R, G, and B, to facilitate field sequential color imaging.
- Each memory segment can hold the red, green, or blue color value, for example an 8-bit quantity, for each of the 480,000 pixels in an SVGA (800 ⁇ 600) field.
- An input pixel store function 196 stores video frames alternately in memory bank 192 or 194 .
- an output pixel fetch function 198 reads data from the respective memory bank 192 , 194 , that was previously filled with data.
- input pixel store and output pixel fetch functions 196 and 198 respectively never overlap their use of the same memory bank 192 or 194 . Rather, they alternate memory banks, thereby de-coupling input data from output data.
- a scan rate control logic function 199 accepts data at a 50/60 Hz rate supplied by a video source and outputs the data at a higher rate, namely, 83.3/80 Hz.
- Scan rate control logic function 199 controls the flow of data in the sequential red, green, and blue fields, instructing output pixel fetch function 198 to insert repeating color fields to convert the effective frame rate.
- headset display 140 requires four pixels of data to be loaded during each input cycle.
- output pixel fetch function 198 is instructed to read four pixels, e.g., 32 bits, simultaneously and to present these pixel data in a burst format to a pixel data serializer 170 .
- Red, green, and blue color field data are serialized in pixel data serializer 170 (see FIG. 2A ) into a stream suitable for transmission over a wireless link.
- pixel data serializer 170 inserts video synchronization information to define horizontal, vertical, and color syncs.
- Pixel data serializer 170 converts the parallel digital pixel color data into amplitude modulated signals using conventional digital to analog converters (DAC's). It then outputs the analog signals, e.g., four pixel values, into a high speed serial data stream suitable for transmission over the wireless (serial) link.
- Conventional sample-and-hold amplifiers are used to delay the analog signals for conversion to a serial analog data stream, as described above in connection with FIG. 1C .
- an input audio signal 171 from a microphone or other conventional source enters an input amplifier IC 172 with level control and high frequency boost to improve overall S/N ratio.
- the audio signal is then applied to a pulse amplitude audio modulator 174 which converts the amplitude modulated signal to an amplitude pulse.
- This pulse is inserted just prior to the beginning of video data within each horizontal scan as determined by sync, timing and black level generator 176 , which controls all system timing.
- the generator's 176 timing functions are mirrored in exact synchronization by generator 334 of FIG. 3D in the remote receiver, i.e., headset, for extraction and reconstruction of data.
- D/A Converter & Buffer 178 which consists of a high-speed video quality D/A (digital/analog) converter, summing amplifier, and video line driver.
- D/A Converter & Buffer 178 which consists of a high-speed video quality D/A (digital/analog) converter, summing amplifier, and video line driver.
- the audio signal is recovered from the video via sampling using reconstructed timing information within the remote receiver.
- the audio pulse amplitude is then conventionally converted to a normal analog audio signal, e.g., for headphones, using a sample-and hold circuit in detector 324 of FIG. 3D .
- infrared LED array 130 is selected for its high frequency modulation capability, which transmits the signal as an amplitude modulated IR beam.
- infrared LED array 130 comprises one or more high-frequency infrared LED's, for example, Infineon, Inc., 10950 North Tantau Avenue, Cupertino, Calif. 95014 Model SFH4200 series of wide-angle SMT devices, or model SFH4500 series of lensed T1-3/4 devices.
- a return audio signal 151 from headset 104 (transmitted through the reverse-path free space infrared link) is received by photodiode and preamplifier 156 , which converts the modulated optical signal to a low level modulated electrical signal, amplifies the low level signal, buffers it for transmission over coaxial cable 182 , and applies it to pulse amplitude audio demodulator 157 , which recovers the audio signal using sample-and-hold technology driven by timing generator 176 .
- the audio signal is frequency conditioned to improve the signal to noise ratio at high frequency and to restore the overall audio fidelity of the signal.
- the audio signal is processed by a line driver IC amplifier with level control circuits 159 for output coupling to conventional audio output equipment (not shown).
- the return audio signal is digitized with a simple A/D converter within headset 104 . It is then formatted, stored and restructured digitally by a PIC controller within timing generator 334 of FIG. 3D as a series of full-amplitude pulses occupying a time slot normally used for a full horizontal line of video; for example, every eighth line. The video normally occupying these lines is then blanked and delayed for IR transmission until the following lines, thereby adding 600/8 or 75 lines to the SVGA format. As with other timing functions, the video blanking and audio pulse formatting is controlled in base station 160 of FIG. 2A by generator 176 and synchronously controlled by headset timing generator 334 in FIG. 3D .
- Audio information is transmitted as an IR series of pulses in the same way by headset LEDs 306 in FIG. 3A , received by collecting lens and lightcone 252 in tranceiver module 110 , and sent through coaxial cable 151 in cable bundle 182 to base station 112 , where it is digitally decoded and converted to analog audio for equivalent use. While this digital embodiment has the disadvantage of increasing the video bandwidth by 675 lines/600 lines, or 12.5%, it has the advantage of decreasing headset LEDs 306 power requirements for equivalent S/N ratio to the pulse amplitude format; thereby extending the operational life of headset batteries 104 .
- transceiver module 110 is used to distribute (broadcast) combined audio and video signals from base station 112 to headset 104 .
- Transceiver module 110 also serves as a collection and relay module for optional IR audio signal 120 returning from headset 104 .
- transceiver module 110 is located below the area where headset 104 is used.
- transceiver module 110 is located above and proximate to the area where headset 104 is used. This placement optimizes the signal distribution and provides the best image quality by minimizing reflections along the IR signal path.
- Transceiver module 110 can be implemented with a bundle of cables 182 from base station 112 .
- FIG. 2C is a block diagram of tranceiver module 110 .
- the combined video and optional audio infrared signals from base station 112 are carried by coaxial cable 270 through bundle of cables 182 to an IR LED array 130 .
- the IR LED array 130 transmits IR signal pattern 132 toward remote receiver 146 , e.g., headset 104 .
- a return modulated audio IR signal 120 from headset 104 is collected by a collecting lens and light cone 252 in transceiver module 110 , and the collected light directed to a photodiode and amplifier 254 .
- Photodiode and amplifier 254 convert and amplify the optical signal to produce an electric audio signal, which drives coaxial cable 151 through bundle of cables 182 to base station 112 .
- FIG. 2D is a more detailed transceiver schematic block diagram, in accordance with a further embodiment of the present invention.
- Transceiver circuit 110 is connected with base station 112 through electrically conducting coaxial cable 182 , which carries respective video signals 270 , audio signals 151 , and DC electrical power 272 .
- DC power 272 from coaxial cable 182 is filtered from audio and video signals by a low pass filter and power supply 260 , which provides power for the active elements of transceiver circuit 110 .
- Video signal 270 from coaxial cable 182 is filtered through a high pass filter 262 and is applied through a DC level restore module 266 to LED drivers 278 , which drive an IR LED array 130 .
- the IR LED array produces IR beam 132 .
- Optional return audio signal on return IR beam 120 is transmitted through collecting lens and lightcone 252 onto photodiode and amplifier 254 where it is converted into an electrical signal.
- the audio electrical signal from photodiode and amplifier 254 is amplified by a line driver 258 and is then filtered through a bandpass filter 264 prior to transmission as audio carrier signal 151 through coaxial cable 182 to base station 112 .
- FIG. 2E is a more detailed transceiver schematic block diagram, in accordance with a further embodiment of the present invention.
- Transceiver circuit 110 is connected with base station 112 through a bundle of cables 186 , which comprises a video coaxial cable 184 carrying video signals 270 , an audio coaxial cable 183 carrying audio signals 151 , and power cables 185 carrying DC electrical power 272 .
- Video signal 270 from video coaxial cable 184 is applied to LED drivers 278 , which drives an IR LED array 130 .
- the IR LED array produces an IR beam 132 .
- Optional return audio signal on return IR beam 120 is transmitted through collecting lens and lightcone 252 onto photodiode and amplifier 254 where it is converted into an electrical signal.
- the audio electrical signal from photodiode and amplifier 254 is amplified by a line driver 258 and is transmitted as audio carrier signal 151 through audio coaxial cable 183 to base station 112 .
- FIG. 3A is a schematic front view of an IR module 300 containing components of IR video interface 100 incorporated in headset 104 .
- FIGS. 3B and 3C are top and side schematic views, respectively, of a user wearing an embodiment of headset 104 including IR module 300 is mounted away from the user's peripheral vision field and above the LCD and associated display optics (see Hebert, cited above), thereby providing a substantially unobstructed wide angle reception path to the ceiling or to an overhead transmitting LED array.
- FIG. 3D is a functional block diagram of remote electronic circuit 142 , located for example at headset 104 .
- the combined video and optional audio low level electronic signals from IR photodetector 304 enter adjacent to the middle left hand side of FIG. 3D .
- the signals are applied to preamplifier/AGC/audio carrier filter module 305 where the signal is amplified and level adjusted by a conventional automatic gain control (AGC) circuit.
- AGC automatic gain control
- Black level control restores the correct baseline DC voltage to the video waveform.
- the composite output from preamp module 305 is further separated into three signals; a video signal portion by processor 330 , synchronization components by stripper 332 , and the optional audio component by detector 324 .
- the synchronization components are applied to a headset timing generator 334 , which generates signals that control display, LED illumination sub-systems, and optional audio sub-system.
- the separated video signal portion continues into a video processing module 330 , where gamma correction is performed using conventional techniques.
- Gamma correction compensates for the different response of headset display 140 relative to the response of a traditional phosphor based CRT.
- Gamma correction adjusts the video signal, such that headset display 140 exhibits a more accurate range of brightness than would be realized without gamma correction.
- the optional audio carrier portion is separated from the video via pulse amplitude detector 324 (e.g., sample-and-hold) using reconstructed timing information from headset timing generated 104 .
- the recovered audio signal is then applied to a headphone amplifier 326 configured for driving a conventional dynamic headphone speaker element 328 .
- FIG. 3E is a detailed functional block diagram of headset timing generator 334 , which is implemented with a field programmable gate array (FPGA), similar to that described in connection with FIG. 2A .
- the outputs from headset timing generator 334 include control signals for display drive module 336 , display LEDs 340 , and optional audio pulse separation.
- display 140 cycles through erase, load, and illuminate phases.
- Timing generator 334 is responsible for correctly sequencing these phases.
- Composite sync components from sync stripper module 332 including horizontal start pulses and field start pulses, are applied to headset timing generator 334 .
- a horizontal timer and pixel counter module 350 locates the beginnings of lines and counts the pixels within each line, for example, 800 active pixels per line of display 140 plus inactive pixels used for timing and control.
- a timing window is generated at the expected field start pulse time (with respect to the horizontal start pulse), that is used by field start detection and color identification module 354 to detect a start of field condition.
- a vertical timing module 356 is reset to zero and commences counting lines in a field, for example, 600 lines.
- a small number of non-video lines from video processor module 330 are encoded with color identifiers.
- RGB LED control module 352 uses RGB LEDs to synchronize display LEDs 340 with the appropriate color fields.
- Vertical timing pulses from vertical timing module 356 and color IDs from field start detection and color identification module 354 are used by display sync control module 358 to generate signals that control the operation of a display drive module 336 and microdisplay 140 .
- Display drive module 336 demultiplexes the video pixels, converting serial pixel analog voltages to parallel analog voltages suitable for driving a display.
- Microdisplay 140 and associated drivers 336 receive video data from video processor module 330 and control signals from headset timing generator 334 . These are used to load the sequential color field data, which are stored in memory cells internal to microdisplay 140 . After the data are loaded and stabilized, a selected red, green, or blue LED 340 is pulsed to illuminate display 140 with the correct color.
- the overall brightness of display 140 is adjusted using an optional ambient light detector circuit 342 .
- Light from the room is measured by a photodetector 308 to provide a control signal to boost or reduce the average brightness level of LEDs 340 .
- An optional return audio signal is generated by a microphone 360 mounted at headset 104 , and is processed by audio electronics including a microphone amplifier 361 and an audio pulse amplitude modulator 362 , in a manner similar to that described in connection with FIG. 2A .
- the processed return audio signal drives infrared LEDs 306 , thereby generating modulated return IR beam 120 , which is transmitted through the atmosphere to transceiver module 110 , where it is collected and relayed to base station 112 .
- the return audio signal is digitized with a simple A/D converter within headset 104 . It is then formatted, stored and restructured digitally by a PIC controller within timing generator 334 of FIG. 3D as a series of full-amplitude pulses occupying a time slot normally used for a full horizontal line of video; for example, every eighth line. The video normally occupying these lines is then blanked and delayed for IR transmission until the following lines, thereby adding 600/8 or 75 lines to the SVGA format. As with other timing functions, the video blanking and audio pulse formatting is controlled in base station 160 of FIG. 2A by generator 176 and synchronously controlled by headset timing generator 334 in FIG. 3D .
- Audio information is transmitted as an IR series of pulses in the same way by headset LEDs 306 in FIG. 3A , received by collecting lens and lightcone 252 in tranceiver module 110 , and sent through coaxial cable 151 in cable bundle 182 to base station 112 , where it is digitally decode and converted to analog audio for equivalent use. While this digital embodiment has the disadvantage of increasing the video bandwidth by 675 lines/600 lines, or 12.5%, it has the advantage of decreasing headset LEDs 306 power requirements for equivalent S/N ratio to the pulse amplitude format; thereby extending the operational life of headset batteries 104 .
- FIG. 4 is a schematic diagram of LED array 130 , in accordance with an embodiment of the present invention.
- FIG. 4 shows a cluster 132 of high-speed infrared-emitting LEDs 131 in the array with each LED 131 emitting an identical optical signal.
- FIG. 2D shows the LED array 130 is interconnected with the video processing circuit 160 in base station 112 through a single coaxial cable 182 or through one or more electrical cables 186 .
- each cluster 132 consists of five LEDs 131 driven in series from a common modulated electrical source through electronic buffers 133 from a common +12 volt power supply 260 .
- the common modulated electrical source is electrical coaxial cable 182 , as shown in FIG. 2D .
- the common modulated electrical source is a single coaxial cable 184 in bundle of cables 186 , as in FIG. 2E .
- the clusters 132 of LEDs 131 and their common modulated electronic driver sources 133 are driven in parallel from the common modulated electrical source. This invention encompasses clusters with more than five LEDs and clusters with less than five LEDs.
- the optimum number of LEDs 131 in the array 130 depends on the desired minimum S/N ratio at the desired maximum range of separation between tranceiver 110 and remote receiver 302 .
- random noise from silicon detectors such as detector 304 in FIG. 3A is nearly constant, while signal strength generally falls off with the square of the range of separation. Therefore, doubling the number of LEDs 131 in array 130 will generally increase the S/N ratio by the square root of 2, or 44%, at a given range.
- the desired S/N ratio is reached at a ratio where an increase in the S/N ratio is no longer noticeable to the eyes of user 102 . This is generally in excess of 40 db.
- pairs of clusters 132 are physically arranged in an electronic dipole configuration so that the electromagnetic field from one cluster tends to cancel out the electromagnetic field from the other cluster. The cancellation of the electromagnetic fields minimizes spurious electronic emissions.
- FIG. 4 shows a partially loaded circuit board with spaces 134 with loaded LEDs 131 and spaces 135 with unloaded LEDs.
- the clusters 132 of LEDs 131 are symmetrically laid out as pairs of clusters so as to further minimize spurious electromagnetic radiation.
- the circuit board is fully loaded with LEDs 131 .
- the spatially distributed nature of the array 130 minimizes the possibility of total blockage of the signal path to the headset 104 from an obstruction, such as a swinging arm of a surgical lighting system.
- FIG. 5A is a cross-sectional schematic diagram of collecting lens assembly 302 at headset 104 , in accordance with an embodiment of the present invention.
- Collecting lens assembly 302 is configured to achieve both a large angular field of view 510 , e.g., to accommodate head motion, and a large entrance pupil to receive maximum energy from IR signal 118 or 132 .
- collecting lens assembly 302 is nominally rotationally symmetric about a symmetry axis 512 , it can alternatively be configured to provide an azimuthally variable detectivity pattern about axis 512 .
- Photodetector 304 can be any photosensitive device having the optical and electronic responses required for the application, but is shown in FIG. 5A as a silicon device selected for its high frequency electronic performance and for high sensitivity to infrared radiation over a wavelength region of approximately 700 nm to approximately 1100 nm.
- the collecting lens assembly 302 consists of three coaxial components: a refractive lens 518 with a conic sidewall and planar exit pupil, an inner cone 515 with a diffusely reflective sidewall mating to lens' 518 conic sidewall, and an outer cone 520 with smooth inner reflective walls extended above the lens which reflects IR signal 118 or 132 into lens 518 and inner light cone 514 over a wider aperture.
- Inner walls 521 of outer cone 520 are oriented, for example, at an angle of approximately 19 degrees to approximately 27 degrees relative to symmetry axis 512 .
- Photodetector 304 is optically cemented to the lens' 518 planar exit pupil to couple some energy into photodetector 304 that would otherwise be lost to total internal reflection within lens 518 . This combination of elements dramatically enhances photodetector's 304 effective apperture over a wide angular field of view, thereby permitting the use of a smaller, wider bandwidth, less expensive photodetector to collect sufficient levels of infrared signal
- one or more optical elements of the collecting lens assembly are aspheric.
- Collecting lens 518 has an aspheric first surface with a numerical aperture of approximately f/0.8, which provides a large entrance pupil.
- Transmissive optical elements are typically made from transparent polymeric material, for example polymethyl methacrylate, polycarbonate, and ULTEM® grade polycarbonate manufactured by General Electric Company. Reflective elements can be made of any material that is reflective in the infrared spectrum of interest; the preferred embodiment being a plastic such as polycarbonate that is reflectively overcoated with aluminum.
- Collecting lens and light cone 252 has substantially the same structure as collecting lens assembly 302 .
- FIGS. 5B-5D are cross-sectional schematic diagrams illustrating the transmission and capture of IR radiation incident from various angles onto collecting lens assembly 302 .
- FIG. 5B shows the case of radiation 530 incident normal to the plane of photodetector 304 . This radiation is captured efficiently by wide-angle lens 518 alone.
- FIG. 5C shows the case of radiation 532 incident at a 30-degree angle from normal, which is efficiently captured by wide-angle lens 518 and inner light cone 514 cooperatively.
- FIG. 5D shows the case of radiation 534 incident at 40-degree angle from normal.
- outer conic cavity 520 reflects radiation into wide-angle lens 518 that would otherwise be lost. This reclaimed radiation is then efficiently captured by wide-angle lens 518 and inner light cone 514 cooperatively.
- FIGS. 5E and 5F are cross-sectional schematic diagrams of a collecting lens assembly 302 with prismatic dispersion plate 560 .
- the prismatic dispersion plate 560 overlies the outer conic cavity 520 , as shown in FIG. 5E .
- the prismatic dispersion plate is molded together and integrated with the outer headset cover, an infrared pass filter window 522 , as shown in FIG. 5F .
- the effect of this dispersion is further illustrated in FIG. 3C with the IR reception angle being widened from 80 degrees to 120 degrees, for example.
- the prismatic dispersion plate is typically made of an optically transmissive polymeric material (for example ULTEM® grade polycarbonate manufactured by the General Electric Company).
- FIG. 5G is a plane view illustrating a prismatic dispersion plate 560 .
- the grooves 562 widen the angular field of vision 510 , or collection angle, of the collecting lens assembly.
- the plate 560 has both vertical 580 and horizontal 582 grooves.
- Horizontal grooves 580 widen the angular field of vision 510 during heads up-down motion, while vertical grooves 582 widen the angular field of vision 510 in side-to-side head motion.
- the prismatic pattern is widened symmetrically with an equal number of horizontal grooves 582 and vertical grooves 580 .
- the angular field of vision 510 , or collection angle is widened asymmetrically with an asymmetrical prismatic pattern.
- One embodiment has more horizontal grooves 582 than vertical grooves 580 to favor heads up-down motion over side-to-side head motion.
- side-to-side head motion is favored over heads up-down motion by having more vertical grooves 580 than horizontal grooves 582 .
- FIG. 5H illustrates the transmission and capture of IR radiation incident from various angles onto the prismatic dispersion plate 560 .
- IR rays 564 and 566 are within the normal angular range of the assembly (“on axis”), while IR ray 568 is an extreme angle outside the normal angular range of the assembly (“off axis”).
- On-axis IR ray 564 enters the prismatic dispersion plate 560 at surface 570 and is refracted slightly before exiting plate 560 at surface 572 .
- On-axis IR ray 564 also enters the plate 560 at surface 570 , but is internally reflected by groove 562 , so it is does not reach the user 102 .
- the grooves decrease on-axis sensitivity by internally reflecting some of the on-axis rays. However, the grooves enhance off-axis sensitivity. Off-axis IR ray 568 would normally not be within the angular range of the assembly. However, when an off-axis IR ray 568 enters the plate 560 at surface 570 , the ray 568 is refracted by groove 562 to be within the angular acceptance of the collection lens assembly 302 .
- An optional optical bandpass filter is positioned over the assembly to minimize undesirable radiation (i.e., noise) outside of the infrared spectrum that constitutes the audio/video signal.
- An optical filter 522 such as KODAK WRATTEN® No. 87B, covers the entrance aperture of outer conic cavity 520 .
- an optical filtering material such as an absorptive dye is incorporated into the material of wide-angle collecting lens 518 or dispersion plate 560 .
- FIG. 5I is a graphic representation of the calculated radiative capture by various elements of collecting lens assembly 302 , relative to the radiative capture by photodetector 304 absent the other elements of collecting lens assembly 302 .
- Relative radiative capture is shown along the vertical axis, and off-axis an incidence relative to symmetry axis 512 is shown along the horizontal axis.
- the baseline radiative capture of identically 1.0 by unaided photodetector 304 is shown as curve 540 .
- Curve 542 shows the combined relative radiative capture by photodetector 304 and wide-angle collecting lens 518 .
- Curve 544 shows the combined relative radiative capture by photodetector 304 , wide-angle collecting lens 518 , and inner light cone 514 .
- Curve 546 shows the combined relative radiative capture by entire collecting lens assembly 302 , including photodetector 304 , wide-angle collecting lens 518 , inner light cone 514 , and outer conic cavity 520 . As shown in FIG. 5C , the radiative capture by complete collecting lens assembly 302 relative to unaided photodetector 304 exceeds a factor of 15 on-axis and approximates a factor of four at a 40-degree angle off-axis.
- Curve 548 shows the relative radiative capture by the vertical grooves 580 in the prismatic dispersion plate 560 .
- Curve 550 shows the relative radiative capture by the horizontal grooves 582 in the prismatic dispersion plate 560 .
- the prismatic dispersion plate increases off-axis enhancement at the cost of on-axis sensitivity.
- a tether 124 (shown dashed) is connected directly from base station 112 to remote electronic circuit 142 at headset 104 , where it delivers the combined audio/visual signal.
- Tether 124 can contain either a single or dual optical fiber cable, such that a second optical fiber transmits return IR signal 120 .
- tether 124 contains a bi-directional, electrically conducting coaxial cable. This configuration greatly simplifies IR video interface 100 by eliminating most components of transceiver module 110 and collecting lens assembly 302 .
- the coaxial cable implementation additionally eliminates all optical and optoelectronic components. It also improves signal to noise performance at reduced power.
- the direct tether connection to headset 104 adversely restricts the freedom of motion of user 102 .
Abstract
Description
- This application is related to application Ser. No. 09/305,092, filed May 3, 1999, now U.S. Pat. No. 6,101,038 issued on Aug. 8, 2000, the specification of which is expressly incorporated herein in its entirety.
- This application is also related to Hebert, “Biocular Viewing System with Intermediate Image Planes for an Electronic Display Device”, U.S. patent application Ser. No. 09/056,934, filed Apr. 6, 1998, commonly assigned, now U.S. Pat. No. 5,926,318 issued on Jul. 20, 1999, the specification of which is incorporated herein in its entirety.
- This invention relates generally to image display systems and more particularly to infrared video and audio interfaces for head-mounted displays, principally eyeglass-mounted displays.
- High quality medical remote imaging has gained increasing importance. This is particularly true of imaging during surgical procedures, most importantly minimally invasive procedures in which direct viewing of the surgical field is difficult. For example, a method for performing coronary artery bypass relies on viewing the cardiac region through a thoracoscope or other viewing scope (see for example Sterman et al. U.S. Pat. No. 5,452,733 and Gifford, III et al. U.S. Pat. No. 5,695,504). As a further example, a surgeon may perform a delicate vascular- or neuro-microsurgical reconstruction through a minimal incision under remote viewing. Remote imaging is now common in orthopedics, ophthalmology, urology, gynecology, anesthesiology, and other medical specifications.
- In a conventional surgical environment, remote imaging is accomplished by attaching a video camera to an endoscope or other minimally invasive instrument and transmitting the video image via cable to a conventional CRT video monitor. This is often cumbersome in a crowded, brightly lighted operating room, where surgical team members are frequently moving around and the surgeon's view of the image screen is obstructed. Additionally, the CRT monitor is incapable of providing the surgeon with critical depth perception, since it is not stereographic.
- Head-mounted displays (HMDs) potentially offer a method to overcome viewing obstructions typical in a surgical environment. While head-mounted displays have been designed, developed and deployed in military applications for many years, such displays are generally bulky, expensive, application-specific devices poorly suited to commercial or surgical applications. Additionally, users of head-mounted displays are frequently restricted in their range of motion by cumbersome interface cabling.
- A compact HMD system requires a very small display device, such as those found in modern camcorder viewfinders, but with significantly higher resolution. A number of such devices are now becoming available, including transmissive and reflective liquid-crystal microdisplay devices and micro-mirror devices having resolutions at or in excess of VGA quality (640 pixels by 480 pixels) with pixel sizes on the order of 15 microns or less. However, they require integration into an ergonomic, well engineered and economical design. Most of these devices exhibit satisfactory image contrast only when illuminated and viewed at narrow angles of incidence, which compromises field of view, eye relief, and viewing comfort. Peripheral vision is also an important consideration.
- A medical stereographic HMD system having dual display devices is described in Heacock et al. “Viewing Ocular Tissues with A Stereoscopic Endoscope Coupled to a Head Mounted Display (HMD),” http://www.hitl.washington.edu/publications/heacock/, Feb. 17, 1998. Kaiser Electro-Optics (2752 Loker Avenue West, Carlsbad, Calif. 92008 manufactures the “CardioView,” “Series 8000,” and “StereoSite” HMD display systems for Vista Medical Technologies. These systems are bulky, heavy, and expensive, and require two LCD display devices. For peripheral vision correction they require the user to wear the HMD over conventional corrective eyeglasses, aggravating user inconvenience and discomfort. Meyerhofer et al. U.S. Pat. No. 5,619,373, issued Apr. 8, 1997, describes a single display device involving beamsplitters for non-stereographic, biocular viewing.
- The scan formats of video source devices, e.g., cameras and cassette players, are not directly compatible with typical solid state display devices. In particular, frame rates conforming with NTSC or PAL standards are too slow, and produce undesirable perceived flicker in solid state displays, which do not have the luminous persistence of phosphor screen displays, for example conventional TV displays. Therefore scan format and frame rate conversion are needed.
- Heckman, U.S. Pat. No. 3,674,925, describes a wireless interface between a video camera source and a remote viewing display, employing a modulated optical video signal transmitter which doubles as a target illuminator. Hanson et al., U.S. Pat. No. 5,005,213, describes a wireless infrared/optical video interface directed to military applications. Puar et al., U.S. Pat. No. 5,650,955 describes an infrared interface for generating video images on a LCD or CRT display. However, the above cited U.S. patents do not address, among other things, serial multiplexed color data, frame rate or scan format conversion.
- Therefore, what is needed in the art is a compact, high resolution, high contrast microdisplay system, particularly for surgical viewing, that is suitable for head-mounted display use without requiring undue complexity or expense and that preferably supports biocular and/or truly stereographic viewing. The system should incorporate format and frame rate conversion to provide compatibility between solid state display devices and conventional video input sources. The system should provide good color fidelity and should incorporate ergonomic design for comfort and efficiency, including peripheral vision accommodation and minimal cabling.
- Apparatus according to the present invention includes a video or audio/video interface linking a base station with a remote video display. Video processing circuitry at the base station converts an input color video signal conforming to NTSC (525 lines) or PAL (625 lines) formats from a conventional source, e.g., a video camera, into a modulated video signal having a format appropriate to drive a solid state video display, e.g., a sequential color LCD display. The modulated video signal has a data structure containing a repetitive sequence of uniform frame times. Each frame time consists of substantially equal consecutive field times for each of three component color fields. Image information for each color field is encoded as a burst of pixel luminance data occupying roughly one-half of each field time. The data structure also typically contains embedded scan and frame control signals and an embedded pulse-amplitude audio channel. To minimize flicker with many such sequential color LCD displays, a frame rate of the order of 80 frames per second, or 240 color fields per second is required. This in turn requires a field time of roughly 4 msec. To handle this video data throughput typically requires a bandwidth of at least 100 MHz. A few such displays can operate at 60 frames per second without objectional flicker. These displays can then utilize roughly 5.3 msec. per field with at least 75 MHz. bandwidth.
- To achieve the required frame rate, conversion is needed from conventional frame rates of 60 Hz or 50 Hz from input sources conforming to NTSC and PAL formats respectively.
- The modulated video signal is transmitted to a remote receiver, located adjacent to a remote video display. The remote video display can be a mobile display, for example mounted in a headset worn by a user. A remote circuit interconnected between the receiver and the display demodulates the modulated video signal and controls the image data loading and color illumination of the display device. During each burst of data, the display device is loaded with pixel luminance data for a single color field. During the period between the end of one data burst and the onset of a subsequent burst, the display device is illuminated by a color LED corresponding to the color field just loaded. The process is repeated sequentially for the three color fields of the image frame, such that bursts of pixel luminance data alternate synchronously with illumination by an appropriate LED.
- In some embodiments, two separate video display devices can be driven in parallel for a simplified binocular optical configuration.
- In other embodiments, two separate video display devices (e.g., for stereographic viewing or alternate images on independently viewed displays) can be driven alternately. Each display is illuminated while the other display is loaded with video data. The two alternating burst datastreams are derived from a single time-multiplexed modulated video signal.
- Although the video interface can transmit the modulated video signal through a conductive coaxial cable from the base station to the remote receiver, cables are cumbersome for mobile receivers, for example head-mounted displays, where they restrict the motion of a user. Alternatively, the video interface can transmit the modulated video signal on a modulated beam of infrared or other electromagnetic energy. Particularly, an infrared (IR) wavelength can be selected outside the visible region in the range of approximately 700 nm to approximately 1100 nm, where good conventional photodetectors, LEDs, and laser diodes are available. The IR beam can be guided through an optical fiber connecting the base station with the remote receiver, but this method has the same drawbacks as the coaxial cable.
- Alternatively, the IR beam is generated by an array of IR LEDs and transmitted through a free atmospheric path. All the LEDs in the array emit an identical optical signal.
- In one embodiment, the IR beam is projected onto a diffuse reflecting surface (for example the ceiling or an efficient scattering panel adjacent to the ceiling) by an array of conventional infrared LEDs with integral collimating lenses (typically for a ±10 degree radiation pattern from each LED) connected to the base station. The lensed LEDs typically create a 2 ft diameter on a scattering surface 6 feet away. The scattering surface is positioned and oriented such that the scattered IR signal is concentrated in a volume including the probable location of a remote receiver.
- In one embodiment, the receiver includes a collecting lens assembly consisting of three coaxial components: a refractive lens with a conic sidewall and planar exit pupil, an inner cone with a diffusely reflective sidewall mating to the lens' conic sidewall, and an outer smooth reflective cone extended above the lens. The photodetector is optically cemented to the lens' planar exit pupil to couple some energy into the photodetector that would otherwise be lost to total internal reflection within the lens. This combination of elements dramatically enhances the photodetector's effective aperture over a wide angular field of view, thereby permitting the use of a smaller, wider bandwidth, less expensive photodetector to collect sufficient levels of infrared signal.
- An optional prismatic dispersion plate overlying the outer conic cavity widens the collecting angle of the assembly. An optional optical bandpass filter is positioned over the assembly.
- In some embodiments, one or more optical elements of the collecting lens assembly are aspheric. Transmissive optical elements are typically made from transparent polymeric material, for example polymethyl methacrylate, polycarbonate, and ULTEM® grade polycarbonate manufactured by General Electric Company.
- In some embodiments, the interface includes a return audio link that provides return audio communication from the receiver location to the base station. The return audio can modulate a LED, which emits an audio modulated IR signal through the atmosphere. The audio modulated IR signal is then detected by a separate receiver connected to the base station, where the audio data is processed and restored. In one embodiment, the audio receiver has the same three-component coaxial structure as the remote receiver.
- The present invention is better understood upon consideration of the detailed description below, in conjunction with the accompanying drawings.
- The present invention may be better understood, and its numerous objects, features, and advantages made apparent to those skilled in the art by referencing the accompanying drawings. For simplicity and ease of understanding, common numbering of elements within the illustrations is employed where an element is the same in different drawings.
-
FIG. 1A is a schematic view of a surgical environment including an infrared video interface for a head-mounted display, in accordance with the present invention; -
FIG. 1B is a schematic view of an alternative surgical environment to that ofFIG. 1A ; -
FIG. 1C is a graphic representation of a typical burst mode data structure for a solid state video display, in accordance with an embodiment of the present invention; -
FIG. 1D is graphic representation of a typical horizontal line within the data burst structure ofFIG. 1C , illustrated on an expanded time scale; -
FIG. 2A is a functional block diagram of a video processing circuit located, for example, in a base station; -
FIG. 2B is a functional block diagram illustrating the operation of a scan rate converter; -
FIG. 2C is a block diagram of a transceiver module; -
FIGS. 2D and 2E are more detailed transceiver schematic block diagram, in accordance with further embodiments of the present invention; -
FIG. 3A is a schematic front view of an IR module incorporated in a headset, containing components of the IR video interface; -
FIG. 3B is a top schematic view of an embodiment of a headset including an IR module; -
FIG. 3C is a side schematic view of a user wearing an embodiment of a headset including an IR module; -
FIG. 3D is a functional block diagram of a remote electronics circuit, located for example at a headset; -
FIG. 3E is a detailed functional block diagram of a headset timing generator; -
FIG. 4 is a schematic diagram of an array of IR LEDs; -
FIG. 5A is a cross-sectional schematic diagram of a collecting lens assembly, in accordance with an embodiment of the present invention; -
FIGS. 5B-5D are cross-sectional schematic diagrams illustrating the transmission and capture of IR radiation incident from various angles onto a collecting lens assembly; -
FIGS. 5E and 5F are cross-sectional schematic diagrams of a collecting lens assembly with a prismatic dispersion plate; -
FIG. 5G is a plane view of a prismatic dispersion plate; -
FIG. 5H is a cross-sectional schematic diagram illustrating the transmission and capture of IR radiation incident from various angles onto a prismatic dispersion plate; and -
FIG. 5I is a graphic representation of the calculated radiative capture by various combinations of elements of the assembly ofFIG. 5A , relative to the radiative capture by an unaided photodetector. - The following is a detailed description of illustrative embodiments of the present invention. As these embodiments of the present invention are described with reference to the aforementioned drawings, various modifications or adaptations of the methods and or specific structures described may become apparent. These descriptions and drawings are not to be considered in a limiting sense as it is understood that the present invention is in no way limited to the embodiments illustrated.
-
FIG. 1A is a schematic view of a surgical environment including aninfrared video interface 100 for a head-mounted display, in accordance with the present invention. Auser 102, e.g., a surgeon or assistant, wears aheadset 104, containing a remotevideo display device 140 and a remoteelectronic circuit 142, including ancillary optical, audio, and electronic apparatus, described in more detail below. In some embodiments, all of the receiving, processing, audio, and display functions relating to the head-mounted display are performed withinheadset 104. Alternatively, some of these functions are performed within anoptional utility module 105 a attached, for example, to the clothing or belt ofuser 102, and connected toheadset 104 by autility cable 105 b.Batteries 144 configured to power the respective head-mounted display functions can be mounted atheadset 104 or optionally atutility module 105 a. A remote mobilevideo bandwidth receiver 146 located, e.g., atheadset 104, receives a diffusely reflectedinfrared signal 118 carrying video and/or audio data on a modulated beam of electromagnetic energy. A modulated infrared signal/transmitted through the atmosphere from an array 108 of conventional IR LEDs with integral collimating lenses (typically for a ±10 degree radiation pattern from each LED) withintransceiver module 110, which is connected to abase station 112 by a bundle ofcables 182. The lensed LEDs typically create a 2 ft diameter on a diffusely reflective scattering surface 6 feet away. Alternatively,transceiver module 110 is integral withbase station 112. - The
LED array 130 is significantly less expensive than a single laser diode of equivalent power and offers a significant reliability advantage over the laser diode. In addition, the spatially distributed nature of theLED array 130 avoids the high power density of the laser diode and minimizes the possibility of total blockage of the signal path to the headset. The lower power density of theLED array 130 avoids the design considerations and concerns related to potential eye damage and associated regulatory controls, such as FDA regulations. - In the embodiment of
FIG. 1A , lensed LED array 108 project modulatedIR signal 106 through the atmosphere onto a diffusely reflective target area of theceiling 116 or (not shown) mountedadjacent ceiling 116.Infrared signal 106 is scattered through the atmosphere from the diffuse target area as diffusely reflectedIR signal 118, a portion of which illuminatesheadset 104. In some embodiments, the diffuse target area (e.g., ceiling 116), provides a substantially cosine (Lambertian) pattern of diffusely reflectedIR signal 118. Alternatively, the diffuse target area has a lenticular or other well known surface structure, providing a directionally preferred scattering pattern of scatteredinfrared signal 118. - In some embodiments,
headset 104 provides a return audio signal back tobase station 112. Areturn IR fan 120 carries the modulated audio signal at least in part through an atmospheric transmission path, generally retracing the transmission paths of diffusely reflectedIR signal 118 and projectedinfrared signal 106. - Although a surgical environment is depicted in
FIG. 1A , in other embodiments similar configurations including aninfrared video interface 100 for a head-mounted display, in accordance with the present invention, are applied to a variety of environments. -
FIG. 1B is a schematic view of an alternative surgical environment to that ofFIG. 1A .Transceiver module 110 is attached above the working space ofusers 102, e.g., suspended fromceiling 116 or other elevated support, and is connected withbase station 112 bycoaxial cable 182. Anarray 130 of IR LEDs, described in more detail below, is mounted intransceiver module 110 and is configured to direct anIR beam 132 within the volume including the probable location ofrespective users 102. - Many of the properties of
infrared video interface 100, in accordance with the present invention, derive from the requirements of head-mountedvideo display 140. For some embodiments, these requirements are described in Hebert, U.S. patent application Ser. No. 09/056,934, cited above. In some embodiments,headset 104 does not require a frame memory. - To simulate full color using a monochromatic display, as described in Hebert, cited above, the display device is sequentially illuminated with red, green, and blue light sources, for example LEDs. When this sequential illumination occurs at a high enough frequency, a user's eye merges the sequence of colors and creates a perception of full color. In accordance with the present invention,
infrared video interface 100 carries each sequential field of red, green, or blue video information as a burst of data. The sequence is repeated at a data rate, such that full motion and full color images are simulated. - A solid state display is preferably illuminated at approximately an 80 Hz frame rate (a 240 Hz field rate representing a three-color sequence for each frame) to minimize the amount of flicker perceived by the eye. This is equivalent to one color field each 4.16 msec. However, the NTSC (National Television Standards Committee) video format provides a 60 Hz frame rate, whereas PAL (phase Alternating Line) video format provides a 50 Hz frame rate. Both of these frame rates are too slow to prevent perceived flicker in the solid state display. Because of luminous persistence of phosphors, conventional video displays, e.g. TV screens, are more tolerant of the slower frame rates. Therefore, frame rate conversion is performed in
base station 112. In accordance with embodiments of the present invention, the method adopted to increase the effective frame rate is cyclical repetition of one or two of the sequential red, green, or blue fields. The specific scheme applied depends on whether the input source conforms with an NTSC or PAL format. For further discussion of conventional video standards and sources, see for example K. Jack, “Video Demystified, A Handbook for the Digital Engineer,” Second Edition, 1996, published by HighText Interactive, Inc., San Diego. - From an NTSC source, interlaced color frames are received at a rate of 60 Hz, (or 16.66 msec per frame). If RGB indicates a new input three-color field sequence (red, green, blue), and if rgb indicates repeated color fields (red, green, blue) stored in the memory of
base station 112, then the following sequence can be generated: RGBrGBRgBRGb RGBrGBRgBRGb. The insertion of one repeated output color field for each consecutive set of three input color fields increases the perceived frame rate and thereby reduces perceived flicker. The repeated color selection and the consecutive input color field set are rotated cyclically, thereby preserving the original color sequence and retaining color fidelity. By converting to four separate color fields every 16.66 msec, then each individual color field is sent every 16.66/4=4.16 msec, resulting in 240 individual color fields transmitted per second. The effective three-color frame output rate is 240/3=80 “full color” frames per second. - From a PAL source, interlaced color frames are received at a rate of 50 Hz (or 20.0 msec per frame). If RGB indicates a new input three-color field sequence (red, green, blue), and rgb indicates repeated color fields (red, green, blue) stored in the memory of
base station 112, then the following sequence can be generated: RGBrgBRGbrGBRgb RGBrgBRGbrGBRgb. - The insertion of two repeated output color fields for each consecutive set of three input color fields increases the perceived frame rate and thereby reduces perceived flicker. The repeated color selections and the consecutive input color field set are rotated cyclically, thereby preserving the original color sequence and retaining color fidelity. By converting to five separate color fields every 20 msec, then each individual color field is sent every 20.0/5=4.0 msec, resulting in 250 individual color fields transmitted per second. The effective three-color frame output rate is 250/3=83.3 “full color” frames per second.
- The total available field time (either 4.16 msec for NTSC or 4.0 msec for PAL) is partitioned into several phases, including: (1) erase display, (2) load display, and (3) illuminate display with an appropriate red, green, or blue LED. In the present embodiment, the display loading time is selected to be approximately 2 msec for each color field of information. During this 2 msec interval, each of the 800×600 pixels in the display is loaded with luminance data, namely, some gradation between black and white. Thus, all 480,000 of the pixels must be loaded in 2 msec or less. This corresponds to a throughput of 480,000 pixels/2 msec=240 million pixels per second. Thus, during 2 msec of each color field time,
infrared video interface 100 transmits the equivalent of 240 million pixels per second, roughly a pixel every 4 nsec. - These times may require adjustments for particular microdisplays, since some are more prone to flicker than others. For example, reflective sequential-color microdisplays manufactured by The MicroDisplay Corporation, 3055 Research Drive, San Pablo, Calif., work without objectionable flicker at the native frame rates of NTSC and PAL (60 frames/sec and 50 frames/sec, respectively). This eliminates the requirement for the infill color fields, allowing a simplification of the electronics for a simple RGBRGB sequence. It correspondingly reduces field times (roughly 5.6 msec. for NTSC; 6.7 msec. for PAL) and consequent bandwidth requirements.
- Since the signal is encoded using a baseband (no carrier frequency) amplitude modulation (AM) protocol, the IR interface data transport scheme can be described as “burst mode amplitude modulation”. Amplitude modulation is preferred to simplify the receiver design. The encoding scheme can alternatively utilize pulse modulation without changing the fundamental structure of
IR video interface 100. However, receiver circuits would need to convert the pulse modulation back into amplitude modulation to be compatible with a display of the present embodiment. -
FIG. 1C is a graphic representation of a typical burst mode data structure for a solid state video display, in accordance with an embodiment of the present invention. InFIG. 1C ,signal amplitude 136 is shown as a function oftime 138. Sequential input color fields are labeled R, G, and B, followed by a repeated color field stored in the memory ofbase station 112 labeled r. A repetitive sequence ofhorizontal sync pulses 152 provides basic synchronization for the burst mode structure. Anindividual color field 137 has a time duration t137, or approximately 4 msec. Thepixel luminance data 139 within each color field is transmitted as a data burst and has a time duration t139, or approximately 2 msec. -
FIG. 1D is graphic representation of a typical horizontal line within data burst 139 ofFIG. 1C , illustrated on an expanded time scale. Within each color field burst R, G, B, r ofFIG. 1C is a series ofhorizontal sync pulses 152. A typicalhorizontal sync pulse 152 has a substantially rectangular waveform with an amplitude that defines a grey scale betweenblack 154 and white 156. Between consecutivehorizontal sync pulses 152 is one horizontal line ofanalog pixel data 158, e.g. 800 pixels to a horizontal line. The full width t152 of ahorizontal sync pulse 152 provides a clock basis for data burst timing and has a duration typically equal to that of 32 pixels, which corresponds to approximately 125 nsec. The time duration t158 between consecutive horizontal sync pulses is approximately 3.25 μsec in the example shown. - The design of
IR video interface 100 is adaptable for driving one or two video/audio channels. In the single channel mode, half of each color field time is used for data transfer, and the other half is used for illumination. In the dual channel mode, one half of each color field time is used for data transfer to a first channel, and the other half of the time is used for data transfer to the second channel. First and second channels' displays are alternately illuminated, such that the illumination occurs for one display while data is transferred to the other display, as described below in more detail. Two data channels can be used for presentation of true stereo imagery or, alternatively, a different view (i.e., inverted view or different video source) sent to different observers. - If
IR video interface 100 is required to transmit data for two display channels, then each channel can be alternately loaded and illuminated. Thus the interface carries burst mode image data for a first channel during 2 msec, while a second channel is illuminated. Likewise, during the next approximately 2 msec interval, the first channel is illuminated while the interface transmits image data to the second channel. This scheme can be described as time division multiplexing (or time-duplexing) with burst mode modulation. - In an imaging system, bandwidth manifests itself as the ability to resolve fine spatial details, such as the sharp edges of objects. More specifically, the measured bandwidth of a high quality endoscopy video camera is approximately 5.33 MHz for each individual RGB channel. An example of such a video camera is the Telecam SL NTSC, sold by Karl Storz Endoscopy, 91 Carpenter Hill Road, Charlton, Mass. 01507. The Storz camera bandwidth represents approximately 275 horizontal cycles (black-white transitions). This is based on a 60 Hz (525 line) system using a standard 52 μsec horizontal line time, i.e. (5.33 MHz/52 μsec) approximately 275 cycles per line. The IR video bandwidth required to deliver 275 cycles in one display line time, namely t158 seconds as shown in
FIG. 1D , is about 85 MHz, i.e. (275 cycles/3.25 μsec). Bandwidth calculations for more cycles (higher resolutions) yield about 100 MHz for VGA (320 horizontal cycles) or 125 MHz for SVGA (400 horizontal cycles). In light of these considerations, the infrared interface must support a bandwidth of about 85 MHz to display high quality images from traditional NTSC or PAL sources, and greater than 100 MHz for VGA or SVGA computer generated images. - Conventional bandwidths for audio communication channels are only of the order of 20 KHz. Therefore, an audio channel can easily be added or embedded as a subcarrier in a video channel having bandwidth of the order of 100 MHz. In the discussion below, a video interface, channel, or signal is generally assumed to include dual audio/video capability.
- Transmitting sequential color fields across
IR video interface 100 increases the bandwidth requirement, but reduces the complexity of receiver and color decoder circuits, described below in more detail. Additionally, repeating selected color fields in the sequence enables an increase in the rate of updating a display frame, thereby reducing perceived flicker. - In addition to bandwidth (image resolution), signals transmitted over an interface must have a high signal to noise ratio (S/N) to provide acceptable image quality. S/N ratios are conventionally measured in decibels (dB), which are logarithmic units. For example, a 40 dB S/N ratio represents one part of noise in 100 parts of signal, i.e., one percent noise. This S/N ratio equates to the EIA standard for “fine quality” broadcast television, which is seen under nearly ideal reception conditions. For a detailed discussion, see for example Table 16 and related text of “Satellite Technology, An Introduction,” by Andrew F. Inglis, 1991, published by Focal Press. In accordance with the present invention,
IR video interface 100 is designed to meet a higher S/N standard than 40 dB. - The high bandwidth and signal to noise requirements discussed above require that the IR composite video and audio transmitter and return audio receiver (e.g., transceiver module 110) be located in a position where the IR energy is efficiently directed toward the area of the receiver, namely
headset 104.FIG. 1B illustratestransceiver module 110 positioned aboveusers 102 and connected tobase station 112 bycoaxial cable 182. As shown inFIG. 1B , the IR signal 132 fromtransceiver module 110 is generated by anLED array 130, described below in more detail. InFIG. 1A ,transceiver module 110 is located closer tobase station 112, andinfrared signal 106 is partially collimated by lensed LED array 108 and aimed at a diffuse surface aboveusers 102. This configuration is less efficient than that ofFIG. 1B , since IR signal 106 undergoes an extra diffuse reflection. -
FIG. 2A is a functional block diagram of avideo processing circuit 160 located, for example, inbase station 112. Aninput video signal 161 enters at the upper left hand corner and is applied to avideo decoder 162. Input signal sources (not shown) include conventional video sources, such as NTSC, PAL, or SECAM encoded composite sources, e.g., cameras, and 525/625 line component sources such as YUV or S-Video.Video decoder 162 consists substantially of a conventional Digital Multistandard Color Decoder IC, for example Philips Semiconductors, 811 East Arques Avenue, Sunnyvale, Calif. 94088, Model SAA 7110 Decoder. - In
video decoder 162 the NTSC/PAL/SECAM analog video signals are demodulated and converted to digital format luminance and chroma (color) for processing by subsequent circuit modules.Video decoder 162 also provides scaling and offset of various video components such as hue, brightness, and saturation for user preferences. - Digitally formatted luminance and chroma signals from
video decoder 162 are applied to a de-interlacer 164, which converts the signals into digital RGB format and then combines the odd and even line fields into a sequential frame format.De-interlacer 164 consists substantially of a conventional interlaced to sequential frame converter, for example, Genesis Microchip, Inc., 2071 Landings Drive, Mountain View, Calif. 94043, Model gmVLD8 De-Interlacer, with an external pixel buffer RAM memory. Since the odd and even scan fields are separated in time by either {fraction (1/50)}, or {fraction (1/60)} sec.,de-interlacer 164 interpolates between the two fields to minimize tearing of vertical lines when the image is rapidly panned. - The resulting RBG color pixel data are applied to a
format scalar 166, which interpolates to a higher pixel resolution as required for the display. For example, VGA format data (640×480 pixels) is scaled to SVGA format data (800×600 pixels).Format scaler 166 consists substantially of a conventional image scaling and anti-aliasing IC, for example, Genesis Microchip, Inc., 2071 Landings Drive, Mountain View, Calif. 94043, Model gmZ1 Scaler. Scaling algorithms familiar in the art are used to minimize video artifacts created during the scaling process. - After scaling, the RGB pixel data are applied to a
scan rate converter 168, which converts the video input frame rates of 50/60 Hz to 83.3/80 Hz to minimize the potential for visible flicker in the display.Scan rate converter 168 then outputs the frame data in separate red, green, and blue fields.Scan rate converter 168 is implemented using a Field Programmable Gate Array (FPGA), for example Xilinx, Inc., 2100 Logic Drive, San Jose, Calif., Model 4010 PQ160 FPGA, and a synchronous high speed SDRAM. -
FIG. 2B is a functional block diagram illustrating the operation ofscan rate converter 168. The digital RGBcolor pixel data 190 fromformat scaler 166 entersscan rate converter 168 from the left and is stored in one of twomemory banks memory bank pixel store function 196 stores video frames alternately inmemory bank - Whereas input
pixel store function 196 writes data intomemory banks function 198 reads data from therespective memory bank functions same memory bank - A scan rate
control logic function 199 accepts data at a 50/60 Hz rate supplied by a video source and outputs the data at a higher rate, namely, 83.3/80 Hz. Scan ratecontrol logic function 199 controls the flow of data in the sequential red, green, and blue fields, instructing output pixel fetchfunction 198 to insert repeating color fields to convert the effective frame rate. In some embodiments,headset display 140 requires four pixels of data to be loaded during each input cycle. In these embodiments, output pixel fetchfunction 198 is instructed to read four pixels, e.g., 32 bits, simultaneously and to present these pixel data in a burst format to apixel data serializer 170. - Red, green, and blue color field data are serialized in pixel data serializer 170 (see
FIG. 2A ) into a stream suitable for transmission over a wireless link. In addition to serialization, pixel data serializer 170 inserts video synchronization information to define horizontal, vertical, and color syncs. Pixel data serializer 170 converts the parallel digital pixel color data into amplitude modulated signals using conventional digital to analog converters (DAC's). It then outputs the analog signals, e.g., four pixel values, into a high speed serial data stream suitable for transmission over the wireless (serial) link. Conventional sample-and-hold amplifiers are used to delay the analog signals for conversion to a serial analog data stream, as described above in connection withFIG. 1C . - Referring to
FIG. 2A , optionally, aninput audio signal 171 from a microphone or other conventional source (not shown) enters aninput amplifier IC 172 with level control and high frequency boost to improve overall S/N ratio. The audio signal is then applied to a pulseamplitude audio modulator 174 which converts the amplitude modulated signal to an amplitude pulse. This pulse is inserted just prior to the beginning of video data within each horizontal scan as determined by sync, timing andblack level generator 176, which controls all system timing. The generator's 176 timing functions are mirrored in exact synchronization bygenerator 334 ofFIG. 3D in the remote receiver, i.e., headset, for extraction and reconstruction of data. All timing and reference level data is combined with the analog-converted digital pixel data in D/A Converter &Buffer 178, which consists of a high-speed video quality D/A (digital/analog) converter, summing amplifier, and video line driver. When combined with the pixel data, the audio signal is recovered from the video via sampling using reconstructed timing information within the remote receiver. The audio pulse amplitude is then conventionally converted to a normal analog audio signal, e.g., for headphones, using a sample-and hold circuit indetector 324 ofFIG. 3D . - The combined serial video data and optional modulated audio data from D/A Converter and
Buffer 178 is transmitted through a bundle of coaxial cables toinfrared LED array 130.IR LED array 130 is selected for its high frequency modulation capability, which transmits the signal as an amplitude modulated IR beam. In some of such embodiments,infrared LED array 130 comprises one or more high-frequency infrared LED's, for example, Infineon, Inc., 10950 North Tantau Avenue, Cupertino, Calif. 95014 Model SFH4200 series of wide-angle SMT devices, or model SFH4500 series of lensed T1-3/4 devices. - Optionally, a
return audio signal 151 from headset 104 (transmitted through the reverse-path free space infrared link) is received by photodiode andpreamplifier 156, which converts the modulated optical signal to a low level modulated electrical signal, amplifies the low level signal, buffers it for transmission overcoaxial cable 182, and applies it to pulseamplitude audio demodulator 157, which recovers the audio signal using sample-and-hold technology driven bytiming generator 176. The audio signal is frequency conditioned to improve the signal to noise ratio at high frequency and to restore the overall audio fidelity of the signal. Finally, the audio signal is processed by a line driver IC amplifier withlevel control circuits 159 for output coupling to conventional audio output equipment (not shown). - In another embodiment, the return audio signal is digitized with a simple A/D converter within
headset 104. It is then formatted, stored and restructured digitally by a PIC controller withintiming generator 334 ofFIG. 3D as a series of full-amplitude pulses occupying a time slot normally used for a full horizontal line of video; for example, every eighth line. The video normally occupying these lines is then blanked and delayed for IR transmission until the following lines, thereby adding 600/8 or 75 lines to the SVGA format. As with other timing functions, the video blanking and audio pulse formatting is controlled inbase station 160 ofFIG. 2A bygenerator 176 and synchronously controlled byheadset timing generator 334 inFIG. 3D . Audio information is transmitted as an IR series of pulses in the same way byheadset LEDs 306 inFIG. 3A , received by collecting lens andlightcone 252 intranceiver module 110, and sent throughcoaxial cable 151 incable bundle 182 tobase station 112, where it is digitally decoded and converted to analog audio for equivalent use. While this digital embodiment has the disadvantage of increasing the video bandwidth by 675 lines/600 lines, or 12.5%, it has the advantage of decreasingheadset LEDs 306 power requirements for equivalent S/N ratio to the pulse amplitude format; thereby extending the operational life ofheadset batteries 104. - In
FIGS. 1A and 1B ,transceiver module 110 is used to distribute (broadcast) combined audio and video signals frombase station 112 toheadset 104.Transceiver module 110 also serves as a collection and relay module for optional IRaudio signal 120 returning fromheadset 104. In the embodiment shown inFIG. 1A ,transceiver module 110 is located below the area whereheadset 104 is used. Alternatively, in the embodiment shown inFIG. 1B ,transceiver module 110 is located above and proximate to the area whereheadset 104 is used. This placement optimizes the signal distribution and provides the best image quality by minimizing reflections along the IR signal path.Transceiver module 110 can be implemented with a bundle ofcables 182 frombase station 112. -
FIG. 2C is a block diagram oftranceiver module 110. The combined video and optional audio infrared signals frombase station 112 are carried bycoaxial cable 270 through bundle ofcables 182 to anIR LED array 130. TheIR LED array 130 transmitsIR signal pattern 132 towardremote receiver 146, e.g.,headset 104. - Optionally, a return modulated audio IR signal 120 from
headset 104 is collected by a collecting lens andlight cone 252 intransceiver module 110, and the collected light directed to a photodiode andamplifier 254. Photodiode andamplifier 254 convert and amplify the optical signal to produce an electric audio signal, which drivescoaxial cable 151 through bundle ofcables 182 tobase station 112. -
FIG. 2D is a more detailed transceiver schematic block diagram, in accordance with a further embodiment of the present invention.Transceiver circuit 110 is connected withbase station 112 through electrically conductingcoaxial cable 182, which carries respective video signals 270,audio signals 151, and DCelectrical power 272.DC power 272 fromcoaxial cable 182 is filtered from audio and video signals by a low pass filter andpower supply 260, which provides power for the active elements oftransceiver circuit 110.Video signal 270 fromcoaxial cable 182 is filtered through ahigh pass filter 262 and is applied through a DC level restoremodule 266 toLED drivers 278, which drive anIR LED array 130. The IR LED array producesIR beam 132. - Optional return audio signal on
return IR beam 120 is transmitted through collecting lens andlightcone 252 onto photodiode andamplifier 254 where it is converted into an electrical signal. The audio electrical signal from photodiode andamplifier 254 is amplified by aline driver 258 and is then filtered through abandpass filter 264 prior to transmission asaudio carrier signal 151 throughcoaxial cable 182 tobase station 112. -
FIG. 2E is a more detailed transceiver schematic block diagram, in accordance with a further embodiment of the present invention.Transceiver circuit 110 is connected withbase station 112 through a bundle ofcables 186, which comprises a videocoaxial cable 184 carryingvideo signals 270, an audiocoaxial cable 183 carryingaudio signals 151, andpower cables 185 carrying DCelectrical power 272.Video signal 270 from videocoaxial cable 184 is applied toLED drivers 278, which drives anIR LED array 130. The IR LED array produces anIR beam 132. - Optional return audio signal on
return IR beam 120 is transmitted through collecting lens andlightcone 252 onto photodiode andamplifier 254 where it is converted into an electrical signal. The audio electrical signal from photodiode andamplifier 254 is amplified by aline driver 258 and is transmitted asaudio carrier signal 151 through audiocoaxial cable 183 tobase station 112. - The optical components mounted at
headset 104 are complementary to those mounted attransceiver module 110.FIG. 3A is a schematic front view of anIR module 300 containing components ofIR video interface 100 incorporated inheadset 104.FIGS. 3B and 3C are top and side schematic views, respectively, of a user wearing an embodiment ofheadset 104 includingIR module 300 is mounted away from the user's peripheral vision field and above the LCD and associated display optics (see Hebert, cited above), thereby providing a substantially unobstructed wide angle reception path to the ceiling or to an overhead transmitting LED array. -
FIG. 3D is a functional block diagram of remoteelectronic circuit 142, located for example atheadset 104. The combined video and optional audio low level electronic signals fromIR photodetector 304 enter adjacent to the middle left hand side ofFIG. 3D . The signals are applied to preamplifier/AGC/audiocarrier filter module 305 where the signal is amplified and level adjusted by a conventional automatic gain control (AGC) circuit. Black level control restores the correct baseline DC voltage to the video waveform. - The composite output from
preamp module 305 is further separated into three signals; a video signal portion byprocessor 330, synchronization components bystripper 332, and the optional audio component bydetector 324. The synchronization components are applied to aheadset timing generator 334, which generates signals that control display, LED illumination sub-systems, and optional audio sub-system. - The separated video signal portion continues into a
video processing module 330, where gamma correction is performed using conventional techniques. Gamma correction compensates for the different response ofheadset display 140 relative to the response of a traditional phosphor based CRT. Gamma correction adjusts the video signal, such thatheadset display 140 exhibits a more accurate range of brightness than would be realized without gamma correction. - The optional audio carrier portion is separated from the video via pulse amplitude detector 324 (e.g., sample-and-hold) using reconstructed timing information from headset timing generated 104. The recovered audio signal is then applied to a
headphone amplifier 326 configured for driving a conventional dynamicheadphone speaker element 328. -
FIG. 3E is a detailed functional block diagram ofheadset timing generator 334, which is implemented with a field programmable gate array (FPGA), similar to that described in connection withFIG. 2A . The outputs fromheadset timing generator 334 include control signals fordisplay drive module 336,display LEDs 340, and optional audio pulse separation. In operation, display 140 cycles through erase, load, and illuminate phases.Timing generator 334 is responsible for correctly sequencing these phases. Composite sync components fromsync stripper module 332, including horizontal start pulses and field start pulses, are applied toheadset timing generator 334. Using the horizontal start pulses, a horizontal timer andpixel counter module 350 locates the beginnings of lines and counts the pixels within each line, for example, 800 active pixels per line ofdisplay 140 plus inactive pixels used for timing and control. A timing window is generated at the expected field start pulse time (with respect to the horizontal start pulse), that is used by field start detection andcolor identification module 354 to detect a start of field condition. When start of field is detected, avertical timing module 356 is reset to zero and commences counting lines in a field, for example, 600 lines. During the vertical timing interval, a small number of non-video lines fromvideo processor module 330 are encoded with color identifiers. These color identifiers are detected by field start detection andcolor identification module 354 and are used by a RGBLED control module 352 to synchronizedisplay LEDs 340 with the appropriate color fields. Vertical timing pulses fromvertical timing module 356 and color IDs from field start detection andcolor identification module 354 are used by displaysync control module 358 to generate signals that control the operation of adisplay drive module 336 andmicrodisplay 140. -
Display drive module 336 demultiplexes the video pixels, converting serial pixel analog voltages to parallel analog voltages suitable for driving a display.Microdisplay 140 and associateddrivers 336 receive video data fromvideo processor module 330 and control signals fromheadset timing generator 334. These are used to load the sequential color field data, which are stored in memory cells internal tomicrodisplay 140. After the data are loaded and stabilized, a selected red, green, orblue LED 340 is pulsed to illuminatedisplay 140 with the correct color. - The overall brightness of
display 140 is adjusted using an optional ambientlight detector circuit 342. Light from the room is measured by aphotodetector 308 to provide a control signal to boost or reduce the average brightness level ofLEDs 340. - An optional return audio signal is generated by a
microphone 360 mounted atheadset 104, and is processed by audio electronics including amicrophone amplifier 361 and an audiopulse amplitude modulator 362, in a manner similar to that described in connection withFIG. 2A . The processed return audio signal drivesinfrared LEDs 306, thereby generating modulatedreturn IR beam 120, which is transmitted through the atmosphere totransceiver module 110, where it is collected and relayed tobase station 112. - In an alternative return audio implementation, the return audio signal is digitized with a simple A/D converter within
headset 104. It is then formatted, stored and restructured digitally by a PIC controller withintiming generator 334 ofFIG. 3D as a series of full-amplitude pulses occupying a time slot normally used for a full horizontal line of video; for example, every eighth line. The video normally occupying these lines is then blanked and delayed for IR transmission until the following lines, thereby adding 600/8 or 75 lines to the SVGA format. As with other timing functions, the video blanking and audio pulse formatting is controlled inbase station 160 ofFIG. 2A bygenerator 176 and synchronously controlled byheadset timing generator 334 inFIG. 3D . Audio information is transmitted as an IR series of pulses in the same way byheadset LEDs 306 inFIG. 3A , received by collecting lens andlightcone 252 intranceiver module 110, and sent throughcoaxial cable 151 incable bundle 182 tobase station 112, where it is digitally decode and converted to analog audio for equivalent use. While this digital embodiment has the disadvantage of increasing the video bandwidth by 675 lines/600 lines, or 12.5%, it has the advantage of decreasingheadset LEDs 306 power requirements for equivalent S/N ratio to the pulse amplitude format; thereby extending the operational life ofheadset batteries 104. -
FIG. 4 is a schematic diagram ofLED array 130, in accordance with an embodiment of the present invention.FIG. 4 shows acluster 132 of high-speed infrared-emittingLEDs 131 in the array with eachLED 131 emitting an identical optical signal.FIG. 2D shows theLED array 130 is interconnected with thevideo processing circuit 160 inbase station 112 through a singlecoaxial cable 182 or through one or moreelectrical cables 186. - In one embodiment, each
cluster 132 consists of fiveLEDs 131 driven in series from a common modulated electrical source throughelectronic buffers 133 from a common +12volt power supply 260. The common modulated electrical source is electricalcoaxial cable 182, as shown inFIG. 2D . Alternatively, the common modulated electrical source is a singlecoaxial cable 184 in bundle ofcables 186, as inFIG. 2E . Theclusters 132 ofLEDs 131 and their common modulatedelectronic driver sources 133 are driven in parallel from the common modulated electrical source. This invention encompasses clusters with more than five LEDs and clusters with less than five LEDs. The optimum number ofLEDs 131 in thearray 130 depends on the desired minimum S/N ratio at the desired maximum range of separation betweentranceiver 110 andremote receiver 302. To those skilled in the art, it is generally understood that random noise from silicon detectors such asdetector 304 inFIG. 3A is nearly constant, while signal strength generally falls off with the square of the range of separation. Therefore, doubling the number ofLEDs 131 inarray 130 will generally increase the S/N ratio by the square root of 2, or 44%, at a given range. Typically, the desired S/N ratio is reached at a ratio where an increase in the S/N ratio is no longer noticeable to the eyes ofuser 102. This is generally in excess of 40 db. - In one embodiment, pairs of
clusters 132 are physically arranged in an electronic dipole configuration so that the electromagnetic field from one cluster tends to cancel out the electromagnetic field from the other cluster. The cancellation of the electromagnetic fields minimizes spurious electronic emissions. -
FIG. 4 shows a partially loaded circuit board with spaces 134 with loadedLEDs 131 andspaces 135 with unloaded LEDs. When the circuit board is partially loaded, theclusters 132 ofLEDs 131 are symmetrically laid out as pairs of clusters so as to further minimize spurious electromagnetic radiation. Alternatively, the circuit board is fully loaded withLEDs 131. The spatially distributed nature of thearray 130 minimizes the possibility of total blockage of the signal path to theheadset 104 from an obstruction, such as a swinging arm of a surgical lighting system. -
FIG. 5A is a cross-sectional schematic diagram of collectinglens assembly 302 atheadset 104, in accordance with an embodiment of the present invention. Collectinglens assembly 302 is configured to achieve both a large angular field ofview 510, e.g., to accommodate head motion, and a large entrance pupil to receive maximum energy from IR signal 118 or 132. Although collectinglens assembly 302 is nominally rotationally symmetric about asymmetry axis 512, it can alternatively be configured to provide an azimuthally variable detectivity pattern aboutaxis 512. -
Photodetector 304 can be any photosensitive device having the optical and electronic responses required for the application, but is shown inFIG. 5A as a silicon device selected for its high frequency electronic performance and for high sensitivity to infrared radiation over a wavelength region of approximately 700 nm to approximately 1100 nm. - The collecting
lens assembly 302 consists of three coaxial components: arefractive lens 518 with a conic sidewall and planar exit pupil, aninner cone 515 with a diffusely reflective sidewall mating to lens' 518 conic sidewall, and anouter cone 520 with smooth inner reflective walls extended above the lens which reflects IR signal 118 or 132 intolens 518 andinner light cone 514 over a wider aperture.Inner walls 521 ofouter cone 520 are oriented, for example, at an angle of approximately 19 degrees to approximately 27 degrees relative tosymmetry axis 512.Photodetector 304 is optically cemented to the lens' 518 planar exit pupil to couple some energy intophotodetector 304 that would otherwise be lost to total internal reflection withinlens 518. This combination of elements dramatically enhances photodetector's 304 effective apperture over a wide angular field of view, thereby permitting the use of a smaller, wider bandwidth, less expensive photodetector to collect sufficient levels of infrared signal. - In some embodiments, one or more optical elements of the collecting lens assembly are aspheric. Collecting
lens 518 has an aspheric first surface with a numerical aperture of approximately f/0.8, which provides a large entrance pupil. Transmissive optical elements are typically made from transparent polymeric material, for example polymethyl methacrylate, polycarbonate, and ULTEM® grade polycarbonate manufactured by General Electric Company. Reflective elements can be made of any material that is reflective in the infrared spectrum of interest; the preferred embodiment being a plastic such as polycarbonate that is reflectively overcoated with aluminum. - Collecting lens and
light cone 252 has substantially the same structure as collectinglens assembly 302. -
FIGS. 5B-5D are cross-sectional schematic diagrams illustrating the transmission and capture of IR radiation incident from various angles onto collectinglens assembly 302.FIG. 5B shows the case ofradiation 530 incident normal to the plane ofphotodetector 304. This radiation is captured efficiently by wide-angle lens 518 alone.FIG. 5C shows the case ofradiation 532 incident at a 30-degree angle from normal, which is efficiently captured by wide-angle lens 518 andinner light cone 514 cooperatively.FIG. 5D shows the case ofradiation 534 incident at 40-degree angle from normal. In this situation, outerconic cavity 520 reflects radiation into wide-angle lens 518 that would otherwise be lost. This reclaimed radiation is then efficiently captured by wide-angle lens 518 andinner light cone 514 cooperatively. - An optional prismatic dispersion plate overlying the outer conic cavity further widens the collecting angle of the assembly.
FIGS. 5E and 5F are cross-sectional schematic diagrams of a collectinglens assembly 302 withprismatic dispersion plate 560. In one embodiment, theprismatic dispersion plate 560 overlies the outerconic cavity 520, as shown inFIG. 5E . Alternatively, the prismatic dispersion plate is molded together and integrated with the outer headset cover, an infraredpass filter window 522, as shown inFIG. 5F . The effect of this dispersion is further illustrated inFIG. 3C with the IR reception angle being widened from 80 degrees to 120 degrees, for example. The prismatic dispersion plate is typically made of an optically transmissive polymeric material (for example ULTEM® grade polycarbonate manufactured by the General Electric Company). -
FIG. 5G is a plane view illustrating aprismatic dispersion plate 560. Thegrooves 562 widen the angular field ofvision 510, or collection angle, of the collecting lens assembly. Theplate 560 has both vertical 580 and horizontal 582 grooves.Horizontal grooves 580 widen the angular field ofvision 510 during heads up-down motion, whilevertical grooves 582 widen the angular field ofvision 510 in side-to-side head motion. In one embodiment, the prismatic pattern is widened symmetrically with an equal number ofhorizontal grooves 582 andvertical grooves 580. Alternatively, the angular field ofvision 510, or collection angle, is widened asymmetrically with an asymmetrical prismatic pattern. One embodiment has morehorizontal grooves 582 thanvertical grooves 580 to favor heads up-down motion over side-to-side head motion. Alternatively, side-to-side head motion is favored over heads up-down motion by having morevertical grooves 580 thanhorizontal grooves 582. -
FIG. 5H illustrates the transmission and capture of IR radiation incident from various angles onto theprismatic dispersion plate 560. IR rays 564 and 566 are within the normal angular range of the assembly (“on axis”), whileIR ray 568 is an extreme angle outside the normal angular range of the assembly (“off axis”). On-axis IR ray 564 enters theprismatic dispersion plate 560 atsurface 570 and is refracted slightly before exitingplate 560 atsurface 572. On-axis IR ray 564 also enters theplate 560 atsurface 570, but is internally reflected bygroove 562, so it is does not reach theuser 102. The grooves decrease on-axis sensitivity by internally reflecting some of the on-axis rays. However, the grooves enhance off-axis sensitivity. Off-axis IR ray 568 would normally not be within the angular range of the assembly. However, when an off-axis IR ray 568 enters theplate 560 atsurface 570, theray 568 is refracted bygroove 562 to be within the angular acceptance of thecollection lens assembly 302. - An optional optical bandpass filter is positioned over the assembly to minimize undesirable radiation (i.e., noise) outside of the infrared spectrum that constitutes the audio/video signal. An
optical filter 522, such as KODAK WRATTEN® No. 87B, covers the entrance aperture of outerconic cavity 520. Alternatively, an optical filtering material such as an absorptive dye is incorporated into the material of wide-angle collecting lens 518 ordispersion plate 560. -
FIG. 5I is a graphic representation of the calculated radiative capture by various elements of collectinglens assembly 302, relative to the radiative capture byphotodetector 304 absent the other elements of collectinglens assembly 302. Relative radiative capture is shown along the vertical axis, and off-axis an incidence relative tosymmetry axis 512 is shown along the horizontal axis. The baseline radiative capture of identically 1.0 byunaided photodetector 304 is shown ascurve 540.Curve 542 shows the combined relative radiative capture byphotodetector 304 and wide-angle collecting lens 518.Curve 544 shows the combined relative radiative capture byphotodetector 304, wide-angle collecting lens 518, andinner light cone 514.Curve 546 shows the combined relative radiative capture by entire collectinglens assembly 302, includingphotodetector 304, wide-angle collecting lens 518,inner light cone 514, and outerconic cavity 520. As shown inFIG. 5C , the radiative capture by completecollecting lens assembly 302 relative tounaided photodetector 304 exceeds a factor of 15 on-axis and approximates a factor of four at a 40-degree angle off-axis.Curve 548 shows the relative radiative capture by thevertical grooves 580 in theprismatic dispersion plate 560.Curve 550 shows the relative radiative capture by thehorizontal grooves 582 in theprismatic dispersion plate 560. The prismatic dispersion plate increases off-axis enhancement at the cost of on-axis sensitivity. - Referring to
FIG. 1B , in an alternative embodiment, a tether 124 (shown dashed) is connected directly frombase station 112 to remoteelectronic circuit 142 atheadset 104, where it delivers the combined audio/visual signal. Tether 124 can contain either a single or dual optical fiber cable, such that a second optical fiber transmits returnIR signal 120. Alternatively,tether 124 contains a bi-directional, electrically conducting coaxial cable. This configuration greatly simplifiesIR video interface 100 by eliminating most components oftransceiver module 110 and collectinglens assembly 302. The coaxial cable implementation additionally eliminates all optical and optoelectronic components. It also improves signal to noise performance at reduced power. However, the direct tether connection toheadset 104 adversely restricts the freedom of motion ofuser 102. - Although a head-mounted display is described above, it will be understood by those having ordinary skill in the art that the principles, methods, and apparatus described herein can be applied as well to remote displays that are not head-mounted.
- While embodiments of the present invention have been shown and described, changes and modifications to these illustrative embodiments can be made without departing from the present invention in its broader aspects. Thus it should be evident that there are other embodiments of this invention which, while not expressly described above, are within the scope of the present invention. Therefore, it will be understood that the appended claims necessarily encompass all such changes and modifications as fall within the described invention's true scope; and further that this scope is not limited merely to the illustrative embodiments presented to demonstrate that scope.
Claims (63)
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US09/756,648 US6900777B1 (en) | 2001-01-03 | 2001-01-03 | Infrared audio/video interface for head-mounted display |
PCT/US2002/000157 WO2002054959A2 (en) | 2001-01-03 | 2002-01-03 | Infrared audio-video interface for head-mounted display |
AU2002243461A AU2002243461A1 (en) | 2001-01-03 | 2002-01-03 | Infrared audio-video interface for head-mounted display |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US09/756,648 US6900777B1 (en) | 2001-01-03 | 2001-01-03 | Infrared audio/video interface for head-mounted display |
Publications (2)
Publication Number | Publication Date |
---|---|
US20050104802A1 true US20050104802A1 (en) | 2005-05-19 |
US6900777B1 US6900777B1 (en) | 2005-05-31 |
Family
ID=25044429
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US09/756,648 Expired - Lifetime US6900777B1 (en) | 2001-01-03 | 2001-01-03 | Infrared audio/video interface for head-mounted display |
Country Status (3)
Country | Link |
---|---|
US (1) | US6900777B1 (en) |
AU (1) | AU2002243461A1 (en) |
WO (1) | WO2002054959A2 (en) |
Cited By (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20040030219A1 (en) * | 2002-08-06 | 2004-02-12 | Korea Institute Of Science And Technology | Endoscope system |
US20040190750A1 (en) * | 1999-05-19 | 2004-09-30 | Rodriguez Tony F. | Watermarked printed objects and methods |
DE102006001079A1 (en) * | 2006-01-09 | 2007-07-12 | Carl Zeiss Ag | Head mounted display device, has image processing unit for producing interpolated complete picture from one of extracted semi-images, and display device displaying interpolated picture as virtual image to user carrying display device |
US20150085186A1 (en) * | 2013-09-24 | 2015-03-26 | Marc R. Amling | Simultaneous Display of Two or More Different Sequentially Processed Images |
US20150287351A1 (en) * | 2014-04-08 | 2015-10-08 | Apple Inc. | System and method of reducing power using a display inactive indication |
US20150346700A1 (en) * | 2014-06-02 | 2015-12-03 | Rovio Entertainment Ltd | Control of a computer program |
KR20160093039A (en) * | 2013-11-25 | 2016-08-05 | 테세랜드 엘엘씨 | Immersive compact display glasses |
Families Citing this family (47)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7035286B2 (en) * | 2000-11-14 | 2006-04-25 | Broadcom Corporation | Linked network switch configuration |
US8411151B2 (en) * | 2002-03-29 | 2013-04-02 | IQinVision, Inc. | System for, and method of, processing an image |
WO2003083875A1 (en) * | 2002-04-03 | 2003-10-09 | Schem Nz Limited | Remote monitoring apparatus |
US7129910B2 (en) * | 2002-07-10 | 2006-10-31 | Hewlett-Packard Development Company, L.P. | Active display system and method with optical addressing |
DE202004010922U1 (en) * | 2004-07-12 | 2005-11-24 | Leica Geosystems Ag | tilt sensor |
WO2006039522A2 (en) * | 2004-09-30 | 2006-04-13 | Boston Scientific Scimed, Inc. | Adapter for use with digital imaging medical device |
US8121534B2 (en) * | 2006-09-27 | 2012-02-21 | Sirius Xm Radio Inc. | System and method for sending and receiving images via wireless audio channels |
US8378924B2 (en) * | 2007-01-12 | 2013-02-19 | Kopin Corporation | Monocular display device |
US9217868B2 (en) * | 2007-01-12 | 2015-12-22 | Kopin Corporation | Monocular display device |
JP2010527557A (en) | 2007-05-14 | 2010-08-12 | コピン コーポレーション | Mobile radio display for accessing data from host and method for controlling the same |
US9235262B2 (en) | 2009-05-08 | 2016-01-12 | Kopin Corporation | Remote control of host application using motion and voice commands |
US8855719B2 (en) * | 2009-05-08 | 2014-10-07 | Kopin Corporation | Wireless hands-free computing headset with detachable accessories controllable by motion, body gesture and/or vocal commands |
US8825468B2 (en) * | 2007-07-31 | 2014-09-02 | Kopin Corporation | Mobile wireless display providing speech to speech translation and avatar simulating human attributes |
US8355671B2 (en) * | 2008-01-04 | 2013-01-15 | Kopin Corporation | Method and apparatus for transporting video signal over Bluetooth wireless interface |
US9886231B2 (en) | 2008-03-28 | 2018-02-06 | Kopin Corporation | Head worn wireless computer having high-resolution display suitable for use as a mobile internet device |
US8600761B2 (en) * | 2008-09-09 | 2013-12-03 | The Boeing Company | Hands-free and non-visually occluding object information interaction system |
WO2011097564A1 (en) * | 2010-02-05 | 2011-08-11 | Kopin Corporation | Touch sensor for controlling eyewear |
US9377862B2 (en) | 2010-09-20 | 2016-06-28 | Kopin Corporation | Searchlight navigation using headtracker to reveal hidden or extra document data |
US8706170B2 (en) | 2010-09-20 | 2014-04-22 | Kopin Corporation | Miniature communications gateway for head mounted display |
US10013976B2 (en) | 2010-09-20 | 2018-07-03 | Kopin Corporation | Context sensitive overlays in voice controlled headset computer displays |
US9316827B2 (en) | 2010-09-20 | 2016-04-19 | Kopin Corporation | LifeBoard—series of home pages for head mounted displays (HMD) that respond to head tracking |
US8736516B2 (en) | 2010-09-20 | 2014-05-27 | Kopin Corporation | Bluetooth or other wireless interface with power management for head mounted display |
US8862186B2 (en) | 2010-09-21 | 2014-10-14 | Kopin Corporation | Lapel microphone micro-display system incorporating mobile information access system |
CN103238339B (en) | 2010-12-02 | 2015-12-09 | 尤特瑞登特生产公司 | Check and follow the tracks of the system and method for stereoscopic video images |
CN109116985A (en) | 2011-05-10 | 2019-01-01 | 寇平公司 | The method that control information is shown |
US8929954B2 (en) | 2012-04-25 | 2015-01-06 | Kopin Corporation | Headset computer (HSC) as auxiliary display with ASR and HT input |
US9442290B2 (en) | 2012-05-10 | 2016-09-13 | Kopin Corporation | Headset computer operation using vehicle sensor feedback for remote control vehicle |
US9378028B2 (en) | 2012-05-31 | 2016-06-28 | Kopin Corporation | Headset computer (HSC) with docking station and dual personality |
AU2013202775B2 (en) | 2012-06-01 | 2015-09-17 | Ultradent Products, Inc. | Stereoscopic video imaging |
US9998687B2 (en) | 2012-09-12 | 2018-06-12 | Bae Systems Information And Electronic Systems Integration Inc. | Face mounted extreme environment thermal sensor system |
USD713406S1 (en) | 2012-11-30 | 2014-09-16 | Kopin Corporation | Headset computer with reversible display |
US9160064B2 (en) | 2012-12-28 | 2015-10-13 | Kopin Corporation | Spatially diverse antennas for a headset computer |
US9134793B2 (en) | 2013-01-04 | 2015-09-15 | Kopin Corporation | Headset computer with head tracking input used for inertial control |
JP6423799B2 (en) | 2013-01-04 | 2018-11-14 | コピン コーポレーション | Ad hoc network |
US9301085B2 (en) | 2013-02-20 | 2016-03-29 | Kopin Corporation | Computer headset with detachable 4G radio |
USD737955S1 (en) | 2013-03-15 | 2015-09-01 | Bae Systems Information And Electronic Systems Integration Inc. | Gas mask with thermal sensor |
US9326661B2 (en) * | 2013-11-18 | 2016-05-03 | Gyrus Acmi, Inc. | Line of sight wireless endoscopy |
GB2536650A (en) | 2015-03-24 | 2016-09-28 | Augmedics Ltd | Method and system for combining video-based and optic-based augmented reality in a near eye display |
JP2019505843A (en) | 2016-01-22 | 2019-02-28 | コーニング インコーポレイテッド | Wide-view personal display device |
US11071596B2 (en) | 2016-08-16 | 2021-07-27 | Insight Medical Systems, Inc. | Systems and methods for sensory augmentation in medical procedures |
US10439657B2 (en) * | 2017-08-03 | 2019-10-08 | Intel Corporation | Overhead communications with wireless wearable devices |
US10976551B2 (en) | 2017-08-30 | 2021-04-13 | Corning Incorporated | Wide field personal display device |
US10728518B2 (en) * | 2018-03-22 | 2020-07-28 | Microsoft Technology Licensing, Llc | Movement detection in low light environments |
US10565720B2 (en) | 2018-03-27 | 2020-02-18 | Microsoft Technology Licensing, Llc | External IR illuminator enabling improved head tracking and surface reconstruction for virtual reality |
DE102019134142A1 (en) * | 2018-12-12 | 2020-06-18 | Analog Devices, Inc. | BUILT-IN CALIBRATION OF RUN TIME DEPTH IMAGING SYSTEMS |
US11382712B2 (en) | 2019-12-22 | 2022-07-12 | Augmedics Ltd. | Mirroring in image guided surgery |
US11896445B2 (en) | 2021-07-07 | 2024-02-13 | Augmedics Ltd. | Iliac pin and adapter |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5710671A (en) * | 1995-09-05 | 1998-01-20 | Temic Telefunken Microelectronic Gmbh | Optical system |
US6101038A (en) * | 1999-05-03 | 2000-08-08 | Optimize, Incorporated | Infrared audio/video interface for head-mounted display |
US6154300A (en) * | 1996-11-12 | 2000-11-28 | Samsung Electronics Co., Ltd. | Wireless data communication apparatus using the diffused infrared-ray antenna |
US6430433B1 (en) * | 1999-09-07 | 2002-08-06 | Carl-Zeiss-Stiftung | Apparatus for image-supported treatment of a work object |
Family Cites Families (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US3674925A (en) * | 1970-12-03 | 1972-07-04 | Us Navy | Cable-less television system |
US5005213A (en) * | 1986-07-10 | 1991-04-02 | Varo, Inc. | Head mounted video display and remote camera system |
JPH01167739U (en) | 1988-05-18 | 1989-11-27 | ||
US5579026A (en) * | 1993-05-14 | 1996-11-26 | Olympus Optical Co., Ltd. | Image display apparatus of head mounted type |
US5619183A (en) * | 1994-09-12 | 1997-04-08 | Richard C. Ziegra | Video audio data remote system |
US5671158A (en) * | 1995-09-18 | 1997-09-23 | Envirotest Systems Corp. | Apparatus and method for effecting wireless discourse between computer and technician in testing motor vehicle emission control systems |
JPH09266460A (en) * | 1996-03-28 | 1997-10-07 | Sony Corp | Transmission reception system |
US6046712A (en) * | 1996-07-23 | 2000-04-04 | Telxon Corporation | Head mounted communication system for providing interactive visual communications with a remote system |
EP0929836B1 (en) * | 1996-09-30 | 2000-08-09 | Siemens Aktiengesellschaft | Method for producing a beam-splitting shaped body and use of the beam-splitting shaped body in an opto-electronic module |
US5745519A (en) * | 1996-11-12 | 1998-04-28 | Opto Power Corp. | Laser diode system |
JP3217723B2 (en) * | 1997-03-13 | 2001-10-15 | ▲すすむ▼ 舘 | Telecommunications system and telecommunications method |
JPH11265248A (en) * | 1998-03-18 | 1999-09-28 | Sony Corp | Spectacle type picture display device |
US5926318A (en) * | 1998-04-06 | 1999-07-20 | Optimize Incorporated | Biocular viewing system with intermediate image planes for an electronic display device |
-
2001
- 2001-01-03 US US09/756,648 patent/US6900777B1/en not_active Expired - Lifetime
-
2002
- 2002-01-03 WO PCT/US2002/000157 patent/WO2002054959A2/en not_active Application Discontinuation
- 2002-01-03 AU AU2002243461A patent/AU2002243461A1/en not_active Abandoned
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5710671A (en) * | 1995-09-05 | 1998-01-20 | Temic Telefunken Microelectronic Gmbh | Optical system |
US6154300A (en) * | 1996-11-12 | 2000-11-28 | Samsung Electronics Co., Ltd. | Wireless data communication apparatus using the diffused infrared-ray antenna |
US6101038A (en) * | 1999-05-03 | 2000-08-08 | Optimize, Incorporated | Infrared audio/video interface for head-mounted display |
US6430433B1 (en) * | 1999-09-07 | 2002-08-06 | Carl-Zeiss-Stiftung | Apparatus for image-supported treatment of a work object |
Cited By (16)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20040190750A1 (en) * | 1999-05-19 | 2004-09-30 | Rodriguez Tony F. | Watermarked printed objects and methods |
US20040030219A1 (en) * | 2002-08-06 | 2004-02-12 | Korea Institute Of Science And Technology | Endoscope system |
US7022068B2 (en) * | 2002-08-06 | 2006-04-04 | Korea Institute Of Science And Technology | Endoscope system with a hollow cylinder and a bellows moving mechanism |
DE102006001079A1 (en) * | 2006-01-09 | 2007-07-12 | Carl Zeiss Ag | Head mounted display device, has image processing unit for producing interpolated complete picture from one of extracted semi-images, and display device displaying interpolated picture as virtual image to user carrying display device |
DE102006001079B4 (en) | 2006-01-09 | 2020-08-06 | Carl Zeiss Ag | HMD device and display method therefor |
US9270919B2 (en) * | 2013-09-24 | 2016-02-23 | Karl Storz Imaging, Inc. | Simultaneous display of two or more different sequentially processed images |
US9948881B2 (en) | 2013-09-24 | 2018-04-17 | Karl Storz Imaging, Inc. | Simultaneous display of two or more different sequentially processed images |
US20150085186A1 (en) * | 2013-09-24 | 2015-03-26 | Marc R. Amling | Simultaneous Display of Two or More Different Sequentially Processed Images |
KR20160093039A (en) * | 2013-11-25 | 2016-08-05 | 테세랜드 엘엘씨 | Immersive compact display glasses |
US20170171533A1 (en) * | 2013-11-25 | 2017-06-15 | Tesseland Llc | Immersive compact display glasses |
US10432920B2 (en) * | 2013-11-25 | 2019-10-01 | Tesseland, Llc | Immersive compact display glasses |
KR102349765B1 (en) * | 2013-11-25 | 2022-01-12 | 테세랜드 엘엘씨 | Immersive compact display glasses |
US9196187B2 (en) * | 2014-04-08 | 2015-11-24 | Apple Inc. | System and method of reducing power using a display inactive indication |
US20150287351A1 (en) * | 2014-04-08 | 2015-10-08 | Apple Inc. | System and method of reducing power using a display inactive indication |
US20150346700A1 (en) * | 2014-06-02 | 2015-12-03 | Rovio Entertainment Ltd | Control of a computer program |
US10838378B2 (en) * | 2014-06-02 | 2020-11-17 | Rovio Entertainment Ltd | Control of a computer program using media content |
Also Published As
Publication number | Publication date |
---|---|
WO2002054959A2 (en) | 2002-07-18 |
AU2002243461A1 (en) | 2002-07-24 |
WO2002054959A3 (en) | 2003-02-20 |
US6900777B1 (en) | 2005-05-31 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US6900777B1 (en) | Infrared audio/video interface for head-mounted display | |
US6101038A (en) | Infrared audio/video interface for head-mounted display | |
US7071897B2 (en) | Immersive augmentation for display systems | |
EP0601179B1 (en) | Single sensor video imaging system and method using sequential color object illumination | |
US6972735B2 (en) | Head-mounted viewing system for single electronic displays using biocular lens with binocular folding mirrors | |
US10798355B2 (en) | Color night vision cameras, systems, and methods thereof | |
EP0683962B1 (en) | Video imaging system and method using a single full frame sensor and sequential color object illumination | |
US7081870B2 (en) | Wearable display and method of displaying images using a wearable display | |
CA2267876C (en) | Biocular viewing system with intermediate image planes for an electronic display device | |
CN100563333C (en) | Endoscope apparatus | |
US5978023A (en) | Color video camera system and method for generating color video signals at increased line and/or frame rates | |
WO2003053057A1 (en) | Camera positioning system and method for eye-to-eye communication | |
US7091929B2 (en) | Method and apparatus for displaying images | |
JP2020532821A (en) | Lighting methods and systems that improve the perspective color perception of images observed by the user | |
JP2005167867A (en) | Optical transmission system | |
JP3506766B2 (en) | Stereoscopic endoscope imaging device | |
KR102592736B1 (en) | Integrated board control system | |
JP4135040B2 (en) | Imaging device, display device, imaging / display device, and three-dimensional image display system using them | |
JPH07313451A (en) | Stereoscopic endoscope image pickup device | |
Ruskuls et al. | Multifocal Near-Eye Display: Timing of Optical Diffuser Elements and Synchronization to DLP-Based Projection Microunit | |
JPH1093994A (en) | Multiplex |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: OPTIMIZE INCORPORATED, CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:HEBERT, RAYMOND T.;HEMPSON, KEVIN R.;REEL/FRAME:011774/0541 Effective date: 20010420 |
|
AS | Assignment |
Owner name: SHERWOOD PARTNERS, INC., CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:OPTIMIZE, INC.;REEL/FRAME:014431/0831 Effective date: 20030717 Owner name: STRYKER CORPORATION, MICHIGAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:OPTIMIZE, INC.;REEL/FRAME:014431/0831 Effective date: 20030717 |
|
AS | Assignment |
Owner name: STRYKER CORPORATION, MICHIGAN Free format text: CORRECTIVE ASSIGNMENT TO CORRECT THE ASSIGNEE DATA BY DELETING \"SHERWOOD PARTNERS, INC.\" PREVIOUSLY RECORDED ON REEL 014431 FRAME 0831;ASSIGNOR:OPTIMIZE, INC.;REEL/FRAME:015700/0842 Effective date: 20030717 |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
FPAY | Fee payment |
Year of fee payment: 4 |
|
FPAY | Fee payment |
Year of fee payment: 8 |
|
FPAY | Fee payment |
Year of fee payment: 12 |