US3697681A

US3697681A - Placement of image on matrix display

Info

Publication number: US3697681A
Application number: US66729A
Authority: US
Inventors: Reginald F H Mccoy
Original assignee: Singer Co
Current assignee: Link Flight Simulation Corp
Priority date: 1970-08-25
Filing date: 1970-08-25
Publication date: 1972-10-10
Anticipated expiration: 1989-10-10
Also published as: JPS549533B1; GB1351157A; FR2106031A5

Abstract

Apparatus which will provide, to a visual display comprising a plurality of scanned raster devices such as television monitors arranged to form a continuous display, a means of placing a high resolution image, which may be larger than the size of one device or monitor but smaller than the size of the total display, in a controlled position on one or more of the monitors in a manner such that the image will appear to be continuous from one monitor to another.

Description

United States Patent McCoy [451 Oct. 10, 1972 [54] PLACEMENT OF IMAGE ON MATRIX 3,517,121 6/ 1970 Petrocelli et al.... 1 78/DIG. 35 DISPLAY 3,507,990 4/1970 Wolff ..l78/6.8 [72] Inventor: Reginald F H. McCoy Highland 3,479,454 11/1969 Wolff ..178/D1G. 35

Park, Ill FOREIGN PATENTS OR APPLICATIONS [73] Assignee: The Singer Company, New ork, 467,195 6/1937 Great Britain ..l78/7.85

Primary Examiner-Robert L. Griffin [22] Filed 1970 Assistant Examiner-Richard K. Eckert, Jr. [21] Appl. No.: 66,729 Attorney-Francis L. Masselle, William Grobman and Charles S. McGuire [52] US. Cl ..l78/6.8, 178/D1G. 6, 178/:)7I8G/.7385 [57] ABSTRACT 51 int. Cl. ..H04n 3/16, H01 j 29/89 Apparatus which will Provide, to a visual p y [58] Field of Search...178/D1G. 6, DIG. 23, DIG. 35, prising a plurality of scanned raster devices Such as 178/63, 733 television monitors arranged to form a continuous display, a means of placing a high resolution image, 6 R f d which may be larger than the size of one device or [5 1 e erences l e monitor but smaller than the size of the total display,

UNITED STATES PATENTS in a controlled position on one or more of the monitors in a manner such that the image will appear to be 3,580,978 5/ 1971 Ebelmg" ..178/DIG. 35 continuous from one monitor to another 3,560,644 2/1971 Petrocelli et al ..l78/6.8 3,525,804 8/1970 Owen l 78/6.8 X 2 Claims, 9 Drawing Figures 54a i 54b VIDEO VIDEO SWITCH SWITCH 52a 66']- 52b 56 SYNC -ju SYNC svimc DELA 42 Y 1 -1o :i /2 DELAY LOGIC is, F 1 I T Um I L e? i so SYNC L SYNC *DELAY L J DELAY H ( 54c 54d 52d v10 0 vlinso E? SW- SW.

7 A .1 C570. f57b SYNC AZ SYNC SYNC El GENERATOR I DELAY DELAY 361 l l AZ,EI 7 COMPUTER l PATENTEDHN 1 912 3 .6 97; 681

SHEET 1 OF 5 TARGET HORIZ. SCAN 70 ON CRT 520.

H-ORIZ. SCAN 72 ON CRT 52b HORIZ. 74 i I CAMERA SCAN I I I Z W l l O I 7 L I L I o Z }l l O 78 sw 54b L o [.KYEAI'OR fil 7w FIG. 4 BY jam ATTORNEY PATENTEDBBHO I912 3.697.681

MK GUN ATTORNEY PATENTEDUIIT 10 I972 SHEET 5 0F 6 MC 2 ATTORNEY PATENTEDBBI 10 m2 SHEET 6 (IF 6 PLACEMENT OF IMAGE ON MATRIX DISPLAY This invention relates to visual displays and in particular to a system for positioning a high resolution image having a narrow field of view on a wide angle display.

Due to increased costs of operating aircraft, a need has arisen for trainers with greater capabilities in the area of visual simulation. An ideal display would surround the pilot with images duplicating the real world scene being simulated. To do so requires extremely complex image generation devices. An approximate solution to this problem which provides essentially equivalent training value is to have a large display field of view surrounding the pilot and to present on his display, in detail, the portion of the real world scene that is of primary interest to the trainee. Outside of this detailed scene there may, if desired, be displayed an image of lower detail having the primary purpose of providing the trainee with visual cues as to the attitude or motion of the simulated aircraft. In this way a simple image generation device having a narrow field of view may be used to generate the detailed scene and another simple device used to fill in the areas of lower detail.

In systems of this type it is often desirable to use scanned raster devices such as television monitors or television projectors to form the displayed image. By using a television system rather than direct projection, the image source may be located at a distance from the display and special effects may be easily introduced in the video signals. To form a wide angle display a plurality of such television display devices must be combined to provide the required field of view. The problem then arises as to how to position the detailed portion of the total image on one or more of the separate devices which comprise the total display. Some previous systems have attempted this by projecting the detailed image on an intermediate display such as a large screen and viewing this intermediate image with a plurality of scanned image pickup devices such as television cameras, one camera being provided for each of the scanned raster devices making up the total display. For a large display this method requires a large number of image pickup devices and causes a difficult registration problem and a degradation of the quality of the displayed image. This is particularly true if the image is in color.

The present invention avoids this problem by allow ing the detailed image to be viewed by a single pickup camera and providing means to position it as desired without the need for an intermediate display.

The object of this invention is to provide, to a display comprising a plurality of scanned raster devices arranged to form a continuous display, means for positioning a detailed image, smaller than the size of the total display, in a controlled position on the display.

Another object is to provide an improved visual display system for use in flight trainers.

A further object is to provide, in a visual system in cluding image transformation means for altering the apparent viewpoint, means for controllably positioning a detailed image of an area of interest on a background of lower detail.

Other objects will in part appear hereinafter and will in part be obvious.

The invention accordingly comprises the features of construction, combination of elements, and arrangement of parts, which will be exemplified in the construction hereinafter set forth, and the scope of the invention will be indicated in the claims. For a fuller understanding of the nature and object of the invention reference should be had to the following detailed description taken in connection with the accompanying drawings in which:

FIG. 1 is a plan view of the geometry during filming;

FIG. 2 is a block diagram of a preferred embodiment of the total system;

FIG. 3 is a detailed block diagram of the image positioning portion of the system;

FIG. 4 is a timing diagram associated with FIG. 3;

FIG. 5 is a block diagram of the switching logic;

FIG. 6 is a side view of the display arrangement;

' FIG. 7 is a plan view of the display arrangement;

FIG. 8 is a perspective view of the vertical segment in the display; and

FIG. 9 is a perspective view of an additional display embodiment showing both horizontal and vertical segments. In the figures which follow it will be shown how the invention in its present embodiment, is able to provide to a trainee in an aircraft simulator an image of an area of interest to him in his training. Basically, a decision is first made as to the area to be displayed during training and a flight made in an actual aircraft over that area with a motion picture camera attached to the aircraft always pointed at the area of interest. The processed film is then used as an image source for the present invention. The trainee sits in a simulated aircraft cockpit which is a virtually exact duplicate of that of the real aircraft and is instructed to fly in a simulated path which somewhat approximates the actual path followed by the aircraft from which the motion picture was taken. Arranged around the windows of this simulated cockpit is the visual display which has as its inputs a multiplicity of TV monitors. The remainder of the system herein described, which may be at a remote location to the cockpit, operates upon the film image to give it the proper perspective from the trainees simulated viewpoint and to properly position it in his field of view. This is done by viewing the image, as projected, with a television camera, distorting the video signal thus generated using optical and electronic techniques and then by means of proper switching, displaying the video signal so generated on the proper monitor, or monitors in the display. The distortion alters the trainees apparent viewpoint in accordance with the computed position of the simulator, thereby providing a realistic display.

FIG. 1 shows how, in the present embodiment, an aircraft flies a path 20 as shown with the line of sight 22 of a motion picture camera, mounted in the aircraft, always pointed at a target 24, or another area of interest to the trainee. While this path is being flown, data relating to the line of sight, either in the form of angles or in the form of lateral, longitudinal and altitude information for both the taking camera and target, are recorded. The required information is then coded on the film for use in the system using standard optical or magnetic techniques. One such technique is described in U.S. Pat. No. 2,999,322 which was granted on Sept. 12, 1961 to H. S. Hemstreet and assigned to the same assignee as the present invention.

The system shown in FIG. 2 must now take this film which was previously generated and present an image to a trainee, in a simulator to which the display portion of the system is attached, properly positioned in his field of view and having the proper perspective. The film is placed in film transport 30 which is a standard pulsed pulldown type of transport and in this case is pulsed by film speed input 32 and vertical sync input 34. Film speed input 32 is generated in computer 36 as a function of aircraft velocity and position based on simulator inputs 38. Computer 36 may be a standard analog or digital computer. In addition to a pulsed pulldown, the system also has a pulsed lamp 40 which is synchronized with film transport 30 by vertical sync 34 causing it to be off during the retrace time of television camera 42. As the film passes through the transport 30, data previously recorded relating to the position from which the film was taken is picked off by data pickoff 44 and fed to computer 36. This data is then used, along with simulator data which contains the position and attitude of the simulated aircraft, to generate outputs on line 46 which are needed for the proper positioning and perspective transformation of the image. Perspective transformation is accomplished by optical or electronic means, or a combination of both, indicated by the block numbered 48 in FIG. 2, and described in US. Pat. Nos. 2,999,322, 3,091,662 and 3,266,912 all granted to H. S. Hemstreet and all assigned to the same assignee as the present invention.

The resulting image generated by television camera 42 will have the proper perspective. This video image is then distributed on lines 50 to video switches 54 for each of a multiplicity of displays 52 which make up a total display surrounding the pilot trainee in the simulator. Video switches 54 are controlled by switch logic 56 to properly position, along with sync delays 57, the image on the total display as will be described below. This is necessary since the size of the image picked up by television camera 42 is small as compared with the total display.

Background video generator 58 is used to fill in portions on the total display where no detailed imagery from the television camera 42 is presented. It will provide different colors and/or gray levels for ground and sky and will be controlled by attitude inputs from computer 36. Visibility modulators 60 will introduce special effects to simulate the desired degree of visibility and will be controlled as a function of inputs 62 which may be manually set by the instructor. Visibility will range from full visibility to a point where no image is visible, simulating heavy fog.

To understand how proper positioning is accomplished FIGS. 3 and 4 must be examined together. Displays 52a through 52d, e.g., television monitors, are arranged to form a continuous display. The particular means of physically arranging the display devices used in the present embodiment to provide continuity of the display area will be described later. The image generated by television camera 42 may appear anywhere within the total display area, indicated by dotted line 66, as a function of the simulated aircraft position and attitude, and the position of the image being viewed relative to the position and attitude of the taking camera. As previously indicated, this information is available in the computer 36 from the simulator data input and the film data input. It is possible then to compute, using standard methods, the line of sight from the simulated aircraft to the center of the image to be displayed. This can then be resolvedinto two components, an azimuth and an elevation, respectively representing the horizontal and vertical angles between the simulated aircraft axis (or other fixed reference) and the line of sight. This information is provided from computer 36 to the sync delays 57 a and 57b and to switch logic 56.

This discussion will deal only with the horizontal positioning which involves the horizontal scan and the azimuth data but in all cases the same analysis applies to the vertical positioning which involves the vertical scan and elevation data. The master horizontal and vertical scans are generated by sync generator 64 and are then distributed to the camera 42,

sync delays

57a and 57b and switch logic 56.

To be able to position the image on two or more of TV monitors 52a 52d it is necessary to have a continuous raster covering any area within the total display area 66. The manner in which this is accomplished can best be seen by examining traces and 72 on FIG. 4. Trace 70 represents the horizontal scan of monitor 52a, showing the relationship of the horizontal position of the scanning spot to time. The long, increasing portion of the trace represents movement of the spot from left to right across the monitor. The short, decreasing portion is the retrace of the spot back to its starting point. During retrace, information is normally not displayed.

Traces

72 and 74 similarly represent the horizontal scans of monitor 52b and the TV camera, respectively. Normally, if the monitors and camera were all connected to the same source of synchronization, e.g., sync generator 64, the time relationship of each would be the same and both monitors would display the image picked up by the camera. Considering only the rasters of the two

monitors

52a and 52b, it can be seen that if they were synchronized both would retrace during the same time. If it is desired to have a continuous raster from one display to another it is necessary that as a horizontal line is completed on the first monitor, the line in the same horizontal position on the second monitor must start its motion across the face of the tube. This requires that a delay be inserted so that the two traces are no longer synchronized. Examining these traces it will be seen that as trace 70 reaches its peak, indicating the end of the scan, and starts its retrace, that trace 72 starts its scan. The result as displayed on

monitors

52a and 52b will be an apparently continuous raster line across the two monitors. The delay in 70 causing it to reach its peak at a later time than 72, is accomplished by a fixed sync delay 57c. This is a normal fixed delay line and one is provided for both the horizontal and vertical scan of each monitor used. By proper adjustment of these delays, a continuous raster in both the horizontal and vertical directions over a number of displays can be created.

Having created a continuous raster it is now necessary to position the image generated by camera 42 on that raster. Assuming an azimuth of 0 represents centering on the total display 66 it can be seen that half of a horizontal line in the upper portion of the image area 67 must be on monitor 52a and half on 521). To achieve this result a camera scan 74, related in time to

scans

70 and 72 as shown on FIG. 4, is necessary. At time 73 camera scan 74 is starting its scan across the image. For the azimuth given, this portion of the image should be displayed at about the middle of monitor 52a. If the trace 70 of monitor 52a is half way across the monitor at time 73 this result will be achieved. As camera trace 74 reaches its mid point at time 75 monitor 52as trace will be starting its retrace. But, since the relative timing of

traces

70 and 72 has been adjusted to form a continuous raster, the remainder of the camera trace will coincide in time with the first half of monitor 52bs trace 72. The result then is that one horizontal camera line 74a on trace 74 will coincide for the first half of its time with the last half of monitor 52as trace 70, shown by 70a, and will coincide for the second half of its time with the first half of monitor 52bs trace 72, shown by 72a. However, if nothing further is done, unwanted information will also be displayed. Looking at the portion of trace 74 labeled 74b and its relation to

traces

70 and 72 it can be seen during the time information is displayed on the last half of monitor 52a, shown by 70a, we will be displaying the same information on the last portion of monitor 52b, shown by 72b. Likewise when showing the desired information on the first portion of monitor 52b, as shown by 72a, the same information will be displayed on an undesired portion of 52a, shown by 70b.

To avoid this problem video switches which will switch on the video signal only when the proper information is present may be used.

Video switches

54a, 54b, 54c, and 54d, shown in FIG. 3, are used for this purpose. The inputs to

switches

54a and 54b are shown by traces 76 and 78 (FIG. 4), respectively. By operating the switches in accordance with the timing of the traces shown, only the desired information will appear on each monitor. Each switch is on only during that portion of camera trace 74 when information is to be displayed on its respective monitor. For example, switch 540, associated with monitor 52a, is on only during the portion of the camera trace 74 between times 73 and 75. Therefore, during the portion of monitor 52a trace 70 labeled 70b, the switch will be off and no improper information displayed. Likewise, switch 54b is only on when it is desired to display information on monitor 52b. All switches are off during camera retrace time, and retrace times of their respective monitors. During the times when a monitor is being scanned but has its respective switch off, another switch may be operated to insert fill-in information.

It was assumed above that camera trace 74 was in the proper time relationship to monitor

traces

70 and 72. To accomplish this relationship a variable sync delay 570 shown in FIG. 3 is used. This is a standard variable delay line which is controlled as a function of azimuth. In the example given with 0 azimuth the camera trace was positioned so that equal portions coincided with monitor traces 70 and 72. As azimuth is changed left or right the delay line 57a will cause a corresponding shift of trace 74 in relation to

traces

70 and 72 and different amounts of the image will be displayed on

monitors

52a and 52b. The operation of

video switches

54a and 54b must also be synchronized with the camera scan and with azimuth to maintain the proper time relationships shown on FIG. 4.

FIG. 5 shows how these video switches may be operated as required. One line of the horizontal motion in scanning across the image. This motion can be related to a horizontal field of view on the image being scanned. The amplitude of camera trace 74 may then be used to represent an angle. If the camera horizontal field of view is 40 then the maximum amplitude of trace 74 may be said to equal 40, half the max 20, one quarter 10, etc. If it is also assumed that

monitors

52a and 52b have a 40 field view, then the junction of the two may be given the value of 0 and the left side of

monitor

52a, 40 and the right side of monitor 52b +40.

Knowing the limits of the monitors it is possible to set up a means of detecting whether a given signal is within the monitors limits. FIG. 5 shows such a means.

Comparators

82 and 84, the former set to detect values less than 40 and the latter values greater than 0, may be used as inputs to AND gate 96 which will have an output when the input to

comparators

82 and 84 is not less than 40 and not greater than 0. The third input to gate 96 from slope detector 94 will inhibit gate 96 from turning on when the slope is in a negative direction, i.e., during retrace. The output from gate 96 may then be used to operate video switch 54a. In a

like manner comparators

86 and 88 may be set to detect values of less than 0 and greater than +40, thus causing gate 98 to operate switch 54b when the input is greater than 0 and less than +40 and the slope is positive. It is now necessary to provide the proper inputs to the comparators.

At the assumed value of 0 azimuth, and an assumed image of 40 horizontal degrees, an image should be displayed on monitor 52a from 20 to 0 and on monitor 52b from 0 to +20. This means that

switches

54a and 54b must be on during these respective times. The input 92 to the comparators which will provide this result is shown on FIG. 5. Input 92 is obtained by summing azimuth 90 and the horizontal camera scan 74 in summing amplifier 80. By scaling azimuth such that 0 of azimuth is equal to minus one half the maximum value of trace 74 the resulting trace 92 after summing will be half negative and half positive. Since the maximum value of the trace is 40 then it will represent scanning from 20 to +20. As the scan begins its value is greater than -40" and less than 0 causing switch 54a to be on and 54b to be off. As the trace 92 passes 0 switch 54a will turn off since the limit of comparator 84 has been reached and switch 54b will turn on since the trace is within the limits of being greater than 0 and less than 40. When the camera scan 74 retraces switch 54b will be turned off and 54a held off due to the signal from slope detector 94.

Video switches

54a and 54b are standard, solid-state video switches of the type commonly used in video insertion and special effects applications.

As azimuth is changed, sync delay 570 will cause the time relationships between the camera scan and the monitor scans to change. At the same time the azimuth signal will shift trace 92 up or down displaying more or less on each of the monitors. For example, if azimuth were increased to 30 then trace 92 would represent SO to l0. The result would be that as the trace starts neither switch would be on since SO is less than the -40 lower limit of comparator 82 and its associated switch 540, and it is less than the lower limit of comparator 86 and its associated switch 54b. As the trace 92 reached 40, switch 54a would come on and stay on until retrace. Since the trace never gets past 0, switch 54b would remain off.

For this method of positioning the image to be useful it is necessary to have a means of combining a number of display devices to form a wide angle total display. FIGS. 6 and 7 show how this may be done. FIG. 6 shows a side view of three tiers of mirror beam splitter infinity image displays. Considering the display comprising spherical mirror 110, beamsplitter 112 and input cathode ray tube 114, it is well known in the art that making the radius of curvature of the face of the cathode ray tube one half that of the mirror and placing the display surface of the CRT at a distance from the mirror equal to one half the radius of the spherical mirror, will result in an image focused at infinity. By placing the mirror at a distance 111, equal to its radius, from the nominal eye position 113 of an observer, the maximum possible amount of allowable head motion without distortion results.

Because of practical limitations of size, such a display has a limited field of view. To obtain an increased field of view it is desirable to combine a number of these displays. FIG. 7 shows how it is possible to butt together two or more displays and increase the horizontal field of view by shaping mirror 110 and beamsplitter 112 into a section of a sphere equal in radius to the spherical mirror with the sides shaped such that the joints represent lines of longitude on a sphere. However, when an attempt is made to increase the vertical field of view by butting another mirror and beamsplitter directly above the first tier, input cathode ray tube 114 is in the way as can be seen by examining FIG. 6.

The present invention solves this problem by constructing a second tier of sections of a smaller sphere comprising mirror 116, beamsplitter 118, and input cathode ray tube 120. The spherical mirror is placed as close as possible to and above beamsplitter 112 and the mirrors required radius is determined by the distance 119 to the nominal eye position. Since its radius is proper for its distance from the observer it will also allow a maximum change in eye position without distortion. The same method is used in constructing the third tier of displays comprising spherical mirror 122, beamsplitter 124 and input cathode ray tube 126. Additional tiers may be constructed until a practical limitation due to small size is reached. 1

FIG. 8 shows how this display presents a continuous display in the vertical direction. The image of one of the displays in the first tier is represented by shaded area 130, that of one in the second tier by 132, and that of one in the third tier by 134. What results from the construction are sections of three spheres all centered at the nominal eye position 113. Because the centers are coincidental, angular relationship will remain the same. This means point 140 will appear on surface 132 in the same angular position as point 138 would have appeared on surface 136 if the same size mirror were used. If positioning is done using angular information, as was previously disclosed then, since all images are focused at infinity and all angular relationships are r0 er, the dis la will a ear continuous. p A n understahdfng of tfl se angular relationships also makes it possible to see how rasters may be matched from one display to another. If the raster is scanned such that the beam traverses equal angles in equal time in the horizontal direction, then the horizontal scans can be matched whether the displays 142 are arranged in one spherical segment as shown in FIG. 9 or three spherical segments as shown on FIG. 8. Only angles and time are important, size does not matter. In the vertical direction each' successive raster line can be displaced through an equal vertical angle. In this way a raster which is in a controlled angular relationship to time may be made continuous from one display to another using the method previously described.

Although the present embodiment utilizes beamsplitter-mirror type displays, this portion of the invention relating to the formation of a continuous raster may be practiced using refractive as well as reflective displays as long as the segments are constructed such that their edges form what are equivalent to lines of longitude on a sphere. In addition, the segments may be arranged to form one sphere or may be arranged in tiers as shown as long as they have their images focused at infinity and are all centered at the same nominal eye position.

What is claimed is:

1. Apparatus providing a displayed image of an area of interest which can be positioned anywhere within a larger display area of a system including a plurality of display units, said apparatus comprising, in combination:

a. an image source;

b. a television camera image pickup;

c. a multiplicity of television raster displays each of said displays comprising a reflective mirror beamsplitter display having a cathode ray tube as input and arranged to form a continuous display area;

. a timing generator and waveform generator to provide an horizontal and a vertical scan to said televi sion camera and said raster displays;

e. fixed delay means inserted between said generators and each of said displays to cause the rasters of said displays to form a continuous raster across said visually continuous display;

f. variable delay means operable to delay the relative scan times of said camera and said plurality of display devices;

g. switching means connecting the video signal generated by said camera to said plurality of display devices operable to selectively switch said video signal to selected ones of said display devices; and

h. control means operable to control said variable delay means and said switching means to selectively position the resulting image anywhere on said visually continuous display.

2. The invention according to claim 1 wherein said image is a representation of a portion of the real world and said control means are responsive to inputs representing the simulated relative position of said observer with respect to said portion.

l i i i l

Claims

1. Apparatus providing a displayed image of an area of interest which can be positioned anywhere within a larger display area of a system including a plurality of display units, said apparatus comprising, in combination: a. an image source; b. a television camera image pickup; c. a multiplicity of television raster displays each of said displays comprising a reflective mirror beamsplitter display having a cathode ray tube as input and arranged to form a continuous display area; d. a timing generator and waveform generator to provide an horizontal and a vertical scan to said televisioN camera and said raster displays; e. fixed delay means inserted between said generators and each of said displays to cause the rasters of said displays to form a continuous raster across said visually continuous display; f. variable delay means operable to delay the relative scan times of said camera and said plurality of display devices; g. switching means connecting the video signal generated by said camera to said plurality of display devices operable to selectively switch said video signal to selected ones of said display devices; and h. control means operable to control said variable delay means and said switching means to selectively position the resulting image anywhere on said visually continuous display.