System and method for displaying concurrent video and reconstructed surgical ...

United States Patent [w]

Gildenberg

US005961456A [ii] Patent Number: [45] Date of Patent:

5,961,456 *Oct. 5,1999

[54] SYSTEM AND METHOD FOR DISPLAYING CONCURRENT VIDEO AND RECONSTRUCTED SURGICAL VIEWS

[76] Inventor: Philip L. Gildenberg, 3776 Darcus St., Houston, Tex. 77005

[ * ] Notice: This patent issued on a continued prosecution application filed under 37 CFR 1.53(d), and is subject to the twenty year patent term provisions ol 35 U.S.C. 154(a)(2).

[21] Appl. No.: 08/625,572 [22] Filed: Mar. 28, 1996

Related U.S. Application Data

[63] Continuation-in-part of application No. 08/395,770, Feb. 28, 1995, abandoned, and a continuation-in-part of application No. 08/552,168, Nov. 2, 1995, abandoned, said application No. 08/395,770, is a continuation of application No. 08/060, 562, May 12,1993, abandoned, said application No. 08/552, 168, is a continuation of application No. 08/303,862, Sep. 9, 1994, abandoned, which is a continuation-in-part of application No. 08/060,562, abandoned.

[51] Int. CI. A61B 5/00

[52] U.S. CI 600/429; 600/426; 600/476;

606/130

A video camera is positioned stereotactically to view a surgical opening in a patient and provide data related to stereotactic space and is stereotactically representative ol a current actual view, which data is processed and displayed as a current actual image along with a relerence image that is computer generated from graphics image data stored on the patient. The two images appear in a display variously, for ease ol comparison and are coordinated by stereotactically tracking the camera to maintain position and orientation. A composite camera structure incorporates a surgical microscope and stereoscopic viewing along with light tracking coordination. Displays included side-by-side and overlapped.

19 Claims, 2 Drawing Sheets

1

SYSTEM AND METHOD FOR DISPLAYING
CONCURRENT VIDEO AND
RECONSTRUCTED SURGICAL VIEWS

CROSS-REFERENCE TO RELATED 5
APPLICATIONS

This is a continuation-in-part of application Ser. Nos. 08/395,770 now abandoned and 08/552,168 now abandoned, filed on Feb. 28, 1995, and Nov. 2, 1995, respectively, and both entitled "CAMERA SYSTEM AND 10 COMPUTER GRAPHIC DISPLAY FOR VIDEO AND IMAGE RECONSTRUCTED SURGICAL VIEWS", application Ser. No. 08/395,770, is a continuation of application Ser. No. 08/060,562, filed on May 12, 1993, and entitled "CAMERA SYSTEM AND COMPUTER GRAPHIC DIS- 15 PLAY FOR VIDEO AND IMAGE RECONSTRUCTED SURGICAL VIEWS", now abandoned, and application Ser. No. 08/552,168, is a continuation of application Ser. No. 08/303,862, filed on Sep. 9, 1994, and entitled "CAMERA SYSTEM AND COMPUTER GRAPHIC DISPLAY FOR 20 VIDEO AND IMAGE RECONSTRUCTED SURGICAL VIEWS", now abandoned, which is a continuation-in-part of application Ser. No. 08/060,562 now abandoned.

BACKGROUND AND SUMMARY OF THE 25
INVENTION

The use of stereotactic instrumentation based on tomographic imaging is now common place in surgery, especially in neurosurgery of the brain. Such methods typically involve 30 attaching a headring apparatus to the patient's skull and acquiring imaging data that is space related to the headring, for example, by the utilization of indexing devices, localizer structure or other fiducial apparatus. Thus, quantitative coordinates of targets within the patient's head can be 35 specified relative to the fiducial apparatus. An arc system or other pointing device then can be used to guide probes or other instruments selectively into the brain quantitatively based on the imaging data.

Also common place in prior techniques is the acquisition 40 and use of large, three-dimensional data sets of imaging data that is stored in computer graphics workstations and visualized on computer graphics screens, such as cathode ray tube (CRT) screens in an operating room. Computer graphics data has been used in conjunction with other apparatus. 45 For example, computer graphics images, based on imaging data have been placed in the direct view field of a surgical microscope. Specifically, see U.S. Pat. No. 4,722,056 granted Jan. 26, 1988 to Roberts et al. Also, to some extent, computer graphic methods have been used with stereotactic 50 arc systems. For example, the work of Dr. Patrick Kelly (Tumor Stereotaxis) referred to in the referenced patent to Roberts et al is pertinent in that regard. Accordingly, a graphic representation of a surgical approach into the head is displayed on a computer screen based on reconstructed 55 images of tomographic scan data. Thus, by knowing the quantitative stereotactic direction into the surgical area, a corresponding graphics display image can be displayed on a computer screen. Consequently, a surgeon can view a reconstructed picture of the image he will see upon looking go directly into an actual surgical field, either with a naked eye or through a microscope. Quantitative maneuvers, such as volumetric resection, can thus be made by measurements in the actual surgical field, compared to measurements extracted from the graphic display. 65

Generally, computer-assisted stereotactic surgery is becoming popular in neurosurgery. In that regard, examples

2

of various available computer graphic systems for use in surgery are produced by Radionics/RSI, Burlington, Mass. A system known as the Compass System is a commercial product embodying Dr. Kelly's technique as mentioned above. Another specific implementation of Dr. Kelly's technique involves importing a reconstructed graphics image into a "heads-up" display, which the surgeon can see as he views the surgical field directly or through a surgical microscope. Forms of heads-up displays, using goggles, are common place in military aircraft applications. Accordingly, a small display can indicate a computer-reconstructed image of what should be seen in a surgical field. Thus, the surgeon can view directly the surgical field and make decisions and quantitative maneuvers in the surgery, based on comparison with the heads-up display.

Generally, these prior computer-assisted surgical methods have certain disadvantages. In the screen display apparatus, the surgeon must view the computer graphics screen, then change his view to the surgical opening. Accordingly, an inconvenient two-step operation is involved requiring alternate viewing. In the case of the heads-up displays, (as for example through a microscope as described by Kelly, stated below) the surgeon is encumbered by having to wear goggles or visualize only a rudimentary outline of the target volume.

The Kelly technique imparting an image, based on a computer-generated target, into visualization of a surgical field, ordinarily involves a microscope. The technique restricts the target image to a rudimentary outline and restricts the view line to the target volume. It is also noteworthy that surgery, while looking at a graphic display (video) is commonly performed in other fields employing endoscopic techniques, however, those techniques do not relate a video image to a computer graphics display.

Generally, the present invention involves stereotactically mounting a video camera to provide image data. Thus, plural surgical images can be provided, as for comparison, in a single (or plural) computer graphics display, one, the surgical view image, the other, a reconstructed image from graphics data. The surgical view image for the single display may be accomplished using a minimally invasive, extracorporeal camera located outside the body near the surgical opening. Thus, in accordance herewith, the camera structure providing the video (actual image) related to stereotactic space, in one embodiment, may be minimally obtrusive so that the surgeon can view into the surgical field directly by naked eye, or by microscopic view. The surgical view or actual current image is combined in accordance with one embodiment in a common unitary display with the reconstructed or reference image processed from computer graphics data. The comparable images in a single display are enabled by tracking and correlating the position of the camera to the patient (camera subject). Thus, for example, by the utilization of stereotactic placement or registration of camera positions and patient anatomy, effective dual images (current actual and reference) may be provided in a single display. For example, a video image may be displayed on a single display means along with a reconstructed computer graphics image provided from prescanned or imaged data. Alternatively, the surgical view or actual current image and the reconstructed or reference image may be provided in plural displays.

BRIEF DESCRIPTION OF THE DRAWINGS

In the figures, which constitute a part of this specification, exemplary embodiments exhibiting various subjectives and features hereof are set forth, specifically: