US20050027186A1 - Video-based surgical targeting system - Google Patents
Video-based surgical targeting system Download PDFInfo
- Publication number
- US20050027186A1 US20050027186A1 US10/775,754 US77575404A US2005027186A1 US 20050027186 A1 US20050027186 A1 US 20050027186A1 US 77575404 A US77575404 A US 77575404A US 2005027186 A1 US2005027186 A1 US 2005027186A1
- Authority
- US
- United States
- Prior art keywords
- patient
- virtual
- image
- images
- real
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
- A61B90/36—Image-producing devices or illumination devices not otherwise provided for
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/10—Computer-aided planning, simulation or modelling of surgical operations
- A61B2034/101—Computer-aided simulation of surgical operations
- A61B2034/102—Modelling of surgical devices, implants or prosthesis
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/10—Computer-aided planning, simulation or modelling of surgical operations
- A61B2034/101—Computer-aided simulation of surgical operations
- A61B2034/105—Modelling of the patient, e.g. for ligaments or bones
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
- A61B2034/2046—Tracking techniques
- A61B2034/2055—Optical tracking systems
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
- A61B90/36—Image-producing devices or illumination devices not otherwise provided for
- A61B2090/364—Correlation of different images or relation of image positions in respect to the body
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
- A61B90/36—Image-producing devices or illumination devices not otherwise provided for
- A61B2090/364—Correlation of different images or relation of image positions in respect to the body
- A61B2090/365—Correlation of different images or relation of image positions in respect to the body augmented reality, i.e. correlating a live optical image with another image
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
- A61B90/36—Image-producing devices or illumination devices not otherwise provided for
- A61B2090/364—Correlation of different images or relation of image positions in respect to the body
- A61B2090/367—Correlation of different images or relation of image positions in respect to the body creating a 3D dataset from 2D images using position information
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/25—User interfaces for surgical systems
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
- A61B90/36—Image-producing devices or illumination devices not otherwise provided for
- A61B90/361—Image-producing devices, e.g. surgical cameras
Definitions
- This invention relates to surgical systems in general, and more particularly to surgical targeting systems.
- the physician typically uses some sort of scanning device to examine the patient's anatomy at the interior site prior to, and in preparation for, conducting the actual procedure itself.
- These scanning devices typically include MRI devices, CT scanners, X-ray machines, ultrasound devices and the like, and serve to provide the physician with a preliminary knowledge of the patient's internal anatomical structure prior to commencing the procedure.
- the physician can then use this information to plan the procedure in advance, taking into account patient-specific anatomical structure.
- the physician can also use the information obtained from these scans to more precisely identify the location of selected structures (e.g. tumors and the like) which may themselves be located within internal organs or other internal body structures.
- the physician can more easily “zero in” on such selected structures during the subsequent procedure, with less trauma to host organs or other internal body structures.
- the structures of interest may be quite small and difficult to identify with the naked eye.
- preliminary scanning of the patient's internal anatomy using high resolution scanning devices can help the physician locate such structures during the subsequent procedure.
- scanning devices of the sort described above tend to generate a series of two-dimensional (i.e., “2-D”) images of the patient's anatomical structure.
- 2-D two-dimensional
- CT scanners generate 2-D images which are viewed directly by the physician.
- the physician can mentally generate a three-dimensional (i.e., “3-D”) sense of the patient's anatomical structure.
- Some scanning devices create large numbers of 2-D images during the scanning process, with each 2-D image representing a plane or slice taken through the scanned structure. Furthermore, some scanning devices also have associated computer hardware and software for building a 3-D computer model of the patient's scanned structure using a plurality of these 2-D images. For example, some MRI devices and some CT scanners leave such associated computer hardware and software. In these cases, an operator using this scanning equipment and associated computer hardware and software can create a number of 2-D images, assemble them into a 3-D computer model of the scanned structure, and then generate various images of that 3-D computer model as seen from various angles so as to enhance understanding of the patient's anatomical structure.
- one object of the present invention is to provide a surgical targeting system to facilitate locating a particular anatomical structure during a medical procedure.
- Another object of the present invention is to provide a video-based surgical targeting system to facilitate locating a particular anatomical structure during a medical procedure.
- Another object of the present invention is to provide a video-based surgical targeting system which permits a series of patient-specific 2-D images (obtained by scanning patient anatomy using one or more scanning devices of the type described above) to be assembled into a 3-D computer model of the patient's scanned structure.
- Still another object of the present invention is to provide a video-based surgical targeting system which allows a physician to view the aforementioned patient-specific 2-D images on a display in any desired access sequence.
- Yet another object of the present invention is to provide a video-based surgical targeting system which allows a physician to assemble a series of patient-specific 2-D images into a patient-specific database, and then to generate virtual images from the aforementioned patient-specific database, as seen from any desired virtual camera position, for viewing on a display.
- Yet another object of the present invention is to provide a video-based surgical targeting system which allows a physician to generate virtual images from the aforementioned 3-D computer model, as seen from any desired virtual camera position, for viewing on a display.
- Another object of the present invention is to provide a video-based surgical targeting system which permits a physician to place virtual planning markers about any sites of interest while viewing one or more of the aforementioned patient-specific 2-D images, with those virtual planning markers then being incorporated into the 3-D computer model, whereby those virtual planning markers can be displayed in their appropriate 3-D positions when generating virtual images of the 3-D computer model.
- Still another object of the present invention is to provide a video-based surgical targeting system which permits a physician to place virtual planning markers about any sites of interest while viewing virtual images of the 3-D computer model, with those virtual planning markers then being incorporated into the 3-D computer model, whereby those virtual planning markers can be displayed in their appropriate positions when generating subsequent virtual images of the 3-D computer model.
- Still another object of the present invention is to provide a video-based surgical targeting system which permits a physician to place virtual planning markers about any sites of interest while viewing virtual images of the 3-D computer model, with those virtual planning markers then being incorporated into the 3-D computer model and into the database of 2-D images, whereby those virtual planning markers can be displayed in their appropriate positions when subsequently generating virtual images of the 3-D computer model or when subsequently displaying 2-D images from the patient-specific database.
- Yet another object of the present invention is to provide a video-based surgical targeting system which permits a real image obtained by a real-time imaging device (e.g. a video camera) to be displayed to a physician, and which permits a virtual image generated from the 3-D computer model to be displayed to a physician, according to the directive of the physician.
- a real-time imaging device e.g. a video camera
- Another object of the present invention is to provide a video-based surgical targeting system which permits a real image obtained by a real-time imaging device (e.g. a video camera) to be appropriately merged with a corresponding virtual image generated from the 3-D computer model.
- a real-time imaging device e.g. a video camera
- Another object of the present invention is to provide a video-based surgical targeting system which permits a real image obtained by a real-time imaging device (e.g. a video camera) to be appropriately merged with a corresponding virtual image generated from the 3-D computer model, whereby the two images will be in registration with one another.
- a real-time imaging device e.g. a video camera
- Still another object of the present invention is to provide a video-based surgical targeting system which permits a real image obtained by a real-time imaging device (e.g. a video camera) to be merged with a corresponding virtual image generated from the 3-D computer model, whereby the two images will be in registration with one another, and whereby the physician can choose to display either one of the two images exclusive of the other, or a composite of both images simultaneously.
- a real-time imaging device e.g. a video camera
- Yet another object of the present invention is to provide a video-based surgical targeting system which permits a real image obtained by a real-time imaging device (e.g. a video camera) to be merged with a corresponding virtual image generated from the 3-D computer model, whereby the two images will be simultaneously displayed in registration with one another, and whereby the physician can modify the virtual image generated from the 3-D computer model as needed, by clipping or fading, so as to expose the virtual planning markers to view, with the virtual planning markers being superimposed on the real image generated by the real-time viewing device.
- a real-time imaging device e.g. a video camera
- Yet another object of the present invention is to provide a video-based surgical targeting system which permits a real image generated by a real-time imaging device (e.g. a video camera) to be merged with a corresponding virtual image generated from the 3-D computer model, whereby the two images will be simultaneously displayed in registration with one another, and whereby the physician can modify the virtual image generated from the 3-D computer model so as to expose only the virtual planning markers to view, with the virtual planning markers being superimposed on the real image generated by the real-time imaging device.
- a real-time imaging device e.g. a video camera
- Still another object of the present invention is to provide a video-based surgical targeting system which permits a real image obtained by a real-time imaging device (e.g. a video camera) to be merged with a corresponding virtual image generated from the 3-D computer model, whereby the two images will be simultaneously displayed in registration with one another, and whereby this registration will be automatically maintained even as the real-time imaging device is moved about relative to the anatomical site, with the virtual image being automatically generated so as to follow the real image.
- a real-time imaging device e.g. a video camera
- Still another object of the present invention is to provide a video-based surgical targeting system which permits a real image obtained by a real-time imaging device (e.g. a video camera) to be merged with a corresponding virtual image generated from the 3-D computer model, whereby the two images will be simultaneously displayed in registration with one another, and whereby this registration will be automatically maintained through the use of a computerized position and orientation tracker connected to the imaging device even as the real-tine imaging device is moved about relative to the anatomical site, with the virtual image being automatically generated so as to follow the real image.
- a real-time imaging device e.g. a video camera
- Still another object of the present invention is to provide a video-based surgical targeting system which permits a real image obtained by a real-time imaging device (e.g. a video camera) to be merged with a corresponding virtual image generated from the 3-D computer model, whereby the two images will be simultaneously displayed in registration with one another, and whereby this registration will be automatically maintained through the use of a computer search algorithm based on the real image and the virtual image even as the real-time imaging device is moved about relative to the anatomical site, with the virtual image being automatically generated so as to follow the real image.
- a real-time imaging device e.g. a video camera
- Yet another object of the present invention is to provide a video-based surgical targeting system which permits a real image obtained by a real-tine imaging device (e.g. a video camera) to be merged with a corresponding virtual image generated from the 3-D computer model, whereby the two images will be simultaneously displayed in registration with one another, and whereby a surgical instrument can be tracked along with the real-time imaging device in order that the surgical instrument can be guided about the anatomical site using the virtual image generated from the 3-D computer model.
- a real-tine imaging device e.g. a video camera
- Another object of the present invention is to provide a video-based surgical targeting system which permits a virtual image to be generated from the 3-D computer model, and further wherein this virtual image can correspond to an “over the shoulder” view of the working tip of a surgical instrument located at the anatomical site.
- Another object of the present invention is to provide an improved method for locating anatomical structures during a medical procedure.
- the video-based surgical targeting system comprises a patient-specific database comprising a plurality of 2-D images of the anatomical structure of a patient; a patient-specific 3-D computer model of the anatomical structure of the patient, the patient-specific 3-D computer model being assembled from the plurality of 2-D images contained in the patient-specific database; marker placement means for (i) inserting virtual planning markers into the 2-D images contained in the patient-specific database, and/or (ii) adjusting the positions of virtual planning markers inserted into the 2-D images contained in the patient-specific database and thereafter incorporated into the patient-specific 3-D computer model, or inserting virtual planning markers into the 3-D computer model; an image generator for generating a virtual image of the anatomical structure modeled by the patient-specific 3-D computer model; real-time image generating means for generating a real image of the anatomical structure of a patient; video mixing means for mixing the virtual image and the real image into an output image, whereby the output image may comprise either one of the two images exclusive of the other
- the video-based surgical targeting system also includes registration means for placing the virtual image in registration with the real image.
- registration means may include means for manually aligning the virtual image with the real image, or means for automatically aligning the virtual image with the real image.
- the registration means may also include tracking means for tracking the position of the real-time image generating means.
- the video-based surgical targeting system further comprises instrument tracking means for tracking the position of a surgical instrument.
- the video-based surgical targeting system comprises a patient-specific database comprising a plurality of 2-D images of the anatomical structure of a patient; marker placement means for inserting virtual planning markers into the 2-D images contained in the patient-specific database; an image generator for generating a virtual image of the anatomical structure represented by the plurality of 2-D images contained in the patient-specific database; real-time image generating means for generating a real image of the anatomical structure of a patient; video mixing means for mixing the virtual image and the real image into an output image, whereby the output image may comprise either one of the two images exclusive of the other, or a composite of both images; and display means for displaying the output image.
- the video-based surgical targeting system comprises a patient-specific database comprising a plurality of images of the anatomical structure of a patient; a patient-specific 3-D computer model of the anatomical structure of the patient, the patient-specific 3-D computer model being assembled from the plurality of images contained in the patient-specific database; marker placement means for (i) inserting virtual planning markers into the images contained in the patient-specific database, and/or (ii) adjusting the positions of virtual planning markers inserted into the images contained in the patient-specific database and thereafter incorporated into the patient-specific 3-D computer model, or inserting virtual planning markers into the 3-D computer model; an image generator for generating a virtual image of the anatomical structure modeled by the patient-specific 3-D computer model; real-time image generating means for generating a real image of the anatomical structure of a patient; video mixing means for mixing the virtual image and the real image into an output image, whereby the output image may comprise either one of the two images exclusive of the other, or a composite of both
- the video-based surgical targeting system comprises a patient-specific database comprising a plurality of 2-D images of the anatomical structure of a patient; marker placement means for inserting virtual planning markers into the 2-D images contained in the patient-specific database; an image generator for generating a virtual image of the anatomical structure defined by the patient-specific database; real-time image generating means for generating a real image of the anatomical structure of a patient, the real-time image generating means being adapted so as to dynamically update the patient-specific database via the real image; video mixing means for mixing the virtual image and the real image into an output image, whereby the output image may comprise either one of the two images exclusive of the other, or a composite of both images; and display means for displaying said output image.
- the present invention also comprises a method for targeting an anatomical structure using the video-based surgical targeting system.
- FIG. 1 is a schematic view of the major components of a video-based surgical targeting system formed in accordance with the present invention
- FIG. 2 is a view of an exemplary patient-specific 2-D image of the sort contained in the patient-specific database
- FIG. 3 is a view of an exemplary patient-specific 2-D image of the sort contained in the patient-specific database, wherein the image has had several virtual planning markers placed into the image;
- FIG. 4 is a composite view of (i) a real image obtained by a real-time imaging device (e.g. a video camera), and (ii) a virtual image generated from the 3-D computer model, wherein the two images are not yet in registration with one another;
- a real-time imaging device e.g. a video camera
- FIG. 5 is a composite view of (i) a real image obtained by a real-time imaging device (e.g. a video camera), and (ii) a virtual image generated from the 3-D computer model, wherein the two images have been placed in registration with one another;
- a real-time imaging device e.g. a video camera
- FIG. 6 is a view like that of FIG. 5 , except that the virtual image generated from the 3-D computer model has been sliced away to expose the virtual planning markers;
- FIG. 7 is a view like that of FIG. 5 , except that the virtual image generated from the 3-D computer model has been faded away to expose the virtual planning markers;
- FIG. 8 is a schematic view showing the positional relationships between various elements of the video-based surgical targeting system and the patient's anatomical structure, as well as the positional relationship between an exemplary surgical instrument and the patient's anatomical structure;
- FIG. 9 is a virtual image generated from the 3-D computer model, showing a virtual “over the shoulder” view from the tip of a tracked surgical instrument.
- FIG. 10 is a flowchart illustrating one way of operating a video-based surgical targeting system formed in accordance with the present invention.
- the starting point for the present invention involves a data acquisition device 5 which is adapted to generate patient-specific 2-D images.
- data acquisition device 5 might comprise an MRI device, a CT scanner or any other scanning device of the sort adapted to provide a series of 2-D images of the patient's anatomical structure, where each 2-D image corresponds to a plane or slice taken through the scanned structure.
- data acquisition device 5 might comprise an X-ray machine or other imaging machine adapted to provide a set of 2-D images of the patient's anatomical structure, where each 2-D image corresponds to a data set relating to the scanned structure.
- patient-specific 2-D images generated by data acquisition device 5 are stored in a patient-specific database 10 .
- Patient-specific database 10 is constructed so that the patient-specific 2-D images can be accessed individually or in any particular sequence desired.
- patient-specific database 10 comprises an appropriate set of data contained in a computer storage system.
- Patient-specific 2-D images stored in patient-specific database 10 are used in building a patient-specific 3-D computer model 15 .
- Patient-specific 3-D computer model 15 may be built from the information stored in patient-specific database 10 using any one of the many algorithms well known in the art, so long as the patient-specific 3-D computer model is capable of generating virtual images of the patient's anatomical structure from substantially any location where a real camera may be positioned relative to that anatomical structure during a given procedure.
- patient-specific 3-D computer model 15 may be built from the information stored in patient-specific database 10 using any one of the many algorithms well known in the art, where the patient-specific 3-D computer model is capable of generating virtual images of the patient's anatomical structure from a predetermined set of possible virtual camera positions.
- patient-specific 3-D computer model 15 is constructed so that it is capable of generating virtual images of the patient's anatomical structure from any location whatsoever, with any desired field of view, and including any other camera-specific criteria desired (e.g. particular camera characteristics such as focus, optical characteristics, optical distortions, etc.).
- camera-specific criteria e.g. particular camera characteristics such as focus, optical characteristics, optical distortions, etc.
- one possible algorithm for building the patient-specific 3-D computer model out of the information stored in the patient-specific database 10 might be the Marching Cubes algorithm.
- patient-specific 3-D computer model 15 comprises an appropriate computer software model resident on an appropriate digital computer.
- Patient-specific 3-D computer model 15 is preferably constructed using a plurality of polygonal models to model the patient's real anatomical structure.
- polygonal models generally comprise a collection of points defining the surface of the 3-D computer model, along with some connectivity information relating to how these surface points are connected to one another.
- the patient-specific 3-D computer model could be dynamic as well as static.
- dynamic changes could occur in the model as a result of computer simulation (e.g. through the use of a Finite Element Model) or because more current real-time data (e.g. from video camera 45 ) is used to update the model, or both.
- patient-specific 3-D computer model 15 might include data from sources other than the 2-D images contained in patient-specific database 10 .
- patient-specific 3-D computer model 15 might include information obtained from a 3-D surface digitizer such as that used in certain types of cranial surgery, or patient-specific 3-D computer model 15 might include information obtained from a temperature probe or a chemical probe.
- patient-specific 3-D computer model 15 might include substantially any type of information gathered by almost any type of device or sensor or process.
- An integral part of the present invention involves the placement of virtual planning markers into the patient-specific data, using apparatus 20 . This may be done by placing such virtual planning markers into one or more of the patient-specific 2-D images contained in patient-specific database 10 before patient-specific 3-D computer model 15 is created; or by placing such virtual planning markers into patient-specific 3-D computer model 15 after that model has been created; or by some combination of the two.
- apparatus 20 preferably comprises computer hardware and software adapted to allow a physician to access one or more of the patient-specific 2-D images contained in patient-specific database 10 and present them for viewing, in the manner shown in FIG. 2 . Then, using a mouse or other data entry device, the physician can place one or more virtual planning markers 25 into an accessed patient-specific 2-D image, in the manner shown in FIG. 3 .
- These virtual planning markers 25 can consist of substantially any geometric form such as a point, a line, a circle, a plane, a path (either straight or curved) etc., and are positioned about anatomical structures of particular interest to the physician.
- virtual planning markers 25 might be placed about a suspected tumor, or they might be placed about particularly sensitive anatomical structures such as vascular tissue or nerve bundles, etc.
- the particular geometric form chosen for a particular virtual planning marker may depend on a variety of factors, such as the particular anatomical structure involved, the computer hardware available, the volume of data to be handled, the particular medical procedure to be conducted, etc.
- each virtual planning marker 25 which is placed on the 2-D image using apparatus 20 is then stored in patient-specific database 10 along with its associated 2-D image.
- patient-specific 3-D computer model 15 is subsequently created from the data contained in patient-specific database 10 , patient-specific 3-D computer model 15 will incorporate virtual planning markers 25 into the model.
- a physician may use apparatus 20 to access patient-specific 3-D computer model 15 and then, using a mouse or other data entry device, adjust the position of one or more of the virtual planning markers 25 which were previously placed into patient-specific database 10 (and thus incorporated into patient-specific 3-D computer model 15 when that 3-D computer model was created).
- the adjusted positions of these virtual planning markers 25 are then stored in patient-specific 3-D computer model 15 .
- an updated 2-D image of these virtual planning markers can then be incorporated into the patient-specific database 10 .
- patient-specific 3-D computer model 15 has not already had a particular virtual planning marker 25 incorporated therein, a physician may also use apparatus 20 to access patient-specific 3-D computer model 15 , and then use a mouse or other data entry device to insert one or more virtual planning markers 25 directly into patient-specific 3-D computer model 15 .
- virtual images incorporating these virtual planning markers can be created by an image generator 30 , fed through a video mixing device 35 , and then presented to the physician on display 40 .
- real images obtained from a video camera 45 can be fed through video mixing device 35 and then presented to the physician on display 40 .
- image generator 30 comprises computer hardware and software of the sort well known in the art for generating virtual images from patient-specific 3-D computer model 15 .
- Video mixing device 35 comprises a video mixing device of the sort well known in the art, whereby the surgeon can selectively display (i) a virtual image created by image generator 30 from patient-specific 3-D computer model 15 , (ii) a real image obtained from video camera 45 , or (iii) a composite of the virtual image and the real image, where the virtual image is superimposed against the real image.
- the virtual image generated from patient-specific 3-D computer model 15 should be placed into registration with the real image obtained from video camera 45 . This is done in several steps.
- a virtual image 50 is created by image generator 30 , fed through video mixing device 35 , and presented on display 40 .
- a real image 55 is captured by video camera 45 , fed through video mixing device 35 , and presented on display 40 .
- Video mixing device 35 is arranged so as to present virtual image 50 and real image 55 simultaneously on display 40 , with the virtual image being superimposed on the real image (i.e., so that the virtual image is in the foreground and the real image is in the background, in the manner shown in FIG. 4 ).
- the two images are placed into proper registration with one another.
- This image registration can be accomplished in either one of two ways.
- a first, and generally more preferable, technique involves holding the position of video camera 45 (and hence real image 55 ) constant and moving the position of the “virtual object” or the “virtual camera” by means of apparatus 57 until the virtual image 50 is brought into registration with real image 55 .
- apparatus 57 includes computer hardware and software of the sort well known in the art to cause image generator 30 to work through a search algorithm to match the virtual image to the real image; or it can be done manually, in which case apparatus 57 includes computer hardware and software of the sort well known in the art to allow the physician to drag the virtual image 50 into registration with real image 55 , using a mouse or other data entry device attached to image generator 30 .
- One convenient way that mouse motion can be used to control the 3-D movements or the patient-specific model is to map such motion into vectors defined by the view direction of the virtual camera, as will be well known to persons skilled in the art.
- a second, and generally less preferable, technique involves holding the virtual image 50 constant and moving video camera 45 (or the patient) until real image 55 matches virtual image 50 .
- image generator 30 can be used to present the virtual and real images on display 40 in various presentation formats so as to facilitate a particular medical procedure.
- image generator 30 can use image generator 30 , video camera 45 and video mixing device 35 to superimpose a virtual image (generated from patient-specific 3-D computer model 15 ) against a real image (generated by video camera 45 ), with image generator 30 being directed to modify the virtual image so as to expose one or more of the virtual planning markers 25 present in patient-specific 3-D computer model 15 , whereby the anatomy highlighted by virtual planning markers 25 will be brought quickly to the attention of the physician.
- exposing the otherwise-hidden virtual planning markers to view will involve rendering some or all of the virtual anatomical structure transparent or semi-transparent. This can be done either by (i) slicing away any portions of the virtual image required in order to expose virtual planning markers 25 , whereby those virtual planning markers will be rendered visible against the background real image, in the manner shown in FIG. 6 ; or (ii) fading away some or all of the virtual image so as to expose such virtual planning markers against the real image, in the manner shown in FIG. 7 .
- a physician can then conduct a medical procedure with the confidence of using real images generated by video camera 45 , while having virtual planning markers 25 superimposed against the real image of the anatomical structure so as to help guide the procedure.
- the position of video camera 45 may change on a fairly frequent basis. Since it is important for virtual image 50 (generated from patient-specific 3-D computer model 15 ) to remain in proper registration with real image 55 (obtained from video camera 45 ), it is necessary for the video-based surgical targeting system to reestablish proper correspondence between the two images each time the video camera moves. As noted above, this proper correspondence can be reestablished each time video camera 45 moves, by either (i) having the physician manually drag virtual image 50 into registration with real image 55 , using a mouse or other data entry device attached to image generator 30 , or (ii) having the system use a search algorithm to match the virtual image to the real image.
- Tracker system 65 comprises a tracker 70 which is attached to video camera 45 , and a tracker base 75 which defines the coordinate system of the tracker system.
- M PC can be considered to represent the matrix transformation from the patient's anatomical structure 60 to camera 45 ;
- M CT can be considered to represent the matrix transformation from camera 45 to tracker base 75 ; and
- M PT can be considered to represent the matrix transformation from anatomical structure 60 to tracker base 75 .
- M CT is known from the tracker system. Furthermore, once the virtual image generated by image generator 30 has been placed in registration with the real image generated by camera 45 , the virtual camera position will be known relative to the virtual anatomical structure, and hence the real camera position will be known relative to the real anatomical structure. Thus, real matrix M PC will also be known. In addition, since M CT and M PC are then both known, it is possible to solve for M PT . Accordingly, the position of anatomical structure 60 will then also be known within the relative coordinate system defined by the tracker system.
- the virtual image generated by image generator 30 can be quickly and easily maintained in registration with the real image, regardless of how often camera 45 is moved.
- tracker 80 can be positioned on a surgical instrument 85 .
- tracker 80 will provide M T1 , where M T1 represents the matrix transformation from tracker base 75 to instrument 85 .
- M T1 represents the matrix transformation from tracker base 75 to instrument 85 .
- M P1 represents the matrix transformation from the patient's anatomical structure 60 to instrument 85 .
- M P1 known it is then possible to track the position of surgical instrument 85 relative to anatomical structure 60 .
- surgical instrument 85 As the position of surgical instrument 85 is tracked relative to anatomical structure 60 , it is also possible to provide a virtual image of surgical instrument 85 as that surgical instrument 85 moves through the anatomical structure, even when some or all of that instrument might be hidden from the view of video camera 45 .
- FIG. 9 it is possible to generate a virtual “over the shoulder” view of the distal tip of surgical instrument 85 moving through anatomical structure 60 , where that tip will be shown in proper correspondence to various structures, e.g. a tumor 95 , vascular structures 100 , etc.
- an additional virtual object 90 showing a line extended along the axis of the surgical instrument 85 so that the physician can see where the surgical instrument would go if moved further along its current trajectory.
- video mixing device 35 should be directed to totally suppress the real image generated by video camera 45 so that it will no longer be shown on display 40 , in order to avoid confusing the physician.
- camera 45 may comprise a video camera, or it may comprise an endoscope, or it may comprise some other type of real-time image capturing means, e.g. it may comprise an ultrasound device.
- patient-specific database 10 comprises non-2-D images, e.g. database 10 might comprise one or more images generated by a Cyberware 3-D scanner.
- patient-specific 3-D computer model 15 might be omitted from the apparatus entirely.
- the virtual images could be generated from patient-specific database 10 by image generator 30 through the use of a volume rendering procedure of the sort well known in the art.
- a tracked surgical instrument 85 to determine the location of anatomical structure 60 . This can be accomplished by using the tracked surgical instrument 85 to engage known fiducial points on the anatomical structure. Alternatively, a tracked surgical instrument 85 can be used to sample multiple surface points located on the anatomical structure and then use a data matching procedure to correlate the sampled points with either patient-specific database 10 or patient-specific 3-D computer model 15 . By way of example, a least squares fit might be used to correlate the sampled points with the 3-D computer model.
- the video-based targeting system could be used to target objects in non-medical applications, e.g. it could be used to target objects concealed within the interior of complex machines, or objects (e.g. a conduit) concealed beneath surface of a structure (e.g. the floor of a building).
- tracking means directly on the patient so as to track the position of the patient's anatomical structure.
- a surgical targeting system is provided to facilitate locating a particular anatomical structure during a medical procedure.
- a video-based surgical targeting system is provided to facilitate locating a particular anatomical structure during a medical procedure.
- a video-based surgical targeting system which permits a series of patient-specific 2-D images (obtained by scanning patient anatomy using one or more scanning devices of (the type described above) to be assembled into a 3-D computer model of the patient's scanned structure.
- a video-based surgical targeting system which allows a physician to view the aforementioned patient-specific 2-D images on a display in any desired access sequence.
- a video-based surgical targeting system which allows a physician to assemble a series of patient-specific 2-D images into a patient-specific database, and then to generate virtual images from the aforementioned patient-specific database, as seen from any desired virtual camera position, for viewing on a display.
- a video-based surgical targeting system which allows a physician to generate virtual images from the aforementioned 3-D computer model, as seen from any desired virtual camera position, for viewing on a display.
- a video-based surgical targeting system which permits a physician to place virtual planning markers about any sites of interest while viewing one or more of the aforementioned patient-specific 2-D images, with those virtual planning markers then being incorporated into the 3-D computer model, whereby those virtual planning markers can be displayed in their appropriate 3-D positions when generating virtual images of the 3-D computer model.
- a video-based surgical targeting system which permits a physician to place virtual planning markers about any sites of interest while viewing virtual images of the 3-D computer model, with those virtual planning markers then being incorporated into the 3-D computer model, whereby those virtual planning markers can be displayed in their appropriate positions when subsequently generating virtual images of the 3-D computer model.
- a video-based surgical targeting system which permits a physician to place virtual planning markers about any sites of interest while viewing virtual images of the 3-D computer model, with those virtual planning markers then being incorporated into the 3-D computer model and into the database of 2-D images, whereby those virtual planning markers can be displayed in their appropriate positions when subsequently generating virtual images of the 3-D computer model or when subsequently displaying 2-D images from the patient-specific database.
- a video-based surgical targeting system which permits a real image obtained by a real-time imaging device (e.g. a video camera) to be displayed to a physician, and which permits a virtual image generated from the 3-D computer model to be displayed to a physician, according to the directive of the physician.
- a real-time imaging device e.g. a video camera
- a video-based surgical targeting system which permits a real image obtained by a real-time imaging device (e.g. a video camera) to be appropriately merged with a corresponding virtual image generated from the 3-D computer model.
- a real-time imaging device e.g. a video camera
- a video-based surgical targeting system which permits a real image obtained by a real-time imaging device (e.g. a video camera) to be appropriately merged with a corresponding virtual image generated from the 3-D computer model, whereby the two images will be in registration with one another.
- a real-time imaging device e.g. a video camera
- a video-based surgical targeting system which permits a real image obtained by a real-time imaging device (e.g. a video camera) to be merged with a corresponding virtual image generated from the 3-D computer model, whereby the two images will be in registration with one another, and whereby the physician can choose to display either one of the two images exclusive of the other, or a composite of both images simultaneously.
- a real-time imaging device e.g. a video camera
- a video-based surgical targeting system which permits a real image obtained by a real-time imaging device (e.g. a video camera) to be merged with a corresponding virtual image generated from the 3-D computer model, whereby the two images will be simultaneously displayed in registration with one another, and whereby the physician can modify the virtual image generated from the 3-D computer model as needed, by clipping or fading, so as to expose the virtual planning markers to view, with the virtual planning markers being superimposed on the real image generated by the real-time imaging device.
- a real-time imaging device e.g. a video camera
- a video-based surgical targeting system which permits a real image generated by a real-time imaging device (e.g. a video camera) to be merged with a corresponding virtual image generated from the 3-D computer model, whereby the two images will be simultaneously displayed in registration with one another, and whereby the physician can modify the virtual image generated from the 3-D computer model so as to expose only the virtual planning markers to view, with the virtual planning markers being superimposed on the real image generated by the real-time imaging device.
- a real-time imaging device e.g. a video camera
- a video-based surgical targeting system which permits a real image obtained by a real-time imaging device (e.g. a video camera) to be merged with a corresponding virtual image generated from the 3-D computer model, whereby the two images will be simultaneously displayed in registration with one another, and whereby this registration will be automatically maintained even as the real-time imaging device is moved about relative to the anatomical site, with the virtual image being automatically generated so as to follow the real image.
- a real-time imaging device e.g. a video camera
- a video-based surgical targeting system which permits a real image obtained by a real-time imaging device (e.g. a video camera) to be merged with a corresponding virtual image generated from the 3-D computer model, whereby the two images will be simultaneously displayed in registration with one another, and whereby this registration will be automatically maintained through the use of a computerized position and orientation tracker connected to the imaging device even as the real-time imaging device is moved about relative to the anatomical site, with the virtual image being automatically generated so as to follow the real image.
- a real-time imaging device e.g. a video camera
- a video-based surgical targeting system which permits a real image obtained by a real-time imaging device (e.g. a video camera) to be merged with a corresponding virtual image generated from the 3-D computer model, whereby the two images will be simultaneously displayed in registration with one another, and whereby this registration will be automatically maintained through the use of a computer search algorithm based on the real image and the virtual image even as the real-time imaging device is moved about relative to the anatomical site, with the virtual image being automatically generated so as to follow the real image.
- a real-time imaging device e.g. a video camera
- a video-based surgical targeting system which permits a real image obtained by a real-time imaging device (e.g. a video camera) to be merged with a corresponding virtual image generated from the 3-D computer model, whereby the two images will be simultaneously displayed in registration with one another, and whereby a surgical instrument can be tracked along with the real-time imaging device in order that the surgical instrument can be guided about the anatomical site using the virtual image generated from the 3-D computer model.
- a real-time imaging device e.g. a video camera
- a video-based surgical targeting system which permits a virtual image to be generated from the 3-D computer model, and further wherein this virtual image can correspond to an “over the shoulder” view of the working tip of a surgical instrument located at the anatomical site.
- data acquisition device 5 was used to generate patient-specific 2-D images, and these images were stored in patient-specific database 10 ; the patient-specific 2-D images stored in patient-specific database 10 were used to build patient-specific 3-D computer model 15 viewable with image generator 30 ; and the output of image generator 30 and video source 45 were connected through video mixing device 35 , whereby the surgeon can selectively display (i) a virtual image created by image generator 30 from patient-specific 3-D computer model 15 , or (ii) a real image obtained from video source 45 , or (iii) a composite of the virtual image and the real image, where the virtual image is superimposed against the real image.
- means 20 are provided for the placement of virtual planning markers 25 into the patient-specific data; this may be done by placing such virtual planning markers into one or more of the patient-specific 2-D images contained in patient-specific database 10 , or by placing such virtual planning markers into patient-specific 3-D computer model 15 , or by some combination of the two.
- virtual planning markers 25 placed in the patient-specific 2-D images stored in patient-specific database 10 will be automatically incorporated into patient-specific 3-D computer model 15
- virtual planning markers 25 placed into patient-specific 3-D computer model 15 will be automatically incorporated into the patient-specific 2-D images stored in patient-specific database 10 .
- video source 45 may comprise a video camera, an endoscope, an ultrasound device, a fluoroscope, or any other type of appropriate image capturing means.
- FIG. 11 there is shown the aorta 105 of a patient as generated by a video source 45 , e.g., a fluoroscope. Also shown is a series of sphere-like markers 25 A placed into the system (e.g., by a human operator using a mouse) and a series of line segments 25 B extending between selected ones of the sphere-like markers 25 A. These sphere-like markers 25 A and line segments 25 B may be used to plan a surgical procedure, to determine anatomical lengths or angles, etc.
- a video source 45 e.g., a fluoroscope.
- a series of sphere-like markers 25 A placed into the system (e.g., by a human operator using a mouse) and a series of line segments 25 B extending between selected ones of the sphere-like markers 25 A.
- These sphere-like markers 25 A and line segments 25 B may be used to plan a surgical procedure, to determine anatomical lengths or angles, etc.
- a straight tube 25 C which may also be used for planning and measurement purposes, etc.
- a curved tube 25 D which may be used for planning and measurement purposes
- a box 25 E which may be used for planning and measurement purposes, e.g., for volume calculations.
- FIG. 12 there is shown a virtual graft 25 F which represents an arterial stent which may be deployed in the aorta, e.g., to treat an aortic aneurysm.
- FIG. 12 is a view showing a virtual graft 25 F positioned on aorta 105 .
Abstract
A video-based surgical targeting system for targeting anatomical structures within a patient.
Description
- This application:
- (1) is a continuation in part-of-pending prior U.S. patent application Ser. No. 09/746,992, filed Dec. 21, 2000 by David T. Chen et al. for VIDEO-BASED SURGICAL TARGETING SYSTEM (Attorney's Docket No. MMS-4 CON 2), which patent application is hereby incorporated herein by reference; and
- (2) claims benefit of prior pending U.S. Provisional Patent Application Ser. No. 60/277,664, filed Mar. 21, 2001 by David T. Chen et al. for VIDEO-BASED SURGICAL TARGETING SYSTEM (Attorney's Docket No. MMS-B PROV), which patent application is hereby incorporated herein by reference.
- This invention relates to surgical systems in general, and more particularly to surgical targeting systems.
- Many medical procedures must be carried out at an interior site which is normally hidden from the view of the physician. In these situations, the physician typically uses some sort of scanning device to examine the patient's anatomy at the interior site prior to, and in preparation for, conducting the actual procedure itself. These scanning devices typically include MRI devices, CT scanners, X-ray machines, ultrasound devices and the like, and serve to provide the physician with a preliminary knowledge of the patient's internal anatomical structure prior to commencing the procedure. The physician can then use this information to plan the procedure in advance, taking into account patient-specific anatomical structure. In addition, the physician can also use the information obtained from these scans to more precisely identify the location of selected structures (e.g. tumors and the like) which may themselves be located within internal organs or other internal body structures. As a result, the physician can more easily “zero in” on such selected structures during the subsequent procedure, with less trauma to host organs or other internal body structures. Furthermore, in many cases the structures of interest may be quite small and difficult to identify with the naked eye. In these cases, preliminary scanning of the patient's internal anatomy using high resolution scanning devices can help the physician locate such structures during the subsequent procedure.
- In general, scanning devices of the sort described above tend to generate a series of two-dimensional (i.e., “2-D”) images of the patient's anatomical structure. For example, CT scanners generate 2-D images which are viewed directly by the physician. By viewing a plurality of these 2-D images, the physician can mentally generate a three-dimensional (i.e., “3-D”) sense of the patient's anatomical structure.
- Some scanning devices create large numbers of 2-D images during the scanning process, with each 2-D image representing a plane or slice taken through the scanned structure. Furthermore, some scanning devices also have associated computer hardware and software for building a 3-D computer model of the patient's scanned structure using a plurality of these 2-D images. For example, some MRI devices and some CT scanners leave such associated computer hardware and software. In these cases, an operator using this scanning equipment and associated computer hardware and software can create a number of 2-D images, assemble them into a 3-D computer model of the scanned structure, and then generate various images of that 3-D computer model as seen from various angles so as to enhance understanding of the patient's anatomical structure.
- While the information generated by the aforementioned scanning devices is generally of great benefit to physicians, certain significant limitations still exist. For one thing, it can be very difficult for a physician to physically locate a particular anatomical structure during a medical procedure, even when that structure is readily identifiable in a scanned image. This may be because the structure is very small and difficult to see with the naked eye, or because it is not readily visible against a particular background, or because it is itself located within an internal organ or other internal body structure, etc. For another thing, even when the structure of interest is successfully located by the physician, it can sometimes still be extremely difficult for the physician to reliably direct a medical instrument to that structure. This may be because the structure is quite small and difficult to target accurately, or because intervening body structure mandates a complex approach, etc.
- Accordingly, one object of the present invention is to provide a surgical targeting system to facilitate locating a particular anatomical structure during a medical procedure.
- Another object of the present invention is to provide a video-based surgical targeting system to facilitate locating a particular anatomical structure during a medical procedure.
- And another object of the present invention is to provide a video-based surgical targeting system which permits a series of patient-specific 2-D images (obtained by scanning patient anatomy using one or more scanning devices of the type described above) to be assembled into a 3-D computer model of the patient's scanned structure.
- Still another object of the present invention is to provide a video-based surgical targeting system which allows a physician to view the aforementioned patient-specific 2-D images on a display in any desired access sequence.
- And another object of the present invention is to provide a video-based surgical targeting system which allows a physician to assemble a series of patient-specific 2-D images into a patient-specific database, and then to generate virtual images from the aforementioned patient-specific database, as seen from any desired virtual camera position, for viewing on a display.
- Yet another object of the present invention is to provide a video-based surgical targeting system which allows a physician to generate virtual images from the aforementioned 3-D computer model, as seen from any desired virtual camera position, for viewing on a display.
- And another object of the present invention is to provide a video-based surgical targeting system which permits a physician to place virtual planning markers about any sites of interest while viewing one or more of the aforementioned patient-specific 2-D images, with those virtual planning markers then being incorporated into the 3-D computer model, whereby those virtual planning markers can be displayed in their appropriate 3-D positions when generating virtual images of the 3-D computer model.
- Still another object of the present invention is to provide a video-based surgical targeting system which permits a physician to place virtual planning markers about any sites of interest while viewing virtual images of the 3-D computer model, with those virtual planning markers then being incorporated into the 3-D computer model, whereby those virtual planning markers can be displayed in their appropriate positions when generating subsequent virtual images of the 3-D computer model.
- Still another object of the present invention is to provide a video-based surgical targeting system which permits a physician to place virtual planning markers about any sites of interest while viewing virtual images of the 3-D computer model, with those virtual planning markers then being incorporated into the 3-D computer model and into the database of 2-D images, whereby those virtual planning markers can be displayed in their appropriate positions when subsequently generating virtual images of the 3-D computer model or when subsequently displaying 2-D images from the patient-specific database.
- Yet another object of the present invention is to provide a video-based surgical targeting system which permits a real image obtained by a real-time imaging device (e.g. a video camera) to be displayed to a physician, and which permits a virtual image generated from the 3-D computer model to be displayed to a physician, according to the directive of the physician.
- Another object of the present invention is to provide a video-based surgical targeting system which permits a real image obtained by a real-time imaging device (e.g. a video camera) to be appropriately merged with a corresponding virtual image generated from the 3-D computer model.
- And another object of the present invention is to provide a video-based surgical targeting system which permits a real image obtained by a real-time imaging device (e.g. a video camera) to be appropriately merged with a corresponding virtual image generated from the 3-D computer model, whereby the two images will be in registration with one another.
- Still another object of the present invention is to provide a video-based surgical targeting system which permits a real image obtained by a real-time imaging device (e.g. a video camera) to be merged with a corresponding virtual image generated from the 3-D computer model, whereby the two images will be in registration with one another, and whereby the physician can choose to display either one of the two images exclusive of the other, or a composite of both images simultaneously.
- Yet another object of the present invention is to provide a video-based surgical targeting system which permits a real image obtained by a real-time imaging device (e.g. a video camera) to be merged with a corresponding virtual image generated from the 3-D computer model, whereby the two images will be simultaneously displayed in registration with one another, and whereby the physician can modify the virtual image generated from the 3-D computer model as needed, by clipping or fading, so as to expose the virtual planning markers to view, with the virtual planning markers being superimposed on the real image generated by the real-time viewing device.
- And another object of the present invention is to provide a video-based surgical targeting system which permits a real image generated by a real-time imaging device (e.g. a video camera) to be merged with a corresponding virtual image generated from the 3-D computer model, whereby the two images will be simultaneously displayed in registration with one another, and whereby the physician can modify the virtual image generated from the 3-D computer model so as to expose only the virtual planning markers to view, with the virtual planning markers being superimposed on the real image generated by the real-time imaging device.
- Still another object of the present invention is to provide a video-based surgical targeting system which permits a real image obtained by a real-time imaging device (e.g. a video camera) to be merged with a corresponding virtual image generated from the 3-D computer model, whereby the two images will be simultaneously displayed in registration with one another, and whereby this registration will be automatically maintained even as the real-time imaging device is moved about relative to the anatomical site, with the virtual image being automatically generated so as to follow the real image.
- Still another object of the present invention is to provide a video-based surgical targeting system which permits a real image obtained by a real-time imaging device (e.g. a video camera) to be merged with a corresponding virtual image generated from the 3-D computer model, whereby the two images will be simultaneously displayed in registration with one another, and whereby this registration will be automatically maintained through the use of a computerized position and orientation tracker connected to the imaging device even as the real-tine imaging device is moved about relative to the anatomical site, with the virtual image being automatically generated so as to follow the real image.
- Still another object of the present invention is to provide a video-based surgical targeting system which permits a real image obtained by a real-time imaging device (e.g. a video camera) to be merged with a corresponding virtual image generated from the 3-D computer model, whereby the two images will be simultaneously displayed in registration with one another, and whereby this registration will be automatically maintained through the use of a computer search algorithm based on the real image and the virtual image even as the real-time imaging device is moved about relative to the anatomical site, with the virtual image being automatically generated so as to follow the real image.
- Yet another object of the present invention is to provide a video-based surgical targeting system which permits a real image obtained by a real-tine imaging device (e.g. a video camera) to be merged with a corresponding virtual image generated from the 3-D computer model, whereby the two images will be simultaneously displayed in registration with one another, and whereby a surgical instrument can be tracked along with the real-time imaging device in order that the surgical instrument can be guided about the anatomical site using the virtual image generated from the 3-D computer model.
- And another object of the present invention is to provide a video-based surgical targeting system which permits a virtual image to be generated from the 3-D computer model, and further wherein this virtual image can correspond to an “over the shoulder” view of the working tip of a surgical instrument located at the anatomical site.
- And another object of the present invention is to provide an improved method for locating anatomical structures during a medical procedure.
- These and other objects of the present invention are achieved through the provision and use of a novel video-based surgical targeting system.
- In one form of the invention, the video-based surgical targeting system comprises a patient-specific database comprising a plurality of 2-D images of the anatomical structure of a patient; a patient-specific 3-D computer model of the anatomical structure of the patient, the patient-specific 3-D computer model being assembled from the plurality of 2-D images contained in the patient-specific database; marker placement means for (i) inserting virtual planning markers into the 2-D images contained in the patient-specific database, and/or (ii) adjusting the positions of virtual planning markers inserted into the 2-D images contained in the patient-specific database and thereafter incorporated into the patient-specific 3-D computer model, or inserting virtual planning markers into the 3-D computer model; an image generator for generating a virtual image of the anatomical structure modeled by the patient-specific 3-D computer model; real-time image generating means for generating a real image of the anatomical structure of a patient; video mixing means for mixing the virtual image and the real image into an output image, whereby the output image may comprise either one of the two images exclusive of the other, or a composite of both images; and display means for displaying the output image.
- Preferably the video-based surgical targeting system also includes registration means for placing the virtual image in registration with the real image. These registration means may include means for manually aligning the virtual image with the real image, or means for automatically aligning the virtual image with the real image. The registration means may also include tracking means for tracking the position of the real-time image generating means.
- Preferably, the video-based surgical targeting system further comprises instrument tracking means for tracking the position of a surgical instrument.
- In an alternative form of the invention, the video-based surgical targeting system comprises a patient-specific database comprising a plurality of 2-D images of the anatomical structure of a patient; marker placement means for inserting virtual planning markers into the 2-D images contained in the patient-specific database; an image generator for generating a virtual image of the anatomical structure represented by the plurality of 2-D images contained in the patient-specific database; real-time image generating means for generating a real image of the anatomical structure of a patient; video mixing means for mixing the virtual image and the real image into an output image, whereby the output image may comprise either one of the two images exclusive of the other, or a composite of both images; and display means for displaying the output image.
- In another alternative form of the invention, the video-based surgical targeting system comprises a patient-specific database comprising a plurality of images of the anatomical structure of a patient; a patient-specific 3-D computer model of the anatomical structure of the patient, the patient-specific 3-D computer model being assembled from the plurality of images contained in the patient-specific database; marker placement means for (i) inserting virtual planning markers into the images contained in the patient-specific database, and/or (ii) adjusting the positions of virtual planning markers inserted into the images contained in the patient-specific database and thereafter incorporated into the patient-specific 3-D computer model, or inserting virtual planning markers into the 3-D computer model; an image generator for generating a virtual image of the anatomical structure modeled by the patient-specific 3-D computer model; real-time image generating means for generating a real image of the anatomical structure of a patient; video mixing means for mixing the virtual image and the real image into an output image, whereby the output image may comprise either one of the two images exclusive of the other, or a composite of both images; and display means for displaying said output image.
- In yet another alternative form of the invention, the video-based surgical targeting system comprises a patient-specific database comprising a plurality of 2-D images of the anatomical structure of a patient; marker placement means for inserting virtual planning markers into the 2-D images contained in the patient-specific database; an image generator for generating a virtual image of the anatomical structure defined by the patient-specific database; real-time image generating means for generating a real image of the anatomical structure of a patient, the real-time image generating means being adapted so as to dynamically update the patient-specific database via the real image; video mixing means for mixing the virtual image and the real image into an output image, whereby the output image may comprise either one of the two images exclusive of the other, or a composite of both images; and display means for displaying said output image.
- The present invention also comprises a method for targeting an anatomical structure using the video-based surgical targeting system.
- These and other objects and features of the present invention will be more fully disclosed or rendered obvious by the following detailed description of the preferred embodiments of the invention, which are to be considered together with the accompanying drawings wherein like numbers refer to like parts and further wherein:
-
FIG. 1 is a schematic view of the major components of a video-based surgical targeting system formed in accordance with the present invention; -
FIG. 2 is a view of an exemplary patient-specific 2-D image of the sort contained in the patient-specific database; -
FIG. 3 is a view of an exemplary patient-specific 2-D image of the sort contained in the patient-specific database, wherein the image has had several virtual planning markers placed into the image; -
FIG. 4 is a composite view of (i) a real image obtained by a real-time imaging device (e.g. a video camera), and (ii) a virtual image generated from the 3-D computer model, wherein the two images are not yet in registration with one another; -
FIG. 5 is a composite view of (i) a real image obtained by a real-time imaging device (e.g. a video camera), and (ii) a virtual image generated from the 3-D computer model, wherein the two images have been placed in registration with one another; -
FIG. 6 is a view like that ofFIG. 5 , except that the virtual image generated from the 3-D computer model has been sliced away to expose the virtual planning markers; -
FIG. 7 is a view like that ofFIG. 5 , except that the virtual image generated from the 3-D computer model has been faded away to expose the virtual planning markers; -
FIG. 8 is a schematic view showing the positional relationships between various elements of the video-based surgical targeting system and the patient's anatomical structure, as well as the positional relationship between an exemplary surgical instrument and the patient's anatomical structure; -
FIG. 9 is a virtual image generated from the 3-D computer model, showing a virtual “over the shoulder” view from the tip of a tracked surgical instrument; and -
FIG. 10 is a flowchart illustrating one way of operating a video-based surgical targeting system formed in accordance with the present invention. - Looking first at
FIG. 1 , the starting point for the present invention involves adata acquisition device 5 which is adapted to generate patient-specific 2-D images. For example,data acquisition device 5 might comprise an MRI device, a CT scanner or any other scanning device of the sort adapted to provide a series of 2-D images of the patient's anatomical structure, where each 2-D image corresponds to a plane or slice taken through the scanned structure. Alternatively,data acquisition device 5 might comprise an X-ray machine or other imaging machine adapted to provide a set of 2-D images of the patient's anatomical structure, where each 2-D image corresponds to a data set relating to the scanned structure. - The patient-specific 2-D images generated by
data acquisition device 5 are stored in a patient-specific database 10. Patient-specific database 10 is constructed so that the patient-specific 2-D images can be accessed individually or in any particular sequence desired. Preferably patient-specific database 10 comprises an appropriate set of data contained in a computer storage system. - The patient-specific 2-D images stored in patient-
specific database 10 are used in building a patient-specific 3-D computer model 15. Patient-specific 3-D computer model 15 may be built from the information stored in patient-specific database 10 using any one of the many algorithms well known in the art, so long as the patient-specific 3-D computer model is capable of generating virtual images of the patient's anatomical structure from substantially any location where a real camera may be positioned relative to that anatomical structure during a given procedure. Alternatively, patient-specific 3-D computer model 15 may be built from the information stored in patient-specific database 10 using any one of the many algorithms well known in the art, where the patient-specific 3-D computer model is capable of generating virtual images of the patient's anatomical structure from a predetermined set of possible virtual camera positions. Preferably, patient-specific 3-D computer model 15 is constructed so that it is capable of generating virtual images of the patient's anatomical structure from any location whatsoever, with any desired field of view, and including any other camera-specific criteria desired (e.g. particular camera characteristics such as focus, optical characteristics, optical distortions, etc.). By way of example, one possible algorithm for building the patient-specific 3-D computer model out of the information stored in the patient-specific database 10 might be the Marching Cubes algorithm. - In one preferred embodiment of the invention, patient-specific 3-
D computer model 15 comprises an appropriate computer software model resident on an appropriate digital computer. Patient-specific 3-D computer model 15 is preferably constructed using a plurality of polygonal models to model the patient's real anatomical structure. As is well known in the art, such polygonal models generally comprise a collection of points defining the surface of the 3-D computer model, along with some connectivity information relating to how these surface points are connected to one another. Preferably there is one polygonal model per anatomical structure being modeled. Of course, the choice of which particular computer software model is used to model a particular anatomical structure ultimately depends on a variety of factors, such as the particular anatomical structure being modeled, the computer hardware available, the volume of data to be handled, the particular medical procedure which is to be conducted, etc. - Of course, it is also anticipated that the patient-specific 3-D computer model could be dynamic as well as static. For example, dynamic changes could occur in the model as a result of computer simulation (e.g. through the use of a Finite Element Model) or because more current real-time data (e.g. from video camera 45) is used to update the model, or both.
- It is also anticipated that patient-specific 3-
D computer model 15 might include data from sources other than the 2-D images contained in patient-specific database 10. For example, patient-specific 3-D computer model 15 might include information obtained from a 3-D surface digitizer such as that used in certain types of cranial surgery, or patient-specific 3-D computer model 15 might include information obtained from a temperature probe or a chemical probe. In fact, in addition to the 2-D image data obtained from patient-specific database 10, patient-specific 3-D computer model 15 might include substantially any type of information gathered by almost any type of device or sensor or process. - An integral part of the present invention involves the placement of virtual planning markers into the patient-specific data, using
apparatus 20. This may be done by placing such virtual planning markers into one or more of the patient-specific 2-D images contained in patient-specific database 10 before patient-specific 3-D computer model 15 is created; or by placing such virtual planning markers into patient-specific 3-D computer model 15 after that model has been created; or by some combination of the two. - More particularly,
apparatus 20 preferably comprises computer hardware and software adapted to allow a physician to access one or more of the patient-specific 2-D images contained in patient-specific database 10 and present them for viewing, in the manner shown inFIG. 2 . Then, using a mouse or other data entry device, the physician can place one or morevirtual planning markers 25 into an accessed patient-specific 2-D image, in the manner shown inFIG. 3 . Thesevirtual planning markers 25 can consist of substantially any geometric form such as a point, a line, a circle, a plane, a path (either straight or curved) etc., and are positioned about anatomical structures of particular interest to the physician. For example,virtual planning markers 25 might be placed about a suspected tumor, or they might be placed about particularly sensitive anatomical structures such as vascular tissue or nerve bundles, etc. Naturally, the particular geometric form chosen for a particular virtual planning marker may depend on a variety of factors, such as the particular anatomical structure involved, the computer hardware available, the volume of data to be handled, the particular medical procedure to be conducted, etc. Regardless of the particular form of virtual planning marker chosen, eachvirtual planning marker 25 which is placed on the 2-Dimage using apparatus 20 is then stored in patient-specific database 10 along with its associated 2-D image. As a result, when patient-specific 3-D computer model 15 is subsequently created from the data contained in patient-specific database 10, patient-specific 3-D computer model 15 will incorporatevirtual planning markers 25 into the model. - Alternatively, a physician may use
apparatus 20 to access patient-specific 3-D computer model 15 and then, using a mouse or other data entry device, adjust the position of one or more of thevirtual planning markers 25 which were previously placed into patient-specific database 10 (and thus incorporated into patient-specific 3-D computer model 15 when that 3-D computer model was created). The adjusted positions of thesevirtual planning markers 25 are then stored in patient-specific 3-D computer model 15. Furthermore, an updated 2-D image of these virtual planning markers can then be incorporated into the patient-specific database 10. - Furthermore, where patient-specific 3-
D computer model 15 has not already had a particularvirtual planning marker 25 incorporated therein, a physician may also useapparatus 20 to access patient-specific 3-D computer model 15, and then use a mouse or other data entry device to insert one or morevirtual planning markers 25 directly into patient-specific 3-D computer model 15. - Once
virtual planning markers 25 have been properly positioned in patient-specific 3-D computer model 15, virtual images incorporating these virtual planning markers can be created by animage generator 30, fed through avideo mixing device 35, and then presented to the physician ondisplay 40. - At the same time, real images obtained from a
video camera 45 can be fed throughvideo mixing device 35 and then presented to the physician ondisplay 40. - More particularly,
image generator 30 comprises computer hardware and software of the sort well known in the art for generating virtual images from patient-specific 3-D computer model 15. -
Video mixing device 35 comprises a video mixing device of the sort well known in the art, whereby the surgeon can selectively display (i) a virtual image created byimage generator 30 from patient-specific 3-D computer model 15, (ii) a real image obtained fromvideo camera 45, or (iii) a composite of the virtual image and the real image, where the virtual image is superimposed against the real image. - In order for the foregoing composite image to be more useful to the physician, the virtual image generated from patient-specific 3-
D computer model 15 should be placed into registration with the real image obtained fromvideo camera 45. This is done in several steps. - First, and looking now at
FIG. 4 , avirtual image 50 is created byimage generator 30, fed throughvideo mixing device 35, and presented ondisplay 40. Simultaneously, areal image 55 is captured byvideo camera 45, fed throughvideo mixing device 35, and presented ondisplay 40.Video mixing device 35 is arranged so as to presentvirtual image 50 andreal image 55 simultaneously ondisplay 40, with the virtual image being superimposed on the real image (i.e., so that the virtual image is in the foreground and the real image is in the background, in the manner shown inFIG. 4 ). - Second, and looking now at
FIG. 5 , the two images are placed into proper registration with one another. This image registration can be accomplished in either one of two ways. - A first, and generally more preferable, technique involves holding the position of video camera 45 (and hence real image 55) constant and moving the position of the “virtual object” or the “virtual camera” by means of
apparatus 57 until thevirtual image 50 is brought into registration withreal image 55. This can be done automatically by havingimage generator 30 use a search algorithm to match the virtual image to the real image, in whichcase apparatus 57 includes computer hardware and software of the sort well known in the art to causeimage generator 30 to work through a search algorithm to match the virtual image to the real image; or it can be done manually, in whichcase apparatus 57 includes computer hardware and software of the sort well known in the art to allow the physician to drag thevirtual image 50 into registration withreal image 55, using a mouse or other data entry device attached to imagegenerator 30. One convenient way that mouse motion can be used to control the 3-D movements or the patient-specific model is to map such motion into vectors defined by the view direction of the virtual camera, as will be well known to persons skilled in the art. - A second, and generally less preferable, technique involves holding the
virtual image 50 constant and moving video camera 45 (or the patient) untilreal image 55 matchesvirtual image 50. - Regardless of which technique is used, once
virtual image 50 has been matched toreal image 55, the position of the “virtual camera” (i.e., the location from which the virtual image appears to be seen) will be matched to the actual position ofvideo camera 45. - Once
virtual image 50 has been placed into proper registration withreal image 55,image generator 30,video camera 45 andvideo mixing device 35 can be used to present the virtual and real images ondisplay 40 in various presentation formats so as to facilitate a particular medical procedure. In particular, one can useimage generator 30,video camera 45 andvideo mixing device 35 to superimpose a virtual image (generated from patient-specific 3-D computer model 15) against a real image (generated by video camera 45), withimage generator 30 being directed to modify the virtual image so as to expose one or more of thevirtual planning markers 25 present in patient-specific 3-D computer model 15, whereby the anatomy highlighted byvirtual planning markers 25 will be brought quickly to the attention of the physician. - In this respect it is to be appreciated that since
virtual planning markers 25 will frequently be located within interior portions of the structure modeled by patient-specific 3-D computer model 15, exposing the otherwise-hidden virtual planning markers to view will involve rendering some or all of the virtual anatomical structure transparent or semi-transparent. This can be done either by (i) slicing away any portions of the virtual image required in order to exposevirtual planning markers 25, whereby those virtual planning markers will be rendered visible against the background real image, in the manner shown inFIG. 6 ; or (ii) fading away some or all of the virtual image so as to expose such virtual planning markers against the real image, in the manner shown inFIG. 7 . As a result, a physician can then conduct a medical procedure with the confidence of using real images generated byvideo camera 45, while havingvirtual planning markers 25 superimposed against the real image of the anatomical structure so as to help guide the procedure. - While the foregoing system constitutes a major improvement over the prior art, it has also been recognized that it can itself be improved upon significantly by adding camera tracking means to
video camera 45. - More particularly, in many medical procedures, the position of
video camera 45 may change on a fairly frequent basis. Since it is important for virtual image 50 (generated from patient-specific 3-D computer model 15) to remain in proper registration with real image 55 (obtained from video camera 45), it is necessary for the video-based surgical targeting system to reestablish proper correspondence between the two images each time the video camera moves. As noted above, this proper correspondence can be reestablished eachtime video camera 45 moves, by either (i) having the physician manually dragvirtual image 50 into registration withreal image 55, using a mouse or other data entry device attached to imagegenerator 30, or (ii) having the system use a search algorithm to match the virtual image to the real image. While both of these procedures are generally capable (assuming that the anatomical structure remains stationary) of accurately yielding the desired image correspondence, each one suffers from a significant disadvantage. In the first case, requiring the physician to physically drag the virtual image into registration with the real image each time the video camera moves can be inconvenient where the video camera is moved about frequently. In the second case, it can take the system a substantial amount of time to solve for the correct image correspondence when using a search algorithm, even where high speed hardware is utilized. Even where speed is not a problem, automated search strategies of the type generally used are prone to errors through false matches. - Accordingly, it has been recognized that by putting a tracker on
video camera 45, the position of the real camera can be continuously monitored, whereby each time the real camera moves, the position of the virtual camera can be correspondingly adjusted byimage generator 30. In this way, the virtual image generated byimage generator 30 can quickly and easily be maintained in registration with the real image, regardless of how oftenvideo camera 45 is moved. - More particularly, and looking now at
FIG. 8 , there is shown avideo camera 45, ananatomical structure 60, and atracker system 65.Tracker system 65 comprises atracker 70 which is attached tovideo camera 45, and atracker base 75 which defines the coordinate system of the tracker system. - In this setting, MPC can be considered to represent the matrix transformation from the patient's
anatomical structure 60 tocamera 45; MCT can be considered to represent the matrix transformation fromcamera 45 totracker base 75; and MPT can be considered to represent the matrix transformation fromanatomical structure 60 totracker base 75. - MCT is known from the tracker system. Furthermore, once the virtual image generated by
image generator 30 has been placed in registration with the real image generated bycamera 45, the virtual camera position will be known relative to the virtual anatomical structure, and hence the real camera position will be known relative to the real anatomical structure. Thus, real matrix MPC will also be known. In addition, since MCT and MPC are then both known, it is possible to solve for MPT. Accordingly, the position ofanatomical structure 60 will then also be known within the relative coordinate system defined by the tracker system. - In view of the foregoing, the virtual image generated by
image generator 30 can be quickly and easily maintained in registration with the real image, regardless of how oftencamera 45 is moved. - Furthermore, another
tracker 80 can be positioned on asurgical instrument 85. In this case,tracker 80 will provide MT1, where MT1 represents the matrix transformation fromtracker base 75 toinstrument 85. Accordingly, once MPT and MT1 are known, it is possible to solve for MP1, where MP1 represents the matrix transformation from the patient'sanatomical structure 60 toinstrument 85. With MP1 known, it is then possible to track the position ofsurgical instrument 85 relative toanatomical structure 60. This is a very powerful tool, since it allows a virtual representation ofsurgical instrument 85 to be accurately added to the properly registered virtual and real images presented ondisplay 40, i.e., it allows a virtual representation ofinstrument 85 to be shown in proper registration with a real image ofanatomical structure 60 ondisplay 40. - Furthermore, as the position of
surgical instrument 85 is tracked relative toanatomical structure 60, it is also possible to provide a virtual image ofsurgical instrument 85 as thatsurgical instrument 85 moves through the anatomical structure, even when some or all of that instrument might be hidden from the view ofvideo camera 45. For example, and looking now atFIG. 9 , it is possible to generate a virtual “over the shoulder” view of the distal tip ofsurgical instrument 85 moving throughanatomical structure 60, where that tip will be shown in proper correspondence to various structures, e.g. atumor 95,vascular structures 100, etc. It is also possible to generate an additionalvirtual object 90 showing a line extended along the axis of thesurgical instrument 85 so that the physician can see where the surgical instrument would go if moved further along its current trajectory. Of course, since the virtual image generated by the video-based surgical targeting system will then differ significantly from the real image generated byvideo camera 45,video mixing device 35 should be directed to totally suppress the real image generated byvideo camera 45 so that it will no longer be shown ondisplay 40, in order to avoid confusing the physician. - It is, of course, possible to modify the preferred embodiments disclosed above without departing from the scope of the present invention.
- Thus, for example, while only one
camera 45 has been disclosed above, it should be appreciated that two ormore cameras 45 may be used. Furthermore, it should also be appreciated thatcamera 45 may comprise a video camera, or it may comprise an endoscope, or it may comprise some other type of real-time image capturing means, e.g. it may comprise an ultrasound device. - Additionally, it should be recognized that the system might be adapted so that patient-
specific database 10 comprises non-2-D images,e.g. database 10 might comprise one or more images generated by a Cyberware 3-D scanner. - Alternatively, patient-specific 3-
D computer model 15 might be omitted from the apparatus entirely. In this case, the virtual images could be generated from patient-specific database 10 byimage generator 30 through the use of a volume rendering procedure of the sort well known in the art. - It is also anticipated that one could use a tracked
surgical instrument 85 to determine the location ofanatomical structure 60. This can be accomplished by using the trackedsurgical instrument 85 to engage known fiducial points on the anatomical structure. Alternatively, a trackedsurgical instrument 85 can be used to sample multiple surface points located on the anatomical structure and then use a data matching procedure to correlate the sampled points with either patient-specific database 10 or patient-specific 3-D computer model 15. By way of example, a least squares fit might be used to correlate the sampled points with the 3-D computer model. - Also, it should be appreciated that the video-based targeting system could be used to target objects in non-medical applications, e.g. it could be used to target objects concealed within the interior of complex machines, or objects (e.g. a conduit) concealed beneath surface of a structure (e.g. the floor of a building).
- Furthermore, it is possible to place tracking means directly on the patient so as to track the position of the patient's anatomical structure.
- Still other changes will be obvious to a person skilled in the art.
- Numerous advantages are obtained through the use of the present invention.
- For one thing, a surgical targeting system is provided to facilitate locating a particular anatomical structure during a medical procedure.
- For another thing, a video-based surgical targeting system is provided to facilitate locating a particular anatomical structure during a medical procedure.
- And a video-based surgical targeting system is provided which permits a series of patient-specific 2-D images (obtained by scanning patient anatomy using one or more scanning devices of (the type described above) to be assembled into a 3-D computer model of the patient's scanned structure.
- Also, a video-based surgical targeting system is provided which allows a physician to view the aforementioned patient-specific 2-D images on a display in any desired access sequence.
- Furthermore, a video-based surgical targeting system is provided which allows a physician to assemble a series of patient-specific 2-D images into a patient-specific database, and then to generate virtual images from the aforementioned patient-specific database, as seen from any desired virtual camera position, for viewing on a display.
- And a video-based surgical targeting system is provided which allows a physician to generate virtual images from the aforementioned 3-D computer model, as seen from any desired virtual camera position, for viewing on a display.
- In addition, a video-based surgical targeting system is provided which permits a physician to place virtual planning markers about any sites of interest while viewing one or more of the aforementioned patient-specific 2-D images, with those virtual planning markers then being incorporated into the 3-D computer model, whereby those virtual planning markers can be displayed in their appropriate 3-D positions when generating virtual images of the 3-D computer model.
- And a video-based surgical targeting system is provided which permits a physician to place virtual planning markers about any sites of interest while viewing virtual images of the 3-D computer model, with those virtual planning markers then being incorporated into the 3-D computer model, whereby those virtual planning markers can be displayed in their appropriate positions when subsequently generating virtual images of the 3-D computer model.
- And a video-based surgical targeting system is provided which permits a physician to place virtual planning markers about any sites of interest while viewing virtual images of the 3-D computer model, with those virtual planning markers then being incorporated into the 3-D computer model and into the database of 2-D images, whereby those virtual planning markers can be displayed in their appropriate positions when subsequently generating virtual images of the 3-D computer model or when subsequently displaying 2-D images from the patient-specific database.
- For another thing, a video-based surgical targeting system is provided which permits a real image obtained by a real-time imaging device (e.g. a video camera) to be displayed to a physician, and which permits a virtual image generated from the 3-D computer model to be displayed to a physician, according to the directive of the physician.
- And a video-based surgical targeting system is provided which permits a real image obtained by a real-time imaging device (e.g. a video camera) to be appropriately merged with a corresponding virtual image generated from the 3-D computer model.
- Also, a video-based surgical targeting system is provided which permits a real image obtained by a real-time imaging device (e.g. a video camera) to be appropriately merged with a corresponding virtual image generated from the 3-D computer model, whereby the two images will be in registration with one another.
- Furthermore, a video-based surgical targeting system is provided which permits a real image obtained by a real-time imaging device (e.g. a video camera) to be merged with a corresponding virtual image generated from the 3-D computer model, whereby the two images will be in registration with one another, and whereby the physician can choose to display either one of the two images exclusive of the other, or a composite of both images simultaneously.
- And a video-based surgical targeting system is provided which permits a real image obtained by a real-time imaging device (e.g. a video camera) to be merged with a corresponding virtual image generated from the 3-D computer model, whereby the two images will be simultaneously displayed in registration with one another, and whereby the physician can modify the virtual image generated from the 3-D computer model as needed, by clipping or fading, so as to expose the virtual planning markers to view, with the virtual planning markers being superimposed on the real image generated by the real-time imaging device.
- And a video-based surgical targeting system is provided which permits a real image generated by a real-time imaging device (e.g. a video camera) to be merged with a corresponding virtual image generated from the 3-D computer model, whereby the two images will be simultaneously displayed in registration with one another, and whereby the physician can modify the virtual image generated from the 3-D computer model so as to expose only the virtual planning markers to view, with the virtual planning markers being superimposed on the real image generated by the real-time imaging device.
- Moreover, a video-based surgical targeting system is provided which permits a real image obtained by a real-time imaging device (e.g. a video camera) to be merged with a corresponding virtual image generated from the 3-D computer model, whereby the two images will be simultaneously displayed in registration with one another, and whereby this registration will be automatically maintained even as the real-time imaging device is moved about relative to the anatomical site, with the virtual image being automatically generated so as to follow the real image.
- And a video-based surgical targeting system is provided which permits a real image obtained by a real-time imaging device (e.g. a video camera) to be merged with a corresponding virtual image generated from the 3-D computer model, whereby the two images will be simultaneously displayed in registration with one another, and whereby this registration will be automatically maintained through the use of a computerized position and orientation tracker connected to the imaging device even as the real-time imaging device is moved about relative to the anatomical site, with the virtual image being automatically generated so as to follow the real image.
- Also, a video-based surgical targeting system is provided which permits a real image obtained by a real-time imaging device (e.g. a video camera) to be merged with a corresponding virtual image generated from the 3-D computer model, whereby the two images will be simultaneously displayed in registration with one another, and whereby this registration will be automatically maintained through the use of a computer search algorithm based on the real image and the virtual image even as the real-time imaging device is moved about relative to the anatomical site, with the virtual image being automatically generated so as to follow the real image.
- And a video-based surgical targeting system is provided which permits a real image obtained by a real-time imaging device (e.g. a video camera) to be merged with a corresponding virtual image generated from the 3-D computer model, whereby the two images will be simultaneously displayed in registration with one another, and whereby a surgical instrument can be tracked along with the real-time imaging device in order that the surgical instrument can be guided about the anatomical site using the virtual image generated from the 3-D computer model.
- In addition, a video-based surgical targeting system is provided which permits a virtual image to be generated from the 3-D computer model, and further wherein this virtual image can correspond to an “over the shoulder” view of the working tip of a surgical instrument located at the anatomical site.
- And an improved method for locating anatomical structures during a medical procedure is provided.
- In the preceding description,
data acquisition device 5 was used to generate patient-specific 2-D images, and these images were stored in patient-specific database 10; the patient-specific 2-D images stored in patient-specific database 10 were used to build patient-specific 3-D computer model 15 viewable withimage generator 30; and the output ofimage generator 30 andvideo source 45 were connected throughvideo mixing device 35, whereby the surgeon can selectively display (i) a virtual image created byimage generator 30 from patient-specific 3-D computer model 15, or (ii) a real image obtained fromvideo source 45, or (iii) a composite of the virtual image and the real image, where the virtual image is superimposed against the real image. - In addition, means 20 are provided for the placement of
virtual planning markers 25 into the patient-specific data; this may be done by placing such virtual planning markers into one or more of the patient-specific 2-D images contained in patient-specific database 10, or by placing such virtual planning markers into patient-specific 3-D computer model 15, or by some combination of the two. In accordance with one aspect of the invention,virtual planning markers 25 placed in the patient-specific 2-D images stored in patient-specific database 10 will be automatically incorporated into patient-specific 3-D computer model 15, andvirtual planning markers 25 placed into patient-specific 3-D computer model 15 will be automatically incorporated into the patient-specific 2-D images stored in patient-specific database 10. - In this respect it should be appreciated that
video source 45 may comprise a video camera, an endoscope, an ultrasound device, a fluoroscope, or any other type of appropriate image capturing means. - It should also be appreciated that it is possible to create
virtual planning markers 25 having a form other than the sphere-like markers shown inFIG. 9 . - By way of example, and looking now at
FIG. 11 , there is shown theaorta 105 of a patient as generated by avideo source 45, e.g., a fluoroscope. Also shown is a series of sphere-like markers 25A placed into the system (e.g., by a human operator using a mouse) and a series ofline segments 25B extending between selected ones of the sphere-like markers 25A. These sphere-like markers 25A andline segments 25B may be used to plan a surgical procedure, to determine anatomical lengths or angles, etc. - Also shown is a
straight tube 25C which may also be used for planning and measurement purposes, etc., acurved tube 25D which may be used for planning and measurement purposes, and abox 25E which may be used for planning and measurement purposes, e.g., for volume calculations. - In addition, other geometric elements such as curved lines, intersecting lines, etc. may also be provided for planning and measurement purposes.
- Significantly, it is also possible to insert into the system virtual planning markers representing virtual grafts, virtual implants, etc. By way of example, in
FIG. 12 there is shown avirtual graft 25F which represents an arterial stent which may be deployed in the aorta, e.g., to treat an aortic aneurysm. -
FIG. 12 is a view showing avirtual graft 25F positioned onaorta 105.
Claims (1)
1. A video-based surgical targeting system comprising:
a patient-specific database comprising a plurality of 2-D images of the anatomical structure of a patient;
a patient-specific 3-D computer model of said anatomical structure of said patient, said patient-specific 3-D computer model being assembled from said plurality of 2-D images contained in said patient-specific database;
graft placement means for (i) inserting virtual grafts into said 2-D images contained in said patient-specific database, and/or (ii) or inserting virtual grafts into said 3-D computer model;
an image generator for generating a virtual image of the anatomical structure modeled by said patient-specific 3-D computer model;
real-time image generating means for generating a real image of the anatomical structure of a patient;
video mixing means for mixing said virtual image and said real image into an output image, whereby said output image may comprise either one of the two images exclusive of the other, or a composite of both images; and
display means for displaying said output image.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US10/775,754 US20050027186A1 (en) | 2000-12-21 | 2004-02-10 | Video-based surgical targeting system |
Applications Claiming Priority (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US09/746,992 US6675032B2 (en) | 1994-10-07 | 2000-12-21 | Video-based surgical targeting system |
US27766401P | 2001-03-21 | 2001-03-21 | |
US10/104,256 US6690960B2 (en) | 2000-12-21 | 2002-03-21 | Video-based surgical targeting system |
US10/775,754 US20050027186A1 (en) | 2000-12-21 | 2004-02-10 | Video-based surgical targeting system |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/104,256 Continuation US6690960B2 (en) | 2000-12-21 | 2002-03-21 | Video-based surgical targeting system |
Publications (1)
Publication Number | Publication Date |
---|---|
US20050027186A1 true US20050027186A1 (en) | 2005-02-03 |
Family
ID=26958631
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/104,256 Expired - Lifetime US6690960B2 (en) | 2000-12-21 | 2002-03-21 | Video-based surgical targeting system |
US10/775,754 Abandoned US20050027186A1 (en) | 2000-12-21 | 2004-02-10 | Video-based surgical targeting system |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/104,256 Expired - Lifetime US6690960B2 (en) | 2000-12-21 | 2002-03-21 | Video-based surgical targeting system |
Country Status (1)
Country | Link |
---|---|
US (2) | US6690960B2 (en) |
Cited By (21)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20040070611A1 (en) * | 2002-09-30 | 2004-04-15 | Canon Kabushiki Kaisha | Video combining apparatus and method |
US20050038337A1 (en) * | 2003-08-11 | 2005-02-17 | Edwards Jerome R. | Methods, apparatuses, and systems useful in conducting image guided interventions |
US20060119621A1 (en) * | 2003-05-30 | 2006-06-08 | Claude Krier | Method and device for displaying medical patient data on a medical display unit |
US20070060799A1 (en) * | 2005-09-13 | 2007-03-15 | Lyon Torsten M | Apparatus and method for automatic image guided accuracy verification |
US20090281566A1 (en) * | 2003-08-11 | 2009-11-12 | Edwards Jerome R | Bodily sealants and methods and apparatus for image-guided delivery of same |
US20100228257A1 (en) * | 2000-01-14 | 2010-09-09 | Bonutti Peter M | Joint replacement component |
US20100315416A1 (en) * | 2007-12-10 | 2010-12-16 | Abb Research Ltd. | Computer implemented method and system for remote inspection of an industrial process |
WO2011048047A1 (en) * | 2009-10-19 | 2011-04-28 | Siemens Aktiengesellschaft | Hollow needle positioning system |
US20120062739A1 (en) * | 2009-05-18 | 2012-03-15 | Peugeot Citroen Automobiles Sa | Method And Device For Extending A Visibility Area |
US20120240077A1 (en) * | 2011-03-16 | 2012-09-20 | Nokia Corporation | Method and apparatus for displaying interactive preview information in a location-based user interface |
US20120303336A1 (en) * | 2009-12-18 | 2012-11-29 | Airbus Operations Gmbh | Assembly and method for verifying a real model using a virtual model and use in aircraft construction |
US8623030B2 (en) | 2001-08-28 | 2014-01-07 | Bonutti Skeletal Innovations Llc | Robotic arthroplasty system including navigation |
US8696549B2 (en) | 2010-08-20 | 2014-04-15 | Veran Medical Technologies, Inc. | Apparatus and method for four dimensional soft tissue navigation in endoscopic applications |
US8781186B2 (en) | 2010-05-04 | 2014-07-15 | Pathfinder Therapeutics, Inc. | System and method for abdominal surface matching using pseudo-features |
US9138165B2 (en) | 2012-02-22 | 2015-09-22 | Veran Medical Technologies, Inc. | Systems, methods and devices for forming respiratory-gated point cloud for four dimensional soft tissue navigation |
US10217209B2 (en) * | 2015-12-11 | 2019-02-26 | Daqri, Llc | System and method for tool mapping |
CN110638529A (en) * | 2019-09-20 | 2020-01-03 | 和宇健康科技股份有限公司 | Operation remote control method and device, storage medium and terminal equipment |
US10617324B2 (en) | 2014-04-23 | 2020-04-14 | Veran Medical Technologies, Inc | Apparatuses and methods for endobronchial navigation to and confirmation of the location of a target tissue and percutaneous interception of the target tissue |
US10624701B2 (en) | 2014-04-23 | 2020-04-21 | Veran Medical Technologies, Inc. | Apparatuses and methods for registering a real-time image feed from an imaging device to a steerable catheter |
US20210181843A1 (en) * | 2019-12-13 | 2021-06-17 | Fuji Xerox Co., Ltd. | Information processing device and non-transitory computer readable medium |
US11304629B2 (en) | 2005-09-13 | 2022-04-19 | Veran Medical Technologies, Inc. | Apparatus and method for image guided accuracy verification |
Families Citing this family (153)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6690960B2 (en) * | 2000-12-21 | 2004-02-10 | David T. Chen | Video-based surgical targeting system |
US10835307B2 (en) | 2001-06-12 | 2020-11-17 | Ethicon Llc | Modular battery powered handheld surgical instrument containing elongated multi-layered shaft |
US20050277823A1 (en) * | 2002-06-10 | 2005-12-15 | Robert Sutherland | Angiogram display overlay technique for tracking vascular intervention sites |
US8403828B2 (en) * | 2003-07-21 | 2013-03-26 | Vanderbilt University | Ophthalmic orbital surgery apparatus and method and image-guide navigation system |
US20050057684A1 (en) * | 2003-09-17 | 2005-03-17 | Konica Minolta Medical & Graphic, Inc. | Digital camera for medical service, photography operating device and photographic image diagnosis system for medical service |
US8182501B2 (en) | 2004-02-27 | 2012-05-22 | Ethicon Endo-Surgery, Inc. | Ultrasonic surgical shears and method for sealing a blood vessel using same |
AU2005295010B2 (en) | 2004-10-08 | 2012-05-31 | Ethicon Endo-Surgery, Inc. | Ultrasonic surgical instrument |
JP2006218129A (en) * | 2005-02-10 | 2006-08-24 | Olympus Corp | Surgery supporting system |
US8073528B2 (en) | 2007-09-30 | 2011-12-06 | Intuitive Surgical Operations, Inc. | Tool tracking systems, methods and computer products for image guided surgery |
US10555775B2 (en) | 2005-05-16 | 2020-02-11 | Intuitive Surgical Operations, Inc. | Methods and system for performing 3-D tool tracking by fusion of sensor and/or camera derived data during minimally invasive robotic surgery |
US7756563B2 (en) * | 2005-05-23 | 2010-07-13 | The Penn State Research Foundation | Guidance method based on 3D-2D pose estimation and 3D-CT registration with application to live bronchoscopy |
US7877128B2 (en) * | 2005-08-02 | 2011-01-25 | Biosense Webster, Inc. | Simulation of invasive procedures |
US8583220B2 (en) * | 2005-08-02 | 2013-11-12 | Biosense Webster, Inc. | Standardization of catheter-based treatment for atrial fibrillation |
US20070191713A1 (en) | 2005-10-14 | 2007-08-16 | Eichmann Stephen E | Ultrasonic device for cutting and coagulating |
US7621930B2 (en) | 2006-01-20 | 2009-11-24 | Ethicon Endo-Surgery, Inc. | Ultrasound medical instrument having a medical ultrasonic blade |
US8526688B2 (en) * | 2006-03-09 | 2013-09-03 | General Electric Company | Methods and systems for registration of surgical navigation data and image data |
WO2007115825A1 (en) * | 2006-04-12 | 2007-10-18 | Nassir Navab | Registration-free augmentation device and method |
US7945310B2 (en) * | 2006-09-18 | 2011-05-17 | Stryker Corporation | Surgical instrument path computation and display for endoluminal surgery |
US7824328B2 (en) * | 2006-09-18 | 2010-11-02 | Stryker Corporation | Method and apparatus for tracking a surgical instrument during surgery |
US20080071141A1 (en) * | 2006-09-18 | 2008-03-20 | Abhisuek Gattani | Method and apparatus for measuring attributes of an anatomical feature during a medical procedure |
US8248413B2 (en) * | 2006-09-18 | 2012-08-21 | Stryker Corporation | Visual navigation system for endoscopic surgery |
US8248414B2 (en) * | 2006-09-18 | 2012-08-21 | Stryker Corporation | Multi-dimensional navigation of endoscopic video |
EP1925256A1 (en) * | 2006-11-24 | 2008-05-28 | BrainLAB AG | Method and device for registering an anatomical structure with markers |
US9477686B2 (en) * | 2007-01-12 | 2016-10-25 | General Electric Company | Systems and methods for annotation and sorting of surgical images |
US8057498B2 (en) | 2007-11-30 | 2011-11-15 | Ethicon Endo-Surgery, Inc. | Ultrasonic surgical instrument blades |
US8911460B2 (en) | 2007-03-22 | 2014-12-16 | Ethicon Endo-Surgery, Inc. | Ultrasonic surgical instruments |
US8142461B2 (en) | 2007-03-22 | 2012-03-27 | Ethicon Endo-Surgery, Inc. | Surgical instruments |
US20080234576A1 (en) * | 2007-03-23 | 2008-09-25 | General Electric Company | System and method to track movement of a tool in percutaneous replacement of a heart valve |
US8523889B2 (en) | 2007-07-27 | 2013-09-03 | Ethicon Endo-Surgery, Inc. | Ultrasonic end effectors with increased active length |
US8808319B2 (en) | 2007-07-27 | 2014-08-19 | Ethicon Endo-Surgery, Inc. | Surgical instruments |
US8512365B2 (en) | 2007-07-31 | 2013-08-20 | Ethicon Endo-Surgery, Inc. | Surgical instruments |
US9044261B2 (en) | 2007-07-31 | 2015-06-02 | Ethicon Endo-Surgery, Inc. | Temperature controlled ultrasonic surgical instruments |
US8430898B2 (en) | 2007-07-31 | 2013-04-30 | Ethicon Endo-Surgery, Inc. | Ultrasonic surgical instruments |
US8623027B2 (en) | 2007-10-05 | 2014-01-07 | Ethicon Endo-Surgery, Inc. | Ergonomic surgical instruments |
US10010339B2 (en) | 2007-11-30 | 2018-07-03 | Ethicon Llc | Ultrasonic surgical blades |
US8219179B2 (en) | 2008-03-06 | 2012-07-10 | Vida Diagnostics, Inc. | Systems and methods for navigation within a branched structure of a body |
US9089360B2 (en) | 2008-08-06 | 2015-07-28 | Ethicon Endo-Surgery, Inc. | Devices and techniques for cutting and coagulating tissue |
US20100285438A1 (en) * | 2009-03-12 | 2010-11-11 | Thenkurussi Kesavadas | Method And System For Minimally-Invasive Surgery Training |
US9700339B2 (en) | 2009-05-20 | 2017-07-11 | Ethicon Endo-Surgery, Inc. | Coupling arrangements and methods for attaching tools to ultrasonic surgical instruments |
US8663220B2 (en) | 2009-07-15 | 2014-03-04 | Ethicon Endo-Surgery, Inc. | Ultrasonic surgical instruments |
US8951248B2 (en) | 2009-10-09 | 2015-02-10 | Ethicon Endo-Surgery, Inc. | Surgical generator for ultrasonic and electrosurgical devices |
US10441345B2 (en) | 2009-10-09 | 2019-10-15 | Ethicon Llc | Surgical generator for ultrasonic and electrosurgical devices |
US11090104B2 (en) | 2009-10-09 | 2021-08-17 | Cilag Gmbh International | Surgical generator for ultrasonic and electrosurgical devices |
US8951272B2 (en) | 2010-02-11 | 2015-02-10 | Ethicon Endo-Surgery, Inc. | Seal arrangements for ultrasonically powered surgical instruments |
US8486096B2 (en) | 2010-02-11 | 2013-07-16 | Ethicon Endo-Surgery, Inc. | Dual purpose surgical instrument for cutting and coagulating tissue |
US8469981B2 (en) | 2010-02-11 | 2013-06-25 | Ethicon Endo-Surgery, Inc. | Rotatable cutting implement arrangements for ultrasonic surgical instruments |
US8795327B2 (en) | 2010-07-22 | 2014-08-05 | Ethicon Endo-Surgery, Inc. | Electrosurgical instrument with separate closure and cutting members |
US9192431B2 (en) | 2010-07-23 | 2015-11-24 | Ethicon Endo-Surgery, Inc. | Electrosurgical cutting and sealing instrument |
DE102010042372A1 (en) * | 2010-10-13 | 2012-04-19 | Kuka Laboratories Gmbh | Method for creating a medical image and medical workstation |
US9259265B2 (en) | 2011-07-22 | 2016-02-16 | Ethicon Endo-Surgery, Llc | Surgical instruments for tensioning tissue |
US9554866B2 (en) | 2011-08-09 | 2017-01-31 | Covidien Lp | Apparatus and method for using a remote control system in surgical procedures |
DE102012200921B4 (en) * | 2012-01-23 | 2014-08-21 | Siemens Aktiengesellschaft | A method for determining a deviation of a medical instrument from a target position |
EP2811932B1 (en) | 2012-02-10 | 2019-06-26 | Ethicon LLC | Robotically controlled surgical instrument |
US9439668B2 (en) | 2012-04-09 | 2016-09-13 | Ethicon Endo-Surgery, Llc | Switch arrangements for ultrasonic surgical instruments |
US10758315B2 (en) * | 2012-06-21 | 2020-09-01 | Globus Medical Inc. | Method and system for improving 2D-3D registration convergence |
US20140005705A1 (en) | 2012-06-29 | 2014-01-02 | Ethicon Endo-Surgery, Inc. | Surgical instruments with articulating shafts |
US9820768B2 (en) | 2012-06-29 | 2017-11-21 | Ethicon Llc | Ultrasonic surgical instruments with control mechanisms |
US9408622B2 (en) | 2012-06-29 | 2016-08-09 | Ethicon Endo-Surgery, Llc | Surgical instruments with articulating shafts |
US9351754B2 (en) | 2012-06-29 | 2016-05-31 | Ethicon Endo-Surgery, Llc | Ultrasonic surgical instruments with distally positioned jaw assemblies |
US9226767B2 (en) | 2012-06-29 | 2016-01-05 | Ethicon Endo-Surgery, Inc. | Closed feedback control for electrosurgical device |
US20140005702A1 (en) | 2012-06-29 | 2014-01-02 | Ethicon Endo-Surgery, Inc. | Ultrasonic surgical instruments with distally positioned transducers |
US9198714B2 (en) | 2012-06-29 | 2015-12-01 | Ethicon Endo-Surgery, Inc. | Haptic feedback devices for surgical robot |
US9393037B2 (en) | 2012-06-29 | 2016-07-19 | Ethicon Endo-Surgery, Llc | Surgical instruments with articulating shafts |
US9326788B2 (en) | 2012-06-29 | 2016-05-03 | Ethicon Endo-Surgery, Llc | Lockout mechanism for use with robotic electrosurgical device |
US9492224B2 (en) | 2012-09-28 | 2016-11-15 | EthiconEndo-Surgery, LLC | Multi-function bi-polar forceps |
US9095367B2 (en) | 2012-10-22 | 2015-08-04 | Ethicon Endo-Surgery, Inc. | Flexible harmonic waveguides/blades for surgical instruments |
US10201365B2 (en) * | 2012-10-22 | 2019-02-12 | Ethicon Llc | Surgeon feedback sensing and display methods |
US20140135804A1 (en) | 2012-11-15 | 2014-05-15 | Ethicon Endo-Surgery, Inc. | Ultrasonic and electrosurgical devices |
US10226273B2 (en) | 2013-03-14 | 2019-03-12 | Ethicon Llc | Mechanical fasteners for use with surgical energy devices |
JP6138566B2 (en) * | 2013-04-24 | 2017-05-31 | 川崎重工業株式会社 | Component mounting work support system and component mounting method |
US9814514B2 (en) | 2013-09-13 | 2017-11-14 | Ethicon Llc | Electrosurgical (RF) medical instruments for cutting and coagulating tissue |
DE102013222230A1 (en) | 2013-10-31 | 2015-04-30 | Fiagon Gmbh | Surgical instrument |
US9265926B2 (en) | 2013-11-08 | 2016-02-23 | Ethicon Endo-Surgery, Llc | Electrosurgical devices |
GB2521228A (en) | 2013-12-16 | 2015-06-17 | Ethicon Endo Surgery Inc | Medical device |
GB2521229A (en) | 2013-12-16 | 2015-06-17 | Ethicon Endo Surgery Inc | Medical device |
US9795436B2 (en) | 2014-01-07 | 2017-10-24 | Ethicon Llc | Harvesting energy from a surgical generator |
JP6740131B2 (en) | 2014-02-21 | 2020-08-12 | スリーディインテグレイテッド アーペーエス3Dintegrated Aps | Set with surgical instrument, surgical system, and training method |
WO2015132778A1 (en) * | 2014-03-02 | 2015-09-11 | Avishay Sidlesky | Endoscopic measurement system and method |
US9554854B2 (en) | 2014-03-18 | 2017-01-31 | Ethicon Endo-Surgery, Llc | Detecting short circuits in electrosurgical medical devices |
US10463421B2 (en) | 2014-03-27 | 2019-11-05 | Ethicon Llc | Two stage trigger, clamp and cut bipolar vessel sealer |
US10092310B2 (en) | 2014-03-27 | 2018-10-09 | Ethicon Llc | Electrosurgical devices |
US9737355B2 (en) | 2014-03-31 | 2017-08-22 | Ethicon Llc | Controlling impedance rise in electrosurgical medical devices |
DE102014207274A1 (en) * | 2014-04-15 | 2015-10-15 | Fiagon Gmbh | Navigation support system for medical instruments |
US9913680B2 (en) | 2014-04-15 | 2018-03-13 | Ethicon Llc | Software algorithms for electrosurgical instruments |
US10285724B2 (en) | 2014-07-31 | 2019-05-14 | Ethicon Llc | Actuation mechanisms and load adjustment assemblies for surgical instruments |
US9950194B2 (en) | 2014-09-09 | 2018-04-24 | Mevion Medical Systems, Inc. | Patient positioning system |
US10314523B2 (en) | 2014-11-14 | 2019-06-11 | Synaptive Medical (Barbados) Inc. | Method, system and apparatus for image capture and registration in image-guided surgery |
US10639092B2 (en) | 2014-12-08 | 2020-05-05 | Ethicon Llc | Electrode configurations for surgical instruments |
US10245095B2 (en) | 2015-02-06 | 2019-04-02 | Ethicon Llc | Electrosurgical instrument with rotation and articulation mechanisms |
US10342602B2 (en) | 2015-03-17 | 2019-07-09 | Ethicon Llc | Managing tissue treatment |
US10321950B2 (en) | 2015-03-17 | 2019-06-18 | Ethicon Llc | Managing tissue treatment |
US10595929B2 (en) | 2015-03-24 | 2020-03-24 | Ethicon Llc | Surgical instruments with firing system overload protection mechanisms |
US11020140B2 (en) | 2015-06-17 | 2021-06-01 | Cilag Gmbh International | Ultrasonic surgical blade for use with ultrasonic surgical instruments |
US11141213B2 (en) | 2015-06-30 | 2021-10-12 | Cilag Gmbh International | Surgical instrument with user adaptable techniques |
US10898256B2 (en) | 2015-06-30 | 2021-01-26 | Ethicon Llc | Surgical system with user adaptable techniques based on tissue impedance |
US11129669B2 (en) | 2015-06-30 | 2021-09-28 | Cilag Gmbh International | Surgical system with user adaptable techniques based on tissue type |
US10034704B2 (en) | 2015-06-30 | 2018-07-31 | Ethicon Llc | Surgical instrument with user adaptable algorithms |
US11051873B2 (en) | 2015-06-30 | 2021-07-06 | Cilag Gmbh International | Surgical system with user adaptable techniques employing multiple energy modalities based on tissue parameters |
US10357303B2 (en) | 2015-06-30 | 2019-07-23 | Ethicon Llc | Translatable outer tube for sealing using shielded lap chole dissector |
US10154852B2 (en) | 2015-07-01 | 2018-12-18 | Ethicon Llc | Ultrasonic surgical blade with improved cutting and coagulation features |
CN108024806B (en) | 2015-07-21 | 2022-07-01 | 3D集成公司 | Cannula assembly kit, trocar assembly kit, sleeve assembly, minimally invasive surgical system and method thereof |
US11020144B2 (en) | 2015-07-21 | 2021-06-01 | 3Dintegrated Aps | Minimally invasive surgery system |
US10687884B2 (en) | 2015-09-30 | 2020-06-23 | Ethicon Llc | Circuits for supplying isolated direct current (DC) voltage to surgical instruments |
DK178899B1 (en) * | 2015-10-09 | 2017-05-08 | 3Dintegrated Aps | A depiction system |
US10595930B2 (en) | 2015-10-16 | 2020-03-24 | Ethicon Llc | Electrode wiping surgical device |
US10179022B2 (en) | 2015-12-30 | 2019-01-15 | Ethicon Llc | Jaw position impedance limiter for electrosurgical instrument |
US10575892B2 (en) | 2015-12-31 | 2020-03-03 | Ethicon Llc | Adapter for electrical surgical instruments |
US10716615B2 (en) | 2016-01-15 | 2020-07-21 | Ethicon Llc | Modular battery powered handheld surgical instrument with curved end effectors having asymmetric engagement between jaw and blade |
US11129670B2 (en) | 2016-01-15 | 2021-09-28 | Cilag Gmbh International | Modular battery powered handheld surgical instrument with selective application of energy based on button displacement, intensity, or local tissue characterization |
US11229471B2 (en) | 2016-01-15 | 2022-01-25 | Cilag Gmbh International | Modular battery powered handheld surgical instrument with selective application of energy based on tissue characterization |
US10709469B2 (en) | 2016-01-15 | 2020-07-14 | Ethicon Llc | Modular battery powered handheld surgical instrument with energy conservation techniques |
US10555769B2 (en) | 2016-02-22 | 2020-02-11 | Ethicon Llc | Flexible circuits for electrosurgical instrument |
US10702329B2 (en) | 2016-04-29 | 2020-07-07 | Ethicon Llc | Jaw structure with distal post for electrosurgical instruments |
US10485607B2 (en) | 2016-04-29 | 2019-11-26 | Ethicon Llc | Jaw structure with distal closure for electrosurgical instruments |
US10646269B2 (en) | 2016-04-29 | 2020-05-12 | Ethicon Llc | Non-linear jaw gap for electrosurgical instruments |
US10456193B2 (en) | 2016-05-03 | 2019-10-29 | Ethicon Llc | Medical device with a bilateral jaw configuration for nerve stimulation |
US10245064B2 (en) | 2016-07-12 | 2019-04-02 | Ethicon Llc | Ultrasonic surgical instrument with piezoelectric central lumen transducer |
US10893883B2 (en) | 2016-07-13 | 2021-01-19 | Ethicon Llc | Ultrasonic assembly for use with ultrasonic surgical instruments |
US10842522B2 (en) | 2016-07-15 | 2020-11-24 | Ethicon Llc | Ultrasonic surgical instruments having offset blades |
US10376305B2 (en) | 2016-08-05 | 2019-08-13 | Ethicon Llc | Methods and systems for advanced harmonic energy |
US10285723B2 (en) | 2016-08-09 | 2019-05-14 | Ethicon Llc | Ultrasonic surgical blade with improved heel portion |
USD847990S1 (en) | 2016-08-16 | 2019-05-07 | Ethicon Llc | Surgical instrument |
US10779847B2 (en) | 2016-08-25 | 2020-09-22 | Ethicon Llc | Ultrasonic transducer to waveguide joining |
US10952759B2 (en) | 2016-08-25 | 2021-03-23 | Ethicon Llc | Tissue loading of a surgical instrument |
US10603064B2 (en) | 2016-11-28 | 2020-03-31 | Ethicon Llc | Ultrasonic transducer |
US11266430B2 (en) | 2016-11-29 | 2022-03-08 | Cilag Gmbh International | End effector control and calibration |
US11026747B2 (en) * | 2017-04-25 | 2021-06-08 | Biosense Webster (Israel) Ltd. | Endoscopic view of invasive procedures in narrow passages |
EP3621542B1 (en) * | 2017-05-09 | 2023-03-15 | Brainlab AG | Generation of augmented reality image of a medical device |
US10820920B2 (en) | 2017-07-05 | 2020-11-03 | Ethicon Llc | Reusable ultrasonic medical devices and methods of their use |
US11659023B2 (en) * | 2017-12-28 | 2023-05-23 | Cilag Gmbh International | Method of hub communication |
US11189029B2 (en) * | 2018-07-02 | 2021-11-30 | Tempus Labs, Inc. | 3D radiomic platform for imaging biomarker development |
US20200015702A1 (en) * | 2018-07-13 | 2020-01-16 | Children's Hospital Medical Center | System and method for landmarking a patient |
EP3640767A1 (en) * | 2018-10-17 | 2020-04-22 | Siemens Schweiz AG | Method for determining at least one area in at least one input model for at least one element to be placed |
WO2020097425A2 (en) | 2018-11-09 | 2020-05-14 | Vida Diagnostics, Inc. | Cut-surface display of tubular structures |
US20220139532A1 (en) * | 2019-02-25 | 2022-05-05 | The Johns Hopkins University | Interactive flying frustums visualization in augmented reality |
EP3719749A1 (en) | 2019-04-03 | 2020-10-07 | Fiagon AG Medical Technologies | Registration method and setup |
US11937866B2 (en) | 2019-12-30 | 2024-03-26 | Cilag Gmbh International | Method for an electrosurgical procedure |
US11779387B2 (en) | 2019-12-30 | 2023-10-10 | Cilag Gmbh International | Clamp arm jaw to minimize tissue sticking and improve tissue control |
US11779329B2 (en) | 2019-12-30 | 2023-10-10 | Cilag Gmbh International | Surgical instrument comprising a flex circuit including a sensor system |
US11911063B2 (en) | 2019-12-30 | 2024-02-27 | Cilag Gmbh International | Techniques for detecting ultrasonic blade to electrode contact and reducing power to ultrasonic blade |
US11660089B2 (en) | 2019-12-30 | 2023-05-30 | Cilag Gmbh International | Surgical instrument comprising a sensing system |
US11589916B2 (en) | 2019-12-30 | 2023-02-28 | Cilag Gmbh International | Electrosurgical instruments with electrodes having variable energy densities |
US11786291B2 (en) | 2019-12-30 | 2023-10-17 | Cilag Gmbh International | Deflectable support of RF energy electrode with respect to opposing ultrasonic blade |
US11937863B2 (en) | 2019-12-30 | 2024-03-26 | Cilag Gmbh International | Deflectable electrode with variable compression bias along the length of the deflectable electrode |
US11684412B2 (en) | 2019-12-30 | 2023-06-27 | Cilag Gmbh International | Surgical instrument with rotatable and articulatable surgical end effector |
US11452525B2 (en) | 2019-12-30 | 2022-09-27 | Cilag Gmbh International | Surgical instrument comprising an adjustment system |
US11812957B2 (en) | 2019-12-30 | 2023-11-14 | Cilag Gmbh International | Surgical instrument comprising a signal interference resolution system |
US11696776B2 (en) | 2019-12-30 | 2023-07-11 | Cilag Gmbh International | Articulatable surgical instrument |
US11744636B2 (en) | 2019-12-30 | 2023-09-05 | Cilag Gmbh International | Electrosurgical systems with integrated and external power sources |
US20210196344A1 (en) | 2019-12-30 | 2021-07-01 | Ethicon Llc | Surgical system communication pathways |
US11944366B2 (en) | 2019-12-30 | 2024-04-02 | Cilag Gmbh International | Asymmetric segmented ultrasonic support pad for cooperative engagement with a movable RF electrode |
US11875459B2 (en) | 2020-04-07 | 2024-01-16 | Vida Diagnostics, Inc. | Subject specific coordinatization and virtual navigation systems and methods |
US11102381B1 (en) | 2021-01-05 | 2021-08-24 | Board Of Regents, The University Of Texas System Clearcam Inc. | Methods, systems and controllers for facilitating cleaning of an imaging element of an imaging device |
Citations (40)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4722056A (en) * | 1986-02-18 | 1988-01-26 | Trustees Of Dartmouth College | Reference display systems for superimposing a tomagraphic image onto the focal plane of an operating microscope |
US4729098A (en) * | 1985-06-05 | 1988-03-01 | General Electric Company | System and method employing nonlinear interpolation for the display of surface structures contained within the interior region of a solid body |
US4882679A (en) * | 1987-11-27 | 1989-11-21 | Picker International, Inc. | System to reformat images for three-dimensional display |
US4922909A (en) * | 1987-07-17 | 1990-05-08 | Little James H | Video monitoring and reapposition monitoring apparatus and methods |
US4945478A (en) * | 1987-11-06 | 1990-07-31 | Center For Innovative Technology | Noninvasive medical imaging system and method for the identification and 3-D display of atherosclerosis and the like |
US4965844A (en) * | 1985-04-03 | 1990-10-23 | Sony Corporation | Method and system for image transformation |
US4985855A (en) * | 1987-08-24 | 1991-01-15 | International Business Machines Corp. | Method for producing installation instructions for three dimensional assemblies |
US4989083A (en) * | 1989-06-29 | 1991-01-29 | Olympus Optical Co., Ltd. | Method of inspecting objects by image pickup means |
US5005559A (en) * | 1989-07-27 | 1991-04-09 | Massachusetts Institute Of Technology | Video-graphic arthroscopy system |
US5151856A (en) * | 1989-08-30 | 1992-09-29 | Technion R & D Found. Ltd. | Method of displaying coronary function |
US5153721A (en) * | 1990-06-04 | 1992-10-06 | Olympus Optical Co., Ltd. | Method and apparatus for measuring an object by correlating displaced and simulated object images |
US5179638A (en) * | 1990-04-26 | 1993-01-12 | Honeywell Inc. | Method and apparatus for generating a texture mapped perspective view |
US5230623A (en) * | 1991-12-10 | 1993-07-27 | Radionics, Inc. | Operating pointer with interactive computergraphics |
US5231483A (en) * | 1990-09-05 | 1993-07-27 | Visionary Products, Inc. | Smart tracking system |
US5255352A (en) * | 1989-08-03 | 1993-10-19 | Computer Design, Inc. | Mapping of two-dimensional surface detail on three-dimensional surfaces |
US5261404A (en) * | 1991-07-08 | 1993-11-16 | Mick Peter R | Three-dimensional mammal anatomy imaging system and method |
US5274551A (en) * | 1991-11-29 | 1993-12-28 | General Electric Company | Method and apparatus for real-time navigation assist in interventional radiological procedures |
US5291889A (en) * | 1991-05-23 | 1994-03-08 | Vanguard Imaging Ltd. | Apparatus and method for spatially positioning images |
US5295199A (en) * | 1991-11-15 | 1994-03-15 | Sony Corporation | Image transforming apparatus and method |
US5297215A (en) * | 1990-06-13 | 1994-03-22 | Kabushiki Kaisha Toshiba | Apparatus and method of displaying medical images of sliced tissue |
US5319551A (en) * | 1989-10-27 | 1994-06-07 | Hitachi, Ltd. | Region extracting method and three-dimensional display method |
US5329310A (en) * | 1992-06-30 | 1994-07-12 | The Walt Disney Company | Method and apparatus for controlling distortion of a projected image |
US5363476A (en) * | 1992-01-28 | 1994-11-08 | Sony Corporation | Image converter for mapping a two-dimensional image onto a three dimensional curved surface created from two-dimensional image data |
US5378915A (en) * | 1992-11-25 | 1995-01-03 | Adac Laboratories | Apparatus and method for automatic tracking of a zoomed scan area in a medical camera system |
US5383454A (en) * | 1990-10-19 | 1995-01-24 | St. Louis University | System for indicating the position of a surgical probe within a head on an image of the head |
US5417210A (en) * | 1992-05-27 | 1995-05-23 | International Business Machines Corporation | System and method for augmentation of endoscopic surgery |
US5447154A (en) * | 1992-07-31 | 1995-09-05 | Universite Joseph Fourier | Method for determining the position of an organ |
US5448687A (en) * | 1988-09-13 | 1995-09-05 | Computer Design, Inc. | Computer-assisted design system for flattening a three-dimensional surface and for wrapping a flat shape to a three-dimensional surface |
US5461706A (en) * | 1992-04-24 | 1995-10-24 | Sony United Kingdom Ltd. | Lighting effects for digital video effects system |
US5491510A (en) * | 1993-12-03 | 1996-02-13 | Texas Instruments Incorporated | System and method for simultaneously viewing a scene and an obscured object |
US5493595A (en) * | 1982-02-24 | 1996-02-20 | Schoolman Scientific Corp. | Stereoscopically displayed three dimensional medical imaging |
US5497452A (en) * | 1991-03-14 | 1996-03-05 | International Business Machines Corporation | Method and apparatus for generating a geometric model |
US5511153A (en) * | 1994-01-18 | 1996-04-23 | Massachusetts Institute Of Technology | Method and apparatus for three-dimensional, textured models from plural video images |
US5526814A (en) * | 1993-11-09 | 1996-06-18 | General Electric Company | Automatically positioned focussed energy system guided by medical imaging |
US5526812A (en) * | 1993-06-21 | 1996-06-18 | General Electric Company | Display system for enhancing visualization of body structures during medical procedures |
US5531227A (en) * | 1994-01-28 | 1996-07-02 | Schneider Medical Technologies, Inc. | Imaging device and method |
US5537638A (en) * | 1991-10-25 | 1996-07-16 | Hitachi, Ltd. | Method and system for image mapping |
US5558619A (en) * | 1991-04-23 | 1996-09-24 | Olympus Optical Co., Ltd. | Endoscope system with automatic control according to movement of an operator |
US6675032B2 (en) * | 1994-10-07 | 2004-01-06 | Medical Media Systems | Video-based surgical targeting system |
US6690960B2 (en) * | 2000-12-21 | 2004-02-10 | David T. Chen | Video-based surgical targeting system |
Family Cites Families (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5099846A (en) | 1988-12-23 | 1992-03-31 | Hardy Tyrone L | Method and apparatus for video presentation from a variety of scanner imaging sources |
-
2002
- 2002-03-21 US US10/104,256 patent/US6690960B2/en not_active Expired - Lifetime
-
2004
- 2004-02-10 US US10/775,754 patent/US20050027186A1/en not_active Abandoned
Patent Citations (42)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5493595A (en) * | 1982-02-24 | 1996-02-20 | Schoolman Scientific Corp. | Stereoscopically displayed three dimensional medical imaging |
US4965844A (en) * | 1985-04-03 | 1990-10-23 | Sony Corporation | Method and system for image transformation |
US4729098A (en) * | 1985-06-05 | 1988-03-01 | General Electric Company | System and method employing nonlinear interpolation for the display of surface structures contained within the interior region of a solid body |
US4722056A (en) * | 1986-02-18 | 1988-01-26 | Trustees Of Dartmouth College | Reference display systems for superimposing a tomagraphic image onto the focal plane of an operating microscope |
US4922909A (en) * | 1987-07-17 | 1990-05-08 | Little James H | Video monitoring and reapposition monitoring apparatus and methods |
US4985855A (en) * | 1987-08-24 | 1991-01-15 | International Business Machines Corp. | Method for producing installation instructions for three dimensional assemblies |
US4945478A (en) * | 1987-11-06 | 1990-07-31 | Center For Innovative Technology | Noninvasive medical imaging system and method for the identification and 3-D display of atherosclerosis and the like |
US4882679A (en) * | 1987-11-27 | 1989-11-21 | Picker International, Inc. | System to reformat images for three-dimensional display |
US5448687A (en) * | 1988-09-13 | 1995-09-05 | Computer Design, Inc. | Computer-assisted design system for flattening a three-dimensional surface and for wrapping a flat shape to a three-dimensional surface |
US4989083A (en) * | 1989-06-29 | 1991-01-29 | Olympus Optical Co., Ltd. | Method of inspecting objects by image pickup means |
US5005559A (en) * | 1989-07-27 | 1991-04-09 | Massachusetts Institute Of Technology | Video-graphic arthroscopy system |
US5255352A (en) * | 1989-08-03 | 1993-10-19 | Computer Design, Inc. | Mapping of two-dimensional surface detail on three-dimensional surfaces |
US5151856A (en) * | 1989-08-30 | 1992-09-29 | Technion R & D Found. Ltd. | Method of displaying coronary function |
US5319551A (en) * | 1989-10-27 | 1994-06-07 | Hitachi, Ltd. | Region extracting method and three-dimensional display method |
US5179638A (en) * | 1990-04-26 | 1993-01-12 | Honeywell Inc. | Method and apparatus for generating a texture mapped perspective view |
US5153721A (en) * | 1990-06-04 | 1992-10-06 | Olympus Optical Co., Ltd. | Method and apparatus for measuring an object by correlating displaced and simulated object images |
US5297215A (en) * | 1990-06-13 | 1994-03-22 | Kabushiki Kaisha Toshiba | Apparatus and method of displaying medical images of sliced tissue |
US5384594A (en) * | 1990-09-05 | 1995-01-24 | Sieber; Jonathan D. | Smart tracking system |
US5231483A (en) * | 1990-09-05 | 1993-07-27 | Visionary Products, Inc. | Smart tracking system |
US5383454B1 (en) * | 1990-10-19 | 1996-12-31 | Univ St Louis | System for indicating the position of a surgical probe within a head on an image of the head |
US5383454A (en) * | 1990-10-19 | 1995-01-24 | St. Louis University | System for indicating the position of a surgical probe within a head on an image of the head |
US5497452A (en) * | 1991-03-14 | 1996-03-05 | International Business Machines Corporation | Method and apparatus for generating a geometric model |
US5558619A (en) * | 1991-04-23 | 1996-09-24 | Olympus Optical Co., Ltd. | Endoscope system with automatic control according to movement of an operator |
US5291889A (en) * | 1991-05-23 | 1994-03-08 | Vanguard Imaging Ltd. | Apparatus and method for spatially positioning images |
US5261404A (en) * | 1991-07-08 | 1993-11-16 | Mick Peter R | Three-dimensional mammal anatomy imaging system and method |
US5537638A (en) * | 1991-10-25 | 1996-07-16 | Hitachi, Ltd. | Method and system for image mapping |
US5295199A (en) * | 1991-11-15 | 1994-03-15 | Sony Corporation | Image transforming apparatus and method |
US5274551A (en) * | 1991-11-29 | 1993-12-28 | General Electric Company | Method and apparatus for real-time navigation assist in interventional radiological procedures |
US5230623A (en) * | 1991-12-10 | 1993-07-27 | Radionics, Inc. | Operating pointer with interactive computergraphics |
US5363476A (en) * | 1992-01-28 | 1994-11-08 | Sony Corporation | Image converter for mapping a two-dimensional image onto a three dimensional curved surface created from two-dimensional image data |
US5461706A (en) * | 1992-04-24 | 1995-10-24 | Sony United Kingdom Ltd. | Lighting effects for digital video effects system |
US5417210A (en) * | 1992-05-27 | 1995-05-23 | International Business Machines Corporation | System and method for augmentation of endoscopic surgery |
US5329310A (en) * | 1992-06-30 | 1994-07-12 | The Walt Disney Company | Method and apparatus for controlling distortion of a projected image |
US5447154A (en) * | 1992-07-31 | 1995-09-05 | Universite Joseph Fourier | Method for determining the position of an organ |
US5378915A (en) * | 1992-11-25 | 1995-01-03 | Adac Laboratories | Apparatus and method for automatic tracking of a zoomed scan area in a medical camera system |
US5526812A (en) * | 1993-06-21 | 1996-06-18 | General Electric Company | Display system for enhancing visualization of body structures during medical procedures |
US5526814A (en) * | 1993-11-09 | 1996-06-18 | General Electric Company | Automatically positioned focussed energy system guided by medical imaging |
US5491510A (en) * | 1993-12-03 | 1996-02-13 | Texas Instruments Incorporated | System and method for simultaneously viewing a scene and an obscured object |
US5511153A (en) * | 1994-01-18 | 1996-04-23 | Massachusetts Institute Of Technology | Method and apparatus for three-dimensional, textured models from plural video images |
US5531227A (en) * | 1994-01-28 | 1996-07-02 | Schneider Medical Technologies, Inc. | Imaging device and method |
US6675032B2 (en) * | 1994-10-07 | 2004-01-06 | Medical Media Systems | Video-based surgical targeting system |
US6690960B2 (en) * | 2000-12-21 | 2004-02-10 | David T. Chen | Video-based surgical targeting system |
Cited By (69)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9795394B2 (en) | 2000-01-14 | 2017-10-24 | Bonutti Skeletal Innovations Llc | Method for placing implant using robotic system |
US8425522B2 (en) | 2000-01-14 | 2013-04-23 | Bonutti Skeletal Innovations Llc | Joint replacement method |
US20100228257A1 (en) * | 2000-01-14 | 2010-09-09 | Bonutti Peter M | Joint replacement component |
US8632552B2 (en) | 2000-01-14 | 2014-01-21 | Bonutti Skeletal Innovations Llc | Method of preparing a femur and tibia in knee arthroplasty |
US9101443B2 (en) | 2000-01-14 | 2015-08-11 | Bonutti Skeletal Innovations Llc | Methods for robotic arthroplasty |
US8784495B2 (en) | 2000-01-14 | 2014-07-22 | Bonutti Skeletal Innovations Llc | Segmental knee arthroplasty |
US9192459B2 (en) | 2000-01-14 | 2015-11-24 | Bonutti Skeletal Innovations Llc | Method of performing total knee arthroplasty |
US10321918B2 (en) | 2001-08-28 | 2019-06-18 | Bonutti Skeletal Innovations Llc | Methods for robotic surgery using a cannula |
US9060797B2 (en) | 2001-08-28 | 2015-06-23 | Bonutti Skeletal Innovations Llc | Method of preparing a femur and tibia in knee arthroplasty |
US8858557B2 (en) | 2001-08-28 | 2014-10-14 | Bonutti Skeletal Innovations Llc | Method of preparing a femur and tibia in knee arthroplasty |
US10470780B2 (en) | 2001-08-28 | 2019-11-12 | Bonutti Skeletal Innovations Llc | Systems and methods for ligament balancing in robotic surgery |
US10231739B1 (en) | 2001-08-28 | 2019-03-19 | Bonutti Skeletal Innovations Llc | System and method for robotic surgery |
US9763683B2 (en) | 2001-08-28 | 2017-09-19 | Bonutti Skeletal Innovations Llc | Method for performing surgical procedures using optical cutting guides |
US8623030B2 (en) | 2001-08-28 | 2014-01-07 | Bonutti Skeletal Innovations Llc | Robotic arthroplasty system including navigation |
US8641726B2 (en) | 2001-08-28 | 2014-02-04 | Bonutti Skeletal Innovations Llc | Method for robotic arthroplasty using navigation |
US8834490B2 (en) | 2001-08-28 | 2014-09-16 | Bonutti Skeletal Innovations Llc | Method for robotic arthroplasty using navigation |
US8840629B2 (en) | 2001-08-28 | 2014-09-23 | Bonutti Skeletal Innovations Llc | Robotic arthroplasty system including navigation |
US20040070611A1 (en) * | 2002-09-30 | 2004-04-15 | Canon Kabushiki Kaisha | Video combining apparatus and method |
US7487468B2 (en) * | 2002-09-30 | 2009-02-03 | Canon Kabushiki Kaisha | Video combining apparatus and method |
US20060119621A1 (en) * | 2003-05-30 | 2006-06-08 | Claude Krier | Method and device for displaying medical patient data on a medical display unit |
US20090281566A1 (en) * | 2003-08-11 | 2009-11-12 | Edwards Jerome R | Bodily sealants and methods and apparatus for image-guided delivery of same |
US8483801B2 (en) | 2003-08-11 | 2013-07-09 | Veran Medical Technologies, Inc. | Methods, apparatuses, and systems useful in conducting image guided interventions |
US8150495B2 (en) | 2003-08-11 | 2012-04-03 | Veran Medical Technologies, Inc. | Bodily sealants and methods and apparatus for image-guided delivery of same |
US10470725B2 (en) | 2003-08-11 | 2019-11-12 | Veran Medical Technologies, Inc. | Method, apparatuses, and systems useful in conducting image guided interventions |
US7853307B2 (en) | 2003-08-11 | 2010-12-14 | Veran Medical Technologies, Inc. | Methods, apparatuses, and systems useful in conducting image guided interventions |
US20080298655A1 (en) * | 2003-08-11 | 2008-12-04 | Edwards Jerome R | Methods, apparatuses, and systems useful in conducting image guided interventions |
US11154283B2 (en) | 2003-08-11 | 2021-10-26 | Veran Medical Technologies, Inc. | Bodily sealants and methods and apparatus for image-guided delivery of same |
US11426134B2 (en) | 2003-08-11 | 2022-08-30 | Veran Medical Technologies, Inc. | Methods, apparatuses and systems useful in conducting image guided interventions |
US20050038337A1 (en) * | 2003-08-11 | 2005-02-17 | Edwards Jerome R. | Methods, apparatuses, and systems useful in conducting image guided interventions |
US9218664B2 (en) | 2005-09-13 | 2015-12-22 | Veran Medical Technologies, Inc. | Apparatus and method for image guided accuracy verification |
US20070060799A1 (en) * | 2005-09-13 | 2007-03-15 | Lyon Torsten M | Apparatus and method for automatic image guided accuracy verification |
US11304630B2 (en) | 2005-09-13 | 2022-04-19 | Veran Medical Technologies, Inc. | Apparatus and method for image guided accuracy verification |
US10617332B2 (en) | 2005-09-13 | 2020-04-14 | Veran Medical Technologies, Inc. | Apparatus and method for image guided accuracy verification |
US11304629B2 (en) | 2005-09-13 | 2022-04-19 | Veran Medical Technologies, Inc. | Apparatus and method for image guided accuracy verification |
US7920909B2 (en) | 2005-09-13 | 2011-04-05 | Veran Medical Technologies, Inc. | Apparatus and method for automatic image guided accuracy verification |
US20070066881A1 (en) * | 2005-09-13 | 2007-03-22 | Edwards Jerome R | Apparatus and method for image guided accuracy verification |
US9218663B2 (en) | 2005-09-13 | 2015-12-22 | Veran Medical Technologies, Inc. | Apparatus and method for automatic image guided accuracy verification |
US20100315416A1 (en) * | 2007-12-10 | 2010-12-16 | Abb Research Ltd. | Computer implemented method and system for remote inspection of an industrial process |
US8860810B2 (en) * | 2009-05-18 | 2014-10-14 | Peugeot Citroen Automobiles Sa | Method and device for extending a visibility area |
US20120062739A1 (en) * | 2009-05-18 | 2012-03-15 | Peugeot Citroen Automobiles Sa | Method And Device For Extending A Visibility Area |
WO2011048047A1 (en) * | 2009-10-19 | 2011-04-28 | Siemens Aktiengesellschaft | Hollow needle positioning system |
US20120303336A1 (en) * | 2009-12-18 | 2012-11-29 | Airbus Operations Gmbh | Assembly and method for verifying a real model using a virtual model and use in aircraft construction |
US8849636B2 (en) * | 2009-12-18 | 2014-09-30 | Airbus Operations Gmbh | Assembly and method for verifying a real model using a virtual model and use in aircraft construction |
US8781186B2 (en) | 2010-05-04 | 2014-07-15 | Pathfinder Therapeutics, Inc. | System and method for abdominal surface matching using pseudo-features |
US10264947B2 (en) | 2010-08-20 | 2019-04-23 | Veran Medical Technologies, Inc. | Apparatus and method for airway registration and navigation |
US10165928B2 (en) | 2010-08-20 | 2019-01-01 | Mark Hunter | Systems, instruments, and methods for four dimensional soft tissue navigation |
US8696549B2 (en) | 2010-08-20 | 2014-04-15 | Veran Medical Technologies, Inc. | Apparatus and method for four dimensional soft tissue navigation in endoscopic applications |
US11690527B2 (en) | 2010-08-20 | 2023-07-04 | Veran Medical Technologies, Inc. | Apparatus and method for four dimensional soft tissue navigation in endoscopic applications |
US11109740B2 (en) | 2010-08-20 | 2021-09-07 | Veran Medical Technologies, Inc. | Apparatus and method for four dimensional soft tissue navigation in endoscopic applications |
US10898057B2 (en) | 2010-08-20 | 2021-01-26 | Veran Medical Technologies, Inc. | Apparatus and method for airway registration and navigation |
US20120240077A1 (en) * | 2011-03-16 | 2012-09-20 | Nokia Corporation | Method and apparatus for displaying interactive preview information in a location-based user interface |
US8601380B2 (en) * | 2011-03-16 | 2013-12-03 | Nokia Corporation | Method and apparatus for displaying interactive preview information in a location-based user interface |
US9972082B2 (en) | 2012-02-22 | 2018-05-15 | Veran Medical Technologies, Inc. | Steerable surgical catheter having biopsy devices and related systems and methods for four dimensional soft tissue navigation |
US11551359B2 (en) | 2012-02-22 | 2023-01-10 | Veran Medical Technologies, Inc | Systems, methods and devices for forming respiratory-gated point cloud for four dimensional soft tissue navigation |
US11830198B2 (en) | 2012-02-22 | 2023-11-28 | Veran Medical Technologies, Inc. | Systems, methods and devices for forming respiratory-gated point cloud for four dimensional soft tissue navigation |
US10977789B2 (en) | 2012-02-22 | 2021-04-13 | Veran Medical Technologies, Inc. | Systems, methods and devices for forming respiratory-gated point cloud for four dimensional soft tissue navigation |
US9138165B2 (en) | 2012-02-22 | 2015-09-22 | Veran Medical Technologies, Inc. | Systems, methods and devices for forming respiratory-gated point cloud for four dimensional soft tissue navigation |
US10460437B2 (en) | 2012-02-22 | 2019-10-29 | Veran Medical Technologies, Inc. | Method for placing a localization element in an organ of a patient for four dimensional soft tissue navigation |
US10249036B2 (en) | 2012-02-22 | 2019-04-02 | Veran Medical Technologies, Inc. | Surgical catheter having side exiting medical instrument and related systems and methods for four dimensional soft tissue navigation |
US11403753B2 (en) | 2012-02-22 | 2022-08-02 | Veran Medical Technologies, Inc. | Surgical catheter having side exiting medical instrument and related systems and methods for four dimensional soft tissue navigation |
US10140704B2 (en) | 2012-02-22 | 2018-11-27 | Veran Medical Technologies, Inc. | Systems, methods and devices for forming respiratory-gated point cloud for four dimensional soft tissue navigation |
US10617324B2 (en) | 2014-04-23 | 2020-04-14 | Veran Medical Technologies, Inc | Apparatuses and methods for endobronchial navigation to and confirmation of the location of a target tissue and percutaneous interception of the target tissue |
US11553968B2 (en) | 2014-04-23 | 2023-01-17 | Veran Medical Technologies, Inc. | Apparatuses and methods for registering a real-time image feed from an imaging device to a steerable catheter |
US10624701B2 (en) | 2014-04-23 | 2020-04-21 | Veran Medical Technologies, Inc. | Apparatuses and methods for registering a real-time image feed from an imaging device to a steerable catheter |
US10217209B2 (en) * | 2015-12-11 | 2019-02-26 | Daqri, Llc | System and method for tool mapping |
US10679337B2 (en) | 2015-12-11 | 2020-06-09 | Daqri, Llc | System and method for tool mapping |
CN110638529A (en) * | 2019-09-20 | 2020-01-03 | 和宇健康科技股份有限公司 | Operation remote control method and device, storage medium and terminal equipment |
US20210181843A1 (en) * | 2019-12-13 | 2021-06-17 | Fuji Xerox Co., Ltd. | Information processing device and non-transitory computer readable medium |
US11868529B2 (en) * | 2019-12-13 | 2024-01-09 | Agama-X Co., Ltd. | Information processing device and non-transitory computer readable medium |
Also Published As
Publication number | Publication date |
---|---|
US6690960B2 (en) | 2004-02-10 |
US20030029464A1 (en) | 2003-02-13 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US6690960B2 (en) | Video-based surgical targeting system | |
US5765561A (en) | Video-based surgical targeting system | |
JP2950340B2 (en) | Registration system and registration method for three-dimensional data set | |
EP1103229B1 (en) | System and method for use with imaging devices to facilitate planning of interventional procedures | |
JP4469423B2 (en) | Stereotaxic treatment apparatus and method | |
CA2003497C (en) | Probe-correlated viewing of anatomical image data | |
EP2637593B1 (en) | Visualization of anatomical data by augmented reality | |
Bichlmeier et al. | The virtual mirror: a new interaction paradigm for augmented reality environments | |
EP0741540B1 (en) | Imaging device and method | |
US7715602B2 (en) | Method and apparatus for reconstructing bone surfaces during surgery | |
USRE43952E1 (en) | Interactive system for local intervention inside a non-homogeneous structure | |
US20070038059A1 (en) | Implant and instrument morphing | |
KR20190058528A (en) | Systems for Guided Procedures | |
WO1998038908A1 (en) | Imaging device and method | |
US20060036162A1 (en) | Method and apparatus for guiding a medical instrument to a subsurface target site in a patient | |
JPH11178837A (en) | Reference structure constitution system and reference structure assembly | |
US8704827B2 (en) | Cumulative buffering for surface imaging | |
EP1465541B1 (en) | Method and apparatus for reconstructing bone surfaces during surgery | |
GB2614025A (en) | Surgery guidance system | |
EP4169470A1 (en) | Apparatus and method for positioning a patient's body and tracking the patient's position during surgery | |
Hirai et al. | Image-guided neurosurgery system integrating AR-based navigation and open-MRI monitoring | |
WO2009085037A2 (en) | Cumulative buffering for surface imaging | |
Gerritsen | Virtual Reality in medical imaging for image-guided surgery |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |