CA1239217A - Method for operating a microscopical mapping system - Google Patents

Abstract

TITLE
METHOD FOR OPERATING A
MICROSCOPICAL MAPPING SYSTEM
ABSTRACT OF THE DISCLOSURE
A "flying field" computer aided interactive microscopical mapping system has been developed that enables an operator to map over a region of the specimen covering many microscopical fields of view without the discontinuities associated with moving the stage and then generating mapping marks as separate and discrete operations. By fixing the mark generation point at a single location on the display device corresponding to the instantaneous intersection of the microscope's optical axis with the specimen stage, a trail of marks are produced that move in unison with the stage motion to remain superimposed with the viewed live specimen image. The points thus recorded are available for later display and analysis.

Description

~ITLE
METHOD FOR OPERRTING A
MICROSCOPICAL MAPP I NG S Y S TE~
BACKGROUND OF THE INVENTION
5The present inv2ntion relate~ tD computer aided interactiva microscopical mapping in ~peci~ens viewed thcough a microscope.
A number of computer-aided ~icro~cope mapping in~truments in the prior art involYe op~ically combining ~he image of a video display with ~he live image fro~ the microscope objective, 6uch that when ~he opera~or look6 anto the ~icro~cope ocular~ he 6ees both images togeeher, superimpQ6ed o~ sach other. A
cur60r on the video di6play i6 ~ypically controlled by the operator using a 30y6t~ck or other graphic input device (GID~. He can map information into the computer by moving the cuc~or over ~he viewed 6pecimen and pre~si~g a button eo en~er coordinates into the computer. Once all featuce6 of interest i~ the current field of view have been entered, he ~ova~ the ~:~tage and Lesumes mapping. I~ ha~ been ob6erved that the di~continuieies asso~iated ~ith the~e Nfield hop6 are very di ruptive of the operator~ 6 6ense of orientation within the 6pecimen ~nd of the continuity of da~a collectio~. ~or example, the ~ystem of Glaser and Va~ der Loo6 (U.S. Paten~ 4,202,037) employ6 a configuration of computer and ~icroscope, wherei~ the u~er con~rol~ by means of the graphical input device ~GID), the po6ition of a ~blinking ~" on the di~play device. The po~ition of the ~blinking ~" can repre~ent, vaciou~ly, the position on the display at which ~h~ u~er wishe6 to generaee a mapp;ng ~ark, or ~he 6election o a 6y~em control command which i6 di~played on a ~edicated portion of the di6play CR-8333 35 device. If the u~er wi~he~ to place a mark over a ~l~3~2~ ~

location in the ~pecimen outside ~he cu~rent field of view, he ~u~t ~irst command the micro~cope ~tage to move such that the desired location i~ bcough~ within the field of view~ and then he mu6~ manipulate ~he GID
in order to bring ~he ~blinking ~" ovel ~he loca~ion of interest. Finally, he mu6t indicate. by pressing a button or by other mean6, that the 6y~tem 6hould record ~he current loca~ion of the Uhlinking X~l. The ~y~tem doe~ so, and ~imultaneously generate~
appropriat0 ~arks on the display in ~ubstanti~l coincidence with the 6pecimen a~ an indica~ion to the operator of ~he action taken. ~hi~ ~ethod of ~icro~copical mapping or 6~etching may be generically ~efeered to a6 "moving ~ur~or ~apping with visual feedback." Those 6killed i~ the art ~ill recognize that to ~lap over a region of the 6pecimen that cover6 ~any mic~o6copical fields of view it will be nec~66ary to move the 6pecimen stage many times and as noted above the problem~ with discon~inuitie~ a~ociated 20 with these 6tage move6 or "field hop6" are very di~ruptive.
In thi~ invention thi6 problem i~ overcome by keeping the video cuc~or fixed at all time~ at the Gen~er of the micro6copical field o~ view, and cau6ing the stage to ~ove under this cursor. ~hinking of ~he cur~or a~ the tip of hi~ ~'pen", the operator instantaneou~ly see6 his "ink" in the form of luminous marks produced on the video display trailing behind the cur60r as he roam6 freely throughout the ~pecimen. ~ark~ which leave the field of view are stored by mean6 which allow ~hem to be ins~antaneou~ly and peeci~ely redi~played upon retu~n to a pe~viou~ly mapped req;on.

~3~

SUMMARY OF THE INVENTION
The p~esent invention it6elf compri~es an impLoved method f OL a user to interact 6imultaneou~1y WiCh the compute~ means and the microscope in a 5 computer-aided microscopical mapping system. More paL~icularly, in ~he present inven~ion, a microscope is employed which 6uperimpose6 the images from a mapping di~play device and the specimen. The ufier employs a graphical input device to convey graphical or posi~ional information ~o the computer. In re6pon62 to this positional information input, ~he computer es~entially ~imultaneously ~ontrols the po~ition of the ~pecimen stage, generate& ~ark6 at a fixed locat;on on the mapping display device, moves the previously generated m~rk6 with respect to the di~play, and produces numerical indicia. The position~ of the marks on the mapping di6play device are moved such that the macks move in unison and in apparent coincidence with the image of the specimen a~
viewed by the user in the microscope oculars. Those mark~ which leave ehe visible window ~f the mapping display de~i~e are p~eserved in their proper rela~ive posi~ions and are redisplayed as eequired by the condition of coincidence of the marks with the previously mapped portion~ of the specimen. The numerical indicia comprise a~ accurate repre~entation of the location and charac~er of user-selected portions of ~he historical trajectory of the specimen 6tage. For reasons which will be made mo~e apparent below, ehis method of interaction between the u~er and the compute~/microscope ~ys~em i~ ~efe~red to herein as "flyiag field ~apping with Yi~ual ~eedback."
In operation, a drawing tube projects ~he image of t~e mapping display device into ~he microficope in 6uch a way that its image appeacs to ~he ~39~'L 7 user &upe~imposed on that of the magni~ied ~pecim~n.
Displayed on a monitor i6 a video cur~o~ (a ~mall cros6 haic) which appear6 at the center of the field of viewr Also di6played on the moniLtor i6 an N x N
pixel portion of a 2048 x 2048 pixe]L digital image memory which is called the mappiny di6play window.
The window ~ize N i~ typîcally between 16 and 512 pixel~ and depend6 on the power of the objective ~elec~ed.
The digital image memory is organized as 204 row6 by 2048 columns of pixel~, each pixel repre6ent6 ~ bits or 1 byte of ~torage. ~he operator deEine6 a one to one corIe~pondence between ~ and y dimen6ion6 of t~e 6pecimen and column and row addre~6e~ in the image memory such that each pixel in the image memory coere6pond~ ~o a unique Carte6ian coordinate in the specimen. Storing a value in an image memory location corre6pond~ to generating a mark on the mapping ~li6play .
In order to map, the operatoe manipulate6 the 6tylu~ of a gcaphics tablet, which transmit6 to the computer information on the di6tance and direction of tAe ~tylu~ ~otion. On the ba~is of thi~ input, the computer doe~ three principal thing6. Fir~t, it cau6e6 the ~i~roscope ~tage to ~ove in a corIe~ponding : way. Second, it drive~ the ~apping di6play window ~hrough the laege image memory in 6uch a way ~ha~ the di~play appear6 to moYe in perfect regi6t~y with ~he live image of the 6pecimen a~ viewed in the ~icro6cope. Third~ i~ determines whether a button on ~he stylu~ ha~ been de~re~6ed by the opecatoe. If ~o, the computer then:
Record~ the curren~ stage coordinate~ in a vec~oc li6t in the computer'~ main memoey.
~ Store6 a value in the digital image memory.

Draw~ a line on the display from the location that previou~ly ~a6 at the center of the field to the curren~ center of ~he field.
It is important to note that since all data are captured from the optical axi~ of the mi~roscope, ~he data are immune ~o corruption due to geometrical distortion in the mapping di~play device or in the optical ~y~em. The corre~pondence between ~tylu~
motion and s~age motion i~ carefully cbo~en ~uch that the u~er feel6 as though he i~ flying over the imaqe of ~he 6pecimen a~ i~ a helicopter. It is thi~ ~act whieh allow~ the u6er to perform highly detailed mapping i~ the context of a speci~en who6e dimension~
are much larger tha~ that vifiible within one micro~copic field of v~ew.
Digital control of ~he mapping display window 8ize allow~ registration of ~he mapping display with the live image under several different micro~cope 20 objective6 of different magnifications. Heans are also provided to ~ap ~evecal discrete type6 of info~matio~ u6ing the color and 6hape attribute~ of ~he mapping mark to di~tingui6h among them.
The principal advantage of this i~Yention 25 over the prior art method lies in it~ ability to caeidly map in a ~egion of the ~pecimen that extend6 over many microscopical fields of view without ~he di~continuities and inaccuracies a660ciated with ~he requireme~t of moving the 6~age and then generating 30 ~he mapping ~ark~ a~ sepa~ate and di~crete operations. It will be appreciated al60 that, contrary~to a ~moving cu~sor" ~e~hod o~ mapping. the "flying ~ield" method of ~apping avoid~ cor~uption of th geometrical da~a due to di~tortion inherent in the 35 di~play devise or in the op~ical ~y6tem, due to the fact tha~ the poin~ of diyitization ~emain6 fixed with reapect to the op~ical axi6.
The apparatus of ~he pre~ent invention co~pri6e~ a computer with a memocy for ~oring data, a manually-controlled g~aphical input device lGID) for manually conveying graphical information into the compu~er, a po~ition-cont~olled ~pecimen ~tage, a ~apping di6play deYice with a vi~ible window corresponding to ~he field of YieW (objective 10 ~agnif ication), and a ~icro6cope where the optic~
therein p~oduce a 6uperimposi~ion of the image of the ~peçimen and the image p~oduced by the mapping di~play device a~ the ocuiar6 of 6aid micro~ope.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 i6 a block diagran of one embodiment of the 6y6tem of thi~ invention.
FIGS. 2A-ZK are ~chematic diagram~ of micco6cope view~ of an object on a ~tage illu~t~ating the advantage of the method of the inventio~.
FIGS. 3A-3H repre~ent a flow charl of a program fQr contcolling the ~ardware components and managing a data ba~e according to the invention.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIM~NT
he Sketchinq Sy6 tem Referring to FIG. 1, the 6ketching 6yfitem choEen for purpo6e of illu~tratio~ include~ a ~icro8~0pe 10, typically a Nikon Fluophot, with an ultrawide Optiphot head 12 that i~ modified fos moun~ing two ~tanda~d format video cameras 14, and 16 with ~he ocular6. Video camera 14 is a DAGE/~TI ~odel 6~ ~ilicon diode array vidi~on, while video camera 16 is a DAGE~MTI CorpoLation ~odel 66 with a ~ilico~
inten~ified target ~SIT) tube for low light level 6en~ing. The outpu~s of came~as 14 and 16 are connected to the analog to digital (A~D) input * denotes trade mark ' ~erminal~ of image proce~or 18, typically an Imaging Technology, lnc. IP512 syztem that include~ 3 FB-512 board6, 2 AP-512 board~ and an arithmetic logic unit (ALU) board. It communicates with the other component~ of the sketching 6y~tem over the 22 bit Digital Equipmen~ Co. (DEC) Q-bu~ 20. Image pcoces60r 18 function6 ~o ~ynch~onize ~he inpu~ and ou~pu~ flow of digital image data. ~o digitize the analog camera output informa~ion. and to convert the processed digital data into analog form for viewing on ~he monito~. A drawing tube 22 with a beam ~plitter ~onvey6 the image appearing on a fir~t mapping di~play monitor 2~ i~to t~e micro~cope barrel ~o enable coincident viewing of both the magnified 6pecimen and the corre6ponding mapping data. A 6econd display monitor 26 i~ u~ed to di~play the identical information a6 that projected by monitor 24. ~oth di6play monitors, 24 and 26 are high-resolution color monitor6, (Mit6ubi~hi Electronic~ of America Inc.
model No. 3919), and are bo~h connected to one of the : output digital to analog ~D/A) terminal6 o~ image p~oce~o~ 18. Connected to a fiecond D/A output terminal i6 tracking di~play monitor, 28, typically a Sanyo Inc. black and white monitor. Thi~ monitor ;~ 25 displays a lcw magnifi~ation static image of the complete section which remain6 fixed independent of the continuou61y changing mapping data displayed on monitor~ 24 and 26. ln addition, the trackinq di6play has 6uperimpofied on it a video ma~er, which indicates the in6tantaneous location and size of the higher-magnification field of view in ~he context of the specimen.
The micro~cope stage as6embly 30 comprise~ a NI~ON DIAPHOT 6~age on which are mounted two BURLEIGH
In~rumen~ Inc. IW-502 inchworm x,y micropo~itioner6 * denotes trade mark ~3~

and EN-372 encoder~ 34, 36. Stage controlle~ 38 compri6es an INTEL Corporation model No. 80B5 microprocessor fo~ providing clo6ed-loop control for bo~h ~ets of po~itioner~ and encoder~ with I/0 code~
on a STD data bus. The mic~op~oces60r i6 al~o pcogrammed to communicate with the ho~t computer 6y6tem 40 over the bu~ 20 The inch~worm micro-positioner~ enable the ope~ator to rapidly and ~moothly move the stage acro6~ i~6 full range of t~avel. Typical performance yield~ 1.0 micrometer (~m) absolute accuracy on 2 axes bidirec~ionally ovec a ~ange of 50 mm on each axi6. Both axes are under ~imultaneous clo~ed loop control, with po~ition feedback derived from 1 ~m lineaL optical encoder~.
Con~equently. the stage can be moved and returned to it~ init.ial position with } ~m accuLacy and with no backla~h. The communication~ interface with ~he host processo~ ~upport~ at least 150 coordinate pair update6 pec ~econd.
Computec ~y~tem 40 comp~i6efi: a DEC LSl 11/23, which includes a Q-bu~: a 256 kbyte main memory; a DLV-llJ quad serial data interface a Data System~ De6ign, Inc. ~odel No. DSD 880 30 ~by~e Winche6teL*disk d~ive with a l~Z ~byte floppy di~k; a 25 Cipher Data Pcoducts Inc. M3~1, 9 track 3200 bpi cache ; 6ereamer magnetic tape drive; a Microline printer manufactu~ed by Ok;data Corporation: a Hewlett Packa~d 7475A 6 pen plotter; and a ~EC model ~o. ~RV 11 parallel interface to the ~tage cont~oller.
Communicating with computer 40 i~ a VEC model ~o. VT100 te~minaI 42 to which i~ attached an Inter~tate Electronic6, a division of Figq;e International Inc.. model No. VRT-300 voice data entcy unit (VDE~ 44. Tactile input i~ provided by a g~aphic~ ~ablet 46, typically a Summagcaphic6 * denotes trade mark p;~

Corporataon model No. ~1201 with ~ylu6. A
three-dimensional (3-D) graphic~ proce~60r 4B, typically a Lexida~a Corporation graphic6 proe~o~
i~ u~ed to di~play a 3-D recon6truction frvm a serial

2-D data ~et.
Image memory S0 and ~DMA) data mover cont~oller 52 ~Image hnalytic~ Corporation model QD512) are ~ardwired circuits that co~unicate directly with the I~0 ~ec~ion of the ho~t computer on the bu~ 20 to impro~e i~age data handli~g capacity and to ~peed ~p the di~play of image information on the ~onitors.
The image memory 50 compri6es ~ ~tack of four addre~able memory board~, each with a 1 Mbyte ~8 bit~
per byte) capacity, to pEovide ~torage for a 4 Mbyte, 2048 x 2048 pixel image with full eight bit6~pixel re~olution.
The capacity of image memory 50 i6 u6ed to ad~antage in the invention. In ~flying-ield"
mapping, t~e capacity i~ available to 6tore graphi~al mapping data which can cover lacge a~ea~ of the 6peci~en at high resolution. Since the memory i~
organized by lOW~ and column6, each memory cell is assignable, by virtue of it~ location within the memo~y, ~o a unique coocdinate in the specimen.
Mapping i~ ~hen accompli6hed by sto~ing as~igned value~ in those ~emory cell location~ that corre6pond with the trajectory of the ~appi~g cur60r over the 6peci~en. Fu~thermore, the ~toced value~ ae~
a~6ignable fro~ 8 bit6 ~256 value6) available and may be ~ade to corre~pond with a uni~ue color on the mapping display ~onitor 24.
Data ~over contcoller 52 ifi a direc~ memory acce6s (DMA~ device capable of high 6peed tran~fer of data ~rom a sou~ce loca~ion to a destination location on bu~ 20. Specifically, it perfo~m the functions of pan, sc~oll, and zoom of ~he ~ x N pixel mapping di~play ~indow within the 2048 x 2048 pixel image mem~ry 50. Thi~ i6 of primary impoltance in our '`flying-~ield" mapping concept. which requires that the mapping inormatio~ stored i~ i~age ~emory 50 be made available eo the mapping di6play monitor 24 in a rapid and con~inuou~ ~anner.
Tracking di6play moni~or 28 di~play~ a digitizad vi~eo image cap~ured by video came~a 14 through a 1~ objective. Sub~equently, thi~ image ~ay be Gontras~ enhanced by i~age proce660r 18 unde~
operator control wit~ the GID to vary brightne~6 and gain. Upo~ initiating the mapping ~equence, image proce~or 18 generates a video box 6uperimpo6ed on the tracking di6play, the 6ize and position of which indicate6 the 6ize and location o~ the micro~cope field of view 6een by the operator. This i6 accompli6hed by varying the 6ize of the box to reflect change6 in the field of view with objective magnification, while the po6ition of the center of the box remains ~ixed with ~tage po6ition.
SY6tem Operation ; To operate t~e "flying-field" mapping sy~eem, the operator initially po~ition~ the ~icroscope 10 field o~ view over ehe specimen of inte~e6~ with the aid of tracking di6play monitor 28. He doe~ thi~ by positionin~ 6~aqe assembly 30 via cont~olle~ 3B using the graphics tablet and stylu6 46, or the terminal 42, a~ the ca~e may be. Once the ~pecimen i6 in po~ition, the operator c~n ~pecify by VDE ~4 or terminal 4Z the ~agnification and type of objec~ve len~, the color foc ~e ~apping mark, and whethe~ the mapping i6 to be i~ the form of line6 or location ~aYk~.

3~

To initiate mappi~g. the operator pres~e6 a but~on o~ ~he 6tylu~ of e~e graphics tablet 46. Then a~ he move6 the ~tylus over the graphi~ tablet~ a 6e~eam of x,y coordinate pair6 i~ ~ransmieted to the 5 ~Dmputer 6y~tem 40. Upon receipt of each new x,y ~oordinate pair input, the compu~er does the ~ollowing:
o It ~au6e~ the ~icroseope ~tage to ~ove i.~
the diec~ed way f~om its current po~itiDn.
~ It cau6es the datamoYer S2 ~o roam ~he mapping di~play vindo~ tllrough the image ~e~ory 50, zu~h that ~he mark~ on the mappinq di6play monitor~ 24, 26 appea~ to ~ove in pe~fec~ regifitEy wi~h the im~ge ~rom the 6pecimen as viewed by ~he operator through the micro6cope ocular6.
It continuously update6 ~he ~arker po6itio~ on the tracking di6play to indicate the cucren~ location of the field o view in the context of ~he ~ntire specimen.
It ~ontinuou~ly records the in6ta~taneous ~tage coordinates in the vector li6t provided by the main memory and locates mar~ in the 4 Mbyte digital i~age memory 50~
draws a line o~ the ~apping dasplay from the la6t incremental center of ~he field of view (FOV) to ~he current cen~er.
Wi~h boundarie~ and points ~apped for a given 30 ~pecimen of a sequence of ~e~tion6, ~he reEult~ may be cataloged by ~-coordinate and 6ent to 3-D graphic6 proces~oz ~8 along wit~ si~ilar data ~om the other 6pecinen~ with different z-coo~dina~e value~. Thi~
daea can then be cendered a~ a~ es~entially 3-di~en~io~al image rep~esen~a~ive sf the original 3-dimensional sec~ioned 6~ ucture.

2~

FIGS. 2A through 2K a~e u~ed to exemplify the ~y6tem ope~ation. More particula~ly, each pair of FIGS. 2A, 2B: 2C, 2D: ZE, 2F; 2G, 2H: and 2I, 2J, represen~ a micro~copic field of YieW and a tracking di~play for that field of view at the magnification~
indicated below each illu~t~ated field of view.
FIG. 2K i~ the diplay of ehe data obtained from the following ~ypical mapping 6equence of operations as seen on the mapping display monitor 26.
Upon looking into ~he oculals, the opera~o~
may ~ee an image ~uch as FI~. ZA, and a~k, ~Iwhere am I?~. Without changing magnification, he can li~t hi~
eyes from the ocular6 and ~ee in the image di~playc~
on the tracking monitor 2B (FIG. 2B) the location and extent of hi~ FOV that appea~6 outlined by the video box 60. He may then decide to ~ap the features apparent in the lower right hand co~ner o FlG. 2A.
By moving the stylu6 on the graphic6 ~ablet 46 downwa~d and to the right, the field of view ~hift~
accordingly (FIG. 2C). ~hen by enabling ~he ~appiny ma~k funceion, a mark is gene~ated and appeac6 a6 a dot 62 on the mapping display and in the micco6cope field of view 6upe~impo6ed with ~he image of the 6pecimen. Note that the video box 60' of FIG. 2D has al~o moved to match the ~ew coordinateE of the FOV.
A~ the operato~ continues to map the contour, he create~ a series of dot6, or line~ (with the line mapping ~ode) that ~emain ~ixed in position with lespect to the specimen (~tage) coordinate6. At any : 30 point, he may wi~h to change magnification ~o enable him to follow fine featu~e~ such a~ the small contour~
64 o ~IG. 2E. When he doe~ thi~, the position of the mapping ma~k~ are adju~ted to remain aligned with the pecimen, and the 6ize of the video box 60" on ~he tracking display becomes smaller (FIG. 2F~. Should ~3~

the ope~atoc elect to ~ap a remote feature, he i~ e to do ~o by po6itiGning ~he FOV over the feature u~ing hi6 stylu6 while watching the tracking di6play ~ideo box 60''' (FIG. ZH~ and commence mapping. He cau~es point6 gl gn ~FIG. 2G) to be genera~ed by depre~sing a button on ~he 8tylu6 a6 he move6 the ~tylu~ in th~ direction selec~ed. Should ~he operaeor wi~h to return the video box 60iY to the initial 6ketching po6i~ion, (FI~. 2J), he ifi then able to in6tantaneou61y re~ume mappi~g the initial feature ~FIG. 2I).
Having completed the ~apping ~e~6ion, the operator obSain~ a readout fro~ ~emory of the data point~ recorded from the ~pecimen examined. Data output may be in hard ~opy orm or vi6ually di6played on the mapping display monitor (FIG. 2K~. Fcom a ~eries o~ 6pecimen mapped ~ection6, one can a~emble a 3-D figure of the mapped object u~ing known techniques and analyze it~ ~tructure.
DESCRIPTION OF FLOW CH~RTS
The number~ preceding each paragraph refer to the numbe~ed ~tep~ of the flow chart6 6hown in FIGS.
3A-3H.
Referring f il6t to Figures 3A and 3B is the initialization procedure that i6 completed ~or each.
~tage mounted 6pecimen, before ~aeping commence~, where:
Step 100 - re6tore6 ~o~h intecnal and exte~nal 6y~eem pointe~ and regi~ters eo their proper initi~l condition6.
In Step~ 101 - through 105, the system allows the operator to acquir* and optimiZe ~Ol viewi~g a low-magnification video image a~ a tracking di6play image. Thi6 image cover~ a 6ub6tantial portion of the ~pecimen.

3.2~L~

S~ep 101 - display~ on monitor 2~ a live video image ~ensed by camera 1~ through a low-power IlliClO6COpe objective.
Step 102 - In respon~e to opera~or command~, drives the ~tage in ~uch a direction tha~ the de~ired region of the ~pecimen image i6 centered in the tracking difiplay monitor 28. Once centered, the po~ition of the ~tage define6 the Y and y offset values (I~OFF, IYOFF) ~ha~ are applied ~o ~ub~equent 6tage positions before they are recorded in the ~e~tor li~'c .
Step~ 103 throu~h 104d - pecform image digitization and frame averagi~g at video rate~ to minimize 6ta~i6tical noise in the T~ image.
S~ep 105 - Enabls6 ~he operatoc to optimize image contra6t and brightne6~ in the TD image.
_tep 106 - Specifie6 a set of default 6y6tem parametec6 that include~: (a) micro6cope objective (magnification and type); (b) the color ~ink~ to be 20 used with the mapping diselay~ 2~, 26 and (c) the line ~apping mode (Figure6 3C, 3D)9 rather than the mark mapping mode (Figure6 3E, 3F).
Step 107 - Enable6 a voice data entry (VDE) interrupt ~ervice routine that i~ further detailed in 25 fitep6 107a throug~ 107f in the flow diag~am of ~igure 3B. Thi6 routine enable6 the opera~or to ~hange the default ~onditions e6~ablished in step 106 u6ing the choices available ~o him from tho6e li6ted in Figure 3G. A particular ASCII character i~6ued by VDE 4~ in 30 re~pon~e ~o a recognized ut~erance i~ defined to : repre6ent one of the li~ted parameter~.
~ fter initialization, the sy6~em defaultfi ~o the line mapping mode of Figures 3C and 3D, ~here:
Step 200 - In ~e6ponse to a request fcom CPU
35 40, 6tage controller 3~ re~urn6 the ab~olute x,y ~tage coordinate~.

~3~

5teps 201 and 202 - Taking into accoun~ IXOFF
and IYOFF. apply the ~tage posi~ion coordinate6 to ad~u~t ~he po~ition of the mapping di~play window in digital image memory 50 60 a~ to conform the ~apping display image ~ith the previou61y de!termined relation~hip be~ween 6~age coordinate~ and diqital image ~emory address. ~ny mapping ~lark6 previou~ly e~tered will now align accurately wi.th the image of the specimen ~8 ~iewed through the micro~cope ocula~.
Step ?03 - Check6 for the exi~tence of any parameter change~ entered via ~DE 44. If none exist, then -Step 204 - Call~ graphics tablet 46 which retu~ns the current ~tylu6 X,y po6itio~ togethe~ with the current 8tatu6 of the ~tylu~ ISWI and ITOG
6witche~. The 1,0 6tate of ISWI is ~ontrolled by tip pre66ure of the ~tylus against the gcaphic6 tablet ~u~face, wherea~ ITOG i~ a logical va~iable ~either "true" oc "fal6e") who~e value i6 ~witched each time ISW~, (a 6witch located on the barrel of the 6tylus within reach of the operato~'~ index finger~ i~
depre~ed.
Step 205 - C~mpute6 the difference between ~he current and previou61y recorded 6tylu~ po6itions to obtain a ~ Y data pair.
Step 206 - Determine~ whethez the stylu6 tip i~ depre~sed, and if 60, then, Step 207 - Dete~mine~ whether ~ Y, or both are 6uficiently large (if their ab~olute value6 are greater than pre~cribed ~, ~Y minima) for moving ~he ~tage. Should a~y of the above ~wo tests fail, the algorithm return6 to ~he t~p of step 203 for another ta~let ~eading. Con~inuing, Step 208 - Check~ ehe sta~e of ITOG to detecmine whether the operator want~ to map the current cur~or po~ition with luminou~ mark~ on the mapping di~play, o~ "IMK", whece, Step~ 209a, b - Control the "INK" ~upply -Next~
Step 210 - Cau6e6 the N x N pixel mapping di6play window to roam through the image memo~y 50 tas controlled by DMA 52) in an amount and direction 6pecified by the tablet ~, ~Y. Recall that the 6ize of N i~ dependent upon the objective len~
~agnification selected.
Step 211 - Responds to the tablet ~X, ~Y
to ~ove the ~eecimen ~tage via ~tage controller 3B, while Step 212 - ~oves the TD video box acco~dingly on monitc~ 2~ via image pcoce~or 13. The TD ~ideo box i6 an indicato~ di6played over ehe low-~agnificatîo~ ~tatic image which 6how~ the operator the po~ition and 6ize of his higher magnification microscopic field of view in the context of the 70 6pecimen.
Step Z13 - Recente~ ~he mapping di~play cur~or in t~e field of view a~ a result of moYement of the mapping dî~play window in the ~ Y
directio~. This correction is accompli6hed by image 25 p~oces60~ 18 Step~ 214 throuq~ 218 - P~ovide for (a) ; recordi~g either a null vector vertex (I~, IY, ICODE =
O) or a "new" vecto~ vertex (I~ , ICODE ~ O) in ~he vector li~t, ~b) incrementing the.vector li6t pointer 30 and Sc) geneLating a ~apping line, before retu~n ~o junc~ion A3, located just before ~tep 203. The vectoL
ICODE parameter cla~fiifie6 a Vertex into one of three categorie6:
ICODE = O a null vecto~ vertex(no length or direc~ion~

~3~

O<ICODE < 20 a Iine vecto~ Yertex (origin or terminu~
20<ICODE < 40 a ma~k.
The color and 6hape of a line or mark i8 S further defined by it~ po~ition wiehin the range to corre6pond with a particular mapped object.
Recycle time through the ~tep~ of Figure6 3C, 3D is between 40 and 150 time~ a 6econd.
Figura6 3E and 3F cover the ~ark mapping mode algorithm which i~ ~imilar in mo~t ~espect~ to that of Figure 3C, 3D~ In contra6t wath the u6e of ~SWl and ITOG for controlling a continuou~ flow of "IN~" or line mapping when the ~tylu6 t;p i~ pre66ed again~t the tablet (controlled by ISWl); ISWl and ISW2 are used to produce a mark.
Step 306 - Allow~ ISW2 to act a6 a ~lutch.
With ISW2 clo6ed (true), the tablet i6 disengaged from i~6 control function~, and when opened, (false), control i~ enabled.
Step~ 307 through 311 - are analogou~ to 6tep6 2Q7, and 210 through 213 of Figure6 3C, 3D.
Step 3 - Te~6 whether the 6tylu~ tip ha~
bean depre ~ed again6t ~he tablet. If so, then steP
313 t~s~ whe~her the tip ha6 been lifted from the tablet, thu~ opening ISWl. If ~o ~hen Step 314 - Store~ a new entry in the vector li~t with ICODE ~dentifying it a6 a mark.
Step 315 - Increment~ the vector li~t pointer, and 3Q Step 316 - Generate6 the appropriate mappinq ~ark o~ mappi~g di~play monitor~ ~4, 26.
Figure 3G a6 briefly mentioned e~rlier, coneains the ~tep~ 400 through 411 that re~psnd to the variou~ parameter-~pecifying ASCII key character~.

~3~

After an appropriate action ha~ been ~elected, ~he flow pa~se~ to Node H and thence to ~tep 412.
Step 41_ - Re~e~6 the value of KEY 80 that the 6y6tem will be ready ~o intelcept a new command character.
Step 413 - direce~ 10w to either the line mapping mode of Figure 3C or the mark mapping ~ode of Figure 3E.
~ he erase and qui~ operation6 o~ ~tep~ 409, 410 appear in Figure6 3H and 3I, re~pectively.
seep 500 - Provide6 two era6ing routine6:
point era~e and ~egment era~e. A "point" i~ either a ~ingle mapping ~ark or vectoc ver~ex that ha~ been written onto the mapping di6play 6creen. A "~egment"
i6 either a contiguous 6equence of vertice~ compri~ing a logical line ~egment, or a 6ingle mark.
Steps 501 throuqh 504 - provide the 6tep~ for a point erase routine, while S~eps 505 506. 504 - Provide for segment era~ure.
The QUIT routine of Figure 3I provide6 the 6eep6 600 through 603 needed ~o ~tore data on di~k and ~o di6able the VDE interlupt.
Although the de6criptio~ ha~ referred 6pecifically to a light micro~cope embodiment, the principal~ of the method are applicable to 6uch technology a6 electron ~icro~copy, cartogzaphy, and photo i~aging and analy~e~.

~ 3 ; 35 i

Claims (3)

CLAIMS:

1. In a method of making a microscopic analysis that includes the steps of producing, by means of a microscope, an image of a specimen mounted on a position-controlled stage, superimposing the produced image with the image of a display device with a visible window and controlling the position of the stage by means of a graphic input device, the improvement comprising the steps of:
generating marks from a fixed point on the display device as controlled by means of the graphic input device:
defining said marks as x,y, coordinate axes pairs, that correspond to the instantaneous points of intersection of the optical axis with the specimen stage at the time the marks are generated; and moving the marks in unison with the specimen stage so that the marks move in apparent coincidence with the stage movement as viewed in the superimposed image.

2. The method as defined in claim 1 wherein the definition of said marks as x,y, coordinates provides a data base for a digital computer, said computer being programmed to manage the data base to preserve the coincidence of the marks with the viewed specimen image.

3. The method of claim 1. further including a second display device and an image sensor connected to the microscope; the additional steps of displaying on the second display device a digitized static video image of the specimen covering a substantial portion thereof, generating a visible marker on the second display device, and moving the visible marker such that the size and location of the visible marker indicates the instantaneous size and location of the microscopic field of view.