CA2089787C - Method and apparatus for user control by deriving next states of a process from a current state and by providing a visual presentation of the derived next states - Google Patents
Method and apparatus for user control by deriving next states of a process from a current state and by providing a visual presentation of the derived next statesInfo
- Publication number
- CA2089787C CA2089787C CA002089787A CA2089787A CA2089787C CA 2089787 C CA2089787 C CA 2089787C CA 002089787 A CA002089787 A CA 002089787A CA 2089787 A CA2089787 A CA 2089787A CA 2089787 C CA2089787 C CA 2089787C
- Authority
- CA
- Canada
- Prior art keywords
- derived
- current state
- representation
- states
- set forth
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Classifications
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B13/00—Burglar, theft or intruder alarms
- G08B13/18—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
- G08B13/189—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
- G08B13/194—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
- G08B13/196—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
- G08B13/19678—User interface
- G08B13/19689—Remote control of cameras, e.g. remote orientation or image zooming control for a PTZ camera
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B13/00—Burglar, theft or intruder alarms
- G08B13/18—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
- G08B13/189—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
- G08B13/194—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
- G08B13/196—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
- G08B13/19678—User interface
- G08B13/19682—Graphic User Interface [GUI] presenting system data to the user, e.g. information on a screen helping a user interacting with an alarm system
Abstract
A user interface includes a process model unit (34) for predicting one or more allowable next states, from a current state of a process, and a display processing unit (26) for deriving, for each of the allowable next states, a representation of the allowable next state. The display processing unit has an output coupled to a display screen (30) for displaying each of the representations (30b-30g) in conjunction with a representation (30a) of a current state of the process. The user interface further includes an actuator control unit (22) that is coupled to an input mechanism whereby a user selects one of the displayed representations of one of the allowable next states. The motor control unit controls the process to cause it to enter a new current state that corresponds to the selected derived representation. In one embodiment, the display screen has a touchscreen capability whereby the user selects one of the representations by physically touching the display screen within an area associated with a selected one of the derived allowable states.
Description
2 ~ '7 ~ 7 Y09-92-05g ]
~1~0~ AND APPARA'J'US FOR USER CO~OL ~Y D~IVI~G ~,XT
STATES OF A PROCESS EROM A CURR~' STAl~ ~ND BY P~OVIDI~JG A
VISUAL ~ ~h:L'ATIO~ OF T.~ D~RIVE~
N13%'r S~T
FIELD OF TTE l~VL~ OM:
This invention relates generally to process con-trol systems and, in particular, to process control systems that provide a user interface for displayiny information related to a controlled process and that receive input from a user of the system.
RA~7ROUND OF T~E INVENTION:
A process control system may include a digital data processor that includes actuators for controlling a process, and a graphical display device for displayin~
process-related information to an operator. The system may also include one or more operator inpu-t devices for receiving input from the operator of the system. The display may graphically illustrate a current state of the process being controlled, and may also provide a list of one or more state variables for control by -the operator. Examples include, but are not limited to, an environmental control system for a large building, a control system for a pharmaceutical manufacturing process, a petrochemical refining process, a control system for an automated manufacturing operation, and a control sy~tem for imaging or photographic devices.
In general, in known types of systems the operator is restricted to attempting to reacll an alternate state by trial and error manipu]ation of ,some system variables. By example, in an environmental control system the operator may be enabled to vary the hot air flow to a selected zone of a building. However, the use of trial and error methods may not always provide an op-timum environmental solution.
YOg-92-0s9 ~20897~ ~
Furthermore, the use of preproyrammec~, or ~cces~ed, next system states may no-t always provide an optimum ~olution to a particular proce~~ control problem.
For either the trial and error approach, or the preprogrammed approach, the operako~ may not be sufficiently aware of a consequence of a particular next state upon the controlled system so as to make an informed decision as to an appropriate next state, based on a current state of the system.
Known systems typically provide graphical representations o~
objects, and provide means for implementing "con-trols" on these objects to obtain a desired behavior. As an example, in U.S. Patent No. 4,649,499, is~ued March 10, 1987, entitled "Touchscreen Two-Dimensional Emulation of Three-Dimensional Objects", J. Su-tton et al. described the use of a touchscreen to emulate three-dimensional object~ on a two-dimensional computer screen. The ~ystem is employed to emulate the operation of a desktop rotary card file and a hand held calculator. In the calculator embodiment an image of a calculator is displayed and an operator touches the displayed "keys" to operate the calculator. What is not disclosed is the generation and display of derived (potential) next states of a process, in conjunction with a current state of the process, or the ability to enable a user to select one of a plurality of derived next states to become a next current state.
It is thus an object of -this inverltion to provide a process control system user interface that derive~ one or more next states of a system based upon a current state of the system and upon a computational model of the system, and which ~urthermore prov:ides a visual display o~ the one or more derived next ~tates in conjunctioll with a visual display of the current state.
It is a further object of this invention to provide a process control system user interface that derives one or more next states of a system based upon a current state of -- 2~78'~
the system and upon a computatiorlal model of the ~ystern, which provides a visual display of the one or more derived next states ~nd a v:isllal display of the current state, and which drives the system to a neY.1 state :Ln re.~pon~e to the operator selectirl<J one c~f the onf? or more dl~played d~rived next states.
SUMM~RY OF ~H~ INV~NTION
The foregoing and other problems are overcome and the objects of the invention are realized by a method for controlling a process, and by a user inter~ace that is operated in accordance wi-th the invention The method includes the steps of (a~ presenting a representation of a current state of a process; (b) deriving, from the current state of the process, one or more possible alternative states; (c) presenting a representation of the one or more derived possible alternative states; and, in response to a user selecting one of -the representations of the one or more derived alternative states, (d) controlling the proces~ to assume a new current state that corresponds to the derived alternative state that i.s a.ssocia-ted with khe selected representation.
In response to the user selecting the representation of the current state of the process, the method includes a step of accepting the current state.
An aspect of the invention relates to the perception of a controlled process from the point of view of the user. In accordance with the inverl-tiorl, the user i~ presented wi-th an array of possibilities, each o~ which represents either the current state of the process or one or more possible next states that are derived from the current state. Each possibility may be graphically or otherwise presented to the user. The user selects from amongst the presented possibilities in order to cause the representation of the current state to more closely approximate a desired next state or a desired end state. The system and method of the invention operates to support the user by selecting actions '' 2~$~7~3~
required to brin~ about the conver-~ence of the cur:rent state and the desirec~ next or en-.l state The system and method also operates -to refresh the at.~r~y of po3~ibll:ities, hased on previous selections by the user and/or as a function of time. As a result, the -total cogrli.ti~e burden of the user is reduced to choosing amongst di.sp]ayed alternatives until a satisfactory condition is achieved Preferably, the representations of the current state and the representations of the al-ternative states that are derived from the current state are arrayed on the display screen in a manner that supports the user's physical intuition concerning the process ln -the context of th.is invention, "derived" refers -to computer~generated predicted transformations. In an exemplary embodiment, the process is the aiming of a camera to obtain a self portrait of the user. For this case, derived camera tilt images are disposed along a y-axis of the screen, derived camera pan imayes are disposed along an x-axis of the screen, and derived camera zoom images are disposed along an inferred z-axis, with a physically larger derived zoom-in i.mage being positioned to appear "nearer" to the operator A user interface that is taught by the invention includes a process model unit, for predicti.ng one or more allowable next states from a current state of a process, and a display processing unit for deriving, for each of the allowable next states, a representation of the allowable next state. The display processing uni.t has an output coupled to a display screen ~or displ.ayiny each o~ -the represerlta-t:Lons in conjunction with a single represerltatioll of the current state of the proces,s The user interface further includes a process controll.er that is coup]ed to an inpu-t mechanism whereby a user ,selects one o~ the representations of one of the allowable next s-tates. The proces3 controller controls mechanisms embodying the controlled process in a manner that causes the process to enter a new current state that corresponds to the selected derived representation.
~ 20~7~
YO9-92-059 rj By example, the displ.ay screen l~s touch~creen capability whereby the user se:Lectf.; one of the repre~entations by physicall.y touchiny tl~e display screen withlrl an area associated with a selected orJe of the derived allowable states or the current s-tate.
In the example above, selection of the current state has the effect of storing the portra-te imaye. In general, selection of the current state has effects which are characteristic of the particular process being controlled These effects could, for example, include rendering the representation o~
the user in greater detail, or changing the representation from black and whi.te to color.
BRIE~ DESCRIPTION OF THE DRAWTNG
The above set forth and other features of the invention are made more apparent in the ensuiny Detailed Description of the Invention when read in conjunction with the attached Drawing, wherein:
Fig. 1 is a block diagram of an exemplary process control system that is constructed and operated in accordance with the invention;
Figs. 2a and 2b illustra-te an operator display screen, for the e~emplary process control system of Fig. 1, having a graphlcal display o~ a current system state and also a plurality of derivecl possible next states, wherein Fig. 2a shows the current state and a plurality of derived next states at a fi.rst i.nstant in time, and wherein ~ig. 2b shows the result of the selection of one of the deri.ved next states of Fig. 2a;
Fig. 3 i5 block diagram tha-t S}IOwS in greater detail the display processing unit of Fig. l;
Fig. 4 is a block diagram which show.s in greater detail the display model unit of F.ig. l;
YO9-92-059 6 2 ~ 8 ~ 7 ~ 7 Fig. 5 is a block diagram which shows in greater detail the motor control unit of Fig. I; and Fig. 6 is a flow char-t khat illustrates a method of the invention.
DETAILED DE~CRIPTION OF THF INVENTION
Fig. 1 is a block diagram of an exemplary process control system lO that is constructed and operated in accordance with the invention. In Fig. 1 the process controls the pointing direction and the size of the field of view of a camera 12. However, and as will become apparent below, the illustrated embodiment is not intended to be read in a limiting sense upon the practice of the invention.
Camera 12 is gimbal-mounted and has an associated tilt control motor 14 for rotating the camera 12 around an x-axis, a pan control motor 16 for rotating the camera 12 around a y-axis, and a lens 12a zoom control motor 18 for providing an apparent translation along a z-axis. The zoom control motor 18 enables camera 12 zoom-in and zoom-out operations. Motors 14, 16, and 1~ receive electrical command signals from a tilt, pan, and zoom controller 20, which in turn receives higher level motor commands from a motor control unit 22. By example, a given motor command generated by the motor control unit 22 may be one to change the pan angle by +3~. In response, the controller 20 interprets the motor command and generates an appropriate pan motor 16 actuation signal so as -to pan the camera 12 by the specified amount.
The operation of the tilt, pan, zoom controller 20, and the attached motors 14, 16, and 1~, is well known, and the operation of same will not be described in further detail.
Camera 12 provides a video output, or video signal 12b, to a frame capture buffer 24, also referred to in the art as a frame grabber. E'rame capture buffer 24 includes a memory for storing digital information representing at leas-t one 20~97~7 full image frame that is generatecl by the camera 12. If the video signal 12b is provided in analog form, the the buffer 24 also includes an analog -to digital converter for converting, prior to stora~e, -the analog video signal to a digital signal. The frame capt~lre buffer 24 provides the stored digital information as a current image 24a to a display processing unit 26.
In accordance with an aspect of the invention the display processing unit 26 operates generally to derive visual representations of one or more possible next process states from a current state of the process. For the illustrated embodiment, the display processing unit 26 derives a plurality of next camera image states from the current image 24a, and provides at an output the current imaye and the one or more derived camera images. This information is stored in a conventional display buffer 28, from where it is provided to a display screen 30 having graphical capability.
For the illustrated embodiment, and as is also shown in Fig.
2, the display screen 30 displays the current camera image state in a region 30a, a derived pan right state in a region 30b, a derived pan ]eft state in a region 30c, a derived ~.oom-out state in a region 30d, a derived zoom~in state in a region 30e, a derived tilt down state in a region 30f, and a derived tilt up state in a region 30g. Displayed states 30b-30g are all der;.ved from the current image s-tate 30a, and display to the operator a res~llt of the execution of the associated action. That is, each derived state 30b-30g displays what the current image state 30a will become if the associated derived state is selected by the operator for execution.
In the context of this invention, "derived" refers to computer-generated predicted transformations.
In this embodiment of the invention the display screen 30 has touchscreen capabilities. That is, each of the areas 30a-30g is defined and operated to have switching capabilities so that, in response to the operator physically ..
Y09-92-059 8 2~8~
touching one of the areas 30a-30g, touchscreen signals 31 are generated. The touchscreen signals 31 include x-y coordinate information that specify a location on the display screen 30 that was touched by the operator. The touchscreen signals 31 are provided to a touchscreen decoding unit 32. In the touchscreen decoding unit 32 the screen coordinates associated with each area 30a-30g are predetermined to encompass one of a plurality of specific touchscreen areas so that, in response to the touch screen signals 31, the touch screen control unit is enabled to identify which of -the areas 30a-30g was touched by the operator. That is, the touch screen decoding unit 32 converts the touch screen signals 31 into a specific switch closure signal. For the illustrated embodiment the display screen 30 has seven (software generated) independent momentary-type switches associated therewith, corresponding to the areas 30a-30g.
In other unillustrated embodiments of the invention the touch screen capability may be replaced by, for example, a keyboard-driven cursor and/or a pointing device (mouse) cursor, whereby the operator .is enabled to specify to the system 10 one of the areas 30a-30g. The specific implementation o~ the operator input function is thus not germane to the understanding of, or the operation of, the invention. That is, other methods for selection, such as voice recogni-tion, light pen, e-tc., may occur to those sXilled in the art.
Reference is now made to Figs. 2a and 2b for illustrating the display screen 30 for the exemplary process control system of Fig. 1. In this example the subject that is imaged by the camera 12 is the operator of the system 10, as in done in a self-portrait ~ooth (kiosk). As a result, the system 10 may also include a printer or other suitable device for producing a hard copy of the camera image, after the camera 12 is correctly positioned with respect to the subject being imaged. Other means may be provided for storing and/or for transmitting the image for later use.
Y09 92-059 9 ~8~7~7 This self-por-trait appllcation par-ticularly emphasizes the utility of the invention. Conventional self-portrait techniques suffer from problems resulting from large differences in heiyht between subjects and a lack of a suitable feedback mechanism for de-termining where the camera is aimed. These problems have become more apparent in that there is a developing class of computer applications wherein self-portraits or pictures of the participants are an integral part of the application. These applications in~lude, but are not limited to, security systems, multi-media applications, and teleconferencing systems.
In accordance with the invention the area 30a displays the current image of the subject. Areas 30b-30g each display an image that is derived from the current image 30a, and present to the operator an image of a next possible state of the current image.
Preferably, the current and derived images are arrayed on the display screen 30 in a manner that supports the user's physical intuition concerning the process, in this case the movement of the camera. That is, the derived tilt areas 30g and 30f are disposed along a y axis of the screen, the derived pan areas 30b and 30c are disposed along an x-axis of the screen, and the derivecl zoom areas 30d and 30e are disposed along an inferred z-axis, with the physically larger derived zoom-in image area 30e being positioned to appear "nearer" to the operator.
In operation, the operator touches or otherwise selects one of the areas 30b-30g. In response, the camera 12 is repositioned accordingly, and the current image 30a reflects the result of the repositioning operation. Furthermore, each of the derived images displayed in the areas 30b-30g is updated in accordance with the new current lmage, and also at regular intervals of time. For example, the derived pan right image area 30b is updated to reflect the result of a pan right operation, based on the current updated image 30a.
This process continues until the current image 30a meets a criteria applied by the operator. At this point, the . .. .
, Y09-9~-059 10 2~787 operator selects the current image area 30a, and the current image 30a is stored and/or is prin-ted out.
In Fig. 2a the operator selects the derlved pan right image 30b to better center the image presented in the current image area 30a. In response, the camera 12 is repositioned by a current pan right increment, resulting in the capture and display of a new current image in the area 30a in Fig.
2a. The images presented in areas 30b-30g are all derived from the new current image, and present possible alternative next states for the imaging process.
Further in accordance with the invention each of the derived next states is quantisized. That is, each next state is derived so as to be within a predetermined distance from the current state. For the example of Fig. 2, the pan right and pan left derived next states are a predetermined angular displacement about the y-axis from -the current state, and the derived tilt down and the tilt up next states are a predetermined angular displacement abo~St the x-axis.
Similarly, the derived zoom~out and zoom-in next states are a predetermined linear distance along the z-axis.
Referring again to Fig. 1, the information relating to the derived next state quanti.zation factors is maintained by a process model uni.t 34. The process model unit 34 is coupled to the display processing unit 26, as is also seen in Fig.
~1~0~ AND APPARA'J'US FOR USER CO~OL ~Y D~IVI~G ~,XT
STATES OF A PROCESS EROM A CURR~' STAl~ ~ND BY P~OVIDI~JG A
VISUAL ~ ~h:L'ATIO~ OF T.~ D~RIVE~
N13%'r S~T
FIELD OF TTE l~VL~ OM:
This invention relates generally to process con-trol systems and, in particular, to process control systems that provide a user interface for displayiny information related to a controlled process and that receive input from a user of the system.
RA~7ROUND OF T~E INVENTION:
A process control system may include a digital data processor that includes actuators for controlling a process, and a graphical display device for displayin~
process-related information to an operator. The system may also include one or more operator inpu-t devices for receiving input from the operator of the system. The display may graphically illustrate a current state of the process being controlled, and may also provide a list of one or more state variables for control by -the operator. Examples include, but are not limited to, an environmental control system for a large building, a control system for a pharmaceutical manufacturing process, a petrochemical refining process, a control system for an automated manufacturing operation, and a control sy~tem for imaging or photographic devices.
In general, in known types of systems the operator is restricted to attempting to reacll an alternate state by trial and error manipu]ation of ,some system variables. By example, in an environmental control system the operator may be enabled to vary the hot air flow to a selected zone of a building. However, the use of trial and error methods may not always provide an op-timum environmental solution.
YOg-92-0s9 ~20897~ ~
Furthermore, the use of preproyrammec~, or ~cces~ed, next system states may no-t always provide an optimum ~olution to a particular proce~~ control problem.
For either the trial and error approach, or the preprogrammed approach, the operako~ may not be sufficiently aware of a consequence of a particular next state upon the controlled system so as to make an informed decision as to an appropriate next state, based on a current state of the system.
Known systems typically provide graphical representations o~
objects, and provide means for implementing "con-trols" on these objects to obtain a desired behavior. As an example, in U.S. Patent No. 4,649,499, is~ued March 10, 1987, entitled "Touchscreen Two-Dimensional Emulation of Three-Dimensional Objects", J. Su-tton et al. described the use of a touchscreen to emulate three-dimensional object~ on a two-dimensional computer screen. The ~ystem is employed to emulate the operation of a desktop rotary card file and a hand held calculator. In the calculator embodiment an image of a calculator is displayed and an operator touches the displayed "keys" to operate the calculator. What is not disclosed is the generation and display of derived (potential) next states of a process, in conjunction with a current state of the process, or the ability to enable a user to select one of a plurality of derived next states to become a next current state.
It is thus an object of -this inverltion to provide a process control system user interface that derive~ one or more next states of a system based upon a current state of the system and upon a computational model of the system, and which ~urthermore prov:ides a visual display o~ the one or more derived next ~tates in conjunctioll with a visual display of the current state.
It is a further object of this invention to provide a process control system user interface that derives one or more next states of a system based upon a current state of -- 2~78'~
the system and upon a computatiorlal model of the ~ystern, which provides a visual display of the one or more derived next states ~nd a v:isllal display of the current state, and which drives the system to a neY.1 state :Ln re.~pon~e to the operator selectirl<J one c~f the onf? or more dl~played d~rived next states.
SUMM~RY OF ~H~ INV~NTION
The foregoing and other problems are overcome and the objects of the invention are realized by a method for controlling a process, and by a user inter~ace that is operated in accordance wi-th the invention The method includes the steps of (a~ presenting a representation of a current state of a process; (b) deriving, from the current state of the process, one or more possible alternative states; (c) presenting a representation of the one or more derived possible alternative states; and, in response to a user selecting one of -the representations of the one or more derived alternative states, (d) controlling the proces~ to assume a new current state that corresponds to the derived alternative state that i.s a.ssocia-ted with khe selected representation.
In response to the user selecting the representation of the current state of the process, the method includes a step of accepting the current state.
An aspect of the invention relates to the perception of a controlled process from the point of view of the user. In accordance with the inverl-tiorl, the user i~ presented wi-th an array of possibilities, each o~ which represents either the current state of the process or one or more possible next states that are derived from the current state. Each possibility may be graphically or otherwise presented to the user. The user selects from amongst the presented possibilities in order to cause the representation of the current state to more closely approximate a desired next state or a desired end state. The system and method of the invention operates to support the user by selecting actions '' 2~$~7~3~
required to brin~ about the conver-~ence of the cur:rent state and the desirec~ next or en-.l state The system and method also operates -to refresh the at.~r~y of po3~ibll:ities, hased on previous selections by the user and/or as a function of time. As a result, the -total cogrli.ti~e burden of the user is reduced to choosing amongst di.sp]ayed alternatives until a satisfactory condition is achieved Preferably, the representations of the current state and the representations of the al-ternative states that are derived from the current state are arrayed on the display screen in a manner that supports the user's physical intuition concerning the process ln -the context of th.is invention, "derived" refers -to computer~generated predicted transformations. In an exemplary embodiment, the process is the aiming of a camera to obtain a self portrait of the user. For this case, derived camera tilt images are disposed along a y-axis of the screen, derived camera pan imayes are disposed along an x-axis of the screen, and derived camera zoom images are disposed along an inferred z-axis, with a physically larger derived zoom-in i.mage being positioned to appear "nearer" to the operator A user interface that is taught by the invention includes a process model unit, for predicti.ng one or more allowable next states from a current state of a process, and a display processing unit for deriving, for each of the allowable next states, a representation of the allowable next state. The display processing uni.t has an output coupled to a display screen ~or displ.ayiny each o~ -the represerlta-t:Lons in conjunction with a single represerltatioll of the current state of the proces,s The user interface further includes a process controll.er that is coup]ed to an inpu-t mechanism whereby a user ,selects one o~ the representations of one of the allowable next s-tates. The proces3 controller controls mechanisms embodying the controlled process in a manner that causes the process to enter a new current state that corresponds to the selected derived representation.
~ 20~7~
YO9-92-059 rj By example, the displ.ay screen l~s touch~creen capability whereby the user se:Lectf.; one of the repre~entations by physicall.y touchiny tl~e display screen withlrl an area associated with a selected orJe of the derived allowable states or the current s-tate.
In the example above, selection of the current state has the effect of storing the portra-te imaye. In general, selection of the current state has effects which are characteristic of the particular process being controlled These effects could, for example, include rendering the representation o~
the user in greater detail, or changing the representation from black and whi.te to color.
BRIE~ DESCRIPTION OF THE DRAWTNG
The above set forth and other features of the invention are made more apparent in the ensuiny Detailed Description of the Invention when read in conjunction with the attached Drawing, wherein:
Fig. 1 is a block diagram of an exemplary process control system that is constructed and operated in accordance with the invention;
Figs. 2a and 2b illustra-te an operator display screen, for the e~emplary process control system of Fig. 1, having a graphlcal display o~ a current system state and also a plurality of derivecl possible next states, wherein Fig. 2a shows the current state and a plurality of derived next states at a fi.rst i.nstant in time, and wherein ~ig. 2b shows the result of the selection of one of the deri.ved next states of Fig. 2a;
Fig. 3 i5 block diagram tha-t S}IOwS in greater detail the display processing unit of Fig. l;
Fig. 4 is a block diagram which show.s in greater detail the display model unit of F.ig. l;
YO9-92-059 6 2 ~ 8 ~ 7 ~ 7 Fig. 5 is a block diagram which shows in greater detail the motor control unit of Fig. I; and Fig. 6 is a flow char-t khat illustrates a method of the invention.
DETAILED DE~CRIPTION OF THF INVENTION
Fig. 1 is a block diagram of an exemplary process control system lO that is constructed and operated in accordance with the invention. In Fig. 1 the process controls the pointing direction and the size of the field of view of a camera 12. However, and as will become apparent below, the illustrated embodiment is not intended to be read in a limiting sense upon the practice of the invention.
Camera 12 is gimbal-mounted and has an associated tilt control motor 14 for rotating the camera 12 around an x-axis, a pan control motor 16 for rotating the camera 12 around a y-axis, and a lens 12a zoom control motor 18 for providing an apparent translation along a z-axis. The zoom control motor 18 enables camera 12 zoom-in and zoom-out operations. Motors 14, 16, and 1~ receive electrical command signals from a tilt, pan, and zoom controller 20, which in turn receives higher level motor commands from a motor control unit 22. By example, a given motor command generated by the motor control unit 22 may be one to change the pan angle by +3~. In response, the controller 20 interprets the motor command and generates an appropriate pan motor 16 actuation signal so as -to pan the camera 12 by the specified amount.
The operation of the tilt, pan, zoom controller 20, and the attached motors 14, 16, and 1~, is well known, and the operation of same will not be described in further detail.
Camera 12 provides a video output, or video signal 12b, to a frame capture buffer 24, also referred to in the art as a frame grabber. E'rame capture buffer 24 includes a memory for storing digital information representing at leas-t one 20~97~7 full image frame that is generatecl by the camera 12. If the video signal 12b is provided in analog form, the the buffer 24 also includes an analog -to digital converter for converting, prior to stora~e, -the analog video signal to a digital signal. The frame capt~lre buffer 24 provides the stored digital information as a current image 24a to a display processing unit 26.
In accordance with an aspect of the invention the display processing unit 26 operates generally to derive visual representations of one or more possible next process states from a current state of the process. For the illustrated embodiment, the display processing unit 26 derives a plurality of next camera image states from the current image 24a, and provides at an output the current imaye and the one or more derived camera images. This information is stored in a conventional display buffer 28, from where it is provided to a display screen 30 having graphical capability.
For the illustrated embodiment, and as is also shown in Fig.
2, the display screen 30 displays the current camera image state in a region 30a, a derived pan right state in a region 30b, a derived pan ]eft state in a region 30c, a derived ~.oom-out state in a region 30d, a derived zoom~in state in a region 30e, a derived tilt down state in a region 30f, and a derived tilt up state in a region 30g. Displayed states 30b-30g are all der;.ved from the current image s-tate 30a, and display to the operator a res~llt of the execution of the associated action. That is, each derived state 30b-30g displays what the current image state 30a will become if the associated derived state is selected by the operator for execution.
In the context of this invention, "derived" refers to computer-generated predicted transformations.
In this embodiment of the invention the display screen 30 has touchscreen capabilities. That is, each of the areas 30a-30g is defined and operated to have switching capabilities so that, in response to the operator physically ..
Y09-92-059 8 2~8~
touching one of the areas 30a-30g, touchscreen signals 31 are generated. The touchscreen signals 31 include x-y coordinate information that specify a location on the display screen 30 that was touched by the operator. The touchscreen signals 31 are provided to a touchscreen decoding unit 32. In the touchscreen decoding unit 32 the screen coordinates associated with each area 30a-30g are predetermined to encompass one of a plurality of specific touchscreen areas so that, in response to the touch screen signals 31, the touch screen control unit is enabled to identify which of -the areas 30a-30g was touched by the operator. That is, the touch screen decoding unit 32 converts the touch screen signals 31 into a specific switch closure signal. For the illustrated embodiment the display screen 30 has seven (software generated) independent momentary-type switches associated therewith, corresponding to the areas 30a-30g.
In other unillustrated embodiments of the invention the touch screen capability may be replaced by, for example, a keyboard-driven cursor and/or a pointing device (mouse) cursor, whereby the operator .is enabled to specify to the system 10 one of the areas 30a-30g. The specific implementation o~ the operator input function is thus not germane to the understanding of, or the operation of, the invention. That is, other methods for selection, such as voice recogni-tion, light pen, e-tc., may occur to those sXilled in the art.
Reference is now made to Figs. 2a and 2b for illustrating the display screen 30 for the exemplary process control system of Fig. 1. In this example the subject that is imaged by the camera 12 is the operator of the system 10, as in done in a self-portrait ~ooth (kiosk). As a result, the system 10 may also include a printer or other suitable device for producing a hard copy of the camera image, after the camera 12 is correctly positioned with respect to the subject being imaged. Other means may be provided for storing and/or for transmitting the image for later use.
Y09 92-059 9 ~8~7~7 This self-por-trait appllcation par-ticularly emphasizes the utility of the invention. Conventional self-portrait techniques suffer from problems resulting from large differences in heiyht between subjects and a lack of a suitable feedback mechanism for de-termining where the camera is aimed. These problems have become more apparent in that there is a developing class of computer applications wherein self-portraits or pictures of the participants are an integral part of the application. These applications in~lude, but are not limited to, security systems, multi-media applications, and teleconferencing systems.
In accordance with the invention the area 30a displays the current image of the subject. Areas 30b-30g each display an image that is derived from the current image 30a, and present to the operator an image of a next possible state of the current image.
Preferably, the current and derived images are arrayed on the display screen 30 in a manner that supports the user's physical intuition concerning the process, in this case the movement of the camera. That is, the derived tilt areas 30g and 30f are disposed along a y axis of the screen, the derived pan areas 30b and 30c are disposed along an x-axis of the screen, and the derivecl zoom areas 30d and 30e are disposed along an inferred z-axis, with the physically larger derived zoom-in image area 30e being positioned to appear "nearer" to the operator.
In operation, the operator touches or otherwise selects one of the areas 30b-30g. In response, the camera 12 is repositioned accordingly, and the current image 30a reflects the result of the repositioning operation. Furthermore, each of the derived images displayed in the areas 30b-30g is updated in accordance with the new current lmage, and also at regular intervals of time. For example, the derived pan right image area 30b is updated to reflect the result of a pan right operation, based on the current updated image 30a.
This process continues until the current image 30a meets a criteria applied by the operator. At this point, the . .. .
, Y09-9~-059 10 2~787 operator selects the current image area 30a, and the current image 30a is stored and/or is prin-ted out.
In Fig. 2a the operator selects the derlved pan right image 30b to better center the image presented in the current image area 30a. In response, the camera 12 is repositioned by a current pan right increment, resulting in the capture and display of a new current image in the area 30a in Fig.
2a. The images presented in areas 30b-30g are all derived from the new current image, and present possible alternative next states for the imaging process.
Further in accordance with the invention each of the derived next states is quantisized. That is, each next state is derived so as to be within a predetermined distance from the current state. For the example of Fig. 2, the pan right and pan left derived next states are a predetermined angular displacement about the y-axis from -the current state, and the derived tilt down and the tilt up next states are a predetermined angular displacement abo~St the x-axis.
Similarly, the derived zoom~out and zoom-in next states are a predetermined linear distance along the z-axis.
Referring again to Fig. 1, the information relating to the derived next state quanti.zation factors is maintained by a process model uni.t 34. The process model unit 34 is coupled to the display processing unit 26, as is also seen in Fig.
3, for providing quantization factors (current increments) thereto. The process model urlit 3~ is also coupled to the motor control unit 22 for providi.ng the quantization factors thereto.
In general, the process model. unik 34 is responsible for predicting the allowed next states, for providing the information required by the display processing unit 26 to derive the appearance of those states, and for providing the information required by the motor control unit 22 to position the camera to achieve a state having the appearance of a selected one of the derived states.
Y09-92-059 1l ~ O $ 9 7 ~ ~
As seen in Fig. 3, the display processing unit 26 receives the current image 24a. The current lmage 24a is applied to a scaling block 40 wherein the current image 2~a is reduced to a size compatible with the current image area 30a (Fig. 2).
The output of the image scaling block 40 is the scaled current image 40a. The scaled current image 40a is applied to the display buffer 28 for display within the current image area 30a, and is also applied to an image pan transformation block 42, an image tilt -transformation block 44, and an image zoom transformation block 46. Blocks 42, 44, and 46 each apply a predetermined image transformation function to the scaled current image 40a, in accordance with an associated current pan increment, a current tilt increment, and a current zoom increment, respectively. The current increments are received from the process model unit 34. The operation of blocks 42, 44, and 4~ generates the derived next states of the scaled current image 40a for display in the areas 30b-30g of the display screen 30.
By example, image pan transformation block 42 horizontally shifts the scaled current image ~Oa by a number of pixels that correspond to an angular displacement that is specified by the current pan increment. The image tilt transformation block 44 vertically shifts the scaled current image 40a by a number of pixels that correspond to an angular displacement that is specified by the current tilt increment. The image zoom transformation block 46 applies both a pixel expansion and contraction operation to the scaled current image 40a to generate the derived zoom-in state and the derived zoom-out state, respectively. The amoullt of pixel expansion and contraction is specified by the current ~oom increment signal.
Fig. 4 is a block diagram that illustrates the process model unit 34. The process model unit 34 lncludes a controller 50 that receives swi-tch closure lnformation 32a from the touchscreen decoding unit 32. The switch closure information 32a is representative of the operator touchin~
the display screen 30 :in one of the areas 30a-30g.
Controller 50 includes a plurality of counters 50a for Y09-92-059 12 2 0 ~ 7 counting a number of switch closure., for each of the regions 30b-30g. The use of the counters 50a enables different increments to be employed as a function of a number o~
switch closures for a particular touchscreen switch. For example, for the first three pan 1eft and pan right switch closures a first pan incremen-t resolution may be employed (5~), and for succeeding pan left and pan right switch closures a second, finer increment resolution may be employed (2~). A further predetermined number of depressions of an individual image state touchscreen switch may be employed to reset the increment to the larger value, as repeated depressions may indicate that the user is attempting to make a larye change in image aspect or position.
A depression of the switch associated with the current image area 30a may be employed to reset the counters 50a associated with the pan, tilt, and zoom touchscreen switches, and to otherwise reinitialize the system 10.
Bidirectionally coupled to controller 50 is a current process model 52 that is maintainecl by the controller 50 in response to switch closure information. The current process model 52 represen-ts the current angular orientation of the camera 12 in the pan and tilt axes, and also represents the current zoom setting. The use of the current process model 52, in conjunction wi-th an allowecl states block 56, enables the controller 50 to control the display processing unit 26 to derive and display only allowable next states for the process. For example, if it is known that an obs-truction would prevent the camera 12 from moving in the tilt axis past a predetermined point, the controller 50 sets the tilt increment to zero when the current process model 52 indicates that the camera has reached the predetermined point. The result is that the display processing unit 26 does not generate a derived next sta-te (in tilt up for example) that would enable the operator to drive the camera into the obstruction.
.
Y09-92-059 13 ~ 7 ~ 7 Also coupled to the controller 50 is a list of increments for pan, tilt, ancl zoom. If a constant increment is employed then each of the predetermined increments 54 is a constant, and -the counters 50a may be dispensed with.
However, if it is desired -to appJ.y different increments then the increments block 54 is accessed by the controlle~ 50 to obtain a current increment, as a function of the count of the associated counter 50a and the current process model 52.
Fig. 5 shows in greater detail the motor control unit 22.
Motor control unit 22 includes a controller 60 that receives the switch closure information 32a and also current increments 34b from the process model unit 34. Controller translates a particular switch closure, such as pan right, in accordance with a current pan right increment, and generates a corresponding pan motor command to the tilt, pan, zoom controller 20 of Fig. 1. The switch closure for the current image state area 30a may be employed by the controller 60 to reset the camera 12 to a predetermined initial orientation.
Fig. 6 is a flow chart that illustrates the operation of the method of the invention for the embodiment of Figs. 1-5. At Block A the camera 12 position and process model 52 are initialized. The display screen 30 is thus also initialized with a current image state 30a and a corresponding plurality of derived image states 30b-30g. A-t Block B a determination is made if the operator has caused a swltch closure. If N0, a determination is made at Blocl~ C i~ a timer 36 (Fig. 1) has generated a timing pulse. Timer 36 is provided to cause the frame capture buffer 24 to capture a new image at regular intervals so as to update -the representations of the current image, and the derived images. This enables the current image area 30a and the derived image areas 30b-30g to accurately reflect changes in position and ~acial expression of the operator. If N0 at Block C, the method continues at Block B.
If YES at Bloc}{ B, a determination is made at Block D if the operator has made a switch closure for a desired one of the Y09~92-059 l~ 9 ~ ~ 7 derived states 30b-30g, or for the current image state 30a.
If the switch closure is for the current state 30a, the YES
branch is taken and the method continues at Block E where the current image state 30a is accepted. That is, the current image may be stored, and/or converted to a tangible form, and/or transmitted to a remote location. The method then returns to Block A.
In general, the action of accepting the current state initiates an accept-specific sequence o~ operations. sy example, to achieve a rapid derivation of next states, and the display of same, the captured and the derived video may be manipulated and displayed as a black and white image.
Upon the occurrence of an accept action by the operator, the image appearing in the current state area 30a, and optionally the next state areas 30b-30g, may be converted to color. Upon a further accept action by the operator, the accepted current state color image may be printed, stored, transmitted, or otherwise acted upon. As such, it should be realized that the action of accepting the current state need not terminate the process being performed.
If N0 at Block D the SWi tch closure .iS acted upon by the process model unit 34 which generates an appropriate set of pan, tilt, and zoom increments 34a and 34b (Block F). In response to the updated incre~ents 34b the motor control unit 22 generates an appropriate motor command 22a for driving the camera 12 to a position that corresponds to the selected derived next state (Block G), and process model 52 is updated to reflect the new camera 12 orientation (Block H).
The action of Block G causes a change in the current image 24a which is processed by the display processing unit 26 to update the display current image 30a and, in conjunction with the updated increments 3~a, to update the derived next states 30b-30g (Block I). The method then continues at Block B to process the next switch closure from the operator.
Y09-92-059 15 2 ~ 7 If at Block C a timeout condltion is detected, the frame capture buffer 24 is automatically updated to capture and store a new frame of video informa~ion. This causes a change in the current image 24a which is processed by the display processing unit 26 to update the display current image 30a and, in conjunc-tion with a current set of increments 34a, to update the derived next states 30b-30g ~Block I). The method then continues at Block B to process -the next switch closure from the operator.
Although dascribed in the context of a video camera control system, it should be realized that the teaching of the invention is also applicable to, by example, controlling the movement of an antenna or a robotic manipulator. The teaching of the invention is also applicable to, by example, a game playing system s~lch as chess. For example, block 30a displays an image that shows a current state of the game pieces, while each of the derived next states displays the game board after one of a number of possible allowable next moves, based on the current positions of the user's game pieces. By selecting one of the derived next states, the system moves the user's game piece accordingly and, in response to the opponent's move, generates a plurality of updated derived next states. The updated derived next states are displayed to the user, in conjunction with the updated current state, so that the user is enabled to select a next move. ~electing the current state may indicate that the user wishes to concede the game.
For this latter embodiment, the model unit 34 includes à
chess rules data base for updating the current process model 52, and also may include a filter for selecting (n) best derived next states from the potentially large number of possible derived next states. The filter criteria may be a score associated with each derived next state (move). ~lso for this latter embodiment, the predetermined increments 54 may be set to one for deriving only next moves, or may be set to two or more for deriving next moves to a depth of two or more. The increments are provided to a processing unit that corresponds to the display processing unit 26 where the Y09-92 05~ ]6 2~787 derived next move images of the yame board are derived. For this embodiment, the current process model 52 represents the current state of the game pieces ancl is thus also supplied to the processing unit ~or display.
As in the case illustrated in Figs. 2a and 2b, the display screen 30 may be organized so as to support the user's intuition of the process, such as by displaying, for each derived next state, the reyion of the chess board from which the game piece would be moved.
It should thus be clear that the method of the invention may be applied with benefit to any process control application wherein there are one or more possible next states of a process that arise from a current state of the process.
An important aspect of the invention relates to the perception of the controlled process from the point of view of the user. In accordance with the teaching of the invention, the user is presented with an array of possibilities, each of which represents either the current state of the process or one or more possible next states that are derived from the c~rrent state. Each possibility may be graphically represented to the user, although other sensory presentations are also within the scope of the invention. For e~ample, each of a plurality of next states may be presented in an auditory or a tactile manner to the user. Mixed sensory presentations are also within the scope of the invention. In all cases, however, the user selects from amongst the presented possibilities in order to make the representation of the current state more closely approximate a desired end state. The system and method of the invention operates to support the user by selecting actions required to bring about the convergence of the current state and the desired end state. The system and method also operates to refresh the array of possibilities, based on previous selections by the user and/or as a function of time. As a result, the total cognitive burden of the user is reduced to choosing amongst presented courses of Y09-92-059 17 2~7~7 action, and to observing the effect o~ a chosen course of action upon the process being controlled.
In systems in which the act:ions of controls are unfamiliar, complex, or have combinatorial spaces which are very large, the method of the invention prevents accidents and errors of oversight. In any system it permits the user to remain focussed on goals, as opposed to the mechanics of an underlying process for achieving those goals.
It should be realized that the teaching of the invention is amenable to expression in a wide variety of embodiments, other than those expressly set forth above. By example, the display of the current state ancl the derived next states need not be static. That is, a railroad traffic control system may show for next states a train travelling :Easter or slower. In an air traffic control system, the system may show a current traffic pattern and, for derived next states, a number of different traffic patterns for a plurality of aircraft flying at different airspeeds and altitudes.
Thus, while the invention has been particularly shown and described with respect to a presently preferred embodiment thereof, it will be understood by those skilled in the art that changes in form and details may be made therein without departing from the scope and spirit of the invention.
In general, the process model. unik 34 is responsible for predicting the allowed next states, for providing the information required by the display processing unit 26 to derive the appearance of those states, and for providing the information required by the motor control unit 22 to position the camera to achieve a state having the appearance of a selected one of the derived states.
Y09-92-059 1l ~ O $ 9 7 ~ ~
As seen in Fig. 3, the display processing unit 26 receives the current image 24a. The current lmage 24a is applied to a scaling block 40 wherein the current image 2~a is reduced to a size compatible with the current image area 30a (Fig. 2).
The output of the image scaling block 40 is the scaled current image 40a. The scaled current image 40a is applied to the display buffer 28 for display within the current image area 30a, and is also applied to an image pan transformation block 42, an image tilt -transformation block 44, and an image zoom transformation block 46. Blocks 42, 44, and 46 each apply a predetermined image transformation function to the scaled current image 40a, in accordance with an associated current pan increment, a current tilt increment, and a current zoom increment, respectively. The current increments are received from the process model unit 34. The operation of blocks 42, 44, and 4~ generates the derived next states of the scaled current image 40a for display in the areas 30b-30g of the display screen 30.
By example, image pan transformation block 42 horizontally shifts the scaled current image ~Oa by a number of pixels that correspond to an angular displacement that is specified by the current pan increment. The image tilt transformation block 44 vertically shifts the scaled current image 40a by a number of pixels that correspond to an angular displacement that is specified by the current tilt increment. The image zoom transformation block 46 applies both a pixel expansion and contraction operation to the scaled current image 40a to generate the derived zoom-in state and the derived zoom-out state, respectively. The amoullt of pixel expansion and contraction is specified by the current ~oom increment signal.
Fig. 4 is a block diagram that illustrates the process model unit 34. The process model unit 34 lncludes a controller 50 that receives swi-tch closure lnformation 32a from the touchscreen decoding unit 32. The switch closure information 32a is representative of the operator touchin~
the display screen 30 :in one of the areas 30a-30g.
Controller 50 includes a plurality of counters 50a for Y09-92-059 12 2 0 ~ 7 counting a number of switch closure., for each of the regions 30b-30g. The use of the counters 50a enables different increments to be employed as a function of a number o~
switch closures for a particular touchscreen switch. For example, for the first three pan 1eft and pan right switch closures a first pan incremen-t resolution may be employed (5~), and for succeeding pan left and pan right switch closures a second, finer increment resolution may be employed (2~). A further predetermined number of depressions of an individual image state touchscreen switch may be employed to reset the increment to the larger value, as repeated depressions may indicate that the user is attempting to make a larye change in image aspect or position.
A depression of the switch associated with the current image area 30a may be employed to reset the counters 50a associated with the pan, tilt, and zoom touchscreen switches, and to otherwise reinitialize the system 10.
Bidirectionally coupled to controller 50 is a current process model 52 that is maintainecl by the controller 50 in response to switch closure information. The current process model 52 represen-ts the current angular orientation of the camera 12 in the pan and tilt axes, and also represents the current zoom setting. The use of the current process model 52, in conjunction wi-th an allowecl states block 56, enables the controller 50 to control the display processing unit 26 to derive and display only allowable next states for the process. For example, if it is known that an obs-truction would prevent the camera 12 from moving in the tilt axis past a predetermined point, the controller 50 sets the tilt increment to zero when the current process model 52 indicates that the camera has reached the predetermined point. The result is that the display processing unit 26 does not generate a derived next sta-te (in tilt up for example) that would enable the operator to drive the camera into the obstruction.
.
Y09-92-059 13 ~ 7 ~ 7 Also coupled to the controller 50 is a list of increments for pan, tilt, ancl zoom. If a constant increment is employed then each of the predetermined increments 54 is a constant, and -the counters 50a may be dispensed with.
However, if it is desired -to appJ.y different increments then the increments block 54 is accessed by the controlle~ 50 to obtain a current increment, as a function of the count of the associated counter 50a and the current process model 52.
Fig. 5 shows in greater detail the motor control unit 22.
Motor control unit 22 includes a controller 60 that receives the switch closure information 32a and also current increments 34b from the process model unit 34. Controller translates a particular switch closure, such as pan right, in accordance with a current pan right increment, and generates a corresponding pan motor command to the tilt, pan, zoom controller 20 of Fig. 1. The switch closure for the current image state area 30a may be employed by the controller 60 to reset the camera 12 to a predetermined initial orientation.
Fig. 6 is a flow chart that illustrates the operation of the method of the invention for the embodiment of Figs. 1-5. At Block A the camera 12 position and process model 52 are initialized. The display screen 30 is thus also initialized with a current image state 30a and a corresponding plurality of derived image states 30b-30g. A-t Block B a determination is made if the operator has caused a swltch closure. If N0, a determination is made at Blocl~ C i~ a timer 36 (Fig. 1) has generated a timing pulse. Timer 36 is provided to cause the frame capture buffer 24 to capture a new image at regular intervals so as to update -the representations of the current image, and the derived images. This enables the current image area 30a and the derived image areas 30b-30g to accurately reflect changes in position and ~acial expression of the operator. If N0 at Block C, the method continues at Block B.
If YES at Bloc}{ B, a determination is made at Block D if the operator has made a switch closure for a desired one of the Y09~92-059 l~ 9 ~ ~ 7 derived states 30b-30g, or for the current image state 30a.
If the switch closure is for the current state 30a, the YES
branch is taken and the method continues at Block E where the current image state 30a is accepted. That is, the current image may be stored, and/or converted to a tangible form, and/or transmitted to a remote location. The method then returns to Block A.
In general, the action of accepting the current state initiates an accept-specific sequence o~ operations. sy example, to achieve a rapid derivation of next states, and the display of same, the captured and the derived video may be manipulated and displayed as a black and white image.
Upon the occurrence of an accept action by the operator, the image appearing in the current state area 30a, and optionally the next state areas 30b-30g, may be converted to color. Upon a further accept action by the operator, the accepted current state color image may be printed, stored, transmitted, or otherwise acted upon. As such, it should be realized that the action of accepting the current state need not terminate the process being performed.
If N0 at Block D the SWi tch closure .iS acted upon by the process model unit 34 which generates an appropriate set of pan, tilt, and zoom increments 34a and 34b (Block F). In response to the updated incre~ents 34b the motor control unit 22 generates an appropriate motor command 22a for driving the camera 12 to a position that corresponds to the selected derived next state (Block G), and process model 52 is updated to reflect the new camera 12 orientation (Block H).
The action of Block G causes a change in the current image 24a which is processed by the display processing unit 26 to update the display current image 30a and, in conjunction with the updated increments 3~a, to update the derived next states 30b-30g (Block I). The method then continues at Block B to process the next switch closure from the operator.
Y09-92-059 15 2 ~ 7 If at Block C a timeout condltion is detected, the frame capture buffer 24 is automatically updated to capture and store a new frame of video informa~ion. This causes a change in the current image 24a which is processed by the display processing unit 26 to update the display current image 30a and, in conjunc-tion with a current set of increments 34a, to update the derived next states 30b-30g ~Block I). The method then continues at Block B to process -the next switch closure from the operator.
Although dascribed in the context of a video camera control system, it should be realized that the teaching of the invention is also applicable to, by example, controlling the movement of an antenna or a robotic manipulator. The teaching of the invention is also applicable to, by example, a game playing system s~lch as chess. For example, block 30a displays an image that shows a current state of the game pieces, while each of the derived next states displays the game board after one of a number of possible allowable next moves, based on the current positions of the user's game pieces. By selecting one of the derived next states, the system moves the user's game piece accordingly and, in response to the opponent's move, generates a plurality of updated derived next states. The updated derived next states are displayed to the user, in conjunction with the updated current state, so that the user is enabled to select a next move. ~electing the current state may indicate that the user wishes to concede the game.
For this latter embodiment, the model unit 34 includes à
chess rules data base for updating the current process model 52, and also may include a filter for selecting (n) best derived next states from the potentially large number of possible derived next states. The filter criteria may be a score associated with each derived next state (move). ~lso for this latter embodiment, the predetermined increments 54 may be set to one for deriving only next moves, or may be set to two or more for deriving next moves to a depth of two or more. The increments are provided to a processing unit that corresponds to the display processing unit 26 where the Y09-92 05~ ]6 2~787 derived next move images of the yame board are derived. For this embodiment, the current process model 52 represents the current state of the game pieces ancl is thus also supplied to the processing unit ~or display.
As in the case illustrated in Figs. 2a and 2b, the display screen 30 may be organized so as to support the user's intuition of the process, such as by displaying, for each derived next state, the reyion of the chess board from which the game piece would be moved.
It should thus be clear that the method of the invention may be applied with benefit to any process control application wherein there are one or more possible next states of a process that arise from a current state of the process.
An important aspect of the invention relates to the perception of the controlled process from the point of view of the user. In accordance with the teaching of the invention, the user is presented with an array of possibilities, each of which represents either the current state of the process or one or more possible next states that are derived from the c~rrent state. Each possibility may be graphically represented to the user, although other sensory presentations are also within the scope of the invention. For e~ample, each of a plurality of next states may be presented in an auditory or a tactile manner to the user. Mixed sensory presentations are also within the scope of the invention. In all cases, however, the user selects from amongst the presented possibilities in order to make the representation of the current state more closely approximate a desired end state. The system and method of the invention operates to support the user by selecting actions required to bring about the convergence of the current state and the desired end state. The system and method also operates to refresh the array of possibilities, based on previous selections by the user and/or as a function of time. As a result, the total cognitive burden of the user is reduced to choosing amongst presented courses of Y09-92-059 17 2~7~7 action, and to observing the effect o~ a chosen course of action upon the process being controlled.
In systems in which the act:ions of controls are unfamiliar, complex, or have combinatorial spaces which are very large, the method of the invention prevents accidents and errors of oversight. In any system it permits the user to remain focussed on goals, as opposed to the mechanics of an underlying process for achieving those goals.
It should be realized that the teaching of the invention is amenable to expression in a wide variety of embodiments, other than those expressly set forth above. By example, the display of the current state ancl the derived next states need not be static. That is, a railroad traffic control system may show for next states a train travelling :Easter or slower. In an air traffic control system, the system may show a current traffic pattern and, for derived next states, a number of different traffic patterns for a plurality of aircraft flying at different airspeeds and altitudes.
Thus, while the invention has been particularly shown and described with respect to a presently preferred embodiment thereof, it will be understood by those skilled in the art that changes in form and details may be made therein without departing from the scope and spirit of the invention.
Claims (37)
1. A method for controlling a process, comprising the steps of:
(a) presenting a representation of a current state of a process;
(b) deriving in accordance with a predetermined transformation, from the current state of the process, one or more alternative states each representing a possible future state of the process that may be entered directly or indirectly from the current state, wherein each of the one or more derived alternative states of the process is not predetermined but is instead made manifest only as a result of executing the step of deriving;
(c) presenting a representations of the one or more derived alternative states; and in response to a selection of one of the representations of the one or more derived alternative states, (d) controlling the process to assume a new current state that corresponds to the derived alternative state that is associated with the selected representation.
(a) presenting a representation of a current state of a process;
(b) deriving in accordance with a predetermined transformation, from the current state of the process, one or more alternative states each representing a possible future state of the process that may be entered directly or indirectly from the current state, wherein each of the one or more derived alternative states of the process is not predetermined but is instead made manifest only as a result of executing the step of deriving;
(c) presenting a representations of the one or more derived alternative states; and in response to a selection of one of the representations of the one or more derived alternative states, (d) controlling the process to assume a new current state that corresponds to the derived alternative state that is associated with the selected representation.
2. A method as set forth in claim 1 wherein the representation of the current state of the process is also selectable and, in response to the selection of the representation of the current state of the process, further including a step of initiating a current state accept-specific sequence of actions.
3. A method as set forth in claim 1 wherein the step of controlling includes a step of energizing an actuator means to drive the process into the new current state.
4. A method as set forth in claim 3 wherein the step of energizing the actuator means energizes the actuator means to drive the process by a predetermined increment.
5. A method as set forth in claim 1 wherein the step of controlling includes a step of updating a model of the process to reflect an assumption of the new current state.
6. A method as set forth in claim 1 wherein the step of deriving derives the one or more alternative states in accordance with a predetermined increment.
7. A method as set forth in claim 5 wherein the step of controlling includes a step of updating the increment for use in deriving one or more alternative states from the new current state.
8. A method as set forth in claim 1 and further including the steps of:
periodically updating the current state of the process;
presenting a representation of the updated current state of the process;
deriving in accordance with the predetermined transformation, from the updated current state of the process, one or more alternative states each representing a possible future state of the process that may be entered directly or indirectly from the updated current state; and presenting a representation of the one or more alternative states that are derived from the updated current state.
periodically updating the current state of the process;
presenting a representation of the updated current state of the process;
deriving in accordance with the predetermined transformation, from the updated current state of the process, one or more alternative states each representing a possible future state of the process that may be entered directly or indirectly from the updated current state; and presenting a representation of the one or more alternative states that are derived from the updated current state.
9. A method as set forth in claim 1 wherein the step of presenting a representation of a current state of a process, and the step of presenting a representation of the one or more derived alternative states, presents the representations in such a manner as to support a user's physical intuition concerning the process.
10. A method as set forth in claim 1 wherein the presentations are presented on display means that includes a display screen having touchscreen capability, and wherein a user selects one of the representations of the one or more derived alternative states by physically touching the display screen within an area associated with the selected one of the derived alternative states.
11. A method as set forth in claim 1, wherein the steps (a), (b), (c), and (d) are repeated until the representation of the current state of the process is selected.
12. A method for controlling a process, comprising the steps of:
(a) presenting a presentation of a current state of a process;
(b) deriving, from the current state of the process, one or more alternative states;
(c) presenting a representation of the one or more derived alternative states; and in response to a selection of one of the representations of the one or more derived alternative states, (d) controlling the process to assume a new current state that corresponds to the derived alternative state that is associated with the selected representation, wherein the process controls the positioning of a camera along or about one or more axes.
(a) presenting a presentation of a current state of a process;
(b) deriving, from the current state of the process, one or more alternative states;
(c) presenting a representation of the one or more derived alternative states; and in response to a selection of one of the representations of the one or more derived alternative states, (d) controlling the process to assume a new current state that corresponds to the derived alternative state that is associated with the selected representation, wherein the process controls the positioning of a camera along or about one or more axes.
13. A method for controlling a process, comprising the steps of:
(a) presenting a representation of a current state of a process;
(b) deriving, from the current state of the process, one or more alternative states;
(c) presenting a representation of the one or more derived alternative states; and in response to a selection of one of the representations of the one or more derived alternative states, (d) controlling the process to assume a new current state that corresponds to the derived alternative state that is associated with the selected representation, wherein the representation of the current state of the process is an image obtained by a camera at a current location, and wherein the representations of the one or more derived alternative states are each an image that would be obtained by the camera if the camera were translated, from the current position, about or along an axis.
(a) presenting a representation of a current state of a process;
(b) deriving, from the current state of the process, one or more alternative states;
(c) presenting a representation of the one or more derived alternative states; and in response to a selection of one of the representations of the one or more derived alternative states, (d) controlling the process to assume a new current state that corresponds to the derived alternative state that is associated with the selected representation, wherein the representation of the current state of the process is an image obtained by a camera at a current location, and wherein the representations of the one or more derived alternative states are each an image that would be obtained by the camera if the camera were translated, from the current position, about or along an axis.
14. A method for controlling a camera, comprising the steps of:
(a) presenting with a display means a representation of a current image obtained by the camera;
(b) deriving, from the current image, one or more alternative images that would be obtained by the camera if the camera were translated about or along an axis;
(c) presenting with the display means a representation of the one or more derived alternative images; and in response to a user selecting one of the representations of the one or more derived alternative images, (d) translating the camera to a position where the camera obtains a new current image that corresponds to the derived alternative image that is associated with the selected representation.
(a) presenting with a display means a representation of a current image obtained by the camera;
(b) deriving, from the current image, one or more alternative images that would be obtained by the camera if the camera were translated about or along an axis;
(c) presenting with the display means a representation of the one or more derived alternative images; and in response to a user selecting one of the representations of the one or more derived alternative images, (d) translating the camera to a position where the camera obtains a new current image that corresponds to the derived alternative image that is associated with the selected representation.
15. A method as set forth in claim 14 wherein derived alternative images include images that would be obtained if a pan angle of the camera were changed and if a tilt angle of the camera were changed.
16. A method as set forth in claim 15 wherein derived alternative images further include images that would be obtained if a camera zoom function were changed.
17. A method as set forth in claim 14 wherein each of the derived alternative images is derived so as to be located within an associated predetermined increment of camera displacement away from a current camera position.
18. A method as set forth in claim 17 wherein the step of translating includes a step of updating a model of the camera orientation to reflect the translation, and a step of updating, if required, one or more of the predetermined camera displacement increments.
19. A method as set forth in claim 18 wherein the step of updating the one or more predetermined camera displacement increments includes an initial step of determining, based on the updated model and on one or more predetermined allowed camera states, if a further translation of the camera would result in the camera entering a disallowed state.
20. A method as set forth in claim 14 wherein the steps (a) and (c) of displaying display the representations in such a manner as to support a user's physical intuition concerning the control of the camera.
21. A method as set forth in claim 14 wherein the display means includes a display screen having touchscreen capability, and wherein the user selects one of the displayed representations by physically touching the display screen within an area associated with the selected one of the derived alternative images.
22. A method as set forth in claim 14 and, in response to the user selecting the representation of the current image, further including a step of initiating an accept-specific sequence of actions for accepting the current image.
23. A method as set forth in claim 14 and, in response to the user selecting the representation of the current image, further including a step of accepting the current image by reproducing the current image in a tangible form, and/or storing the current image for later reference, and/or transmitting the current image to a remote location.
24. A user interface for a process control system, comprising:
means for deriving in accordance with a predetermined transformation, from a current state of the process, one or more alternative states each representing a possible future state of the process that may be entered directly or indirectly from the current state, wherein each of the one or more derived alternative states of the process is not predetermined but is instead made manifest only as a result of an operation of said means for deriving;
means for presenting a representation of the current state of the process and for presenting a representation of the one or more derived alternative states; and control means, responsive to a user selecting one of the representations of the one or more derived alternative states, for controlling the process to assume a new current state that corresponds to the derived alternative state that is associated with the selected representation.
means for deriving in accordance with a predetermined transformation, from a current state of the process, one or more alternative states each representing a possible future state of the process that may be entered directly or indirectly from the current state, wherein each of the one or more derived alternative states of the process is not predetermined but is instead made manifest only as a result of an operation of said means for deriving;
means for presenting a representation of the current state of the process and for presenting a representation of the one or more derived alternative states; and control means, responsive to a user selecting one of the representations of the one or more derived alternative states, for controlling the process to assume a new current state that corresponds to the derived alternative state that is associated with the selected representation.
25. A user interface as set forth in claim 24 wherein said control means is responsive to the user selecting the representation of the current state of the process, for initiating a current state accept-specific sequence of operations.
26. A user interface as set forth in claim 24 wherein said control means includes actuator means for driving the process into the new current state.
27. A user interface as forth in claim 24 wherein said control means includes means for storing and updating a model of the process to reflect an assumption of the new current state.
28. A user interface as set forth in claim 24 wherein said deriving means includes means for deriving the one or more alternative states in accordance with a predetermined increment, and wherein said predetermined increment is generated by said control means.
29. A user interface as set forth in claim 24 wherein said presenting means presents the representations in such a manner as to support a user's physical intuition concerning the process.
30. A user interface as set forth in claim 24 wherein said presenting means includes a display screen, and wherein the user selects one of the representations of the one or more derived alternative states by physically touching the display screen to specify one of the derived alternative states, or by otherwise specifying one of the derived alternative states.
31. A user interface comprising:
process model means for predicting in accordance with a predetermined transformation one or more allowable future states of a process from a current state of the process, each of the predicted one or more future states, representing a future state of the process that may be entered directly or indirectly from the current state, wherein each of the predicted one or more future states of the process is not predetermined but is instead made manifest only as a result of an operation of said process model means;
display processing means, having an input coupled to an output of said process model means, for deriving, for each of the predicted allowable future states, a representation of said predicted allowable future state, said display processing means having an output coupled to display means for displaying each of said derived representations in conjunction with a representation of the current state of the process; and control means, having a first input coupled to an output of said process model means and a second input coupled to a means whereby a user selects one of said derived representations of one of said predicted allowable future states, for controlling the process to enter a new current state that corresponds to the selected derived representation.
process model means for predicting in accordance with a predetermined transformation one or more allowable future states of a process from a current state of the process, each of the predicted one or more future states, representing a future state of the process that may be entered directly or indirectly from the current state, wherein each of the predicted one or more future states of the process is not predetermined but is instead made manifest only as a result of an operation of said process model means;
display processing means, having an input coupled to an output of said process model means, for deriving, for each of the predicted allowable future states, a representation of said predicted allowable future state, said display processing means having an output coupled to display means for displaying each of said derived representations in conjunction with a representation of the current state of the process; and control means, having a first input coupled to an output of said process model means and a second input coupled to a means whereby a user selects one of said derived representations of one of said predicted allowable future states, for controlling the process to enter a new current state that corresponds to the selected derived representation.
32. A user interface as set forth in claim 31 wherein said display means includes a display screen having touchscreen capability, and wherein the user selects one of the derived representations of the one or more predicted allowable future states by physically touching the display screen within an area associated with the selected one of the derived representations.
33. A user interface as set forth in claim 31 wherein said display means includes a display screen, and further including means for use by the user for specifying one of the derived representations of the one or more predicted allowable future states to select one of the predicted allowable future states.
34. A user interface as set forth in claim 31 wherein said display means displays the derived representations in such a manner as to support a user's physical intuition concerning the process.
35. A user interface for use with a video image capture means, comprising: display means having a display screen, said display screen displaying in a first region a representation of a captured video signal, said display screen displaying in a second region a transformed representation of said captured video signal;
video image transform means, having an input for receiving the captured video signal, for generating said transformed representation from the captured video signal, the transformed representation being generated to represent a video image that would be obtained from a predetermined allowable future state of the video image capture means; and means for specifying to said user interface either the first region or the second region so as to select the representation of the captured video signal, or to cause the video image capture means to enter the predetermined allowable future state, respectively.
video image transform means, having an input for receiving the captured video signal, for generating said transformed representation from the captured video signal, the transformed representation being generated to represent a video image that would be obtained from a predetermined allowable future state of the video image capture means; and means for specifying to said user interface either the first region or the second region so as to select the representation of the captured video signal, or to cause the video image capture means to enter the predetermined allowable future state, respectively.
36. A method for controlling a process, comprising the steps of:
(a) presenting a representation of a current state of a process;
(b) deriving, from the current state of the process, one or more alternative states, wherein each of the one or more derived alternative states of the process is not predetermined but is instead made manifest only as a result of executing the step of deriving;
(c) presenting a representation of the one or more derived alternative states; and in response to a selection of one of the representations of the one or more derived alternative states, (d) controlling the process to assume a new current state that corresponds to the derived alternative state that is associated with the selected representation, wherein the process controls the movement of one or more physical objects within a space defined by a coordinate system.
(a) presenting a representation of a current state of a process;
(b) deriving, from the current state of the process, one or more alternative states, wherein each of the one or more derived alternative states of the process is not predetermined but is instead made manifest only as a result of executing the step of deriving;
(c) presenting a representation of the one or more derived alternative states; and in response to a selection of one of the representations of the one or more derived alternative states, (d) controlling the process to assume a new current state that corresponds to the derived alternative state that is associated with the selected representation, wherein the process controls the movement of one or more physical objects within a space defined by a coordinate system.
37. A method for controlling a process, comprising the steps of:
(a) presenting a representation of a current state of a process;
(b) deriving, from the current state of the process, one or more alternative states, wherein each of the one or more derived alternative states of the process is not predetermined but is instead made manifest only as a result of executing the step of deriving;
(c) presenting a representation of the one or more derived alternative states; and in response to a selection of one of the representations of the one or more derived alternative states, (d) controlling the process to assume a new current state that corresponds to the derived alternative state that is associated with the selected representation, wherein the process controls the orientation of one or more physical objects within a space defined by a coordinate system.
(a) presenting a representation of a current state of a process;
(b) deriving, from the current state of the process, one or more alternative states, wherein each of the one or more derived alternative states of the process is not predetermined but is instead made manifest only as a result of executing the step of deriving;
(c) presenting a representation of the one or more derived alternative states; and in response to a selection of one of the representations of the one or more derived alternative states, (d) controlling the process to assume a new current state that corresponds to the derived alternative state that is associated with the selected representation, wherein the process controls the orientation of one or more physical objects within a space defined by a coordinate system.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US87050392A | 1992-04-17 | 1992-04-17 | |
| US870,503 | 1992-04-17 |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CA2089787A1 CA2089787A1 (en) | 1993-10-18 |
| CA2089787C true CA2089787C (en) | 1999-01-26 |
Family
ID=25355521
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA002089787A Expired - Fee Related CA2089787C (en) | 1992-04-17 | 1993-02-18 | Method and apparatus for user control by deriving next states of a process from a current state and by providing a visual presentation of the derived next states |
Country Status (7)
| Country | Link |
|---|---|
| US (1) | US5426732A (en) |
| EP (1) | EP0565830B1 (en) |
| JP (1) | JP3050348B2 (en) |
| AT (1) | ATE224076T1 (en) |
| CA (1) | CA2089787C (en) |
| DE (1) | DE69332282T2 (en) |
| ES (1) | ES2179043T3 (en) |
Families Citing this family (85)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8352400B2 (en) | 1991-12-23 | 2013-01-08 | Hoffberg Steven M | Adaptive pattern recognition based controller apparatus and method and human-factored interface therefore |
| US5903454A (en) * | 1991-12-23 | 1999-05-11 | Hoffberg; Linda Irene | Human-factored interface corporating adaptive pattern recognition based controller apparatus |
| US10361802B1 (en) | 1999-02-01 | 2019-07-23 | Blanding Hovenweep, Llc | Adaptive pattern recognition based control system and method |
| US5757418A (en) * | 1992-07-31 | 1998-05-26 | Canon Kabushiki Kaisha | Television conference system and method therefor |
| DE69434779T2 (en) * | 1993-09-20 | 2007-06-14 | Canon K.K. | video system |
| US6266085B1 (en) * | 1994-11-17 | 2001-07-24 | Canon Kabushiki Kaisha | Camera imaging and magnification device |
| JP3335017B2 (en) * | 1994-11-29 | 2002-10-15 | キヤノン株式会社 | Camera device control device |
| US6094600A (en) * | 1996-02-06 | 2000-07-25 | Fisher-Rosemount Systems, Inc. | System and method for managing a transaction database of records of changes to field device configurations |
| US6868538B1 (en) | 1996-04-12 | 2005-03-15 | Fisher-Rosemount Systems, Inc. | Object-oriented programmable controller |
| US5838563A (en) * | 1996-04-12 | 1998-11-17 | Fisher-Rosemont Systems, Inc. | System for configuring a process control environment |
| US5940294A (en) * | 1996-04-12 | 1999-08-17 | Fisher-Rosemont Systems, Inc. | System for assisting configuring a process control environment |
| EP0825506B1 (en) | 1996-08-20 | 2013-03-06 | Invensys Systems, Inc. | Methods and apparatus for remote process control |
| US5832189A (en) * | 1996-09-26 | 1998-11-03 | Interval Research Corporation | Affect-based robot communication methods and systems |
| US6310609B1 (en) | 1997-04-17 | 2001-10-30 | Nokia Mobile Phones Limited | User interface with guide lights |
| JP2000089889A (en) * | 1998-09-14 | 2000-03-31 | Sharp Corp | Input display device |
| US7528508B2 (en) | 1998-10-09 | 2009-05-05 | Azoteq Pty Ltd. | Touch sensor user interface with compressible material construction |
| US7265494B2 (en) | 1998-10-09 | 2007-09-04 | Azoteq Pty Ltd. | Intelligent user interface with touch sensor technology |
| US7904187B2 (en) | 1999-02-01 | 2011-03-08 | Hoffberg Steven M | Internet appliance system and method |
| US7089530B1 (en) | 1999-05-17 | 2006-08-08 | Invensys Systems, Inc. | Process control configuration system with connection validation and configuration |
| AU5273100A (en) | 1999-05-17 | 2000-12-05 | Foxboro Company, The | Methods and apparatus for control configuration with versioning, security, composite blocks, edit selection, object swapping, formulaic values and other aspects |
| US6754885B1 (en) | 1999-05-17 | 2004-06-22 | Invensys Systems, Inc. | Methods and apparatus for controlling object appearance in a process control configuration system |
| US6788980B1 (en) | 1999-06-11 | 2004-09-07 | Invensys Systems, Inc. | Methods and apparatus for control using control devices that provide a virtual machine environment and that communicate via an IP network |
| US6618630B1 (en) | 1999-07-08 | 2003-09-09 | Fisher-Rosemount Systems, Inc. | User interface that integrates a process control configuration system and a field device management system |
| US6510352B1 (en) | 1999-07-29 | 2003-01-21 | The Foxboro Company | Methods and apparatus for object-based process control |
| JP2001054013A (en) * | 1999-08-13 | 2001-02-23 | Fuji Photo Film Co Ltd | Image pickup signal processor and its processing method |
| US7071969B1 (en) * | 2001-09-27 | 2006-07-04 | National Semiconductor Corporation | Parameterized preview array for iterative image optimization in remote applications |
| CA2517579A1 (en) * | 2002-02-28 | 2003-09-04 | Zetacon Corporation | Predictive control system and method |
| US20040002950A1 (en) | 2002-04-15 | 2004-01-01 | Brennan Sean F. | Methods and apparatus for process, factory-floor, environmental, computer aided manufacturing-based or other control system using hierarchically enumerated data set |
| US7221398B2 (en) * | 2003-05-01 | 2007-05-22 | Hewlett-Packard Development Company, L.P. | Accurate preview for digital cameras |
| US20040218080A1 (en) * | 2003-05-02 | 2004-11-04 | Stavely Donald J. | Digital camera with preview alternatives |
| WO2005015157A2 (en) * | 2003-08-08 | 2005-02-17 | Quantum Intech, Inc. | Eletrophysiological intuition indicator |
| US7761923B2 (en) | 2004-03-01 | 2010-07-20 | Invensys Systems, Inc. | Process control methods and apparatus for intrusion detection, protection and network hardening |
| US9165280B2 (en) * | 2005-02-22 | 2015-10-20 | International Business Machines Corporation | Predictive user modeling in user interface design |
| WO2007123753A2 (en) * | 2006-03-30 | 2007-11-01 | Invensys Systems, Inc. | Digital data processing apparatus and methods for improving plant performance |
| US7768527B2 (en) * | 2006-05-31 | 2010-08-03 | Beihang University | Hardware-in-the-loop simulation system and method for computer vision |
| US8497928B2 (en) | 2007-07-31 | 2013-07-30 | Palm, Inc. | Techniques to automatically focus a digital camera |
| US7987145B2 (en) * | 2008-03-19 | 2011-07-26 | Honeywell Internationa | Target trajectory generator for predictive control of nonlinear systems using extended Kalman filter |
| WO2009146359A1 (en) | 2008-05-28 | 2009-12-03 | Illinois Tool Works Inc. | Welding training system |
| CN104407518B (en) | 2008-06-20 | 2017-05-31 | 因文西斯系统公司 | The system and method interacted to the reality and Simulation Facility for process control |
| US20090327949A1 (en) * | 2008-06-26 | 2009-12-31 | Honeywell International Inc. | Interactive overlay window for a video display |
| US8645871B2 (en) | 2008-11-21 | 2014-02-04 | Microsoft Corporation | Tiltable user interface |
| FR2939555B1 (en) * | 2008-12-10 | 2015-09-18 | Somfy Sas | METHOD FOR OPERATING A DEVICE FOR CONTROLLING DOMOTIC EQUIPMENT |
| US8127060B2 (en) | 2009-05-29 | 2012-02-28 | Invensys Systems, Inc | Methods and apparatus for control configuration with control objects that are fieldbus protocol-aware |
| US8463964B2 (en) | 2009-05-29 | 2013-06-11 | Invensys Systems, Inc. | Methods and apparatus for control configuration with enhanced change-tracking |
| NO332170B1 (en) | 2009-10-14 | 2012-07-16 | Cisco Systems Int Sarl | Camera control device and method |
| US9101994B2 (en) | 2011-08-10 | 2015-08-11 | Illinois Tool Works Inc. | System and device for welding training |
| US9573215B2 (en) | 2012-02-10 | 2017-02-21 | Illinois Tool Works Inc. | Sound-based weld travel speed sensing system and method |
| US8612093B2 (en) | 2012-02-22 | 2013-12-17 | Bae Systems Controls Inc. | Automatic dynamic control of aircraft cabin systems |
| US9368045B2 (en) | 2012-11-09 | 2016-06-14 | Illinois Tool Works Inc. | System and device for welding training |
| US9583014B2 (en) | 2012-11-09 | 2017-02-28 | Illinois Tool Works Inc. | System and device for welding training |
| US9583023B2 (en) | 2013-03-15 | 2017-02-28 | Illinois Tool Works Inc. | Welding torch for a welding training system |
| US9728103B2 (en) | 2013-03-15 | 2017-08-08 | Illinois Tool Works Inc. | Data storage and analysis for a welding training system |
| US9713852B2 (en) | 2013-03-15 | 2017-07-25 | Illinois Tool Works Inc. | Welding training systems and devices |
| US9666100B2 (en) | 2013-03-15 | 2017-05-30 | Illinois Tool Works Inc. | Calibration devices for a welding training system |
| US9672757B2 (en) | 2013-03-15 | 2017-06-06 | Illinois Tool Works Inc. | Multi-mode software and method for a welding training system |
| US11090753B2 (en) | 2013-06-21 | 2021-08-17 | Illinois Tool Works Inc. | System and method for determining weld travel speed |
| US10056010B2 (en) | 2013-12-03 | 2018-08-21 | Illinois Tool Works Inc. | Systems and methods for a weld training system |
| US10105782B2 (en) | 2014-01-07 | 2018-10-23 | Illinois Tool Works Inc. | Feedback from a welding torch of a welding system |
| US9757819B2 (en) | 2014-01-07 | 2017-09-12 | Illinois Tool Works Inc. | Calibration tool and method for a welding system |
| US10170019B2 (en) | 2014-01-07 | 2019-01-01 | Illinois Tool Works Inc. | Feedback from a welding torch of a welding system |
| US9751149B2 (en) | 2014-01-07 | 2017-09-05 | Illinois Tool Works Inc. | Welding stand for a welding system |
| US9724788B2 (en) | 2014-01-07 | 2017-08-08 | Illinois Tool Works Inc. | Electrical assemblies for a welding system |
| US9589481B2 (en) | 2014-01-07 | 2017-03-07 | Illinois Tool Works Inc. | Welding software for detection and control of devices and for analysis of data |
| US9862049B2 (en) | 2014-06-27 | 2018-01-09 | Illinois Tool Works Inc. | System and method of welding system operator identification |
| US9937578B2 (en) | 2014-06-27 | 2018-04-10 | Illinois Tool Works Inc. | System and method for remote welding training |
| US10307853B2 (en) | 2014-06-27 | 2019-06-04 | Illinois Tool Works Inc. | System and method for managing welding data |
| US10665128B2 (en) | 2014-06-27 | 2020-05-26 | Illinois Tool Works Inc. | System and method of monitoring welding information |
| US11014183B2 (en) | 2014-08-07 | 2021-05-25 | Illinois Tool Works Inc. | System and method of marking a welding workpiece |
| US9724787B2 (en) | 2014-08-07 | 2017-08-08 | Illinois Tool Works Inc. | System and method of monitoring a welding environment |
| US9875665B2 (en) | 2014-08-18 | 2018-01-23 | Illinois Tool Works Inc. | Weld training system and method |
| US10239147B2 (en) | 2014-10-16 | 2019-03-26 | Illinois Tool Works Inc. | Sensor-based power controls for a welding system |
| US11247289B2 (en) | 2014-10-16 | 2022-02-15 | Illinois Tool Works Inc. | Remote power supply parameter adjustment |
| US10373304B2 (en) | 2014-11-05 | 2019-08-06 | Illinois Tool Works Inc. | System and method of arranging welding device markers |
| US10204406B2 (en) | 2014-11-05 | 2019-02-12 | Illinois Tool Works Inc. | System and method of controlling welding system camera exposure and marker illumination |
| US10210773B2 (en) | 2014-11-05 | 2019-02-19 | Illinois Tool Works Inc. | System and method for welding torch display |
| US10417934B2 (en) | 2014-11-05 | 2019-09-17 | Illinois Tool Works Inc. | System and method of reviewing weld data |
| US10490098B2 (en) | 2014-11-05 | 2019-11-26 | Illinois Tool Works Inc. | System and method of recording multi-run data |
| US10402959B2 (en) | 2014-11-05 | 2019-09-03 | Illinois Tool Works Inc. | System and method of active torch marker control |
| US10427239B2 (en) | 2015-04-02 | 2019-10-01 | Illinois Tool Works Inc. | Systems and methods for tracking weld training arc parameters |
| US10373517B2 (en) | 2015-08-12 | 2019-08-06 | Illinois Tool Works Inc. | Simulation stick welding electrode holder systems and methods |
| US10593230B2 (en) | 2015-08-12 | 2020-03-17 | Illinois Tool Works Inc. | Stick welding electrode holder systems and methods |
| US10657839B2 (en) | 2015-08-12 | 2020-05-19 | Illinois Tool Works Inc. | Stick welding electrode holders with real-time feedback features |
| US10438505B2 (en) | 2015-08-12 | 2019-10-08 | Illinois Tool Works | Welding training system interface |
| US11288978B2 (en) | 2019-07-22 | 2022-03-29 | Illinois Tool Works Inc. | Gas tungsten arc welding training systems |
| US11776423B2 (en) | 2019-07-22 | 2023-10-03 | Illinois Tool Works Inc. | Connection boxes for gas tungsten arc welding training systems |
Family Cites Families (54)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| NZ218742A (en) * | 1986-06-03 | 1990-09-26 | Fisher & Paykel | Programmed logic controller |
| US3911215A (en) * | 1974-03-18 | 1975-10-07 | Elographics Inc | Discriminating contact sensor |
| US4085438A (en) * | 1976-11-11 | 1978-04-18 | Copes-Vulcan Inc. | Digital sootblower control systems and methods therefor |
| US4220815B1 (en) * | 1978-12-04 | 1996-09-03 | Elographics Inc | Nonplanar transparent electrographic sensor |
| US4479197A (en) * | 1980-11-25 | 1984-10-23 | Hewlett-Packard Company | Method and apparatus for selecting and setting the mode of operation for a mechanism |
| JPS5833715A (en) * | 1981-08-21 | 1983-02-28 | Hitachi Ltd | Automatic selecting and display device for screen |
| US4516156A (en) * | 1982-03-15 | 1985-05-07 | Satellite Business Systems | Teleconferencing method and system |
| JPS6066298A (en) * | 1983-09-21 | 1985-04-16 | キヤノン株式会社 | Information processor |
| EP0203914B1 (en) * | 1984-01-30 | 1989-10-18 | Touch Display Systems Ab | Touch controlled display device |
| US4649499A (en) * | 1984-03-07 | 1987-03-10 | Hewlett-Packard Company | Touchscreen two-dimensional emulation of three-dimensional objects |
| US5003505A (en) * | 1984-05-07 | 1991-03-26 | Hewlett-Packard Company | Touchscreen/keyboard scanner |
| US4873623A (en) * | 1985-04-30 | 1989-10-10 | Prometrix Corporation | Process control interface with simultaneously displayed three level dynamic menu |
| JPH0648445B2 (en) * | 1985-06-27 | 1994-06-22 | 株式会社東芝 | Plant monitoring control device |
| JPH0772844B2 (en) * | 1985-10-23 | 1995-08-02 | 株式会社日立製作所 | Robot teaching device |
| US4803039A (en) * | 1986-02-03 | 1989-02-07 | Westinghouse Electric Corp. | On line interactive monitoring of the execution of process operating procedures |
| US4929935A (en) * | 1986-05-02 | 1990-05-29 | John Fluke Mfg. Co., Inc. | Apparatus for aligning raster scanned image to touch panel |
| JPS6361597A (en) * | 1986-09-01 | 1988-03-17 | Mitsubishi Electric Corp | Master station equipment for remote supervisory and controlling equipment |
| US4823283A (en) * | 1986-10-14 | 1989-04-18 | Tektronix, Inc. | Status driven menu system |
| US4868912A (en) * | 1986-11-26 | 1989-09-19 | Digital Electronics | Infrared touch panel |
| US4763356A (en) * | 1986-12-11 | 1988-08-09 | AT&T Information Systems, Inc. American Telephone and Telegraph Company | Touch screen form entry system |
| US4903012A (en) * | 1987-01-20 | 1990-02-20 | Alps Electric Co., Ltd. | Coordinate system input device providing registration calibration and a mouse function |
| US4815014A (en) * | 1987-02-27 | 1989-03-21 | Westinghouse Electric Corp. | Machine assisted execution of process operating procedures |
| US4988982A (en) * | 1987-03-25 | 1991-01-29 | The Grass Valley Group, Inc. | Touch pad machine control |
| FR2615941B1 (en) * | 1987-05-25 | 1991-12-06 | Sfena | DEVICE FOR DETECTING THE POSITION OF A CONTROL MEMBER ON A TOUCH TABLET |
| JPH0789268B2 (en) * | 1987-07-10 | 1995-09-27 | 株式会社日立製作所 | Screen display control method |
| US4847604A (en) * | 1987-08-27 | 1989-07-11 | Doyle Michael D | Method and apparatus for identifying features of an image on a video display |
| JPS6484325A (en) * | 1987-09-28 | 1989-03-29 | Oki Electric Ind Co Ltd | Multiplex input detecting system in pressure sensitive type input device |
| US4931018A (en) * | 1987-12-21 | 1990-06-05 | Lenco, Inc. | Device for training welders |
| US5053758A (en) * | 1988-02-01 | 1991-10-01 | Sperry Marine Inc. | Touchscreen control panel with sliding touch control |
| GB2215942B (en) * | 1988-02-12 | 1992-01-08 | Pearpoint Ltd | Scanning t.v.camera |
| US4980646A (en) * | 1988-03-30 | 1990-12-25 | The Trustees Of The University Of Pennsylvania | Impedance tomographic tactile sensor |
| US4914624A (en) * | 1988-05-06 | 1990-04-03 | Dunthorn David I | Virtual button for touch screen |
| US4853498A (en) * | 1988-06-13 | 1989-08-01 | Tektronix, Inc. | Position measurement apparatus for capacitive touch panel system |
| US5123088A (en) * | 1988-06-27 | 1992-06-16 | Nippon Telegraph And Telephone Corp. | Method and apparatus for creating and displaying navigators for guiding to related image information |
| DE3823219C1 (en) * | 1988-07-08 | 1989-05-18 | Telenorma Telefonbau Und Normalzeit Gmbh, 6000 Frankfurt, De | |
| US4957690A (en) * | 1988-12-07 | 1990-09-18 | General Electric Company | System and method for monitoring and control of safety-related components of a nuclear power plant |
| US5027279A (en) * | 1988-12-30 | 1991-06-25 | Pitney Bowes Inc. | Method and apparatus for remotely controlling a document inserter |
| US5051912A (en) * | 1989-02-07 | 1991-09-24 | Hurco Acquisition Corporation | Vectoring/orbiting control unit for electrical discharge machining |
| US5038142A (en) * | 1989-03-14 | 1991-08-06 | International Business Machines Corporation | Touch sensing display screen apparatus |
| US5121318A (en) * | 1989-04-19 | 1992-06-09 | Westinghouse Electric Corp. | On-line plant operating procedure guidance system |
| JPH0378373A (en) * | 1989-08-22 | 1991-04-03 | Fuji Photo Optical Co Ltd | Television camera operating device |
| US5121319A (en) * | 1989-10-23 | 1992-06-09 | International Business Machines Corporation | Method and apparatus for selective suspension and resumption of computer based manufacturing processes |
| US5055840A (en) * | 1990-01-16 | 1991-10-08 | Carroll Touch Incorporated | Infrared touch input device and light emitted activation circuit |
| JP2820994B2 (en) * | 1990-02-19 | 1998-11-05 | 三菱電機株式会社 | Dynamic simulation method |
| US5276789A (en) * | 1990-05-14 | 1994-01-04 | Hewlett-Packard Co. | Graphic display of network topology |
| JP2673464B2 (en) * | 1990-06-18 | 1997-11-05 | ファナック株式会社 | Molding monitor device |
| WO1992003816A1 (en) * | 1990-08-16 | 1992-03-05 | General Parametrics Corp. | A display control device incorporating an auxiliary display |
| US5208903A (en) * | 1990-09-10 | 1993-05-04 | Eastman Kodak Company | Video image display for predicting color hardcopy image quality |
| US5117285A (en) * | 1991-01-15 | 1992-05-26 | Bell Communications Research | Eye contact apparatus for video conferencing |
| US5224053A (en) * | 1991-01-22 | 1993-06-29 | Dayton Reliable Tool & Mfg. Co. | Interactive control for can end systems |
| JPH04260176A (en) * | 1991-02-15 | 1992-09-16 | Hitachi Ltd | Goods specification decision device and environmental image generator |
| US5191645A (en) * | 1991-02-28 | 1993-03-02 | Sony Corporation Of America | Digital signal processing system employing icon displays |
| US5185628A (en) * | 1991-11-18 | 1993-02-09 | Easatman Kodak Company | Reproduction apparatus with improved operator interactive display for use in job set-up |
| US5202726A (en) * | 1991-12-18 | 1993-04-13 | Xerox Corporation | Facilitation of the diagnosis of malfunctions and set-up of a reproduction machine |
-
1993
- 1993-01-25 JP JP5010230A patent/JP3050348B2/en not_active Expired - Lifetime
- 1993-02-18 CA CA002089787A patent/CA2089787C/en not_active Expired - Fee Related
- 1993-02-20 EP EP93102726A patent/EP0565830B1/en not_active Expired - Lifetime
- 1993-02-20 ES ES93102726T patent/ES2179043T3/en not_active Expired - Lifetime
- 1993-02-20 AT AT93102726T patent/ATE224076T1/en not_active IP Right Cessation
- 1993-02-20 DE DE69332282T patent/DE69332282T2/en not_active Expired - Lifetime
-
1994
- 1994-10-05 US US08/320,891 patent/US5426732A/en not_active Expired - Lifetime
Also Published As
| Publication number | Publication date |
|---|---|
| DE69332282D1 (en) | 2002-10-17 |
| EP0565830B1 (en) | 2002-09-11 |
| JPH0683437A (en) | 1994-03-25 |
| DE69332282T2 (en) | 2003-05-22 |
| EP0565830A2 (en) | 1993-10-20 |
| US5426732A (en) | 1995-06-20 |
| CA2089787A1 (en) | 1993-10-18 |
| ATE224076T1 (en) | 2002-09-15 |
| EP0565830A3 (en) | 1999-01-07 |
| ES2179043T3 (en) | 2003-01-16 |
| JP3050348B2 (en) | 2000-06-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CA2089787C (en) | Method and apparatus for user control by deriving next states of a process from a current state and by providing a visual presentation of the derived next states | |
| US6515689B1 (en) | Control apparatus | |
| US5872594A (en) | Method for open loop camera control using a motion model to control camera movement | |
| EP0737005B1 (en) | Camera control method and apparatus | |
| US5396287A (en) | TV camera work control apparatus using tripod head | |
| EP0610863B1 (en) | Omniview motionless camera surveillance system | |
| US4992866A (en) | Camera selection and positioning system and method | |
| US7714936B1 (en) | Omniview motionless camera orientation system | |
| US8085300B2 (en) | Surveillance camera system, remote-controlled monitoring device, control method, and their control program | |
| US5714977A (en) | Video processing system for movement simulation | |
| WO2002079794A1 (en) | Method for assisting an automated video tracking system in reaquiring a target | |
| JPH04308895A (en) | Method for video operation, video search, video process definition, remote operation monitoring, and device or system | |
| EP0360599B1 (en) | A video processing system for movement simulation | |
| JPH05236319A (en) | Telecamera action controller using panhead | |
| JP3876493B2 (en) | Information display method | |
| JP3419516B2 (en) | Control method of imaging means and imaging control device | |
| KR20070106209A (en) | Control device of a pan/tilt camera and recording medium thereof | |
| JP3744995B2 (en) | Imaging method and apparatus | |
| JP2981408B2 (en) | Method and apparatus for controlling high-speed introduction of a target object in a camera image | |
| JPH05236317A (en) | Telecamera action controller using panhead | |
| JPH05344402A (en) | Television camera operation controller using universal head | |
| JPS6231272A (en) | Universal head controller | |
| JPH05236318A (en) | Telecamera action controller using panhead | |
| JP2001128152A (en) | Monitor camera system | |
| JPH0784713A (en) | Command processor |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| EEER | Examination request | ||
| MKLA | Lapsed |