US20110115721A1 - Translating User Interaction With A Touch Screen Into Input Commands - Google Patents
Translating User Interaction With A Touch Screen Into Input Commands Download PDFInfo
- Publication number
- US20110115721A1 US20110115721A1 US12/621,759 US62175909A US2011115721A1 US 20110115721 A1 US20110115721 A1 US 20110115721A1 US 62175909 A US62175909 A US 62175909A US 2011115721 A1 US2011115721 A1 US 2011115721A1
- Authority
- US
- United States
- Prior art keywords
- touch screen
- graphical display
- screen system
- user interaction
- command
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
- G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
- G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
- G06F3/04883—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures for inputting data by handwriting, e.g. gesture or text
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
- G06F3/0412—Digitisers structurally integrated in a display
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
- G06F3/0416—Control or interface arrangements specially adapted for digitisers
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
- G06F2203/048—Indexing scheme relating to G06F3/048
- G06F2203/04808—Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
Definitions
- This specification relates to translating user interaction with a touch screen into input commands.
- Touch screens are graphical display devices that act as both inputs and outputs.
- touch screens include a graphical display and detect the presence and location of manual interaction with that graphical display. A user can thus interact with the elements shown on the graphical display intuitively, in ways that resemble the ways that humans interact with the world outside of graphical displays and electronic devices.
- an electronic device may be able to receive input that commands some portion of the displayed elements be scrolled or panned.
- Scrolling is the movement of elements shown on a display screen, generally either in either a horizontal or vertical direction.
- a list of contacts can be scrolled from top to bottom on the display screen of a cell phone so that some contacts appear to move off of the bottom display screen as other contacts appear to move onto the top of the display screen.
- Panning is the movement of the portion of a map shown in on a display screen. Panning does not change the arrangement of features shown on the map but rather moves the user's field of view across what appears to be a larger image of the map.
- a user commands the graphical display to scroll or pan by dragging a finger across the display.
- the graphical display will scroll or pan in the same direction.
- a method includes a touch screen system identifying stationary contact with a touch screen graphical display, the touch screen system starting a timer in response to the identification of the stationary contact, the touch screen system determining whether a second user interaction with the touch screen graphical display is timely using the timer, and in response to the second user interaction being timely, the touch screen system interpreting the second user interaction as a glyph.
- the touch screen system can interpret the stationary contact as a zoom-in command in response to the second user interaction being untimely.
- the method can also include, in response to the second user interaction being untimely, the touch screen system interpreting the stationary contact as a command directing that that further details regarding an entry contacted by the stationary contact.
- Interpreting the second user interaction as a glyph can include interpreting the second user interaction under a set of rules tailored to the identification of alphanumeric characters.
- Identifying the stationary contact can include determining that the stationary contact is in contact with an edge of the touch screen graphical display.
- Interpreting the second user interaction as a glyph can include disabling of finger dragging interactions as scrolling or panning commands.
- the method can also include the touch screen system pinning the touch screen graphical display in response to identification of the stationary contact.
- the method can also include the touch screen system changing to a new translation mode in response to the second user interaction being timely.
- the new translation mode can include a glyph translation mode tailored to interpret user interaction involving multiple finger drags across the touch screen graphical display as glyphs.
- a touch screen system in another aspect, includes a touch screen graphical display, a command interpreter configured to identify modifier interactions with the graphical display and, in response to identification of the modifier interaction, modify the interpretation of other interaction with the graphical display, one or more persistent data storage devices.
- the data storage devices store a first set of rules for interpreting user interaction with the graphical display and a second set of rules for interpreting user interaction with the graphical display.
- the first set of rules interpret motion across a map or a proper subset of a collection of information as a scrolling or panning command.
- the second set of rules interpret the motion as a glyph.
- the command interpreter is configured to modify the interpretation of the other interaction by switching between interpretation under the first set of rules and interpretation under the second set of rules.
- the touch screen system can include a collection of map information stored at the one or more persistent data storage devices.
- the touch screen system can also include a hand-held housing mounting the touch screen graphical display.
- the command interpreter can be configured to identify contact on a lateral side of the touch screen graphical display as modifier interactions.
- the command interpreter can be configured to identify stationary contact followed by a timely second contact as modifier interactions.
- the second set of rules can include rules for interpreting the motion as a custom glyph.
- the command interpreter can be configured to identify a pinning command, the pinning command disabling a touch screen system from interpreting finger dragging as a scroll command or as a pan command.
- the touch screen system can also include a clock.
- the command interpreter can be configured to start a timer that relies on the clock in response to identification of a stationary contact with the graphical display.
- the command interpreter can be configured to identify a modifier interaction in response to a second user interaction being received before the timer reaches a threshold period.
- the threshold period can be within 15% of one second.
- the command interpreter can be configured to interpret the stationary contact as a zoom-in command in response to a failure to receive a second user interaction before the timer reaches a threshold period.
- the command interpreter can be configured to interpret the stationary contact as a command directing that that further details regarding an entry contacted by the stationary contact in response to a failure to receive a second user interaction before the timer reaches a threshold period.
- the touch means for translating can include a clock and a timer that relies on the clock and configured to start in response to identification of a stationary contact with the touch screen graphical display.
- FIG. 1 is a schematic diagram of a touch screen system.
- FIGS. 2-9 are schematic representations of user interaction with touch screen systems.
- FIGS. 10-12 are flow charts of processes that can be performed by touch screen systems to translate user interaction into input commands
- FIG. 13 shows an example of generic devices that may be used with the described techniques.
- FIG. 1 is a schematic diagram of a touch screen system 100 .
- Touch screen system 100 includes a housing 105 onto which a touch screen graphical display 110 is moveably mounted. Touch screen graphical display 110 is moveable to expose a collection of input keys 110 .
- Housing 105 houses a collection of electronic components 120 that cooperate to provide the functionality needed to operate touch screen system 100 , including the ability to distinguish different commands input over touch screen graphical display 110 .
- Touch screen system 100 can be, e.g., a computer, a telephone, a music player, a PDA, or the like. In some implementations, touch screen system 100 can be mobile, hand-held device, as shown.
- Electronic component collection 120 includes a display manager 125 , a drawn user input controller 130 , an input manager 135 , and a clock 155 .
- Display manager 125 , drawn user input controller 130 , and input manager 135 can be implemented using digital electronic circuitry, or in computer software, firmware, or hardware.
- Clock 155 can be an integral part of one or more of display manager 125 , drawn user input controller 130 , and input manager 135 or clock 155 can be a separate component, as shown.
- Display manager 125 is a device that is configured to render content for presentation on touch screen graphical display 110 .
- Display manager 125 can receive graphic-related content from a number of sources and may determine how the content is to be provided to a user. For example, different windows for various applications 140 may need to be displayed, and the display manager 125 can determine which to display, which to hide, and what to display or hide when there is overlap between various graphical objects.
- Display manager 125 can include various components to provide particular functionality for interacting with displayed components, which may be shared across multiple applications, and may be supplied, for example, by an operating system.
- Drawn user input controller 130 is a device that is configured to translate user interaction with touch screen graphical display 110 into commands. Drawn user input controller 130 can thus distinguish between different commands input over touch screen graphical display 110 . In distinguishing between different commands, drawn user input controller 130 may refer to a command interpreter 145 . Command interpreter 145 implements sets of rules 150 for interpreting user interaction with touch screen graphical display 110 as commands. In some implementations, rule sets 150 can include both default and user-specific rules for translating user interaction into commands and can be stored in a data storage device.
- input manager 135 can include one or more maps for implementing the interpretations of the user interaction with touch screen graphical display 110 using control signals that can be interpreted by other components in collection 120 .
- different maps are specific to different applications 140 . In this manner, the controller 417 can act as an intermediary between input elements and applications 140 .
- a lists, maps, etc. storage 160 can store various definitions for objects that may be stored on and presented by touch screen system 100 .
- Such objects include lists of other objects, such as lists of songs, television shows, movies, or other media on the device.
- the lists may also include lists of file names where touch screen system 100 stores a large number of files.
- other objects such as maps may be stored (e.g., as graphical map tiles that can be pre-fetched from a remote server), and meta data about the maps, such as the names of towns and streets on a map, along with the names of points of interest and business on the maps.
- storage 160 can also store profile information for a user.
- Profile information includes various parameters about a user of touch screen system 100 .
- the profile information may include data defining various custom commands that a user has entered, along with objects or actions to be associated with the entry of those commands.
- Profile information can be stored in a variety of forms, such as in one or more XML files or in Sqlite files.
- the touch screen system will interpret user interaction with graphical display 200 according to a set of rules. Typically, these rules are tailored to allow a user to navigate through the collection of information. Referring to FIG. 3 , the touch screen system translates the dragging of a finger 305 in a generally downward direction across graphical display 200 into a scroll down command. This movement of finger 305 is represented by an arrow 310 which, in general, is not actually displayed on graphical display 200 . In response to the scroll down command, entries 235 , 240 , 245 appear to move off the bottom of the window in graphical display 200 , whereas new entries 315 , 320 , 325 appear to enter the top of the window in graphical display 200 . Thus, a different proper subset 330 of the collection of identification information is displayed for a user to see.
- the touch screen system may interpret this L-shaped pattern as including a scroll down command.
- the downward stroke of the finger in the first part of the L-shaped pattern can be mistaken for a downward scroll command.
- the sideways stroke of the finger in the second part of the L-shaped pattern can be ignored or taken as a command that is independent of the downward scroll command.
- the touch screen system may not be able to distinguish between a downward scroll command and a second command embodied by an L-shaped pattern. This limits the number of commands that can be input over graphical display 200 .
- This movement of finger 305 is represented by an arrow 605 which, in general, is not actually displayed on graphical display 500 .
- the bottom portion of the map appear to move off the bottom of the window in graphical display 500 , whereas a new portion of the map appears to enter the top of the window in graphical display 500 .
- the arrow indicium identifying the city Minneapolis moves downward and a different section 610 of the map is displayed for a user to see.
- the touch screen system further interprets the dragging of a finger 305 sideways across graphical display 500 as a pan sideways command.
- This movement of finger 305 is represented by the lower, sideways-directed portion of arrow 605 .
- the rightmost portion of the map appear to move off the right of the window in graphical display 500 , whereas a new portion of the map appears to enter the left of the window in graphical display 500 .
- the arrow indicium identifying the city Minneapolis moves rightward on graphical display 500 and a different section 705 of the map is displayed for a user to see.
- FIG. 8 is a schematic representations of user interaction with a touch screen system such as touch screen system 100 ( FIG. 1 ) in which additional number of commands can be input over a graphical display.
- the user has placed a thumb 805 on the lateral side of graphical display 200 while dragging finger 305 in a generally downward direction across graphical display 200 .
- the touch screen system does not interpret this dragging of finger 305 as a scroll down command. Instead, the touch screen system translates the combination of the placement of thumb 805 and the dragging of finger 305 into a new command altogether, e.g., as the input of a glyph command.
- FIG. 9 is a schematic representations of user interaction with a touch screen system such as touch screen system 100 ( FIG. 1 ) in which additional number of commands can be input over a graphical display.
- the user ha's placed a thumb 805 on the lateral side of graphical display 500 while dragging finger 305 in a generally L-shaped pattern across graphical display 500 .
- the touch screen system does not interpret this movement of finger 305 as a pair of panning commands. Instead, the touch screen system translates the combination of the placement of thumb 805 and the dragging of finger 305 into a new command altogether, e.g., as the input of a glyph command.
- a single glyph can also be assigned to multiple objects.
- the touch screen system can construe the glyph as a common to display a pop up control that lists the multiple objects.
- the proper object may be selected automatically by the system based on the context in which the glyph was entered. For example, if the glyph is entered when an address book application is active, the touch screen system can construe the glyph as a command to jump to a contact entry corresponding to the glyph rather than a music playlist corresponding to the glyph.
- the system performing process 1000 can receive a first user interaction with the graphical display of a touch screen system at box 1005 .
- the system determines whether the first user interaction is a modifier interaction at decision 1010 .
- a modifier interaction is user interaction with the touch screen graphical display that modifies the interpretation of other interaction with the graphical display, e.g., when the first and the second interaction occur at the same time.
- the positioning of thumb 805 on the lateral side of graphical displays 200 , 500 is a modifier interaction in that it modifies the interpretation of the dragging of finger 305 across displays 200 , 500 .
- a modifier interaction can be identified based on the characteristics of the user interaction with the graphical display. For example, in the context of FIGS. 2-8 , the positioning of thumb 805 on the lateral side of graphical displays 200 , 500 can be identified according to the positioning of thumb 805 relative to—or even in contact with—the lateral side of the graphical displays 200 , 500 . A lack of movement of thumb 805 can also be used to identify a modifier interaction. Further, as described further below, the magnitude of the time delay between the positioning of thumb 805 and the dragging of finger 305 can be used to identify a modifier interaction.
- FIG. 11 is a flow chart of a process 1100 that can be performed by a touch screen system to translate user interaction into input commands.
- Process 1100 can be performed by one or more digital data processing devices that perform operations by executing one or more sets of machine-readable instructions.
- process 1100 can be performed by command interpreter 145 in touch screen system 100 ( FIG. 1 ).
- Process 1100 can be performed in isolation or in conjunction with other digital data processing operations.
- the system performing process 1100 can receive a first user interaction with the graphical display of a touch screen system at box 1105 .
- the system identifies that the first user interaction is a pinning command at box 1110 .
- a pinning command is a command that disables a touch screen system from interpreting finger dragging as a scroll or a pan command even if a subset of a collection of information or a map is displayed.
- the positioning of thumb 805 on the lateral side of graphical displays 200 , 500 is a pinning command in that touch screen systems do not interpret dragging of finger 305 as a scrolling or panning commands.
- the magnitude of the time delay between a pair of interactions can be used to identify a pinning command, as described further below.
- the mode change is persistent even in the absence of continued user interaction.
- the touch screen system changes mode in response to the positioning of thumb 805 on the lateral side of graphical displays 200 , 500 , the user can remove thumb 805 from graphical displays 200 , 500 but graphical displays 200 , 500 will remain pinned.
- the touch screen system is in a quasimode in that the user interaction must continue for the mode change to persist.
- thumb 805 must remain positioned on the lateral side of graphical displays 200 , 500 for graphical displays 200 , 500 to remain pinned.
- the system performing process 1100 can receive a second user interaction with the graphical display of a touch screen system at box 1120 and interpret the second user interaction in the new mode at 1125 .
- the interpretation of the second user interaction in the new mode will generally differ from the interpretation of the second user interaction in the previous mode.
- the interpretation of the dragging of finger 305 as glyphs i.e., in the new mode
- the interpretation of the second user interaction in the new mode can be done in accordance with a second set of rules.
- FIG. 12 is a flow chart of a process 1200 that can be performed by a touch screen system to translate user interaction into input commands.
- Process 1200 can be performed by one or more digital data processing devices that perform operations by executing one or more sets of machine-readable instructions.
- process 1200 can be performed by command interpreter 145 in touch screen system 100 ( FIG. 1 ).
- Process 1200 can be performed in isolation or in conjunction with other digital data processing operations.
- process 1200 can be performed at decision box 1010 in process 1000 ( FIG. 10 ) or at box 1110 in process 1100 ( FIG. 11 ).
- the system performing process 1200 can also determine whether a second user interaction with the graphical display of a touch screen system has been timely received at decision box 1215 .
- the timely receipt of the second interaction requires that the second interaction be received before a period measured by the timer has passed. For example, in some implementations, the second interaction must be received within one second, or within 600 ms, in order to be considered timely.
- the duration of the period is fixed. In other implementations, the duration of the period changes, e.g., in response to the amount of motion in the stationary contact. For example, if the stationary contact is relatively unstable with a large amount of motion, the duration of the period can be increased. On the other hand, if the stationary contact is relatively stable with a small amount of motion, the duration of the period can be decreased.
- a glyph rule set is a set of rules for translating user interaction with the graphical display of a touch screen system as glyph commands.
- the rules can be tailored to the identification of relatively complex alphanumeric characters and, as the case may be, custom glyphs.
- Computing device 700 includes a processor 702 , memory 704 , a storage device 706 , a high-speed interface 708 connecting to memory 704 and high-speed expansion ports 710 , and a low speed interface 712 connecting to low speed bus 714 and storage device 706 .
- Each of the components 702 , 704 , 706 , 708 , 710 , and 712 are interconnected using various busses, and may be mounted on a common motherboard or in other manners as appropriate.
- the processor 702 can process instructions for execution within the computing device 700 , including instructions stored in the memory 704 or on the storage device 706 to display graphical information for a GUI on an external input/output device, such as display 716 coupled to high speed interface 708 .
- multiple processors and/or multiple buses may be used, as appropriate, along with multiple memories and types of memory.
- multiple computing devices 700 may be connected, with each device providing portions of the necessary operations (e.g., as a server bank, a group of blade servers, or a multi-processor system).
- the memory 704 stores information within the computing device 700 .
- the memory 704 is a volatile memory unit or units.
- the memory 704 is a non-volatile memory unit or units.
- the memory 704 may also be another form of computer-readable medium, such as a magnetic or optical disk.
- the storage device 706 is capable of providing mass storage for the computing device 700 .
- the storage device 706 may be or contain a computer-readable medium, such as a floppy disk device, a hard disk device, an optical disk device, or a tape device, a flash memory or other similar solid state memory device, or an array of devices, including devices in a storage area network or other configurations.
- a computer program product can be tangibly embodied in an information carrier.
- the computer program product may also contain instructions that, when executed, perform one or more methods, such as those described above.
- the information carrier is a computer- or machine-readable medium, such as the memory 704 , the storage device 706 , memory on processor 702 , or a propagated signal.
- the high speed controller 708 manages bandwidth-intensive operations for the computing device 700 , while the low speed controller 712 manages lower bandwidth-intensive operations.
- the high-speed controller 708 is coupled to memory 704 , display 716 (e.g., through a graphics processor or accelerator), and to high-speed expansion ports 710 , which may accept various expansion cards (not shown).
- low-speed controller 712 is coupled to storage device 706 and low-speed expansion port 714 .
- the low-speed expansion port which may include various communication ports (e.g., USB, Bluetooth, Ethernet, wireless Ethernet) may be coupled to one or more input/output devices, such as a keyboard, a pointing device, a scanner, or a networking device such as a switch or router, e.g., through a network adapter.
- input/output devices such as a keyboard, a pointing device, a scanner, or a networking device such as a switch or router, e.g., through a network adapter.
- the computing device 700 may be implemented in a number of different forms, as shown in the figure. For example, it may be implemented as a standard server 720 , or multiple times in a group of such servers. It may also be implemented as part of a rack server system 724 . In addition, it may be implemented in a personal computer such as a laptop computer 722 . Alternatively, components from computing device 700 may be combined with other components in a mobile device (not shown), such as device 750 . Each of such devices may contain one or more of computing device 700 , 750 , and an entire system may be made up of multiple computing devices 700 , 750 communicating with each other.
- Computing device 750 includes a processor 752 , memory 764 , an input/output device such as a display 754 , a communication interface 766 , and a transceiver 768 , among other components.
- the device 750 may also be provided with a storage device, such as a microdrive or other device, to provide additional storage.
- a storage device such as a microdrive or other device, to provide additional storage.
- Each of the components 750 , 752 , 764 , 754 , 766 , and 768 are interconnected using various buses, and several of the components may be mounted on a common motherboard or in other manners as appropriate.
- the processor 752 can execute instructions within the computing device 750 , including instructions stored in the memory 764 .
- the processor may be implemented as a chipset of chips that include separate and multiple analog and digital processors.
- the processor may provide, for example, for coordination of the other components of the device 750 , such as control of user interfaces, applications run by device 750 , and wireless communication by device 750 .
- Processor 752 may communicate with a user through control interface 758 and display interface 756 coupled to a display 754 .
- the display 754 may be, for example, a TFT LCD (Thin-Film-Transistor Liquid Crystal Display) or an OLED (Organic Light Emitting Diode) display, or other appropriate display technology.
- the display interface 756 may comprise appropriate circuitry for driving the display 754 to present graphical and other information to a user.
- the control interface 758 may receive commands from a user and convert them for submission to the processor 752 .
- an external interface 762 may be provide in communication with processor 752 , so as to enable near area communication of device 750 with other devices. External interface 762 may provide, for example, for wired communication in some implementations, or for wireless communication in other implementations, and multiple interfaces may also be used.
- the memory 764 stores information within the computing device 750 .
- the memory 764 can be implemented as one or more of a computer-readable medium or media, a volatile memory unit or units, or a non-volatile memory unit or units.
- Expansion memory 774 may also be provided and connected to device 750 through expansion interface 772 , which may include, for example, a SIMM (Single In Line Memory Module) card interface.
- SIMM Single In Line Memory Module
- expansion memory 774 may provide extra storage space for device 750 , or may also store applications or other information for device 750 .
- expansion memory 774 may include instructions to carry out or supplement the processes described above, and may include secure information also.
- expansion memory 774 may be provide as a security module for device 750 , and may be programmed with instructions that permit secure use of device 750 .
- secure applications may be provided via the SIMM cards, along with additional information, such as placing identifying information on the SIMM card in a non-hackable manner.
- the memory may include, for example, flash memory and/or NVRAM memory, as discussed below.
- a computer program product is tangibly embodied in an information carrier.
- the computer program product contains instructions that, when executed, perform one or more methods, such as those described above.
- the information carrier is a computer- or machine-readable medium, such as the memory 764 , expansion memory 774 , memory on processor 752 , or a propagated signal that may be received, for example, over transceiver 768 or external interface 762 .
- Device 750 may communicate wirelessly through communication interface 766 , which may include digital signal processing circuitry where necessary. Communication interface 766 may provide for communications under various modes or protocols, such as GSM voice calls, SMS, EMS, or MMS messaging, CDMA, TDMA, PDC, WCDMA, CDMA2000, or GPRS, among others. Such communication may occur, for example, through radio-frequency transceiver 768 . In addition, short-range communication may occur, such as using a Bluetooth, WiFi, or other such transceiver (not shown). In addition, GPS (Global Positioning System) receiver module 770 may provide additional navigation- and location-related wireless data to device 750 , which may be used as appropriate by applications running on device 750 .
- GPS Global Positioning System
- the computing device 750 may be implemented in a number of different forms, as shown in the figure. For example, it may be implemented as a cellular telephone 780 . It may also be implemented as part of a smartphone 782 , personal digital assistant, or other similar mobile device.
- Device 750 may also include one or more different devices that are capable of sensing motion. Examples include, but are not limited to, accelerometers and compasses. Accelerometers and compasses, or other devices that are capable of detecting motion or position are available from any number of vendors and can sense motion in a variety of ways. For example, accelerometers can detect changes in acceleration while compasses can detect changes in orientation respective to the magnetic North or South Pole. These changes in motion can be detected by the device 750 and used to update the display of the respective devices 750 according to processes and techniques described herein.
- implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, specially designed ASICs (application specific integrated circuits), computer hardware, firmware, software, and/or combinations thereof.
- ASICs application specific integrated circuits
- These various implementations can include implementation in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.
- the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer.
- a display device e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor
- a keyboard and a pointing device e.g., a mouse or a trackball
- Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.
- the systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a client computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components.
- the components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (“LAN”), a wide area network (“WAN”), and the Internet.
- LAN local area network
- WAN wide area network
- the Internet the global information network
- the computing system can include clients and servers.
- a client and server are generally remote from each other and typically interact through a communication network.
- the relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.
Abstract
Description
- This specification relates to translating user interaction with a touch screen into input commands.
- People spends hours at a time with their electronic devices—computers, telephones, music players, PDA's, and the like. They like best those devices that are intuitive to use and whose interactions best meet their expectations regarding how machines should work. They interact with electronics through inputs and outputs from the devices, where the outputs generally are provided audibly and/or on a graphical display screen, and the inputs may occur via touch screens, joysticks, mice, 4-directional keypads, and other such input mechanisms.
- Touch screens are graphical display devices that act as both inputs and outputs. In particular, touch screens include a graphical display and detect the presence and location of manual interaction with that graphical display. A user can thus interact with the elements shown on the graphical display intuitively, in ways that resemble the ways that humans interact with the world outside of graphical displays and electronic devices.
- Although dramatic advances have been made in touch screen technology, the variety of commands that could potentially be input into a touch screen is much larger than the commands that can be distinguished by touch screens using present techniques. For example, an electronic device may be able to receive input that commands some portion of the displayed elements be scrolled or panned. Scrolling is the movement of elements shown on a display screen, generally either in either a horizontal or vertical direction. For example, a list of contacts can be scrolled from top to bottom on the display screen of a cell phone so that some contacts appear to move off of the bottom display screen as other contacts appear to move onto the top of the display screen. Panning is the movement of the portion of a map shown in on a display screen. Panning does not change the arrangement of features shown on the map but rather moves the user's field of view across what appears to be a larger image of the map.
- In many touch screens, a user commands the graphical display to scroll or pan by dragging a finger across the display. In response, the graphical display will scroll or pan in the same direction. Although such interaction is intuitively easy for people to understand, a touch screen that interprets all finger dragging as a scroll or pan command is unable to correctly interpret other commands that may resemble scroll or pan commands.
- Accordingly, systems and techniques that allow touch screen systems to translate user interaction into a relatively larger array of input commands are described. In one aspect, a method includes a touch screen system identifying stationary contact with a touch screen graphical display, the touch screen system starting a timer in response to the identification of the stationary contact, the touch screen system determining whether a second user interaction with the touch screen graphical display is timely using the timer, and in response to the second user interaction being timely, the touch screen system interpreting the second user interaction as a glyph.
- This and other aspects can include one or more of the following features. The touch screen system can interpret the stationary contact as a zoom-in command in response to the second user interaction being untimely. The method can also include, in response to the second user interaction being untimely, the touch screen system interpreting the stationary contact as a command directing that that further details regarding an entry contacted by the stationary contact. Interpreting the second user interaction as a glyph can include interpreting the second user interaction under a set of rules tailored to the identification of alphanumeric characters.
- Identifying the stationary contact can include determining that the stationary contact is in contact with an edge of the touch screen graphical display. Interpreting the second user interaction as a glyph can include disabling of finger dragging interactions as scrolling or panning commands. The method can also include the touch screen system pinning the touch screen graphical display in response to identification of the stationary contact. The method can also include the touch screen system changing to a new translation mode in response to the second user interaction being timely. The new translation mode can include a glyph translation mode tailored to interpret user interaction involving multiple finger drags across the touch screen graphical display as glyphs.
- In another aspect, a touch screen system includes a touch screen graphical display, a command interpreter configured to identify modifier interactions with the graphical display and, in response to identification of the modifier interaction, modify the interpretation of other interaction with the graphical display, one or more persistent data storage devices. The data storage devices store a first set of rules for interpreting user interaction with the graphical display and a second set of rules for interpreting user interaction with the graphical display. The first set of rules interpret motion across a map or a proper subset of a collection of information as a scrolling or panning command. The second set of rules interpret the motion as a glyph. The command interpreter is configured to modify the interpretation of the other interaction by switching between interpretation under the first set of rules and interpretation under the second set of rules.
- This and other aspects can include one or more of the following features. The touch screen system can include a collection of map information stored at the one or more persistent data storage devices. The touch screen system can also include a hand-held housing mounting the touch screen graphical display. The command interpreter can be configured to identify contact on a lateral side of the touch screen graphical display as modifier interactions. The command interpreter can be configured to identify stationary contact followed by a timely second contact as modifier interactions. The second set of rules can include rules for interpreting the motion as a custom glyph. The command interpreter can be configured to identify a pinning command, the pinning command disabling a touch screen system from interpreting finger dragging as a scroll command or as a pan command. The touch screen system can also include a clock. The command interpreter can be configured to start a timer that relies on the clock in response to identification of a stationary contact with the graphical display.
- The command interpreter can be configured to identify a modifier interaction in response to a second user interaction being received before the timer reaches a threshold period. The threshold period can be within 15% of one second. The command interpreter can be configured to interpret the stationary contact as a zoom-in command in response to a failure to receive a second user interaction before the timer reaches a threshold period. The command interpreter can be configured to interpret the stationary contact as a command directing that that further details regarding an entry contacted by the stationary contact in response to a failure to receive a second user interaction before the timer reaches a threshold period.
- In another aspect, a touch screen system includes a touch screen graphical display, means for translating a same user interaction with the touch screen graphical display into two different input commands, and one or more persistent data storage devices storing rules accessible by the means for translating.
- This and other aspects can include one or more of the following features. The touch means for translating can include a clock and a timer that relies on the clock and configured to start in response to identification of a stationary contact with the touch screen graphical display.
- The details of one or more implementations are set forth in the accompanying drawings and the description below. Other features and advantages will be apparent from the description and drawings, and from the claims.
-
FIG. 1 is a schematic diagram of a touch screen system. -
FIGS. 2-9 are schematic representations of user interaction with touch screen systems. -
FIGS. 10-12 are flow charts of processes that can be performed by touch screen systems to translate user interaction into input commands -
FIG. 13 shows an example of generic devices that may be used with the described techniques. - Like reference symbols in the various drawings indicate like elements.
-
FIG. 1 is a schematic diagram of atouch screen system 100.Touch screen system 100 includes ahousing 105 onto which a touch screengraphical display 110 is moveably mounted. Touch screengraphical display 110 is moveable to expose a collection ofinput keys 110.Housing 105 houses a collection ofelectronic components 120 that cooperate to provide the functionality needed to operatetouch screen system 100, including the ability to distinguish different commands input over touch screengraphical display 110.Touch screen system 100 can be, e.g., a computer, a telephone, a music player, a PDA, or the like. In some implementations,touch screen system 100 can be mobile, hand-held device, as shown. -
Electronic component collection 120 includes adisplay manager 125, a drawnuser input controller 130, aninput manager 135, and aclock 155.Display manager 125, drawnuser input controller 130, andinput manager 135 can be implemented using digital electronic circuitry, or in computer software, firmware, or hardware.Clock 155 can be an integral part of one or more ofdisplay manager 125, drawnuser input controller 130, andinput manager 135 orclock 155 can be a separate component, as shown. -
Display manager 125 is a device that is configured to render content for presentation on touch screengraphical display 110.Display manager 125 can receive graphic-related content from a number of sources and may determine how the content is to be provided to a user. For example, different windows forvarious applications 140 may need to be displayed, and thedisplay manager 125 can determine which to display, which to hide, and what to display or hide when there is overlap between various graphical objects.Display manager 125 can include various components to provide particular functionality for interacting with displayed components, which may be shared across multiple applications, and may be supplied, for example, by an operating system. - Drawn
user input controller 130 is a device that is configured to translate user interaction with touch screengraphical display 110 into commands. Drawnuser input controller 130 can thus distinguish between different commands input over touch screengraphical display 110. In distinguishing between different commands, drawnuser input controller 130 may refer to acommand interpreter 145.Command interpreter 145 implements sets ofrules 150 for interpreting user interaction with touch screengraphical display 110 as commands. In some implementations, rule sets 150 can include both default and user-specific rules for translating user interaction into commands and can be stored in a data storage device. -
Input manager 135 is a device that is configured to implement the commands embodied by user interaction with various input elements oftouch screen system 100—includinggraphical display 110—using control signals that can be interpreted by other components incollection 120. For example, such commands may come frominput keys 115, from touch screengraphical display 110, from a trackball, or from other such sources, including dedicated buttons or soft buttons (e.g., buttons whose functions may change over time, and whose functions may be displayed on areas ofgraphical display 110 that are adjacent to the particular buttons.Input manager 135 can determine, for example, in what area of the display commands are being received, and thus in what application being shown on the display the commands are intended for. - In some implementations,
input manager 135 may translate input motions ongraphical display 110 into a common format and pass those interpreted motions (e.g., short press, long press, flicks, and straight-line drags) to the appropriate application.Input manager 135 can also report such inputs to an event manager (not shown) that in turn reports them to the appropriate modules or applications. - In some implementations,
input manager 135 can include one or more maps for implementing the interpretations of the user interaction with touch screengraphical display 110 using control signals that can be interpreted by other components incollection 120. In some implementations, different maps are specific todifferent applications 140. In this manner, the controller 417 can act as an intermediary between input elements andapplications 140. - A variety of
applications 140 may operate, generally on a common microprocessor, ontouch screen system 100. Theapplications 140 may take a variety of forms, such as mapping applications, e-mail and other messaging applications, web browser applications, music and video players, and various applications running within a web browser or running extensions of a web browser.Applications 140 may include applications that stored records or other components having alphanumeric names. - In some implementations,
electronic component collection 120 also includes a wireless interface. A wireless interface is a device configured to manage communication with a wireless network, which may be a data network that also carries voice communications. A wireless interface can provide for communication by thetouch screen system 100 with messaging services such as text messaging, e-mail, and telephone voice mail messaging. In addition, the wireless interface may support downloads and uploads of content and computer code over a wireless network. - Various forms of persistent data storage may be provided in
touch screen system 100, such as using fixed disk drives and/or solid state memory devices. For example, a lists, maps, etc.storage 160 can store various definitions for objects that may be stored on and presented bytouch screen system 100. Such objects include lists of other objects, such as lists of songs, television shows, movies, or other media on the device. The lists may also include lists of file names wheretouch screen system 100 stores a large number of files. Also, other objects such as maps may be stored (e.g., as graphical map tiles that can be pre-fetched from a remote server), and meta data about the maps, such as the names of towns and streets on a map, along with the names of points of interest and business on the maps. - In some implementations,
storage 160 can also store profile information for a user. Profile information includes various parameters about a user oftouch screen system 100. For example, the profile information may include data defining various custom commands that a user has entered, along with objects or actions to be associated with the entry of those commands. Profile information can be stored in a variety of forms, such as in one or more XML files or in Sqlite files. - Using the pictured components, and others that are omitted here for clarity, the
touch screen system 100 may distinguish different commands input withgraphical display 110. -
FIGS. 2-4 are schematic representations of user interaction with a touch screen system such as touch screen system 100 (FIG. 1 ). InFIG. 2 , the touch screen system includes a touch screengraphical display 200 that displays aproper subset 210 of a collection of information for a user to see. In particular, a collection ofentries graphical display 200. Eachentry - The touch screen system will interpret user interaction with
graphical display 200 according to a set of rules. Typically, these rules are tailored to allow a user to navigate through the collection of information. Referring toFIG. 3 , the touch screen system translates the dragging of afinger 305 in a generally downward direction acrossgraphical display 200 into a scroll down command. This movement offinger 305 is represented by anarrow 310 which, in general, is not actually displayed ongraphical display 200. In response to the scroll down command,entries graphical display 200, whereasnew entries graphical display 200. Thus, a differentproper subset 330 of the collection of identification information is displayed for a user to see. - This response to the dragging of
finger 305 is effective for scrolling through a collection of information and intuitively easy for users to understand. However, always translating the dragging offinger 305 in this manner limits the variety of commands that could potentially be input viagraphical display 200.FIG. 4 illustrates an example situation in which translating the dragging offinger 305 in this manner limits the variety of commands that can be input. In particular, a user is draggingfinger 305 acrossgraphical display 200 in a generally L-shaped pattern. This movement offinger 305 is represented by an L-shapedarrow 405 which, in general, is not actually displayed on graphical display 200: - The touch screen system may interpret this L-shaped pattern as including a scroll down command. In particular, the downward stroke of the finger in the first part of the L-shaped pattern can be mistaken for a downward scroll command. The sideways stroke of the finger in the second part of the L-shaped pattern can be ignored or taken as a command that is independent of the downward scroll command. Thus, the touch screen system may not be able to distinguish between a downward scroll command and a second command embodied by an L-shaped pattern. This limits the number of commands that can be input over
graphical display 200. - Such difficulties are also apparent during user interaction with touch screen graphical displays that display maps.
FIGS. 5-7 are schematic representations of user interaction with a touch screen system such as touch screen system 100 (FIG. 1 ). InFIG. 5 , the touch screen system includes a touch screengraphical display 500 that displays asection 505 of a map. Insection 505, an arrow indicium identifies the city Minneapolis in the top left corner ofgraphical display 500. The touch screen system will translate user interaction withgraphical display 200 into input commands according to a set of rules. Typically, these rules are tailored to allow a user to change the map display. Referring toFIG. 6 , the touch screen system interprets the dragging of afinger 305 in a generally downward direction acrossgraphical display 500 as a pan down command. This movement offinger 305 is represented by anarrow 605 which, in general, is not actually displayed ongraphical display 500. In response to the pan down command, the bottom portion of the map appear to move off the bottom of the window ingraphical display 500, whereas a new portion of the map appears to enter the top of the window ingraphical display 500. The arrow indicium identifying the city Minneapolis moves downward and adifferent section 610 of the map is displayed for a user to see. - Referring to
FIG. 7 , the touch screen system further interprets the dragging of afinger 305 sideways acrossgraphical display 500 as a pan sideways command. This movement offinger 305 is represented by the lower, sideways-directed portion ofarrow 605. In response to the pan sideways command, the rightmost portion of the map appear to move off the right of the window ingraphical display 500, whereas a new portion of the map appears to enter the left of the window ingraphical display 500. The arrow indicium identifying the city Minneapolis moves rightward ongraphical display 500 and a different section 705 of the map is displayed for a user to see. - Although it may sometimes be appropriate to interpret this L-shaped pattern as a pair of panning commands, such an interpretation limits the number of commands that can be input over
graphical display 200. -
FIG. 8 is a schematic representations of user interaction with a touch screen system such as touch screen system 100 (FIG. 1 ) in which additional number of commands can be input over a graphical display. In particular, the user has placed athumb 805 on the lateral side ofgraphical display 200 while draggingfinger 305 in a generally downward direction acrossgraphical display 200. The touch screen system does not interpret this dragging offinger 305 as a scroll down command. Instead, the touch screen system translates the combination of the placement ofthumb 805 and the dragging offinger 305 into a new command altogether, e.g., as the input of a glyph command. -
FIG. 9 is a schematic representations of user interaction with a touch screen system such as touch screen system 100 (FIG. 1 ) in which additional number of commands can be input over a graphical display. In particular, the user ha's placed athumb 805 on the lateral side ofgraphical display 500 while draggingfinger 305 in a generally L-shaped pattern acrossgraphical display 500. With this combination, the touch screen system does not interpret this movement offinger 305 as a pair of panning commands. Instead, the touch screen system translates the combination of the placement ofthumb 805 and the dragging offinger 305 into a new command altogether, e.g., as the input of a glyph command. - The placement of
thumb 805 on the lateral side ofgraphical displays graphical displays thumb 805 thus modifies the translation of the movement offinger 305. For example, in the absence ofthumb 805 ongraphical displays finger 305 in accordance with a first set of rules 150 (FIG. 1 ). Withthumb 805 ongraphical displays finger 305 in accordance with a second set of rules 150 (FIG. 1 ). In some implementations, the representation ongraphical display 500 can be completely immobile while pinned. - In some implementations, the touch screen systems are configured to interpret the movements of
finger 305 while pinned bythumb 805 as “glyphs.” A glyph is a visual writing element that a user can enter, such as by dragging their finger on a touchscreen, generally in a single traced motion without lifting their finger. Glyphs can include alphanumeric characters and other symbols. - Glyphs can be construed by a touch screen system in a number of different ways. For example, a glyph can be construed as a command to display a proper subset of a collection of information that start or are otherwise associated with a glyph. By way of example, a user who is looking at a long list of items such as a list of songs in a playlist, may draw a letter of the alphabet on the display. In response, a touch screen system can display entries that start with that letter.
- As another example, a glyph can be construed as a command to display a certain portion of a map. By way of example, a user who is viewing a portion of a map may trace a letter onto the screen of their device. In response, the touch screen system can display a section of a map that includes the nearest city that starts with that letter.
- A user may also be permitted to assign one or more glyphs to particular commands or objects on a computer system. For example, a user may open a contacts entry for a friend and may select a control for submitting a glyph. The user may then trace any glyph that he or she prefers (e.g., the first letter of the friend's nickname or a simple shape) to have that glyph assigned to the friend's contact record. If the user enters that same glyph later while using the touch screen system, the touch screen system can construe the glyph as a command to display the contact record. Multiple glyphs may also be assigned to the same object so that the user has multiple ways to get to such an object.
- A single glyph can also be assigned to multiple objects. In such a situation, upon receipt of the glyph from a user, the touch screen system can construe the glyph as a common to display a pop up control that lists the multiple objects. Alternatively, the proper object may be selected automatically by the system based on the context in which the glyph was entered. For example, if the glyph is entered when an address book application is active, the touch screen system can construe the glyph as a command to jump to a contact entry corresponding to the glyph rather than a music playlist corresponding to the glyph. Where the context could refer to multiple objects (e.g., if the glyph is entered on a desktop), the objects may be prioritized according to the object that is most likely to be selected (e.g., because it is frequently accessed by the user of the device, or because it is a type of object that is generally accessed more than another type of object).
-
FIG. 10 is a flow chart of aprocess 1000 that can be performed by a touch screen system to translate user interaction into input commands.Process 1000 can be performed by one or more digital data processing devices that perform operations by executing one or more sets of machine-readable instructions. For example,process 1000 can be performed bycommand interpreter 145 in touch screen system 100 (FIG. 1 ).Process 1000 can be performed in isolation or in conjunction with other digital data processing operations. - The
system performing process 1000 can receive a first user interaction with the graphical display of a touch screen system atbox 1005. The system then determines whether the first user interaction is a modifier interaction atdecision 1010. A modifier interaction is user interaction with the touch screen graphical display that modifies the interpretation of other interaction with the graphical display, e.g., when the first and the second interaction occur at the same time. In the context ofFIGS. 2-8 , the positioning ofthumb 805 on the lateral side ofgraphical displays finger 305 acrossdisplays - A modifier interaction can be identified based on the characteristics of the user interaction with the graphical display. For example, in the context of
FIGS. 2-8 , the positioning ofthumb 805 on the lateral side ofgraphical displays thumb 805 relative to—or even in contact with—the lateral side of thegraphical displays thumb 805 can also be used to identify a modifier interaction. Further, as described further below, the magnitude of the time delay between the positioning ofthumb 805 and the dragging offinger 305 can be used to identify a modifier interaction. - In response to determining that the first user interaction is not a modifier interaction, the
system performing process 1000 interprets the first user interaction under a first set of interpretation rules atbox 1015. The first set of interpretation rules can be a default set of rules, in much the same way that the letters on a keyboard are interpreted, by default, and lower case letters unless the shift key is pressed. - In response to determining that the first user interaction is indeed a modifier interaction, the
system performing process 1000 receives second user interaction atbox 1020 and interprets that second user interaction under a second set of interpretation rules atbox 1025. In some implementations, the second set of interpretation rules can disable scrolling or panning commands. For example, in the context ofFIGS. 2-8 , the second set of interpretation rules can allow second user interaction withgraphical displays -
FIG. 11 is a flow chart of aprocess 1100 that can be performed by a touch screen system to translate user interaction into input commands.Process 1100 can be performed by one or more digital data processing devices that perform operations by executing one or more sets of machine-readable instructions. For example,process 1100 can be performed bycommand interpreter 145 in touch screen system 100 (FIG. 1 ).Process 1100 can be performed in isolation or in conjunction with other digital data processing operations. - The
system performing process 1100 can receive a first user interaction with the graphical display of a touch screen system atbox 1105. The system then identifies that the first user interaction is a pinning command atbox 1110. A pinning command is a command that disables a touch screen system from interpreting finger dragging as a scroll or a pan command even if a subset of a collection of information or a map is displayed. For example, in the context ofFIGS. 2-8 , the positioning ofthumb 805 on the lateral side ofgraphical displays finger 305 as a scrolling or panning commands. In some implementations, the magnitude of the time delay between a pair of interactions can be used to identify a pinning command, as described further below. - The
system performing process 1100 can change to a new translation mode in response to the pinning command atbox 1115. Translation modes are states of a touch screen system that specify the interpretation of user interaction with the graphical display. For example, a touch screen system can interpret a user interaction as a first command while in a first mode but interpret the same user interaction as a second command while in a second mode. In other words, the translation of user interaction depends on the mode of the touch screen system. In some implementations, the different modes can be associated with different sets of rules for translating user interaction. - In some implementations, the new translation mode is a mode that is tailored to interpret user interaction as glyphs. For example, the new translation mode can interpret user interaction involving multiple finger drags in different directions across a graphical display. The new translation mode may also be able to interpret user interaction in which the user temporarily removes a finger from the graphical display in order to complete a glyph. For example, the new translation mode may allow a user to remove a finger during the input of a T-shaped glyph.
- In some implementations, the mode change is persistent even in the absence of continued user interaction. For example, in the context of
FIGS. 2-8 , once the touch screen system changes mode in response to the positioning ofthumb 805 on the lateral side ofgraphical displays thumb 805 fromgraphical displays graphical displays FIGS. 2-8 ,thumb 805 must remain positioned on the lateral side ofgraphical displays graphical displays - The
system performing process 1100 can receive a second user interaction with the graphical display of a touch screen system atbox 1120 and interpret the second user interaction in the new mode at 1125. The interpretation of the second user interaction in the new mode will generally differ from the interpretation of the second user interaction in the previous mode. For example, in the context ofFIGS. 2-8 , the interpretation of the dragging offinger 305 as glyphs (i.e., in the new mode) differs from the interpretation of the dragging offinger 305 as a scrolling or panning command. In some implementations, the interpretation of the second user interaction in the new mode can be done in accordance with a second set of rules. -
FIG. 12 is a flow chart of aprocess 1200 that can be performed by a touch screen system to translate user interaction into input commands.Process 1200 can be performed by one or more digital data processing devices that perform operations by executing one or more sets of machine-readable instructions. For example,process 1200 can be performed bycommand interpreter 145 in touch screen system 100 (FIG. 1 ).Process 1200 can be performed in isolation or in conjunction with other digital data processing operations. For example,process 1200 can be performed atdecision box 1010 in process 1000 (FIG. 10 ) or atbox 1110 in process 1100 (FIG. 11 ). - The
system performing process 1200 can identify stationary contact between a user's appendage and the graphical display of a touch screen system atbox 1205. For example, the system can identify that a user's thumb is stationary on the side of a graphical display. In general, the system will not require that the contact be completely motionless. Rather, the system will generally allow a limited range of motion to occur but yet consider the contact stationary. - In response to the identification of the stationary contact, the
system performing process 1200 can start a timer atbox 1210. The timer thus measures how long the stationary contact has occurred. The timer can be implemented using a clock signal provided by an internal or an external clock such as, e.g.,clock 155 of touch screen system 100 (FIG. 1 ). - The
system performing process 1200 can also determine whether a second user interaction with the graphical display of a touch screen system has been timely received atdecision box 1215. The timely receipt of the second interaction requires that the second interaction be received before a period measured by the timer has passed. For example, in some implementations, the second interaction must be received within one second, or within 600 ms, in order to be considered timely. In some implementations, the duration of the period is fixed. In other implementations, the duration of the period changes, e.g., in response to the amount of motion in the stationary contact. For example, if the stationary contact is relatively unstable with a large amount of motion, the duration of the period can be increased. On the other hand, if the stationary contact is relatively stable with a small amount of motion, the duration of the period can be decreased. - In response to determining that the second interaction was indeed timely received, the
system performing process 1200 interprets the second user interaction in accordance with a glyph rule set atbox 1220. A glyph rule set is a set of rules for translating user interaction with the graphical display of a touch screen system as glyph commands. The rules can be tailored to the identification of relatively complex alphanumeric characters and, as the case may be, custom glyphs. - In response to determining that the second interaction was not timely received, the
system performing process 1200 interprets the stationary contact, and any second user interaction, using a first rule set atbox 1225. The first rule set can translate the stationary contact, e.g., into a command directing that the graphic display zoom into the contacted portion of the map or into a command directing that that further details regarding a contacted entry be provided. If second—but untimely—user interaction was received, the second user interaction can also be interpreted using the first rule set. -
FIG. 13 shows an example of ageneric computer device 700 and a generic mobile computer device 750, which may be used with the techniques described here. In particular, one or both ofgeneric computer device 700 and generic mobile computer device 750 may be a touch screen system and include a touch screen graphical display.Computing device 700 is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers; mainframes, and other appropriate computers. Computing device 750 is intended to represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed in this document. -
Computing device 700 includes aprocessor 702,memory 704, a storage device 706, a high-speed interface 708 connecting tomemory 704 and high-speed expansion ports 710, and alow speed interface 712 connecting to low speed bus 714 and storage device 706. Each of thecomponents processor 702 can process instructions for execution within thecomputing device 700, including instructions stored in thememory 704 or on the storage device 706 to display graphical information for a GUI on an external input/output device, such asdisplay 716 coupled tohigh speed interface 708. In other implementations, multiple processors and/or multiple buses may be used, as appropriate, along with multiple memories and types of memory. Also,multiple computing devices 700 may be connected, with each device providing portions of the necessary operations (e.g., as a server bank, a group of blade servers, or a multi-processor system). - The
memory 704 stores information within thecomputing device 700. In one implementation, thememory 704 is a volatile memory unit or units. In another implementation, thememory 704 is a non-volatile memory unit or units. Thememory 704 may also be another form of computer-readable medium, such as a magnetic or optical disk. - The storage device 706 is capable of providing mass storage for the
computing device 700. In one implementation, the storage device 706 may be or contain a computer-readable medium, such as a floppy disk device, a hard disk device, an optical disk device, or a tape device, a flash memory or other similar solid state memory device, or an array of devices, including devices in a storage area network or other configurations. A computer program product can be tangibly embodied in an information carrier. The computer program product may also contain instructions that, when executed, perform one or more methods, such as those described above. The information carrier is a computer- or machine-readable medium, such as thememory 704, the storage device 706, memory onprocessor 702, or a propagated signal. - The
high speed controller 708 manages bandwidth-intensive operations for thecomputing device 700, while thelow speed controller 712 manages lower bandwidth-intensive operations. Such allocation of functions is exemplary only. In one implementation, the high-speed controller 708 is coupled tomemory 704, display 716 (e.g., through a graphics processor or accelerator), and to high-speed expansion ports 710, which may accept various expansion cards (not shown). In the implementation, low-speed controller 712 is coupled to storage device 706 and low-speed expansion port 714. The low-speed expansion port, which may include various communication ports (e.g., USB, Bluetooth, Ethernet, wireless Ethernet) may be coupled to one or more input/output devices, such as a keyboard, a pointing device, a scanner, or a networking device such as a switch or router, e.g., through a network adapter. - The
computing device 700 may be implemented in a number of different forms, as shown in the figure. For example, it may be implemented as a standard server 720, or multiple times in a group of such servers. It may also be implemented as part of a rack server system 724. In addition, it may be implemented in a personal computer such as a laptop computer 722. Alternatively, components fromcomputing device 700 may be combined with other components in a mobile device (not shown), such as device 750. Each of such devices may contain one or more ofcomputing device 700, 750, and an entire system may be made up ofmultiple computing devices 700, 750 communicating with each other. - Computing device 750 includes a processor 752,
memory 764, an input/output device such as a display 754, a communication interface 766, and atransceiver 768, among other components. The device 750 may also be provided with a storage device, such as a microdrive or other device, to provide additional storage. Each of thecomponents - The processor 752 can execute instructions within the computing device 750, including instructions stored in the
memory 764. The processor may be implemented as a chipset of chips that include separate and multiple analog and digital processors. The processor may provide, for example, for coordination of the other components of the device 750, such as control of user interfaces, applications run by device 750, and wireless communication by device 750. - Processor 752 may communicate with a user through control interface 758 and
display interface 756 coupled to a display 754. The display 754 may be, for example, a TFT LCD (Thin-Film-Transistor Liquid Crystal Display) or an OLED (Organic Light Emitting Diode) display, or other appropriate display technology. Thedisplay interface 756 may comprise appropriate circuitry for driving the display 754 to present graphical and other information to a user. The control interface 758 may receive commands from a user and convert them for submission to the processor 752. In addition, an external interface 762 may be provide in communication with processor 752, so as to enable near area communication of device 750 with other devices. External interface 762 may provide, for example, for wired communication in some implementations, or for wireless communication in other implementations, and multiple interfaces may also be used. - The
memory 764 stores information within the computing device 750. Thememory 764 can be implemented as one or more of a computer-readable medium or media, a volatile memory unit or units, or a non-volatile memory unit or units. Expansion memory 774 may also be provided and connected to device 750 through expansion interface 772, which may include, for example, a SIMM (Single In Line Memory Module) card interface. Such expansion memory 774 may provide extra storage space for device 750, or may also store applications or other information for device 750. Specifically, expansion memory 774 may include instructions to carry out or supplement the processes described above, and may include secure information also. Thus, for example, expansion memory 774 may be provide as a security module for device 750, and may be programmed with instructions that permit secure use of device 750. In addition, secure applications may be provided via the SIMM cards, along with additional information, such as placing identifying information on the SIMM card in a non-hackable manner. - The memory may include, for example, flash memory and/or NVRAM memory, as discussed below. In one implementation, a computer program product is tangibly embodied in an information carrier. The computer program product contains instructions that, when executed, perform one or more methods, such as those described above. The information carrier is a computer- or machine-readable medium, such as the
memory 764, expansion memory 774, memory on processor 752, or a propagated signal that may be received, for example, overtransceiver 768 or external interface 762. - Device 750 may communicate wirelessly through communication interface 766, which may include digital signal processing circuitry where necessary. Communication interface 766 may provide for communications under various modes or protocols, such as GSM voice calls, SMS, EMS, or MMS messaging, CDMA, TDMA, PDC, WCDMA, CDMA2000, or GPRS, among others. Such communication may occur, for example, through radio-
frequency transceiver 768. In addition, short-range communication may occur, such as using a Bluetooth, WiFi, or other such transceiver (not shown). In addition, GPS (Global Positioning System) receiver module 770 may provide additional navigation- and location-related wireless data to device 750, which may be used as appropriate by applications running on device 750. - Device 750 may also communicate audibly using audio codec 760, which may receive spoken information from a user and convert it to usable digital information. Audio codec 760 may likewise generate audible sound for a user, such as through a speaker, e.g., in a handset of device 750. Such sound may include sound from voice telephone calls, may include recorded sound (e.g., voice messages, music files, etc.) and may also include sound generated by applications operating on device 750.
- The computing device 750 may be implemented in a number of different forms, as shown in the figure. For example, it may be implemented as a
cellular telephone 780. It may also be implemented as part of a smartphone 782, personal digital assistant, or other similar mobile device. - Device 750 may also include one or more different devices that are capable of sensing motion. Examples include, but are not limited to, accelerometers and compasses. Accelerometers and compasses, or other devices that are capable of detecting motion or position are available from any number of vendors and can sense motion in a variety of ways. For example, accelerometers can detect changes in acceleration while compasses can detect changes in orientation respective to the magnetic North or South Pole. These changes in motion can be detected by the device 750 and used to update the display of the respective devices 750 according to processes and techniques described herein.
- Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, specially designed ASICs (application specific integrated circuits), computer hardware, firmware, software, and/or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.
- These computer programs (also known as programs, software, software applications or code) include machine instructions for a programmable processor, and can be implemented in a high-level procedural and/or object-oriented programming language, and/or in assembly/machine language. As used herein, the terms “machine-readable medium” “computer-readable medium” refers to any computer program product, apparatus and/or device (e.g., magnetic discs, optical disks, memory, Programmable Logic Devices (PLDs)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term “machine-readable signal” refers to any signal used to provide machine instructions and/or data to a programmable processor.
- To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.
- The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a client computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (“LAN”), a wide area network (“WAN”), and the Internet.
- The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.
- A number of embodiments have been described. Nevertheless, it will be understood that various modifications may be made without departing from the spirit and scope of the invention. For example, much of this document has been described with respect to messaging and mapping applications, but other forms of graphical applications may also be addressed, such as interactive program guides, web page navigation and zooming, and other such applications.
- In addition, the logic flows depicted in the figures do not require the particular order shown, or sequential order, to achieve desirable results. In addition, other steps may be provided, or steps may be eliminated, from the described flows, and other components may be added to, or removed from, the described systems. Accordingly, other embodiments are within the scope of the following claims.
Claims (24)
Priority Applications (6)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US12/621,759 US8432367B2 (en) | 2009-11-19 | 2009-11-19 | Translating user interaction with a touch screen into input commands |
PCT/US2010/056400 WO2011062837A2 (en) | 2009-11-19 | 2010-11-11 | Translating user interaction with a touch screen into input commands |
KR1020127013089A KR101668398B1 (en) | 2009-11-19 | 2010-11-11 | Translating user interaction with a touch screen into input commands |
EP10832023.5A EP2502131A4 (en) | 2009-11-19 | 2010-11-11 | Translating user interaction with a touch screen into input commands |
CA2781347A CA2781347C (en) | 2009-11-19 | 2010-11-11 | Translating user interaction with a touch screen into input commands |
AU2010322267A AU2010322267B2 (en) | 2009-11-19 | 2010-11-11 | Translating user interaction with a touch screen into input commands |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US12/621,759 US8432367B2 (en) | 2009-11-19 | 2009-11-19 | Translating user interaction with a touch screen into input commands |
Publications (2)
Publication Number | Publication Date |
---|---|
US20110115721A1 true US20110115721A1 (en) | 2011-05-19 |
US8432367B2 US8432367B2 (en) | 2013-04-30 |
Family
ID=44010958
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/621,759 Active 2031-06-11 US8432367B2 (en) | 2009-11-19 | 2009-11-19 | Translating user interaction with a touch screen into input commands |
Country Status (6)
Country | Link |
---|---|
US (1) | US8432367B2 (en) |
EP (1) | EP2502131A4 (en) |
KR (1) | KR101668398B1 (en) |
AU (1) | AU2010322267B2 (en) |
CA (1) | CA2781347C (en) |
WO (1) | WO2011062837A2 (en) |
Cited By (55)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20110138321A1 (en) * | 2009-12-04 | 2011-06-09 | International Business Machines Corporation | Zone-based functions in a user interface |
US20110172902A1 (en) * | 2006-01-03 | 2011-07-14 | Strategic Design Federation W, Inc. | Computer-Aided Route Selection |
US20120169632A1 (en) * | 2010-12-31 | 2012-07-05 | Yu Dong-Won | Method and apparatus for performing processes in a user equipment by using touch patterns |
EP2549717A1 (en) * | 2011-07-19 | 2013-01-23 | Lg Electronics Inc. | Mobile terminal and controlling method thereof |
EP2584441A1 (en) * | 2011-10-18 | 2013-04-24 | Research In Motion Limited | Electronic device and method of controlling same |
US20130141463A1 (en) * | 2011-12-06 | 2013-06-06 | Microsoft Corporation | Combined interactive map and list view |
US8810535B2 (en) | 2011-10-18 | 2014-08-19 | Blackberry Limited | Electronic device and method of controlling same |
EP2664986A3 (en) * | 2012-05-14 | 2014-08-20 | Samsung Electronics Co., Ltd | Method and electronic device thereof for processing function corresponding to multi-touch |
US20150177927A1 (en) * | 2010-04-07 | 2015-06-25 | Apple Inc. | Device, method, and graphical user interface for managing concurrently open software applications |
US9109921B1 (en) * | 2013-06-19 | 2015-08-18 | Amazon Technologies, Inc. | Contextual based navigation element |
US9405453B1 (en) * | 2010-04-08 | 2016-08-02 | Twitter, Inc. | User interface mechanics |
US9513801B2 (en) | 2010-04-07 | 2016-12-06 | Apple Inc. | Accessing electronic notifications and settings icons with gestures |
US9569102B2 (en) | 2010-01-06 | 2017-02-14 | Apple Inc. | Device, method, and graphical user interface with interactive popup views |
US9602729B2 (en) | 2015-06-07 | 2017-03-21 | Apple Inc. | Devices and methods for capturing and interacting with enhanced digital images |
US9619076B2 (en) | 2012-05-09 | 2017-04-11 | Apple Inc. | Device, method, and graphical user interface for transitioning between display states in response to a gesture |
US9645732B2 (en) | 2015-03-08 | 2017-05-09 | Apple Inc. | Devices, methods, and graphical user interfaces for displaying and using menus |
US9658740B2 (en) | 2013-03-15 | 2017-05-23 | Apple Inc. | Device, method, and graphical user interface for managing concurrently open software applications |
US9674426B2 (en) | 2015-06-07 | 2017-06-06 | Apple Inc. | Devices and methods for capturing and interacting with enhanced digital images |
US9753639B2 (en) | 2012-05-09 | 2017-09-05 | Apple Inc. | Device, method, and graphical user interface for displaying content associated with a corresponding affordance |
US9778771B2 (en) | 2012-12-29 | 2017-10-03 | Apple Inc. | Device, method, and graphical user interface for transitioning between touch input to display output relationships |
US9785305B2 (en) | 2015-03-19 | 2017-10-10 | Apple Inc. | Touch input cursor manipulation |
US20170315718A1 (en) * | 2016-04-27 | 2017-11-02 | Kyocera Document Solutions Inc. | Handwritten character input device, image forming apparatus and handwritten character input method |
US9823831B2 (en) | 2010-04-07 | 2017-11-21 | Apple Inc. | Device, method, and graphical user interface for managing concurrently open software applications |
US9823839B2 (en) | 2012-05-09 | 2017-11-21 | Apple Inc. | Device, method, and graphical user interface for displaying additional information in response to a user contact |
US9886184B2 (en) | 2012-05-09 | 2018-02-06 | Apple Inc. | Device, method, and graphical user interface for providing feedback for changing activation states of a user interface object |
US9959025B2 (en) | 2012-12-29 | 2018-05-01 | Apple Inc. | Device, method, and graphical user interface for navigating user interface hierarchies |
US9990121B2 (en) | 2012-05-09 | 2018-06-05 | Apple Inc. | Device, method, and graphical user interface for moving a user interface object based on an intensity of a press input |
US9996231B2 (en) | 2012-05-09 | 2018-06-12 | Apple Inc. | Device, method, and graphical user interface for manipulating framed graphical objects |
US10007400B2 (en) | 2010-12-20 | 2018-06-26 | Apple Inc. | Device, method, and graphical user interface for navigation of concurrently open software applications |
US10042542B2 (en) | 2012-05-09 | 2018-08-07 | Apple Inc. | Device, method, and graphical user interface for moving and dropping a user interface object |
US10048757B2 (en) | 2015-03-08 | 2018-08-14 | Apple Inc. | Devices and methods for controlling media presentation |
US10078442B2 (en) | 2012-12-29 | 2018-09-18 | Apple Inc. | Device, method, and graphical user interface for determining whether to scroll or select content based on an intensity theshold |
US10095396B2 (en) | 2015-03-08 | 2018-10-09 | Apple Inc. | Devices, methods, and graphical user interfaces for interacting with a control object while dragging another object |
US10095391B2 (en) | 2012-05-09 | 2018-10-09 | Apple Inc. | Device, method, and graphical user interface for selecting user interface objects |
US10126930B2 (en) | 2012-05-09 | 2018-11-13 | Apple Inc. | Device, method, and graphical user interface for scrolling nested regions |
US10162452B2 (en) | 2015-08-10 | 2018-12-25 | Apple Inc. | Devices and methods for processing touch inputs based on their intensities |
US10175757B2 (en) | 2012-05-09 | 2019-01-08 | Apple Inc. | Device, method, and graphical user interface for providing tactile feedback for touch-based operations performed and reversed in a user interface |
US10175864B2 (en) | 2012-05-09 | 2019-01-08 | Apple Inc. | Device, method, and graphical user interface for selecting object within a group of objects in accordance with contact intensity |
US10310732B2 (en) | 2013-03-15 | 2019-06-04 | Apple Inc. | Device, method, and graphical user interface for concurrently displaying a plurality of settings controls |
US10437333B2 (en) | 2012-12-29 | 2019-10-08 | Apple Inc. | Device, method, and graphical user interface for forgoing generation of tactile output for a multi-contact gesture |
US10496260B2 (en) | 2012-05-09 | 2019-12-03 | Apple Inc. | Device, method, and graphical user interface for pressure-based alteration of controls in a user interface |
US10620781B2 (en) | 2012-12-29 | 2020-04-14 | Apple Inc. | Device, method, and graphical user interface for moving a cursor according to a change in an appearance of a control icon with simulated three-dimensional characteristics |
US10637986B2 (en) | 2016-06-10 | 2020-04-28 | Apple Inc. | Displaying and updating a set of application views |
US11068153B2 (en) | 2012-05-09 | 2021-07-20 | Apple Inc. | Device, method, and graphical user interface for displaying user interface objects corresponding to an application |
US11442591B2 (en) * | 2018-04-09 | 2022-09-13 | Lockheed Martin Corporation | System, method, computer readable medium, and viewer-interface for prioritized selection of mutually occluding objects in a virtual environment |
US11740776B2 (en) | 2012-05-09 | 2023-08-29 | Apple Inc. | Context-specific user interfaces |
US11775141B2 (en) | 2017-05-12 | 2023-10-03 | Apple Inc. | Context-specific user interfaces |
AU2022203957B2 (en) * | 2014-08-02 | 2023-10-12 | Apple Inc. | Context-specific user interfaces |
US11822778B2 (en) | 2020-05-11 | 2023-11-21 | Apple Inc. | User interfaces related to time |
US11842032B2 (en) | 2020-05-11 | 2023-12-12 | Apple Inc. | User interfaces for managing user interface sharing |
US11908343B2 (en) | 2015-08-20 | 2024-02-20 | Apple Inc. | Exercised-based watch face and complications |
US11918857B2 (en) | 2016-06-11 | 2024-03-05 | Apple Inc. | Activity and workout updates |
US11921992B2 (en) | 2021-05-14 | 2024-03-05 | Apple Inc. | User interfaces related to time |
US11922004B2 (en) | 2014-08-15 | 2024-03-05 | Apple Inc. | Weather user interface |
US11960701B2 (en) | 2020-04-29 | 2024-04-16 | Apple Inc. | Using an illustration to show the passing of time |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10078420B2 (en) * | 2012-03-16 | 2018-09-18 | Nokia Technologies Oy | Electronic devices, associated apparatus and methods |
US8907914B2 (en) * | 2012-08-31 | 2014-12-09 | General Electric Company | Methods and apparatus for documenting a procedure |
US9625884B1 (en) | 2013-06-10 | 2017-04-18 | Timothy Harris Ousley | Apparatus for extending control and methods thereof |
Citations (27)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5252951A (en) * | 1989-04-28 | 1993-10-12 | International Business Machines Corporation | Graphical user interface with gesture recognition in a multiapplication environment |
US5379057A (en) * | 1988-11-14 | 1995-01-03 | Microslate, Inc. | Portable computer with touch screen and computer system employing same |
US5488204A (en) * | 1992-06-08 | 1996-01-30 | Synaptics, Incorporated | Paintbrush stylus for capacitive touch sensor pad |
US5596346A (en) * | 1994-07-22 | 1997-01-21 | Eastman Kodak Company | Method and apparatus for applying a function to a localized area of a digital image using a window |
US5880411A (en) * | 1992-06-08 | 1999-03-09 | Synaptics, Incorporated | Object position detector with edge motion feature and gesture recognition |
US5953735A (en) * | 1991-03-20 | 1999-09-14 | Forcier; Mitchell D. | Script character processing method and system with bit-mapped document editing |
US6057845A (en) * | 1997-11-14 | 2000-05-02 | Sensiva, Inc. | System, method, and apparatus for generation and recognizing universal commands |
US6188391B1 (en) * | 1998-07-09 | 2001-02-13 | Synaptics, Inc. | Two-layer capacitive touchpad and method of making same |
US6323846B1 (en) * | 1998-01-26 | 2001-11-27 | University Of Delaware | Method and apparatus for integrating manual input |
US6594616B2 (en) * | 2001-06-18 | 2003-07-15 | Microsoft Corporation | System and method for providing a mobile input device |
US20030222917A1 (en) * | 2002-05-30 | 2003-12-04 | Intel Corporation | Mobile virtual desktop |
US20040193413A1 (en) * | 2003-03-25 | 2004-09-30 | Wilson Andrew D. | Architecture for controlling a computer using hand gestures |
US20060026521A1 (en) * | 2004-07-30 | 2006-02-02 | Apple Computer, Inc. | Gestures for touch sensitive input devices |
US20060097991A1 (en) * | 2004-05-06 | 2006-05-11 | Apple Computer, Inc. | Multipoint touchscreen |
US20060101354A1 (en) * | 2004-10-20 | 2006-05-11 | Nintendo Co., Ltd. | Gesture inputs for a portable display device |
US20060103633A1 (en) * | 2004-11-17 | 2006-05-18 | Atrua Technologies, Inc. | Customizable touch input module for an electronic device |
US20060125803A1 (en) * | 2001-02-10 | 2006-06-15 | Wayne Westerman | System and method for packing multitouch gestures onto a hand |
US20060177753A1 (en) * | 2002-09-27 | 2006-08-10 | Canon Kabushiki Kaisha | Toner |
US20060209014A1 (en) * | 2005-03-16 | 2006-09-21 | Microsoft Corporation | Method and system for providing modifier key behavior through pen gestures |
US20070177803A1 (en) * | 2006-01-30 | 2007-08-02 | Apple Computer, Inc | Multi-touch gesture dictionary |
US20070177804A1 (en) * | 2006-01-30 | 2007-08-02 | Apple Computer, Inc. | Multi-touch gesture dictionary |
US20070184878A1 (en) * | 2006-02-03 | 2007-08-09 | Lg Electronics Inc. | Text inputting |
US20080163130A1 (en) * | 2007-01-03 | 2008-07-03 | Apple Inc | Gesture learning |
US20090061947A1 (en) * | 2007-09-03 | 2009-03-05 | Lg Electronics Inc. | Mobile terminal and touch recognition method therefor |
US20090091540A1 (en) * | 2007-10-04 | 2009-04-09 | Linh Doan | Method and apparatus for controlling timing of status change of electronics apparatus based on user's finger location and input speed |
US20090210810A1 (en) * | 2008-02-15 | 2009-08-20 | Lg Electronics Inc. | Mobile communication device equipped with touch screen and method of controlling the same |
US20090278806A1 (en) * | 2008-05-06 | 2009-11-12 | Matias Gonzalo Duarte | Extended touch-sensitive control area for electronic device |
Family Cites Families (16)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4988981B1 (en) | 1987-03-17 | 1999-05-18 | Vpl Newco Inc | Computer data entry and manipulation apparatus and method |
JPH07146750A (en) | 1993-11-25 | 1995-06-06 | Casio Comput Co Ltd | Switch device |
JP3486459B2 (en) | 1994-06-21 | 2004-01-13 | キヤノン株式会社 | Electronic information equipment and control method thereof |
US5864635A (en) | 1996-06-14 | 1999-01-26 | International Business Machines Corporation | Distinguishing gestures from handwriting in a pen based computer by stroke analysis |
US6573883B1 (en) | 1998-06-24 | 2003-06-03 | Hewlett Packard Development Company, L.P. | Method and apparatus for controlling a computing device with gestures |
US7469381B2 (en) * | 2007-01-07 | 2008-12-23 | Apple Inc. | List scrolling and document translation, scaling, and rotation on a touch-screen display |
JP2002259046A (en) | 2001-02-28 | 2002-09-13 | Tomoya Sonoda | System for entering character and symbol handwritten in air |
US6690387B2 (en) * | 2001-12-28 | 2004-02-10 | Koninklijke Philips Electronics N.V. | Touch-screen image scrolling system and method |
US11275405B2 (en) | 2005-03-04 | 2022-03-15 | Apple Inc. | Multi-functional hand-held device |
JP4206053B2 (en) | 2004-03-31 | 2009-01-07 | 株式会社国際電気通信基礎技術研究所 | User interface device and user interface program |
US9008447B2 (en) | 2004-04-01 | 2015-04-14 | Google Inc. | Method and system for character recognition |
KR20190061099A (en) | 2005-03-04 | 2019-06-04 | 애플 인크. | Multi-functional hand-held device |
US20070273674A1 (en) | 2005-03-18 | 2007-11-29 | Searete Llc, A Limited Liability Corporation | Machine-differentiatable identifiers having a commonly accepted meaning |
US7694240B2 (en) * | 2006-11-22 | 2010-04-06 | General Electric Company | Methods and systems for creation of hanging protocols using graffiti-enabled devices |
US7949157B2 (en) | 2007-08-10 | 2011-05-24 | Nitin Afzulpurkar | Interpreting sign language gestures |
JP2011503709A (en) * | 2007-11-07 | 2011-01-27 | エヌ−トリグ リミテッド | Gesture detection for digitizer |
-
2009
- 2009-11-19 US US12/621,759 patent/US8432367B2/en active Active
-
2010
- 2010-11-11 EP EP10832023.5A patent/EP2502131A4/en not_active Withdrawn
- 2010-11-11 CA CA2781347A patent/CA2781347C/en active Active
- 2010-11-11 WO PCT/US2010/056400 patent/WO2011062837A2/en active Application Filing
- 2010-11-11 AU AU2010322267A patent/AU2010322267B2/en not_active Ceased
- 2010-11-11 KR KR1020127013089A patent/KR101668398B1/en active IP Right Grant
Patent Citations (29)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5379057A (en) * | 1988-11-14 | 1995-01-03 | Microslate, Inc. | Portable computer with touch screen and computer system employing same |
US5675362A (en) * | 1988-11-14 | 1997-10-07 | Microslate, Inc. | Portable computer with touch screen and computing system employing same |
US5252951A (en) * | 1989-04-28 | 1993-10-12 | International Business Machines Corporation | Graphical user interface with gesture recognition in a multiapplication environment |
US5953735A (en) * | 1991-03-20 | 1999-09-14 | Forcier; Mitchell D. | Script character processing method and system with bit-mapped document editing |
US5488204A (en) * | 1992-06-08 | 1996-01-30 | Synaptics, Incorporated | Paintbrush stylus for capacitive touch sensor pad |
US5880411A (en) * | 1992-06-08 | 1999-03-09 | Synaptics, Incorporated | Object position detector with edge motion feature and gesture recognition |
US5596346A (en) * | 1994-07-22 | 1997-01-21 | Eastman Kodak Company | Method and apparatus for applying a function to a localized area of a digital image using a window |
US6057845A (en) * | 1997-11-14 | 2000-05-02 | Sensiva, Inc. | System, method, and apparatus for generation and recognizing universal commands |
US6323846B1 (en) * | 1998-01-26 | 2001-11-27 | University Of Delaware | Method and apparatus for integrating manual input |
US6188391B1 (en) * | 1998-07-09 | 2001-02-13 | Synaptics, Inc. | Two-layer capacitive touchpad and method of making same |
US20060125803A1 (en) * | 2001-02-10 | 2006-06-15 | Wayne Westerman | System and method for packing multitouch gestures onto a hand |
US6594616B2 (en) * | 2001-06-18 | 2003-07-15 | Microsoft Corporation | System and method for providing a mobile input device |
US20030222917A1 (en) * | 2002-05-30 | 2003-12-04 | Intel Corporation | Mobile virtual desktop |
US20060177753A1 (en) * | 2002-09-27 | 2006-08-10 | Canon Kabushiki Kaisha | Toner |
US20040193413A1 (en) * | 2003-03-25 | 2004-09-30 | Wilson Andrew D. | Architecture for controlling a computer using hand gestures |
US20060097991A1 (en) * | 2004-05-06 | 2006-05-11 | Apple Computer, Inc. | Multipoint touchscreen |
US20060026536A1 (en) * | 2004-07-30 | 2006-02-02 | Apple Computer, Inc. | Gestures for touch sensitive input devices |
US20060026521A1 (en) * | 2004-07-30 | 2006-02-02 | Apple Computer, Inc. | Gestures for touch sensitive input devices |
US20060101354A1 (en) * | 2004-10-20 | 2006-05-11 | Nintendo Co., Ltd. | Gesture inputs for a portable display device |
US20060103633A1 (en) * | 2004-11-17 | 2006-05-18 | Atrua Technologies, Inc. | Customizable touch input module for an electronic device |
US20060209014A1 (en) * | 2005-03-16 | 2006-09-21 | Microsoft Corporation | Method and system for providing modifier key behavior through pen gestures |
US20070177803A1 (en) * | 2006-01-30 | 2007-08-02 | Apple Computer, Inc | Multi-touch gesture dictionary |
US20070177804A1 (en) * | 2006-01-30 | 2007-08-02 | Apple Computer, Inc. | Multi-touch gesture dictionary |
US20070184878A1 (en) * | 2006-02-03 | 2007-08-09 | Lg Electronics Inc. | Text inputting |
US20080163130A1 (en) * | 2007-01-03 | 2008-07-03 | Apple Inc | Gesture learning |
US20090061947A1 (en) * | 2007-09-03 | 2009-03-05 | Lg Electronics Inc. | Mobile terminal and touch recognition method therefor |
US20090091540A1 (en) * | 2007-10-04 | 2009-04-09 | Linh Doan | Method and apparatus for controlling timing of status change of electronics apparatus based on user's finger location and input speed |
US20090210810A1 (en) * | 2008-02-15 | 2009-08-20 | Lg Electronics Inc. | Mobile communication device equipped with touch screen and method of controlling the same |
US20090278806A1 (en) * | 2008-05-06 | 2009-11-12 | Matias Gonzalo Duarte | Extended touch-sensitive control area for electronic device |
Cited By (85)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8538693B2 (en) * | 2006-01-03 | 2013-09-17 | Strategic Design Federation W, Inc. | Computer-aided route selection |
US20110172902A1 (en) * | 2006-01-03 | 2011-07-14 | Strategic Design Federation W, Inc. | Computer-Aided Route Selection |
US9829331B2 (en) * | 2006-01-03 | 2017-11-28 | Strategic Design Federation W, Inc. | Computer-aided route selection |
US20180080786A1 (en) * | 2006-01-03 | 2018-03-22 | Strategic Design Federation W, Inc. | Computer-aided route selection |
US11371856B1 (en) * | 2006-01-03 | 2022-06-28 | Huawei Technologies Co., Ltd. | Computer-aided route selection |
US20150219465A1 (en) * | 2006-01-03 | 2015-08-06 | Strategic Design Federation W, Inc. | Computer-aided route selection |
US20140019045A1 (en) * | 2006-01-03 | 2014-01-16 | Strategic Design Federation W, Inc. | Computer-aided route selection |
US20220299331A1 (en) * | 2006-01-03 | 2022-09-22 | Huawei Technologies Co., Ltd. | Computer-aided route selection |
US10634511B2 (en) * | 2006-01-03 | 2020-04-28 | Strategic Design Federation W, Llc | Computer-aided route selection |
US9008967B2 (en) * | 2006-01-03 | 2015-04-14 | Strategic Design Federation W, Inc. | Computer-aided route selection |
US11920941B2 (en) * | 2006-01-03 | 2024-03-05 | Huawei Technologies Co., Ltd. | Computer-aided route selection |
US20110138321A1 (en) * | 2009-12-04 | 2011-06-09 | International Business Machines Corporation | Zone-based functions in a user interface |
US9569102B2 (en) | 2010-01-06 | 2017-02-14 | Apple Inc. | Device, method, and graphical user interface with interactive popup views |
US9513801B2 (en) | 2010-04-07 | 2016-12-06 | Apple Inc. | Accessing electronic notifications and settings icons with gestures |
US10891023B2 (en) | 2010-04-07 | 2021-01-12 | Apple Inc. | Device, method and graphical user interface for shifting a user interface between positions on a touch-sensitive display in response to detected inputs |
US9823831B2 (en) | 2010-04-07 | 2017-11-21 | Apple Inc. | Device, method, and graphical user interface for managing concurrently open software applications |
US20150177927A1 (en) * | 2010-04-07 | 2015-06-25 | Apple Inc. | Device, method, and graphical user interface for managing concurrently open software applications |
US10156962B2 (en) * | 2010-04-07 | 2018-12-18 | Apple Inc. | Device, method and graphical user interface for sliding an application view by a predefined amount of sliding based on a touch input to a predefined button of a multifunction device |
US10901601B2 (en) | 2010-04-07 | 2021-01-26 | Apple Inc. | Device, method, and graphical user interface for managing concurrently open software applications |
US10101879B2 (en) | 2010-04-07 | 2018-10-16 | Apple Inc. | Device, method, and graphical user interface for managing concurrently open software applications using a three-dimensional stack of images of open applications |
US9405453B1 (en) * | 2010-04-08 | 2016-08-02 | Twitter, Inc. | User interface mechanics |
US11023120B1 (en) | 2010-04-08 | 2021-06-01 | Twitter, Inc. | User interface mechanics |
US11487404B2 (en) | 2010-12-20 | 2022-11-01 | Apple Inc. | Device, method, and graphical user interface for navigation of concurrently open software applications |
US10852914B2 (en) | 2010-12-20 | 2020-12-01 | Apple Inc. | Device, method, and graphical user interface for navigation of concurrently open software applications |
US10007400B2 (en) | 2010-12-20 | 2018-06-26 | Apple Inc. | Device, method, and graphical user interface for navigation of concurrently open software applications |
US10261668B2 (en) | 2010-12-20 | 2019-04-16 | Apple Inc. | Device, method, and graphical user interface for navigation of concurrently open software applications |
US11880550B2 (en) | 2010-12-20 | 2024-01-23 | Apple Inc. | Device, method, and graphical user interface for navigation of concurrently open software applications |
US8982065B2 (en) * | 2010-12-31 | 2015-03-17 | Kt Corporation | Method and apparatus for performing processes in a user equipment by using touch patterns |
US20120169632A1 (en) * | 2010-12-31 | 2012-07-05 | Yu Dong-Won | Method and apparatus for performing processes in a user equipment by using touch patterns |
US9792036B2 (en) | 2011-07-19 | 2017-10-17 | Lg Electronics Inc. | Mobile terminal and controlling method to display memo content |
EP2549717A1 (en) * | 2011-07-19 | 2013-01-23 | Lg Electronics Inc. | Mobile terminal and controlling method thereof |
EP2584441A1 (en) * | 2011-10-18 | 2013-04-24 | Research In Motion Limited | Electronic device and method of controlling same |
US8810535B2 (en) | 2011-10-18 | 2014-08-19 | Blackberry Limited | Electronic device and method of controlling same |
US20130141463A1 (en) * | 2011-12-06 | 2013-06-06 | Microsoft Corporation | Combined interactive map and list view |
US9753639B2 (en) | 2012-05-09 | 2017-09-05 | Apple Inc. | Device, method, and graphical user interface for displaying content associated with a corresponding affordance |
US10175864B2 (en) | 2012-05-09 | 2019-01-08 | Apple Inc. | Device, method, and graphical user interface for selecting object within a group of objects in accordance with contact intensity |
US9823839B2 (en) | 2012-05-09 | 2017-11-21 | Apple Inc. | Device, method, and graphical user interface for displaying additional information in response to a user contact |
US10496260B2 (en) | 2012-05-09 | 2019-12-03 | Apple Inc. | Device, method, and graphical user interface for pressure-based alteration of controls in a user interface |
US9990121B2 (en) | 2012-05-09 | 2018-06-05 | Apple Inc. | Device, method, and graphical user interface for moving a user interface object based on an intensity of a press input |
US9996231B2 (en) | 2012-05-09 | 2018-06-12 | Apple Inc. | Device, method, and graphical user interface for manipulating framed graphical objects |
US11068153B2 (en) | 2012-05-09 | 2021-07-20 | Apple Inc. | Device, method, and graphical user interface for displaying user interface objects corresponding to an application |
US10042542B2 (en) | 2012-05-09 | 2018-08-07 | Apple Inc. | Device, method, and graphical user interface for moving and dropping a user interface object |
US10481690B2 (en) | 2012-05-09 | 2019-11-19 | Apple Inc. | Device, method, and graphical user interface for providing tactile feedback for media adjustment operations performed in a user interface |
US9886184B2 (en) | 2012-05-09 | 2018-02-06 | Apple Inc. | Device, method, and graphical user interface for providing feedback for changing activation states of a user interface object |
US11740776B2 (en) | 2012-05-09 | 2023-08-29 | Apple Inc. | Context-specific user interfaces |
US10095391B2 (en) | 2012-05-09 | 2018-10-09 | Apple Inc. | Device, method, and graphical user interface for selecting user interface objects |
US9619076B2 (en) | 2012-05-09 | 2017-04-11 | Apple Inc. | Device, method, and graphical user interface for transitioning between display states in response to a gesture |
US10126930B2 (en) | 2012-05-09 | 2018-11-13 | Apple Inc. | Device, method, and graphical user interface for scrolling nested regions |
US10175757B2 (en) | 2012-05-09 | 2019-01-08 | Apple Inc. | Device, method, and graphical user interface for providing tactile feedback for touch-based operations performed and reversed in a user interface |
US10191627B2 (en) | 2012-05-09 | 2019-01-29 | Apple Inc. | Device, method, and graphical user interface for manipulating framed graphical objects |
EP2664986A3 (en) * | 2012-05-14 | 2014-08-20 | Samsung Electronics Co., Ltd | Method and electronic device thereof for processing function corresponding to multi-touch |
US10078442B2 (en) | 2012-12-29 | 2018-09-18 | Apple Inc. | Device, method, and graphical user interface for determining whether to scroll or select content based on an intensity theshold |
US10437333B2 (en) | 2012-12-29 | 2019-10-08 | Apple Inc. | Device, method, and graphical user interface for forgoing generation of tactile output for a multi-contact gesture |
US9959025B2 (en) | 2012-12-29 | 2018-05-01 | Apple Inc. | Device, method, and graphical user interface for navigating user interface hierarchies |
US10620781B2 (en) | 2012-12-29 | 2020-04-14 | Apple Inc. | Device, method, and graphical user interface for moving a cursor according to a change in an appearance of a control icon with simulated three-dimensional characteristics |
US9778771B2 (en) | 2012-12-29 | 2017-10-03 | Apple Inc. | Device, method, and graphical user interface for transitioning between touch input to display output relationships |
US9857897B2 (en) | 2012-12-29 | 2018-01-02 | Apple Inc. | Device and method for assigning respective portions of an aggregate intensity to a plurality of contacts |
US11137898B2 (en) | 2013-03-15 | 2021-10-05 | Apple Inc. | Device, method, and graphical user interface for displaying a plurality of settings controls |
US10310732B2 (en) | 2013-03-15 | 2019-06-04 | Apple Inc. | Device, method, and graphical user interface for concurrently displaying a plurality of settings controls |
US9658740B2 (en) | 2013-03-15 | 2017-05-23 | Apple Inc. | Device, method, and graphical user interface for managing concurrently open software applications |
US9109921B1 (en) * | 2013-06-19 | 2015-08-18 | Amazon Technologies, Inc. | Contextual based navigation element |
AU2022203957B2 (en) * | 2014-08-02 | 2023-10-12 | Apple Inc. | Context-specific user interfaces |
US11922004B2 (en) | 2014-08-15 | 2024-03-05 | Apple Inc. | Weather user interface |
US10048757B2 (en) | 2015-03-08 | 2018-08-14 | Apple Inc. | Devices and methods for controlling media presentation |
US10095396B2 (en) | 2015-03-08 | 2018-10-09 | Apple Inc. | Devices, methods, and graphical user interfaces for interacting with a control object while dragging another object |
US9645732B2 (en) | 2015-03-08 | 2017-05-09 | Apple Inc. | Devices, methods, and graphical user interfaces for displaying and using menus |
US9785305B2 (en) | 2015-03-19 | 2017-10-10 | Apple Inc. | Touch input cursor manipulation |
US9860451B2 (en) | 2015-06-07 | 2018-01-02 | Apple Inc. | Devices and methods for capturing and interacting with enhanced digital images |
US9602729B2 (en) | 2015-06-07 | 2017-03-21 | Apple Inc. | Devices and methods for capturing and interacting with enhanced digital images |
US9674426B2 (en) | 2015-06-07 | 2017-06-06 | Apple Inc. | Devices and methods for capturing and interacting with enhanced digital images |
US10162452B2 (en) | 2015-08-10 | 2018-12-25 | Apple Inc. | Devices and methods for processing touch inputs based on their intensities |
US11182017B2 (en) | 2015-08-10 | 2021-11-23 | Apple Inc. | Devices and methods for processing touch inputs based on their intensities |
US11908343B2 (en) | 2015-08-20 | 2024-02-20 | Apple Inc. | Exercised-based watch face and complications |
JP2017199217A (en) * | 2016-04-27 | 2017-11-02 | 京セラドキュメントソリューションズ株式会社 | Handwritten character input device, image forming apparatus, and handwritten character input method |
CN107315528A (en) * | 2016-04-27 | 2017-11-03 | 京瓷办公信息系统株式会社 | Handwriting character inputting device and hand-written character input method |
US20170315718A1 (en) * | 2016-04-27 | 2017-11-02 | Kyocera Document Solutions Inc. | Handwritten character input device, image forming apparatus and handwritten character input method |
US11323559B2 (en) | 2016-06-10 | 2022-05-03 | Apple Inc. | Displaying and updating a set of application views |
US10637986B2 (en) | 2016-06-10 | 2020-04-28 | Apple Inc. | Displaying and updating a set of application views |
US11918857B2 (en) | 2016-06-11 | 2024-03-05 | Apple Inc. | Activity and workout updates |
US11775141B2 (en) | 2017-05-12 | 2023-10-03 | Apple Inc. | Context-specific user interfaces |
US11442591B2 (en) * | 2018-04-09 | 2022-09-13 | Lockheed Martin Corporation | System, method, computer readable medium, and viewer-interface for prioritized selection of mutually occluding objects in a virtual environment |
US11960701B2 (en) | 2020-04-29 | 2024-04-16 | Apple Inc. | Using an illustration to show the passing of time |
US11842032B2 (en) | 2020-05-11 | 2023-12-12 | Apple Inc. | User interfaces for managing user interface sharing |
US11822778B2 (en) | 2020-05-11 | 2023-11-21 | Apple Inc. | User interfaces related to time |
US11921992B2 (en) | 2021-05-14 | 2024-03-05 | Apple Inc. | User interfaces related to time |
Also Published As
Publication number | Publication date |
---|---|
CA2781347C (en) | 2017-09-05 |
WO2011062837A2 (en) | 2011-05-26 |
WO2011062837A3 (en) | 2011-07-28 |
US8432367B2 (en) | 2013-04-30 |
AU2010322267B2 (en) | 2014-07-10 |
AU2010322267A1 (en) | 2012-06-07 |
KR101668398B1 (en) | 2016-10-21 |
EP2502131A4 (en) | 2018-01-24 |
KR20120116916A (en) | 2012-10-23 |
EP2502131A2 (en) | 2012-09-26 |
CA2781347A1 (en) | 2011-05-26 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US8432367B2 (en) | Translating user interaction with a touch screen into input commands | |
JP6925186B2 (en) | Surface of visible objects outside the screen | |
US20220035522A1 (en) | Device, Method, and Graphical User Interface for Displaying a Plurality of Settings Controls | |
EP3465408B1 (en) | Systems, devices, and methods for dynamically providing user interface controls at a touch-sensitive secondary display | |
JP5912083B2 (en) | User interface providing method and apparatus | |
US9569102B2 (en) | Device, method, and graphical user interface with interactive popup views | |
US8589374B2 (en) | Multifunction device with integrated search and application selection | |
US20110175826A1 (en) | Automatically Displaying and Hiding an On-screen Keyboard | |
US20110163966A1 (en) | Apparatus and Method Having Multiple Application Display Modes Including Mode with Display Resolution of Another Apparatus | |
US11079926B2 (en) | Method and apparatus for providing user interface of portable device | |
CN108885505B (en) | Intuitive document navigation with interactive content elements |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: GOOGLE INC., CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:LI, YANG;CLERON, MICHAEL A.;HACKBORN, DIANNE K.;SIGNING DATES FROM 20091022 TO 20091113;REEL/FRAME:023558/0536 |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
CC | Certificate of correction | ||
FPAY | Fee payment |
Year of fee payment: 4 |
|
AS | Assignment |
Owner name: GOOGLE LLC, CALIFORNIA Free format text: CHANGE OF NAME;ASSIGNOR:GOOGLE INC.;REEL/FRAME:044101/0299 Effective date: 20170929 |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 8TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1552); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 8 |