US20150185882A1 - Method and apparatus for recognition of writing, for remote communication, and for user defined input templates - Google Patents

Method and apparatus for recognition of writing, for remote communication, and for user defined input templates Download PDF

Info

Publication number
US20150185882A1
US20150185882A1 US14/657,243 US201514657243A US2015185882A1 US 20150185882 A1 US20150185882 A1 US 20150185882A1 US 201514657243 A US201514657243 A US 201514657243A US 2015185882 A1 US2015185882 A1 US 2015185882A1
Authority
US
United States
Prior art keywords
writing
user
position indicator
writing surface
housing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US14/657,243
Inventor
Christopher D. Kasabach
John M. Stivoric
Francine Duskey Gemperle
Christopher Pacione
Eric Teller
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zivio Technologies LLC
Original Assignee
Zivio Technologies LLC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zivio Technologies LLC filed Critical Zivio Technologies LLC
Priority to US14/657,243 priority Critical patent/US20150185882A1/en
Publication of US20150185882A1 publication Critical patent/US20150185882A1/en
Priority to US15/390,766 priority patent/US20170108957A1/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/0354Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of 2D relative movements between the device, or an operating part thereof, and a plane or surface, e.g. 2D mice, trackballs, pens or pucks
    • G06F3/03545Pens or stylus
    • G06K9/00402
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10Character recognition
    • G06V30/14Image acquisition
    • G06V30/142Image acquisition using hand-held instruments; Constructional details of the instruments
    • G06V30/1423Image acquisition using hand-held instruments; Constructional details of the instruments the instrument generating sequences of position coordinates corresponding to handwriting
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10Character recognition
    • G06V30/32Digital ink

Definitions

  • the present invention is directed generally to a method and device for recognition of writing and communication with remote devices.
  • computers for data entry and communication has become limiting and inconvenient in some aspects.
  • the use of a computer keyboard requires repetitive motion that is known to cause physical discomfort and, in some cases, physical injury.
  • computers are relatively large devices.
  • Even the advent of laptop computers has not made data entry and communication easy.
  • a laptop cannot be smaller than a keyboard and, to be efficient, a keyboard must have keys at least as large as a typical user's fingers.
  • the size of a laptop computer is not foreseen to reduce significantly in the future. In any event, it is not foreseen to become small enough to be carried by a person with little or no inconvenience.
  • hand-held devices for data entry typically in the form of handwriting recognition.
  • Many of those devices require a written or sensory grid in order to determine the motion of the device and read the lines drawn by a user.
  • Some devices require special purpose input pads to monitor the writing and drawing of a user.
  • Other devices require that a user only print using special characters in order for handwriting recognition to work. All of those devices also suffer the inability to monitor multiple writings by a user on an ordinary piece of paper.
  • Other devices allow for multiple writings on an ordinary piece of paper, but do not compensate for movement of the paper relative to the writing instrument.
  • the prior art devices inaccurately recognize the writings because the user compensates for the movement of the paper, but the device is not able to determine that user has so compensated.
  • the present invention is directed to a device including a housing, a writing tip connected to the housing, a writing surface position indicator, a processor in the housing, a memory device in the housing connected to the processor, and a sensor in the housing and cooperative with the writing surface position indicator.
  • the device may be used to record writings and drawings applied to the surface by the user, to transmit that data to a remote device, to download data from remote devices, and to otherwise communicate with remote devices.
  • the present invention is also directed to a device including a housing, writing tip connected to the housing, a processor in the housing, and a memory device in the housing connected to the processor having stored thereon instructions which when executed by the processor cause the processor to recognize an interface defined by writing on a writing surface.
  • the present invention provides the advantage in that it permits data entry with a hand-held device which does not require a special writing surface.
  • the present invention also provides the advantage that data can be entered accurately because the device compensates for movement of the writing surface during the writing process.
  • the present invention also provides the advantage that the data can be stored into the writing instrument itself, rather than having to transmit the data to a remote device.
  • the present invention also provides the advantage that the device can communicate with remote devices.
  • the present invention also provides the advantage that a user can define a personal interface, which can be stored by the device, to enter data or instruct the device to perform certain commands.
  • FIG. 1 is a side plan view of a device constructed in accordance with the present invention
  • FIG. 2 is a block diagram illustrating components of the device
  • FIG. 3 is a top plan view of the device being used to write on a piece of paper
  • FIG. 4 is a top plan view of the device being used to define a personal interface on a piece of paper;
  • FIG. 5 is a top plan view of the device being used to define an alternative personal interface on a piece of paper
  • FIG. 6 is a side plan view of the device in a base.
  • FIG. 7 is a side plan view of an alternative embodiment of the device formed as a ring on a user's finger.
  • FIG. 1 is a side plan view of a device 10 constructed in accordance with the present invention.
  • the device 10 includes a writing instrument 12 and a writing surface position indicator 14 .
  • the writing instrument 12 may be a writing device such as, for example, a pen, a mechanical pencil, a marker, and chalk.
  • the writing instrument 12 includes a housing 15 and a writing tip 16 connected to the housing 15 to enable a user of the device 10 to place marks on a surface by applying the writing tip 16 to any surface.
  • the writing surface position indicator 14 provides information indicative of the relative position of the writing tip 16 to the surface on which the user is using the writing instrument 12 .
  • the writing surface position indicator 14 communicates with the writing instrument 12 to better determine the position of the writing tip 16 relative to the writing surface.
  • the communication between the writing surface position indicator 14 and the writing instrument 12 helps the device 10 track the marks being applied to the writing surface, even if the surface is translated or rotated during use.
  • the device 10 may track movement of the writing instrument 12 with a number of accelerometers 36 , and periodically correct the perceived position of the writing instrument 12 with the writing surface position indicator 14 , as discussed hereinbelow.
  • the writing surface position indicator 14 may be used alone to determine position and movement of the writing instrument 12 .
  • the writing surface position indicator 14 is affixed to or engaged with the writing surface and transmits a signal to one or more sensors 17 in the housing 15 .
  • the writing surface position indicator 14 may be a clip fastenable to an edge or corner of a writing surface.
  • the writing surface position indicator 14 may be fastened to the writing instrument 12 .
  • the writing surface position indicator 14 may be fastened to an end of the writing instrument 12 opposite the writing tip 16 , such as is illustrated in FIG. 1 . Further, the writing surface position indicator 14 may be fastened to the end of the writing instrument 12 to protect the writing tip 16 when the device 10 is not in use.
  • the writing surface position indicator 14 may be a clip, such as on a clipboard, for holding the writing surface.
  • the writing surface position indicator 14 may be, for example, a sticker fastenable to the writing surface.
  • the writing surface position indicator 14 may be placed on the writing surface and have sufficient mass, such as a paper weight, to maintain a constant position relative to the writing surface even if the writing surface is translated or rotated during writing.
  • the writing surface position indicator 14 may be active or passive.
  • the writing surface position indicator 14 may include a number of transmitters 18 to transmit a signal to the sensor 17 in the writing instrument 12 .
  • the device 10 determines the relative position of the writing tip 16 based on the received signal, as further described hereinbelow.
  • the writing surface position indicator 14 may be passive, such as by utilizing magnetic components.
  • the writing surface position indicator 14 may be a magnetic substance, such as magnetic ink applied to the surface or a magnetic sticker fixably placed on the writing surface.
  • the sensor 17 detects the position and orientation of the writing surface position indicator 14 , such as by sensing a magnetic field generated by the writing surface position indicator 14 , to determine the relative position of the writing tip 16 .
  • the device 10 may also include one or more other features to facilitate its use.
  • the device 10 may include an input 19 , such as a push button, to allow the user to select from various modes of operation and functions to be performed by the device 10 , as further described hereinbelow.
  • the device 10 may also include a display 20 , such as a liquid crystal display, for providing a user with visual information, including information regarding user input (feedback).
  • the display 20 may also, for example, serially display received and sent information, such as e-mail messages.
  • a facsimile message may be received by the device 10 , for example, by converting the facsimile message into electronic form and displaying it on the display 20 , much like an e-mail message.
  • the device 10 may further include an audio interface system 21 , including a speaker 22 and a microphone 24 .
  • the audio interface 21 may be detachably connectable to the housing 15 . When the audio interface 21 is connected to the housing 15 , it may allow the device 10 to provide conventional telephony communications using, for example, cellular technology contained within the housing 15 , as discussed hereinbelow.
  • the device 10 may, for example, be programmed to record audio dictation, to translate audio input into text to send via e-mail, or to convert the text of a received e-mail message into audio output.
  • the audio interface system 21 may provide an audio signal when an e-mail message is received by the device 10 .
  • FIG. 2 is a block diagram illustrating one embodiment of the device 10 including a processor 30 .
  • the processor 30 may be any one of many processors, such as one of the Pentium® family of processors manufactured by Intel Corp., Santa Clara, Calif.
  • a memory device 32 is coupled to the processor 30 , and may be used to store instructions for the operation of the device 10 , as well as to store data collected by the device 10 .
  • the memory device 32 may be, for example, 10 Mbytes of solid state memory.
  • the memory device 32 may be volatile or non-volatile.
  • the memory device 32 may include ROM for storing instructions to be executed by the processor 30 .
  • the memory device 32 may include RAM, such as dynamic RAM or static RAM, for storing instructions and data.
  • the memory device 32 may be or include an EPROM for storing instructions and data.
  • the device 10 may track the movement of the writing tip 16 relative to the writing surface with the writing surface position indicator 14 in conjunction with the pressure sensor 34 and accelerometers 36 .
  • one pressure sensor 34 may be connected to and located near the writing tip 16 of the writing instrument 12 .
  • the pressure sensor 34 may be located along or at the end of a shaft 38 connected to the writing tip 16 .
  • the pressure sensor 34 allows the device 10 to determine when the writing tip 16 is against a writing surface and the force with which the writing tip 16 is being applied to the writing surface, thereby allowing the device 10 to determine when the user is writing as well as the fineness or thickness of a line being drawn.
  • the pressure sensor 34 may be, for example, a strain gauge based force sensor.
  • Movement of the writing instrument 12 may be determined from the accelerometers 36 , which may be connected to the shaft 38 .
  • Each accelerometers 36 may be, for example, a dual axis, micro-electromechanical accelerometer. Three accelerometers 36 may be used, as illustrated in FIG. 1 , so that collectively they provide data indicative of motion, including speed and direction, of the writing instrument 12 in all three planes of three dimensional space.
  • three additional accelerometers 36 may be employed to provide data indicative of roll, pitch, and yaw. Information relating to roll, pitch, and yaw of the writing instrument 12 during the writing operation may provide for better handwriting recognition because information relating to rotation of the writing instrument 12 about a particular point is obtained.
  • the present invention may only utilize two accelerometers 36 to determine movement on the writing surface, and the pressure sensor 34 alone is used to determine when the writing instrument 12 is engaging the writing surface.
  • the writing surface position indicator 14 facilitates determining the position of the writing tip 16 relative to the writing surface, even if the writing surface moves during use, such as is often the case during writing. Furthermore, the writing surface position indicator 14 may be used by the device 10 to periodically calibrate the position of the writing tip 16 relative to surface as determined by the accelerometers 36 to compensate for the error in the position determined thereby, which typically accumulates over time.
  • the writing surface position indicator 14 may alone be used to track the movement of the writing instrument 12 .
  • the writing surface position indicator 14 continually determines the position of the writing tip 16 relative to the writing surface in three dimensions, thus enabling the device 10 to determine when the writing tip 16 is in contact with the surface, and the movement of the writing tip 16 as it is applied to the surface.
  • the writing surface position indicator 14 communicates the information to the writing instrument 12 as described hereinbelow.
  • the memory device 32 may be programmed with instructions which when executed by the processor 30 cause the processor 30 to recognize handwriting using conventional handwriting recognition algorithms.
  • the memory device 32 may also be programmed to cause the processor 30 to record in the memory device 32 non-recognized writings, such as drawings and doodles written by the user.
  • a communication system 40 may also be provided so that the device 10 may engage in external communication.
  • cellular telecommunication technology such as PCS digital cellular technology
  • the communication system 40 may be used, for example, to allow a user to use the device 10 as a telephone, with the aid of the audio interface system 21 .
  • the communication system 40 may also allow a user to connect to the Internet, via an internal modem and Internet connection (not shown), so that the device 10 may, for example, receive and send e-mail.
  • FIG. 3 is a top plan view of the device 10 being used to write on a piece of paper 42 .
  • the writing instrument 12 may be used to write on the paper 42 in a conventional manner by applying the writing tip 16 to the paper 42 .
  • the writing surface position indicator 14 is a clip fastenable to an edge of the paper 42 . According to that embodiment, the writing surface position indicator 14 may be removed from the writing instrument 12 and clipped, or otherwise fastened, to the paper 42 .
  • An embodiment of the writing surface position indicator 14 includes its own power supply, such as a rechargeable Lithium-ion solid polymer or liquid gel battery.
  • the writing surface position indicator 14 may communicate one or more signals that are used by the device 10 to determine the relative position of the writing tip 16 .
  • the signals may be, for example, infrared signals, ultrasonic signals, or electromagnetic signals.
  • the signals from the writing surface position indicator 14 are received by the sensor 17 , which may be, for example, a receiver as through part of the communication system 40 (illustrated in FIG. 2 ).
  • the device 10 may use the signals communicated by the writing surface position indicator 14 in any of several ways.
  • the signals from the writing surface position indicator 14 may be used to provide a reference for the device 10 of the relative position between the writing tip 16 and the writing surface because the writing surface position indicator 14 maintains a constant position relative to the paper 42 .
  • the writing surface position indicator 14 also allows for periodic calibration of the device 10 .
  • the writing surface position indicator 14 helps the device 10 compensate for small errors in its accelerometers 36 that accumulate over time. Without this feature, the movements recorded and used by the device 10 may become unusably noisy within a few minutes of use.
  • the writing surface position indicator 14 has two signal transmitters 18 separated by a small distance.
  • the writing surface position indicator 14 periodically transmits signals from the two transmitters 18 which are received by the sensor 17 .
  • the signals may be transmitted at different frequencies so that they may be distinguished.
  • the device 10 determines the exact position of the writing instrument 12 relative to the writing surface position indicator 14 by, for example, triangulation.
  • the device 10 then resets the internal value of where the writing instrument 12 is relative to the surface (having over the past small fraction of a second been approximated by the accelerometers 36 ) to the value calculated based on the information provided by the writing surface position indicator 14 . In this way, “creep” can be avoided.
  • the frequency of signals transmitted by the writing surface position indicator 14 may vary depending on the accuracy required for a specific application and the accuracy of the accelerometers 36 . Furthermore, the device 10 can maintain the position of the writing tip 16 relative to the paper 42 even if the paper 42 is rotated, or otherwise moved, during the writing. In an alternative embodiment, the writing surface position indicator 14 may produce a single signal from one transmitter 18 , and the device 10 may sense that signal with two spaced apart sensors 17 on the writing instrument 12 (as illustrated in FIG. 1 ), so that the position of the writing tip 16 relative to the writing surface position indicator 14 may be determined using, for example, triangulation.
  • the device 10 may operate in several modes. The different modes may be selected with the input 19 .
  • the device 10 may include an operation mode, a command mode, and an off mode.
  • instructions are stored in the memory device 32 which when executed by the processor 30 cause the processor 30 to operate the device 10 in the various modes.
  • the device 10 in the operation mode monitors and records what is being written or drawn by the user on the writing surface.
  • the device 10 can be programmed to recognize the writing using handwriting recognition algorithms stored in the memory device 32 .
  • the device 10 monitors the writing of the user and interprets that writing as commands. For example, the user may send previous writings and drawings, such as those monitored and stored in operation mode, by writing “send” followed by parameters for sending. For example, by writing “send 555-1234”, the device 10 will send the data via facsimile. In contrast, if “send” is followed by an e-mail address, the data will be sent via e-mail. Of course, separate commands, such as “fax” and “e-mail,” may be used to avoid confusion.
  • An “end” command, or any other command defined by the user may be used to initiate action by the device. For example, the symbol “.fwdarw.” may be used to end and execute the command.
  • the device may also use other commands, such as “save”, “read”, “contact”, “download”, and “phone”.
  • the device 10 In “off” mode, the device 10 operates as a conventional writing device and does not perform other functions, such as handwriting recognition.
  • the device 10 may also be used to define a personal interface.
  • the interface may be used, for example, to enter data into the device 10 or instruct the device 10 to execute specified commands.
  • the interface may be defined by the user as a template including, for example, icons, words, and symbols on a surface, which are drawn with the device 10 .
  • the user may enter command mode, write the word “interface”, and draw a template of numbers and functions that define a calculator, as illustrated in FIG. 4 .
  • the handwriting recognition feature of the device 10 allows it to recognize the numbers and letters, and the device 10 can determine the writing tip's 16 position relative to those drawn numbers and functions.
  • each element such as each number 44 and each function 46 may be surrounded by a circle or box 48 to more clearly define a command area 50 so that, for example, touching the writing tip 16 inside the command area 50 invokes that element or function, and touching the writing tip 16 outside of the command area 50 does not invoke it.
  • the user may touch the writing tip 16 of the writing instrument 12 to the command area 50 , and the device 10 will perform the command indicated by the user.
  • the user may draw a calculator then interact with the drawn interface as though it were a physical calculator. Answers may be provided on the display 20 .
  • a user may balance their checkbook on a piece of scrap paper while waiting at a bus stop. When finished, the user may throw away or recycle the paper.
  • the template may be a keyboard, with the keys arranged in any order desired by the user, and the user may interact with the keyboard with the writing tip 16 of the writing instrument 12 .
  • the entries made by the user may be shown on the display 20 , and the user may review the results by drawing on the paper 42 up and down arrows, and then touching the arrows.
  • the interface may be a user-defined macroinstruction.
  • the user could write the name of a person on the paper and circle it to define a command area 50 , and instruct the device 10 that whenever the user touches the writing tip 16 to the command area 50 the device 10 is to transmit stored data to the person's e-mail address.
  • the macroinstruction may be stored in the memory device 32 of the device 10 , such that it is saved, and will be recognized by the device 10 without the user having to re-define the interface during each subsequent use of the device 10 .
  • the various modes may also be used in conjunction. For example, a user could select the operate mode using the input 19 , and then write on a piece of paper. In the operate mode, the device 10 recognizes the writing, as described hereinbefore. Also, the user could instruct the device 10 to store the writing in the memory device 32 under a particular index or file.
  • the user could instruct the device 10 to perform a macroinstruction saved as a user-defined interface, such as to e-mail the writing to a particular address.
  • FIG. 6 is a side plan view of the device 10 in a base 54 .
  • the base 54 may be used to download data from the device 10 and transmit it to a remote device, such as a personal computer.
  • the base 54 may also charge the power supplies of the device 10 .
  • the power supplies for the device 10 may be, for example, an internal rechargeable Lithium-ion solid polymer or liquid gel battery.
  • the base 54 may be utilized to upload instructions into the memory device 32 of the device 10 .
  • a cord 56 on the base 54 may be used to connect the device 10 to a remote device, such as to a computer and/or to a power supply.
  • the base 54 could be wireless.
  • the present invention also includes a method of operating the present invention.
  • the method includes defining and using a personal interface.
  • the interface may be defined with a pen, as described hereinbefore, or it may be defined in other ways.
  • the invention may be embodied as a ring or sleeve that fits on a user's finger.
  • the writing instrument 12 may be a ring with an extension 60 that engages the paper 42 and houses a pressure sensor 34 .
  • a writing surface position indicator 14 may be removably attached to the writing instrument 12 and attached to paper 42 when writing. In that way, the user may write and draw with the finger as if it were the device 10 described hereinbefore. Feedback may be provided to the user by a display 20 mounted on the ring.
  • a user need not define an interface on a writing surface, but could instead define an interface in space, such as by moving the writing instrument 12 similar to waving a baton.
  • a user could distinguish to the device 10 when he is defining the interface and interacting with it from merely moving the writing instrument 12 from point to point, such as by indicating so to the writing instrument 12 by depressing the input 19 .

Abstract

A device including a housing, a writing tip connected to the housing, a writing surface position indicator, a processor in the housing, a memory device in the housing connected to the processor, and a sensor in the housing and cooperative with the writing surface position indicator. The device may be used to record writings and drawings applied to a surface by a user, to transmit that data to a remote device, to download data from remote devices, and to otherwise communicate with remote devices.

Description

    CLAIM OF PRIORITY
  • This application is a continuation of pending U.S. application Ser. No. 11/474,833 filed Jun. 26, 2006. U.S. application Ser. No. 11/474,833 is a continuation of pending U.S. application Ser. No. 10/430,182 filed May 5, 2003, which is a continuation of U.S. application Ser. No. 10/146,243 filed May 5, 2002 and issued as U.S. Pat. No. 6,628,847 on Sep. 30, 2003. U.S. application Ser. No. 10/146,243 was a divisional of Ser. No. 09/258,560, which was filed on Feb. 26, 1999 and which issued as U.S. Pat. No. 6,456,749 on Sep. 24, 2002. U.S. application Ser. No. 09/258,560 claimed priority to U.S. Provisional Application Ser. No. 60/076,338 filed Sep. 27, 1998.
  • STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT
  • Not Applicable
  • BACKGROUND OF THE INVENTION
  • 1. Field of the Invention
  • The present invention is directed generally to a method and device for recognition of writing and communication with remote devices.
  • 2. Description of the Background
  • The use of computers for data entry and communication has become limiting and inconvenient in some aspects. For example, the use of a computer keyboard requires repetitive motion that is known to cause physical discomfort and, in some cases, physical injury. In addition, computers are relatively large devices. Even the advent of laptop computers has not made data entry and communication easy. One reason is that a laptop cannot be smaller than a keyboard and, to be efficient, a keyboard must have keys at least as large as a typical user's fingers. As a result, the size of a laptop computer is not foreseen to reduce significantly in the future. In any event, it is not foreseen to become small enough to be carried by a person with little or no inconvenience.
  • To address these problems, some relevant art discloses hand-held devices for data entry, typically in the form of handwriting recognition. Many of those devices, however, require a written or sensory grid in order to determine the motion of the device and read the lines drawn by a user. Some devices require special purpose input pads to monitor the writing and drawing of a user. Other devices require that a user only print using special characters in order for handwriting recognition to work. All of those devices also suffer the inability to monitor multiple writings by a user on an ordinary piece of paper. Other devices allow for multiple writings on an ordinary piece of paper, but do not compensate for movement of the paper relative to the writing instrument. Thus, the prior art devices inaccurately recognize the writings because the user compensates for the movement of the paper, but the device is not able to determine that user has so compensated.
  • As a result, the need exists for a device that provides for easy and convenient data entry and communication, as well as the ability to use any surface and to be able to accurately monitor a user's actions.
  • BRIEF SUMMARY OF THE INVENTION
  • The present invention is directed to a device including a housing, a writing tip connected to the housing, a writing surface position indicator, a processor in the housing, a memory device in the housing connected to the processor, and a sensor in the housing and cooperative with the writing surface position indicator. The device may be used to record writings and drawings applied to the surface by the user, to transmit that data to a remote device, to download data from remote devices, and to otherwise communicate with remote devices.
  • The present invention is also directed to a device including a housing, writing tip connected to the housing, a processor in the housing, and a memory device in the housing connected to the processor having stored thereon instructions which when executed by the processor cause the processor to recognize an interface defined by writing on a writing surface.
  • The present invention provides the advantage in that it permits data entry with a hand-held device which does not require a special writing surface. The present invention also provides the advantage that data can be entered accurately because the device compensates for movement of the writing surface during the writing process. The present invention also provides the advantage that the data can be stored into the writing instrument itself, rather than having to transmit the data to a remote device. The present invention also provides the advantage that the device can communicate with remote devices. The present invention also provides the advantage that a user can define a personal interface, which can be stored by the device, to enter data or instruct the device to perform certain commands. These and other advantages and benefits of the present invention will become apparent from the detailed description of the invention hereinbelow.
  • BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING
  • For the present invention to be clearly understood and readily practiced, the present invention will be described in conjunction with the following figures, wherein:
  • FIG. 1 is a side plan view of a device constructed in accordance with the present invention;
  • FIG. 2 is a block diagram illustrating components of the device;
  • FIG. 3 is a top plan view of the device being used to write on a piece of paper;
  • FIG. 4 is a top plan view of the device being used to define a personal interface on a piece of paper;
  • FIG. 5 is a top plan view of the device being used to define an alternative personal interface on a piece of paper;
  • FIG. 6 is a side plan view of the device in a base; and
  • FIG. 7 is a side plan view of an alternative embodiment of the device formed as a ring on a user's finger.
  • DETAILED DESCRIPTION OF THE INVENTION
  • It is to be understood that the figures and descriptions of the present invention have been simplified to illustrate elements that are relevant for a clear understanding of the present invention, while eliminating, for purposes of clarity, other elements. Those of ordinary skill in the art will recognize that other elements may be desirable in order to implement a particular embodiment of the present invention. However, because such elements are well known in the art, and because they do not facilitate a better understanding of the present invention, a discussion of such elements is not provided herein.
  • FIG. 1 is a side plan view of a device 10 constructed in accordance with the present invention. The device 10 includes a writing instrument 12 and a writing surface position indicator 14. The writing instrument 12 may be a writing device such as, for example, a pen, a mechanical pencil, a marker, and chalk. The writing instrument 12 includes a housing 15 and a writing tip 16 connected to the housing 15 to enable a user of the device 10 to place marks on a surface by applying the writing tip 16 to any surface.
  • The writing surface position indicator 14 provides information indicative of the relative position of the writing tip 16 to the surface on which the user is using the writing instrument 12. The writing surface position indicator 14 communicates with the writing instrument 12 to better determine the position of the writing tip 16 relative to the writing surface. The communication between the writing surface position indicator 14 and the writing instrument 12 helps the device 10 track the marks being applied to the writing surface, even if the surface is translated or rotated during use. The device 10 may track movement of the writing instrument 12 with a number of accelerometers 36, and periodically correct the perceived position of the writing instrument 12 with the writing surface position indicator 14, as discussed hereinbelow.
  • Alternatively, the writing surface position indicator 14 may be used alone to determine position and movement of the writing instrument 12. To communicate the relative position of the writing tip 16 to the surface, the writing surface position indicator 14 is affixed to or engaged with the writing surface and transmits a signal to one or more sensors 17 in the housing 15. For example, the writing surface position indicator 14 may be a clip fastenable to an edge or corner of a writing surface. When the writing surface position indicator 14 is not being used to provide information indicative of the relative position of the writing tip 16 to the writing surface, it may be fastened to the writing instrument 12. For example, the writing surface position indicator 14 may be fastened to an end of the writing instrument 12 opposite the writing tip 16, such as is illustrated in FIG. 1. Further, the writing surface position indicator 14 may be fastened to the end of the writing instrument 12 to protect the writing tip 16 when the device 10 is not in use.
  • In one embodiment, the writing surface position indicator 14 may be a clip, such as on a clipboard, for holding the writing surface. In another embodiment, the writing surface position indicator 14 may be, for example, a sticker fastenable to the writing surface. In another embodiment, the writing surface position indicator 14 may be placed on the writing surface and have sufficient mass, such as a paper weight, to maintain a constant position relative to the writing surface even if the writing surface is translated or rotated during writing.
  • In addition, the writing surface position indicator 14 may be active or passive. For example, the writing surface position indicator 14 may include a number of transmitters 18 to transmit a signal to the sensor 17 in the writing instrument 12. The device 10 determines the relative position of the writing tip 16 based on the received signal, as further described hereinbelow. Alternatively, the writing surface position indicator 14 may be passive, such as by utilizing magnetic components. For example, the writing surface position indicator 14 may be a magnetic substance, such as magnetic ink applied to the surface or a magnetic sticker fixably placed on the writing surface. In an embodiment in which the writing surface position indicator 14 is passive, the sensor 17 detects the position and orientation of the writing surface position indicator 14, such as by sensing a magnetic field generated by the writing surface position indicator 14, to determine the relative position of the writing tip 16.
  • The device 10 may also include one or more other features to facilitate its use. For example, the device 10 may include an input 19, such as a push button, to allow the user to select from various modes of operation and functions to be performed by the device 10, as further described hereinbelow. The device 10 may also include a display 20, such as a liquid crystal display, for providing a user with visual information, including information regarding user input (feedback). The display 20 may also, for example, serially display received and sent information, such as e-mail messages. In addition, a facsimile message may be received by the device 10, for example, by converting the facsimile message into electronic form and displaying it on the display 20, much like an e-mail message.
  • The device 10 may further include an audio interface system 21, including a speaker 22 and a microphone 24. The audio interface 21 may be detachably connectable to the housing 15. When the audio interface 21 is connected to the housing 15, it may allow the device 10 to provide conventional telephony communications using, for example, cellular technology contained within the housing 15, as discussed hereinbelow. In addition, the device 10 may, for example, be programmed to record audio dictation, to translate audio input into text to send via e-mail, or to convert the text of a received e-mail message into audio output. Also, the audio interface system 21 may provide an audio signal when an e-mail message is received by the device 10.
  • FIG. 2 is a block diagram illustrating one embodiment of the device 10 including a processor 30. The processor 30 may be any one of many processors, such as one of the Pentium® family of processors manufactured by Intel Corp., Santa Clara, Calif. A memory device 32 is coupled to the processor 30, and may be used to store instructions for the operation of the device 10, as well as to store data collected by the device 10. The memory device 32 may be, for example, 10 Mbytes of solid state memory. The memory device 32 may be volatile or non-volatile. The memory device 32 may include ROM for storing instructions to be executed by the processor 30. The memory device 32 may include RAM, such as dynamic RAM or static RAM, for storing instructions and data. The memory device 32 may be or include an EPROM for storing instructions and data.
  • The device 10 may track the movement of the writing tip 16 relative to the writing surface with the writing surface position indicator 14 in conjunction with the pressure sensor 34 and accelerometers 36. For example, one pressure sensor 34 may be connected to and located near the writing tip 16 of the writing instrument 12. Alternatively, the pressure sensor 34 may be located along or at the end of a shaft 38 connected to the writing tip 16. The pressure sensor 34 allows the device 10 to determine when the writing tip 16 is against a writing surface and the force with which the writing tip 16 is being applied to the writing surface, thereby allowing the device 10 to determine when the user is writing as well as the fineness or thickness of a line being drawn. The pressure sensor 34 may be, for example, a strain gauge based force sensor.
  • Movement of the writing instrument 12 may be determined from the accelerometers 36, which may be connected to the shaft 38. Each accelerometers 36 may be, for example, a dual axis, micro-electromechanical accelerometer. Three accelerometers 36 may be used, as illustrated in FIG. 1, so that collectively they provide data indicative of motion, including speed and direction, of the writing instrument 12 in all three planes of three dimensional space. In an alternative embodiment, three additional accelerometers 36 may be employed to provide data indicative of roll, pitch, and yaw. Information relating to roll, pitch, and yaw of the writing instrument 12 during the writing operation may provide for better handwriting recognition because information relating to rotation of the writing instrument 12 about a particular point is obtained. Alternatively, the present invention may only utilize two accelerometers 36 to determine movement on the writing surface, and the pressure sensor 34 alone is used to determine when the writing instrument 12 is engaging the writing surface.
  • Used in conjunction with the pressure sensor 34 and accelerometers 36, the writing surface position indicator 14 facilitates determining the position of the writing tip 16 relative to the writing surface, even if the writing surface moves during use, such as is often the case during writing. Furthermore, the writing surface position indicator 14 may be used by the device 10 to periodically calibrate the position of the writing tip 16 relative to surface as determined by the accelerometers 36 to compensate for the error in the position determined thereby, which typically accumulates over time.
  • In an alternative embodiment, the writing surface position indicator 14 may alone be used to track the movement of the writing instrument 12. In that embodiment, the writing surface position indicator 14 continually determines the position of the writing tip 16 relative to the writing surface in three dimensions, thus enabling the device 10 to determine when the writing tip 16 is in contact with the surface, and the movement of the writing tip 16 as it is applied to the surface. The writing surface position indicator 14 communicates the information to the writing instrument 12 as described hereinbelow.
  • The memory device 32 may be programmed with instructions which when executed by the processor 30 cause the processor 30 to recognize handwriting using conventional handwriting recognition algorithms. The memory device 32 may also be programmed to cause the processor 30 to record in the memory device 32 non-recognized writings, such as drawings and doodles written by the user.
  • A communication system 40 may also be provided so that the device 10 may engage in external communication. For example, cellular telecommunication technology, such as PCS digital cellular technology, may be contained in the writing instrument 12, and used to allow the device 10 to send and receive information. The communication system 40 may be used, for example, to allow a user to use the device 10 as a telephone, with the aid of the audio interface system 21. The communication system 40 may also allow a user to connect to the Internet, via an internal modem and Internet connection (not shown), so that the device 10 may, for example, receive and send e-mail.
  • FIG. 3 is a top plan view of the device 10 being used to write on a piece of paper 42. The writing instrument 12 may be used to write on the paper 42 in a conventional manner by applying the writing tip 16 to the paper 42. In the embodiment of the present invention illustrated in FIG. 3, the writing surface position indicator 14 is a clip fastenable to an edge of the paper 42. According to that embodiment, the writing surface position indicator 14 may be removed from the writing instrument 12 and clipped, or otherwise fastened, to the paper 42. An embodiment of the writing surface position indicator 14 includes its own power supply, such as a rechargeable Lithium-ion solid polymer or liquid gel battery. The writing surface position indicator 14 may communicate one or more signals that are used by the device 10 to determine the relative position of the writing tip 16. The signals may be, for example, infrared signals, ultrasonic signals, or electromagnetic signals. The signals from the writing surface position indicator 14 are received by the sensor 17, which may be, for example, a receiver as through part of the communication system 40 (illustrated in FIG. 2).
  • The device 10 may use the signals communicated by the writing surface position indicator 14 in any of several ways. For example, the signals from the writing surface position indicator 14 may be used to provide a reference for the device 10 of the relative position between the writing tip 16 and the writing surface because the writing surface position indicator 14 maintains a constant position relative to the paper 42. As a result, even if the paper 42 is moved during use, as almost always happens during writing, the device 10 will be able to maintain its position relative to the paper 42, allowing the device 10 to maintain an accurate record of what is being written. The writing surface position indicator 14 also allows for periodic calibration of the device 10. For example, by maintaining a known reference point, established at the beginning of a writing session, the writing surface position indicator 14 helps the device 10 compensate for small errors in its accelerometers 36 that accumulate over time. Without this feature, the movements recorded and used by the device 10 may become unusably noisy within a few minutes of use.
  • In one embodiment of the present invention, the writing surface position indicator 14 has two signal transmitters 18 separated by a small distance. The writing surface position indicator 14 periodically transmits signals from the two transmitters 18 which are received by the sensor 17. The signals may be transmitted at different frequencies so that they may be distinguished. The device 10 determines the exact position of the writing instrument 12 relative to the writing surface position indicator 14 by, for example, triangulation. The device 10 then resets the internal value of where the writing instrument 12 is relative to the surface (having over the past small fraction of a second been approximated by the accelerometers 36) to the value calculated based on the information provided by the writing surface position indicator 14. In this way, “creep” can be avoided. The frequency of signals transmitted by the writing surface position indicator 14 may vary depending on the accuracy required for a specific application and the accuracy of the accelerometers 36. Furthermore, the device 10 can maintain the position of the writing tip 16 relative to the paper 42 even if the paper 42 is rotated, or otherwise moved, during the writing. In an alternative embodiment, the writing surface position indicator 14 may produce a single signal from one transmitter 18, and the device 10 may sense that signal with two spaced apart sensors 17 on the writing instrument 12 (as illustrated in FIG. 1), so that the position of the writing tip 16 relative to the writing surface position indicator 14 may be determined using, for example, triangulation.
  • The device 10 may operate in several modes. The different modes may be selected with the input 19. For example, the device 10 may include an operation mode, a command mode, and an off mode. To operate in the different modes, instructions are stored in the memory device 32 which when executed by the processor 30 cause the processor 30 to operate the device 10 in the various modes. For example, the device 10 in the operation mode monitors and records what is being written or drawn by the user on the writing surface. The device 10 can be programmed to recognize the writing using handwriting recognition algorithms stored in the memory device 32.
  • In the command mode, the device 10 monitors the writing of the user and interprets that writing as commands. For example, the user may send previous writings and drawings, such as those monitored and stored in operation mode, by writing “send” followed by parameters for sending. For example, by writing “send 555-1234”, the device 10 will send the data via facsimile. In contrast, if “send” is followed by an e-mail address, the data will be sent via e-mail. Of course, separate commands, such as “fax” and “e-mail,” may be used to avoid confusion. An “end” command, or any other command defined by the user, may be used to initiate action by the device. For example, the symbol “.fwdarw.” may be used to end and execute the command. The device may also use other commands, such as “save”, “read”, “contact”, “download”, and “phone”.
  • In “off” mode, the device 10 operates as a conventional writing device and does not perform other functions, such as handwriting recognition.
  • The device 10 may also be used to define a personal interface. The interface may be used, for example, to enter data into the device 10 or instruct the device 10 to execute specified commands. The interface may be defined by the user as a template including, for example, icons, words, and symbols on a surface, which are drawn with the device 10. For example, the user may enter command mode, write the word “interface”, and draw a template of numbers and functions that define a calculator, as illustrated in FIG. 4. The handwriting recognition feature of the device 10 allows it to recognize the numbers and letters, and the device 10 can determine the writing tip's 16 position relative to those drawn numbers and functions. To better define the personal interface, each element, such as each number 44 and each function 46 may be surrounded by a circle or box 48 to more clearly define a command area 50 so that, for example, touching the writing tip 16 inside the command area 50 invokes that element or function, and touching the writing tip 16 outside of the command area 50 does not invoke it. The user may touch the writing tip 16 of the writing instrument 12 to the command area 50, and the device 10 will perform the command indicated by the user. For example, the user may draw a calculator then interact with the drawn interface as though it were a physical calculator. Answers may be provided on the display 20. As a result, a user may balance their checkbook on a piece of scrap paper while waiting at a bus stop. When finished, the user may throw away or recycle the paper.
  • Other, potentially limitless, interfaces may also be created. For example, the template may be a keyboard, with the keys arranged in any order desired by the user, and the user may interact with the keyboard with the writing tip 16 of the writing instrument 12. The entries made by the user may be shown on the display 20, and the user may review the results by drawing on the paper 42 up and down arrows, and then touching the arrows.
  • Also, the interface may be a user-defined macroinstruction. For example, as illustrated in FIG. 5, the user could write the name of a person on the paper and circle it to define a command area 50, and instruct the device 10 that whenever the user touches the writing tip 16 to the command area 50 the device 10 is to transmit stored data to the person's e-mail address. Moreover, the macroinstruction may be stored in the memory device 32 of the device 10, such that it is saved, and will be recognized by the device 10 without the user having to re-define the interface during each subsequent use of the device 10.
  • Another example of an interface is that the user could write on the paper 42 “FONT=COURIER,” circle it, and instruct the device 10 that whenever the user touches the writing tip 16 to the paper 42 within the circle, the device 10 is to convert information stored in the memory device into ASCII text in courier font.
  • The various modes may also be used in conjunction. For example, a user could select the operate mode using the input 19, and then write on a piece of paper. In the operate mode, the device 10 recognizes the writing, as described hereinbefore. Also, the user could instruct the device 10 to store the writing in the memory device 32 under a particular index or file.
  • Alternatively, in the command mode, the user could instruct the device 10 to perform a macroinstruction saved as a user-defined interface, such as to e-mail the writing to a particular address.
  • FIG. 6 is a side plan view of the device 10 in a base 54. The base 54 may be used to download data from the device 10 and transmit it to a remote device, such as a personal computer. The base 54 may also charge the power supplies of the device 10. The power supplies for the device 10 may be, for example, an internal rechargeable Lithium-ion solid polymer or liquid gel battery. In addition, the base 54 may be utilized to upload instructions into the memory device 32 of the device 10. A cord 56 on the base 54 may be used to connect the device 10 to a remote device, such as to a computer and/or to a power supply. Alternatively, the base 54 could be wireless.
  • The present invention also includes a method of operating the present invention. The method includes defining and using a personal interface. The interface may be defined with a pen, as described hereinbefore, or it may be defined in other ways. For example, as illustrated in FIG. 7, the invention may be embodied as a ring or sleeve that fits on a user's finger. The writing instrument 12, for example, may be a ring with an extension 60 that engages the paper 42 and houses a pressure sensor 34. A writing surface position indicator 14 may be removably attached to the writing instrument 12 and attached to paper 42 when writing. In that way, the user may write and draw with the finger as if it were the device 10 described hereinbefore. Feedback may be provided to the user by a display 20 mounted on the ring.
  • Those of ordinary skill in the art will recognize that many modifications and variations of the present invention may be implemented. For example, a user need not define an interface on a writing surface, but could instead define an interface in space, such as by moving the writing instrument 12 similar to waving a baton. A user could distinguish to the device 10 when he is defining the interface and interacting with it from merely moving the writing instrument 12 from point to point, such as by indicating so to the writing instrument 12 by depressing the input 19. The foregoing description and the following claims are intended to cover all such modifications and variations.

Claims (1)

What is claimed is:
1. A device comprising:
a. a housing;
b. a writing tip connected to the housing;
c. a processor in the housing;
d. at least one of a speaker and a display supported by the housing; and
e. a memory device in the housing connected to the processor and having stored thereon instructions which when executed by the processor cause the processor to (i) recognize an interface defined by writing on a writing surface, and (ii) to cause the processor to recognize user interaction with the interface, and (iii) to cause the at least one of a speaker and display to provide output based on said user interaction.
US14/657,243 1998-02-27 2015-03-13 Method and apparatus for recognition of writing, for remote communication, and for user defined input templates Abandoned US20150185882A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US14/657,243 US20150185882A1 (en) 1998-02-27 2015-03-13 Method and apparatus for recognition of writing, for remote communication, and for user defined input templates
US15/390,766 US20170108957A1 (en) 1998-02-27 2016-12-27 Method and apparatus for recognition of writing, for remote communication, and for user defined input templates

Applications Claiming Priority (9)

Application Number Priority Date Filing Date Title
US7633898P 1998-02-27 1998-02-27
US09/258,560 US6456749B1 (en) 1998-02-27 1999-02-26 Handheld apparatus for recognition of writing, for remote communication, and for user defined input templates
US10/146,243 US6628847B1 (en) 1998-02-27 2002-05-15 Method and apparatus for recognition of writing, for remote communication, and for user defined input templates
US10/430,182 US7068860B2 (en) 1998-02-27 2003-05-05 Method and apparatus for recognition of writing, for remote communication, and for user defined input templates
US11/474,833 US7349586B2 (en) 1998-02-27 2006-06-26 Method and apparatus for recognition of writing, for remote communication, and for user defined input templates
US12/009,603 US8036498B2 (en) 1998-02-27 2008-01-18 Method and apparatus for recognition of writing, for remote communication, and for user defined input templates
US13/224,220 US8818140B2 (en) 1998-02-27 2011-09-01 Method and apparatus for recognition of writing, for remote communication, and for user defined input templates
US14/311,462 US20140301668A1 (en) 1998-02-27 2014-06-23 Method and apparatus for recognition of writing, for remote communication, and for user defined input templates
US14/657,243 US20150185882A1 (en) 1998-02-27 2015-03-13 Method and apparatus for recognition of writing, for remote communication, and for user defined input templates

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
US14/311,462 Continuation US20140301668A1 (en) 1998-02-27 2014-06-23 Method and apparatus for recognition of writing, for remote communication, and for user defined input templates

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US15/390,766 Continuation US20170108957A1 (en) 1998-02-27 2016-12-27 Method and apparatus for recognition of writing, for remote communication, and for user defined input templates

Publications (1)

Publication Number Publication Date
US20150185882A1 true US20150185882A1 (en) 2015-07-02

Family

ID=26757984

Family Applications (9)

Application Number Title Priority Date Filing Date
US09/258,560 Expired - Lifetime US6456749B1 (en) 1998-02-27 1999-02-26 Handheld apparatus for recognition of writing, for remote communication, and for user defined input templates
US10/146,243 Expired - Lifetime US6628847B1 (en) 1998-02-27 2002-05-15 Method and apparatus for recognition of writing, for remote communication, and for user defined input templates
US10/430,182 Expired - Lifetime US7068860B2 (en) 1998-02-27 2003-05-05 Method and apparatus for recognition of writing, for remote communication, and for user defined input templates
US11/474,833 Expired - Fee Related US7349586B2 (en) 1998-02-27 2006-06-26 Method and apparatus for recognition of writing, for remote communication, and for user defined input templates
US12/009,603 Expired - Fee Related US8036498B2 (en) 1998-02-27 2008-01-18 Method and apparatus for recognition of writing, for remote communication, and for user defined input templates
US13/224,220 Expired - Fee Related US8818140B2 (en) 1998-02-27 2011-09-01 Method and apparatus for recognition of writing, for remote communication, and for user defined input templates
US14/311,462 Abandoned US20140301668A1 (en) 1998-02-27 2014-06-23 Method and apparatus for recognition of writing, for remote communication, and for user defined input templates
US14/657,243 Abandoned US20150185882A1 (en) 1998-02-27 2015-03-13 Method and apparatus for recognition of writing, for remote communication, and for user defined input templates
US15/390,766 Abandoned US20170108957A1 (en) 1998-02-27 2016-12-27 Method and apparatus for recognition of writing, for remote communication, and for user defined input templates

Family Applications Before (7)

Application Number Title Priority Date Filing Date
US09/258,560 Expired - Lifetime US6456749B1 (en) 1998-02-27 1999-02-26 Handheld apparatus for recognition of writing, for remote communication, and for user defined input templates
US10/146,243 Expired - Lifetime US6628847B1 (en) 1998-02-27 2002-05-15 Method and apparatus for recognition of writing, for remote communication, and for user defined input templates
US10/430,182 Expired - Lifetime US7068860B2 (en) 1998-02-27 2003-05-05 Method and apparatus for recognition of writing, for remote communication, and for user defined input templates
US11/474,833 Expired - Fee Related US7349586B2 (en) 1998-02-27 2006-06-26 Method and apparatus for recognition of writing, for remote communication, and for user defined input templates
US12/009,603 Expired - Fee Related US8036498B2 (en) 1998-02-27 2008-01-18 Method and apparatus for recognition of writing, for remote communication, and for user defined input templates
US13/224,220 Expired - Fee Related US8818140B2 (en) 1998-02-27 2011-09-01 Method and apparatus for recognition of writing, for remote communication, and for user defined input templates
US14/311,462 Abandoned US20140301668A1 (en) 1998-02-27 2014-06-23 Method and apparatus for recognition of writing, for remote communication, and for user defined input templates

Family Applications After (1)

Application Number Title Priority Date Filing Date
US15/390,766 Abandoned US20170108957A1 (en) 1998-02-27 2016-12-27 Method and apparatus for recognition of writing, for remote communication, and for user defined input templates

Country Status (1)

Country Link
US (9) US6456749B1 (en)

Families Citing this family (96)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6686910B2 (en) * 1996-04-22 2004-02-03 O'donnell, Jr. Francis E. Combined writing instrument and digital documentor apparatus and method of use
US6573887B1 (en) * 1996-04-22 2003-06-03 O'donnell, Jr. Francis E. Combined writing instrument and digital documentor
US6456749B1 (en) * 1998-02-27 2002-09-24 Carnegie Mellon University Handheld apparatus for recognition of writing, for remote communication, and for user defined input templates
US7971784B2 (en) 1999-05-25 2011-07-05 Silverbrook Research Pty Ltd Sensing device with mode changes via nib switch
US7721948B1 (en) * 1999-05-25 2010-05-25 Silverbrook Research Pty Ltd Method and system for online payments
US7832626B2 (en) * 1999-05-25 2010-11-16 Silverbrook Research Pty Ltd Anonymous competition entry
US8113950B2 (en) * 1999-05-25 2012-02-14 Silverbrook Research Pty Ltd Competition entry with limited return messaging
US6737591B1 (en) 1999-05-25 2004-05-18 Silverbrook Research Pty Ltd Orientation sensing device
US7760969B2 (en) 1999-05-25 2010-07-20 Silverbrook Research Pty Ltd Method of providing information via context searching from a printed substrate
US6428155B1 (en) * 1999-05-25 2002-08-06 Silverbrook Research Pty Ltd Printer cartridge including machine readable ink
US20070233513A1 (en) * 1999-05-25 2007-10-04 Silverbrook Research Pty Ltd Method of providing merchant resource or merchant hyperlink to a user
US6816274B1 (en) * 1999-05-25 2004-11-09 Silverbrook Research Pty Ltd Method and system for composition and delivery of electronic mail
US7762453B2 (en) * 1999-05-25 2010-07-27 Silverbrook Research Pty Ltd Method of providing information via a printed substrate with every interaction
AUPQ363299A0 (en) * 1999-10-25 1999-11-18 Silverbrook Research Pty Ltd Paper based information inter face
US7821507B2 (en) * 1999-05-25 2010-10-26 Silverbrook Research Pty Ltd Method of providing information via a printed substrate and two-mode sensing device
AU774258B2 (en) * 1999-06-30 2004-06-24 Silverbrook Research Pty Ltd Method and system for collaborative document markup
GB9922214D0 (en) * 1999-09-20 1999-11-17 Ncr Int Inc Creation transmission and retrieval of information
US7322524B2 (en) * 2000-10-20 2008-01-29 Silverbrook Research Pty Ltd Graphic design software using an interface surface
US7503493B2 (en) * 1999-10-25 2009-03-17 Silverbrook Research Pty Ltd Method and system for digitizing freehand graphics with user-selected properties
US6811516B1 (en) * 1999-10-29 2004-11-02 Brian M. Dugan Methods and apparatus for monitoring and encouraging health and fitness
US6985576B1 (en) * 1999-12-02 2006-01-10 Worldcom, Inc. Method and apparatus for automatic call distribution
US7295193B2 (en) * 1999-12-23 2007-11-13 Anoto Ab Written command
WO2001061631A1 (en) * 2000-02-16 2001-08-23 Telefonaktiebolaget Lm Ericsson (Publ) Printer pen
US6832116B1 (en) 2000-02-16 2004-12-14 Telefonaktiebolaget L M Ericsson (Publ) Method and system for controlling an electronic utility device using an electronic reading device
US6839623B1 (en) 2000-02-16 2005-01-04 Telefonaktiebolaget Lm Ericsson (Publ) Positioning applications for an electronic reading device
US6813396B1 (en) 2000-02-16 2004-11-02 Telefonatiebolaget L.M. Ericsson (Publ) Method for sharing information between electronic reading devices
US6952497B1 (en) 2000-02-16 2005-10-04 Telefonaktiebolaget L M Ericsson (Publ) Method and system for electronically recording transactions and performing security function
US20010033293A1 (en) * 2000-02-16 2001-10-25 Magnus Hollstrom Electronic pen help feedback and information retrieval
AU775392B2 (en) * 2000-05-24 2004-07-29 Silverbrook Research Pty Ltd Printed page tag encoder
US7162442B2 (en) * 2000-06-30 2007-01-09 Silverbrook Research Pty Ltd Method and system for submitting bids using coded forms
US7289110B2 (en) * 2000-07-17 2007-10-30 Human Messaging Ab Method and arrangement for identifying and processing commands in digital images, where the user marks the command, for example by encircling it
CN1181662C (en) * 2000-07-21 2004-12-22 中兴通讯股份有限公司 Hand written character input method for mobile telephone and mobile phone with said function
US7515292B2 (en) * 2000-11-25 2009-04-07 Silverbrook Research Pty Ltd Apparatus for cooling and storing produce
US6927871B1 (en) * 2000-11-25 2005-08-09 Silverbrook Research Pty Ltd Apparatus for interaction with a network computer system
US20020160883A1 (en) 2001-03-08 2002-10-31 Dugan Brian M. System and method for improving fitness equipment and exercise
US20070111858A1 (en) * 2001-03-08 2007-05-17 Dugan Brian M Systems and methods for using a video game to achieve an exercise objective
US8939831B2 (en) 2001-03-08 2015-01-27 Brian M. Dugan Systems and methods for improving fitness equipment and exercise
US7916124B1 (en) 2001-06-20 2011-03-29 Leapfrog Enterprises, Inc. Interactive apparatus using print media
US7202861B2 (en) * 2001-06-25 2007-04-10 Anoto Ab Control of a unit provided with a processor
JP4261145B2 (en) * 2001-09-19 2009-04-30 株式会社リコー Information processing apparatus, information processing apparatus control method, and program for causing computer to execute the method
US6977187B2 (en) * 2002-06-19 2005-12-20 Foster-Miller, Inc. Chip package sealing method
WO2004023382A1 (en) * 2002-09-04 2004-03-18 Fachhochschule Regensburg Biometric acoustic writing system and method for identifying individuals and recognizing handwriting by using biometric data
CN100377043C (en) * 2002-09-28 2008-03-26 皇家飞利浦电子股份有限公司 Three-dimensional hand-written identification process and system thereof
US7203384B2 (en) * 2003-02-24 2007-04-10 Electronic Scripting Products, Inc. Implement for optically inferring information from a planar jotting surface
EP1661059B1 (en) * 2003-07-21 2010-10-06 Gannon Technologies Group Systems and methods for assessing disorders affecting fine motor skills using handwriting recognition
US7038846B2 (en) 2003-11-24 2006-05-02 Electronic Scripting Products, Inc. Solid catadioptric lens with a single viewpoint
US7088440B2 (en) * 2003-12-22 2006-08-08 Electronic Scripting Products, Inc. Method and apparatus for determining absolute position of a tip of an elongate object on a plane surface with invariant features
US7826641B2 (en) 2004-01-30 2010-11-02 Electronic Scripting Products, Inc. Apparatus and method for determining an absolute pose of a manipulated object in a real three-dimensional environment with invariant features
US9229540B2 (en) 2004-01-30 2016-01-05 Electronic Scripting Products, Inc. Deriving input from six degrees of freedom interfaces
US7961909B2 (en) 2006-03-08 2011-06-14 Electronic Scripting Products, Inc. Computer interface employing a manipulated object with absolute pose detection component and a display
US8542219B2 (en) * 2004-01-30 2013-09-24 Electronic Scripting Products, Inc. Processing pose data derived from the pose of an elongate object
US20060066591A1 (en) * 2004-03-17 2006-03-30 James Marggraff Method and system for implementing a user interface for a device through recognized text and bounded areas
US20060127872A1 (en) * 2004-03-17 2006-06-15 James Marggraff Method and device for associating a user writing with a user-writable element
US20060078866A1 (en) * 2004-03-17 2006-04-13 James Marggraff System and method for identifying termination of data entry
US20060125805A1 (en) * 2004-03-17 2006-06-15 James Marggraff Method and system for conducting a transaction using recognized text
US7453447B2 (en) * 2004-03-17 2008-11-18 Leapfrog Enterprises, Inc. Interactive apparatus with recording and playback capability usable with encoded writing medium
US20060077184A1 (en) * 2004-03-17 2006-04-13 James Marggraff Methods and devices for retrieving and using information stored as a pattern on a surface
US7853193B2 (en) 2004-03-17 2010-12-14 Leapfrog Enterprises, Inc. Method and device for audibly instructing a user to interact with a function
US20060067576A1 (en) * 2004-03-17 2006-03-30 James Marggraff Providing a user interface having interactive elements on a writable surface
US7831933B2 (en) * 2004-03-17 2010-11-09 Leapfrog Enterprises, Inc. Method and system for implementing a user interface for a device employing written graphical elements
US20060033725A1 (en) * 2004-06-03 2006-02-16 Leapfrog Enterprises, Inc. User created interactive interface
US20050281437A1 (en) * 2004-05-17 2005-12-22 Renate Fruchter Talking paper
JP2007504565A (en) * 2004-06-03 2007-03-01 リープフロッグ エンタープライズィーズ,インコーポレイテッド Interactive interface created by the user
US7922099B1 (en) 2005-07-29 2011-04-12 Leapfrog Enterprises, Inc. System and method for associating content with an image bearing surface
US7936339B2 (en) * 2005-11-01 2011-05-03 Leapfrog Enterprises, Inc. Method and system for invoking computer functionality by interaction with dynamically generated interface regions of a writing surface
US11826652B2 (en) 2006-01-04 2023-11-28 Dugan Health, Llc Systems and methods for improving fitness equipment and exercise
US20080058808A1 (en) * 2006-06-14 2008-03-06 Spartek Medical, Inc. Implant system and method to treat degenerative disorders of the spine
US8781568B2 (en) 2006-06-23 2014-07-15 Brian M. Dugan Systems and methods for heart rate monitoring, data transmission, and use
US8261967B1 (en) 2006-07-19 2012-09-11 Leapfrog Enterprises, Inc. Techniques for interactively coupling electronic content with printed media
US8337335B2 (en) * 2006-10-07 2012-12-25 Dugan Brian M Systems and methods for measuring and/or analyzing swing information
US8430770B2 (en) * 2006-10-07 2013-04-30 Brian M. Dugan Systems and methods for measuring and/or analyzing swing information
KR20100029219A (en) * 2007-05-29 2010-03-16 라이브스크라이브, 인크. Multi-modal smartpen computing system
US8374992B2 (en) * 2007-05-29 2013-02-12 Livescribe, Inc. Organization of user generated content captured by a smart pen computing system
US20090251440A1 (en) * 2008-04-03 2009-10-08 Livescribe, Inc. Audio Bookmarking
US8944824B2 (en) * 2008-04-03 2015-02-03 Livescribe, Inc. Multi-modal learning system
US8976007B2 (en) 2008-08-09 2015-03-10 Brian M. Dugan Systems and methods for providing biofeedback information to a cellular telephone and for using such information
US20090270743A1 (en) * 2008-04-17 2009-10-29 Dugan Brian M Systems and methods for providing authenticated biofeedback information to a mobile device and for using such information
TWI374391B (en) * 2008-05-27 2012-10-11 Ind Tech Res Inst Method for recognizing writing motion and trajectory and apparatus for writing and recognizing system
CN101604212B (en) * 2008-06-11 2013-01-09 财团法人工业技术研究院 Recognition method of writing actions and tracks, writing device and recognition system thereof
US8165929B2 (en) * 2008-08-04 2012-04-24 Chudy Group, LLC Adaptive pharmaceutical product management methods and system
TWI457793B (en) * 2008-08-08 2014-10-21 Ind Tech Res Inst Real-time motion recognition method and inertia sensing and trajectory
US8454437B2 (en) 2009-07-17 2013-06-04 Brian M. Dugan Systems and methods for portable exergaming
CN101876875B (en) * 2009-04-28 2012-05-30 太瀚科技股份有限公司 Electronic whiteboard with partial printing function
US20120253489A1 (en) 2011-03-28 2012-10-04 Dugan Brian M Systems and methods for fitness and video games
US9533228B2 (en) 2011-03-28 2017-01-03 Brian M. Dugan Systems and methods for fitness and video games
US9610506B2 (en) 2011-03-28 2017-04-04 Brian M. Dugan Systems and methods for fitness and video games
US8947226B2 (en) 2011-06-03 2015-02-03 Brian M. Dugan Bands for measuring biometric information
KR20140008987A (en) * 2012-07-13 2014-01-22 삼성전자주식회사 Method and apparatus for controlling application using recognition of handwriting image
US11269431B2 (en) * 2013-06-19 2022-03-08 Nokia Technologies Oy Electronic-scribed input
DE102013214020A1 (en) 2013-07-17 2015-02-19 Stabilo International Gmbh Digital pen
US10254856B2 (en) 2014-01-17 2019-04-09 Osterhout Group, Inc. External user interface for head worn computing
US9939934B2 (en) 2014-01-17 2018-04-10 Osterhout Group, Inc. External user interface for head worn computing
DE102014106839B4 (en) 2014-05-15 2019-02-07 Stabilo International Gmbh Drift compensation for an electronic pen
US10228775B2 (en) * 2016-01-22 2019-03-12 Microsoft Technology Licensing, Llc Cross application digital ink repository
US11577159B2 (en) 2016-05-26 2023-02-14 Electronic Scripting Products Inc. Realistic virtual/augmented/mixed reality viewing and interactions
CN111989641A (en) * 2018-05-14 2020-11-24 株式会社和冠 Learning support system

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6456749B1 (en) * 1998-02-27 2002-09-24 Carnegie Mellon University Handheld apparatus for recognition of writing, for remote communication, and for user defined input templates

Family Cites Families (31)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4211195A (en) 1977-10-12 1980-07-08 Colt Industries Operating Corp. Overspeed ignition system
US4404940A (en) 1980-04-30 1983-09-20 Allied Corporation Engine speed limiting circuit
JPS59180066A (en) 1983-03-29 1984-10-12 Nippon Denso Co Ltd Ignition device for internal-combustion engine
JPH0670422B2 (en) 1985-01-31 1994-09-07 スズキ株式会社 Internal combustion engine speed control device
US4606323A (en) 1985-04-30 1986-08-19 Allied Corporation Magneto for ignition system
US6167439A (en) * 1988-05-27 2000-12-26 Kodak Limited Data retrieval, manipulation and transmission with facsimile images
EP0374241B1 (en) * 1988-05-27 1997-08-27 Kodak Limited Document annotation and manipulation in a data processing system
US5054088A (en) 1989-09-20 1991-10-01 International Business Machines Corporation Signature verification data compression for storage on an identification card
DE69021547T2 (en) 1990-02-16 1996-01-18 Rolls Royce Plc Device and method for determining the authenticity of a person's handwriting.
US5198877A (en) * 1990-10-15 1993-03-30 Pixsys, Inc. Method and apparatus for three-dimensional non-contact shape sensing
JP2726594B2 (en) 1991-04-01 1998-03-11 八洲電機株式会社 Memory pen
US5234345A (en) * 1991-09-27 1993-08-10 Weinblatt Lee S Survey technique for readership of publications
KR960003070B1 (en) 1992-05-22 1996-03-04 샤프 가부시끼가이샤 Tablet for display unit
US5438430A (en) * 1992-09-25 1995-08-01 Xerox Corporation Paper user interface for image manipulations such as cut and paste
WO1994015272A1 (en) * 1992-12-22 1994-07-07 Morgan Michael W Pen-based electronic teaching system
EP0622754B1 (en) 1993-04-30 1999-11-24 Sharp Kabushiki Kaisha Display-integrated type tablet device with high co-ordinate detection accuracy and method for driving the same
US5561466A (en) * 1993-06-23 1996-10-01 Nec Corporation Video and audio data multiplexing into ATM cells with no dummy cell used and ATM cell demultiplexing
US5561446A (en) * 1994-01-28 1996-10-01 Montlick; Terry F. Method and apparatus for wireless remote information retrieval and pen-based data entry
US5649023A (en) * 1994-05-24 1997-07-15 Panasonic Technologies, Inc. Method and apparatus for indexing a plurality of handwritten objects
US5518008A (en) * 1994-08-25 1996-05-21 Spectral Sciences Research Corporation Structural analyzer, in particular for medical implants
US6081261A (en) * 1995-11-01 2000-06-27 Ricoh Corporation Manual entry interactive paper and electronic document handling and processing system
US6686910B2 (en) * 1996-04-22 2004-02-03 O'donnell, Jr. Francis E. Combined writing instrument and digital documentor apparatus and method of use
US6573887B1 (en) * 1996-04-22 2003-06-03 O'donnell, Jr. Francis E. Combined writing instrument and digital documentor
US6130666A (en) * 1996-10-07 2000-10-10 Persidsky; Andre Self-contained pen computer with built-in display
US5755199A (en) 1996-10-10 1998-05-26 R. E. Phelon Co., Inc. Discharge ignition apparatus for internal combustion engine having built-in overspeed disable capability
US5953000A (en) * 1997-06-02 1999-09-14 Weirich; John P. Bounded-display-surface system for the input and output of computer data and video graphics
US6298173B1 (en) * 1997-10-03 2001-10-02 Matsushita Electric Corporation Of America Storage management system for document image database
US6446871B1 (en) * 1997-12-19 2002-09-10 A.T. Cross Company Method and apparatus for storing reference codes in a writing instrument and for retrieving information identifed by the reference codes
US5936527A (en) * 1998-02-10 1999-08-10 E-Tag Systems, Inc. Method and apparatus for locating and tracking documents and other objects
US6408820B1 (en) 2000-08-02 2002-06-25 Walbro Corporation Capacitor discharge engine ignition system with automatic ignition advance and/or minimum ignition speed control
US6932064B1 (en) 2004-04-28 2005-08-23 Walbro Engine Management, L.L.C. Capacitor discharge ignition

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6456749B1 (en) * 1998-02-27 2002-09-24 Carnegie Mellon University Handheld apparatus for recognition of writing, for remote communication, and for user defined input templates
US6628847B1 (en) * 1998-02-27 2003-09-30 Carnegie Mellon University Method and apparatus for recognition of writing, for remote communication, and for user defined input templates
US7068860B2 (en) * 1998-02-27 2006-06-27 Chris Dominick Kasabach Method and apparatus for recognition of writing, for remote communication, and for user defined input templates
US7349586B2 (en) * 1998-02-27 2008-03-25 Chris Dominick Kasabach Method and apparatus for recognition of writing, for remote communication, and for user defined input templates
US8036498B2 (en) * 1998-02-27 2011-10-11 Zivio Technologies LLC Method and apparatus for recognition of writing, for remote communication, and for user defined input templates
US8818140B2 (en) * 1998-02-27 2014-08-26 Zivio Technologies LLC Method and apparatus for recognition of writing, for remote communication, and for user defined input templates

Also Published As

Publication number Publication date
US20110317939A1 (en) 2011-12-29
US6628847B1 (en) 2003-09-30
US20140301668A1 (en) 2014-10-09
US7349586B2 (en) 2008-03-25
US20040022454A1 (en) 2004-02-05
US20080187251A1 (en) 2008-08-07
US7068860B2 (en) 2006-06-27
US20060269168A1 (en) 2006-11-30
US8036498B2 (en) 2011-10-11
US20170108957A1 (en) 2017-04-20
US8818140B2 (en) 2014-08-26
US6456749B1 (en) 2002-09-24

Similar Documents

Publication Publication Date Title
US20170108957A1 (en) Method and apparatus for recognition of writing, for remote communication, and for user defined input templates
US6486875B1 (en) Wireless computer peripheral that functions as a mouse and pen with ink processor memory power display and speaker all in one
US6633282B1 (en) Ballpoint pen type input device for computer
US5581484A (en) Finger mounted computer input device
US6806867B1 (en) Palm pad system
US6201903B1 (en) Method and apparatus for pen-based faxing
US6686910B2 (en) Combined writing instrument and digital documentor apparatus and method of use
US6212296B1 (en) Method and apparatus for transforming sensor signals into graphical images
US20030214490A1 (en) Stylus providing variable line width as a function of pressure
US6211863B1 (en) Method and software for enabling use of transcription system as a mouse
US20070103431A1 (en) Handheld tilt-text computing system and method
US20050024346A1 (en) Digital pen function control
US20110298709A1 (en) System and method for digital recording of handpainted, handdrawn and handwritten information
US20050179644A1 (en) Data input device
US20060232558A1 (en) Virtual keyboard
US6177927B1 (en) Transcription system kit
EP1257965B1 (en) Electronic pen with ink on/off function and paper touch sensing
EP0676781A1 (en) Keyboard
JP2006172230A (en) Notebook device with handwriting input function and electronic notebook system
US20050020303A1 (en) Mobile electronic device with integrated stylus input device
US20060087496A1 (en) Electronic notepad
KR20150063699A (en) Smart Pen
KR100721668B1 (en) Writing apparatus of providing data transfer using wpan and mobile communication terminal of processing written data with it
JPH0594564A (en) Data processing system
GB2354824A (en) Paper marking pen with motion sensors and wireless link to a computer

Legal Events

Date Code Title Description
STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION