WO2012122706A1 - Superimposed annotation output - Google Patents

Superimposed annotation output Download PDF

Info

Publication number
WO2012122706A1
WO2012122706A1 PCT/CN2011/071854 CN2011071854W WO2012122706A1 WO 2012122706 A1 WO2012122706 A1 WO 2012122706A1 CN 2011071854 W CN2011071854 W CN 2011071854W WO 2012122706 A1 WO2012122706 A1 WO 2012122706A1
Authority
WO
WIPO (PCT)
Prior art keywords
output
displaying
display
annotations
video
Prior art date
Application number
PCT/CN2011/071854
Other languages
French (fr)
Inventor
Xiaoru Yuan
Original Assignee
Peking University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Peking University filed Critical Peking University
Priority to KR1020137010900A priority Critical patent/KR101452667B1/en
Priority to PCT/CN2011/071854 priority patent/WO2012122706A1/en
Priority to JP2013549694A priority patent/JP5852135B2/en
Priority to CN201180068426.5A priority patent/CN103384874B/en
Priority to US13/502,011 priority patent/US9715326B2/en
Priority to EP11860725.8A priority patent/EP2686761A4/en
Publication of WO2012122706A1 publication Critical patent/WO2012122706A1/en

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/431Generation of visual interfaces for content selection or interaction; Content or additional data rendering
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/70Information retrieval; Database structures therefor; File system structures therefor of video data
    • G06F16/78Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G06F3/04883Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures for inputting data by handwriting, e.g. gesture or text
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/16Sound input; Sound output
    • G06F3/167Audio in a user interface, e.g. using voice commands for navigating, audio feedback
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/4302Content synchronisation processes, e.g. decoder synchronisation
    • H04N21/4307Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
    • H04N21/43072Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen of multiple content streams on the same device
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/4302Content synchronisation processes, e.g. decoder synchronisation
    • H04N21/4307Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
    • H04N21/43074Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen of additional data with content streams on the same device, e.g. of EPG data or interactive icon with a TV program
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47End-user applications
    • H04N21/478Supplemental services, e.g. displaying phone caller identification, shopping application
    • H04N21/4788Supplemental services, e.g. displaying phone caller identification, shopping application communicating with other users, e.g. chatting
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/63Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
    • H04N21/632Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing using a connection between clients on a wide area network, e.g. setting up a peer-to-peer communication via Internet for retrieving video segments from the hard-disk of other client devices
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00Television systems
    • H04N7/14Systems for two-way working
    • H04N7/15Conference systems

Definitions

  • Modern computing devices provide the capability to run multiple programs simultaneously.
  • Desktop computers, laptop computers, tablet computers, personal digital assistants, and even mobile phones have multi-processing capabilities that allow for multiple applications to run at the same time. Accordingly, in an example scenario, users of such devices may execute a first program for playing video and simultaneously execute a second program for recording user inputs such as notes.
  • Computing devices that have a generous display such as, for example, a desktop or laptop computer may accommodate simultaneously displaying a video in a first application and recording user inputs in a second application.
  • simultaneous viewing of two applications is difficult, if not sometimes impossible.
  • switching between application programs is not a useful alternative.
  • a user may wish to view a video stream, which may be, for example a video telephone conference, while simultaneously take notes regarding the video stream.
  • a device such as a mobile phone or PDA that has a relatively small display, simultaneously presenting both the application to view the video stream and the application for taking notes is not possible.
  • switching between the two applications prevents the user from simultaneously engaging in both activities (viewing the video and taking notes), which if not for the limits of the technology, the user would otherwise wish to do.
  • the first output may be a video stream of a video telephone conference.
  • the second output comprises annotations or notes that are received from a user during the telephone conference.
  • the annotations or notes may correspond to user inputs received on the display on which the first output is displayed.
  • the display may be a touch sensitive display, for example, and the user inputs may be handwritten notes or annotations entered on the display while the video stream of a telephone conference is displayed on the display. Accordingly, while video from a video telephone conference is being displayed, the operator may make notes or annotations on the display, and the markings or outputs
  • the video i.e., the first output
  • the annotations i.e., the second output
  • the annotations are opaque and the video is visible through the opaque second output.
  • the first output is visible.
  • the first output and the second output may be stored along with information synchronizing the two outputs.
  • the first output is a video output and the second output comprises user-entered notes or annotations
  • both the video and the annotations are stored along with the synchronizing information recording the time that the particular notes or annotations were displayed relative to the video.
  • the first output, the second output, and the synchronization information may be retrieved at a later time and used to display the outputs according to the timing sequence of the originally displayed outputs.
  • the stored first output is a video and the stored second output represents notes or annotations that were made during the playing of the video
  • the retrieved stored video is played and the annotations superimposed on the playing video at the appropriate intervals to recreate the timing that the original markings were made relative to the playing video.
  • the notes or annotations may be moved on, or removed from, the display in response to user commands and/or predetermined preferences and rules enforced (or applied) by the displaying device.
  • the first output is a video output and the second output comprises user-entered notes or annotations
  • the user may request that annotations or notes be moved on the display.
  • the user may drag on the annotations to indicate a desire to move the annotations to a different portion of the display.
  • the system may remove the currently displayed annotations from the display.
  • the device may comprise predefined rules that specify the second output should be moved to a particular portion of the display, or removed from the display, after having been displayed for a prescribed period of time.
  • FIG. 1 depicts an illustrative computing arrangement in which aspects of the subject matter described herein may be implemented.
  • FIG. 2 depicts an illustrative display interface apparatus.
  • FIG. 3 depicts an illustrative display apparatus with a first output displayed thereon.
  • FIG. 4 depicts an illustrative display apparatus with a second output displayed thereon.
  • FIG. 5 depicts an illustrative display apparatus with two outputs simultaneously displayed therein thereon.
  • FIG. 6 depicts a block diagram of example logical components comprised on an illustrative computing device that is adapted to provide superimposed display outputs.
  • FIG. 7 is a flow diagram of an example process for presenting a second output superimposed on a first output.
  • FIG. 8 is a flow diagram of an example process for storing and retrieving a first output and a second output.
  • FIG. 9 is a flow diagram of an example process for moving a second output on a display apparatus and removing a second output from a display apparatus.
  • FIG. 10 is a block diagram of a computing environment with which aspects of the subject matter described herein may be deployed.
  • Modern computing devices provide the capability to run multiple programs simultaneously. Accordingly, in an example scenario, users may execute a first program for playing video and simultaneously execute a second program for recording user inputs such as notes.
  • Computing devices that have a generous display interface such as, for example, a desktop or laptop computer may accommodate simultaneously displaying a video in a first application and recording user inputs in a second application.
  • a generous display interface such as, for example, a desktop or laptop computer may accommodate simultaneously displaying a video in a first application and recording user inputs in a second application.
  • computing devices such as mobile phones and personal digital assistants that have relatively limited or small display interfaces, simultaneous viewing of two applications is difficult.
  • Technologies are generally disclosed for systems and methods effective to display a first output and simultaneously display a second output corresponding to user input superimposed on the first output.
  • the second output In an area of the display that the first output and the second output overlap, the second output is opaque and the first output is visible through the opaque second output. In an area of the display that the first output and the second output do not overlap, the first output is visible.
  • the first output may be a video portion of a video telephone conference
  • the second output may be annotations or notes that are received from a user during the telephone conference. The annotations or notes may be received on the display on which the first output is displayed.
  • FIG. 1 depicts an illustrative computing arrangement in which aspects of the subject matter described herein may be implemented.
  • FIG. 1 illustrates an example computing arrangement 100 comprised of computing devices 110 each of which may be adapted to provide superimposed output as described herein.
  • Devices 110 may comprise, for example, any of desktop computer 110a, laptop computer 110b, phone 110c, tablet computing device l lOd, personal digital assistant (PDA) HOe, and mobile phone l lOf, each of which may be adapted to simultaneously display two output feeds, for example, from two executing applications.
  • PDA personal digital assistant
  • any one of devices l lOa-f may display output of a first application, e.g., display a video output, and simultaneously display input to a second application, e.g., display notes or annotations that were entered by the device operator. More particularly, any one of devices l lOa-f may be programmed to display a video while simultaneously receiving
  • any of devices l lOa-f may display a still image and simultaneously display inputs that were entered by the device operator while the image was displayed.
  • Each of devices 110 may be adapted to communicate with communications network 150.
  • Communications network 150 may be any type of network that is suitable for providing communications between computing devices l lOa-f and any servers accessed by devices l lOa-f.
  • Communications network 150 may comprise a combination of discrete networks which may use different technologies.
  • communications network 150 may comprise local area networks (LANs), wide area networks (WAN's), cellular networks, or combinations thereof.
  • Communications network 150 may comprise wireless, wireline, or combination thereof.
  • communications network 150 comprises the Internet and may additionally comprise any networks adapted to communicate with the Internet.
  • Communications network 150 may comprise a wireless telephony network that is adapted to communicate video, audio, and other data between devices 110.
  • a telephony network is adapted to communicate video telephone conferences between devices 110.
  • FIG. 2 depicts an illustrative display apparatus 200.
  • Display apparatus 200 may be employed as a display component in any of the devices depicted in FIG. 1, although the size of display apparatus 200 may vary depending on the particular system of which it is a part. For example, if display apparatus 200 is comprised in desktop system 110a, it likely will be larger than in the circumstance that display apparatus 200 is comprised in phone 11 Of or PDA l lOe.
  • Display apparatus 200 may be any device suitable for electronically displaying output and may be adapted to display, for example, video, still images, text, graphics, as well as any other type of output that may be electronically represented.
  • display apparatus 200 may also comprise an input device such as, for example, a touch sensitive screen 202.
  • touch sensitive screen 202 is adapted to receive inputs on its surface and generate electronic markings on display apparatus 200 at locations corresponding to the location of inputs. For example, a user may use his/her fingers and hands and/or a pointing device such as a stylus on touch sensitive screen 202 to enter inputs, which are reflected by electronic markings being displayed on display apparatus 200 at locations corresponding to the inputs.
  • FIG. 3 depicts illustrative display apparatus 200 with first output 310 displayed thereon.
  • first output 310 represents a person's image.
  • first output 310 may be a video image, and may be, for example, the video stream of a video telephone conference. While in the example depicted in FIG. 3 first output 310 illustrates the likeness of a person, and represents a video output, first output 310 may represent any type of likeness and may be any type of content including, for example, still images, graphics, and/or text. Moreover, first output 310 may be any output that is presented on display apparatus 200. [0025] FIG.
  • second output 410 is text.
  • second output 410 may be text corresponding to user input. More particularly, second output 410 may comprise text corresponding to user input received on touch sensitive screen 202 or entered through some other input means such as, for example, a keyboard or computerized voice recognition. While in the example depicted in FIG. 4 second output 410 illustrates text, second output 410 may represent any type of content including, for example, video, still images, graphics, and/or text.
  • display apparatus 200 is adapted to display first output 310, which in an example embodiment, may be a video feed of a telephone conference, and second output 410, which in an example embodiment may be text or annotations corresponding to user input.
  • first output 310 which in an example embodiment, may be a video feed of a telephone conference
  • second output 410 which in an example embodiment may be text or annotations corresponding to user input.
  • display apparatus 200 will likely not be large enough to accommodate simultaneously displaying both first output 310 and also second output 410.
  • switching between the two outputs that are displayed on display apparatus 200 is less than advantageous as it prevents the user from engaging in more than one activity at one time.
  • any one of devices l lOa-f may comprise display apparatus 200 which is adapted to superimpose two outputs.
  • FIG. 5 depicts illustrative display apparatus 200 with two outputs simultaneously displayed thereon. As shown, first output 310 is displayed and second output 410 is displayed superimposed on top of first output 310. In the area of display apparatus 200 wherein second output 410 overlaps first output 310, second output 310 is opaque, but first output 310 is visible beneath second output 410. In areas where first output 310 is not overlapped by second output 410, first output 310 is plainly visible. Second output 410 may have a level of transparency that allows for first output 310 to be viewable beneath second output 410.
  • the level of opaqueness of second output 410 may be varied and thereby increase or decrease the visibility of first output 310 in the portions that they two overlap. Furthermore, second output 410 may be displayed with a particular coloring scheme so as to be more or less discernable while still allowing first output 310 to be visible. In the scenario wherein second output 410 corresponds to user inputs entered on touch sensitive screen 202, second output 410 may be displayed using a particular color of grey that stands out relative to the background on which it is displayed. In alternative embodiments, the color of second output 410 may be red, yellow, black, or any other color that is suitable for the application in order to make second output 410 more or less discernable.
  • First output 310 may be any type of data including, for example, video, still images, graphics, text, etc.
  • first output 310 may be a video stream, and more particularly, a video stream portion of an audio/video telephone conference.
  • Second output 410 may be output corresponding to user inputs received while viewing the video stream.
  • the user input may be received by any suitable means.
  • the user inputs may be received on touch sensitive screen 202.
  • the user of display apparatus 200 may view first output 310 and simultaneously enter notes or annotations on touch sensitive screen 202. Second output 410 corresponding to the received user inputs is superimposed on first output 310.
  • first output 310 is a video stream of a telephone conference
  • the person participating in the call is able to continue to participate in the telephone call and view the video while also taking notes or annotations on touch sensitive screen 202.
  • first output 310 continues to be visible and discernable even though second output 410 is superimposed thereon.
  • the combination of displaying a video on display apparatus 200 while simultaneously presenting handwritten annotations creates the sensation of talking with someone behind a glass separator on which notes may be recorded.
  • FIG. 6 depicts a block diagram of example logical components comprised on an illustrative computing device 110 that is adapted to provide superimposed display outputs.
  • the logic components may be comprised on any type of computing device including, for example, desktop 110a, laptop 110b, phone 110c, tablet computing device l lOd, PDA l lOe, and mobile phone 11 Of.
  • an illustrative device 110 may comprise received data control logic 610 that operates to receive data for display on display apparatus 200.
  • received data control logic 610 may be adapted to coordinate the receipt of a video stream or any type of data including, for example, still images, text, and/or graphics.
  • illustrative device 110 may further comprise user interface input control logic 612 which operates to control the user inputs received from, for example, a touch sensitive screen, a keyboard, etc.
  • user interface input control logic 612 may be adapted to receive inputs from a touch sensitive screen 202.
  • User interface input control logic 612 may be adapted to receive any type of user inputs including, for example, video, images, still images, text, and graphics.
  • user interface input logic 612 may be adapted to control inputs from any suitable input device including, for example, touch sensitive screens or displays, physical keyboards, virtual keyboards, pointing mice, computerized voice recognition systems, etc.
  • User interface input logic 612 and received data control logic 610 may be comprised in the same or separate software applications.
  • Output control logic 614 is adapted to control the device output.
  • output control logic 614 may allow for the display of first output 310 on display apparatus 200 and superimpose second output 410 on first output 310.
  • output control logic 614 may be adapted to display a video stream of a video conference and annotations corresponding to user inputs superimposed on the video stream.
  • output control logic 614 may be adapted to display still images and annotations received from touch sensitive screen 202.
  • Output control logic 614 may be comprised in a common program application with received data control logic 610 and user interface input control logic 612, but alternatively may be separate from one or more applications comprising received data control logic 610 and user interface input control logic 612.
  • Output control logic 614 may be adapted to maintain the integrity of first output 310 and second output 410 while the two outputs are displayed on display apparatus 200. In other words, the distinctness of second output 410 is maintained even though changes may be made to first output 310 while second output 410 is superimposed on first output 310. For example, where first output 310 comprises video output, which by its nature, changes over time, the distinctness and sharpness of second output 410 is not impacted by the successive change in a video stream comprised in first output 310. Output control logic 614 may buffer data
  • Received data store 616 comprises logic and computing memory for storing data that is received and displayed on display apparatus 200.
  • received data store 616 stores a copy of first output 310.
  • the video stream and corresponding audio stream may be stored in received data store 616.
  • User input data store 618 comprises logic and computing memory for storing data corresponding to user inputs and their corresponding outputs. For example, in an example scenario wherein user inputs are received on a touch sensitive screen 202, data corresponding to the inputs, as well as the corresponding outputs that were generated on the display apparatus 200 are stored in user input data store 618. Also, any data that may be used for synchronizing the user input data and corresponding outputs with any other outputs such as, for example, video outputs, may be stored in user input data store 618. For example, information regarding timing of the receipt of inputs and generation of corresponding output relative to the display of a video may be recorded.
  • the timing information may be any type of information that is suitable for synchronizing the display of two outputs consistent with the timing intervals of the original display.
  • the timing information may comprise absolute timing information or may be timing information that is relative to the playing time of the first output. For example, the timing information may specify that a particular second output was displayed at three minutes into the play time of a corresponding first output.
  • Synchronization control logic 614 is adapted to control the synchronized display of stored data. For example, in response to a request to display stored data 616, synchronization control logic is adapted to retrieve data from stores 616 and 618 and use the synchronization information stored in store 618 in order to have output control logic 614 display the stored data consistent with the timing of the original display of the two output streams. For example, in an example scenario wherein output corresponding to user inputs received on touch sensitive screen 202 was displayed superimposed on a video stream presented on display apparatus 200, synchronization control logic 614 may be adapted to present the corresponding stored data from data stores 616 and 618 in a manner that is consistent with the timing and sequence that applied when the two outputs were originally displayed.
  • FIG. 7 is a flow diagram of an example process for presenting second output 410 superimposed on first output 310.
  • the example process may be implemented on any suitable device including, for example, any of devices l lOa-f.
  • An example process may include one or more operations, actions, or functions as illustrated by one or more of blocks 710, 712, and/or 714.
  • blocks 710, 712, and/or 714 may be divided into additional blocks, combined into fewer blocks, or eliminated, depending on the desired implementation. Processing may begin at block 710.
  • first output 310 is displayed on display apparatus 200.
  • First output 310 may be any output suitable to be displayed on display apparatus 200.
  • first output 310 may be video, images, still images, text, graphics, or any other output type.
  • first output 310 may comprise the video stream of a telephone conference.
  • audio corresponding to the video may be output by a speaker component of device 110. Processing may continue from block 710 to block 712
  • user input is received at device 110.
  • the inputs may be any that are suitable to be received on device 110 and may comprise, for example, touch inputs, mechanical inputs, and/or audio inputs. Further, the inputs may be received via any suitable input device, including, for example, a touch sensitive screen, a physical keyboard, a virtual keyboard, a mouse, a microphone with voice recognition software, etc.
  • inputs are received on touch sensitive screen 202 while first output 310 is displayed on display apparatus 200. More particularly, handwritten inputs may be received on touch sensitive screen 202 while a video stream, which may be a video telephone conference stream, is displayed on display apparatus 200. Processing may continue from block 712 to block 714.
  • second output 410 which corresponds to the user inputs received at block 712, is displayed on display apparatus 200.
  • Second output 410 is displayed simultaneously with, and superimposed on first output 310.
  • second output 410 is opaque and first output 310 is visible through opaque second output 410.
  • Second output 410 may be, at least in part, transparent so first output 310 may be visible on display apparatus 200.
  • first output 310 is visible.
  • the degree or level of opaqueness may vary depending upon the preferences of the device operator.
  • the level of the opaqueness may vary from very opaque to very transparent.
  • the degree to which first output 310 is partially obscured or darkened by second output 410 in the area that the two overlap may vary from greatly to only slightly. Indeed, in some embodiments, first output 310 may be almost completely obscured, while in other embodiments, first output 310 is only slightly obscured.
  • device 110 operates to maintain the integrity of first output 310 and second output 410 while the two outputs are displayed on display apparatus 200.
  • the distinctness of second output 410 is maintained even though changes may be made to first output 310 while second output 410 is superimposed on first output 310.
  • the distinctness and integrity of first output 310 is maintained even though changes may be made to second output 410 while second output 410 is superimposed on first output 310.
  • first output 310 comprises video output. By its nature, video output changes over time so as to depict motion. The distinctness and sharpness of second output 410 is not impacted by the successive change in a video stream comprised in first output 310.
  • second output 410 changes over time such as, for example, when successive handwritten notes are received, the distinctness of first output 310 is maintained.
  • device 110 may maintain the integrity and distinctness of outputs 310 and 410 in any suitable manner.
  • display apparatus 220 consistent with the typical operation of display devices, continuously refreshes the data that is presented.
  • device 110 may buffer data corresponding to outputs 310 and 410 prior to display.
  • Device 110, and in particular output control 614 logic may map the buffered data for second output 410 into or over the buffered data for first output 310.
  • output control logic 614 may map data for second output 410 and first output 310 to pixels locations on display apparatus 220. The buffering and mapping allows for controlling the integrity of separate outputs 310 and 410.
  • Data comprising second output 410 data mapped onto first output 310 data is then displayed by display apparatus 220.
  • the logic for buffering and mapping the data prior to display may be implemented, for example, by a graphics and or display controller that controls the pixel display on display apparatus 220.
  • first and second outputs 310 and 410 may be stored and later retrieved and displayed.
  • the outputs may be retrieved and displayed separately and independently. However, the outputs may also be retrieved and displayed simultaneously, with the outputs being synchronized and displayed in the same relative timing as was used when the outputs were originally displayed.
  • FIG. 8 is a flow diagram of an example process for storing and retrieving first output 310 and second output 410.
  • An example process may include one or more operations, actions, or functions as illustrated by one or more of blocks 810, 812, 814, and/or 816. Processing may begin at block 810.
  • first output 310 is stored in, for example, database store 616.
  • database store 616 For example, in an example scenario wherein first output 310 is the output of a video telephone conference, the video stream and, in some scenarios, the audio stream are stored in database store 616. Processing may continue from block 810 to block 812.
  • second output 410 and the corresponding user inputs are stored or recorded, for example, in database store 618.
  • database store 618 For example, in an example scenario, wherein user inputs were received and displayed as notes or annotations on display apparatus 200, at block 812, those annotations are stored in database store 618. Processing may continue from block 812 to block 814.
  • synchronization information may be stored in user input data store 618.
  • the synchronization information that is stored may be any data useful in recording and recreating the temporal arrangement of second output 410 relative to first output 310. In other words, data indicating when second output 410 was displayed relative to first output 310 may be recorded.
  • first output 310 is a video stream of an audio/video conference
  • second output 410 comprises annotations or notes corresponding to user inputs
  • synchronization information captures the times at which various second outputs 410 were displayed relative to the playtime of first output 310.
  • the synchronization information comprises information recording that second output 410 was displayed 30 seconds into the video stream.
  • information recording that second output 410 was displayed 30 seconds into the video stream In an embodiment wherein audio portions of first output 310 are recorded, information
  • synchronizing audio with second output 410 may also be recorded. Processing may continue from block 814 to 816.
  • the stored data for first output 310 and/or stored data for second output 410 may be retrieved and displayed.
  • stored first output 310 comprises a video stream and an audio stream
  • stored second output 410 comprises annotations that were made and superimposed on the video stream
  • the stored information may be retrieved from memory and displayed.
  • second output 410 and first output 310 are synchronized so that second output 410 is displayed at an appropriate time relative to first output 310 consistent with the relative timing of display when first output 310 and second output 410 were originally displayed.
  • stored first output 310 may comprise video and audio
  • stored second output 410 may comprise annotations. All may be retrieved from memory and output consistent with the stored synchronization information.
  • second output 410 which may comprise notes or annotations, may be displayed separately from first output 310.
  • second output 410 may be retrieved and displayed while an audio portion of first output 310 is replayed.
  • the annotations or notes are displayed synchronized with the audio replay so as to be consistent with the temporal placement of the notes or annotations relative to the audio stream.
  • second output 410 may be moved and/or removed from display apparatus 200. Such functionality may be particularly useful wherein display apparatus 200 is relatively small and second output 410 covers too much of, or a particular portion of, display apparatus 200. Second output 410 may be moved on display apparatus 200 so as to be less obtrusive. Alternatively, or in addition, second output 410 may be removed from display apparatus 200.
  • FIG. 9 is a flow diagram of an example processes for moving a second output on a display apparatus and/or removing a second output from a display apparatus.
  • An example process may include one or more operations, actions, or functions as illustrated by one or more of blocks 910 and/or 912. Processing may begin at block 910.
  • second output 410 may be moved on display apparatus 200.
  • second output 410 may be moved so as to have a smaller amount of overlap with first output 310.
  • second output 410 may be moved on display apparatus 200 in response to a user input.
  • second output 410 may be moved in response to a mouse cursor dragging on second output 410.
  • second output 410 may be moved automatically by device 110 after a prescribed period of time.
  • second output 410 may be moved automatically by device 110 to a particular portion of display apparatus 200 such as, for example, the bottom portion, after second output 410 has been displayed for 5 minutes. Processing may continue from block 910 to 912.
  • second output 410 may be removed from display apparatus 200.
  • second output 410 may be removed in response to user input or in response to rules or requirements enforced or applied by device 110.
  • second output 410 may be removed from display apparatus 200 in response to user input.
  • a user may select to clear all or a portion of the notes or annotations that are displayed on display apparatus 200 so as to make space for additional notes or annotations.
  • device 110 may comprise rules that specify outputs corresponding to user inputs should be removed after a prescribed period of time has expired. For example, after an output has been displayed for 10 minutes, device 110 may remove display apparatus 200.
  • FIG. 10 depicts a block diagram of example computing system 1000 that may be used to implement the systems and methods described herein.
  • computing system 1000 may be used to implement any of devices l lOa-f.
  • Computing system 1000 may be controlled primarily by computer readable instructions that may be in the form of software.
  • computing system 1000 may be controlled by a computing application 1080 which comprises instructions for performing functionality consistent with that described herein.
  • the computer readable instructions may include instructions for computing system 1000 for storing and accessing computer readable instructions themselves.
  • Such software may be executed within central processing unit (CPU) 1010 to cause computing system 1000 to perform the processes or functions associated therewith.
  • CPU 1010 may be implemented by micro-electronic chip CPUs called microprocessors.
  • CPU 1010 may fetch, decode, and/or execute instructions and may transfer information to and from other resources via a main data-transfer path or system bus 1005.
  • a system bus may connect the components in computing system 1000 and may define the medium for data exchange.
  • Computing system 1000 may further include memory devices coupled to system bus 1005.
  • the memory devices may include random access memory (RAM) 1025 and read only memory (ROM) 1030.
  • RAM 1025 and ROM 1030 may include circuitry that allows information to be stored and retrieved.
  • ROM 1030 may include stored data that cannot be modified. Additionally, data stored in RAM 1025 typically may be read or changed by CPU 1010 or other hardware devices. Access to RAM 1025 and/or ROM 1030 may be controlled by memory controller 1020.
  • Memory controller 1020 may provide an address translation function that translates virtual addresses into physical addresses as instructions are executed.
  • computing system 1000 may include peripherals controller 1035 that may be responsible for communicating instructions from CPU 1010 to peripherals, such as, touch sensitive input 1040, keyboard 1045, mouse 1050, and audio input and output device 1055.
  • Computing system 1000 may further include display 1065 that may be controlled by display controller 1063.
  • Display 1065 may be used to display visual output generated by computing system 1000 and may correspond to display apparatus 200 discussed herein. Such visual output may include text, graphics, animated graphics, video, or the like.
  • Display controller 1063 may include electronic components that generate a video signal that may be sent to display 1065.
  • computing system 1000 may include network adaptor 1070 that may be used to connect computing system 2000 to an external communication network such as network 150, described above in connection with Figure 1.
  • the computing device generally includes a processor, a storage medium readable by the processor (including volatile and non- volatile memory and/or storage elements), at least one input device, and at least one output device.
  • One or more programs that may implement or utilize the processes described in connection with the subject matter described herein, e.g., through the use of an API, reusable controls, or the like. Such programs are preferably implemented in a high level procedural or object oriented programming language to communicate with a computer system. However, the program(s) can be
  • the language may be a compiled or interpreted language, and combined with hardware implementations.
  • example embodiments may refer to utilizing aspects of the subject matter described herein in the context of one or more stand-alone computer systems or devices, the subject matter described herein is not so limited, but rather may be implemented in connection with any computing environment, such as a network or distributed computing environment. Still further, aspects of the subject matter described herein may be implemented in or across a plurality of processing chips or devices, and storage may similarly be spread across a plurality of devices. Such devices might include personal computers, network servers, handheld devices, supercomputers, or computers integrated into other systems such as automobiles and airplanes.
  • a first output may be a video stream that is not portion of a telephone conference.
  • the first output may be a still image.
  • the second output may be entered through a keyboard as opposed to a touch sensitive screen.

Abstract

Technologies are generally described for systems and methods effective to display a first output and simultaneously display a second output corresponding to user input superimposed on the first output. In some examples, a second output is displayed on a display simultaneously with and superimposed on a first output. The second output corresponds to user input. In an area of the display that the first output and the second output overlap, the second output is opaque and the first output is visible through the opaque second output. In an area of the display that the first output and the second output do not overlap, the first output is visible. The first output may be a video portion of a video telephone conference, and the second output may correspond to annotations or notes that are received from a user during the telephone conference.

Description

SUPERIMPOSED ANNOTATION OUTPUT
BACKGROUND
[0001] Modern computing devices provide the capability to run multiple programs simultaneously. Desktop computers, laptop computers, tablet computers, personal digital assistants, and even mobile phones have multi-processing capabilities that allow for multiple applications to run at the same time. Accordingly, in an example scenario, users of such devices may execute a first program for playing video and simultaneously execute a second program for recording user inputs such as notes.
[0002] Computing devices that have a generous display such as, for example, a desktop or laptop computer may accommodate simultaneously displaying a video in a first application and recording user inputs in a second application. But for computing devices such as mobile phones and personal digital assistants that have relatively small displays, simultaneous viewing of two applications is difficult, if not sometimes impossible. Furthermore, depending upon the nature of the applications and the user's intended use of those applications, switching between application programs is not a useful alternative. For example, a user may wish to view a video stream, which may be, for example a video telephone conference, while simultaneously take notes regarding the video stream. On a device such as a mobile phone or PDA that has a relatively small display, simultaneously presenting both the application to view the video stream and the application for taking notes is not possible. Furthermore, switching between the two applications prevents the user from simultaneously engaging in both activities (viewing the video and taking notes), which if not for the limits of the technology, the user would otherwise wish to do.
SUMMARY
[0003] Technologies are generally described for systems and methods effective to display a first output and simultaneously display a second output corresponding to user input superimposed on the first output. In an area of the display that the first output and the second output overlap, the second output is opaque and the first output is visible through the opaque second output. In an area of the display that the first output and the second output do not overlap, the first output is visible.
[0004] In an example scenario, the first output may be a video stream of a video telephone conference. The second output comprises annotations or notes that are received from a user during the telephone conference. The annotations or notes may correspond to user inputs received on the display on which the first output is displayed. The display may be a touch sensitive display, for example, and the user inputs may be handwritten notes or annotations entered on the display while the video stream of a telephone conference is displayed on the display. Accordingly, while video from a video telephone conference is being displayed, the operator may make notes or annotations on the display, and the markings or outputs
corresponding to those user inputs are displayed superimposed on the video output. In an area of the display that the video, i.e., the first output, and the annotations, i.e., the second output, overlap, the annotations are opaque and the video is visible through the opaque second output. In an area of the display that the first output and the second output do not overlap, the first output is visible.
[0005] According to another aspect of the disclosed embodiments, the first output and the second output may be stored along with information synchronizing the two outputs. For example, in the example scenario wherein the first output is a video output and the second output comprises user-entered notes or annotations, both the video and the annotations are stored along with the synchronizing information recording the time that the particular notes or annotations were displayed relative to the video. The first output, the second output, and the synchronization information may be retrieved at a later time and used to display the outputs according to the timing sequence of the originally displayed outputs. For example, in the example scenario wherein the stored first output is a video and the stored second output represents notes or annotations that were made during the playing of the video, the retrieved stored video is played and the annotations superimposed on the playing video at the appropriate intervals to recreate the timing that the original markings were made relative to the playing video.
[0006] According to another aspect of the disclosed embodiments, the notes or annotations may be moved on, or removed from, the display in response to user commands and/or predetermined preferences and rules enforced (or applied) by the displaying device. For example, in the example scenario wherein the first output is a video output and the second output comprises user-entered notes or annotations, the user may request that annotations or notes be moved on the display. The user may drag on the annotations to indicate a desire to move the annotations to a different portion of the display. Similarly, in response to user input, the system may remove the currently displayed annotations from the display. In another embodiment, the device may comprise predefined rules that specify the second output should be moved to a particular portion of the display, or removed from the display, after having been displayed for a prescribed period of time.
[0007] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description of Illustrative Embodiments. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. Other features are described below.
BRIEF DESCRIPTION OF THE DRAWINGS
[0008] FIG. 1 depicts an illustrative computing arrangement in which aspects of the subject matter described herein may be implemented.
[0009] FIG. 2 depicts an illustrative display interface apparatus.
[0010] FIG. 3 depicts an illustrative display apparatus with a first output displayed thereon.
[0011] FIG. 4 depicts an illustrative display apparatus with a second output displayed thereon.
[0012] FIG. 5 depicts an illustrative display apparatus with two outputs simultaneously displayed therein thereon.
[0013] FIG. 6 depicts a block diagram of example logical components comprised on an illustrative computing device that is adapted to provide superimposed display outputs. [0014] FIG. 7 is a flow diagram of an example process for presenting a second output superimposed on a first output.
[0015] FIG. 8 is a flow diagram of an example process for storing and retrieving a first output and a second output.
[0016] FIG. 9 is a flow diagram of an example process for moving a second output on a display apparatus and removing a second output from a display apparatus.
[0017] FIG. 10 is a block diagram of a computing environment with which aspects of the subject matter described herein may be deployed.
DETAILED DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS
[0018] In the following detailed description, reference is made to the accompanying drawings, which form a part hereof. In the drawings, similar symbols typically identify similar components, unless context dictates otherwise. The illustrative embodiments described in the detailed description, drawings, and claims are not meant to be limiting. Other embodiments may be utilized, and other changes may be made, without departing from the spirit or scope of the subject matter presented herein. It will be readily understood that the aspects of the present disclosure, as generally described herein, and illustrated in the Figures, can be arranged, substituted, combined, separated, and designed in a wide variety of different configurations, all of which are explicitly contemplated herein.
Overview
[0019] Modern computing devices provide the capability to run multiple programs simultaneously. Accordingly, in an example scenario, users may execute a first program for playing video and simultaneously execute a second program for recording user inputs such as notes. Computing devices that have a generous display interface such as, for example, a desktop or laptop computer may accommodate simultaneously displaying a video in a first application and recording user inputs in a second application. But for computing devices such as mobile phones and personal digital assistants that have relatively limited or small display interfaces, simultaneous viewing of two applications is difficult. [0020] Technologies are generally disclosed for systems and methods effective to display a first output and simultaneously display a second output corresponding to user input superimposed on the first output. In an area of the display that the first output and the second output overlap, the second output is opaque and the first output is visible through the opaque second output. In an area of the display that the first output and the second output do not overlap, the first output is visible. In an example scenario, the first output may be a video portion of a video telephone conference, and the second output may be annotations or notes that are received from a user during the telephone conference. The annotations or notes may be received on the display on which the first output is displayed.
Example Computing Arrangement
[0021] FIG. 1 depicts an illustrative computing arrangement in which aspects of the subject matter described herein may be implemented. In particular, FIG. 1 illustrates an example computing arrangement 100 comprised of computing devices 110 each of which may be adapted to provide superimposed output as described herein. Devices 110 may comprise, for example, any of desktop computer 110a, laptop computer 110b, phone 110c, tablet computing device l lOd, personal digital assistant (PDA) HOe, and mobile phone l lOf, each of which may be adapted to simultaneously display two output feeds, for example, from two executing applications. In an example scenario, any one of devices l lOa-f may display output of a first application, e.g., display a video output, and simultaneously display input to a second application, e.g., display notes or annotations that were entered by the device operator. More particularly, any one of devices l lOa-f may be programmed to display a video while simultaneously receiving
handwritten annotations on a touch screen display and displaying markings corresponding to the annotations on the display. In an alternate scenario, any of devices l lOa-f may display a still image and simultaneously display inputs that were entered by the device operator while the image was displayed.
[0022] Each of devices 110 may be adapted to communicate with communications network 150. Communications network 150 may be any type of network that is suitable for providing communications between computing devices l lOa-f and any servers accessed by devices l lOa-f. Communications network 150 may comprise a combination of discrete networks which may use different technologies. For example, communications network 150 may comprise local area networks (LANs), wide area networks (WAN's), cellular networks, or combinations thereof. Communications network 150 may comprise wireless, wireline, or combination thereof. In an example embodiment, communications network 150 comprises the Internet and may additionally comprise any networks adapted to communicate with the Internet. Communications network 150 may comprise a wireless telephony network that is adapted to communicate video, audio, and other data between devices 110. In a particular scenario, a telephony network is adapted to communicate video telephone conferences between devices 110.
[0023] FIG. 2 depicts an illustrative display apparatus 200. Display apparatus 200 may be employed as a display component in any of the devices depicted in FIG. 1, although the size of display apparatus 200 may vary depending on the particular system of which it is a part. For example, if display apparatus 200 is comprised in desktop system 110a, it likely will be larger than in the circumstance that display apparatus 200 is comprised in phone 11 Of or PDA l lOe. Display apparatus 200 may be any device suitable for electronically displaying output and may be adapted to display, for example, video, still images, text, graphics, as well as any other type of output that may be electronically represented. In an example embodiment, display apparatus 200 may also comprise an input device such as, for example, a touch sensitive screen 202. In an example embodiment, touch sensitive screen 202 is adapted to receive inputs on its surface and generate electronic markings on display apparatus 200 at locations corresponding to the location of inputs. For example, a user may use his/her fingers and hands and/or a pointing device such as a stylus on touch sensitive screen 202 to enter inputs, which are reflected by electronic markings being displayed on display apparatus 200 at locations corresponding to the inputs.
[0024] FIG. 3 depicts illustrative display apparatus 200 with first output 310 displayed thereon. In the illustrative embodiment of FIG. 3, first output 310 represents a person's image. In an example scenario, first output 310 may be a video image, and may be, for example, the video stream of a video telephone conference. While in the example depicted in FIG. 3 first output 310 illustrates the likeness of a person, and represents a video output, first output 310 may represent any type of likeness and may be any type of content including, for example, still images, graphics, and/or text. Moreover, first output 310 may be any output that is presented on display apparatus 200. [0025] FIG. 4 depicts illustrative display apparatus 200 with second output 410 displayed thereon. In the illustrative embodiment of FIG. 4, second output 410 is text. In an example scenario, second output 410 may be text corresponding to user input. More particularly, second output 410 may comprise text corresponding to user input received on touch sensitive screen 202 or entered through some other input means such as, for example, a keyboard or computerized voice recognition. While in the example depicted in FIG. 4 second output 410 illustrates text, second output 410 may represent any type of content including, for example, video, still images, graphics, and/or text.
[0026] Thus, as depicted in FIG. 3 and FIG. 4, display apparatus 200 is adapted to display first output 310, which in an example embodiment, may be a video feed of a telephone conference, and second output 410, which in an example embodiment may be text or annotations corresponding to user input. For devices such as, for example, mobile phone l lOf and PDA l lOe, display apparatus 200 will likely not be large enough to accommodate simultaneously displaying both first output 310 and also second output 410. Furthermore, switching between the two outputs that are displayed on display apparatus 200 is less than advantageous as it prevents the user from engaging in more than one activity at one time.
[0027] In an illustrative embodiment, any one of devices l lOa-f may comprise display apparatus 200 which is adapted to superimpose two outputs. FIG. 5 depicts illustrative display apparatus 200 with two outputs simultaneously displayed thereon. As shown, first output 310 is displayed and second output 410 is displayed superimposed on top of first output 310. In the area of display apparatus 200 wherein second output 410 overlaps first output 310, second output 310 is opaque, but first output 310 is visible beneath second output 410. In areas where first output 310 is not overlapped by second output 410, first output 310 is plainly visible. Second output 410 may have a level of transparency that allows for first output 310 to be viewable beneath second output 410. The level of opaqueness of second output 410 may be varied and thereby increase or decrease the visibility of first output 310 in the portions that they two overlap. Furthermore, second output 410 may be displayed with a particular coloring scheme so as to be more or less discernable while still allowing first output 310 to be visible. In the scenario wherein second output 410 corresponds to user inputs entered on touch sensitive screen 202, second output 410 may be displayed using a particular color of grey that stands out relative to the background on which it is displayed. In alternative embodiments, the color of second output 410 may be red, yellow, black, or any other color that is suitable for the application in order to make second output 410 more or less discernable.
[0028] First output 310 may be any type of data including, for example, video, still images, graphics, text, etc. In an example scenario, first output 310 may be a video stream, and more particularly, a video stream portion of an audio/video telephone conference. Second output 410 may be output corresponding to user inputs received while viewing the video stream. The user input may be received by any suitable means. In an example scenario, the user inputs may be received on touch sensitive screen 202. In such a scenario, the user of display apparatus 200 may view first output 310 and simultaneously enter notes or annotations on touch sensitive screen 202. Second output 410 corresponding to the received user inputs is superimposed on first output 310. The user is able to view first output 310 and simultaneously capture and display notes and annotations, i.e., second output 410, that may be prompted by first output 310. In the scenario wherein first output 310 is a video stream of a telephone conference, the person participating in the call is able to continue to participate in the telephone call and view the video while also taking notes or annotations on touch sensitive screen 202. In this scenario, first output 310 continues to be visible and discernable even though second output 410 is superimposed thereon. The combination of displaying a video on display apparatus 200 while simultaneously presenting handwritten annotations creates the sensation of talking with someone behind a glass separator on which notes may be recorded.
[0029] FIG. 6 depicts a block diagram of example logical components comprised on an illustrative computing device 110 that is adapted to provide superimposed display outputs. The logic components may be comprised on any type of computing device including, for example, desktop 110a, laptop 110b, phone 110c, tablet computing device l lOd, PDA l lOe, and mobile phone 11 Of. In an example embodiment, an illustrative device 110 may comprise received data control logic 610 that operates to receive data for display on display apparatus 200. For example, received data control logic 610 may be adapted to coordinate the receipt of a video stream or any type of data including, for example, still images, text, and/or graphics.
[0030] In an example embodiment, illustrative device 110 may further comprise user interface input control logic 612 which operates to control the user inputs received from, for example, a touch sensitive screen, a keyboard, etc. In an example scenario wherein device 110 is adapted to display a still image or video stream of a video conference, user interface input control logic 612 may be adapted to receive inputs from a touch sensitive screen 202. User interface input control logic 612 may be adapted to receive any type of user inputs including, for example, video, images, still images, text, and graphics. Likewise, user interface input logic 612 may be adapted to control inputs from any suitable input device including, for example, touch sensitive screens or displays, physical keyboards, virtual keyboards, pointing mice, computerized voice recognition systems, etc. User interface input logic 612 and received data control logic 610 may be comprised in the same or separate software applications.
[0031] Output control logic 614 is adapted to control the device output. For example, output control logic 614 may allow for the display of first output 310 on display apparatus 200 and superimpose second output 410 on first output 310. In an example scenario, output control logic 614 may be adapted to display a video stream of a video conference and annotations corresponding to user inputs superimposed on the video stream. In another example scenario, output control logic 614 may be adapted to display still images and annotations received from touch sensitive screen 202. Output control logic 614 may be comprised in a common program application with received data control logic 610 and user interface input control logic 612, but alternatively may be separate from one or more applications comprising received data control logic 610 and user interface input control logic 612.
[0032] Output control logic 614 may be adapted to maintain the integrity of first output 310 and second output 410 while the two outputs are displayed on display apparatus 200. In other words, the distinctness of second output 410 is maintained even though changes may be made to first output 310 while second output 410 is superimposed on first output 310. For example, where first output 310 comprises video output, which by its nature, changes over time, the distinctness and sharpness of second output 410 is not impacted by the successive change in a video stream comprised in first output 310. Output control logic 614 may buffer data
corresponding to outputs 310 and 410 prior to display, and map the buffered data for second output 410 over the buffered data for first output 310. For example, output control logic 614 may map data for second output 410 and first output 310 to pixels locations on display apparatus 220. The mapping allows for controlling the integrity of separate outputs 310 and 410. Output control logic 614 may then communicate the second output 410 data mapped onto first output 310 data for display. [0033] Received data store 616 comprises logic and computing memory for storing data that is received and displayed on display apparatus 200. For example, received data store 616 stores a copy of first output 310. In an example scenario wherein device 110 is adapted to receive a video stream of a video conference, the video stream and corresponding audio stream may be stored in received data store 616.
[0034] User input data store 618 comprises logic and computing memory for storing data corresponding to user inputs and their corresponding outputs. For example, in an example scenario wherein user inputs are received on a touch sensitive screen 202, data corresponding to the inputs, as well as the corresponding outputs that were generated on the display apparatus 200 are stored in user input data store 618. Also, any data that may be used for synchronizing the user input data and corresponding outputs with any other outputs such as, for example, video outputs, may be stored in user input data store 618. For example, information regarding timing of the receipt of inputs and generation of corresponding output relative to the display of a video may be recorded. The timing information may be any type of information that is suitable for synchronizing the display of two outputs consistent with the timing intervals of the original display. The timing information may comprise absolute timing information or may be timing information that is relative to the playing time of the first output. For example, the timing information may specify that a particular second output was displayed at three minutes into the play time of a corresponding first output.
[0035] Synchronization control logic 614 is adapted to control the synchronized display of stored data. For example, in response to a request to display stored data 616, synchronization control logic is adapted to retrieve data from stores 616 and 618 and use the synchronization information stored in store 618 in order to have output control logic 614 display the stored data consistent with the timing of the original display of the two output streams. For example, in an example scenario wherein output corresponding to user inputs received on touch sensitive screen 202 was displayed superimposed on a video stream presented on display apparatus 200, synchronization control logic 614 may be adapted to present the corresponding stored data from data stores 616 and 618 in a manner that is consistent with the timing and sequence that applied when the two outputs were originally displayed.
Superimposed Outputs Processing [0036] FIG. 7 is a flow diagram of an example process for presenting second output 410 superimposed on first output 310. The example process may be implemented on any suitable device including, for example, any of devices l lOa-f. An example process may include one or more operations, actions, or functions as illustrated by one or more of blocks 710, 712, and/or 714. Moreover, for this and other processes and/or methods described herein, although illustrated as discrete blocks, various blocks may be divided into additional blocks, combined into fewer blocks, or eliminated, depending on the desired implementation. Processing may begin at block 710.
[0037] At block 710 (PROVIDE FIRST OUTPUT), first output 310 is displayed on display apparatus 200. First output 310 may be any output suitable to be displayed on display apparatus 200. For example, first output 310 may be video, images, still images, text, graphics, or any other output type. In one example scenario, first output 310 may comprise the video stream of a telephone conference. Simultaneously with displaying video, audio corresponding to the video may be output by a speaker component of device 110. Processing may continue from block 710 to block 712
[0038] At block 712 (RECEIVE INPUT), user input is received at device 110. The inputs may be any that are suitable to be received on device 110 and may comprise, for example, touch inputs, mechanical inputs, and/or audio inputs. Further, the inputs may be received via any suitable input device, including, for example, a touch sensitive screen, a physical keyboard, a virtual keyboard, a mouse, a microphone with voice recognition software, etc. In an example scenario, inputs are received on touch sensitive screen 202 while first output 310 is displayed on display apparatus 200. More particularly, handwritten inputs may be received on touch sensitive screen 202 while a video stream, which may be a video telephone conference stream, is displayed on display apparatus 200. Processing may continue from block 712 to block 714.
[0039] At block 714 (PROVIDE SECOND OUTPUT SUPERIMPOSED ON FIRST OUTPUT), second output 410, which corresponds to the user inputs received at block 712, is displayed on display apparatus 200. Second output 410 is displayed simultaneously with, and superimposed on first output 310. In an area of display apparatus 220 that first output 310 and second output 410 overlap, second output 410 is opaque and first output 310 is visible through opaque second output 410. Second output 410 may be, at least in part, transparent so first output 310 may be visible on display apparatus 200. In an area of display apparatus 200 that first output 310 and second output 410 do not overlap, first output 310 is visible.
[0040] In the area of display apparatus 200 that second output 410 overlaps first output 310, the degree or level of opaqueness, and conversely, the level of transparency may vary depending upon the preferences of the device operator. Thus, the level of the opaqueness may vary from very opaque to very transparent. Analogously, the degree to which first output 310 is partially obscured or darkened by second output 410 in the area that the two overlap, may vary from greatly to only slightly. Indeed, in some embodiments, first output 310 may be almost completely obscured, while in other embodiments, first output 310 is only slightly obscured.
[0041] In an example scenario, wherein at block 712, notes or annotations are received as inputs into device 110 while first output 310 is displayed, at block 714, markings, annotations, and/or notes corresponding to the inputs are displayed on display apparatus 200 at locations corresponding to the inputs. More particularly, wherein handwritten annotations or notes are received by a user on a touch sensitive screen 202, corresponding annotations are displayed on display apparatus 200 superimposed on first output 310 which may be, for example, the video output of an audio/video conference. In an alternate embodiment, annotations are made on a physical or virtual keyboard and corresponding notes or annotations are displayed on display apparatus 200 superimposed on first output 310. In yet another embodiment, user inputs are entered through computerized voice recognition and corresponding outputs are displayed on display apparatus 200 superimposed on first output 310.
[0042] In an example scenario, device 110 operates to maintain the integrity of first output 310 and second output 410 while the two outputs are displayed on display apparatus 200. In other words, the distinctness of second output 410 is maintained even though changes may be made to first output 310 while second output 410 is superimposed on first output 310. Likewise, the distinctness and integrity of first output 310 is maintained even though changes may be made to second output 410 while second output 410 is superimposed on first output 310. In an example scenario, first output 310 comprises video output. By its nature, video output changes over time so as to depict motion. The distinctness and sharpness of second output 410 is not impacted by the successive change in a video stream comprised in first output 310. Likewise, in a scenario wherein second output 410 changes over time such as, for example, when successive handwritten notes are received, the distinctness of first output 310 is maintained.
[0043] In connection with providing outputs at block 714, device 110 may maintain the integrity and distinctness of outputs 310 and 410 in any suitable manner. In an example embodiment, display apparatus 220, consistent with the typical operation of display devices, continuously refreshes the data that is presented. In an example scenario, device 110 may buffer data corresponding to outputs 310 and 410 prior to display. Device 110, and in particular output control 614 logic, may map the buffered data for second output 410 into or over the buffered data for first output 310. For example, output control logic 614 may map data for second output 410 and first output 310 to pixels locations on display apparatus 220. The buffering and mapping allows for controlling the integrity of separate outputs 310 and 410. Data comprising second output 410 data mapped onto first output 310 data is then displayed by display apparatus 220. The logic for buffering and mapping the data prior to display may be implemented, for example, by a graphics and or display controller that controls the pixel display on display apparatus 220.
[0044] According to another aspect of the potential embodiments, first and second outputs 310 and 410 may be stored and later retrieved and displayed. The outputs may be retrieved and displayed separately and independently. However, the outputs may also be retrieved and displayed simultaneously, with the outputs being synchronized and displayed in the same relative timing as was used when the outputs were originally displayed. FIG. 8 is a flow diagram of an example process for storing and retrieving first output 310 and second output 410. An example process may include one or more operations, actions, or functions as illustrated by one or more of blocks 810, 812, 814, and/or 816. Processing may begin at block 810.
[0045] At block 810 (STORE FIRST OUTPUT), first output 310 is stored in, for example, database store 616. For example, in an example scenario wherein first output 310 is the output of a video telephone conference, the video stream and, in some scenarios, the audio stream are stored in database store 616. Processing may continue from block 810 to block 812.
[0046] At block 812 (STORE SECOND OUTPUT), second output 410 and the corresponding user inputs are stored or recorded, for example, in database store 618. For example, in an example scenario, wherein user inputs were received and displayed as notes or annotations on display apparatus 200, at block 812, those annotations are stored in database store 618. Processing may continue from block 812 to block 814.
[0047] At block 814 (STORE SYNCHRONIZING INFORMATION), information synchronizing second output 410 with first output 310 is stored. For example, synchronization information may be stored in user input data store 618. The synchronization information that is stored may be any data useful in recording and recreating the temporal arrangement of second output 410 relative to first output 310. In other words, data indicating when second output 410 was displayed relative to first output 310 may be recorded. In an example scenario, wherein first output 310 is a video stream of an audio/video conference, and second output 410 comprises annotations or notes corresponding to user inputs, synchronization information captures the times at which various second outputs 410 were displayed relative to the playtime of first output 310. By way of example, wherein second output 410 was displayed thirty seconds into displaying first output 310 which comprises a video stream, the synchronization information comprises information recording that second output 410 was displayed 30 seconds into the video stream. In an embodiment wherein audio portions of first output 310 are recorded, information
synchronizing audio with second output 410 may also be recorded. Processing may continue from block 814 to 816.
[0048] At block 816 (RETRIEVE AND OUTPUT STORED FIRST AND/OR
SECOND OUTPUTS), the stored data for first output 310 and/or stored data for second output 410 may be retrieved and displayed. For example, where stored first output 310 comprises a video stream and an audio stream, and stored second output 410 comprises annotations that were made and superimposed on the video stream, at block 816, the stored information may be retrieved from memory and displayed. As mentioned, second output 410 and first output 310 are synchronized so that second output 410 is displayed at an appropriate time relative to first output 310 consistent with the relative timing of display when first output 310 and second output 410 were originally displayed. In an example embodiment, stored first output 310 may comprise video and audio, and stored second output 410 may comprise annotations. All may be retrieved from memory and output consistent with the stored synchronization information. In another scenario, only one or a portion of the stored outputs may be replayed. For example, second output 410, which may comprise notes or annotations, may be displayed separately from first output 310. In another example scenario, second output 410 may be retrieved and displayed while an audio portion of first output 310 is replayed. In such a scenario, the annotations or notes are displayed synchronized with the audio replay so as to be consistent with the temporal placement of the notes or annotations relative to the audio stream.
[0049] According to another aspect of the disclosed embodiments, second output 410 may be moved and/or removed from display apparatus 200. Such functionality may be particularly useful wherein display apparatus 200 is relatively small and second output 410 covers too much of, or a particular portion of, display apparatus 200. Second output 410 may be moved on display apparatus 200 so as to be less obtrusive. Alternatively, or in addition, second output 410 may be removed from display apparatus 200.
[0050] FIG. 9 is a flow diagram of an example processes for moving a second output on a display apparatus and/or removing a second output from a display apparatus. An example process may include one or more operations, actions, or functions as illustrated by one or more of blocks 910 and/or 912. Processing may begin at block 910.
[0051] At block 910 (MOVE ALL OR A PORTION OF THE SECOND OUTPUT), all or a portion of second output 410 may be moved on display apparatus 200. For example, second output 410 may be moved so as to have a smaller amount of overlap with first output 310. Thus, in an example embodiment, second output 410 may be moved on display apparatus 200 in response to a user input. For example, second output 410 may be moved in response to a mouse cursor dragging on second output 410. In an example embodiment, second output 410 may be moved automatically by device 110 after a prescribed period of time. For example, second output 410 may be moved automatically by device 110 to a particular portion of display apparatus 200 such as, for example, the bottom portion, after second output 410 has been displayed for 5 minutes. Processing may continue from block 910 to 912.
[0052] At block 912 (REMOVE ALL OR A PORTION OF THE SECOND OUTPUT), all or a portion of second output 410 may be removed from display apparatus 200. For example, second output 410 may be removed in response to user input or in response to rules or requirements enforced or applied by device 110. In an example scenario, second output 410 may be removed from display apparatus 200 in response to user input. For example, a user may select to clear all or a portion of the notes or annotations that are displayed on display apparatus 200 so as to make space for additional notes or annotations. In an example embodiment, device 110 may comprise rules that specify outputs corresponding to user inputs should be removed after a prescribed period of time has expired. For example, after an output has been displayed for 10 minutes, device 110 may remove display apparatus 200.
Example Computing Environment
[0053] Figure 10 depicts a block diagram of example computing system 1000 that may be used to implement the systems and methods described herein. For example, computing system 1000 may be used to implement any of devices l lOa-f. Computing system 1000 may be controlled primarily by computer readable instructions that may be in the form of software. For example, computing system 1000 may be controlled by a computing application 1080 which comprises instructions for performing functionality consistent with that described herein. The computer readable instructions may include instructions for computing system 1000 for storing and accessing computer readable instructions themselves. Such software may be executed within central processing unit (CPU) 1010 to cause computing system 1000 to perform the processes or functions associated therewith. In many known computer servers, workstations, personal computers, or the like, CPU 1010 may be implemented by micro-electronic chip CPUs called microprocessors.
[0054] In operation, CPU 1010 may fetch, decode, and/or execute instructions and may transfer information to and from other resources via a main data-transfer path or system bus 1005. Such a system bus may connect the components in computing system 1000 and may define the medium for data exchange. Computing system 1000 may further include memory devices coupled to system bus 1005. According to an example embodiment, the memory devices may include random access memory (RAM) 1025 and read only memory (ROM) 1030. RAM 1025 and ROM 1030 may include circuitry that allows information to be stored and retrieved. In one embodiment, ROM 1030 may include stored data that cannot be modified. Additionally, data stored in RAM 1025 typically may be read or changed by CPU 1010 or other hardware devices. Access to RAM 1025 and/or ROM 1030 may be controlled by memory controller 1020.
Memory controller 1020 may provide an address translation function that translates virtual addresses into physical addresses as instructions are executed.
[0055] In addition, computing system 1000 may include peripherals controller 1035 that may be responsible for communicating instructions from CPU 1010 to peripherals, such as, touch sensitive input 1040, keyboard 1045, mouse 1050, and audio input and output device 1055. Computing system 1000 may further include display 1065 that may be controlled by display controller 1063. Display 1065 may be used to display visual output generated by computing system 1000 and may correspond to display apparatus 200 discussed herein. Such visual output may include text, graphics, animated graphics, video, or the like. Display controller 1063 may include electronic components that generate a video signal that may be sent to display 1065. Further, computing system 1000 may include network adaptor 1070 that may be used to connect computing system 2000 to an external communication network such as network 150, described above in connection with Figure 1.
[0056] Thus, applicants have disclosed example embodiments of systems and methods for performing user interface processing. Applicants disclose systems and methods for displaying a first output and simultaneously displaying a second output corresponding to user input superimposed on the first output. In an area of the display that the first output and the second output overlap, the second output is opaque and the first output is visible through the opaque second output. In an area of the display that the first output and the second output do not overlap, the first output is visible.
[0057] It should be understood that the various techniques described herein may be implemented in connection with hardware or software or, where appropriate, with a combination of both. Thus, the methods and apparatus of the subject matter described herein, or certain aspects or portions thereof, may take the form of program code (i.e., instructions) embodied in tangible media, such as floppy diskettes, CD-ROMs, hard drives, or any other machine-readable storage medium wherein, when the program code is loaded into and executed by a machine, such as a computer, the machine becomes an apparatus for practicing the subject matter described herein. In the case where program code is stored on media, it may be the case that the program code in question is stored on one or more media that collectively perform the actions in question, which is to say that the one or more media taken together contain code to perform the actions, but that - in the case where there is more than one single medium - there is no requirement that any particular part of the code be stored on any particular medium. In the case of program code execution on programmable computers, the computing device generally includes a processor, a storage medium readable by the processor (including volatile and non- volatile memory and/or storage elements), at least one input device, and at least one output device. One or more programs that may implement or utilize the processes described in connection with the subject matter described herein, e.g., through the use of an API, reusable controls, or the like. Such programs are preferably implemented in a high level procedural or object oriented programming language to communicate with a computer system. However, the program(s) can be
implemented in assembly or machine language, if desired. In any case, the language may be a compiled or interpreted language, and combined with hardware implementations.
[0058] Although example embodiments may refer to utilizing aspects of the subject matter described herein in the context of one or more stand-alone computer systems or devices, the subject matter described herein is not so limited, but rather may be implemented in connection with any computing environment, such as a network or distributed computing environment. Still further, aspects of the subject matter described herein may be implemented in or across a plurality of processing chips or devices, and storage may similarly be spread across a plurality of devices. Such devices might include personal computers, network servers, handheld devices, supercomputers, or computers integrated into other systems such as automobiles and airplanes.
[0059] Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. For example, several example scenarios have been described involving display of a video portion of an audio/video telephone conference and superimposing annotations received from a touch sensitive screen or display. However, the methods and systems described herein may apply to any combination of output types. For example, a first output may be a video stream that is not portion of a telephone conference. The first output may be a still image. The second output may be entered through a keyboard as opposed to a touch sensitive screen. The specific features and acts described above are disclosed as example forms of implementing the subject matter of the below-listed claims.
[0060] While various aspects and embodiments have been disclosed herein, other aspects and embodiments will be apparent to those skilled in the art. The various aspects and embodiments disclosed herein are for purposes of illustration and are not intended to be limiting, with the true scope and spirit being indicated by the following claims.

Claims

What is Claimed:
1. A method for user interface processing, comprising: displaying a first output on a display; and displaying on the display a second output superimposed on the first output, the second output corresponding to user input, wherein in an area of the display that the first output and the second output overlap, the second output is opaque and the first output is visible through the opaque second output, and wherein in an area of the display that the first output and the second output do not overlap, the first output is visible.
2. The method of claim 1, wherein displaying a first output on a display comprises displaying video.
3. The method of claim 1, wherein displaying a first output on a display comprises displaying one or more of the following: video; images; still images; text, and graphics.
4. The method of claims 1, 2, or 3, wherein displaying on the display a second output superimposed on the first output comprises displaying one or more of the following: video;
images; still images; text; and graphics.
5. The method of claims 1, 2, 3, or 4, wherein displaying on the display a second output superimposed on the first output comprises displaying annotations corresponding to user input.
6. The method of claim 5, wherein displaying annotations corresponding to user input comprises displaying annotations corresponding to user input received on the display.
7. The method of claim 5, wherein displaying annotations corresponding to user input comprises displaying annotations corresponding to handwritten input received on a touch sensitive screen.
8. The method of claims 5, 6, or 7, wherein displaying annotations corresponding to user input comprises displaying annotations corresponding to user input received on a keyboard.
9. The method claim 8, wherein displaying annotations corresponding to user input received on a keyboard comprises displaying annotations corresponding to user input received on at least one of a physical keyboard and a computer- generated keyboard.
10. The method of claims 5, 6, 7, or 8, wherein displaying annotations corresponding to user input comprises displaying annotations corresponding to user input received via computerized voice recognition.
11. The method of claims 6, 7, 8, 9, or 10, further comprising receiving user input simultaneously with displaying the first output, wherein displaying on the display a second output comprises displaying output corresponding to the received user input.
12. The method of claims 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, or 11, wherein in the area of the display that the first output and the second output overlap, the second output is opaque and the first output is obscured.
13. The method of claims 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, or 11, wherein in the area of the display that the first output and the second output overlap, the second output is partially opaque and the first output is partially obscured.
14. The method of claims 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, or 11, wherein displaying on the display a second output comprises displaying on the display opaque markings, and wherein in an area of the display that the first output and the second output overlap, the opaque markings are visible and the first output is obscured.
15. The method of claim 14, wherein in the area of the display that the first output and the second output overlap, the first output is completely obscured.
16. The method of claim 14, wherein displaying on the display opaque markings comprises displaying partially opaque markings, and wherein in an area of the display that the first output and the second output overlap, the first output is partially obscured.
17. The method of claim 1, further comprising receiving user input simultaneously with displaying the first output.
18. The method of claim 17, wherein displaying on the display a second output comprises displaying on the display annotations corresponding to the received user input.
19. The method of claim 17 or 18, wherein displaying on the display a second output comprises displaying on the display notes corresponding to the received user input.
20. The method of claim 18, wherein displaying a first output on the display comprises displaying video.
21. The method of claim 20, wherein displaying video comprises displaying a video stream of an audio/video conference.
22. The method of claims 18, 20 or 21, wherein receiving user input simultaneously with displaying the first output comprises receiving user input on the display.
23. The method of claims 18, 20, 21, or 22, wherein receiving user input simultaneously with displaying the first output comprises receiving user input on a keyboard.
24. The method of claims 18, 20, or 21, wherein displaying on the display annotations corresponding to the received user input comprises displaying on the display markings at locations on the display at which user input is received.
25. The method of claims 20 or 21, wherein receiving user input simultaneously with displaying the first output comprises receiving handwritten input on the display.
26. The method of claims 18, 20, 21, 22, 23, 24, or 25, further comprising storing the annotations.
27. The method of claim 26, further comprising displaying the stored annotations.
28. The method of claims 20, 21, or 25, further comprising: storing the annotations; storing the video; storing information synchronizing the annotations with the video; retrieving the video, the annotations, and the information synchronizing the annotations with the video; and displaying the retrieved video and the retrieved annotations, the annotations
superimposed on and synchronized with the video.
29. The method of claims 20, 21, or 25, further comprising: generating audio simultaneously with displaying video; storing the video, the audio, and the annotations; storing information synchronizing the annotations with the audio and the video; retrieving the video, the audio, the annotations, and the information synchronizing the annotations with the audio and video; playing the retrieved audio; displaying the retrieved video on the display, and displaying the retrieved annotations on the display, the retrieved annotations
superimposed on the retrieved video and synchronized with the retrieved video and the retrieved audio.
30. The method of claim 1, wherein displaying a first output on a display comprises displaying video on the display, and further comprising receiving user input on the display simultaneously with displaying the video.
31. The method of claim 30, wherein displaying on the display a second output superimposed on the first output comprises displaying on the display annotations corresponding to the received user input, the annotations superimposed on the video, and further comprising recording the video and the annotations, the recorded annotations synchronized with the recorded video.
32. The method of claim 31, further comprising displaying the recorded annotations on the display.
33. The method of claim 31, further comprising displaying the recorded video and the recorded annotations, the recorded annotations superimposed on and synchronized with the recorded video.
34. The method of claim 31, further comprising: generating audio, the audio synchronized with the video; and recording the audio.
35. The method of claim 34, further comprising displaying the recorded annotations on the display and playing the recorded audio, the playing of the recorded audio synchronized with the displayed recorded annotations.
36. The method of claim 34, further comprising: displaying the recorded video; playing the recorded audio synchronized with the displayed recorded video; and displaying the recorded annotations superimposed on and synchronized with the displayed recorded video.
37. The method of claims 30, 31, 32, 33, 34, 35, or 36, wherein the video is a video stream of an audio/video conference.
38. The method of claims 18, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 31, 32, 33, 34, 35, 36, or 37, further comprising removing at least a portion of the annotations from the display after the annotations have been displayed for a prescribed period of time.
39. The method of claims 18, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 31, 32, 33, 34, 35, 36, or 37, further comprising removing at least a portion of the annotations in response to receiving a user input.
40. The method of claims 18, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, or 37, further comprising moving at least a portion of the annotations on the display in response to user input.
41. The method of claims 38 or 39, further comprising maintaining at least a portion of the annotations on the display.
42. The method of claim 41, wherein the at least a portion of the annotations maintained on the display is identified by user input.
43. The method of claims 18, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 31, 32, 33, 34, 35, 36, or 37, wherein the annotations are displayed using a color that is discemable when superimposed on the first output.
44. The method of claim 43, wherein the color is selected depending upon the color of the first output.
45. The method of any one of claims 1 through 44, wherein displaying a first output on a display comprises displaying output on a mobile device.
46. The method of any one of claims 1 through 44, wherein displaying a first output on a display comprises displaying output on one or more of the following: a telephone; a mobile telephone; a personal digital assistant; a laptop computer; a tablet computer; and a desktop computer.
47. An apparatus, comprising: a display; an input device; a computing processor, the computing processor communicatively coupled with the display and the input device; and memory communicatively coupled with the processor, the memory comprising instructions that, if executed by the apparatus, cause the apparatus to perform operations comprising: displaying a first output on a display; and displaying on the display a second output superimposed on the first output, the second output configured to opaquely display user input, wherein the second output is visible through the superimposed second output.
48. The apparatus of claim 47, wherein displaying a first output on a display comprises displaying video.
49. The apparatus of claim 47, wherein displaying a first output on a display comprises displaying one or more of the following: video; images; still images; text, and graphics.
50. The apparatus of claims 47, 48, or 49, wherein displaying on the display a second output superimposed on the first output comprises displaying one or more of the following: video; images; still images; text; and graphics.
51. The apparatus of claims 47, 48, 49, or 50, wherein displaying on the display a second output superimposed on the first output comprises displaying annotations corresponding to user input.
52. The apparatus of claim 51, wherein displaying annotations corresponding to user input comprises displaying annotations corresponding to user input received on the display.
53. The apparatus of claim 51, wherein displaying annotations corresponding to user input comprises displaying annotations corresponding to handwritten input received on a touch sensitive screen.
54. The apparatus of claims 51, 52, or 53, wherein displaying annotations corresponding to user input comprises displaying annotations corresponding to user input received on a keyboard.
55. The apparatus of claim 54, wherein displaying annotations corresponding to user input received on a keyboard comprises displaying annotations corresponding to user input received on at least one of a physical keyboard and a computer-generated keyboard.
56. The apparatus of claims 51, 52, 53, or 54, wherein displaying annotations corresponding to user input comprises displaying annotations corresponding to user input received via computerized voice recognition.
57. The apparatus of claims 52, 53, 54, 55, or 56, wherein the memory further comprises instructions that cause the apparatus to perform further operations comprising receiving user input simultaneously with displaying the first output, wherein displaying on the display a second output comprises displaying output corresponding to the received user input
58. The apparatus of claims 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, or 57, wherein in an area of the display that the first output and the second output overlap, the second output is opaque and the first output is obscured.
59. The apparatus of claims 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, or 57, wherein in an area of the display that the first output and the second output overlap, the second output is partially opaque and the first output is partially obscured.
60. The apparatus of claims 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, or 57, wherein displaying on the display a second output comprises displaying on the display opaque markings, and wherein in an area of the display that the first output and the second output overlap, the opaque markings are visible and the first output is obscured.
61. The apparatus of claim 47, wherein the second output corresponds to an input received on the display.
62. The apparatus of claim 60, wherein displaying on the display opaque markings comprises displaying partially opaque markings, and wherein in an area of the display that the first output and the second output overlap, the first output is partially obscured.
63. The apparatus of claim 47, wherein the memory further comprises instructions that cause the apparatus to perform further operations comprising receiving user input simultaneously with displaying the first output.
64. The apparatus of claim 63, wherein displaying on the display a second output comprises displaying on the display annotations corresponding to the received user input.
65. The apparatus of claims 63 or 64, wherein displaying on the display a second output comprises displaying on the display notes corresponding to the received user input.
66. The apparatus of claim 64, wherein displaying a first output on the display comprises displaying video.
67. The apparatus of claim 66, wherein displaying video comprises displaying a video stream of an audio/video conference.
68. The apparatus of claims 64, 66, or 67, wherein receiving user input simultaneously with displaying the first output comprises receiving user input on the display.
69. The apparatus of claims 64, 66, 67, or 68, wherein receiving user input simultaneously with displaying the first output comprises receiving user input on a keyboard.
70. The apparatus of claims 64, 66, or 67, wherein displaying on the display annotations corresponding to the received user input comprises displaying on the display markings at locations on the display at which user input is received.
71. The apparatus of claims 66 or 67, wherein receiving user input simultaneously with displaying the first output comprises receiving handwritten input on the display.
72. The apparatus of claim 64, 66, 67, 68, 69, 70, or 71, wherein the memory further comprises instructions that cause the apparatus to perform further operations comprising storing the annotations.
73. The apparatus of claim 72, wherein the memory further comprises instructions that cause the apparatus to perform further operations comprising displaying the stored annotations.
74. The apparatus of claims 66, 67, or 71, wherein the memory further comprises instructions that cause the apparatus to perform further operations comprising: storing the annotations; storing the video; storing information synchronizing the annotations with the video; retrieving the video, the annotations, and the information synchronizing the annotations with the video; and displaying the retrieved video and the retrieved annotations, the retrieved annotations superimposed on and synchronized with the retrieved video.
75. The apparatus of claims 66, 67, or 71, wherein the memory further comprises instructions that cause the apparatus to perform further operations comprising: generating audio simultaneously with displaying video; storing the video, the audio, and the annotations; storing information synchronizing the annotations with the audio and video; retrieving the video, the audio, the annotations, and the information synchronizing the annotations with the audio and video; playing the retrieved audio; displaying the retrieved video on the display, and displaying the retrieved annotations on the display, the retrieved annotations
superimposed on the retrieved video and synchronized with the retrieved video and the retrieved audio.
76. The apparatus of claim 47, wherein displaying a first output on a display comprises displaying video on the display, and wherein the memory further comprises instructions that cause the apparatus to perform further operations comprising receiving user input on the display simultaneously with displaying the video.
77. The apparatus of claim 76, wherein displaying on the display a second output
superimposed on the first output comprises displaying on the display annotations corresponding to the received user input, the annotations superimposed on the video, and wherein the memory further comprises instructions that cause the apparatus to perform further operations comprising recording the video and the annotations, the recorded annotations synchronized with the recorded video.
78. The apparatus of claim 77, wherein the memory further comprises instructions that cause the apparatus to perform further operations comprising displaying the recorded annotations on the display.
79. The apparatus of claim 77, wherein the memory further comprises instructions that cause the apparatus to perform further operations comprising displaying the recorded video and the recorded annotations, the recorded annotations superimposed on and synchronized with the recorded video.
80. The apparatus of claim 77, wherein the memory further comprises instructions that cause the apparatus to perform further operations comprising: generating audio, the audio synchronized with the video; and recording the audio.
81. The apparatus of claim 80, wherein the memory further comprises instructions that cause the apparatus to perform further operations comprising displaying the recorded annotations on the display and playing the recorded audio, the playing of the recorded audio synchronized with the displayed recorded annotations.
82. The apparatus of claim 80, wherein the memory further comprises instructions that cause the apparatus to perform further operations comprising: displaying the recorded video; playing the recorded audio synchronized with the displayed recorded video; and displaying the recorded annotations superimposed on and synchronized with the displayed recorded video.
83. The apparatus of claims 76, 77, 78, 79, 80, 81, or 82, wherein the video is a video stream of an audio/video conference.
84. The apparatus of claims 64, 66, 67, 68, 69, 70 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, or 83, wherein the memory further comprises instructions that cause the apparatus to perform further operations comprising removing at least a portion of the annotations from the display after the annotations have been displayed for a prescribed period of time.
85. The apparatus of claims 64, 66, 67, 68, 69, 70 71, 72, 73, 74, 75, 77, 78, 79, 80, 81, 82, or 83, wherein the memory further comprises instructions that cause the apparatus to perform further operations comprising removing annotations in response to receiving a user input.
86. The apparatus of claims 64, 66, 67, 71, 72, 73, 74, 75, 77, 78, 79, 80, 81, 82, or 83, wherein the memory further comprises instructions that cause the apparatus to perform further operations comprising moving at least a portion of the annotations on the display in response to user input.
87. The apparatus of claims 84 or 85, wherein the memory further comprises instructions that cause the apparatus to perform further operations comprising maintaining at least a portion of the annotations on the display.
88. The apparatus of claim 87, wherein the at least a portion of the annotations maintained on the display is identified by user input.
89. The apparatus of claims 64, 66, 67, 71, 72, 73, 74, 75, 77, 78, 79, 80, 81, 82, or 83, wherein the annotations are displayed using a color that is discernable when superimposed on the first output.
90. The apparatus of claim 89, wherein the color is selected depending upon the color of the first output.
91. The apparatus of any one of claims 47 through 90, wherein displaying a first output on a display comprises displaying output on a mobile device.
92. The apparatus of any one of claims 47 through 90, wherein displaying a first output on a display comprises displaying output on one or more of the following: a telephone; a mobile telephone; a personal digital assistant; a laptop computer; a tablet computer; and a desktop computer.
93. The apparatus of any one of claims 47 through 90, wherein the apparatus is a mobile device.
94. The apparatus of claim 93, wherein the apparatus is a telephone.
95. The apparatus of claim 93, wherein the apparatus is a personal digital assistant.
96. The apparatus of claim 93, wherein the apparatus is a laptop computer.
97. The apparatus of claim 93, wherein the apparatus is a tablet computer.
98. A tangible computer readable medium having instructions stored thereon that, if executed by a computing device, cause the computing device to perform operations, comprising: displaying a first output on a display; and displaying on the display a second output superimposed on the first output, the second output corresponding to user input, wherein in an area of the display that the first output and the second output overlap, the second output is visible and the first output is partially obscured but visible through the displayed second output, and wherein in an area of the display that the first output and the second output do not overlap, the first output is visible.
99. The tangible computer readable medium of claim 98, wherein displaying a first output on a display comprises displaying video.
100. The tangible computer readable medium of claim 98, wherein displaying a first output on a display comprises displaying one or more of the following: video; images; still images; text, and graphics.
101. The tangible computer readable medium of claims 98, 99, or 100, wherein displaying on the display a second output superimposed on the first output comprises displaying one or more of the following: video; images; still images; text; and graphics.
102. The tangible computer readable medium of claims 98, 99, 100, or 101, wherein displaying on the display a second output superimposed on the first output comprises displaying annotations corresponding to user input.
103. The tangible computer readable medium of claim 102, wherein displaying annotations corresponding to user input comprises displaying annotations corresponding to user input received on the display.
104. The tangible computer readable medium of claims 102, wherein displaying annotations corresponding to user input comprises displaying annotations corresponding to handwritten input received on a touch sensitive screen.
105. The tangible computer readable medium of claims 102, 103, or 104, wherein displaying annotations corresponding to user input comprises displaying annotations corresponding to user input received on a keyboard.
106. The tangible computer readable medium of claim 105, wherein displaying annotations corresponding to user input received on a keyboard comprises displaying annotations
corresponding to user input received on at least one of a physical keyboard and a computer- generated keyboard.
107. The tangible computer readable medium of claims 102, 103, 104, or 105, wherein displaying annotations corresponding to user input comprises displaying annotations
corresponding to user input received via computerized voice recognition.
108. The tangible computer readable medium of claims 103, 104, 105, 106, or 107, further comprising instructions that cause the computing device to perform further operations comprising receiving user input simultaneously with displaying the first output, wherein displaying on the display a second output comprises displaying output corresponding to the received user input
109. The tangible computer readable medium of claims 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, or 108, wherein in the area of the display that the first output and the second output overlap, the second output is opaque.
110. The tangible computer readable medium of claims 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, or 108, wherein in the area of the display that the first output and the second output overlap, the second output is partially opaque.
111. The tangible computer readable medium of claims 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, or 108, wherein displaying on the display a second output comprises displaying on the display opaque markings, and wherein in an area of the display that the first output and the second output overlap, the opaque markings are visible and the first output is partially obscured.
112. The tangible computer readable medium of claim 111, wherein in the area of the display that the first output and the second output overlap, the first output is darkened relative to the remainder of the first output.
113. The tangible computer readable medium of claim 111, wherein displaying on the display opaque markings comprises displaying partially opaque markings, and wherein in an area of the display that the first output and the second output overlap, the first output is partially obscured.
114. The tangible computer readable medium of claim 98, further comprising instructions that cause the computing device to perform further operations comprising receiving user input simultaneously with displaying the first output.
115. The tangible computer readable medium of claim 114, wherein displaying on the display a second output comprises displaying on the display annotations corresponding to the received user input.
116. The tangible computer readable medium of claims 114 or 115, wherein displaying on the display a second output comprises displaying on the display notes corresponding to the received user input.
117. The tangible computer readable medium of claim 115, wherein displaying a first output on the display comprises displaying video.
118. The tangible computer readable medium of claim 117, wherein displaying video comprises displaying a video stream of an audio/video conference.
119. The tangible computer readable medium of claims 115, 117, or 118, wherein receiving user input simultaneously with displaying the first output comprises receiving user input on the display.
120. The tangible computer readable medium of claims 115, 117, 118, or 119, wherein receiving user input simultaneously with displaying the first output comprises receiving user input on a keyboard.
121. The tangible computer readable medium of claims 115, 117, or 118, wherein displaying on the display annotations corresponding to the received user input comprises displaying on the display markings at locations on the display at which user input is received.
122. The tangible computer readable medium of claims 117 or 118, wherein receiving user input simultaneously with displaying the first output comprises receiving handwritten input on the display.
123. The tangible computer readable medium of claims 115, 117, 118, 119, 120, 121, or 122, further comprising instructions that cause the computing device to perform further operations comprising storing the annotations.
124. The tangible computer readable medium of claim 123, further comprising instructions that cause the computing device to perform further operations comprising displaying the stored annotations.
125. The tangible computer readable medium of claims 117, 118, or 122, further comprising instructions that cause the computing device to perform further operations comprising: storing the annotations; storing the video; storing information synchronizing the annotations with the video; retrieving the video, the annotations, and the information synchronizing the annotations with the video; and displaying the retrieved video and the retrieved annotations, the retrieved annotations superimposed on and synchronized with the retrieved video.
126. The tangible computer readable medium of claims 117, 118, or 122, further comprising instructions that cause the computing device to perform further operations comprising: generating audio simultaneously with displaying video; storing the video, the audio, and the annotations; storing information synchronizing the annotations with the audio and the video; retrieving the video, the audio, the annotations, and the information synchronizing the annotations with the audio and video; playing the retrieved audio; displaying the retrieved video on the display, and displaying the retrieved annotations on the display, the retrieved annotations superimposed on the retrieved video and synchronized with the retrieved video and the retrieved audio.
127. The tangible computer readable medium of claim 98, wherein displaying a first output on a display comprises displaying video on the display, and wherein the tangible computer-readable medium further comprises instructions that cause the computing device to perform further operations comprising receiving user input on the display simultaneously with displaying the video.
128. The tangible computer readable medium of claim 127, wherein displaying on the display a second output superimposed on the first output comprises displaying on the display annotations corresponding to the received user input, the annotations superimposed on the video, and wherein the tangible computer readable medium further comprises instructions that cause the computing device to perform further operations comprising recording the video and the annotations, the recorded annotations synchronized with the recorded video.
129. The tangible computer readable medium of claim 128, further comprising instructions that cause the computing device to perform further operations comprising displaying the recorded annotations on the display.
130. The tangible computer readable medium of claim 128, further comprising instructions that cause the computing device to perform further operations comprising displaying the recorded video and the recorded annotations, the recorded annotations superimposed on and synchronized with the recorded video.
131. The tangible computer readable medium of claim 128, further comprising instructions that cause the computing device to perform further operations comprising: generating audio, the audio synchronized with the video; and recording the audio.
132. The tangible computer readable medium of claim 131, further comprising instructions that cause the computing device to perform further operations comprising displaying the recorded annotations on the display and playing the recorded audio, the playing of the recorded audio synchronized with the displayed recorded annotations.
133. The tangible computer readable medium of claim 131, further comprising instructions that cause the computing device to perform further operations comprising: displaying the recorded video; playing the recorded audio synchronized with the displayed recorded video; and displaying the recorded annotations superimposed on and synchronized with the displayed recorded video.
134. The tangible computer readable medium of claims 127, 128, 129, 130, 131, 132, or 133, wherein the video is a video stream of an audio/video conference.
135. The tangible computer readable medium of claims 115, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 128, 129, 130, 131, 132, 133, or 134, further comprising instructions that cause the computing device to perform further operations comprising removing at least a portion of the annotations from the display after the annotations have been displayed for a prescribed period of time.
136. The tangible computer readable medium of claims 115, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 128, 129, 130, 131, 132, 133, or 134, further comprising instructions that cause the computing device to perform further operations comprising removing annotations in response to receiving a user input.
137. The tangible computer readable medium of claims 115, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 128, 129, 130, 131, 132, 133, or 134, further comprising instructions that cause the computing device to perform further operations comprising moving at least a portion of the annotations on the display in response to user input.
138. The tangible computer readable medium of claims 135 or 136, further comprising instructions that cause the computing device to perform further operations comprising
maintaining at least a portion of the annotations on the display.
139. The tangible computer readable medium of claim 138, wherein the at least a portion of the annotations maintained on the display is identified by user input.
140. The tangible computer readable medium of claims 115, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 128, 129, 130, 131, 132, 133, or 134, wherein the annotations are displayed using a color that is discernable when superimposed on the first output.
141. The tangible computer readable medium of claim 140, wherein the color is selected depending upon the color of the first output.
142. The tangible computer readable medium of any one of claims 98 through 141, wherein displaying a first output on a display comprises displaying output on a mobile device.
143. The tangible computer readable medium of any one of claims 98 through 141, wherein displaying a first output on a display comprises displaying output on one or more of the following: a telephone; a mobile telephone; a personal digital assistant; a laptop computer; a tablet computer; and a desktop computer.
PCT/CN2011/071854 2011-03-16 2011-03-16 Superimposed annotation output WO2012122706A1 (en)

Priority Applications (6)

Application Number Priority Date Filing Date Title
KR1020137010900A KR101452667B1 (en) 2011-03-16 2011-03-16 Superimposed annotation output
PCT/CN2011/071854 WO2012122706A1 (en) 2011-03-16 2011-03-16 Superimposed annotation output
JP2013549694A JP5852135B2 (en) 2011-03-16 2011-03-16 Superimposed annotation output
CN201180068426.5A CN103384874B (en) 2011-03-16 2011-03-16 Output is explained in superposition
US13/502,011 US9715326B2 (en) 2011-03-16 2011-03-16 Superimposed annotation output
EP11860725.8A EP2686761A4 (en) 2011-03-16 2011-03-16 Superimposed annotation output

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2011/071854 WO2012122706A1 (en) 2011-03-16 2011-03-16 Superimposed annotation output

Publications (1)

Publication Number Publication Date
WO2012122706A1 true WO2012122706A1 (en) 2012-09-20

Family

ID=46830030

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2011/071854 WO2012122706A1 (en) 2011-03-16 2011-03-16 Superimposed annotation output

Country Status (6)

Country Link
US (1) US9715326B2 (en)
EP (1) EP2686761A4 (en)
JP (1) JP5852135B2 (en)
KR (1) KR101452667B1 (en)
CN (1) CN103384874B (en)
WO (1) WO2012122706A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2015001856A (en) * 2013-06-17 2015-01-05 コニカミノルタ株式会社 Image display device, display control program, and display control method
EP3246796A4 (en) * 2015-02-12 2018-02-28 Samsung Electronics Co., Ltd. Device and method for inputting note information into image of photographed object

Families Citing this family (29)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
ES2434259T3 (en) * 2010-09-29 2013-12-16 Teliasonera Ab Social television service
CN103297855B (en) * 2012-03-02 2015-05-20 腾讯科技(深圳)有限公司 Application display method and terminal
KR102042265B1 (en) * 2012-03-30 2019-11-08 엘지전자 주식회사 Mobile terminal
US10152467B2 (en) * 2012-08-13 2018-12-11 Google Llc Managing a sharing of media content among client computers
US20140232813A1 (en) * 2013-02-20 2014-08-21 Sprint Communications Company L.P. Using metadata for video message modifications among wireless communication devices
US9213472B2 (en) 2013-03-12 2015-12-15 Sap Se User interface for providing supplemental information
KR101401728B1 (en) * 2013-09-06 2014-05-30 주식회사 엘지유플러스 Communication apparatus for image call, control method thereof, and recording medium for recording program for executing the control method
US10291597B2 (en) 2014-08-14 2019-05-14 Cisco Technology, Inc. Sharing resources across multiple devices in online meetings
CN104361565B (en) * 2014-12-05 2018-10-23 华东理工大学 A kind of content scrawling picture digital picture detection and restorative procedure automatically
US10542126B2 (en) 2014-12-22 2020-01-21 Cisco Technology, Inc. Offline virtual participation in an online conference meeting
CN104601930A (en) * 2015-01-09 2015-05-06 西安融成科技有限公司 System and method for automatically switching and displaying video image and desktop image
US9948786B2 (en) 2015-04-17 2018-04-17 Cisco Technology, Inc. Handling conferences using highly-distributed agents
US9654727B2 (en) 2015-06-01 2017-05-16 Apple Inc. Techniques to overcome communication lag between terminals performing video mirroring and annotation operations
CN107305457A (en) * 2016-04-20 2017-10-31 斑马网络技术有限公司 Application processing method, equipment, interface system, control device and operating system
US10852936B2 (en) * 2016-09-23 2020-12-01 Apple Inc. Devices, methods, and graphical user interfaces for a unified annotation layer for annotating content displayed on a device
CN106569796A (en) * 2016-09-30 2017-04-19 努比亚技术有限公司 Display method and terminal
US10592867B2 (en) 2016-11-11 2020-03-17 Cisco Technology, Inc. In-meeting graphical user interface display using calendar information and system
US9961302B1 (en) * 2016-12-13 2018-05-01 Cisco Technology, Inc. Video conference annotation
US10516707B2 (en) 2016-12-15 2019-12-24 Cisco Technology, Inc. Initiating a conferencing meeting using a conference room device
US10440073B2 (en) 2017-04-11 2019-10-08 Cisco Technology, Inc. User interface for proximity based teleconference transfer
US10375125B2 (en) 2017-04-27 2019-08-06 Cisco Technology, Inc. Automatically joining devices to a video conference
CN108989900A (en) * 2017-06-02 2018-12-11 中兴通讯股份有限公司 A kind of method for processing video frequency and terminal
US10375474B2 (en) 2017-06-12 2019-08-06 Cisco Technology, Inc. Hybrid horn microphone
US10477148B2 (en) 2017-06-23 2019-11-12 Cisco Technology, Inc. Speaker anticipation
US10516709B2 (en) 2017-06-29 2019-12-24 Cisco Technology, Inc. Files automatically shared at conference initiation
US10706391B2 (en) 2017-07-13 2020-07-07 Cisco Technology, Inc. Protecting scheduled meeting in physical room
US10091348B1 (en) 2017-07-25 2018-10-02 Cisco Technology, Inc. Predictive model for voice/video over IP calls
JP7212255B2 (en) * 2019-02-04 2023-01-25 株式会社Mixi Information processing system, control program and information processing device
US11704626B2 (en) * 2021-07-09 2023-07-18 Prezi, Inc. Relocation of content item to motion picture sequences at multiple devices

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5600775A (en) 1994-08-26 1997-02-04 Emotion, Inc. Method and apparatus for annotating full motion video and other indexed data structures
EP0836114A1 (en) * 1996-10-11 1998-04-15 Eastman Kodak Company Full frame annotation system for camera
WO2001065832A1 (en) * 2000-02-29 2001-09-07 Watchpoint Media, Inc. A method and apparatus for receiving a hyperlinked television broadcast
US20080276272A1 (en) 2007-05-02 2008-11-06 Google Inc. Animated Video Overlays
CN101473294A (en) * 2006-06-20 2009-07-01 诺基亚公司 System, device, method, and computer program product for annotating media files

Family Cites Families (31)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2823018B2 (en) * 1989-01-20 1998-11-11 東京電力株式会社 Video transmission equipment
JPH037487A (en) * 1989-06-05 1991-01-14 Fuji Photo Film Co Ltd Television telephone set
JP2911135B2 (en) * 1989-07-04 1999-06-23 株式会社日立製作所 Video printer
JPH05242143A (en) 1992-02-26 1993-09-21 Hitachi Ltd Multi-media presentation system
US5831615A (en) * 1994-09-30 1998-11-03 Intel Corporation Method and apparatus for redrawing transparent windows
US6323911B1 (en) * 1995-10-02 2001-11-27 Starsight Telecast, Inc. System and method for using television schedule information
JP3636830B2 (en) * 1996-07-10 2005-04-06 株式会社日立製作所 Intelligent video camera and intelligent still camera
US6119147A (en) * 1998-07-28 2000-09-12 Fuji Xerox Co., Ltd. Method and system for computer-mediated, multi-modal, asynchronous meetings in a virtual space
JP2000341572A (en) * 1999-05-31 2000-12-08 Ricoh Co Ltd Digital camera
JP2001186430A (en) * 1999-12-22 2001-07-06 Mitsubishi Electric Corp Digital broadcast receiver
JP2001250332A (en) * 2000-03-06 2001-09-14 Sony Corp Reproducing device
US20020173345A1 (en) 2001-05-17 2002-11-21 Swerup Jan Ingemar Method and system for phone note taking
US7502386B2 (en) 2002-03-27 2009-03-10 Mitsubishi Denki Kabushiki Kaisha Communication apparatus and communication method
JP2004193979A (en) 2002-12-11 2004-07-08 Canon Inc Video distribution system
US7797064B2 (en) * 2002-12-13 2010-09-14 Stephen Loomis Apparatus and method for skipping songs without delay
FR2868231B1 (en) 2004-03-26 2006-11-10 Rachid Hallalel DEVICE FOR VISUAL CONSULTATION AND EXCHANGE OF IMAGES BETWEEN TWO MOBILE TELEPHONES THEN THE EXTRACTION AND AUTOMATIC SCROLLING OF TEXT IN IMAGE OVERLAY
US20050273700A1 (en) * 2004-06-02 2005-12-08 Amx Corporation Computer system with user interface having annotation capability
US7716376B1 (en) * 2006-03-28 2010-05-11 Amazon Technologies, Inc. Synchronized video session with integrated participant generated commentary
US20090064245A1 (en) * 2007-08-28 2009-03-05 International Business Machines Corporation Enhanced On-Line Collaboration System for Broadcast Presentations
US20090119100A1 (en) 2007-11-07 2009-05-07 Raji L Akella Associating annotation recording with a cell phone number
GB0801429D0 (en) * 2008-01-25 2008-03-05 Decisive Media Ltd Media Annotation system, method and media player
US9275684B2 (en) * 2008-09-12 2016-03-01 At&T Intellectual Property I, L.P. Providing sketch annotations with multimedia programs
CN101505396B (en) 2009-03-10 2011-06-15 中兴通讯股份有限公司 Method and system for displaying mobile phone television service and visible telephone service
US8380866B2 (en) * 2009-03-20 2013-02-19 Ricoh Company, Ltd. Techniques for facilitating annotations
JP5487695B2 (en) 2009-04-13 2014-05-07 コニカミノルタ株式会社 Relay device, conference support system, and program
US9094715B2 (en) * 2009-05-29 2015-07-28 Cognitive Networks, Inc. Systems and methods for multi-broadcast differentiation
US20110154399A1 (en) * 2009-12-22 2011-06-23 Verizon Patent And Licensing, Inc. Content recommendation engine
US20130347029A1 (en) * 2012-06-21 2013-12-26 United Video Properties, Inc. Systems and methods for navigating to content without an advertisement
JP2014127987A (en) * 2012-12-27 2014-07-07 Sony Corp Information processing apparatus and recording medium
FR3005436B1 (en) * 2013-05-13 2016-01-01 Fasver METHOD AND DEVICE FOR SECURELY PROTECTING AN OFFICIAL DOCUMENT AND OFFICIAL DOCUMENT SO PROTECTED
KR102282003B1 (en) * 2014-08-07 2021-07-27 삼성전자 주식회사 Electronic device and method for controlling display thereof

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5600775A (en) 1994-08-26 1997-02-04 Emotion, Inc. Method and apparatus for annotating full motion video and other indexed data structures
EP0836114A1 (en) * 1996-10-11 1998-04-15 Eastman Kodak Company Full frame annotation system for camera
WO2001065832A1 (en) * 2000-02-29 2001-09-07 Watchpoint Media, Inc. A method and apparatus for receiving a hyperlinked television broadcast
CN101473294A (en) * 2006-06-20 2009-07-01 诺基亚公司 System, device, method, and computer program product for annotating media files
US20080276272A1 (en) 2007-05-02 2008-11-06 Google Inc. Animated Video Overlays

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2015001856A (en) * 2013-06-17 2015-01-05 コニカミノルタ株式会社 Image display device, display control program, and display control method
EP3246796A4 (en) * 2015-02-12 2018-02-28 Samsung Electronics Co., Ltd. Device and method for inputting note information into image of photographed object
US10778928B2 (en) 2015-02-12 2020-09-15 Samsung Electronics Co., Ltd. Device and method for inputting note information into image of photographed object

Also Published As

Publication number Publication date
JP2014504760A (en) 2014-02-24
KR101452667B1 (en) 2014-10-22
CN103384874B (en) 2016-09-28
CN103384874A (en) 2013-11-06
US9715326B2 (en) 2017-07-25
KR20130076878A (en) 2013-07-08
JP5852135B2 (en) 2016-02-03
EP2686761A4 (en) 2014-11-19
US20130027425A1 (en) 2013-01-31
EP2686761A1 (en) 2014-01-22

Similar Documents

Publication Publication Date Title
US9715326B2 (en) Superimposed annotation output
US10375313B1 (en) Creative camera
US10839572B2 (en) Contextual virtual reality interaction
US20220319139A1 (en) Multi-endpoint mixed-reality meetings
US10359905B2 (en) Collaboration with 3D data visualizations
US20230098395A1 (en) Wide angle video conference
US10855481B2 (en) Live ink presence for real-time collaboration
EP3285175A2 (en) Multiple-device screen capture
US11694371B2 (en) Controlling interactivity of digital content overlaid onto displayed data via graphics processing circuitry using a frame buffer
US9495064B2 (en) Information processing method and electronic device
US20160182579A1 (en) Method of establishing and managing messaging sessions based on user positions in a collaboration space and a collaboration system employing same
US20210034318A1 (en) Shared volume computing architecture of a virtual reality environment and related systems and methods
US20230102820A1 (en) Parallel renderers for electronic devices
US20230040610A1 (en) Object placement for electronic devices
JP2016033831A (en) Overlapped annotation output
US20220327748A1 (en) Determining a change in position of displayed digital content in subsequent frames via graphics processing circuitry
US20220244903A1 (en) Application casting
US20230326094A1 (en) Integrating overlaid content into displayed data via graphics processing circuitry and processing circuitry using a computing memory and an operating system memory
US11928303B2 (en) Shared-content session user interfaces
US20230221830A1 (en) User interface modes for three-dimensional display
US20240064395A1 (en) Wide angle video conference
US20230298221A1 (en) Method and system for controlling access to virtual and real-world environments for head mounted device
WO2023014618A1 (en) Object placement for electronic devices
WO2023215637A1 (en) Interactive reality computing experience using optical lenticular multi-perspective simulation
CN117369930A (en) Interface control method and device, electronic equipment and readable storage medium

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 13502011

Country of ref document: US

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 11860725

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 20137010900

Country of ref document: KR

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 2013549694

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE