US20140359477A1 - Universal environment extender - Google Patents
Universal environment extender Download PDFInfo
- Publication number
- US20140359477A1 US20140359477A1 US13/909,889 US201313909889A US2014359477A1 US 20140359477 A1 US20140359477 A1 US 20140359477A1 US 201313909889 A US201313909889 A US 201313909889A US 2014359477 A1 US2014359477 A1 US 2014359477A1
- Authority
- US
- United States
- Prior art keywords
- transmitter
- receiver
- extender
- content
- display
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/436—Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
- H04L67/125—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks involving control of end-device applications over a network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
- G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/40—Support for services or applications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/104—Peer-to-peer [P2P] networks
- H04L67/1074—Peer-to-peer [P2P] networks for supporting data block transmission mechanisms
- H04L67/1078—Resource delivery mechanisms
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/222—Secondary servers, e.g. proxy server, cable television Head-end
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/41—Structure of client; Structure of client peripherals
- H04N21/422—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
- H04N21/42204—User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor
- H04N21/42206—User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor characterized by hardware details
- H04N21/4222—Remote control device emulator integrated into a non-television apparatus, e.g. a PDA, media center or smart toy
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/02—Network architectures or network communication protocols for network security for separating internal from external traffic, e.g. firewalls
- H04L63/029—Firewall traversal, e.g. tunnelling or, creating pinholes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/104—Peer-to-peer [P2P] networks
Definitions
- the present invention is related to smart devices and more specifically to a system and method for extending such devices.
- smart devices such as smart phones and tablets become ubiquitous and prevalent, it becomes desirable that the user interface and experience of such device can be extended beyond the physical bound of the smart devices.
- the smart device serves as one of the senses of the human body, it can act as an environment extender that can greatly extend the physical boundary of the user.
- Smart devices come from different vendors and are loaded with different operating systems (OS), such as Android, iOS and Linux. It becomes a big challenge for the extender to be able to work with different smart devices, under different OS and/or environment.
- OS operating systems
- the concept of the environment extender is not new. It has been limitedly realized by Apple TV and Android Miracast. While in Apple TV, the audio and video content of an iPhone or iPad can be mirrored to an Apple TV settop box that is hooked up to a high definition TV (HDTV). The audio and video content are streamed from the smart device, iPhone or iPad, to the HDTV.
- the Apple TV can also receive selected media file format, such mp4, mp3 or jpeg, from the smart device and play on the HDTV.
- Apple TV serves as an environment extender. It can either “mirror” the smart device environment on to the HDTV through Apple TV, or “stream certain content” from the smart device to the HDTV.
- the smart device environment is “mirrored” to the HDTV through an Android settop box.
- the Android settop box can also received “certain streamed content” from the smart device and play it on the HDTV.
- a universal environment extender comprises a smart device; and a main control utility running on the smart device. Then the main control utility allows the extender as a receiver to operate in a mode that allows the extender to explore and reverse-explore the source content/applications and to extend the display environment from another smart device as a transmitter.
- FIG. 1 is a block diagram of a conventional media player with receiver main/mirror/player app.
- FIG. 2 is a diagram of a universal environment extender in native mirror mode.
- FIG. 3 is a diagram of the universal environment extender when exploring a receiver main menu.
- FIG. 4 is a diagram of the universal environment extender when in a setting mode.
- FIG. 5 is a diagram of the universal environment extender when in a receiver app explorer mode.
- FIG. 6 is a diagram of the universal environment extender when in a receiver app player mode.
- FIG. 7 is a diagram of the universal environment extender when in a receiver content explorer mode.
- FIG. 8 is a diagram of the universal environment extender when in a receiver content player mode.
- FIG. 9 is a flow chart of the operation of transmitter main control utility.
- FIG. 10 is a flow chart of the operation of receiver main control utility.
- FIG. 11 is a flow chart of the operation of receiver app explorer/player utility.
- FIG. 12 is a flow chart of the operation of receiver content explorer/player utility.
- FIG. 13 is a Universal Environment Extender (Receiver) in a cloud Infrastructure
- the present invention is related to smart devices and more specifically to a system and method for extending such devices.
- the following description is presented to enable one of ordinary skill in the art to make and use the invention and is provided in the context of a patent application and its requirements.
- Various modifications to the preferred embodiments and the generic principles and features described herein will be readily apparent to those skilled in the art.
- the present invention is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features described herein.
- a system and method in accordance with an embodiment comprises a receiver, a transmitter and a pair of applications (apps) running on both devices.
- One receiving app is running under the receiver while another transmitter app is running under the target smart device.
- the receiver is able to interact with any smart device with the transmitter app installed. It is able to mirror any smart device environment and to stream any intended file content to the remote display such as HDTV, smart watch, smart glasses, smart conference room projector, smart settop box, smart car entertainment system, medical console, or smart gadgets.
- a system and method in accordance with the present invention will also expand the network interface between the transmitter and the receiver from wireless local area network (WLAN) to wide area network (WAN).
- WLAN wireless local area network
- WAN wide area network
- a receiver can be accessed as a private cloud server from anywhere at anytime by the transmitter as a smart device client.
- the transmitter can also access the receiver as a private cloud server behind the firewall with fixed or dynamic IP addresses.
- the receiver as a smart device client requires no outside or third party routing server in the WAN; requires no additional router setup in the LAN to establish a secure peer-to-peer communication channel with the receiver as a private cloud server.
- the transmitter as a smart device client can access services or storage from the receiver as a private cloud server.
- a universal environment extender is provided that is able to work with the smart devices from different vendors with different OS. It is able to extend the user interface and user experience beyond the physical boundary of the smart device onto a remote environment that is also accessible by the user.
- An extender in accordance with an embodiment can be utilized with HDTV, smart watch, smart glasses, smart conference room projector, smart settop box, smart car entertainment system, medical console, or smart gadgets.
- the content in general is divided into two categories:
- a transmitter would “push” content through the interface to a receiver.
- the receiver then decodes the content and play on the local display.
- This mechanism has been applied by Apple TV as well as Android media player.
- the problem becomes obvious when the transmitter is able to “explore” and discover other content beyond its local access domain.
- the content would have to be downloaded or streamed to the transmitter and then send to the receiver.
- the routing distance from the source of the content to the ultimate display would have been much longer, if not double. It is one of the purposes of this invention to make the streaming of the content more efficient by allowing the receiver to “pull” the content directly from the source, be it on the transmitter or on the network resources where the transmitter has access to.
- a conventional media player 12 has the main/mirror/player app 13 running.
- the media player 12 may be connected by a keyboard 14 , a mouse 15 , and/or a remote control stick 16 . It also may be wirelessly connected by a smart phone 19 , or a tablet 17 .
- the smart phone 19 and the tablet 17 are collectively caller smart devices.
- the smart devices have app running in UI (user interface) mode 191 , or in content mode 18 .
- the receiver display device 10 can be a HDTV, projector, automobile panel, or a watch panel.
- the display panel 11 shows the viewable content from the result of the receiver main/mirror/player app.
- the transmitter 17 , 19 is able to “push” content to the receiver 12 in mirror mode, which displays the exact mirror of the content on receiver display panel 11 as shown on the transmitter display panel 170 or 190 .
- the transmitter 17 , 19 is also able to run in UI mode 191 in order to control the receiver main/mirror/player utility 13 to do configuration and system function.
- the transmitter 17 , 19 is further able to run in app mode 18 to “push” the selected content, video, audio and/or photo, to the receiver display panel 11 or audio function (not shown) connected to the receiver 12 .
- the terms “display” or “display panel” apply to both video and audio functions on the display devices.
- content refers generally to video, audio, photos, and document files.
- application refers generally to application installed on the transmitter and receiver.
- stream refers generally to “play” and/or “read/write” the content from/to the source.
- wireless refers to any wireless network including WiFi local area network (WLAN) or wide area network (WAN).
- explorer refers to storage, file or content explorer. In some situation, it refers to app explorer.
- storage refers to accessible storage including local or network storage.
- FIG. 1 it is cumbersome to require a keyboard 14 , a mouse 15 , and/or a remote control stick 16 to conduct configuration or operation onto the functions of the receiver 12 . It would be beneficial to do away with the keyboard, mouse and/or remote control stick and yet be able to conduct configuration or operation onto the functions of the receiver 12 . By eliminating this user interface devices it saves costs and will also provide a more elegant solution for the user.
- two smart devices behaving as a transmitter and a receiver can work in two different modes.
- One is peer-to-peer mode, while another is joining the wireless access point (AP) mode.
- the peer-to-peer mode is sufficient with one serving as a router while many clients may connect to the router. But the “joining the wireless access point” mode is more common at work or at home for the smart devices.
- the transmitter is able to configure the receiver setting either through a browser or an app.
- another wireless access point exists on the wireless network that serves as a main router, it is beneficial to have both the transmitter and the receiver to join this AP network such that both can access to the resources on the network.
- the resources available include network storage, Internet capability, network printing, network audio and others.
- the transmitter it becomes difficult and not obvious for the transmitter to configure the newly discovered receiver through the peer-to-peer network and to discover the available access point (AP) on the open wireless environment. It is a second objective to allow the transmitter to be able to discover the receiver and the wireless access point separately and be able to configure the receiver as well as to connect the receiver to the access point all through the transmitter user interface panel on the transmitter, without resorting to any external means like mouse, keyboard or a remote control stick.
- the playable content includes video, audio, photo, and document.
- the document includes Word, Excel, Power Point, PDF, Text and others.
- the object/link content includes: URL link, app links such as YouTube link, Facebook link and others.
- the playable content limited to video, audio and photo, is “pushed” from the transmitter 17 , 19 to the receiver 12 .
- the receiver player utility 13 will decode the playable content and “play” or display on the receiver display panel 11 .
- the object/link is more appropriately referred to as app.
- the transmitter and receiver tend to work in pairs. It implies that the transmitter and the receiver have to run in the same domain of the OS (operating system), such as iOS, Android or Linux.
- OS operating system
- Video, audio and photo are the most obvious streaming applications between the transmitter and receiver.
- the applications are limited by the traditional receiver to only support these three content formats for streaming. It is a fourth objective to stream Word, text, power point, Excel, PDF and other documentation file formats in addition to the traditional video, audio, photo content.
- a sixth objective of the invention is to expand the interface between the transmitter and the receiver from wireless local area network (WLAN) to wide area network (WAN).
- WLAN wireless local area network
- WAN wide area network
- the realization of this objective is described in patent application: Ser. No. entitled PRIVATE CLOUD SERVER AND CLIENT ARCHITECTURE WITHOUT UTILIZING A ROUTING SERVER Ser. No. 13/229,285. It allows accessing the receiver as a private cloud server from anywhere at anytime by the transmitter as a smart device client.
- the transmitter can also access the receiver as a private cloud server behind the firewall with fixed or dynamic IP addresses.
- the receiver as a smart device client requires no outside or third party routing server in the WAN; requires no additional router setup in the LAN to establish a secure peer-to-peer communication channel with the receiver as a private cloud server.
- the transmitter as a smart device client can access services or storage from the receiver as a private cloud server.
- a transmitter 24 , 27 has a main control utility as shown in FIG. 9
- a receiver 22 has a corresponding main control utility as shown in FIG. 10 .
- the transmitter main control utility the transmitter 24 , 27 first attempts to discover the receiver 90 . Once the receiver 22 is discovered, it checks if the native mirror mode is activated 91 . If so, the transmitter 24 , 27 sends native mirror mode request to the receiver 910 . It starts interacting with the receiver 911 . As shown in FIG. 2 , the transmitter 24 , 27 stays in native mirror mode 26 , 29 .
- Its display panel 25 , 28 is mirrored on the receiver display panel 21 , with the universal environment extender 22 as the receiver running under native mirror mode 23 .
- the display content 21 is displayed and mirrored on the receiver display device 20 .
- the receiver 22 has access to the storage 220 as well as applications 221 .
- the storage includes the local storage and the network accessible storage on the network.
- the transmitter 24 , 27 will install the virtual UI driver 92 which allows it to communicate with the receiver 22 .
- the transmitter 24 will have icons displayed on its reverse mirror panel 36 , and runs in reverse mirror mode 920 to control the receiver 22 .
- finger can move freely in display panel area 35 .
- several icons are displayed and can be selected, including “volume down” 391 , “volume up” 391 , “back to last” 387 , “home” 388 , “soft keyboard” 389 , “reverse mirror/toggle” 381 , “choose display” 382 and “shut down” 390 .
- the transmitter 27 will have icons displayed on its remote control panel 38 , and runs in remote control mode 921 to control the receiver 22 .
- finger mouse 383 can move freely in mouse pad area 380 .
- remote control panel 38 several icons are displayed and can be selected, including “search left” 384 , “play/pause” 385 , “search right” 389 , “volume down” 391 , “volume up” 391 , “back to last” 387 , “home” 388 , “soft keyboard” 389 , “reverse mirror/toggle” 381 , “choose display” 382 and “shut down” 390 .
- the transmitter 24 in reverse mirror mode 36 can toggle to remote control mode 39 in transmitter 27 at any time, if necessary 921 , by selecting “reverse mirror/toggle” icon 381 .
- the unit can safely be shut down by selecting icon “shut down” 390 .
- the icons “search left” 384 , “play/pause” 385 , “search right” 386 , once selected, are used mainly by video, audio and photos content explorer and player.
- the icons “volume down” 391 and “volume up” 392 are for speaker volume control on the receiver 22 , once selected.
- the icon “back to last” 387 is selected to go back to last action.
- the icon “choose display” 382 is selected to choose which receiver 22 to actively play the content in the storage 220 from the transmitter 24 , 27 .
- the playable display panel includes the transmitter's own local display 35 , 38 , as well as any discoverable receiver 22 on the network.
- the transmitter 27 is toggled to reverse mirror mode 36 . Its display panel 35 will mirror whatever is displayed on the receiver display panel 31 . Some of the icons from the transmitter 27 remote control panel 38 are also shown on the bottom row of the receiver display panel 31 for control in the reverse mirror mode 36 . Examples are: “volume down” 391 , “volume up” 391 , “back to last” 387 , “home” 388 , “soft keyboard” 389 , “reverse mirror/toggle” 381 , “choose display” 382 and “shut down” 390 . While in reverse mirror mode 36 , the transmitter 24 can also toggle to remote control mode 39 at any time, if necessary, by selecting “reverse mirror/toggle” icon 381 .
- the “choose display” icon 382 and “reverse mirror/toggle” icon 381 combine to designate which display is active during the operation.
- the “choose display” icon 382 selects the main active display panel.
- “choose display” icon 382 selects the transmitter, only the transmitter display panel is selected as the main active display.
- the receiver display panel is the main active display while the transmitter display panel is also selected.
- “choose display” icon 382 selects the receiver while “reverse mirror/toggle” icon 381 is off, only the receiver display panel is selected.
- the receiver 22 starts its main control utility. It first listens to the transmitter request 100 and tries to be discovered by the transmitter 101 . Once it is discovered, it checks if the native mirror mode has been activated 102 ? If so, it starts processing the native mirror mode 1020 .
- the receiver 22 installs the virtual UI driver 93 . It displays the main menu 31 , 104 . The displayed menu is mirrored back to transmitter 24 , 27 , if necessary 106 , and waits for the transmitter input 105 .
- one of the main control icons 32 is selected on the receiver display panel 31 .
- the same icon 34 is selected on the transmitter display panel 35 that is running under reverse mirror mode 36 . If the transmitter 27 is running under remote control mode 39 , the control icons can be selected on the transmitter display panel 38 .
- the corresponding controls icons are also shown on the bottom row of the receiver display panel 31 .
- the transmitter 24 , 27 starts exploring receiver main menu 36 , 39 , 93 . If the setting mode is selected, the transmitter 24 , 27 sends request to the receiver 940 . It can toggle between the reverse mirror mode 36 and remote control mode 39 , by selecting the reverse mirror/toggle icon 381 in either mode 941 .
- the transmitter 24 , 27 starts interacting with the receiver 942 .
- the receiver 22 starts processing the request 1070 . The interaction between the transmitter 24 , 27 and the receiver 22 is shown as in FIG. 4 .
- more setting icons are available, including wireless setting 42 , display setting 440 , system setting 47 and advanced setting 48 .
- the wireless setting 42 is for receiver wireless attributes including ID, password, and channel in addition to the available wireless access point (AP) information.
- the corresponding receiver setting utility 43 and transmitter setting utility 46 , 49 allow transmitter 24 , 27 to configure (not shown) the receiver 22 to join another wireless network access point (AP) while on a peer-to-peer wireless network with the receiver 22 .
- the display setting 440 is for receiver 22 and receiver display device 20 .
- the system setting 47 is for the system setting of receiver 22 , including system information and firmware upgrade.
- the advanced setting 48 covers the rest of the setting aspects of the receiver 22 that is more for expert mode setting.
- one of the setting icons 42 is selected on the receiver display panel 41 .
- the same icon 44 is selected on the transmitter display panel 45 that is running under reverse mirror mode 46 . If the transmitter 27 is running under remote control mode 49 , the control icons can be selected on the transmitter display panel 38 .
- the corresponding controls icons are also shown on the bottom row of the receiver display panel 41 .
- the transmitter 24 , 27 sends request to the receiver 96 . It can toggle between the reverse mirror mode 36 and remote control mode 39 , by selecting the reverse mirror/toggle icon 381 in either mode 961 .
- the transmitter 24 , 27 starts interacting with the receiver 962 . On the receiver side, it starts the receiver app explorer/player utility 53 , 109 as in FIG. 5 and FIG. 10 .
- the receiver 22 starts the app explorer/player utility 53 and app explorer 111 . It waits for the transmitter selection 113 to determine if the app from applications 221 is playable on the receiver 114 ? If it is playable, the app from applications 221 is run on the receiver 116 . If it is not playable, the receiver 22 will locate outside resources to play the app, if necessary 115 .
- one of the app icons 52 from applications 221 is selected on the receiver display panel 51 .
- the same icon 54 is selected on the transmitter display panel 55 that is running under reverse mirror mode 56 . If the transmitter 27 is running under remote control mode 59 , the control icons can be selected on the transmitter display panel 38 .
- the corresponding controls icons are also shown on the bottom row of the receiver display panel 51 .
- the selected app 52 from applications 221 is played on the receiver display panel 61 by the receiver app explorer/player utility 53 .
- the selected app 52 can be reverse mirrored 66 back to the transmitter 24 on its panel 65 .
- remote control mode 69 or reverse mirror mode 66 , most control icons are available on transmitter display panel 38 , 65 or on the bottom row of the receiver display panel 61 .
- the transmitter 24 , 27 will first send the request to the receiver 950 . It can toggle between the reverse mirror mode 36 and remote control mode 39 , by selecting the reverse mirror/toggle icon 381 in either mode 951 . The transmitter 24 , 27 then explores the local content 9510 in the storage 220 and sends the content location to the receiver 9511 and starts interacting with the receiver 962 . On the receiver side, as shown in FIG. 10 , if the explorer mode is selected, it starts receiver explorer utility 1080 .
- one of the content type icons 72 is selected on the receiver display panel 71 .
- the same icon 74 is selected on the transmitter display panel 75 that is running under reverse mirror mode 76 . If the transmitter 27 is running under remote control mode 79 , the control icons can be selected on the transmitter display panel 38 .
- the corresponding controls icons are also shown on the bottom row of the receiver display panel 71 .
- the selected content type 72 will further be explored (not shown) by the receiver content explorer/player utility 73 .
- the selected content in the storage 220 is eventually played on the receiver display panel 81 by the receiver app explorer/player utility 73 .
- the played content in the storage 220 can be reverse mirrored 86 back to the transmitter 24 on its display panel 85 .
- remote control mode 89 or reverse mirror mode 86 , most control icons are available on transmitter display panel 38 , 85 or on the bottom row of the receiver display panel 81 .
- the “choose display” icon 382 and “reverse mirror/toggle” icon 381 combine to designate which display is active during the operation.
- “choose display” icon 382 selects the transmitter, only the transmitter display panel is selected.
- “choose display” icon 382 selects the receiver while “reverse mirror/toggle” icon 381 is on, the receiver display panel as well as the transmitter display panel are selected.
- “choose display” icon 382 selects the receiver while “reverse mirror/toggle” icon 381 is off, only the receiver display panel is selected.
- the receiver content explorer/player utility 73 accepts content location info from the transmitter 120 . It then starts reverse content explorer onto the transmitter 121 . It will also explore the content in the storage 220 from the receiver 22 and its accessible network domain 1210 . It waits for the selection from the transmitter 122 . The content in the storage 220 is “pulled” from the source and streamed to the receiver 123 . The receiver 22 will decrypt the content in the storage 220 , if necessary 126 . It checks if the content is playable on the receiver 128 ? If so, the content is played 1280 . If not, the receiver 22 will locate outside resources to play the content, if necessary 129 .
- the private cloud server 1308 and client 1327 , 1321 functions as a receiver.
- the smart device client 1306 , 1309 , 1310 , 1301 functions as a transmitter.
- the transmitter and receiver each has a corresponding display panel 1336 , 1339 , 1330 , 1340 and 1337 , 1338 , 1331 .
- the transmitter and receiver pair can reside on the same LAN 1304 and connect directly wired or wireless, as with transmitter 1306 and receiver 1308 , 1327 .
- the transmitter 1301 can also connect to a receiver 1308 behind a Router_P 1302 and LAN 1304 through the WAN and public cloud 1300 .
- the transmitter 1309 , 1310 behind a Router_S 1303 and LAN 1305 can connect to a receiver 1308 behind a Router_P 1302 and LAN 1304 through the WAN and public cloud 1300 .
- the “choose display” and “reverse mirror/toggle” icons combine to select the active display in one of three combinations of transmitter display panel only, receiver display panel only or both display panels.
- While the transmitter 1301 , 1309 , 1310 and the receiver 1308 are connected through the public cloud 1300 , it is beneficial to set the “choose display” option to transmitter display panel such that the played content video, audio, photos or documents can be streamed directly from the source to the transmitter display panel in the cloud access application.
- Another alternative is to set the “choose display” to the receiver display panel and the “reverse mirror/toggle” to ON such that the receiver display panel is chosen and the transmitter display panel is also turned on to mirror the receiver display panel. This alternative setting is best for streaming the receiver accessible applications from the cloud to the transmitter display panel.
- FIG. 13 shows a number of scenarios of the universal environment extender systems.
- One scenario is between transmitter 1309 and the receiver 1321 on LAN 1305 .
- the source of the content/applications can be from any member on the LAN 1305 .
- the target display panels can be either on the transmitter display panel 1339 itself or on the receiver display panel 1331 or on both transmitter and receiver display panels.
- Another scenario is between transmitter 1310 and the receiver 1308 across the cloud 1300 through the LAN 1305 and 1304 .
- the source of the content/applications can be from any member on the LAN 1305 and LAN 1304 .
- the target display panels can be either on the transmitter display panel 1330 itself or on the receiver display panel 1338 , 1337 across the cloud 1300 on LAN 1304 , or on the receiver panel 1331 on the same LAN 1305 , or on both transmitter and receiver display panels.
- the target display panel can be either the transmitter display panel 1339 itself, or the receiver display panel 1331 .
- the receiver display panel 1331 can be a HDTV, smart watch, smart glasses, smart conference room projector, smart settop box, smart car entertainment system, medical console, or smart gadget that is physically viewable from where the transmitter 1309 is located.
- the transmitter 1310 can view the source content/applications stored on any one member on the LAN 1304 routing through the private cloud server/extender/receiver 1308 .
- the source content/applications can then be viewable either on the transmitter display panel 1330 itself, or on the receiver display panel 1331 on the same LAN.
- the receiver display panel 1331 can be a HDTV, smart watch, smart glasses, smart conference room projector, smart settop box, smart car entertainment system, medical console, or smart gadget that is physically viewable from where the transmitter 1309 is located.
- the universal environment extender can apply to HDTV, smart watch, smart glasses, smart conference room projector, smart settop box, smart car entertainment system, medical console, or smart gadgets.
Abstract
Description
- This application is related to U.S. patent application Ser. No. 13/229,285 (Docket No. 5039P), filed Sep. 9, 2011, entitled “PRIVATE CLOUD SERVER AND CLIENT ARCHITECTURE WITHOUT UTILIZING A ROUTING SERVER,” and U.S. patent application Ser. No. 12/912,614 (Docket No. 4929P), filed Oct. 26, 2010, entitled “DUAL-MODE WIRELESS NETWORKED DEVICE INTERFACE AND AUTOMATIC CONFIGURATION THEREOF,” all of which are incorporated herein by reference in there entireties.
- The present invention is related to smart devices and more specifically to a system and method for extending such devices.
- As smart devices such as smart phones and tablets become ubiquitous and prevalent, it becomes desirable that the user interface and experience of such device can be extended beyond the physical bound of the smart devices. If the smart device serves as one of the senses of the human body, it can act as an environment extender that can greatly extend the physical boundary of the user.
- Smart devices come from different vendors and are loaded with different operating systems (OS), such as Android, iOS and Linux. It becomes a big challenge for the extender to be able to work with different smart devices, under different OS and/or environment.
- The concept of the environment extender is not new. It has been limitedly realized by Apple TV and Android Miracast. While in Apple TV, the audio and video content of an iPhone or iPad can be mirrored to an Apple TV settop box that is hooked up to a high definition TV (HDTV). The audio and video content are streamed from the smart device, iPhone or iPad, to the HDTV. The Apple TV can also receive selected media file format, such mp4, mp3 or jpeg, from the smart device and play on the HDTV. In this case, Apple TV serves as an environment extender. It can either “mirror” the smart device environment on to the HDTV through Apple TV, or “stream certain content” from the smart device to the HDTV.
- In the case of Miracast on Android platform, the smart device environment is “mirrored” to the HDTV through an Android settop box. The Android settop box can also received “certain streamed content” from the smart device and play it on the HDTV.
- While both platforms can “mirror” and “stream”, it is limited to the smart device, as the transmitter, running under the same OS with the settop box, as the receiver. Furthermore, the file format support is limited. It is usually limited to just mp4, mp3 and jepeg files.
- It is usually difficult, if not impossible, to transmit content from one smart device to another receiver that is running under a different OS. Examples are: transmitting from an iDevice to an Android receiver; transmitting from an Android device to an Apple receiver such as an Apple TV.
- A universal environment extender is disclosed. The universal environment extender comprises a smart device; and a main control utility running on the smart device. Then the main control utility allows the extender as a receiver to operate in a mode that allows the extender to explore and reverse-explore the source content/applications and to extend the display environment from another smart device as a transmitter.
-
FIG. 1 is a block diagram of a conventional media player with receiver main/mirror/player app. -
FIG. 2 is a diagram of a universal environment extender in native mirror mode. -
FIG. 3 is a diagram of the universal environment extender when exploring a receiver main menu. -
FIG. 4 is a diagram of the universal environment extender when in a setting mode. -
FIG. 5 is a diagram of the universal environment extender when in a receiver app explorer mode. -
FIG. 6 is a diagram of the universal environment extender when in a receiver app player mode. -
FIG. 7 is a diagram of the universal environment extender when in a receiver content explorer mode. -
FIG. 8 is a diagram of the universal environment extender when in a receiver content player mode. -
FIG. 9 is a flow chart of the operation of transmitter main control utility. -
FIG. 10 is a flow chart of the operation of receiver main control utility. -
FIG. 11 is a flow chart of the operation of receiver app explorer/player utility. -
FIG. 12 is a flow chart of the operation of receiver content explorer/player utility. -
FIG. 13 is a Universal Environment Extender (Receiver) in a cloud Infrastructure - The present invention is related to smart devices and more specifically to a system and method for extending such devices. The following description is presented to enable one of ordinary skill in the art to make and use the invention and is provided in the context of a patent application and its requirements. Various modifications to the preferred embodiments and the generic principles and features described herein will be readily apparent to those skilled in the art. Thus, the present invention is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features described herein.
- A system and method in accordance with an embodiment comprises a receiver, a transmitter and a pair of applications (apps) running on both devices. One receiving app is running under the receiver while another transmitter app is running under the target smart device. The receiver is able to interact with any smart device with the transmitter app installed. It is able to mirror any smart device environment and to stream any intended file content to the remote display such as HDTV, smart watch, smart glasses, smart conference room projector, smart settop box, smart car entertainment system, medical console, or smart gadgets.
- A system and method in accordance with the present invention will also expand the network interface between the transmitter and the receiver from wireless local area network (WLAN) to wide area network (WAN). By applying principles disclosed for example in U.S. patent application Ser. No. 13/229,285, entitled “PRIVATE CLOUD SERVER AND CLIENT ARCHITECTURE WITHOUT UTILIZING A ROUTING SERVER,” assigned to the assignee of the present application, a receiver can be accessed as a private cloud server from anywhere at anytime by the transmitter as a smart device client. The transmitter can also access the receiver as a private cloud server behind the firewall with fixed or dynamic IP addresses. The receiver as a smart device client requires no outside or third party routing server in the WAN; requires no additional router setup in the LAN to establish a secure peer-to-peer communication channel with the receiver as a private cloud server. The transmitter as a smart device client can access services or storage from the receiver as a private cloud server.
- A universal environment extender is provided that is able to work with the smart devices from different vendors with different OS. It is able to extend the user interface and user experience beyond the physical boundary of the smart device onto a remote environment that is also accessible by the user.
- An extender in accordance with an embodiment can be utilized with HDTV, smart watch, smart glasses, smart conference room projector, smart settop box, smart car entertainment system, medical console, or smart gadgets.
- The content in general is divided into two categories:
-
- 1. Content accessible to the receiver
- 2. Content accessible to the transmitter, but not to the receiver
- Traditionally, a transmitter would “push” content through the interface to a receiver. The receiver then decodes the content and play on the local display. This mechanism has been applied by Apple TV as well as Android media player. The problem becomes obvious when the transmitter is able to “explore” and discover other content beyond its local access domain. The content would have to be downloaded or streamed to the transmitter and then send to the receiver. The routing distance from the source of the content to the ultimate display would have been much longer, if not double. It is one of the purposes of this invention to make the streaming of the content more efficient by allowing the receiver to “pull” the content directly from the source, be it on the transmitter or on the network resources where the transmitter has access to.
- For those content that are accessible to the receiver, it will build a content explorer. The content is then explored and selected by the transmitter. The content is then “pulled” by the receiver from the original source domain,
- Only for those content that are accessible to the transmitter, but not to the receiver, it will build a reverse content explorer, based on the information from the transmitter. The content is then explored and selected by the transmitter. The content is then “pulled” by the receiver through transmitter from the original source domain.
- As shown in
FIG. 1 , aconventional media player 12 has the main/mirror/player app 13 running. Themedia player 12 may be connected by akeyboard 14, amouse 15, and/or aremote control stick 16. It also may be wirelessly connected by asmart phone 19, or atablet 17. Thesmart phone 19 and thetablet 17 are collectively caller smart devices. The smart devices have app running in UI (user interface)mode 191, or incontent mode 18. Connected to the media player (receiver) 12, thereceiver display device 10 can be a HDTV, projector, automobile panel, or a watch panel. Thedisplay panel 11 shows the viewable content from the result of the receiver main/mirror/player app. Thetransmitter receiver 12 in mirror mode, which displays the exact mirror of the content onreceiver display panel 11 as shown on thetransmitter display panel - The
transmitter UI mode 191 in order to control the receiver main/mirror/player utility 13 to do configuration and system function. Thetransmitter app mode 18 to “push” the selected content, video, audio and/or photo, to thereceiver display panel 11 or audio function (not shown) connected to thereceiver 12. - Throughout the rest of this description, the terms “display” or “display panel” apply to both video and audio functions on the display devices. The term “content” refers generally to video, audio, photos, and document files. The term “app” refers generally to application installed on the transmitter and receiver. The term “stream” refers generally to “play” and/or “read/write” the content from/to the source. The term “wireless” refers to any wireless network including WiFi local area network (WLAN) or wide area network (WAN). The term “explorer” refers to storage, file or content explorer. In some situation, it refers to app explorer. The term “storage” refers to accessible storage including local or network storage.
- As can be seen from
FIG. 1 , it is cumbersome to require akeyboard 14, amouse 15, and/or aremote control stick 16 to conduct configuration or operation onto the functions of thereceiver 12. It would be beneficial to do away with the keyboard, mouse and/or remote control stick and yet be able to conduct configuration or operation onto the functions of thereceiver 12. By eliminating this user interface devices it saves costs and will also provide a more elegant solution for the user. - In a wireless networking environment, two smart devices behaving as a transmitter and a receiver can work in two different modes. One is peer-to-peer mode, while another is joining the wireless access point (AP) mode. In a self-contained environment, the peer-to-peer mode is sufficient with one serving as a router while many clients may connect to the router. But the “joining the wireless access point” mode is more common at work or at home for the smart devices.
- It is not difficult for two smart devices, one as a transmitter and another as a receiver, to build a peer-to-peer network. Once the peer-to-peer network is established, the transmitter is able to configure the receiver setting either through a browser or an app. But if another wireless access point exists on the wireless network that serves as a main router, it is beneficial to have both the transmitter and the receiver to join this AP network such that both can access to the resources on the network. The resources available include network storage, Internet capability, network printing, network audio and others.
- It becomes difficult and not obvious for the transmitter to configure the newly discovered receiver through the peer-to-peer network and to discover the available access point (AP) on the open wireless environment. It is a second objective to allow the transmitter to be able to discover the receiver and the wireless access point separately and be able to configure the receiver as well as to connect the receiver to the access point all through the transmitter user interface panel on the transmitter, without resorting to any external means like mouse, keyboard or a remote control stick.
- The realization of this objective is described in U.S. patent application Ser. No. 12/912,614, entitled “DUAL-MODE WIRELESS NETWORKED DEVICE INTERFACE AND AUTOMATIC CONFIGURATION THEREOF,” which is assigned to the assignee of the present application. An autoconfiguration function is provided which through a portable device can associate with one or more portable devices on a downstream wireless network and associate with one or more access points on an upstream wireless network.
- When it comes to content, it is divided into two types: playable content and object/link content. The playable content includes video, audio, photo, and document. The document includes Word, Excel, Power Point, PDF, Text and others. The object/link content includes: URL link, app links such as YouTube link, Facebook link and others. In traditional media player, the playable content, limited to video, audio and photo, is “pushed” from the
transmitter receiver 12. Thereceiver player utility 13 will decode the playable content and “play” or display on thereceiver display panel 11. The object/link is more appropriately referred to as app. - Due to IP and compatibility issues, the transmitter and receiver tend to work in pairs. It implies that the transmitter and the receiver have to run in the same domain of the OS (operating system), such as iOS, Android or Linux.
- It is a third objective to eliminate this limitation to allow the transmitter running under any kind of OS to be able to play any kind of content and app on the receiver in all operational modes.
- Video, audio and photo are the most obvious streaming applications between the transmitter and receiver. The applications are limited by the traditional receiver to only support these three content formats for streaming. It is a fourth objective to stream Word, text, power point, Excel, PDF and other documentation file formats in addition to the traditional video, audio, photo content.
- By “pushing” the content from the transmitter to the receiver, traditional media player has another deficiency in being a middle man when streaming the content from outside the transmitter to the receiver. It would have to “pull” the content from outside the transmitter and then “push” it to the receiver. In terms of efficiency, it wastes the transfer bandwidth and the resources of the transmitter during the streaming process. It is a fifth objective of this invention to implement “pull” mechanism in the receiver. In the above scenario, all the transmitter has to do is to pass the content location information to the receiver. In essence, it allows the receiver to build a reverse content explorer on the contents that the transmitter has the right to access to, including the content on the transmitter as well as those accessible devices residing on the same network with the receiver. Once selected, the content will be “pulled” by the receiver directly, without any resources dedicated from the transmitter.
- A sixth objective of the invention is to expand the interface between the transmitter and the receiver from wireless local area network (WLAN) to wide area network (WAN). The realization of this objective is described in patent application: Ser. No. entitled PRIVATE CLOUD SERVER AND CLIENT ARCHITECTURE WITHOUT UTILIZING A ROUTING SERVER Ser. No. 13/229,285. It allows accessing the receiver as a private cloud server from anywhere at anytime by the transmitter as a smart device client. The transmitter can also access the receiver as a private cloud server behind the firewall with fixed or dynamic IP addresses. The receiver as a smart device client requires no outside or third party routing server in the WAN; requires no additional router setup in the LAN to establish a secure peer-to-peer communication channel with the receiver as a private cloud server. The transmitter as a smart device client can access services or storage from the receiver as a private cloud server.
- As shown in
FIG. 2 , atransmitter FIG. 9 , while areceiver 22 has a corresponding main control utility as shown inFIG. 10 . In the transmitter main control utility, thetransmitter receiver 90. Once thereceiver 22 is discovered, it checks if the native mirror mode is activated 91. If so, thetransmitter receiver 910. It starts interacting with thereceiver 911. As shown inFIG. 2 , thetransmitter native mirror mode display panel receiver display panel 21, with theuniversal environment extender 22 as the receiver running undernative mirror mode 23. Thedisplay content 21 is displayed and mirrored on thereceiver display device 20. Thereceiver 22 has access to thestorage 220 as well asapplications 221. The storage includes the local storage and the network accessible storage on the network. - As shown in
FIG. 3 andFIG. 9 , if the native mirror mode is not activated, thetransmitter virtual UI driver 92 which allows it to communicate with thereceiver 22. Thetransmitter 24 will have icons displayed on itsreverse mirror panel 36, and runs inreverse mirror mode 920 to control thereceiver 22. In reverse mirror mode, finger can move freely indisplay panel area 35. In thepanel 35, several icons are displayed and can be selected, including “volume down” 391, “volume up” 391, “back to last” 387, “home” 388, “soft keyboard” 389, “reverse mirror/toggle” 381, “choose display” 382 and “shut down” 390. Thetransmitter 27 will have icons displayed on itsremote control panel 38, and runs inremote control mode 921 to control thereceiver 22. In remote control mode,finger mouse 383 can move freely inmouse pad area 380. Inremote control panel 38, several icons are displayed and can be selected, including “search left” 384, “play/pause” 385, “search right” 389, “volume down” 391, “volume up” 391, “back to last” 387, “home” 388, “soft keyboard” 389, “reverse mirror/toggle” 381, “choose display” 382 and “shut down” 390. - The
transmitter 24 inreverse mirror mode 36 can toggle toremote control mode 39 intransmitter 27 at any time, if necessary 921, by selecting “reverse mirror/toggle”icon 381. The unit can safely be shut down by selecting icon “shut down” 390. The icons “search left” 384, “play/pause” 385, “search right” 386, once selected, are used mainly by video, audio and photos content explorer and player. The icons “volume down” 391 and “volume up” 392 are for speaker volume control on thereceiver 22, once selected. The icon “back to last” 387 is selected to go back to last action. The icon “choose display” 382 is selected to choose whichreceiver 22 to actively play the content in thestorage 220 from thetransmitter local display discoverable receiver 22 on the network. - Once the reverse mirror/
toggle icon 381 is selected in theremote control mode 39, thetransmitter 27 is toggled to reversemirror mode 36. Itsdisplay panel 35 will mirror whatever is displayed on thereceiver display panel 31. Some of the icons from thetransmitter 27remote control panel 38 are also shown on the bottom row of thereceiver display panel 31 for control in thereverse mirror mode 36. Examples are: “volume down” 391, “volume up” 391, “back to last” 387, “home” 388, “soft keyboard” 389, “reverse mirror/toggle” 381, “choose display” 382 and “shut down” 390. While inreverse mirror mode 36, thetransmitter 24 can also toggle toremote control mode 39 at any time, if necessary, by selecting “reverse mirror/toggle”icon 381. - The “choose display”
icon 382 and “reverse mirror/toggle”icon 381 combine to designate which display is active during the operation. The “choose display”icon 382 selects the main active display panel. When “choose display”icon 382 selects the transmitter, only the transmitter display panel is selected as the main active display. When “choose display”icon 382 selects the receiver while “reverse mirror/toggle”icon 381 is on, the receiver display panel is the main active display while the transmitter display panel is also selected. When “choose display”icon 382 selects the receiver while “reverse mirror/toggle”icon 381 is off, only the receiver display panel is selected. - As shown in
FIG. 10 , thereceiver 22 starts its main control utility. It first listens to thetransmitter request 100 and tries to be discovered by thetransmitter 101. Once it is discovered, it checks if the native mirror mode has been activated 102? If so, it starts processing thenative mirror mode 1020. - If the native mirror mode is not activated, the
receiver 22 installs thevirtual UI driver 93. It displays themain menu transmitter transmitter input 105. - As shown in
FIG. 3 , one of themain control icons 32 is selected on thereceiver display panel 31. Thesame icon 34 is selected on thetransmitter display panel 35 that is running underreverse mirror mode 36. If thetransmitter 27 is running underremote control mode 39, the control icons can be selected on thetransmitter display panel 38. The corresponding controls icons are also shown on the bottom row of thereceiver display panel 31. - As shown in
FIG. 3 andFIG. 9 , thetransmitter main menu transmitter receiver 940. It can toggle between thereverse mirror mode 36 andremote control mode 39, by selecting the reverse mirror/toggle icon 381 in eithermode 941. Thetransmitter receiver 942. Thereceiver 22 starts processing therequest 1070. The interaction between thetransmitter receiver 22 is shown as inFIG. 4 . - As shown in
FIG. 4 , more setting icons are available, including wireless setting 42, display setting 440, system setting 47 and advanced setting 48. The wireless setting 42 is for receiver wireless attributes including ID, password, and channel in addition to the available wireless access point (AP) information. The correspondingreceiver setting utility 43 andtransmitter setting utility transmitter receiver 22 to join another wireless network access point (AP) while on a peer-to-peer wireless network with thereceiver 22. - The display setting 440 is for
receiver 22 andreceiver display device 20. The system setting 47 is for the system setting ofreceiver 22, including system information and firmware upgrade. The advanced setting 48 covers the rest of the setting aspects of thereceiver 22 that is more for expert mode setting. - As shown in
FIG. 4 , one of the settingicons 42 is selected on thereceiver display panel 41. Thesame icon 44 is selected on thetransmitter display panel 45 that is running underreverse mirror mode 46. If thetransmitter 27 is running underremote control mode 49, the control icons can be selected on thetransmitter display panel 38. The corresponding controls icons are also shown on the bottom row of thereceiver display panel 41. - If the app mode is selected, as shown in
FIG. 5 , thetransmitter receiver 96. It can toggle between thereverse mirror mode 36 andremote control mode 39, by selecting the reverse mirror/toggle icon 381 in eithermode 961. Thetransmitter receiver 962. On the receiver side, it starts the receiver app explorer/player utility FIG. 5 andFIG. 10 . - As shown in
FIG. 5 ,FIG. 6 &FIG. 11 , thereceiver 22 starts the app explorer/player utility 53 andapp explorer 111. It waits for thetransmitter selection 113 to determine if the app fromapplications 221 is playable on thereceiver 114? If it is playable, the app fromapplications 221 is run on thereceiver 116. If it is not playable, thereceiver 22 will locate outside resources to play the app, if necessary 115. - As shown in
FIG. 5 , one of theapp icons 52 fromapplications 221 is selected on thereceiver display panel 51. Thesame icon 54 is selected on thetransmitter display panel 55 that is running underreverse mirror mode 56. If thetransmitter 27 is running underremote control mode 59, the control icons can be selected on thetransmitter display panel 38. The corresponding controls icons are also shown on the bottom row of thereceiver display panel 51. - As shown in
FIG. 5 &FIG. 6 , the selectedapp 52 fromapplications 221, is played on thereceiver display panel 61 by the receiver app explorer/player utility 53. The selectedapp 52 can be reverse mirrored 66 back to thetransmitter 24 on itspanel 65. In eitherremote control mode 69, or reversemirror mode 66, most control icons are available ontransmitter display panel receiver display panel 61. - As shown in
FIG. 9 , if the explorer mode is selected, thetransmitter receiver 950. It can toggle between thereverse mirror mode 36 andremote control mode 39, by selecting the reverse mirror/toggle icon 381 in eithermode 951. Thetransmitter local content 9510 in thestorage 220 and sends the content location to thereceiver 9511 and starts interacting with thereceiver 962. On the receiver side, as shown inFIG. 10 , if the explorer mode is selected, it startsreceiver explorer utility 1080. - As shown in
FIG. 7 , one of thecontent type icons 72 is selected on thereceiver display panel 71. Thesame icon 74 is selected on thetransmitter display panel 75 that is running underreverse mirror mode 76. If thetransmitter 27 is running underremote control mode 79, the control icons can be selected on thetransmitter display panel 38. The corresponding controls icons are also shown on the bottom row of thereceiver display panel 71. - As shown in
FIG. 7 andFIG. 8 , the selectedcontent type 72, will further be explored (not shown) by the receiver content explorer/player utility 73. The selected content in thestorage 220 is eventually played on thereceiver display panel 81 by the receiver app explorer/player utility 73. The played content in thestorage 220 can be reverse mirrored 86 back to thetransmitter 24 on itsdisplay panel 85. In eitherremote control mode 89, or reversemirror mode 86, most control icons are available ontransmitter display panel receiver display panel 81. - The “choose display”
icon 382 and “reverse mirror/toggle”icon 381 combine to designate which display is active during the operation. When “choose display”icon 382 selects the transmitter, only the transmitter display panel is selected. When “choose display”icon 382 selects the receiver while “reverse mirror/toggle”icon 381 is on, the receiver display panel as well as the transmitter display panel are selected. When “choose display”icon 382 selects the receiver while “reverse mirror/toggle”icon 381 is off, only the receiver display panel is selected. - As shown in
FIG. 7 andFIG. 12 , the receiver content explorer/player utility 73 accepts content location info from thetransmitter 120. It then starts reverse content explorer onto thetransmitter 121. It will also explore the content in thestorage 220 from thereceiver 22 and itsaccessible network domain 1210. It waits for the selection from thetransmitter 122. The content in thestorage 220 is “pulled” from the source and streamed to thereceiver 123. Thereceiver 22 will decrypt the content in thestorage 220, if necessary 126. It checks if the content is playable on thereceiver 128? If so, the content is played 1280. If not, thereceiver 22 will locate outside resources to play the content, if necessary 129. - As shown in
FIG. 13 , theprivate cloud server 1308 andclient smart device client corresponding display panel same LAN 1304 and connect directly wired or wireless, as withtransmitter 1306 andreceiver transmitter 1301 can also connect to areceiver 1308 behind aRouter_P 1302 andLAN 1304 through the WAN andpublic cloud 1300. Thetransmitter Router_S 1303 andLAN 1305 can connect to areceiver 1308 behind aRouter_P 1302 andLAN 1304 through the WAN andpublic cloud 1300. - While the transmitter and the receiver are on the
same LAN 1304, the “choose display” and “reverse mirror/toggle” icons combine to select the active display in one of three combinations of transmitter display panel only, receiver display panel only or both display panels. - While the
transmitter receiver 1308 are connected through thepublic cloud 1300, it is beneficial to set the “choose display” option to transmitter display panel such that the played content video, audio, photos or documents can be streamed directly from the source to the transmitter display panel in the cloud access application. Another alternative is to set the “choose display” to the receiver display panel and the “reverse mirror/toggle” to ON such that the receiver display panel is chosen and the transmitter display panel is also turned on to mirror the receiver display panel. This alternative setting is best for streaming the receiver accessible applications from the cloud to the transmitter display panel. -
FIG. 13 shows a number of scenarios of the universal environment extender systems. One scenario is betweentransmitter 1309 and thereceiver 1321 onLAN 1305. The source of the content/applications can be from any member on theLAN 1305. The target display panels can be either on thetransmitter display panel 1339 itself or on thereceiver display panel 1331 or on both transmitter and receiver display panels. Another scenario is betweentransmitter 1310 and thereceiver 1308 across thecloud 1300 through theLAN LAN 1305 andLAN 1304. The target display panels can be either on thetransmitter display panel 1330 itself or on thereceiver display panel cloud 1300 onLAN 1304, or on thereceiver panel 1331 on thesame LAN 1305, or on both transmitter and receiver display panels. - In the first scenario on LAN, the target display panel can be either the
transmitter display panel 1339 itself, or thereceiver display panel 1331. Thereceiver display panel 1331 can be a HDTV, smart watch, smart glasses, smart conference room projector, smart settop box, smart car entertainment system, medical console, or smart gadget that is physically viewable from where thetransmitter 1309 is located. - In the second scenario across the WAN or
cloud 1300, thetransmitter 1310 can view the source content/applications stored on any one member on theLAN 1304 routing through the private cloud server/extender/receiver 1308. The source content/applications can then be viewable either on thetransmitter display panel 1330 itself, or on thereceiver display panel 1331 on the same LAN. Thereceiver display panel 1331 can be a HDTV, smart watch, smart glasses, smart conference room projector, smart settop box, smart car entertainment system, medical console, or smart gadget that is physically viewable from where thetransmitter 1309 is located. - The universal environment extender can apply to HDTV, smart watch, smart glasses, smart conference room projector, smart settop box, smart car entertainment system, medical console, or smart gadgets.
- Although the present invention has been described in accordance with the embodiments shown, one of ordinary skill in the art will readily recognize that there could be variations to the embodiments and those variations would be within the spirit and scope of the present invention. Accordingly, many modifications may be made by one of ordinary skill in the art without departing from the spirit and scope of the present invention.
Claims (20)
Priority Applications (5)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US13/909,889 US10021180B2 (en) | 2013-06-04 | 2013-06-04 | Universal environment extender |
GB1521425.7A GB2534672B (en) | 2013-06-04 | 2013-07-09 | Universal environment extender system and transmitter for a UEE system |
GB1312300.5A GB2514856B (en) | 2013-06-04 | 2013-07-09 | Universal Environment Extender |
TW102125517A TWI538499B (en) | 2013-06-04 | 2013-07-17 | Universal environment extender, transmitter and universal environment extender sysetm thereof |
CN201310410701.8A CN104219791B (en) | 2013-06-04 | 2013-09-10 | general environment expander |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US13/909,889 US10021180B2 (en) | 2013-06-04 | 2013-06-04 | Universal environment extender |
Publications (2)
Publication Number | Publication Date |
---|---|
US20140359477A1 true US20140359477A1 (en) | 2014-12-04 |
US10021180B2 US10021180B2 (en) | 2018-07-10 |
Family
ID=49033550
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/909,889 Active 2035-07-09 US10021180B2 (en) | 2013-06-04 | 2013-06-04 | Universal environment extender |
Country Status (4)
Country | Link |
---|---|
US (1) | US10021180B2 (en) |
CN (1) | CN104219791B (en) |
GB (2) | GB2514856B (en) |
TW (1) | TWI538499B (en) |
Cited By (43)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20140282103A1 (en) * | 2013-03-16 | 2014-09-18 | Jerry Alan Crandall | Data sharing |
US20140372558A1 (en) * | 2013-06-17 | 2014-12-18 | Thomson Licensing | Wifi display compatible network gateway |
US20150042633A1 (en) * | 2013-08-09 | 2015-02-12 | Lenovo (Beijing) Limited | Display method and electronic device |
US20150168935A1 (en) * | 2013-12-17 | 2015-06-18 | Inwellcom Technology Co., Ltd. | Remotely managing and controlling system and method |
US20150229997A1 (en) * | 2014-02-07 | 2015-08-13 | Samsung Electronics Co., Ltd. | User terminal and control method thereof |
US20150293741A1 (en) * | 2014-04-10 | 2015-10-15 | Screenovate Technologies Ltd. | Method for real-time multimedia interface management |
US20150339026A1 (en) * | 2014-05-22 | 2015-11-26 | Samsung Electronics Co., Ltd. | User terminal device, method for controlling user terminal device, and multimedia system thereof |
US20150347188A1 (en) * | 2014-05-30 | 2015-12-03 | Appsfreedom, Inc. | Delegated business process management for the internet of things |
US20160127851A1 (en) * | 2014-11-05 | 2016-05-05 | Samsung Electronics Co., Ltd. | Method for device to control another device and the device |
US9622278B2 (en) | 2010-10-26 | 2017-04-11 | Kingston Digital Inc. | Dual-mode wireless networked device interface and automatic configuration thereof |
US20170244779A1 (en) * | 2015-12-03 | 2017-08-24 | Google Inc. | Methods, systems and media for presenting a virtual operating system on a display device |
US9781087B2 (en) | 2011-09-09 | 2017-10-03 | Kingston Digital, Inc. | Private and secure communication architecture without utilizing a public cloud based routing server |
US9935930B2 (en) | 2011-09-09 | 2018-04-03 | Kingston Digital, Inc. | Private and secure communication architecture without utilizing a public cloud based routing server |
US10021180B2 (en) * | 2013-06-04 | 2018-07-10 | Kingston Digital, Inc. | Universal environment extender |
US10237253B2 (en) | 2011-09-09 | 2019-03-19 | Kingston Digital, Inc. | Private cloud routing server, private network service and smart device client architecture without utilizing a public cloud based routing server |
US20190155495A1 (en) * | 2017-11-22 | 2019-05-23 | Microsoft Technology Licensing, Llc | Dynamic device interaction adaptation based on user engagement |
USD858554S1 (en) * | 2018-04-25 | 2019-09-03 | Allstate Insurance Company | Display screen with graphical user interface |
US10601810B2 (en) | 2011-09-09 | 2020-03-24 | Kingston Digital, Inc. | Private cloud routing server connection mechanism for use in a private communication architecture |
US10650621B1 (en) | 2016-09-13 | 2020-05-12 | Iocurrents, Inc. | Interfacing with a vehicular controller area network |
US10788797B1 (en) | 2019-05-06 | 2020-09-29 | Apple Inc. | Clock faces for an electronic device |
US10802703B2 (en) | 2015-03-08 | 2020-10-13 | Apple Inc. | Sharing user-configurable graphical constructs |
US10838586B2 (en) | 2017-05-12 | 2020-11-17 | Apple Inc. | Context-specific user interfaces |
US10852905B1 (en) | 2019-09-09 | 2020-12-01 | Apple Inc. | Techniques for managing display usage |
US10931806B2 (en) * | 2014-12-09 | 2021-02-23 | Inwellcom Technology Co., Ltd. | Remotely managing and controlling system and method |
US11042281B2 (en) | 2014-08-15 | 2021-06-22 | Apple Inc. | Weather user interface |
US11061372B1 (en) | 2020-05-11 | 2021-07-13 | Apple Inc. | User interfaces related to time |
US11150798B2 (en) | 2016-03-28 | 2021-10-19 | Apple Inc. | Multifunction device control of another electronic device |
US11257464B2 (en) | 2017-05-16 | 2022-02-22 | Apple Inc. | User interface for a flashlight mode on an electronic device |
US20220067163A1 (en) * | 2020-08-28 | 2022-03-03 | Coretronic Corporation | Electronic-device control system and electronic-device control method |
US11301130B2 (en) | 2019-05-06 | 2022-04-12 | Apple Inc. | Restricted operation of an electronic device |
US11307757B2 (en) | 2016-09-23 | 2022-04-19 | Apple Inc. | Watch theater mode |
US11327650B2 (en) | 2018-05-07 | 2022-05-10 | Apple Inc. | User interfaces having a collection of complications |
US11372659B2 (en) | 2020-05-11 | 2022-06-28 | Apple Inc. | User interfaces for managing user interface sharing |
US20220214848A1 (en) * | 2019-04-18 | 2022-07-07 | Samsung Electronics Co., Ltd. | Display device, user terminal device, method for controlling display device, and method for controlling user terminal device |
US11526256B2 (en) | 2020-05-11 | 2022-12-13 | Apple Inc. | User interfaces for managing user interface sharing |
US11580867B2 (en) | 2015-08-20 | 2023-02-14 | Apple Inc. | Exercised-based watch face and complications |
US11683292B2 (en) | 2011-09-09 | 2023-06-20 | Kingston Digital, Inc. | Private cloud routing server connection mechanism for use in a private communication architecture |
US11694590B2 (en) | 2020-12-21 | 2023-07-04 | Apple Inc. | Dynamic user interface with time indicator |
US11720239B2 (en) | 2021-01-07 | 2023-08-08 | Apple Inc. | Techniques for user interfaces related to an event |
US11740776B2 (en) | 2012-05-09 | 2023-08-29 | Apple Inc. | Context-specific user interfaces |
US11863529B2 (en) | 2011-09-09 | 2024-01-02 | Kingston Digital, Inc. | Private cloud routing server connection mechanism for use in a private communication architecture |
US11921992B2 (en) | 2021-05-14 | 2024-03-05 | Apple Inc. | User interfaces related to time |
US11955100B2 (en) | 2022-02-18 | 2024-04-09 | Apple Inc. | User interface for a flashlight mode on an electronic device |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR102598082B1 (en) * | 2016-10-28 | 2023-11-03 | 삼성전자주식회사 | Image display apparatus, mobile device and operating method for the same |
CN108196826A (en) * | 2017-11-27 | 2018-06-22 | 中科开元信息技术(北京)有限公司 | A kind of realization method and system of real time remote control communication system framework |
KR20200084587A (en) * | 2019-01-03 | 2020-07-13 | 삼성전자주식회사 | Display apparatus and controlling method thereof |
Citations (14)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6563515B1 (en) * | 1998-05-19 | 2003-05-13 | United Video Properties, Inc. | Program guide system with video window browsing |
US20050041736A1 (en) * | 2003-05-07 | 2005-02-24 | Bernie Butler-Smith | Stereoscopic television signal processing method, transmission system and viewer enhancements |
US7120429B2 (en) * | 2001-08-13 | 2006-10-10 | Qualcomm Inc. | System and method for licensing applications on wireless devices over a wireless network |
US20060271968A1 (en) * | 2005-05-31 | 2006-11-30 | Zellner Samuel N | Remote control |
US20100188987A1 (en) * | 2009-01-28 | 2010-07-29 | Agere Systems, Inc. | Power learning security in wireless routers |
US20120030584A1 (en) * | 2010-07-30 | 2012-02-02 | Brian Bian | Method and apparatus for dynamically switching between scalable graphical user interfaces for mobile devices |
US20120042275A1 (en) * | 2010-08-10 | 2012-02-16 | Microsoft Corporation | Cloning specific windows on a wireless display surface |
US8170209B2 (en) * | 2002-12-23 | 2012-05-01 | Koninklijke Philips Electronics N.V. | Extensible disc player |
US20120164961A1 (en) * | 2010-12-25 | 2012-06-28 | Jie Gao | Wireless Display Performance Enhancement |
US20130150161A1 (en) * | 2011-12-13 | 2013-06-13 | Empire Technology Development, Llc | Graphics render matching for displays |
US20130177891A1 (en) * | 2011-07-02 | 2013-07-11 | Joachim Hammerschmidt | Audio-visual learning system |
US20140141721A1 (en) * | 2012-11-21 | 2014-05-22 | Lg Electronics Inc. | Method for controlling system including electronic tag, mobile device, and display device, and mobile device and display device of the same |
US20140306865A1 (en) * | 2013-04-12 | 2014-10-16 | Shanghai Powermo Information Tech. Co. Ltd. | Method for communicating with a plurality of first display devices and at least one second display device, and electronic apparatus using the same method |
US20150327313A1 (en) * | 2012-12-09 | 2015-11-12 | Lg Electronics Inc. | Method and device for session initialization in wireless communication system |
Family Cites Families (56)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5408618A (en) | 1992-07-31 | 1995-04-18 | International Business Machines Corporation | Automatic configuration mechanism |
GB2341523B (en) | 1998-09-12 | 2003-10-29 | Ibm | Apparatus and method for establishing communication in a computer network |
US6892230B1 (en) | 1999-06-11 | 2005-05-10 | Microsoft Corporation | Dynamic self-configuration for ad hoc peer networking using mark-up language formated description messages |
US6725281B1 (en) | 1999-06-11 | 2004-04-20 | Microsoft Corporation | Synchronization of controlled device state using state table and eventing in data-driven remote device control model |
US6438594B1 (en) | 1999-08-31 | 2002-08-20 | Accenture Llp | Delivering service to a client via a locally addressable interface |
US7068680B1 (en) | 1999-10-01 | 2006-06-27 | Accenture Llp | Communication service architectures for netcentric computing systems |
US7020697B1 (en) | 1999-10-01 | 2006-03-28 | Accenture Llp | Architectures for netcentric computing systems |
US6981041B2 (en) | 2000-04-13 | 2005-12-27 | Aep Networks, Inc. | Apparatus and accompanying methods for providing, through a centralized server site, an integrated virtual office environment, remotely accessible via a network-connected web browser, with remote network monitoring and management capabilities |
WO2002015491A1 (en) | 2000-08-17 | 2002-02-21 | Advanced Network Technology Laboratories Pte Ltd. | Reconfigurable computer networks |
AU2001223622A1 (en) | 2000-12-04 | 2002-06-18 | Nokia Corporation | Communication system and method for establishing a connection to a serving network element |
US6954790B2 (en) | 2000-12-05 | 2005-10-11 | Interactive People Unplugged Ab | Network-based mobile workgroup system |
US7194263B2 (en) | 2001-09-17 | 2007-03-20 | Microsoft Corporation | System and method for concurrent operation of a wireless device in two disjoint wireless networks |
US7139818B1 (en) | 2001-10-04 | 2006-11-21 | Cisco Technology, Inc. | Techniques for dynamic host configuration without direct communications between client and server |
US7219140B2 (en) | 2001-12-05 | 2007-05-15 | Dennis Craig Marl | Configuration and management systems for mobile and embedded devices |
US7474327B2 (en) | 2002-02-12 | 2009-01-06 | Given Imaging Ltd. | System and method for displaying an image stream |
US6978314B2 (en) | 2002-02-26 | 2005-12-20 | Xerox Corporation | System and method for locating devices on a local area network |
US7408882B2 (en) | 2002-06-24 | 2008-08-05 | Paradyne Corporation | Automatic discovery of network node addresses |
US7562393B2 (en) | 2002-10-21 | 2009-07-14 | Alcatel-Lucent Usa Inc. | Mobility access gateway |
EP1586214B1 (en) | 2003-01-16 | 2008-12-03 | Research In Motion Limited | System and method of exchanging identification information for mobile stations |
US7328256B2 (en) | 2003-06-02 | 2008-02-05 | Apple Inc. | Method and apparatus for distributing computer files across a network to multiple clients |
US7453852B2 (en) | 2003-07-14 | 2008-11-18 | Lucent Technologies Inc. | Method and system for mobility across heterogeneous address spaces |
FR2861934B1 (en) | 2003-10-30 | 2006-01-27 | Wavecom | METHOD AND DEVICE FOR ACCESSING A MOBILE SERVER TERMINAL OF A FIRST COMMUNICATION NETWORK USING A CLIENT TERMINAL OF ANOTHER COMMUNICATION NETWORK. |
US8590032B2 (en) | 2003-12-10 | 2013-11-19 | Aventail Llc | Rule-based routing to resources through a network |
US7640546B2 (en) | 2004-01-16 | 2009-12-29 | Barclays Capital Inc. | Method and system for identifying active devices on network |
US7231632B2 (en) | 2004-04-16 | 2007-06-12 | Apple Computer, Inc. | System for reducing the number of programs necessary to render an image |
US7881267B2 (en) | 2004-06-04 | 2011-02-01 | Hewlett-Packard Development Company, L.P. | Portable computing device for wireless communications and method of operation |
EP1771979B1 (en) | 2004-07-23 | 2011-11-23 | Citrix Systems, Inc. | A method and systems for securing remote access to private networks |
US7810148B2 (en) | 2005-02-25 | 2010-10-05 | Microsoft Corporation | Enabling terminal services through a firewall |
US20080201751A1 (en) | 2006-04-18 | 2008-08-21 | Sherjil Ahmed | Wireless Media Transmission Systems and Methods |
US7676690B2 (en) | 2006-05-30 | 2010-03-09 | Dell Products L.P. | Mirroring of activity between electronic devices |
US8589489B2 (en) | 2006-06-19 | 2013-11-19 | Bomgar | Method and system for providing secure remote access and control |
US8860752B2 (en) | 2006-07-13 | 2014-10-14 | Apple Inc. | Multimedia scripting |
US7761087B2 (en) | 2006-07-21 | 2010-07-20 | Kyocera Corporation | Apparatus, system and method for providing services through a multi-mode wireless terminal device |
US7647203B1 (en) | 2006-08-24 | 2010-01-12 | Nvidia Corporation | Multifaceted video benchmarking system, method, and computer program product |
US8141143B2 (en) | 2007-05-31 | 2012-03-20 | Imera Systems, Inc. | Method and system for providing remote access to resources in a secure data center over a network |
US20090019492A1 (en) | 2007-07-11 | 2009-01-15 | United Video Properties, Inc. | Systems and methods for mirroring and transcoding media content |
TW200920030A (en) | 2007-10-18 | 2009-05-01 | D Link Corp | The method to puncture the firewall for building the linking channel between the network terminal devices |
US20090129301A1 (en) | 2007-11-15 | 2009-05-21 | Nokia Corporation And Recordation | Configuring a user device to remotely access a private network |
US20090303973A1 (en) | 2008-06-10 | 2009-12-10 | Nokia Siemens Networks Oy | Packet data network selection |
US7636764B1 (en) | 2008-09-29 | 2009-12-22 | Gene Fein | Cloud resource usage in data forwarding storage |
US8300056B2 (en) | 2008-10-13 | 2012-10-30 | Apple Inc. | Seamless display migration |
US8069217B2 (en) | 2009-04-16 | 2011-11-29 | Dell Products L.P. | System and method for providing access to a shared system image |
US8412798B1 (en) | 2009-10-03 | 2013-04-02 | Frank C. Wang | Content delivery system and method |
US10264029B2 (en) | 2009-10-30 | 2019-04-16 | Time Warner Cable Enterprises Llc | Methods and apparatus for packetized content delivery over a content delivery network |
US8650565B2 (en) | 2009-12-14 | 2014-02-11 | Citrix Systems, Inc. | Servicing interrupts generated responsive to actuation of hardware, via dynamic incorporation of ACPI functionality into virtual firmware |
GB201003962D0 (en) | 2010-03-10 | 2010-04-21 | Tangentix Ltd | Multimedia content delivery system |
US8725880B2 (en) | 2010-04-07 | 2014-05-13 | Apple, Inc. | Establishing online communication sessions between client computing devices |
US8583811B2 (en) | 2010-04-23 | 2013-11-12 | Qualcomm Incorporated | Gateway device for multimedia content |
KR20120015931A (en) | 2010-08-13 | 2012-02-22 | 삼성전자주식회사 | Portable electric device and display mirorring method therof |
US9466127B2 (en) | 2010-09-30 | 2016-10-11 | Apple Inc. | Image alteration techniques |
WO2012044557A2 (en) | 2010-10-01 | 2012-04-05 | Imerj, Llc | Auto-configuration of a docked system in a multi-os environment |
US8958559B2 (en) | 2011-06-03 | 2015-02-17 | Apple Inc. | System and method for secure instant messaging |
US20120307141A1 (en) | 2011-06-06 | 2012-12-06 | Apple Inc. | Frame retiming for mirror mode |
US9203807B2 (en) | 2011-09-09 | 2015-12-01 | Kingston Digital, Inc. | Private cloud server and client architecture without utilizing a routing server |
US9967388B2 (en) * | 2012-02-21 | 2018-05-08 | Qualcomm Incorporated | Mirrored interface navigation of multiple user interfaces |
US10021180B2 (en) * | 2013-06-04 | 2018-07-10 | Kingston Digital, Inc. | Universal environment extender |
-
2013
- 2013-06-04 US US13/909,889 patent/US10021180B2/en active Active
- 2013-07-09 GB GB1312300.5A patent/GB2514856B/en active Active
- 2013-07-09 GB GB1521425.7A patent/GB2534672B/en active Active
- 2013-07-17 TW TW102125517A patent/TWI538499B/en active
- 2013-09-10 CN CN201310410701.8A patent/CN104219791B/en active Active
Patent Citations (14)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6563515B1 (en) * | 1998-05-19 | 2003-05-13 | United Video Properties, Inc. | Program guide system with video window browsing |
US7120429B2 (en) * | 2001-08-13 | 2006-10-10 | Qualcomm Inc. | System and method for licensing applications on wireless devices over a wireless network |
US8170209B2 (en) * | 2002-12-23 | 2012-05-01 | Koninklijke Philips Electronics N.V. | Extensible disc player |
US20050041736A1 (en) * | 2003-05-07 | 2005-02-24 | Bernie Butler-Smith | Stereoscopic television signal processing method, transmission system and viewer enhancements |
US20060271968A1 (en) * | 2005-05-31 | 2006-11-30 | Zellner Samuel N | Remote control |
US20100188987A1 (en) * | 2009-01-28 | 2010-07-29 | Agere Systems, Inc. | Power learning security in wireless routers |
US20120030584A1 (en) * | 2010-07-30 | 2012-02-02 | Brian Bian | Method and apparatus for dynamically switching between scalable graphical user interfaces for mobile devices |
US20120042275A1 (en) * | 2010-08-10 | 2012-02-16 | Microsoft Corporation | Cloning specific windows on a wireless display surface |
US20120164961A1 (en) * | 2010-12-25 | 2012-06-28 | Jie Gao | Wireless Display Performance Enhancement |
US20130177891A1 (en) * | 2011-07-02 | 2013-07-11 | Joachim Hammerschmidt | Audio-visual learning system |
US20130150161A1 (en) * | 2011-12-13 | 2013-06-13 | Empire Technology Development, Llc | Graphics render matching for displays |
US20140141721A1 (en) * | 2012-11-21 | 2014-05-22 | Lg Electronics Inc. | Method for controlling system including electronic tag, mobile device, and display device, and mobile device and display device of the same |
US20150327313A1 (en) * | 2012-12-09 | 2015-11-12 | Lg Electronics Inc. | Method and device for session initialization in wireless communication system |
US20140306865A1 (en) * | 2013-04-12 | 2014-10-16 | Shanghai Powermo Information Tech. Co. Ltd. | Method for communicating with a plurality of first display devices and at least one second display device, and electronic apparatus using the same method |
Non-Patent Citations (13)
Title |
---|
"Panasonic App Viera 2.0" an online document published on Mar 21, 2012 at http://imultiscreen.com/panasonic-app-viera-2-0-feb-2012/ which is a transcript of https://www.youtube.com/watch?v=j9kgEtNMTZ4 * |
"Panasonic VIERA Connect Web Platform (2012)" online article by Adrienne Maxwell published on July 31, 2012 at https://hometheaterreview.com/panasonic-viera-connect-web-platform-2012/ * |
"Viera Remote App Basic- screenshots", includes screenshots of https://www.youtube.com/watch?v=QZ2hCyOhZmc "VIERA remote App Version 2.0 - Basic Operation" published on May 30, 2012 * |
filename "2012 Sep WiFi-Miracast brochure.pdf" Miracast-paper: "Wi-Fi CERTIFIED Miracastâ¢: Extending the Wi-Fi experience to seamless video display" published by Wi-Fi Alliance on Sep 19, 2012 * |
filename "2012 Sep WiFi-Miracast brochure.pdf" Miracast-paper: "Wi-Fi CERTIFIED Miracast™: Extending the Wi-Fi experience to seamless video display" published by Wi-Fi Alliance on Sep 19, 2012 * |
filename: "2011-July Samsung Remote App.pdf" "Samsung App Profile: Remote App" published published online on July 30, 2011 * |
filename: "2012-Jul samsung-tv-remote-app" "Samsung TV Remote app not working for some 2012" online article published on July 18, 2012 * |
filename: "2013-Mar panasonic-my-home-screen-smart-tv.pdf" "Panasonic My Home Screen Smart TV Interface" published online on Mar 6, 2013 * |
filename: "WiFi Miracast Demo video - Screenshots and Transcript.pdf" Screenshots and Transcripts of "Wi-Fi CERTIFIED Miracast⢠Demo at 2013 CES" video https://www.youtube.com/watch?v=cYagdOp9y7E published by Wi-Fi Alliance on Jan 9, 2013 * |
filename: "WiFi Miracast Demo video - Screenshots and Transcript.pdf" Screenshots and Transcripts of "Wi-Fi CERTIFIED Miracast™ Demo at 2013 CES" video https://www.youtube.com/watch?v=cYagdOp9y7E published by Wi-Fi Alliance on Jan 9, 2013 * |
Google Search - panasonic viera tv remote app * |
Google Search - viera remote app 2.0 * |
Google Search - viera tv remote app * |
Cited By (76)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9622278B2 (en) | 2010-10-26 | 2017-04-11 | Kingston Digital Inc. | Dual-mode wireless networked device interface and automatic configuration thereof |
US10601810B2 (en) | 2011-09-09 | 2020-03-24 | Kingston Digital, Inc. | Private cloud routing server connection mechanism for use in a private communication architecture |
US9781087B2 (en) | 2011-09-09 | 2017-10-03 | Kingston Digital, Inc. | Private and secure communication architecture without utilizing a public cloud based routing server |
US11863529B2 (en) | 2011-09-09 | 2024-01-02 | Kingston Digital, Inc. | Private cloud routing server connection mechanism for use in a private communication architecture |
US11356417B2 (en) | 2011-09-09 | 2022-06-07 | Kingston Digital, Inc. | Private cloud routing server connection mechanism for use in a private communication architecture |
US11683292B2 (en) | 2011-09-09 | 2023-06-20 | Kingston Digital, Inc. | Private cloud routing server connection mechanism for use in a private communication architecture |
US10237253B2 (en) | 2011-09-09 | 2019-03-19 | Kingston Digital, Inc. | Private cloud routing server, private network service and smart device client architecture without utilizing a public cloud based routing server |
US9935930B2 (en) | 2011-09-09 | 2018-04-03 | Kingston Digital, Inc. | Private and secure communication architecture without utilizing a public cloud based routing server |
US11740776B2 (en) | 2012-05-09 | 2023-08-29 | Apple Inc. | Context-specific user interfaces |
US20160110074A1 (en) * | 2013-03-16 | 2016-04-21 | Jerry Alan Crandall | Data Sharing |
US20160110075A1 (en) * | 2013-03-16 | 2016-04-21 | Jerry Alan Crandall | Data Sharing |
US20160110073A1 (en) * | 2013-03-16 | 2016-04-21 | Jerry Alan Crandall | Data Sharing |
US20160110153A1 (en) * | 2013-03-16 | 2016-04-21 | Jerry Alan Crandall | Data Sharing |
US20160110072A1 (en) * | 2013-03-16 | 2016-04-21 | Jerry Alan Crandall | Data Sharing |
US9563341B2 (en) * | 2013-03-16 | 2017-02-07 | Jerry Alan Crandall | Data sharing |
US20140282103A1 (en) * | 2013-03-16 | 2014-09-18 | Jerry Alan Crandall | Data sharing |
US9645720B2 (en) * | 2013-03-16 | 2017-05-09 | Jerry Alan Crandall | Data sharing |
US10021180B2 (en) * | 2013-06-04 | 2018-07-10 | Kingston Digital, Inc. | Universal environment extender |
US10187925B2 (en) * | 2013-06-17 | 2019-01-22 | Interdigital Ce Patent Holdings | WiFi display compatible network gateway |
US20140372558A1 (en) * | 2013-06-17 | 2014-12-18 | Thomson Licensing | Wifi display compatible network gateway |
US9639113B2 (en) * | 2013-08-09 | 2017-05-02 | Lenovo (Beijing) Limited | Display method and electronic device |
US20150042633A1 (en) * | 2013-08-09 | 2015-02-12 | Lenovo (Beijing) Limited | Display method and electronic device |
US20150168935A1 (en) * | 2013-12-17 | 2015-06-18 | Inwellcom Technology Co., Ltd. | Remotely managing and controlling system and method |
US20150229997A1 (en) * | 2014-02-07 | 2015-08-13 | Samsung Electronics Co., Ltd. | User terminal and control method thereof |
US9948979B2 (en) * | 2014-02-07 | 2018-04-17 | Samsung Electronics Co., Ltd. | User terminal and control method thereof |
US20150293741A1 (en) * | 2014-04-10 | 2015-10-15 | Screenovate Technologies Ltd. | Method for real-time multimedia interface management |
US10127002B2 (en) * | 2014-04-10 | 2018-11-13 | Screenovate Technologies Ltd. | Method for real-time multimedia interface management |
US20150339026A1 (en) * | 2014-05-22 | 2015-11-26 | Samsung Electronics Co., Ltd. | User terminal device, method for controlling user terminal device, and multimedia system thereof |
US20150347188A1 (en) * | 2014-05-30 | 2015-12-03 | Appsfreedom, Inc. | Delegated business process management for the internet of things |
US11550465B2 (en) | 2014-08-15 | 2023-01-10 | Apple Inc. | Weather user interface |
US11042281B2 (en) | 2014-08-15 | 2021-06-22 | Apple Inc. | Weather user interface |
US11922004B2 (en) | 2014-08-15 | 2024-03-05 | Apple Inc. | Weather user interface |
US10667104B2 (en) | 2014-11-05 | 2020-05-26 | Samsung Electronics Co., Ltd. | Method for device to control another device and the device |
US11490235B2 (en) | 2014-11-05 | 2022-11-01 | Samsung Electronics Co., Ltd. | Method for device to control another device and the device |
US10313857B2 (en) * | 2014-11-05 | 2019-06-04 | Samsung Electronics Co., Ltd. | Method for device to control another device and the device |
US20160127851A1 (en) * | 2014-11-05 | 2016-05-05 | Samsung Electronics Co., Ltd. | Method for device to control another device and the device |
US10931806B2 (en) * | 2014-12-09 | 2021-02-23 | Inwellcom Technology Co., Ltd. | Remotely managing and controlling system and method |
US10802703B2 (en) | 2015-03-08 | 2020-10-13 | Apple Inc. | Sharing user-configurable graphical constructs |
US11580867B2 (en) | 2015-08-20 | 2023-02-14 | Apple Inc. | Exercised-based watch face and complications |
US11908343B2 (en) | 2015-08-20 | 2024-02-20 | Apple Inc. | Exercised-based watch face and complications |
US10326822B2 (en) * | 2015-12-03 | 2019-06-18 | Google Llc | Methods, systems and media for presenting a virtual operating system on a display device |
US20170244779A1 (en) * | 2015-12-03 | 2017-08-24 | Google Inc. | Methods, systems and media for presenting a virtual operating system on a display device |
US11150798B2 (en) | 2016-03-28 | 2021-10-19 | Apple Inc. | Multifunction device control of another electronic device |
US10650621B1 (en) | 2016-09-13 | 2020-05-12 | Iocurrents, Inc. | Interfacing with a vehicular controller area network |
US11232655B2 (en) | 2016-09-13 | 2022-01-25 | Iocurrents, Inc. | System and method for interfacing with a vehicular controller area network |
US11307757B2 (en) | 2016-09-23 | 2022-04-19 | Apple Inc. | Watch theater mode |
US11775141B2 (en) | 2017-05-12 | 2023-10-03 | Apple Inc. | Context-specific user interfaces |
US10838586B2 (en) | 2017-05-12 | 2020-11-17 | Apple Inc. | Context-specific user interfaces |
US11327634B2 (en) | 2017-05-12 | 2022-05-10 | Apple Inc. | Context-specific user interfaces |
US11257464B2 (en) | 2017-05-16 | 2022-02-22 | Apple Inc. | User interface for a flashlight mode on an electronic device |
US10732826B2 (en) * | 2017-11-22 | 2020-08-04 | Microsoft Technology Licensing, Llc | Dynamic device interaction adaptation based on user engagement |
US20190155495A1 (en) * | 2017-11-22 | 2019-05-23 | Microsoft Technology Licensing, Llc | Dynamic device interaction adaptation based on user engagement |
USD858554S1 (en) * | 2018-04-25 | 2019-09-03 | Allstate Insurance Company | Display screen with graphical user interface |
US11327650B2 (en) | 2018-05-07 | 2022-05-10 | Apple Inc. | User interfaces having a collection of complications |
US11726735B2 (en) * | 2019-04-18 | 2023-08-15 | Samsung Electronics Co., Ltd. | Display device, user terminal device, method for controlling display device, and method for controlling user terminal device |
US20220214848A1 (en) * | 2019-04-18 | 2022-07-07 | Samsung Electronics Co., Ltd. | Display device, user terminal device, method for controlling display device, and method for controlling user terminal device |
US11340778B2 (en) | 2019-05-06 | 2022-05-24 | Apple Inc. | Restricted operation of an electronic device |
US10788797B1 (en) | 2019-05-06 | 2020-09-29 | Apple Inc. | Clock faces for an electronic device |
US11340757B2 (en) | 2019-05-06 | 2022-05-24 | Apple Inc. | Clock faces for an electronic device |
US11131967B2 (en) | 2019-05-06 | 2021-09-28 | Apple Inc. | Clock faces for an electronic device |
US11301130B2 (en) | 2019-05-06 | 2022-04-12 | Apple Inc. | Restricted operation of an electronic device |
US10878782B1 (en) | 2019-09-09 | 2020-12-29 | Apple Inc. | Techniques for managing display usage |
US10936345B1 (en) * | 2019-09-09 | 2021-03-02 | Apple Inc. | Techniques for managing display usage |
US10852905B1 (en) | 2019-09-09 | 2020-12-01 | Apple Inc. | Techniques for managing display usage |
US10908559B1 (en) | 2019-09-09 | 2021-02-02 | Apple Inc. | Techniques for managing display usage |
US11061372B1 (en) | 2020-05-11 | 2021-07-13 | Apple Inc. | User interfaces related to time |
US11372659B2 (en) | 2020-05-11 | 2022-06-28 | Apple Inc. | User interfaces for managing user interface sharing |
US11526256B2 (en) | 2020-05-11 | 2022-12-13 | Apple Inc. | User interfaces for managing user interface sharing |
US11822778B2 (en) | 2020-05-11 | 2023-11-21 | Apple Inc. | User interfaces related to time |
US11842032B2 (en) | 2020-05-11 | 2023-12-12 | Apple Inc. | User interfaces for managing user interface sharing |
US11442414B2 (en) | 2020-05-11 | 2022-09-13 | Apple Inc. | User interfaces related to time |
US20220067163A1 (en) * | 2020-08-28 | 2022-03-03 | Coretronic Corporation | Electronic-device control system and electronic-device control method |
US11694590B2 (en) | 2020-12-21 | 2023-07-04 | Apple Inc. | Dynamic user interface with time indicator |
US11720239B2 (en) | 2021-01-07 | 2023-08-08 | Apple Inc. | Techniques for user interfaces related to an event |
US11921992B2 (en) | 2021-05-14 | 2024-03-05 | Apple Inc. | User interfaces related to time |
US11955100B2 (en) | 2022-02-18 | 2024-04-09 | Apple Inc. | User interface for a flashlight mode on an electronic device |
Also Published As
Publication number | Publication date |
---|---|
GB2514856A (en) | 2014-12-10 |
GB2514856B (en) | 2017-12-20 |
TW201448584A (en) | 2014-12-16 |
GB201521425D0 (en) | 2016-01-20 |
GB2534672B (en) | 2017-12-20 |
US10021180B2 (en) | 2018-07-10 |
CN104219791A (en) | 2014-12-17 |
GB2534672A (en) | 2016-08-03 |
CN104219791B (en) | 2018-03-09 |
TWI538499B (en) | 2016-06-11 |
GB201312300D0 (en) | 2013-08-21 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10021180B2 (en) | Universal environment extender | |
EP3029650B1 (en) | Electronic device, audio device, and methods of controlling electronic device and audio device power supply | |
CN108141629B (en) | Method, system, and medium for presenting a virtual operating system user interface on a display device | |
US10887932B2 (en) | Electronic device and method of registering personal cloud apparatus in user portal server thereof | |
EP3169086B1 (en) | Connection method for multimedia playing device, master device, control terminal, and system | |
US9629195B2 (en) | Connection switching method applicable to remote controllable system and mobile device, remote controllable system using the same, and mobile device using the same | |
JP6383109B2 (en) | Network connection method, network connection device, terminal, communication device, network connection system, program, and recording medium | |
WO2016090826A1 (en) | Configuration method and device | |
KR20150043439A (en) | Method, Terminal, and System for Sharing Information with Digital Television Terminal | |
US20170019443A1 (en) | Device and method for transferring the rendering of multimedia content | |
CN105573609A (en) | Content sharing method and device | |
US11297373B2 (en) | Techniques for authorizing controller devices | |
WO2017096851A1 (en) | Method, system, and server for pushing video file | |
KR20160078024A (en) | Device and method thereof for transmitting/receiving video data | |
EP3116188B1 (en) | Method and device for providing contents in communication system | |
CN107094132B (en) | Network security of the internet of things | |
KR20120034338A (en) | Security operating method for access point and system thereof | |
KR102531029B1 (en) | Methods, systems and media for presenting media content | |
KR20160101754A (en) | Audio video navigation devide in a vehicle, and avn auxiliary device and operating method thereof | |
US20170127120A1 (en) | User terminal and control method therefor | |
WO2015169005A1 (en) | Method, system and main sharing end for sharing screen and operation control | |
US20240089526A1 (en) | Method for playback in split screen and display device | |
US20150188968A1 (en) | Electronic device and playback method of streaming media thereof |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: KINGSTON TECHNOLOGY CORPORATION, CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:CHEN, BEN WEI;REEL/FRAME:030545/0096 Effective date: 20130531 |
|
AS | Assignment |
Owner name: KINGSTON DIGITAL, INC., CALIFORNIA Free format text: CORRECTIVE ASSIGNMENT TO CORRECT THE ASSIGNEE NAME PREVIOUSLY RECORDED ON REEL 030545 FRAME 0096. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT;ASSIGNOR:CHEN, BEN WEI;REEL/FRAME:030573/0346 Effective date: 20130531 |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 4 |