US20150088966A1

US20150088966A1 - Service activity user interface

Info

Publication number: US20150088966A1
Application number: US14/093,943
Authority: US
Inventors: Edward James Gayles; Ngai Yan Gladys Pang; Zaka Ur Rehman Ashraf; Christopher Karl Lang; Michael William Miller; Tomer Moscovich; Rajasimman Madhivanan; Vlad Troyanker; Ruoruo Zhang
Original assignee: Amazon Technologies Inc
Current assignee: Amazon Technologies Inc
Priority date: 2013-09-20
Filing date: 2013-12-02
Publication date: 2015-03-26
Also published as: US20150089038A1; EP3047404A4; US9578117B2; EP3047404A1

Abstract

Described are techniques and systems for providing a user interface which provides information about one or more activities being performed by services available among affiliated devices. The information is based on activity data which may be obtained through a subscription-publication mechanism with the affiliated devices. The user interface may also provide activity controls to modify the one or more activities.

Description

PRIORITY

This application claims priority to U.S. Provisional Patent Application Ser. No. 61/880,681 filed on Sep. 20, 2013, entitled “Enhanced Multiple Media Device Infrastructure.” The entirety of this previously filed application is hereby incorporated by reference. This matter is also known by matter numbers 579-6105 and PM6821-US.

BACKGROUND

A media device may provide one or more services accessible to other media devices. These services may provide for various activities on the device associated with consumption of content. For example, a first media device may provide a content streaming service while a second media device provides a content display service.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a system for providing activity data about one or more activities executing on one or more devices, and presenting information about the one or more activities using an activity view user interface.

FIG. 2 is a block diagram of activity data.

FIG. 3 illustrates a block diagram of a media device.

FIG. 4 illustrates a flow diagram of a process of media devices subscribing to one another to receive published activity data.

FIG. 5 illustrates an activity view user interface.

FIG. 6 illustrates an activity detail view interface.

FIG. 7 illustrates an activity view interface including a private activity.

FIG. 8 illustrates a flow diagram of a process of acquiring activity data.

FIG. 9 illustrates a flow diagram of a process of responding to a subscription request.

FIG. 10 illustrates a flow diagram of a process of providing an activity view user interface using the media device.

Certain implementations and embodiments will now be described more fully below with reference to the accompanying figures, in which various aspects are shown. However, various aspects may be implemented in many different forms and should not be construed as limited to the implementations set forth herein. Like numbers refer to like elements throughout.

DETAILED DESCRIPTION

Content such as video, music, games, applications, and so forth may be consumed using a variety of media devices. These media devices may include remote controls, televisions, set-top boxes, tablet computers, laptop computers, desktop computers, media players, home automation devices, sensors, and so forth. The media devices may be configured to communicate with one another or other devices using one or more networks. A group of media devices may be affiliated with one another. This affiliation may be based on a common account, network, geographic location, and so forth. The media devices may have one or more services which are available to provide various functions. These services may be used by the media device itself to perform one or more activities, or may be used by other media devices. For example, a set-top box may provide a streaming service which supports the activity of delivering video content to a television. The television may be running a video presentation service which supports the activity of presenting the video content on a display. Users may benefit from awareness of these activities in the affiliated devices, as well as notification of other events. For example, the user may wish to see what content the television in the next room is presenting, or what a child is reading on a tablet computer. Furthermore, the users may benefit from being able to control these activities, without necessarily opening an application dedicated to the activity. Continuing the example, the user may wish to pause the content playing on the television in the next room, without having to access a dedicated remote control application.
Described in this disclosure are techniques and systems for exchanging activity data about one or more activities being provided by services executing at least in part on affiliated media devices. The activity data, or a portion thereof, may be provided using a subscription-publication arrangement, such that the affiliated media devices exchange properties of the various activities. In other implementations, other mechanisms such as message queues, network advertisement, broadcast, and so forth may be used to provide the activity data. The activity data, or a portion thereof, may be sent from one media device to another without request, sent upon request, or a combination thereof.
This activity data may be used to provide an activity view interface which informs the user as to the status of the activities on the affiliated devices. The activity view interface may also provide some activity controls, allowing the user to readily modify the activities. For example, the user may transfer an activity such as presentation of content from the tablet in-hand to the television. These activity controls may be a subset of controls associated with the activity, such as pause and play for audio or video content.
The activity view user interface may also be configured to provide additional details. For example, the user may activate a control to view additional information about the content being presented, purchase options for the content, and so forth. Controls may also be provided which invoke an application associated with the service for more comprehensive control, and so forth. For example, touching on an icon for the video content being presented on the television may open a remote control application which allows for control of the various functions on the television.
These techniques and user interfaces described herein may improve the user experience by providing the user with concise information about the affiliated devices. This information may include content consumption activities, what devices are available to perform various activities, notifications about events, and so forth. Furthermore, the user interfaces may allow the user to make changes to the activities, such as changing which device is performing a particular activity, sharing activities or content with other devices, and so forth. The user interface is configured to emphasize the content, and support the user in consuming that content using one or more of the affiliated devices.

Illustrative System

FIG. 1 is a system 100 for providing activity data about one or more activities and an activity view user interface based on this activity data. One or more users 102(1), 102(2), . . . , 102(Y) may use one or more media devices 104(1), 104(2), . . . , 104(D). As used in this disclosure, letters in parenthesis such as “(D)” indicate an integer value. These media devices 104 may include televisions, tablet computers, personal computers, electronic book readers, gaming consoles, set-top boxes, media players, in-vehicle entertainment systems, portable media players, smartphones, servers, sensors, home automation systems, and so forth.
A group of media devices 104 may be affiliated with one another. This affiliation may be based on one or more of a common account, connection to a common network, connection to a common device, geographic location, and so forth. For example, a group of media devices 104 may comprise those media devices 104 which are in a particular home, or which are associated with a single account.
Content 106 may be consumed, processed, distributed, and so forth by the media devices 104. The content 106 may include audio data, video data, electronic book (“eBook”) data, application programs such as games, and so forth. The content 106 may be downloaded or streamed from a content provider for consumption, processing, storage, and so forth on the media device 104.
The media devices 104 may couple to one or more local area networks 108. The local area networks 108 may be personal area networks (“PANs”). The local area network(s) 108 may interconnect with one another facilitating communication between the media devices 104. The local area networks 108 may be supported using an access point topology, ad-hoc peer-to-peer topology, and so forth. The local area networks 108 may be wired, wireless, or a combination. The local area networks 108 may be implemented using Ethernet, Wi-Fi™, Bluetooth®, ZigBee, and so forth. Within the system 100, several different local area networks 108 may coexist. For example, media devices 104(1) and 104(6) may be connected to one another using Wi-Fi™, while the media device 104(2) and 104(5) may be interconnected using Bluetooth®.
The system 100 may also include one or more wide area networks 110. The wide area networks 110 may be communicatively coupled with one or more of the affiliated media devices 104, the local area network(s) 108, and so forth. The wide area network 110 may include private networks, public networks such as the Internet, or a combination thereof. Connections may be established between media devices 104 which are on the same or different LANs 108 or WANs 110. For example, information may be passed from the media device 104(2) to the media device 104(1) using Bluetooth®. The media device 104(1) may in turn pass this information to the media device 104(7) via the wide area network 110.
The media devices 104 may execute, or have available for execution, one or more service modules 112. These service modules 112 provide services for use on the local media device 104, remote media devices 104, or both. These services may include managing digital rights management (“DRM”) credentials of the content, storing and retrieving content, rendering content for presentation on an output device, and so forth. These service modules 112 may perform one or more activities associated with the content 106. These activities may include receiving and processing the content 106 for presentation, generating a stream of content 106 for presentation, transferring presentation from one media device 104 to another, installing applications, accessing a uniform resource locator (“URL”), and so forth. Other services may provide for data processing, data retrieval, data storage, monitor operation of the media devices 104, such as battery status, connectivity to the networks, and so forth. In some implementations the service modules 112 may support various services such as Miracast™ as promulgated by the Wi-Fi Alliance, Universal Plug and Play (“UPnP”) as promulgated by the UPnP Forum, and so forth.
The service modules 112 may provide activity data 114. The activity data 114 comprises information indicative of operation of the service module 112, such as the activities being performed. This information may include descriptions of the activity, properties of the state of the activity, and so forth. For example, the activity data 114 may indicate that a particular content title such as “The Art of Fugue” by Johann Sebastian Bach is playing on a particular media device 104(2). In another example, the activity data 114 may include information indicative of an active Miracast™ session. The activity data 114 is discussed in more detail below with regard to FIG. 2.
The media devices 104 may also execute an activity manager module 116. The activity manager module 116 is configured to coordinate the acquisition and distribution of activity data 114 from the local media device 104 as well as from remote media devices 104. The activity manager module 116 may be configured to operate with application programming interfaces (“API”) for the service modules 112, to exchange service commands 118. The service commands 118 are instructions or other information configured to affect operation of the service module 112. For example, service commands 118 may include instructions to pause presentation of content, resume presentation of content, forward fast to a later point in the content, transfer the activity to another media device 104, and so forth.
The activity manager module 116 may also be configured to maintain permissions associated with the activities. These permissions determine what kind of interaction other users 102, media devices 104, or a combination thereof, may have with the activities. These permissions may include Read-Only, Read and Control, Private and so forth. The “Read-Only” control allows the activity data 114 associated with the activity to be distributed and provided to the other affiliated media devices 104. In comparison, “Read and Control” allows the distribution of the activity data 114 as well as modification to the activity based on the service commands 118 or other inputs. The “Private” permission prevents distribution of the activity data 114 to remote media devices 104 and may also result in a denial of action based on the service commands 118 or other inputs received from remote media devices 104.
The activity manager module 116 may be configured to operate in a peer-to-peer arrangement. In the peer-to-peer arrangement, the local activity manager module 116 may operate in conjunction with one or more remote local activity manager modules 116 to distribute the activity data 114 or other information. For example, the local activity manager module 116 may send to remote activity manager modules 116 information following a change in the permissions for an activity. In some implementations, the activity manager module 116 may be configured to operate as a centralized authority, or operate in conjunction with another device designated as a centralized authority. For example, one of the media devices 104 may be designated as a master, and be configured to maintain information about the activities, such as the permissions, current properties of the activities, and so forth.
The service module 112 may register with the local activity manager module 116. The terms “local” and “remote” are relative to one another such that “local” indicates execution on a first device while “remote” indicates execution on a second device external to the first. The service modules 112 may register one or more event handlers with the activity manager module 116, and may also provide at least a portion of the activity data 114. An activity provided by the service module 112 may also publish one or more properties using an event notification module. The event notification module provides a common service with which to publish properties such that those properties are available to other services. The event notification module is discussed below with regard to FIG. 3.
An activity view module 120 is configured to provide one or more user interfaces which enable the user 102 to access information about the activities on the affiliated media devices 104. The activity view module 120 may also provide controls with which the user 102 may affect the activities on the affiliated media devices 102. The activity view module 120 may receive activity data 114 or information based thereon from the activity manager module 116. Upon a determination of access to the user interface provided by the activity view module 120, the activity view module 120 may request activity data 114 about the activities from the activity manager module 116. Based on the activity data 114, a user interface such as the activity view user interface 122 may be provided. The activity view user interface 122 may comprise one or more of a graphical user interface, a character user interface, an audible interface, or a haptic interface. The user interfaces are discussed in more detail below with regard to FIGS. 5-7.
The user 102 may access the user interface 122 to view activities on the affiliated devices. For example, the user 102(6) may see in the user interface information indicating that the media device 104(1) is playing the movie “Life of a Cowboy”, as well as a private showing of a photo album on the local media device 104(6). Likewise, the user 102(1) using the media device 104(1) sees in the user interface that the media device 104(2) is paused while playing “The Art of Fugue”, and that the media device 104(7) has someone playing the game “Whimsical Penguins”. As described, the activity controls may allow the user 102 to affect the activities, including joining those activities. For example, the user 102(1) may activate the control to join the game and play “Whimsical Penguins” with the user 102(7) on the media device 104(7). In another example, the user 102(6) may use the user interface on the media device 104(6) to change permissions from “Private” to “Read-Only” for the activity of presenting the photo album, such that other affiliated media devices 104 may present the pictures along with, or instead of, the media device 104(6).
The user interface may also provide notifications. These notifications provide information indicative of one or more services provided by the service modules 112 of the affiliated media devices 104. These notifications may include information such as battery/power status, a media device 104 re-establishing communication with the affiliated media device(s) 104, and so forth. For example, the notifications may indicate the media device 104(1) is running low on battery power, that communication with the media device 104(7) has been established, home automation information such as status of a dishwasher, and so forth.
The media devices 104 may be in communication with one or more servers 124(1), 124(2), . . . , 124(S). These servers 124 may provide various functions to the media devices 104. For example, the servers 124 may send or stream content 106, distribute updates to the service modules 112, and so forth. In some implementations the servers 124 may send, receive, or send and receive one or more of the activity data 114 or the service commands 118 and direct this information to one or more of the other affiliated media devices 104. For example, the server 124 may receive activity data 114(7) from the media device 104(7) and provide to the media devices 104(1)-(6). In another example, the server 124 may provide activity data 114 to the media devices 104.
By using the user interface provided by the activity view module 120 and supported by the activity manager module 116, the affiliated media devices 104 may readily provide information to the users 102 about what activities are taking place, notifications for events of interest, and so forth. The user 102 may initiate control over one or more of these activities using the interface, or may use the interface to invoke a particular application which controls that activity and operates in conjunction with the service module 112 executing the activity.
FIG. 2 is a block diagram 200 of the activity data 114 which may be acquired and managed by the activity manager module 116. As described above, the activity data 114 may be used to provide information presented by the activity view module 120 in the activity view user interface 122.
The activity data 114 may include an activity description 202 and properties 204. The activity data 114 may be expressed as a string, metadata, using a markup language, as executable code, script, and so forth.
The activity description 202 may include a service description 202(1). The service description 202(1) provides information indicative of the service module 112 which started the activity. For example, the service description 202(1) may indicate that the service module 112(1) initiated the activity. The activity identifier 202(2) uniquely identifies the activity provided by the service module 112. For example, the service module 112 may be contemporaneously providing two activities such as streaming video to the media device 104(1) and the media device 104(6). The activity identifier 202(2) provides information which disambiguates the two activities.
The activity description 202 may also include a description 202(3). The description 202(3) provides information in a human-friendly or human-readable format. For example, the description 202(3) may comprise the string “Playing a movie” or “Downloading a game.”
Permissions 202(4) information may also be included in the activity description 202. The permissions 202(4) provide information which indicates the permissions associated with the activity. These permissions may be used to configure how information about the activities is shared, which (if any) media devices 104, user accounts, and so forth may affect the activities, and so forth. As described above, these permissions 202(4) may include, but are not limited to Read-Only, Read and Control, Private, and so forth. As described above, the “Read-Only” control allows the activity data 114 associated with the activity to be distributed and provided to the other affiliated media devices 104. In comparison, “Read and Control” allows the distribution of the activity data 114 as well as modification to the activity based on the service commands 118 or other inputs. The “Private” permission prevents distribution of the activity data 114 to remote media devices 104 and may also result in a denial of action based on the service commands 118 or other inputs received from remote media devices 104.
Activity type 202(5) indicates whether the activity involves content delivery sharing or device controlling operation. With content delivery sharing, one media device 104(9) provides content 106 to another media device 104(10) for some activity, such as presentation on a display. In comparison, device controlling involves one media device 104(11) providing commands which control another media device 104(12). For example, the media device 104(11) may send service commands 118 to the media device 104(12) to stop or start playback.
The activity description 202 may also include information on the devices involved 202(6) in the activity. Continuing the prior example, in the content delivery sharing scenario, the devices involved 202(6) are the media device 104(9) and 104(10).
The activity description 202 may include other information, such as version information about the service module 112 performing the activity, time the activity commenced, account(s) associated with the activity, and so forth. The activity description 202 thus provides information about the activity, but may not provide information about other aspects of the activity, such as the content 106 associated with the activity.
The properties 204 provide additional information about the activity. This information may change during the activity. For example, activity may be the media device 104(1) streaming music for presentation on the media device 104(5), while the properties 204 provide information about the music currently being streamed. The properties 204 may include a content title 204(1). For example, the content title 204(1) may comprise the name of a song such as “The Art of Fugue”. The content description 204(2) provides information about the content 106, such as the genre of music, name of the composer, and so forth.
The properties 204 may provide information about the current state of the activity. Remaining time 204(3) may be included in the properties 204, which indicates how much time remains in presenting the content 106 before concluding. For example, that the song may have 1 minute 35 seconds remaining before finishing.
Other state data 204(4) may be included in the properties 204. For example, current volume setting of the output device, information about the encoding rate, and so forth.
Portions of the activity data 114 may be sent at different times in some implementations. For example, upon start of the activity, the activity manager module 116 may distribute the activity description 202 and the properties 204 to remote media devices 104. In some implementations, at a later time the properties 204 may be sent without the activity description 202. For example, the properties 204 may be distributed upon a change in one or more values, such as when the title of the song being played changes. The properties 204 may also be distributed periodically, such as every 10 seconds. In some implementations, a combination thereof may be used.
FIG. 3 illustrates a block diagram 300 of the media device 104. The media device 104 may include one or more processors 302 configured to execute one or more stored instructions. The processors 302 may comprise one or more cores. The media device 104 may include one or more input/output (“I/O”) interface(s) 304 to allow the processor 302 or other portions of the media device 104 to communicate with other devices. The I/O interfaces 304 may comprise inter-integrated circuit (“I2C”), serial peripheral interface bus (“SPI”), Universal Serial Bus (“USB”) as promulgated by the USB Implementers Forum, RS-232, one or more media device interfaces such as High Definition Multimedia Interface (“HDMI”) as promulgated by HDMI Licensing LLC, TOSLINK as promulgated by Toshiba Corp., analog video, analog audio, IEEE 3394 as promulgated by the Institute for Electrical and Electronics Engineers, and so forth.
The I/O interface(s) 304 may couple to one or more I/O devices 306. The I/O devices 306 may include input devices such as one or more of a camera, a microphone, a touch sensor, a button, and so forth. The I/O devices 306 may also include output devices such as one or more of a display, audio speakers, haptic output device and so forth. In some embodiments, the I/O devices 306 may be physically incorporated with the media device 304 or may be externally placed.
The media device 104 may also include one or more communication interfaces 308. The communication interfaces 308 are configured to provide communications between the media device 104 and other devices, such as other media devices 104, routers, access points, the servers 314, and so forth. The communication interfaces 308 may include devices such as transceivers configured to send and receive data using the personal area networks, wired and wireless LANs, wired and wireless wide area networks (“WANs”), and so forth. For example, the communication interfaces 308 may include hardware to connect using Ethernet, Wi-Fi™, Bluetooth®, ZigBee, and so forth.
The media device 104 may also include one or more busses or other internal communications hardware or software that allow for the transfer of data between the various modules and components of the media device 104.
As shown in FIG. 3, the media device 104 includes one or more memories 310. The memory 310 comprises one or more computer-readable storage media (“CRSM”). The CRSM may be any one or more of an electronic storage medium, a magnetic storage medium, an optical storage medium, a quantum storage medium, a mechanical computer storage medium and so forth. The memory 310 provides storage of computer readable instructions, data structures, program modules and other data for the operation of the media device 304.
The memory 310 may include at least one operating system (OS) module 312. The OS module 312 is configured to manage hardware resource devices such as the I/O interfaces 304, the I/O devices 306, the communication interfaces 308, and provide various services to applications or modules executing on the processors 302. Also stored in the memory 310 may be one or more of the following modules. These modules may be executed as foreground applications, background tasks, daemons, and so forth.
A communication module 314 is configured to discover other media devices 104 and the services they support and provide a framework for establishing and using connections with the other media devices 104. The connections may be authenticated, encrypted, and so forth. The communication module 314 may include one or more transport modules. The transport modules are configured to provide functionality at a transport layer and establish and maintain communication channels that transfer bits from one media device 104 to another device. In one implementation the transport modules may operate at layer 4 of the Open Systems Interconnection (“OSI”) model. The media device 104 may have multiple transport modules available contemporaneously.
A registrar module 316 may be provided to work in conjunction with the other modules such as the service modules 112, the activity manager module 116, and so forth. For example, the registrar module 316 may be configured to accept information from the service modules 112 and register their presence, event handlers, and so forth on the local media device 104.
An event notification module 318 may be provided to provide a mechanism for distributing the properties 204 or other information to associated devices. In some implementations the event notification module 318 may provide the functions associated with subscription, publication, or both. In other implementations, the event notification module 318 may be configured to use message queues, network advertisements, process received broadcasts, and so forth to distribute the properties 204 or other information.
As described above, the service modules 112 provide services for use on the local media device 104, remote media devices 104, or both. The service modules 112 may perform one or more activities associated with the content 106. These activities may include receiving and processing the content 106 for presentation, generating a stream of content 106 for presentation, transferring presentation from one media device 104 to another, and so forth.
A content direct sharing module 320 may be present in the memory 310. The content direct sharing module 320 may be configured to operate in conjunction with the communication module 314 to send a media stream to a receiving media device 104. For example, the content direct sharing module 320 may use connections maintained by the communication module 314 to send audio and video data from the tablet media device 104(1) to the television media device 104(6) for presentation.
A device controller module 322 may also be present in the memory 310. The device controller module 322 is configured to generate media presentation data. The media presentation data provides information indicative of one or more pieces of content 106 to be presented. For example, the media presentation data may include the uniform resource locator (“URL”) of “http://www.example.com/play?v=39A9GKd”. A first media device 104(12) may generate and send the media presentation data to a second media device 104(13). The second media device 104(13) may then use the media presentation data to present the content 106. The content 106 may be delivered as a download or stream. The device controller module 322 may also be configured to provide one or more service commands 118. For example, the device controller module 322 may send service commands 118 to the optical disk player media device 104(3) to open a disk tray.
The memory 310 may also store the activity manager module 116 which is configured to coordinate the acquisition and distribution of activity data 114 from the local media device 104 as well as from remote media devices 104. The activity manager module 116 may be configured to operate with APIs for the service modules 112, to exchange service commands 118.
The activity view module 120 is configured to provide one or more user interfaces which enable the user 102 to access information about the activities on the affiliated media devices 102. The activity view module 120 may also provide controls with which the user 102 may affect the activities on the affiliated media devices 102. The activity view module 120 may receive activity data 114 or information based thereon from the activity manager module 116. Upon a determination of access to the user interface provided by the activity view module 120, the activity view module 120 may request the activity data 114 about the activities from the activity manager module 116. Based on the activity data 114, a user interface such as the activity view user interface 122 may be provided.
The activity view module 120 may provide the user interface to the user 102 using the I/O devices 306 and may accept inputs received from the I/O devices 306. The user interface may include one or more visual, audible, or haptic elements. For example, the user interface may be configured to provide a graphic user interface, an audible user interface, and so forth.
One or more application modules 324 may be stored in the memory 310. The one or more application modules 324 provide functionality which interacts with the user 102. For example, an application module 324 may be a game which is playable by the user 102, a content player application, and so forth.
A digital rights management module 326 may provide support for presenting or processing content 106 which is protected using one or more digital rights management schemes. Other modules 328 may also be present. For example, a speech recognition module may be present and used to accept spoken input from the user 102 as received from a microphone I/O device 306.
The memory 310 may also include a datastore 330 to store information. The datastore 330 may use a flat file, database, linked list, tree, or other data structure to store the information. In some implementations, the datastore 330 or a portion of the datastore 330 may be distributed across one or more other devices including servers, network attached storage devices and so forth.
The datastore 330 may store the content 106, either in its entirety or a portion. Subscription data 332 may also be stored in the datastore 330. The subscription data 332 provides information on media devices 104 which have subscribed to publication of the activity data 114 by the local media device 104. The subscription data 332 may include expiration times for the subscriptions, or other information. The datastore 330 may also store the activity data 114, service commands 118, and so forth. Other data 334 may also be stored, such as user preferences, configuration files, and so forth.
FIG. 4 illustrates a flow diagram 400 of a process of media devices 104 subscribing to one another to receive published activity data. This diagram again depicts the two devices 104(1) and 104(2) for illustration and not by way of limitation. Time increases down the page as indicated by the arrow 402.
Block 404, at the media device 104, determines local activity provided by one or more of the service modules 112. For example, the media device 104(1) determines the service module 112(1) is playing a video. Likewise, the media device 104(2) determines the service module 112(2) is playing music.
Block 406 subscribes to other affiliated media devices 104 to receive activity data 114. For example, the activity manager module 116(1) of the media device 104(1) may initiate transmission of a subscription request 408(1) from the media device 104(1) to the media device 104(2). Likewise, the activity manager module 116(2) of the media device 104(2) may initiate transmission of a subscription request 408(2) from the media device 104(2) to the media device 104(1).
Based on the processing of the subscription request 408, the activity manager module 116 for the media device 104 will keep those subscribed media devices 104 informed as to the activities occurring thereon.
In some implementations, a mechanism other than a subscription request may be used. For example, a messaging queue, broadcast, and so forth may be used to distribute the activity data 114 between the media devices.
Block 410 sends activity data 114 to the subscribed media devices 104. Continuing the example, the media device 104(1) sends the activity data 114(1) to the media device 104(2). Likewise, the media device 104(2) sends the activity data 114(2) to the media device 104(1). As described above, in some implementations a portion of activity data 114 may be provided at a later time. For example, the current content titles 204(1) being played on each of the subscribed media devices 104 may be exchanged.
Block 412 presents an activity view user interface 122. For example, the user 102 may have activated the activity view user interface 122. In this example, the user 102(2) of the media device 104(2) takes no action; perhaps simply looking to see what other activities are taking place on the affiliated media devices 104. However, the user 102(1) of the media devices 104(1) decides to interact with one of the available activities as provided in the activity data 114.
Block 414 receives input to issue a service command 118 to modify the activity. For example, the user 102(1) may select a control in the activity view user interface 122 to request a pause in the playback of the music on the media device 104(2).
Block 416 sends the service command 118 to modify the activity to the remote media device 104. For example, the activity manager 116 may send the service commands 118(2) to pause playback on the media device 104(2).
Based on the command, block 418 modifies the activity performed by the one or more service modules 112. Continuing the example, the service module 112 playing the music on the media device 104(2) may pause playback.
While blocks 404 through 410 are depicted as occurring contemporaneously, the operations depicted need not occur symmetrically across the affiliated media devices 104. For example, media device 104(2) may be configured to respond to subscription requests by publishing the activity data 114, but may not initiate subscription requests.
FIG. 5 illustrates a user interface 500 including the activity view user interface 122. As described above, the activity view user interface 122 may be provided by the activity view module 120. The information presented by the activity view user interface 122 may be based on activity data 114 provided by the activity manager module 116. The example depicted here is from the point of view of the media device 104(1).
The user interface 500 may include a menu 502. This menu 502 may include controls for performing various functions, such as executing an application, configuring the media device 104, or invoking the activity view user interface 122.
The activity view user interface 122 may include one or more of an activity view 504 or a notification view 506. The activity view 504 provides information on one or more activities occurring among the affiliated media devices 104. In one implementation, the activity view 504 may be configured to present those activities involved in the consumption of content 106. For example, playing a movie, playing a game, presenting an eBook, and so forth. In this implementation, non-consumptive activities such as backing up data files, provisioning a device, and so forth may be omitted from the activity view 504.
The activity view 504 may comprise activity tiles 508. The activity tile 508 provides a space within a graphical interface within which information about a particular activity may be presented. The activity tile 508 may be fixed or variable in size. For example, the fixed activity tile 508 may comprise a predetermined region within the user interface. The region may be defined by a particular height and width when presented on a display device, such as a rectangle 500 pixels wide by 50 pixels tall. In one implementation, the activity tile 508 may be associated with a URL, such that the URL may be accessed and at least a portion of the contents retrieved therefrom may be used to populate the activity tile 508.
The activity tile 508 may comprise one or more standardized elements, custom elements as provided by or associated with a particular service module 112, or both. In one implementation the custom elements may be received from the one or more service modules 112. The activity tile 508 may be configured such that information based on the activity data 114 associated with a given activity is presented within an individual activity tile 508.
The activity tile 508 may include an activity graphic 510. For example, the activity graphic 510 may comprise an icon associated with an application, image of cover art for a music album, image of a movie poster, and so forth. The activity graphic 510 may comprise one or more of a still image, animation, or video.
Activity details 512 may be presented within the activity tile 508. The activity details 512 provide information about the activity. For example, the activity details 512 may provide the title of the content 106 being consumed, media devices 104 involved in the activity, and so forth.
The activity tile 508 may also include one or more activity controls 514. The activity controls 514 are configured to, when activated, affect the activity represented by the activity tile 508. The activity controls 514 may include controls to pause or resume playback, send to another media device 104 for presentation, pull from another media device 104 currently presenting, install an application, invoke an application associated with the content 106, open a URL, and so forth. For example, the activity controls 514 may be configured to control the activity by transferring the activity to a remote media device 104, from a remote media device 104, or from one remote media device 104 to another remote media device 104. In another example, the activity tile 508 may comprise an activity control 514 configured to, upon activation by user input 516, invoke an application associated with the content 106. The invoked application may provide a user interface with which to control consumption of the content 106. The invocation of the application may include one or more of initiating execution, bringing the application to a foreground, transitioning user interface focus to the foreground, and so forth.
The activity control 514 may be standard elements provided by the activity view module 120, or custom elements. For example, the activity control 514 associated with pausing content may be represented by a standardized icon of two vertical parallel lines, or may be replaced with a customized control in a particular activity tile 508 associated with a particular service module 112.
In some implementations the activity controls 514 may present information to the user 102. For example, one of the activity controls 514 configured to pause the presentation of the video playing on the media device 104(1) is depicted with a pie chart depicting time remaining in the content 106.
The activity controls 514 may provide a limited subset of controls associated with the activity. This limited subset may include the ability to pause, resume, fast forward, rewind, and so forth. For more comprehensive or detailed control over the activity, the application associated with providing a primary user interface for the service module 112 may be invoked. For example, for the activity of playing a video, a video content playback application may be accessed.
The activity controls 514 may also provide functions to send the activity to another media device 104, pull the activity from another media device 104, mirror the activity across multiple media devices 104, and so forth. Other functions provided may include joining a game, installing an application, photo album presentation, other activity. The activity controls 514 may thus be configured to provide a simple set of adjunct controls, working with, but not necessarily replacing more comprehensive controls.
The user 102 may provide user input 516. This user input 516 may comprise one or more of activation of a key, voice command, accelerometer input, camera input, or touch input (as illustrated here). Different portions of the activity tile 508 may be configured to provide different functions. For example, user input 516 associated with the activity controls 514 may activate the activity controls. Other portions of the activity tiles 508 may be configured to provide other functions. For example, an activity detail control may be associated with the activity details 512 which is configured to present an activity detail view, as described below in FIG. 6.
As mentioned earlier, the activity view user interface 122 may also include a notification view 506. The notification view 506 may be used to provide information which may be of use to the user 102, but which is not necessarily associated with an activity. In one implementation, the notification view 506 may present information based on activity data 114 and the corresponding one or more properties 204 for one or more services unassociated with content activities. For example, these activities may include home automation events, an alert indicating a low battery on one of the media devices 104, connectivity status of the affiliated media devices 104, and so forth.
In some implementations the activity view 504, the notification view 506, or both may be initially presented as a single tile. User input 516 may expand the single tile to depict the activity tiles 508 and so forth shown here. Similarly, user input 516 may collapse the activity tiles 508 back to a single tile.
FIG. 6 illustrates a user interface 600 including an activity detail view interface. In some implementations, the user interface 600 may be invoked based on user input 516 to one or more of the activity tiles 508, such as through activation of an activity detail control. In some implementations at least a portion of the activity tile 508 may be configured as the activity detail control, such that a touch, click, and so forth, open the activity detail view depicted here. In other implementations a gesture, button input, spoken command, and so forth may invoke the activity detail view interface. The example depicted here is from the point of view of the media device 104(1).
As above, a menu 602 may be provided to provide the user 102 with navigational options within the overall user interface. The activity detail view 604 provides more comprehensive information about the particular activity. In this example, the activity detail view 604 has been invoked following user input 516 with regard to the activity tile 508 for the activity of playing “The Art of Fugue”. As a result, the activity detail view 604 in this example provides additional information about that activity.
Expanded activity detail 606 provides additional information about the activity. In this example, this may include various properties 204 associated with the activity, such as details about the content 106 being presented.
Additional information such as a progress indicator 608 depicting a relative position of the playback in the content 106 and time remaining may be provided. The activity controls 514 described above may be provided as well. In some implementations, such as on a device with a touch sensitive display, the activity controls 514 may be presented in a physically larger format.
The activity detail view 604 may also provide additional activity controls. A device selection control 610 provides functionality for the user 102 to change the media devices 104 involved in the activity. For example, the user 102 may select this control to pull an activity from a remote media device 104 to the local media device 104, or push the activity from the local media device 104 to the remote media device 104. Other changes may include mirroring presentation of content 106 to multiple devices 104, and so forth.
An application invocation control 612 provides the functionality of invoking the application or other user interface associated with the activity. This application may be one configured to administer or control the service module 112 performing the activity. For example, an audio content player application may be configured to control an audio presentation service module 112. Activation of the application invocation control 612 would thus open this application. Continuing the example illustrated here, activation of the application invocation control 612 would launch the music player application.
The activity detail view 604 may also include a permission control 614. Activation of the permission control 614 provides a user interface which enables the user 102 to change the permissions 202(4) associated with the activity. For example, the user 102 may select to make the music presentation “Read-Only” such that the local media device 104(2) may affect the activity and prevent other affiliated media devices 104 from doing so.
Other controls may also be presented. A purchase control may be configured to allow the user 102 to attempt to acquire rights in the content 106 currently being presented. For example, a “buy now” button may, when activated, charge an account associated with the local media device for a purchase of the content 106.
The controls described in this disclosure may be activate by a touch, gesture, click, button press, spoken command, movement, and so forth. For example, the control may comprise portions of a touchscreen configured to accept input, such that when touched activation occurs.
FIG. 7 illustrates another user interface 700 including an activity view user interface 122 showing a private activity. The example depicted here is from the point of view of the media device 104(6), and at least the media devices 104(1)-(7) are affiliated with one another.
In comparison to the user interface 500, the user interface 700 depicts five activities. One of these activities “Vacation Photo Album” has permissions 202(4) set to “Private”. As described above, the “Private” permission prevents the activity data 114 associated with the activity from being provided to remote media devices 104. Thus, the local media device 104(1) is able to present the activity tile 508 associated with the “private” activity. The local activity manager module 116 determines, based on the permissions 202(4), what activity data 114 to provide to remote affiliated media devices 104. As above, user input 516 may be used to invoke various functions, such as presentation of the activity detail view 604.

Illustrative Processes

FIG. 8 illustrates a flow diagram 800 of a process of acquiring activity data 114. This process may be executed at least in part by the media device 104.
Block 802 accesses local activity data 114. As described above, the local activity data 114 is indicative of operation of a service module 112 executing at least in part on the processor 302 of the local media device 104 to perform an activity. This activity may be associated with content 106, such as presenting content 106, streaming content 106, receiving content 106, and so forth.
Block 804 subscribes the local media device 104 to one or more remote media devices 104 for delivery of remote activity data 114 from the one or more remote media devices 104. In some implementations a mechanism other than subscription may be used. For example, a message queue may be used to distribute the activity data 114. As described above, the remote activity data 114 comprises information indicative of operation of a service executing at least in part on the one or more remote media devices 104 to perform an activity. The activity may be associated with content 106.
In some implementations, the local media device 104 may determine one or more affiliated remote media devices 104. For example, the determination may include receiving one or more broadcasts by the remote media devices 104 connected to the same local area network 108.
In other implementations, other mechanisms may be used to distribute the activity data 114. For example, instead of or in addition to the subscription-publication mechanism, media devices 104 may periodically broadcast their activity data 114 or a portion thereof. In another implementation, the local media devices 104 may poll the remote media devices 104 for activity data 114 or changes thereto.
Block 806 receives remote activity data 114 from the one or more subscribed remote media devices 104. For example, the communication interface 308 may receive the data by way of the local area network 108.
Block 808 stores the local activity data 114 and the remote activity data 114. For example, this information may be stored in the memory 310.
Block 810 provides at least a portion of the stored local activity data 114, the remote activity data 114, or both to one or more service modules 112 executing on the processor 302 of the local media device 104. For example, the activity data 114 may be provided to the activity manager module 116.
FIG. 9 illustrates a flow diagram 900 of a process of responding to a subscription request. This process may be executed at least in part by the media device 104. As described above, a subscription-publication mechanism may be used to distribute activity data 114 between the affiliated media devices 104.
Block 902 receives from one or more remote media devices 104 a subscription request 408. The subscription request 408 may include information such as the network address of the requesting media device 104. The subscription request 408 indicates that the requesting media device 104 would like to receive published activity data 114 from the local media device 104.
Block 904, the local media device 104 subscribes the one or more remote devices to receive the activity data 114. In some implementations the subscription may expire. For example, the subscription may expire after ten minutes. Subscribing devices would then periodically renew their subscriptions. This expiration allows for affiliated media devices 104 which have become disconnected from the network or otherwise become unavailable to be removed from the activity view user interface 122.
Block 906 determines a change in the local activity data 114. For example, one of the properties 204 may have changed, such as following a transition from playback of one song to another.
Block 908 publishes at least a portion of the changed local activity data 114 to the subscribed one or more remote media devices 104. For example, the local media device 104 may send updated portions of the activity data 114 using the networks to the remote affiliated media devices 104. This publication may include establishment of direct connections between the local media device 104 and the subscribed remote media devices 104, broadcast on the network(s), multicast on the network(s), and so forth.
In some implementations the subscription expirations may be extended or reset following the local media device 104 resuming operation after having been in a low power or sleep mode. This provides for the media devices 104 to enter lower power states, conserving electrical power, while maintaining their ability to participate in distribution of the activity data 114.
As described above, activities having a permission 202(4) of “Private” may not be published. As a result, they remain visible to the local media device 104 upon which they are executing, but not to other affiliated media devices 104.
FIG. 10 illustrates a flow diagram 1000 of a process of providing an activity view user interface 122 using the media device 104. This process may be executed at least in part by the media device 104.
Block 1002 determines access to an activity view user interface 122. As described above, the access may comprise presentation of the activity view user interface 122 on the display device of one or more media devices 104. The access may be determined by an interrupt, handler, or other mechanism. In some implementations this may comprise receiving a request to access the activity view user interface 122.
Block 1004 accesses activity data 114 for one or more of the service modules 112. These service modules 112 may be providing one or more activities. In one implementation, updates for at least a portion of the local activity data 114 may be accessed. The at least a portion may comprise one or more properties 204 indicative of the activity. For example, the remaining time 204(3) for video being presented.
Block 1006 provides the activity view user interface 122. This interface is configured to present at least a portion of the one or more properties 204 for one or more of the activities. In some implementations, the interface may also present at least a portion of the activity description 202. The activity view user interface 122 may also include one or more controls configured to control the one or more activities. For example, the activity controls 514, device selection control 610, application invocation control 612, permission control 614, and so forth As described above, the activity may be one or more of presenting video content, presenting audio content, presenting an electronic book, or executing a game.
Block 1008 may provide a notification view 506 interface configured to present at least a portion of information indicative of the one or more properties of the one or more service modules 112. As described above, these notifications may be for activities unassociated with content 106. Thus, the activity data 114 may comprise one or more properties 204 indicative of one or more services unassociated with content activities.
Block 1010 receives input indicative of activation of the one or more activity controls 514. For example, the user input 516 of the user 102 touching the activity controls 514 on the touchscreen.
Block 1012 sends one or more service commands 118 to the service module 112 performing the activity. As described above, the one or more service commands 118 are configured to instruct the service module 112 to alter performance of the activity. For example, the alteration may include pausing, resuming, rewinding, transferring the activity to another media device 104, and so forth.
Those having ordinary skill in the art will readily recognize that certain steps or operations illustrated in the figures above can be eliminated or taken in an alternate order. Moreover, the methods described above may be implemented as one or more software programs for a computer system and are encoded in a computer readable storage medium as instructions executable on one or more processors.
Embodiments may be provided as a computer program product including a nontransitory computer readable storage medium having stored thereon instructions (in compressed or uncompressed form) that may be used to program a computer (or other electronic device) to perform processes or methods described herein. The computer readable storage medium can be any one of an electronic storage medium, a magnetic storage medium, an optical storage medium, a quantum storage medium and so forth. For example, the computer readable storage media may include, but is not limited to, hard drives, floppy diskettes, optical disks, read-only memories (ROMs), random access memories (RAMs), EPROMs, EEPROMs, flash memory, magnetic or optical cards, solid-state memory devices, or other types of physical media suitable for storing electronic instructions. Further, embodiments may also be provided as a computer program product including a transitory machine-readable signal (in compressed or uncompressed form). Examples of machine-readable signals, whether modulated using a carrier or not, include, but are not limited to, signals that a computer system or machine hosting or running a computer program can be configured to access, including signals transferred by one or more networks. For example, the transitory machine-readable signal may comprise transmission of software by the Internet.
Separate instances of these programs can be executed on or distributed across separate computer systems. Thus, although certain steps have been described as being performed by certain devices, software programs, processes, or entities, this need not be the case and a variety of alternative implementations will be understood by those having ordinary skill in the art.
Additionally, those having ordinary skill in the art readily recognize that the techniques described above can be utilized in a variety of devices, environments and situations. Although the present disclosure is written with respect to specific embodiments and implementations, various changes and modifications may be suggested to one skilled in the art and it is intended that the present disclosure encompass such changes and modifications that fall within the scope of the appended claims.

Claims

What is claimed is:

1. A local device comprising:

a communication interface comprising a transceiver configured to send and receive data;

a display device;

a memory storing computer-executable instructions; and

a processor communicatively coupled to the communication interface, the display device, and the memory, the processor configured to execute the computer-executable instructions to:

access local activity data indicative of operation of a service executing at least in part on the processor to perform an activity associated with content, the activity comprising one or more of:

presenting video content,

presenting audio content,

presenting an electronic book, or

executing a game;

subscribe, using the communication interface, to one or more remote devices for delivery of remote activity data from the one or more remote devices, wherein the remote activity data comprises information indicative of operation of a service executing at least in part on the one or more remote devices to perform an activity associated with content;

receive, using the communication interface, remote activity data from the one or more subscribed to remote devices;

determine a request to present the user interface on the display device;

access at least a portion of the remote activity data from the one or more subscribed remote devices; and

present, with the user interface:

information about one or more of the activities, and

a control configured to change the one or more activities, responsive to input.

2. The system of claim 1, further comprising computer-executable instructions to:

receive input activating the control; and

send one or more service commands configured to instruct the service performing the activity to alter performance of the activity.

3. The system of claim 1, further comprising computer-executable instructions to:

receive from one or more remote devices a subscription request, wherein the subscription request is configured to direct the local device to publish the local activity data to the one or more remote devices;

determine a change in the local activity data; and

publish at least a portion of the changed local activity data to the subscribed one or more remote devices.

4. A computer-implemented method executable on a device, the method comprising:

receiving a request to access a user interface;

accessing activity data indicative of operation of one or more services, wherein the services are configured to perform an activity associated with content; and

presenting at least a portion of the activity data and one or more controls configured to control the activity in the user interface.

5. The computer-implemented method of claim 4, the user interface comprising one or more activity tiles such that each activity has an activity tile, wherein each activity tile has a predefined height and width within the user interface and the activity data for the activity is presented within the predefined height and width.

6. The computer-implemented method of claim 5, wherein the activity tile further comprises an activity detail control;

receiving input activating the activity detail control;

responsive to the activation of the activity detail control, providing an activity detail view comprising one or more of:

a progress indicator indicative of position of the activity within the content,

a permission control configured to accept input to set one or more permissions associated with the activity,

a device selection control, or

an application invocation control.

7. The computer-implemented method of claim 4, further comprising:

receiving a uniform resource locator (“URL”) from the one or more services;

retrieving one or more additional user interface elements using the URL; and

presenting the one or more additional user interface elements with the user interface.

8. The computer-implemented method of claim 4, wherein the one or more controls are configured to control the activity by discontinuing the activity on a local device and initiating performance of the activity on a remote device.

9. The computer-implemented method of claim 4, wherein the one or more controls are configured to invoke an application associated with the content.

10. The computer-implemented method of claim 4, the one or more services comprising applications for performing one or more of:

managing digital rights management credentials of content,

storing and retrieving content, or

rendering content for presentation on an output device.

11. The computer-implemented method of claim 4, wherein the activity comprises one or more of presenting or executing the content, and wherein the content comprises one or more of:

video data,

audio data,

electronic book data, or

an application program.

12. The computer-implemented method of claim 4, further comprising:

sending a request for activity data to one or more remote devices; and

receiving activity data from at least a portion of the one or more remote devices.

13. The computer-implemented method of claim 12, wherein the device and the one or more remote devices are affiliated with one another using one or more of:

an account,

geographic location,

connection to a common network, or

connection to a common device.

14. The computer-implemented method of claim 4, further comprising:

receiving a request for activity data from one or more remote devices; and

sending activity data to at least a portion of the one or more remote devices.

15. The computer-implemented method of claim 4, wherein the one or more controls include a permission control configured to accept input, the method further comprising:

receiving input using the permission control; and

setting one or more permissions for the activity, wherein the permissions are configured to affect one or more of:

publication of activity data to other devices, or

control of the activity by one or more remote devices.

16. The computer-implemented method of claim 4, the activity data further indicative of one or more services unassociated with content activities; and the method further comprising:

providing a notification view interface configured to present at least a portion of the activity data of the one or more services unassociated with content activities.

17. A computer-implemented method executable on a local device, the method comprising:

determining one or more affiliated remote devices;

sending a request for delivery of remote activity data to at least a portion of the one or more remote devices;

receiving the remote activity data, wherein the remote activity data comprises information indicative of operation of a service executing at least in part on the one or more remote devices to perform a remote activity;

accessing local activity data indicative of operation of a local service, wherein the local activity data comprises information indicative of operation of a service executing locally to perform a local activity;

receiving, using the communication interface, remote activity data from the one or more remote devices; and

storing activity data comprising at least a portion of the local activity data and the remote activity data.

18. The computer-implemented method of claim 17, wherein the local device and the one or more remote devices are communicatively coupled in a peer-to-peer configuration.

19. The computer-implemented method of claim 17, further comprising:

accessing the activity data; and

presenting, with the user interface, at least a portion of the activity data and one or more controls configured to control the remote activity or the local activity, wherein the user interface comprises one or more graphical user interface elements.

20. The computer-implemented method of claim 19, wherein the one or more controls are configured to control the activity by discontinuing the activity on a remote device and initiating performance of the activity on a local device.