US20130283169A1 - Voice-based virtual area navigation - Google Patents

Voice-based virtual area navigation Download PDF

Info

Publication number
US20130283169A1
US20130283169A1 US13/860,801 US201313860801A US2013283169A1 US 20130283169 A1 US20130283169 A1 US 20130283169A1 US 201313860801 A US201313860801 A US 201313860801A US 2013283169 A1 US2013283169 A1 US 2013283169A1
Authority
US
United States
Prior art keywords
user
zone
virtual area
communicant
virtual
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US13/860,801
Inventor
David Van Wie
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Social Communications Co
Original Assignee
Social Communications Co
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Social Communications Co filed Critical Social Communications Co
Priority to US13/860,801 priority Critical patent/US20130283169A1/en
Assigned to SOCIAL COMMUNICATIONS COMPANY reassignment SOCIAL COMMUNICATIONS COMPANY ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: VAN WIE, DAVID
Publication of US20130283169A1 publication Critical patent/US20130283169A1/en
Priority to US14/997,301 priority patent/US10649724B2/en
Priority to US16/842,554 priority patent/US11397507B2/en
Priority to US17/845,981 priority patent/US20220404950A1/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/04815Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/16Sound input; Sound output
    • G06F3/167Audio in a user interface, e.g. using voice commands for navigating, audio feedback
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
    • G06Q50/01Social networking
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00Speech recognition
    • G10L15/22Procedures used during a speech recognition process, e.g. man-machine dialogue
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00Data switching networks
    • H04L12/02Details
    • H04L12/16Arrangements for providing special services to substations
    • H04L12/18Arrangements for providing special services to substations for broadcast or conference, e.g. multicast
    • H04L12/1813Arrangements for providing special services to substations for broadcast or conference, e.g. multicast for computer conferences, e.g. chat rooms
    • H04L12/1827Network arrangements for conference optimisation or adaptation
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L51/00User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
    • H04L51/04Real-time or near real-time messaging, e.g. instant messaging [IM]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L51/00User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
    • H04L51/04Real-time or near real-time messaging, e.g. instant messaging [IM]
    • H04L51/043Real-time or near real-time messaging, e.g. instant messaging [IM] using or handling presence information
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L51/00User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
    • H04L51/04Real-time or near real-time messaging, e.g. instant messaging [IM]
    • H04L51/046Interoperability with other network applications or services
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L51/00User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
    • H04L51/07User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail characterised by the inclusion of specific contents
    • H04L51/18Commands or executable codes
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00Speech recognition
    • G10L15/22Procedures used during a speech recognition process, e.g. man-machine dialogue
    • G10L2015/223Execution procedure of a spoken command
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00Speech recognition
    • G10L15/22Procedures used during a speech recognition process, e.g. man-machine dialogue
    • G10L2015/225Feedback of the input speech

Definitions

  • FIG. 1 is a diagrammatic view of an example of a network communications environment.
  • FIG. 2 is a diagrammatic view of an example of a graphical user interface.
  • FIG. 3 is a diagrammatic view of an example of a graphical user interface.
  • FIG. 4 is a diagrammatic view of an example of a graphical user interface.
  • FIG. 5 is a diagrammatic view of an example of a graphical user interface.
  • FIG. 6 is a diagrammatic view of an example of a graphical user interface.
  • FIG. 7 is a block diagram of an example of a voice-based virtual area navigation architecture.
  • FIG. 8 is a flow diagram of an example of a voice-based virtual area navigation method.
  • FIG. 9 is a flow diagram of an example of a voice-based virtual area navigation method.
  • a “communicant” is a person who communicates or otherwise interacts with other persons over one or more network connections, where the communication or interaction may or may not occur in the context of a virtual area.
  • a “user” is a communicant who is operating a particular network node that defines a particular perspective for descriptive purposes.
  • a “computer” is any machine, device, or apparatus that processes data according to computer-readable instructions that are stored on a computer-readable medium either temporarily or permanently.
  • a “computer operating system” is a software component of a computer system that manages and coordinates the performance of tasks and the sharing of computing and hardware resources.
  • a “software application” (also referred to as software, an application, computer software, a computer application, a program, and a computer program) is a set of instructions that a computer can interpret and execute to perform one or more specific tasks.
  • a “data file” is a block of information that durably stores data for use by a software application.
  • computer-readable medium refers to any tangible, non-transitory medium capable storing information (e.g., instructions and data) that is readable by a machine (e.g., a computer).
  • Storage devices suitable for tangibly embodying such information include, but are not limited to, all forms of physical, non-transitory computer-readable memory, including, for example, semiconductor memory devices, such as random access memory (RAM), EPROM, EEPROM, and Flash memory devices, magnetic disks such as internal hard disks and removable hard disks, magneto-optical disks, DVD-ROM/RAM, and CD-ROM/RAM.
  • a “data sink” (referred to herein simply as a “sink”) is any of a device (e.g., a computer), part of a device, or software that receives data.
  • a “data source” (referred to herein simply as a “source”) is any of a device (e.g., a computer), part of a device, or software that originates data.
  • a “network node” (also referred to simply as a “node”) is a junction or connection point in a communications network. Examples of network nodes include, but are not limited to, a terminal, a computer, and a network switch.
  • a “server” network node is a host computer on a network that responds to requests for information or service.
  • a “client network node” is a computer on a network that requests information or service from a server.
  • URI Uniform Resource Identifier
  • a “network resource” is anything that can be identified by a uniform resource identifier (URI) and accessed over a network, including an electronic document, an image, a source of information, a service, operators and operands of a mathematical equation, classes, properties, numeric values, and a collection of other resources.
  • URI uniform resource identifier
  • a “network connection” is a link between two communicating network nodes.
  • a “connection handle” is a pointer or identifier (e.g., a uniform resource identifier (URI)) that can be used to establish a network connection with a network resource.
  • a “network communication” can include any type of information (e.g., text, voice, audio, video, electronic mail message, data file, motion data stream, and data packet) that is transmitted or otherwise conveyed from one network node to another network node over a network connection.
  • a “communicant interaction” is any type of direct or indirect action or influence between a communicant and another network entity, which may include for example another communicant, a virtual area, or a network service.
  • Examples of types of communicant interactions include communicants communicating with each other in realtime, a communicant entering a virtual area, and a communicant requesting access to a resource from a network service.
  • Presence refers to the ability and willingness of a networked entity (e.g., a communicant, service, or device) to communicate, where such willingness affects the ability to detect and obtain information about the state of the entity on a network and the ability to connect to the entity.
  • a networked entity e.g., a communicant, service, or device
  • a “realtime data stream” is data that is structured and processed in a continuous flow and is designed to be received with no delay or only imperceptible delay.
  • Realtime data streams include digital representations of voice, video, user movements, facial expressions and other physical phenomena, as well as data within the computing environment that may benefit from rapid transmission, rapid execution, or both rapid transmission and rapid execution, including for example, avatar movement instructions, text chat, realtime data feeds (e.g., sensor data, machine control instructions, transaction streams and stock quote information feeds), screen shares, and file transfers.
  • a “virtual area” (also referred to as an “area,” a “place,” or a “space”) is a representation of a computer-managed space or scene.
  • Virtual areas typically are one-dimensional, two-dimensional, or three-dimensional representations; although in some examples a virtual area may correspond to a single point.
  • a virtual area is designed to simulate a physical, real-world space. For example, using a traditional computer monitor, a virtual area may be visualized as a two-dimensional graphic of a three-dimensional computer-generated space. However, virtual areas do not require an associated visualization.
  • a virtual area typically refers to an instance of a virtual area schema, where the schema defines the structure and contents of a virtual area in terms of variables and the instance defines the structure and contents of a virtual area in terms of values that have been resolved from a particular context.
  • a “position” in a virtual area refers to a location of a point or an area or a volume in the virtual area.
  • a point typically is represented by a single set of one-dimensional, two-dimensional, or three-dimensional coordinates (e.g., x, y, z) that define a spot in the virtual area.
  • An area typically is represented by the three-dimensional coordinates of three or more coplanar vertices that define a boundary of a closed two-dimensional shape in the virtual area.
  • a volume typically is represented by the three-dimensional coordinates of four or more non-coplanar vertices that define a closed boundary of a three-dimensional shape in the virtual area.
  • “Navigating” in the context of a virtual area means moving or changing an attentional focus or state of a user or another communicant with respect to the virtual area. For example, a user may navigate his or her attention from one virtual communication context to another (e.g., from one zone of a virtual area to another or from one virtual area to another). A user also may navigate his or her presence or the presence of another communicant from one virtual communication context to another (e.g., from one zone of a virtual area to another or from one virtual area to another).
  • the term “includes” means includes but not limited to, the term “including” means including but not limited to.
  • the term “based on” means based at least in part on.
  • the examples that are described herein provide systems and methods for voice-based navigation in one or more virtual areas that define respective persistent virtual communication contexts. These examples enable communicants to use voice commands to, for example, search for communication opportunities in the different virtual communication contexts, enter specific ones of the virtual communication contexts, and bring other communicants into specific ones of the virtual communication contexts. In this way, these examples allow communicants to exploit the communication opportunities that are available in virtual areas, even when hands-based or visual methods of interfacing with the virtual areas are not available (e.g., when people are driving, passing through an airport, putting out a forest fire, delivering packages, flying fighter jets, jogging, driving a forklift, and otherwise living their mobile lives).
  • FIG. 1 shows an example of a network communications environment 10 that includes a first client network node 12 (Client Node A), a second client network node 14 (Client Network Node B), a virtual area platform 18 and an optional proxy node 19 that are interconnected by a network 20 .
  • the network 20 may include one or more of any of a local area network (LAN), a metropolitan area network (MAN), and a wide area network (WAN) (e.g., the internet).
  • the network 20 typically includes a number of different computing platforms and transport facilities that support the transmission of a wide variety of different media types (e.g., text, voice, audio, video, and other data) between network nodes.
  • the first client network node 12 includes a computer-readable medium 22 (or “memory”), a processor 24 , and input/output (I/O) hardware 26 (including, e.g., a display and network communication hardware).
  • the processor 24 executes at least one virtual area communications application 26 that is stored in the memory 22 .
  • the second client network node 14 typically is configured in substantially the same general way as the first client network node 12 , with a computer-readable medium 30 storing at least one virtual area communications application 32 , a processor 34 , and I/O hardware 36 .
  • Each of the network nodes 12 , 14 has a respective set of one or more sources and an exemplary set of one or more sinks.
  • Each source is a device or component that originates data of a particular data stream content type and each sink is a device or component that receives data of a particular data stream content type.
  • a source and a sink of the same data stream content type are referred to herein as being “complementary.”
  • Exemplary sources include an audio source (e.g., an audio capture device, such as a microphone), a video source (e.g., a video capture device, such as a video camera), a chat source (e.g., a text capture device, such as a keyboard), a motion data source (e.g., a pointing device, such as a computer mouse), and other sources (e.g., file sharing source or a source of a customized real-time data stream).
  • an audio source e.g., an audio capture device, such as a microphone
  • a video source e.g., a video capture device, such as a video camera
  • a chat source e.g., a text capture device, such as a keyboard
  • a motion data source e.g., a pointing device, such as a computer mouse
  • other sources e.g., file sharing source or
  • Exemplary sinks include an audio sink (e.g., an audio rendering device, such as a speaker or headphones), a video sink (e.g., a video rendering device, such as a display monitor), a chat sink (e.g., a text rendering device, such as a display monitor), a motion data sink (e.g., a movement rendering device, such as a display monitor), and other sinks (e.g., a printer for printing shared files, a device for rendering real-time data streams different from those already described, or software that processes real-time streams for analysis or customized display).
  • Each source has an active state in which the source is available for originating data and an inactive state in which the source is not available for originating data.
  • each sink has an active state in which the sink is available for receiving data and an inactive state in which the sink is not available for receiving data.
  • the communicants operating the client nodes 12 , 14 typically can control the states of the sources and sinks using controls provided by the communications applications 26 , 32 .
  • the communications applications 26 , 32 provide user controls for turning on/off the local microphones and the local speakers (e.g., headsets) on the client network nodes 12 , 14 .
  • the virtual area platform 18 includes at least one server network node 40 that provides a network infrastructure service environment 42 that manages sessions of the first and second client nodes 12 , 14 in one or more virtual areas 44 in accordance with respective virtual area applications 46 .
  • One or more of the virtual area applications 44 typically are synchronous conferencing applications that support one or more types of communications between the client nodes 12 , 14 (e.g., text chat, audio conferencing, video conferencing, application sharing, and file sharing).
  • the network infrastructure service environment 42 typically includes one or more network infrastructure services that cooperate with the communications applications 26 , 32 in the process of establishing and administering network connections between the client nodes 12 , 14 and other network nodes.
  • the network infrastructure services that are included in the example of the network infrastructure service environment 42 are an account service, a security service, an area service, a rendezvous service, an interaction service, and a capabilities engine.
  • the area service administers a virtual area 44 by managing sessions of the first and second client nodes 12 , 14 in the virtual area 44 in accordance with the virtual area application 46 .
  • Examples of the virtual area platform 18 and the virtual area applications 46 are described in U.S. Provisional Patent Application No. 61/563,088, filed Nov. 23, 2011.
  • Examples of an account service, a security service, an area service, a rendezvous service, and an interaction service are described in U.S. patent application Ser. No. 12/630,973, filed Dec. 4, 2009.
  • Examples of a capabilities engine are described in U.S. Provisional Patent Application No. 61/535,910, filed Sep. 16, 2011.
  • the network infrastructure service environment 42 maintains a relationship database 47 that contains the records 48 of interactions between communicants and social network profiles 50 that are associated with respective communicants. Each interaction record describes the context of an interaction between a pair of communicants.
  • Each social network profile 50 typically includes: identity characteristics (e.g., name, age, gender, and geographic location information such as postal mailing address) that describe a respective communicant or a persona that is assumed by the communicant; explicit relationship information that is declared by the communicant; and relationship information that is inferred from the communicant's interactions in the network communication environment 10 . Additional details regarding the relationship database 47 and the search and retrieval functionalities associated with the relationship database as described in U.S.
  • Some examples provide systems and methods for encapsulating assemblies (e.g., meetings, informal gatherings, and the like) of communicants in one or more virtual areas based on information obtained from the interactions records 48 . These examples provide a wide variety of ways in which to capture features of ongoing and completed assemblies and visualize those features at different levels of detail.
  • Exemplary visualizations include an activity view that shows a time-based visualization of assemblies within a particular domain, and a timeline view that shows a scalable visualization of a selected subset of the assemblies within a particular domain.
  • the domain may be predefined or specified by a user and may be used to select the assemblies that are presented in the visualizations.
  • the assemblies may be filtered according to a variety of different attributes of the assemblies, including by virtual location (e.g., a particular virtual area or a particular zone of a virtual area in which the assemblies occurred), by communicant (e.g., one or more communicants who participated in the assemblies), and by time (e.g., a period during which the assemblies started or ended).
  • virtual location e.g., a particular virtual area or a particular zone of a virtual area in which the assemblies occurred
  • communicant e.g., one or more communicants who participated in the assemblies
  • time e.g., a period during which the assemblies started or ended.
  • the assembly visualizations allow communicants to readily obtain a quick summary of the activities occurring in a particular context (e.g., one or more of time, virtual location, participants involved, and types of information shared).
  • the communications applications 26 , 32 , the area applications 46 , and the network infrastructure service environment 42 together provide a platform that administers the realtime connections with network nodes in an instance of a virtual area subject to a set of constraints 43 (e.g., capabilities and other types of permissions, rules, and preferences).
  • Each of the virtual area applications 46 is hosted by a respective one of the virtual areas 44 and includes a description of the respective virtual area 44 .
  • Communicants respectively operating the client nodes 12 , 14 connect to the virtual areas 44 through the virtual area communications applications 26 , 32 .
  • the communications applications 26 , 32 typically present respective views of the virtual areas 44 in accordance with data received from the network infrastructure service environment 42 .
  • the communications applications 26 , 32 also provide respective interfaces (e.g., one or more of a voice input interface, and audio output interface, and a visual graphical user interface) for receiving commands from the communicants.
  • visual graphical user interfaces communicants typically are represented in the virtual areas 44 by respective avatars (e.g., sprites).
  • communicants' states and activities are described using audio signals (e.g., synthesized speech).
  • Communicant avatars typically move about the virtual areas 44 in response to commands that are input by the communicants at their respective network nodes.
  • the communications applications 26 , 32 establish realtime data stream connections between the first and second client network nodes 12 , 14 and other network nodes connected to the virtual area 44 based on the positions of the communicants' avatars in the virtual areas 44 .
  • each of the client network nodes 12 , 14 includes a respective realtime kernel of the type described in U.S. patent application Ser. No. 12/630,973, filed Dec. 4, 2009, which supports remote configuration of stream handlers for processing data streams (e.g., rendering audio and video data streams) on a client network node.
  • a virtual area 44 may correspond to an abstract (non-geometric) virtual area that is defined with respect to abstract coordinates, or a visual virtual area that is defined with respect to one-, two- or three-dimensional geometric coordinates.
  • Abstract virtual areas may or may not be associated with respective visualizations, whereas visual virtual areas are associated with respective visualizations.
  • the virtual areas are visual virtual areas of the type disclosed in U.S. Pat. Nos. 7,769,806 and 7,844,724.
  • These visual virtual areas include physical geometry and collision geometry.
  • the physical geometry describes the shape of the virtual area.
  • the physical geometry typically is formed from surfaces of triangles, quadrilaterals, or polygons. Colors and textures are mapped onto the physical geometry to create a more realistic appearance for the virtual area. Lighting effects may be painted onto the visual geometry and the texture, color, or intensity near the lighting effects may be modified.
  • the collision geometry describes invisible surfaces that determine the ways in which objects can move in the virtual area.
  • the collision geometry may coincide with the visual geometry, correspond to a simpler approximation of the visual geometry, or relate to application-specific requirements of a virtual area designer.
  • Some examples of the virtual area platform 18 enable software application designers to define the semantics of position in an abstract virtual area (e.g., a software application or a computer data file). Through associations with respective connection rules, these position definitions can be used, for example, to drive connections to virtual areas, entries into virtual areas, connections to communicants and other sources or sinks of realtime data streams, and determinations of presence data relating to communicants, network resources, and network services. Additional details regarding systems and methods of defining the semantics of position in abstract virtual areas are described in U.S. application Ser. No. 12/631,008, which was filed on Dec. 4, 2009.
  • a virtual area typically includes one or more zones.
  • a zone may be a rendered spatial extent, a set of rules applied to a spatial extent, or both. Zones may be arranged hierarchically in a virtual area, with an outermost zone (referred to herein as the “Global Governance Zone”) enclosing all other zones in the virtual area.
  • Global governance Zone there can be location zones (e.g., rooms of a virtual area) or smaller governance zones that enclose a group of location zones and provide regions of governance on the map.
  • a zone definition typically also includes one or more channel definitions that describe how to create respective channels in the zone and specify the information about the channel that is published to a client network node that becomes present in the zone.
  • a channel is always uniquely defined point-to-point and is unique to a virtual area application and a session between a client network node and the virtual area platform.
  • Examples of the types of rules that may be associated with a zone include switching rules, governance rules, and permission rules.
  • Switching rules govern realtime stream connections between network nodes that are linked to the virtual area (e.g., network nodes that are associated with objects, such as avatars, in the virtual area).
  • the switching rules typically include a description of conditions for connecting sources and sinks of realtime data streams in terms of positions in the virtual area.
  • Each switching rule typically includes attributes that define the realtime data stream type to which the rule applies and the location or locations in the virtual area where the rule applies.
  • each of the rules optionally may include one or more attributes that specify a required role of the source, a required role of the sink, a priority level of the stream, and a requested data routing topology.
  • one or more implicit or default switching rules may apply to that part of the virtual area.
  • One exemplary default switching rule is a rule that connects every source to every compatible sink within an area, subject to policy rules. Policy rules may apply globally to all connections between the area clients or only to respective connections with individual area clients.
  • An example of a policy rule is a proximity policy rule that only allows connections of sources with compatible sinks that are associated with respective objects that are within a prescribed distance (or radius) of each other in the virtual area.
  • the network connections between network nodes may be arranged in a variety of different data routing topologies, including a peer-to-peer topology, a mediated topology (i.e., a topology in which connections between network nodes are mediated by another network node, such as a server network node, a client network node, or a network switch), and hybrid architectures that combine aspects of peer-to-peer and mediated architectures.
  • the switching rules dictate how local connection processes executing on each of the network nodes establishes communications with the other network nodes based on the locations of the associated objects in the zones of the virtual area.
  • a switching rule also may define a direct connection between network nodes or an indirect connection through an intermediate network node (e.g., the proxy node 19 shown in FIG. 1 ).
  • governance rules control who has access to resources (e.g., the virtual area itself, regions with the virtual area, and objects within the virtual area), who has access to data (e.g., data streams and other content) that is associated with the virtual area, what is the scope of that access to the data associated the virtual area (e.g., what can a user do with the data), and what are the follow-on consequences of accessing that data (e.g., record keeping, such as audit logs, and payment requirements).
  • an entire virtual area or a zone of the virtual area is associated with a “governance mesh” that enables a software application developer to associate governance rules with a virtual area or a zone of a virtual area. This avoids the need for the creation of individual permissions for every file in a virtual area and avoids the need to deal with the complexity that potentially could arise when there is a need to treat the same document differently depending on the context.
  • a permission rule defines a respective capability requirement (e.g., for a respective action, behavior, or state) in terms of one or more capabilities, attributes, and settings, which may be persistent or transient.
  • Examples of permission rules include: a rule that conditions a communicant's ability to enter a target zone on the communicant having a CanEnterZone capability for the target zone; a rule that conditions the ability of a grantee communicant to open a target door of a target room on the grantee communicant having a CanOpenDoor capability for the target room; and a rule that conditions the transmission of a message describing the state of a particular communicant's avatar in a zone to a recipient having a CanSeeState capability for the particular communicant in the zone.
  • a capability provides permission for a client to perform some action within the application. For example, a client may be granted the capability “CanEnterZone” for a specific zone within a virtual area that has been defined with that capability requirement. The client that has the capability can enter the zone, whereas a client without the capability would have their RDS state change rejected when they tried to enter the zone. Examples of capabilities systems for administering permission rules are described in U.S. Provisional Patent Application No. 61/535,910, filed Sep. 16, 2011.
  • the zones of a virtual area can be associated with respective switching rules, each of which instructs the area service to connect sources of a respective data stream type that are associated with a designated source zone with sinks of the respective realtime data stream type that are associated with a designated sink zone.
  • Network nodes can establish respective presences in the zones of a virtual area.
  • network nodes associated with respective objects e.g., avatars representing the communicants operating the network nodes
  • the area service administers data stream connections between the network nodes based on the switching rules, the respective sources and sinks associated with the network nodes, and the respective zones of the virtual area in which the objects are located.
  • FIG. 2 shows an exemplary graphical user interface 270 that is generated by an example of the communications application 26 in a window 259 on a display of the client network node from which a user of the client application (“Art” in this example) is operating.
  • the graphical user interface 270 includes a people panel 265 , a viewer panel 266 , a people interaction toolbar 267 , an audio interaction toolbar 268 , and a set of panel view controls 269 .
  • the panel view controls 269 include a people panel button 271 for opening and closing the people panel 265 , a chat panel button 273 for opening and closing a chat panel, and a viewer panel button 275 for opening and closing the viewer panel 266 .
  • the people panel 265 depicts the realtime availabilities and activities of some or all of Art's contacts across different communication contexts.
  • the people panel 265 shows Art's communicants segmented into a virtual area section 272 and a contacts group section 276 .
  • the virtual area section 272 shows the presence and activity states in each of the area applications 44 of which Art is a member with respect to which at least one of Art and Art's contacts is present.
  • the contacts group section 276 contains all or a selected portion of Art's contacts that are not represented in any of the virtual area sections. In the example shown in FIG.
  • the virtual area section 272 includes a section 278 corresponding to the user's current zone of presence (i.e., the Main zone in the SococoHQ virtual area) and a section (labeled with a header bar 279 entitled “SococoHQ”) that identifies all the communicants who have a presence in the area application “SococoHQ,” and contains the graphical representations of the communicants (including at least one of Art or Art's contacts) who currently have presence in the SococoHQ area application 44 and are not in the current zone of presence.
  • the contacts group section 276 is labeled with a header bar entitled “Contacts” and contains graphical representations of some or all of Art's contacts who are not shown in the virtual area section 272 (i.e., they either are not members of or not present in SococoHQ).
  • Each communicant is represented graphically by a respective circular sprite that is labeled with a respective user name of the communicant (i.e., “Art,” “Beth,” “Carl,” “Dan,” “Ed,” “Fran,” and “Garth”).
  • Each sprite also may be associated with a respective status line that includes additional information about the communicant.
  • each status line can include one or more of the following information: location of presence (e.g., a virtual area application or a zone of that sever application); availability (e.g., busy, idle); a status message (e.g., “Out of the office next Wednesday”); and the name of the client node from which the communicant is operating (e.g., “workstation 1” or “mobile phone”).
  • location of presence e.g., a virtual area application or a zone of that sever application
  • availability e.g., busy, idle
  • a status message e.g., “Out of the office next Wednesday”
  • the name of the client node from which the communicant is operating e.g., “workstation 1” or “mobile phone”.
  • the ordering of the spatial positions (e.g., from top to bottom) of the communicant avatars in each of the sections 272 , 276 is alphabetical by user name.
  • the spatial positions of the communicant avatars in the area application section 272 are ordered in accordance with the temporal ordering of the communicants in terms of the times when the communicants established their presences in the area application.
  • the spatial positions of the communicant avatars in the contacts section 276 may be sorted alphabetically by user name, according to frequency of contact, according to recentness of contact, or according to other sorting or filtering criteria.
  • the activities of the communicants in the contexts of the area applications 44 may be inferred by the client communications application from activities on various communication channels over which the respective communicants are configured to communicate.
  • the activities on the communication channel are represented in the graphical user interface 270 by visual cues that are depicted in association with the graphical representations of the communicants in the virtual area section 272 and the contacts section 276 .
  • the “on” or “off” state of a communicant's local speaker channel is depicted by the presence or absence of a headphones graphic 290 on the communicant's sprite.
  • the headphones graphic 290 When a communicant's speakers are on, the headphones graphic 290 is present (see sprites Art, Carl, and Dan) and, when a communicant's speakers are off, the headphones graphic 290 is absent (see sprites Beth and Ed).
  • the “on” or “off” state of the communicant's microphone is depicted by the presence or absence of a microphone graphic 292 on the communicant's sprite.
  • the microphone graphic 292 When a communicant's microphone is on, the microphone graphic 292 is present (see sprite Dan); and, when a communicant's microphone is off, the microphone graphic 292 is absent (see sprites Art, Beth, Carl, and Ed).
  • the headphones graphic 290 and the microphone graphic 292 provide visual cues of the activity states of the communicant's sound playback and microphone devices.
  • the current activity state of a communicant's microphone channel is indicated by a dynamic visualization that lightens and darkens the communicant's avatar in realtime to reflect the presence or absence of audio data on the microphone channel.
  • communicants can determine when another communicant is speaking by the “blinking” of the coloration of that communicant's avatar.
  • the activity state of a communicant's text chat channel is depicted by the presence or absence of the hand graphic 294 adjacent the communicant's sprite (see sprite Beth).
  • text chat data is transmitted only when keyboard keys are depressed, in which case the visualization of the communicant's text channel appears as a flashing on and off of the hand graphic 294 .
  • the graphical user interface 270 that is presented to Art can show visual cues indicating the communication channel activity states of the communicants who are present in the SococoHQ virtual area (where Art is present) and the communication channel states of the communicants who are present in other areas of which Art is a member but not currently present.
  • the audio interaction toolbar 268 includes a headphone control 284 that enables Art to toggle on and off the local speakers of the client network node, and a microphone control 286 that enables Art to toggle on and off the local microphone of the client network node.
  • the people interaction toolbar 267 includes a Chat button 298 and a Get button 302 .
  • Selection of the Chat button 298 opens a Chat panel 340 (see FIG. 5 ) that enables Art to initiate a chat with other communicants who are present in the area application where Art is present (i.e., SococoHQ in the illustrated example).
  • Selection of the Get button 302 opens an Invite window that enables Art to invite one or more communicants to a selected virtual area location (e.g., an area application or zone within that area application). Additional details regarding embodiments of the methods and functions invoked by the Chat button 298 and the Get button 302 are described in U.S. patent application Ser. No. 12/354,709, filed Jan. 15, 2009, and U.S. Provisional Patent Application No. 61/373,914, filed Aug. 16, 2010.
  • the graphical user interface 270 includes a Phone button 304 , a respective View Screen button 306 , 308 , 310 , 312 for each viewscreen in the current zone of presence, a Map button 314 , and a meeting button 316 .
  • the Phone button 304 is associated with telephony related functionality of the platform that enables a user of a Public Switched Telephone Network (PSTN) terminal device to participate in virtual area based communications (e.g., by the PSTN terminal device user calling into a zone of the virtual area or by a user of the communications application 26 to call out to the PSTN terminal device user), as described in U.S. patent application Ser. No. 13/165,729, filed Jun. 21, 2011, and communicants involved in telephony communications are represented by avatars that are decorated with a graphical representation of a telephone (see, e.g., the avatar 113 in Carl's Office shown in FIG. 3 ).
  • PSTN Public Switched Telephone Network
  • Each of the View Screen buttons 306 - 312 sets the viewer panel 266 to display the content the content being shared in connection with a corresponding one of the viewscreen objects in the current zone of presence or, if no content is being shared in connection with the current viewscreen object, to display a Share button that allows the user to initiate a screen sharing session in connection with the selected view screen object.
  • the Map button 314 sets the view presented in the viewer panel 266 to a map view of the virtual area.
  • the Meeting button 316 sets the view presented in the viewer panel 266 to a meeting user interface for visualizing assemblies of communicants in the virtual area, as described in U.S. patent application Ser. No. 13/432,837, filed Mar. 28, 2012.
  • the graphical user interface 270 also includes a home button 285 that is associated with a control that returns the user's presence to a designated “home” location in the virtual environment (e.g., a designed zone, such as a personal zone or other office that is assigned to the user). Additional details regarding the structure, function, and operation of examples of the navigation controls are described in U.S. patent application Ser. No. 12/354,709, filed Jan. 15, 2009.
  • FIG. 2 shows an example of the graphical user interface 270 in the Map view mode, which presents in the viewer panel 266 a rendered view of the user's current zone of presence (Main) of the SococoHQ virtual area, which is located at the location SococoHQ/Main as indicated in the location bar 318 .
  • This address indicates that Main is a zone within the area SococoHQ.
  • Each of the communicants who is present in the virtual area is represented graphically in the viewer panel 266 by a respective avatar that corresponds to the communicant's avatar shown in the people panel 265 .
  • the virtual area is represented graphically by a two-dimensional top view of a rectangular space.
  • the communicants' sprites automatically are positioned in predetermined locations (or “seats”) in the virtual area when the communicants initially enter the virtual area.
  • the Main zone includes four view screen props (or “objects”) 388 , 390 , 392 , 394 and a table prop 396 .
  • Communicants interact with the props by selecting them with an input device (e.g., by single-clicking on the props with a computer mouse, touch pad, touch screen, or the like).
  • the view screen props 388 - 394 are associated with application sharing functionality of the platform that enables communicants to share applications operating their respective client network nodes.
  • the application sharing functionality is invoked by activating a view screen (e.g., by single-clicking the view screen object with an input device).
  • the platform provides visual cues that indicate whether or not a communicant is sharing an application over an application sharing channel.
  • the communicant's sprite In response to a communicant's selection of the view screen prop, the communicant's sprite automatically is moved to a position in the graphical representation of the virtual area that is adjacent the view screen prop.
  • the position of a communicant's sprite adjacent the view screen prop indicates that the communicant currently is sharing or is about to share an application with the other communicants in the virtual area.
  • the avatar of each communicant who is viewing a shared application is depicted with a pair of “eyes” to indicate that the represented communicants are viewing the content being shared in connection with the view screen props (see, e.g., the avatars of Art and Dan in FIG. 2 ).
  • the graphical depiction of view screen prop is changed depending on whether or not an active application sharing session is occurring.
  • the depicted color of the view screen may change from a brighter color during an active application sharing session to a darker color when there is no application sharing taking place. Examples of the application sharing process are described in connection with FIGS. 26-28 of U.S. patent application Ser. No. 12/354,709, filed Jan. 15, 2009, and in U.S. patent application Ser. No. 12/418,270, filed Apr. 3, 2009.
  • one or more of the viewscreen props 388 - 394 may be associated with respective uniform resource identifiers (URIs) of network resources to enable communicants to interact with and share information associated with the network resources via the application sharing (e.g., web browser sharing) functionality associated with the viewscreen objects as described in U.S. Provisional Patent Application No. 61/444,989, filed Feb. 21, 2011.
  • URIs uniform resource identifiers
  • the table prop 396 is associated with file share functionality of the platform that enables communicants to upload computer data files to server storage in association with the virtual area and to download data files that are associated with the virtual area from the server storage to the respective client network nodes.
  • the document objects 398 , 400 are linked to respective documents that are have been shared in the virtual area and stored in server storage. Any of the document objects 398 , 400 may be selected by a communicant (e.g., by double-clicking the document object 398 with an input device, such as a computer mouse) to initiate downloading of the associated document to the communicant's client network node. Additional details regarding the structure, function, and operation of the table prop 396 may be obtained from U.S.
  • the virtual area platform 18 enables a wide variety of highly customizable virtual area applications to be created. Examples of such applications include virtual area applications for creating a virtual office, a virtual personal space, a virtual art gallery, a virtual concert hall, a virtual auditorium, a virtual conference room, and a virtual clubhouse.
  • the virtual area platform 18 supports the creation of network connections between network nodes in the same zone of a virtual area, as well as the creation of one-way or two-way data stream connections between network nodes in different zones.
  • a virtual area typically is associated with a specific set of communicants (e.g., members of the virtual area) and a plurality of persistent zones that define respective sub-contexts within the virtual area.
  • Each zone of a virtual area may support an independent communication session between the network nodes in the zone.
  • a virtual area may include zones in which audio, video, and text chat channel connections are established only between the sources and sinks of network nodes that are in the same zone.
  • one or more attributes or other data e.g., a name, a designated purpose, membership, or associated history of interaction in the zone
  • the visual and audio interfaces that are provided on the client network nodes are configured to present all the independent communication sessions that are occurring in the virtual area. This allows a user to visualize (through sight or imagination) multiple concurrent independent communication interactions and thereby quickly learn who is meeting with whom and the contexts of those meetings (as defined by the zones in which the meetings are occurring).
  • FIG. 3 shows an example of a two-dimensional visualization of the Sococo HQ virtual area 100 .
  • the Sococo HQ virtual area includes a lobby 102 , a Main conference room 104 , a West Conference room 106 , an East Conference room 108 , a West Nook zone 110 , an East Nook zone 112 , a Courtyard zone 114 , and sixteen offices.
  • Each of the conference rooms 104 - 108 includes respective viewscreen objects 116 - 122 and 388 - 304 , table objects 124 , 126 , and 396 , and respective telephone objects 128 , 130 , 132 and supports realtime audio, chat, and application and network resource sharing communications between the network nodes in the same conference room.
  • Each of the offices includes respective viewscreen objects (not shown) and a respective telephony object (not shown) and supports realtime audio, chat, and application and network resource sharing communications between the network nodes in the same office.
  • Each of the telephony objects supports shared dial-in and dial-out telephony communications as described in U.S. patent application Ser. No. 13/165,729, filed Jun. 21, 2011.
  • Each of the West Nook 110 , East Nook 112 , and Lobby 102 zones respectively supports realtime audio and chat communications between the network nodes in the same zone.
  • the communication properties of an example of the virtual area 100 are described in U.S. Provisional Patent Application No. 61/603,024, filed Feb. 24, 2012.
  • FIG. 4 shows and example of a virtual area 600 that includes zones and is associated with a visualization of an auditorium (“Sococo Auditorium”).
  • the virtual area 600 includes a stage (or podium) zone 602 , a general seating zone 604 , a question-and-answer zone 606 , a West Conference Room 608 , an East Conference Room 610 , and a Lobby 611 .
  • the stage zone 602 includes viewscreen objects 612 , 614 and speaker props 616 , 618 .
  • the general seating zone 604 includes a plurality of sub-zones 620 (referred to herein as “bench zones”) defining locations where network nodes are permitted to be present in the general seating zone 604 .
  • the question-and-answer zone 606 includes a microphone sub-zone 622 , a queuing sub-zone 624 , and a microphone prop 626 .
  • Each of the West Conference Room 608 and the East Conference Room 610 includes a respective viewscreen object 607 , 609 .
  • the communication properties of an example of the virtual area 600 are described in U.S. Provisional Patent Application No. 61/603,024, filed Feb. 24, 2012.
  • FIG. 5 shows an example 200 of the people panel 265 of the graphical user interface 270 that depicts the realtime availabilities and activities of some or all of the contacts of a target communicant (“Art” in this example) across the different communication contexts provided by the Sococo HQ virtual area 100 and the Sococo Auditorium virtual area 600 .
  • the people panel 200 includes a separate section for each of the areas of which Art is a member and a separate Contacts section. Art's current area of presence (“Sococo HQ”) is shown at the top.
  • the people panel 265 shows Art's current zone of presence (“Art's Office”) at the top, followed by the zones of co-presence (“Main,” Paul's Office,” and “West Conference”).
  • the Zone of Co-Presence filtering criteria identifies those communicants who are co-present within a particular zone of a virtual area application.
  • each group of two or more communicants who are present within the same zone of the virtual area application is listed in a separate, selectively expandable and collapsible Zone of Co-Presence area 130 of the graphical user interface 128 .
  • the members of the area application are segmented by the status of their presence with respect to the area application.
  • the members of Sococo HQ are sorted into a Present group 202 and a Not Present group 204 .
  • the members of Sococo Auditorium also are sorted into a Present group and a Not Present group.
  • Each of the groups 202 - 204 are labeled with a respective “Present” or “Not Present” label that is associated with a respective toggle control that can be toggled to selectively collapse and expand the associated group 202 - 204 .
  • Additional filtering criteria include a Frequent contacts group 206 in which Art's contacts are sorted by the frequency of their respective interactions with Art, a Recent contacts group 208 in which Art's contacts are sorted by the recentness of their respective interactions with Art, an Online contacts section 212 that lists all of Art's contacts who currently are online (i.e., connected to the network 20 ), and an Offline contacts group 214 that lists all of Art's contacts who currently are offline (i.e., disconnected from the network).
  • FIG. 6 shows an embodiment of a graphical user interface 220 that depicts the realtime availabilities and activities of some or all of the contacts of a target communicant (“Art” in this example) across different communication contexts defined by a data file A and a software application B.
  • the software application B may be any type of software application, including consumer and business software applications (e.g., a personal information manager software application, such as the Microsoft® Outlook® software application; a document processing software application, such as the Microsoft® Word® software application; a spreadsheet software application, such as the Microsoft® Excel® software application, and a web browser software application, such as the Microsoft® Internet Explorer® software application).
  • the members of the virtual area application are segmented by the status of their presence with respect to the data file or software application.
  • Art is member of Data File A and Software Application B.
  • the members of Data File A are sorted into a Present group 222 and a Not Present group 224
  • the members of Software Application B are sorted into a “Zone of Co-Presence” group 226 , a Present group 228 and a Not Present group 230 , where the zones correspond to respective sections of the Data File A and the Software Application B.
  • Art readily can visualize who is present and which members are not present in each of the data files and software applications of which he is a member.
  • the examples that are described herein provide systems and methods for voice-based navigation in one or more virtual areas that enable communicants to exploit the communication opportunities that are available in such virtual area contexts, even when hands-based or visual methods of accessing such opportunities are not available.
  • the voice-based navigation functionalities enable a user to obtain sufficient information to visualize in his or her mind essential aspects of the visual information shown in the graphical user interfaces described above in connection with FIGS. 2-6 .
  • a user can navigate multiple multi-zone virtual areas to determine what is going on in a particular virtual area or zone (e.g., where are communicants located and what are they doing—talking, text chatting, application sharing, communicating over a telephony network, or simply co-present with another communicants), or determine what a specific communicant is doing (e.g., is the communicant online or offline, where the communicant present, and what is the communicant currently doing).
  • Among the statuses that a user of the voice-based navigation system can ascertain about a communicant are statuses of the states and activities of the communicant in one or more of the virtual area applications 46 .
  • Exemplary communicant states include:
  • Audio Sink e.g., Speaker
  • Audio Source e.g., Microphone
  • Video Sink e.g., Display
  • Video Source e.g., Screen Share
  • Video Source e.g., Webcam
  • File Transfer Client of Focus Which network Client Name e.g., Sococo, service is Yahoo!, Windows Live communicant Messenger, Google Talk, currently using? Facebook, etc.
  • Client Version Station of Focus Which client Operating System e.g., XP, network node is Vista, Windows 7, MacOS, communicant iPhone, Windows currently using? Mobile, etc.
  • OS Version Device Type e.g., Desktop Computer, Smart Phone, Phone, etc.
  • the Online/Offline state indicates whether or not the communicant currently is available for communication.
  • a communicant is in the Online state when the communicant is connected to the network communication environment (e.g., by establishing a presence with respect to one or more of the virtual area applications 46 ), and the communicant is in the Offline state when the communicant is unconnected to the network communication environment (e.g., by not having a presence with respect to any of the virtual area applications 46 ).
  • the Location of Presence state indicates where the communicant has established a presence in the network communication environment.
  • the Location of Presence typically has a respective attribute value (e.g., an application_ID and/or a zone_ID value) for each of the virtual area applications 44 and/or zones of the virtual area applications 46 in which the user has presence.
  • the Application-Specific Identity Information is communicated to other communicants to indicate information about the communicant in the context of a particular virtual area application with which the communicant is associated.
  • the Application-Specific Identity Information includes:
  • the Application-Specific Availability state indicates the availability of the communicant in the context of the virtual area application.
  • the Application-Specific Availability state typically includes an attribute value that indicates when the communicant is idle or sleeping and an attribute value that indicates when the communicant is busy.
  • the Station of Focus state characterizes the client network node from which the communicant is operating.
  • the Station of Focus state typically includes attributes that identify the Device Type of the client network node, the Operating System (OS) running on the client network node, and the OS Version of the operating system.
  • OS Operating System
  • the Client of Focus state indicates the virtual area application with which the communicant currently has focus.
  • the Client of Focus state typically includes a Client Name attribute that identifies client communication application with which the communicant is communicating with the virtual area application, and a Client Version attribute that identifies the version of the client communication application.
  • the Channels Supported by Client of Focus state indicates the communication channels that are available for communicating with the communicant in the context of the user's client application.
  • the Channels Supported by Client of Focus state typically includes a Chat attribute that indicates whether or not the user's client application supports chat communications with the communicant, Audio Sink and Audio Source attributes that respectively indicate whether or not the user's client application supports audio sink and audio source communications with the communicant, Video Source and Video Sink attributes that respectively indicate whether or not the user's client application supports video sink and video source communications with the communicant, and File Transfer attribute that indicates whether or not the user's client application supports file transfer communications with the communicant.
  • Exemplary communicant activities include the channels that the communicant currently has open for communication, which indicates how the communicant currently is interacting with other communicants.
  • the activities that a user of the voice-based navigation system can ascertain about a communicant include the following attributes of an Open Channels object:
  • FIG. 7 shows an example of a voice-based virtual area navigation architecture 700 that includes a speech recognizer 702 , a virtual area navigation command interpreter 704 , an area service 706 , a navigation results translator 708 , and a speech synthesizer 710 .
  • a voice-based virtual area navigation session begins with the detection of a distinctive sound pattern or phrase that is recognized by the speech recognizer 702 as a command to initiate the command recognition process.
  • the command recognition process is initiated in response to receipt of a distinctive nonlinguistic vocal sound (e.g., one or more tongue clicking sounds).
  • the speech recognizer 702 converts voice input 712 into grammatically correct text in accordance with constraints that are imposed by a virtual area navigation vocabulary and a language model 714 .
  • the speech recognizer 702 generates recognized output (e.g., text) that represents the most likely words that were uttered in the voice input 712 based on the vocabulary and grammar 714 .
  • the speech recognizer 702 may include one or more of a continuous speech recognizer that attempts to recognize fluently spoken speech in the voice input 712 and an isolated word recognizer that attempts to detect commands (also referred to as actions) and operands (also referred to as arguments or modifiers) by focusing on keywords in the voice input 712 .
  • the vocabulary specifies the set of words that can be recognized by the speech recognizer 702 .
  • Some examples use a small vocabulary of commands and operands without an explicit grammar (e.g., all word sequences are possible), whereas other examples may use larger vocabularies with word grammars that impose different levels of constraint on the types of word sequences that are allowed.
  • the speech recognizer 702 uses a vocabulary that includes a small set of fixed commands and a set of operands that is configured dynamically by the area service 706 based on data associated with the user.
  • the set of operands are constrained to the names of the virtual areas, virtual area zones, or contacts that are associated with the user.
  • the operands are not limited to the values that are associated with the user. In some of these examples, however, the speech recognizer 702 preferentially selects the operand values that are associated with the user.
  • the operands may consist of all or a subset of all the values that are possible in the virtual communications environment 10 (e.g., the names of all of the virtual areas, virtual area zones, and communicants that are associated with the virtual area platform 18 ), but the values that are associated with the user are assigned higher probabilities in the lexical decoding process that is used to estimate recognized word probabilities.
  • the language model specifies which combinations of words (e.g., commands and operands) are allowable.
  • the language model specifies a syntax (e.g., the navigation command syntax described below) that defines the combinations of words that form grammatically correct strings (e.g., commands and their respective operands).
  • a syntax e.g., the navigation command syntax described below
  • syntactically correct word combination hypotheses are allowed, whereas syntactically incorrect word combination hypotheses are disallowed.
  • the allowable syntactically correct word combinations are assigned respective probability values based on information about he user (e.g., the user's current status—such as state and activity, as described above—in one or more virtual areas, the user's current set of capabilities in the virtual area as defined by the permission rules, the communicants in the user's contact list, and attributes associated with the user).
  • the user's current status such as state and activity, as described above—in one or more virtual areas, the user's current set of capabilities in the virtual area as defined by the permission rules, the communicants in the user's contact list, and attributes associated with the user.
  • the virtual area navigation command interpreter 704 uses statistical or rules-based methods to extract one or more virtual area navigation commands and zero of more operands from the recognized output based on a set of semantic rules 716 .
  • the virtual area navigation command interpreter uses the semantic rules 716 to validate the recognized output received from the speech recognizer 702 .
  • the semantic rules 716 ensure that the recognized output is consistent with the task being performed or the user's state.
  • the semantic rules 716 specify how the likelihoods of certain combinations of commands and operands should be modified based on information about external conditions, facts, or elements of the network communications environment associated with the user and the virtual area application in relation to the semantic values of those combinations of commands and operands.
  • the semantic rules 716 assign low probabilities to commands or command/operand combinations that are not allowed based on the status of the user at the time the command or command/operand combination was received. For example, the semantic rules may dynamically assign a lower probability to a command to Go to a virtual area or zone of which the user is not a member or for which the user does not have the requisite capability than it assigns to a command to Go to a virtual area or zone of which the user is a member or has the requisite capability. In addition, the semantic rules may assign a lower probability to a command to GET or FIND a communicant that is not a contact of the user than it assigns to a command to GET or FIND a communicant that is a contact of the user. The extracted commands and operands are passed to the area service 706
  • the area service 706 is a component of the network infrastructure service environment.
  • the area service 706 administers virtual areas, manages network connections that are associated with the virtual area subject to the capabilities of the requesting entities, maintains global state information for the virtual area, and serves as a data server for the client network nodes participating in a shared communication session in a context defined by the virtual area.
  • the area service 706 provides the context-dependent information to the speech recognizer 702 and the virtual area navigation command interpreter 704 .
  • This information includes, for example, user-specific data 730 (e.g., the user's contact list information 722 and information 720 that relates to the areas/zones that are associated to the user) and user status data 732 (e.g., the current state and activity of the user).
  • the area service 706 also executes the tasks corresponding to the extracted virtual area navigation commands and operands; in this process, navigation results are generated (e.g., by one or more of the area service, other components of the network infrastructure service, or other network entities).
  • the navigation results are passed to a navigation results translator 708 , which formats the navigation results into a format that can be converted into audio output 718 (e.g., a synthesized speech data that the client network node 12 can render) by the speech synthesizer 710 .
  • audio output 718 e.g., a synthesized speech data that the client network node 12 can render
  • the components 702 - 710 of the architecture 700 may be allocated to the client network node 12 and the virtual area platform 18 in different ways. In some examples, all of the components 702 - 710 of the voice-based virtual area navigation architecture 700 are allocated to the virtual area platform 18 ; the client network node 12 sends recorded voice files to the virtual area platform 18 for processing and receives in return sound files containing synthesized speech report of the navigation results.
  • the client network node 12 performs front-end signal processing (e.g., speech detection, such as voice activity detection and speech boundary detection, and feature extraction, such as MFCC (Mel-Frequency Cepstral Coefficient) feature extraction) on the voice input and the remaining functions of the speech recognizer 702 , as well as the functions of the other components 704 - 710 of the voice-based virtual area navigation architecture 700 , are performed by the virtual area platform 18 .
  • the speech recognizer 702 is allocated to the client network node 12 and the other components 704 - 710 of the voice-based virtual area navigation architecture 700 are allocated to the virtual area platform.
  • LOOK optionally can be modified by: a zone name (e.g., “LOOK Main”), which reports who is in the specified zone in the current virtual area in which the user is present; or an area/zone pair (e.g., “LOOK Sococo Corporate Lobby”), which reports who else in the specified zone in the specified virtual area.
  • a zone name e.g., “LOOK Main”
  • an area/zone pair e.g., “LOOK Sococo Corporate Lobby”
  • connector words such as “in” or “of” are accepted in the syntax, although anything that requires true parsing rather than a comparison within a restricted vocabulary will lower the recognition success rate.
  • the LOOK command additionally reports one or more attributes of the status of each of the communicants in the pertinent zone (e.g., is the communicant currently available, which channels (modes or channels of communication) can be used to communicate with the communicant, and what is communicant currently doing—e.g., is the communicant currently interacting with other communicants and, if so, how).
  • attributes of the status of each of the communicants in the pertinent zone e.g., is the communicant currently available, which channels (modes or channels of communication) can be used to communicate with the communicant, and what is communicant currently doing—e.g., is the communicant currently interacting with other communicants and, if so, how).
  • FIND with a specified contact operand reports the current zone, or virtual area and zone if in a different virtual area than the user's current virtual area of presence, of the specified contact.
  • FIND with a specified meeting name operand reports the zone or virtual area and zone if in a different virtual area than the user's current zone of presence in which a meeting having the specified meeting name is taking place.
  • the FIND command additionally reports one or more attributes of the status of each of the communicants in the pertinent zone (e.g., is the communicant currently available, which channels (modes or channels of communication) can be used to communicate with the communicant, and what is communicant currently doing—e.g., is the communicant currently interacting with other communicants and, if so, how).
  • attributes of the status of each of the communicants in the pertinent zone e.g., is the communicant currently available, which channels (modes or channels of communication) can be used to communicate with the communicant, and what is communicant currently doing—e.g., is the communicant currently interacting with other communicants and, if so, how).
  • GO moves the user's presence to either the zone of the specified contact operand (e.g., “GO Paul Brody”), a specified zone of the user's current virtual area of presence (e.g., “GO West Conference”), a default zone of a specified area (e.g., “GO Sococo Corporate” might move the user's presence to the Lobby of the Sococo Corporate virtual area), a specified zone of the specified virtual area (e.g., “GO Sococo Corporate Lobby”), or the user's home zone (e.g., “GO Home”), which is a zone of a virtual area that is designated as the user's home zone (e.g., the user's virtual office).
  • the zone of the specified contact operand e.g., “GO Paul Brody”
  • a specified zone of the user's current virtual area of presence e.g., “GO West Conference”
  • a default zone of a specified area e.g., “GO Sococo Corporate” might move the user's
  • the GO command additionally reports one or more attributes of the status of each of the communicants in the zone to which the user's presence has been moved (e.g., is the communicant currently available, which channels (modes or channels of communication) can be used to communicate with the communicant, and what is communicant currently doing—e.g., is the communicant currently interacting with other communicants and, if so, how).
  • attributes of the status of each of the communicants in the zone to which the user's presence has been moved e.g., is the communicant currently available, which channels (modes or channels of communication) can be used to communicate with the communicant, and what is communicant currently doing—e.g., is the communicant currently interacting with other communicants and, if so, how).
  • a service in response to receipt of a GET command, a service (e.g., the area service or another service of the network infrastructure service environment 42 ) asks the user whether a voice message should accompany the get message and, if the user gives an affirmative reply (e.g., by saying “YES”), the service prompts the user to speak a voice message, records the voice message, and includes the voice message with the get message that is sent to the specified contact.
  • the service instead of sending the voice message, the service translates the voice message into a text message using a speech-to-text processor and sends the text message with the get message that is sent to the specified contact.
  • YES indicates an affirmative response to an input choice.
  • NO indicates a negative response to an input choice.
  • FIG. 8 shows an example of a method by which the server network node 42 operates to provide voice-based virtual area navigation.
  • the server network node 42 establishes a respective presence of a user in a given zone of a virtual area in a virtual communications environment ( FIG. 8 , block 800 ).
  • the virtual area typically includes multiple zones, where each of respective ones of the zones defines a respective context for realtime communications between network nodes of respective communicants who are present in the zone.
  • the server network node 42 receives a representation of voice input in association with the user ( FIG. 8 , block 802 ).
  • the server network node 42 determines a virtual area navigation command based on the representation of voice input ( FIG. 8 , block 804 ).
  • the server network node 42 executes of the virtual area navigation command, which involves producing navigation results ( FIG. 8 , block 806 ).
  • the server network node 42 transmits the navigation results to a network node associated with the user ( FIG. 8 , block 808 ).
  • Examples of the method of FIG. 8 enable the user to use voice commands to search for communication opportunities in the different communications contexts that are available in one or more virtual areas, enter specific ones of such virtual communication contexts, and bring other communicants into specific ones of such virtual communication contexts.
  • the voice input representation that is received by the server network node 42 includes a sound file of speech recorded by the client network node 12 .
  • the voice input representation is a feature-based representation of speech that is produced by the client network node 12 .
  • the voice input includes a text-based representation of speech that was recognized by the client network node 12 .
  • the server network node 42 in the process of determining the virtual area navigation command ( FIG. 8 , block 804 ), the server network node 42 ascertains one or more constraints for recognizing words in the representation of voice input based on data associated with the user, and determines the virtual area navigation command based on the one or more constraints and the representation of voice input.
  • the process of ascertaining the one or more constraints involves determining a restricted vocabulary of words (to which the recognition of words in the representation of voice input is limited) based on the data associated with the user. In some examples, this process involves including names of virtual areas associated with the user in the vocabulary. In some examples, this process involves including names of virtual area zones associated with the user in the vocabulary. In some examples, the process involves including names of communicants associated with the user in the vocabulary.
  • the process of ascertaining the one or more constraints involves determining a language model for recognizing words in the representation of voice input based on the data associated with the user.
  • the determined language model includes a specification of allowable combinations of virtual area names and zone names to which words in the representation of voice input are limited.
  • the virtual area navigation command involves navigating the user's attention in the virtual communications environment. Examples of such commands include the LOOK and FIND commands described above.
  • the virtual area navigation command is a communicant presence reporting command.
  • the navigation results include a list of communicants who are present in a particular one of the zones of the virtual area.
  • the navigation results include indications of activities being performed by respective ones of the communicants who are present the particular zone.
  • the navigation results also may include indications of which communicants in the zone are performing one or more of talking, text chatting, and application sharing in the particular zone.
  • the server network node 42 determines whether the representation of voice input indicates a respective one of the zones of the virtual area in connection with the communicant presence reporting command.
  • the server network node 42 executes the communicant presence reporting command with respect to the given zone in which the user currently is present. In response to a determination that the representation of voice input indicates a respective one of the zones of the virtual area in connection with the communicant presence reporting command, the server network node 42 executes the communicant presence reporting command with respect to the indicated zone.
  • the indicated zone may be different from the given zone in which the user currently is present.
  • the server network node 42 determines in the representation of voice input an indication of a particular zone of a second virtual area of multiple zones in the virtual communications environment. The user may or may not be present in the second virtual area.
  • the navigation results include a list of communicants who are present in the particular zone of the second virtual area.
  • the server network node 42 determines in the representation of voice input an indication of a particular communicant. In some of these examples, the server network node 42 executes the communicant presence reporting command with respect to the particular communicant, and the navigation results include an indication of a zone in which the particular communicant is present. In some examples, the indicated zone is a zone of the virtual area in which the user currently is present. In some examples, the indicated zone is a zone of a second virtual area in which the user is not present.
  • the server network node 42 determines in the voice input an indication of a particular assembly of communicants in the virtual area. In some of these examples, the server network node 42 executes the communicant presence reporting command with respect to the particular assembly of communicants, and the navigation results include an indication of a zone in which the particular assembly of communicants is occurring.
  • the process of executing the virtual area navigation command involves navigating a communicant's presence in the virtual communications environment.
  • Examples of such commands include the GO and GET commands described above.
  • the virtual area navigation command is a user presence navigation command for navigating the user's presence in the virtual communications environment.
  • the server network node 42 determines in the voice input an indication of a respective one of the zones of the virtual area; the server network node executes the user presence navigation command with respect to the indicated zone, and in the process moves the user's presence to the indicated zone.
  • the navigation results may include an indication that the user's presence has been moved to the indicated zone.
  • the navigation results also may include a list of communicants who are present in the indicated zone.
  • the server network node 42 determines in the voice input a verbal connotation of a home in connection with the user presence navigation command. Based on this determination, the server network node 42 executes the user presence navigation command with respect to a particular one of the zones of the virtual area that is designated as the user's home zone, and in the process moves the user's presence to the particular zone.
  • the navigation results may include an indication that the user's presence has moved to the particular zone.
  • the server network node 42 determines in the voice input an indication of a particular zone of a second virtual area of multiple zones in the virtual communications environment in connection with the user presence navigation command; the server network node 42 executes the user presence navigation command with respect to the indicated zone, and in the process moves the user's presence to the indicated zone.
  • the user may or may not be present in the second virtual area.
  • the navigation results may include an indication that the user's presence has moved to the indicated zone.
  • the server network node 42 determines in the voice input an indication of a particular communicant in connection with the user presence navigation command. In some of these examples, the server network node 42 executes the user presence navigation command with respect to the particular communicant, and in the process moves the user's presence to a particular zone in which the particular communicant currently is present.
  • the navigation results typically include an indication that the user's presence has moved to the particular zone.
  • the virtual area navigation command is a contact presence navigation command for navigating the presence of a particular one of the user's contacts to a location (e.g., a zone) in the virtual communication environment.
  • the server network node 42 determines in the voice input an indication of a particular contact of the user. Based on this determination, the server network node 42 executes the contact presence navigation command with respect to the particular communicant contact.
  • the process of executing the contact presence navigation command involves inviting the particular communicant to the particular zone, in which case the navigation results may include an indication that the particular contact has been invited to the particular zone.
  • the server network node 42 determines whether the voice input indicates a respective one of the zones of the virtual area. In response to a determination that the voice input fails to indicate any of the zones of the virtual area in connection with the contact presence navigation command, the server network node 42 invites the particular contact to the given zone in which the user is present. In response to a determination that the voice input indicates a respective one of the zones of the virtual area in connection with the contact presence navigation command, the server network node 42 invites the particular contact to the indicated zone.
  • FIG. 9 shows an example of a method by which the client network node 12 provides voice-based virtual area navigation.
  • the client network node 12 establishes a respective presence of a user in a given zone of a virtual area in a virtual communications environment ( FIG. 9 , block 900 ).
  • the virtual area typically includes multiple zones, where each of respective ones of the zones defines a respective context for realtime communications between network nodes of respective communicants who are present in the zone.
  • the client network node 12 processes voice input ( FIG. 8 , block 802 ). Based at least in part on the voice input, the client network node 12 calls one or more services to perform a task involving navigation in the virtual area ( FIG. 9 , block 904 ).
  • the client network node 12 receives navigation results relating to at least one of the zones of the virtual area ( FIG. 9 , block 906 ).
  • the client network node 12 receives the navigation results from the server network node providing one or more of the services that are called to perform the determined task.
  • a remote server network node provides at least one of the one or more services, where the client network node 12 receives the navigation results from the remote server network node.
  • the client network node 12 generates audio output that includes an audio report of the navigation results ( FIG. 9 , block 908 ).
  • the processing of the voice input ( FIG. 9 , block 904 ) is initiated in response to receipt of a distinctive nonlinguistic vocal sound (e.g., a tongue clicking sound).
  • the processing involves recording input speech into a sound file and sending the sound file to one or more of the called services.
  • the processing involves generating a feature-based representation of input speech and sending the feature-based representation to one or more of the called services.
  • the processing involves recognizing commands and parameters in the voice input and sending the recognized commands and parameters to one or more of the called services.
  • a remote server network node (e.g., the server network node 40 ) provides at least one of the one or more services for performing the determined task ( FIG. 9 , block 904 ).
  • the client network node 12 based at least in part on the voice input, ascertains a command that involves navigating the user's attention in the virtual communications environment. Examples of such commands include the LOOK and FIND commands described above.
  • the ascertained command is a communicant presence reporting command.
  • the navigation results include a list of communicants who are present in a particular one of the zones of the virtual area, in which case the generated audio output ( FIG. 9 , block 908 ) includes an audio report of the communicants who are present in the particular zone.
  • the navigation results also may include indications of activities being performed by respective ones of the communicants present the particular zone, in which case the generated audio output comprises an audio reports of the activities.
  • the navigation results also may include indications of which communicants in the zone are performing one or more of talking, text chatting, and application sharing in the particular zone.
  • the client network node 12 determines in the voice input an indication of a particular zone of a second virtual area of multiple zones in the virtual communications environment in connection with the communicant presence reporting command.
  • the user may or may not be present in the second virtual area.
  • the navigation results may include a list of communicants who are present in the particular zone of the second virtual area, in which case the generated audio output includes an audio report of the communicants who are present in the particular zone of the second virtual area.
  • the client network node 12 determines in the voice input an indication of a particular communicant in connection with the communicant presence reporting command. In some of these examples, the determined task includes executing the communicant presence reporting command with respect to the particular communicant. In some of these examples, the navigation results include an indication of a zone in which the particular communicant is present, and the generated audio output includes an audio report relating to the indicated zone.
  • the client network node 12 determines in the voice input an indication of a particular assembly of communicants in the virtual area. In some of these examples, the determined task includes executing the communicant presence reporting command with respect to the particular assembly of communicants. In some of these examples, the navigation results include an indication of a zone in which the particular assembly of communicants is occurring, and the generated audio output includes an audio report relating to the indicated zone.
  • the client network node 12 based at least in part on the voice input, ascertains a command that involves navigating a communicant's presence in the virtual communications environment. Examples of such commands include the GO and GET commands described above.
  • the ascertained command is a user presence navigation command for navigating the user's presence in the virtual communications environment.
  • the client network node 12 determines in the voice input an indication of a respective one of the zones of the virtual area in connection with the user presence navigation command.
  • the determined task includes executing the user presence navigation command with respect to the indicated zone.
  • the navigation results may include an indication that the user's presence has moved to the indicated zone, in which case the generated audio output comprises an audio report that the user has entered the indicated zone.
  • the navigation results may include a list of communicants who are present in the indicated zone, in which case the generated audio output comprises an audio report of the communicants who are present in the indicated zone.
  • the client network node 12 determines in the voice input a verbal connotation of a home in connection with the user presence navigation command, in which case the determined task includes executing the user presence navigation command with respect to a respective one of the zones of the virtual area that is designated as the user's home zone.
  • the client network node 12 determines in the voice input an indication of a particular zone of a second virtual area of multiple zones in the virtual communications environment in connection with the user presence navigation command, where the user is not present in the second virtual area.
  • the determined task includes executing the user presence navigation command with respect to the indicated zone.
  • the navigation results include an indication that the user's presence has moved to the indicated zone, in which case the generated audio output include an audio report that the user has entered the indicated zone.
  • the client network node 12 determines in the voice input an indication of a particular communicant in connection with the user presence navigation command. In some of these examples, the determined task includes executing the user presence navigation command with respect to the particular communicant. In some of these examples, the navigation results include an indication that the user's presence has moved to a zone in which the particular communicant is present, in which case the generated audio output include an audio report that the user has entered the indicated zone.
  • the ascertained command is a contact presence navigation command for navigating the presence of a respective one of the user contacts in the virtual communication environment.
  • the client network node determines in the voice input an indication of a particular contact of the user, in which case the determined task includes executing the contact presence navigation command with respect to the particular contact.
  • the navigation results include an indication that the particular contact has been invited to a particular zone of the virtual area, in which case the generated audio output comprises an audio report that the particular contact has been invited to a particular zone of the virtual area.
  • the client network node 12 determines whether the voice input indicates a respective one of the zones of the virtual area in connection with the contact presence navigation command. In response to a determination that the voice input fails to indicate any of the zones of the virtual area in connection with the contact presence navigation command, the determined task includes executing the contact presence navigation command with respect to the particular contact and the given zone in which the user is present.
  • the navigation results comprise an indication that the particular communicant contact has been invited to the given zone, in which case the generated audio output includes an audio report that the particular contact has been invited to the given zone.
  • the determined task includes executing the contact presence navigation command with respect to the particular contact and the indicated zone.
  • the navigation results include an indication that the particular contact has been invited the indicated zone, in which case the generated audio output includes an audio report that the particular contact has been invited to the indicated zone.

Abstract

Examples of systems and methods for voice-based navigation in one or more virtual areas that define respective persistent virtual communication contexts are described. These examples enable communicants to use voice commands to, for example, search for communication opportunities in the different virtual communication contexts, enter specific ones of the virtual communication contexts, and bring other communicants into specific ones of the virtual communication contexts. In this way, these examples allow communicants to exploit the communication opportunities that are available in virtual areas, even when hands-based or visual methods of interfacing with the virtual areas are not available.

Description

    CROSS-REFERENCE TO RELATED APPLICATIONS
  • Under 35 U.S.C. §119(e), this application claims the benefit of U.S. Provisional Application No. 61/637,461, filed Apr. 24, 2012, the entirety of which is incorporated herein by reference.
  • This application relates to the following co-pending patent applications, the entirety of each of which is incorporated herein by reference:
    • U.S. application Ser. No. 12/354,709, filed Jan. 15, 2009;
    • U.S. application Ser. No. 12/418,243, filed Apr. 3, 2009;
    • U.S. application Ser. No. 12/509,658, filed Jul. 27, 2009;
    • U.S. application Ser. No. 12/630,973, filed Dec. 4, 2009;
    • U.S. patent application Ser. No. 12/631,026, filed Dec. 4, 2009;
    • U.S. application Ser. No. 12/818,517, filed Jun. 18, 2010;
    • U.S. application Ser. No. 12/825,512, filed Jun. 29, 2010;
    • U.S. application Ser. No. 12/855,210, filed Aug. 12, 2010;
    • U.S. application Ser. No. 13/209,812, filed Aug. 15, 2011;
    • U.S. application Ser. No. 13/229,349, filed Sep. 9, 2011;
    • U.S. application Ser. No. 13/432,837, filed Mar. 28, 2012;
    • U.S. application Ser. No. 13/604,360, filed Sep. 5, 2012;
    • U.S. application Ser. No. 13/680,463, filed Nov. 19, 2012; and
    • U.S. application Ser. No. 13/770,761, filed Feb. 19, 2013.
    BACKGROUND
  • When face-to-face communications are not practical, people often rely on one or more technological solutions to meet their communications needs. Traditional telephony systems enable voice communications between callers. Instant messaging (also referred to as “chat”) communications systems enable users to communicate text messages in real time through instant message computer clients that are interconnected by an instant message server. Some instant messaging systems and interactive virtual reality communications systems allow users to be represented by user-controllable graphical objects (referred to as “avatars”). What are needed are improved systems and methods for realtime network communications.
  • DESCRIPTION OF DRAWINGS
  • FIG. 1 is a diagrammatic view of an example of a network communications environment.
  • FIG. 2 is a diagrammatic view of an example of a graphical user interface.
  • FIG. 3 is a diagrammatic view of an example of a graphical user interface.
  • FIG. 4 is a diagrammatic view of an example of a graphical user interface.
  • FIG. 5 is a diagrammatic view of an example of a graphical user interface.
  • FIG. 6 is a diagrammatic view of an example of a graphical user interface.
  • FIG. 7 is a block diagram of an example of a voice-based virtual area navigation architecture.
  • FIG. 8 is a flow diagram of an example of a voice-based virtual area navigation method.
  • FIG. 9 is a flow diagram of an example of a voice-based virtual area navigation method.
  • DETAILED DESCRIPTION
  • In the following description, like reference numbers are used to identify like elements. Furthermore, the drawings are intended to illustrate major features of exemplary embodiments in a diagrammatic manner. The drawings are not intended to depict every feature of actual embodiments nor relative dimensions of the depicted elements, and are not drawn to scale.
  • I. DEFINITION OF TERMS
  • A “communicant” is a person who communicates or otherwise interacts with other persons over one or more network connections, where the communication or interaction may or may not occur in the context of a virtual area. A “user” is a communicant who is operating a particular network node that defines a particular perspective for descriptive purposes.
  • A “computer” is any machine, device, or apparatus that processes data according to computer-readable instructions that are stored on a computer-readable medium either temporarily or permanently. A “computer operating system” is a software component of a computer system that manages and coordinates the performance of tasks and the sharing of computing and hardware resources. A “software application” (also referred to as software, an application, computer software, a computer application, a program, and a computer program) is a set of instructions that a computer can interpret and execute to perform one or more specific tasks. A “data file” is a block of information that durably stores data for use by a software application.
  • The term “computer-readable medium” refers to any tangible, non-transitory medium capable storing information (e.g., instructions and data) that is readable by a machine (e.g., a computer). Storage devices suitable for tangibly embodying such information include, but are not limited to, all forms of physical, non-transitory computer-readable memory, including, for example, semiconductor memory devices, such as random access memory (RAM), EPROM, EEPROM, and Flash memory devices, magnetic disks such as internal hard disks and removable hard disks, magneto-optical disks, DVD-ROM/RAM, and CD-ROM/RAM.
  • A “data sink” (referred to herein simply as a “sink”) is any of a device (e.g., a computer), part of a device, or software that receives data.
  • A “data source” (referred to herein simply as a “source”) is any of a device (e.g., a computer), part of a device, or software that originates data.
  • A “network node” (also referred to simply as a “node”) is a junction or connection point in a communications network. Examples of network nodes include, but are not limited to, a terminal, a computer, and a network switch. A “server” network node is a host computer on a network that responds to requests for information or service. A “client network node” is a computer on a network that requests information or service from a server.
  • A Uniform Resource Identifier (URI) is a string of characters that identifies a network resource.
  • A “network resource” is anything that can be identified by a uniform resource identifier (URI) and accessed over a network, including an electronic document, an image, a source of information, a service, operators and operands of a mathematical equation, classes, properties, numeric values, and a collection of other resources.
  • A “network connection” is a link between two communicating network nodes. A “connection handle” is a pointer or identifier (e.g., a uniform resource identifier (URI)) that can be used to establish a network connection with a network resource. A “network communication” can include any type of information (e.g., text, voice, audio, video, electronic mail message, data file, motion data stream, and data packet) that is transmitted or otherwise conveyed from one network node to another network node over a network connection.
  • A “communicant interaction” is any type of direct or indirect action or influence between a communicant and another network entity, which may include for example another communicant, a virtual area, or a network service. Examples of types of communicant interactions include communicants communicating with each other in realtime, a communicant entering a virtual area, and a communicant requesting access to a resource from a network service.
  • “Presence” refers to the ability and willingness of a networked entity (e.g., a communicant, service, or device) to communicate, where such willingness affects the ability to detect and obtain information about the state of the entity on a network and the ability to connect to the entity.
  • A “realtime data stream” is data that is structured and processed in a continuous flow and is designed to be received with no delay or only imperceptible delay. Realtime data streams include digital representations of voice, video, user movements, facial expressions and other physical phenomena, as well as data within the computing environment that may benefit from rapid transmission, rapid execution, or both rapid transmission and rapid execution, including for example, avatar movement instructions, text chat, realtime data feeds (e.g., sensor data, machine control instructions, transaction streams and stock quote information feeds), screen shares, and file transfers.
  • A “virtual area” (also referred to as an “area,” a “place,” or a “space”) is a representation of a computer-managed space or scene. Virtual areas typically are one-dimensional, two-dimensional, or three-dimensional representations; although in some examples a virtual area may correspond to a single point. Oftentimes, a virtual area is designed to simulate a physical, real-world space. For example, using a traditional computer monitor, a virtual area may be visualized as a two-dimensional graphic of a three-dimensional computer-generated space. However, virtual areas do not require an associated visualization. A virtual area typically refers to an instance of a virtual area schema, where the schema defines the structure and contents of a virtual area in terms of variables and the instance defines the structure and contents of a virtual area in terms of values that have been resolved from a particular context.
  • A “position” in a virtual area refers to a location of a point or an area or a volume in the virtual area. A point typically is represented by a single set of one-dimensional, two-dimensional, or three-dimensional coordinates (e.g., x, y, z) that define a spot in the virtual area. An area typically is represented by the three-dimensional coordinates of three or more coplanar vertices that define a boundary of a closed two-dimensional shape in the virtual area. A volume typically is represented by the three-dimensional coordinates of four or more non-coplanar vertices that define a closed boundary of a three-dimensional shape in the virtual area.
  • “Navigating” in the context of a virtual area means moving or changing an attentional focus or state of a user or another communicant with respect to the virtual area. For example, a user may navigate his or her attention from one virtual communication context to another (e.g., from one zone of a virtual area to another or from one virtual area to another). A user also may navigate his or her presence or the presence of another communicant from one virtual communication context to another (e.g., from one zone of a virtual area to another or from one virtual area to another).
  • As used herein, the term “includes” means includes but not limited to, the term “including” means including but not limited to. The term “based on” means based at least in part on.
  • II. VOICE-BASED VIRTUAL AREA NAVIGATION
  • The examples that are described herein provide systems and methods for voice-based navigation in one or more virtual areas that define respective persistent virtual communication contexts. These examples enable communicants to use voice commands to, for example, search for communication opportunities in the different virtual communication contexts, enter specific ones of the virtual communication contexts, and bring other communicants into specific ones of the virtual communication contexts. In this way, these examples allow communicants to exploit the communication opportunities that are available in virtual areas, even when hands-based or visual methods of interfacing with the virtual areas are not available (e.g., when people are driving, passing through an airport, putting out a forest fire, delivering packages, flying fighter jets, jogging, driving a forklift, and otherwise living their mobile lives).
  • FIG. 1 shows an example of a network communications environment 10 that includes a first client network node 12 (Client Node A), a second client network node 14 (Client Network Node B), a virtual area platform 18 and an optional proxy node 19 that are interconnected by a network 20. The network 20 may include one or more of any of a local area network (LAN), a metropolitan area network (MAN), and a wide area network (WAN) (e.g., the internet). The network 20 typically includes a number of different computing platforms and transport facilities that support the transmission of a wide variety of different media types (e.g., text, voice, audio, video, and other data) between network nodes.
  • The first client network node 12 includes a computer-readable medium 22 (or “memory”), a processor 24, and input/output (I/O) hardware 26 (including, e.g., a display and network communication hardware). The processor 24 executes at least one virtual area communications application 26 that is stored in the memory 22. The second client network node 14 typically is configured in substantially the same general way as the first client network node 12, with a computer-readable medium 30 storing at least one virtual area communications application 32, a processor 34, and I/O hardware 36.
  • Each of the network nodes 12, 14 has a respective set of one or more sources and an exemplary set of one or more sinks. Each source is a device or component that originates data of a particular data stream content type and each sink is a device or component that receives data of a particular data stream content type. A source and a sink of the same data stream content type are referred to herein as being “complementary.” Exemplary sources include an audio source (e.g., an audio capture device, such as a microphone), a video source (e.g., a video capture device, such as a video camera), a chat source (e.g., a text capture device, such as a keyboard), a motion data source (e.g., a pointing device, such as a computer mouse), and other sources (e.g., file sharing source or a source of a customized real-time data stream). Exemplary sinks include an audio sink (e.g., an audio rendering device, such as a speaker or headphones), a video sink (e.g., a video rendering device, such as a display monitor), a chat sink (e.g., a text rendering device, such as a display monitor), a motion data sink (e.g., a movement rendering device, such as a display monitor), and other sinks (e.g., a printer for printing shared files, a device for rendering real-time data streams different from those already described, or software that processes real-time streams for analysis or customized display). Each source has an active state in which the source is available for originating data and an inactive state in which the source is not available for originating data. Likewise, each sink has an active state in which the sink is available for receiving data and an inactive state in which the sink is not available for receiving data. The communicants operating the client nodes 12, 14 typically can control the states of the sources and sinks using controls provided by the communications applications 26, 32. For example, in some examples, the communications applications 26, 32 provide user controls for turning on/off the local microphones and the local speakers (e.g., headsets) on the client network nodes 12, 14.
  • The virtual area platform 18 includes at least one server network node 40 that provides a network infrastructure service environment 42 that manages sessions of the first and second client nodes 12, 14 in one or more virtual areas 44 in accordance with respective virtual area applications 46. One or more of the virtual area applications 44 typically are synchronous conferencing applications that support one or more types of communications between the client nodes 12, 14 (e.g., text chat, audio conferencing, video conferencing, application sharing, and file sharing). The network infrastructure service environment 42 typically includes one or more network infrastructure services that cooperate with the communications applications 26, 32 in the process of establishing and administering network connections between the client nodes 12, 14 and other network nodes. Among the network infrastructure services that are included in the example of the network infrastructure service environment 42 are an account service, a security service, an area service, a rendezvous service, an interaction service, and a capabilities engine. The area service administers a virtual area 44 by managing sessions of the first and second client nodes 12, 14 in the virtual area 44 in accordance with the virtual area application 46. Examples of the virtual area platform 18 and the virtual area applications 46 are described in U.S. Provisional Patent Application No. 61/563,088, filed Nov. 23, 2011. Examples of an account service, a security service, an area service, a rendezvous service, and an interaction service are described in U.S. patent application Ser. No. 12/630,973, filed Dec. 4, 2009. Examples of a capabilities engine are described in U.S. Provisional Patent Application No. 61/535,910, filed Sep. 16, 2011.
  • The network infrastructure service environment 42 maintains a relationship database 47 that contains the records 48 of interactions between communicants and social network profiles 50 that are associated with respective communicants. Each interaction record describes the context of an interaction between a pair of communicants. Each social network profile 50 typically includes: identity characteristics (e.g., name, age, gender, and geographic location information such as postal mailing address) that describe a respective communicant or a persona that is assumed by the communicant; explicit relationship information that is declared by the communicant; and relationship information that is inferred from the communicant's interactions in the network communication environment 10. Additional details regarding the relationship database 47 and the search and retrieval functionalities associated with the relationship database as described in U.S. patent application Ser. No. 12/354,709, filed Jan. 15, 2009, U.S. patent application Ser. No. 12/418,243, filed Apr. 3, 2009, U.S. patent application Ser. No. 12/631,026, filed Dec. 4, 2009, and U.S. patent application Ser. No. 13/432,837, filed Mar. 28, 2012.
  • Some examples provide systems and methods for encapsulating assemblies (e.g., meetings, informal gatherings, and the like) of communicants in one or more virtual areas based on information obtained from the interactions records 48. These examples provide a wide variety of ways in which to capture features of ongoing and completed assemblies and visualize those features at different levels of detail. Exemplary visualizations include an activity view that shows a time-based visualization of assemblies within a particular domain, and a timeline view that shows a scalable visualization of a selected subset of the assemblies within a particular domain. The domain may be predefined or specified by a user and may be used to select the assemblies that are presented in the visualizations. The assemblies may be filtered according to a variety of different attributes of the assemblies, including by virtual location (e.g., a particular virtual area or a particular zone of a virtual area in which the assemblies occurred), by communicant (e.g., one or more communicants who participated in the assemblies), and by time (e.g., a period during which the assemblies started or ended). The assembly visualizations allow communicants to readily obtain a quick summary of the activities occurring in a particular context (e.g., one or more of time, virtual location, participants involved, and types of information shared). They also increase the availability of assembly related information (e.g., notes, shared files, recordings, follow-up tasks, comments and other feedback) by allowing communicants to see and retrieve the information associated with the assemblies in a variety of different ways. Examples of systems and methods of encapsulating and visualizing assemblies of communicants in virtual areas are described in U.S. patent application Ser. No. 13/432,837, filed Mar. 28, 2012.
  • The communications applications 26, 32, the area applications 46, and the network infrastructure service environment 42 together provide a platform that administers the realtime connections with network nodes in an instance of a virtual area subject to a set of constraints 43 (e.g., capabilities and other types of permissions, rules, and preferences). Each of the virtual area applications 46 is hosted by a respective one of the virtual areas 44 and includes a description of the respective virtual area 44. Communicants respectively operating the client nodes 12, 14 connect to the virtual areas 44 through the virtual area communications applications 26, 32.
  • The communications applications 26, 32 typically present respective views of the virtual areas 44 in accordance with data received from the network infrastructure service environment 42. The communications applications 26, 32 also provide respective interfaces (e.g., one or more of a voice input interface, and audio output interface, and a visual graphical user interface) for receiving commands from the communicants. In visual graphical user interfaces, communicants typically are represented in the virtual areas 44 by respective avatars (e.g., sprites). In audio output interfaces, communicants' states and activities are described using audio signals (e.g., synthesized speech). Communicant avatars typically move about the virtual areas 44 in response to commands that are input by the communicants at their respective network nodes. In some examples, the communications applications 26, 32 establish realtime data stream connections between the first and second client network nodes 12, 14 and other network nodes connected to the virtual area 44 based on the positions of the communicants' avatars in the virtual areas 44. In some examples, each of the client network nodes 12, 14 includes a respective realtime kernel of the type described in U.S. patent application Ser. No. 12/630,973, filed Dec. 4, 2009, which supports remote configuration of stream handlers for processing data streams (e.g., rendering audio and video data streams) on a client network node.
  • A virtual area 44 may correspond to an abstract (non-geometric) virtual area that is defined with respect to abstract coordinates, or a visual virtual area that is defined with respect to one-, two- or three-dimensional geometric coordinates. Abstract virtual areas may or may not be associated with respective visualizations, whereas visual virtual areas are associated with respective visualizations.
  • In some of the examples that are described herein, the virtual areas are visual virtual areas of the type disclosed in U.S. Pat. Nos. 7,769,806 and 7,844,724. These visual virtual areas include physical geometry and collision geometry. The physical geometry describes the shape of the virtual area. The physical geometry typically is formed from surfaces of triangles, quadrilaterals, or polygons. Colors and textures are mapped onto the physical geometry to create a more realistic appearance for the virtual area. Lighting effects may be painted onto the visual geometry and the texture, color, or intensity near the lighting effects may be modified. The collision geometry describes invisible surfaces that determine the ways in which objects can move in the virtual area. The collision geometry may coincide with the visual geometry, correspond to a simpler approximation of the visual geometry, or relate to application-specific requirements of a virtual area designer.
  • Some examples of the virtual area platform 18 enable software application designers to define the semantics of position in an abstract virtual area (e.g., a software application or a computer data file). Through associations with respective connection rules, these position definitions can be used, for example, to drive connections to virtual areas, entries into virtual areas, connections to communicants and other sources or sinks of realtime data streams, and determinations of presence data relating to communicants, network resources, and network services. Additional details regarding systems and methods of defining the semantics of position in abstract virtual areas are described in U.S. application Ser. No. 12/631,008, which was filed on Dec. 4, 2009.
  • A virtual area typically includes one or more zones. A zone may be a rendered spatial extent, a set of rules applied to a spatial extent, or both. Zones may be arranged hierarchically in a virtual area, with an outermost zone (referred to herein as the “Global Governance Zone”) enclosing all other zones in the virtual area. Within the Global Governance Zone, there can be location zones (e.g., rooms of a virtual area) or smaller governance zones that enclose a group of location zones and provide regions of governance on the map. A zone definition typically also includes one or more channel definitions that describe how to create respective channels in the zone and specify the information about the channel that is published to a client network node that becomes present in the zone. A channel is always uniquely defined point-to-point and is unique to a virtual area application and a session between a client network node and the virtual area platform.
  • Examples of the types of rules that may be associated with a zone include switching rules, governance rules, and permission rules.
  • Switching rules govern realtime stream connections between network nodes that are linked to the virtual area (e.g., network nodes that are associated with objects, such as avatars, in the virtual area). The switching rules typically include a description of conditions for connecting sources and sinks of realtime data streams in terms of positions in the virtual area. Each switching rule typically includes attributes that define the realtime data stream type to which the rule applies and the location or locations in the virtual area where the rule applies. In some examples, each of the rules optionally may include one or more attributes that specify a required role of the source, a required role of the sink, a priority level of the stream, and a requested data routing topology. In some examples, if there are no explicit switching rules defined for a particular part of the virtual area, one or more implicit or default switching rules may apply to that part of the virtual area. One exemplary default switching rule is a rule that connects every source to every compatible sink within an area, subject to policy rules. Policy rules may apply globally to all connections between the area clients or only to respective connections with individual area clients. An example of a policy rule is a proximity policy rule that only allows connections of sources with compatible sinks that are associated with respective objects that are within a prescribed distance (or radius) of each other in the virtual area. The network connections between network nodes may be arranged in a variety of different data routing topologies, including a peer-to-peer topology, a mediated topology (i.e., a topology in which connections between network nodes are mediated by another network node, such as a server network node, a client network node, or a network switch), and hybrid architectures that combine aspects of peer-to-peer and mediated architectures. In some examples, the switching rules dictate how local connection processes executing on each of the network nodes establishes communications with the other network nodes based on the locations of the associated objects in the zones of the virtual area. A switching rule also may define a direct connection between network nodes or an indirect connection through an intermediate network node (e.g., the proxy node 19 shown in FIG. 1).
  • Governance rules control who has access to resources (e.g., the virtual area itself, regions with the virtual area, and objects within the virtual area), who has access to data (e.g., data streams and other content) that is associated with the virtual area, what is the scope of that access to the data associated the virtual area (e.g., what can a user do with the data), and what are the follow-on consequences of accessing that data (e.g., record keeping, such as audit logs, and payment requirements). In some examples, an entire virtual area or a zone of the virtual area is associated with a “governance mesh” that enables a software application developer to associate governance rules with a virtual area or a zone of a virtual area. This avoids the need for the creation of individual permissions for every file in a virtual area and avoids the need to deal with the complexity that potentially could arise when there is a need to treat the same document differently depending on the context.
  • A permission rule defines a respective capability requirement (e.g., for a respective action, behavior, or state) in terms of one or more capabilities, attributes, and settings, which may be persistent or transient. Examples of permission rules include: a rule that conditions a communicant's ability to enter a target zone on the communicant having a CanEnterZone capability for the target zone; a rule that conditions the ability of a grantee communicant to open a target door of a target room on the grantee communicant having a CanOpenDoor capability for the target room; and a rule that conditions the transmission of a message describing the state of a particular communicant's avatar in a zone to a recipient having a CanSeeState capability for the particular communicant in the zone. A capability provides permission for a client to perform some action within the application. For example, a client may be granted the capability “CanEnterZone” for a specific zone within a virtual area that has been defined with that capability requirement. The client that has the capability can enter the zone, whereas a client without the capability would have their RDS state change rejected when they tried to enter the zone. Examples of capabilities systems for administering permission rules are described in U.S. Provisional Patent Application No. 61/535,910, filed Sep. 16, 2011.
  • As explained above, the zones of a virtual area can be associated with respective switching rules, each of which instructs the area service to connect sources of a respective data stream type that are associated with a designated source zone with sinks of the respective realtime data stream type that are associated with a designated sink zone. Network nodes can establish respective presences in the zones of a virtual area. In some examples, network nodes associated with respective objects (e.g., avatars representing the communicants operating the network nodes) that can be moved to different locations in the virtual area, and the network nodes are present in the zones in which the associated objects are located. The area service administers data stream connections between the network nodes based on the switching rules, the respective sources and sinks associated with the network nodes, and the respective zones of the virtual area in which the objects are located.
  • FIG. 2 shows an exemplary graphical user interface 270 that is generated by an example of the communications application 26 in a window 259 on a display of the client network node from which a user of the client application (“Art” in this example) is operating. The graphical user interface 270 includes a people panel 265, a viewer panel 266, a people interaction toolbar 267, an audio interaction toolbar 268, and a set of panel view controls 269. The panel view controls 269 include a people panel button 271 for opening and closing the people panel 265, a chat panel button 273 for opening and closing a chat panel, and a viewer panel button 275 for opening and closing the viewer panel 266.
  • The people panel 265 depicts the realtime availabilities and activities of some or all of Art's contacts across different communication contexts. In the example shown in FIG. 2, the people panel 265 shows Art's communicants segmented into a virtual area section 272 and a contacts group section 276. The virtual area section 272 shows the presence and activity states in each of the area applications 44 of which Art is a member with respect to which at least one of Art and Art's contacts is present. The contacts group section 276 contains all or a selected portion of Art's contacts that are not represented in any of the virtual area sections. In the example shown in FIG. 2, the virtual area section 272 includes a section 278 corresponding to the user's current zone of presence (i.e., the Main zone in the SococoHQ virtual area) and a section (labeled with a header bar 279 entitled “SococoHQ”) that identifies all the communicants who have a presence in the area application “SococoHQ,” and contains the graphical representations of the communicants (including at least one of Art or Art's contacts) who currently have presence in the SococoHQ area application 44 and are not in the current zone of presence. The contacts group section 276 is labeled with a header bar entitled “Contacts” and contains graphical representations of some or all of Art's contacts who are not shown in the virtual area section 272 (i.e., they either are not members of or not present in SococoHQ).
  • Each communicant is represented graphically by a respective circular sprite that is labeled with a respective user name of the communicant (i.e., “Art,” “Beth,” “Carl,” “Dan,” “Ed,” “Fran,” and “Garth”). Each sprite also may be associated with a respective status line that includes additional information about the communicant. In some embodiments, each status line can include one or more of the following information: location of presence (e.g., a virtual area application or a zone of that sever application); availability (e.g., busy, idle); a status message (e.g., “Out of the office next Wednesday”); and the name of the client node from which the communicant is operating (e.g., “workstation 1” or “mobile phone”). In some embodiments, the ordering of the spatial positions (e.g., from top to bottom) of the communicant avatars in each of the sections 272, 276 is alphabetical by user name. In other embodiments, the spatial positions of the communicant avatars in the area application section 272 are ordered in accordance with the temporal ordering of the communicants in terms of the times when the communicants established their presences in the area application. The spatial positions of the communicant avatars in the contacts section 276 may be sorted alphabetically by user name, according to frequency of contact, according to recentness of contact, or according to other sorting or filtering criteria.
  • The activities of the communicants in the contexts of the area applications 44 may be inferred by the client communications application from activities on various communication channels over which the respective communicants are configured to communicate. The activities on the communication channel are represented in the graphical user interface 270 by visual cues that are depicted in association with the graphical representations of the communicants in the virtual area section 272 and the contacts section 276. For example, the “on” or “off” state of a communicant's local speaker channel is depicted by the presence or absence of a headphones graphic 290 on the communicant's sprite. When a communicant's speakers are on, the headphones graphic 290 is present (see sprites Art, Carl, and Dan) and, when a communicant's speakers are off, the headphones graphic 290 is absent (see sprites Beth and Ed). The “on” or “off” state of the communicant's microphone is depicted by the presence or absence of a microphone graphic 292 on the communicant's sprite. When a communicant's microphone is on, the microphone graphic 292 is present (see sprite Dan); and, when a communicant's microphone is off, the microphone graphic 292 is absent (see sprites Art, Beth, Carl, and Ed). The headphones graphic 290 and the microphone graphic 292 provide visual cues of the activity states of the communicant's sound playback and microphone devices. In addition, the current activity state of a communicant's microphone channel is indicated by a dynamic visualization that lightens and darkens the communicant's avatar in realtime to reflect the presence or absence of audio data on the microphone channel. Thus, whether or not their local speakers are turned on, communicants can determine when another communicant is speaking by the “blinking” of the coloration of that communicant's avatar. The activity state of a communicant's text chat channel is depicted by the presence or absence of the hand graphic 294 adjacent the communicant's sprite (see sprite Beth). Thus, when a communicant is transmitting text chat data to another network node the hand graphic 294 is present, and when a communicant is not transmitting text chat data the hand graphic 294 is not present. In some embodiments, text chat data is transmitted only when keyboard keys are depressed, in which case the visualization of the communicant's text channel appears as a flashing on and off of the hand graphic 294.
  • Members of an area application are able to receive the visual cues of the communicant activities occurring in the context of that area application whether or not the member is present. For example, the graphical user interface 270 that is presented to Art can show visual cues indicating the communication channel activity states of the communicants who are present in the SococoHQ virtual area (where Art is present) and the communication channel states of the communicants who are present in other areas of which Art is a member but not currently present.
  • The audio interaction toolbar 268 includes a headphone control 284 that enables Art to toggle on and off the local speakers of the client network node, and a microphone control 286 that enables Art to toggle on and off the local microphone of the client network node.
  • Additional details regarding embodiments of the people panel 265 are described in U.S. Provisional Patent Application No. 61/373,914, filed Aug. 16, 2010, and U.S. patent application Ser. No. 12/354,709, filed Jan. 15, 2009.
  • The people interaction toolbar 267 includes a Chat button 298 and a Get button 302. Selection of the Chat button 298 opens a Chat panel 340 (see FIG. 5) that enables Art to initiate a chat with other communicants who are present in the area application where Art is present (i.e., SococoHQ in the illustrated example). Selection of the Get button 302 opens an Invite window that enables Art to invite one or more communicants to a selected virtual area location (e.g., an area application or zone within that area application). Additional details regarding embodiments of the methods and functions invoked by the Chat button 298 and the Get button 302 are described in U.S. patent application Ser. No. 12/354,709, filed Jan. 15, 2009, and U.S. Provisional Patent Application No. 61/373,914, filed Aug. 16, 2010.
  • In addition to the control and panel elements of the graphical user interface 270 (e.g., the people panel 265, the viewer panel 266, the people interaction toolbar 267, the audio interaction toolbar 268, and the panel view controls 271, 273, 275), the graphical user interface 270 includes a Phone button 304, a respective View Screen button 306, 308, 310, 312 for each viewscreen in the current zone of presence, a Map button 314, and a meeting button 316. The Phone button 304 is associated with telephony related functionality of the platform that enables a user of a Public Switched Telephone Network (PSTN) terminal device to participate in virtual area based communications (e.g., by the PSTN terminal device user calling into a zone of the virtual area or by a user of the communications application 26 to call out to the PSTN terminal device user), as described in U.S. patent application Ser. No. 13/165,729, filed Jun. 21, 2011, and communicants involved in telephony communications are represented by avatars that are decorated with a graphical representation of a telephone (see, e.g., the avatar 113 in Carl's Office shown in FIG. 3). Each of the View Screen buttons 306-312 sets the viewer panel 266 to display the content the content being shared in connection with a corresponding one of the viewscreen objects in the current zone of presence or, if no content is being shared in connection with the current viewscreen object, to display a Share button that allows the user to initiate a screen sharing session in connection with the selected view screen object. The Map button 314 sets the view presented in the viewer panel 266 to a map view of the virtual area. The Meeting button 316 sets the view presented in the viewer panel 266 to a meeting user interface for visualizing assemblies of communicants in the virtual area, as described in U.S. patent application Ser. No. 13/432,837, filed Mar. 28, 2012. The graphical user interface 270 also includes a home button 285 that is associated with a control that returns the user's presence to a designated “home” location in the virtual environment (e.g., a designed zone, such as a personal zone or other office that is assigned to the user). Additional details regarding the structure, function, and operation of examples of the navigation controls are described in U.S. patent application Ser. No. 12/354,709, filed Jan. 15, 2009.
  • FIG. 2 shows an example of the graphical user interface 270 in the Map view mode, which presents in the viewer panel 266 a rendered view of the user's current zone of presence (Main) of the SococoHQ virtual area, which is located at the location SococoHQ/Main as indicated in the location bar 318. This address indicates that Main is a zone within the area SococoHQ.
  • Each of the communicants who is present in the virtual area is represented graphically in the viewer panel 266 by a respective avatar that corresponds to the communicant's avatar shown in the people panel 265. The virtual area is represented graphically by a two-dimensional top view of a rectangular space. In some examples, the communicants' sprites automatically are positioned in predetermined locations (or “seats”) in the virtual area when the communicants initially enter the virtual area.
  • The Main zone includes four view screen props (or “objects”) 388, 390, 392, 394 and a table prop 396. Communicants interact with the props by selecting them with an input device (e.g., by single-clicking on the props with a computer mouse, touch pad, touch screen, or the like). The view screen props 388-394 are associated with application sharing functionality of the platform that enables communicants to share applications operating their respective client network nodes. The application sharing functionality is invoked by activating a view screen (e.g., by single-clicking the view screen object with an input device). In some embodiments, the platform provides visual cues that indicate whether or not a communicant is sharing an application over an application sharing channel. In response to a communicant's selection of the view screen prop, the communicant's sprite automatically is moved to a position in the graphical representation of the virtual area that is adjacent the view screen prop. The position of a communicant's sprite adjacent the view screen prop indicates that the communicant currently is sharing or is about to share an application with the other communicants in the virtual area. In addition, the avatar of each communicant who is viewing a shared application is depicted with a pair of “eyes” to indicate that the represented communicants are viewing the content being shared in connection with the view screen props (see, e.g., the avatars of Art and Dan in FIG. 2). The graphical depiction of view screen prop is changed depending on whether or not an active application sharing session is occurring. For example, the depicted color of the view screen may change from a brighter color during an active application sharing session to a darker color when there is no application sharing taking place. Examples of the application sharing process are described in connection with FIGS. 26-28 of U.S. patent application Ser. No. 12/354,709, filed Jan. 15, 2009, and in U.S. patent application Ser. No. 12/418,270, filed Apr. 3, 2009.
  • In some examples, one or more of the viewscreen props 388-394 may be associated with respective uniform resource identifiers (URIs) of network resources to enable communicants to interact with and share information associated with the network resources via the application sharing (e.g., web browser sharing) functionality associated with the viewscreen objects as described in U.S. Provisional Patent Application No. 61/444,989, filed Feb. 21, 2011.
  • The table prop 396 is associated with file share functionality of the platform that enables communicants to upload computer data files to server storage in association with the virtual area and to download data files that are associated with the virtual area from the server storage to the respective client network nodes. In example shown in FIG. 2, there are two document objects 398, 400 that are associated with the table prop 396. The document objects 398, 400 are linked to respective documents that are have been shared in the virtual area and stored in server storage. Any of the document objects 398, 400 may be selected by a communicant (e.g., by double-clicking the document object 398 with an input device, such as a computer mouse) to initiate downloading of the associated document to the communicant's client network node. Additional details regarding the structure, function, and operation of the table prop 396 may be obtained from U.S. patent application Ser. No. 12/354,709, filed Jan. 15, 2009.
  • The virtual area platform 18 enables a wide variety of highly customizable virtual area applications to be created. Examples of such applications include virtual area applications for creating a virtual office, a virtual personal space, a virtual art gallery, a virtual concert hall, a virtual auditorium, a virtual conference room, and a virtual clubhouse. The virtual area platform 18 supports the creation of network connections between network nodes in the same zone of a virtual area, as well as the creation of one-way or two-way data stream connections between network nodes in different zones.
  • A virtual area typically is associated with a specific set of communicants (e.g., members of the virtual area) and a plurality of persistent zones that define respective sub-contexts within the virtual area. Each zone of a virtual area may support an independent communication session between the network nodes in the zone. For example, a virtual area may include zones in which audio, video, and text chat channel connections are established only between the sources and sinks of network nodes that are in the same zone. In addition, one or more attributes or other data (e.g., a name, a designated purpose, membership, or associated history of interaction in the zone) may be associated with a virtual area zone to define a specific persistent virtual communication context that represents a particular persistent communication opportunity within the virtual area for organizing a communicant's communications with others. In some examples, the visual and audio interfaces that are provided on the client network nodes are configured to present all the independent communication sessions that are occurring in the virtual area. This allows a user to visualize (through sight or imagination) multiple concurrent independent communication interactions and thereby quickly learn who is meeting with whom and the contexts of those meetings (as defined by the zones in which the meetings are occurring).
  • FIG. 3 shows an example of a two-dimensional visualization of the Sococo HQ virtual area 100. The Sococo HQ virtual area includes a lobby 102, a Main conference room 104, a West Conference room 106, an East Conference room 108, a West Nook zone 110, an East Nook zone 112, a Courtyard zone 114, and sixteen offices. Each of the conference rooms 104-108 includes respective viewscreen objects 116-122 and 388-304, table objects 124, 126, and 396, and respective telephone objects 128, 130, 132 and supports realtime audio, chat, and application and network resource sharing communications between the network nodes in the same conference room. Each of the offices includes respective viewscreen objects (not shown) and a respective telephony object (not shown) and supports realtime audio, chat, and application and network resource sharing communications between the network nodes in the same office. Each of the telephony objects supports shared dial-in and dial-out telephony communications as described in U.S. patent application Ser. No. 13/165,729, filed Jun. 21, 2011. Each of the West Nook 110, East Nook 112, and Lobby 102 zones respectively supports realtime audio and chat communications between the network nodes in the same zone. The communication properties of an example of the virtual area 100 are described in U.S. Provisional Patent Application No. 61/603,024, filed Feb. 24, 2012.
  • FIG. 4 shows and example of a virtual area 600 that includes zones and is associated with a visualization of an auditorium (“Sococo Auditorium”). The virtual area 600 includes a stage (or podium) zone 602, a general seating zone 604, a question-and-answer zone 606, a West Conference Room 608, an East Conference Room 610, and a Lobby 611. The stage zone 602 includes viewscreen objects 612, 614 and speaker props 616, 618. The general seating zone 604 includes a plurality of sub-zones 620 (referred to herein as “bench zones”) defining locations where network nodes are permitted to be present in the general seating zone 604. The question-and-answer zone 606 includes a microphone sub-zone 622, a queuing sub-zone 624, and a microphone prop 626. Each of the West Conference Room 608 and the East Conference Room 610 includes a respective viewscreen object 607, 609. The communication properties of an example of the virtual area 600 are described in U.S. Provisional Patent Application No. 61/603,024, filed Feb. 24, 2012.
  • FIG. 5 shows an example 200 of the people panel 265 of the graphical user interface 270 that depicts the realtime availabilities and activities of some or all of the contacts of a target communicant (“Art” in this example) across the different communication contexts provided by the Sococo HQ virtual area 100 and the Sococo Auditorium virtual area 600. The people panel 200 includes a separate section for each of the areas of which Art is a member and a separate Contacts section. Art's current area of presence (“Sococo HQ”) is shown at the top.
  • Within the current area of presence section, the people panel 265 shows Art's current zone of presence (“Art's Office”) at the top, followed by the zones of co-presence (“Main,” Paul's Office,” and “West Conference”). The Zone of Co-Presence filtering criteria identifies those communicants who are co-present within a particular zone of a virtual area application. Thus, for each virtual area application, each group of two or more communicants who are present within the same zone of the virtual area application is listed in a separate, selectively expandable and collapsible Zone of Co-Presence area 130 of the graphical user interface 128. The depiction of the zones of co-presence involving some or all of Art's contacts enables Art to readily visualize all of the conversations that are occurring across all of the communication contexts that are defined by the virtual area applications of which Art is a member. In this way, Art can determine whether or not there are any ongoing conversations that he would like to join.
  • In addition, for each area application of which Art is a member and with respect to which at least one of Art and Art's contacts is present, the members of the area application are segmented by the status of their presence with respect to the area application. Thus, the members of Sococo HQ are sorted into a Present group 202 and a Not Present group 204. (The members of Sococo Auditorium also are sorted into a Present group and a Not Present group.) In this way, Art readily can visualize who is present and which members are not present in each area application. Each of the groups 202-204 are labeled with a respective “Present” or “Not Present” label that is associated with a respective toggle control that can be toggled to selectively collapse and expand the associated group 202-204. The contacts section 210 labeled with a header bar entitled “Contacts” that identifies all of Art's contacts who are not shown in any of the Sococo HQ and Sococo Auditorium sections (i.e., they either are not members of or not present in any of Sococo HQ and Sococo Auditorium).
  • Additional filtering criteria include a Frequent contacts group 206 in which Art's contacts are sorted by the frequency of their respective interactions with Art, a Recent contacts group 208 in which Art's contacts are sorted by the recentness of their respective interactions with Art, an Online contacts section 212 that lists all of Art's contacts who currently are online (i.e., connected to the network 20), and an Offline contacts group 214 that lists all of Art's contacts who currently are offline (i.e., disconnected from the network).
  • FIG. 6 shows an embodiment of a graphical user interface 220 that depicts the realtime availabilities and activities of some or all of the contacts of a target communicant (“Art” in this example) across different communication contexts defined by a data file A and a software application B. The software application B may be any type of software application, including consumer and business software applications (e.g., a personal information manager software application, such as the Microsoft® Outlook® software application; a document processing software application, such as the Microsoft® Word® software application; a spreadsheet software application, such as the Microsoft® Excel® software application, and a web browser software application, such as the Microsoft® Internet Explorer® software application). In this embodiment, for each data file and software application B of which Art is a member and with respect to which at least one of Art and Art's contacts is present, the members of the virtual area application are segmented by the status of their presence with respect to the data file or software application. In this example, Art is member of Data File A and Software Application B. Thus, the members of Data File A are sorted into a Present group 222 and a Not Present group 224, and the members of Software Application B are sorted into a “Zone of Co-Presence” group 226, a Present group 228 and a Not Present group 230, where the zones correspond to respective sections of the Data File A and the Software Application B. In this way, Art readily can visualize who is present and which members are not present in each of the data files and software applications of which he is a member.
  • Examples of the people panels 200, 220 are described in U.S. patent application Ser. No. 13/209,812, filed Aug. 15, 2011.
  • As explained above, the examples that are described herein provide systems and methods for voice-based navigation in one or more virtual areas that enable communicants to exploit the communication opportunities that are available in such virtual area contexts, even when hands-based or visual methods of accessing such opportunities are not available. In some examples, the voice-based navigation functionalities enable a user to obtain sufficient information to visualize in his or her mind essential aspects of the visual information shown in the graphical user interfaces described above in connection with FIGS. 2-6. Through voice commands and audio feedback, for example, a user can navigate multiple multi-zone virtual areas to determine what is going on in a particular virtual area or zone (e.g., where are communicants located and what are they doing—talking, text chatting, application sharing, communicating over a telephony network, or simply co-present with another communicants), or determine what a specific communicant is doing (e.g., is the communicant online or offline, where the communicant present, and what is the communicant currently doing).
  • Among the statuses that a user of the voice-based navigation system can ascertain about a communicant are statuses of the states and activities of the communicant in one or more of the virtual area applications 46.
  • Exemplary communicant states include:
  • STATE DESCRIPTION ATTRIBUTES
    Online/Offline State Is this communicant Online/Offline
    available now?
    Location of Where is this Application (or Zone within
    Presence communicant? Application) of Primary Focus
    Application-Specific Who is this Name
    Identity Information communicant? Avatar Color
    Photo
    Status Message
    Role (e.g., Guest, Member,
    Moderator, Owner)
    Application-specific Is this person Idle/Sleeping
    Availability available now?
    Busy
    Channels Supported How can I Chat
    by Client of Focus communicate with
    this communicant? Audio Sink (e.g., Speaker)
    Audio Source (e.g.,
    Microphone)
    Video Sink (e.g., Display)
    Video Source (e.g., Screen
    Share)
    Video Source (e.g.,
    Webcam)
    File Transfer
    Client of Focus Which network Client Name (e.g., Sococo,
    service is Yahoo!, Windows Live
    communicant Messenger, Google Talk,
    currently using? Facebook, etc.)
    Client Version
    Station of Focus Which client Operating System (e.g., XP,
    network node is Vista, Windows 7, MacOS,
    communicant iPhone, Windows
    currently using? Mobile, etc.)
    OS Version
    Device Type (e.g., Desktop
    Computer, Smart Phone,
    Phone, etc.)
  • The Online/Offline state indicates whether or not the communicant currently is available for communication. In the illustrated embodiments, a communicant is in the Online state when the communicant is connected to the network communication environment (e.g., by establishing a presence with respect to one or more of the virtual area applications 46), and the communicant is in the Offline state when the communicant is unconnected to the network communication environment (e.g., by not having a presence with respect to any of the virtual area applications 46).
  • The Location of Presence state indicates where the communicant has established a presence in the network communication environment. The Location of Presence typically has a respective attribute value (e.g., an application_ID and/or a zone_ID value) for each of the virtual area applications 44 and/or zones of the virtual area applications 46 in which the user has presence.
  • The Application-Specific Identity Information is communicated to other communicants to indicate information about the communicant in the context of a particular virtual area application with which the communicant is associated. The Application-Specific Identity Information includes:
      • an application-specific Name attribute that is used to identify the communicant to other communicants in the context of the virtual area application;
      • an application-specific Avatar Color attribute that is used to set the color of an iconographic representation of the communicant in the context of the virtual area application;
      • an application-specific Photo attribute that references a user-selected image that is associated with representations of the communicant in the context of the virtual area application;
      • an application-specific Status Message attribute that references a message that is associated with representations of the communicant in the context of the virtual area application; and
      • an application-specific Role attribute that indicates a role (e.g., Guest, Member, Moderator, Owner) of the communicant that may be used for switching data streams to communicants in the context of the virtual area application.
  • The Application-Specific Availability state indicates the availability of the communicant in the context of the virtual area application. The Application-Specific Availability state typically includes an attribute value that indicates when the communicant is idle or sleeping and an attribute value that indicates when the communicant is busy.
  • The Station of Focus state characterizes the client network node from which the communicant is operating. The Station of Focus state typically includes attributes that identify the Device Type of the client network node, the Operating System (OS) running on the client network node, and the OS Version of the operating system.
  • The Client of Focus state indicates the virtual area application with which the communicant currently has focus. The Client of Focus state typically includes a Client Name attribute that identifies client communication application with which the communicant is communicating with the virtual area application, and a Client Version attribute that identifies the version of the client communication application.
  • The Channels Supported by Client of Focus state indicates the communication channels that are available for communicating with the communicant in the context of the user's client application. The Channels Supported by Client of Focus state typically includes a Chat attribute that indicates whether or not the user's client application supports chat communications with the communicant, Audio Sink and Audio Source attributes that respectively indicate whether or not the user's client application supports audio sink and audio source communications with the communicant, Video Source and Video Sink attributes that respectively indicate whether or not the user's client application supports video sink and video source communications with the communicant, and File Transfer attribute that indicates whether or not the user's client application supports file transfer communications with the communicant.
  • Exemplary communicant activities include the channels that the communicant currently has open for communication, which indicates how the communicant currently is interacting with other communicants. In some examples, the activities that a user of the voice-based navigation system can ascertain about a communicant include the following attributes of an Open Channels object:
      • an IsListening attribute that indicates when the communicant's audio sink channel (e.g., speaker) is open;
      • a HasMicOpen attribute that indicates when the communicant's audio source channel (e.g., microphone) is open;
      • an IsSpeaking attribute that indicates when the amplitude of the audio being transmitted on the communicant's audio source channel;
      • an IsWatching attribute that indicates when video data being streamed to the communicant's video sink channel is displayed on the screen;
      • an IsTyping attributes that indicates when a user is actively typing into an input box for the communicant's chat channel;
      • a HasApplicationShareFocus attribute that indicates when application sharing data is being streamed to the communicant on the communicant's application sharing video sink channel;
      • a HasWebCamOn attribute that indicates when the communicant's webcam video source channel is open;
      • a HasChatInputFocus attribute that indicates when the communicant's client network node indicates that the communicant has focus on the communicant's chat channel; and
      • an ObjectOfFocus attribute that indicates what object in a zone the communicant is looking at in the Viewer pane.
  • FIG. 7 shows an example of a voice-based virtual area navigation architecture 700 that includes a speech recognizer 702, a virtual area navigation command interpreter 704, an area service 706, a navigation results translator 708, and a speech synthesizer 710.
  • In some examples, a voice-based virtual area navigation session begins with the detection of a distinctive sound pattern or phrase that is recognized by the speech recognizer 702 as a command to initiate the command recognition process. In some of these examples, the command recognition process is initiated in response to receipt of a distinctive nonlinguistic vocal sound (e.g., one or more tongue clicking sounds).
  • The speech recognizer 702 converts voice input 712 into grammatically correct text in accordance with constraints that are imposed by a virtual area navigation vocabulary and a language model 714. The speech recognizer 702 generates recognized output (e.g., text) that represents the most likely words that were uttered in the voice input 712 based on the vocabulary and grammar 714. The speech recognizer 702 may include one or more of a continuous speech recognizer that attempts to recognize fluently spoken speech in the voice input 712 and an isolated word recognizer that attempts to detect commands (also referred to as actions) and operands (also referred to as arguments or modifiers) by focusing on keywords in the voice input 712.
  • The vocabulary specifies the set of words that can be recognized by the speech recognizer 702. Some examples use a small vocabulary of commands and operands without an explicit grammar (e.g., all word sequences are possible), whereas other examples may use larger vocabularies with word grammars that impose different levels of constraint on the types of word sequences that are allowed.
  • In the illustrated example, the speech recognizer 702 uses a vocabulary that includes a small set of fixed commands and a set of operands that is configured dynamically by the area service 706 based on data associated with the user. In some examples, the set of operands are constrained to the names of the virtual areas, virtual area zones, or contacts that are associated with the user. In other examples, the operands are not limited to the values that are associated with the user. In some of these examples, however, the speech recognizer 702 preferentially selects the operand values that are associated with the user. For example, the operands may consist of all or a subset of all the values that are possible in the virtual communications environment 10 (e.g., the names of all of the virtual areas, virtual area zones, and communicants that are associated with the virtual area platform 18), but the values that are associated with the user are assigned higher probabilities in the lexical decoding process that is used to estimate recognized word probabilities.
  • The language model specifies which combinations of words (e.g., commands and operands) are allowable. In some examples, the language model specifies a syntax (e.g., the navigation command syntax described below) that defines the combinations of words that form grammatically correct strings (e.g., commands and their respective operands). In some examples, syntactically correct word combination hypotheses are allowed, whereas syntactically incorrect word combination hypotheses are disallowed. In some of these examples, the allowable syntactically correct word combinations are assigned respective probability values based on information about he user (e.g., the user's current status—such as state and activity, as described above—in one or more virtual areas, the user's current set of capabilities in the virtual area as defined by the permission rules, the communicants in the user's contact list, and attributes associated with the user).
  • The virtual area navigation command interpreter 704 uses statistical or rules-based methods to extract one or more virtual area navigation commands and zero of more operands from the recognized output based on a set of semantic rules 716. The virtual area navigation command interpreter uses the semantic rules 716 to validate the recognized output received from the speech recognizer 702. In some examples, the semantic rules 716 ensure that the recognized output is consistent with the task being performed or the user's state. In some examples, the semantic rules 716 specify how the likelihoods of certain combinations of commands and operands should be modified based on information about external conditions, facts, or elements of the network communications environment associated with the user and the virtual area application in relation to the semantic values of those combinations of commands and operands. In some examples, the semantic rules 716 assign low probabilities to commands or command/operand combinations that are not allowed based on the status of the user at the time the command or command/operand combination was received. For example, the semantic rules may dynamically assign a lower probability to a command to Go to a virtual area or zone of which the user is not a member or for which the user does not have the requisite capability than it assigns to a command to Go to a virtual area or zone of which the user is a member or has the requisite capability. In addition, the semantic rules may assign a lower probability to a command to GET or FIND a communicant that is not a contact of the user than it assigns to a command to GET or FIND a communicant that is a contact of the user. The extracted commands and operands are passed to the area service 706
  • The area service 706 is a component of the network infrastructure service environment. In some examples, the area service 706 administers virtual areas, manages network connections that are associated with the virtual area subject to the capabilities of the requesting entities, maintains global state information for the virtual area, and serves as a data server for the client network nodes participating in a shared communication session in a context defined by the virtual area. The area service 706 provides the context-dependent information to the speech recognizer 702 and the virtual area navigation command interpreter 704. This information includes, for example, user-specific data 730 (e.g., the user's contact list information 722 and information 720 that relates to the areas/zones that are associated to the user) and user status data 732 (e.g., the current state and activity of the user). The area service 706 also executes the tasks corresponding to the extracted virtual area navigation commands and operands; in this process, navigation results are generated (e.g., by one or more of the area service, other components of the network infrastructure service, or other network entities).
  • The navigation results are passed to a navigation results translator 708, which formats the navigation results into a format that can be converted into audio output 718 (e.g., a synthesized speech data that the client network node 12 can render) by the speech synthesizer 710.
  • The components 702-710 of the architecture 700 may be allocated to the client network node 12 and the virtual area platform 18 in different ways. In some examples, all of the components 702-710 of the voice-based virtual area navigation architecture 700 are allocated to the virtual area platform 18; the client network node 12 sends recorded voice files to the virtual area platform 18 for processing and receives in return sound files containing synthesized speech report of the navigation results. In other examples, the client network node 12 performs front-end signal processing (e.g., speech detection, such as voice activity detection and speech boundary detection, and feature extraction, such as MFCC (Mel-Frequency Cepstral Coefficient) feature extraction) on the voice input and the remaining functions of the speech recognizer 702, as well as the functions of the other components 704-710 of the voice-based virtual area navigation architecture 700, are performed by the virtual area platform 18. In still other embodiments, the speech recognizer 702 is allocated to the client network node 12 and the other components 704-710 of the voice-based virtual area navigation architecture 700 are allocated to the virtual area platform.
  • In some examples, the following virtual area navigation commands and associated syntax are defined:
  • LOOK {<zone> | <area/zone>}
    FIND <contact> | <meeting>
    GO <contact> | HOME | <zone> | <area> | <area/zone>
    GET <contact>
    YES
    NO

    These commands have the following semantics:
  • LOOK
  • LOOK with no modifier reports who else is in the user's current zone of presence.
  • LOOK optionally can be modified by: a zone name (e.g., “LOOK Main”), which reports who is in the specified zone in the current virtual area in which the user is present; or an area/zone pair (e.g., “LOOK Sococo Corporate Lobby”), which reports who else in the specified zone in the specified virtual area.
  • In some examples, connector words such as “in” or “of” are accepted in the syntax, although anything that requires true parsing rather than a comparison within a restricted vocabulary will lower the recognition success rate.
  • In some examples, the LOOK command additionally reports one or more attributes of the status of each of the communicants in the pertinent zone (e.g., is the communicant currently available, which channels (modes or channels of communication) can be used to communicate with the communicant, and what is communicant currently doing—e.g., is the communicant currently interacting with other communicants and, if so, how).
  • FIND
  • FIND with a specified contact operand (e.g., “FIND Paul Brody”) reports the current zone, or virtual area and zone if in a different virtual area than the user's current virtual area of presence, of the specified contact.
  • FIND with a specified meeting name operand (e.g., FIND “Daily Standup”) reports the zone or virtual area and zone if in a different virtual area than the user's current zone of presence in which a meeting having the specified meeting name is taking place.
  • In some examples, the FIND command additionally reports one or more attributes of the status of each of the communicants in the pertinent zone (e.g., is the communicant currently available, which channels (modes or channels of communication) can be used to communicate with the communicant, and what is communicant currently doing—e.g., is the communicant currently interacting with other communicants and, if so, how).
  • GO
  • GO moves the user's presence to either the zone of the specified contact operand (e.g., “GO Paul Brody”), a specified zone of the user's current virtual area of presence (e.g., “GO West Conference”), a default zone of a specified area (e.g., “GO Sococo Corporate” might move the user's presence to the Lobby of the Sococo Corporate virtual area), a specified zone of the specified virtual area (e.g., “GO Sococo Corporate Lobby”), or the user's home zone (e.g., “GO Home”), which is a zone of a virtual area that is designated as the user's home zone (e.g., the user's virtual office).
  • In some examples, the GO command additionally reports one or more attributes of the status of each of the communicants in the zone to which the user's presence has been moved (e.g., is the communicant currently available, which channels (modes or channels of communication) can be used to communicate with the communicant, and what is communicant currently doing—e.g., is the communicant currently interacting with other communicants and, if so, how).
  • GET
  • GET sends a get message to the specified contact operand.
  • In some examples, in response to receipt of a GET command, a service (e.g., the area service or another service of the network infrastructure service environment 42) asks the user whether a voice message should accompany the get message and, if the user gives an affirmative reply (e.g., by saying “YES”), the service prompts the user to speak a voice message, records the voice message, and includes the voice message with the get message that is sent to the specified contact. In some examples, instead of sending the voice message, the service translates the voice message into a text message using a speech-to-text processor and sends the text message with the get message that is sent to the specified contact.
  • YES
  • YES indicates an affirmative response to an input choice.
  • NO
  • NO indicates a negative response to an input choice.
  • FIG. 8 shows an example of a method by which the server network node 42 operates to provide voice-based virtual area navigation.
  • In accordance with the method of FIG. 8, the server network node 42 establishes a respective presence of a user in a given zone of a virtual area in a virtual communications environment (FIG. 8, block 800). The virtual area typically includes multiple zones, where each of respective ones of the zones defines a respective context for realtime communications between network nodes of respective communicants who are present in the zone. The server network node 42 receives a representation of voice input in association with the user (FIG. 8, block 802). The server network node 42 determines a virtual area navigation command based on the representation of voice input (FIG. 8, block 804). The server network node 42 executes of the virtual area navigation command, which involves producing navigation results (FIG. 8, block 806). The server network node 42 transmits the navigation results to a network node associated with the user (FIG. 8, block 808).
  • Examples of the method of FIG. 8 enable the user to use voice commands to search for communication opportunities in the different communications contexts that are available in one or more virtual areas, enter specific ones of such virtual communication contexts, and bring other communicants into specific ones of such virtual communication contexts.
  • In some examples, the voice input representation that is received by the server network node 42 (FIG. 8, block 802) includes a sound file of speech recorded by the client network node 12. In other examples, the voice input representation is a feature-based representation of speech that is produced by the client network node 12. In still other examples, the voice input includes a text-based representation of speech that was recognized by the client network node 12.
  • In some examples, in the process of determining the virtual area navigation command (FIG. 8, block 804), the server network node 42 ascertains one or more constraints for recognizing words in the representation of voice input based on data associated with the user, and determines the virtual area navigation command based on the one or more constraints and the representation of voice input.
  • In some examples, the process of ascertaining the one or more constraints involves determining a restricted vocabulary of words (to which the recognition of words in the representation of voice input is limited) based on the data associated with the user. In some examples, this process involves including names of virtual areas associated with the user in the vocabulary. In some examples, this process involves including names of virtual area zones associated with the user in the vocabulary. In some examples, the process involves including names of communicants associated with the user in the vocabulary.
  • In some examples, the process of ascertaining the one or more constraints involves determining a language model for recognizing words in the representation of voice input based on the data associated with the user. In some examples, the determined language model includes a specification of allowable combinations of virtual area names and zone names to which words in the representation of voice input are limited.
  • In some examples, the virtual area navigation command involves navigating the user's attention in the virtual communications environment. Examples of such commands include the LOOK and FIND commands described above.
  • In some of these examples, the virtual area navigation command is a communicant presence reporting command. In some examples, the navigation results include a list of communicants who are present in a particular one of the zones of the virtual area. In some of these examples, the navigation results include indications of activities being performed by respective ones of the communicants who are present the particular zone. The navigation results also may include indications of which communicants in the zone are performing one or more of talking, text chatting, and application sharing in the particular zone. In some examples, the server network node 42 determines whether the representation of voice input indicates a respective one of the zones of the virtual area in connection with the communicant presence reporting command. In response to a determination that the representation of voice input fails to indicate any of the zones of the virtual area in connection with the communicant presence reporting command, the server network node 42 executes the communicant presence reporting command with respect to the given zone in which the user currently is present. In response to a determination that the representation of voice input indicates a respective one of the zones of the virtual area in connection with the communicant presence reporting command, the server network node 42 executes the communicant presence reporting command with respect to the indicated zone. The indicated zone may be different from the given zone in which the user currently is present.
  • In some examples, in connection with the communicant presence reporting command, the server network node 42 determines in the representation of voice input an indication of a particular zone of a second virtual area of multiple zones in the virtual communications environment. The user may or may not be present in the second virtual area. In some of these examples, the navigation results include a list of communicants who are present in the particular zone of the second virtual area.
  • In some examples, in connection with the communicant presence reporting command, the server network node 42 determines in the representation of voice input an indication of a particular communicant. In some of these examples, the server network node 42 executes the communicant presence reporting command with respect to the particular communicant, and the navigation results include an indication of a zone in which the particular communicant is present. In some examples, the indicated zone is a zone of the virtual area in which the user currently is present. In some examples, the indicated zone is a zone of a second virtual area in which the user is not present.
  • In some examples, the server network node 42 determines in the voice input an indication of a particular assembly of communicants in the virtual area. In some of these examples, the server network node 42 executes the communicant presence reporting command with respect to the particular assembly of communicants, and the navigation results include an indication of a zone in which the particular assembly of communicants is occurring.
  • In some examples the process of executing the virtual area navigation command involves navigating a communicant's presence in the virtual communications environment. Examples of such commands include the GO and GET commands described above.
  • In some of these examples, the virtual area navigation command is a user presence navigation command for navigating the user's presence in the virtual communications environment. In some examples, in connection with the user presence navigation command, the server network node 42 determines in the voice input an indication of a respective one of the zones of the virtual area; the server network node executes the user presence navigation command with respect to the indicated zone, and in the process moves the user's presence to the indicated zone. The navigation results may include an indication that the user's presence has been moved to the indicated zone. The navigation results also may include a list of communicants who are present in the indicated zone.
  • In some of these examples, the server network node 42 determines in the voice input a verbal connotation of a home in connection with the user presence navigation command. Based on this determination, the server network node 42 executes the user presence navigation command with respect to a particular one of the zones of the virtual area that is designated as the user's home zone, and in the process moves the user's presence to the particular zone. In these examples, the navigation results may include an indication that the user's presence has moved to the particular zone.
  • In some examples, the server network node 42 determines in the voice input an indication of a particular zone of a second virtual area of multiple zones in the virtual communications environment in connection with the user presence navigation command; the server network node 42 executes the user presence navigation command with respect to the indicated zone, and in the process moves the user's presence to the indicated zone. The user may or may not be present in the second virtual area. The navigation results may include an indication that the user's presence has moved to the indicated zone.
  • In some examples, the server network node 42 determines in the voice input an indication of a particular communicant in connection with the user presence navigation command. In some of these examples, the server network node 42 executes the user presence navigation command with respect to the particular communicant, and in the process moves the user's presence to a particular zone in which the particular communicant currently is present. The navigation results typically include an indication that the user's presence has moved to the particular zone.
  • In some examples, the virtual area navigation command is a contact presence navigation command for navigating the presence of a particular one of the user's contacts to a location (e.g., a zone) in the virtual communication environment. In some of these examples, in connection with the contact presence navigation command, the server network node 42 determines in the voice input an indication of a particular contact of the user. Based on this determination, the server network node 42 executes the contact presence navigation command with respect to the particular communicant contact. In some examples, the process of executing the contact presence navigation command involves inviting the particular communicant to the particular zone, in which case the navigation results may include an indication that the particular contact has been invited to the particular zone.
  • In some examples, in connection with the contact presence navigation command, the server network node 42 determines whether the voice input indicates a respective one of the zones of the virtual area. In response to a determination that the voice input fails to indicate any of the zones of the virtual area in connection with the contact presence navigation command, the server network node 42 invites the particular contact to the given zone in which the user is present. In response to a determination that the voice input indicates a respective one of the zones of the virtual area in connection with the contact presence navigation command, the server network node 42 invites the particular contact to the indicated zone.
  • FIG. 9 shows an example of a method by which the client network node 12 provides voice-based virtual area navigation.
  • In accordance with the method of FIG. 9, the client network node 12 establishes a respective presence of a user in a given zone of a virtual area in a virtual communications environment (FIG. 9, block 900). The virtual area typically includes multiple zones, where each of respective ones of the zones defines a respective context for realtime communications between network nodes of respective communicants who are present in the zone. The client network node 12 processes voice input (FIG. 8, block 802). Based at least in part on the voice input, the client network node 12 calls one or more services to perform a task involving navigation in the virtual area (FIG. 9, block 904). In association with the service call, the client network node 12 receives navigation results relating to at least one of the zones of the virtual area (FIG. 9, block 906). In some examples, the client network node 12 receives the navigation results from the server network node providing one or more of the services that are called to perform the determined task. In some examples, a remote server network node provides at least one of the one or more services, where the client network node 12 receives the navigation results from the remote server network node. The client network node 12 generates audio output that includes an audio report of the navigation results (FIG. 9, block 908).
  • In some examples, the processing of the voice input (FIG. 9, block 904) is initiated in response to receipt of a distinctive nonlinguistic vocal sound (e.g., a tongue clicking sound). In some examples, the processing involves recording input speech into a sound file and sending the sound file to one or more of the called services. In other examples, the processing involves generating a feature-based representation of input speech and sending the feature-based representation to one or more of the called services. In still other examples, the processing involves recognizing commands and parameters in the voice input and sending the recognized commands and parameters to one or more of the called services.
  • In some examples, a remote server network node (e.g., the server network node 40) provides at least one of the one or more services for performing the determined task (FIG. 9, block 904).
  • In some examples, based at least in part on the voice input, the client network node 12 ascertains a command that involves navigating the user's attention in the virtual communications environment. Examples of such commands include the LOOK and FIND commands described above.
  • In some of these examples, the ascertained command is a communicant presence reporting command. In some of these examples, the navigation results include a list of communicants who are present in a particular one of the zones of the virtual area, in which case the generated audio output (FIG. 9, block 908) includes an audio report of the communicants who are present in the particular zone. The navigation results also may include indications of activities being performed by respective ones of the communicants present the particular zone, in which case the generated audio output comprises an audio reports of the activities. The navigation results also may include indications of which communicants in the zone are performing one or more of talking, text chatting, and application sharing in the particular zone.
  • In some examples, the client network node 12 determines in the voice input an indication of a particular zone of a second virtual area of multiple zones in the virtual communications environment in connection with the communicant presence reporting command. The user may or may not be present in the second virtual area. In these examples, the navigation results may include a list of communicants who are present in the particular zone of the second virtual area, in which case the generated audio output includes an audio report of the communicants who are present in the particular zone of the second virtual area.
  • In some examples, the client network node 12 determines in the voice input an indication of a particular communicant in connection with the communicant presence reporting command. In some of these examples, the determined task includes executing the communicant presence reporting command with respect to the particular communicant. In some of these examples, the navigation results include an indication of a zone in which the particular communicant is present, and the generated audio output includes an audio report relating to the indicated zone.
  • In some examples, the client network node 12 determines in the voice input an indication of a particular assembly of communicants in the virtual area. In some of these examples, the determined task includes executing the communicant presence reporting command with respect to the particular assembly of communicants. In some of these examples, the navigation results include an indication of a zone in which the particular assembly of communicants is occurring, and the generated audio output includes an audio report relating to the indicated zone.
  • In some examples, based at least in part on the voice input, the client network node 12 ascertains a command that involves navigating a communicant's presence in the virtual communications environment. Examples of such commands include the GO and GET commands described above.
  • In some of these examples, the ascertained command is a user presence navigation command for navigating the user's presence in the virtual communications environment. In some examples, the client network node 12 determines in the voice input an indication of a respective one of the zones of the virtual area in connection with the user presence navigation command. In some of these examples, the determined task includes executing the user presence navigation command with respect to the indicated zone. In some of these examples, the navigation results may include an indication that the user's presence has moved to the indicated zone, in which case the generated audio output comprises an audio report that the user has entered the indicated zone. The navigation results may include a list of communicants who are present in the indicated zone, in which case the generated audio output comprises an audio report of the communicants who are present in the indicated zone.
  • In some examples, the client network node 12 determines in the voice input a verbal connotation of a home in connection with the user presence navigation command, in which case the determined task includes executing the user presence navigation command with respect to a respective one of the zones of the virtual area that is designated as the user's home zone.
  • In some examples, the client network node 12 determines in the voice input an indication of a particular zone of a second virtual area of multiple zones in the virtual communications environment in connection with the user presence navigation command, where the user is not present in the second virtual area. In some of these examples, the determined task includes executing the user presence navigation command with respect to the indicated zone. In some of these examples, the navigation results include an indication that the user's presence has moved to the indicated zone, in which case the generated audio output include an audio report that the user has entered the indicated zone.
  • In some examples, the client network node 12 determines in the voice input an indication of a particular communicant in connection with the user presence navigation command. In some of these examples, the determined task includes executing the user presence navigation command with respect to the particular communicant. In some of these examples, the navigation results include an indication that the user's presence has moved to a zone in which the particular communicant is present, in which case the generated audio output include an audio report that the user has entered the indicated zone.
  • In some examples, the ascertained command is a contact presence navigation command for navigating the presence of a respective one of the user contacts in the virtual communication environment.
  • In some examples, the client network node determines in the voice input an indication of a particular contact of the user, in which case the determined task includes executing the contact presence navigation command with respect to the particular contact. In some of these examples, the navigation results include an indication that the particular contact has been invited to a particular zone of the virtual area, in which case the generated audio output comprises an audio report that the particular contact has been invited to a particular zone of the virtual area.
  • In some examples, the client network node 12 determines whether the voice input indicates a respective one of the zones of the virtual area in connection with the contact presence navigation command. In response to a determination that the voice input fails to indicate any of the zones of the virtual area in connection with the contact presence navigation command, the determined task includes executing the contact presence navigation command with respect to the particular contact and the given zone in which the user is present. In some of these examples, the navigation results comprise an indication that the particular communicant contact has been invited to the given zone, in which case the generated audio output includes an audio report that the particular contact has been invited to the given zone. In response to a determination that the voice input indicates a respective one of the zones of the virtual area in connection with the contact presence navigation command, the determined task includes executing the contact presence navigation command with respect to the particular contact and the indicated zone. In some of these examples, the navigation results include an indication that the particular contact has been invited the indicated zone, in which case the generated audio output includes an audio report that the particular contact has been invited to the indicated zone.
  • III. CONCLUSION
  • Other embodiments are within the scope of the claims.

Claims (42)

1. A method, comprising:
establishing a presence for a user in a given zone of a virtual area in a virtual communications environment, wherein the virtual area comprises multiple zones, and each of respective ones of the zones defines a respective context for realtime communications between network nodes of respective communicants who are present in the zone;
receiving a representation of voice input in association with the user;
determining a virtual area navigation command based on the representation of voice input;
executing of the virtual area navigation command, wherein the executing comprises producing navigation results; and
transmitting the navigation results to a network node associated with the user.
2. The method of claim 1, wherein the determining comprises:
based on data associated with the user, ascertaining one or more constraints for recognizing words in the representation of voice input; and
determining the virtual area navigation command based on the one or more constraints and the representation of voice input.
3. The method of claim 2, wherein the ascertaining comprises, based on the data associated with the user, determining a restricted vocabulary of words that to which the recognition of words in the representation of voice input is limited.
4. The method of claim 3, wherein the ascertaining comprises including in the vocabulary names of virtual areas associated with the user.
5. The method of claim 3, wherein the ascertaining comprises including in the vocabulary names of virtual area zones associated with the user.
6. The method of claim 3, wherein the ascertaining comprises including in the vocabulary names of communicants associated with the user.
7. The method of claim 2, wherein the ascertaining comprises, based on the data associated with the user, determining a language model for recognizing words in the representation of voice input.
8. The method of claim 7, wherein the determined language model comprises a specification of allowable combinations of virtual area names and zone names to which words in the representation of voice input are limited.
9. The method of claim 1, wherein the virtual area navigation command involves navigating the user's attention in the virtual communications environment.
10. The method of claim 9, wherein the virtual area navigation command is a communicant presence reporting command.
11. The method of claim 10, wherein the navigation results comprise a list of communicants who are present in a particular one of the zones of the virtual area.
12. The method of claim 11, wherein the navigation results comprise indications of activities being performed by respective ones of the communicants present the particular zone.
13. The method of claim 12, wherein the navigation results comprise indications of which communicants in the zone are performing one or more of talking, text chatting, and application sharing in the particular zone.
14. The method of claim 10, wherein the determining comprises determining whether the representation of voice input indicates a respective one of the zones of the virtual area in connection with the communicant presence reporting command.
15. The method of claim 14, wherein, in response to a determination that the representation of voice input fails to indicate any of the zones of the virtual area in connection with the communicant presence reporting command, the executing comprises executing the communicant presence reporting command with respect to the given zone in which the user is present, and the particular zone is the given zone in which the user is present.
16. The method of claim 14, wherein, in response to a determination that the representation of voice input indicates a respective one of the zones of the virtual area in connection with the communicant presence reporting command, the executing comprises executing the communicant presence reporting command with respect to the indicated zone, and the particular zone is the indicated zone.
17. The method of claim 16, wherein the indicated zone is different from the given zone in which the user is present.
18. The method of claim 10, wherein the determining comprises determining in the representation of voice input an indication of a particular zone of a second virtual area of multiple zones in the virtual communications environment in connection with the communicant presence reporting command, the user is not present in the second virtual area, and the navigation results comprise a list of communicants who are present in the particular zone of the second virtual area.
19. The method of claim 10, wherein the determining comprises determining in the representation of voice input an indication of a particular communicant in connection with the communicant presence reporting command.
20. The method of claim 19, wherein the executing comprises executing the communicant presence reporting command with respect to the particular communicant, and the navigation results comprise an indication of a zone in which the particular communicant is present.
21. The method of claim 19, wherein the indicated zone is a zone of the virtual area.
22. The method of claim 19 wherein the indicated zone is a zone of a second virtual area in which the user is not present.
23. The method of claim 10, wherein the determining comprises determining in the voice input an indication of a particular assembly of communicants in the virtual area.
24. The method of claim 23, wherein the executing comprises executing the communicant presence reporting command with respect to the particular assembly of communicants, and the navigation results comprise an indication of a zone in which the particular assembly of communicants is occurring.
25. The method of claim 1, wherein the executing comprises navigating a communicant's presence in the virtual communications environment.
26. The method of claim 25, wherein the virtual area navigation command is a user presence navigation command for navigating the user's presence in the virtual communications environment.
27. The method of claim 26, wherein the determining comprises determining in the voice input an indication of a respective one of the zones of the virtual area in connection with the user presence navigation command, the executing comprises executing the user presence navigation command with respect to the indicated zone and moving the user's presence to the indicated zone.
28. The method of claim 27, wherein the navigation results comprise an indication that the user's presence have been moved to the indicated zone.
29. The method of claim 27, wherein the navigation results comprise a list of communicants who are present in the indicated zone.
30. The method of claim 26, wherein the determining comprises determining in the voice input a verbal connotation of a home in connection with the user presence navigation command, and the executing comprises executing the user presence navigation command with respect to a particular one of the zones of the virtual area that is designated as the user's home zone and moving the user's presence to the particular zone, and the navigation results comprise an indication that the user's presence has moved to the particular zone.
31. The method of claim 26, wherein the determining comprises determining in the voice input an indication of a particular zone of a second virtual area of multiple zones in the virtual communications environment in connection with the user presence navigation command, the user is not present in the second virtual area, and the executing comprises executing the user presence navigation command with respect to the indicated zone and moving the user's presence to the indicated zone.
32. The method of claim 31, wherein the navigation results comprise an indication that the user's presence has moved to the indicated zone.
33. The method of claim 26, wherein the determining comprises determining in the voice input an indication of a particular communicant in connection with the user presence navigation command.
34. The method of claim 33, wherein the executing comprises executing the user presence navigation command with respect to the particular communicant and moving the user's presence to a particular zone in which the particular communicant currently is present, and the navigation results comprise an indication that the user's presence has moved to the particular zone.
35. The method of claim 25, wherein the virtual area navigation command is a contact presence navigation command for navigating a contact's presence in the virtual communication environment.
36. The method of claim 35, wherein the determining comprises determining in the voice input an indication of a particular contact of the user, and the executing comprises executing the contact presence navigation command with respect to the particular communicant contact.
37. The method of claim 36, wherein the executing comprises inviting the particular communicant to the particular zone, and the navigation results comprise an indication that the particular contact has been invited to the particular zone.
38. The method of claim 36, wherein the determining comprises determining whether the voice input indicates a respective one of the zones of the virtual area in connection with the contact presence navigation command.
39. The method of claim 38, wherein, in response to a determination that the voice input fails to indicate any of the zones of the virtual area in connection with the contact presence navigation command, the executing comprises inviting the particular contact to the given zone in which the user is present, and the particular zone is the given zone in which the user is present.
40. The method of claim 38, wherein, in response to a determination that the voice input indicates a respective one of the zones of the virtual area in connection with the contact presence navigation command, the executing comprises inviting the particular contact to the indicated zone, and the particular zone is the indicated zone.
41. Apparatus, comprising:
a memory storing processor-readable instructions; and
a processor coupled to the memory, operable to execute the instructions, and based at least in part on the execution of the instructions operable to perform operations comprising
establishing a presence for a user in a given zone of a virtual area in a virtual communications environment, wherein the virtual area comprises multiple zones, and each of respective ones of the zones defines a respective context for realtime communications between network nodes of respective communicants who are present in the zone;
receiving a representation of voice input in association with the user;
determining a virtual area navigation command based on the representation of voice input;
executing of the virtual area navigation command, wherein the executing comprises producing navigation results; and
transmitting the navigation results to a network node associated with the user.
42. At least one non-transitory computer-readable medium having processor-readable program code embodied therein, the processor-readable program code adapted to be executed by a processor to implement a method comprising:
establishing a presence for a user in a given zone of a virtual area in a virtual communications environment, wherein the virtual area comprises multiple zones, and each of respective ones of the zones defines a respective context for realtime communications between network nodes of respective communicants who are present in the zone;
receiving a representation of voice input in association with the user;
determining a virtual area navigation command based on the representation of voice input;
executing of the virtual area navigation command, wherein the executing comprises producing navigation results; and
transmitting the navigation results to a network node associated with the user.
US13/860,801 2009-01-15 2013-04-11 Voice-based virtual area navigation Abandoned US20130283169A1 (en)

Priority Applications (4)

Application Number Priority Date Filing Date Title
US13/860,801 US20130283169A1 (en) 2012-04-24 2013-04-11 Voice-based virtual area navigation
US14/997,301 US10649724B2 (en) 2009-01-15 2016-01-15 Voice interface for virtual area interaction
US16/842,554 US11397507B2 (en) 2012-04-24 2020-04-07 Voice-based virtual area navigation
US17/845,981 US20220404950A1 (en) 2012-04-24 2022-06-21 Voice-Based Virtual Area Navigation

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201261637461P 2012-04-24 2012-04-24
US13/860,801 US20130283169A1 (en) 2012-04-24 2013-04-11 Voice-based virtual area navigation

Related Child Applications (3)

Application Number Title Priority Date Filing Date
US13/860,873 Continuation-In-Part US20130283166A1 (en) 2009-01-15 2013-04-11 Voice-based virtual area navigation
US13/860,873 Continuation US20130283166A1 (en) 2009-01-15 2013-04-11 Voice-based virtual area navigation
US14/997,301 Continuation-In-Part US10649724B2 (en) 2009-01-15 2016-01-15 Voice interface for virtual area interaction

Publications (1)

Publication Number Publication Date
US20130283169A1 true US20130283169A1 (en) 2013-10-24

Family

ID=49381324

Family Applications (5)

Application Number Title Priority Date Filing Date
US13/860,801 Abandoned US20130283169A1 (en) 2009-01-15 2013-04-11 Voice-based virtual area navigation
US13/860,873 Abandoned US20130283166A1 (en) 2009-01-15 2013-04-11 Voice-based virtual area navigation
US14/997,301 Active 2035-10-06 US10649724B2 (en) 2009-01-15 2016-01-15 Voice interface for virtual area interaction
US16/842,554 Active US11397507B2 (en) 2012-04-24 2020-04-07 Voice-based virtual area navigation
US17/845,981 Pending US20220404950A1 (en) 2012-04-24 2022-06-21 Voice-Based Virtual Area Navigation

Family Applications After (4)

Application Number Title Priority Date Filing Date
US13/860,873 Abandoned US20130283166A1 (en) 2009-01-15 2013-04-11 Voice-based virtual area navigation
US14/997,301 Active 2035-10-06 US10649724B2 (en) 2009-01-15 2016-01-15 Voice interface for virtual area interaction
US16/842,554 Active US11397507B2 (en) 2012-04-24 2020-04-07 Voice-based virtual area navigation
US17/845,981 Pending US20220404950A1 (en) 2012-04-24 2022-06-21 Voice-Based Virtual Area Navigation

Country Status (1)

Country Link
US (5) US20130283169A1 (en)

Cited By (27)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150012822A1 (en) * 2012-01-08 2015-01-08 Thomson Licensing Method and apparatus for providing media asset recommendations
US9319357B2 (en) 2009-01-15 2016-04-19 Social Communications Company Context based virtual area creation
US9411489B2 (en) 2007-10-24 2016-08-09 Sococo, Inc. Interfacing with a spatial virtual communication environment
US9411490B2 (en) 2007-10-24 2016-08-09 Sococo, Inc. Shared virtual area communication environment based apparatus and methods
US20180124002A1 (en) * 2016-11-01 2018-05-03 Microsoft Technology Licensing, Llc Enhanced is-typing indicator
US20210295848A1 (en) * 2018-09-25 2021-09-23 Sonos, Inc. Voice detection optimization based on selected voice assistant service
US11570219B2 (en) * 2020-05-07 2023-01-31 Re Mago Holding Ltd Method, apparatus, and computer readable medium for virtual conferencing with embedded collaboration tools
US11792590B2 (en) 2018-05-25 2023-10-17 Sonos, Inc. Determining and adapting to changes in microphone performance of playback devices
US11790937B2 (en) 2018-09-21 2023-10-17 Sonos, Inc. Voice detection optimization using sound metadata
US11790911B2 (en) 2018-09-28 2023-10-17 Sonos, Inc. Systems and methods for selective wake word detection using neural network models
US11797263B2 (en) 2018-05-10 2023-10-24 Sonos, Inc. Systems and methods for voice-assisted media content selection
US11798553B2 (en) 2019-05-03 2023-10-24 Sonos, Inc. Voice assistant persistence across multiple network microphone devices
US11816393B2 (en) 2017-09-08 2023-11-14 Sonos, Inc. Dynamic computation of system response volume
US11817083B2 (en) 2018-12-13 2023-11-14 Sonos, Inc. Networked microphone devices, systems, and methods of localized arbitration
US11817076B2 (en) 2017-09-28 2023-11-14 Sonos, Inc. Multi-channel acoustic echo cancellation
US11832068B2 (en) 2016-02-22 2023-11-28 Sonos, Inc. Music service selection
US11862161B2 (en) 2019-10-22 2024-01-02 Sonos, Inc. VAS toggle based on device orientation
US11863593B2 (en) 2016-02-22 2024-01-02 Sonos, Inc. Networked microphone device control
US11869503B2 (en) 2019-12-20 2024-01-09 Sonos, Inc. Offline voice control
US11881222B2 (en) 2020-05-20 2024-01-23 Sonos, Inc Command keywords with input detection windowing
US11881223B2 (en) 2018-12-07 2024-01-23 Sonos, Inc. Systems and methods of operating media playback systems having multiple voice assistant services
US11887598B2 (en) 2020-01-07 2024-01-30 Sonos, Inc. Voice verification for media playback
US11893308B2 (en) 2017-09-29 2024-02-06 Sonos, Inc. Media playback system with concurrent voice assistance
US11900937B2 (en) 2017-08-07 2024-02-13 Sonos, Inc. Wake-word detection suppression
US11899519B2 (en) 2018-10-23 2024-02-13 Sonos, Inc. Multiple stage network microphone device with reduced power consumption and processing load
US11934742B2 (en) 2016-08-05 2024-03-19 Sonos, Inc. Playback device supporting concurrent voice assistants
US11947870B2 (en) 2016-02-22 2024-04-02 Sonos, Inc. Audio response playback

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9990059B2 (en) 2014-05-23 2018-06-05 Microsoft Technology Licensing, Llc Ink modes
GB2528867A (en) * 2014-07-31 2016-02-10 Ibm Smart device control
US10374872B2 (en) * 2016-05-24 2019-08-06 Apstra, Inc. Configuring system resources for different reference architectures
US11579744B2 (en) * 2017-06-21 2023-02-14 Navitaire Llc Systems and methods for seat selection in virtual reality

Citations (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5774628A (en) * 1995-04-10 1998-06-30 Texas Instruments Incorporated Speaker-independent dynamic vocabulary and grammar in speech recognition
US6057856A (en) * 1996-09-30 2000-05-02 Sony Corporation 3D virtual reality multi-user interaction with superimposed positional information display for each user
US6310629B1 (en) * 1997-12-19 2001-10-30 Texas Instruments Incorporated System and method for advanced interfaces for virtual environments
US20020087328A1 (en) * 2000-12-28 2002-07-04 Denenberg Lawrence A. Automatic dynamic speech recognition vocabulary based on external sources of information
US20020143916A1 (en) * 2000-05-11 2002-10-03 Dennis Mendiola Method and system for tracking the online status of active users of an internet-based instant messaging system
US20030158731A1 (en) * 2002-02-15 2003-08-21 Falcon Stephen Russell Word training interface
US20050128212A1 (en) * 2003-03-06 2005-06-16 Edecker Ada M. System and method for minimizing the amount of data necessary to create a virtual three-dimensional environment
US20050166154A1 (en) * 2004-01-22 2005-07-28 Wilson Richard M. Enhanced instant message status message area containing time/date stamped entries and editable by others
US20060053016A1 (en) * 2002-02-04 2006-03-09 Microsoft Corporation Systems and methods for managing multiple grammars in a speech recognition system
US20090106671A1 (en) * 2007-10-22 2009-04-23 Olson Donald E Digital multimedia sharing in virtual worlds
US20090300521A1 (en) * 2008-05-30 2009-12-03 International Business Machines Corporation Apparatus for navigation and interaction in a virtual meeting place
US20100020075A1 (en) * 2003-03-06 2010-01-28 Xydne, Inc. Apparatus and method for creating a virtual three-dimensional environment, and method of generating revenue therefrom
US20130143669A1 (en) * 2010-12-03 2013-06-06 Solocron Entertainment, Llc Collaborative electronic game play employing player classification and aggregation
US20130275886A1 (en) * 2012-04-11 2013-10-17 Myriata, Inc. System and method for transporting a virtual avatar within multiple virtual environments

Family Cites Families (30)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6219045B1 (en) * 1995-11-13 2001-04-17 Worlds, Inc. Scalable virtual world chat client-server system
US5896129A (en) * 1996-09-13 1999-04-20 Sony Corporation User friendly passenger interface including audio menuing for the visually impaired and closed captioning for the hearing impaired for an interactive flight entertainment system
US20040176106A1 (en) * 2003-01-29 2004-09-09 Tendler Robert K. System for location reporting
US8401860B1 (en) * 2005-05-06 2013-03-19 Paul R Evans Voice-activated command and control for remotely controlled model vehicles
US9116989B1 (en) * 2005-08-19 2015-08-25 At&T Intellectual Property Ii, L.P. System and method for using speech for data searching during presentations
JP5115058B2 (en) * 2006-08-28 2013-01-09 株式会社Jvcケンウッド Electronic device control apparatus and electronic device control method
US20080092341A1 (en) * 2006-10-20 2008-04-24 Biometric Lingerie, Inc. Biometric garment and method of operation
US7930212B2 (en) * 2007-03-29 2011-04-19 Susan Perry Electronic menu system with audio output for the visually impaired
US20130073707A1 (en) 2011-09-16 2013-03-21 Social Communications Company Capabilities based management of virtual areas
US7769806B2 (en) 2007-10-24 2010-08-03 Social Communications Company Automated real-time data stream switching in a shared virtual area communication environment
US20090288007A1 (en) 2008-04-05 2009-11-19 Social Communications Company Spatial interfaces for realtime networked communications
US8397168B2 (en) 2008-04-05 2013-03-12 Social Communications Company Interfacing with a spatial virtual communication environment
US8732236B2 (en) 2008-12-05 2014-05-20 Social Communications Company Managing network communications between network nodes and stream transport protocol
US7844724B2 (en) 2007-10-24 2010-11-30 Social Communications Company Automated real-time data stream switching in a shared virtual area communication environment
US10083420B2 (en) 2007-11-21 2018-09-25 Sermo, Inc Community moderated information
US8638908B2 (en) * 2008-02-28 2014-01-28 Computer Products Introductions, Corp Contextual conversation processing in telecommunication applications
US9514444B2 (en) 2009-01-15 2016-12-06 Sococo, Inc. Encapsulating virtual area based communicant assemblies
US20120246582A1 (en) 2008-04-05 2012-09-27 Social Communications Company Interfacing with a spatial virtual communications environment
WO2009146130A2 (en) * 2008-04-05 2009-12-03 Social Communications Company Shared virtual area communication environment based apparatus and methods
US8082245B2 (en) * 2008-09-11 2011-12-20 International Business Machines Corporation Providing location information within a virtual world
WO2010065909A2 (en) 2008-12-05 2010-06-10 Social Communications Company Managing interactions in a network communications environment
KR20130010910A (en) 2008-12-05 2013-01-29 소우셜 커뮤니케이션즈 컴퍼니 Realtime kernel
US9853922B2 (en) 2012-02-24 2017-12-26 Sococo, Inc. Virtual area communications
US8405623B2 (en) * 2009-03-25 2013-03-26 International Business Machines Corporation Directional audio viewport for the sight impaired in virtual worlds
US9858925B2 (en) * 2009-06-05 2018-01-02 Apple Inc. Using context information to facilitate processing of commands in a virtual assistant
US9111538B2 (en) * 2009-09-30 2015-08-18 T-Mobile Usa, Inc. Genius button secondary commands
WO2012024205A2 (en) 2010-08-16 2012-02-23 Social Communications Company Promoting communicant interactions in a network communications environment
CN103221967A (en) 2010-09-11 2013-07-24 社会传播公司 Relationship based presence indicating in virtual area contexts
US20130031475A1 (en) * 2010-10-18 2013-01-31 Scene 53 Inc. Social network based virtual assembly places
WO2013078062A1 (en) 2011-11-23 2013-05-30 Social Communications Company Creating and managing virtual areas

Patent Citations (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5774628A (en) * 1995-04-10 1998-06-30 Texas Instruments Incorporated Speaker-independent dynamic vocabulary and grammar in speech recognition
US6057856A (en) * 1996-09-30 2000-05-02 Sony Corporation 3D virtual reality multi-user interaction with superimposed positional information display for each user
US6310629B1 (en) * 1997-12-19 2001-10-30 Texas Instruments Incorporated System and method for advanced interfaces for virtual environments
US20020080193A1 (en) * 1997-12-19 2002-06-27 Muthusamy Yeshwant K. System and method for advanced interfaces for virtual environments
US20020143916A1 (en) * 2000-05-11 2002-10-03 Dennis Mendiola Method and system for tracking the online status of active users of an internet-based instant messaging system
US20020087328A1 (en) * 2000-12-28 2002-07-04 Denenberg Lawrence A. Automatic dynamic speech recognition vocabulary based on external sources of information
US7363229B2 (en) * 2002-02-04 2008-04-22 Microsoft Corporation Systems and methods for managing multiple grammars in a speech recognition system
US8447616B2 (en) * 2002-02-04 2013-05-21 Microsoft Corporation Systems and methods for managing multiple grammars in a speech recognition system
US7720678B2 (en) * 2002-02-04 2010-05-18 Microsoft Corporation Systems and methods for managing multiple grammars in a speech recognition system
US20060053016A1 (en) * 2002-02-04 2006-03-09 Microsoft Corporation Systems and methods for managing multiple grammars in a speech recognition system
US7587317B2 (en) * 2002-02-15 2009-09-08 Microsoft Corporation Word training interface
US20030158731A1 (en) * 2002-02-15 2003-08-21 Falcon Stephen Russell Word training interface
US20100020075A1 (en) * 2003-03-06 2010-01-28 Xydne, Inc. Apparatus and method for creating a virtual three-dimensional environment, and method of generating revenue therefrom
US20050128212A1 (en) * 2003-03-06 2005-06-16 Edecker Ada M. System and method for minimizing the amount of data necessary to create a virtual three-dimensional environment
US20050166154A1 (en) * 2004-01-22 2005-07-28 Wilson Richard M. Enhanced instant message status message area containing time/date stamped entries and editable by others
US20090106671A1 (en) * 2007-10-22 2009-04-23 Olson Donald E Digital multimedia sharing in virtual worlds
US20090300521A1 (en) * 2008-05-30 2009-12-03 International Business Machines Corporation Apparatus for navigation and interaction in a virtual meeting place
US20130143669A1 (en) * 2010-12-03 2013-06-06 Solocron Entertainment, Llc Collaborative electronic game play employing player classification and aggregation
US20130275886A1 (en) * 2012-04-11 2013-10-17 Myriata, Inc. System and method for transporting a virtual avatar within multiple virtual environments

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
http://wiki.secondlife.com/wiki/Main_Page last modified on Feb. 23, 2010 ("2LWiki") *
Second Life For Dummies, Published Jan. 15, 2008 by Wiley Publishing, Inc. ("Robbins") *

Cited By (29)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9411489B2 (en) 2007-10-24 2016-08-09 Sococo, Inc. Interfacing with a spatial virtual communication environment
US9411490B2 (en) 2007-10-24 2016-08-09 Sococo, Inc. Shared virtual area communication environment based apparatus and methods
US9483157B2 (en) 2007-10-24 2016-11-01 Sococo, Inc. Interfacing with a spatial virtual communication environment
US9319357B2 (en) 2009-01-15 2016-04-19 Social Communications Company Context based virtual area creation
US20150012822A1 (en) * 2012-01-08 2015-01-08 Thomson Licensing Method and apparatus for providing media asset recommendations
US11832068B2 (en) 2016-02-22 2023-11-28 Sonos, Inc. Music service selection
US11863593B2 (en) 2016-02-22 2024-01-02 Sonos, Inc. Networked microphone device control
US11947870B2 (en) 2016-02-22 2024-04-02 Sonos, Inc. Audio response playback
US11934742B2 (en) 2016-08-05 2024-03-19 Sonos, Inc. Playback device supporting concurrent voice assistants
US20180124002A1 (en) * 2016-11-01 2018-05-03 Microsoft Technology Licensing, Llc Enhanced is-typing indicator
US11900937B2 (en) 2017-08-07 2024-02-13 Sonos, Inc. Wake-word detection suppression
US11816393B2 (en) 2017-09-08 2023-11-14 Sonos, Inc. Dynamic computation of system response volume
US11817076B2 (en) 2017-09-28 2023-11-14 Sonos, Inc. Multi-channel acoustic echo cancellation
US11893308B2 (en) 2017-09-29 2024-02-06 Sonos, Inc. Media playback system with concurrent voice assistance
US11797263B2 (en) 2018-05-10 2023-10-24 Sonos, Inc. Systems and methods for voice-assisted media content selection
US11792590B2 (en) 2018-05-25 2023-10-17 Sonos, Inc. Determining and adapting to changes in microphone performance of playback devices
US11790937B2 (en) 2018-09-21 2023-10-17 Sonos, Inc. Voice detection optimization using sound metadata
US20210295848A1 (en) * 2018-09-25 2021-09-23 Sonos, Inc. Voice detection optimization based on selected voice assistant service
US11727936B2 (en) * 2018-09-25 2023-08-15 Sonos, Inc. Voice detection optimization based on selected voice assistant service
US11790911B2 (en) 2018-09-28 2023-10-17 Sonos, Inc. Systems and methods for selective wake word detection using neural network models
US11899519B2 (en) 2018-10-23 2024-02-13 Sonos, Inc. Multiple stage network microphone device with reduced power consumption and processing load
US11881223B2 (en) 2018-12-07 2024-01-23 Sonos, Inc. Systems and methods of operating media playback systems having multiple voice assistant services
US11817083B2 (en) 2018-12-13 2023-11-14 Sonos, Inc. Networked microphone devices, systems, and methods of localized arbitration
US11798553B2 (en) 2019-05-03 2023-10-24 Sonos, Inc. Voice assistant persistence across multiple network microphone devices
US11862161B2 (en) 2019-10-22 2024-01-02 Sonos, Inc. VAS toggle based on device orientation
US11869503B2 (en) 2019-12-20 2024-01-09 Sonos, Inc. Offline voice control
US11887598B2 (en) 2020-01-07 2024-01-30 Sonos, Inc. Voice verification for media playback
US11570219B2 (en) * 2020-05-07 2023-01-31 Re Mago Holding Ltd Method, apparatus, and computer readable medium for virtual conferencing with embedded collaboration tools
US11881222B2 (en) 2020-05-20 2024-01-23 Sonos, Inc Command keywords with input detection windowing

Also Published As

Publication number Publication date
US11397507B2 (en) 2022-07-26
US20130283166A1 (en) 2013-10-24
US10649724B2 (en) 2020-05-12
US20200233640A1 (en) 2020-07-23
US20160139878A1 (en) 2016-05-19
US20220404950A1 (en) 2022-12-22

Similar Documents

Publication Publication Date Title
US11397507B2 (en) Voice-based virtual area navigation
US11489893B2 (en) Bridging physical and virtual spaces
US20210185105A1 (en) Automatic Session Establishment in Peer-to-Peer Communication
US11380020B2 (en) Promoting communicant interactions in a network communications environment
US9602447B2 (en) Context based virtual area creation
US8930472B2 (en) Promoting communicant interactions in a network communications environment
US11575531B2 (en) Dynamic virtual environment
US20120246582A1 (en) Interfacing with a spatial virtual communications environment
US20200213256A1 (en) Context Based Virtual Area Creation
US20130174059A1 (en) Communicating between a virtual area and a physical space
US20240087180A1 (en) Promoting Communicant Interactions in a Network Communications Environment

Legal Events

Date Code Title Description
AS Assignment

Owner name: SOCIAL COMMUNICATIONS COMPANY, CALIFORNIA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:VAN WIE, DAVID;REEL/FRAME:030954/0944

Effective date: 20130715

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION