WO2002042921A1

WO2002042921A1 - System and method for synthesizing environments to facilitate distributed, context-sensitive, multi-user interactive applications

Info

Publication number: WO2002042921A1
Application number: PCT/US2001/044091
Authority: WO
Inventors: David A. Levine; Gabriel D. Minton; Mark Conway Wirt
Original assignee: Butterfly.Net, Inc.
Priority date: 2000-11-27
Filing date: 2001-11-27
Publication date: 2002-05-30
Also published as: AU2002219857A1

Abstract

A distributed, location-based, contextual multi-user gaming, entertainment and e-commerce server system, method and computer programming product. The system includes a server (102) which allows multiple users (112) to participate in one instance of an interactive application program and a source of position information to geographically locate the users. The method and computer program product include allowing users to register multiple devices (e.g., mobile phones, video game consoles, personal data assistants, eye phones, etc. (112a-112e)) as well as their personal computers (112f) as clients to interact and communicate, via the server, with the other users involved in the instance of the application program. The system, method and computer program product, through the various registered users, allow both non-virtual (i.e., real world) events and stimuli, and virtual world signals and stimuli to affect the environment of the instance of the application program.

Description

SYSTEM AND METHOD FOR SYNTHESIZING ENVIRONMENTS TO

FACILITATE DISTRIBUTED, CONTEXT-SENSITIVE, MULTI-USER

INTERACTIVE APPLICATIONS

Background of the Invention

Field of the Invention

The present invention relates generally to computer network systems, and more particularly to computer network systems which facilitate multi-person interaction within multiple environments.

Related Art

Over the past several years, there has been an explosion of computers, and thus people, connected to the global Internet and the World-Wide Web (WWW). This increase of connectivity has allowed computer users to access various types of information, disseminate information, be exposed to electronic commerce, as well as engage in various forms of entertainment and interaction.

Using the Internet, people can send electronic messages, play games and collaborate on work projects concurrently with other users spread across a wide geographic area. More particularly, there has been a dramatic rise in the number of servers connected to the Internet through which service providers offer users the opportunity to interact in a mediated environment. That is, several people can simultaneously provide inputs into a shared computer program and each person' s movements, decisions, etc. affect the shared virtual environment and thus every other person. These programs are known as multi-user, interactive applications. These multi-user, interactive application programs generally take the form of Massively Multi-Player Online Role-Playing Games (MMORPGs). These virtual environments generally consist of a 3-D navigable representation of a fantasy realm based on themes, rules, and roles taken from medieval fantasy literature and role-playing games popularized in the dice game Dungeons and Dragons . They also contain a chat interface for textual communications between players and to display messages generated by the system (as represented by Non- Player Characters (NPCs)). MMORPGs also provide tools for customizing the interface, characters and environment. The chat screen also provides a text window for messages generated by the system. RPGs are also typically distributed independently of multi-user environments on CD-ROM or available for download over the Internet. These MMORPGs connect to their own servers. hi addition, services such as BattleZone provide a service for connecting players of CD-ROM games. Further examples of such online, multi-player games include EverQuest™ "The Ruins of Kunark" from Verant Interactive/Sony Computer Entertainment America, "Ultima Online" from Electronic Arts, Inc., "Asheron's Call" from the Microsoft Corporation, and the like.

One common characteristic (and short-coming) among the various multi- person interactive applications is that they are based on the client-server paradigm. This means that most of the processing involved in executing these multi-person interactive applications is centralized on the server computers to which the users (i.e., players) are connected. This method of creating a virtual community is not entirely scalable and does not provide for decentralized management of people and devices. Typically, only a small subset of simultaneous players can interact with one another at any time. Players can only interact with those connected to the same server, so the community must be segmented.

Another common characteristic (and short-coming) among the various multi-person, interactive applications is that the user (client) interface to the server-based virtual environment is typically a personal computer, workstation or terminal where the user must distinguish between the real world and the virtual world. Consequently, users of multi-person interactive environments employ terms such as IRL ("in real life") to distinguish between their actual physical location (e.g., "I'm in my bedroom IRL."), and the virtual world (e.g., "I'm in the living room") which suggests that such a user is in the living room in the MMORPG interactive application program, and not in the living room of their real house. Yet another shortcoming among the various multi-person interactive applications is that users cannot interact or otherwise respond to events that occur in the virtual (e.g., MMORPG) environment when they are away from their personal computers, workstations or terminals. That is, users can not participate in the virtual, interactive, multi-person environment unless they are sitting at the computer. Aside from personal computers, workstations and terminals connected to the Internet, mobile phones, computer tablets, two-way pagers, personal data assistants (PDAs) and the like, are commonly owned by today's consumers and each represent an opportunity to allow users to participate in multi-person, interactive applications. Conventional multi-user interactive applications, however, do not allow users to access the virtual environment using these devices.

Finally, another shortcoming among the various multi-person, interactive applications is that users cannot control physical devices such as toys, appliances and vehicles (IRL), through their interactions with virtual world objects.

Given the foregoing, what is needed is a system, method and computer program product for providing a bridge between virtual and real environments in the context of multi-user gaming, entertainment and e-commerce applications.

Summary of the Invention

The present invention, which meets the above-identified needs, is a system, method and computer program product for bridging activities in real and virtual environments in the context of multi-user gaming, entertainment and e- commerce applications.

The system includes an application database that stores information about the users participating (e.g., playing) in the interactive, multi-user application (e.g., game) and also stores information about the types of client devices owned by each of the plurality of users. The system also includes one or more servers, each connected to the application database, for executing the interactive, multiuser application. One or more translators, each connected to one of the servers, are also included in the system for supporting connections from the various types of client devices. The system further includes one or more transportation networks, each connected to one of the translators, for facilitating communications between the translators and the type of client devices supported by each of the translators. In an embodiment, the system also includes an orientationally-aware peripheral device within the client devices for tracking the location and orientation of users within the system.

The system, as described herein, thereby allows the creation of scenarios that bridge the virtual environment representation of each of the plurality of users with the physical environment through contextually relevant interfaces. Thus, the system of the present invention also includes a distributed component software framework to connect all client devices and servers to form a bridge between the real world and virtual environments.

The method and computer program product involve users registering with an application service provider (ASP) providing the system as described herein.

This registration involves receiving a request for presence within the interactive, multi-user application from a first user and a second user. The method and computer programming product then establish a presence for the first and second users within the application. That is, a computer-generated synthetic representation is created for the first and second users within the application.

Next, the system stores in the application database information about one or more devices which the first user and the second user can use to gain access to the application. Each of the users, as part of the registration process, also receives components of a multi-tier, or "X2Y" software framework, appropriate for their client device types, in order to facilitate messages and other communications between them and the rest of the system (i.e., translators, servers, and application database).

During execution of the application, a signal from the first user via a first device is received by the server via a translator and transportation network appropriate for the first client device type being used. The signal is a request for interaction with an entity in the application. The system, in response to the request, effects the request within the application and communicates the effect to the second user using a second device via a translator and transportation network appropriate for the second client device type being used.

The system, method and computer program product of the present invention allow for the interaction of any device with any person participating in any application of the system permitted by users of the system and the processing of requests and commands are fully distributed among the users, computers and other devices participating in the community.

The system, method and computer program product of the present invention accounts for both the physical and virtual location and context of the participating devices and people. The system, method and computer program product also provide for both synchronous and asynchronous communications between people, computers, other devices and computers for the purpose of coordinating activities in the real (i.e., physical) and virtual worlds. One advantage of the present invention is that it can combine both real

(non- virtual) and virtual environments while facilitating user (player) interaction.

Another advantage of the present invention is that, unlike conventional interactive games, it allows "X2F' communications and commerce, where X and

Y may be any device, person or organization. That is, universal access to the gaming environment is allowed via any device to which an interface can be provided (e.g., mobile phones, video game consoles, personal data assistants, video glasses, ear pieces, etc.). This offers an advantage over previous Internet application offerings whose models are focused on either: "B2C" (business-to- consumer) transactions over the Web, such as Amazon.com, where consumers buy products from one business or several businesses; "C2C" (consumer-to- consumer) transactions such as eBay where an individual can buy or sell from another individual; "B2B" (business-to-business) transactions where institutions buy and sell from each other; or "P2P" (peer-to-peer) systems where applications are networked between personal computers. Another advantage of the present invention is that, aside from personal computers, workstations and terminals connected to the Internet, it allows mobile phones, wireless data devices, personal data assistants (PDAs) and the like, which are commonly owned by today's consumers, to represent opportunities to where users can participate in multi-person, interactive applications.

Another advantage of the present invention is that users' locations can be geographically tracked, via a Global Positioning Satellite (GPS) system, cell- based triangulation, dead-reckoning (i.e., inertial tracking) or the like as described herein, in order to provide more realistic content, and thus more realistic interactive experience to users.

Another advantage of the present invention is that it provides a persistent, yet editable, gaming environment that allows users to define and customize their interactive application environment.

Yet another advantage of the present invention is that it maximizes the use of computing resources on targeted platforms and takes advantage of available bandwidth to create sustained scenarios that represent and interface with the real world.

Further features and advantages of the invention as well as the structure and operation of various embodiments of the present invention are described in detail below with reference to the accompanying drawings.

Brief Description of the Figures

The features and advantages of the present invention will become more apparent from the detailed description set forth below when taken in conjunction with the drawings in which like reference numbers indicate identical or functionally similar elements. Additionally, the left-most digit of a reference number identifies the drawing in which the reference number first appears.

FlG. 1 is a block diagram representing the system architecture of an embodiment of the present invention, showing connectivity among the parts;

FlG. 2 is a block diagram representing the system architecture of an embodiment of the present invention, highlighting the communications flow of the present invention; FlG. 3 is a flowchart depicting an embodiment of the operation and control flow of the multi-user bridging system of the present invention;

FlGS.4A-E are block diagrams representing an embodiment of a software framework useful for implementing the present invention; FlG. 5 is a diagram illustrating the structure of a sample help desk ticket according to an embodiment of the present invention; and

FlG. 6 is a block diagram representing the architecture of an orientationally-aware peripheral (OAP) device according to an embodiment of the present invention; FlG. 7 is a block diagram of an exemplary computer system useful for implementing the present invention.

Detailed Description of the Preferred Embodiments

I. Overview

The present invention is a distributed, platform-sensitive, location-based, contextual system, method and computer program product for bridging activities in real and virtual environments within the context of multi-user gaming, entertainment and e-commerce applications.

An application service provider (ASP), using the present invention, would utilize an infrastructure of hardware components connected over wireless networks and the Internet, and an infrastructure of telemetry, metering, monitoring, remote control, signaling and visualization software to create immersive, compelling and ubiquitous interactive, multi-user applications for business, government and consumer markets. The present invention takes advantage of low-cost, mass-marketed electronic devices, public networks and readily available spectrum space to create new, powerful capabilities that have not previously been envisioned or deployed. That is, the ASP may utilize a combination of centralized data-processing capabilities, software, personal computers, laptops, workstations, and autonomous agents on mobile devices to create scenarios that bridge mobile and remote users of the service with contextually relevant interfaces.

In one particular embodiment of the present invention, an organization provides a server (or collection of servers) accessible via a Web site, that facilitates an interactive, multi-player gaming environment application. That is, an ASP allows access, perhaps on a subscription or per-use basis, to a multi-user bridging tool via the global Internet. The ASP would provide the hardware (e.g., servers) and software (e.g., database) infrastructure, application software, content files, customer support, and billing mechanism to offer users (i.e., players) a new set of services and applications that bridge real-life ("physical") entities, features, spaces and events with computer-generated ("synthetic") environments, logic and processes based on relative position, motion and (real or virtual) orientation. Thus, the system of the present invention allows all entities to have a unique identity and stores synthetic entities in the same manner as physical entities. As suggested above, in an embodiment of the present invention, an ASP may provide users with access to the multi-user bridging tool of the present invention and charge on a subscription or per-use basis.

In an alternate embodiment of the present invention, the multi-user bridging tool of the present invention, instead of being accessed via the global Internet, would run locally on proprietary equipment and be networked among the local or wide area network (e.g., over an Ethernet, intranet, or extranet) of an entity allowing multiple users (e.g., employees of a single company that owns proprietary equipment) to access and use the multi-user bridging tool of the present invention. In an alternate embodiment of the present invention, each user device provides some or all of the functionality of the components of the multi-user bridging tool of the present invention as described herein. Such devices, as will be apparent to one skilled in the relevant art(s) after reading the description herein, would allow for distributed implementations of the present invention. In an alternate embodiment of the present invention, the client devices provide some or all of the functionality of the components of the content experience management tool as described herein. In such an alternate embodiment, the client devices would maintain connectivity with a centrally- managed, multi-user bridging tool or alternatively the devices would share data, as described" herein, among multiple devices (i.e., a "peer-to-peer" model). The present invention is described in terms of a gaming example. This is for convenience only and is not intended to limit the application of the present invention, hi fact, after reading the following description, it will be apparent to one skilled in the relevant art(s) how to implement the following invention in alternative embodiments (e.g., multi-user interactive applications focused on entertainment, project management, e-commerce, etc.). For example, in an alternate embodiment, a computer-aided design (CAD) application program executes within the multi-user bridging tool of the present invention while maintaining referential integrity between a real life (physical) environment (e.g., a field engineer) and a computer-generated (synthetic) environment (e.g., a remotely-located designer using a CAD program). This allows the creation of synthetic models based on physically-derived (or observed) data, the maintenance and enhancement of synthetic models as change occurs in the physical world, and most importantly, the real-time interaction between physical and synthetic entities (e.g., persons). As used herein, the term "event" shall refer to an occurrence in the real world (i.e., physical world), and the term "signal" shall refer to an occurrence or user stimulation that occurs in or originates from the virtual or synthetic world (e.g., from an interactive, multi-person application).

As used herein, the term "gaming" shall refer to any activity performed by a user on a client device which provides some entertainment value. Such activity ranges from participating in a synthetic environment with structured rules and roles, to simply forwarding a content file to another for entertainment purposes.

As used herein, "entity" shall refer to a physical user or any part of an synthetic environment that can be manipulated within an environment. The terms "user," "person," "player," "participant" and the plural form of these terms are used interchangeably to refer to those who would access, use, or benefit from the present invention.

II. Example System Architecture

Referring to FIG. 1, a block diagram illustrating the physical architecture of a multi-user bridging (MUB) system 100, according to an embodiment of the present invention is shown. FlG. 1 also shows connectivity among the various components of MUB system 100. It should be understood that the particular

MUB system 100 in FIG. 1 (i.e., a bridging system for an interactive, multi-player gaming application) is shown for illustrative purposes only and does not limit the invention. Other implementations for performing the functions described herein will be apparent to persons skilled in the relevant art(s) based on the teachings contained herein, and the invention is directed to such other implementations.

As will be apparent to one skilled in the relevant art(s), all of the components "inside" MUB system 100 are connected and communicate via a communication medium such as a local area network (LAN) or a wide area network (WAN) 101. MUB system 100 includes a plurality of application servers 102 (shown as servers 102a...n) that serve as the "middle-tier" (i.e., processing system) of the present invention. Servers 102, as explained in detail below, include the independent software components (e.g., agents, factories, and handlers) that implements the multi-user gaming operation of MUB system 100. While a plurality of separate servers are shown in FlG. 1, it will be apparent to one skilled in the relevant art(s) that the MUB system 100 may utilize one or more servers in a distributed fashion (or possibly mirrored for fault tolerance) connected via LAN 101.

Also connected to LAN 101 is an application database 104. This database, as explained in more detail below, stores information related to the players utilizing MUB system 100. Such information includes player registration, permission, ownership, and location information, as well as game environments and rules.

MUB system 100 also includes a plurality of administrative workstations 106 (shown as workstations 106a...n) that may be used by the MUB organization to update, maintain, monitor, and log statistics related to servers 102 and MUB system 100 in general. Also, administrative workstations 106 may be used "offline" by ASP personnel in order to enter configuration data and gaming rules, as described below, in order to customize MUB system 100 performance.

MUB system 100 also includes a translator (gateway) 108 which acts as the interface between the servers 102 and the external (i.e., outside of the ASP's infrastructure) devices. Consequently, translator 108 is connected to a firewall 110. Generally speaking, a firewall, which is well-known in the relevant art(s), is a dedicated gateway machine with special security precaution software. It is typically used, for example, to service connections and protect a cluster of more loosely-administered machines hidden behind it from an external invasion. Thus, firewall 110 serves as the connection and separation between the LAN 101 , which includes the plurality of network elements (i.e., elements 102-108) "inside" of LAN 101, and a transportation network 103 (e.g., the global Intemet) "outside" of LAN 101. Connected to the transportation network (e.g., global Internet 103), outside of the LAN 101, includes a plurality of external client devices 112 that allow users (i.e., players) to remotely access and use MUB system 100. External client devices 112 would include, for example, a mobile phone 112a, a video game console (with Internet connection) 112b, a personal data assistant 112c, a personal area network with video glasses and/or ear piece 112d, a laptop 112e, and a desktop computer 112f .

While only one translator 108 is shown in FlG. 1, it will be apparent to one skilled in the relevant art(s) that MUB system 100 may utilize one or more translators in a distributed fashion (or possibly mirrored for fault tolerance) connected via LAN 101. In such an embodiment, as will be apparent to one skilled in the relevant art(s) after reading the description herein, each translator 108 would be dedicated to, and support connections from, a specific type of external client device 112 using a different transportation network 103.

For example, in one embodiment of the present invention, translator 108 may be a Web server which sends out Web pages in response to Hypertext Transfer Protocol (HTTP) requests from remote browsers (e.g., desktop computers 112f). The Web server would provide the "front end" to the users of the present invention. That is, the Web server would provide the graphical user interface (GUI) to users of MUB system 100 in the form of Web pages. Such users may access the Web server at the MUB organization's site via the transportation network 103 (e.g., the Internet and thus, the World Wide Web).

Lastly, while one database 104 is shown in FIG.1 for ease of explanation, it will be apparent to one skilled in the relevant art(s) that MUB system 100 may utilize databases physically located on one or more computers which may or may not be the same as any of servers 102. More detailed descriptions of MUB system 100 components, as well as their functionality, are provided below.

///. Communications Flow

Referring to FlG.2, a block diagram 200 further illustrating the physical architecture 100 according to an embodiment of MUB system 100 is shown. More specifically, FlG. 2 illustrates a more simplified version of MUB system 100 than that shown in FlG. 1 in order to highlight the communications flow of the present invention. During operation of an instance of an interactive, multi-player game executing within MUB system 100, translator 108 acts as the interface between the players' client devices 112 (through transportation network 103 which is not shown in FlG. 2). That is, translator 108 facilitates communications between at least one of the servers 102 and the plurality of different client devices 112. Thus, translator 108 is responsible for translating (and thus bridging) between the game's signals and physical events into the protocol(s) being used by client devices 112 in order to communicate player movements, game rules, scene changes, player status, audio content, video displays, game score data, etc. Such player movements, scene changes, player status, audio content, video displays, etc. would be dictated by and/or stored in application database 104 in communication with servers 102.

As will be apparent to one skilled in the art(s) after reading the description herein, one or more translator(s) 108 would be needed to handle devices and software that do not natively communicate via (proprietary) protocols over TCP/TP. These include both existing first generation (IG) wireless data protocols such as Wireless Access Protocol (WAP), Cellular Digital Packet Data (CDPD) and Mobitex, as well as next generation technologies and standards (2.5G and 3G) such as General Packet Radio Service (GPRS), Enhanced Data rates for Global Evolution (EDGE), Universal Mobile Telecommunications System (UTMS) and Bluetooth. Translators are also needed for, Internet protocols such as Simple Mail Transfer Protocol (SMTP), HyperText Transfer Protocol (HTTP),

Simple Object Access Protocol (SOAP), Jini, Instant Messaging (IM), etc., in order for servers 102 to communicate (via the appropriate protocol transportation network 103) with the different types of client devices 112 (e.g., mobile phones, video game consoles, personal data assistants, ear-pieces, retinal projection display devices, etc.) and for clients 112 to communicate in a P2P fashion within

MUB system 100.

IV. Location Awareness

As will be apparent to one skilled in the relevant art(s) after reading the description herein, latitude, longitude and other sets of location data are integral to the applications (i.e., multi-user game play) executed within MUB system 100.

Such location awareness allows software agents to traverse physical terrains and physical entities such as people, buildings and vehicles to be represented in virtual worlds. Therefore, in addition to existing systems such as GPS and the like, inertial tracking can be used to track the location and orientation of players within system 100.

In an embodiment of the present invention, an orientationally-aware peripheral (OAP) device, described in detail below, may be included within MUB system 100 within each client device 112.

Referring to FlG. 6, a block diagram representing the architecture of an orientationally-aware peripheral (OAP) device 600 according to an embodiment of the present invention is shown. OAP device 600 consists of an inertial tracking subsystem 630 and a communication subsystem 620. Inertial tracking subsystem 630 employs six accelerometers that will track the placement and orientation of the peripheral device in six degrees of freedom ("6-DOF"). Such a design eliminates the need for separate gyroscopic sensors to determine orientation information. The six accelerometers are divided into three groups of two sensors each (i.e., accelerometers pairs 602, 604 and 606) oriented along each of three perpendicular axes. Each pair of accelerometers is separated as far as is possible on the platform. By correctly integrating the acceleration of all six sensors, both position and angular orientation can easily be calculated.

The above-described arrangement of accelerometers allows for the simple orientation and integration of the OAP device 600 in the client device 112, but should not be taken as a limitation of the invention. That is, other possible arrangements exist for locating a client device (and thus, a player), as will be apparent to persons skilled in the relevant art(s) based on the teachings contained herein, and the invention is directed to such other implementations.

For example, in an alternate embodiment, the six accelerometers are placed along the vertices of a triangular pyramid. Such an embodiment would either be a closed- or open-pyramid, with the six accelerometers along the vertices of the pyramid.

Inertial trackers tend to "drift" from the reference frame in time (via systematic- or bias-errors). These errors are cumulative. They are also subject to random errors (noise). Therefore, synchronization is important in ensuring OAP device 600 works in a wide range of environments. Synchronization would occur when OAP device 600 is brought to a known location and the system 100 is made aware of this fact. With OAP device 600 in a known location, its position can be reset, while expunging any current positional errors. Importantly, such synchronization can be brought into the narrative context of the game, making it an integral part of the action (as opposed to a distinct interruption).

For the pyramidal embodiment described above, synchronization can be performed by placing OAP device 600 at a known location and in a known orientation. Software code logic included in OAP device 600, in an embodiment, would assume that it is synchronized when OAP device 600 has not moved over a certain pre-selected time period.

Also, for certain applications offered by the ASP where accurate orientation is needed but positional data is not essential, it is possible that OAP device 600 could be self-synchronizing. That is, whenever the device is stationary for a pre-selected time period, a correction is applied so that the normal vector of the downward-facing face is aligned with the current gravitation vector.

As long as the device is placed on a flat surface fairly often in a random orientation, these corrections will be often in every direction; the net affect of these corrections would be a continual synchronization.

As mentioned above, OAP device 600 also includes communication subsystem 620, where the output of the inertial tracker is received by a data translator 614 and communicated to other client devices 112 participating in the same instance of the multi-player, interactive game. One embodiment of the communication subsystem 620 would employ wireless communication protocols (such as Bluetooth, IEEE 802.11 or the like) to communicate with a nearby computer or base-station (and thus with translator 108) via a transmitter 612.

V. Application Database

Application database 104 stores the various types of information that MUB system 100 would need to store in order to provide the bridging of activities in real and virtual environments in the context of multi-user gaming, entertainment and e-commerce applications. Such information, includes user registration information (name, address, billing information, etc.), device 112 registrations, device 112 capabilities (e.g., polygon rendering capability, media formats, operating systems, available peripherals, color versus black-and-white display, etc.), player permissions (e.g., who is allowed to create new characters) and player ownership of synthetic entities and environment objects, entity location information, game environments, game rules, themes and roles, etc., as will be apparent to one skilled in the relevant art(s) after reading the teachings herein. In an embodiment of the present invention, application database 104 is implemented using a relational database product (e.g., Microsoft^® Access, Microsoft^® SQL Server, IBM^® DB2^®, ORACLE^®, INGRES^®, or the like). As is well known in the relevant art(s), relational databases allow the definition of data structures, storage and retrieval operations, and integrity constraints, where data and relations between them are organized in tables. Further, tables are a collection of records and each record in a table possesses the same fields.

In an alternate embodiment of the present invention, application database 104 is implemented using an object database product (e.g., Ode available from Bell Laboratories of Murray Hill, NJ, POET available from the POET Software Corporation of San Mateo, CA, ObjectStore available from Object Design, h e. of

Burlington, MA, and the like). As is well known in the relevant art(s), data in object databases are stored as objects and can be interpreted only using the methods specified by each data object's class.

As will be appreciated by one skilled in the relevant art(s), whether application database 104 is an object, relational, and or even flat-files depends on the character of the data being stored by the ASP which, in turn, is driven by the specific interactive, multi-user applications being offered by the ASP. Server 102 contains specific code logic to assemble components from any combination of these database models and to build the required answer to a query. In any event, translator 108, client devices 112 , and/or administration workstation 106 are unaware of how, where, or in what format such data is stored. VI. Software Architecture

A. General Considerations

In an embodiment of the present invention, servers 102 can be implemented using Intel^® x86 or Pentium^® hardware running Microsoft^® Windows NT™ server platform or a Sun Ultra SPARC server running the Solaris operating system. Servers 102 execute a middle-tier software applications implemented in a high level programming language such as Java or the C++ programming languages. In an embodiment of the present invention, the software application communicates with database 104 using an MUB database handler implemented in C++ or Java.

In an embodiment of the present invention where transportation network 103 is the Internet and translator 108 is a Web sever, a secure GUI "front-end" for MUB system 100 is provided, hi an embodiment of the present invention, the front-end is implemented as a fully-rendered C++ environment, through a Web translator using the Active Server Pages (ASP), Visual BASIC (VB) script, and

JavaScript™ sever-side scripting environments that allow for the creation of dynamic Web pages, or through a translator to another client device.

A software framework for providing connectivity and maintaining referential integrity between physical and synthetic entities is crucial for MUB system 100 to support applications (e.g., interactive, multi-player games) that take advantage of external client devices 112 that are: (1) high-polygon-count hardware devices (e.g., game console 112b, etc.) to depict, navigate through, point at and interact with synthetic models of physical spaces and events; and (2) mobile, specialized devices for audio (e.g., MP3 player, etc.), video (e.g., digital video camera, videophone, etc.) and communications (e.g., mobile phone, PDAs, etc.). B. X2Y Software Framework

The bridge between the real world and virtual environments requires a distributed component framework to connect all client devices 112 and servers 102. Thus, the present invention contemplates, in one embodiment, a MUB system 100 software framework that allows scalability, as each component can be replicated any number of times. This also allows for the dynamic extension of the framework, and the connection of disparate systems without altering executable code.

Rather than integrating different applications, protocols, data types and standards into inter-related systems, the framework provides an efficient means of data collection, distribution and management. Also, rather than relying on users (participants) to have a particular application for sending, retrieving or presenting information, the framework provides an authenticated and encrypted distribution channel, or "component bus", through which participants have access to independent software agents (autonomous processes designed to handle particular tasks). These agents maintain the context of the information and relationships between participants.

Within the framework, ad hoc sessions can be established across any group of participants. Participants have access to common knowledge stores in multiple repositories at different locations. Any type of database or file server

104 can be connected to the framework, which applies flexible rule-sets to restrict or enable updating and retrieval of data, documents and multi-media files. Messaging, scheduling, reporting and logging all occur within the context of the working participants and the roles of the participants within that shared narrative, virtual environment, zone or mission. Unlike systems that require a central server or set of servers for providing applications, the MUB framework provides realtime and time-shifted information management, multimedia publishing, collaboration and communications services over an unlimited number of computers. Participants can access any client device 112 or software entity or reach any other participant from any client device 112 connected to the transportation network 103 such as the Internet.

The MUB framework allows participants to reach any data store or any other participant through any agent on the component bus. The framework has no central point of control or potential point of failure.

This new distributed component infrastructure allows any transportation network 103, such as the Internet, to evolve from segregated information systems, applications and protocols all running over the same pipeline to a collective knowledge and interactive services processor that harnesses the power of all the systems and users connected to it.

In an embodiment of the present invention, the X2Y framework provides:

• An encrypted, authenticated "object bus" (distribution channel) to connect diverse relational or object database management systems (RDBMS or ODBMS) owned by different entities doing business or playing games with each other across the Internet, private or virtual private networks.

• A document transportation subsystem to handle multimedia and complex binary and structured file formats, and a document transformation subsystem to dynamically create rich, structured documents from distributed data. • A contextual interface to provide an intuitive, comprehensive view of the diverse data repositories connected to the object bus. This view is dynamically created based on the user's role within the system so that the user sees only the appropriate data and documents presented in the clearest, most relevant layout to a particular task. • A real-time communications subsystem that allows users to collaborate with each other, using the data stores and document repositories as a shared environment.

• An automated software versioning and updating system so that all users are provided with the latest releases and most current information. In addition, MUB framework provides a robust security system to allow organizations or individuals to share information privately across the Internet.

Once users are authenticated, a session key is generated that encrypts all transmissions among participants and between data stores, including all documents, messages and real-time discussions.

A specific X2Y framework which can be implemented in MUB system 100, according to one embodiment of the present invention, is described in detail below.

1. Overview of X2Y Software Framework Operation

Referring to FlG. 4A, an X2Y software framework 400 according to an embodiment of MUB system 100 of the present invention is shown. The framework 400 employs a "context channel" which is a group-specific object bus that distributes software components, messages and media among participants and data repositories.

Each time a user logs into MUB system 100, an authentication agent 402 confirms their identity and generates a session key that is used to encrypt all information transfers, including database access, documents and real-time discussions. Login information is retrieved by authentication agent 402 through a

Lightweight Directory Access Protocol (LDAP) server 422 or from any other data store, allowing large groups of users to be added at once, in case another service or entity wishes to join the network.

After authentication, a presence factory 406 generates a personalized interface and delivers it to the user's computer. Presence factory 406 stores configuration information such as shortcuts and preferences and handles messages that were received while the user was off-line. Because presence factory 406 can store the participant's information on a high availability server, participants can use framework 400 with their personalized environment on any computer connected to transportation network 103 (e.g., a private network or the

Internet). A context factory 408 centrally maintains context-specific information, such as schedule messages used in the multi-user, interactive application (i.e., "service") calendars, files stored for sharing in the collaborative workspace and document profiles. When an authorized participant selects a service, an interface is generated that consistently reflects relevant issues, other service participants and available information repositories.

A notification agent 410 routes messages between participants and to participants from autonomous processes. Notification agent 410 accepts messages generated by "events" (through a participant's interaction with their interface) or "signals" (through activities caused by agents or systems) directed at participants.

A meeting manager 412 is a beacon (a type of agent) that establishes realtime (synchronous) connections between two or more users and then brokers the messages that comprise the discussions between all participants in a private or communal service meeting room.

In an embodiment, database handlers can be deployed to extend context- specific information spaces to multiple repositories. Participants would then have access to all information repositories with a unified view of resources. Translation agents can be deployed across framework 400 to transform structured documents (HTML, XML, etc.) into database formats and vice versa.

The main advantage of framework 400 over other systems is the ability to deploy a collaboration framework across multiple organizations while applying internal rules for conduct between participants and access to resources within each working group. Documents, media files and data can also be re-used across working groups without replicating resources.

Referring again to FlG. 4A, the communications flow between the components of the framework 400 is shown. A GUI 418 represents the local client. Through the login process, authentication agent 402 confirms the user's identity and generates a session key that is used for encryption. Login information is then retrieved by the authentication agent 402 through LDAP server 422 following which the presence factory 406 then delivers user persona data to the local client to generate GUI 418.

All files, documents and messages sent from the local client are received by notification agent 410 and then directed to the appropriate data store. User state checks are performed by either presence factory 406 or context factory 408 to verify the user is logged on to MUB system 100 prior to forwarding data to the

GUI 418.

Discussion requests are first initiated within GUI 418 and then sent to meeting manager 412 which returns a meeting ID to a client meeting manager 414. Client meeting manager 414 and meeting manager 412 then act to broker messages between participants during a real-time (synchronous) meeting.

More detailed descriptions of the X2Y software framework 400 and its components, as well as their functionality, are provided below.

2. Security Subsystem and Encryption

The security subsystem provides services throughout framework 400. It provides for the encryption of messages and files passed between users. It is also responsible for authenticating users within system 100. Any connection to the MUB framework service, whether it be to send or read messages, retrieve documents, create document profiles or open a collaboration window, has to be passed to authentication agent 402 for authorization before it is executed.

MUB framework 400 implements application-level security in which encryption is point-to-point between users and services, or between services and services. Data is encrypted at the point of origin, and left in encrypted form until it is accepted by its recipient. The data is decrypted once it is held in memory by the recipient (i.e., user or service). MUB framework 400 employs a strong security model for encrypting data passed between users and the system, which in an embodiment can be based on the Data Encryption Standard (DES) or any other means of encryption. DES, which is well known in the relevant art(s), is an encryption specification defined by the United States Department of Commerce's National Institute of Standards and Technology (NIST).

3. Authentication Agent

"Authentication" refers to any protocol the system relies on to verify that access to user accounts in MUB system 100 is limited to those actual users.

Authentication is usually provided by a password mechanism. In an embodiment, framework 400 provides for a strong authentication mechanism, including not only authentication at login, but also the authentication of users by the various system components when connections are made to their services (this authentication occurs in the background and is not evident to the user) as explained in more detail below.

Authentication agent 402 is a centrally important component in framework 400. It communicates directly with the an authentication data store of usernames, passwords, roles and other relevant user data via LDAP Application Program Interface (APT) 422 to establish user authentication at login, and maintains information on user state (whether and where the user is located, etc.) for other components during runtime. Such data store is LDAP server 422, which is a relational database, object database, flat file or other information store.

a. Authentication at Login

The first time authentication agent 402 is invoked in a session is at user login. When a user connects to MUB system 100, a login GUI 418 appears which accepts the username and password from the user. The program also starts an internal component, a notification handler 416, enabling it to receive messages from the system.

The password itself is never sent across the network. Instead it is used on client device 112 as an encryption "key" for the current session. Authentication agent 402 (which has its own record of the password) encrypts a test message (a random data segment) using this key, and sends it to the user's notification handler 416, which decrypts it and sends it back. If the returned test message is correct, authentication agent 402 sends a different (random) data segment called a session key (also encrypted for transmission), which is subsequently used on both sides to encrypt all exchanges during the session. The session key becomes invalid when the session is closed.

This model effectively prevents capture of data from the "sniffing" or monitoring of data lines. Even in the unlikely event that the session key is discovered and session data decrypted, the decryption will not work on later or earlier sessions. For such decrypting operations, the combination of time and computing overhead required is prohibitive.

In an embodiment, authentication information is held in a separate repository, but in a format compliant with LDAP server 422, and provided to authentication agent 402 via LDAP API 404. This model enables the authentication process to be extended to other applications compliant with this standard, enabling one login for multiple services.

b. Authentication During Runtime

Authentication also occurs during runtime. Whenever a user tries to establish a link with any another component within system 100, that component contacts authentication agent 402 and queries it as to the user's present status. The connection is established and the service delivered only if authentication agent 402 confirms the user's identity. In an embodiment, this is achieved by comparing its record of username, JP address and port, login date and time, and session key to records provided by the component. 4. Database Handlers

In an embodiment, a database handler (not shown in FlG. 4A) is an integrating component that resides on server 102 and brings a database 104 into system 100 by providing connectivity to other components within the framework 400. The database handler' s function is to translate various requests for services from other components in framework 400 into specific queries which can be delivered to database 104, and then to relay the data returned back to the asking component. In practice, the introduction of the database handler maximizes the interoperability of components that need access to this data, since they can share a communications protocol (e.g., the proprietary Unicode protocol (PUP) described in Section VI.B.17.b below). Database 104 can then be upgraded, replaced or supplemented by one or more additional databases (including legacy systems) without the need to reprogram other components of framework 400 (or system 100 in general).

5. Asynchronous Messaging Subsystem

Asynchronous messaging refers to messages sent by users to one another, or to a service in general (meaning the message will be accessible to any user with access to the service), and which have persistence in system 100 (they remain stored in the system until intentionally deleted). Asynchronous messaging does not require that the recipient(s) be logged in (or presently connected) to MUB system 100 in order to receive the message. They are simply stored in the corresponding database 104 (i.e., presence factory 406 (i.e., repository 430) or context factory 408 (i.e., repository 426), as described in Sections VI.B.6 and

VI.BJ, respectively, below) and then made available to the user when logged onto system 100. In an embodiment, asynchronous messages supported include service messages, requests for on-line meetings (see Synchronous Communications, Section VLB.9 below), and scheduling messages (used to schedule events with other users). FlG. 4B represents an overview of the asynchronous message process within framework 400.

6. Presence Factory & Message Repository

Presence factory 406 is the component within framework 400 responsible for storing information pertaining to specific user's configurations of messages addressed to the user, and shortcuts to services. Presence factory 406 includes a presence message repository 430 and a process 428 that is used to access and store this information (see FlG. 4A).

Upon receiving a message, presence factory 428 directs a copy of the message to the presence message repository 430 for storage. Presence factory 428 then performs a "state check" against authentication agent 402, checking whether each recipient is actually logged in (and if so, where). For each recipient currently logged in, presence factory 428 sends a copy of the message to the appropriate notification handler 416 (see Section VLB.12 below). As a result, messages are updated dynamically, appearing within the recipient's interface on their client device 112 as soon as it is received.

In an embodiment, users' configuration information is stored in the presence message repository 430 as a collection of ASCII text files structured according to their message type. Because this information is centrally stored and delivered to users upon login, users gain the capability to access their materials from any interface.

In an embodiment, the underlying organizational structure for the presence message repository 430 is a directory tree held within a root subdirectory identified in a "system.ini" file. System data stores and document storage are organized within two structures, one assigned to services and one assigned to users.

User data stores (holding messages to be directed to individual users) are held within the structure:

%system%/persona/username where %system% designates the root subdirectory identified in the setup.ini file. The username directory is further subdivided:

%system%/persona/username/messages %system%/persona/username/shortcuts Messages sent to the user are stored within the messages directory while user interface shortcut settings used for accessing services are stored within the shortcuts directory.

7. Context Factory and Message and File Repository

While presence factory 406 maintains information related to user messages, context factory 408 maintains information related to group messages.

As will be apparent to one skilled in the relevant art(s), groups of users may represent players on a specific team within an interactive, multi-user application, users within a specific room within a synthetic environment, or any other contextual-based logical grouping of participants within an instance of a service executing on MUB system 100. In addition, context factory 408 also maintains information about files stored in the services workspace, and profiling information. Context factory 408 is responsible for delivering all of this information to the GUI 418 when a service is accessed.

Similar to presence factory 406, context factory 408 updates messages to recipients' interfaces dynamically. Since context factory 408 handles messages that are not addressed to specific users, but rather to the service as a whole, the necessary "state check" performed is more complex than that of presence factory 406.

Context factory 408 includes a context message and file repository 426 and a process 424 that is used to access and store information (see FlG. 4A).

Context factory 408 maintains a list of users who have retrieved data from it at any given time (i.e., current "listeners"). As a message is received, context factory 424 directs a copy of the message to the context message and file repository 426. Context factory 424 then performs the state check against all users currently on the service and for which the message is targeted. If the user is currently logged in, the message is sent to that user's notification handler 416

(Section VLB.12 below) for update in GUI 418. If the user is currently not logged in, an exception is passed which establishes the user as being logged out (a process which includes updating the list of service "listeners") and the message will be sent to the user's interface the next time they log in and open the service.

The context message and file repository 426 functions like the presence message repository 430, with an added service: it stores files, public and private, accessible through the service in GUI 418. Subdirectories are organized according to a "service.context" code notation, in which each project is assigned a unique ID by virtue of its being attached to a specific game or organizational scheme. For example, the service.context code is represented as follows:

All 111.22222 where A 11111 represents the six-digit service number, and 22222 represents the five-digit context ID number.

The path name then, is named according to the following structure: %system%/service_no/context_ID or, for this example: %system%/Al 1111/22222

(again, where %system% designates the MUB framework 400 root subdirectory identified in the setup.ini file). The contextJGD directory is further subdivided:

%system%/service/context_ID/messages/ %system%/service/context_ID/workspace/

%system%/service/context_ID/docs/

The "messages" directory represents the context message repository 426 in which copies of all messages sent to the service (contextJQD) are stored. Copies of all documents that have been registered through the framework 400 document profile are placed in the "docs" directory. The file names reflect the numeric profile_ID assigned to the document when it is profiled.

The "workspace" directory is divided into a private and public area for storage of work in progress and documents to be shared within the service as shown below:

%system%/service/context_ID/workspace/private/ %system%/service/context_ID/workspace/public/

8. Notification Agent

Notification agent 410 is a component responsible for routing messages within framework 400. That is, notification agent 410 acts as a messaging relay or broker, accepting a command from one system 100 component and directing it to either presence factory 406 or context factory 408, designated for the appropriate recipient(s). Specifications for the PUP protocol, as it applies to notification agent 410, appear in Section VLB.17.b below.

Synchronous Communications Subsystem

"Synchronous" communications are communications that happen in "real time ' That is, synchronous communications occur when all participants are logged in at once, and can confer directly with one another. A telephone conference call is an example of a synchronous communication, h an embodiment of the present invention, synchronous messaging provided by MUB framework 400 is generally text-based, but voice and video conferencing is also supported. Synchronous communications ("chat") sessions are initiated by means of asynchronous messages ("chat requests") sent to the invited users. If a user is logged in, he or she has the option of joining the session in progress. 10. Meeting Manager

Meeting manager 412 is a central component within MUB framework 400 that sets up (establishes) meetings and brokers messages directed to meetings. Meeting manager 412 has the capability to send and receive PUP (Section VLB.17.b below) messages and also has a built-in notification agent which is used in relaying meeting text messages to the notification handlers 416 (Section VLB.12 below) of the meeting participants. When a new meeting is initiated, meeting manager 412 is also responsible for generating a meeting ID, which is unique to each meeting and ensures that messages are directed to the correct meeting. Only one meeting manager 412 exists within MUB framework 400, whether or not meetings are occurring within the system.

11. Client Meeting Manager

While only one meeting manager 412, responsible for all meetings, exists within MUB framework 400, there may be several client meeting managers 414. Client meeting manager 414 acts as a "relay" between a user's meeting frame

(i.e., GUI window or display) on the user's interface 418 and meeting manager

412. One client meeting manager 414 exists for each user logged into a meeting.

The main functions of client meeting manager 414 include accepting incoming meeting messages from a user's notification handler 416 (Section VLB.12 below) and sending them to the user's meeting frame; and accepting outgoing messages from the user's meeting frame and sending them to meeting manager 412 for distribution. (See FlG. 4D.) If a user is logged into more than one meeting, the user's client meeting manager 414 also becomes responsible for directing meeting messages to the proper meeting frame on the user's interface 418. 12. Notification Handler

Notification handler 416 integrates GUI 418 into framework 400 by enabling it to receive PUP messages. Specifications for the PUP protocol appear in Section VI.B.17.b below.

13. Synchronous Communications Process

Referring to FlG. 4C, a new meeting is initiated when a user sends a discussion request message to other system users and/or services. The request is sent to meeting manager 412 as a PUP (Section VI.B.17.b below) command requesting the meeting manager 412 to establish a new meeting. Meeting manager 412 responds by returning a unique meeting ID. At this time, the user's client meeting manager 414 is created if it does not already exist.

The discussion request message is then sent to notification agent 410 which forwards it to presence factory 406 (i.e., repository 430) and/or context factory 408 (i.e., repository 426) for storage (depending on whether the discussion request was directed toward users and/or services). From each factory (406 and

408), a copy of the message is then routed to those users currently logged into system 100. Client meeting manager 414 for each recipient then receives, in an embodiment, the discussion request message and through a dialog box on GUI 418, prompts the user to accept or decline joining the meeting. If the user joins the meeting, a new meeting frame is created that allows the user to interact within the meeting.

Meeting text messages originate in the meeting frame and are sent to meeting manager 412 via client meeting manager 414. Meeting manager 412 maintains a list of all meetings and participants and is then responsible for directing the message to each user logged into the meeting. An overview of the synchronous communications dialog within framework 400 is illustrated in FIG. 4D. 14. X2Y Framework Data Stores and Data Structures

Contact Information - A contact is considered to be someone associated with a service that may not be a direct participant in the system. Therefore, in an embodiment, a contact database maintains information with regard to contacts and their association with a service and/or relation to.a context. Generally, users may be contacts, but contacts are not users by default.

File Profiles - A profiles database 420 contains a range of information pertinent to managing projects in MUB framework 400. Profiles will include primary profiles of documents; in principle, however, profiles maybe configured to refer to any type of information, such as contacts (people) or financial information.

Because many organizations (e.g., ASPs) have systems in place to collect and coordinate this information (in particular, contact and billing information), the information management capabilities provided by MUB framework 400 through profiles database 420 provide considerable opportunities for integration of services within the organization. There are three models for managing this information in conjunction with framework 400. The three models may be mixed as appropriate to the requirements of the organization.

The first model is to provide framework 400 with read write access to a legacy system. The database handler can be fitted to any Open Database

Connectivity (ODBC) or Java Database Connectivity (JDBC) compliant database. Built on this infrastructure, interfaces can be configured to allow authorized users direct access to the legacy system through MUB framework 400. Authentication protocols and checks can be put in place to secure access. The second model is to provide read-only capability to address legacy systems. This may be achieved either by providing framework 400 with a database handler to communicate with the legacy system on-the-fly (as in the first model), except disabling write capability; or by performing a periodic data migration from the legacy system into a compliant database. The latter approach may also be adapted on an ad-hoc basis for the import of project-specific data, such as document profiles created in another application program.

The third model is to duplicate or divide services between the two above- described systems.

15. Help Notices to Application Service Provider

In general, help notice ticket tracking provides a method for MUB framework 400 users to submit help requests and/or problem notifications. Each request is a ticket that is stored as a file in a directory structure which MUB framework 400 then tracks through use of a ticket tracking component (i.e., window) of GUI 418. FlG.5 illustrates the structure of a sample help desk ticket 500. The open format to store tickets allows for easy modification of tracking information for various implementations.

16. LDAP Database

The Lightweight Directory Access Protocol (LDAP) is a directory service protocol that runs over TCP/IP for accessing information directories. LDAP is an open Internet standard developed by the Internet Engineering Task Force (IETF) and is based on the X.500 standard that defines how global directories should be structured. The open standard, in short, eliminates proprietary control over the protocol.

There are many different ways to provide a directory service, but LDAP is reasonably simple while providing a wealth of features capable of supporting any kind of application. LDAP offers both access and update capabilities, search capabilities, and also includes authentication that enables the security of directory data via access controls. In addition, LDAP is one of the very few protocols that have a well documented, easy-to-use API. LDAP API 404 is separate from the protocol and is currently defined in Request for Comments (RFCs) 1777, 1778 and 2259 (Internet Architecture Board).

Like X.500, LDAP bases its directory model on entries, where a distinguished name (DN) is used to refer to an entry without ambiguity. But rather than use the highly structured X.500 data encoding, LDAP adopts a simple, string-based approach for representing directory entries. This simplifies an application's requirement for processing directory entries, as the strings use a minimal subset of Basic Encoding Rules (BER). The full DN format is currently described in RFC 2253.

Framework 400 makes use of the LDAP protocol for the user authentication process. This allows the authentication process to be extended to other applications compliant with this standard, enabling one login for multiple services.

Shown below in TABLE 1 is an excerpt from the "*.ldif ' file used to build LDAP server 422. The file excerpt represents three DN entries, each of which consist of multiple attribute/value pairs. The entries shown represent a typical LDAP schema configuration for an implementation.

subtreeaci: +(&(privilege=write)

(client=ldap :///uid=dirmgr, o=Your Company,c=US)) dn: cn=John Doe, o=Your Company, c=US objectclass: top objectclass: person objectclass: organizationalPerson objectclass: inetOrgPerson en: John Doe sn: Doe givenname: John title: No Title Given uid: jdoe userpassword: jdoepw

The first entry represents a typical directory manager, the second is a typical organization entry and the third is a typical user entry. Both the directory manager and the user are members of the organization "Your Company", and both are members of the object class "inetOrgPerson", but the directory manager has additional directory privileges over the typical user. These privileges are defined in the "subtreeaci" attribute of the organization entry.

17. Communications Protocols

A protocol, generally defined, is a set of mutually accepted rules and procedures that apply to both ends of a communication channel. Protocols allow for reliable communication between internal framework 400 components and from components to external entities such as databases.

a. TCP/IP

In an embodiment of the present invention, all messaging between components within MUB system 100 occurs in TCP/IP, a publicly specified protocol suite that ensures the operation of framework 400 components across public and private networks compliant with the Internet. The Internet Protocol (IP), currently defined in RFC 791, is a network-level, packet (i.e., a unit of transmitted data) switching protocol. The Transmission Control Protocol (TCP), currently defined in RFC 793, is a transport-level, flow control protocol.

TCP/IP is widely used today because it has the ability to connect computer systems regardless of their individual hardware or software configurations (heterogeneous systems). TCP/IP transmits information in "packets." These packets have the ability to travel different routes to reach their destination. Both TCP and IP rely on data encapsulation, data routing across networks, data exchange across non-uniform platforms, and fragmentation and re-assembly of data. Both also participate in the layered approach to communication. TCP is responsible for fragmenting data (breaking into manageable chunks), reassembling the data back in the correct order once it reaches its destination, and re-sending anything that gets lost. IP is responsible for routing the packets of data from node-to-node once broken-up (fragmented) by TCP. IP forwards each packet based on its four byte destination address (the IP number). TCP treats the data as a stream of bytes. It logically assigns a sequence number to each byte, which is identified in the TCP packet header. TCP acknowledges data that has been received and retransmits data that has been lost. The TCP design means that error recovery is done end-to-end between the client and server machine.

b. Proprietary Unicode Protocol

In an embodiment of the present invention, a proprietary Unicode protocol (PUP) specification supports all communication within MUB framework 400. The protocol is based on Unicode (which is described in Section VI.B.17.C below): each message is described and associated with a straight Unicode text value (e.g., "MODIFY"). This allows for unlimited, human-readable messages.

Java software components call native read/write Unicode text field methods for input and output of PUP messages. Messages between components are classified into two types—Events and Signals. "Events" are messages generated by users logged into framework 400 and sent to an internal component. For example, a message sent to meeting manager 412 to request the initiation of a meeting (synchronous messaging window) takes the form of an event. (See FlG. 4C.) Events in MUB framework 400 are generated exclusively through GUI 418.

"Signals" are messages sent by an internal component to other internal components, including notification handler 416. h the above example, messages sent by meeting manager 412 to invite participants to a meeting (initiated by a user's event message) take the form of signals.

TABLE 2 lists the PUP base commands, while TABLE 3 lists an MUB framework implementation of PUP commands according to one embodiment of the present invention.

TABLE 3 - THE MUB FRAMEWORK IMPLEMENTATION OF PUP

c. Unicode

Unicode is a two-byte (i.e., 16-bit) character set that contains all of the characters commonly used in information processing. It was developed by the Unicode Consortium which consists of individual experts and information technology companies such as Apple, Hewlett Packard, Microsoft, Sun

Microsystems, Digital, and IBM. Unicode integrates ASCII, Latin character sets and Far Eastern languages into a single character set. The coding system was designed to support the interchange, processing, and display of the written texts of the many diverse living languages throughout the world. MUB framework 400 is Unicode compliant in the sense that a high-level programming language such as the Java language incorporates Unicode as its primitive built-in character type and the language has included APIs that have the ability to adapt text, numbers, dates, currency, and user-defined objects to any country's conventions. As described in the previous section, the framework's PUP commands used to communicate between the MUB framework 400 components utilize the Unicode character set.

Information about the current Unicode standard is available from the Unicode Consortium of Mountain View, CA (www.unicode.org).

d. Web Translation Protocol

In an embodiment of the present invention, a proprietary Web translation protocol (WTP) may also be utilized. At an abstract level, the purpose of WTP is to form a Web-compliant shell around Java objects. The protocol is a cross between a scripting language (such as JavaScript or AppleScript) and a macro language (such as a C pre-processor). The shell provides a simple yet flexible template-based information delivery system.

WTP templates can be used to generate many types of objects, hi the case of HTML, for example, instead of appearing in specially delimited sections of the document (as does JavaScript), WTP verbs appear as part of the HTML markup itself.

C. Relation of Software and System Architectures

As will be apparent to one skilled in the relevant art(s) after reading the teachings herein and referring to FIG. 1 and FlG. 4A, in one embodiment of the present invention, the code logic representing authentication agent 402, notification agent 410, meeting manager 412, LDAP API 404, presence factory 428 and context factory 424 may reside and execute on servers 102; LDAP server

422, repository 426, repository 430 and profiles database 420 may reside on application database 104; and client meeting manager 414, notification handler 416, and GUI 418 execute and reside on client devices 112 (e.g., PC 112f).

Referring to FlG. 4E, such an allocation of X2Y framework 400 components among the MUB system 100, according to an embodiment of the present invention, is shown. More specifically, the framework 400 components allocated to the server 102, database 104 and translators 108 are shown. Further, translator 108, in an embodiment, includes an authentication translator 436, a notification translator 438 and an other protocols translator 440 in order for servers 102 to communicate (via the appropriate protocol transportation network

103) with the different types of client devices 112.

In an embodiment of the present invention, as will be appreciated by those skilled in the relevant art(s), client meeting manager 414 and notification handler 416 software modules of framework 400 would be delivered to a user's client device 112 (e.g., PC 112f) upon first connecting to the ASP's MUB system 100 and registering as a user (player). Upon registration, a virtual representation (i.e., "presence") of each user is created within the instance of the interactive, multiuser application. Thus, framework 400 (and system 100 as a whole) is the solution to the problem of maintaining referential integrity between the physical and synthetic environments. The creation of synthetic models based on physically-derived data, the maintenance and enhancement of synthetic models as change occurs in IRL, and the real-time interaction between physical and synthetic entities, can all be accomplished by the present invention.

VII. Example System Operation

A. Gaming Example

Referring to FlG.3, a flowchart depicting an embodiment of the operation and control flow 300 of MUB system 100 of the present invention is shown. More specifically, control flow 300 depicts, in flowchart form, an example of multiple users in both the physical and synthetic worlds being bridged during the execution of one instance of an interactive multi-user gaming application. The description of FlG.3 is presented with particularized reference to individual MUB system 100 components. Control flow 300 begins at step 302, with control passing immediately to step 304.

In step 304, a user on a PC client device 112f ("PC user") designs a new character for the instance of an interactive, multi-user gaming application being executed within MUB system 100. As will be apparent to one skilled in the relevant art(s), after reading the teachings herein, one of the servers 102 within MUB system 100 would ensure (by checking database 104) that the PC user had

"creation" permissions within the instance of the interactive, multi-user gaming application being executed (i.e., played). Such a new character is termed an electronic life form (ELF) within the instance of the interactive, multi-user gaming application. Each ELF can be classified in terms of three definitions: (1) role - this encapsulates the role of that person or character (e.g. manager, administrator, guardian, wizard, secretary, etc.); (2) attributes - this encapsulates the person or character within the synthetic environment (e.g., hair color, eyes, description, inventory, location, etc.); and (3) name - which is the identifier used when registering the ELF with MUB system 100. In an embodiment of the present invention, such user would design a

"monster" character using one or more of the following steps: (a) use graphics software such as 3D Studio to create a 3D visual representation of the "monster" character; (b) use a JPEG file to create a 2D visual representation of the "monster" character; (c) create an MP3 file that contains audio content (i.e., sounds) that the "monster" character makes; (d) type text associated with the "monster" character (e.g., "85 Ft. Monster"); (e) use any commercially available gaming character creation utilities to create the "monster" character (e.g., www.creaturelabs.com by CyberLife Technology Ltd. of Cambridge, England); (f) define user response rules to the "monster" character (e.g., "pressing *9999 will kill "monster" in 30 seconds); and (f) define how the "monster" character moves within the synthetic environment (e.g., x,y position to x',y' position at z rate).

In step 306, the PC user would register the new "monster" character with MUB system 100. That is, the communications flow described with reference to FlG. 2 would allow the server 102 to centrally store the attributes of the new character in application database 104.

In step 308, server 102 would cause the new "monster" character to be delivered to all other users playing the same instance of the interactive multi-user gaming application as the PC user. Such deliver would be affected by translator 108, under the control of server 102, via transportation network 103. Further, the server would place the new "monster" character in a PC user-dictated location within the synthetic environment, say for example, Wall Street of New York City.

As one skilled in the relevant art(s) would appreciate after reading the description herein, the PC user would need to have "creator" rights within the specific instance of the interactive multi-user gaming application in order to create the new "monster" character in step 308. Such rights would be dictated by the identity, permissions, and gaming rules stored by MUB system 100 in application database 104. hi step 310, a user on a laptop client device 112e ("laptop user") would now "see" the new "monster" character on their laptop. More specifically, the laptop user would see the "monster" character on the synthetic representation of

Wall Street in New York City. MUB system 100 ensures that the "monster" character is properly rendered for each user utilizing a different type of client device 112.

In step 312, the laptop user sends a message to a user on a mobile phone client device 112a ("mobile user"). Such message, for example, would convey that "a new 'monster' character is two blocks from you." This message may be sent because the mobile user is represented in the synthetic environment as being on Wall Street in New York City because in the physical world, they are.

In step 314, the mobile user receives a signal (e.g., audio indication, text message, voice mail message, graphic display, etc.) on client device 112a reflecting the laptop user's message sent in step 312.

In step 316, the mobile user can interact with "monster" character (i.e., manipulate the "monster" character entity). Such interaction would involve, for example, pressing *9999 on their mobile phone client device 112a to kill the "monster" character, hi step 318, the synthetic representation of the "monster" character would disappear from the PC user's, laptops user's and mobile user's client devices. Again, MUB system 100 would ensure that the "monster" character's death would be properly rendered (using the proper signal) for each player's different type of client device.

Control flow 300 then ends as indicated by step 320.

B. Alternate Embodiments

It should be understood that control flow 300, which highlights the functionality, scalability, and other advantages of MUB system 100, is presented for example purposes only. The architecture of the present invention is sufficiently flexible and configurable such that users may utilize MUB system 100 in ways other than that shown in FlG. 3. Such alternate embodiments are presented below.

In one embodiment, users of MUB system 100 may further bridge the synthetic environment with the physical environment. More specifically, in step

316 of flow 300, the mobile user may have taken a taxi in order to "run away" from (i.e., interact with) the "monster" character. If the mobile user also possessed a video camera client device 112, the video stream of the taxi ride may be uploaded to server 102 (via transportation network 103 and translator 108), so that the video stream of the mobile user running away from the "monster" character may be seen on the PC user's and laptops user's client devices.

In another embodiment of the present invention, as one skilled in the relevant art(s) will appreciate after reading the description herein, if the mobile user's taxi ride takes them outside of the Wall Street area of New York City, then the synthetic representation of the mobile user would disappear from the PC user' s and laptops user' s client devices.

In another embodiment of the present invention, as one skilled in the relevant art(s) will appreciate after reading the description herein, a user may create an MP3 file that contains audio content (e.g., a recorded voice message) that is played on a registered client device owned by another player when that player enters a specific area of the synthetic or physical environment. For example, the PC user could specify that the "monster" character speaks each time another player enters a specific building located on Wall Street in New York City. That sound would be played, for example, on a player' s mobile phone 112a when they walk into the physical building, or on a players PC 112f speaker when a player's synthetic representation walks into the specified building.

In yet another embodiment of the present invention, as one skilled in the relevant art(s) will appreciate after reading the description herein, application database 104 would contain billing information (i.e., address, telephone, credit card or bank account number) for each player registered with the ASP providing MUB system 100. This would allow players to actually incorporate financial transactions into the synthetic and physical environment bridging of the interactive multi-user gaming application being executed (i.e., played). More specifically, using the above taxi ride example, the mobile user could charge the PC user for the physical environment taxi ride he was forced to take in order to run away from the synthetic environment "monster" character. VIII. Environment

The present invention (i.e., MUB system 100, flow 300, MUB framework 400, device 600 or any of the parts thereof) may be implemented using hardware, software or a combination thereof and may be implemented in one or more computer systems or other processing systems. In fact, an example of a computer system 700 is shown in FlG.7. The computer system 700 represents any single or multi-processor computer. In conjunction, single-threaded and multi-threaded applications can be used. Unified or distributed memory systems can be used. Computer system 700, or portions thereof, may be used to implement the present invention. For example, the System 100 of the present invention may comprise software running on a computer system such as computer system 700.

In one example, the System 100 of the present invention is implemented in a multi-platform (platform independent) programming language such as JAVA™, programming language/structured query language (PL/SQL), hyper-text mark-up language (HTML), practical extraction report language (PERL), common translator interface/structured query language (CGI/SQL) or the like. Java™-enabled and JavaScript™-enabled browsers are used, such as, Netscape™, HofJava™, and Microsoft™ Explorer™ browsers. Active content Web pages can be used. Such active content Web pages can include Java™ applets or ActiveX™ controls, or any other active content technology developed now or in the future. The present invention, however, is not intended to be limited to Java™, JavaScript™, or their enabled browsers, and can be implemented in any programming language and browser, developed now or in the future, as would be apparent to a person skilled in the relevant art(s) given this description.

In another example, the System 100 of the present invention, may be implemented using a high-level programming language (e.g., C++) and applications written for the Microsoft Windows™ NT or SUN™ OS environments. It will be apparent to persons skilled in the relevant art(s) how to implement the invention in alternative embodiments from the teachings herein. Computer system 700 includes one or more processors, such as processor 744. One or more processors 744 can execute software implementing the routines described above. Each processor 744 is connected to a communication infrastructure 742 (e.g., a communications bus, cross-bar, or network). Various software embodiments are described in terms of this exemplary computer system.

After reading this description, it will become apparent to a person skilled in the relevant art how to implement the invention using other computer systems and or computer architectures.

Computer system 700 can include a display interface 702 that forwards graphics, text, and other data from the communication infrastructure 742 (or from a frame buffer not shown) for display on the display unit 730.

Computer system 700 also includes a main memory 746, preferably random access memory (RAM), and can also include a secondary memory 748. The secondary memory 748 can include, for example, a hard disk drive 750 and/or a removable storage drive 752, representing a floppy disk drive, a magnetic tape drive, an optical disk drive, etc. The removable storage drive 752 reads from and/or writes to a removable storage unit 754 in a well known manner.

Removable storage unit 754 represents a floppy disk, magnetic tape, optical disk, etc., which is read by and written to by removable storage drive 752. As will be appreciated, the removable storage unit 754 includes a computer usable storage medium having stored therein computer software and/or data.

In alternative embodiments, secondary memory 748 may include other similar means for allowing computer programs or other instructions to be loaded into computer system 700. Such means can include, for example, a removable storage unit 762 and an interface 760. Examples can include a program cartridge and cartridge interface (such as that found in video game console devices), a removable memory chip (such as an EPROM, or PROM) and associated socket, and other removable storage units 762 and interfaces 760 which allow software and data to be transferred from the removable storage unit 762 to computer system 700. Computer system 700 can also include a communications interface 764. Communications interface 764 allows software and data to be transferred between computer system 700 and external devices via communications path 766. Examples of communications interface 764 can include a modem, a network interface (such as Ethernet card), a communications port, interfaces described above, etc. Software and data transferred via communications interface 764 are in the form of signals which can be electronic, electromagnetic, optical or other signals capable of being received by communications interface 764, via communications path 766. Note that communications interface 764 provides a means by which computer system 700 can interface to a network such as the

Internet.

The present invention can be implemented using software running (that is, executing) in an environment similar to that described above. In this document, the term "computer program product" is used to generally refer to removable storage unit 754, a hard disk installed in hard disk drive 750, or a carrier wave carrying software over a communication path 766 (wireless link or cable) to communication interface 764. A computer useable medium can include magnetic media, optical media, or other recordable media, or media that transmits a carrier wave or other signal. These computer program products are means for providing software to computer system 700.

Computer programs (also called computer control logic) are stored in main memory 746 and/or secondary memory 748. Computer programs can also be received via communications interface 764. Such computer programs, when executed, enable the computer system 700 to perform the features of the present invention as discussed herein. In particular, the computer programs, when executed, enable the processor 744 to perform features of the present invention. Accordingly, such computer programs represent controllers of the computer system 700.

The present invention can be implemented as control logic in software, firmware, hardware or any combination thereof, hi an embodiment where the invention is implemented using software, the software may be stored in a computer program product and loaded into computer system 700 using removable storage drive 752, hard disk drive 750, or interface 760. Alternatively, the computer program product may be downloaded to computer system 700 over communications path 766. The control logic (software), when executed by the one or more processors 744, causes the processor(s) 744 to perform functions of the invention as described herein.

In another embodiment, the invention is implemented primarily in firmware and/or hardware using, for example, hardware components such as application specific integrated circuits (ASICs). Implementation of a hardware state machine so as to perform the functions described herein will be apparent to persons skilled in the relevant art(s) from the teachings herein.

IX. Conclusion

While various embodiments of the present invention have been described above, it should be understood that they have been presented by way of example, and not limitation. It will be apparent to persons skilled in the relevant art that various changes in form and detail may be made therein without departing from the spirit and scope of the invention. This is especially true in light of technology and terms within the relevant art(s) that may be later developed. Thus, the present invention should not be limited by any of the above-described exemplary embodiments, but should be defined only in accordance with the following claims and their equivalents.

Claims

What Is Claimed:

1. A system for bridging a physical environment and a virtual environment in order to facilitate the execution of an interactive, multi-user application, comprising: (a) an application database wherein said application database stores information about a plurality of users and one or more client devices owned by each of said plurality of users;

(b) a plurality of servers, each connected to said application database;

(c) a plurality of translators, each connected to one of said plurality of servers, wherein each of said translators supports connections between at least one client device and at least one server; and

(d) a plurality of transportation networks, each connected to one of said plurality of translators; wherein the system allows the creation of scenarios that bridge the virtual environment representation of each of said plurality of users with the physical environment through contextually relevant interfaces.

2. The system of claim 1, wherein at least one of said transportation networks is at least a portion of the Internet and at least one of said translators is a Web server.

3. The system of claim 1 , wherein at least one the client devices includes an orientationally-aware peripheral device to allow the system to account for the physical location and context of one of the plurality of users who owns said one of the client devices.

4. The system of claim 1, wherein the client devices comprises at least one of the following:

(i) a mobile phone; (ii) a video game console; (iii) a personal data assistant; (iv) a personal computer; (v) eye phone; and (vi) video camera.

5. The system of claim 1, wherein said application database, comprises: a presence message repository that stores messages that were received for each of said plurality of users when they are off-line; and a context message and file repository that stores messages targeted to logical groupings of said plurality of users, and files related to the interactive, multi-user application.

6. A method for bridging a physical environment and a virtual representation in order to facilitate playing a multi-user, interactive game, comprising the steps of:

(a) receiving a request for presence from a first user and a second user;

(b) establishing, in response to said requests, presence for said first user and said second user within the game; (c) delivering virtual entities in order to effect the presence for each user;

(d) storing data about one or more client devices which said first user and said second user can use to gain access to the game;

(e) receiving a signal from said first user, via a first client device, wherein said signal is a request for interaction with an entity in the game;

(f) effecting said request within the game; and

(g) communicating said effect to said second user using a second client device.

7. A method for bridging physical and virtual environments in order to facilitate the execution of an instance of an interactive, multi-user gaming application, comprising the steps of:

(a) receiving requests from a first user using a first client device, and a second user utilizing a second client device, wherein said requests are for presence within the gaming application;

(b) storing information about the characteristics and capabilities of said first and second client devices;

(c) establishing, in response to said requests, a virtual representation for said first user and said second user within the gaming application;

(d) receiving an indication of an event in the physical world from said first user via said first client device;

(e) updating said virtual representation of said first user within the gaming application according to said event; (f) creating, by using said stored information about the characteristics and capabilities of said second client device, a signal which reflects the updating of said virtual representation of said first user; and

(g) communicating the updating of said virtual representation of said first user to said second user by delivering said signal to said second client device; wherein the physical and virtual environments are bridged by maintaining referential integrity within the instance of the interactive, multi-user gaming application.

8. A method for bridging physical and virtual environments in order to facilitate the execution of an instance of an interactive, multi-user gaming application, comprising the steps of:

(a) receiving requests from a first user using a first client device, and a second user utilizing a second client device, wherein said requests are for presence within the gaming application; (b) storing information about the characteristics and capabilities of said first and second client devices; (c) establishing, in response to said requests, a virtual representation for said first user and said second user within the gaming application;

(d) receiving an indication of an event in the physical world from said first user via said first client device; (e) updating the virtual environment within the gaming application to reflect said event;

(f) creating, by using said stored information about the characteristics and capabilities of said second client device, a signal which reflects the updating of the virtual environment; and (g) communicating the updating of the virtual environment to said second user by delivering said signal to said second client device; wherein the physical and virtual environments are bridged by maintaining referential integrity within the instance of the interactive, multi-user gaming application.

9. A computer program product comprising a computer usable medium having control logic stored therein for causing a computer to bridge a physical environment and a virtual environment while executing an interactive, multi-user application, said control logic comprising: first computer readable program code means for causing the computer to receive a request for presence from a first user and a second user; second computer readable program code means for causing the computer to establish, in response to said requests, presence for said first user and said second user within the game; third computer readable program code means for causing the computer to deliver virtual entities in order to effect the presence for each user; fourth computer readable program code means for causing the computer to store data about one or more client devices which said first user and said second user can use to gain access to the game; fifth computer readable program code means for causing the computer to receive a signal from said first user, via a first client device, wherein said signal is a request for interaction with an entity in the game; sixth computer readable program code means for causing the computer to effect said request within the game; and seventh computer readable program code means for causing the computer to communicate said effect to said second user using a second client device.

10. A computer program product comprising a computer usable medium having control logic stored therein for causing a computer to physical and virtual environments in order to facilitate the execution of an instance of an interactive, multi-user gaming application, said control logic comprising: first computer readable program code means for causing the computer to receive requests from a first user using a first client device, and a second user utilizing a second client device, wherein said requests are for presence within the gaming application; second computer readable program code means for causing the computer to store information about the characteristics and capabilities of said first and second client devices; third computer readable program code means for causing the computer to establish, in response to said requests, a virtual representation for said first user and said second user within the gaming application; fourth computer readable program code means for causing the computer to receive an indication of an event in the physical world from said first user via said first client device; fifth computer readable program code means for causing the computer to update said virtual representation of said first user within the gaming application according to said event; sixth computer readable program code means for causing the computer to create, by using said stored information about the characteristics and capabilities of said second client device, a signal which reflects the updating of said virtual representation of said first user; and seventh computer readable program code means for causing the computer to communicate the updating of said virtual representation of said first user to said second user by delivering said signal to said second client device; wherein the physical and virtual environments are bridged by maintaining referential integrity within the instance of the interactive, multi-user gaming application.

11. A computer program product comprising a computer usable medium having control logic stored therein for causing a computer to physical and virtual environments in order to facilitate the execution of an instance of an interactive, multi-user gaming application, said control logic comprising: first computer readable program code means for causing the computer to receive requests from a first user using a first client device, and a second user utilizing a second client device, wherein said requests are for presence within the gaming application; second computer readable program code means for causing the computer to store information about the characteristics and capabilities of said first and second client devices; third computer readable program code means for causing the computer to establish, in response to said requests, a virtual representation for said first user and said second user within the gaming application; fourth computer readable program code means for causing the computer to receive an indication of an event in the physical world from said first user via said first client device; fifth computer readable program code means for causing the computer to update the virtual environment within the gaming application to reflect said event; sixth computer readable program code means for causing the computer to create, by using said stored information about the characteristics and capabilities of said second client device, a signal which reflects the updating of the virtual environment; and seventh computer readable program code means for causing the computer to communicate the updating of the virtual environment to said second user by delivering said signal to said second client device; wherein the physical and virtual environments are bridged by maintaining referential integrity within the instance of the interactive, multi-user gaming application.

12. An orientationally-aware peripheral (OAP) apparatus, comprising: six accelerometers, separated into three groups of two, and oriented along each of three perpendicular axes, wherein each pair of said accelerometers is separated as far as physically possible on one of said three perpendicular axes within the apparatus, and wherein both position and angular orientation can be calculated to track the placement and orientation of the apparatus in six degrees of freedom by correctly integrating the acceleration of all six said accelerometers, wherein said OAP apparatus is useful in a system for bridging a physical environment and a virtual environment in order to facilitate the execution of an interactive, multi-user application.

13. The OAP apparatus of claim 12, further comprising: software code logic that synchronizes the OAP apparatus when the apparatus has not moved for a pre-selected time period.

14. The OAP apparatus of claim 12, further comprising: a communication subsystem wherein output of the apparatus is communicated via said communications subsystem to players participating in the same instance of the interactive, multi-player application.

15. The OAP apparatus of claim 14, wherein said communications subsystem is wireless.

16. The OAP apparatus of claim 15, wherein said wireless communications subsystem is at least one of the following: (i) Bluetooth; and (ii) IEEE 802.11.

17. An orientationally-aware peripheral (OAP) apparatus, comprising: a housing in the shape of a triangular pyramid; and six accelerometers placed along the vertices of said triangular pyramid housing, wherein both position and angular orientation can be calculated to track the placement and orientation of the apparatus in six degrees of freedom by integrating the acceleration of all six of said accelerometers, wherein said OAP apparatus device is useful in a system for bridging a physical environment and a virtual environment in order to facilitate the execution of an interactive, multi-user application.

18. The OAP apparatus of claim 17, further comprising: software code logic that synchronizes the OAP apparatus when the apparatus has not moved for a pre-selected time period.

19. The OAP apparatus of claim 18, further comprising: a communication subsystem, wherein the output of said communications system is communicated to players participating in the same instance of the interactive, multi-player application.^

20. The OAP apparatus of claim 19, wherein said communication subsystem is wireless.

21. The OAP apparatus of claim 20, wherein said wireless communication subsystem is at least one of the following: (i) Bluetooth; and (ii) IEEE 802.11.