US20140280972A1 - Broker module for managing and monitoring resources between internet service providers - Google Patents
Broker module for managing and monitoring resources between internet service providers Download PDFInfo
- Publication number
- US20140280972A1 US20140280972A1 US14/204,354 US201414204354A US2014280972A1 US 20140280972 A1 US20140280972 A1 US 20140280972A1 US 201414204354 A US201414204354 A US 201414204354A US 2014280972 A1 US2014280972 A1 US 2014280972A1
- Authority
- US
- United States
- Prior art keywords
- internet service
- service provider
- resources
- client device
- service providers
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
- H04L67/1004—Server selection for load balancing
- H04L67/1012—Server selection for load balancing based on compliance of requirements or conditions with available server resources
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
- H04L43/0805—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters by checking availability
- H04L43/0817—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters by checking availability by checking functioning
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/08—Configuration management of networks or network elements
- H04L41/0893—Assignment of logical groups to network elements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/50—Network service management, e.g. ensuring proper service fulfilment according to agreements
- H04L41/5003—Managing SLA; Interaction between SLA and QoS
- H04L41/5019—Ensuring fulfilment of SLA
- H04L41/5022—Ensuring fulfilment of SLA by giving priorities, e.g. assigning classes of service
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
- H04L43/0852—Delays
- H04L43/0864—Round trip delays
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
- H04L43/0876—Network utilisation, e.g. volume of load or congestion level
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/10—Active monitoring, e.g. heartbeat, ping or trace-route
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/16—Threshold monitoring
Definitions
- the invention relates to internet service providers, and in particular to a broker module for managing and monitoring resources between internet service provider.
- Remote gaming is a mechanism by which users of a client device interact with a game that is running at a remote location.
- Client devices are allocated resources at a remote location through an internet service provider (ISP) for servicing remote gameplay.
- ISP internet service provider
- An ISP includes several data centers for servicing various client devices utilizing that ISP.
- Each data center may include multiple blades (e.g., individual processing entity of the data center), each blade including various resources (e.g., CPU and GPU) for servicing the client.
- an ISP manages resources for its associated client devices by determining an appropriate data center and blade within its own resource pool for servicing each of those client devices. Because an ISP can only manage its own resource pool and additionally because an ISP can only allocate resources from its own resource pool several non-optimal situations may occur.
- the resource pool may be utilized at maximum capacity such that additional users of client devices may need to wait for resources to free up prior to being serviced.
- the optimal set of resources for a given user of a client device may be currently occupied by another user, such that the requesting user of the client device is assigned to a sub-optimal set of resources. In such circumstances, users are not experiencing an optimal remote gameplay experience as the user is experiencing a wait time prior to gameplay or the user is being serviced by a suboptimal set of resources.
- the resource pool may outnumber user demand.
- the unused resources are being underutilized (e.g., remain idle) resulting in inefficiency.
- Some embodiments of the present invention are directed to a broker module for managing and monitoring resources between internet service providers.
- FIG. 1 illustrates an example remote gaming environment.
- FIG. 2 illustrates a remote gaming environment with a broker module for managing and monitoring resources between different internet service providers according to some embodiments.
- FIG. 3 illustrates an example set of constraints and resources provided by an internet service provider to a broker module in accordance with some embodiments.
- FIG. 4 is a flow chart illustrating a method for managing and monitoring resources between internet service providers by a broker module in accordance with some embodiments.
- FIG. 5 is a flow chart illustrating a method for sharing resources between internet service providers by a broker module in accordance with some embodiments.
- FIG. 6 illustrates a remote gaming environment wherein a client device is allocated resources from another internet service provider in accordance with some embodiments.
- FIG. 7 is a flow chart illustrating a method for performing pre-emption by a broker module in accordance with some embodiments.
- FIGS. 8A and 8B illustrate a remote gaming environment where a broker module performs pre-emption of an internet service provider's resources in accordance with some embodiments.
- FIG. 9 is a block diagram of an illustrative computing system suitable for implementing an embodiment of the present invention.
- Some embodiments of the present invention are directed to a broker module for managing and monitoring resources between internet service providers.
- ISP internet service provider
- users of client devices associated with an internet service provider (ISP) with a resource pool utilized to full capacity or users of client devices associated with an internet service provider where the optimal set of resources for those users are currently occupied may borrow resources from the resource pool of another internet service provider (ISP).
- ISP internet service provider
- the resource pool for a given internet service provider (ISP) may be utilized more efficiently by reducing the amount of unused resources through lending of those resources to users of client devices associated with other ISPs.
- FIG. 1 illustrates an example remote gaming environment 100 .
- the remote gaming environment 100 includes several client devices 101 and multiple internet service providers ISP 1 , ISP 2 .
- Each internet service provider ISP 1 , ISP 2 includes a corresponding resource manager 103 , a plurality of corresponding data center managers 105 , and a plurality of data centers DC.
- Each data center DC may include several blades 105 (e.g., individual processing entities of the data center DC) that may run games to service one or more client devices 101 at a time.
- Each blade 105 includes various resources (e.g., CPU and GPU) for servicing its corresponding client devices 101 .
- Client devices 101 are associated with an internet service provider ISP 1 , ISP 2 that provides the infrastructure for servicing remote gaming for those client devices 101 .
- Each internet service provider ISP 1 , ISP 2 includes a resource manager 103 that manages the data centers DC (e.g., resource pool) for that internet service provider ISP 1 , ISP 2 .
- the resource manager 103 for an internet service provider ISP 1 , ISP 2 may communicate with several data center managers 105 , each of which is managing the blades 105 and resources of an individual data center DC.
- a corresponding resource manager 103 is used to determine which data center to assign the client device 101 .
- the corresponding resource manager 103 may receive information from data center managers 105 to facilitate its assignment of the client device 101 .
- Assignment of a client device 101 to a data center DC may be based on various factors. For example, a client device 101 may be assigned to a particular data center DC based on its proximity to that data center. The client device 101 may also be assigned to a particular data center DC based on the resources available at that data center DC.
- the data center manager 105 for that particular data center DC is used to assign the client device to a particular blade 107 and set of resources within that data center DC.
- Assignment of a client device 101 to a particular blade 107 may be based on various factors. For example, a client device 101 may be assigned to a particular blade 107 based on the resources (e.g., CPU and GPU) associated with that blade 107 . The client device 101 may also be assigned to a particular blade 107 based on the availability of that blade 107 .
- ISP 2 can only manage its own resource pool (e.g., data center, blades, etc.) and additionally because an internet service provider ISP 1 , ISP 2 can only allocate resources from its own resource pool several non-optimal situations may occur.
- resource pool e.g., data center, blades, etc.
- the resource pool may be utilized at maximum capacity such that additional users of client devices 101 may need to wait for resources to free up prior to being serviced.
- the optimal set of resources for a given user of a client device 101 may be currently occupied by another client device 101 , such that the requesting client device 101 is assigned to a sub-optimal set of resources. In such circumstances, users are not experiencing an optimal remote gameplay experience as the user is experiencing a wait time prior to gameplay or the user is being serviced by a suboptimal set of resources.
- the resource pool may outnumber user demand.
- the unused resources are being underutilized (e.g., remain idle) resulting in inefficiency.
- FIG. 2 illustrates a remote gaming environment 200 with a broker module 201 for managing and monitoring resources between different internet service providers ISP 1 , ISP 2 according to some embodiments.
- the remote gaming environment 200 of FIG. 2 is very similar to the remote gaming environment 100 of FIG. 1 , with the addition of a broker module 201 .
- the broker module 201 provides a link between each internet service provider ISP 1 , ISP 2 and its corresponding resource manager 103 . Rather than having each internet service provider ISP 1 , ISP 2 manage its own resources (e.g., data centers, blades, etc.), the broker module 201 is used to manage resources for all of the internet service providers ISP 1 , ISP 2 .
- the remote gaming environment 200 of FIG. 2 may be expanded to support a broker module 201 that manages resources for any number internet service providers ISP 1 , ISP 2 .
- client devices 101 are associated with each internet service provider ISP 1 , ISP 2
- the remote gaming environment 200 of FIG. 2 may be expanded to support any number of client devices 101 for a given internet service provider ISP 1 , ISP 2 .
- Each internet service provider ISP 1 , ISP 2 initially provides the broker 201 with a set of resources. For example, each internet service provider ISP 1 , ISP 2 may allocate a set of data centers for the broker module 201 to manage. Each internet service provider ISP 1 , ISP 2 may then provide the broker module 201 with a set of constraints. For example, each internet service provider ISP 1 , ISP 2 may provide the broker module 201 with a set of service level agreements (e.g., priority constraints, minimal amount of resources, etc.). Additional constraints will be described in greater detail below.
- service level agreements e.g., priority constraints, minimal amount of resources, etc.
- the broker module 201 may then provide end users of client devices 101 associated with an internet service provider ISP 1 , ISP 2 a proper set of resources.
- the broker module 201 communicates with the resource manager 103 of each internet service provider ISP 1 , ISP 2 to assign client devices 101 to particular data centers associated with those internet service providers ISP 1 , ISP 2 .
- the resource manager 103 may then communicate with its corresponding data center managers 105 to assign client devices 101 to particular blades and resources within a data center DC.
- the broker module 201 may communicate directly with each data center manager 105 to assign client devices 101 to particular blades and resources within a data center DC.
- ISP 2 Having a broker module 201 that provides management and monitoring to several internet service providers ISP 1 , ISP 2 allows for resources to be utilized more efficiently and optimally. So long as the constraints for a given internet service provider ISP 1 , ISP 2 are met by the broker module 201 , unused resources for that internet service provider ISP 1 , ISP 2 may be redistributed to facilitate servicing of client devices associated with other internet service providers ISP 1 , ISP 2 .
- the broker module 201 may borrow resources from the resource pool of another internet service provider ISP 1 , ISP 2 to service the user of the client device 101 .
- the broker module 201 may again borrow resources from the resource pool of another internet service provider ISP 1 , ISP 2 to service the user of the client device 101 .
- unused resources of a resource pool of a given internet service provider ISP may be lent to users of client devices 101 associated with other internet service providers ISP 1 , ISP 2 .
- the broker module 201 may optimize how resources are shared between internet service providers ISP 1 , ISP 2 and how client devices 101 are assigned resources by utilizing a learning process.
- the resource manager 103 may provide statistics and metrics to the broker module 201 depicting operating conditions which may be utilized by the broker module 201 to optimize subsequent resource assignment and resource borrowing.
- the resource manager 103 may receive statistics and metrics from its corresponding data center managers 105 and forward those statistics and metrics to the broker module 201 .
- the data center managers 105 may directly forward statistics and metrics to the broker module 201 .
- the statistics and metrics may be provided periodically by the resource manager 103 or data center managers 105 to the broker module 201 . In other embodiments, the statistics and metrics may be provided by the resource manager 103 or data center managers 105 to the broker module 201 upon request.
- the statistics and metrics may include the number of resources an internet service provider ISP 1 , ISP 2 has available.
- the statistics and metrics may indicate the total number of data centers DC available for a given internet service provider ISP, the total number of blades 107 for each data center DC, and the number and types of resources associated with each blade 107 .
- the statistics and metrics may include the number of resources used by an internet service provider ISP 1 , ISP 2 out of those available.
- the statistics and metrics may indicate the total number of blades 107 being used for each data center, and the amount of resources being used by each blade 107 at any given time.
- the statistics and metrics may include the number of resources loaned by an internet service provider ISP 1 , ISP 2 to another internet service provider ISP 1 , ISP 2 .
- the statistics and metrics may indicate the total number of blades 107 being loaned by a data center DC or the total number of resources being loaned by a blade 107 at any given time.
- the statistics and metrics may include the number of resources borrowed by an internet service provider ISP 1 , ISP 2 from another internet service provider ISP 1 , ISP 2 .
- the statistics and metrics may indicate the total number of blades being borrowed by an internet service provider ISP 1 , ISP 2 or the total number of resources being borrowed from a blade by an internet service provider ISP 1 , ISP 2 .
- broker module 201 may optimize subsequent resource assignment and resource borrowing as discussed above. Furthermore, in addition to providing for resources to be utilized more efficiently and optimally, broker modules 201 also serve the purpose of alleviating the responsibility of internet service providers ISP 1 , ISP 2 managing and monitoring its own resources.
- FIG. 3 illustrates an example set of constraints and resources provided by an internet service provider to a broker module in accordance with some embodiments.
- the set of resources provided by an internet service provider may include the available data centers for that internet service provider and the available blades associated with each available data center. Additionally, the set of resources provided by the internet service provider may further include the set of resources available for each blade.
- the set of constraints provided by an internet service provider may include location policies for that internet service provider, service level agreements for that internet service provider and custom content for that internet service provider.
- Location policies may include IP proximity constraints, ping time constraints, data center capacity constraints, and link quality constraints.
- IP proximity constraints indicate the proximity a client device's IP address must be in relation to a data center servicing that client device.
- Ping time constraints indicate the ping time that must be met when a data center/blade is servicing a client device.
- Data center capacity indicates the maximum operating capacity for data centers associated with the internet service provider.
- Link quality indicates the level of lossiness tolerable for a client device connection. It is important to note that different users of an internet service provider may be associated with different location policies based on their subscribed level of service.
- Service level agreements may include priority for an internet service provider, resources requested by the internet service provider, bursting capabilities to be provided to the internet service provider, and lending/sharing constraints for the internet service provider.
- Priority indicates the level of priority an internet service provider is provided in comparison to other internet service providers utilizing the broker module. For example, an internet service provider with a high level of priority may utilize unused resources earlier than an internet service provided with a lower level of priority.
- Resources requested by the internet service provider indicate a minimum amount of resources to be available at all times to the internet service provider. For example, an internet service provider may indicate that at least 5 out of 10 data centers are available to service its end users at all times. Bursting capabilities indicate a level of bandwidth transmission over a short period of time to be provided to end users of the internet service provider.
- Lending/sharing constraints indicate a number of resources to be shared or borrowed by an internet service provider. For example, an internet service provider may indicate a small percentage of resources to be shared with other internet service providers while also indicating an unlimited number of resources to be borrowed from other internet service providers
- custom content constraints may be used to indicate certain custom content available only to end users of an internet service provider such that users of other internet service providers may not be given access to that content.
- the set of resources and set of constraints are used by the broker module to determine how to manage and monitor resources between internet service providers.
- the broker module utilizes the set of resources for an internet service provider to understand what resources are available for that particular internet service provider.
- the broker module then utilizes the set of constraints to determine how resources may be shared amongst internet service providers and also to determine how resources should be provided to client devices.
- the broker module acts to provide a level of service to end users of an internet service provider in line with the constraints provided by the internet service provider, while allowing resources to be shared amongst internet service providers.
- FIG. 4 is a flow chart illustrating a method for managing and monitoring resources between internet service providers by a broker module in accordance with some embodiments.
- the broker module receives a set of resources and constraints from all internet service providers utilizing the broker module.
- the broker module sets up connections with resource managers, data center managers, data centers, blades and their associated resources.
- the broker module operates to manage and monitor resources between internet service providers that are utilizing the broker module.
- the broker module may first receive a user session request from a client device as shown at 401 .
- a client of a user device may request to interact with a certain game through its internet service provider.
- the request may be forwarded to the broker module, which then determines how resources should be allocated for that client device as shown at 403 .
- the determination of how resources should be allocated for the user may be based on the set of resources and constraints provided by the internet service providers.
- the broker module may determine that the client device is best serviced by a blade of a data center provided by the internet service provider.
- the blade of the data center may have the required resources for the client device and meet the constraints provided by the internet service provider.
- the broker module may determine that the client device is best serviced by a blade of a data center provided by another internet service provider.
- the blade of the data center provided by the other internet service provider may have the required resources for the client device and meet the constraints provided by its internet service provider as well as the constraints provided by the internet service provider associated with the client device.
- This situation may arise where the resource pool for the internet service provider associated with the client device is utilized to maximum capacity such that additional users of client devices need to wait for resources to free up prior to being serviced.
- This situation may also arise where the optimal set of resources for the client device may be currently occupied by another client device, such that the requesting client device can only be assigned to a sub-optimal set of resources when using resources provided by its own internet service provider.
- the broker module continues to monitor the user session as shown at 405 .
- Monitoring may involve observing the user session to ensure that no errors or failures occur and also providing the proper support in case of error or failure.
- Support may include moving a user session from a failing blade or data center to an alternative blade or data center that is properly functioning and can optimally service the client device.
- the broker module may also receive feedback about its resources as shown at 407 . In some embodiments, this may occur periodically during operation of the broker module. As discussed above, feedback may involve receiving statistics and metrics from resources managed by the broker module that depict operating conditions which may be utilized by the broker module to optimize subsequent resource assignment and resource borrowing.
- the resource manager for each internet service provider may receive statistics and metrics from its corresponding data center managers and forward those statistics and metrics to the broker module.
- the data center managers for each internet service provider may directly forward statistics and metrics to the broker module.
- the broker module may modify its subsequent resource allocation schemes based on the feedback to better optimize resource allocation for subsequent client device requests as shown at 409 .
- the broker module may learn that a particular data center is rarely utilized and as such may prioritize blades of that data center for sharing.
- the broker module may learn that a particular data center is highly utilized and as such may de-prioritize blades of that data center for sharing.
- the broker module acts to provide a level of service to end users of an internet service provider in line with the constraints provided by the internet service provider, while allowing resources to be shared amongst internet service providers. Furthermore, the broker module employs a learning process to allow for subsequent resource assignment and resource borrowing optimization as discussed above.
- FIG. 5 is a flow chart illustrating a method for sharing resources between internet service providers by a broker module in accordance with some embodiments.
- the broker module may first receive a user session request from a client device as shown at 501 .
- a client of a user device may request to interact with a certain game through its internet service provider.
- the request may be forwarded to the broker module, which then determines how resources should be allocated for that client device as shown at 503 .
- determining how resources should be allocated for that client device a determination is first made by the broker module as to whether the internet service provider associated with the client device can provide the resources needed by the client device as shown at 505 .
- the broker module may communicate with the resource manager or data center managers of the internet service provider associated with the client device to determine whether optimal resources are available at a blade for servicing the client device.
- the broker module determines that the internet service provider associated with the client device is capable of providing the resources needed by the client device in line with the constraints provided by the internet service provider, then the client device is provided resources from its associated internet service provider as shown at 507 .
- the client device is provided resources from another internet service provider as shown at 509 .
- the client device may be provided resources from another internet service provided based on statistics and metrics received by the broker module. It is important to note that the client device is provided resources from another internet service provider so long as constraints are met for the other internet service provider and the internet service provider associated with the client device.
- the situation where the internet service provider associated with the client device is unable to provide the resources needed by the client device may arise where the resource pool for the internet service provider associated with the client device is utilized to maximum capacity such that additional users of client devices need to wait for resources to free up prior to being serviced. This situation may also arise where the optimal set of resources for the client device may be currently occupied by another client device, such that the requesting client device can only be assigned to a sub-optimal set of resources when using resources provided by its own internet service provider.
- the client device may experience optimal gameplay while resources are utilized more efficiently and optimally.
- FIG. 6 illustrates a remote gaming environment 600 wherein a client device 101 is allocated resources from another internet service provider in accordance with some embodiments.
- the remote gaming environment 600 of FIG. 6 is substantially similar to the remote gaming environment 200 of FIG. 2 and as such will not be described again in detail.
- a client device (depicted by gray shading) 101 belonging to a first internet service provider ISP 1 is allocated resources of a second internet service provider ISP 2 by the broker module 201 .
- this situation may arise where the resource pool for the first internet service provider ISP 1 is utilized to maximum capacity such that requesting client device (depicted by grey shading) 101 needs to wait for resources of the first internet service provider ISP 1 to free up prior to being serviced by the first internet service provider ISP 1 .
- This situation may also arise where the optimal set of resources for the requesting client device (depicted by grey shading) 101 may be currently occupied by another client device, such that the requesting client device (depicted by grey shading) 101 can only be assigned to a sub-optimal set of resources when using resources provided by the first internet service provider ISP 1 .
- the client device (depicted by grey shading) 101 may experience optimal gameplay while resources are utilized more efficiently and optimally.
- the broker module may also perform pre-emption in certain circumstances. Pre-emption refers to the action of prematurely disassociating a client device session from allocated resources of an internet service provider such that those resources may be provided to another client device.
- FIG. 7 is a flow chart illustrating a method for performing pre-emption by a broker module in accordance with some embodiments.
- the broker module may first receive a user session request from a client device as shown at 701 .
- a client of a user device may request to interact with a certain game through its internet service provider.
- the request may be forwarded to the broker module, which then determines how resources should be allocated for that client device as shown at 703 .
- determining how resources should be allocated for that client device a determination is first made by the broker module as to whether the internet service provider associated with the client device can currently provide the resources needed by the client device as shown at 705 .
- the broker module may communicate with the resource manager or data center managers of the internet service provider associated with the client device to determine whether optimal resources are available at a blade for servicing the client device.
- the broker module determines that the internet service provider associated with the client device is capable of providing the resources needed by the client device in line with the constraints provided by the internet service provider, then the client device is provided resources from its associated internet service provider as shown at 707 .
- the broker module determines whether pre-emption of the internet service provider's resources is available as shown at 709 . For example, the broker module may determine that a client device belonging to a non-paying guest is currently occupying resources of the internet service provider that are optimal for the requesting client device. If the requesting client device belongs to a paying user, then the broker module may determine that the resources occupied by the guest client device may be pre-empted.
- the broker module determines that pre-emption of the internet service provider's resources is not available, then the broker module borrows resources from another internet service provider as shown at 717 .
- the broker module determines whether it is more efficient to perform pre-emption or whether it is more efficient to borrow resources from another internet service provider as shown at 711 .
- the broker module determines that it is more efficient to borrow resources from another internet service provider, then the broker module borrows resources from another internet service provider to service the requesting client device as shown at 717 .
- the broker module may perform pre-emption of the internet service provider's resources as shown at 713 . This may involve disassociating the client device currently occupying the internet service provider's resources. The pre-empted resources of the internet service provider may then be provided to the requesting client device as shown at 715 . In this way, the broker module is able to provide resources of an internet service provider to a client device associated with users of high priority.
- FIGS. 8A and 8B illustrate a remote gaming environment 800 , 800 ′ where a broker module 201 performs pre-emption of an internet service provider's resources in accordance with some embodiments.
- the determination by the broker module 201 as to whether or not to perform pre-emption of an internet service provider's resources may be made in accordance with the method depicted in FIG. 7 .
- the remote gaming environments 800 , 800 ′ of FIGS. 8A and 8B are substantially similar to the remote gaming environment 200 of FIG. 2 and as such will not be described again in detail.
- a first client device belonging to a non-paying guest (depicted by gray shading) 101 is allocated resources of a first internet service provider ISP 1 by the broker module 201 .
- resources previously occupied by the first client belonging to the non-paying guest (depicted by grey shading) 101 are pre-empted and provided to a second client device belonging to a paying user (depicted by patterned shading) 101 ′.
- the first client belonging to the non-paying guest (depicted by grey shading) 101 may then be provided sub-optimal resources at another blade 107 of another data center DC associated with the first internet service provider ISP 1 as illustrated in FIG. 8B .
- the first client belonging to the non-paying guest (depicted by grey shading) 101 may be provided resources of another internet service provider or not provided resources at all.
- broker module may provide management and monitoring to several internet service providers in any remote application context.
- FIG. 9 is a block diagram of an illustrative computing system 1400 suitable for implementing an embodiment of the present invention.
- Computer system 1400 includes a bus 1406 or other communication mechanism for communicating information, which interconnects subsystems and devices, such as processor 1407 , system memory 1408 (e.g., RAM), static storage device 1409 (e.g., ROM), disk drive 1410 (e.g., magnetic or optical), communication interface 1414 (e.g., modem or Ethernet card), display 1411 (e.g., CRT or LCD), input device 1412 (e.g., keyboard), and cursor control.
- processor 1407 e.g., system memory 1408 (e.g., RAM), static storage device 1409 (e.g., ROM), disk drive 1410 (e.g., magnetic or optical), communication interface 1414 (e.g., modem or Ethernet card), display 1411 (e.g., CRT or LCD), input device 1412 (e.g., keyboard), and cursor control.
- computer system 1400 performs specific operations by processor 1407 executing one or more sequences of one or more instructions contained in system memory 1408 .
- Such instructions may be read into system memory 1408 from another computer readable/usable medium, such as static storage device 1409 or disk drive 1410 .
- static storage device 1409 or disk drive 1410 may be used in place of or in combination with software instructions to implement the invention.
- hard-wired circuitry may be used in place of or in combination with software instructions to implement the invention.
- embodiments of the invention are not limited to any specific combination of hardware circuitry and/or software.
- the term “logic” shall mean any combination of software or hardware that is used to implement all or part of the invention.
- Non-volatile media includes, for example, optical or magnetic disks, such as disk drive 1410 .
- Volatile media includes dynamic memory, such as system memory 1408 .
- Computer readable media includes, for example, floppy disk, flexible disk, hard disk, magnetic tape, any other magnetic medium, CD-ROM, any other optical medium, punch cards, paper tape, any other physical medium with patterns of holes, RAM, PROM, EPROM, FLASH-EPROM, any other memory chip or cartridge, or any other medium from which a computer can read.
- execution of the sequences of instructions to practice the invention is performed by a single computer system 1400 .
- two or more computer systems 1400 coupled by communication link 1415 may perform the sequence of instructions required to practice the invention in coordination with one another.
- Computer system 1400 may transmit and receive messages, data, and instructions, including program, i.e., application code, through communication link 1415 and communication interface 1414 .
- Received program code may be executed by processor 1407 as it is received, and/or stored in disk drive 1410 , or other non-volatile storage for later execution.
- Computer system 1400 may communicate through a data interface 1433 to a database 1432 on an external storage device 1431 .
Abstract
Description
- The present application claims the benefit of U.S. Provisional Application 61/778,145 filed on Mar. 12, 2013, which is hereby incorporated by reference in its entirety.
- The invention relates to internet service providers, and in particular to a broker module for managing and monitoring resources between internet service provider.
- Remote gaming is a mechanism by which users of a client device interact with a game that is running at a remote location. Client devices are allocated resources at a remote location through an internet service provider (ISP) for servicing remote gameplay. An ISP includes several data centers for servicing various client devices utilizing that ISP. Each data center may include multiple blades (e.g., individual processing entity of the data center), each blade including various resources (e.g., CPU and GPU) for servicing the client.
- Conventionally, an ISP manages resources for its associated client devices by determining an appropriate data center and blade within its own resource pool for servicing each of those client devices. Because an ISP can only manage its own resource pool and additionally because an ISP can only allocate resources from its own resource pool several non-optimal situations may occur.
- In some circumstances, the resource pool may be utilized at maximum capacity such that additional users of client devices may need to wait for resources to free up prior to being serviced. In some other circumstances, the optimal set of resources for a given user of a client device may be currently occupied by another user, such that the requesting user of the client device is assigned to a sub-optimal set of resources. In such circumstances, users are not experiencing an optimal remote gameplay experience as the user is experiencing a wait time prior to gameplay or the user is being serviced by a suboptimal set of resources.
- In certain other circumstances, the resource pool may outnumber user demand. In such circumstances, the unused resources are being underutilized (e.g., remain idle) resulting in inefficiency.
- Some embodiments of the present invention are directed to a broker module for managing and monitoring resources between internet service providers.
- Further details of aspects, objects and advantages of the invention are described below in the detailed description, drawings and claims. Both the foregoing general description and the following detailed description are exemplary and explanatory and are not intended to be limiting as to the scope of the invention.
- The drawings illustrate the design and utility of embodiments, in which similar elements are referred to by common reference numerals. These drawings are not necessarily drawn to scale. In order to better appreciate how the above-recited and other advantages and objects are obtained, a more particular description of the embodiments will be rendered, which are illustrated in the accompanying drawings. These drawings depict only typical embodiments and are not therefore to be considered limiting of its scope.
-
FIG. 1 illustrates an example remote gaming environment. -
FIG. 2 illustrates a remote gaming environment with a broker module for managing and monitoring resources between different internet service providers according to some embodiments. -
FIG. 3 illustrates an example set of constraints and resources provided by an internet service provider to a broker module in accordance with some embodiments. -
FIG. 4 is a flow chart illustrating a method for managing and monitoring resources between internet service providers by a broker module in accordance with some embodiments. -
FIG. 5 is a flow chart illustrating a method for sharing resources between internet service providers by a broker module in accordance with some embodiments. -
FIG. 6 illustrates a remote gaming environment wherein a client device is allocated resources from another internet service provider in accordance with some embodiments. -
FIG. 7 is a flow chart illustrating a method for performing pre-emption by a broker module in accordance with some embodiments. -
FIGS. 8A and 8B illustrate a remote gaming environment where a broker module performs pre-emption of an internet service provider's resources in accordance with some embodiments. -
FIG. 9 is a block diagram of an illustrative computing system suitable for implementing an embodiment of the present invention. - Various embodiments are described hereinafter with reference to the figures. It should be noted that the figures are not necessarily drawn to scale. It should also be noted that the figures are only intended to facilitate the description of the embodiments, and are not intended as an exhaustive description of the invention or as a limitation on the scope of the invention. In addition, an illustrated embodiment need not have all the aspects or advantages shown. An aspect or an advantage described in conjunction with a particular embodiment is not necessarily limited to that embodiment and can be practiced in any other embodiments even if not so illustrated. Also, reference throughout this specification to “some embodiments” or “other embodiments” means that a particular feature, structure, material, or characteristic described in connection with the embodiments is included in at least one embodiment. Thus, the appearances of the phrase “in some embodiments” or “in other embodiments” in various places throughout this specification are not necessarily referring to the same embodiment or embodiments.
- Some embodiments of the present invention are directed to a broker module for managing and monitoring resources between internet service providers. In this way, users of client devices associated with an internet service provider (ISP) with a resource pool utilized to full capacity or users of client devices associated with an internet service provider where the optimal set of resources for those users are currently occupied may borrow resources from the resource pool of another internet service provider (ISP). Additionally, the resource pool for a given internet service provider (ISP) may be utilized more efficiently by reducing the amount of unused resources through lending of those resources to users of client devices associated with other ISPs.
- Remote gaming is a mechanism by which users of a client device interact with a game that is running at a remote location.
FIG. 1 illustrates an exampleremote gaming environment 100. Theremote gaming environment 100 includesseveral client devices 101 and multiple internet service providers ISP1, ISP2. Each internet service provider ISP1, ISP2 includes acorresponding resource manager 103, a plurality of correspondingdata center managers 105, and a plurality of data centers DC. Each data center DC may include several blades 105 (e.g., individual processing entities of the data center DC) that may run games to service one ormore client devices 101 at a time. Eachblade 105 includes various resources (e.g., CPU and GPU) for servicing itscorresponding client devices 101. -
Client devices 101 are associated with an internet service provider ISP1, ISP2 that provides the infrastructure for servicing remote gaming for thoseclient devices 101. Each internet service provider ISP1, ISP2 includes aresource manager 103 that manages the data centers DC (e.g., resource pool) for that internet service provider ISP1, ISP2. Theresource manager 103 for an internet service provider ISP1, ISP2 may communicate with severaldata center managers 105, each of which is managing theblades 105 and resources of an individual data center DC. - When a user of a
client device 101 requests to be serviced by its internet service provider ISP1, ISP2, acorresponding resource manager 103 is used to determine which data center to assign theclient device 101. Thecorresponding resource manager 103 may receive information fromdata center managers 105 to facilitate its assignment of theclient device 101. Assignment of aclient device 101 to a data center DC may be based on various factors. For example, aclient device 101 may be assigned to a particular data center DC based on its proximity to that data center. Theclient device 101 may also be assigned to a particular data center DC based on the resources available at that data center DC. - After the
resource manager 103 has assigned theclient device 101 to a particular data center DC, thedata center manager 105 for that particular data center DC is used to assign the client device to aparticular blade 107 and set of resources within that data center DC. Assignment of aclient device 101 to aparticular blade 107 may be based on various factors. For example, aclient device 101 may be assigned to aparticular blade 107 based on the resources (e.g., CPU and GPU) associated with thatblade 107. Theclient device 101 may also be assigned to aparticular blade 107 based on the availability of thatblade 107. - Because an internet service provider ISP1, ISP2 can only manage its own resource pool (e.g., data center, blades, etc.) and additionally because an internet service provider ISP1, ISP2 can only allocate resources from its own resource pool several non-optimal situations may occur.
- In some circumstances, the resource pool may be utilized at maximum capacity such that additional users of
client devices 101 may need to wait for resources to free up prior to being serviced. In some other circumstances, the optimal set of resources for a given user of aclient device 101 may be currently occupied by anotherclient device 101, such that the requestingclient device 101 is assigned to a sub-optimal set of resources. In such circumstances, users are not experiencing an optimal remote gameplay experience as the user is experiencing a wait time prior to gameplay or the user is being serviced by a suboptimal set of resources. - In certain other circumstances, the resource pool may outnumber user demand. In such circumstances, the unused resources are being underutilized (e.g., remain idle) resulting in inefficiency.
- One way to overcome these limitations is to implement a broker module for managing and monitoring resources between different internet service providers.
FIG. 2 illustrates aremote gaming environment 200 with abroker module 201 for managing and monitoring resources between different internet service providers ISP1, ISP2 according to some embodiments. - The
remote gaming environment 200 ofFIG. 2 is very similar to theremote gaming environment 100 ofFIG. 1 , with the addition of abroker module 201. Thebroker module 201 provides a link between each internet service provider ISP1, ISP2 and itscorresponding resource manager 103. Rather than having each internet service provider ISP1, ISP2 manage its own resources (e.g., data centers, blades, etc.), thebroker module 201 is used to manage resources for all of the internet service providers ISP1, ISP2. - While only two internet service providers ISP1, ISP2 are depicted in
FIG. 2 , one ordinarily skilled in the art will recognize that theremote gaming environment 200 ofFIG. 2 may be expanded to support abroker module 201 that manages resources for any number internet service providers ISP1, ISP2. Furthermore, while only twoclient devices 101 are associated with each internet service provider ISP1, ISP2, one ordinarily skilled in the art will recognize that theremote gaming environment 200 ofFIG. 2 may be expanded to support any number ofclient devices 101 for a given internet service provider ISP1, ISP2. - Each internet service provider ISP1, ISP2 initially provides the
broker 201 with a set of resources. For example, each internet service provider ISP1, ISP2 may allocate a set of data centers for thebroker module 201 to manage. Each internet service provider ISP1, ISP2 may then provide thebroker module 201 with a set of constraints. For example, each internet service provider ISP1, ISP2 may provide thebroker module 201 with a set of service level agreements (e.g., priority constraints, minimal amount of resources, etc.). Additional constraints will be described in greater detail below. - Based on the resources and constraints provided by each internet service provider ISP1, ISP2, the
broker module 201 may then provide end users ofclient devices 101 associated with an internet service provider ISP1, ISP2 a proper set of resources. Thebroker module 201 communicates with theresource manager 103 of each internet service provider ISP1, ISP2 to assignclient devices 101 to particular data centers associated with those internet service providers ISP1, ISP2. Theresource manager 103 may then communicate with its correspondingdata center managers 105 to assignclient devices 101 to particular blades and resources within a data center DC. Alternatively, thebroker module 201 may communicate directly with eachdata center manager 105 to assignclient devices 101 to particular blades and resources within a data center DC. - Having a
broker module 201 that provides management and monitoring to several internet service providers ISP1, ISP2 allows for resources to be utilized more efficiently and optimally. So long as the constraints for a given internet service provider ISP1, ISP2 are met by thebroker module 201, unused resources for that internet service provider ISP1, ISP2 may be redistributed to facilitate servicing of client devices associated with other internet service providers ISP1, ISP2. - For a user of a
client device 101 associated with an internet service provider ISP1, ISP2 with a resource pool utilized to full capacity, thebroker module 201 may borrow resources from the resource pool of another internet service provider ISP1, ISP2 to service the user of theclient device 101. Likewise, for a user of aclient device 101 associated with an internet service provider ISP1, ISP2 where the optimal set of resources for that user is currently occupied, thebroker module 201 may again borrow resources from the resource pool of another internet service provider ISP1, ISP2 to service the user of theclient device 101. Additionally, unused resources of a resource pool of a given internet service provider ISP may be lent to users ofclient devices 101 associated with other internet service providers ISP1, ISP2. - The
broker module 201 may optimize how resources are shared between internet service providers ISP1, ISP2 and howclient devices 101 are assigned resources by utilizing a learning process. During operation, theresource manager 103 may provide statistics and metrics to thebroker module 201 depicting operating conditions which may be utilized by thebroker module 201 to optimize subsequent resource assignment and resource borrowing. Theresource manager 103 may receive statistics and metrics from its correspondingdata center managers 105 and forward those statistics and metrics to thebroker module 201. Alternatively, thedata center managers 105 may directly forward statistics and metrics to thebroker module 201. - In some embodiments, the statistics and metrics may be provided periodically by the
resource manager 103 ordata center managers 105 to thebroker module 201. In other embodiments, the statistics and metrics may be provided by theresource manager 103 ordata center managers 105 to thebroker module 201 upon request. - In some embodiments, the statistics and metrics may include the number of resources an internet service provider ISP1, ISP2 has available. For example, the statistics and metrics may indicate the total number of data centers DC available for a given internet service provider ISP, the total number of
blades 107 for each data center DC, and the number and types of resources associated with eachblade 107. - In some embodiments, the statistics and metrics may include the number of resources used by an internet service provider ISP1, ISP2 out of those available. For example, the statistics and metrics may indicate the total number of
blades 107 being used for each data center, and the amount of resources being used by eachblade 107 at any given time. - In some embodiments, the statistics and metrics may include the number of resources loaned by an internet service provider ISP1, ISP2 to another internet service provider ISP1, ISP2. For example, the statistics and metrics may indicate the total number of
blades 107 being loaned by a data center DC or the total number of resources being loaned by ablade 107 at any given time. - In some embodiments, the statistics and metrics may include the number of resources borrowed by an internet service provider ISP1, ISP2 from another internet service provider ISP1, ISP2. For example, the statistics and metrics may indicate the total number of blades being borrowed by an internet service provider ISP1, ISP2 or the total number of resources being borrowed from a blade by an internet service provider ISP1, ISP2.
- By utilizing such statistics and metrics, the
broker module 201 may optimize subsequent resource assignment and resource borrowing as discussed above. Furthermore, in addition to providing for resources to be utilized more efficiently and optimally,broker modules 201 also serve the purpose of alleviating the responsibility of internet service providers ISP1, ISP2 managing and monitoring its own resources. -
FIG. 3 illustrates an example set of constraints and resources provided by an internet service provider to a broker module in accordance with some embodiments. - The set of resources provided by an internet service provider may include the available data centers for that internet service provider and the available blades associated with each available data center. Additionally, the set of resources provided by the internet service provider may further include the set of resources available for each blade.
- The set of constraints provided by an internet service provider may include location policies for that internet service provider, service level agreements for that internet service provider and custom content for that internet service provider. Location policies may include IP proximity constraints, ping time constraints, data center capacity constraints, and link quality constraints. IP proximity constraints indicate the proximity a client device's IP address must be in relation to a data center servicing that client device. Ping time constraints indicate the ping time that must be met when a data center/blade is servicing a client device. Data center capacity indicates the maximum operating capacity for data centers associated with the internet service provider. Link quality indicates the level of lossiness tolerable for a client device connection. It is important to note that different users of an internet service provider may be associated with different location policies based on their subscribed level of service.
- Service level agreements may include priority for an internet service provider, resources requested by the internet service provider, bursting capabilities to be provided to the internet service provider, and lending/sharing constraints for the internet service provider. Priority indicates the level of priority an internet service provider is provided in comparison to other internet service providers utilizing the broker module. For example, an internet service provider with a high level of priority may utilize unused resources earlier than an internet service provided with a lower level of priority. Resources requested by the internet service provider indicate a minimum amount of resources to be available at all times to the internet service provider. For example, an internet service provider may indicate that at least 5 out of 10 data centers are available to service its end users at all times. Bursting capabilities indicate a level of bandwidth transmission over a short period of time to be provided to end users of the internet service provider. Lending/sharing constraints indicate a number of resources to be shared or borrowed by an internet service provider. For example, an internet service provider may indicate a small percentage of resources to be shared with other internet service providers while also indicating an unlimited number of resources to be borrowed from other internet service providers.
- Finally, custom content constraints may be used to indicate certain custom content available only to end users of an internet service provider such that users of other internet service providers may not be given access to that content.
- The set of resources and set of constraints are used by the broker module to determine how to manage and monitor resources between internet service providers. The broker module utilizes the set of resources for an internet service provider to understand what resources are available for that particular internet service provider. The broker module then utilizes the set of constraints to determine how resources may be shared amongst internet service providers and also to determine how resources should be provided to client devices. Ultimately, the broker module acts to provide a level of service to end users of an internet service provider in line with the constraints provided by the internet service provider, while allowing resources to be shared amongst internet service providers.
-
FIG. 4 is a flow chart illustrating a method for managing and monitoring resources between internet service providers by a broker module in accordance with some embodiments. - Initially, the broker module receives a set of resources and constraints from all internet service providers utilizing the broker module. The broker module sets up connections with resource managers, data center managers, data centers, blades and their associated resources. Once the initial setup has been performed, the broker module operates to manage and monitor resources between internet service providers that are utilizing the broker module.
- The broker module may first receive a user session request from a client device as shown at 401. For example, a client of a user device may request to interact with a certain game through its internet service provider.
- The request may be forwarded to the broker module, which then determines how resources should be allocated for that client device as shown at 403. The determination of how resources should be allocated for the user may be based on the set of resources and constraints provided by the internet service providers.
- In some embodiments, the broker module may determine that the client device is best serviced by a blade of a data center provided by the internet service provider. For example, the blade of the data center may have the required resources for the client device and meet the constraints provided by the internet service provider.
- In other embodiments, the broker module may determine that the client device is best serviced by a blade of a data center provided by another internet service provider. For example, the blade of the data center provided by the other internet service provider may have the required resources for the client device and meet the constraints provided by its internet service provider as well as the constraints provided by the internet service provider associated with the client device. This situation may arise where the resource pool for the internet service provider associated with the client device is utilized to maximum capacity such that additional users of client devices need to wait for resources to free up prior to being serviced. This situation may also arise where the optimal set of resources for the client device may be currently occupied by another client device, such that the requesting client device can only be assigned to a sub-optimal set of resources when using resources provided by its own internet service provider. By servicing the client device using resources provided by another internet service provider while still meeting constraints for both the other internet service provider and the internet service provider associated with the client device, the client device may experience optimal gameplay while resources are utilized more efficiently and optimally.
- Once resources have been allocated to the client device, the broker module continues to monitor the user session as shown at 405. Monitoring may involve observing the user session to ensure that no errors or failures occur and also providing the proper support in case of error or failure. Support may include moving a user session from a failing blade or data center to an alternative blade or data center that is properly functioning and can optimally service the client device.
- The broker module may also receive feedback about its resources as shown at 407. In some embodiments, this may occur periodically during operation of the broker module. As discussed above, feedback may involve receiving statistics and metrics from resources managed by the broker module that depict operating conditions which may be utilized by the broker module to optimize subsequent resource assignment and resource borrowing.
- In some embodiments, the resource manager for each internet service provider may receive statistics and metrics from its corresponding data center managers and forward those statistics and metrics to the broker module. Alternatively, in other embodiments, the data center managers for each internet service provider may directly forward statistics and metrics to the broker module.
- After the broker module receives feedback, it may modify its subsequent resource allocation schemes based on the feedback to better optimize resource allocation for subsequent client device requests as shown at 409. For example, the broker module may learn that a particular data center is rarely utilized and as such may prioritize blades of that data center for sharing. Likewise, the broker module may learn that a particular data center is highly utilized and as such may de-prioritize blades of that data center for sharing.
- Having a broker module that provides management and monitoring to several internet service providers allows for resources to be utilized more efficiently and optimally. The broker module acts to provide a level of service to end users of an internet service provider in line with the constraints provided by the internet service provider, while allowing resources to be shared amongst internet service providers. Furthermore, the broker module employs a learning process to allow for subsequent resource assignment and resource borrowing optimization as discussed above.
-
FIG. 5 is a flow chart illustrating a method for sharing resources between internet service providers by a broker module in accordance with some embodiments. - The broker module may first receive a user session request from a client device as shown at 501. For example, a client of a user device may request to interact with a certain game through its internet service provider.
- The request may be forwarded to the broker module, which then determines how resources should be allocated for that client device as shown at 503. In determining how resources should be allocated for that client device, a determination is first made by the broker module as to whether the internet service provider associated with the client device can provide the resources needed by the client device as shown at 505. For example, the broker module may communicate with the resource manager or data center managers of the internet service provider associated with the client device to determine whether optimal resources are available at a blade for servicing the client device.
- If the broker module determines that the internet service provider associated with the client device is capable of providing the resources needed by the client device in line with the constraints provided by the internet service provider, then the client device is provided resources from its associated internet service provider as shown at 507.
- If the broker module determines that the internet service provider associated with the client device is unable to provide the resources needed by the client device in line with the constraints provided by the internet service provider, then the client device is provided resources from another internet service provider as shown at 509. The client device may be provided resources from another internet service provided based on statistics and metrics received by the broker module. It is important to note that the client device is provided resources from another internet service provider so long as constraints are met for the other internet service provider and the internet service provider associated with the client device.
- The situation where the internet service provider associated with the client device is unable to provide the resources needed by the client device may arise where the resource pool for the internet service provider associated with the client device is utilized to maximum capacity such that additional users of client devices need to wait for resources to free up prior to being serviced. This situation may also arise where the optimal set of resources for the client device may be currently occupied by another client device, such that the requesting client device can only be assigned to a sub-optimal set of resources when using resources provided by its own internet service provider.
- By servicing the client device using resources provided by another internet service provider while still meeting constraints for both the other internet service provider and the internet service provider associated with the client device, the client device may experience optimal gameplay while resources are utilized more efficiently and optimally.
-
FIG. 6 illustrates aremote gaming environment 600 wherein aclient device 101 is allocated resources from another internet service provider in accordance with some embodiments. Theremote gaming environment 600 ofFIG. 6 is substantially similar to theremote gaming environment 200 ofFIG. 2 and as such will not be described again in detail. - In the
remote gaming environment 600 ofFIG. 6 , a client device (depicted by gray shading) 101 belonging to a first internet service provider ISP1 is allocated resources of a second internet service provider ISP2 by thebroker module 201. - As discussed above, this situation may arise where the resource pool for the first internet service provider ISP1 is utilized to maximum capacity such that requesting client device (depicted by grey shading) 101 needs to wait for resources of the first internet service provider ISP1 to free up prior to being serviced by the first internet service provider ISP1. This situation may also arise where the optimal set of resources for the requesting client device (depicted by grey shading) 101 may be currently occupied by another client device, such that the requesting client device (depicted by grey shading) 101 can only be assigned to a sub-optimal set of resources when using resources provided by the first internet service provider ISP1. By instead servicing the client device (depicted by grey shading) 101 using resources provided by a second internet service provider ISP2 while still meeting constraints for both the second internet service provider ISP2 and the first internet service provider ISP1, the client device (depicted by grey shading) 101 may experience optimal gameplay while resources are utilized more efficiently and optimally.
- In addition to servicing a client device using resources provided by another internet service provider, the broker module may also perform pre-emption in certain circumstances. Pre-emption refers to the action of prematurely disassociating a client device session from allocated resources of an internet service provider such that those resources may be provided to another client device.
-
FIG. 7 is a flow chart illustrating a method for performing pre-emption by a broker module in accordance with some embodiments. - The broker module may first receive a user session request from a client device as shown at 701. For example, a client of a user device may request to interact with a certain game through its internet service provider.
- The request may be forwarded to the broker module, which then determines how resources should be allocated for that client device as shown at 703. In determining how resources should be allocated for that client device, a determination is first made by the broker module as to whether the internet service provider associated with the client device can currently provide the resources needed by the client device as shown at 705. For example, the broker module may communicate with the resource manager or data center managers of the internet service provider associated with the client device to determine whether optimal resources are available at a blade for servicing the client device.
- If the broker module determines that the internet service provider associated with the client device is capable of providing the resources needed by the client device in line with the constraints provided by the internet service provider, then the client device is provided resources from its associated internet service provider as shown at 707.
- If the broker module determines that the internet service provider associated with the client device is incapable of currently providing the resources needed by the client device in line with constraints provided by the internet service provider, then the broker module determines whether pre-emption of the internet service provider's resources is available as shown at 709. For example, the broker module may determine that a client device belonging to a non-paying guest is currently occupying resources of the internet service provider that are optimal for the requesting client device. If the requesting client device belongs to a paying user, then the broker module may determine that the resources occupied by the guest client device may be pre-empted.
- If the broker module determines that pre-emption of the internet service provider's resources is not available, then the broker module borrows resources from another internet service provider as shown at 717.
- If the broker module determines that pre-emption of the internet service provider's resources is available, the broker module then determines whether it is more efficient to perform pre-emption or whether it is more efficient to borrow resources from another internet service provider as shown at 711.
- If the broker module determines that it is more efficient to borrow resources from another internet service provider, then the broker module borrows resources from another internet service provider to service the requesting client device as shown at 717.
- If the broker module determines that it is more efficient to perform pre-emption, then the broker module may perform pre-emption of the internet service provider's resources as shown at 713. This may involve disassociating the client device currently occupying the internet service provider's resources. The pre-empted resources of the internet service provider may then be provided to the requesting client device as shown at 715. In this way, the broker module is able to provide resources of an internet service provider to a client device associated with users of high priority.
-
FIGS. 8A and 8B illustrate aremote gaming environment broker module 201 performs pre-emption of an internet service provider's resources in accordance with some embodiments. The determination by thebroker module 201 as to whether or not to perform pre-emption of an internet service provider's resources may be made in accordance with the method depicted inFIG. 7 . Theremote gaming environments FIGS. 8A and 8B are substantially similar to theremote gaming environment 200 ofFIG. 2 and as such will not be described again in detail. - In the
remote gaming environment 800 ofFIG. 8A , a first client device belonging to a non-paying guest (depicted by gray shading) 101 is allocated resources of a first internet service provider ISP1 by thebroker module 201. - In the
remote gaming environment 800′ ofFIG. 8B , resources previously occupied by the first client belonging to the non-paying guest (depicted by grey shading) 101 are pre-empted and provided to a second client device belonging to a paying user (depicted by patterned shading) 101′. The first client belonging to the non-paying guest (depicted by grey shading) 101 may then be provided sub-optimal resources at anotherblade 107 of another data center DC associated with the first internet service provider ISP1 as illustrated inFIG. 8B . Alternatively, after pre-emption, the first client belonging to the non-paying guest (depicted by grey shading) 101 may be provided resources of another internet service provider or not provided resources at all. - While the above description has been provided with reference to the context of gaming, it is important to note that the broker module may provide management and monitoring to several internet service providers in any remote application context.
-
FIG. 9 is a block diagram of anillustrative computing system 1400 suitable for implementing an embodiment of the present invention.Computer system 1400 includes a bus 1406 or other communication mechanism for communicating information, which interconnects subsystems and devices, such asprocessor 1407, system memory 1408 (e.g., RAM), static storage device 1409 (e.g., ROM), disk drive 1410 (e.g., magnetic or optical), communication interface 1414 (e.g., modem or Ethernet card), display 1411 (e.g., CRT or LCD), input device 1412 (e.g., keyboard), and cursor control. - According to one embodiment of the invention,
computer system 1400 performs specific operations byprocessor 1407 executing one or more sequences of one or more instructions contained insystem memory 1408. Such instructions may be read intosystem memory 1408 from another computer readable/usable medium, such asstatic storage device 1409 or disk drive 1410. In alternative embodiments, hard-wired circuitry may be used in place of or in combination with software instructions to implement the invention. Thus, embodiments of the invention are not limited to any specific combination of hardware circuitry and/or software. In one embodiment, the term “logic” shall mean any combination of software or hardware that is used to implement all or part of the invention. - The term “computer readable medium” or “computer usable medium” as used herein refers to any medium that participates in providing instructions to
processor 1407 for execution. Such a medium may take many forms, including but not limited to, non-volatile media and volatile media. Non-volatile media includes, for example, optical or magnetic disks, such as disk drive 1410. Volatile media includes dynamic memory, such assystem memory 1408. - Common forms of computer readable media includes, for example, floppy disk, flexible disk, hard disk, magnetic tape, any other magnetic medium, CD-ROM, any other optical medium, punch cards, paper tape, any other physical medium with patterns of holes, RAM, PROM, EPROM, FLASH-EPROM, any other memory chip or cartridge, or any other medium from which a computer can read.
- In an embodiment of the invention, execution of the sequences of instructions to practice the invention is performed by a
single computer system 1400. According to other embodiments of the invention, two ormore computer systems 1400 coupled by communication link 1415 (e.g., LAN, PTSN, or wireless network) may perform the sequence of instructions required to practice the invention in coordination with one another. -
Computer system 1400 may transmit and receive messages, data, and instructions, including program, i.e., application code, throughcommunication link 1415 andcommunication interface 1414. Received program code may be executed byprocessor 1407 as it is received, and/or stored in disk drive 1410, or other non-volatile storage for later execution.Computer system 1400 may communicate through adata interface 1433 to adatabase 1432 on anexternal storage device 1431. - In the foregoing specification, the invention has been described with reference to specific embodiments thereof. It will, however, be evident that various modifications and changes may be made thereto without departing from the broader spirit and scope of the invention. The specification and drawings are, accordingly, to be regarded in an illustrative rather than restrictive sense.
Claims (24)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US14/204,354 US9635102B2 (en) | 2013-03-12 | 2014-03-11 | Broker module for managing and monitoring resources between internet service providers |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201361778145P | 2013-03-12 | 2013-03-12 | |
US14/204,354 US9635102B2 (en) | 2013-03-12 | 2014-03-11 | Broker module for managing and monitoring resources between internet service providers |
Publications (2)
Publication Number | Publication Date |
---|---|
US20140280972A1 true US20140280972A1 (en) | 2014-09-18 |
US9635102B2 US9635102B2 (en) | 2017-04-25 |
Family
ID=51533716
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/204,354 Active 2034-08-08 US9635102B2 (en) | 2013-03-12 | 2014-03-11 | Broker module for managing and monitoring resources between internet service providers |
Country Status (1)
Country | Link |
---|---|
US (1) | US9635102B2 (en) |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20170187644A1 (en) * | 2015-01-26 | 2017-06-29 | Hewlett Packard Enterprise Development Lp | Resource allocation |
WO2017180774A1 (en) * | 2016-04-12 | 2017-10-19 | Quest Software Inc. | Optimal service provider selection |
WO2017180772A1 (en) * | 2016-04-12 | 2017-10-19 | Quest Software Inc. | Single tier routing |
US10157083B2 (en) * | 2016-09-14 | 2018-12-18 | Microsoft Technology Licensing, Llc | Outage detection and compute resource protection in a computer service environment |
USRE49334E1 (en) | 2005-10-04 | 2022-12-13 | Hoffberg Family Trust 2 | Multifactorial optimization system and method |
US20220400161A1 (en) * | 2021-06-15 | 2022-12-15 | Citrix Systems, Inc. | Systems and methods for routing remote application data |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2015165029A1 (en) * | 2014-04-29 | 2015-11-05 | 华为技术有限公司 | Resource reuse method and apparatus |
US20190334785A1 (en) * | 2018-04-26 | 2019-10-31 | Microsoft Technology Licensing, Llc | Forecasting underutilization of a computing resource |
Citations (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20020178263A1 (en) * | 2001-03-30 | 2002-11-28 | William Hreha | Dynamic resource allocation architecture for differentiated services over broadband communication networks |
US20030101124A1 (en) * | 2000-05-12 | 2003-05-29 | Nemo Semret | Method and system for market based resource allocation |
US20040107281A1 (en) * | 2001-08-17 | 2004-06-03 | Pratik Bose | Dynamic allocation of network resources in a multiple-user communication sytem |
US20040111308A1 (en) * | 2002-12-09 | 2004-06-10 | Brighthaul Ltd. | Dynamic resource allocation platform and method for time related resources |
US7043225B1 (en) * | 2000-02-25 | 2006-05-09 | Cisco Technology, Inc. | Method and system for brokering bandwidth in a wireless communications network |
US7647411B1 (en) * | 2001-02-26 | 2010-01-12 | Symantec Corporation | System and method for controlling distribution of network communications |
US20110153802A1 (en) * | 2009-12-17 | 2011-06-23 | Steiner Moritz M | Localization of peer to peer traffic |
US8260959B2 (en) * | 2002-01-31 | 2012-09-04 | British Telecommunications Public Limited Company | Network service selection |
US9032077B1 (en) * | 2012-06-28 | 2015-05-12 | Amazon Technologies, Inc. | Client-allocatable bandwidth pools |
-
2014
- 2014-03-11 US US14/204,354 patent/US9635102B2/en active Active
Patent Citations (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7043225B1 (en) * | 2000-02-25 | 2006-05-09 | Cisco Technology, Inc. | Method and system for brokering bandwidth in a wireless communications network |
US20030101124A1 (en) * | 2000-05-12 | 2003-05-29 | Nemo Semret | Method and system for market based resource allocation |
US7647411B1 (en) * | 2001-02-26 | 2010-01-12 | Symantec Corporation | System and method for controlling distribution of network communications |
US20020178263A1 (en) * | 2001-03-30 | 2002-11-28 | William Hreha | Dynamic resource allocation architecture for differentiated services over broadband communication networks |
US20040107281A1 (en) * | 2001-08-17 | 2004-06-03 | Pratik Bose | Dynamic allocation of network resources in a multiple-user communication sytem |
US8260959B2 (en) * | 2002-01-31 | 2012-09-04 | British Telecommunications Public Limited Company | Network service selection |
US20040111308A1 (en) * | 2002-12-09 | 2004-06-10 | Brighthaul Ltd. | Dynamic resource allocation platform and method for time related resources |
US20110153802A1 (en) * | 2009-12-17 | 2011-06-23 | Steiner Moritz M | Localization of peer to peer traffic |
US9032077B1 (en) * | 2012-06-28 | 2015-05-12 | Amazon Technologies, Inc. | Client-allocatable bandwidth pools |
Cited By (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
USRE49334E1 (en) | 2005-10-04 | 2022-12-13 | Hoffberg Family Trust 2 | Multifactorial optimization system and method |
US20170187644A1 (en) * | 2015-01-26 | 2017-06-29 | Hewlett Packard Enterprise Development Lp | Resource allocation |
US10536394B2 (en) * | 2015-01-26 | 2020-01-14 | Hewlett Packard Enterprise Development Lp | Resource allocation |
WO2017180774A1 (en) * | 2016-04-12 | 2017-10-19 | Quest Software Inc. | Optimal service provider selection |
WO2017180772A1 (en) * | 2016-04-12 | 2017-10-19 | Quest Software Inc. | Single tier routing |
US10157083B2 (en) * | 2016-09-14 | 2018-12-18 | Microsoft Technology Licensing, Llc | Outage detection and compute resource protection in a computer service environment |
US20190073246A1 (en) * | 2016-09-14 | 2019-03-07 | Microsoft Technology Licensing, Llc | Outage detection and compute resource protection in a computer service environment |
US10565023B2 (en) * | 2016-09-14 | 2020-02-18 | Microsoft Technology Licensing, Llc | Outage detection and compute resource protection in a computer service environment |
US20220400161A1 (en) * | 2021-06-15 | 2022-12-15 | Citrix Systems, Inc. | Systems and methods for routing remote application data |
US11582325B2 (en) * | 2021-06-15 | 2023-02-14 | Citrix Systems, Inc. | Systems and methods for routing remote application data |
Also Published As
Publication number | Publication date |
---|---|
US9635102B2 (en) | 2017-04-25 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US9635102B2 (en) | Broker module for managing and monitoring resources between internet service providers | |
US11252220B2 (en) | Distributed code execution involving a serverless computing infrastructure | |
US20190324819A1 (en) | Distributed-system task assignment method and apparatus | |
US20210149737A1 (en) | Method for fast scheduling for balanced resource allocation in distributed and collaborative container platform environment | |
EP3577561B1 (en) | Resource management for virtual machines in cloud computing systems | |
JP6490913B2 (en) | Task execution by idle resources of grid computing system | |
US8156179B2 (en) | Grid-enabled, service-oriented architecture for enabling high-speed computing applications | |
US20230033296A1 (en) | Managing composition service entities with complex networks | |
US8185905B2 (en) | Resource allocation in computing systems according to permissible flexibilities in the recommended resource requirements | |
US7996525B2 (en) | Systems and methods for dynamically provisioning cloud computing resources | |
US9906589B2 (en) | Shared management service | |
EP3782030A1 (en) | System for managing deployment of distributed computing resources | |
US20170031622A1 (en) | Methods for allocating storage cluster hardware resources and devices thereof | |
US20140173620A1 (en) | Resource allocation method and resource management platform | |
WO2019091387A1 (en) | Method and system for provisioning resources in cloud computing | |
CN105612539B (en) | Producer system partitioning among leasing agent systems | |
US20220075665A1 (en) | Scheduling method for selecting optimal cluster within cluster of distributed collaboration type | |
US20200272526A1 (en) | Methods and systems for automated scaling of computing clusters | |
US20210191751A1 (en) | Method and device for allocating resource in virtualized environment | |
US20220326986A1 (en) | Scheduling workloads on partitioned resources of a host system in a container-orchestration system | |
US11108700B2 (en) | Method and system to implement messaging for cluster elections | |
EP4260185A1 (en) | System and method for performing workloads using composed systems | |
JP2010097566A (en) | Information processing apparatus, and method for assigning batch processing in information processing system | |
JP6374059B2 (en) | Computer resource allocation determination method, computer resource allocation determination method program, and control computer | |
US11809912B2 (en) | System and method for allocating resources to perform workloads |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: CII NOW, INC., CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:CALIPPE, JOEL;SOMADDER, GURUDAS;SIGNING DATES FROM 20140417 TO 20140421;REEL/FRAME:032731/0375 |
|
AS | Assignment |
Owner name: GOOGLE INC., CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:CIINOW, INC.;REEL/FRAME:033621/0128 Effective date: 20140729 |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
AS | Assignment |
Owner name: GOOGLE LLC, CALIFORNIA Free format text: CHANGE OF NAME;ASSIGNOR:GOOGLE INC.;REEL/FRAME:044097/0658 Effective date: 20170929 |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 4 |