US20140274304A1

US20140274304A1 - Method and apparatus for evaluation of skill level progression and matching of participants in a multi-media interactive environment

Info

Publication number: US20140274304A1
Application number: US13/899,395
Authority: US
Inventors: Jonathan Haswell
Original assignee: Ignite Game Technologies Inc
Current assignee: Ignite Game Technologies Inc
Priority date: 2013-03-13
Filing date: 2013-05-21
Publication date: 2014-09-18
Also published as: US9216353B2; US20140274242A1

Abstract

In an online interactive simulation or game environment, in which an object's motion is controlled by a player, a player's skill level is quickly quantified as they control the object to properly place the player in games with those of like skill without having to compete with others. An optimal motion is established for the object's travel during a portion of a game. At each increment of travel, an optimum velocity or time delta is established. The motion of the object that is being controlled by a player who is being rated is then tracked relative to the established optimal motion. Deviations therebetween are calculated on an incremental basis, and the aggregate score is used to determine the player's skill level for a set of equivalent conditions.

Description

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims priority to U.S. provisional patent application Ser. No. 61/779,332, filed Mar. 13, 2013, which application is incorporated herein in its entirety by this reference thereto.

BACKGROUND OF THE INVENTION

1. Technical Field
The invention relates to interactive electronic games and interactive simulation environments. More particularly, the invention relates to skill quantification, skill progression analysis and skill matching with respect to an interactive simulation environment.
2. Description of the Background Art
Player skill can be evaluated in games for a number of purposes. For multi-player games, a ranking system is predominately used both to identify and to track the skills of gamers in a game to match them into competitive matches better.
Probably the most popular skill ranking and matching system used today for multi-player games is Elo. The Elo rating system calculates the relative skill levels of players in two-player games, such as chess. It is named after its creator Arpad Elo, a Hungarian-born American physics professor. The Elo system was developed as an improved chess rating system, but today it is also used in many other games. Per the Elo system, each player has a rating, which is a number. A higher number indicates a better player, based on results against other rated players. The winner of a contest between two players gains a certain number of points in his rating and the losing player loses the same amount. The number of points won or lost in a contest depends on the difference in the ratings of the players. Thus, a player gains more points by beating a higher-rated player than by beating a lower-rated player. Over a series of games, a player's rating goes up if the player does better than expected.
Another popular skill matching system is the TrueSkill™ system developed by Microsoft Research. The TrueSkill ranking system only uses the final standings of all teams in a game to update the skill estimates (ranks) of all gamers playing in the game. One difference from other ranking systems is that the TrueSkill ranking system characterizes skill by two numbers, i.e. the average skill of the gamer, and the degree of uncertainty in the gamer's skill. If the uncertainty is high, the ranking system does not yet know exactly the skill of the gamer. In contrast, if the uncertainty is small, the ranking system has a strong belief that the skill of the gamer is close to their rated average skill.
Both the Elo and TrueSkill rating systems, as well as other known skill rating systems, assume that the skill of a player in a multi-player competitive game can only be determined as a result of competition. It would be advantageous if a player's skill could be determined quickly without competing against others, especially for games that include online interactive simulation environments, such as vehicle racing.

SUMMARY OF THE INVENTION

Embodiments of the invention provide an online interactive simulation and/or game environment in which an object's motion is controlled by a player along a path. A player's skill level is quickly quantified as they control the object traversing the path. In this way, the player can be properly placed in games with those of like skill without having to compete with others. An optimal path is established for the object's travel during a portion of a game, and at each increment of travel, an optimum velocity or time delta is established. The path of the object that is being controlled by a player who is being rated is then tracked over the same path. At each distance increment the object's position and velocity or time delta are recorded and compared with an optimum value. Deviations therebetween are calculated on an incremental basis, and the aggregate score is used to determine the player's skill level for a set of equivalent conditions.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram depicting the path of a simulated vehicle as it negotiates a section of simulated race track or course, including a straight portion and a turn, and also including incremental measurements that are made to measure player skill relative to target or optimal skill according to the invention;

FIG. 2 is a flow chart describing how player skill is measured for a player who is controlling a virtual object as it negotiates a path in a virtual environment according to the invention;

FIG. 3 is a block schematic diagram that shows a system for measuring player skill according to the invention;

FIG. 4 is a table that lists player controllable inputs according to the invention; and

FIG. 5 is a block schematic diagram that depicts a machine in the exemplary form of a computer system within which a set of instructions for causing the machine to perform any of the herein disclosed methodologies may be executed.

DETAILED DESCRIPTION OF THE INVENTION

An apparatus and method is described for operating an online interactive simulation and/or game environment, in which an object's motion along a path is controlled by a player. A player's skill level is quickly quantified while the player controls the object traversing the path, so that the player can be properly placed in games with those of like skill. This is accomplished without having the player compete with others.
In embodiments of the invention, an optimal path is established for the object's travel during a portion of a game and, at each increment of travel, an optimum velocity or time delta is established. The path of the object that is being controlled by a player being rated is then tracked over the same path. At each distance increment the object's position and velocity or time delta are recorded and compared with the optimum. Deviations therebetween are calculated on an incremental basis, and the aggregate determines the player's skill level for a set of equivalent conditions.
For simplicity, the embodiments described herein focus on vehicle racing as an exemplary and non-limiting embodiment of the invention. The method and apparatus described herein is also applicable to any interactive simulation environment where a player controls the movements of an object along a path, and where an optimum path and velocity profile can be constructed for a particular set of conditions. The method and apparatus described herein is also applicable to any skill quantification where an individual's skill may be measured with respect to an optimum or target skill level without that player having to compete with other players.
A vehicle may race over a closed-loop or open loop course. For a closed-loop course, each circumvention of the course is referred to as a lap. In one embodiment of the invention, over the course of a lap, for example, a vehicle's path is tracked both on straight sections or straights, as well an in turns. It is in turns where a vehicle's optimum path and incremental velocity is most influenced by track conditions. It is also in turns that a player's skill tends to deviate the most when compared with an optimal level of skill because a mistake made in a turn can have a major negative affect on velocity for a straight that follows the turn.
The fastest path around a race track is dependent on a number of variables, including but not limited to:

- The vehicle type and capabilities;
- The track surface condition, e.g. dry, wet, oil covered, debris covered, etc.;
- One or more instances of passing another vehicle or being passed; and
- The entry point for a turn;

The optimal or target path and time/velocity profile for a player controlled object's travel along that path shall be herein called tS or Target Skill. For a vehicle racing simulation or game environment, this normally includes a target time duration for traversing the path that is the minimum possible under the chosen conditions. However, for some simulation or game environments the duration for traversing the path may not be the minimum possible but, instead, may include a specific profile of velocity along an optimal physical path. The path and velocity profile for a player controlled object's travel along a path as guided and influenced by a specific player shall be herein called pS or Player Skill.

Knowing Skill

A key premise of Skillquant™, an apparatus and method according to the invention that is also associated with the online virtual multi-player racing environment called Simraceway®, is that skill is knowable and precisely quantifiable. Those skilled in the art will appreciate that the invention is readily applied to other systems and that references to Skillquant™ and Simraceway® are solely for purposes of example and explanation, and not by way of limitation. While one specific Skillquant implementation cannot necessarily be used for all game types universally, adaptations of the apparatus and method described herein enable variations on Skillquant implementations that support other types of vehicle performance and racing games, and other genres of games where an object is guided over a path by a player, and where an optimal profile for velocity over that path can be determined.

Players

A player is a participant who is able to influence the outcome of the game. However, the identification of players may not always be simple. For instance, in the online interactive racing game Simraceway®, pit crews and officials are players as well as drivers because they can influence the outcome of a match or race. Also, the abstract third party of chance can in some cases be considered a participant as well. For instance, a pit crew's performance can be determined by a real player participating in an event. Alternately, the performance of tasks normally performed by a pit crew can be performed automatically by the game provider, e.g. by artificial intelligence (AI) players, with variations on that performance being applied according to chance. Just as the best F1 driver occasionally has a bad pit stop through no fault of the driver, a chance element can accomplish a similar effect.
Target Skill (Hereinafter “tS”)
Target Skill is the optimal skill that a player can in theory exhibit over the game or simulation at any given point in the game. This is the element of the gameplay wherein the player can control the outcome by performing as close as possible to this optimal level. For games of material skill (S) and material chance (C), tS is concerned with the player's successful control of S. Note that for games where the element of chance C is exerted by the provider, the exercise of a high degree of player skill pS even approaching the tS level does not necessarily derive an outcome in the player's favor, if for instance chance C is the predominating factor.
Overall, target skill (tS) is the optimal driving line for a vehicle racing game or simulation. However, a specific target skill (tS) calculation is different for each game type when the techniques described herein are applied to other forms of games or simulations.
Different methods and/or algorithms are available to determine the fastest line on a race track. One method is to have a computer system run a simulation to drive a ghost vehicle around a track while taking different lines through turns, braking at different points, and applying the throttle at different times exiting a turn until, through a process of successive approximation, a convergence is observed on a line and velocity profile that provides the fastest lap for a set of conditions.
Player Skill (Hereinafter “pS”)
Player Skill is the skill exhibited by a player relative to target skill (tS). When a player is for example controlling a vehicle, pS is determined based on the vehicle's position relative to the limit line, where the limit line is the optimal physical path for a given set of conditions. Here, for example, the value 0 means that the vehicle is perfectly aligned with the optimal path and deviation values above this, measured for example in centimeters, are less well aligned.
The track diagram 100 of FIG. 1 shows a portion of a vehicle racetrack, including a straight 112 leading up to a turn or corner 114. Here, the path representing target skill (tS) 102 is shown as a solid line, and the path representing an exemplary player skill (pS) 104 is shown as a dotted line. A vehicle enters this section of a race track at an entry point 106. According to the tS line 102, the fastest path turns-in from the far side of the track and subsequently approaches the inner edge 108 at the apex of the turn. For this example, the inner edge of the turn comprises a shaded area representing a curbing. After passing near the apex of the corner, the tS line drifts to the outside of the track as it reaches the exit 110 of the turn.
FIG. 1 shows a number of exemplary deviations 116 where the player controlled physical path (pS) deviates from the optimal tS path. According to an embodiment of the invention, physical deviations from the optimal path are calculated at each distance increment based upon which skill is being monitored. This distance increment could be any physical scale, including for example centimeters. While the increment for calculating deviations may occur at regular distance intervals along a path, the resolution for some simulations may be limited to the frame-rate for a video display output. In FIG. 1, two adjacent deviation measurements 118 are shown with an exemplary distance increment 120 separating them.
FIG. 2 is a flow diagram 200 that describes a process for determining pS relative to tS for a player controlled object traversing a physical path, where the physical path is represented in a virtual simulation environment.
In step S210, for a given set of conditions an optimal path is established for a player controlled virtual object and an optimal timing or velocity profile is established for traversing that path. The combination of the optimal physical path and optimal timing or velocity profile represents target skill (tS).
In step S220, the motion of the virtual object is monitored as it traverses the path under player control with respect to time and position.
In step S230, at each increment of distance along the path the system measures and records any distance deviation of the controlled object's position from the ideal path, and any timing deviation from the ideal timing profile. The timing deviation may be represented as a time delta relative to an optimal time for the controlled object to pass through a specific distance increment along the path. The timing deviation may also be represented as a velocity relative to an optimal velocity at a specific distance increment along the path.
Subsequently in step S240, each of the distance and timing deviations at all distance increments along the path are aggregated.
In step S250, a score rating for the particular player controlling the object along this path is created based on the aggregated result, providing an overall pS value for the player with respect to that path. As a player participates in more games, the player's pS ratings can be aggregated to determine a pS skill rating that benefits from experience over multiple games.

Skillquant Assumptions

In an embodiment of the invention, the following is assumed:

- The system has perfect information, i.e. all material input variables are knowable in a virtual game or race. They are not in real life.
- Every vehicle and track combination has been tested and given a tS limit line or target skill value for each of a set of conditions. If a condition changes, the tS limit line may have changed as well. The handling of changed conditions is described below.
- Events for a vehicle racing game can include for example the following types:
  - Hotlap—Single player events in pursuit of fastest lap
  - Challenge Races—Multi-lap, multiplayer races of <10 laps
  - Endurance Challenges—Multi-lap, multiplayer races of >10 laps and involving a pit stop
- Only Endurance Challenges require pitting.
- By default, pit crews have equal skill when they are artificially generated. Alternately an element of chance can be injected at random by occasionally having some driver's pit stops take longer.
- All cars by default have fixed set ups, e.g. suspension adjustment, brake bias adjustment, wing settings, etc. If any vehicle parameter may be adjusted by a driver, then that parameter becomes a “Player Controllable Input.”
- If it is desirable to minimize the need to adjust tS dynamically or choose an alternate tS due to changing track conditions then, by default, track or environment condition changes either do not occur or have definable start and end times during an event. This enables their impact on the tS limit line to be quantified more easily. Note that more complex solutions that take changing conditions into account are described below.

For an exemplary vehicle racing game, a player (p) may include any of the following:

- Online racers—Human or artificial intelligence (AI) or simulated racers
- Pit Crews—Human or AI
- Actual Chance—Network interruptions, power surges or failures, client or server processing stutters or delays
- Simulated Chance—Programmed random variables representing the real chance factors exhibited in real motorsport, for example tire blowouts, mechanical failures, third party player accidents, etc.

Event Instance and Game Instance

An event instance is a single complete session of a simulation. This is different for each event type and is defined uniquely for each event. In Simraceway terms, this may be, for example, a Hotlap Competition, Challenge, or Quick Race, where the number of laps is typically different for each type of event. In Simraceway for example, a Hotlap allows the player to attempt to achieve a single perfect lap, whilst a Challenge event may ask entrants to complete several hundred laps with the added difficulty of each entrant trying to interfere with each other's line. However, these can both be calculated in the same way by distilling the problem to the lap level and summing the equally weighted variance of the event's constituent laps. In embodiments of the invention described herein for closed-loop vehicle racing events, a game may be thought of as a single lap where the Skillquant scores for each constituent lap are aggregated to determine the corresponding event's Skillquant score.
Note that the definitions for game and event instances for some game and event types is straightforward in some cases but not for others. For example, a golf game may be viewed not as a game of 18 holes, but rather 18 games of a single hole played in an arbitrary sum succession. Alternately, a game of golf may be viewed as a succession of games where each game comprises a single stroke, with the event still comprising 18 holes. Here, to provide a total pS rating for an event, the individual pS ratings for all strokes may be summed over the total of all 18 holes.

Dynamically Adjusting tS for Changing Conditions

When a player, for example in a vehicle racing game, finds himself forced off the normal tS line, the player's skill can still be judged effectively if tS is temporarily adjusted for that section of the track. This can be accomplished by at least two methods:

- A plurality of alternate tS lines can be predefined before an event or game such that when a vehicle deviates from the optimal line, they are thereafter compared with an alternate tS line, at least for that portion of the track.
- The tS line for that portion of the track can be dynamically altered according to an algorithm that takes into account the changing conditions. For instance, if a vehicle moves off the optimal tS line toward the inside while entering a turn to pass a slower car, the algorithm can dynamically adjust tS so that the vehicle's position deviation is essentially zero at that point, and the optimal velocity or time profile is set to be consistent with the driver's velocity as his vehicle pulls even with the vehicle being passed.

Computing pS

In an embodiment, Player Skill (pS) is derived as the distance and time variance from Target Skill (tS) at each distance increment along a path on a single point basis, or optionally on a covered volumetric approach where instead of determining an object's position as a single point, the system according to the invention uses a total 2-dimensional volume of the object in determining the object's position. In the case of a vehicle racing game, the latter method is slightly more accurate, particularly where rotation or yaw of a racing vehicle is concerned.
For a PDC hotlap event (2.3 miles per lap) a lap provides, for example in raw form, approximately 370 k data points (2.3 miles*160,934 cm per mile) for monitoring pS. Each data point for pS includes a position or distance deviation, typically measured for example in cm, and a timing deviation which can be implemented as either units of Δ time, or units of instantaneous Δ velocity for a particular distance increment along the path, the velocity being compared with the instantaneous velocity corresponding to tS at that point. To compute pS, as shown in FIG. 2, these deviations are recorded as shown in step S230 and then aggregated per step S240. The aggregation can be performed according to a number different methods, for example:

- Distance and timing deviations can be recorded separately at each distance increment, a standard deviation can be calculated for each parameter for that distance increment, and the results then aggregated for the entire path.
- Distance and timing deviations can be aggregated at each distance increment, and then a standard deviation can be calculated that aggregates the combined deviations over the entire path.
- Distance and timing deviations can be recorded separately at each distance increment and multiplied together for that distance increment. A standard deviation can be calculated for the results of the multiplications for each distance increment, and the results then aggregated for the entire path.

Regardless of which method is chosen for aggregation of the deviations, it may be advantageous to compute the standard deviation for the aggregated deviations. It may also be advantageous to convert the aggregated deviations to a logarithmic form because there can be for some events a significant range between pS scores of different players. A logarithmic scale enables metering very high fidelity at one end, for example the centimeters/millimeters and 10ths of seconds of leaders, and yet also handle new users who are a lap behind. Scores below a certain level may be cut off or eliminated because they are essentially too low to be meaningful.
After computing a pS score, it is added to the Skillquant database and then typically rendered to the user as a percentile specific to a game or event.

Generalized System Configuration

FIG. 3 is an overview diagram 300 that shows the major components involved in operating a virtual simulation environment or game according to the invention, including Mods (virtual models) for vehicles 310 and tracks 312 for inclusion in an exemplary vehicle racing game. An optimal tS line and associated timing profiles are established with respect to at least a portion of a track 312. A game client 302 resides on a user machine 304 and communicates via a network with a game server 306 located, for example, on a host game website 308. Most often the network connecting client 302 and server 306 is the Internet.
To install a new model of a vehicle 310 or a track 312, an appropriate compressed file containing the necessary files and directory structure are downloaded from a content delivery network (CDN) 314 or, alternatively, from the game website 308. Software operating on any of user machine 304, game server 306, or CDN 314 may operate on one or more processors and/or one or more servers, and use memory and database resources located thereon. Computing and database resources according to the invention may also operate in the cloud. The one or more processors and/or one or more servers may be co-located, spread over different locations, located in the cloud, or a combination thereof. For purposes of the invention, the use of servers that are part of a cloud computing platform provides a capability for easily expanding or contracting computing resources required. Thus, a system implemented according to the invention can quickly and efficiently adapt to client needs and it is only necessary pay for the computing resources needed at any point in time. A provider of cloud computing resources essentially provides infrastructure as a Service (IaaS)

Player Controllable Inputs (Hereinafter “PCIs”)

Each game has its own definition of player controllable inputs. While one embodiment of the invention described herein evaluates pS for a player in terms of incremental variation of an object or vehicle's path with respect to a position and velocity profile when compared with an optimal tS path, another embodiment for judging player skill compares a player's operation of his game console or controller device (their PCIs) with those of a virtual player operating an equivalent set of PCIs, for an equivalent set of operating conditions. For this alternative embodiment or complex form of the invention it is assumed when a player operates their PCIs in a like manner to a virtual player performing optimal operation of equivalent PCI's and under the same conditions, that their skill level approaches that of a target skill level (tS).
A player's operation of PCIs may be measured in different terms, and these terms should be appropriately adapted to a particular game and game controller console. For a vehicle racing game for example, PCIs such as brake and throttle may be measured in terms of percentages relative to a maximum brake position and a maximum throttle position, respectively. PCI's such as steering input can also be measured on a percentage basis, or alternately measured on a basis measuring degrees of rotation where 360° represents a full rotation of a rotatable steering input device, and 540° represents one and a half rotations of a rotatable steering input device. If a joystick is used instead controlled steering input, a percentage basis might be more appropriate. When pS is determined and compared with tS on the basis of PCIs, it is important that methods implemented at the provider properly take into account the relative influence of each PCI. This is critical for a player to be correctly assessed when this embodiment is used.
Thus, for a complex form based on PCIs, pS can be alternately calculated whereby pS variance from tS is measured not by the object's variance with respect to a tS comprising a physical path and a timing profile (the simple form), but, instead, by the variance on PCIs compared with a tS defined for PCIs. For example, a player may hold a throttle position on a game console (pS) at 72% vs. an optimal tS position of 76%, at a specific increment (time or distance) of the path that the controlled object traverses.
Table 400 of FIG. 4 provides a list of player controllable inputs. Column 402 shows inputs controlled by beginners and column 404 shows inputs controlled by intermediate players. For the beginner and intermediate players, a (%) is shown for each input type and represents the % of control a system provides to those broad skill class users. This is sometimes described as a skills appropriate experience for the user. Controllable inputs for advanced players are shown in column 406, the simulation column 408 refers to a broad label for the top group of Skillquant ranges for ease of reference. Note that the % values are estimates and vary with driver and/or player experience level. For example different aides can be offered to beginners vs. intermediates vs. experts.
The complex form with metering at player controlled inputs, rather than simply using positional output, can require up to eight separate calculations vs. just one calculation for the simple form. However, for combining with data taken from real racing events, telemetry data can be easily integrated because it already meters these driver and/or player inputs. The complex form is not necessarily required for everyday use, but has applications in professional services for driver development. Whilst more complex to build from the ground up, given that all telemetry can typically be accessed fairly easily, it may present an engineering shortcut.
Key differences of the complex form compared with the simple form include:

- pS derivation becomes a function of actual player controllable inputs rather than their output, being the motion of the object, vehicle, or car. This adds eight calculation steps.
- tS itself becomes more complex to calculate, consistent with a simulation which considers all environmental dynamic variables rather than assuming they are constant. This adds seven calculation steps for the following conditions, including but not limited to: track condition, tire wear, brake fade, fuel load, track temperature and altitude/fuel mix.
- Variance is calculated as both latitudinal and longitudinal from tS on the limit line.

Other Benefits of Measuring Skill

Metering the skill that creates an outcome, e.g. lap time or position for a vehicle racing simulation, as opposed to simply measuring the outcome has other benefits to both the game provider and the players. Some benefits include the following:

- Data Insight: The game and game provider can get an understanding of the skill behind the outcome, which helps the game and game provider improve skill matching. Improved skill matching directly influences engagement and where monetization is included, improved monetization.

Matching Engine

For asynchronous events, it is useful to use pS to match skill among players to:

- Determine (dynamically) the initial skill range of entrants.
- Calculate, according to a user operated ranges, skill cohorts, i.e. groupings of players with like skill, according to a desired minimum number of players and according to prize dynamics, the goal being to achieve the tightest range of skill cohort that achieves the required minimum number of entrants after which point a new cohort should be created.
- Admit users to their relevant skill cohort.
- After event entry registration closes, true up above the skill cohorts.
- Identify users who may cross skill cohorts.

Player Liquidity

Player liquidity represents the pool of players that are available at any one time for any one skill; race class; prize type; location; etc., for example the real time pool of players from which an operator and/or provider of a simulation or game (the “provider”) can reasonably match players against each other. The term tightly manage refers to the variance of skill that must be applied to draw enough matches. The more liquid the environment, for example the more players online at once and available to play, the tighter matches are. A dynamic element in the process of skill matching refers to widening and/or narrowing the range of skill variance used by the provider to drive matches according to the prevailing player liquidity. Quantifying a player's skill enables the game and game provider to manage player liquidity tightly via dynamic skill classing.

Other Skill Quantification Methods

While embodiments of the invention focus primarily on comparing a player's skill level with respect to guiding a vehicle along an optimal path that includes an optimal velocity profile, other methods for establishing a target skill level and rating a player skill level relative to the target skill level are possible. These include for a racing game comparing lap times for a player with an optimal lap time under equivalent conditions and with equivalent equipment. Another method can include evaluating instantaneous acceleration of an object being guided by a player at different points in the game. Skill can also be evaluated such that consistency of performance is included as a component of evaluating a player's skill. Regardless of what methods are used to evaluate a player's skill level, it is also useful to measure the skill progression of a player over time. Based on evaluations of skill progression, benefits arise for both a player and a provider.

Skill Progression and Real Time Experience Curves

Players improve their skill at different rates. Even a player who eventually has great skill may initially improve slowly. When first placing a player in a multi-player competition, it is critical for encouraging and maintaining player participation that their initial experience is as positive as possible. Therefore, it is advantageous that a player be matched against other players of similar skill, even in their very first competition and, therefore, their skill must be known prior to their first competition. Then, during the player's first competition they can enjoy a fair and competitive matching with other players, and simultaneously can be re-evaluated on their skill as the game progresses so that they are subsequently optimally matched for their second competition.
Continually quantifying a player's skill reveals to the game and game provider a user's likely skill progression through the game, and provides an opportunity to optimize the content and events that can be served to the player. Continually quantifying a player's skill also enables the establishment of real time experience curves which can be used to improve the comfort level of a player by having them feel competitive and, where a game includes any degree of monetization, get a player racing for cash sooner.
This is accomplished for example by dynamically matching car and race types to:

- A player's current skill level.
- Use a player's experience curve to understand to what level the provider can take the player's skill with a given car and/or track combination.

For example, a global skill data set may show that players who have progressed from skill level A to skill level B at rate X progress to skill point C faster if their next race is with car type A with track type A, and progress slower if their next race is with Car type B and Track type B.
The following example contrasts the methods of tracking skill described herein with prior art methods based on event outcomes. Assume a first player is running in 9^thposition in a 12 hr endurance race, however eventually places first because position's 1 and 2 caused a pile-up (accident) on the last lap that took-out themselves and the cars in positions 3-8 closely following. For this scenario, the winning driver is not really the most skilled in that race. They may have been for that last lap because it may have taken great skill to navigate the pile up, but they may also have just been lucky. Their pS line relative to a dynamically-adjusted tS line for the last lap would determine that. So, the winner's score here would be 9^thposition for the laps preceding the last and then 1^stfor one lap, which is not the same level of skill as the binary outcome would suggest.

Highlighting Restrained Players

Continually quantifying a player's skill level enables a game or game provider to highlight players and/or racers who are very skilled but are restrained by poor equipment and or vehicles. For example, a player may have chosen a vehicle simulation model that is inferior to others they compete against. Alternately, they may be using an inferior game control console or input device that provides an additional degree of difficulty that other game consoles may not impose.
By tracking the rate of skill progression with a given controller or input device type, which are tracked by the provider, any leaps in skill level that occur as a user upgrades to a different controller device are compared to a global dataset of such data. The results of such a comparison can reveal where players are hitting skill ceilings because of their interface equipment. For example, they may have progressed well with keyboards but have achieved their maximum available skill level with a keyboard and need to upgrade to a joypad or steering wheel input device to increase the skill level they can effectively exert.

Highlighting Sandbagging Players

Continually quantifying a player's skill enables a game or game provider to highlight players and/or racers who are very skilled but are attempting to sandbag, wherein a player intentionally makes mistakes to lower their apparent skill level which eventually enables them to be matched with other players they can easily beat. In embodiments of the invention, this is accomplished by looking for unusual, positive skill variance within events. For example, assume that an event consists of 2 levels, i.e. qualifying and race. Here, qualifying determines which of three classes (A to C, C is lowest) of race a player enters, where all classes may offer the same level of prize. If a user sandbags or artificially lowers their skill level to qualify for class C, but is in fact a Class B or Class A skilled player, the provider can track the skill leap from qualifying to race and trigger a variance flag. The variance flag indicates that having compared this particular player's skill progression to a global data set, it has been determined that no user with their equivalent starting skill level has ever progressed to that degree from one session to another. As a result of the variance flag, the provider may either warn or penalize the player in question, or alternately log the event and wait until a pattern of sandbagging has been established for that player before issuing a warning or penalty.

Increased Richness of Game Data

Continually quantifying a player's skill enables a game or game provider to accumulate a richer set of performance data for the player in a game, rather than just scoring the outcome. It provides skill data on a player's intra game performance, not simply a game's binary outcome, thereby providing a richer insight on how the black and white data of results was achieved.
According to embodiments of the invention, Skill Quantification (“Skillquant”) is a proprietary, modular, skill quantification and management engine that may be implemented as part of interactive electronic games and interactive simulation environments. Its central principle that differentiates from other skill and ranking systems is that the maximum skill a player can exert over a game is, in fact, knowable. As such, the skill of a specific player can be evaluated and measured against the known maximum level without that player having ever competed head-to-head with another competitor.
According to embodiments of the invention, player skill may be metered at distance or time increments having any chosen duration, however it is preferred to meter player skill in digital environments at the highest possible fidelity. This degree of unique data visibility facilitates the following functions, in a modular system configuration:

- Accurate, real-time, player skill quantification
- Accurate player skill matching with minimal player history
- Optimization of player engagement through real time experience curves
- Player liquidity management
- Compliance monitoring for both operator and competent authority
- Detection of cheating and fraudulent player behavior patterns

Consistency

Consistency can be a simple and robust constituent of a skill evaluation. However, this test must be applied within the correct skill tolerance. For example, the variance of skill that equals good or highly skilled for one game may well be different for another. A skill consistency number, supported by a statistically robust data set, can be hard to find. This is particularly the case when trying to make a legal classification for an entire population. For instance, using the skill tolerance or skill consistency of PGA golf players is not a good proxy for the entire user base of golf. Similarly, using the skill tolerance of F1 drivers is not a good proxy for the entire user base of race car drivers.
If a government, for example, wanted to determine whether racing enthusiasts in general should be allowed to play online interactive race car games for cash prizes, it is appropriate to determine what the predominant skill range of racing enthusiasts is. This should be tested, not simply at the top level of skill, which is generally where most data lies and so is tempting as a proxy, and which however represents a skill that is not possible for the general populace that would be participating online. It is necessary to determine what the general skill level is as practically operated by the predominant number of players playing the game. Digital environments, with perfect information available, are attractive for this reason.

Skill and Chance

Virtually every game of every genre, digital or otherwise, has some element of skill and some element of chance. There are some exceptions to this rule but fewer than one would expect, and these are contingent upon the definition of a game. For instance, one exception is “Naughts and Crosses” (alias tic-tac-toe) which on the surface seems to be a game where competent players always score a draw. However, it may actually be a simple form of mental exercise for developing young minds. Overall, an objective view of the subject requires one to detach their preconceived notions of games classification and instead focus entirely on the data.

Skill/Chance Paradox and Legal Classification

Where a game requires so much skill that to compete effectively it is practically beyond the reach of a majority of users, then upon testing it behaves as a game of chance for that majority of users. This can be relevant for legal classification purposes where competent authorities are not only concerned with testing the technical position of the game, but also scrutinizing its practical operation (predominance, predominance test) which is a well known legal test. A good example of this is exemplified by some variants of roulette that can actually be played as a game of skill by a very small number of people able to calculate the landing position of the ball based on its trajectory, but whereas the vast majority cannot. Another example are synthetic CDO's, which in a context similar to a racing game would be regulated as a game of chance, not a financial markets instrument. For such scenarios, if a test was applied to these financial markets to determine the degree of gambling involved, the predominant number of users would not be correctly determining and/or predicting their outcome from the engagement, and so would not able to skillfully control the outcome of that game. Thus, because there is an entry cost, i.e. acquisition of the CDO, and prize, i.e. the profit, these would be gambling. For these financial markets, core winners would probably be the market making banks who made the margin on each buy and sell trade, a lot like a casino. Note that financial products such as these evade gambling testing not by successfully passing the test of being a game of skill. Being correctly determined as a game of skill, specifically the thing that gambling is not, would represent the test that truly should determine how they should pass. Instead, governments simply carve them out by un-proven classification. Financial product definitions simply say that if something is a gambling game, it is not a financial investment product and vice versa.
With regard to compliance monitoring with respect to governmental regulations for online gambling, it is important to be able to rate the components of skill and chance in an online game when players wager on the outcome of a competition. As mentioned above, if the skill level required to compete effectively is extremely high, a competition takes on more characteristics of a game of chance for the average player. By matching players by skill levels and doing so right from the beginning of their participation, the ability to compete effectively based on skill is effective from the beginning and the game is may be classified more as a game of skill.
To make a game more interesting and enticing for some players, but still focus predominantly on skill, additional elements of chance may be injected into a competition. For instance, in a vehicle racing game, simulated weather conditions may be suddenly changed during an event, or a vehicle may suddenly leak simulated oil on the track creating a locally slippery condition. These sudden chance events may catch some drivers unaware regardless of their skill level. Also, automated pit stops may randomly encounter delays that cost even the best drivers time.

Skill/Chance Compliance Monitoring

Some governmental authorities monitor online games to ensure that they qualify as games of skill, especially when wagers are made on the outcomes. Thus, it is useful to quantify the composition of a game with respect to skill versus chance. In simplest terms, if by exercising their control console a player can control the outcome of a game instance, in this context, their finishing position after a lap or game, and relative position to tS, then this is a game of skill. If they cannot, it is a game of chance. The percentage degree to which they can exert that control is the percentage of skill contained in the game and the inverse value is the percentage of chance.

Computer Implementation

FIG. 5 is a block schematic diagram that depicts a machine in the exemplary form of a computer system 1600 within which a set of instructions for causing the machine to perform any of the herein disclosed methodologies may be executed. In alternative embodiments, the machine may comprise or include a network router, a network switch, a network bridge, personal digital assistant, a cellular telephone, a Web appliance or any machine capable of executing or transmitting a sequence of instructions that specify actions to be taken.
The computer system 1600 includes a processor 1602, a main memory 1604 and a static memory 1606, which communicate with each other via a bus 1608. The computer system 1600 may further include a display unit 1610, for example, a liquid crystal display (LCD) or a cathode ray tube (CRT). The computer system 1600 also includes an alphanumeric input device 1612, for example, a keyboard; a cursor control device 1614, for example, a mouse; a disk drive unit 1616, a signal generation device 1618, for example, a speaker, and a network interface device 1628.
The disk drive unit 1616 includes a machine-readable medium 1624 on which is stored a set of executable instructions, i.e. software, 1626 embodying any one, or all, of the methodologies described herein below. The software 1626 is also shown to reside, completely or at least partially, within the main memory 1604 and/or within the processor 1602. The software 1626 may further be transmitted or received over a network 1630 by means of a network interface device 1628.
In contrast to the system 1600 discussed above, a different embodiment uses logic circuitry instead of computer-executed instructions to implement processing entities. Depending upon the particular requirements of the application in the areas of speed, expense, tooling costs, and the like, this logic may be implemented by constructing an application-specific integrated circuit (ASIC) having thousands of tiny integrated transistors. Such an ASIC may be implemented with CMOS (complementary metal oxide semiconductor), TTL (transistor-transistor logic), VLSI (very large systems integration), or another suitable construction. Other alternatives include a digital signal processing chip (DSP), discrete circuitry (such as resistors, capacitors, diodes, inductors, and transistors), field programmable gate array (FPGA), programmable logic array (PLA), programmable logic device (PLD), and the like.
It is to be understood that embodiments may be used as or to support software programs or software modules executed upon some form of processing core (such as the CPU of a computer) or otherwise implemented or realized upon or within a machine or computer readable medium. A machine-readable medium includes any mechanism for storing or transmitting information in a form readable by a machine, e.g. a computer. For example, a machine readable medium includes read-only memory (ROM); random access memory (RAM); magnetic disk storage media; optical storage media; flash memory devices; electrical, optical, acoustical or other form of propagated signals, for example, carrier waves, infrared signals, digital signals, etc.; or any other type of media suitable for storing or transmitting information.
Although the invention is described herein with reference to the preferred embodiment, one skilled in the art will readily appreciate that other applications may be substituted for those set forth herein without departing from the spirit and scope of the present invention. Accordingly, the invention should only be limited by the Claims included below.

Claims

1. A computer implemented method for quantifying and using a progression of a player's skill with respect to an interactive computer game simulation, comprising:

continually quantifying a player's skill during one or more events of a game type;

based on a plurality of quantifications of the player's skill, predicting a skill level of the player for a next event; and

matching a player with other players of similar skill for the next event, based at least in part on the predicted skill level.

2. The method of claim 1, wherein quantifying the progression of the player's skill comprises comparing said player's skill level with a target skill level at one or more points in time during an event.

3. The method of claim 2, wherein said one or more points in time coincide with increments of any of time and distance.

4. The method of claim 1, further comprising:

creating skill cohorts comprising groupings of players with like skills; and

during a registration period, admitting a player to an appropriate skill cohort based on the player's predicted skill level.

5. The method of claim 4, further comprising:

after close of the registration period, identifying players who are likely to cross between skill cohorts during a next event; and

adjusting the skill cohorts as appropriate based in part on the identifying of players likely to cross between skill cohorts.

6. The method of claim 5, further comprising:

after the close of the registration period and based on available player liquidity, adjusting the skill cohorts as appropriate to create matches optimized for minimal skill variance within a cohort.

7. A method for quantifying and using progressions of skill for a plurality of players with respect to an interactive computer game simulation, comprising:

continually quantifying a player's skill during one or more events of a game type; and

determining for a plurality of players over a first time period comprising a first plurality of events, a first set of players who have progressed from a first skill level to a second skill level at a first rate.

8. The method of claim 7, further comprising:

determining for the first set of players over a second time period comprising a second plurality of events, equipment comprising any of a car type and a track type that has enabled a second set of players within the first set of players to advance faster to a third skill level during the second time period; and

encouraging the second set of players to continue using their current equipment.

9. The method of claim 7, further comprising:

determining over a second time period comprising a second plurality of events and for the first set of players, equipment comprising any of a car type and a track type that has enabled a third set of players within the first set of players to advance more slowly to the third skill level during the second time period; and

encouraging the third set of players to upgrade their equipment.

10. A method for quantifying and using a progression of a player's skill with respect to an interactive computer game simulation, comprising:

continually quantifying a first player's skill during one or more events of a game type;

analyzing a plurality of quantifications of the first player's skill to determine the first player's skill progression rate of increase or decrease over time; and

determining by a drop in the skill progression rate of increase that the first player is constrained by equipment, the equipment comprising any of an input device and characteristics of a game object model.

11. The method of claim 10, further comprising:

comparing the drop in the skill progression rate of increase for the first player exhibiting the recorded drop in the skill progression rate of increase with other players having similar equipment, wherein the other players exhibited increased rates of skill progression increase after upgrading their equipment; and

determining that the first player is constrained by his equipment based on similarities of skill progression rates of increase between the first player and the other players.

12. The method of claim 11, further comprising:

suggesting to the first player that he should upgrade his equipment.

13. A method for quantifying and using a progression of a player's skill with respect to an interactive computer game simulation, comprising:

analyzing a plurality of quantifications of the first player's skill to determine the player's skill progression rate of increase or decrease over time; and

determining by abnormal variations in the skill level of a player that the player appears to be sandbagging.

14. The method of claim 13, further comprising:

warning the player if he appears to be sandbagging.

15. The method of claim 13, further comprising:

penalizing the player if he appears to be sandbagging.

16. The method of claim 13, wherein the abnormal variations in the player's skill level comprise negative variations determined during a qualifying event followed by a skill level determined for the player measured during a race event that is consistent with a high skill level previously exhibited by the player.

17. The method of claim 13, wherein determining the abnormal variations in the player's skill level comprises comparing the variations in the player's skill level with a global data set of player skill variations.

18. An apparatus for quantifying and using a progression of a player's skill with respect to an interactive computer game simulation, comprising:

a processor continually quantifying a player's skill during one or more events of a game type;

based on a plurality of quantifications of the player's skill, said processor predicting a skill level of the player for a next event; and

said processor matching a player with other players of similar skill for the next event, based at least in part on the predicted skill level.

19. The apparatus of claim 18, wherein quantifying the progression of the player's skill comprises said processor comparing said player's skill level with a target skill level at one or more points in time during an event.

20. The apparatus of claim 19, wherein said one or more points in time coincide with increments of any of time and distance.

21. The apparatus of claim 18, further comprising:

said processor creating skill cohorts comprising groupings of players with like skills; and

during a registration period, said processor admitting a player to an appropriate skill cohort based on the player's predicted skill level.

22. The apparatus of claim 21, further comprising:

after close of the registration period, said processor identifying players who are likely to cross between skill cohorts during a next event; and

said processor adjusting the skill cohorts as appropriate based in part on the identifying of players likely to cross between skill cohorts.

23. The apparatus of claim 22, further comprising:

after the close of the registration period and based on available player liquidity, said processor adjusting the skill cohorts as appropriate to create matches optimized for minimal skill variance within a cohort.

24. An apparatus for quantifying and using progressions of skill for a plurality of players with respect to an interactive computer game simulation, comprising:

a processor continually quantifying a player's skill during one or more events of a game type; and

said processor determining for a plurality of players over a first time period comprising a first plurality of events, a first set of players who have progressed from a first skill level to a second skill level at a first rate.

25. The apparatus of claim 24, further comprising:

said processor determining for the first set of players over a second time period comprising a second plurality of events, equipment comprising any of a car type and a track type that has enabled a second set of players within the first set of players to advance faster to a third skill level during the second time period; and

said processor encouraging the second set of players to continue using their current equipment.

26. The apparatus of claim 24, further comprising:

said processor determining over a second time period comprising a second plurality of events and for the first set of players, equipment comprising any of a car type and a track type that has enabled a third set of players within the first set of players to advance more slowly to the third skill level during the second time period; and

said processor encouraging the third set of players to upgrade their equipment.

27. An apparatus for quantifying and using a progression of a player's skill with respect to an interactive computer game simulation, comprising:

a processor continually quantifying a first player's skill during one or more events of a game type;

said processor analyzing a plurality of quantifications of the first player's skill to determine the first player's skill progression rate of increase or decrease over time; and

said processor determining by a drop in the skill progression rate of increase that the first player is constrained by equipment, the equipment comprising any of an input device and characteristics of a game object model.

28. The apparatus of claim 27, further comprising:

said processor comparing the drop in the skill progression rate of increase for the first player exhibiting the recorded drop in the skill progression rate of increase with other players having similar equipment, wherein the other players exhibited increased rates of skill progression increase after upgrading their equipment; and

said processor determining that the first player is constrained by his equipment based on similarities of skill progression rates of increase between the first player and the other players.

29. The apparatus of claim 28, further comprising:

said processor suggesting to the first player that he should upgrade his equipment.

30. An apparatus for quantifying and using a progression of a player's skill with respect to an interactive computer game simulation, comprising:

said processor analyzing a plurality of quantifications of the first player's skill to determine the player's skill progression rate of increase or decrease over time; and

said processor determining by abnormal variations in the skill level of a player that the player appears to be sandbagging.

31. The apparatus of claim 30, further comprising:

said processor warning the player if he appears to be sandbagging.

32. The apparatus of claim 30, further comprising:

said processor penalizing the player if he appears to be sandbagging.

33. The apparatus of claim 30, wherein the abnormal variations in the player's skill level comprise negative variations determined during a qualifying event followed by a skill level determined for the player measured during a race event that is consistent with a high skill level previously exhibited by the player.

34. The apparatus of claim 30 wherein determining the abnormal variations in the player's skill level comprises comparing the variations in the player's skill level with a global data set of player skill variations.