WO2002086748A1

WO2002086748A1 - Method and apparatus for testing transaction capacity of site on a global communication network

Info

Publication number: WO2002086748A1
Application number: PCT/US2002/011947
Authority: WO
Inventors: Randall A. Hayes; Hunter Ellinger
Original assignee: Distributed Computing, Inc.
Priority date: 2001-04-18
Filing date: 2002-04-17
Publication date: 2002-10-31

Abstract

Method and apparatus for testing transaction capacity of site on a global communication network. A plurality of agents (118) are provided that are interfaced with the network, each of the agents (118) operable to access the customer server (104) over the network. A master server (110) selects a plurality of the agents (118) on the network as a load testing group. The plurality of agents (118) in the load testing group are activated with the master server (110) to initiate a load test operation of the customer server (104) for a predetermined amount of time. The agents (118) operable are to send the results of the load testing operation thereon to the master server (110) and the results stored at the master server (110).

Description

METHOD AND APPARATUS FOR TESTING TRANSACTION CAPACITY OF SITE ON A GLOBAL COMMUNICATION NETWORK

TECHNICAL FIELD OF THE INVENTION

[0001] The present invention pertains in general to load testing systems for a global communication network and, more particularly, to a distributed load testing system.

BACKGROUND OF THE INVENTION

[0002] The global communication network, typically referred to as the "internet" has seen an exploding use in recent years. The applications of this network extend from consumer- to-consumer to business-to-consumer to business-to-business applications. An important part of each of these applications is a server which must reside at a given location on the network and monitor incoming and outgoing traffic from the network. This traffic is typically the result of the request sent from a user on the network through the network to the server and, once received by the server, the request is processed therein. The server will then respond with the requested information by relaying it back through the network to the user.

[0003] In the normal course of handling a request, there are certain delays that may occur due to either the route taken through the overall network or with delays in handling information by the server. These delays, if too long, can detract from the overall efficiency and productivity of a network. The operation of a given business usually requires that their server handle the load, i.e., the number of users requesting information from the server, in an efficient manner such that delays are minimized. These delays can be the function of the type of request that is serviced, the manner by which it is serviced, etc.

[0004] The present systems that are provided to load test a server operate to "exercise" the server by residing on the network at remote locations and forwarding requests to the server. These requests are serviced and then the appropriate information returned to the requesting test node. A delay between the request and the receipt of the information by the test node constitutes the delay in the system. The disadvantage to these systems is that they comprise dedicated computers, which are relatively expensive, and are disposed at only a limited number of nodes in the system. SUMMARY OF THE INVENTION

[0005] The present invention disclosed and claimed herein, in one aspect thereof, comprises a method for load testing a customer server on a network. A plurality of agents are provided that are interfaced with the network, each of the agents operable to access the customer service over the network. A master server selects a plurality of the agents on the network as a load testing group. The plurality of agents in the load testing group are activated with the master server to initiate a load test operation of the customer server for a predetermined amount of time. The agents operable are to send the results of the load testing operation thereon to the master server and the results stored at the master server.

BRIEF DESCRIPTION OF THE DRAWINGS

[0006] For a more complete understanding of the present invention and the advantages thereof, reference is now made to the following description taken in conjunction with the accompanying Drawings in which:

[0007] FIGURE 1 illustrates a diagrammatic view of the overall system in the present disclosure;

[0008] FIGURE 2 illustrates a detail of a master server and an agent;

[0009] FIGURE 3 illustrates a flow chart depicting the general operation of the interface between an agent and a master server;

[0010] FIGURE 4 illustrates a diagrammatic view of a single agent during a transaction test;

[0011] FIGURE 5 illustrates a flow chart for initiating the load tests;

[0012] FIGURE 6 illustrates a flow chart for the operation of initiating a load test at an agent site;

[0013] FIGURES 7 and 8 illustrate flow charts depicting the load transaction test at the agent and the initiating thereof by the master;

[0014] FIGURE 9 illustrates a diagrammatic view of the manner by which multiple virtual agents are created at a single location;

[0015] FIGURE 10 illustrates a diagrammatic view of a remote customer terminal on the network for controlling the overall operation;

[0016] FIGURE 11 illustrates a flow chart operating at the master customer terminal;

[0017] FIGURE 12 illustrates a display generated at the master customer terminal during load testing;

[0018] FIGURES 13 and 14 illustrate a diagrammatic view of the load testing;

[0019] FIGUREs 15 and 16 illustrate displays of the delay as a function of the request type and the geographic location of the agents;

[0020] FIGURE 17 illustrates a flow chart for the set up operation that configures the load test;

[0021] FIGURE 18 illustrates a flow chart for the agent download operation;

[0022] FIGURE 19 illustrates a flow chart for creating the profile at the master server; [0023] FIGURE 20 illustrates a diagrammatic view of an alternate embodiment utilizing a calibration agent and a normal agent; and

[0024] FIGURE 21 illustrates a flow chart depicting the operation of receiving data from a calibration and a normal agent.

DETAILED DESCRIPTION OF THE INVENTION

[0025] Referring now to FIGURE 1, there is illustrated a diagrammatic view of the system of the present disclosure. There is provided a Global Communication Network (GCN) 102. A main customer server 104 is interfaced with the GCN 102 through a port 106. The port 106 represents a plurality of ports 108, it being understood that the main customer server 104 can access the GCN 102 with multiple parallel transactions. This is a conventional configuration.

[0026] The GCN 102 is also interfaced with a master server 110 which is operable to control and initiate load testing of the main customer server 104. The master server has associated therewith a controller function 112, a dispatcher function 114 and a scheduler function 116. The master server 110 is often operable to control a plurality of agent nodes 118 that are disposed about the GCN 102. The agent nodes 118 are labeled Al, A2, A3,..., AN. Each of the agent nodes is operable to run resident software that can be controlled by the master server 110, which software is operable to create a request, forward that request to the main customer server 104, and then receive a reply. This "transaction" is timed, such that a delay can be measured between the transmission of the request and the return thereof. This delay is a function of the path through the GCN 102, i.e., the time that is required for the main customer server 104 to process the request and the time for the request to be returned back through the GCN 102. It should be understood that, although each of the agent nodes 118 have a dynamic IP address on the network, the path from the agent to the main customer server 104 may not necessarily be the same path for the return reply. This is the nature of the Global Communication Network 102, in that every communication across the network involves a plurality of intermediate computers on the network. This data communication is typically performed utilizing a defined protocol, such as TCP/IP, a conventional protocol.

[0027] In the general operation of the system, the master server 110 initiates a load testing by selecting a plurality of agent nodes 118 for the load testing operation. Each of the agent nodes 118 is initiated by a dispatcher 114 in order to receive a request script and instructions as to how and when to play the script to the main customer server 104. This script consists of a plurality of requests in a predetermined order that are to be routed to the main customer server 104. Each of the agents 118 measures a delay between initiation of the script and completion thereof with the completion being the receipt of the final response. This delay is stored locally and, at a later time, transferred to the master server 110. typically, a given agent node may repeat the request multiple times and only send the measured delays back periodically.

[0028] This information at the master server 110 can then be accessed by the customer, either the main customer server 104 or another remote terminal (not shown).

[0029] Referring now to FIGURE 2, there is illustrated a diagrammatic view of the interaction between the master server 110 and one of the agent nodes 118. The agent node 118 has associated therewith a database 202 that is operable to store resident programs therein. Agent 118 is a general computer that resides on the GCN 102 and is basically a personal computer associated with either a business or an individual, this computer having access CPU time that can be utilized. This access CPU time is what is utilized by the master server 110 in order to provide requesting capacity for the load testing. These agents 118 are contracted with by a service provider that operates the master server 110. This service provider provides a contract with the agent 118 that remunerates the agent 118 for the amount of testing that is provided thereby for the load testing operation(s).

[0030] Once a contractual relationship has been determined between the agent 118 and the master server 110, the master server will provide software to the agent 118 that is disposed as a resident program in the database 202. This resident program typically runs as a process on the agent computer. This process can be initiated by the agent 118 to initiate a ready-to- test command along a status path 204 from the agent 118 over to the master server 110. The master server 110 that recognizes that the agent 118 is in a ready-to-test mode and logs this into a storage location therein. When the master server 110 is ready to begin a load test, it selects among one of the plurality of agent nodes 118. If the given agent node in FIGURE 2 is selected, then the master server 110 will send an acknowledgment command over a path 206 from the master server 110 to the agent 118 in order to instruct the agent to initiate a load test with the parameters that are downloaded thereto. These parameters instruct the agent as to what type of request to make and where to send those requests. Typically, this is a set of HTTP commands, although any type of network command language could be utilized, such as XML or the such.. The master server 110 will basically schedule the load test and, when time for testing occurs, the script will be dispatched to the select ones of the agent nodes 110.

[0031] Referring now to FIGURE 3, there is illustrated a flow chart indicating the operation of the interaction between the agent and the master server 110. This is initiated at a block 302 and then proceeds to a function block 304 wherein the agent sends the ready-to- test status information to the master server 110 which is stored therein, as indicated by function block 306. Once the database is updated, an acknowledgment is sent back to the agent 118 that it has been logged into the database. The program then proceeds to an End block 308. This in effect allows the agent or agent nodes 118 to participate on an as ready basis. Therefore, if a multitude of agents are contracted with, then the master server 110 need only have knowledge of which agents on the network have excess capacity that can be used for load testing. Typically, each of these agents will be connected to the GCN 102 over a high speed interconnection such as a DSL line or an ISDN connection.

[0032] Referring now to FIGURE 4, there is illustrated a diagrammatic view of the overall operation of the load testing. This load testing is illustrated with a single agent node 118, although it should be understood that a multitude of agent nodes 118 will be utilized in any load testing operation. The load testing operation is initiated by the customer server 104 (or another customer terminal that is interfaced with the GCN 102) by sending in a request along a path "1" from the customer server 104 over to the master server 110 through the network 102. An acknowledgment is sent back from the master server 110 to the customer server 104 along a path "2." The request comprises configuration information instructing the master server 110 as to how the load test is to be run and may even provide a script, if one was not previously set up in the master server 110 as a profile for that customer server 104. The request also provides some scheduling information to the master server 110. [0033] Once the master server has received all of the information necessary from a customer server 104 to initiate a load test, the master server 110 will then schedule this operation in the scheduler 116. Upon the predetermined time for the load test, the dispatcher 112 will contact the appropriate agent node (118) along a path "3" from the master server 110 over to the agent node (118) with sufficient information for the agent node 118 to conduct a load test. As described hereinabove, this constitutes such things as the request or requests to be sent to the customer server 104 and the number of requests to be made, i.e. a start time and an end time. This will be acknowledged by the agent 118 via a path. It should be understood that the master server 110 has a lookup table disposed therein with the ones of the agent nodes 118 that are in the ready-to-test mode. If an acknowledgment is not received back from the agent node 118, this indicates that the addressed one of the agent nodes 118 is not available and it will then be dropped from the list. However, if an acknowledgment is received, then the master server 110 considers this as one of the multitude of agent nodes 118 that are in the load test.

[0034] Once the load test is initiated, the agent node 118 will send an HTTP command or request along a path "5" from agent node 118 over to the customer server 104. This will be processed by the customer server 104 and then returned back along a path "6" to the agent node 118. The agent node 118 will measure the delay between the transmission of the HTTP command and the receipt of a response. It may be that there are a sequence of commands that must be sent in one "script," the overall delay of that script execution measured as the performance parameter of the system. The script may, and typically will be, repeated a large number of times. Periodically, the delays are averaged and accumulated and forwarded back to the master server 110 over the path "4."

[0035] Referring now to FIGURE 5, there is illustrated a flow chart initiating the operation at the customer server 104. This is initiated at a block 502 and then proceeds to a decision block 506 to determine if a log in has been received. Once the customer server 104 is verified at the login, the program will flow along the "Y" path to a decision block 508, wherein it is determined whether a profile is present for the customer server 104, this representing a predetermined script. If not, the program will flow to a function block 510 to create the profile and if not, decision block 508 will flow to a function block 512, the same destination as for the function block 510. Function block 512 indicates the operation wherein the load test is configured. This configuration will constitute such things as the length of time that the load test should be performed, the number of agents that are to be involved in the load test, the location of the agents, etc. Once configured, the program will then flow to a decision block 514 to determine if the configuration is complete. If not, the program will flow along the "N" path back to the input of block 512 and, when done, the program will flow along the "Y" path to an End block 516.

[0036] Referring now to FIGURE 6, there is illustrated a flow chart depicting the operation wherein the master server initiates the operation of the load test at the agent. The program is initiated at a block 602 and then proceeds to a decision block 604 to receive an initiating command. Once the initiating request is received, the program will flow to a function block 606 to receive the various parameters, these indicating all of the necessary parameters to effect a load test by that particular agent. As will be described hereinbelow, each of agent nodes 18 could comprise a plurality of virtual agents and, therefore, a plurality of parameters may be received to allow each of the virtual agents to operate independently of each other and even operate in different load tests for different customer servers.

[0037] Once the run parameters have been sent, an acknowledgment is sent back to the master server, as indicated by a function block 608. The program then flows to a function block 610 to begin the run operation, wherein the load test is executed at the agent location of the CPU operating thereat. The program then flows to a decision block 612 to determine if the load testing transactions have been completed. Once the load testing has been completed, the program will flow along the "Y" path to an End block 614.

[0038] Referring now to FIGURE 7, there is illustrated a flow chart that depicts the operation of the run operation on the agent node 118. The program is initiated at a block 702 and then proceeds to a function block 704 to set up a given load testing session. This load testing session, as is described hereinabove, comprises the operation of sending requests, waiting for responses and measuring the delay between the time of the transmission of the request to the time of the receipt of the response. Once the session is set up, i.e. all of the parameters have been received and the agent node 118 "parameterized," then the load testing session is ready to execute. This is indicated by a function block 706, wherein an HTTP request is sent out over the network. Once the request is sent, a timer is started, as indicated by function block 708, and then the program flows to a decision block 710 to determine if a response has been received. The program will continue in a loop until the response has been received, at which time it will flow to a function block 712 to stop the timer and then to a function block 714 to log the delay for the given response. It should be noted that the timer is started to the time it is stopped could involve multiple requests for a given load test. For example, the customer may determine that more than just a simple command is required to test the system. There may be a series of requests and responses that must be transmitted and received in order to determine the load on the system. These series of requests are determined in order to accurately reflect a typical transaction between a customer and the customer server. Once all of the requests and responses for a given script have been executed, then this will constitute a full response having been received, at which time the timer will be stopped, as indicated by the function block 712 and then this response logged in the block 714.

[0039] The program, after the response has been logged, will flow to a decision block 718 to determine if more load requests are to be executed. In the initial set up, the parameters indicated the number of times that a request script must be executed and the response thereto measured. Until the maximum number has been achieved, the program will flow along the "Y" path from decision block 718 back to the input of the function block 706. Once the request script has been executed a predefined number of times, the program will flow to the End block 720. The program will flow from the function block 714 to a decision block 716 to determine if this log information should be transmitted to the log. If so, the program will flow to a function block 717 to transmit the log to the master server. This is a periodic operation which occurs less frequently than the actual time it takes for a request script to be executed. The decision block 716 and the function block 717 both then flow to a decision block 718. [0040] Referring now to FIGURE 8, there is illustrated a flow chart when the master server interfaces with the agent to determine the number of agents and location thereof that are ready-to-test. The flow chart is initiated at a block 802 and then flows to function block 804 to prepare for the session. In the preparation for the session, the parameters are selected for the given load test, i.e., the profile and the configuration parameters associated therewith are pulled up in response to the scheduler determining that the flow transaction is to be initiated. The program then flows to a function block 806 to select the agents that are necessary. As described hereinabove, agents are selected upon availability, applied geographic location, etc. Once a "poll" of agents has been selected from the internal log, then these agents will be polled, as indicated by function block 808. It is noted that the ready-to-test status was stored in conjunction with the agents ID and their IP address. However, it could be that the agent for some reason went offline, or that the agent's IP address had changed. This change of IP address can occur, since a number of the agents may have what are referred to as "dynamic" IP addresses. These are addresses that are assigned to the agent whenever it accesses its IP server, which then assigns it a dynamic address. As long as the agent 118 does not disconnect with the IP server, then this dynamic address should remain the same. As such, it is necessary to determine if the agent is still available. This polling operation will basically address the agent, send information thereto in the form of the session parameters and then update a local poll indicating that this particular agent is participating in the load transaction. Agents that do not respond in the selected ones are then deleted as being non responding, as indicated by function block 810. The program then flows to function block 812 to add new agents and then to a decision block 816 to determine if sufficient new agents have been added. Once complete, the program will flow to a return block 18.

[0041] Referring now to FIGURE 9, there is depicted a diagrammatic view of a single agent node illustrating the concept of virtual agents. Typically, a TCP/IP request requires relatively little CPU time in an agent computer. Therefore, the agent node 118 has running on its CPU a plurality of separate requesting applications 902 that are each operable to conduct a session over the GCN 102. This is indicated by a plurality of output session ports 904. Each of these session ports 904 represents the ability for one of the sessions 902 to send a request to the GCN and uniquely request information from a customer server and then receive from the customer server a response directed from a request application. This is basically the operation wherein an internet browser can be "launched" on a computer a number of times, with each launched browser application constituting a separate addressable entity on the agent node 118 to allow the customer server to return information thereto. In effect, this constitutes a deeper level of dynamic addressing than would be provided at the IP server that is intermediate between the GCN 102 and the node 118.

[0042] Also illustrated in FIGURE 9 is the concept that multiple customer servers could be provided, as indicated by a block 906 and a block 908 labeled Cl and C2. It could be that the request application process is 902 labeled Al and A2 interface with Cl, wherein the remaining request applications 902 interface with the customer 908. Therefore, a particular agent node 118 might constitute a thousand virtual agents which can be configured in any manner. It could be that, during a load testing operation where only a percentage of the capacity of a given agent node 118 is utilized, another load test session could be downloaded to the agent node 118 for testing the loading of another customer on the network, or even the same customer by increasing the loading thereto by increasing the number of virtual users. It is noted that there will be some type of log set up in the master server 110 that will determine how much load testing has been requested and accepted by each agent node 118, which each agent node 118 is uniquely identified on the system. This log in operation will then result in a different remuneration to the overall service provider under the operating business model, depending upon the type of load, etc. that is required of the agent node 118.

[0043] Referring now to FIGURE 10, there is illustrated a diagrammatic view of the initiation operation, wherein a separate customer terminal 1002 is provided that interfaces with the GCN 102. In this operation, the customer terminal 1002 will interface through the GCN 102 with the master 110 to initiate and configure the load test transaction. Additionally, the customer from the customer terminal 1002 will have been provided therewith a display 1004 that will enable the customer terminal 1002 to monitor the load transaction. I/O device 1006, such as a keyboard and a storage device, is provided to allow the customer to interface with the customer terminal 1002. This functionality of the terminal 1002 is completely different than that associated with the customer server 104, which is operable to interface with the customers and complete requests over the network 102.

[0044] Referring now to FIGURE 11, there is illustrated a flow chart depicting the operation at the customer terminal. The program is initiated at a block 1102 and then proceeds to a function block 1104 to run the set up operation. This, as described hereinabove, is where the configuration parameters are set to the master server 110. Once the information is sent to the master server 110, then the program flows to a function block 1106 to determine if results are to be displayed. If not, the program flows to an End block 1108. If the results are to be displayed, the program flows along a "Y" path of function block 1110 to display the results, which results are transmitted thereto over the GCN 106 by the master server 110. The program then flows to a decision block 1112 to determine if this setup is to be modified. There are provisions that allow the user to reset some of the parameters during the load test operation. For example, it may be that a given script is not adequately load testing the customer server 104 , due to such things as an insufficient number of agents accessing the customer server 104, the need for a different script to be downloaded to the agents by a different selection of agents by some type of filter that will filter agents by geographic location, and any other type of parameters that the customer may have access to. If the request script is to be modified, the program will flow along a "Y" path to a function block 1114 to update the configuration and then to the input of a display decision block 1106. to again display results. If the program is not to be modified, the program will flow from the decision block 1112 along the "N" path back to the input of function block 1110 to continue to display results until the program ends.

[0045] Referring now to FIGURE 12, there is illustrated an example of a display that might be provided to the customer on the display 1004. The display is arranged in a plurality of rows and columns. There are provided, for example, four columns of data, Dl, D2, D3 and D4. There is also provided a column for Load and a column for Delay. The rows will be dynamically added in real time with the bottom row continually replaced with new data and the upper rows incremented such that the top row will disappear. The data will appear that is necessary to define the transaction, as defined by the customer, and in association therewith will be a "bar" which will indicate the amount of load, i.e., the number of users that are on line in the form of agents. There will be provided a delay "bar" in the delay column indicating the amount of delay in a graphical manner. For predetermined increments of time, the load will be indicated graphically and the delay will also appear, therefore, the customer can see the effects of delay upon load in a dynamic manner. It should be understood that load can be effected by other things than the customer's server, i.e., it could be caused by factors associated with network traffic. Suppose, for example, that there were a catastrophic change in the stock market that resulted in on-line traders attempting to access all of the trading servers on the network in a short period of time. This would "spike-up" the network traffic which could result in delays that were not due to the operation of the customer server, but, rather, to network traffic. These could be recognized on the display.

[0046] Referring now to FIGURE 13, there is illustrated an alternate display of delay versus load. As the load, i.e., the number of agents, is increased in a predetermined manner, the delay can be measured and graphically displayed. It may be that the profile set up or configured by the customer required that the agents be introduced in an exponential manner. As such, the initial access of the customer server by agents would be relatively small and would increase.

[0047] Referring now to FIGURE 14, there is illustrated an alternate display and method wherein the load is varied in an incremental manner. In the illustration in FIGURE 14, there is illustrated a plot of Delay versus Load. The Load is initially set to a static value, as indicated by a line 1402 and that is increased at a point 1404 to a second level, a level 1406. It can be seen that the delay will increase to a set level. This can again be increased at a time 1408 to a second and higher level which will result in a delay line 1410. These increments can be actually instituted on the part of the customer by inputting additional commands to the master server 110. This can be increased and decreased at the will of the customer server and in response thereto. Alternatively, this can be set up in a profile. Further, the increments can be the result of a different script that is played, resulting in additional delay. [0048] Referring now to FIGURE 15, there is illustrated a display which illustrates the delays as a function of a type of request. It could be that each profile that is run is comprised of multiple different request scripts, there being provided five for the display of FIGURE 15, RQl, RQ2, RQ3, RQ4 and RQ5. It could be that the profile was set up so that there was a certain percentage distribution between the request, where they could be evenly distributed. The request numbers are disposed in one column 1502 with the delay bars disposed in a column 1504 to the right thereof. These are horizontal bars which indicate a level of delay per request. As such, a consumer can have a graphic representation of how a particular request effects the overall delay.

[0049] Referring now to FIGURE 16, there is illustrated a display associated with geographical locations of the agents. The agents can be divided into geographical locations referred to as agent locations "AL," of which there are defined five, AL1, AL2, AL3, AL4 and AL5. The location number is indicated in a geographical location column 1602 with a graphical delay "bar" illustrated to the right thereof in a section 1604. As such, the consumer can gain an immediate graph of a representation of delays as a function of their geographical location.

[0050] Referring now to FIGURE 17, there is illustrated a flow chart depicting the operation of configuring the load test, which is initiated at a block 1702 and then proceeds to a function block 1704 wherein the base number of agents, i.e., the minimum is entered. The program then flows to a function block 1706 to determine the increment value, i.e., the number of agents that are added at each increment of time. Also, the increment time is set. In this configuration, which is by example only, the profile determines that agents will be added on at a certain incremental value. The program then flows to a function block 1708 to set the test duration and then to a function block 1710 to set the maximum level of agents that are to be utilized. The program then flows to a function block 1712 to set the type of increment, i.e., whether it is an exponential increment or a linear increment. The program then flows to a function block 1716 to set the profiles. In this operation, as indicated by function block 1718, a selection is made among a number of different stored profiles, such that a mix of profiles or request scripts can be utilized. Once the profiles have been selected for the overall load transaction, the program flows to function block 1720 in order to schedule the load test and then proceeds to an End block 1722.

[0051] Referring now to FIGURE 18, there is illustrated a flow chart depicting the operation of downloading the parameters to the agent. The program is initiated at a block 1802 and then proceeds to a decision block 1804 to determine if the scheduled time has occurred. Once the schedule time for dispatching has occurred, the program flows to a function block 1806 to select the profiles that are to be downloaded and then these profiles distributed to the group of select agents, as indicated by a function block 1808. This information, i.e., the agents to which the load test task has been assigned, is logged, as indicated by function block 1810. The responses received from the agents are then logged, as indicated by a function block 1812 and then the program flows to a decision block 1814 to determine if the profiles are to be modified, as indicated by instructions received from the customer terminal. If so, the program will flow along a "Y" path to select new profiles and distribute these new profiles to agents. If not modified, the program will flow along the "N" path to a decision block 1816 to determine if the maximum duration has been achieved, i.e., all of the agents have completed their task. If not, the program will flow along the "N" path back to the input of function block 1812 until the entire load test is completed, i.e., the predetermined number of request scripts have been played by the participating agents. Once complete, the program will flow to an End block 1818.

[0052] Referring now to FIGURE 19, there is illustrated a flow chart depicting the operation of creating the profile in the master server 110, which is initiated at a block 1902 and then proceeds to a function block 1904. In function block 1904, the program is initiated, this typically being the situation when a new client comes on board, a client is creating a new profile, etc. The program then flows to a function block 1906 wherein the customer creates a profile name. The program then flows to a function block 1908 to determine if the customer wishes to begin playing the request script. Typically, in order to define what the load test is, the customer will actually send HTTP commands to their browser for a particular transaction. This may require many requests. For example, a customer would access their customer server with a request by directing that request to the URL of the customer server and await a response. When the response is returned, there may be an interactive aspect wherein another HTTP command is returned. This may continue for any number of HTTP commands. Once complete, the customer will indicate such to the master server 110. This is referred to as the "tracker" operation, which is indicated at a function block 1910, wherein the sequence of requests that are in the created script, are indicated by the function block 1912. When the script is complete, the program will flow from a decision block 1914 to a function block 1916 wherein the terminate command will be transmitted by the customer to the master server 110. The master server 110 will then inquire of the customer whether they accept the recorded request script, as indicated by decision block 1918. If not accepted, the program flows to an End block 1920 and, if accepted, the program flows along a "Y" path from decision block 1918 to a function block 1922 to save the script at the master under the named profile. The program will then flow from function block 1922 to the End block 1920.

[0053] Referring now to FIGURE 20, there is illustrated a diagrammatic view of a system wherein agents are categorized as two types of agents, a calibration agent and a normal agent. As described hereinabove, each agent can be comprised of a plurality of "virtual" agents; that is, each agent can be configured to initiate a large number of processes, each process operable to conduct a load test by itself. As such, a large amount of data from a given agent could be output to its broadband network link. There could be a problem with the amount of data that can be handled by the particular "pipe" between the agent and the network. In effect, there is a possibility that a large number of agents could be actually bandwidth limited due to the fact that the agent was forced to conduct a large number of transactions.

[0054] To alleviate the potential inaccuracy due to this, a small number of agents, typically one out often or one out of one hundred are configured as calibration agents. This calibration agent is illustrated as a block 2002 in FIGURE 20. The calibration agent is a distinct agent or computer on the network that has access to the network through a single "pipe" 2004. Although illustrated as a pipe, this is merely an internet connection. This could be facilitated with the interface from the calibration agent 2002 computer to a broadband connection such as DSL, which is an interface through an ISP to the GCN 102. This "pipe" refers to the amount of bandwidth that can be accommodated. The larger the pipe, the more data that can be transferred between the GCN 102 and the computer associated with the device. By only associating one load test process with the calibration agent 2002, it can be virtually insured that there will be no delay due to the connection between the calibration agent 2002 and the GCN 102. A normal agent is illustrated by a block 2006, which has a "pipe" 2008 disposed between the agent 2006 and the GCN 102. There will be a plurality of virtual agents that will access the pipe 2008, as illustrated by a plurality of I/O 2010. There could be as many as 10,000 virtual agents associated with the normal agent block 2006. As such, there could be delays associated with the operation of a normal agent 2006 due to this level of activity. However, the virtual agents still "load" the customer server, even though the delay measurement may be off. Therefore, any delay provided by the loading will be reflected more accurately with the calibration agent 2002, since the calibration agent 2002 will not experience delays as a result of overtaxing of its pipe 2004. Since they are bearing the same load test, the delay for the calibration agent 2002 can be utilized for the delay of the normal agent 2006.

[0055] Referring now to FIGURE 21, there is illustrated a flow chart depicting the operation of utilizing the calibration agent and normal agents. The program is initiated at a block 2102 and then proceeds to a function block 2104 to receive data from the calibration agents and then to a function block 2106 to receive the data from the normal agents. This data is then compared in a block 2108 and then a determination made as to whether there is a difference, as indicated in a decision block 2110. If there is no difference, the program will flow along the "N" path to function block 2112 wherein both the normal and calibration data are utilized in combination and treated equally. However, if there is a difference, then the program will flow along the "Y" path to a function block 2114 wherein a balance will be generated between a normal and a calibration agent data period in one embodiment, the calibration delays will be utilized. In another embodiment, there will be a "blending" of the two values by some predetermined algorithm. After the balanced value has been determined, the program will flow from the function block 2114 to a function block 2116 to log the information, which is also the path that the function block 2112 will take. Once logged, the program will flow back to the input of function block 2104 to continue receiving data on a periodic basis. [0056] Although the preferred embodiment has been described in detail, it should be understood that various changes, substitutions and alterations can be made therein without departing from the spirit and scope of the invention as defined by the appended claims.

Claims

WHAT IS CLAIMED IS:

1. A method for load testing a customer server on a network, comprising the steps of: providing a plurality of agents interfaced with the network that can access the customer server over the network; selecting with a master server on the network a plurality of the agents as a load testing group; activating the plurality of agents in the load testing group with the master server to initiate a load test operation of the customer server for a predetermined amount of time that requires access to the customer server by each of the agents in the load testing group; the agents operable to send the results of the load testing operation thereon to the master server; and storing the results at the master server when received.

2. The method of Claim 1 further comprising the step of retrieving the results of the load test stored at the master server from the master server.

3. The method of Claim 1 , wherein the step of activating the plurality of agents in the load testing group comprises the steps of: downloading predetermined load test parameters to each of the agents; each of the agents operable to utilize the downloaded parameters that define the load testing operation to be performed on the customer server by the receiving one of the agents.

4. The method of Claim 1, wherein the step of selecting with the master server a plurality of agents as a load testing group includes the step of available ones of the plurality of agents notifying the master server that they are available for the load testing operation and then selecting only among the ones of the notifying agents for formation of the load testing group.

5. The method of Claim 4, and further comprising the step of the master server attempting to access each of the ones of the plurality of agents that have indicated availability for the load testing operation to determine if they are available at the time of selection and, if not, removing them from the available status.

6. The method of Claim 1 , wherein the step of sending the results to the master server by the agents occurs on a periodic basis.

7. The method of Claim 6, and further comprising the step of retrieving the results from the master server and displaying the results in a predetermined manner.

8. The method of Claim 7, wherein the display of the results occurs on a real time basis as the data is received on the periodic basis.

9. The method of Claim 1 , wherein in each of the plurality of agents has associated therewith a predetermined load testing profile when operating as a part of the load testing group and further comprising the step of modifying the profile at each of the agents during the load testing operation.

10. The method of Claim 9, wherein the step of modifying is controlled by a location on the network remote from the master server.

11. The method of Claim 1 , wherein each of the agents is interfaced with the network through an agent connection with a defined bandwidth that can introduce a finite delay into the load testing operation as the number of requests to be serviced increases.

12. The method of Claim 11 , wherein each of the agents can comprise a plurality of virtual agents that each: are interfaced with the network through the associated agent connection and can access the customer server over the network through the associated agent connection; can be activated in the load testing group by the master server as a separate and distinct agent to initiate a load test operation of the customer server for a predetermined amount of time through the associated agent connection; are operable to send the results of the load testing operation thereon to the master server over the associated agent connection, such that all of the virtual agents operate independently in the load testing of the associated load testing group.

13. The method of Claim 12, wherein each of the virtual agents associated with an agent and operable to communicate with the network through the associated agent connection can be a pert of separate load testing groups.

14. The method of Claim 12, wherein, for a predetermined number of virtual agents in a given load testing group on a given agent, there is defined and associated therewith a separate agent in the same load testing group as a calibration agent, which calibration agent is limited to the number of virtual agents that can be operated thereat, such that the number of virtual agents operating thereat is less than the number of virtual agents operating at the agent associated with the calibration agent, such the number of accesses to the customer server are less through the associated agent connection for the calibration server as compared to the associated agent connection for the associated agent.

15. The method of Claim 14, wherein the number of virtual agent operating at the calibration agent is one.

16. The method of Claim 14 further comprising the step of retrieving the results of the load test stored at the master server from the master server and analyzing the results with the analysis of a group of virtual agents operating at a given agent weighted toward the results returned by the associated calibration agent.

17. The method of Claim 16, wherein the weighting is the use of the results from the associated calibration agent exclusive of the results of the associated group of virtual agents.

18. A load testing system for load testing a customer server on a network, comprising: a plurality of agents interfaced with the network that can access the customer server over the network; a master server disposed on the network and operable to select a plurality of the agents as a load testing group; said master server operable to activate said plurality of agents in the load testing group to initiate a load test operation of the customer server for a predetermined amount of time that requires access to the customer server by each of said agents in the load testing group; said agents operable to send the results of the load testing operation thereon to said master server; and said master server operable to store the results when received.

19. The load testing system of Claim 18 and further comprising an external customer terminal interfaced with the network for retrieving the results of the load test stored at said master server from said master server.

20. The load testing system of Claim 18, wherein said master server, when activating said plurality of agents in said load testing group is operable to: download predetermined load test parameters to each of said agents; each of said agents operable to utilize said downloaded parameters that define said load testing operation to be performed on the customer server by the receiving one of said agents.

21. The load testing systeh of Claim 18, wherein available ones of said plurality of agents are operable to notify said master server that they are available for said load testing operation and then said master server is operable to select only among the ones of said notifying agents for formation of said load testing group.

22. The load testing system of Claim 21, wherein said master server is further operable to attempt to access each of the ones of said plurality of agents that have indicated availability for said load testing operation to determine if they are available at the time of selection and, if not, removing them from said available status.

23. The load testing system of Claim 18, wherein said agents send the results to said master server on a periodic basis.

24. The load testing system of Claim 23, and further comprising a display for displaying said results from said master server in a predetermined manner.

25. The load testing system of Claim 24, wherein said display displays the results on a real time basis as the data is received on the periodic basis.

26. The load testing system of Claim 18, wherein in each of said plurality of agents has associated therewith a predetermined load testing profile when operating as a part of said load testing group and said master server is further operable to modify the profile at each of said agents during said load testing operation.

27. The load testing system of Claim 26, and further comprising a modifying server for controlling said master server to modify the profile at each of said agents during said load testing operation.

28. The load testing system of Claim 18, wherein each of said agents is interfaced with the network through an agent connection with a defined bandwidth that can introduce a finite delay into said load testing operation as the number of requests to be serviced increases.

29. The load testing system of Claim 28, wherein each of said agents can comprise a plurality of virtual agents that each: are interfaced with the network through said associated agent connection and can access the customer server over the network through said associated agent connection; can be activated in said load testing group by said master server as a separate and distinct agent to initiate a load test operation of the customer server for a predetermined amount of time through said associated agent connection; are operable to send said results of said load testing operation thereon to said master server over said associated agent connection, such that all of said virtual agents operate independently in the load testing of said associated load testing group.

30. The load testing system of Claim 29, wherein each of said virtual agents associated with an agent and operable to communicate with the network through said associated agent connection can be a pert of separate load testing groups.

31. The load testing system of Claim 29, wherein, for a predetermined number of virtual agents in a given load testing group on a given agent, there is defined and associated therewith a separate agent in said same load testing group as a calibration agent, which calibration agent is limited to said number of virtual agents that can be operated thereat, such that said number of virtual agents operating thereat is less than said number of virtual agents operating at said agent associated with said calibration agent, such the number of accesses to the customer server are less through said associated agent connection for said calibration agent as compared to said associated agent connection for associated agent.

32. The load testing system of Claim 31 , wherein said number of virtual agent operating at said calibration agent is one.

33. The load testing system of Claim 31 further comprising an external customer terminal interfaced with the network for retrieving the results of said load test stored at said master server from said master server and analyzing said results with the analysis of a group of virtual agents operating at a given agent weighted toward the results returned by said associated calibration agent.

34. The load testing system of Claim 33, wherein the weighting is the use of the results from said associated calibration agent exclusive of said results of said associated group of virtual agents.