US20160034328A1 - Systems and methods for spatially displaced correlation for detecting value ranges of transient correlation in machine data of enterprise systems - Google Patents
Systems and methods for spatially displaced correlation for detecting value ranges of transient correlation in machine data of enterprise systems Download PDFInfo
- Publication number
- US20160034328A1 US20160034328A1 US14/446,131 US201414446131A US2016034328A1 US 20160034328 A1 US20160034328 A1 US 20160034328A1 US 201414446131 A US201414446131 A US 201414446131A US 2016034328 A1 US2016034328 A1 US 2016034328A1
- Authority
- US
- United States
- Prior art keywords
- data
- data points
- correlation coefficient
- metric
- range
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
- G06F11/0706—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation the processing taking place on a specific hardware platform or in a specific software environment
- G06F11/0748—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation the processing taking place on a specific hardware platform or in a specific software environment in a remote unit communicating with a single-box computer node experiencing an error/fault
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
- G06F11/0751—Error or fault detection not based on redundancy
- G06F11/0754—Error or fault detection not based on redundancy by exceeding limits
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
- G06F11/079—Root cause analysis, i.e. error or fault diagnosis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2201/00—Indexing scheme relating to error detection, to error correction, and to monitoring
- G06F2201/81—Threshold
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Hardware Design (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Debugging And Monitoring (AREA)
Abstract
Aspects of the present disclosure include systems and/or methods for detecting ranges of data that represent transient correlations in machine data corresponding to various hardware and/or software systems, such as enterprise systems employed by an information technology (“IT”) organization. In various aspects, the machine data may comprise one or more operational metrics that represent system performance, usage, and/or business activity of the enterprise system. The operational metrics may be used to identify operational issues within the enterprise system.
Description
- Aspects of the present disclosure relate to systems and methods for analyzing metrics.
- Operations system monitoring represents a critical component of any large scale software system, including enterprise-level information technology systems. Monitoring the operations of such systems enables administrators to perform various diagnostic procedures, such as determining whether the system is functioning properly and automatically initiating various repair procedures when the system is functioning improperly. Monitoring is complex and typically requires the collection of numerous operational metrics, and the continuous aggregation, interpretation, and reporting of the collected operational metric data.
- A major challenge in the design and implementation of such monitoring services is ensuring that the operational metrics being collected accurately identify operational issues within the system. Stated differently, the monitored operational metrics must accurately reflect the behaviors of the system and cannot falsely indicate that the system is behaving improperly when the system is actually behaving as intended. Monitoring services that are too sensitive, static and inflexible, and/or improperly configured, cause such errors. Moreover, monitoring is often a manual process that, particularly for enterprise systems, is prone to error, prone to overlooking of some metrics, suffers from human inability to sometimes process large disparate data sets, and is slow to adapt to changes in topology, configuration, and otherwise.
- It is with these concepts in mind, among others, that aspects of the present disclosure were conceived.
- Aspects of the present disclosure involve systems, methods, and/or non-transitory computer-readable mediums. The methods and/or non-transitory computer-readable mediums include generating a plurality of ranges of data based on a first dataset corresponding to a first metric describing a first performance of a system, wherein each range of data includes a first number of data points. The methods and/or non-transitory computer-readable mediums further include for each range of data of the plurality of ranges of data, identifying a second number of data points from a second dataset corresponding to a second metric describing a second performance of the system, the second number of data points corresponding to the respective range of data and determining a correlation coefficient based on the first number of data points of the respective range of data and the second number of data points corresponding to the respective range of data. The methods and/or non-transitory computer-readable mediums further include for each range of data of the plurality of ranges of data, determining a representative correlation coefficient based on the correlation coefficients. The methods and/or non-transitory computer-readable mediums further include based on the representative correlation coefficient, storing an indication of whether or not the first dataset is correlated to the second dataset on the respective range of data of the representative correlation coefficient.
- The foregoing and other objects, features, and advantages of the present disclosure set forth herein will be apparent from the following description of particular embodiments of those inventive concepts, as illustrated in the accompanying drawings. The drawings depict only typical embodiments of the present disclosure and, therefore, are not to be considered limiting in scope.
-
FIG. 1 is a flowchart illustrating an example process for identifying ranges of data that represent transient correlations, according to aspects of the present disclosure. -
FIGS. 2A-2B are block diagrams illustrating a computing architecture for identifying ranges of data that represent transient correlations, according to aspects of the present disclosure. -
FIG. 3A is a flowchart illustrating a process for determining correlation coefficients, according to aspects of the present disclosure. -
FIG. 3B is an illustration of a range of data, according to aspects of the present disclosure. -
FIG. 4 is an illustration of a computing device, according to aspects of the present disclosure. - Aspects of the present disclosure include systems and/or methods for detecting ranges of data that represent transient correlations in machine data corresponding to various possible monitored computing systems including enterprise systems, hardware systems, networking systems, software systems, and combinations thereof. In various aspects, the machine data may involve one or more operational metrics that represent system performance, usage, and/or business activity of the monitored system. The operational metrics, when correlated according to the embodiments discussed herein, may be used to identify operational issues within the system and may be used to proactively manage the monitored systems. For example, the correlations may be analyzed by administrators or used to automatically configure or alter system performance to ensure that an enterprise system is behaving in accordance with a service-level agreement (SLA) with a customer, for example. As another example, the correlation information may be used to manage resources consumed by the enterprise system so that resources are being optimally used, to proactively identify additional resource needs, to retask additional resources when some resources are becoming over utilized, and the like.
- Identifying correlations between operational metrics is particularly challenging when the correlations are transient, only occur during specific operational conditions, are rare or involve multiple metrics, among other challenges. For example, CPU usage and application response time are two operational metrics of a system that may not typically be correlated. During specific operational conditions, however, there may be instances when CPU usage and application response time do correlate. For example, when a CPU is particularly active (e.g., 80%), the CPU usage may impact application response time and therefore a correlation exists between the metrics above a CPU usage of 80%. Once the CPU usage decreases below the 80% threshold, there is no longer an impact and therefore no correlation between the CPU usage operational metric and the application response time metric. Since the correlation between the operational metrics only exists under certain conditions, the correlation is considered conditional, i.e., a conditional or transient correlation. Aspects of the present disclosure identify the ranges of data that represent such conditional or transient correlations. The ranges of data that represent the transient correlations may be processed or otherwise used by the system to identify non-obvious relationships and/or dependencies that may be causing enterprise-level system issues.
- An illustrative process and/or system for identifying transient or conditional correlation between two or more operational metrics is depicted in FIGS. 1 and 2A-2B. In particular,
FIG. 1 illustrates anexample process 100 for identifying the ranges of data that represent the correlations.FIG. 2A illustrates acomputing architecture 200 including hardware and executable instructions running thereon that may be involved in the execution of theprocess 100 of Fig.FIG. 2B illustrates theenterprise system 204 capable of generating machine data for use in identifying transient correlations between operational metrics, according to one embodiment. Generally speaking, theenterprise system 202 includes both heterogeneous and stand-alone computer systems that run various software components, and/or applications on behalf of an enterprise, such as account payables systems, human resources systems, information technology, accounting systems, inventory management systems, etc. Stated differently, theenterprise system 202 provides various software and/or hardware components that enables an entity, such as large enterprise organization, to manage, integrate and/or coordinate business activities, functionalities, processes, and/or the like. - Referring initially to
FIG. 2A , thecomputing architecture 200 includes aserver 202, which may include one ormore processors 203, alocal memory 207, and astorage 209. Although thestorage 209 is depicted as being a part of theserver 202, it is contemplated that the storage may be located externally or elsewhere in operable communication with theserver 202. For example, thestorage 209 may be a storage appliance. Theserver 202 may interact or otherwise communicate with and/or within anenterprise system 204 capable of generating or otherwise providing machine data to theserver 202. While the correlation methodology is depicted herein as running on a server, and being used in the context of an enterprise system, such an embodiment is only one example of the operating environment and type of system from which metrics may be obtained and correlated, and subsequently used for proactive or remedial actions. - The
server 202 may obtain machine data from theenterprise system 204, identify data corresponding to at least two operational metrics included within the machine data, and process the data corresponding to the operational metrics to identify ranges of data that represent transient correlations between the operational metrics. The identified transient correlations may be processed to determine operational issues that may exist within theenterprise system 204 and any identified operational issues may be provided to a user-device 206, for example, in the form of an interactive display, alert, user-interface, and/or the like, any of which may be accessible by a system administrator. Alternatively, some correlations may drive automatic configuration or topology changes. For example, if CPU usage is causing application performance issues as identified by a transient correlation, additional CPUs may be deployed or tasked to the system of concern. Theserver 202 may communicate with theenterprise system 204 through acommunications network 220, which may be the Internet, an intranet, a local area network, a wireless local network, a wide area network, or another communication network, as well as combinations of networks. The metrics may also be aggregated or otherwise stored in a database, such as one maintained in theappliance 209, and accessed therefrom in real-time or otherwise. - Referring now to
FIG. 2B , in the illustrated embodiment, theenterprise system 204 includes one or more components (e.g., applications, hardware, servers, clients, storage, processes) 212, 214, and 216 that communicate with aruntime environment 113 configured to execute one ormore services more services enterprise system - The
enterprise system 204 may further includemultiple detection modules enterprise systems enterprise system data storage system 218. Additionally, any obtained machine data may be transmitted to theserver 202 for processing to identify ranges of data that represent transient correlations between various metrics, and therefore, could be an indication of operational issues that exist within theenterprise system 204. While thedetection modules enterprise system 204, it is contemplated that they may be located elsewhere and external to theenterprise system 204, such as for example within theserver 202. - Referring now to
FIG. 1 , theprocess 100 for identifying ranges of data that represent transient correlations in machine data is provided, according to one embodiment. As illustrated,process 100 begins with obtaining machine data including one or more operational metrics used in the monitoring of a system (operation 102). In particular, theserver 202 may execute an application, process, thread, instruction and/or the like, to gather or otherwise obtain machine data from theenterprise system 204. Alternatively, the application may access thestorage 209 having the metrics stored therein. Examples of metrics that may be processed by the system include CPU usage, memory usage, application response time, heap usage, network congestion, file system usage and capacity, latency, wait I/O, workload, physical storage performance, and the like. - The machine data obtained from the enterprise system may include data corresponding to multiple operational metrics obtained over a particular period of time, such a minute, hour, day, week, month, or some other temporal delineation. For example and as introduced above, the system may capture data corresponding to a CPU usage metric that describes the amount of time a CPU takes to process a request to execute various instructions during the execution of an application, or service (e.g., the
services enterprise system 202 over a specific period of time. As another example, the machine data may include data corresponding to the application response time metric that quantifies the amount of time required to perform each step in processing a transaction during the execution of an application, or service. Thus, the machine data may include set of data points describing application response time of theservices enterprise system 202 over a specific period of time. While only two example operational metrics are used herein to illustrate the processes and systems of the present disclosure, other operational metrics and corresponding data may be included within the machine data, correlations identified, and various actions taken based thereon. - The various operational metrics included within the machine data obtained by the
server 202 may include operational metrics that typically do not correlate. For example, as explained above, CPU usage and application response time operational metrics generally do not correlate, but a correlation may arise during periods of time (i.e., transient or conditional correlations) when CPU usage is so high (e.g., above a threshold of 80%-85%) that the CPU usage starts to impact application response time. Once the CPU usage decreases below a certain threshold, there is no longer an impact on application performance, and therefore, no longer any correlation between the CPU usage metric and the application response time metric. Identifying these sometimes brief and transient correlations that occur between otherwise uncorrelated operational metrics, enables users, such as administrators, to engage in more accurate and pro-active enterprise system management, allows automatic configurations to be set, provides alerts, and the like. - To identify such correlations, the machine data is processed to identify any ranges of data that represent a statistically significant and transient correlation between one or more metrics (i.e., the first metric and the second metric) (operation 104). In one embodiment, the
server 202 processes the machine data to identify a primary operational metric and a secondary operational metric with which the primary operational metric may be correlated. Generally speaking, a primary metric is a metric to which other secondary metrics may be compared to determine whether any correlations exists between the primary metric and the secondary metric. Machine data corresponding to the primary metric provides the baseline line data and to which data from the secondary metric may be compared. Moreover, the primary metric is used to establish thresholds; exceeding a threshold on the primary metric causes an effect on the secondary metric. Theserver 202 partitions data corresponding to the primary operational metric into a series of data ranges referred to herein as “bins.” Each bin represents a range of data points quantifying the primary operational metric over a specific period of time, as will be explained in more detail below. Theserver 202 then matches data corresponding to the secondary operational metric to the bins of the primary operational metric. A correlation coefficient is calculated for each bin to determine which bin, if any, may be identifying a transient correlation between the primary operational metric and the secondary operational metric and to determine which bin is the most statistically significant. -
FIGS. 3A-3B provide aflowchart 360 for identifying statistically significant transient correlations between a primary operational metric and a secondary operational metric and a corresponding scatter plot of machine data, for the primary operational metric, quantifying discrete points over a period of time and binned for correlation to a secondary metric. The illustrated embodiments will be described in the context of a Java™ Virtual Machine (“JVM”) performing garbage collection on a heap memory during runtime of a Java™ application. - A JVM allocates a heap memory and subsequently manages the heap for an executing Java™ application. When the Java™ application creates a new data object, the JVM sub-allocates an area of the heap memory to store the object. An object in the heap is maintained as long as it is referenced by other objects. Objects that are no longer referenced, however, are considered to be garbage and are therefore cleared out of the heap, using a garbage collection process, to reclaim the memory that the unreferenced objects occupy.
- The size of the heap may have an impact on garbage collection process invocation times, which ultimately may impact the performance of the executing Java™ application. For example, a large heap typically introduces a slight overhead in garbage collection invocation times. Additionally, a heap that is larger than the available physical memory in the system must be paged out to disk, which leads to long access times, especially during garbage collection. To determine whether any transient correlations exist between heap usage and garbage collection invocation times, the heap usage is set as the primary operational metric and garbage collection-invocation time is set as the secondary operational metric. If there is a correlation between the primary metric and the secondary metric, when the primary metric exceeds some threshold, then the primary metric is impacting performance of the secondary metric in the particular conditions associated with the exceeded threshold.
- Referring now to
FIG. 3A , to start at 302, machine data corresponding to a primary operational metric is randomly partitioned or otherwise divided into one or more bins. The number of initially generated bins, may be user-defined, and may include at least two bins, although many more bins may be generated. Each bin encompasses a range of values between an upper value and a lower value.FIG. 3B provides agraphical illustration 320 of one ormore bins axis 340 and a corresponding time indication and/or stamp identifying when a particular discrete data value of the primary operational metric was generated is provided along thex-axis 342. Thus, each discrete data point within a given bin may correspond to a specific timestamp or other temporal indication identifying the specific time that particular discrete data point for the primary operational metric was recorded or otherwise obtained during monitoring of theenterprise system 204. - Each
bin bin 322 has a maximum edge value of 1 (e.g., 100% heap usage) at 344 and a minimum edge value of 0.8 (e.g., 80% heap usage) at 346.Bin 324 has a maximum edge value of 0.8 at 346 and a minimum edge value of 0.6 348.Bin 326 has a maximum edge value of 0.6 at 348 and a minimum edge value of 0.4 at 350.Bin 328 has a maximum edge value of 0.4 at 350 and a minimum edge value of 0.2 at 352.Bin 330 has a maximum edge value of 0.2 at 352 and a minimum edge value of 0.0 at 354. - Referring again to
FIG. 3A , once one or more bins have been generated and populated with the primary operational metric, data points from machine data corresponding to the secondary operational metric (e.g., garbage collection invocation time) are identified and grouped according to the bins generated for the primary operational metric. In one embodiment, grouping is based on the time stamps of the data points within each bin. Referring again toFIG. 3B ,bin 324 includes eight data points, each data point having a specific value quantifying the primary operational metric and a specific timestamp indicating when the data point was recorded. For example, in the illustrated embodiment,data points bin 324 and correspond to a timestamp of 21 seconds and 27 seconds. The illustrated embodiment ofbin 324 includes eight data points for simplicity reasons only. It is contemplated that a bin may contain any number of data points including hundreds, thousands, and the like. - Referring again to
FIG. 3A , at 306, once the bins have been generated for the primary operational metric and data corresponding to each bin for the secondary operational metric has been identified, all of the data is used to calculate a correlation coefficient for each bin to determine whether any correlations exists between the primary operational metric and the secondary operational metric. Generally speaking, a correlation coefficient is a value or measure that determines the degree to which two variables movements are associated and may be calculated using various correlation coefficient methodologies, such as the Spearman correlation approach or the Pearson correlation approach. Thus, in the context ofFIG. 3A and 3B , any calculated correlation coefficient quantifies the degree to which the primary operational metric (e.g., heap usage) and the secondary operational metric (e.g., garbage collection invocation-time) may be associated. Referring toFIG. 3B , correlation coefficients may be calculated for each bin 322, 324, 326, 328, and 330. In particular, correlation coefficients may be calculated forbin 322 based on the data included in thebin 322 corresponding to the primary metric and the corresponding data of the secondary metric. Correlation coefficients may be calculated forbin 324 based on the data included in thebin 324 corresponding to the primary metric and the corresponding data of the secondary metric. Correlation coefficients may be calculated forbin 326 based on the data included in thebin 326 corresponding to the primary metric and the corresponding data of the secondary metric, and so on, until a series of correlation coefficients has been generated for each of thebins - At 308 (same as
operation 106 ofFIG. 1 ), the calculated correlation coefficients for all bins are processed to identify the maximum coefficient that represents the most significant correlation between the primary operational metric and the second operational metric. Stated differently, the maximum correlation coefficient represents the value that most accurately quantifies the degree to which the data point value movement of the primary operational metric and the secondary operational metric are associated. In one embodiment, theserver 202 may perform a sum of squares of correlation coefficients calculation to identify the maximum correlation coefficient using the following steps: -
- 1.0 Consider the function F: given three (3) bin Edges (Ek, Ek+1, Ek+3) move middle edge to maximize sum of squares of correlation coefficients for two given adjacent bins
- 2.0 Compute sum of squares of correlation of coefficients of all of the bins=S
- 3.0→While S is changing:
- 3.1→Randomly select Ek and run F (Ek, Ek+1, Ek+3)
- 3.2→Estimate a new value for Snew
- 3.3→If Snew=S then S=Snew, and go to step 3.0, else go to 4.
- 4.0→Estimate the computation time of step the process of 1.0
- An example sum of squares of correlation coefficients calculation will now be provided, according to one embodiment. The sum of squares process may be performed for one or more of the
bins FIG. 3B . Assume a correlation coefficient has been calculated for thebin 324. In order to calculate the maximum correlation coefficient, theserver 202 may randomly increase themaximum edge 346 defining the upper boundary for thebin 324 or decrease theminimum edge 348 defining the lower boundary for thebin 324. In either case, the edge is modified to include new data points corresponding to the primary operational metric. Including new data points in the bin effectively generates a new data range from which a new correlation coefficient may be calculated and used within the sum of squares of correlation coefficients calculation to determine the maximum correlation coefficient. For example, theminimum edge 348 of thebin 324 may be decreased to include three of the data points originally included inbin 326. Alternatively, themaximum edge 346 of thebin 324 may be increased to include some of the data points originally included inbin 322. - The
server 202 will then identify the data points of the second operational metric that group, based on timestamp, with the three new data points included in thebin 326. Thus, if theminimum edge 348 of thebin 324 is decreased to include three of the data points originally included inbin 326, three corresponding data points of the second operational metric may be identified, according to the timestamps of the three data points originally included inbin 326 that are now included inbin 324. - A new correlation coefficient will be calculated based on the new data points included within the
bin 324 and the new data points of the second operational metric that correspond to the new data included within thebin 324. The new correlation coefficient will be used, in addition to the first correlation coefficient, in the sum of squares of correlation coefficients calculation to determine a new value for the sum of squares of correlation coefficients. If the new calculated sum of squares of correlation coefficients is greater than the previous calculation of the sum of squares of correlation coefficients, the new value is considered the maximum. Otherwise, the sum of squares of correlation of coefficients process repeats. Thus, theminimum edge 348 of thebin 324 may again be decreased to include new data points in thebin 324. A new correlation coefficient will be calculated for thebin 324 based on the new data points included within thebin 324 and any new data points of the second operational metric that correspond to the new data points included within thebin 324. The process will continue until the new sum of squares of correlation coefficient value is greater than the previously calculated value, at which point, the previously calculated value for the sum of squares of correlation coefficients is identified as the maximum. - Adjusting the edge of one bin necessarily adjusts an edge of another bin. In the example above, adjusting the minimum edge of
bin 324 adjusts the maximum edge ofbin 326. Thus, when the sum of squares of correlation coefficient calculation coefficient is calculated forbin 326, the calculation will be performed using the data values included within the newly adjusted edges of thebin 326. - Referring back to
FIG. 1 , the calculated maximum correlation coefficients may be used to determine whether or not the generated bins identify a statistically significant correlation between the primary operational metric and the second operational metric (operation 108). Specifically, theserver 202 may calculate or otherwise identify p-values corresponding to the maximum correlation coefficient of each bin, to determine whether it satisfies a certain level of statistical significance. More specifically, the p-values may be used in a hypothesis testing statistical analysis in which the p-values statistically prove the correlation between the primary operational metric and the second operational metric. For example, when the p-value for the maximum correlation coefficient of a given bin is less than 0.025, the correlation coefficient may be considered statistically significant and therefore accepted. If there are no bins for which the correlation coefficient can be accepted, then there is no acceptable correlation between the two operational metrics (i.e., the primary operational metric and the secondary operational metric). - An acceptable maximum correlation coefficient may be used to determine whether there is a high correlation between the two operational metrics within the value range of the bin (operation 110). In one embodiment, the
server 202 may analyze the accepted maximum correlation coefficient to determine whether it proves the hypothesis of correlation, as is generally understood in the art. For example, when the maximum correlation coefficient is greater than 0.05, then there is a high correlation between the two operational metrics in the value range of the bin for which the maximum correlation coefficient was accepted. - When the maximum correlation coefficient for a bin is determined to have a high correlation, an indication of the correlation may be generated and stored (operation 112). More specifically, the
server 202 may generate a notification, which may be sent from theserver 202 to the client device 206 (e.g., an administrator user at a desktop computer) indicating the existence of high correlations between various operational metrics in the value range bin, which represents a specific period of time, or transient correlation. - For example and referring to the heap usage and garbage collection example referenced with
FIGS. 3A and 3B , the following table provides example results of the identified bins and their corresponding maximum correlation coefficients (in the provided table, both the maximum correlation coefficient value for both Pearson and Spearman is provided). -
TABLE 1 Correlations between Heap Usage and Garbage Collection - Invocation Time Bin - Bin - Pearson Pearson Spearman Spearman Min Max Coefficient p-value Coefficient p- value Bin 1 0.0 .2 0.170 0.191 0.01 0.940 Bin 2 .2 .4 −0.290 0.021 −0.205 0.106 Bin 3 .4 .6 −0.340 0.006 −0.525 7.232 Bin 4 .6 .8 −0.075 0.562 −0.085 0.511 Bin 5.8 1 0.547 0.024 0.031 0.806 - Based on the results outlined in table 1 above,
bin 5 has a p-value of less than 0.025 (the threshold to accept a correlation coefficient) for Pearson correlation and hence the Pearson correlation coefficient can be accepted.Bin 5 also illustrates that there is a high correlation between heap space usage and garbage collection invocation time since the Pearson correlation is greater than 0.5, according to one embodiment. Hence, there is a transient, or conditional, correlation between the two metrics when the heap usage is between 80% and 100%. At other values ranges, there is no correlation. Since there is no general correlation between the two metrics and a correlation only inbin 5, the system will generate a notification identifying a transient correlation between the two metrics—heap usage and garbage collection invocation times—when the heap usage is between 80% and 100%. Here, a correlation is identified between the primary metric (heap space) and the secondary metric (garbage collection invocation or other), such that when the primary metric exceeds a threshold it has a performance impact on the secondary metric. - Thus, aspects of the present disclosure enable users to discover ranges of data that represent transient correlations between operational metrics used during system monitoring. Identifying such correlations allows users to more accurately monitor and manage system resource usage and perform more pro-active service-level agreement management, among others.
-
FIG. 4 illustrates an example of acomputing node 400 which may comprise an implementation of theserver 202. Thecomputing node 400 represents one example of a suitable computing device and is not intended to suggest any limitation as to the scope of use or functionality of embodiments of the invention described herein. Regardless, thecomputing node 400 is capable of being implemented and/or performing any of the functionality described above. - As illustrated, the
computer node 400 includes a computer system/server 402, which is operational with numerous other general purpose or special purpose computing system environments or configurations. Examples of well-known computing systems, environments, and/or configurations that may be suitable for use with computer system/server 402 may include personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments that include any of the above systems or devices, and the like. - Computer system/
server 402 may be described in the general context of computer system executable instructions, such as program modules, being executed by a computer system. Generally, program modules may include routines, programs, objects, components, logic, data structures, and so on that perform particular tasks or implement particular abstract data types. Computer system/server 402 may be practiced in distributed cloud computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed cloud computing environment, program modules may be located in both local and remote computer system storage media including memory storage devices. - As shown in
FIG. 4 , computer system/server 402 incomputing node 400 is shown in the form of a general-purpose computing device. The components of computer system/server 402 may include one or more processors orprocessing units 404, asystem memory 406, and abus 408 that couples various system components includingsystem memory 406 toprocessor 404. -
Bus 408 represents one or more of any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus using any of a variety of bus architectures. Such architectures may include Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnects (PCI) bus. - Computer system/
server 402 typically includes a variety of computer system readable media. Such media may be any available media that is accessible by computer system/server 402, and it includes both volatile and non-volatile media, removable and non-removable media. -
System memory 406 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 410 and/orcache memory 412. Computer system/server 402 may further include other removable/non-removable, volatile/non-volatile computer system storage media. By way of example only,storage system 413 can be provided for reading from and writing to a non-removable, non-volatile magnetic media (not shown and typically called a “hard drive”). Although not shown, a magnetic disk drive for reading from and writing to a removable, non-volatile magnetic disk (e.g., a “floppy disk”), and an optical disk drive for reading from or writing to a removable, non-volatile optical disk such as a CD-ROM, DVD-ROM or other optical media can be provided. In such instances, each can be connected tobus 408 by one or more data media interfaces. As will be further depicted and described below,memory 406 may include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the invention. - Program/
utility 414, having a set (at least one) ofprogram modules 416, may be stored inmemory 406, as well as an operating system, one or more application programs, other program modules, and program data. Each of the operating system, one or more application programs, other program modules, and program data or some combination thereof, may include an implementation of a networking environment.Program modules 416 generally carry out the functions and/or methodologies of embodiments of the invention as described herein. - Computer system/
server 402 may also communicate with one or moreexternal devices 418 such as a keyboard, a pointing device, adisplay 420, etc.; one or more devices that enable a user to interact with computer system/server 402; and/or any devices (e.g., network card, modem, etc.) that enable computer system/server 402 to communicate with one or more other computing devices. Such communication can occur via Input/Output (I/O) interfaces 422. Still yet, computer system/server 402 can communicate with one or more networks such as a local area network (LAN), a general wide area network (WAN), and/or a public network (e.g., the Internet) vianetwork adapter 424. As depicted,network adapter 424 communicates with the other components of computer system/server 402 viabus 408. It should be understood that although not shown, other hardware and/or software components could be used in conjunction with computer system/server 402. Examples, include, but are not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc. - The embodiments of the present disclosure described herein are implemented as logical steps in one or more computer systems. The logical operations of the present disclosure are implemented (1) as a sequence of processor-implemented steps executing in one or more computer systems and (2) as interconnected machine or circuit engines within one or more computer systems. The implementation is a matter of choice, dependent on the performance requirements of the computer system implementing aspects of the present disclosure. Accordingly, the logical operations making up the embodiments of the disclosure described herein are referred to variously as operations, steps, objects, or engines. Furthermore, it should be understood that logical operations may be performed in any order, unless explicitly claimed otherwise or a specific order is inherently necessitated by the claim language.
- The foregoing merely illustrates the principles of the disclosure. Various modifications and alterations to the described embodiments will be apparent to those skilled in the art in view of the teachings herein. It will thus be appreciated that those skilled in the art will be able to devise numerous systems, arrangements and methods which, although not explicitly shown or described herein, embody the principles of the disclosure and are thus within the spirit and scope of the present disclosure. From the above description and drawings, it will be understood by those of ordinary skill in the art that the particular embodiments shown and described are for purposes of illustrations only and are not intended to limit the scope of the present disclosure. References to details of particular embodiments are not intended to limit the scope of the disclosure.
Claims (22)
1. A method comprising:
generating, by one or more computing devices, a plurality of ranges of data based on a first dataset corresponding to a first metric describing a first performance of a system, wherein each range of data includes a first number of data points; and
for each range of data of the plurality of ranges of data:
identify a second number of data points from a second dataset corresponding to a second metric describing a second performance of the system, the second number of data points corresponding to the respective range of data; and
determine a correlation coefficient based on the first number of data points of the respective range of data and the second number of data points corresponding to the respective range of data;
determine a representative correlation coefficient based on the correlation coefficient; and
based on the representative correlation coefficient, storing an indication of whether or not the first dataset is correlated to the second dataset on the respective range of data of the representative correlation coefficient.
2. The method of claim 1 , wherein each range of data includes a first edge defining a low boundary for the first number of data points and a second edge defining a high boundary for the first number of data points and wherein determining the representative correlation coefficient comprises:
increasing the first edge of the first number of data points to include a third number of data points within the respective range of data;
increasing the second number of data points from the second dataset to generate a fourth number of data points;
determining a second correlation coefficient based on the third number of data points and the fourth number of data points;
determining a first value, the first value being a sum of squares of correlation coefficients, based on the first correlation coefficient and the second correlation coefficient; and
when the first value is greater than the representative correlation coefficient, moving the first edge of the data points to include a fifth number of data points within the respective range of data.
3. The method of claim 2 , further comprising:
when the first value is less than the representative correlation coefficient, identifying the first value as a new representative correlation coefficient.
4. The method of claim 1 , wherein each range of data includes a first edge defining a low boundary for the number of data points and a second edge defining a high boundary for the number of data points and wherein determining the representative correlation coefficient comprises:
decreasing the second edge of the first number of data points to include a third number of data point within the respective range of data;
decreasing the second number of data points from the second dataset to generate a fourth number of data points;
determining a second correlation coefficient based on the third number of data points and the fourth number of data points;
determining a first value, the first value being a sum of squares of correlation coefficients, based on the first correlation coefficient and the second correlation coefficient; and
when the first value is greater than the representative correlation coefficient, moving the second edge of the data points to include a fifth number of data points within the respective range of data.
5. The method of claim 4 , further comprising:
when the first value is less than the representative correlation coefficient, identifying the first value as a new representative correlation coefficient.
6. The method of claim 1 , wherein each range of data of the plurality of ranges of data corresponds to a time interval during which the first number of data points were recorded and wherein the second number of data points corresponds to the respective range of data based on the time interval.
7. The method of claim 1 , wherein the correlation coefficient is calculated using at least one of a Pearson or Spearman correlation algorithm and wherein the representative correlation coefficient is a maximum correlation coefficient of the correlation coefficients.
8. The method of claim 1 , wherein the number of data points included within each range of data is randomly generated and includes at least enough data points to perform a statistical analysis.
9. The method of claim 1 , further comprising:
based on the representative correlation coefficient of each discrete range of data, identifying a specific range of data from the plurality of ranges of data that identifies an acceptable correlation between the first metric and the second metric.
10. The method of claim 9 , further comprising:
determining a p-value corresponding to the range of data; and
when the p-value proves a hypothesis of correlation between the first metric and the second metric, generating the indication.
11. The method of claim 1 , further comprising:
based on the representative correlation coefficient of each discrete range of data, identifying specific ranges of data from the plurality of ranges of data that identifies an acceptable correlation between the first metric and the second metric; and
determining a plurality of p-values corresponding to the specific ranges of data; and
when at least one of the plurality of p-values proves a hypothesis of correlation between the first metric and the second metric, generating the indication.
12. A non-transitory computer readable medium encoded with instructions, the instructions executable by a processor, comprising:
generating a plurality of ranges of data based on a first dataset corresponding to a first metric describing a first performance of a system, wherein each range of data includes a first number of data points; and
for each range of data of the plurality of ranges of data:
identifying a second number of data points from a second dataset corresponding to a second metric describing a second performance of the system, the second number of data points corresponding to the respective range of data;
determining a correlation coefficient based on the first number of data points of the respective range of data and the second number of data points corresponding to the respective range of data; and
determining representative correlation coefficient based on the correlation coefficient; and
based on the representative correlation coefficient, storing an indication of whether or not the first dataset is correlated to the second dataset on the respective range of data of the representative correlation coefficient.
13. The non-transitory computer readable medium of claim 12 , wherein each range of data includes a first edge defining a low boundary for the first number of data points and a second edge defining a high boundary for the first number of data points and wherein determining the representative correlation coefficient comprises:
increasing the first edge of the first number of data points to include a third number of data points within the respective range of data;
increasing the second number of data points from the second dataset to generate a fourth number of data points;
determining a second correlation coefficient based on the third number of data points and the fourth number of data points;
determining a first value, the first value being a sum of squares of correlation coefficients, based on the first correlation coefficient and the second correlation coefficient; and
when the first value is greater than the representative correlation coefficient, moving the first edge of the data points to include a fifth number of data points within the respective range of data.
14. The non-transitory computer readable medium of claim 12 , further comprising:
when the first value is less than the representative correlation coefficient, identifying the first value as a new representative correlation coefficient.
15. The non-transitory computer readable medium of claim 12 , wherein each range of data includes a first edge defining a low boundary for the number of data points and a second edge defining a high boundary for the number of data points and wherein determining the representative correlation coefficient comprises:
decreasing the second edge of the first number of data points to include a third number of data point within the respective range of data;
decreasing the second number of data points from the second dataset to generate a fourth number of data points;
determining a second correlation coefficient based on the third number of data points and the fourth number of data points;
determining a first value, the first value being a sum of squares of correlation coefficients, based on the first correlation coefficient and the second correlation coefficient; and
when the first value is greater than representative correlation coefficient, moving the second edge of the data points to include a fifth number of data points within the respective range of data.
16. The non-transitory computer readable medium of claim 15 , further comprising:
when the first value is less than the representative correlation coefficient, identifying the first value as a new representative correlation coefficient.
17. The non-transitory computer readable medium of claim 12 , wherein each range of data of the plurality of ranges of data corresponds to a time interval during which the first number of data points were recorded and wherein the second number of data points corresponds to the respective range of data based on the time interval.
18. The non-transitory computer readable medium of claim 12 , wherein the correlation coefficient is calculated using at least one of a Pearson or Spearman correlation algorithm and wherein the representative correlation coefficient is a representative correlation coefficient of the correlation coefficients.
19. The non-transitory computer readable medium of claim 12 , wherein the number of data points included within each range of data is randomly generated and includes at least enough data points to perform a statistical analysis.
20. The non-transitory computer readable medium of claim 12 , further comprising:
based on the representative correlation coefficient of each discrete range of data, identifying a specific range of data from the plurality of ranges of data that identifies an acceptable correlation between the first metric and the second metric.
21. The non-transitory computer readable medium of claim 20 , further comprising:
determining a p-value corresponding to the range of data; and
when the p-value proves a hypothesis of correlation between the first metric and the second metric, generating the indication.
22. The non-transitory computer readable medium of claim 12 , further comprising:
based on the representative correlation coefficient of each discrete range of data, identifying specific ranges of data from the plurality of ranges of data that identifies an acceptable correlation between the first metric and the second metric; and
determining a plurality of p-values corresponding to the specific ranges of data; and
when at least one of the plurality of p-values proves a hypothesis of correlation between the first metric and the second metric, generating the indication.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US14/446,131 US9658910B2 (en) | 2014-07-29 | 2014-07-29 | Systems and methods for spatially displaced correlation for detecting value ranges of transient correlation in machine data of enterprise systems |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US14/446,131 US9658910B2 (en) | 2014-07-29 | 2014-07-29 | Systems and methods for spatially displaced correlation for detecting value ranges of transient correlation in machine data of enterprise systems |
Publications (2)
Publication Number | Publication Date |
---|---|
US20160034328A1 true US20160034328A1 (en) | 2016-02-04 |
US9658910B2 US9658910B2 (en) | 2017-05-23 |
Family
ID=55180133
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/446,131 Active 2035-01-14 US9658910B2 (en) | 2014-07-29 | 2014-07-29 | Systems and methods for spatially displaced correlation for detecting value ranges of transient correlation in machine data of enterprise systems |
Country Status (1)
Country | Link |
---|---|
US (1) | US9658910B2 (en) |
Cited By (19)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20160042289A1 (en) * | 2014-08-05 | 2016-02-11 | Oracle International Corporation | Systems and methods for adaptive thresholding using maximum concentration intervals |
US10621005B2 (en) | 2017-08-31 | 2020-04-14 | Oracle International Corporation | Systems and methods for providing zero down time and scalability in orchestration cloud services |
US10635563B2 (en) | 2016-08-04 | 2020-04-28 | Oracle International Corporation | Unsupervised method for baselining and anomaly detection in time-series data for enterprise systems |
US10692255B2 (en) | 2016-02-29 | 2020-06-23 | Oracle International Corporation | Method for creating period profile for time-series data with recurrent patterns |
US10699211B2 (en) | 2016-02-29 | 2020-06-30 | Oracle International Corporation | Supervised method for classifying seasonal patterns |
US10817803B2 (en) | 2017-06-02 | 2020-10-27 | Oracle International Corporation | Data driven methods and systems for what if analysis |
US10855548B2 (en) | 2019-02-15 | 2020-12-01 | Oracle International Corporation | Systems and methods for automatically detecting, summarizing, and responding to anomalies |
US10885461B2 (en) | 2016-02-29 | 2021-01-05 | Oracle International Corporation | Unsupervised method for classifying seasonal patterns |
US10915830B2 (en) | 2017-02-24 | 2021-02-09 | Oracle International Corporation | Multiscale method for predictive alerting |
US10949436B2 (en) | 2017-02-24 | 2021-03-16 | Oracle International Corporation | Optimization for scalable analytics using time series models |
US10963346B2 (en) | 2018-06-05 | 2021-03-30 | Oracle International Corporation | Scalable methods and systems for approximating statistical distributions |
US10970186B2 (en) | 2016-05-16 | 2021-04-06 | Oracle International Corporation | Correlation-based analytic for time-series data |
US10997517B2 (en) | 2018-06-05 | 2021-05-04 | Oracle International Corporation | Methods and systems for aggregating distribution approximations |
US11082439B2 (en) | 2016-08-04 | 2021-08-03 | Oracle International Corporation | Unsupervised method for baselining and anomaly detection in time-series data for enterprise systems |
US11138090B2 (en) | 2018-10-23 | 2021-10-05 | Oracle International Corporation | Systems and methods for forecasting time series with variable seasonality |
US11232133B2 (en) | 2016-02-29 | 2022-01-25 | Oracle International Corporation | System for detecting and characterizing seasons |
US11533326B2 (en) | 2019-05-01 | 2022-12-20 | Oracle International Corporation | Systems and methods for multivariate anomaly detection in software monitoring |
US11537940B2 (en) | 2019-05-13 | 2022-12-27 | Oracle International Corporation | Systems and methods for unsupervised anomaly detection using non-parametric tolerance intervals over a sliding window of t-digests |
US11887015B2 (en) | 2019-09-13 | 2024-01-30 | Oracle International Corporation | Automatically-generated labels for time series data and numerical lists to use in analytic and machine learning systems |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11086750B2 (en) * | 2016-04-07 | 2021-08-10 | University Of Maryland, College Park | Systems and methods for determination of health indicators using rank correlation analysis |
KR102440335B1 (en) * | 2016-10-28 | 2022-09-02 | 삼성에스디에스 주식회사 | A method and apparatus for detecting and managing a fault |
US11392786B2 (en) * | 2018-10-23 | 2022-07-19 | Oracle International Corporation | Automated analytic resampling process for optimally synchronizing time-series signals |
Citations (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20030110007A1 (en) * | 2001-07-03 | 2003-06-12 | Altaworks Corporation | System and method for monitoring performance metrics |
US6898556B2 (en) * | 2001-08-06 | 2005-05-24 | Mercury Interactive Corporation | Software system and methods for analyzing the performance of a server |
US7386417B1 (en) * | 2004-09-29 | 2008-06-10 | Sun Microsystems, Inc. | Method and apparatus for clustering telemetry signals to facilitate computer system monitoring |
US7391835B1 (en) * | 2004-09-29 | 2008-06-24 | Sun Microsystems, Inc. | Optimizing synchronization between monitored computer system signals |
US20100050023A1 (en) * | 2005-07-29 | 2010-02-25 | Bmc Software, Inc. | System, method and computer program product for optimized root cause analysis |
US20100287416A1 (en) * | 2009-03-17 | 2010-11-11 | Correlsense Ltd | Method and apparatus for event diagnosis in a computerized system |
US20120054331A1 (en) * | 2010-08-27 | 2012-03-01 | Assaf Dagan | Correlation of metrics monitored from a virtual environment |
US20120254414A1 (en) * | 2011-03-30 | 2012-10-04 | Bmc Software, Inc. | Use of metrics selected based on lag correlation to provide leading indicators of service performance degradation |
US20140281739A1 (en) * | 2013-03-14 | 2014-09-18 | Netflix, Inc. | Critical systems inspector |
-
2014
- 2014-07-29 US US14/446,131 patent/US9658910B2/en active Active
Patent Citations (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20030110007A1 (en) * | 2001-07-03 | 2003-06-12 | Altaworks Corporation | System and method for monitoring performance metrics |
US6898556B2 (en) * | 2001-08-06 | 2005-05-24 | Mercury Interactive Corporation | Software system and methods for analyzing the performance of a server |
US7386417B1 (en) * | 2004-09-29 | 2008-06-10 | Sun Microsystems, Inc. | Method and apparatus for clustering telemetry signals to facilitate computer system monitoring |
US7391835B1 (en) * | 2004-09-29 | 2008-06-24 | Sun Microsystems, Inc. | Optimizing synchronization between monitored computer system signals |
US20100050023A1 (en) * | 2005-07-29 | 2010-02-25 | Bmc Software, Inc. | System, method and computer program product for optimized root cause analysis |
US20100287416A1 (en) * | 2009-03-17 | 2010-11-11 | Correlsense Ltd | Method and apparatus for event diagnosis in a computerized system |
US20120054331A1 (en) * | 2010-08-27 | 2012-03-01 | Assaf Dagan | Correlation of metrics monitored from a virtual environment |
US20120254414A1 (en) * | 2011-03-30 | 2012-10-04 | Bmc Software, Inc. | Use of metrics selected based on lag correlation to provide leading indicators of service performance degradation |
US20140281739A1 (en) * | 2013-03-14 | 2014-09-18 | Netflix, Inc. | Critical systems inspector |
Non-Patent Citations (1)
Title |
---|
Fenton et al., Risk Assessment and Decision Analysis with Bayesian Networks, 2012, CRC Press, Chapter 1, pages 12-13. * |
Cited By (28)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20160042289A1 (en) * | 2014-08-05 | 2016-02-11 | Oracle International Corporation | Systems and methods for adaptive thresholding using maximum concentration intervals |
US10069900B2 (en) * | 2014-08-05 | 2018-09-04 | Oracle International Corporation | Systems and methods for adaptive thresholding using maximum concentration intervals |
US11232133B2 (en) | 2016-02-29 | 2022-01-25 | Oracle International Corporation | System for detecting and characterizing seasons |
US10867421B2 (en) | 2016-02-29 | 2020-12-15 | Oracle International Corporation | Seasonal aware method for forecasting and capacity planning |
US10692255B2 (en) | 2016-02-29 | 2020-06-23 | Oracle International Corporation | Method for creating period profile for time-series data with recurrent patterns |
US10970891B2 (en) | 2016-02-29 | 2021-04-06 | Oracle International Corporation | Systems and methods for detecting and accommodating state changes in modelling |
US10699211B2 (en) | 2016-02-29 | 2020-06-30 | Oracle International Corporation | Supervised method for classifying seasonal patterns |
US11113852B2 (en) | 2016-02-29 | 2021-09-07 | Oracle International Corporation | Systems and methods for trending patterns within time-series data |
US11836162B2 (en) | 2016-02-29 | 2023-12-05 | Oracle International Corporation | Unsupervised method for classifying seasonal patterns |
US11080906B2 (en) | 2016-02-29 | 2021-08-03 | Oracle International Corporation | Method for creating period profile for time-series data with recurrent patterns |
US10885461B2 (en) | 2016-02-29 | 2021-01-05 | Oracle International Corporation | Unsupervised method for classifying seasonal patterns |
US11670020B2 (en) | 2016-02-29 | 2023-06-06 | Oracle International Corporation | Seasonal aware method for forecasting and capacity planning |
US10970186B2 (en) | 2016-05-16 | 2021-04-06 | Oracle International Corporation | Correlation-based analytic for time-series data |
US10635563B2 (en) | 2016-08-04 | 2020-04-28 | Oracle International Corporation | Unsupervised method for baselining and anomaly detection in time-series data for enterprise systems |
US11082439B2 (en) | 2016-08-04 | 2021-08-03 | Oracle International Corporation | Unsupervised method for baselining and anomaly detection in time-series data for enterprise systems |
US10949436B2 (en) | 2017-02-24 | 2021-03-16 | Oracle International Corporation | Optimization for scalable analytics using time series models |
US10915830B2 (en) | 2017-02-24 | 2021-02-09 | Oracle International Corporation | Multiscale method for predictive alerting |
US10817803B2 (en) | 2017-06-02 | 2020-10-27 | Oracle International Corporation | Data driven methods and systems for what if analysis |
US10678601B2 (en) | 2017-08-31 | 2020-06-09 | Oracle International Corporation | Orchestration service for multi-step recipe composition with flexible, topology-aware, and massive parallel execution |
US10621005B2 (en) | 2017-08-31 | 2020-04-14 | Oracle International Corporation | Systems and methods for providing zero down time and scalability in orchestration cloud services |
US10963346B2 (en) | 2018-06-05 | 2021-03-30 | Oracle International Corporation | Scalable methods and systems for approximating statistical distributions |
US10997517B2 (en) | 2018-06-05 | 2021-05-04 | Oracle International Corporation | Methods and systems for aggregating distribution approximations |
US11138090B2 (en) | 2018-10-23 | 2021-10-05 | Oracle International Corporation | Systems and methods for forecasting time series with variable seasonality |
US10855548B2 (en) | 2019-02-15 | 2020-12-01 | Oracle International Corporation | Systems and methods for automatically detecting, summarizing, and responding to anomalies |
US11949703B2 (en) | 2019-05-01 | 2024-04-02 | Oracle International Corporation | Systems and methods for multivariate anomaly detection in software monitoring |
US11533326B2 (en) | 2019-05-01 | 2022-12-20 | Oracle International Corporation | Systems and methods for multivariate anomaly detection in software monitoring |
US11537940B2 (en) | 2019-05-13 | 2022-12-27 | Oracle International Corporation | Systems and methods for unsupervised anomaly detection using non-parametric tolerance intervals over a sliding window of t-digests |
US11887015B2 (en) | 2019-09-13 | 2024-01-30 | Oracle International Corporation | Automatically-generated labels for time series data and numerical lists to use in analytic and machine learning systems |
Also Published As
Publication number | Publication date |
---|---|
US9658910B2 (en) | 2017-05-23 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US9658910B2 (en) | Systems and methods for spatially displaced correlation for detecting value ranges of transient correlation in machine data of enterprise systems | |
US10069900B2 (en) | Systems and methods for adaptive thresholding using maximum concentration intervals | |
Yadwadkar et al. | Selecting the best vm across multiple public clouds: A data-driven performance modeling approach | |
US10291463B2 (en) | Large-scale distributed correlation | |
Ibidunmoye et al. | Performance anomaly detection and bottleneck identification | |
US10095598B2 (en) | Transaction server performance monitoring using component performance data | |
Kavulya et al. | An analysis of traces from a production mapreduce cluster | |
US9391866B1 (en) | Method for qualitative analysis of system performance correlation factors | |
US9128792B2 (en) | Systems and methods for installing, managing, and provisioning applications | |
US9798644B2 (en) | Monitoring system performance with pattern event detection | |
US9235491B2 (en) | Systems and methods for installing, managing, and provisioning applications | |
US20120185735A1 (en) | System and method for determining causes of performance problems within middleware systems | |
US20140195860A1 (en) | Early Detection Of Failing Computers | |
US9886195B2 (en) | Performance-based migration among data storage devices | |
US8631280B2 (en) | Method of measuring and diagnosing misbehaviors of software components and resources | |
US20180121856A1 (en) | Factor-based processing of performance metrics | |
US9880879B1 (en) | Identifying task instance outliers based on metric data in a large scale parallel processing system | |
CN113342559A (en) | Diagnostic framework in a computing system | |
US20210366268A1 (en) | Automatic tuning of incident noise | |
US10684933B2 (en) | Smart self-healing service for data analytics systems | |
US20140096125A1 (en) | Systems and methods for installing, managing, and provisioning applications | |
US8949824B2 (en) | Systems and methods for installing, managing, and provisioning applications | |
US9015718B1 (en) | Identifying task instances that interfere with processor performance | |
US11036561B2 (en) | Detecting device utilization imbalances | |
Gupta et al. | Astro: A predictive model for anomaly detection and feedback-based scheduling on Hadoop |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: ORACLE INTERNATIONAL CORPORATION, CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:POOLA, THYAGARAJU;VOLCHEGURSKY, VLADIMIR;KARPURAM, VENKATA RAMANA;REEL/FRAME:033415/0727 Effective date: 20140723 |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 4 |