US20030120774A1 - Networked architecture for enabling automated gathering of information from WEB servers - Google Patents
Networked architecture for enabling automated gathering of information from WEB servers Download PDFInfo
- Publication number
- US20030120774A1 US20030120774A1 US10/360,337 US36033703A US2003120774A1 US 20030120774 A1 US20030120774 A1 US 20030120774A1 US 36033703 A US36033703 A US 36033703A US 2003120774 A1 US2003120774 A1 US 2003120774A1
- Authority
- US
- United States
- Prior art keywords
- servers
- data
- server
- work
- internet
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/954—Navigation, e.g. using categorised browsing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/1066—Session management
- H04L65/1101—Session protocols
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/957—Browsing optimisation, e.g. caching or content distillation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/958—Organisation or management of web site content, e.g. publishing, maintaining pages or automatic linking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/30—Authentication, i.e. establishing the identity or authorisation of security principals
- G06F21/31—User authentication
- G06F21/41—User authentication where a single sign-on provides access to a plurality of computers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2221/00—Indexing scheme relating to security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F2221/21—Indexing scheme relating to G06F21/00 and subgroups addressing additional information or applications relating to security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F2221/2119—Authenticating web pages, e.g. with suspicious links
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/99931—Database or file accessing
- Y10S707/99933—Query processing, i.e. searching
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/99931—Database or file accessing
- Y10S707/99933—Query processing, i.e. searching
- Y10S707/99935—Query augmenting and refining, e.g. inexact access
Definitions
- the present invention is related as a continuation in part (CIP) to a patent application entitled “Method and Apparatus for Obtaining and Presenting WEB Summaries to Users” filed on Jun. 1, 1999, for which Ser. No. 09/323,598 is assigned, and which is incorporated herein by reference, which is a CIP of application Ser. No. 09/208,740, also incorporated herein by reference.
- the present invention is in the field of digital network information gathering from network servers and pertains more particularly to methods and apparatus for providing and operating a networked system of machines dedicated to performing automated data gathering, processing, and presentation of such data.
- WWW World Wide Web
- Anyone with a suitable Internet appliance such as a personal computer with a standard Internet connection may connect to the Internet and navigate to many thousands of information pages (termed web pages) stored on Internet-connected servers for the purpose of garnering information and initiating transactions with hosts of such servers and pages.
- Internet nodes include any hosted machines dedicated to performing a service such as file serving, data storing, data routing, and so on.
- Such nodes are generally loosely associated with each other only by universal resource locator (URL) addressing and mapped network paths.
- URL universal resource locator
- Some data initiated by or requested by users is not protected from being intercepted by some network-connected nodes and therefore may perhaps be observed by third parties due to the nature of publicly-shared bandwidth over the Internet.
- various means for protecting data from being observed by third parties are established and routinely practiced by entities hosting pluralities of nodes connected to the Internet. Such methods include the use of firewall technology, secure servers, and private sub-networks connected to the Internet network.
- An information gathering, summarization and presentation system known to the inventor and described in the related patent application listed under the cross-reference section uses an Internet portal and software suite to allow users to request and obtain data including Web-page summaries containing specific data found by using a unique scripting method supplied by a knowledge worker. In some embodiments such data may also be pushed to a user subscribing to the service.
- a service such as that described above requires a considerable amount of processing power in order to service a very large client base in terms of job processing.
- a desired goal is to automate such an information gathering and presentation service so as to be wholly or largely transparent to individual users.
- Prior art network architectures do not possess the processing power nor the dedicated cross-communication capabilities that would be required for such a service to be wholly automated and be able to serve a mass clientele.
- a data-gathering and reporting system for collecting data from a wide area network (WAN)
- WAN wide area network
- a data-gathering and reporting system for collecting data from a wide area network (WAN)
- a database stored in a data repository
- a first server having access to the data base and organizing data-gathering work assignments from data in the database
- a hierarchical network of distributor servers having a highest level connected to the first server and expanding to a lowest level, with distributor servers at different levels connected by data links and distributing work assignments to lower levels on demand from the distributor servers at lower levels
- a plurality of gatherer servers connected by data links to the lowest level of the hierarchy of distributor servers and to the WAN, the lowest level of distributor servers distributing work assignments to the gatherer servers on demand from the gatherer servers, the gatherer servers accomplishing the work assignments distributed by the distributor servers and queueing data collected from the WAN as a result of the work assignments
- a hierarchical network of collector servers having a lowest level connected to the gatherer servers and contract
- the WAN is the Internet
- data is collected from WEB servers on the Internet.
- gating of work assignments and data between one server and another in the distributor server hierarchy is by the one server having a queue with an adjustable threshold, and demanding data or work assignments from the other server as a result of the queue level falling to the threshold. Latency and database writing efficiency may be adjusted by adjusting queue thresholds among servers, and server power and capacity required in a system is adjusted by scaling the number of servers and number of hierarchical levels of servers.
- priority is assigned to work assignments, and work assignments and collected data are gated from server to server according to assigned priority as well as by need. Also in some embodiments work assignments are expressed in a markup language, allowing all information required to fill an assignment to be encapsulated such that only the one or more filing servers need be connected to the database.
- the system is associated with an Internet subscription server, and the work assignments are for collecting data from WEB pages associated with individual subscribers.
- the work assignments may be automatically scheduled for individual subscribers and some assignments may be on demand from individual subscribers.
- Flow is by work requests from the work request generator down the hierarchy of distributor servers to the gatherer servers where work requests are accomplished by gathering WEB summaries from Internet servers according to the work requests, and by data collected from the gatherer servers up the hierarchy of collector servers to the filing server, and wherein flow is gated on demand down the hierarchy of distributor servers by each server from a previous server in the direction of flow.
- FIG. 1 is an architectural overview of a data-gathering network, components, and connectivity according to an embodiment of the present invention.
- FIG. 2 is a network diagram illustrating hierarchy and communication direction of part of the automated data-gathering system of FIG. 1.
- FIG. 1 is an architectural overview of a data-gathering network 109 and components thereof according to an embodiment of the present invention.
- Network 109 comprises a Data-packet network 111 , an automated data gathering system 115 , a PSTN network 113 , and a plurality of connected users 145 .
- Data-packet network 111 may be any type of wide area network (WAN) that is known in the art that is capable of data-packet communication.
- network 111 is the well-known Internet network, and will hereinafter be referred to as Internet 111 .
- the advantage of using Internet 111 is that it is the largest publicly-accessible data-packet medium available.
- Another advantage to using Internet 111 is that data communication protocols are well established and standardized. However, any data packet network may be used as long as suitable communication protocols, of which many are known, are in place. Other than the Internet such networks include private corporate Intranets and the like.
- Internet 111 comprises a plurality of exemplary WEB servers, 119 , 121 , 123 , and 125 , connected to an Internet backbone 117 as is known in the art.
- Servers 119 - 125 are adapted as normal file servers dedicated to serving WEB pages in a familiar format such as Hyper Text Markup Language (HTML).
- HTML Hyper Text Markup Language
- Internet 111 is connected to a public switched telephone network (PSTN) 113 as is generally known in the art of Internet access.
- PSTN public switched telephone network
- Typical public Internet access involves such as an Internet service provider (ISP) represented herein by element number 141 , which is accessed over a conventional telephone network connection system represented by element number 143 .
- ISP Internet service provider
- a plurality of users 145 shown connected to ISP 141 represent the most common method for public access to Internet 111 .
- There are several other methods known in the art for accomplishing access to Internet 111 such as continual corporate connections, satellite connections, etc, and the system shown is merely exemplary.
- Network 109 uses the Internet 111 and PSTN 113 in order to establish convenient access capability for users 145 .
- Users 145 in this example may be assumed to have typical internet access capability as is known in the art, typically including a PC, a telephone line, and a modem for dialing up the ISP.
- Users 145 may also be operating satellite connections, WEB TV cable connections, or any other known Internet connection that may be completed using one of a variety of Internet-capable appliances, including appliances having wireless connection, such as combinations of cell phones with personal organizer and computer capability.
- Architecture 115 represents an automated data gathering and presentation system adapted to provide optimum performance in the processing of mass information requests coming in continually from users such as users 145 .
- architecture 115 is centralized (housed in one location), however; a centralized architecture is not required in order to practice the present invention.
- architecture 115 may be distributed geographically throughout Internet 111 .
- Architecture 115 comprises a dedicated network of cooperating machines adapted to practice the functions of the present invention.
- Architecture 115 is hierarchical in construction in some parts meaning that pluralities of slave components at intermediate levels are ultimately directed by one master component.
- Architecture 115 comprises at least one scheduled update server 127 adapted to enter into and identify data-gathering job assignments that are stored in a database.
- a database holding such work may be stored in such as a mass repository 129 that is illustrated as connected to server 127 .
- Mass repository 129 is in a preferred embodiment an off-line storage facility and may be accessed and updated by server 127 .
- Mass repository 129 is large enough in terms of data-storage space to contain all user-profile and user initiated requests for information. In alternative embodiments, more than one mass repository such as repository 129 may be used.
- Mass repository 129 may be of any type known in the art such as an optical storage facility, or other known mass storage system, or a combination of different types.
- Database server 127 distributes scheduled work assignments in hierarchical fashion to a plurality of connected distributor servers 135 .
- Distributors 135 are connected to each other and to server 127 by dedicated network 139 , as is described below with reference to FIG. 2.
- Each distributor server 135 contains a work queue (not shown) adapted to hold job assignments until they are requested from another distributor further down the hierarchical line, thus the distribution of tasks for distributors coupled to server 127 is by pull technology, providing efficient loading. This effectively provides a distributed queue that automatically load balances on the number of servers available. In this way work is pulled down from distributor to distributor, as respective work-queues become able to handle more work.
- the ultimate goal of each distributor is to pass all of it's work assignments down until they are ultimately received by a plurality of connected gatherer machines 137 .
- a second scheduling server 130 is connected to server 128 and is dedicated to handling not scheduled, but instant-update requests from users 145 . Users may communicate such information-gathering requests to server 128 via the Internet, and server 130 acts through a second set of instant-update distributors 136 to gatherers 137 . Distributors 136 do not operate by pull technology, but rather on demand to immediately execute instant update requests. These distributors have their queues refilled by user requests rather than by database queries.
- Gatherers 137 are adapted to obtain work assignments from distributors 135 , and perform the assigned functions with respect to each job. Each gatherer 137 has a work queue (not shown) adapted to hold job assignments passed down from distributors 135 . As individual work queues become depleted, gatherers 137 request additional work from associated distributors up the line. Dedicated network 139 connects gatherers 137 to distributors 135 .
- each gatherer is afforded a full-time Internet connection represented herein by a data connection line 117 a illustrated as teeing off backbone 117 .
- Database server 127 also has a full-time Internet connection illustrated herein as a branch of data connection 117 a.
- each gatherer is provided with enough additional processing power and suitable software to perform its organization and rendering of data into a suitable format as to be compatible to users such as users 145 .
- Internet connectivity with respect to server 127 allows users 145 to upload data requests using suitable software on their Internet appliances. Such software is not shown here. However, a suitable example is taught in the cross-referenced patent application.
- the Internet connection afforded to server 127 is a user connection allowing bi-directional communication.
- the Internet connections afforded to gatherers 137 are dedicated to allowing them to navigate Internet 111 and retrieve particular data according to job assignment. There is no user communication with gatherers 137 .
- the navigation process generic to gatherers 137 is wholly automated and transparent to users.
- collectors 133 are computer nodes adapted to efficiently collect data and to pass the data back to the database held in mass repository 129 .
- Collectors 133 are connected to gatherers 137 via digital network 139 . Each collector accepts completed data packages passed on to them by gatherers 137 . The movement of data through the hierarchy of the collectors is by push technology.
- digital network 139 is a separate and dedicated network adapted for swift transmission of data between connected machines. In this way, no competition exists for precious bandwidth resources. In a centralized scenario such as is exemplified in this embodiment, network 139 may be implemented economically and efficiently.
- Network 139 may or may not be adapted to communicate via Internet protocol as long as database server 127 has a means for interpretation and rendering of alternate data formats into HTML, XML, or another suitable format for serving the data information to users 145 (typically in the form of a WEB page).
- the language in any case is a markup language, and is therefore extensible over time.
- architecture 115 may use a metadata system of communication between connected nodes and storage facility 129 .
- exemplary architecture described above may be used with virtually any type of information gathering service that uses a client and parent software application without departing from the spirit and scope of the present invention.
- a large corporation or technical campus may practice the present invention privately using the architecture described above on a private or corporate WAN instead of the Internet.
- One may also run on a Virtual Private Network (VPN) on top of the Internet backbone.
- VPN Virtual Private Network
- the inventor intends that architecture 115 may be used with the WEB-summary service described in the related patent application referenced above, and therefore, is designed for that purpose in this embodiment. Slight modifications may be made to machines and connections in order to adapt architecture 115 to other variations of WEB-based or network-based information gathering and presentation services.
- architecture 115 provides optimum scalability to accommodate increased or decreased user demand. Furthermore, a fact that only one machine is required to have bi-directional communication capability with storage facility 129 insures economy and practicability with regard to socket connection requirements. More detail regarding the hierarchy of architecture 115 is provided below.
- FIG. 2 is a network diagram illustrating hierarchy and communication direction of part of the architecture 115 of FIG. 1.
- architecture 115 is held on a separate digital network 139 as described above with reference to FIG. 1.
- architecture 115 may be distributed over a WAN using the WAN, which could be the Internet, as a communication medium rather than a separate digital network as described in FIG. 1.
- all nodes would be slaved to their master nodes by addressing techniques on the WAN rather than hierarchical connection by a separate network.
- a separate digital network may still be provided to run in parallel with the WAN. The purpose of using a separate dedicated network to connect all nodes is to speed up transmission of data in the loop.
- architecture 115 for scheduled updates utilizes database server 127 at the very top of the hierarchy.
- Server 127 manages data stored in repository 129 and communicates to users via Internet path 117 .
- Server 127 has access to user-profile address lists, and users 145 (FIG. 1) also upload special requests to server 128 (FIG. 1) which are handles via server 130 and distributor hierarchy 136 (not shown in FIG. 2 ).
- server 128 FIG. 1
- work assignments representing unfulfilled request are created and distributed over network 139 for scheduled requests to distributors 135 using a trickle-down pull technique as illustrated by the directional “communication” arrows connecting each distributor.
- distributors 135 there are six distributors 135 represented in this hierarchical tree.
- the top distributor pulls assignments from server 127 and passes them on to two distributors “down the tree”, which in turn pass them on to three distributors further down the tree.
- the passing on is controlled by queues at each distributor having adjustable thresholds. As a queue at a distributor falls below a specified threshold, the distributor requests more work assignments from the higher-level distributors to which it is slaved.
- a lower level of distributors 135 will distribute assignments to gatherers 137 . It is the gatherer's job to accomplish the job assignments by navigating the Internet ( 111 ) by virtue of Internet connection 117 a and the URL lists associated with the job assignments, and to retrieve information requested in each given job assignment held in their queues.
- each gatherer 137 is equipped with suitable navigational software and parsing capability as described in the cross-referenced patent application. The inventors also refer to gatherers 137 as agents. In this embodiment, gathers 137 also summarize and organize retrieved data into WEB-summaries according to user direction as passed on with the work assignments. The exact nature of job performance attributed to gatherers 137 will, of course, be dictated by the software and processing capability afforded them. As previously described, other information sourced from the Internet or any other data network may be obtained and processed according to predetermined rules.
- Gatherers 137 have connection ports provided and adapted for pulling information from distributors 135 . Gatherers 137 are similarly provided with connection ports that are adapted for passing information to collectors 133 as illustrated by the directional “communication” arrows. These ports are associated with network 139 and not with Internet 111 . A third port is provided for each gatherer to access the Internet or other designated WAN.
- the gatherers are queue-managed, as are the distributors, so the gatherers pull work assignments from the distributors according to queue thresholds, just as lower-level distributors work with higher-level distributors.
- the collectors 133 push collected data from completed assignments from the gatherers up the collector network to the filer or filers.
- a top-level collector or collectors 133 pass completed job assignments to filers 131 , which are connected to and write data directly to repository 129 updating the database.
- Filers 131 may be provided as one or more powerful processors, or a lager number of less powerful processors.
- a secondary or failsafe contingent of filers 131 may be provided and adapted to take over in the event that first-line filers fail for any reason.
- Processing power may be regulated with respect to all connected nodes such that data is continually being streamed down and back up the loop created by network 139 without being held up.
- additional failsafe connections may be provided between connected nodes at a same level in the tree such that if one node appears ready to fail or needs to be withdrawn from the hierarchy for any reason, it's queue may be emptied to adjacent nodes.
- a means for detecting and mirroring duplicate requests is provided. This is provided in one embodiment in the form of a second database representing completed assignments and user attributes and a software module that checks for duplicate requests coming into server 127 against a first database containing all unfulfilled requests and those requests already in process. If a duplicate or more than one duplicate request is discovered such as, perhaps, return today's New York Times headlines, then only the leading request (one being processed) of the same nature is allowed to proceed. Once the request is written into repository 129 by one of filers 131 , it is mirrored or made available to all of the users that initiated the same request. In this way, much unnecessary work may be eliminated from the process to affect streamlining.
- a priority system may be used in the queuing and distribution of work assignments.
- on-demand requests may take priority over requests that will be accessed at a later time by users.
- priority requests may be tagged according to priority upon receipt by any means known in the art and caused to trickle through each queue according to that priority such that they may gain on and surpass other requests of lesser priority moving through the system.
- Any priority system may be adopted and used by system 109 according to enterprise rules.
- gatherers 137 may, if overloaded to a point wherein they are causing an unacceptable amount of latency, use their Internet connection to send completed job assignments over Internet paths 117 a and 117 to a duplicate or mirrored site that is distributed elsewhere on Internet 111 .
- a mirrored site may have a separate digital network and nodes connected thereto just as architecture 115 . It may be a case wherein the second site is not operating to capacity and could handle the extra load.
- Such a second site may be connected to a first site via Internet connection as described, or may also have a dedicated data link connecting to the first site and adapted to become active only when required for load balancing.
- Server 127 is, in a preferred embodiment, adapted to notify users 145 when their requests are available in the case of user-initiated requests, and to schedule delivery of updates according to stored user profiles. This is accomplished via Internet path 117 . In some cases, requests may be delivered if so ordered. In other cases they may be pulled from server 127 or another connected server adapted for the purpose. As to network 139 , a push system is used. Work assignments are pushed from each node to the next. This concept acts to discourage any overload. A separate data storage facility may be provided wherein users may access completed requests. Un-accessed requests may be purged after a period of time. Similarly, requests that have been accessed or delivered are also purged from the system.
- server 127 may be programmed to slow or stop the receiving of requests until such time that the system is deemed capable of handling more work at the desired pace. Such a condition would alert system administrators of a need to scale-up according to more demand. Similarly, if there is a lull in workflow, then parts of the system may be shutdown without affecting system performance. Ultimately, a system could be scaled down if needed.
- Primary access to system 109 may be provided at the ISP level such as with the Internet Portal server described in the cross-referenced patent application. Subscribers may first have to verify identity and perhaps use a password before being allowed to access server 127 . In some cases, interface servers may be provided and distributed over different regions wherein requests from those servers are delivered to a server such as server 127 .
- architecture 115 may be wholly automated and adapted to perform a wide variety of information gathering and presentation services.
- architecture 115 may be used for obtaining and presenting WEB-summaries as is the case in this example, or it may be adapted to such as returning review summaries to administrative workers regarding such as completed cases or other such review work.
Abstract
A data-gathering and reporting system for collecting WEB summaries from the Internet for individual subscribers to a Portal subscription system has a plurality of gatherer servers each connected to the Internet, to an ascending hierarchy of work request distribution servers, and to a ascending hierarchy of collector servers. A work request generator at the top of the hierarchy of distribution servers generates work requests for collecting WEB summaries, and a filer server at the top of the hierarchy of collector servers writes data to a database. Work flow is by work requests from the work request generator down the hierarchy of distributor servers to the gatherer servers, where work requests are accomplished by gathering WEB summaries from Internet servers according to the work requests, and by data collected from the gatherer servers up the hierarchy of collector servers to the filing server.
Description
- The present invention is related as a continuation in part (CIP) to a patent application entitled “Method and Apparatus for Obtaining and Presenting WEB Summaries to Users” filed on Jun. 1, 1999, for which Ser. No. 09/323,598 is assigned, and which is incorporated herein by reference, which is a CIP of application Ser. No. 09/208,740, also incorporated herein by reference.
- The present invention is in the field of digital network information gathering from network servers and pertains more particularly to methods and apparatus for providing and operating a networked system of machines dedicated to performing automated data gathering, processing, and presentation of such data.
- The information network known as the World Wide Web (WWW), which is a subset of the well-known Internet, is arguably the most complete source of publicly accessible information available. Anyone with a suitable Internet appliance such as a personal computer with a standard Internet connection may connect to the Internet and navigate to many thousands of information pages (termed web pages) stored on Internet-connected servers for the purpose of garnering information and initiating transactions with hosts of such servers and pages.
- Information travels over the Internet network through many connected computers known as nodes in the art. Internet nodes include any hosted machines dedicated to performing a service such as file serving, data storing, data routing, and so on. Such nodes are generally loosely associated with each other only by universal resource locator (URL) addressing and mapped network paths.
- Some data initiated by or requested by users is not protected from being intercepted by some network-connected nodes and therefore may perhaps be observed by third parties due to the nature of publicly-shared bandwidth over the Internet. However, various means for protecting data from being observed by third parties are established and routinely practiced by entities hosting pluralities of nodes connected to the Internet. Such methods include the use of firewall technology, secure servers, and private sub-networks connected to the Internet network.
- Many companies doing business on the Internet host semi-private data networks comprising a plurality of computer nodes dedicated to the provision of proprietary information and related data. Certain authorized users such as those working for the company or those having password access and/or active and verifiable accounts with the company may access such data. For example, a large company may host a plurality of file servers, including connected data storage systems wherein users may search for and access data stored for the purpose by the company. Such sub-nets, as they are often termed, use the Internet as a connective wide area network (WAN) and the data travels through shared bandwidth connections. Although a user may be protected from third party interceptions of data sent or requested the user must generally navigate to each URL where data is available. If a search engine is provided to assist a user in searching for specific data made available by the company, it is limited to searching only the nodes hosted by the company or data from third party nodes that is made available through cooperative URL linking or posting.
- An information gathering, summarization and presentation system known to the inventor and described in the related patent application listed under the cross-reference section uses an Internet portal and software suite to allow users to request and obtain data including Web-page summaries containing specific data found by using a unique scripting method supplied by a knowledge worker. In some embodiments such data may also be pushed to a user subscribing to the service.
- A service such as that described above requires a considerable amount of processing power in order to service a very large client base in terms of job processing. A desired goal is to automate such an information gathering and presentation service so as to be wholly or largely transparent to individual users. Prior art network architectures do not possess the processing power nor the dedicated cross-communication capabilities that would be required for such a service to be wholly automated and be able to serve a mass clientele.
- What is clearly needed is a dedicated and hierarchical network of cooperating computer-nodes that is adapted to fulfill a very large number of automatically-schedules and user-initiated data requests in a wholly automated and transparent fashion. Such a networked system could be scaleable in that it may be easily expanded in terms of adding machinery according to user demand. Such a system would save users and service providers much time and labor associated with obtaining optimum and efficient results from an information gathering and presentation service.
- In a preferred embodiment of the present invention a data-gathering and reporting system for collecting data from a wide area network (WAN) is provided, comprising a database stored in a data repository; a first server having access to the data base and organizing data-gathering work assignments from data in the database; a hierarchical network of distributor servers having a highest level connected to the first server and expanding to a lowest level, with distributor servers at different levels connected by data links and distributing work assignments to lower levels on demand from the distributor servers at lower levels; a plurality of gatherer servers connected by data links to the lowest level of the hierarchy of distributor servers and to the WAN, the lowest level of distributor servers distributing work assignments to the gatherer servers on demand from the gatherer servers, the gatherer servers accomplishing the work assignments distributed by the distributor servers and queueing data collected from the WAN as a result of the work assignments; a hierarchical network of collector servers having a lowest level connected to the gatherer servers and contracting to a highest level, the gatherer servers communicating data collected to the lowest level of collector servers, with collector servers at different levels connected by data links and delivering collected data to higher levels; and one or more filing servers connected to the highest level of collector servers, the filing servers communicating with the database in the data repository, the collector servers delivering collected data to the one or more filing servers, and the filing servers writing the collected data to the database.
- In one important embodiment the WAN is the Internet, and data is collected from WEB servers on the Internet. Also in a preferred embodiment gating of work assignments and data between one server and another in the distributor server hierarchy is by the one server having a queue with an adjustable threshold, and demanding data or work assignments from the other server as a result of the queue level falling to the threshold. Latency and database writing efficiency may be adjusted by adjusting queue thresholds among servers, and server power and capacity required in a system is adjusted by scaling the number of servers and number of hierarchical levels of servers.
- In some embodiments priority is assigned to work assignments, and work assignments and collected data are gated from server to server according to assigned priority as well as by need. Also in some embodiments work assignments are expressed in a markup language, allowing all information required to fill an assignment to be encapsulated such that only the one or more filing servers need be connected to the database.
- In a preferred embodiment the system is associated with an Internet subscription server, and the work assignments are for collecting data from WEB pages associated with individual subscribers. In this case some work assignments may be automatically scheduled for individual subscribers and some assignments may be on demand from individual subscribers.
- In another aspect of the invention a data-gathering and reporting system for collecting WEB summaries from the Internet for individual subscribers to a Portal subscription system is provided, comprising a plurality of gatherer servers each connected to the Internet, to an ascending hierarchy of work request distribution servers, and to a ascending hierarchy of collector servers; a work request generator at the top of the hierarchy of distribution servers, generating work requests for collecting WEB summaries; and a filer server at the top of the hierarchy of collector servers, the file server connected to and writing data to a database. Flow is by work requests from the work request generator down the hierarchy of distributor servers to the gatherer servers where work requests are accomplished by gathering WEB summaries from Internet servers according to the work requests, and by data collected from the gatherer servers up the hierarchy of collector servers to the filing server, and wherein flow is gated on demand down the hierarchy of distributor servers by each server from a previous server in the direction of flow.
- In this system gating of work assignments and data between one distribution server and another is by the one server having a queue with an adjustable threshold, and demanding data or work assignments from the other server as a result of the queue level falling to the threshold. Latency and database writing efficiency is adjusted by adjusting queue thresholds among servers, and server power and capacity required in a system is adjusted by scaling the number of servers and number of hierarchical levels of servers. In some cases priority may be assigned to work assignments, and work assignments and collected data may be gated from server to server according to assigned priority as well as by need. Also in a preferred work assignments are expressed in a markup language, allowing all information required to fill an assignment to be encapsulated such that only the one or more filing servers need be connected to the database.
- In another aspect of the invention methods are provided for practicing the invention using the system of the invention. In the embodiments of the invention taught below in enabling detail, for the first time a scalable and very efficient system for gathering large amounts of data on the Internet is provided, where the data collected may be directed by work assignments in small increments. There are many advantages. For example, the system of the invention relieves the user of the necessity of navigating the clutter of the Internet to find what is needed on a daily basis. It also provides immediate access for the user to information from multiple sources, because information is gathered on behalf of a user continuously. Various second-level service may also be provided, such as access from wireless internet appliance devices.
- FIG. 1 is an architectural overview of a data-gathering network, components, and connectivity according to an embodiment of the present invention.
- FIG. 2 is a network diagram illustrating hierarchy and communication direction of part of the automated data-gathering system of FIG. 1.
- It was described in the background section that in order to provide a viable data gathering and presentation system for servicing a mass clientele, such a system should be dedicated, automated and possess enough processing power to fill a large and continuous user demand. To this end, the inventors provide a scaleable networked architecture that is dedicated to achieving the goals of the present invention in an automated fashion and that is transparent to the user. Such an architecture is taught in enabling detail below.
- FIG. 1 is an architectural overview of a data-
gathering network 109 and components thereof according to an embodiment of the present invention. Network 109 comprises a Data-packet network 111, an automateddata gathering system 115, aPSTN network 113, and a plurality of connectedusers 145. - Data-
packet network 111 may be any type of wide area network (WAN) that is known in the art that is capable of data-packet communication. In this embodiment,network 111 is the well-known Internet network, and will hereinafter be referred to as Internet 111. The advantage of using Internet 111 is that it is the largest publicly-accessible data-packet medium available. Another advantage to using Internet 111 is that data communication protocols are well established and standardized. However, any data packet network may be used as long as suitable communication protocols, of which many are known, are in place. Other than the Internet such networks include private corporate Intranets and the like. -
Internet 111 comprises a plurality of exemplary WEB servers, 119, 121, 123, and 125, connected to anInternet backbone 117 as is known in the art. Servers 119-125 are adapted as normal file servers dedicated to serving WEB pages in a familiar format such as Hyper Text Markup Language (HTML). These servers are equivalent to servers 23, 25, and 27 of the cross-referenced patent application Ser. No. 09/323,598, from which Web summaries may be gathered. -
Internet 111 is connected to a public switched telephone network (PSTN) 113 as is generally known in the art of Internet access. Typical public Internet access involves such as an Internet service provider (ISP) represented herein byelement number 141, which is accessed over a conventional telephone network connection system represented byelement number 143. A plurality ofusers 145, shown connected toISP 141 represent the most common method for public access toInternet 111. There are several other methods known in the art for accomplishing access toInternet 111 such as continual corporate connections, satellite connections, etc, and the system shown is merely exemplary. -
Network 109 uses theInternet 111 andPSTN 113 in order to establish convenient access capability forusers 145.Users 145, in this example may be assumed to have typical internet access capability as is known in the art, typically including a PC, a telephone line, and a modem for dialing up the ISP.Users 145 may also be operating satellite connections, WEB TV cable connections, or any other known Internet connection that may be completed using one of a variety of Internet-capable appliances, including appliances having wireless connection, such as combinations of cell phones with personal organizer and computer capability. Although there are only fourusers 145 represented in this example, it will be appreciated that there will be many more such that a mass clientele is established creating a heavy demand onsystem 109. - It is disclosed in the cross-referenced patent application that users may obtain WEB summaries relating to virtually any WEB page available on the Internet. Such Web pages include those URLs in individual URL lists maintained for the users (subscribers), any other URL that may be identified to the system by a user, and individual Web accounts. This process is automated except for directional input by the user and scripting supplied by knowledge workers, and is a function of
server 128 shown in FIG. 1 withinarchitecture 115.Server 128 is equivalent to server 31, of FIG. 1 of the cross-referenced patent application, and provides portal functions including the obtaining and presenting of Web summaries to users, as well as automatic authentication of user's accounts as gathering is done, through the features of the Portal server, which is the subject of cross-referenced patent application Ser. No. 09/208,740. In order to insure that an information gathering and summarization service such as the one described in the related application will be able to service an exceptionally large client base, a unique architecture comprising dedicate machines and networked connections must be providedArchitecture 115 represents an automated data gathering and presentation system adapted to provide optimum performance in the processing of mass information requests coming in continually from users such asusers 145. In this embodiment,architecture 115 is centralized (housed in one location), however; a centralized architecture is not required in order to practice the present invention. In analternative embodiment architecture 115 may be distributed geographically throughoutInternet 111. -
Architecture 115 comprises a dedicated network of cooperating machines adapted to practice the functions of the present invention.Architecture 115 is hierarchical in construction in some parts meaning that pluralities of slave components at intermediate levels are ultimately directed by one master component.Architecture 115 comprises at least one scheduledupdate server 127 adapted to enter into and identify data-gathering job assignments that are stored in a database. A database holding such work may be stored in such as amass repository 129 that is illustrated as connected toserver 127.Mass repository 129 is in a preferred embodiment an off-line storage facility and may be accessed and updated byserver 127.Mass repository 129 is large enough in terms of data-storage space to contain all user-profile and user initiated requests for information. In alternative embodiments, more than one mass repository such asrepository 129 may be used.Mass repository 129 may be of any type known in the art such as an optical storage facility, or other known mass storage system, or a combination of different types. -
Database server 127 distributes scheduled work assignments in hierarchical fashion to a plurality ofconnected distributor servers 135.Distributors 135 are connected to each other and toserver 127 bydedicated network 139, as is described below with reference to FIG. 2. Eachdistributor server 135 contains a work queue (not shown) adapted to hold job assignments until they are requested from another distributor further down the hierarchical line, thus the distribution of tasks for distributors coupled toserver 127 is by pull technology, providing efficient loading. This effectively provides a distributed queue that automatically load balances on the number of servers available. In this way work is pulled down from distributor to distributor, as respective work-queues become able to handle more work. The ultimate goal of each distributor is to pass all of it's work assignments down until they are ultimately received by a plurality ofconnected gatherer machines 137. - A
second scheduling server 130 is connected toserver 128 and is dedicated to handling not scheduled, but instant-update requests fromusers 145. Users may communicate such information-gathering requests toserver 128 via the Internet, andserver 130 acts through a second set of instant-update distributors 136 togatherers 137.Distributors 136 do not operate by pull technology, but rather on demand to immediately execute instant update requests. These distributors have their queues refilled by user requests rather than by database queries. -
Gatherers 137 are adapted to obtain work assignments fromdistributors 135, and perform the assigned functions with respect to each job. Eachgatherer 137 has a work queue (not shown) adapted to hold job assignments passed down fromdistributors 135. As individual work queues become depleted,gatherers 137 request additional work from associated distributors up the line.Dedicated network 139 connectsgatherers 137 todistributors 135. - It is the objective goal of all gatherers to navigate
Internet 111, and pull data from WEB servers such as from servers 119-125 and process the data according to their job assignments. To achieve this purpose, each gatherer is afforded a full-time Internet connection represented herein by adata connection line 117 a illustrated as teeing offbackbone 117.Database server 127 also has a full-time Internet connection illustrated herein as a branch ofdata connection 117 a. In addition to having an Internet connection for navigatingInternet 111, each gatherer is provided with enough additional processing power and suitable software to perform its organization and rendering of data into a suitable format as to be compatible to users such asusers 145. - Internet connectivity with respect to
server 127 allowsusers 145 to upload data requests using suitable software on their Internet appliances. Such software is not shown here. However, a suitable example is taught in the cross-referenced patent application. The Internet connection afforded toserver 127 is a user connection allowing bi-directional communication. In contrast, the Internet connections afforded togatherers 137 are dedicated to allowing them to navigateInternet 111 and retrieve particular data according to job assignment. There is no user communication withgatherers 137. The navigation process generic togatherers 137 is wholly automated and transparent to users. - As
gatherers 137 complete their job assignments, the associated data is passed on to a plurality of machines represented herein byelement number 133 and termed collectors by the inventors.Collectors 133 are computer nodes adapted to efficiently collect data and to pass the data back to the database held inmass repository 129.Collectors 133 are connected togatherers 137 viadigital network 139. Each collector accepts completed data packages passed on to them bygatherers 137. The movement of data through the hierarchy of the collectors is by push technology. - Eventually, collectors pass completed jobs on to powerful filer processors.
Filers 131 are dedicated and adapted to writing finished data directly into the database stored inrepository 129. In this example, following the disclosure of the cross-referenced patent application, finished data represents WEB summaries requested ofsystem 109 byusers 145 as taught in the cross-referenced patent application. Similarly, the software used in conjunction withcommunication system 109 could be identical or similar to the software taught therein. - It is noted here, and supported by repeated references to
digital network 139 that theentire architecture 115 is held off-line (not connected to the Internet) save for the described connection toserver 127 and connections provided togatherers 137. In this regard,digital network 139 is a separate and dedicated network adapted for swift transmission of data between connected machines. In this way, no competition exists for precious bandwidth resources. In a centralized scenario such as is exemplified in this embodiment,network 139 may be implemented economically and efficiently. -
Network 139 may or may not be adapted to communicate via Internet protocol as long asdatabase server 127 has a means for interpretation and rendering of alternate data formats into HTML, XML, or another suitable format for serving the data information to users 145 (typically in the form of a WEB page). The language in any case is a markup language, and is therefore extensible over time. In order to savestorage space architecture 115 may use a metadata system of communication between connected nodes andstorage facility 129. - It will be apparent to one with skill in the art that the exemplary architecture described above may be used with virtually any type of information gathering service that uses a client and parent software application without departing from the spirit and scope of the present invention. For example, a large corporation or technical campus may practice the present invention privately using the architecture described above on a private or corporate WAN instead of the Internet. One may also run on a Virtual Private Network (VPN) on top of the Internet backbone. The inventor intends that
architecture 115 may be used with the WEB-summary service described in the related patent application referenced above, and therefore, is designed for that purpose in this embodiment. Slight modifications may be made to machines and connections in order to adaptarchitecture 115 to other variations of WEB-based or network-based information gathering and presentation services. - The unique hierarchical connection scheme provided to
architecture 115 provides optimum scalability to accommodate increased or decreased user demand. Furthermore, a fact that only one machine is required to have bi-directional communication capability withstorage facility 129 insures economy and practicability with regard to socket connection requirements. More detail regarding the hierarchy ofarchitecture 115 is provided below. - FIG. 2 is a network diagram illustrating hierarchy and communication direction of part of the
architecture 115 of FIG. 1. In this example,architecture 115 is held on a separatedigital network 139 as described above with reference to FIG. 1. However, in an alternative embodiment,architecture 115 may be distributed over a WAN using the WAN, which could be the Internet, as a communication medium rather than a separate digital network as described in FIG. 1. - In the above-described embodiment, all nodes would be slaved to their master nodes by addressing techniques on the WAN rather than hierarchical connection by a separate network. In still another embodiment, a separate digital network may still be provided to run in parallel with the WAN. The purpose of using a separate dedicated network to connect all nodes is to speed up transmission of data in the loop.
- Referring back to FIG. 2,
architecture 115 for scheduled updates utilizesdatabase server 127 at the very top of the hierarchy.Server 127 manages data stored inrepository 129 and communicates to users viaInternet path 117.Server 127 has access to user-profile address lists, and users 145 (FIG. 1) also upload special requests to server 128 (FIG. 1) which are handles viaserver 130 and distributor hierarchy 136 (not shown in FIG. 2). As data gathering requirements come due according to user profiles and requests fromusers 145 are logged and stored, work assignments representing unfulfilled request are created and distributed overnetwork 139 for scheduled requests todistributors 135 using a trickle-down pull technique as illustrated by the directional “communication” arrows connecting each distributor. For example, there are sixdistributors 135 represented in this hierarchical tree. The top distributor pulls assignments fromserver 127 and passes them on to two distributors “down the tree”, which in turn pass them on to three distributors further down the tree. The passing on, however, is controlled by queues at each distributor having adjustable thresholds. As a queue at a distributor falls below a specified threshold, the distributor requests more work assignments from the higher-level distributors to which it is slaved. - It will be appreciated by one with skill in the art that there may be more than one distributor at the top of the tree passing assignments to still more distributors down the tree than are illustrated in this embodiment. The inventors intend to illustrate only the nature of cascading assignments to more and more distributors situated down the tree, by the queue-controlled pull technique.
- Ultimately, a lower level of
distributors 135 will distribute assignments togatherers 137. It is the gatherer's job to accomplish the job assignments by navigating the Internet (111) by virtue ofInternet connection 117 a and the URL lists associated with the job assignments, and to retrieve information requested in each given job assignment held in their queues. To achieve this end, eachgatherer 137 is equipped with suitable navigational software and parsing capability as described in the cross-referenced patent application. The inventors also refer togatherers 137 as agents. In this embodiment, gathers 137 also summarize and organize retrieved data into WEB-summaries according to user direction as passed on with the work assignments. The exact nature of job performance attributed togatherers 137 will, of course, be dictated by the software and processing capability afforded them. As previously described, other information sourced from the Internet or any other data network may be obtained and processed according to predetermined rules. -
Gatherers 137 have connection ports provided and adapted for pulling information fromdistributors 135.Gatherers 137 are similarly provided with connection ports that are adapted for passing information tocollectors 133 as illustrated by the directional “communication” arrows. These ports are associated withnetwork 139 and not withInternet 111. A third port is provided for each gatherer to access the Internet or other designated WAN. - The gatherers are queue-managed, as are the distributors, so the gatherers pull work assignments from the distributors according to queue thresholds, just as lower-level distributors work with higher-level distributors. The
collectors 133 push collected data from completed assignments from the gatherers up the collector network to the filer or filers. - It can be seen in this example that a hierarchical loop is created that ultimately ends back at
repository 129. For example, A top-level collector orcollectors 133 pass completed job assignments tofilers 131, which are connected to and write data directly torepository 129 updating the database.Filers 131 may be provided as one or more powerful processors, or a lager number of less powerful processors. Moreover, a secondary or failsafe contingent offilers 131 may be provided and adapted to take over in the event that first-line filers fail for any reason. - Processing power may be regulated with respect to all connected nodes such that data is continually being streamed down and back up the loop created by
network 139 without being held up. In one embodiment, additional failsafe connections may be provided between connected nodes at a same level in the tree such that if one node appears ready to fail or needs to be withdrawn from the hierarchy for any reason, it's queue may be emptied to adjacent nodes. - In another embodiment of the present invention, a means for detecting and mirroring duplicate requests is provided. This is provided in one embodiment in the form of a second database representing completed assignments and user attributes and a software module that checks for duplicate requests coming into
server 127 against a first database containing all unfulfilled requests and those requests already in process. If a duplicate or more than one duplicate request is discovered such as, perhaps, return today's New York Times headlines, then only the leading request (one being processed) of the same nature is allowed to proceed. Once the request is written intorepository 129 by one offilers 131, it is mirrored or made available to all of the users that initiated the same request. In this way, much unnecessary work may be eliminated from the process to affect streamlining. - In still another embodiment, a priority system may be used in the queuing and distribution of work assignments. In this embodiment, on-demand requests may take priority over requests that will be accessed at a later time by users. For example, priority requests may be tagged according to priority upon receipt by any means known in the art and caused to trickle through each queue according to that priority such that they may gain on and surpass other requests of lesser priority moving through the system. Any priority system may be adopted and used by
system 109 according to enterprise rules. - In still a further embodiment of the present invention,
gatherers 137 may, if overloaded to a point wherein they are causing an unacceptable amount of latency, use their Internet connection to send completed job assignments overInternet paths Internet 111. Such a mirrored site may have a separate digital network and nodes connected thereto just asarchitecture 115. It may be a case wherein the second site is not operating to capacity and could handle the extra load. Such a second site may be connected to a first site via Internet connection as described, or may also have a dedicated data link connecting to the first site and adapted to become active only when required for load balancing. -
Server 127 is, in a preferred embodiment, adapted to notifyusers 145 when their requests are available in the case of user-initiated requests, and to schedule delivery of updates according to stored user profiles. This is accomplished viaInternet path 117. In some cases, requests may be delivered if so ordered. In other cases they may be pulled fromserver 127 or another connected server adapted for the purpose. As tonetwork 139, a push system is used. Work assignments are pushed from each node to the next. This concept acts to discourage any overload. A separate data storage facility may be provided wherein users may access completed requests. Un-accessed requests may be purged after a period of time. Similarly, requests that have been accessed or delivered are also purged from the system. - If the entire system is operating at maximum capacity, then
server 127 may be programmed to slow or stop the receiving of requests until such time that the system is deemed capable of handling more work at the desired pace. Such a condition would alert system administrators of a need to scale-up according to more demand. Similarly, if there is a lull in workflow, then parts of the system may be shutdown without affecting system performance. Ultimately, a system could be scaled down if needed. - Primary access to
system 109 may be provided at the ISP level such as with the Internet Portal server described in the cross-referenced patent application. Subscribers may first have to verify identity and perhaps use a password before being allowed to accessserver 127. In some cases, interface servers may be provided and distributed over different regions wherein requests from those servers are delivered to a server such asserver 127. - It will be apparent to one with skill in the art that a networked system architecture such as
architecture 115 may be wholly automated and adapted to perform a wide variety of information gathering and presentation services. For example,architecture 115 may be used for obtaining and presenting WEB-summaries as is the case in this example, or it may be adapted to such as returning review summaries to administrative workers regarding such as completed cases or other such review work. There are many possible and varied implementations. Therefore, the method and apparatus of the present invention should be afforded the broadest scope. The spirit and scope of the present invention is limited only by the claims that follow.
Claims (23)
1. A data-gathering and reporting system for collecting data from a wide area network (WAN) comprising:
a database stored in a data repository;
a first server having access to the data base and organizing data-gathering work assignments from data in the database;
a hierarchical network of distributor servers having a highest level connected to the first server and expanding to a lowest level, with distributor servers at different levels connected by data links and distributing work assignments to lower levels on demand from the distributor servers at lower levels;
a plurality of gatherer servers connected by data links to the lowest level of the hierarchy of distributor servers and to the WAN, the lowest level of distributor servers distributing work assignments to the gatherer servers on demand from the gatherer servers, the gatherer servers accomplishing the work assignments distributed by the distributor servers and queueing data collected from the WAN as a result of the work assignments;
a hierarchical network of collector servers having a lowest level connected to the gatherer servers and contracting to a highest level, the gatherer servers communicating data collected to the lowest level of collector servers, with collector servers at different levels connected by data links and delivering collected data to higher levels by push; and
one or more filing servers connected to the highest level of collector servers, the filing servers communicating with the database in the data repository, the collector servers delivering collected data to the one or more filing servers, and the filing servers writing the collected data to the database.
2. The system of claim 1 wherein the WAN is the Internet, and data is collected from WEB servers on the Internet.
3. The system of claim 1 wherein gating of work assignments and data between one server and another in the distributor network is by the one server having a queue with an adjustable threshold, and demanding data or work assignments from the other server as a result of the queue level falling to the threshold.
4. The system of claim 3 wherein latency and database writing efficiency is adjusted by adjusting queue thresholds among servers.
5. The system of claim 1 wherein server power and capacity required in a system is adjusted by scaling the number of servers and number of hierarchical levels of servers.
6. The system of claim 1 wherein priority is assigned to work assignments, and work assignments and collected data are gated from server to server according to assigned priority as well as by need.
7. The system of claim 1 wherein work assignments are expressed in a markup language, allowing all information required to fill an assignment to be encapsulated such that only the one or more filing servers need be connected to the database.
8. The system of claim 2 wherein the system is associated with an Internet subscription server, and the work assignments are for collecting data from WEB pages associated with individual subscribers.
9. The system of claim 8 wherein some work assignments are automatically scheduled for individual subscribers and some assignments are on demand from individual subscribers.
10. A data-gathering and reporting system for collecting WEB summaries from the Internet for individual subscribers to a Portal subscription system, comprising:
a plurality of gatherer servers each connected to the Internet, to an ascending hierarchy of work request distribution servers, and to a ascending hierarchy of collector servers;
a work request generator at the top of the hierarchy of distribution servers, generating work requests for collecting WEB summaries; and
a filer server at the top of the hierarchy of collector servers, the file server connected to and writing data to a database;
wherein flow is by work requests from the work request generator down the hierarchy of distributor servers to the gatherer servers where work requests are accomplished by gathering WEB summaries from Internet servers according to the work requests, and by data collected from the gatherer servers up the hierarchy of collector servers to the filing server.
11. The system of claim 10 wherein gating of work assignments and data between one server and another in the hierarchy of distributor servers is by the one server having a queue with an adjustable threshold, and demanding data or work assignments from the other server as a result of the queue level falling to the threshold.
12. The system of claim 11 wherein latency and database writing efficiency is adjusted by adjusting queue thresholds among servers.
13. The system of claim 10 wherein server power and capacity required in a system is adjusted by scaling the number of servers and number of hierarchical levels of servers.
14. The system of claim 10 wherein priority is assigned to work assignments, and work assignments and collected data are gated from server to server according to assigned priority as well as by need.
15. The system of claim 10 wherein work assignments are expressed in a markup language, allowing all information required to fill an assignment to be encapsulated such that only the one or more filing servers need be connected to the database.
16. The system of claim 10 wherein some work assignments are automatically scheduled for individual subscribers and some assignments are on demand from individual subscribers.
17. A method for gathering data from the Internet, comprising:
(a) generating data collection requests by a request generator;
(b) passing the requests down a descending hierarchy of distributor servers on demand from servers at lower levels;
(c) accomplishing the data gathering requests by a level of gatherer servers connected to the Internet and the lowest level of distributor servers, the gatherer servers pulling requests from the distributor servers;
(d) passing collected data in discrete packets associated with the requests up an ascending hierarchy of collector servers to a filing server at the top of the hierarchy; and
(e) writing the collected data to a database by the filing server.
18. The method of claim 17 wherein gating of work assignments and data between one server and another in the distributor server hierarchy is by the one server having a queue with an adjustable threshold, and demanding data or work assignments from the other server as a result of the queue level falling to the threshold.
19. The method of claim 18 wherein latency and database writing efficiency is adjusted by adjusting queue thresholds among servers.
20. The method of claim 17 wherein server power and capacity required in a system is adjusted by scaling the number of servers and number of hierarchical levels of servers.
21. The method of claim 17 wherein priority is assigned to work requests, and work requests and collected data are gated from server to server according to assigned priority as well as by need.
22. The method of claim 17 wherein work requests are expressed in a markup language, allowing all information required to fill a request to be encapsulated such that only the filing server needs be connected to the database.
23. The method of claim 17 wherein some work requests are automatically scheduled for individual subscribers and some assignments are on demand from individual subscribers.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US10/360,337 US20030120774A1 (en) | 1998-12-08 | 2003-02-07 | Networked architecture for enabling automated gathering of information from WEB servers |
Applications Claiming Priority (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US09/208,740 US6412073B1 (en) | 1998-12-08 | 1998-12-08 | Method and apparatus for providing and maintaining a user-interactive portal system accessible via internet or other switched-packet-network |
US09/323,598 US6199077B1 (en) | 1998-12-08 | 1999-06-01 | Server-side web summary generation and presentation |
US09/362,914 US6517587B2 (en) | 1998-12-08 | 1999-07-27 | Networked architecture for enabling automated gathering of information from Web servers |
US10/360,337 US20030120774A1 (en) | 1998-12-08 | 2003-02-07 | Networked architecture for enabling automated gathering of information from WEB servers |
Related Parent Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US09/323,598 Continuation-In-Part US6199077B1 (en) | 1998-12-08 | 1999-06-01 | Server-side web summary generation and presentation |
US09/362,914 Division US6517587B2 (en) | 1998-12-08 | 1999-07-27 | Networked architecture for enabling automated gathering of information from Web servers |
Publications (1)
Publication Number | Publication Date |
---|---|
US20030120774A1 true US20030120774A1 (en) | 2003-06-26 |
Family
ID=23428028
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US09/362,914 Expired - Lifetime US6517587B2 (en) | 1998-12-08 | 1999-07-27 | Networked architecture for enabling automated gathering of information from Web servers |
US10/360,337 Abandoned US20030120774A1 (en) | 1998-12-08 | 2003-02-07 | Networked architecture for enabling automated gathering of information from WEB servers |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US09/362,914 Expired - Lifetime US6517587B2 (en) | 1998-12-08 | 1999-07-27 | Networked architecture for enabling automated gathering of information from Web servers |
Country Status (5)
Country | Link |
---|---|
US (2) | US6517587B2 (en) |
EP (1) | EP1236084A1 (en) |
JP (1) | JP2003505784A (en) |
AU (1) | AU5918500A (en) |
WO (1) | WO2001008000A1 (en) |
Cited By (16)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20060173770A1 (en) * | 2005-01-31 | 2006-08-03 | Mckay Anthony | Telephony controlled auction |
US7672879B1 (en) | 1998-12-08 | 2010-03-02 | Yodlee.Com, Inc. | Interactive activity interface for managing personal data and performing transactions over a data packet network |
US7752535B2 (en) | 1999-06-01 | 2010-07-06 | Yodlec.com, Inc. | Categorization of summarized information |
US20100317377A1 (en) * | 2009-06-12 | 2010-12-16 | Zou Lin | Queue Management System Allows queue number to be remotely obtained by Patients or customers |
US7856386B2 (en) | 2006-09-07 | 2010-12-21 | Yodlee, Inc. | Host exchange in bill paying services |
US7979348B2 (en) | 2002-04-23 | 2011-07-12 | Clearing House Payments Co Llc | Payment identification code and payment system using the same |
US8069407B1 (en) | 1998-12-08 | 2011-11-29 | Yodlee.Com, Inc. | Method and apparatus for detecting changes in websites and reporting results to web developers for navigation template repair purposes |
US8190629B2 (en) | 1998-12-08 | 2012-05-29 | Yodlee.Com, Inc. | Network-based bookmark management and web-summary system |
US8261334B2 (en) | 2008-04-25 | 2012-09-04 | Yodlee Inc. | System for performing web authentication of a user by proxy |
US8555359B2 (en) | 2009-02-26 | 2013-10-08 | Yodlee, Inc. | System and methods for automatically accessing a web site on behalf of a client |
US8725607B2 (en) | 2004-01-30 | 2014-05-13 | The Clearing House Payments Company LLC | Electronic payment clearing and check image exchange systems and methods |
CN106209446A (en) * | 2016-07-06 | 2016-12-07 | 北京交通大学 | The construction method of the service application logic network of data center server |
US11042882B2 (en) | 2015-07-01 | 2021-06-22 | The Clearing House Payments Company, L.L.C. | Real-time payment system, method, apparatus, and computer program |
US11295308B1 (en) | 2014-10-29 | 2022-04-05 | The Clearing House Payments Company, L.L.C. | Secure payment processing |
US11436577B2 (en) | 2018-05-03 | 2022-09-06 | The Clearing House Payments Company L.L.C. | Bill pay service with federated directory model support |
US11694168B2 (en) | 2015-07-01 | 2023-07-04 | The Clearing House Payments Company L.L.C. | Real-time payment system, method, apparatus, and computer program |
Families Citing this family (85)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
ATE242511T1 (en) | 1998-10-28 | 2003-06-15 | Verticalone Corp | APPARATUS AND METHOD FOR AUTOMATICALLY COMPOSING AND TRANSMITTING TRANSACTIONS CONTAINING PERSONAL ELECTRONIC INFORMATION OR DATA |
US7200804B1 (en) * | 1998-12-08 | 2007-04-03 | Yodlee.Com, Inc. | Method and apparatus for providing automation to an internet navigation application |
US6802042B2 (en) * | 1999-06-01 | 2004-10-05 | Yodlee.Com, Inc. | Method and apparatus for providing calculated and solution-oriented personalized summary-reports to a user through a single user-interface |
IL143573A0 (en) | 1998-12-09 | 2002-04-21 | Network Ice Corp | A method and apparatus for providing network and computer system security |
US20040078423A1 (en) * | 2002-03-22 | 2004-04-22 | Ramakrishna Satyavolu | Method and apparatus for controlled establishment of a turnkey system providing a centralized data aggregation and summary capability to third party entities |
US6477565B1 (en) * | 1999-06-01 | 2002-11-05 | Yodlee.Com, Inc. | Method and apparatus for restructuring of personalized data for transmission from a data network to connected and portable network appliances |
US7346929B1 (en) * | 1999-07-29 | 2008-03-18 | International Business Machines Corporation | Method and apparatus for auditing network security |
US6947903B1 (en) * | 1999-08-06 | 2005-09-20 | Elcommerce.Com.Inc. | Method and system for monitoring a supply-chain |
US7444390B2 (en) * | 1999-10-20 | 2008-10-28 | Cdimensions, Inc. | Method and apparatus for providing a web-based active virtual file system |
US8006243B2 (en) * | 1999-12-07 | 2011-08-23 | International Business Machines Corporation | Method and apparatus for remote installation of network drivers and software |
WO2001054388A1 (en) | 2000-01-07 | 2001-07-26 | Ineto, Inc. | Customer communication service system |
WO2001084775A2 (en) | 2000-04-28 | 2001-11-08 | Internet Security Systems, Inc. | System and method for managing security events on a network |
JP4700884B2 (en) * | 2000-04-28 | 2011-06-15 | インターナショナル・ビジネス・マシーンズ・コーポレーション | Method and system for managing computer security information |
JP4037999B2 (en) * | 2000-05-15 | 2008-01-23 | インターナショナル・ビジネス・マシーンズ・コーポレーション | Website, robot type search engine response system, robot type search engine registration method, storage medium, and program transmission device |
US7680912B1 (en) * | 2000-05-18 | 2010-03-16 | thePlatform, Inc. | System and method for managing and provisioning streamed data |
US7257766B1 (en) * | 2000-06-29 | 2007-08-14 | Egocentricity Ltd. | Site finding |
US7640200B2 (en) | 2000-07-10 | 2009-12-29 | Byallaccounts, Inc. | Financial portfolio management system and method |
US9027121B2 (en) | 2000-10-10 | 2015-05-05 | International Business Machines Corporation | Method and system for creating a record for one or more computer security incidents |
US7567921B1 (en) * | 2000-10-23 | 2009-07-28 | Business-To-Investor, Inc. | Method and system for providing commercial information and operating an electronic commerce system over a global communications network with company and constituency nodes |
US7146305B2 (en) * | 2000-10-24 | 2006-12-05 | Vcis, Inc. | Analytical virtual machine |
US7325067B1 (en) * | 2000-11-27 | 2008-01-29 | Esaya, Inc. | Personalized account migration system and method |
US7130466B2 (en) * | 2000-12-21 | 2006-10-31 | Cobion Ag | System and method for compiling images from a database and comparing the compiled images with known images |
US20020147803A1 (en) * | 2001-01-31 | 2002-10-10 | Dodd Timothy David | Method and system for calculating risk in association with a security audit of a computer network |
US7089309B2 (en) * | 2001-03-21 | 2006-08-08 | Theplatform For Media, Inc. | Method and system for managing and distributing digital media |
US7657419B2 (en) * | 2001-06-19 | 2010-02-02 | International Business Machines Corporation | Analytical virtual machine |
CA2466079A1 (en) * | 2001-11-16 | 2003-05-30 | Cranel Incorporated | System and method for improving support for information technology through collecting, diagnosing and reporting configuration, metric, and event information |
US7673137B2 (en) * | 2002-01-04 | 2010-03-02 | International Business Machines Corporation | System and method for the managed security control of processes on a computer system |
US7292689B2 (en) | 2002-03-15 | 2007-11-06 | Intellisist, Inc. | System and method for providing a message-based communications infrastructure for automated call center operation |
US8068595B2 (en) | 2002-03-15 | 2011-11-29 | Intellisist, Inc. | System and method for providing a multi-modal communications infrastructure for automated call center operation |
US8170197B2 (en) * | 2002-03-15 | 2012-05-01 | Intellisist, Inc. | System and method for providing automated call center post-call processing |
US7370360B2 (en) * | 2002-05-13 | 2008-05-06 | International Business Machines Corporation | Computer immune system and method for detecting unwanted code in a P-code or partially compiled native-code program executing within a virtual machine |
US20040205581A1 (en) * | 2002-07-15 | 2004-10-14 | Gava Fabio M. | Hierarchical storage |
US20070179961A1 (en) * | 2002-07-15 | 2007-08-02 | Fabio Gava | Hierarchical storage |
US8832178B2 (en) | 2002-11-06 | 2014-09-09 | Noel William Lovisa | Service implementation |
US8683016B1 (en) * | 2002-12-20 | 2014-03-25 | Versata Development Group, Inc. | Data recording components and processes for acquiring selected web site data |
GB2397402A (en) * | 2003-01-20 | 2004-07-21 | Mitel Networks Corp | Internet proxy that supports location-based services |
US7913303B1 (en) | 2003-01-21 | 2011-03-22 | International Business Machines Corporation | Method and system for dynamically protecting a computer system from attack |
US8255978B2 (en) * | 2003-03-11 | 2012-08-28 | Innovatrend, Inc. | Verified personal information database |
US7636786B2 (en) * | 2003-06-19 | 2009-12-22 | International Business Machines Corporation | Facilitating access to a resource of an on-line service |
US7536387B2 (en) * | 2003-08-15 | 2009-05-19 | Intelligent Medical Objects, Inc. | Method for interfacing applications to maintain data integrity |
US20050050456A1 (en) * | 2003-08-29 | 2005-03-03 | Dehamer Brian James | Method and apparatus for supporting XML-based service consumption in a web presentation architecture |
US20050081204A1 (en) * | 2003-09-25 | 2005-04-14 | International Business Machines Corporation | Method and system for dynamically bounded spinning threads on a contested mutex |
US20050086664A1 (en) * | 2003-10-01 | 2005-04-21 | Sundaresan Sankar R. | Method and apparatus for transaction tracking in a web presentation architecture |
US20050091336A1 (en) * | 2003-10-01 | 2005-04-28 | Dehamer Brian J. | Method and apparatus for supporting cookie management in a web presentation architecture |
US20050086292A1 (en) * | 2003-10-01 | 2005-04-21 | Yee Sunny K. | Method and apparatus for supporting preprocessing in a Web presentation architecture |
US7146544B2 (en) * | 2003-10-01 | 2006-12-05 | Hewlett-Packard Development Company, L.P. | Method and apparatus for supporting error handling in a web presentation architecture |
US20050076291A1 (en) * | 2003-10-01 | 2005-04-07 | Yee Sunny K. | Method and apparatus for supporting page localization management in a Web presentation architecture |
US20050076329A1 (en) * | 2003-10-01 | 2005-04-07 | Christina Hsu | Method and apparatus for supporting configuration of a web application in a web presentation architecture |
US20050076294A1 (en) * | 2003-10-01 | 2005-04-07 | Dehamer Brian James | Method and apparatus for supporting layout management in a web presentation architecture |
US7657938B2 (en) * | 2003-10-28 | 2010-02-02 | International Business Machines Corporation | Method and system for protecting computer networks by altering unwanted network data traffic |
WO2006009879A2 (en) * | 2004-06-18 | 2006-01-26 | Washington Mutual, Inc. | System for automatically transferring account information, such as information regarding a financial servicees account |
JP4708862B2 (en) * | 2005-05-26 | 2011-06-22 | キヤノン株式会社 | Optical scanning device and image forming apparatus using the same |
US20070184903A1 (en) * | 2006-02-08 | 2007-08-09 | Derek Liu | Network-based game system capable of serving massive number of game players |
US7519734B1 (en) | 2006-03-14 | 2009-04-14 | Amazon Technologies, Inc. | System and method for routing service requests |
US7996730B2 (en) * | 2006-06-05 | 2011-08-09 | International Business Machines Corporation | Customizable system for the automatic gathering of software service information |
US7849069B2 (en) * | 2006-06-21 | 2010-12-07 | International Business Machines Corporation | Method and system for federated resource discovery service in distributed systems |
US8775214B2 (en) | 2006-07-19 | 2014-07-08 | Thompson Reuters (Market) LLC | Management method and system for a user |
US9424270B1 (en) * | 2006-09-28 | 2016-08-23 | Photobucket Corporation | System and method for managing media files |
US9171040B2 (en) * | 2006-10-10 | 2015-10-27 | International Business Machines Corporation | Methods, systems, and computer program products for optimizing query evaluation and processing in a subscription notification service |
US8159961B1 (en) * | 2007-03-30 | 2012-04-17 | Amazon Technologies, Inc. | Load balancing utilizing adaptive thresholding |
US8285656B1 (en) | 2007-03-30 | 2012-10-09 | Consumerinfo.Com, Inc. | Systems and methods for data verification |
US9990674B1 (en) | 2007-12-14 | 2018-06-05 | Consumerinfo.Com, Inc. | Card registry systems and methods |
US8312033B1 (en) | 2008-06-26 | 2012-11-13 | Experian Marketing Solutions, Inc. | Systems and methods for providing an integrated identifier |
US8060424B2 (en) | 2008-11-05 | 2011-11-15 | Consumerinfo.Com, Inc. | On-line method and system for monitoring and reporting unused available credit |
US9262754B1 (en) | 2009-08-21 | 2016-02-16 | Wells Fargo Bank, N.A. | Request tracking system and method |
US9483606B1 (en) | 2011-07-08 | 2016-11-01 | Consumerinfo.Com, Inc. | Lifescore |
US9106691B1 (en) | 2011-09-16 | 2015-08-11 | Consumerinfo.Com, Inc. | Systems and methods of identity protection and management |
US8738516B1 (en) | 2011-10-13 | 2014-05-27 | Consumerinfo.Com, Inc. | Debt services candidate locator |
US9853959B1 (en) | 2012-05-07 | 2017-12-26 | Consumerinfo.Com, Inc. | Storage and maintenance of personal data |
US9654541B1 (en) | 2012-11-12 | 2017-05-16 | Consumerinfo.Com, Inc. | Aggregating user web browsing data |
US9916621B1 (en) | 2012-11-30 | 2018-03-13 | Consumerinfo.Com, Inc. | Presentation of credit score factors |
US9406085B1 (en) | 2013-03-14 | 2016-08-02 | Consumerinfo.Com, Inc. | System and methods for credit dispute processing, resolution, and reporting |
US10102570B1 (en) | 2013-03-14 | 2018-10-16 | Consumerinfo.Com, Inc. | Account vulnerability alerts |
US10685398B1 (en) | 2013-04-23 | 2020-06-16 | Consumerinfo.Com, Inc. | Presenting credit score information |
US10102536B1 (en) | 2013-11-15 | 2018-10-16 | Experian Information Solutions, Inc. | Micro-geographic aggregation system |
US9477737B1 (en) | 2013-11-20 | 2016-10-25 | Consumerinfo.Com, Inc. | Systems and user interfaces for dynamic access of multiple remote databases and synchronization of data based on user rules |
US10262362B1 (en) | 2014-02-14 | 2019-04-16 | Experian Information Solutions, Inc. | Automatic generation of code for attributes |
CN110383319B (en) | 2017-01-31 | 2023-05-26 | 益百利信息解决方案公司 | Large scale heterogeneous data ingestion and user resolution |
US10650023B2 (en) * | 2018-07-24 | 2020-05-12 | Booz Allen Hamilton, Inc. | Process for establishing trust between multiple autonomous systems for the purposes of command and control |
US10880313B2 (en) | 2018-09-05 | 2020-12-29 | Consumerinfo.Com, Inc. | Database platform for realtime updating of user data from third party sources |
US10963434B1 (en) | 2018-09-07 | 2021-03-30 | Experian Information Solutions, Inc. | Data architecture for supporting multiple search models |
US11315179B1 (en) | 2018-11-16 | 2022-04-26 | Consumerinfo.Com, Inc. | Methods and apparatuses for customized card recommendations |
US11238656B1 (en) | 2019-02-22 | 2022-02-01 | Consumerinfo.Com, Inc. | System and method for an augmented reality experience via an artificial intelligence bot |
US11941065B1 (en) | 2019-09-13 | 2024-03-26 | Experian Information Solutions, Inc. | Single identifier platform for storing entity data |
US11880377B1 (en) | 2021-03-26 | 2024-01-23 | Experian Information Solutions, Inc. | Systems and methods for entity resolution |
Citations (14)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5924090A (en) * | 1997-05-01 | 1999-07-13 | Northern Light Technology Llc | Method and apparatus for searching a database of records |
US5937168A (en) * | 1997-05-30 | 1999-08-10 | Bellsouth Corporation | Routing information within an adaptive routing architecture of an information retrieval system |
US6021409A (en) * | 1996-08-09 | 2000-02-01 | Digital Equipment Corporation | Method for parsing, indexing and searching world-wide-web pages |
US6067545A (en) * | 1997-08-01 | 2000-05-23 | Hewlett-Packard Company | Resource rebalancing in networked computer systems |
US6070191A (en) * | 1997-10-17 | 2000-05-30 | Lucent Technologies Inc. | Data distribution techniques for load-balanced fault-tolerant web access |
US6085188A (en) * | 1998-03-30 | 2000-07-04 | International Business Machines Corporation | Method of hierarchical LDAP searching with relational tables |
US6101500A (en) * | 1998-01-07 | 2000-08-08 | Novell, Inc. | System and method for managing objects in a hierarchical data structure |
US6122673A (en) * | 1998-07-22 | 2000-09-19 | Fore Systems, Inc. | Port scheduler and method for scheduling service providing guarantees, hierarchical rate limiting with/without overbooking capability |
US6173322B1 (en) * | 1997-06-05 | 2001-01-09 | Silicon Graphics, Inc. | Network request distribution based on static rules and dynamic performance data |
US6199077B1 (en) * | 1998-12-08 | 2001-03-06 | Yodlee.Com, Inc. | Server-side web summary generation and presentation |
US6301584B1 (en) * | 1997-08-21 | 2001-10-09 | Home Information Services, Inc. | System and method for retrieving entities and integrating data |
US6412073B1 (en) * | 1998-12-08 | 2002-06-25 | Yodiee.Com, Inc | Method and apparatus for providing and maintaining a user-interactive portal system accessible via internet or other switched-packet-network |
US6651098B1 (en) * | 2000-02-17 | 2003-11-18 | International Business Machines Corporation | Web site management in a world wide web communication network through reassignment of the server computers designated for respective web documents based upon user hit rates for the documents |
US6859783B2 (en) * | 1995-12-29 | 2005-02-22 | Worldcom, Inc. | Integrated interface for web based customer care and trouble management |
Family Cites Families (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CA1341310C (en) * | 1988-07-15 | 2001-10-23 | Robert Filepp | Interactive computer network and method of operation |
SE9300671D0 (en) * | 1993-03-01 | 1993-03-01 | Sven Nauckhoff | WORK FLOW MANAGEMENT |
US5838918A (en) * | 1993-12-13 | 1998-11-17 | International Business Machines Corporation | Distributing system configuration information from a manager machine to subscribed endpoint machines in a distrubuted computing environment |
US5768577A (en) | 1994-09-29 | 1998-06-16 | International Business Machines Corporation | Performance optimization in a heterogeneous, distributed database environment |
WO1997019415A2 (en) * | 1995-11-07 | 1997-05-29 | Cadis, Inc. | Search engine for remote object oriented database management system |
US6085238A (en) * | 1996-04-23 | 2000-07-04 | Matsushita Electric Works, Ltd. | Virtual LAN system |
US6185601B1 (en) * | 1996-08-02 | 2001-02-06 | Hewlett-Packard Company | Dynamic load balancing of a network of client and server computers |
US5787425A (en) * | 1996-10-01 | 1998-07-28 | International Business Machines Corporation | Object-oriented data mining framework mechanism |
US6381640B1 (en) * | 1998-09-11 | 2002-04-30 | Genesys Telecommunications Laboratories, Inc. | Method and apparatus for automated personalization and presentation of workload assignments to agents within a multimedia communication center |
-
1999
- 1999-07-27 US US09/362,914 patent/US6517587B2/en not_active Expired - Lifetime
-
2000
- 2000-07-07 JP JP2001513028A patent/JP2003505784A/en active Pending
- 2000-07-07 WO PCT/US2000/018542 patent/WO2001008000A1/en not_active Application Discontinuation
- 2000-07-07 AU AU59185/00A patent/AU5918500A/en not_active Abandoned
- 2000-07-07 EP EP00945208A patent/EP1236084A1/en not_active Withdrawn
-
2003
- 2003-02-07 US US10/360,337 patent/US20030120774A1/en not_active Abandoned
Patent Citations (14)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6859783B2 (en) * | 1995-12-29 | 2005-02-22 | Worldcom, Inc. | Integrated interface for web based customer care and trouble management |
US6021409A (en) * | 1996-08-09 | 2000-02-01 | Digital Equipment Corporation | Method for parsing, indexing and searching world-wide-web pages |
US5924090A (en) * | 1997-05-01 | 1999-07-13 | Northern Light Technology Llc | Method and apparatus for searching a database of records |
US5937168A (en) * | 1997-05-30 | 1999-08-10 | Bellsouth Corporation | Routing information within an adaptive routing architecture of an information retrieval system |
US6173322B1 (en) * | 1997-06-05 | 2001-01-09 | Silicon Graphics, Inc. | Network request distribution based on static rules and dynamic performance data |
US6067545A (en) * | 1997-08-01 | 2000-05-23 | Hewlett-Packard Company | Resource rebalancing in networked computer systems |
US6301584B1 (en) * | 1997-08-21 | 2001-10-09 | Home Information Services, Inc. | System and method for retrieving entities and integrating data |
US6070191A (en) * | 1997-10-17 | 2000-05-30 | Lucent Technologies Inc. | Data distribution techniques for load-balanced fault-tolerant web access |
US6101500A (en) * | 1998-01-07 | 2000-08-08 | Novell, Inc. | System and method for managing objects in a hierarchical data structure |
US6085188A (en) * | 1998-03-30 | 2000-07-04 | International Business Machines Corporation | Method of hierarchical LDAP searching with relational tables |
US6122673A (en) * | 1998-07-22 | 2000-09-19 | Fore Systems, Inc. | Port scheduler and method for scheduling service providing guarantees, hierarchical rate limiting with/without overbooking capability |
US6199077B1 (en) * | 1998-12-08 | 2001-03-06 | Yodlee.Com, Inc. | Server-side web summary generation and presentation |
US6412073B1 (en) * | 1998-12-08 | 2002-06-25 | Yodiee.Com, Inc | Method and apparatus for providing and maintaining a user-interactive portal system accessible via internet or other switched-packet-network |
US6651098B1 (en) * | 2000-02-17 | 2003-11-18 | International Business Machines Corporation | Web site management in a world wide web communication network through reassignment of the server computers designated for respective web documents based upon user hit rates for the documents |
Cited By (24)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8069407B1 (en) | 1998-12-08 | 2011-11-29 | Yodlee.Com, Inc. | Method and apparatus for detecting changes in websites and reporting results to web developers for navigation template repair purposes |
US7672879B1 (en) | 1998-12-08 | 2010-03-02 | Yodlee.Com, Inc. | Interactive activity interface for managing personal data and performing transactions over a data packet network |
US8190629B2 (en) | 1998-12-08 | 2012-05-29 | Yodlee.Com, Inc. | Network-based bookmark management and web-summary system |
US7752535B2 (en) | 1999-06-01 | 2010-07-06 | Yodlec.com, Inc. | Categorization of summarized information |
US10387879B2 (en) | 2002-04-23 | 2019-08-20 | The Clearing Housse Payments Company L.L.C. | Payment identification code and payment system using the same |
US7979348B2 (en) | 2002-04-23 | 2011-07-12 | Clearing House Payments Co Llc | Payment identification code and payment system using the same |
US9799011B2 (en) | 2004-01-30 | 2017-10-24 | The Clearing House Payments Company L.L.C. | Electronic payment clearing and check image exchange systems and methods |
US11301824B2 (en) | 2004-01-30 | 2022-04-12 | The Clearing House Payments Company LLC | Electronic payment clearing and check image exchange systems and methods |
US10685337B2 (en) | 2004-01-30 | 2020-06-16 | The Clearing House Payments Company L.L.C. | Electronic payment clearing and check image exchange systems and methods |
US10643190B2 (en) | 2004-01-30 | 2020-05-05 | The Clearing House Payments Company L.L.C. | Electronic payment clearing and check image exchange systems and methods |
US8725607B2 (en) | 2004-01-30 | 2014-05-13 | The Clearing House Payments Company LLC | Electronic payment clearing and check image exchange systems and methods |
US10636018B2 (en) | 2004-01-30 | 2020-04-28 | The Clearing House Payments Company L.L.C. | Electronic payment clearing and check image exchange systems and methods |
US20060173770A1 (en) * | 2005-01-31 | 2006-08-03 | Mckay Anthony | Telephony controlled auction |
US7856386B2 (en) | 2006-09-07 | 2010-12-21 | Yodlee, Inc. | Host exchange in bill paying services |
US8261334B2 (en) | 2008-04-25 | 2012-09-04 | Yodlee Inc. | System for performing web authentication of a user by proxy |
US8555359B2 (en) | 2009-02-26 | 2013-10-08 | Yodlee, Inc. | System and methods for automatically accessing a web site on behalf of a client |
US20100317377A1 (en) * | 2009-06-12 | 2010-12-16 | Zou Lin | Queue Management System Allows queue number to be remotely obtained by Patients or customers |
US11295308B1 (en) | 2014-10-29 | 2022-04-05 | The Clearing House Payments Company, L.L.C. | Secure payment processing |
US11816666B2 (en) | 2014-10-29 | 2023-11-14 | The Clearing House Payments Company L.L.C. | Secure payment processing |
US11042882B2 (en) | 2015-07-01 | 2021-06-22 | The Clearing House Payments Company, L.L.C. | Real-time payment system, method, apparatus, and computer program |
US11694168B2 (en) | 2015-07-01 | 2023-07-04 | The Clearing House Payments Company L.L.C. | Real-time payment system, method, apparatus, and computer program |
CN106209446A (en) * | 2016-07-06 | 2016-12-07 | 北京交通大学 | The construction method of the service application logic network of data center server |
US11436577B2 (en) | 2018-05-03 | 2022-09-06 | The Clearing House Payments Company L.L.C. | Bill pay service with federated directory model support |
US11829967B2 (en) | 2018-05-03 | 2023-11-28 | The Clearing House Payments Company L.L.C. | Bill pay service with federated directory model support |
Also Published As
Publication number | Publication date |
---|---|
WO2001008000A1 (en) | 2001-02-01 |
US6517587B2 (en) | 2003-02-11 |
US20020023104A1 (en) | 2002-02-21 |
EP1236084A1 (en) | 2002-09-04 |
JP2003505784A (en) | 2003-02-12 |
AU5918500A (en) | 2001-02-13 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US6517587B2 (en) | Networked architecture for enabling automated gathering of information from Web servers | |
US20030191832A1 (en) | Method and apparatus for controlled establishment of a turnkey system providing a centralized data aggregation and summary capability to third party entities | |
US20040078423A1 (en) | Method and apparatus for controlled establishment of a turnkey system providing a centralized data aggregation and summary capability to third party entities | |
US6081840A (en) | Two-level content distribution system | |
US8055706B2 (en) | Transparent request routing for a partitioned application service | |
US6789103B1 (en) | Synchronized server parameter database | |
US8370470B2 (en) | System and method for managing server configurations | |
US6868444B1 (en) | Server configuration management and tracking | |
DE69835674T2 (en) | SYSTEM AND METHOD FOR SERVER-EFFICIENT OPTIMIZATION OF DATA TRANSMISSION IN A DISTRIBUTED COMPUTER NETWORK | |
US20020004816A1 (en) | System and method for on-network storage services | |
US6223209B1 (en) | Distributed world wide web servers | |
DE69838739T2 (en) | Method and apparatus for presenting and using network topology data | |
US6564251B2 (en) | Scalable computing system for presenting customized aggregation of information | |
US8326846B2 (en) | Virtual list view support in a distributed directory | |
US5933606A (en) | Dynamic link page retargeting using page headers | |
US20030135611A1 (en) | Self-monitoring service system with improved user administration and user access control | |
US5933596A (en) | Multiple server dynamic page link retargeting | |
US20100042927A1 (en) | Third Party Management of Computer System Control | |
US20030093463A1 (en) | Dynamic distribution and network storage system | |
CN1852145A (en) | System and method for identifying authority using relative inquire | |
WO2002077844A2 (en) | Turnkey system providing centralized data aggregation | |
CN115858503B (en) | Heterogeneous database migration management method and system based on migration linked list | |
JP3681313B2 (en) | Data distribution method | |
US20090164457A1 (en) | Information collection, filtering and distribution method and system | |
JPH0877058A (en) | Information providing system |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |
|
AS | Assignment |
Owner name: YODLEE, INC., CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:YODLEE.COM, INC.;REEL/FRAME:047364/0170 Effective date: 20181029 |