US20110191327A1 - Method for Human Ranking of Search Results - Google Patents
Method for Human Ranking of Search Results Download PDFInfo
- Publication number
- US20110191327A1 US20110191327A1 US12/944,842 US94484210A US2011191327A1 US 20110191327 A1 US20110191327 A1 US 20110191327A1 US 94484210 A US94484210 A US 94484210A US 2011191327 A1 US2011191327 A1 US 2011191327A1
- Authority
- US
- United States
- Prior art keywords
- result items
- rankings
- users
- search
- user
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/248—Presentation of query results
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/951—Indexing; Web crawling techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
- G06F16/9535—Search customisation based on user profiles and personalisation
Definitions
- the present invention relates generally to search engines, and more particularly, to methods of ranking search results and to user interfaces that are used to display search results.
- Search engines assist users in finding relevant documents or other informational content from among a larger corpus.
- a common application of search engines is to assist users in finding information, documents, and files that are located on a network.
- a common application of search engines is to assist users in finding information, documents, and files on the World Wide Web (“Web”), which comprises a massive network of Web documents.
- Web World Wide Web
- the user submits a word or phrase he or she is interested in, called a search query, and the search engine seeks to provide a list of high quality Web documents that are relevant to that query.
- the list of Web documents is usually ranked with higher-quality and more relevant documents appearing first.
- the ranking is conventionally done either by examination of the contents of the Web documents or by examining the link structure of the Web documents.
- a search engine may examine the words on a web document and assign an importance ranking according to how similar or related the words on the web document are to the search query.
- a search engine may examine the link structure of a web document and assign an importance ranking according to which other documents link to the document and which other documents the document links to.
- a search engine may assign an importance ranking to a document based on the user's search history.
- a search engine may assign an importance ranking to a document based on whether the user has previously navigated to and made a purchase from that document.
- search engines One problem with search engines is that the methods of ranking documents often do not produce rankings that users believe to be an accurate reflection of relevance or quality. Often, a Web document that users believe to be the highest quality and most relevant is not listed first or second or anywhere near the top of the results list. A second problem is that users who notice this problem do not have a means to fix it so that future users who search for the same query can receive better search results.
- One aspect of the invention is directed to a method that includes receiving at a computer system a search query from a user.
- the method includes performing a search over a set of data in response to the search query.
- the method includes generating a rankings of a set of result items from the search over the set of data.
- the method includes adjusting the rankings of the set of result items by a method of using human suggested rankings comprising: retrieving from a database previously suggested rankings of result items that were provided by other users; and adjusting the rankings of the set of result items based on the retrieved previously suggested rankings of result items that were provided by the other users to obtain an adjusted rankings of the set of result items.
- the method includes displaying the adjusted rankings of the set of result items in a results document, wherein the user may enter suggested rankings for zero, one, or more result items in the results document, such suggested rankings to be stored in the database to be available for potential later use.
- One aspect of the invention is directed to a method that includes receiving at a computer system a search query from a user.
- the method includes performing a search over a set of data in response to the search query.
- the method includes generating a rankings of a set of result items from the search over the set of data.
- the method includes adjusting the rankings of the set of result items by a method of using human suggested rankings comprising: retrieving from a database previously suggested rankings of result items that were provided by other users; the previously suggested rankings of result items containing suggested rankings for some of the result items in the set of result items; the other users having conducted a search using the same search query earlier in time; the other users having entered the suggested rankings of result items after being displayed with the results of their search using the same search query earlier in time; and adjusting the rankings of the set of result items based on the retrieved previously suggested rankings of result items that were provided by the other users to obtain an adjusted rankings of the set of result items.
- the method includes displaying the adjusted rankings of the set of result items in a results document, wherein the user may enter suggested rankings for zero, one, or more result items in the results document, such suggested rankings to be stored in the database to be available for potential later use in adjusting the rankings of a later set of result items returned in response to a later search queries of a later users.
- One aspect of the invention is directed to adjusting the rankings of the set of result items based on the retrieved previously suggested rankings of result items from the other users to obtain an adjusted rankings of the set of result items by a method comprising: re-ranking one or more result items in the set of result items to the last user-suggested ranking the one or more result items received from the other users, the other users having conducted a search using the same search query earlier in time.
- One aspect of the invention is directed to adjusting the rankings of the set of result items based on the retrieved previously suggested rankings of result items from the other users to obtain an adjusted rankings of the set of result items by a method comprising: re-ranking one or more result items in the set of result items to an average user-suggested ranking the one or more result items have received from the other users, the other users having conducted a search using the same search query earlier in time.
- One aspect of the invention is directed to adjusting the rankings of the set of result items based on the retrieved previously suggested rankings of result items from the other users to obtain an adjusted rankings of the set of result items by a method comprising: assigning two weights, the first weight being determined based on user-suggested rankings one or more result items in the set of result items have received from the other users, the other users having conducted a search using the same search query earlier in time, and the second weight being determined by the content, link structure, relevancy, quality, or other substantive qualities of one or more result items in the set of result items, and the two weights being used in combination to adjust the rankings of the one or more result items in the set of result items.
- the invention improves on the prior art in at least the following ways: it may improve the quality of rankings of the set of result items returned as a result of a user search so that more relevant result items are ranked higher and less relevant result items are ranked lower; and it allows users to improve the quality of rankings of the set of result items for later users.
- FIG. 1 is a diagram illustrating an exemplary environment in which one embodiment of the present invention may operate
- FIG. 2 is a flow diagram illustrating operations that may be performed for human ranking of search results in one embodiment of the present invention
- FIG. 3 is a diagram illustrating an exemplary results document that could be produced in accordance with one embodiment of the present invention.
- FIG. 1 is a block diagram illustrating an exemplary network environment for implementation of one embodiment of the present invention.
- the network environment 100 may include multiple clients 110 connected to one or more servers 120 via a network 140 .
- Network 140 may include a local area network (LAN), a wide area network (WAN), a telephone network, such as the Public Switched Telephone Network (PSTN), an intranet, the Internet, or a combination of networks.
- PSTN Public Switched Telephone Network
- An intranet the Internet
- Internet or a combination of networks.
- Two clients 110 and one server 120 have been illustrated for simplicity, though in practice there may be more or fewer clients and servers.
- Clients 110 may be personal computers, digital assistants, personal digital assistants, cellular phones, mobile phones, smart phones, pagers, digital tablets, laptop computers, Internet appliances, and other processor-based devices.
- Clients may be client computing devices.
- Client computing devices may be personal computers, digital assistants, personal digital assistants, cellular phones, mobile phones, smart phones, pagers, digital tablets, laptop computers, Internet appliances, and other processor-based devices.
- Servers 120 may include entities that process, search, and/or maintain documents or files in a manner consistent with the principles of the invention. Server 120 may be implemented as a number of networked server devices, though it is illustrated as a single entity.
- Server 120 may include a search engine 125 that can receive search queries from clients 110 .
- the search engine 125 may function as a query-based web page search engine.
- the search engine may cause the output, such as the display, of search results on the client 110 .
- the search results may be in the form a results document.
- the results document may contain a set of result items.
- the search engine 125 processes search queries for documents located on the network. It may process queries by searching the content database 127 in which documents and/or index information about documents on the network are stored. It may return a set of documents and/or index information (called the set of result items) and may rank this set of result items according to relevancy and quality.
- the search engine then accesses the user rankings database 128 to access previous user-suggested rankings of result items for the search query.
- a user-suggested ranking of a result item is a ranking of the result item that was suggested by a user.
- the search engine may adjust the results rankings based on these suggested rankings.
- the documents and/or index information about documents in the content database 127 and the user-suggested ranks in the user rankings database 128 may be stored in a single database or in more or fewer databases. Alternatively, they may be stored in a networked system of databases. It is for simplicity of illustration that these elements are described as residing in two different databases.
- the term database as used in this patent may refer to a computer database or any data storage or network of data storage.
- FIG. 2 is a flow chart illustrating an example method 200 that may be performed by search engine 125 consistent with aspects of the invention.
- Search engine 125 may receive a search query from a user (act 201 ). The search query may be received at the server 120 . The user may provide the search query either at the server 120 or at a client 110 that is connected to the server 120 over the network 140 .
- the search engine may perform a search over a set of data, where the set of data is at the computer system or accessible to the computer system (act 202 ). The set of data that is searched over may be the content database 127 .
- the search engine may generate a rankings of a set of result items returned from the search over the set of data (act 203 ).
- the step of generating a rankings of a set of result items may be performed by any conventional or nonconventional method of generating a rankings of a set of result items such as but not limited to: analysis of the content of the document and other documents or analysis of the link structure of the document and other documents.
- the search engine may adjust the rankings of the set of result items by a method of using human suggested rankings (act 204 ). Act 204 may include retrieving from the user rankings database 128 a previously suggested rankings of result items that were provided by a other users.
- the previously suggested rankings of result items contains suggested rankings for zero, one, or more of the result items in the set of result items.
- the previously suggested rankings of result items in the user rankings database 128 may have been provided by the other users, the other users having conducted a search using the same search query earlier in time.
- the previously suggested rankings of result items in the database may have been provided by the other users by entering suggested rankings of result items, the result items having been displayed in a prior results document that was returned after the other users conducted a search using the same search query earlier in time.
- Act 204 may include adjusting the rankings of the set of result items based on the retrieved previously suggested rankings of result items that were provided by the other users to obtain an adjusted rankings of the set of result items.
- the search engine may display the adjusted rankings of the set of result items in a results document (act 205 ).
- the user may enter suggested rankings for zero, one, or more result items in the results document (act 206 ). Such suggested rankings are stored in the user rankings database 128 to be available for potential later use in adjusting the rankings of a later set of result items returned in response to a later search queries of a later users.
- a suggested ranking for a result item in the user rankings database 128 may be a database entry with information regarding: the user who made the user-suggested ranking, the suggested ranking of the result item, the name or identifying information of the result item, the search query used by the user who made the user-suggested ranking to obtain the result item, the date and time that the suggested ranking was entered, and other information.
- a suggested ranking of a result item may be a suggestion that the result item be ranked at a certain ranking.
- a suggested ranking for website A may be a suggestion that the website A be ranked third in a set of result items returned when a user conducts a search using search query B.
- a suggested ranking is not merely an indication that a user recommended or did not recommend a website; is not merely an indication that a user voted up or voted down a website; is not merely a rating of a website on a scale for example from 1 to 5. Rather a suggested ranking indicates that a user made a suggestion that the result item be ranked at a certain ranking in the set of result items.
- search query is defined as a search query that is similar or substantially the same. Two search queries need not be exactly identical to be the “same search query.”
- a search query of “Michael Jordan” may be the “same search query” as “the Michael Jordan,” “person Michael Jordan,” “Michael Jordan basketball,” “Michael J basketball player,” and so on in some embodiments of the invention.
- the step in act 204 of adjusting the rankings of the set of result items based on the retrieved previously suggested rankings of result items that were provided by the other users to obtain an adjusted rankings of the set of result items may be accomplished by a method comprising: re-ranking one or more result items in the set of result items to the last user-suggested ranking the one or more result items received from the other users, the other users having conducted a search using the same search query earlier in time.
- Additional procedures may be used in this adjustment method as follows: When a result item is re-ranked, it may displace a result item that was ranked at that location in the original ranking, the original ranking being the ranking of the result items before the adjustment; said result item is moved down one rank if the re-ranked result item was ranked below it in the original ranking, and said result item is moved up one rank if the re-ranked result item was above it in the original ranking; this movement of said first result item may displace a second result item; the second result item is moved down one rank if the first result item was moved down, and the second result item is moved up if the first result item was moved up; the process is repeated for a third, fourth, fifth, and so on result items as necessary until no result items need to be re-ranked.
- the step in act 204 of adjusting the rankings of the set of result items based on the retrieved previously suggested rankings of result items that were provided by the other users to obtain an adjusted rankings of the set of result items may be accomplished by a method comprising: re-ranking one or more result items in the set of result items to an average user-suggested ranking the one or more result items have received from the other users, the other users having conducted a search using the same search query earlier in time.
- the average may be a mean, median, mode, weighted average, or other kind of average.
- Additional procedures may be used in this adjustment method as follows: When a result item is re-ranked, it may displace a result item that was ranked at that location in the original ranking, the original ranking being the ranking of the result items before the adjustment; said result item is moved down one rank if the re-ranked result item was ranked below it in the original ranking, and said result item is moved up one rank if the re-ranked result item was above it in the original ranking; this movement of said first result item may displace a second result item; the second result item is moved down one rank if the first result item was moved down, and the second result item is moved up if the first result item was moved up; the process is repeated for a third, fourth, fifth, and so on result items as necessary until no result items need to be re-ranked.
- the step in act 204 of adjusting the rankings of the set of result items based on the retrieved previously suggested rankings of result items that were provided by the other users to obtain an adjusted rankings of the set of result items may be accomplished by a method comprising: assigning two weights, the first weight being determined based on user-suggested rankings one or more result items in the set of result items have received from the other users, the other users having conducted a search using the same search query earlier in time, and the second weight being determined by the content, link structure, relevancy, quality, or other substantive qualities of one or more result items in the set of result items, and the two weights being used in combination to adjust the rankings of the one or more result items in the set of result items.
- Additional procedures may be used in this adjustment method as follows: When a result item is re-ranked, it may displace a result item that was ranked at that location in the original ranking, the original ranking being the ranking of the result items before the adjustment; said result item is moved down one rank if the re-ranked result item was ranked below it in the original ranking, and said result item is moved up one rank if the re-ranked result item was above it in the original ranking; this movement of said first result item may displace a second result item; the second result item is moved down one rank if the first result item was moved down, and the second result item is moved up if the first result item was moved up; the process is repeated for a third, fourth, fifth, and so on result items as necessary until no result items need to be re-ranked.
- adjusting the rankings of the search items according to user-suggested rankings may be accomplished by retrieving the rankings for all or less than all of the items from the user rankings database.
- act 204 may be performed according to many different methods and processes.
- the foregoing description of preferred embodiments of act 204 provides illustration and description, but is not intended to be exhaustive or to limit the implementation of act 204 to the precise forms disclosed. Modifications and variations are possible in light of the above teachings or may be acquired from practice of the invention. For example, although many of the operations described above were described in a particular order, many of the operations are amenable to being performed simultaneously or in different orders to still achieve the same or equivalent results.
- the user may not suggest rankings of items unless the user has a user account and is logged in to the user account. In one embodiment, the user may not suggest rankings of items unless the user has editing permissions. Editing permissions may be given to all or less than all of the users of the invention. Editing permissions may be given or taken away from a user at different times. Some users may receive editing permissions when they create a user account. Other users may not receive editing permissions when they create a user account. There may be different levels of editing permissions.
- the user must pass a test to show he or she is a human and not a computer program before the user can enter suggested rankings for the result items in the results document.
- the user may only enter suggested rankings for the result items in the results document a limited number of times over a limited time period.
- the user and the other users are divided into one or more classes.
- the user and the other users may be in multiple classes at once.
- the user may be in a class or more than one class.
- suggested rankings entered by a user are only used to adjust rankings made in response to another user's search query if the two users are of the same class, for example both users being of the class “interested in sports.”
- users may be assigned to classes in many ways and not just according to interest area. Other kinds of classes may be: users who are female, users who are over the age of 25, users who have a college education, users who work in the computer software industry, and so on. It may be appreciated that classes may be automatically assigned, assigned by running a computer program, assigned by manual intervention, or specified by the user.
- act 204 may be performed by a method comprising: retrieving from the database a previously suggested rankings of result items that were provided by a other users; wherein the previously suggested rankings of result items contains suggested rankings for zero, one, or more of the result items in the set of result items; wherein the previously suggested rankings of result items in the database were provided by the other users, the other users having conducted a search using the same search query earlier in time, the other users being in the same class as the user; wherein the previously suggested rankings of result items in the database were provided by the other users by entering suggested rankings of result items, the result items having been displayed in a prior results document that was returned after the other users conducted a search using the same search query earlier in time; adjusting the rankings of the set of result items based on the retrieved previously suggested rankings of result items that were provided by the other users to obtain an adjusted rankings of the set of result items.
- FIG. 3 is a diagram illustrating an exemplary results document 300 displaying the ranked results returned in response to a search query.
- the result items on results document 300 are exemplary result items returned by search engine 125 for the exemplary search “Michael Jordan.”
- each result item may be listed with a title 301 , a summary of relevant content 302 , and URL 303 .
- the title 301 may be linked to the document.
- the result items may be listed with more, less, or different information.
- the user may enter suggested rankings for one or more result items listed on the results document by selecting and moving the one or more result items to a higher or lower ranking. In one embodiment, this may be accomplished by clicking on the title 301 , summary of relevant content 302 , or URL 303 to select the result item and then moving the result item. In one embodiment, some or all result items may be accompanied by a move button 304 such that the user may click the move button and then drag the item to a new location in the list that corresponds to a new ranking. Selecting and moving the one or more result items to a higher or lower ranking can be accomplished in many ways, using many different kinds of user interface, and the above examples of ways to select and move a result item are exemplary and not limiting.
- the new user-suggested ranking may be stored as a new user-suggested ranking in the user rankings database 128 .
- the results document includes user interface elements next to one or more result items.
- the user may enter suggested rankings for the one or more result items via the user interface elements.
- some or all result items may be accompanied by a text input box where the user may enter a new suggested ranking.
- the new user-suggested ranking may be stored as a new user-suggested ranking in the user rankings database 128 .
- a merge component 310 is included on the results document.
- the search query received from the user is one search query from a set of merged search queries
- adjusting the rankings of the set of result items by a method of using human suggested rankings is performed by a method comprising: retrieving from the database a previously suggested rankings of result items that were provided by a other users; wherein the previously suggested rankings of result items contains suggested rankings for zero, one, or more of the result items in the set of result items; wherein the previously suggested rankings of result items in the database were provided by the other users, the other users having conducted a search using any search query from the set of merged search queries earlier in time; wherein the previously suggested rankings of result items in the database were provided by the other users by entering suggested rankings of result items, the result items having been displayed in a prior results document that was returned after the other users conducted a search using any search query from the set of merged search queries earlier in time; adjusting the rankings of the set of result items based on the retrieved previously suggested rankings of result items that were
- the results document may include a merge component 310 that displays information about one or more sets of merged search queries or a link to a merge component 310 that displays information about one or more sets of merged search queries.
- the results document may include user interface elements that may be used to add one or more search queries to a set of merged search queries or to remove one or more search queries from a set of merged search queries.
- a text input field 311 allows entering a search query and a merge button 312 causes that search query to be added to the set of merged search queries 313 .
- the set of merged search queries may include the search query submitted by the user to generate the current results document.
- Each search query in the set of merged search queries may return the same results document, and new user suggested rankings of items on the results document may be used to adjust the rankings of items when searching for any of the merged search queries.
- An unmerge button 314 next to each currently merged search query may, when clicked, cause the search query to be removed from the set of merged search queries.
- the results document includes a version control component 320 that displays information about one or more previous versions of the results document or a link to a version control component 320 that displays information about one or more previous versions of the results document.
- Each of the one or more previous versions of the results document may display a past adjusted rankings of the set of result items, the past adjusted rankings of the set of result items being generated at the current time or having been generated at a previous time by a method of using human suggested rankings comprising: retrieving from the database a previously suggested rankings of result items that were provided by a other users; wherein the previously suggested rankings of result items contains suggested rankings for zero, one, or more of the result items in the set of result items; wherein the previously suggested rankings of result items in the database were provided by the other users, the other users having conducted a search using the same search query earlier in time; wherein the previously suggested rankings of result items in the database were provided by the other users by entering suggested rankings of result items, the result items having been displayed in a prior results document that was returned after the other users conducted a search using
- the component may include information about the current version of the results document in a current version display 321 .
- the component may include date information 322 , time information 323 , and user information 324 for previous versions of the results document, a previous version of the results document being a document displaying the ranking of the set of result items adjusted according to the user-suggested rankings of items provided by users up to a specific time that is earlier than the current time, the specific time being associated with the previous version of the results document.
- the date information 322 and time information 323 may reflect the specific earlier time that is associated with the previous version of the results document.
- the user information 324 may reflect the last user to enter a user suggested ranking before or at that specific time.
- Each of the current and previous version listings may include a first checkbox 325 and a second checkbox 326 allowing the selection of two previous versions of the results document simultaneously.
- the user may choose to view a differences between two or more previous versions of the results document.
- a display of the differences between the two selected previous versions may appear when the user clicks the compare button 327 .
- the show button 328 may cause a previous version to be displayed, the user choosing which previous version to display using the first checkbox 325 or another interface element.
- the results document includes a discussion component 330 for user discussion or a link to a discussion component 330 for user discussion, the discussion component for user discussion comprising a wiki, a forum, a bulletin board, a chat room, or other interface component configured for user discussion.
- users may receive rankings of result items that are more accurate with respect to the rankings that users believe the result items should have.
Abstract
A search engine may perform a search for a user search query. A rankings of result items may be returned as a result of the search. The user may enter new suggested rankings for some of the items. The new suggested rankings may change the rankings of result items when other users search for the same search query at a later time.
Description
- This application claims priority from U.S. provisional patent application Ser. No. 61/299,994 filed Jan. 31, 2010 and U.S. provisional patent application Ser. No. 61/302,117 filed Feb. 6, 2010, which are both incorporated herein by reference.
- The present invention relates generally to search engines, and more particularly, to methods of ranking search results and to user interfaces that are used to display search results.
- Search engines assist users in finding relevant documents or other informational content from among a larger corpus. A common application of search engines is to assist users in finding information, documents, and files that are located on a network. A common application of search engines is to assist users in finding information, documents, and files on the World Wide Web (“Web”), which comprises a massive network of Web documents. The user submits a word or phrase he or she is interested in, called a search query, and the search engine seeks to provide a list of high quality Web documents that are relevant to that query. The list of Web documents is usually ranked with higher-quality and more relevant documents appearing first.
- The ranking is conventionally done either by examination of the contents of the Web documents or by examining the link structure of the Web documents. For example, a search engine may examine the words on a web document and assign an importance ranking according to how similar or related the words on the web document are to the search query. For example, a search engine may examine the link structure of a web document and assign an importance ranking according to which other documents link to the document and which other documents the document links to. For example, a search engine may assign an importance ranking to a document based on the user's search history. For example, a search engine may assign an importance ranking to a document based on whether the user has previously navigated to and made a purchase from that document.
- One problem with search engines is that the methods of ranking documents often do not produce rankings that users believe to be an accurate reflection of relevance or quality. Often, a Web document that users believe to be the highest quality and most relevant is not listed first or second or anywhere near the top of the results list. A second problem is that users who notice this problem do not have a means to fix it so that future users who search for the same query can receive better search results.
- One aspect of the invention is directed to a method that includes receiving at a computer system a search query from a user. The method includes performing a search over a set of data in response to the search query. The method includes generating a rankings of a set of result items from the search over the set of data. The method includes adjusting the rankings of the set of result items by a method of using human suggested rankings comprising: retrieving from a database previously suggested rankings of result items that were provided by other users; and adjusting the rankings of the set of result items based on the retrieved previously suggested rankings of result items that were provided by the other users to obtain an adjusted rankings of the set of result items. The method includes displaying the adjusted rankings of the set of result items in a results document, wherein the user may enter suggested rankings for zero, one, or more result items in the results document, such suggested rankings to be stored in the database to be available for potential later use.
- One aspect of the invention is directed to a method that includes receiving at a computer system a search query from a user. The method includes performing a search over a set of data in response to the search query. The method includes generating a rankings of a set of result items from the search over the set of data. The method includes adjusting the rankings of the set of result items by a method of using human suggested rankings comprising: retrieving from a database previously suggested rankings of result items that were provided by other users; the previously suggested rankings of result items containing suggested rankings for some of the result items in the set of result items; the other users having conducted a search using the same search query earlier in time; the other users having entered the suggested rankings of result items after being displayed with the results of their search using the same search query earlier in time; and adjusting the rankings of the set of result items based on the retrieved previously suggested rankings of result items that were provided by the other users to obtain an adjusted rankings of the set of result items. The method includes displaying the adjusted rankings of the set of result items in a results document, wherein the user may enter suggested rankings for zero, one, or more result items in the results document, such suggested rankings to be stored in the database to be available for potential later use in adjusting the rankings of a later set of result items returned in response to a later search queries of a later users.
- One aspect of the invention is directed to adjusting the rankings of the set of result items based on the retrieved previously suggested rankings of result items from the other users to obtain an adjusted rankings of the set of result items by a method comprising: re-ranking one or more result items in the set of result items to the last user-suggested ranking the one or more result items received from the other users, the other users having conducted a search using the same search query earlier in time.
- One aspect of the invention is directed to adjusting the rankings of the set of result items based on the retrieved previously suggested rankings of result items from the other users to obtain an adjusted rankings of the set of result items by a method comprising: re-ranking one or more result items in the set of result items to an average user-suggested ranking the one or more result items have received from the other users, the other users having conducted a search using the same search query earlier in time.
- One aspect of the invention is directed to adjusting the rankings of the set of result items based on the retrieved previously suggested rankings of result items from the other users to obtain an adjusted rankings of the set of result items by a method comprising: assigning two weights, the first weight being determined based on user-suggested rankings one or more result items in the set of result items have received from the other users, the other users having conducted a search using the same search query earlier in time, and the second weight being determined by the content, link structure, relevancy, quality, or other substantive qualities of one or more result items in the set of result items, and the two weights being used in combination to adjust the rankings of the one or more result items in the set of result items.
- The invention improves on the prior art in at least the following ways: it may improve the quality of rankings of the set of result items returned as a result of a user search so that more relevant result items are ranked higher and less relevant result items are ranked lower; and it allows users to improve the quality of rankings of the set of result items for later users.
-
FIG. 1 is a diagram illustrating an exemplary environment in which one embodiment of the present invention may operate; -
FIG. 2 is a flow diagram illustrating operations that may be performed for human ranking of search results in one embodiment of the present invention; -
FIG. 3 is a diagram illustrating an exemplary results document that could be produced in accordance with one embodiment of the present invention. -
FIG. 1 is a block diagram illustrating an exemplary network environment for implementation of one embodiment of the present invention. Thenetwork environment 100 may includemultiple clients 110 connected to one ormore servers 120 via anetwork 140.Network 140 may include a local area network (LAN), a wide area network (WAN), a telephone network, such as the Public Switched Telephone Network (PSTN), an intranet, the Internet, or a combination of networks. Twoclients 110 and oneserver 120 have been illustrated for simplicity, though in practice there may be more or fewer clients and servers. -
Clients 110 may be personal computers, digital assistants, personal digital assistants, cellular phones, mobile phones, smart phones, pagers, digital tablets, laptop computers, Internet appliances, and other processor-based devices. Clients may be client computing devices. Client computing devices may be personal computers, digital assistants, personal digital assistants, cellular phones, mobile phones, smart phones, pagers, digital tablets, laptop computers, Internet appliances, and other processor-based devices.Servers 120 may include entities that process, search, and/or maintain documents or files in a manner consistent with the principles of the invention.Server 120 may be implemented as a number of networked server devices, though it is illustrated as a single entity. -
Server 120 may include asearch engine 125 that can receive search queries fromclients 110. In one embodiment, thesearch engine 125 may function as a query-based web page search engine. The search engine may cause the output, such as the display, of search results on theclient 110. The search results may be in the form a results document. The results document may contain a set of result items. - In one embodiment, the
search engine 125 processes search queries for documents located on the network. It may process queries by searching thecontent database 127 in which documents and/or index information about documents on the network are stored. It may return a set of documents and/or index information (called the set of result items) and may rank this set of result items according to relevancy and quality. - In one embodiment, the search engine then accesses the
user rankings database 128 to access previous user-suggested rankings of result items for the search query. A user-suggested ranking of a result item is a ranking of the result item that was suggested by a user. The search engine may adjust the results rankings based on these suggested rankings. - Those skilled in the art will appreciate that the documents and/or index information about documents in the
content database 127 and the user-suggested ranks in theuser rankings database 128 may be stored in a single database or in more or fewer databases. Alternatively, they may be stored in a networked system of databases. It is for simplicity of illustration that these elements are described as residing in two different databases. The term database as used in this patent may refer to a computer database or any data storage or network of data storage. -
FIG. 2 is a flow chart illustrating anexample method 200 that may be performed bysearch engine 125 consistent with aspects of the invention.Search engine 125 may receive a search query from a user (act 201). The search query may be received at theserver 120. The user may provide the search query either at theserver 120 or at aclient 110 that is connected to theserver 120 over thenetwork 140. In response, the search engine may perform a search over a set of data, where the set of data is at the computer system or accessible to the computer system (act 202). The set of data that is searched over may be thecontent database 127. The search engine may generate a rankings of a set of result items returned from the search over the set of data (act 203). The step of generating a rankings of a set of result items may be performed by any conventional or nonconventional method of generating a rankings of a set of result items such as but not limited to: analysis of the content of the document and other documents or analysis of the link structure of the document and other documents. The search engine may adjust the rankings of the set of result items by a method of using human suggested rankings (act 204). Act 204 may include retrieving from the user rankings database 128 a previously suggested rankings of result items that were provided by a other users. The previously suggested rankings of result items contains suggested rankings for zero, one, or more of the result items in the set of result items. The previously suggested rankings of result items in theuser rankings database 128 may have been provided by the other users, the other users having conducted a search using the same search query earlier in time. The previously suggested rankings of result items in the database may have been provided by the other users by entering suggested rankings of result items, the result items having been displayed in a prior results document that was returned after the other users conducted a search using the same search query earlier in time. Act 204 may include adjusting the rankings of the set of result items based on the retrieved previously suggested rankings of result items that were provided by the other users to obtain an adjusted rankings of the set of result items. The search engine may display the adjusted rankings of the set of result items in a results document (act 205). The user may enter suggested rankings for zero, one, or more result items in the results document (act 206). Such suggested rankings are stored in theuser rankings database 128 to be available for potential later use in adjusting the rankings of a later set of result items returned in response to a later search queries of a later users. - A suggested ranking for a result item in the
user rankings database 128 may be a database entry with information regarding: the user who made the user-suggested ranking, the suggested ranking of the result item, the name or identifying information of the result item, the search query used by the user who made the user-suggested ranking to obtain the result item, the date and time that the suggested ranking was entered, and other information. A suggested ranking of a result item may be a suggestion that the result item be ranked at a certain ranking. For example, a suggested ranking for website A may be a suggestion that the website A be ranked third in a set of result items returned when a user conducts a search using search query B. A suggested ranking is not merely an indication that a user recommended or did not recommend a website; is not merely an indication that a user voted up or voted down a website; is not merely a rating of a website on a scale for example from 1 to 5. Rather a suggested ranking indicates that a user made a suggestion that the result item be ranked at a certain ranking in the set of result items. - In this patent, the term “same search query” is defined as a search query that is similar or substantially the same. Two search queries need not be exactly identical to be the “same search query.” For example, a search query of “Michael Jordan” may be the “same search query” as “the Michael Jordan,” “person Michael Jordan,” “Michael Jordan basketball,” “Michael J basketball player,” and so on in some embodiments of the invention.
- In one embodiment, the step in
act 204 of adjusting the rankings of the set of result items based on the retrieved previously suggested rankings of result items that were provided by the other users to obtain an adjusted rankings of the set of result items may be accomplished by a method comprising: re-ranking one or more result items in the set of result items to the last user-suggested ranking the one or more result items received from the other users, the other users having conducted a search using the same search query earlier in time. Additional procedures may be used in this adjustment method as follows: When a result item is re-ranked, it may displace a result item that was ranked at that location in the original ranking, the original ranking being the ranking of the result items before the adjustment; said result item is moved down one rank if the re-ranked result item was ranked below it in the original ranking, and said result item is moved up one rank if the re-ranked result item was above it in the original ranking; this movement of said first result item may displace a second result item; the second result item is moved down one rank if the first result item was moved down, and the second result item is moved up if the first result item was moved up; the process is repeated for a third, fourth, fifth, and so on result items as necessary until no result items need to be re-ranked. Other procedures may be used to deal with the situation in which a result item that is re-ranked displaces a result item that was ranked at that location in the original ranking. For example, the result item that was ranked at that location in the original ranking could be deleted from the set of result items. - In one embodiment, the step in
act 204 of adjusting the rankings of the set of result items based on the retrieved previously suggested rankings of result items that were provided by the other users to obtain an adjusted rankings of the set of result items may be accomplished by a method comprising: re-ranking one or more result items in the set of result items to an average user-suggested ranking the one or more result items have received from the other users, the other users having conducted a search using the same search query earlier in time. The average may be a mean, median, mode, weighted average, or other kind of average. Additional procedures may be used in this adjustment method as follows: When a result item is re-ranked, it may displace a result item that was ranked at that location in the original ranking, the original ranking being the ranking of the result items before the adjustment; said result item is moved down one rank if the re-ranked result item was ranked below it in the original ranking, and said result item is moved up one rank if the re-ranked result item was above it in the original ranking; this movement of said first result item may displace a second result item; the second result item is moved down one rank if the first result item was moved down, and the second result item is moved up if the first result item was moved up; the process is repeated for a third, fourth, fifth, and so on result items as necessary until no result items need to be re-ranked. Other procedures may be used to deal with the situation in which a result item that is re-ranked displaces a result item that was ranked at that location in the original ranking. For example, the result item that was ranked at that location in the original ranking could be deleted from the set of result items. - In one embodiment, the step in
act 204 of adjusting the rankings of the set of result items based on the retrieved previously suggested rankings of result items that were provided by the other users to obtain an adjusted rankings of the set of result items may be accomplished by a method comprising: assigning two weights, the first weight being determined based on user-suggested rankings one or more result items in the set of result items have received from the other users, the other users having conducted a search using the same search query earlier in time, and the second weight being determined by the content, link structure, relevancy, quality, or other substantive qualities of one or more result items in the set of result items, and the two weights being used in combination to adjust the rankings of the one or more result items in the set of result items. Additional procedures may be used in this adjustment method as follows: When a result item is re-ranked, it may displace a result item that was ranked at that location in the original ranking, the original ranking being the ranking of the result items before the adjustment; said result item is moved down one rank if the re-ranked result item was ranked below it in the original ranking, and said result item is moved up one rank if the re-ranked result item was above it in the original ranking; this movement of said first result item may displace a second result item; the second result item is moved down one rank if the first result item was moved down, and the second result item is moved up if the first result item was moved up; the process is repeated for a third, fourth, fifth, and so on result items as necessary until no result items need to be re-ranked. Other procedures may be used to deal with the situation in which a result item that is re-ranked displaces a result item that was ranked at that location in the original ranking. For example, the result item that was ranked at that location in the original ranking could be deleted from the set of result items. - In one embodiment, when a user suggests a new ranking for an item, the item is re-ranked to the suggested ranking, the other items are displaced accordingly, and the rankings of all or less than all of the items in the search results list are stored in the user rankings database. In this embodiment, adjusting the rankings of the search items according to user-suggested rankings may be accomplished by retrieving the rankings for all or less than all of the items from the user rankings database.
- Those skilled in the art will recognize that adjusting the rankings of the search items according to user suggested rankings (act 204) may be performed according to many different methods and processes. The foregoing description of preferred embodiments of
act 204 provides illustration and description, but is not intended to be exhaustive or to limit the implementation ofact 204 to the precise forms disclosed. Modifications and variations are possible in light of the above teachings or may be acquired from practice of the invention. For example, although many of the operations described above were described in a particular order, many of the operations are amenable to being performed simultaneously or in different orders to still achieve the same or equivalent results. - In one embodiment, the user may not suggest rankings of items unless the user has a user account and is logged in to the user account. In one embodiment, the user may not suggest rankings of items unless the user has editing permissions. Editing permissions may be given to all or less than all of the users of the invention. Editing permissions may be given or taken away from a user at different times. Some users may receive editing permissions when they create a user account. Other users may not receive editing permissions when they create a user account. There may be different levels of editing permissions.
- In one embodiment, the user must pass a test to show he or she is a human and not a computer program before the user can enter suggested rankings for the result items in the results document.
- In one embodiment, the user may only enter suggested rankings for the result items in the results document a limited number of times over a limited time period.
- In one embodiment, the user and the other users are divided into one or more classes. The user and the other users may be in multiple classes at once. The user may be in a class or more than one class. In this embodiment, suggested rankings entered by a user are only used to adjust rankings made in response to another user's search query if the two users are of the same class, for example both users being of the class “interested in sports.” It may be appreciated that users may be assigned to classes in many ways and not just according to interest area. Other kinds of classes may be: users who are female, users who are over the age of 25, users who have a college education, users who work in the computer software industry, and so on. It may be appreciated that classes may be automatically assigned, assigned by running a computer program, assigned by manual intervention, or specified by the user. In this embodiment where the user and the other users are divided into one or more classes, act 204 may be performed by a method comprising: retrieving from the database a previously suggested rankings of result items that were provided by a other users; wherein the previously suggested rankings of result items contains suggested rankings for zero, one, or more of the result items in the set of result items; wherein the previously suggested rankings of result items in the database were provided by the other users, the other users having conducted a search using the same search query earlier in time, the other users being in the same class as the user; wherein the previously suggested rankings of result items in the database were provided by the other users by entering suggested rankings of result items, the result items having been displayed in a prior results document that was returned after the other users conducted a search using the same search query earlier in time; adjusting the rankings of the set of result items based on the retrieved previously suggested rankings of result items that were provided by the other users to obtain an adjusted rankings of the set of result items.
-
FIG. 3 is a diagram illustrating an exemplary results document 300 displaying the ranked results returned in response to a search query. The result items on results document 300 are exemplary result items returned bysearch engine 125 for the exemplary search “Michael Jordan.” In one embodiment, each result item may be listed with atitle 301, a summary ofrelevant content 302, and URL 303. Thetitle 301 may be linked to the document. In other embodiments, the result items may be listed with more, less, or different information. - In one embodiment, the user may enter suggested rankings for one or more result items listed on the results document by selecting and moving the one or more result items to a higher or lower ranking. In one embodiment, this may be accomplished by clicking on the
title 301, summary ofrelevant content 302, or URL 303 to select the result item and then moving the result item. In one embodiment, some or all result items may be accompanied by amove button 304 such that the user may click the move button and then drag the item to a new location in the list that corresponds to a new ranking. Selecting and moving the one or more result items to a higher or lower ranking can be accomplished in many ways, using many different kinds of user interface, and the above examples of ways to select and move a result item are exemplary and not limiting. The new user-suggested ranking may be stored as a new user-suggested ranking in theuser rankings database 128. - In one embodiment, the results document includes user interface elements next to one or more result items. The user may enter suggested rankings for the one or more result items via the user interface elements. In one embodiment, some or all result items may be accompanied by a text input box where the user may enter a new suggested ranking. The new user-suggested ranking may be stored as a new user-suggested ranking in the
user rankings database 128. - Those skilled in the art will recognize that many different kinds of user interface actions may be used to allow the user to enter a new user-suggested ranking and cause that ranking to be stored in the
user rankings database 128. The foregoing description of preferred embodiments provides illustration and description, but is not intended to be exhaustive or to limit implementation to the precise forms disclosed. Modifications and variations are possible in light of the above teachings or may be acquired from practice of the invention. - In one embodiment, a
merge component 310 is included on the results document. In one embodiment, the search query received from the user is one search query from a set of merged search queries, and adjusting the rankings of the set of result items by a method of using human suggested rankings is performed by a method comprising: retrieving from the database a previously suggested rankings of result items that were provided by a other users; wherein the previously suggested rankings of result items contains suggested rankings for zero, one, or more of the result items in the set of result items; wherein the previously suggested rankings of result items in the database were provided by the other users, the other users having conducted a search using any search query from the set of merged search queries earlier in time; wherein the previously suggested rankings of result items in the database were provided by the other users by entering suggested rankings of result items, the result items having been displayed in a prior results document that was returned after the other users conducted a search using any search query from the set of merged search queries earlier in time; adjusting the rankings of the set of result items based on the retrieved previously suggested rankings of result items that were provided by the other users to obtain an adjusted rankings of the set of result items. - The results document may include a
merge component 310 that displays information about one or more sets of merged search queries or a link to amerge component 310 that displays information about one or more sets of merged search queries. The results document may include user interface elements that may be used to add one or more search queries to a set of merged search queries or to remove one or more search queries from a set of merged search queries. In one embodiment of themerge component 310, atext input field 311 allows entering a search query and amerge button 312 causes that search query to be added to the set of merged search queries 313. The set of merged search queries may include the search query submitted by the user to generate the current results document. Each search query in the set of merged search queries may return the same results document, and new user suggested rankings of items on the results document may be used to adjust the rankings of items when searching for any of the merged search queries. Anunmerge button 314 next to each currently merged search query may, when clicked, cause the search query to be removed from the set of merged search queries. - In one embodiment, the results document includes a
version control component 320 that displays information about one or more previous versions of the results document or a link to aversion control component 320 that displays information about one or more previous versions of the results document. Each of the one or more previous versions of the results document may display a past adjusted rankings of the set of result items, the past adjusted rankings of the set of result items being generated at the current time or having been generated at a previous time by a method of using human suggested rankings comprising: retrieving from the database a previously suggested rankings of result items that were provided by a other users; wherein the previously suggested rankings of result items contains suggested rankings for zero, one, or more of the result items in the set of result items; wherein the previously suggested rankings of result items in the database were provided by the other users, the other users having conducted a search using the same search query earlier in time; wherein the previously suggested rankings of result items in the database were provided by the other users by entering suggested rankings of result items, the result items having been displayed in a prior results document that was returned after the other users conducted a search using the same search query earlier in time; wherein the other users entered the suggested rankings of result items prior to a specific time, the specific time being associated with the one or more previous versions of the results document; adjusting the rankings of the set of result items based on the retrieved previously suggested rankings of result items that were provided by the other users to obtain an adjusted rankings of the set of result items. - In one embodiment of the
version control component 320, the component may include information about the current version of the results document in acurrent version display 321. The component may includedate information 322,time information 323, anduser information 324 for previous versions of the results document, a previous version of the results document being a document displaying the ranking of the set of result items adjusted according to the user-suggested rankings of items provided by users up to a specific time that is earlier than the current time, the specific time being associated with the previous version of the results document. Thedate information 322 andtime information 323 may reflect the specific earlier time that is associated with the previous version of the results document. Theuser information 324 may reflect the last user to enter a user suggested ranking before or at that specific time. Each of the current and previous version listings may include afirst checkbox 325 and asecond checkbox 326 allowing the selection of two previous versions of the results document simultaneously. The user may choose to view a differences between two or more previous versions of the results document. A display of the differences between the two selected previous versions may appear when the user clicks the comparebutton 327. Theshow button 328 may cause a previous version to be displayed, the user choosing which previous version to display using thefirst checkbox 325 or another interface element. - In one embodiment, the results document includes a
discussion component 330 for user discussion or a link to adiscussion component 330 for user discussion, the discussion component for user discussion comprising a wiki, a forum, a bulletin board, a chat room, or other interface component configured for user discussion. - By adjusting search result rankings with user-suggested rankings, users may receive rankings of result items that are more accurate with respect to the rankings that users believe the result items should have.
- The foregoing description of preferred embodiments of the present invention provides illustration and description, but is not intended to be exhaustive or to limit the invention to the precise forms disclosed. Modifications and variations are possible in light of the above teachings or may be acquired from practice of the invention. For example, although many of the operations described above were described in a particular order, many of the operations are amenable to being performed simultaneously or in different orders to still achieve the same or equivalent results.
Claims (18)
1. A method for human ranking of search engine results, the method performed by a computer system having access to a database, the method comprising:
receiving at the computer system a search query from a user;
wherein the search query is provided by the user either at the computer system or at a client computing device connected to the computer system over a network;
performing a search over a set of data in response to the search query, the set of data being at the computer system or accessible to the computer system;
generating a rankings of a set of result items returned from the search over the set of data;
adjusting the rankings of the set of result items by a method of using human suggested rankings comprising:
retrieving from the database a previously suggested rankings of result items that were provided by a other users;
wherein the previously suggested rankings of result items contains suggested rankings for zero, one, or more of the result items in the set of result items;
wherein the previously suggested rankings of result items in the database were provided by the other users, the other users having conducted a search using the same search query earlier in time;
wherein the previously suggested rankings of result items in the database were provided by the other users by entering suggested rankings of result items, the result items having been displayed in a prior results document that was returned after the other users conducted a search using the same search query earlier in time;
adjusting the rankings of the set of result items based on the retrieved previously suggested rankings of result items that were provided by the other users to obtain an adjusted rankings of the set of result items;
displaying the adjusted rankings of the set of result items in a results document, wherein the user may enter suggested rankings for zero, one, or more result items in the results document, such suggested rankings to be stored in the database to be available for potential later use in adjusting the rankings of a later set of result items returned in response to a later search queries of a later users.
2. The method of claim 1 , wherein adjusting the rankings of the set of result items based on the retrieved previously suggested rankings of result items from the other users to obtain an adjusted rankings of the set of result items is performed by a method comprising: re-ranking one or more result items in the set of result items to the last user-suggested ranking the one or more result items received from the other users, the other users having conducted a search using the same search query earlier in time.
3. The method of claim 1 , wherein adjusting the rankings of the set of result items based on the retrieved previously suggested rankings of result items from the other users to obtain an adjusted rankings of the set of result items is performed by a method comprising: re-ranking one or more result items in the set of result items to an average user-suggested ranking the one or more result items have received from the other users, the other users having conducted a search using the same search query earlier in time.
4. The method of claim 1 , wherein adjusting the rankings of the set of result items based on the retrieved previously suggested rankings of result items from the other users to obtain an adjusted rankings of the set of result items is performed by a method comprising: assigning two weights, the first weight being determined based on user-suggested rankings one or more result items in the set of result items have received from the other users, the other users having conducted a search using the same search query earlier in time, and the second weight being determined by the content, link structure, relevancy, quality, or other substantive qualities of one or more result items in the set of result items, and the two weights being used in combination to adjust the rankings of the one or more result items in the set of result items.
5. The method of claim 1 , wherein the results document includes a discussion component for user discussion or a link to a discussion component for user discussion, the discussion component for user discussion comprising a wiki, a forum, a bulletin board, a chat room, or other interface component configured for user discussion.
6. The method of claim 1 , wherein the results document includes a version control component that displays information about one or more previous versions of the results document or a link to a version control component that displays information about one or more previous versions of the results document;
wherein each of the one or more previous versions of the results document display a past adjusted rankings of the set of result items, the past adjusted rankings of the set of result items being generated at the current time or having been generated at a previous time by a method of using human suggested rankings comprising:
retrieving from the database a previously suggested rankings of result items that were provided by a other users;
wherein the previously suggested rankings of result items contains suggested rankings for zero, one, or more of the result items in the set of result items;
wherein the previously suggested rankings of result items in the database were provided by the other users, the other users having conducted a search using the same search query earlier in time;
wherein the previously suggested rankings of result items in the database were provided by the other users by entering suggested rankings of result items, the result items having been displayed in a prior results document that was returned after the other users conducted a search using the same search query earlier in time;
wherein the other users entered the suggested rankings of result items prior to a specific time, the specific time being associated with the one or more previous versions of the results document;
adjusting the rankings of the set of result items based on the retrieved previously suggested rankings of result items that were provided by the other users to obtain an adjusted rankings of the set of result items.
7. The method of claim 6 , wherein the user may choose to view a differences between two or more previous versions of the results document.
8. The method of claim 1 , wherein the results document includes user interface elements next to one or more result items;
wherein the user may enter suggested rankings for the one or more result items via the user interface elements.
9. The method of claim 1 , wherein the user may enter suggested rankings for one or more result items listed on the results document by selecting and moving the one or more result items to a higher or lower ranking.
10. The method of claim 1 , wherein the user may only enter suggested rankings for the result items in the results document if the user has editing permissions.
11. The method of claim 1 , wherein the user must pass a test to show he or she is a human and not a computer program before the user can enter suggested rankings for the result items in the results document.
12. The method of claim 1 , wherein the user may only enter suggested rankings for the result items in the results document a limited number of times over a limited time period.
13. The method of claim 1 , wherein the search query received from the user is one search query from a set of merged search queries;
wherein adjusting the rankings of the set of result items by a method of using human suggested rankings is instead performed by a method comprising:
retrieving from the database a previously suggested rankings of result items that were provided by a other users;
wherein the previously suggested rankings of result items contains suggested rankings for zero, one, or more of the result items in the set of result items;
wherein the previously suggested rankings of result items in the database were provided by the other users, the other users having conducted a search using any search query from the set of merged search queries earlier in time;
wherein the previously suggested rankings of result items in the database were provided by the other users by entering suggested rankings of result items, the result items having been displayed in a prior results document that was returned after the other users conducted a search using any search query from the set of merged search queries earlier in time;
adjusting the rankings of the set of result items based on the retrieved previously suggested rankings of result items that were provided by the other users to obtain an adjusted rankings of the set of result items.
14. The method of claim 13 , wherein the results document includes a merge component that displays information about one or more sets of merged search queries or a link to a merge component that displays information about one or more sets of merged search queries.
15. The method of claim 13 , wherein the results document includes user interface elements that may be used to add one or more search queries to a set of merged search queries or to remove one or more search queries from a set of merged search queries.
16. The method of claim 1 , wherein the user and the other users are divided into one or more classes;
wherein the user is in a class;
wherein adjusting the rankings of the set of result items by a method of using human suggested rankings is instead performed by a method comprising:
retrieving from the database a previously suggested rankings of result items that were provided by a other users;
wherein the previously suggested rankings of result items contains suggested rankings for zero, one, or more of the result items in the set of result items;
wherein the previously suggested rankings of result items in the database were provided by the other users, the other users having conducted a search using the same search query earlier in time, the other users being in the same class as the user;
wherein the previously suggested rankings of result items in the database were provided by the other users by entering suggested rankings of result items, the result items having been displayed in a prior results document that was returned after the other users conducted a search using the same search query earlier in time;
adjusting the rankings of the set of result items based on the retrieved previously suggested rankings of result items that were provided by the other users to obtain an adjusted rankings of the set of result items.
17. A system including at least one computer system having access to a database, the system configured to:
receive at the computer system a search query from a user;
wherein the search query is provided by the user either at the computer system or at a client computing device connected to the computer system over a network;
perform a search over a set of data in response to the search query, the set of data being at the computer system or accessible to the computer system;
generate a rankings of a set of result items returned from the search over the set of data;
adjust the rankings of the set of result items by a method of using human suggested rankings comprising:
retrieving from the database a previously suggested rankings of result items that were provided by a other users;
wherein the previously suggested rankings of result items contains suggested rankings for zero, one, or more of the result items in the set of result items;
wherein the previously suggested rankings of result items in the database were provided by the other users, the other users having conducted a search using the same search query earlier in time;
wherein the previously suggested rankings of result items in the database were provided by the other users by entering suggested rankings of result items, the result items having been displayed in a prior results document that was returned after the other users conducted a search using the same search query earlier in time;
adjusting the rankings of the set of result items based on the retrieved previously suggested rankings of result items that were provided by the other users to obtain an adjusted rankings of the set of result items;
display the adjusted rankings of the set of result items in a results document, wherein the user may enter suggested rankings for zero, one, or more result items in the results document, such suggested rankings to be stored in the database for potential later use in adjusting the rankings of a later set of result items returned in response to a later search queries of a later users.
18. A computer program product, comprising a computer usable medium having a computer readable program code embodied therein, said computer readable program code adapted to be executed to implement a method of human ranking of search engine results, said method using a computer system having access to a database, said method comprising:
receiving at the computer system a search query from a user;
wherein the search query is provided by the user either at the computer system or at a client computing device connected to the computer system over a network;
performing a search over a set of data in response to the search query, the set of data being at the computer system or accessible to the computer system;
generating a rankings of a set of result items returned from the search over the set of data;
adjusting the rankings of the set of result items by a method of using human suggested rankings comprising:
retrieving from the database a previously suggested rankings of result items that were provided by a other users;
wherein the previously suggested rankings of result items contains suggested rankings for zero, one, or more of the result items in the set of result items;
wherein the previously suggested rankings of result items in the database were provided by the other users, the other users having conducted a search using the same search query earlier in time;
wherein the previously suggested rankings of result items in the database were provided by the other users by entering suggested rankings of result items, the result items having been displayed in a prior results document that was returned after the other users conducted a search using the same search query earlier in time;
adjusting the rankings of the set of result items based on the retrieved previously suggested rankings of result items that were provided by the other users to obtain an adjusted rankings of the set of result items;
displaying the adjusted rankings of the set of result items in a results document, wherein the user may enter suggested rankings for zero, one, or more result items in the results document, such suggested rankings to be stored in the database for potential later use in adjusting the rankings of a later set of result items returned in response to a later search queries of a later users.
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US12/944,842 US20110191327A1 (en) | 2010-01-31 | 2010-11-12 | Method for Human Ranking of Search Results |
US13/487,868 US8924376B1 (en) | 2010-01-31 | 2012-06-04 | Method for human ranking of search results |
US14/530,782 US9053158B1 (en) | 2010-01-31 | 2014-11-02 | Method for human ranking of search results |
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US29999410P | 2010-01-31 | 2010-01-31 | |
US30211710P | 2010-02-06 | 2010-02-06 | |
US12/944,842 US20110191327A1 (en) | 2010-01-31 | 2010-11-12 | Method for Human Ranking of Search Results |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/487,868 Continuation-In-Part US8924376B1 (en) | 2010-01-31 | 2012-06-04 | Method for human ranking of search results |
Publications (1)
Publication Number | Publication Date |
---|---|
US20110191327A1 true US20110191327A1 (en) | 2011-08-04 |
Family
ID=44342515
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/944,842 Abandoned US20110191327A1 (en) | 2010-01-31 | 2010-11-12 | Method for Human Ranking of Search Results |
US12/983,363 Expired - Fee Related US8099406B2 (en) | 2010-01-31 | 2011-01-03 | Method for human editing of information in search results |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/983,363 Expired - Fee Related US8099406B2 (en) | 2010-01-31 | 2011-01-03 | Method for human editing of information in search results |
Country Status (1)
Country | Link |
---|---|
US (2) | US20110191327A1 (en) |
Cited By (50)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20100153292A1 (en) * | 2008-12-11 | 2010-06-17 | Microsoft Corporation | Making Friend and Location Recommendations Based on Location Similarities |
US8577963B2 (en) | 2011-06-30 | 2013-11-05 | Amazon Technologies, Inc. | Remote browsing session between client browser and network based browser |
US8589385B2 (en) | 2011-09-27 | 2013-11-19 | Amazon Technologies, Inc. | Historical browsing session management |
US8615431B1 (en) | 2011-09-29 | 2013-12-24 | Amazon Technologies, Inc. | Network content message placement management |
US8627195B1 (en) | 2012-01-26 | 2014-01-07 | Amazon Technologies, Inc. | Remote browsing and searching |
US20140023241A1 (en) * | 2012-07-23 | 2014-01-23 | Toshiba Tec Kabushiki Kaisha | Dictionary registration apparatus and method for adding feature amount data to recognition dictionary |
US8706860B2 (en) | 2011-06-30 | 2014-04-22 | Amazon Technologies, Inc. | Remote browsing session management |
US20140177912A1 (en) * | 2012-10-31 | 2014-06-26 | Toshiba Tec Kabushiki Kaisha | Commodity reading apparatus, commodity sales data processing apparatus and commodity reading method |
US8799412B2 (en) | 2011-06-30 | 2014-08-05 | Amazon Technologies, Inc. | Remote browsing session management |
US8839087B1 (en) * | 2012-01-26 | 2014-09-16 | Amazon Technologies, Inc. | Remote browsing and searching |
US8849802B2 (en) | 2011-09-27 | 2014-09-30 | Amazon Technologies, Inc. | Historical browsing session management |
US8914514B1 (en) | 2011-09-27 | 2014-12-16 | Amazon Technologies, Inc. | Managing network based content |
US8943197B1 (en) | 2012-08-16 | 2015-01-27 | Amazon Technologies, Inc. | Automated content update notification |
US8949232B2 (en) * | 2011-10-04 | 2015-02-03 | Microsoft Corporation | Social network recommended content and recommending members for personalized search results |
US8972477B1 (en) | 2011-12-01 | 2015-03-03 | Amazon Technologies, Inc. | Offline browsing session management |
US9009334B1 (en) | 2011-12-09 | 2015-04-14 | Amazon Technologies, Inc. | Remote browsing session management |
US9037696B2 (en) | 2011-08-16 | 2015-05-19 | Amazon Technologies, Inc. | Managing information associated with network resources |
US9037975B1 (en) | 2012-02-10 | 2015-05-19 | Amazon Technologies, Inc. | Zooming interaction tracking and popularity determination |
US9087024B1 (en) | 2012-01-26 | 2015-07-21 | Amazon Technologies, Inc. | Narration of network content |
US9092405B1 (en) | 2012-01-26 | 2015-07-28 | Amazon Technologies, Inc. | Remote browsing and searching |
US9117002B1 (en) | 2011-12-09 | 2015-08-25 | Amazon Technologies, Inc. | Remote browsing session management |
US9137210B1 (en) | 2012-02-21 | 2015-09-15 | Amazon Technologies, Inc. | Remote browsing session management |
US9152970B1 (en) | 2011-09-27 | 2015-10-06 | Amazon Technologies, Inc. | Remote co-browsing session management |
US9178955B1 (en) | 2011-09-27 | 2015-11-03 | Amazon Technologies, Inc. | Managing network based content |
US9183258B1 (en) | 2012-02-10 | 2015-11-10 | Amazon Technologies, Inc. | Behavior based processing of content |
US9195768B2 (en) | 2011-08-26 | 2015-11-24 | Amazon Technologies, Inc. | Remote browsing session management |
US9208316B1 (en) | 2012-02-27 | 2015-12-08 | Amazon Technologies, Inc. | Selective disabling of content portions |
US9235764B2 (en) * | 2012-11-05 | 2016-01-12 | Toshiba Tec Kabushiki Kaisha | Commodity recognition apparatus and commodity recognition method |
US9298843B1 (en) | 2011-09-27 | 2016-03-29 | Amazon Technologies, Inc. | User agent information management |
US9307004B1 (en) | 2012-03-28 | 2016-04-05 | Amazon Technologies, Inc. | Prioritized content transmission |
US9313100B1 (en) | 2011-11-14 | 2016-04-12 | Amazon Technologies, Inc. | Remote browsing session management |
US9330188B1 (en) | 2011-12-22 | 2016-05-03 | Amazon Technologies, Inc. | Shared browsing sessions |
US9336321B1 (en) | 2012-01-26 | 2016-05-10 | Amazon Technologies, Inc. | Remote browsing and searching |
US9374244B1 (en) | 2012-02-27 | 2016-06-21 | Amazon Technologies, Inc. | Remote browsing session management |
US9383958B1 (en) | 2011-09-27 | 2016-07-05 | Amazon Technologies, Inc. | Remote co-browsing session management |
US9460220B1 (en) | 2012-03-26 | 2016-10-04 | Amazon Technologies, Inc. | Content selection based on target device characteristics |
US9509783B1 (en) | 2012-01-26 | 2016-11-29 | Amazon Technlogogies, Inc. | Customized browser images |
US9578137B1 (en) | 2013-06-13 | 2017-02-21 | Amazon Technologies, Inc. | System for enhancing script execution performance |
US9621406B2 (en) | 2011-06-30 | 2017-04-11 | Amazon Technologies, Inc. | Remote browsing session management |
US9635041B1 (en) | 2014-06-16 | 2017-04-25 | Amazon Technologies, Inc. | Distributed split browser content inspection and analysis |
US9641637B1 (en) | 2011-09-27 | 2017-05-02 | Amazon Technologies, Inc. | Network resource optimization |
US9772979B1 (en) | 2012-08-08 | 2017-09-26 | Amazon Technologies, Inc. | Reproducing user browsing sessions |
US10089403B1 (en) | 2011-08-31 | 2018-10-02 | Amazon Technologies, Inc. | Managing network based storage |
US10152463B1 (en) | 2013-06-13 | 2018-12-11 | Amazon Technologies, Inc. | System for profiling page browsing interactions |
US10296558B1 (en) | 2012-02-27 | 2019-05-21 | Amazon Technologies, Inc. | Remote generation of composite content pages |
US20190236158A1 (en) * | 2018-01-30 | 2019-08-01 | Wal-Mart Stores, Inc. | Databases and file management systems and methods for performing a live update of a graphical user interface to boost one or more items |
US10664538B1 (en) | 2017-09-26 | 2020-05-26 | Amazon Technologies, Inc. | Data security and data access auditing for network accessible content |
US10693991B1 (en) | 2011-09-27 | 2020-06-23 | Amazon Technologies, Inc. | Remote browsing session management |
US10726095B1 (en) | 2017-09-26 | 2020-07-28 | Amazon Technologies, Inc. | Network content layout using an intermediary system |
US11841912B2 (en) * | 2011-05-01 | 2023-12-12 | Twittle Search Limited Liability Company | System for applying natural language processing and inputs of a group of users to infer commonly desired search results |
Families Citing this family (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8924376B1 (en) | 2010-01-31 | 2014-12-30 | Bryant Christopher Lee | Method for human ranking of search results |
US8775336B2 (en) * | 2011-12-19 | 2014-07-08 | Yahoo! Inc. | Interactive interface for object search |
US10409808B2 (en) | 2013-10-30 | 2019-09-10 | Salesforce.Com, Inc. | System and method for metadata management via a user interface page |
US10740413B2 (en) * | 2013-10-30 | 2020-08-11 | Salesforce.Com, Inc. | System and method for user information management via a user interface page |
AU2015244356A1 (en) * | 2014-03-10 | 2016-10-27 | Aravind Musuluri | Augmenting search results |
US9858251B2 (en) | 2014-08-14 | 2018-01-02 | Rakuten Kobo Inc. | Automatically generating customized annotation document from query search results and user interface thereof |
US10726074B2 (en) | 2017-01-04 | 2020-07-28 | Microsoft Technology Licensing, Llc | Identifying among recent revisions to documents those that are relevant to a search query |
Citations (20)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6285999B1 (en) * | 1997-01-10 | 2001-09-04 | The Board Of Trustees Of The Leland Stanford Junior University | Method for node ranking in a linked database |
US20050216454A1 (en) * | 2004-03-15 | 2005-09-29 | Yahoo! Inc. | Inverse search systems and methods |
US20060026147A1 (en) * | 2004-07-30 | 2006-02-02 | Cone Julian M | Adaptive search engine |
US20070106659A1 (en) * | 2005-03-18 | 2007-05-10 | Yunshan Lu | Search engine that applies feedback from users to improve search results |
US20080154879A1 (en) * | 2006-12-22 | 2008-06-26 | Yahoo! Inc. | Method and apparatus for creating user-generated document feedback to improve search relevancy |
US20080162540A1 (en) * | 2006-12-29 | 2008-07-03 | Yahoo! Inc. | Identifying offensive content using user click data |
US20080177728A1 (en) * | 1998-03-03 | 2008-07-24 | Dwayne Bowman | Identifying the items most relevant to a current query based on items selected in connection with similar queries |
US7447678B2 (en) * | 2003-12-31 | 2008-11-04 | Google Inc. | Interface for a universal search engine |
US20090077071A1 (en) * | 2006-04-18 | 2009-03-19 | Mainstream Advertising , Inc. | System and method for responding to a search request |
US20090077056A1 (en) * | 2007-09-17 | 2009-03-19 | Yahoo! Inc. | Customization of search results |
US20090119258A1 (en) * | 2007-11-05 | 2009-05-07 | William Petty | System and method for content ranking and reviewer selection |
US20090204599A1 (en) * | 2008-02-13 | 2009-08-13 | Microsoft Corporation | Using related users data to enhance web search |
US20090234834A1 (en) * | 2008-03-12 | 2009-09-17 | Yahoo! Inc. | System, method, and/or apparatus for reordering search results |
US7599911B2 (en) * | 2002-08-05 | 2009-10-06 | Yahoo! Inc. | Method and apparatus for search ranking using human input and automated ranking |
US20100106703A1 (en) * | 2006-05-02 | 2010-04-29 | Mark Cramer | Dynamic search engine results employing user behavior |
US7725346B2 (en) * | 2005-07-27 | 2010-05-25 | International Business Machines Corporation | Method and computer program product for predicting sales from online public discussions |
US7756806B1 (en) * | 2002-11-11 | 2010-07-13 | Zxibix, Inc. | System and method for facilitating collaboration and related multiple user thinking and cooperation regarding an arbitrary problem |
US20100180217A1 (en) * | 2007-12-03 | 2010-07-15 | Ebay Inc. | Live search chat room |
US7801885B1 (en) * | 2007-01-25 | 2010-09-21 | Neal Akash Verma | Search engine system and method with user feedback on search results |
US7798908B2 (en) * | 2007-02-21 | 2010-09-21 | Microsoft Corporation | Promotional games architecture |
Family Cites Families (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6651086B1 (en) * | 2000-02-22 | 2003-11-18 | Yahoo! Inc. | Systems and methods for matching participants to a conversation |
US6957384B2 (en) * | 2000-12-27 | 2005-10-18 | Tractmanager, Llc | Document management system |
US7340438B2 (en) * | 2001-05-21 | 2008-03-04 | Nokia Corporation | Method and apparatus for managing and enforcing user privacy |
US20030023754A1 (en) * | 2001-07-27 | 2003-01-30 | Matthias Eichstadt | Method and system for adding real-time, interactive functionality to a web-page |
US7418661B2 (en) * | 2002-09-17 | 2008-08-26 | Hewlett-Packard Development Company, L.P. | Published web page version tracking |
AU2003297193A1 (en) * | 2002-12-13 | 2004-07-09 | Applied Minds, Inc. | Meta-web |
US20050262427A1 (en) | 2003-03-31 | 2005-11-24 | International Business Machines Corporation | System and method for managing website content |
US7689578B2 (en) * | 2004-01-15 | 2010-03-30 | International Business Machines Corporation | Dealing with annotation versioning through multiple versioning policies and management thereof |
EP2662784A1 (en) * | 2004-03-15 | 2013-11-13 | Yahoo! Inc. | Search systems and methods with integration of user annotations |
US20060206462A1 (en) * | 2005-03-13 | 2006-09-14 | Logic Flows, Llc | Method and system for document manipulation, analysis and tracking |
US7472119B2 (en) * | 2005-06-30 | 2008-12-30 | Microsoft Corporation | Prioritizing search results by client search satisfaction |
US8463764B2 (en) * | 2008-03-17 | 2013-06-11 | Fuhu Holdings, Inc. | Social based search engine, system and method |
-
2010
- 2010-11-12 US US12/944,842 patent/US20110191327A1/en not_active Abandoned
-
2011
- 2011-01-03 US US12/983,363 patent/US8099406B2/en not_active Expired - Fee Related
Patent Citations (20)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6285999B1 (en) * | 1997-01-10 | 2001-09-04 | The Board Of Trustees Of The Leland Stanford Junior University | Method for node ranking in a linked database |
US20080177728A1 (en) * | 1998-03-03 | 2008-07-24 | Dwayne Bowman | Identifying the items most relevant to a current query based on items selected in connection with similar queries |
US7599911B2 (en) * | 2002-08-05 | 2009-10-06 | Yahoo! Inc. | Method and apparatus for search ranking using human input and automated ranking |
US7756806B1 (en) * | 2002-11-11 | 2010-07-13 | Zxibix, Inc. | System and method for facilitating collaboration and related multiple user thinking and cooperation regarding an arbitrary problem |
US7447678B2 (en) * | 2003-12-31 | 2008-11-04 | Google Inc. | Interface for a universal search engine |
US20050216454A1 (en) * | 2004-03-15 | 2005-09-29 | Yahoo! Inc. | Inverse search systems and methods |
US20060026147A1 (en) * | 2004-07-30 | 2006-02-02 | Cone Julian M | Adaptive search engine |
US20070106659A1 (en) * | 2005-03-18 | 2007-05-10 | Yunshan Lu | Search engine that applies feedback from users to improve search results |
US7725346B2 (en) * | 2005-07-27 | 2010-05-25 | International Business Machines Corporation | Method and computer program product for predicting sales from online public discussions |
US20090077071A1 (en) * | 2006-04-18 | 2009-03-19 | Mainstream Advertising , Inc. | System and method for responding to a search request |
US20100106703A1 (en) * | 2006-05-02 | 2010-04-29 | Mark Cramer | Dynamic search engine results employing user behavior |
US20080154879A1 (en) * | 2006-12-22 | 2008-06-26 | Yahoo! Inc. | Method and apparatus for creating user-generated document feedback to improve search relevancy |
US20080162540A1 (en) * | 2006-12-29 | 2008-07-03 | Yahoo! Inc. | Identifying offensive content using user click data |
US7801885B1 (en) * | 2007-01-25 | 2010-09-21 | Neal Akash Verma | Search engine system and method with user feedback on search results |
US7798908B2 (en) * | 2007-02-21 | 2010-09-21 | Microsoft Corporation | Promotional games architecture |
US20090077056A1 (en) * | 2007-09-17 | 2009-03-19 | Yahoo! Inc. | Customization of search results |
US20090119258A1 (en) * | 2007-11-05 | 2009-05-07 | William Petty | System and method for content ranking and reviewer selection |
US20100180217A1 (en) * | 2007-12-03 | 2010-07-15 | Ebay Inc. | Live search chat room |
US20090204599A1 (en) * | 2008-02-13 | 2009-08-13 | Microsoft Corporation | Using related users data to enhance web search |
US20090234834A1 (en) * | 2008-03-12 | 2009-09-17 | Yahoo! Inc. | System, method, and/or apparatus for reordering search results |
Cited By (68)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20100153292A1 (en) * | 2008-12-11 | 2010-06-17 | Microsoft Corporation | Making Friend and Location Recommendations Based on Location Similarities |
US11841912B2 (en) * | 2011-05-01 | 2023-12-12 | Twittle Search Limited Liability Company | System for applying natural language processing and inputs of a group of users to infer commonly desired search results |
US8799412B2 (en) | 2011-06-30 | 2014-08-05 | Amazon Technologies, Inc. | Remote browsing session management |
US8577963B2 (en) | 2011-06-30 | 2013-11-05 | Amazon Technologies, Inc. | Remote browsing session between client browser and network based browser |
US9621406B2 (en) | 2011-06-30 | 2017-04-11 | Amazon Technologies, Inc. | Remote browsing session management |
US10116487B2 (en) | 2011-06-30 | 2018-10-30 | Amazon Technologies, Inc. | Management of interactions with representations of rendered and unprocessed content |
US8706860B2 (en) | 2011-06-30 | 2014-04-22 | Amazon Technologies, Inc. | Remote browsing session management |
US10506076B2 (en) | 2011-06-30 | 2019-12-10 | Amazon Technologies, Inc. | Remote browsing session management with multiple content versions |
US9037696B2 (en) | 2011-08-16 | 2015-05-19 | Amazon Technologies, Inc. | Managing information associated with network resources |
US9870426B2 (en) | 2011-08-16 | 2018-01-16 | Amazon Technologies, Inc. | Managing information associated with network resources |
US10063618B2 (en) | 2011-08-26 | 2018-08-28 | Amazon Technologies, Inc. | Remote browsing session management |
US9195768B2 (en) | 2011-08-26 | 2015-11-24 | Amazon Technologies, Inc. | Remote browsing session management |
US10089403B1 (en) | 2011-08-31 | 2018-10-02 | Amazon Technologies, Inc. | Managing network based storage |
US8589385B2 (en) | 2011-09-27 | 2013-11-19 | Amazon Technologies, Inc. | Historical browsing session management |
US9641637B1 (en) | 2011-09-27 | 2017-05-02 | Amazon Technologies, Inc. | Network resource optimization |
US9383958B1 (en) | 2011-09-27 | 2016-07-05 | Amazon Technologies, Inc. | Remote co-browsing session management |
US9298843B1 (en) | 2011-09-27 | 2016-03-29 | Amazon Technologies, Inc. | User agent information management |
US9253284B2 (en) | 2011-09-27 | 2016-02-02 | Amazon Technologies, Inc. | Historical browsing session management |
US10693991B1 (en) | 2011-09-27 | 2020-06-23 | Amazon Technologies, Inc. | Remote browsing session management |
US8849802B2 (en) | 2011-09-27 | 2014-09-30 | Amazon Technologies, Inc. | Historical browsing session management |
US9178955B1 (en) | 2011-09-27 | 2015-11-03 | Amazon Technologies, Inc. | Managing network based content |
US8914514B1 (en) | 2011-09-27 | 2014-12-16 | Amazon Technologies, Inc. | Managing network based content |
US9152970B1 (en) | 2011-09-27 | 2015-10-06 | Amazon Technologies, Inc. | Remote co-browsing session management |
US8615431B1 (en) | 2011-09-29 | 2013-12-24 | Amazon Technologies, Inc. | Network content message placement management |
US8949232B2 (en) * | 2011-10-04 | 2015-02-03 | Microsoft Corporation | Social network recommended content and recommending members for personalized search results |
US9313100B1 (en) | 2011-11-14 | 2016-04-12 | Amazon Technologies, Inc. | Remote browsing session management |
US10057320B2 (en) | 2011-12-01 | 2018-08-21 | Amazon Technologies, Inc. | Offline browsing session management |
US8972477B1 (en) | 2011-12-01 | 2015-03-03 | Amazon Technologies, Inc. | Offline browsing session management |
US9117002B1 (en) | 2011-12-09 | 2015-08-25 | Amazon Technologies, Inc. | Remote browsing session management |
US9866615B2 (en) | 2011-12-09 | 2018-01-09 | Amazon Technologies, Inc. | Remote browsing session management |
US9479564B2 (en) | 2011-12-09 | 2016-10-25 | Amazon Technologies, Inc. | Browsing session metric creation |
US9009334B1 (en) | 2011-12-09 | 2015-04-14 | Amazon Technologies, Inc. | Remote browsing session management |
US9330188B1 (en) | 2011-12-22 | 2016-05-03 | Amazon Technologies, Inc. | Shared browsing sessions |
US9087024B1 (en) | 2012-01-26 | 2015-07-21 | Amazon Technologies, Inc. | Narration of network content |
US10275433B2 (en) | 2012-01-26 | 2019-04-30 | Amazon Technologies, Inc. | Remote browsing and searching |
US8627195B1 (en) | 2012-01-26 | 2014-01-07 | Amazon Technologies, Inc. | Remote browsing and searching |
US9336321B1 (en) | 2012-01-26 | 2016-05-10 | Amazon Technologies, Inc. | Remote browsing and searching |
US8839087B1 (en) * | 2012-01-26 | 2014-09-16 | Amazon Technologies, Inc. | Remote browsing and searching |
US10104188B2 (en) | 2012-01-26 | 2018-10-16 | Amazon Technologies, Inc. | Customized browser images |
US9509783B1 (en) | 2012-01-26 | 2016-11-29 | Amazon Technlogogies, Inc. | Customized browser images |
US9529784B2 (en) | 2012-01-26 | 2016-12-27 | Amazon Technologies, Inc. | Remote browsing and searching |
US9092405B1 (en) | 2012-01-26 | 2015-07-28 | Amazon Technologies, Inc. | Remote browsing and searching |
US9898542B2 (en) | 2012-01-26 | 2018-02-20 | Amazon Technologies, Inc. | Narration of network content |
US9195750B2 (en) | 2012-01-26 | 2015-11-24 | Amazon Technologies, Inc. | Remote browsing and searching |
US9183258B1 (en) | 2012-02-10 | 2015-11-10 | Amazon Technologies, Inc. | Behavior based processing of content |
US9037975B1 (en) | 2012-02-10 | 2015-05-19 | Amazon Technologies, Inc. | Zooming interaction tracking and popularity determination |
US10567346B2 (en) | 2012-02-21 | 2020-02-18 | Amazon Technologies, Inc. | Remote browsing session management |
US9137210B1 (en) | 2012-02-21 | 2015-09-15 | Amazon Technologies, Inc. | Remote browsing session management |
US9208316B1 (en) | 2012-02-27 | 2015-12-08 | Amazon Technologies, Inc. | Selective disabling of content portions |
US9374244B1 (en) | 2012-02-27 | 2016-06-21 | Amazon Technologies, Inc. | Remote browsing session management |
US10296558B1 (en) | 2012-02-27 | 2019-05-21 | Amazon Technologies, Inc. | Remote generation of composite content pages |
US9460220B1 (en) | 2012-03-26 | 2016-10-04 | Amazon Technologies, Inc. | Content selection based on target device characteristics |
US9307004B1 (en) | 2012-03-28 | 2016-04-05 | Amazon Technologies, Inc. | Prioritized content transmission |
US9723067B2 (en) | 2012-03-28 | 2017-08-01 | Amazon Technologies, Inc. | Prioritized content transmission |
US20140023241A1 (en) * | 2012-07-23 | 2014-01-23 | Toshiba Tec Kabushiki Kaisha | Dictionary registration apparatus and method for adding feature amount data to recognition dictionary |
US9772979B1 (en) | 2012-08-08 | 2017-09-26 | Amazon Technologies, Inc. | Reproducing user browsing sessions |
US8943197B1 (en) | 2012-08-16 | 2015-01-27 | Amazon Technologies, Inc. | Automated content update notification |
US9830400B2 (en) | 2012-08-16 | 2017-11-28 | Amazon Technologies, Inc. | Automated content update notification |
US20140177912A1 (en) * | 2012-10-31 | 2014-06-26 | Toshiba Tec Kabushiki Kaisha | Commodity reading apparatus, commodity sales data processing apparatus and commodity reading method |
US9235764B2 (en) * | 2012-11-05 | 2016-01-12 | Toshiba Tec Kabushiki Kaisha | Commodity recognition apparatus and commodity recognition method |
US10152463B1 (en) | 2013-06-13 | 2018-12-11 | Amazon Technologies, Inc. | System for profiling page browsing interactions |
US9578137B1 (en) | 2013-06-13 | 2017-02-21 | Amazon Technologies, Inc. | System for enhancing script execution performance |
US10164993B2 (en) | 2014-06-16 | 2018-12-25 | Amazon Technologies, Inc. | Distributed split browser content inspection and analysis |
US9635041B1 (en) | 2014-06-16 | 2017-04-25 | Amazon Technologies, Inc. | Distributed split browser content inspection and analysis |
US10664538B1 (en) | 2017-09-26 | 2020-05-26 | Amazon Technologies, Inc. | Data security and data access auditing for network accessible content |
US10726095B1 (en) | 2017-09-26 | 2020-07-28 | Amazon Technologies, Inc. | Network content layout using an intermediary system |
US20190236158A1 (en) * | 2018-01-30 | 2019-08-01 | Wal-Mart Stores, Inc. | Databases and file management systems and methods for performing a live update of a graphical user interface to boost one or more items |
US10956428B2 (en) * | 2018-01-30 | 2021-03-23 | Walmart Apollo Llc | Databases and file management systems and methods for performing a live update of a graphical user interface to boost one or more items |
Also Published As
Publication number | Publication date |
---|---|
US8099406B2 (en) | 2012-01-17 |
US20110191317A1 (en) | 2011-08-04 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20110191327A1 (en) | Method for Human Ranking of Search Results | |
US9053158B1 (en) | Method for human ranking of search results | |
US8370334B2 (en) | Dynamic updating of display and ranking for search results | |
US9348871B2 (en) | Method and system for assessing relevant properties of work contexts for use by information services | |
US8799265B2 (en) | Semantically associated text index and the population and use thereof | |
US7698626B2 (en) | Enhanced document browsing with automatically generated links to relevant information | |
US7765209B1 (en) | Indexing and retrieval of blogs | |
KR101215791B1 (en) | Using reputation measures to improve search relevance | |
US9092756B2 (en) | Information-retrieval systems, methods and software with content relevancy enhancements | |
US20050028156A1 (en) | Automatic method and system for formulating and transforming representations of context used by information services | |
US20100057714A1 (en) | Search results ranking method and system | |
KR20070039072A (en) | Results based personalization of advertisements in a search engine | |
Tatu et al. | Rsdc’08: Tag recommendations using bookmark content | |
US20070179949A1 (en) | Learning retrieval functions incorporating query differentiation for information retrieval | |
WO2007127676A1 (en) | System and method for indexing web content using click-through features | |
US9734211B1 (en) | Personalizing search results | |
WO2008106667A1 (en) | Searching heterogeneous interrelated entities | |
JP2008507041A (en) | Personalize the ordering of place content in search results | |
CA2812422A1 (en) | System, method and computer program for searching within a sub-domain by linking to other sub-domains | |
Behnert et al. | Ranking search results in library information systems—Considering ranking approaches adapted from web search engines | |
US20070174266A1 (en) | Method of optimization of listed result of internet-based search and system based on the method | |
AU2012202738B2 (en) | Results based personalization of advertisements in a search engine | |
KR20120057786A (en) | Method and system to provide search list and search keyword ranking based on information database attached to search result | |
Karwowski | Search Tools for the Web |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |
|
AS | Assignment |
Owner name: ADVANCED RESEARCH LLC, MARYLAND Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:LEE, BRYANT;REEL/FRAME:037508/0973 Effective date: 20101112 |