CA2208709C - Method for providing a compressed rendition of a video program in a format suitable for electronic searching and retrieval - Google Patents
Method for providing a compressed rendition of a video program in a format suitable for electronic searching and retrieval Download PDFInfo
- Publication number
- CA2208709C CA2208709C CA002208709A CA2208709A CA2208709C CA 2208709 C CA2208709 C CA 2208709C CA 002208709 A CA002208709 A CA 002208709A CA 2208709 A CA2208709 A CA 2208709A CA 2208709 C CA2208709 C CA 2208709C
- Authority
- CA
- Canada
- Prior art keywords
- hypertext
- video program
- transcript
- pictorial transcript
- representative frames
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Lifetime
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
- G06F16/74—Browsing; Visualisation therefor
- G06F16/748—Hypervideo
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
- G06F16/73—Querying
- G06F16/738—Presentation of query results
- G06F16/739—Presentation of query results in form of a video summary, e.g. the video summary being a video sequence, a composite still image or having synthesized frames
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
- G06F16/78—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/783—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
- G06F16/7844—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using original textual content or text extracted from visual content or transcript of audio data
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/912—Applications of a database
- Y10S707/913—Multimedia
- Y10S707/914—Video
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/99941—Database schema or data structure
- Y10S707/99944—Object-oriented database structure
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/99941—Database schema or data structure
- Y10S707/99944—Object-oriented database structure
- Y10S707/99945—Object-oriented database structure processing
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/99941—Database schema or data structure
- Y10S707/99948—Application of database or data structure, e.g. distributed, multimedia, or image
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Library & Information Science (AREA)
- Human Computer Interaction (AREA)
- Computational Linguistics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Television Signal Processing For Recording (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
A compressed rendition of a video program is provided in a format suitable for electronic searching and retrieval. An electronic pictorial transcript representation of the video program is initially received. The video program has a video component and a second information-bearing media component associated therewith. The pictorial transcript representation includes a representative frame from each segment of the video component of the video program and a portion of the second media component associated with the segment.
The electronic pictorial transcript is transformed into a hypertext format to form a hypertext pictorial transcript. The hypertext pictorial transcript is subsequently recorded in an electronic medium.
The electronic pictorial transcript is transformed into a hypertext format to form a hypertext pictorial transcript. The hypertext pictorial transcript is subsequently recorded in an electronic medium.
Description
METHOD FOR PROVIDING A COMPRESSED RENDITION
OF A VIDEO PROGRAM IN A FORMAT SUITABLE FOR
ELECTRONIC SEARCHING AND RETRIEVAL
Technical Field This invention relates generally to a method for automatically providing a compressed rendition of a video program in a format suitable for electronic searching and retrieval, and more particularly to a method for providing a compressed rendition of a video program in a format suitable for electronic searching and retrieval on the World Wide 1 o Web.
Background The rapid growth of the World Wide Web began with the development of an on-line browser having a graphical user interface.
Graphical interfaces provide a number of important advantages, including the ability to rapidly scroll through a document to get to a particular point of interest. Moreover, the ability to interact with a medium other than text (i.e. images or audio) increases the rate at which information can be conveyed since an image often conveys an idea faster and more efficiently than text.
2o While graphical browsers provide an adequate interface for text and images, they provide an inadequate interface for video programs.
The sequential nature of the video and audio components of a video program impedes rapid access to such programs on the World Wide Web by graphical browsers. Furthermore, because of the limited bandwidth of networks supporting the World Wide Web, and particularly the limitations of most users' connections to such networks, it takes a long time to transmit a program with its full content. For example, at a connection speed of 28,800 bits per second, it could take up to about 45 minutes to transmit even a three or four minute audiovisual segment with sound and full-motion video. As a result, video program providers sometimes form a compressed version of the video program by manually extracting and retaining selected frames from the program while other frames are discarded. The selected frames and accompanying text, typically taken from a transcript of the program, result in a document that may subsequently be made available over the World Wide Web.
1o However, the generation of this document is typically a tedious and time consuming task since it must be created by a manual process.
Accordingly, it would be advantageous to provide a rendition of a video program which can be automatically generated and which allows easy interaction with graphical browsers with a minimum of information ~ s loss.
Summar3i of the Invention The present inventors have realized that a pictorial transcript representation of a video program is particularly well suited for on-line searching and retrieving applications such as browsing on the World Wide 2o Web. Pictorial transcripts are compact representations of video programs which are automatically generated by selecting representative frames or images from the video program and combining them with a second media component such as audio or text which is associated with each representative frame. Properly chosen, the representative frames convey a 25 substantial portion of the information content of the original video program. Moreover, pictorial transcripts may be generated in an automatic fashion, thus eliminating the substantial time and effort that was previously required to place a document of this type on the World Wide Web.
In accordance with one aspect of the present invention there is provided a method for automatically providing a compressed rendition of a video program in a format suitable for electronic searching and retrieval, said method comprising the steps of: receiving electronic data representing a condensed version of a video program, said video program having a video component and a second information-bearing media component associated therewith, said electronic data representation including a representative frame from each segment of the video component of the video program and a portion of said second media component associated with said segment;
automatically transforming said electronic data representation into a hypertext format to form a hypertext pictorial transcript; recording said hypertext pictorial transcript in an electronic medium; and wherein said 15 hypertext pictorial transcript has at least one standard page layout, selectable from a plurality of standard page layouts, that includes a subset of representative frames selected by at least one criterion that reduces bandwidth.
In a specific embodiment, the inventive method provides a 20 compressed rendition of a video program in a format suitable for electronic searching and retrieval.
In a specific embodiment, the inventive method provides a compressed rendition of a video program in a format suitable for electronic searching and retrieval. An electronic pictorial transcript representation of 25 the video program is initially received. The video program has a video component and a second information-bearing media component associated therewith. The pictorial transcript representation includes a representative frame from each segment of the video component of the video program and a portion of the second media component associated with the segment. The 30 electronic pictorial transcript is transformed into a hypertext format to form a -3a-hypertext pictorial transcript. The hypertext pictorial transcript is subsequently recorded in an electronic medium.
Brief Description of the Drawi~,s FIG. 1 is an example of one page of a printed pictorial transcript generated from a television news program in accordance with method of the present invention.
FIG. 2 illustrates the use of server push for viewing an HTMI, ~4 pictorial transcript.
FIG. 3 shows an example of a page format that may be employed when performing keyword searching.
FIG. 4 shows an example of an index that may be generated for HTML pictorial transcripts.
Detailed Description A method for automatically compressing multimedia data is disclosed in Shahraray B., and Gibbon D.C., "Automatic Generation of Pictorial Transcripts of Video Programs," in Multimedia Computing and Networking 1995, Proc. SPIE 2417, February 1995. In accordance with this known method, a video program is compressed by selecting certain frames from 5 the entire sequence of frames to serve as representative frames. For example, a single frame may be used to represent the visual information contained in any given scene of the video program. A scene may be defined as a segment of the video program over which the visual contents do not change significantly. Thus, a frame selected from the 1 o scene may be used to represent the entire scene without losing a substantially large amount of information. A series of such representative frames from all the scenes in the video program provides a reasonably accurate representation of the entire video program with an acceptable degree of information loss. These compression methods in 15 effect perform a content-based sampling of the video program.
Additional information may be found in B. Shahraray, "Scene Change - Detection and Content-Based Sampling of Video Sequences," Digital Video Compression: Algorithms and Technologies 1995, SPIE 2419.
In the previously cited documents, a plurality of representative 2o frames are selected by sampling the video program in a content-based manner to retain a single representative frame from each scene. While the series of frames selected in this manner may not contain all the visual information in the original video program, when combined with another medium that was a part of the original video program, such as audio or 25 closed-captioned text, the resulting multimedia program adequately conveys the information content of the video program in a condensed format. To generate this condensed multimedia program, a correspondence must be formed between the representative frames and the audio or textual medium. For example, each representative frame should be associated with the portion of the audio or textual medium corresponding to the entire scene from which the representative frame s was selected. This correspondence may be accomplished in a relatively simple manner because in the original video program the video medium is already synchronized with the audio or textual information.
Additional details concerning the formulation of this correspondence may be found in the previously cited references.
1o The representative frames, the audio or textual components associated therewith, and the correspondence between the representative frames and the audio or textual components comprise electronic data representing a condensed version of a video program, which hereinafter will be referred to as the condensed electronic data.
15 In the case of closed-captioned text, a printed rendition of the condensed electronic data may be provided. The printed rendition constitutes a so-called pictorial transcript in which each representative frame is printed with a caption containing the portion of the closed-caption text corresponding to the scene from which that representative 2o frame is taken. FIG.1 is an example of one page of printed pictorial transcript generated from a television news program. Alternatively, rather than printing the condensed electronic data as a pictorial transcript, the data simply may be electronically stored for subsequent retrieval. Thereafter the data may be printed, displayed on a computer, 25 or transmitted in any desired format.
In addition, the condensed electronic data may be generalized further to refer to the series of representative frames and the audio segments corresponding thereto rather than closed-caption segments. In this case the condensed electronic data may be conveniently stored electronically and then displayed by sequentially displaying the representative frames and, simultaneous with each displayed frame, playing the corresponding audio segment.
In accordance with the present invention, electronic data representing a condensed version of a video program is formatted in hypertext markup language (HTML) so that the resulting HTML
document is compatible with the World Wide Web. HTML documents 1o refer to on-line documents having words or graphics that contain links to other on-line documents. Such documents are commonly referred to as hypertext documents. By selecting the link (using a mouse or key command) the user is connected to another document that may be located on the same or a different computer. It should be noted that while the present invention is described in terms of an on-line document formatted in HTML, more generally the present invention is applicable to hypertext documents formatted in languages other than HTML, such as hypercard, for example.
An HTML document is automatically produced from the 2o condensed electronic data by an HTML generator, which converts the data into an HTML document. Procedures to implement such a generator are well known. As used hereinafter, the terms HTML
document and HTML pictorial transcript refer to the condensed electronic data that is formatted in HTML. The HTML document or pictorial transcript may be composed of individual records connected by links. The individual records of the HTML document or pictorial transcript are referred to as pages.
The HTML pictorial transcript may be advantageously divided over two or more HTML pages, depending on the size of the document.
An HTML document consisting of only a single HTML page is impractical for all but the shortest programs (e.g., less than ten minutes in length) because WWW browsers, which sometimes lack parallel loading capability, begin to exhibit unacceptable delays. In fact, even browsers having parallel loading capability such as Netscape will often be taxed. The size of each HTML page may be determined in any appropriate manner. For example, the HTML generator may begin a 1o new page after a predetermined number of images (e.g., 25) have been placed on a single page. Alternatively, the pages may be divided on the basis of story and topic based segmentation. The various pages comprising the HTLM document may connected by hypertext links.
A graphical browser is a graphical interface that can access documents on the WWW in an HTML format. The HTML pictorial transcript may be conveniently accessed and searched using conventional graphical browsers such as Mosaic, Spry and Explorer, for example.
The HTML pictorial transcript may be displayed in a variety of different formats. The user may have the option of selecting among several predetermined formats, or alternatively, the user may customize a format via the web browser. The server, in turn, re-executes the HTML
generator routine, which now produces the HTML document in the desired format. Additionally, if no selection is made, the HTML
2s transcript may be displayed in a default format (which may be one of the standard formats). In some embodiments of the invention, the user may _g_ be provided with a plurality of different default formats from which to choose.
In one embodiment of the invention, a standard or default format displays an HTML pictorial transcript that is the equivalent of the printed rendition of a pictorial transcript such as shown in FIG.1. Other formats may modify this particular format to reduce retrieval time and improve page layout. For example, some formats may be employed to reduce the required bandwidth by displaying only a subset of the representative frames contained in the HTML pictorial transcript. Many 1o different criteria may be employed to determine which representative frames to retain and which to omit.
One criterion that may used to eliminate select representative frames is based on the presence of redundant frames. For example, if the original program contains a shot of a given scene at one time and subsequently contains substantially the same scene after one or more other scenes have intervened, the resulting pictorial transcript will contain two representative frames that are substantially the same.
Accordingly, one of the redundant representative frames may be eliminated to reduce bandwidth. In the resulting HTML pictorial 2o transcript it may be desirable to use a hypertext link in place of the second appearance of the redundant representative frame which links back to the first appearance of the representative frame.
Other criteria that may used to eliminate select representative frames are based on random subsampling (e.g., retain every other representative frame) or, alternatively, the size of the JPEG image file.
For example, it may be desirable to retain only the largest of the image files on the assumption that image size is correlated with the complexity of the image. More complex images typically convey more information.
Conversely, it may be desirable to retain only the smallest of the image files to further minimize bandwidth requirements. Alternatively, it may be advantageous to retain only representative images that differ from 5 one another by more than a prescribed amount, as determined by scene matching techniques. The representative images that are eliminated in this manner may be replaced by hypertext anchors linked to the similar representative images that were retained.
Another criterion that may be employed to select a subset of the t o representative images is based on the length of the scene from which the representative image was taken. For example, only representative images taken from the longest of the scenes in the video program may be retained since these scenes are presumably the most significant. For example, a video program of a speaker making a presentation before an 15 audience may contain many longer scenes of the speaker interrupted by occasional brief shots~of the audience. If the representative frames from only the longest scenes are retained, then representative frames of the speaker will be retained while the representative frames of the audience will be eliminated.
2o In some cases it may be desirable to eliminate representative frames associated with advertisements if the video programs are recorded from commercial television, for example. These representative frames may be easily removed because most commercials are either not captioned or are captioned in a mode different from the remainder of the 25 video program. Accordingly, the change in caption modes can be used to detect advertisements which are to be omitted from the HTLM transcript.
Another format that may be used to display HTML pictorial transcripts takes advantage of a mechanism known as server push, which is available on recent versions of the Netscape browser. Server push allows an HTML page to undergo changes while it is being viewed. This browser feature can be used to maintain a suitable page layout (e.g., a layout having a maximum number of images) without needing to eliminate sequentially occurring images. This feature, which could also be implemented using Java Animations, will be illustrated with reference to FIG. 2. FIG. 2(a) shows an HTML page of a pictorial transcript which 1 o has three sequential images 1, 2, and 3, without any intervening captions.
However, suppose the page format which is selected dictates that only one image is to be displayed on a page, as in FIG. 2(b). Server push may be used display the images as shown in FIGS. 2(c) - 2(e). When the page is first displayed at time t1 in FIG. 2(c), only the first image is displayed.
Using server push, the second image can be displayed at a later time t2 (e.g. one second later), as shown in FIG. 2(d). At yet a later time t3 the third image can be displayed, as in FIG. 2(e). Moreover, if the network bandwidth and client and server throughput are sufficiently high, video shorts (real-time playback) can be made to appear at the caption breaks.
2o In many cases a user will not be interested in viewing the HTML
pictorial transcript in a sequential manner. Rather, the user may be only interested in those portions of the transcript that pertain to a particular topic. In such cases the user may wish to perform a keyword search of the HTML pictorial transcript. The HTML generator can perform the search on the closed-captioned text and emphasize those portions of the transcript that contain the keyword. For example, images that appear immediately prior to and after the occurrence of a keyword may be displayed at full resolution while other images may be displayed at a smaller size and resolution. FIG. 3 shows an example of this format after a search for the word "Tokyo." The smaller images may be hypertext links to the corresponding full sized images. In some cases, particularly for large HTML pictorial transcripts, hypertext anchors may be used in place of the small images to reduce bandwidth. If the keyword appears more than once in the transcript, a chain of links may be created among the individual occurrences of the word. For example, in FIG. 3, the arrows denote a link to other occurrences of the term "Tokyo." The 1 o HTML pictorial transcript may also include hypertext anchors to other HTML documents which contain material supplementary to, or related to, the information in the transcript.
The HTML generator may create an index page for the HTML
pictorial transcript using conventional methods such as linguistic techniques, for example.. FIG. 4 shows one example of such an index page, which may be located as the first page of the document. The index may contain links to the individual pages of the transcript. The index may also include other information such as index terms obtained by linguistic analysis techniques. In FIG. 4, a portion of the index is 2o available for the user to list additional keywords to serve as index terms.
The index terms may be hypertext links to those locations in the transcript where the terms appear.
Similar to the HTML documents previously discussed, HTML
pictorial transcripts in which the representative frames are each 2s associated with a corresponding audio segment may be arranged in a variety of different formats. For example, the individual representative frames may serve as links to the audio segment. Alternatively, anchors may be associated with the representative frames. By clicking on the anchors the respective audio segments are played.
It will be appreciated that those skilled in the art will be able to devise numerous arrangements which, although not explicitly shown or described herein, embody the principles of the invention. Accordingly, all such alternatives, modifications and variations which fall within the spirit and broad scope of the appended claims will be embraced by the principles of the invention. For example, while the invention has been described as electronic data representing a condensed version of a video 1o program that is formatted as an HTML document for the World Wide Web, the invention is more generally applicable to such data that is formatted in any hypertext language suitable for electronic retrieval on a computer or over a communications network.
OF A VIDEO PROGRAM IN A FORMAT SUITABLE FOR
ELECTRONIC SEARCHING AND RETRIEVAL
Technical Field This invention relates generally to a method for automatically providing a compressed rendition of a video program in a format suitable for electronic searching and retrieval, and more particularly to a method for providing a compressed rendition of a video program in a format suitable for electronic searching and retrieval on the World Wide 1 o Web.
Background The rapid growth of the World Wide Web began with the development of an on-line browser having a graphical user interface.
Graphical interfaces provide a number of important advantages, including the ability to rapidly scroll through a document to get to a particular point of interest. Moreover, the ability to interact with a medium other than text (i.e. images or audio) increases the rate at which information can be conveyed since an image often conveys an idea faster and more efficiently than text.
2o While graphical browsers provide an adequate interface for text and images, they provide an inadequate interface for video programs.
The sequential nature of the video and audio components of a video program impedes rapid access to such programs on the World Wide Web by graphical browsers. Furthermore, because of the limited bandwidth of networks supporting the World Wide Web, and particularly the limitations of most users' connections to such networks, it takes a long time to transmit a program with its full content. For example, at a connection speed of 28,800 bits per second, it could take up to about 45 minutes to transmit even a three or four minute audiovisual segment with sound and full-motion video. As a result, video program providers sometimes form a compressed version of the video program by manually extracting and retaining selected frames from the program while other frames are discarded. The selected frames and accompanying text, typically taken from a transcript of the program, result in a document that may subsequently be made available over the World Wide Web.
1o However, the generation of this document is typically a tedious and time consuming task since it must be created by a manual process.
Accordingly, it would be advantageous to provide a rendition of a video program which can be automatically generated and which allows easy interaction with graphical browsers with a minimum of information ~ s loss.
Summar3i of the Invention The present inventors have realized that a pictorial transcript representation of a video program is particularly well suited for on-line searching and retrieving applications such as browsing on the World Wide 2o Web. Pictorial transcripts are compact representations of video programs which are automatically generated by selecting representative frames or images from the video program and combining them with a second media component such as audio or text which is associated with each representative frame. Properly chosen, the representative frames convey a 25 substantial portion of the information content of the original video program. Moreover, pictorial transcripts may be generated in an automatic fashion, thus eliminating the substantial time and effort that was previously required to place a document of this type on the World Wide Web.
In accordance with one aspect of the present invention there is provided a method for automatically providing a compressed rendition of a video program in a format suitable for electronic searching and retrieval, said method comprising the steps of: receiving electronic data representing a condensed version of a video program, said video program having a video component and a second information-bearing media component associated therewith, said electronic data representation including a representative frame from each segment of the video component of the video program and a portion of said second media component associated with said segment;
automatically transforming said electronic data representation into a hypertext format to form a hypertext pictorial transcript; recording said hypertext pictorial transcript in an electronic medium; and wherein said 15 hypertext pictorial transcript has at least one standard page layout, selectable from a plurality of standard page layouts, that includes a subset of representative frames selected by at least one criterion that reduces bandwidth.
In a specific embodiment, the inventive method provides a 20 compressed rendition of a video program in a format suitable for electronic searching and retrieval.
In a specific embodiment, the inventive method provides a compressed rendition of a video program in a format suitable for electronic searching and retrieval. An electronic pictorial transcript representation of 25 the video program is initially received. The video program has a video component and a second information-bearing media component associated therewith. The pictorial transcript representation includes a representative frame from each segment of the video component of the video program and a portion of the second media component associated with the segment. The 30 electronic pictorial transcript is transformed into a hypertext format to form a -3a-hypertext pictorial transcript. The hypertext pictorial transcript is subsequently recorded in an electronic medium.
Brief Description of the Drawi~,s FIG. 1 is an example of one page of a printed pictorial transcript generated from a television news program in accordance with method of the present invention.
FIG. 2 illustrates the use of server push for viewing an HTMI, ~4 pictorial transcript.
FIG. 3 shows an example of a page format that may be employed when performing keyword searching.
FIG. 4 shows an example of an index that may be generated for HTML pictorial transcripts.
Detailed Description A method for automatically compressing multimedia data is disclosed in Shahraray B., and Gibbon D.C., "Automatic Generation of Pictorial Transcripts of Video Programs," in Multimedia Computing and Networking 1995, Proc. SPIE 2417, February 1995. In accordance with this known method, a video program is compressed by selecting certain frames from 5 the entire sequence of frames to serve as representative frames. For example, a single frame may be used to represent the visual information contained in any given scene of the video program. A scene may be defined as a segment of the video program over which the visual contents do not change significantly. Thus, a frame selected from the 1 o scene may be used to represent the entire scene without losing a substantially large amount of information. A series of such representative frames from all the scenes in the video program provides a reasonably accurate representation of the entire video program with an acceptable degree of information loss. These compression methods in 15 effect perform a content-based sampling of the video program.
Additional information may be found in B. Shahraray, "Scene Change - Detection and Content-Based Sampling of Video Sequences," Digital Video Compression: Algorithms and Technologies 1995, SPIE 2419.
In the previously cited documents, a plurality of representative 2o frames are selected by sampling the video program in a content-based manner to retain a single representative frame from each scene. While the series of frames selected in this manner may not contain all the visual information in the original video program, when combined with another medium that was a part of the original video program, such as audio or 25 closed-captioned text, the resulting multimedia program adequately conveys the information content of the video program in a condensed format. To generate this condensed multimedia program, a correspondence must be formed between the representative frames and the audio or textual medium. For example, each representative frame should be associated with the portion of the audio or textual medium corresponding to the entire scene from which the representative frame s was selected. This correspondence may be accomplished in a relatively simple manner because in the original video program the video medium is already synchronized with the audio or textual information.
Additional details concerning the formulation of this correspondence may be found in the previously cited references.
1o The representative frames, the audio or textual components associated therewith, and the correspondence between the representative frames and the audio or textual components comprise electronic data representing a condensed version of a video program, which hereinafter will be referred to as the condensed electronic data.
15 In the case of closed-captioned text, a printed rendition of the condensed electronic data may be provided. The printed rendition constitutes a so-called pictorial transcript in which each representative frame is printed with a caption containing the portion of the closed-caption text corresponding to the scene from which that representative 2o frame is taken. FIG.1 is an example of one page of printed pictorial transcript generated from a television news program. Alternatively, rather than printing the condensed electronic data as a pictorial transcript, the data simply may be electronically stored for subsequent retrieval. Thereafter the data may be printed, displayed on a computer, 25 or transmitted in any desired format.
In addition, the condensed electronic data may be generalized further to refer to the series of representative frames and the audio segments corresponding thereto rather than closed-caption segments. In this case the condensed electronic data may be conveniently stored electronically and then displayed by sequentially displaying the representative frames and, simultaneous with each displayed frame, playing the corresponding audio segment.
In accordance with the present invention, electronic data representing a condensed version of a video program is formatted in hypertext markup language (HTML) so that the resulting HTML
document is compatible with the World Wide Web. HTML documents 1o refer to on-line documents having words or graphics that contain links to other on-line documents. Such documents are commonly referred to as hypertext documents. By selecting the link (using a mouse or key command) the user is connected to another document that may be located on the same or a different computer. It should be noted that while the present invention is described in terms of an on-line document formatted in HTML, more generally the present invention is applicable to hypertext documents formatted in languages other than HTML, such as hypercard, for example.
An HTML document is automatically produced from the 2o condensed electronic data by an HTML generator, which converts the data into an HTML document. Procedures to implement such a generator are well known. As used hereinafter, the terms HTML
document and HTML pictorial transcript refer to the condensed electronic data that is formatted in HTML. The HTML document or pictorial transcript may be composed of individual records connected by links. The individual records of the HTML document or pictorial transcript are referred to as pages.
The HTML pictorial transcript may be advantageously divided over two or more HTML pages, depending on the size of the document.
An HTML document consisting of only a single HTML page is impractical for all but the shortest programs (e.g., less than ten minutes in length) because WWW browsers, which sometimes lack parallel loading capability, begin to exhibit unacceptable delays. In fact, even browsers having parallel loading capability such as Netscape will often be taxed. The size of each HTML page may be determined in any appropriate manner. For example, the HTML generator may begin a 1o new page after a predetermined number of images (e.g., 25) have been placed on a single page. Alternatively, the pages may be divided on the basis of story and topic based segmentation. The various pages comprising the HTLM document may connected by hypertext links.
A graphical browser is a graphical interface that can access documents on the WWW in an HTML format. The HTML pictorial transcript may be conveniently accessed and searched using conventional graphical browsers such as Mosaic, Spry and Explorer, for example.
The HTML pictorial transcript may be displayed in a variety of different formats. The user may have the option of selecting among several predetermined formats, or alternatively, the user may customize a format via the web browser. The server, in turn, re-executes the HTML
generator routine, which now produces the HTML document in the desired format. Additionally, if no selection is made, the HTML
2s transcript may be displayed in a default format (which may be one of the standard formats). In some embodiments of the invention, the user may _g_ be provided with a plurality of different default formats from which to choose.
In one embodiment of the invention, a standard or default format displays an HTML pictorial transcript that is the equivalent of the printed rendition of a pictorial transcript such as shown in FIG.1. Other formats may modify this particular format to reduce retrieval time and improve page layout. For example, some formats may be employed to reduce the required bandwidth by displaying only a subset of the representative frames contained in the HTML pictorial transcript. Many 1o different criteria may be employed to determine which representative frames to retain and which to omit.
One criterion that may used to eliminate select representative frames is based on the presence of redundant frames. For example, if the original program contains a shot of a given scene at one time and subsequently contains substantially the same scene after one or more other scenes have intervened, the resulting pictorial transcript will contain two representative frames that are substantially the same.
Accordingly, one of the redundant representative frames may be eliminated to reduce bandwidth. In the resulting HTML pictorial 2o transcript it may be desirable to use a hypertext link in place of the second appearance of the redundant representative frame which links back to the first appearance of the representative frame.
Other criteria that may used to eliminate select representative frames are based on random subsampling (e.g., retain every other representative frame) or, alternatively, the size of the JPEG image file.
For example, it may be desirable to retain only the largest of the image files on the assumption that image size is correlated with the complexity of the image. More complex images typically convey more information.
Conversely, it may be desirable to retain only the smallest of the image files to further minimize bandwidth requirements. Alternatively, it may be advantageous to retain only representative images that differ from 5 one another by more than a prescribed amount, as determined by scene matching techniques. The representative images that are eliminated in this manner may be replaced by hypertext anchors linked to the similar representative images that were retained.
Another criterion that may be employed to select a subset of the t o representative images is based on the length of the scene from which the representative image was taken. For example, only representative images taken from the longest of the scenes in the video program may be retained since these scenes are presumably the most significant. For example, a video program of a speaker making a presentation before an 15 audience may contain many longer scenes of the speaker interrupted by occasional brief shots~of the audience. If the representative frames from only the longest scenes are retained, then representative frames of the speaker will be retained while the representative frames of the audience will be eliminated.
2o In some cases it may be desirable to eliminate representative frames associated with advertisements if the video programs are recorded from commercial television, for example. These representative frames may be easily removed because most commercials are either not captioned or are captioned in a mode different from the remainder of the 25 video program. Accordingly, the change in caption modes can be used to detect advertisements which are to be omitted from the HTLM transcript.
Another format that may be used to display HTML pictorial transcripts takes advantage of a mechanism known as server push, which is available on recent versions of the Netscape browser. Server push allows an HTML page to undergo changes while it is being viewed. This browser feature can be used to maintain a suitable page layout (e.g., a layout having a maximum number of images) without needing to eliminate sequentially occurring images. This feature, which could also be implemented using Java Animations, will be illustrated with reference to FIG. 2. FIG. 2(a) shows an HTML page of a pictorial transcript which 1 o has three sequential images 1, 2, and 3, without any intervening captions.
However, suppose the page format which is selected dictates that only one image is to be displayed on a page, as in FIG. 2(b). Server push may be used display the images as shown in FIGS. 2(c) - 2(e). When the page is first displayed at time t1 in FIG. 2(c), only the first image is displayed.
Using server push, the second image can be displayed at a later time t2 (e.g. one second later), as shown in FIG. 2(d). At yet a later time t3 the third image can be displayed, as in FIG. 2(e). Moreover, if the network bandwidth and client and server throughput are sufficiently high, video shorts (real-time playback) can be made to appear at the caption breaks.
2o In many cases a user will not be interested in viewing the HTML
pictorial transcript in a sequential manner. Rather, the user may be only interested in those portions of the transcript that pertain to a particular topic. In such cases the user may wish to perform a keyword search of the HTML pictorial transcript. The HTML generator can perform the search on the closed-captioned text and emphasize those portions of the transcript that contain the keyword. For example, images that appear immediately prior to and after the occurrence of a keyword may be displayed at full resolution while other images may be displayed at a smaller size and resolution. FIG. 3 shows an example of this format after a search for the word "Tokyo." The smaller images may be hypertext links to the corresponding full sized images. In some cases, particularly for large HTML pictorial transcripts, hypertext anchors may be used in place of the small images to reduce bandwidth. If the keyword appears more than once in the transcript, a chain of links may be created among the individual occurrences of the word. For example, in FIG. 3, the arrows denote a link to other occurrences of the term "Tokyo." The 1 o HTML pictorial transcript may also include hypertext anchors to other HTML documents which contain material supplementary to, or related to, the information in the transcript.
The HTML generator may create an index page for the HTML
pictorial transcript using conventional methods such as linguistic techniques, for example.. FIG. 4 shows one example of such an index page, which may be located as the first page of the document. The index may contain links to the individual pages of the transcript. The index may also include other information such as index terms obtained by linguistic analysis techniques. In FIG. 4, a portion of the index is 2o available for the user to list additional keywords to serve as index terms.
The index terms may be hypertext links to those locations in the transcript where the terms appear.
Similar to the HTML documents previously discussed, HTML
pictorial transcripts in which the representative frames are each 2s associated with a corresponding audio segment may be arranged in a variety of different formats. For example, the individual representative frames may serve as links to the audio segment. Alternatively, anchors may be associated with the representative frames. By clicking on the anchors the respective audio segments are played.
It will be appreciated that those skilled in the art will be able to devise numerous arrangements which, although not explicitly shown or described herein, embody the principles of the invention. Accordingly, all such alternatives, modifications and variations which fall within the spirit and broad scope of the appended claims will be embraced by the principles of the invention. For example, while the invention has been described as electronic data representing a condensed version of a video 1o program that is formatted as an HTML document for the World Wide Web, the invention is more generally applicable to such data that is formatted in any hypertext language suitable for electronic retrieval on a computer or over a communications network.
Claims (27)
1. A method for automatically providing a compressed rendition of a video program in a format suitable for electronic searching and retrieval, said method comprising the steps of:
receiving electronic data representing a condensed version of a video program, said video program having a video component and a second information-bearing media component associated therewith, said electronic data representation including a representative frame from each segment of the video component of the video program and a portion of said second media component associated with said segment;
automatically transforming said electronic data representation into a hypertext format to form a hypertext pictorial transcript;
recording said hypertext pictorial transcript in an electronic medium ; and wherein said hypertext pictorial transcript has at least one standard page layout, selectable from a plurality of standard page layouts, that includes a subset of representative frames selected by at least one criterion that reduces bandwidth.
receiving electronic data representing a condensed version of a video program, said video program having a video component and a second information-bearing media component associated therewith, said electronic data representation including a representative frame from each segment of the video component of the video program and a portion of said second media component associated with said segment;
automatically transforming said electronic data representation into a hypertext format to form a hypertext pictorial transcript;
recording said hypertext pictorial transcript in an electronic medium ; and wherein said hypertext pictorial transcript has at least one standard page layout, selectable from a plurality of standard page layouts, that includes a subset of representative frames selected by at least one criterion that reduces bandwidth.
2. The method of claim 1 wherein said second media component is closed-caption text.
3. The method of claim 1 wherein said second media component is audio and wherein said portions of said audio component are represented by hypertext anchors.
4. The method of claim 1 wherein said hypertext format is hypertext markup language.
5. The method of claim 1 wherein said hypertext pictorial transcript comprises a plurality of hypertext pages, each of said pages having a prescribed maximum size.
6. The method of claim 5 wherein said prescribed maximum size is determined by a maximum number of frames per page.
7. The method of claim 5 wherein said plurality of hypertext pages are interconnected by hypertext links.
8. The method of claim 1 wherein said hypertext pictorial transcript has at least one standard page layout.
9. The method of claim 1 wherein said hypertext pictorial transcript has a user customizable page layout.
10. The method of claim 8 wherein said standard page out layout includes a subset of representative frames selected by at least one criterion that reduces bandwidth.
11. The method of claim 10 wherein said criterion removes substantially redundant representative frames.
12. The method of claim 11 further comprising the step of replacing said substantially redundant frames with hypertext anchors.
13. The method of claim 10 wherein said criterion removes alternating ones of sequentially occurring representative frames.
14. The method of claim 10 wherein said criterion removes representative frames below a prescribed image size.
15. The method of claim 10 wherein said criterion removes representative frames above a prescribed image size.
16. The method of claim 10 wherein said criterion removes representative frames that differ from other representative frames by less than a prescribed amount.
17. The method of claim 10 wherein said criterion removes representative frames taken from segments below a threshold length.
18. The method of claim 10 wherein said criterion removes representative frames taken from advertisements.
19. The method of claim 1 further comprising the steps of generating and recording an index page to the hypertext pictorial transcript.
20. The method of claim 19 wherein said index page includes links to individual pages of the hypertext pictorial transcript.
21. The method of claim 20 wherein said index page includes hypertext index terms indexed to pages of the hypertext pictorial transcript.
22. The method of claim 1 wherein said hypertext pictorial transcript has a plurality of standard page layouts selectable by a user.
23. The method of claim 1 further comprising the step of transmitting said hypertext pictorial transcript over a communications network.
24. The method of claim 23 wherein said network is the World Wide Web.
25. The method of claim 1 wherein said hypertext pictorial transcript comprises a plurality of hypertext pages.
26. The method of claim 25 wherein said hypertext pages are divided based on topic segmentation.
27. The method of claim 25 wherein said hypertext pages are divided based on a change in closed-caption format.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US08/679,976 US6098082A (en) | 1996-07-15 | 1996-07-15 | Method for automatically providing a compressed rendition of a video program in a format suitable for electronic searching and retrieval |
US679,976 | 1996-07-16 |
Publications (2)
Publication Number | Publication Date |
---|---|
CA2208709A1 CA2208709A1 (en) | 1998-01-16 |
CA2208709C true CA2208709C (en) | 2001-02-06 |
Family
ID=24729149
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CA002208709A Expired - Lifetime CA2208709C (en) | 1996-07-15 | 1997-06-24 | Method for providing a compressed rendition of a video program in a format suitable for electronic searching and retrieval |
Country Status (5)
Country | Link |
---|---|
US (5) | US6098082A (en) |
EP (1) | EP0820025B1 (en) |
JP (1) | JPH1098684A (en) |
CA (1) | CA2208709C (en) |
DE (1) | DE69707770T2 (en) |
Families Citing this family (116)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6098082A (en) * | 1996-07-15 | 2000-08-01 | At&T Corp | Method for automatically providing a compressed rendition of a video program in a format suitable for electronic searching and retrieval |
JP3688822B2 (en) * | 1996-09-03 | 2005-08-31 | 株式会社東芝 | Electronic medical record system |
US6263507B1 (en) * | 1996-12-05 | 2001-07-17 | Interval Research Corporation | Browser for use in navigating a body of information, with particular application to browsing information represented by audiovisual data |
JP4072233B2 (en) * | 1997-03-24 | 2008-04-09 | キヤノン株式会社 | Information processing device |
US7263659B2 (en) * | 1998-09-09 | 2007-08-28 | Ricoh Company, Ltd. | Paper-based interface for multimedia information |
US7596755B2 (en) | 1997-12-22 | 2009-09-29 | Ricoh Company, Ltd. | Multimedia visualization and integration environment |
US7954056B2 (en) | 1997-12-22 | 2011-05-31 | Ricoh Company, Ltd. | Television-based visualization and navigation interface |
CN1867068A (en) | 1998-07-14 | 2006-11-22 | 联合视频制品公司 | Client-server based interactive television program guide system with remote server recording |
US7266782B2 (en) * | 1998-09-09 | 2007-09-04 | Ricoh Company, Ltd. | Techniques for generating a coversheet for a paper-based interface for multimedia information |
US7215436B2 (en) * | 1998-09-09 | 2007-05-08 | Ricoh Company, Ltd. | Device for generating a multimedia paper document |
US7263671B2 (en) * | 1998-09-09 | 2007-08-28 | Ricoh Company, Ltd. | Techniques for annotating multimedia information |
JP2000194726A (en) * | 1998-10-19 | 2000-07-14 | Sony Corp | Device, method and system for processing information and providing medium |
US6453348B1 (en) | 1998-11-06 | 2002-09-17 | Ameritech Corporation | Extranet architecture |
US6859799B1 (en) | 1998-11-30 | 2005-02-22 | Gemstar Development Corporation | Search engine for video and graphics |
US6492998B1 (en) * | 1998-12-05 | 2002-12-10 | Lg Electronics Inc. | Contents-based video story browsing system |
JP2000184351A (en) * | 1998-12-11 | 2000-06-30 | Office Noa:Kk | Moving picture distribution method and its system |
US6473778B1 (en) | 1998-12-24 | 2002-10-29 | At&T Corporation | Generating hypermedia documents from transcriptions of television programs using parallel text alignment |
US6185527B1 (en) | 1999-01-19 | 2001-02-06 | International Business Machines Corporation | System and method for automatic audio content analysis for word spotting, indexing, classification and retrieval |
US6647535B1 (en) * | 1999-03-18 | 2003-11-11 | Xerox Corporation | Methods and systems for real-time storyboarding with a web page and graphical user interface for automatic video parsing and browsing |
US7010801B1 (en) | 1999-06-11 | 2006-03-07 | Scientific-Atlanta, Inc. | Video on demand system with parameter-controlled bandwidth deallocation |
US6817028B1 (en) | 1999-06-11 | 2004-11-09 | Scientific-Atlanta, Inc. | Reduced screen control system for interactive program guide |
US7992163B1 (en) | 1999-06-11 | 2011-08-02 | Jerding Dean F | Video-on-demand navigational system |
US6876991B1 (en) | 1999-11-08 | 2005-04-05 | Collaborative Decision Platforms, Llc. | System, method and computer program product for a collaborative decision platform |
US7412643B1 (en) * | 1999-11-23 | 2008-08-12 | International Business Machines Corporation | Method and apparatus for linking representation and realization data |
US7200857B1 (en) | 2000-06-09 | 2007-04-03 | Scientific-Atlanta, Inc. | Synchronized video-on-demand supplemental commentary |
US8516525B1 (en) | 2000-06-09 | 2013-08-20 | Dean F. Jerding | Integrated searching system for interactive media guide |
US7975277B1 (en) | 2000-04-03 | 2011-07-05 | Jerding Dean F | System for providing alternative services |
US7934232B1 (en) | 2000-05-04 | 2011-04-26 | Jerding Dean F | Navigation paradigm for access to television services |
US8069259B2 (en) | 2000-06-09 | 2011-11-29 | Rodriguez Arturo A | Managing removal of media titles from a list |
US7962370B2 (en) | 2000-06-29 | 2011-06-14 | Rodriguez Arturo A | Methods in a media service system for transaction processing |
US7103906B1 (en) | 2000-09-29 | 2006-09-05 | International Business Machines Corporation | User controlled multi-device media-on-demand system |
KR20190096450A (en) | 2000-10-11 | 2019-08-19 | 로비 가이드스, 인크. | Systems and methods for delivering media content |
US7340759B1 (en) | 2000-11-10 | 2008-03-04 | Scientific-Atlanta, Inc. | Systems and methods for adaptive pricing in a digital broadband delivery system |
NO20020417L (en) * | 2001-01-25 | 2002-07-26 | Ensequence Inc | Selective viewing of video based on one or more themes |
US20030163815A1 (en) * | 2001-04-06 | 2003-08-28 | Lee Begeja | Method and system for personalized multimedia delivery service |
US8060906B2 (en) * | 2001-04-06 | 2011-11-15 | At&T Intellectual Property Ii, L.P. | Method and apparatus for interactively retrieving content related to previous query results |
US20030088687A1 (en) | 2001-12-28 | 2003-05-08 | Lee Begeja | Method and apparatus for automatically converting source video into electronic mail messages |
US8108892B1 (en) | 2001-05-03 | 2012-01-31 | Comcast Cable Holdings, Llc | Interactive television network and method including content searching |
US8479238B2 (en) | 2001-05-14 | 2013-07-02 | At&T Intellectual Property Ii, L.P. | Method for content-based non-linear control of multimedia playback |
US7508946B2 (en) | 2001-06-27 | 2009-03-24 | Sony Corporation | Integrated circuit device, information processing apparatus, memory management method for information storage device, mobile terminal apparatus, semiconductor integrated circuit device, and communication method using mobile terminal apparatus |
US8006262B2 (en) | 2001-06-29 | 2011-08-23 | Rodriguez Arturo A | Graphic user interfaces for purchasable and recordable media (PRM) downloads |
US7526788B2 (en) | 2001-06-29 | 2009-04-28 | Scientific-Atlanta, Inc. | Graphic user interface alternate download options for unavailable PRM content |
US7512964B2 (en) | 2001-06-29 | 2009-03-31 | Cisco Technology | System and method for archiving multiple downloaded recordable media content |
US7496945B2 (en) | 2001-06-29 | 2009-02-24 | Cisco Technology, Inc. | Interactive program guide for bidirectional services |
US7203380B2 (en) * | 2001-11-16 | 2007-04-10 | Fuji Xerox Co., Ltd. | Video production and compaction with collage picture frame user interface |
US7743347B2 (en) | 2001-11-19 | 2010-06-22 | Ricoh Company, Ltd. | Paper-based interface for specifying ranges |
US7495795B2 (en) | 2002-02-21 | 2009-02-24 | Ricoh Company, Ltd. | Interface for printing multimedia information |
US7703044B2 (en) | 2001-11-19 | 2010-04-20 | Ricoh Company, Ltd. | Techniques for generating a static representation for time-based media information |
US7149957B2 (en) | 2001-11-19 | 2006-12-12 | Ricoh Company, Ltd. | Techniques for retrieving multimedia information using a paper-based interface |
US7788080B2 (en) * | 2001-11-19 | 2010-08-31 | Ricoh Company, Ltd. | Paper interface for simulation environments |
US8635531B2 (en) | 2002-02-21 | 2014-01-21 | Ricoh Company, Ltd. | Techniques for displaying information stored in multiple multimedia documents |
US8539344B2 (en) * | 2001-11-19 | 2013-09-17 | Ricoh Company, Ltd. | Paper-based interface for multimedia information stored by multiple multimedia documents |
US7747655B2 (en) | 2001-11-19 | 2010-06-29 | Ricoh Co. Ltd. | Printable representations for time-based media |
US7861169B2 (en) | 2001-11-19 | 2010-12-28 | Ricoh Co. Ltd. | Multimedia print driver dialog interfaces |
KR100411437B1 (en) * | 2001-12-28 | 2003-12-18 | 엘지전자 주식회사 | Intelligent news video browsing system |
US7334251B2 (en) | 2002-02-11 | 2008-02-19 | Scientific-Atlanta, Inc. | Management of television advertising |
JP4597675B2 (en) | 2002-08-24 | 2010-12-15 | マスクレス・リソグラフィー・インコーポレーテッド | Continuous direct write photolithography |
US20040095377A1 (en) * | 2002-11-18 | 2004-05-20 | Iris Technologies, Inc. | Video information analyzer |
US7493646B2 (en) | 2003-01-30 | 2009-02-17 | United Video Properties, Inc. | Interactive television systems with digital video recording and adjustable reminders |
US7275159B2 (en) | 2003-08-11 | 2007-09-25 | Ricoh Company, Ltd. | Multimedia output device having embedded encryption functionality |
US7864352B2 (en) | 2003-09-25 | 2011-01-04 | Ricoh Co. Ltd. | Printer with multimedia server |
US8077341B2 (en) | 2003-09-25 | 2011-12-13 | Ricoh Co., Ltd. | Printer with audio or video receiver, recorder, and real-time content-based processing logic |
JP2005108230A (en) | 2003-09-25 | 2005-04-21 | Ricoh Co Ltd | Printing system with embedded audio/video content recognition and processing function |
US8161388B2 (en) | 2004-01-21 | 2012-04-17 | Rodriguez Arturo A | Interactive discovery of display device characteristics |
US7882436B2 (en) * | 2004-03-10 | 2011-02-01 | Trevor Burke Technology Limited | Distribution of video data |
US8274666B2 (en) | 2004-03-30 | 2012-09-25 | Ricoh Co., Ltd. | Projector/printer for displaying or printing of documents |
US7779355B1 (en) | 2004-03-30 | 2010-08-17 | Ricoh Company, Ltd. | Techniques for using paper documents as media templates |
US8086575B2 (en) | 2004-09-23 | 2011-12-27 | Rovi Solutions Corporation | Methods and apparatus for integrating disparate media formats in a networked media system |
US8185543B1 (en) | 2004-11-10 | 2012-05-22 | Google Inc. | Video image-based querying for video content |
JP2006339817A (en) * | 2005-05-31 | 2006-12-14 | Toshiba Corp | Information processor and display method thereof |
US8189472B2 (en) | 2005-09-07 | 2012-05-29 | Mcdonald James F | Optimizing bandwidth utilization to a subscriber premises |
US7865501B2 (en) * | 2005-11-15 | 2011-01-04 | International Business Machines Corporation | Method and apparatus for locating and retrieving data content stored in a compressed digital format |
US20070157260A1 (en) * | 2005-12-29 | 2007-07-05 | United Video Properties, Inc. | Interactive media guidance system having multiple devices |
AU2011211350B2 (en) * | 2005-12-29 | 2014-01-30 | Rovi Guides, Inc. | An Interactive Media Guidance System Having Multiple Devices |
US9681105B2 (en) | 2005-12-29 | 2017-06-13 | Rovi Guides, Inc. | Interactive media guidance system having multiple devices |
US20070157240A1 (en) * | 2005-12-29 | 2007-07-05 | United Video Properties, Inc. | Interactive media guidance system having multiple devices |
US20070156521A1 (en) | 2005-12-29 | 2007-07-05 | United Video Properties, Inc. | Systems and methods for commerce in media program related merchandise |
US20070156539A1 (en) * | 2005-12-29 | 2007-07-05 | United Video Properties, Inc. | Systems and methods for commerce in media program related merchandise |
US7840977B2 (en) * | 2005-12-29 | 2010-11-23 | United Video Properties, Inc. | Interactive media guidance system having multiple devices |
US8607287B2 (en) | 2005-12-29 | 2013-12-10 | United Video Properties, Inc. | Interactive media guidance system having multiple devices |
US20070174276A1 (en) * | 2006-01-24 | 2007-07-26 | Sbc Knowledge Ventures, L.P. | Thematic grouping of program segments |
US7734579B2 (en) * | 2006-02-08 | 2010-06-08 | At&T Intellectual Property I, L.P. | Processing program content material |
US8200688B2 (en) * | 2006-03-07 | 2012-06-12 | Samsung Electronics Co., Ltd. | Method and system for facilitating information searching on electronic devices |
WO2008001305A2 (en) * | 2006-06-29 | 2008-01-03 | Koninklijke Philips Electronics N.V. | Method and system of key frame extraction |
TW200803468A (en) * | 2006-06-30 | 2008-01-01 | Primax Electronics Ltd | A method for deciding retained image in monitoring apparatus |
US8589973B2 (en) * | 2006-09-14 | 2013-11-19 | At&T Intellectual Property I, L.P. | Peer to peer media distribution system and method |
US8090694B2 (en) * | 2006-11-02 | 2012-01-03 | At&T Intellectual Property I, L.P. | Index of locally recorded content |
US20080112690A1 (en) * | 2006-11-09 | 2008-05-15 | Sbc Knowledge Venturses, L.P. | Personalized local recorded content |
US8402114B2 (en) | 2006-12-28 | 2013-03-19 | Advertising.Com Llc | Systems and methods for selecting advertisements for display over a communications network |
US20090055393A1 (en) * | 2007-01-29 | 2009-02-26 | Samsung Electronics Co., Ltd. | Method and system for facilitating information searching on electronic devices based on metadata information |
US9286385B2 (en) | 2007-04-25 | 2016-03-15 | Samsung Electronics Co., Ltd. | Method and system for providing access to information of potential interest to a user |
US20090019492A1 (en) | 2007-07-11 | 2009-01-15 | United Video Properties, Inc. | Systems and methods for mirroring and transcoding media content |
US20090165049A1 (en) | 2007-12-19 | 2009-06-25 | United Video Properties, Inc. | Methods and devices for presenting and interactive media guidance application |
US20090207316A1 (en) * | 2008-02-19 | 2009-08-20 | Sorenson Media, Inc. | Methods for summarizing and auditing the content of digital video |
US8601526B2 (en) | 2008-06-13 | 2013-12-03 | United Video Properties, Inc. | Systems and methods for displaying media content and media guidance information |
US8259082B2 (en) * | 2008-09-12 | 2012-09-04 | At&T Intellectual Property I, L.P. | Multimodal portable communication interface for accessing video content |
US10063934B2 (en) | 2008-11-25 | 2018-08-28 | Rovi Technologies Corporation | Reducing unicast session duration with restart TV |
JP5470819B2 (en) * | 2008-11-28 | 2014-04-16 | 富士通株式会社 | Spatial data output processing method, program thereof, and data processing system |
US9014546B2 (en) | 2009-09-23 | 2015-04-21 | Rovi Guides, Inc. | Systems and methods for automatically detecting users within detection regions of media devices |
US20110125758A1 (en) * | 2009-11-23 | 2011-05-26 | At&T Intellectual Property I, L.P. | Collaborative Automated Structured Tagging |
US8572488B2 (en) * | 2010-03-29 | 2013-10-29 | Avid Technology, Inc. | Spot dialog editor |
US9361130B2 (en) * | 2010-05-03 | 2016-06-07 | Apple Inc. | Systems, methods, and computer program products providing an integrated user interface for reading content |
WO2012014129A2 (en) * | 2010-07-26 | 2012-02-02 | Koninklijke Philips Electronics N.V. | Determining representative images for a video |
JP2012239028A (en) * | 2011-05-11 | 2012-12-06 | Canon Inc | Image processing apparatus, image processing method, and program |
US8805418B2 (en) | 2011-12-23 | 2014-08-12 | United Video Properties, Inc. | Methods and systems for performing actions based on location-based rules |
US9848276B2 (en) | 2013-03-11 | 2017-12-19 | Rovi Guides, Inc. | Systems and methods for auto-configuring a user equipment device with content consumption material |
JP6168882B2 (en) * | 2013-07-04 | 2017-07-26 | キヤノン株式会社 | Display control apparatus, control method thereof, and control program |
US9674563B2 (en) | 2013-11-04 | 2017-06-06 | Rovi Guides, Inc. | Systems and methods for recommending content |
US20170004859A1 (en) * | 2015-06-30 | 2017-01-05 | Coursera, Inc. | User created textbook |
US10613941B1 (en) * | 2015-09-30 | 2020-04-07 | EMC IP Holding Company LLC | Hybrid NVRAM logging in filesystem namespace |
USD816117S1 (en) * | 2016-06-13 | 2018-04-24 | Apple Inc. | Display screen or portion thereof with icon |
KR101924634B1 (en) * | 2017-06-07 | 2018-12-04 | 네이버 주식회사 | Content providing server, content providing terminal and content providing method |
US11270123B2 (en) * | 2019-10-22 | 2022-03-08 | Palo Alto Research Center Incorporated | System and method for generating localized contextual video annotation |
US11588911B2 (en) | 2021-01-14 | 2023-02-21 | International Business Machines Corporation | Automatic context aware composing and synchronizing of video and audio transcript |
US11736773B2 (en) * | 2021-10-15 | 2023-08-22 | Rovi Guides, Inc. | Interactive pronunciation learning system |
US11902690B2 (en) * | 2021-10-27 | 2024-02-13 | Microsoft Technology Licensing, Llc | Machine learning driven teleprompter |
Family Cites Families (31)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5241671C1 (en) * | 1989-10-26 | 2002-07-02 | Encyclopaedia Britannica Educa | Multimedia search system using a plurality of entry path means which indicate interrelatedness of information |
US5754766A (en) * | 1991-04-17 | 1998-05-19 | Shaw; Venson M. | Integrated circuit system for direct document execution |
US5983004A (en) * | 1991-09-20 | 1999-11-09 | Shaw; Venson M. | Computer, memory, telephone, communications, and transportation system and methods |
US5598557A (en) * | 1992-09-22 | 1997-01-28 | Caere Corporation | Apparatus and method for retrieving and grouping images representing text files based on the relevance of key words extracted from a selected file to the text files |
US5692093A (en) * | 1993-01-08 | 1997-11-25 | Srt, Inc. | Method and apparatus for eliminating television commercial messages |
JP3340532B2 (en) * | 1993-10-20 | 2002-11-05 | 株式会社日立製作所 | Video search method and apparatus |
JPH07175816A (en) * | 1993-10-25 | 1995-07-14 | Hitachi Ltd | Video associative retrieving device and method |
JP3361587B2 (en) * | 1993-12-27 | 2003-01-07 | 株式会社日立製作所 | Moving image search apparatus and method |
US5664227A (en) * | 1994-10-14 | 1997-09-02 | Carnegie Mellon University | System and method for skimming digital audio/video data |
JP3338585B2 (en) * | 1995-05-16 | 2002-10-28 | 富士通株式会社 | Apparatus and method for converting presentation data |
US5708825A (en) * | 1995-05-26 | 1998-01-13 | Iconovex Corporation | Automatic summary page creation and hyperlink generation |
US5710918A (en) * | 1995-06-07 | 1998-01-20 | International Business Machines Corporation | Method for distributed task fulfillment of web browser requests |
US5699458A (en) * | 1995-06-29 | 1997-12-16 | Intel Corporation | Efficient browsing of encoded images |
US5572643A (en) * | 1995-10-19 | 1996-11-05 | Judson; David H. | Web browser with dynamic display of information objects during linking |
US5838682A (en) * | 1995-11-28 | 1998-11-17 | Bell Atlantic Network Services, Inc. | Method and apparatus for establishing communications with a remote node on a switched network based on hypertext dialing information received from a packet network |
US5884056A (en) * | 1995-12-28 | 1999-03-16 | International Business Machines Corporation | Method and system for video browsing on the world wide web |
US5764235A (en) * | 1996-03-25 | 1998-06-09 | Insight Development Corporation | Computer implemented method and system for transmitting graphical images from server to client at user selectable resolution |
DE69702689T2 (en) * | 1996-05-30 | 2000-11-30 | Matsushita Electric Ind Co Ltd | Data conversion device for a pseudo-interactive data communication system |
US5907680A (en) * | 1996-06-24 | 1999-05-25 | Sun Microsystems, Inc. | Client-side, server-side and collaborative spell check of URL's |
US5874986A (en) * | 1996-06-26 | 1999-02-23 | At&T Corp | Method for communicating audiovisual programs over a communications network |
US6098082A (en) * | 1996-07-15 | 2000-08-01 | At&T Corp | Method for automatically providing a compressed rendition of a video program in a format suitable for electronic searching and retrieval |
US5895461A (en) * | 1996-07-30 | 1999-04-20 | Telaric, Inc. | Method and system for automated data storage and retrieval with uniform addressing scheme |
US5872555A (en) * | 1996-10-24 | 1999-02-16 | International Business Machines Corporation | Method and apparatus for customizing colors in a data processing system |
US6122657A (en) * | 1997-02-04 | 2000-09-19 | Networks Associates, Inc. | Internet computer system with methods for dynamic filtering of hypertext tags and content |
US5818935A (en) * | 1997-03-10 | 1998-10-06 | Maa; Chia-Yiu | Internet enhanced video system |
US6138128A (en) * | 1997-04-02 | 2000-10-24 | Microsoft Corp. | Sharing and organizing world wide web references using distinctive characters |
US6154754A (en) * | 1997-09-25 | 2000-11-28 | Siemens Corporate Research, Inc. | Automatic synthesis of semantic information from multimedia documents |
US6166735A (en) * | 1997-12-03 | 2000-12-26 | International Business Machines Corporation | Video story board user interface for selective downloading and displaying of desired portions of remote-stored video data objects |
US6473778B1 (en) * | 1998-12-24 | 2002-10-29 | At&T Corporation | Generating hypermedia documents from transcriptions of television programs using parallel text alignment |
US6647535B1 (en) * | 1999-03-18 | 2003-11-11 | Xerox Corporation | Methods and systems for real-time storyboarding with a web page and graphical user interface for automatic video parsing and browsing |
JP2001256156A (en) * | 2000-03-10 | 2001-09-21 | Victor Co Of Japan Ltd | Control information system and control information transmission method |
-
1996
- 1996-07-15 US US08/679,976 patent/US6098082A/en not_active Ceased
-
1997
- 1997-06-24 CA CA002208709A patent/CA2208709C/en not_active Expired - Lifetime
- 1997-07-15 EP EP97112055A patent/EP0820025B1/en not_active Expired - Lifetime
- 1997-07-15 DE DE69707770T patent/DE69707770T2/en not_active Expired - Lifetime
- 1997-07-15 JP JP9188840A patent/JPH1098684A/en active Pending
-
2000
- 2000-07-24 US US09/625,039 patent/US6581070B1/en not_active Expired - Lifetime
-
2001
- 2001-06-26 US US09/891,371 patent/USRE45316E1/en not_active Expired - Lifetime
-
2003
- 2003-05-05 US US10/429,610 patent/US7657561B2/en not_active Expired - Fee Related
-
2009
- 2009-12-15 US US12/638,107 patent/US20100091182A1/en not_active Abandoned
Also Published As
Publication number | Publication date |
---|---|
US6098082A (en) | 2000-08-01 |
EP0820025A1 (en) | 1998-01-21 |
US7657561B2 (en) | 2010-02-02 |
EP0820025B1 (en) | 2001-10-31 |
DE69707770D1 (en) | 2001-12-06 |
DE69707770T2 (en) | 2002-08-08 |
JPH1098684A (en) | 1998-04-14 |
USRE45316E1 (en) | 2014-12-30 |
US6581070B1 (en) | 2003-06-17 |
US20030208626A1 (en) | 2003-11-06 |
US20100091182A1 (en) | 2010-04-15 |
CA2208709A1 (en) | 1998-01-16 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CA2208709C (en) | Method for providing a compressed rendition of a video program in a format suitable for electronic searching and retrieval | |
US7131059B2 (en) | Scalably presenting a collection of media objects | |
US9372926B2 (en) | Intelligent video summaries in information access | |
US7039633B1 (en) | Hyper video: information retrieval using multimedia | |
US6965890B1 (en) | Hyper video: information retrieval using realtime buffers | |
US6757866B1 (en) | Hyper video: information retrieval using text from multimedia | |
US7779342B2 (en) | Hypervideo: information retrieval at user request | |
US7765574B1 (en) | Automated segmentation and information extraction of broadcast news via finite state presentation model | |
US7802177B2 (en) | Hypervideo: information retrieval using time-related multimedia | |
Yeung et al. | Video visualization for compact presentation and fast browsing of pictorial content | |
Shahraray et al. | Automated authoring of hypermedia documents of video programs | |
US8453189B2 (en) | Method and system for retrieving information about television programs | |
US20040201609A1 (en) | Systems and methods of authoring a multimedia file | |
Srinivasan et al. | " What is in that video anyway?": In Search of Better Browsing | |
US20030191776A1 (en) | Media object management | |
England et al. | I/browse: The bellcore video library toolkit | |
Tseng et al. | Video personalization and summarization system | |
KR20010067762A (en) | A bulletin board system which provides a thumbnail for an picture file and the method thereof | |
Niblack et al. | Web-based searching and browsing of multimedia data | |
EP1481339B1 (en) | Method and device for computer processing of audio-visual content with subtitles | |
Munisamy et al. | Automatic Transformation of Video with Meta-Data into Web Contents | |
WO2006092752A2 (en) | Creating a summarized overview of a video sequence | |
JP2003085191A (en) | User taste making device and summary making device | |
Falchuk | MULTINODeS-a multimedia news-on-demand system. |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
EEER | Examination request | ||
MKEX | Expiry |
Effective date: 20170627 |