WO2008033840A2

WO2008033840A2 - System and methods for creating, collecting, and using metadata

Info

Publication number: WO2008033840A2
Application number: PCT/US2007/078162
Authority: WO
Inventors: David A. Dudas; James H. Kaskade; Kenneth W. O'flaherty
Original assignee: Eyespot Corporation
Priority date: 2006-09-12
Filing date: 2007-09-11
Publication date: 2008-03-20
Also published as: WO2008033840A3

Abstract

A system and related methods comprising an Internet-hosted application service for online storage, editing and sharing of digital video, whereby the application creates, collects and analyzes media metadata pertaining to mixes or portions of mixes (including titles, tags and other keywords) and their usage, as well as static and dynamic user metadata pertaining to personal characteristics and system usage behavior, in order to perform intelligent filtering of the mixes, personalization services, target marketing, and monetization.

Description

SYSTEM AND METHODS FOR CREATING,

COLLECTING, AND USING METADATA

Cross Reference to Related Applications

[01] This application claims the benefit of U.S. Provisional Patent Application number 60/825,364, entitled "SYSTEM AND METHODS FOR COLLECTING DETAILED METADATA IN AN ONLINE DIGITAL VIDEO EDITING PLATFORM AND USING THE METADATA FOR PURPOSES OF CONTENT FILTERING, PERSONALIZATION, TARGET MARKETING AND MONETIZATION," filed on September 12, 2006 and U.S. Provisional Patent Application number 60/869,672 entitled "SYSTEM AND METHODS FOR COLLECTING DETAILED METADATA IN AN ONLINE DIGITAL VIDEO EDITING PLATFORM AND USING THE METADATA FOR PURPOSES OF CONTENT FILTERING, PERSONALIZATION, TARGET MARKETING AND MONETIZATION filed on December 12, 2006.

[02] This application herein incorporates by reference the following PCT Patent

Applications:

Field of the Invention

[03] This invention relates in general to the use of computer technology to create, collect and use metadata in an online video editing environment. Background

[04] Although video technology has been in existence for many years, it has until recently been inaccessible to most people, other than through the medium of television. In particular, ordinary consumers have not had the ability to also be creators of video. But this is changing rapidly.

[05] There are currently approximately 900 million devices in existence worldwide that are capable of producing video: 680 million video camera phones, 180 million video digital cameras, plus 40 million digital camcorders. The ability to record video has proliferated, in particular in the world of cellular phones. The extremely rapid increase in availability of all of these devices, especially camera cellular phones, has generated a mounting need on the part of consumers to find ways of converting their video content into mixes that that they can share with others. This amounts to a core need for two capabilities: video editing and online video sharing.

[06] Figure 1 is a block diagram illustrating a prior art video editing platform including a creation block 199, a consumption block 198, and a media aggregation, storage, manipulation & delivery infrastructure 108. Figure 1 shows with arrows the paths that currently exist for transferring video content from a particular source, including a DSC 100, a DVC 102, a mobile phone 104, and a webcam 106 to a particular destination viewing device including a DVD player 110, a DSTB 112, a DVR 114, a mobile phone 116, a handheld 118, a video iPod 120, or a PC 122. The only destination device that supports content from all input devices is the PC 122. Otherwise, mobile phone 104 can send video content to another mobile phone 116, and a limited number of today's digital camcorders and digital cameras can create video content on DVDs that can then be viewed on the DVD player 110. In general, these paths are fractured and many of the devices in the creation block 199 have no way of interfacing with many of the devices in the consumption block 198. Beyond the highlighted paths through the media aggregation, storage, manipulation & delivery infrastructure 108, no other practical video transfer paths exist today.

[07] In a partial response to these needs, there has been a proliferation of online video-sharing websites. Currently there are over 200 such sites, most of which have come into existence very recently. The video-sharing site YouTube, in particular, has garnered huge popularity, and has become a viewing destination for millions of people worldwide.

[08] Most of the videos available for viewing on such sites are either of poor quality, or are copied from other sources, such as TV programs, TV commercials, or music videos. The copying of commercial video content frequently gives rise to copyright issues - the copying and the redistribution may well infringe the copyright of the owner or creator of the content. Thus video-sharing sites that stream such content to consumers may be infringing copyrights, and may be legally liable.

[09] In addition to potential copyright issues, online video-sharing sites also face the possibility that they may be serving up objectionable content - for example "hate" content or pornographic content that some of their users have posted to the website. With the huge volumes of videos that are posted to websites such as YouTube (over 30,000 per day), it is impossible for the site administrators to review every video prior to making it available for viewing.

[010] The main reason why many of the videos available for viewing on video- sharing sites are of poor quality is that they were shot by amateurs who had no tools available for editing their content. Thus their videos typically suffer from having been shot in one continuous sequence that includes uninteresting or poorly filmed sections. If they had been able to cut out the poor parts, and combine multiple clips from their digital camera and/or camera phone, they would have been able to create a far better video.

[011] Some consumers do use desktop video-editing tools, but they tend to be a minority group of enthusiastic amateur videographers. A solution to this problem has emerged with the advent of a handful of video-sharing sites that also offer online video- editing. By having access to an extremely easy-to-use facility to edit (or "mix") video content online, with features such as trimming, adding transitions or effects to video, adding titles, combining clips, and adding an audio track, consumers are now able to create and share video mixes of considerable quality.

[012] A few video-sharing/editing sites also allow users to reuse the video content of other members of the site (with the author's permission). This has led to active communities of users who "harvest" video content from other users and include the content in their own productions. The content may have been provided by other -A- consumers, or by commercial groups, such as record labels or recording artists, who wish to promote their work by offering up samples (audio, video, or both) for "remixing."

[013] A very recent phenomenon is the proliferation of "mashups" on the online video-sharing sites. Mashups are remixes of commercial content, usually made by amateurs, which are entertaining in their own right (and may even be better than the original from which they are derived). Some mashups infringe the copyright of the owners; other mashups may be legal, and made in response to a promotional campaign by the owners. In either case, there are currently no methods available by which the owners of the original content can derive revenue directly from mashup productions.

[014] Video-sharing sites face a major issue as to how to make money. Most sites offer a free service, but incur major expenses in computer equipment, especially in storage and bandwidth. Various "monetization" solutions exist. One monetization solution is to charge viewers for viewing certain content. Another monetization solution is to support advertising on their website. One advertising approach is to attach an advertisement to a video that is being viewed - in which case there is a need to select an advertisement that is least likely to annoy or offend the viewer (and that may hopefully be of interest or value to the viewer).

[015] When a viewer pays for viewing a video, or when an advertisement is attached to a video, it is appropriate for the website to share the derived revenue with the author of the video. There is thus a need for the site to save detailed information ("metadata") about videos, authors, and viewers, such that it can compute the distribution of advertising or other revenue.

[016] In the case of remixes or mashups, there is a further complication that there are multiple authors involved (e.g., author A reuses videos from author B and author C, where author Cs video is a remix using clips from authors D and E and music from author G). In such cases, there is a need for the site to save additional detailed historical information (metadata) regarding the hierarchy of content and authors contributing to a video production, and to be able to use this metadata to compute the sharing of revenue among all contributors.

[017] In order to increase the enjoyment of visitors to a website, some websites attempt to "personalize" the experience of each user. Personalization typically involves guiding visitors down a viewing path that best matches their interests, and may involve making recommendations (e.g., if user A buys book a certain book ("Book A"), and user B has also bought book A, recommend other books that user B bought). Video-sharing websites have an opportunity to personalize the viewing experiences of their users, as well as the creative experiences of their authors, by similarly making appropriate recommendations regarding videos to view, blogs to look at, or groups to visit (groups usually feature videos around a particular subject). Such personalization needs to be supported by the collection and analysis of metadata regarding videos, groups, blogs and authors. Data mining techniques such as predictive modeling, clustering and association are well suited to such personalization tasks (similar techniques are often referred to today as "Recommendation Systems" and "Collaborative Filtering").

[018] The metadata collected about the media and its authors can also be used to mitigate the problems discussed earlier relating to content: to analyze and mine the content and authorship of videos in order to detect, flag, and possibly remove, potentially objectionable media - media that may infringe copyrights, or may include hate content or pornographic content.

[019] A video-sharing/video-editing site may have partner websites that distribute or publish videos created by the site's users. In such syndication cases, there may be a need to select an advertisement without any knowledge about the viewer of the video. This can be accomplished by collecting and analyzing metadata relating to each video, and matching advertisements solely based on what is known about each video, e.g., the subject matter (or category), title, descriptive tags or other related keywords of the video and of all of the component parts of the video (including any harvested content and its authors). Various data mining techniques, including predictive modeling and clustering, are well suited to this task.

[020] With video syndication, there may also be a need for a site to provide its partner sites access, e.g., via an application programming interface ("API"), to the metadata that it collects and analyzes, regarding the media and the authors, such that the partner sites can perform content filtering, personalization, target marketing/advertising, or monetization. Summary

[021] A system and methods are disclosed for creating, collecting, and using metadata associated with digital content in an online video editing environment. In one aspect a system and related methods create, collect, and analyze media metadata pertaining to the content and its usage, as well as user metadata pertaining to personal characteristics and behavior, in order to perform intelligent filtering of content, personalization services, target marketing, and monetization.

[022] Within the online video editing environment, a user can create "mixes". A

"mix" includes a number of digital content components and transformations that are applied to the content. Video content, clips, pictures, sounds, special effects, title frames, etc., make up the mix. In one aspect, the online video editing environment receives portions of the mixes (video, clips, pictures, sounds) via an upload process. A system administrator, content moderator, or similar person can apply metadata tags to a selected number of the components of a mix and the mixes themselves (e.g., the most popular). The mixes also can have metadata tags applied, either voluntarily or as a requirement, by a user who has created the mix. An algorithm(s) can apply additional metadata tags to the mixes. The mixes along with the associated metadata tags are then made available to users of the online video editing community and an action (e.g., filtering, advertising, etc.) can be performed based on the metadata tags.

[023] The user metadata can include static personal profile information, such as age and gender, as well as dynamic behavioral and social networking information based on the uploading, editing, harvesting, sharing, group membership and video viewing actions of each user.

[024] The media metadata can include static information covering clip titles, photograph titles, audio/music titles, mix titles, album titles, blog titles, and associated commentary text, and descriptive tags applied to content. The media metadata can also include dynamic information relating to the frequency with which individual pieces of content are streamed, copied, harvested or shared, and by whom, and to their geographic and demographic distribution.

[025] A single mix can contain inherited content metadata and user metadata, as described, originating from any and all content sources used to create the final product, and their authors. The system can use the user and media metadata, including inherited metadata, to automatically filter content, in order to identify selected types of content, including suspected pornographic content, hate content, and copyright infringement.

[026] The system also can use the metadata to dynamically personalize and enhance each user's experience on the website, whether connected through personal computer, mobile device, or digital home network, and to make appropriate recommendations to users regarding social networking and regarding viewing, mixing and sharing of content. Metadata can also be used in relation to an upload process, wherein certain tags can cause only a portion, or certain segments, of the digital content to be uploaded.

[027] The system also can use the metadata for purposes of target marketing and advertising, including dynamic selection of the most appropriate advertisement to attach to mixes that are being viewed (pre-rolls, post-rolls, dynamically placed interstitials, or overlays), or to place around the video player, or to communicate as website advertisements, email advertisements, or through other means. The system also can use the metadata for monetization purposes, by automating the allocation of revenues associated with advertising and marketing.

[028] Underlying methods can include various data mining techniques applied to the metadata, including prediction, classification, clustering, association, and the use of automatically generated video categories and user categories, in combination with business rules, to assist in real-time decision-making.

[029] Many of the abilities of the system are enabled by the environment whereby multiple users can contribute to a mix (represented by an authorship hierarchy or "tree"), and whereby the system maintains static and dynamic metadata pertaining to each of the authors in the authorship tree, and pertaining to the mix used at each level in the tree.

[030] The system can be used on a dedicated website or its functionality can be served to different websites seeking to provide users with enhanced video editing capabilities and metadata use, which can include, for example, redirection away from an external website to a website local to the system, wherein the present functionality is employed while still maintaining the look and feel of the external website. Other features and advantages of the present invention will become more readily apparent to those of ordinary skill in the art after reviewing the following detailed description and accompanying drawings.

Brief Description of the Drawings

[031] The details of the present invention, both as to its structure and operation, may be gleaned in part by study of the accompanying drawings, in which like reference numerals refer to like parts, and in which:

[032] Figure 1 is a block diagram illustrating a prior art video editing platform.

[033] Figure 2 is a block diagram illustrating the functional blocks or modules in an example architecture.

[034] Figure 3 is a block diagram illustrating an example online video platform.

[035] Figure 4 is a block diagram illustrating an example online video editor.

[036] Figure 5 is a block diagram illustrating an example video preprocessing application.

[037] Figure 6A is a schematic representation summarizing an example data structure of the metadata associated with an example mix created and collected using the system.

[038] Figure 6B is a flowchart illustrating an exemplary process for creating, collecting, and using metadata.

[039] Figure 7 is a flowchart illustrating an exemplary process for filtering mixes using metadata.

[040] Figure 8 is a schematic representation summarizing an example of personalization using metadata.

[041] Figure 9 is a flowchart illustrating an exemplary process for the implementation of categorization using predictive modeling. [042] Figure 10 is a flowchart illustrating an exemplary process for the implementation of categorization using clustering.

[043] Figure 11 is a flowchart illustrating an exemplary process for the implementation of categorization using a combination of predictive modeling and clustering.

Detailed Description

[044] Certain examples as disclosed herein provide for the use of computer technology to create, collect, and use metadata associated with content. The content includes mixes. Mixes include one or more of all of the different types of content, including video clips (clips), songs, audio, pictures, special effects, etc. As used herein, the term mix refers interchangeably to the mix itself or a portion of the mix. In one aspect, a system and related methods create, collect, and analyze media metadata pertaining to the mixes and their usage, as well as user metadata pertaining to personal characteristics and behavior, in order to perform intelligent filtering of the mixes, personalization services, target marketing, and monetization.

[045] After reading this description it will become apparent to one skilled in the art how to implement the invention in various alternative examples and alternative applications. However, although various examples of the present invention are described herein, it is understood that these examples are presented by way of example only, and not limitation. As such, this detailed description of various alternative examples should not be construed to limit the scope or breadth of the present invention as set forth in the appended claims.

[046] Those of skill will further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the embodiments disclosed herein can often be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled persons can implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the invention. In addition, the grouping of functions within a module, block, circuit or step is for ease of description. Specific functions or steps can be moved from one module, block or circuit without departing from the invention.

[047] Referring now to the Figures, Figure 2 is a block diagram illustrating the functional blocks or modules in an example computer architecture. In the illustrated example, a system 200 includes an online video platform 206, an online video editor 202, a preprocessing application 204, as well as a content creation block 208 and a content consumption block 210. The illustrated blocks 200-210 can be implemented on one or more servers or server systems.

[048] The content creation block 208 can include input data from multiple sources that are provided to the online video platform 206, including personal video creation devices 212, personal photo and music repositories 214, and personally selected online video resources 216.

[049] In one example, video files are uploaded by consumers from their personal video creation devices 212. Note that as used herein the term "video" is used as shorthand to refer to any one of a plurality of types of data which include not only video files, but also images, audio files, special effects, and other data formats capable of being combined and/or included in a mix. The personal video creation devices 212 can include DSCs, DVCs, cellular phones equipped with video cameras, and webcams. In another example, input to the online video platform 206 is obtained from other sources of digital video and non-video content selected by the user. Non-video sources include the personal photo and music repositories 214, which are stored on the user's PC, or on the video server, or on an external server, such as a photo-sharing application service provider ("ASP"). Additional video sources include websites that publish shareable video content, such as news organizations or other external video-sharing sites, which are designated as personally selected online video resources 216.

[050] The preprocessing application 204 can perform a number of tasks. One such task is segmenting the video content into a number of manageable upload segments, which can be uploaded in parallel, one at a time, or in different orders depending on how they are prioritized or used. For example, a user can interact with one of the upload segments via the online video editor 202, which can cause a re-prioritization of that particular upload segment.

[051] The preprocessing application 204 also contains a metadata module 299.

The metadata module is shown as being a component of the preprocessing application 204, but it can also reside, in whole or in part, in the online video editor 202 or the online video platform 204. The metadata module 299 performs a variety of functions such as collecting, generating, and/or creating new metadata tags for uploaded content either through algorithms, through inheritance of tags from earlier mixes that were borrowed from to make the current mix, or manually by a system administrator, a person tasked with indexing/moderating content, or a user.

[052] The online video editor 202 (also referred to as the Internet-hosted application service) can be used on a dedicated website or its functionality can be served to different websites seeking to provide users with enhanced video editing capabilities. For example, a user can go to any number of external websites providing an enhanced video editing service. The present system 200 can be used to enable the external websites to provide the video editing capabilities while maintaining the look and feel of the external websites. In that respect, the user of one of the external websites may not be aware that they are using the present system 200other than the fact that they are using functionality provided by the present system 200. In a transparent manner then, the system 200 serves the application to the external IP address of the external website and provides the needed function while at the same time running the application in a manner consistent with the graphical user interface ("GUI") that is already implemented at the external IP address. Alternatively, a user of the external website can cause the invocation of a redirection and GUI recreation module 230, which can cause the user to be redirected to one of the servers used in the present system 200 which provides the needed functionality while at the same time recreating the look and feel of the external website.

[053] The online video editor 202 also contains a filtering module 298 and a targeting module 297. The filtering module 298 and the targeting module 297 are shown as being components of the online video editor 202, but they can also reside, in whole or in part, in the preprocessing application 204 or the online video platform 206. The filtering module 298 can perform the scoring and application of business rules (defined in more detail later), which can be used to classify mixes and portions of mixes so that actions can be taken. For example, the filtering module 298 can filter out mixes and portions of mixes that are pornographic in nature while allowing non-objectionable mixes to pass through. The targeting module 297 can be used for a variety of purposes, including applying business rules to mixes using the metadata, which, for example, allow it to make recommendations of other mixes, products, or services to a user. The targeting module 297 can use the metadata to place specific advertisements in association with the mix. The advertisements can be placed such that they are of specific potential relevance to the user based on an analysis of the associated metadata.

[054] Video content can be output by the online video platform 206 to the content consumption block 210. Content consumption block 210 can be utilized by a user of a variety of possible destination devices, including, but not limited to, mobile devices 218, computers 220, DVRs 222, DSTBs 224, and DVDs 226. The mobile devices 218 can be, for example, cellular phones or PDAs equipped with video display capability. The computers 220 can include PCs, Apples, or other computers or video viewing devices that download content via the PC or Apple, such as handheld devices (e.g., PalmOne), or an Apple video iPod. The DVDs 226 can be used as a media to output video content to a permanent storage location, as part of a fulfillment service.

[055] Delivery by the online video platform 206 to the mobile devices 218 can use a variety of methods, including a multimedia messaging service ("MMS"), a wireless application protocol ("WAP"), and instant messaging ("IM"). Delivery by the online video platform 206 to the computers 220 can use a variety of methods, including: email, IM, uniform resource locator ("URL") addresses, peer-to-peer file distribution ("P2P"), or really simple syndication ("RSS").

[056] The functions and the operation of the online video platform 206 will now be described in more detail with reference to Figure 3. In the illustrated example of Figure 3, the online video platform 206 includes an opt- in engine module 300, a delivery engine module 302, a presence engine module 304, a transcoding engine module 306, an analytic engine module 308, and an editing engine module 310.

[057] The online video platform 206 can be implemented on one or more servers, for example, Linux servers. The system 200 can leverage open source applications and an open source software development environment. The system 200 has been architected to be extremely scalable, requiring no system reconfiguration to accommodate a growing number of service users, and to support the need for high reliability.

[058] The application suite can be based on AJAX, Flash, or a hybrid ("client") where the online application behaves as if it resides on the user's local computing device, rather than across the Internet on a remote computing device, such as a server. The client architecture allows users to manipulate data and perform "drag and drop" operations, without the need for page refreshes or other interruptions.

[059] The opt-in engine module 300 can be a server, which manages distribution relationships between content producers in the content creation block 208 and content consumers in the content consumption block 210. The delivery engine module 302 can be a server that manages the delivery of content from content producers in the content creation block 208 to content consumers in the content consumption block 210. The presence engine module 304 can be a server that determines device priority for delivery of mixes to each consumer, based on predefined delivery preferences and detection of consumer presence at each delivery device.

[060] The transcoding engine module 306 can be a server that performs decoding and encoding tasks on media to achieve optimal format for delivery to target devices. The analytic engine module 308 can be a server that maintains and analyzes statistical data relating to website activity and viewer behavior. The editing engine module 310 can be a server that performs tasks associated with enabling a user to edit mixes efficiently in an online environment.

[061] The functions and the operation of the online video editor 202 will now be described in more detail with reference to Figure 4. Figure 4 is a block diagram illustrating an example online video editor 202. In the illustrated example, the online video editor 202 includes an interface module 400, input media 402a-h, and a template 404. A digital content aggregation and control module 406 can also be used in conjunction with the online video editor 202 and thumbnails 408 representing the actual video files can be included in the interface module 400.

[062] The online video editor 202 can be an Internet-hosted application, which provides the interface module 400 for selecting video and other digital content (e.g., music, voice, photos) and incorporating the selected contents into mixes via the digital content aggregation and control module 406. The digital content aggregation and control module 406 can be software, hardware, and/or firmware that enables the modification of the mix as well as the visual representation of the user's actions in the interface module 400. The input media 402a-h can come from such input sources as the shutterfly website 402a, remote media 402b, local media 402c, the napster web service 402d, the real rhapsody website 402e, the garage band website 402f, the flickr website 402g and webshots 402h. The input media 402a-h can be media that the user has selected for possible inclusion in the mix and can be represented as the thumbnails 408 in a working "palette" of available content elements, in the main window of the interface module 400. The input media 402a-h can be of diverse types and formats, which can be aggregated together by the digital content aggregation and control module 406.

[063] The thumbnails 408 are used as a way to preview content in parallel with the upload process. The thumbnails 408 can be generated in a number of manners. For example, the thumbnails can be single still frames created from certain sections within the content. Alternatively, the thumbnails 408 can include multiple selections of frames (e.g., a quadrant of four frames). In another example, the thumbnails can include an actual sample of the video in seconds (e.g., a 1 minute video could be represented by the first 5 seconds). In yet another example, the thumbnails 408 can be multiple samples of video (e.g., 4 thumbnails of 3 second videos for a total of 12 seconds). In general, the thumbnails 408 are a method of representing the media to be uploaded, whereby the process of creating the representation and uploading it takes a significantly less amount of time than either uploading the original media or compressing and uploading the original media.

[064] The online video editor 202 allows the user to choose (or can create) the template 404 for the mix. The template 404 can represent a timeline sequence and structure for insertion of contents into the mix. The template 404 can be presented in a separate window at the bottom of the screen, and the online video editor 202 via the digital content aggregation and control module 406 can allow the user to drag and drop the thumbnails 408 (representing mixes or portions of mixes) in order to insert them into the timeline to create the new mix. The online video editor 202 can also allow the user to select from a library of special effects to create transitions between scenes in the video. The work-in-progress of a particular video project can be shown in a separate window. [065] On completion of the project, the online video editor 202 allows the user to publish the video to one or more previously defined galleries / archives 410. Any new video published to the gallery / archive 410 can be made available automatically to all subscribers 412 to the gallery. Alternatively, the user can choose to keep certain mixes private or to only share the mixes with certain users.

[066] The functions and the operation of the preprocessing application 204 will now be described in more detail with reference to Figure 5. Figure 5 is a block diagram illustrating an example preprocessing application. In the illustrated example, the preprocessing application 204 includes a data model module 502, a control module 504, a user interface module 506, foundation classes 508, an operating system module 510, a video segmentation module 512, a video compression module 514, a video segment upload module 516, a video source 518, the metadata module 299, the filtering module 298, the targeting module 297, and video segment files 520.

[067] In one example, the preprocessing application 204 is written in C++ and runs on a Windows PC, wherein the foundation classes 508 includes Microsoft foundation classes ("MFCs"). In this example, an object-oriented programming model is provided to the Windows APIs. In another example, the preprocessing application 204 is written, wherein the foundation classes 508 are in a format suitable for the operating system module 510 to be the Linux operating system. The video segment upload module 516 can be an application that uses a Model-View-Controller ("MVC") architecture. The MVC architecture separates the data model module 502, the user interface module 506, and the control module 504 into three distinct components.

[068] In operation, the preprocessing application 204 automatically segments, compresses, and uploads content from the user's PC, regardless of length. The preprocessing application 204 uses the video segmentation module 512, the video compression module 514, and the video segment upload module 516 respectively to perform these tasks. The uploading method works in parallel with the online video editor 202, allowing the user to begin editing the content immediately, while the content is in the process of being uploaded. The content can be uploaded to the online video platform 206 and stored as one or more video segment files 520, one file per segment. [069] The video source 518 can be a digital video camcorder or other video source device. In one example, the preprocessing application 204 starts automatically when the video source 518 is plugged into the user's PC. Thereafter, it can automatically segment the content stream by scene transition using the video segmentation module 512, and save each of the video segment files 520 as a separate file on the PC.

[070] From the user's perspective, content would be captured on any number of devices at the video source block 518. Once the user captured the content (i.e., on their camcorder, cellular phone, etc.) it would be transferred to a local computing device, such as the hard drive of a client computer with Internet access.

[071] Alternatively content can be transferred to a local computing device whereby an intelligent uploader can be deployed. In some cases, the content can be sent directly from the video source 518 over a wireless network (not shown), then over the Internet, and finally to the online video platform 206. This alternative bypasses the need to involve a local computing device or a client computer. However, this example is most useful when the content is either very short, or highly compressed, or both.

[072] In the case that the content is not compressed or long or both, and, therefore, relatively large, it is typically transferred first to a client computer where an intelligent uploader is useful. In this example, an upload process is initiated from a local computing device using the video segment upload module 516, which facilitates the input of lengthy content. To that end, the user would be provided with the ability to interact with the user interface module 506. Based on user input, the control module 504 controls the video segmentation module 512 and the video compression module 514, wherein the content is segmented and compressed into the video segment files 520. For example, lengthy content can be segmented into 100 upload segments, which are in turn compressed into 100 segmented and compressed upload segments.

[073] Each of the compressed video segment files 520 begin to be uploaded separately via the video segment upload module 516 under the direction of the control module 504. This can occur by each of the upload segments being uploaded in parallel. Alternatively each of the upload segments can be uploaded in order, the largest segment first, the smallest segment first, or any other manner. [074] As the content is being uploaded, the online video editor 202 is presented to the user. Through a user interface provided by the user interface module 506, thumbnails representing the video segments in the process of being uploaded are made available to the user. The user would proceed to edit the video content via an interaction with the thumbnails. For example, the user can be provided with the ability to drag and drop the thumbnails into and out of a timeline or a storyline, to modify the order of the segments that will appear in the final edited content.

[075] The system 200 (Figure 2) is configured to behave as if all of the content represented by the thumbnails is currently in one location (i.e., on the user's local computer) despite the fact that the content is still in the process of being uploaded by the video segment upload module 516. When the user performs an editing action on the thumbnails by dragging one of the thumbnails into a storyline, the upload process can be changed. For example, if the upload process was uploading all of the compressed upload segments in sequential order and the user dropped an upload segment representing the last sequential portion of the content into the storyline, the upload process can immediately begin to upload the last sequential portion of the content, thereby lowering the priority of the segments that were currently being uploaded prior to the user's editing action.

[076] All of the user's editing actions are saved by the online video editor 202.

Once the content is uploaded completely (including the prioritized upload segments and the remaining upload segments), the saved editing actions are applied to the completely uploaded segments. In this manner, the user may have already finished the editing process and logged off or the user can still be logged on. Regardless, the process of applying the edits only when the content is finished uploading saves the user from having to wait for the upload process to finish before editing the content. Once the final edits are applied, various capabilities exist to share, forward, publish, browse, and otherwise use the uploaded content in a number of ways.

[077] The online video editor 202 (also called the Internet-hosted application service) provides a mixing platform, whereby users can create, edit, track, and share mixes. The online video editor 202 can provide multiple opportunities for users to create mixes not only by mixing (e.g., trimming, adding transitions or effects to video, adding titles, combining, and overlaying) their own uploaded portions of mixes (video clips, photos, audio/music tracks), but also by "remixing" other mixes "harvested" (reused) from other users, including commercial users who offer high-quality audio and video portions of mixes representing the works of a variety of artists. A resultant mix may thus consist of content from multiple sources: it can be a mix (created from content originating solely from the user), or a remix (created from content originating solely from other sources, or from a combination of the user's and others' content).

[078] Furthermore, each portion of a mix used in a mix can itself be a mix; and, more importantly, each portion of a mix used in a remix can itself be a mix or even a remix - it can consist of multiple pieces of content from one or more sources. And the remixing activity can be recursive, whereby there can be several or many levels of remixing, resulting in a hierarchy of authors who have contributed content to the resulting mix. (Note: in this document, the term mix will be used regardless of whether the production is a mix or a remix and is intended to refer to both.)

[079] The metadata module 299 maintains detailed information regarding the media sources and authors of the component parts of every mix (including clips or mixes, photos, and audio/music tracks). Each mix can contain inherited media metadata and user metadata, originating from all sources used to create the final product and from all levels in the hierarchy of contributing authors. The online video mixing activities of the user base thus provide a rich source of metadata which the system 200 is in a unique position to mine, and which user's of the system 200 are in a unique position to track, in order to gain an understanding of the media, as well as the authors of the media (i.e., the subscribers to the Internet-hosted service) and the relationships between them.

[080] The metadata module 299 can maintain the metadata in one or more tables in a data warehouse. The metadata module 299 can aggregate all of the metadata into a single data store. Online analytical processing (OLAP) or other data mining tools can then be used by the metadata module 299. The metadata module 299 can also maintain operational data stores for action at defined intervals (e.g., daily). The operational data stores can be used for real-time action, such as personalization, monetization, filtering, etc. For example, any scoring associated with the data warehouse can be input back to the operational data stores for real-time action, such as creating new categorization models, which are used to score users, mixes, or for matching relevant advertisements. [081] In another embodiment, the metadata module 299 maintains the metadata in a data structure of a computing device, (e.g. in a set of tables in a relational database). Figure 6A is a schematic representation exemplifying one such data structure; in this schematic, each of the types of metadata ("Author" metadata, "Media" metadata, "Title & Tags" metadata, "Community" metadata, "Segmentation" metadata, and "Behavioral" metadata) might be stored and maintained in one or more tables in a relational database, and would be used for real-time actions such as personalization, monetization or filtering. The metadata in these tables might also be aggregated on a regular basis into a single data store for purposes of analysis and data mining, (e.g., in order to build up-to-date categorization models that can be fed back to the operational system for scoring users and/or videos.)

[082] Returning again to Figure 5, the media metadata that metadata module 299 maintains for each mix or portion of the mix can include keywords from the title, descriptive tags (assigned by the author, and possibly also by administrators or other users), and a category. The system 200 can automatically assign the media category (based on the mix's tags and keywords), using data mining techniques, including: categorization, classification, clustering and association. The system administrators maintain a living master list of categories based on observation of the overall content base of the website. Categories are useful in matching up advertisements with mixes that users are viewing, as well as in personalizing user experiences and in identifying potentially objectionable mixes.

[083] In one embodiment, advertising categories are both manually created

(based on the types of advertisers), and new advertising categories (e.g., more specific clusters of users) can be automatically discovered through a data mining process. For example, within an automotive advertising segment, the system 200 may find a cluster of automotive mixes or users creating automotive mixes that are highly valuable targets for automotive advertisers. Categories and categorization are discussed in detail later. Various types of categories exist, including video categories, group categories, author categories, and viewer categories. Categories are a form of metadata, and are stored in the metadata data structure.

[084] In addition to media metadata, the metadata module 299 also maintains user demographic metadata (including declared interests), and metadata relating to other creative and sharing activities of its users. The sharing metadata can include mobile phone numbers, email addresses and website uniform resource locators ("URLs") of destinations to which users have sent mixes. The additional creative activity metadata includes metadata related to the user's upload activity including the types of mixes the user uploads, how often the user uploads the mixes, etc. Creative activity metadata also can include sources, including whether the user uses a cellular phone or a local computing device to initiate the upload.

[085] Creative activity metadata also can include names of albums or other types of collections of files that the user creates or elects to include in their account which can consist of groups of mixes, videos, photos, and music around a topic, titles, and keyword content of blogs that they write about, mixes they have created, titles and keyword content of posts they write to the public discussion forum, flagging history (prior cases of creating/posting objectionable mixes), and groups that they create or belong to. The metadata module 299 maintains metadata about every group, including the owner, each member, and each mix posted to the group for viewing (along with keywords from accompanying commentary). Each mix in turn has its own associated metadata, including its author, title, descriptive tags, and creation history (as previously described).

[086] The metadata module 299 makes this additional user metadata available to users of the system 200 for every author in the creation hierarchy of a mix, providing further insights into the activities and relationships of the user base, and providing a rich source for filtering, personalization, target marketing, and monetization.

[087] The metadata module 299 can also mine metadata relating to creative users of the system 200 in combination with additional metadata that is maintained relating to other users who are not authors but are simply viewers of the available mixes. The suite of media and user metadata is used to segment users into viewer categories and author categories. The combination of author categories, viewer categories and video categories are used as inputs into real-time decisions regarding filtering, personalization, target marketing, and monetization.

[088] In one aspect the metadata module 299 maintains metadata relating to: the current mix being viewed; the current viewer (profile and behavior metadata); clips/mixes previously created by the current viewer (if a member); the author of the current mix (media, profile and behavior metadata); inherited media metadata including all video clips, video mixes, audio/music, photos contributing to the current mix; and inherited user metadata including profile and behavior metadata relating to all authors of clips, mixes, audio/music, photos contributing to the current mix (at all levels of remixing).

[089] In one embodiment the metadata module 299 maintains metadata for each mix, for each viewer, and for each author in a data structure in the memory of a computing device. For each mix, the metadata module 299 can maintain the following: a video category, assigned by the application of a predictive or a clustering model in the metadata module 299; a title; tags; music genre, title, artist, and record label; author; a creation date and time, a ranking; public/private status; a number of views (by mix location); a number of views (by viewer location); a number of video ad impressions (by mix location); a number of page ad impressions (by mix location); a number of shares by email; a number of shares by mobile device; a number of shares by publishing destination; a number of times harvested for reuse in other mixes; links to harvested mixes; title, tags and author of each portion of the mix the mix; and title, tags and author of each photo included in the mix.

[090] For each viewer the metadata module 299 can maintain metadata in a data structure in the memory of a computing device for each viewer category, assigned by a predictive and/or a clustering model in the metadata module 299, which can be based on: a category, title, tags, music genre of the currently viewed mix; categories, titles, tags, music, and genre of all mixes viewed during current session; categories and names of all groups visited during the current session; search terms used during the current session; categories, titles, tags, music, and genre of mixes most frequently viewed in prior sessions; categories and names of groups most frequently visited during prior sessions; search terms most frequently used in prior sessions; the number of logins per month; an average time spent on site per login; an average number of mixes viewed per login; and a clickthrough behavior (on advertisements).

[091] For each author, the metadata module 299 can maintain in a data structure of a memory of a computing device the following metadata: an author category, assigned by a predictive or a clustering model in the metadata module 299 based on an author's viewer category and supporting metadata; media metadata, which include titles and tags of media uploaded (videos, photos, music), titles and tags of media harvested (videos, photos, music), and titles and tags of mixes created; community metadata, which includes album titles, album categories, assigned by a predictive or a clustering model, blog titles, keywords from blog entries, names of groups created, keywords from groups created, names of groups belonged to, keywords from groups belonged to, titles of posts to public discussion forum, keywords from posts to public discussion forum, names of destinations shared to, and method shared ("send by email", "send to mobile device (cellular phone)", "published", etc.); behavioral data - (historical) including a number of logins per month, an average time spent on site per login, a total number of clips uploaded, a total number of cellular phone clips uploaded, a total number of photos uploaded, a total number of music tracks uploaded, a total number of mixes created, a total number of individual tags created, a total number of a tag's applications (one tag can be applied multiple times), a most favored music genre, a number of albums created, a number of posts to a public forum, a number of groups created, a number of groups belonged to, a number of destinations shared to, and method shared ("send by email", "send to cellular phone", "published"), a number of times flagged for posting hate content, a number of times flagged for posting pornographic content, and a number of times flagged for posting content with copyright infringement issues; behavioral data - (Real-time) including a number and titles of clips uploaded this date, a number and titles of cellular phone clips uploaded this date, a number and titles of photos uploaded this date, a number and titles of music tracks uploaded this date, a number of mixes created this date, a number and names of tags applied this session, a number and names of new groups created this session, a number and names of new groups joined this session, and categories, titles, tags, music genre of all mixes viewed during current session; and demographic data, including an age group, a gender, an area code+3, a state of residence, a ZIP code, an income group, whether the author is a homeowner, whether the author is a parent of minor children, whether the author is a sports enthusiast, whether the author is an exercise enthusiast, whether the author is a movie enthusiast, whether the author is an automobile enthusiast, whether the author is a travel enthusiast, whether the author is a nature enthusiast, whether the author is a pet owner, and whether the author is a gamer.

[092] Figure 6A is a schematic representation summarizing an example data structure of the metadata associated with an example mix created and collected using the system 200. Figure 6A includes metadata blocks 600, 602, 604 and 606 associated with the mixes A, C, B, and D respectively. Mix A includes two mixes by other authors (mix B and mix C) with associated metadata blocks 604 and 602. Metadata block 600 incorporates by pointers the metadata blocks 602 and 604 associated with the two mixes B and C, which are included in mix A. Alternatively, the content of the metadata blocks of incorporated mixes can be copied in the metadata block of the mix.

[093] Mix B is associated with the metadata block 604. Mix B includes a further mix by another author (mix D). Mix D is associated with metadata block 606, which is associated with block 604 via a pointer.

[094] Each mix has associated behavioral metadata tags, author metadata tags, community metadata tags, media metadata tags, title metadata tags, and categories 699. Figure 6A summarizes examples of the metadata associated with each mix. Figure 6A does not show metadata relating to a viewer of the mix, which can also be included.

[095] The metadata block 600 includes metadata tags, which are associated with mix A and/or inherited (e.g., incorporated by the already existing metadata tags associated with portions of mixes that were incorporated into the mix), including behavioral metadata tags, author metadata tags, community metadata tags, media metadata tags, and title metadata tags. The behavioral metadata tags include tags associated with, for example, views (by location of media, viewer location, etc.), sharing (email, mobile, publishing destination), ranking, harvesting of data, use in mixes, flagging classification, and campaign information. The author metadata tags include tags associated with, for example, demographics, media ownership, media creation behavior, media viewing behavior, media sharing behavior, social network, flagging behavior, and forum statistics.

[096] The community metadata tags include tags associated with, for example, blog post & comment key word information, group post & comment key word information, and album classifications. The community metadata tags include tags associated with, for example, time & date of original mix upload, components of the mix, and private/public status. The community metadata tags include tags associated with, for example, title keywords, tag key words, tags associated with audio to text conversion.

[097] The metadata maintained and created by the metadata module 299 (Figure

2) enables intelligent filtering of mixes or portions of mixes by the filtering module 298. Immediately following upload or creation, the filtering module 298 can score all newly uploaded portions of mixes for the likelihood of including objectionable content (or other types of predetermined content) and for likelihood of copyright infringement. A predictive model accomplishes real-time scoring built from metadata relating to prior cases of flagged portions of mixes and authors (versus non-flagged mixes and authors). After the filtering module 298 scores each new portion on the mix, it applies business rules to the score to determine the appropriate action. Possible actions taken by the filtering module 298 include (but are not limited to): flagging the author and the portion of the mix for later review (during next business day, for example), flagging the author and the portion of the mix for immediate review (triggering email or a phone call to a designated reviewer, for example), and flagging the author and immediately taking the portion of the mix down from the website.

[098] In all cases, the targeting module 297 can automatically assign the categories 699. Thereafter, the filtering module 298 can score and filter the mix. The author, a system administrator, or an automated audio-to-text conversion program can assign the metadata tags.

[099] Figure 6B is a flowchart illustrating an exemplary process for creating, collecting, and using metadata. The preprocessing application 204, the online video platform 206, and the online video editor 202 (Figure 2) can carry out the process. In the illustrated example, the preprocessing application 204 receives a number of mixes at step 1200. The preprocessing application 204 receives one or more of the mixes via an upload process as previously described, or the mixes already reside in the system 200 or at a network location accessible to the preprocessing application 204. At step 1202, a system administrator assigns selected metadata tags to a selected amount of the mixes. The selected amount of the mixes can be a predefined percentage of all of the mixes. The selected amount may be, for example, one hundred of the most popular mixes. The system administrator is a trusted source capable of manually handling the classification of the selected amount of mixes in an effective manner, such as applying the appropriate descriptive words to the mixes or selecting appropriate metadata tags. This includes for example, the system administrator applying a category 699 (Figure 6A) or any number of types of metadata tags.

[0100] At step 1204, the system 200 receives user selected metadata tags for a selected amount of the mixes. The user can input the second metadata tags either optionally or as a required component of the online video editing function. Typically, the user will create keywords or select keywords from a pre-existing list or menu that the user feels are applicable to the mix for searching purposes. The system 200 can also require the user to input a minimum, such as two metadata tags for each mix or portion of a mix that they upload. These metadata tags include, for example, media metadata tags, title metadata tags, and others.

[0101] At step 1206, the filtering module 298 applies third metadata tags to the mixes. In one embodiment, the filtering module 298 uses a predictive modeling algorithm or a cluster modeling algorithm, or both. The results of the algorithm produce new metadata tags, which are used to further classify the mix including for example the category 699 (Figure 6A).

[0102] In one embodiment, the predictive modeling algorithm is a computer program that embodies a neural net or similar algorithm. The algorithm has inputs, which define the independent variables and a single output ("score") which is the dependent variable. The algorithm is first trained by applying known inputs and associated output to the model. A number of numerical coefficients are calculated based on the application of inputs/output. Through a series of "training" passes, the algorithm is ready to apply a set of unknown inputs to which it calculates an output score, all based on history associated with the training data set.

[0103] For example, the filtering module 298 can take a group of users who actually responded to an online advertisement, together with a similar-sized random group of users who did not respond to the advertisement. The output variable would be "responded to an advertisement" and this would be set to "Yes" (or "1") for those users who did respond, and to "No" (or "0") for those users who did not; the input variables would be all the metadata associated with the user, the video they were watching, the advertisement metadata, etc. The filtering module 298 would use the data as the "training data set" to program the predictive model, which would learn a pattern from the training data representing the input variable combinations that most closely distinguish between responders and non-responders. This pattern would be saved as a model - a reusable program. Then the system 200 would use this model to score new users and their probability of responding to the same advertisement. If the score is high, (e.g. one within a range from zero to one), then they would be served the advertisement. [0104] Clustering models use algorithms to take a set of input variables and assign the set of the input variables to one of several output values. The input variables or "attributes" are used to define the population, and the output values are used to define groups or clusters or subsets of the population where each cluster have similar attributes. For example, in one embodiment the filtering module 298 directs the clustering algorithm to define ten clusters using all the metadata available around users. At the conclusion of the algorithm data processing, there may be ten distinct user clusters, which define different types of users. A detailed description of each cluster is obtained by looking at the strength of coefficients associated with each attribute used to create the clusters. For example, the filtering module 298 may find that one cluster was created based predominantly on three attributes (which ultimately were the most statistically significant) including age, gender, and video category viewed the most - specifically, there was a large group of users between the ages of 14-17, males, watching skateboard videos. The filtering module 298 might define this cluster as "skate youth" that advertisers specifically want to target.

[0105] In another embodiment, if someone has uploaded content, which later is flagged as inappropriate, then a score can be increased for that user, "inappropriate content user score". Any future mixes, which contain content from that user can be scored as "suspect". The more material from that user or other users with "inappropriate content user scores" would only increase the "suspect" score, raising that video up in propriety to be reviewed by someone. Ultimately, a counter in a database for each user and a flag for each piece of content may all be tied together with some business logic.

[0106] Thereafter, at step 1208, the online video editing environment makes the mixes available to the users. For example, a user can create a new mix that includes an existing mix. The new mix automatically includes all of the metadata associated with the new and existing mixes. Then, the targeting module 297 or the filtering module 298 performs an action at step 1210. The targeting module 297 or the filtering module 298 base the action, at least in part, on the metadata tags. The action can include associating a targeted advertisement with the mix, making a suggestion based on the nature of the mix, filtering out an inappropriate mix, etc.

[0107] Figure 7 is a flowchart illustrating an exemplary process for filtering mixes using metadata. In one embodiment, the filtering module 298 shown in Figure 2 carries out this process. In the illustrated example, the filtering module 298 determines at step

700 whether the video segment upload module 516 is uploading a new mix or a portion of a mix. Step 700 repeats until the upload is complete. When the upload is complete, the newly uploaded mix is scored at step 702. Scoring assigns a number (or "score") indicating the probability of the uploaded mix including objectionable material; the higher the score, the higher the likelihood of objectionable material. Scoring can take the form of any number of scoring algorithms that are based in part on the metadata that is available, which is associated with the mix. For example, the filtering module 298 can score a category known to contain pornographic or otherwise objectionable mixes highly whereas it can score a category known to contain children's or otherwise non-objectionable mixes lower. Scoring will be defined in more detail later. Following the scoring of the new mix, business rules are applied to the scored, uploaded mix at step 704. The business rules can be input manually or dynamically. The business rules can be accessible to the filtering module 298 for application to the mix. The business rules in one aspect can include a list of commands in the form of If A, then B. Business rules will be defined in more detail later.

[0108] Following the scoring and the application of the business rules, the filtering module 298 determines an appropriate action to take. In the example of Figure 7 three actions are specified although others are possible. If a first action is required at step 706, the filtering module 298 flags the author and the mix for later review at step 708. This can occur when the scoring and the application of the business rules indicate that the mix should be reviewed later but is most likely not objectionable or does not need to be immediately removed.

[0109] If, on the other hand, a second action is required at step 710, the filtering module 298 flags the author and the mix for immediate review at step 712. This can occur when the mix is intermediate in nature (i.e., the system 200 does not score the mix to be outright objectionable but the score also indicates it might be objectionable and should be reviewed as soon as possible). If, on the other hand, a third action is required at step 714, the filtering module 298 flags the author and the mix is immediately taken down from the website at step 716. This can occur when the mix is determined to be most likely objectionable and should not be stored by the system 200 even for a short period of time. [0110] In one example, the filtering module 298 uses metadata factors to score the mix to screen for objectionable or copyright infringing content, where the metadata factors include (but are not limited to) the following: categories, keywords, titles, tags of the current mix; categories of prior mixes created by the author, plus their keywords, titles, and tags; categories of the current author; flagging behavior of the current author (for prior mixes); (if a mix) categories, keywords, titles, and tags of components used in the current mix (including clips, mixes, photos, and audio/music tracks); (if a mix) categories, keywords, titles, and tags associated with authors of components that were used in the current mix; (if a mix) flagging behavior of authors of components that were used in the current mix; and (if a mix) categories, keywords, titles, and tags associated with other mixes/remixes that use any of the same components that are used in the current mix.

[0111] The metadata created and maintained by the metadata module 299 provides insights into the interests, the viewing and mixing history, and the social networking activities of its users. The targeting module 297 can use these insights to dynamically personalize and enhance each user's experience on the website, whether connected through a personal computer, a mobile device, or a digital home network.

[0112] In combination with business rules, the targeting module 297 can apply a variety of data mining techniques to the metadata in order to make appropriate recommendations to users. One scheme uses predictive and clustering models, which the targeting module 297 applies to mixes, viewers, authors and groups in order to classify them into the categories 699. The targeting module 297 uses the categories 699 to recommend similar mixes to view, or groups to join having similar categorical content.

[0113] In another embodiment, the targeting module 297 uses clustering models, which it applies to viewers, authors, and groups in order to identify similar interests and make recommendations, such as to view or join a particular group (this is enhanced by the fact that groups already indicate clusters of users with similar interests). In yet another embodiment, the targeting module 297 uses association models, which it applies to mix hierarchies, to sets of titles of mixes viewed in a single session, and to sets of names of groups visited in a single session, in order to identify groupings of similar mixes or groups, and to make mixing, viewing or joining recommendations. [0114] The following are examples of personalization based on usage of the metadata that the targeting module 297 maintains:

[0115] Example 1 : Group membership is a type of social networking that provides a natural form of clustering/user-based collaborative filtering, without the need for modeling to identify the clusters - by being members of a group, members have a high likelihood of having similar interest(s). Business rule: If users B and D are members of groups XXX and YYY, and user B joins group ZZZ, then recommend to user D to look at or join group ZZZ.

[0116] Example 2: Mix hierarchy provides a looser, unconscious type of social networking similar to item-based collaborative filtering, where content (clips or mixes) that are used together indicate potential common interests. Business rule: If user A makes a mix using a mix from user B, and users B's mix includes clips from users C and D, and users B, C and D are all members of group ZZZ, then recommend to user A to look at the blogs of users B, C and D, and to look at or join group ZZZ.

[0117] Example 3: A clustering model (user-based collaborative filtering) applied to group memberships identifies that members of group XXX tend to also be members of groups YYY and ZZZ. Action: Recommend to members of group XXX who are not already members of groups YYY and ZZZ that they consider joining them.

[0118] Example 4: Association rules (item-based collaborative filtering) induced from sets of hierarchies of mixes (category & title) that were used in making mixes identify clips/mixes that are frequently used together (analogous to market baskets of items purchased together). Action: When User A harvests clip X to use in a mix, recommend looking at clips Y and Z that are frequently used together with clip X in other mix hierarchies.

[0119] Example 5: Association rules (item-based collaborative filtering) induced from sets of titles of mixes viewed within a session identify mixes that are frequently viewed together (analogous to market baskets). Action: When User A views mix V, recommend also viewing mixes U and W that are frequently viewed together with mix V.

[0120] Example 6: Association rules (item-based collaborative filtering) induced from sets of names of groups visited within a session identify groups that are frequently visited together (analogous to market baskets). Action: When user B visits group G, recommend also visiting groups F and H which are frequently viewed together with group G.

[0121] Example 7: Predictive and clustering models induced from metadata relating to prior mixes are used to classify new mixes into a video category (mixes of the same category have similar features - a form of item-based collaborative filtering); video categories are then used to make viewing recommendations to viewers. Action: When viewer F views mix V of category CCC, recommend also viewing newly created mixes S and T that are of same category.

[0122] Example 8: Predictive and clustering models induced from metadata relating to mixes posted to groups are used to classify new groups into a group category, where the group category is from the same set of categories that is defined for mixes; group categories are then used to make recommendations regarding viewing or joining groups. Action: If user A makes a mix of category CCC, or if the mix includes a clip or mix of category CCC, recommend that user A visit group XXX which is also of category CCC (i.e., group XXX features mixes of category CCC).

[0123] Figure 8 is a schematic representation summarizing an example of personalization using metadata. In the illustrated example, personalization is shown, together with additional examples of personalization actions that the targeting module 297 takes based on the available metadata. Figure 8 includes clips 806 and 808 and mixes 802, 804, and 810. In the present example of Figure 8, user A has uploaded the mix 802 via a cellular phone multi-media message service ("MMS"). User A has made a mix that includes a mix 804 from user B, who in turn used clips 806 and 808 from users C and D.

[0124] User B's mix 804 is of the category "YoungKids", and has an audio track with music by the Black Eyed Peas (music genre HipHop), which user A retains on her mix 802. User Cs clip 806 is of the category "Nature"; and user D's clip 806 is of the category "Cats". User A gives her new mix 802 the title "LittleJoey" and the tags "Kids", "Preschool" and "Playground", the mix 802 inherits various tags from its harvested clips, including "Swingset", "HipHop", "BossaNova", "SergioMendes", "Beach", "Hawaii", "Sunset", "Cat", and" CatTrick". Using a predictive and/or clustering model based on earlier examples of mixes and their tags and categories, the targeting module 297 automatically classifies the new mix 802 into category "YoungKids". User A shares her new mix 802 with user E, who has made mixes 810 of the categories "Nature", Cats", and "Cartoons" and who publishes mixes 810 to a website 812, such as MySpace.com. Users B and D are members of the Group ZZZ, and C frequently views mixes from Group ZZZ. User D publishes frequently to a blog site 814, such as Blogger.com.

[0125] Based on analysis of metadata relating to users A, B, C, D and E, and their related media, the targeting module 297 can personalize user A's experience in visiting a website in the online video platform 202, which can be part of the online video platform 202 or can be the result of a redirection from an external website and a recreation of the look and feel of the external website, via the redirection and GUI recreation module 230. The targeting module 297 can make a number of recommendations 899. For example, the targeting module 297 can recommend to user A that she:

a) View or join Group ZZZ b) Read the blogs of users B, C and D c) Look at Group YYY that has music available from artists of genre "HipHop" d) Look at Group XXX that has a weekly cellular phone video contest e) Look at Group WWW that is of category "Cats" (and features mixes about cats) f) Look at Group VVV that is of category "YoungKids" (and features mixes about young children) g) Look at Group UUU that is of category "Cartoons" (and features cartoon-style mixes) h) Look at Group TTT that is of category "Nature" (and has several mixes with the tags "Beach" and "Hawaii") i) Look at user E' s My Space page j) Look at user D's Blogger page

[0126] The insights into the interests, the viewing and mixing history, and the social networking activities of the user base revealed by the system's metadata maintained, collected, and created by the metadata module 299 are also useful for purposes of target marketing and advertising. Multiple forms of advertising are supported, including: dynamic selection of the most appropriate advertisement to attach to mixes that are being viewed. Advertisements can be attached in various ways, including as pre-rolls (inserted at the start), post-rolls (inserted at the end), dynamically placed interstitials (inserted between mixes); overlays (expandable boxed inserted over mixes); advertisements placed around the video player, on the video player page; other website advertisements; email advertisements; and any other advertising means associated with the mixes.

[0127] With reference again to the example in Figure 8, the targeting module 297 can also perform several forms of intelligent target marketing and targeted advertising to user A, based on the available metadata.

[0128] Target marketing includes the following example. Since user A uploaded mixes from her mobile device (cellular phone) using MMS, the targeting module 297 can suggest to user A that she consider purchasing an application that she can download to her phone. The application makes it easier to upload cellular phone mixes, and offers other useful features relating to searching, viewing, sharing mixes on the cellular phone. The targeting module 297 can also promote to her other products and services related to video- enabled cellular phones, or to cellular phones for young children.

[0129] Targeted advertising includes the following example. Based on the available metadata, the targeting module 297 can also target advertisements to user A that relate to: children's products, toy stores (via user A's metadata); cat products, pet stores (via metadata about user D, from the mix tree); nature lovers (e.g., travel and organic/environmental/ergonomic products) - via metadata about user C (from the mix tree), and user E (from mix sharing); music of genres "HipHop" and "BossaNova" (via metadata about user B, from the mix tree); and music by Black Eyed Peas and by Sergio Mendes (via metadata about user B, from the mix tree).

[0130] Another example of targeted advertising is shown in the following example where a viewer selects a mix to view. If the selected mix is less than 10 seconds in length, the targeting module 297 skips the advertising; otherwise: the targeting module 297 retrieves the category of the selected mix. If the video category is "undefined," the targeting module 297 assigns the advertising randomly. If the video category is not "undefined," the targeting module 297 retrieves a rotating list of advertisements requesting this video category as top priority. If the targeting module 297 has already served the advertisement on the top of the rotating list to this viewer on any of the viewer's last several viewings in this session, for example 3, the targeting module 297 can skip this advertisement in favor of the next advertisement in the list until the targeting module 297 has selected 3 advertisements (or until the list is exhausted).

[0131] If the list is exhausted, the targeting module 297 can proceed next to a rotating list, representing advertisements requesting this video category as the next highest priority, and the process repeats until the targeting module 297 has selected several, for example 3, advertisements (or until all lists are exhausted). The targeting module 297 can then score each advertisement based on the priority of the matching video category, as requested for this advertisement (High, Medium or Low), plus additional optional criteria (if available), which include: a viewer demographic group versus priority of demographic groups requested for this advertisement. Next, the targeting module 297 matches the mix's tags, if any, versus the priority of tags requested with this advertisement. For example, one suggested scoring scheme is as follows: the targeting module 297 scores a matching video category as: High = 6, Medium = 4, Low = 2; the targeting module 297 scores a matching demographic group as: High = 6, Medium = 4, Low = 2; the targeting module 297 scores matching tags as: High = 3, Medium = 2, Low = 1 (if there are several matching tags, the targeting module 297 can limit the matching tag score to amaximum of 6 points total). The targeting module 297 can then serve the top-scoring advertisement (based on the combined points score).

[0132] Targeting based on metadata relating to user A is relatively straightforward; other targeting is more subtle, exploiting the explicit (via sharing) and implicit (via the mix tree) social relationships between users that the targeting module 297 uncovers by the maintained metadata. Because user A shares her mix 802 with user E, it is assumed via business rules that there is a likelihood that user A also shares user E's interest in Nature (and, by implication, products associated with nature lovers). The likelihood that user A is interested in Nature is reinforced by the fact that user A indirectly harvests a mix from user C that is also about Nature. Similarly, because user A keeps an audio track harvested from user B 's mix 804, it is assumed via business rules that user A will be interested in music associated with the harvested track (both the specific artists and the music genres).

[0133] Several metadata components show important details surrounding the sharing and harvesting relationships among users; these include video categories, video tags, and other possible keywords extracted from video titles, album names, blog titles, forum posts, and from text associated with these. Video categories and video tags are especially strong indicators of content that can be used to infer shared interests. Video tags describing the content of the mix are supplied by the creator (or can optionally be added by system administrators or by other users).

[0134] One aspect of the targeting module 297 and the filtering module 298 applies a variety of data mining techniques to the media and user metadata in order to perform intelligent content filtering, personalization, target marketing/advertising and monetization. Real-time decisions can be made automatically based on a combination of business rules and data mining outputs.

[0135] Examples of applied data mining techniques include: predictive modeling, which can be applied to media and author metadata in order to score new media for likelihood of including objectionable mix; predictive modeling and clustering, which can be applied to mixes, groups, viewers and authors in order to identify target marketing opportunities with viewers and authors, and to select the most appropriate advertisement to attach to a mix that is being viewed; predictive modeling and clustering, which can be applied to mixes, groups, viewers and authors in order to segment them into categories, which are used for multiple purposes; predictive modeling and clustering, which can be applied to viewers, authors and groups in order to identify similar interests and make recommendations, such as to view or join a particular group (this is sometimes referred to as "user-based collaborative filtering"); and association models, which can be applied to mix hierarchies, to sets of titles of mixes viewed in a single session, and to sets of names of groups visited in a single session, in order to identify groupings of similar mixes or groups, and to make mixing, viewing or joining recommendations (this is sometimes referred to as "item-based collaborative filtering").

[0136] Example of business rules that can be applied according to one aspect include: do not serve the same advertisement to the same viewer within the same session; do not make more than three recommendations to a user within one session; and distribute revenue for advertisements based on a revenue-sharing formula.

[0137] The system 200 can use categorization for a variety of purposes. In one embodiment, the targeting module 298 uses a combination of predictive modeling and clustering in order to segment mixes, groups, viewers and authors in order to place them into categories. Mixes and groups can share the same set of categories, based on tags, titles and other keywords associated with mixes, whereas viewer categories and author categories can be based on these plus other metadata elements relating to user behavior.

[0138] Categorization can be particularly valuable in supporting fast, real-time decisions. The targeting module 298 can perform categorization dynamically in real time, such as in determining the category of a newly uploaded mix or portion of a mix. In other cases, the targeting module 298 can use categories assigned previously through a categorization process, such as in selecting in real time the most appropriate advertisement to attach to a mix that a user has requested for viewing.

[0139] The targeting module 298 can take into account, in the selection of targeted advertisements and other marketing, one or more categories in various ways, such as: viewer category alone (e.g., if the viewer is not also an author of any mixes); video category alone (e.g., for syndicated mixes); author category alone (e.g., if an author is creating a mix); viewer and author categories (e.g., if an author is viewing another author's mix); and mix, viewer and author categories.

[0140] Different combinations of categories can be experimented with in different situations, and advertising responses (e.g., clickthrough rates) can be analyzed in order to decide on the most effective method to use in each situation. The following types of data available to the targeting module 297 can be used in selecting the most appropriate advertisement to serve with a mix that a user has selected for viewing, including: the video category of currently selected mix (plus optionally the title and tags); viewer demographic data; viewer category, based on the viewing history of the viewer (categories (plus optionally titles/tags)of viewed mixes, search terms used, groups visited, in the current and the last n sessions); if the viewer is a member of a group, the member category (based on the set of their video categories, plus optionally titles, tags, blog titles, groups created, and groups joined; the member category of the creator of the currently selected mix; member categories of content contributors to the currently selected mix; and viewer clickthrough data - type of product/service of ads that this viewer has clicked on during the current session or today or past week / month / quarter / year. [0141] In the case of syndicated mixes (where a partner site requests a mix to be served), the video category can be used as an indicator of "genre", and the partner site can request mixes of one or more particular genres; in such a case, the targeting module 298 can attach advertisements to the mix based solely on the video category (absent other metadata).

[0142] The targeting module 297 can also use video categories to create

"channels", which are selections of mixes available for viewing that are grouped according to subject matter. The targeting module 297 can organize channels by name on the website, and all publicly viewable mixes belonging to a channel are easily accessible for viewing under their channel title. In the most straightforward form, the targeting module 297 can equate video categories one-for-one with channels, and it can organize mixes by category for viewing. The targeting module 297 can also use channels for personalization, in a similar way to groups. For example, after a viewer views a mix, the targeting module 297 can recommend the viewer to visit the channel that features mixes of the same video category as the viewed mix.

[0143] Video categories constitute the highest-level classification of mixes, and provide a means of matching advertisements to mixes, such that, in real time, the targeting module 297 can select the most appropriate advertisement and attach it to each mix that is being viewed. For example, an "automotive" video category (or an "automotive enthusiast" viewer category) might be identified, based on discovery of a significant cluster of automotive videos (or viewers of such videos); advertisements from manufacturers such as Ford could then be selected to accompany viewings of videos in the "automotive' category, or could be targeted at viewers who have been categorized as "automotive enthusiast".

[0144] Video categories can be assigned automatically based on the mix's tags and other keywords, using data mining techniques. The system administrators can maintain a living master list of video categories based on observation of the overall content base of the website. The system 200 also can maintain an up-to-date set of predictive models and clustering models incorporating the current master list of video categories, together with the tags and keywords that are used as inputs by the models to classify mixes into a category. [0145] Categorizing mixes and categorizing users can involve human intelligence, especially in the initial definition of the set of categories, and then subsequently in regularly updating this set, in order to represent in the most meaningful way the base of mixes and users available to the online video editing environment. The categories should be sufficiently intuitive to enable advertisers (with or without the assistance of the system staff) to select the most appropriate target categories for their ads (or to facilitate automatic matching of ad category to video category and/or user category).

[0146] Once the initial categorization process has been accomplished by applying human intelligence to a selected amount of existing mixes, a combination of predictive and clustering data mining techniques are used to automatically classify further mixes into the established categories.

[0147] Automatic categorization depends on good tagging. In order to derive a meaningful category for any mix, at least two tags should be available that accurately describe the mix content. If the creator of the mix does not provide tags, administrative staff can do so, at least for those mixes where it is important to advertise intelligently (i.e., for the most popular mixes, which will account for the majority of mix viewings).

[0148] The targeting module 297 assigns video categories using various forms of predictive modeling and clustering. In one aspect the targeting module 297 uses a predictive model. Figure 9 is a flowchart illustrating an exemplary process for the implementation of categorization using predictive modeling. In the illustrated example, the targeting module 297 builds a predictive model from a selected amount of popular, tagged mixes, and then applies the model to new mixes as users add them, to automatically classify them into a category.

[0149] At step 900, a system administrator or data analyst assigns a category manually to a selected amount of the most popular mixes. The manual assignment can occur by viewing the mixes, observing the titles and tags of the mix (if any), assigning new tags to the mix, and then assigning the mix to the most appropriate category. The selected amount of the most popular mixes can be a few hundred. For example, as Figure 6B shows at step 1202, the metadata tags the administrator assigns to the mixes may include media metadata tags, for example, and also may include the assignment of a category. [0150] At step 902, the targeting module 297 tests and trains a predictive model based on the manually categorized mixes. The predictive model uses the newly assigned tags as predictive inputs. The predictive model can also take the form of a decision tree (i.e. it may have been created using a decision tree induction algorithm). If the model is in the from of a decision tree, the tree can be enhanced manually based on observed knowledge.

[0151] At step 904, the predictive model analyzes the mixes (including any mixes that were newly uploaded) and automatically categorizes them. To that end, the video platform can require and/or strongly suggest that users apply at least two tags to each new mix. Where feasible, the system administrator can also manually assign tags to the new mixes prior to applying the predictive model, especially the more popular mixes. At step 906, it is determined whether the targeting module 297 was successful in automatically categorizing the mix using the predictive model.

[0152] If not, the mix is categorized as undefined at step 908. After step 908, or if success does occur at step 906 then the predictive model has categorized the mix, so the targeting module 297 determines whether there are any more mixes at step 910. If so, the process repeats at step 904 wherein the predictive model obtains the next mix and the automatic categorization process repeats until all mixes including newly uploaded ones are processed.

[0153] After the predictive model processes all of the mixes at step 910, the targeting module 297 determines whether a time period has passed at step 912. Depending on the system capabilities and size, the time period can be a week, a month, etc. If step 912 is false, step 910 repeats to ensure no new mixes have been uploaded that need categorization. When step 912 becomes true, the process repeats at step 900, wherein the manual categorization of the most popular mix occurs again. During this iterative process new categories can also be included where appropriate. Therefore, as the predictive model is rebuilt and rerun each time, it should improve since it is based on the latest set of categories and a possibly larger set of popular mixes.

[0154] In another aspect, the targeting module 297 uses a clustering model to assign mix categories. Figure 10 is a flowchart illustrating an exemplary process for the implementation of categorization using clustering. In the illustrated example, clustering is used to automatically identify a set of clusters, and the clusters are then manually examined to identify a category that they represent. The clustering model is then applied to new mixes as they are added to automatically assign them to a cluster representing a category.

[0155] At step 1000, tags are manually assigned where necessary to a selected amount of the most popular mixes. For example, as Figure 6B shows at step 1202, the metadata tags the administrator assigns to the mixes also may include the assignment of a category. At step 1002 the targeting module 297 applies a clustering algorithm to the manually tagged mixes; this algorithm segments the mixes into different clusters based on similarities and differences between the mixes and their assigned tags. The output of the clustering algorithm is a cluster number assigned to each mix, plus a clustering model that can be used on subsequent mixes to automatically assign them to a cluster. At step 1004 the targeting module 297 then runs a decision tree induction algorithm on the manually tagged mixes to classify the rules that define each cluster, the decision tree algorithm sets the cluster number for the manually tagged mixes as the target. The output of the algorithm is a decision tree with an associated set of rules indicating which input variables correlate with which cluster. At step 1006 the induced rules for each cluster are manually examined and a category is assigned to each cluster using human intelligence. Several clusters can be determined to represent different subcategories of a major category, in which case the clusters can be assigned to the same category.

[0156] Subsequently, at step 1008, the next mix is obtained; this is the first of a new batch of mixes that will be classified into a cluster using the cluster model created earlier. At step 1010, the cluster model is run to identify the cluster number for the next mix. The cluster number now automatically identifies the category the mix belongs to. At step 1012, the targeting module 297 determines if there are more mixes. If there are more mixes, the process repeats at step 1008 and the next mix is obtained. The process repeats until all remaining mixes, including any new mixes, are processed. After processing all of the mixes, step 1012 becomes false at which time the targeting module 297 determines whether a time period has elapsed at step 1014. The time period can vary depending on the needs and capabilities of the system 200 (Figure 2). The time period can be one week or one month. If the time period has not elapsed, the process repeats at step 1012 to ensure no new mixes exist that have not been processed. Once the time period passes, step 1000 repeats, wherein the most popular mixes are manually clustered again, mix categories are manually assigned with the assistance of a decision tree, and the new clustering model is rerun with regard to all of the remaining mixes.

[0157] In another aspect the targeting module 297 uses a hybrid approach that combines both predictive modeling and clustering to assign video categories. Figure 11 is a flowchart illustrating an exemplary process for the implementation of categorization using a combination of predictive modeling and clustering. In the illustrated example, the targeting module 297 uses a combination of both approaches, fine-tuning each model based on the results of the other model, and then picking whichever model achieves the most accurate result on unseen mixes.

[0158] At step 1100, tags and categories are manually assigned to a selected amount of the most popular mixes. For example, as Figure 6B shows at step 1202, the metadata tags the administrator assigns to the mixes may include the assignment of a category or the assignment of other metadata tags such as media metadata tags, title metadata tags, etc. At step 1102, a predictive model and a cluster model are built using the manually categorized mixes and the results are compared. At step 1104 the manually assigned categories are compared to the clusters identified by the cluster model. At step 1106, the cluster model is fine-tuned. The fine tuning can occur using different numbers of clusters and different algorithms in order to get a cluster classification that most closely matches the manual classification.

[0159] At step 1108, the remaining mixes are run against both the cluster model and the predictive model. At step 1110, all or some of the cases where the models disagree with the classification are manually examined. At step 1112, both the predictive model and the cluster model are fine tuned based on the differences. At step 1114, the targeting module 297 determines whether a satisfactory agreement level is reached between the models and the manual classifications. If not, the process repeats at step 1108 wherein the mixes continue to be run against the models and the models are fine tuned until step 1114 becomes satisfactory.

[0160] Once step 1114 is true, the targeting module 297 determines which model achieved a more accurate result at step 1116. If the predictive model achieved a more accurate result, then the predictive model is used at step 1118. If the cluster model achieved a more accurate result, then the cluster model is used at step 1118.

[0161] Categorization methods also can be applied to groups. The set of group categories is the same as the set of video categories, and is based on the predominant video category in the set of mixes posted to the group. Group categories are used for various purposes, including personalization in order to recommend to viewers that they visit a particular group which is of the same category as the mix just viewed.

[0162] The same set of categories can also be applied to viewers and authors of mixes, based on the predominant categories of mixes that they view or create, in order to achieve a simple form of user categorization. Multiple video categories can be applied to a viewer or author. More sophisticated viewer and author categorization schemes take into account additional user static and dynamic behavioral metadata, including search terms used, prior clickthrough behavior, and the frequency and duration of visits to the website.

[0163] The online video editing environment can also be configured to support syndication of media. Any of the publicly available media available on the website can be syndicated to partners for redistribution or other purposes. To support syndicated media, the system 200 can have a set of open application programming interfaces ("APIs") that allow partners that can benefit from these services to access metadata associated with users and media. This is potentially a paid service that includes access to "actionable" metadata (filtering metadata, personalization metadata, segmentation / target marketing metadata, and monetization metadata) that follows the media as it is passed from the system's platform to partner platforms (e.g., if publishing a mix to the YouTube site).

[0164] The targeting module 297 also provides semi-automatic and fully automatic methods of matching advertisements to video categories, viewer categories, and author categories. To achieve this, the targeting module 297 can also include a categorization scheme for advertisements, consisting of approximately 40 categories, based on a consolidation of the advertising categories of several leading media companies. The targeting module 297 also can maintain a cross-reference list (built and updated on a regular basis by the system administrators) that defines the most appropriate advertisement categories for each video, viewer and author category. [0165] The sets of video, viewer and author categories each can consist of a number of categories. In one example, there are between 50 and 100 categories. Thus matching an advertisement to a video based on video category involves referring to a video category-advertisement category cross-reference list, which in the current example comprises 50-100 entries showing one or more advertisement categories representing the most appropriate advertisement for each video category. This cross-reference list is either used manually, in conjunction with advertiser or their agent, as an aid in selecting one or more video categories, or automatically, to make an automatic selection of video categories. In this example, the selected video categories represent the categories of mixes to which the advertisement can be attached, and guide the decision-making process when a mix is selected for viewing.

[0166] Through an optional refinement of the matching process, advertisers can also be given the option of specifying one or more additional keywords describing their target mix, where the keywords would be used to match on tags associated with mixes. Thus the match could be based on a combination of the advertisement category/video category match and a match of one or more advertisement keywords to one or more video tags associated with a mix.

[0167] In a further refinement, in cases where demographic metadata is available, advertisers can also be given the option of designating particular demographic groups to which they would like to target each ad. Thus the match could be based on a combination of video or user category, tags, and demographic group preferences.

[0168] The ability to detail the components of a final mix can be used to create methods to allocate any resulting revenue attributed to a particular creation. In one aspect this functionality is implemented by the targeting module 297, although a separate module can be implemented as well to carry out this functionality. For example, a final mix can be constructed of three media components, Ma + Mb + Mc, from three different authors, Aa + Ab + Ac. If the final mix is able to generate revenue through any number of means including but limited to pay to download, subscribe to view, and advertising, the resulting revenue can be divided up among the authors based on their contributions.

[0169] Dividing up the revenue can be accomplished using several different methods including but not limited to the following: (1) the duration of the components (Ma, Mb, Mc); (2) the popularity of the components including the number of times viewed, the number of times harvested in other mixes ("mashups"), the number of times shared, the number of times downloaded, and the number of times published to other locations; (3) the method of viewing, sharing and downloading including to a personal computer, to a mobile device, and to a digital home connection; and (4) the fixed cost for the media (e.g., use of licensed content can command a fixed percentage of revenue).

[0170] Business logic combined with the ability to collect video construction

("video mashup") metadata can result in a variety of different monetization methods, any of which can be inserted in a modular fashion into the monetization algorithm.

[0171] The following is an example:

(1) User A collects media from users B, C, and D.

(2) User A combines collected media assets into a mashup and then publishes the mashup by an API integration. The platform publishes the mashup to a corporate account, as opposed to user A's account.

(3) The mashup earns $100 in net revenue over the course of time (net is gross advertising revenue less ad-sales costs, less syndication costs), and the platform has knowledge of this revenue vis-a-vis the API. For example, the platform can actually have received $200 of gross revenue, but in turn subtracted $40 for publishers who helped syndicate, and $60 for the platform as part of their fees.

(4) The platform then automatically applies a calculus to determine how the net $100 should be split between users A, B, C, and D as well as the platform itself. After deriving the revenue split, the targeting module 297 transmits the revenue splits to the platform via the API. The targeting module 297 in turn credits appropriate percentages to users A, B, C, and D.

(5) Conversely, user A, B, C, and D can all have individual accounts established through the API. Once the targeting module 297 automatically applies a calculus to determine how the net $100 should be split between users A, B, C, and D, the revenue splits are transmitted to the platform via the API for each user.

[0172] In one aspect, the first step is analysis of the mashup to determine which media assets were used as components of the final mashup video. For each digital component of the mashup (video components, photos, and audio), the targeting module 297 determines whether the asset is subject to specific per usage (e.g., $0.03 per video that incorporates a music artist song) or per stream licensing terms (e.g., $0.003/content/stream) which come from specific licensing deals with copyright holders. If so, the targeting module subtracts the appropriate amount from the revenue pool off the net revenue and compensates the asset owner.

[0173] After all licensor requirements are met, the targeting module 297 can subtract a percentage, (E%), from the revenue pool remaining to be assigned to the platform as compensation for providing the technology. Next the targeting module 297 can subtract a percentage, (M%), from the revenue pool and assign it to user A as compensation for creating the mashup. If the mashup is a first-generation creation (i.e., they did not "remix" another user's work), user A is allowed to keep 100% of this allocation.

[0174] If the mashup is a remix, 50% of the allocation is distributed to the creator of the prior generation mashup. This generational, iterative revenue split is applied continuously until the individual allocations reach zero, similar to multi-level marketing schemes. For example, assuming there were four generations of mixers involved in a remix the allocation of M% would be made as follows using $10 as the result of M%:

Fourth Remix: 50% x M% = 50% x $ 10 = $5

Third Remix: 50% x (50% x M%) = 50% x $5 = $2.50

Second Remix: 50% x (50% x (50% x M%)) = 50% x $2.50 = $ 1.25

First Remix: 50% x (50% x (50% x M%)) = $1.25

[0175] Lastly, the targeting module 297 can distribute the remaining revenue in the pool, (R%), to any digital assets (e.g., user-generated content that was used in the mix) that are not subject to specific per usage or per stream licensing terms. Revenue allocation can be determined using a weighted percentage algorithm. [0176] The weighted percentage algorithm is described further below. The duration of all component assets from users whose clips were used in a mix is calculated to determine the total duration of media used in the mashup. Note that this sum can be different from the duration of the mashup itself. For example, assets A, B, and C can be clips of duration 20 seconds each, and asset D can be an audio soundtrack of 60 seconds, in which case the total sum of all media is 120 seconds as compared to the 60 second duration of the mashup itself.

[0177] The relative weight of each media asset is calculated on a percentage basis using the sum duration of all media as the denominator (less any copyright content already compensated) and the durations of each media asset as numerators. For example, asset A accounts for 16.67% (20/120) of the mashup and soundtrack asset D accounts for 50% (60/120). After relative weights are calculated, the remaining revenue pool, R%, is allocated to each asset in accordance with its weight.

[0178] The fact that remixing and publishing are easy using the present system 200 creates an opportunity for gamers to remix popular revenue-generating mashups, make insignificant changes, and unjustly gain a revenue share. To prevent this from happening the targeting module 297 can require remixes to be quantifiably different from their originals before they can be published. In one aspect, a remix can only be published, and thus eligible for monetization, if one or more of the following are true: the entire soundtrack has been changed; 50% or more of the effects have been changed; the number of non-audio media assets (i.e., video, photo, title frames) changed is greater than or equal to 25%; or the total duration of the mashup changes is greater than or equal to 25%.

[0179] The following illustrates an example of the compensation algorithm described above. A mix in the online video platform is a remix that is in its fourth generation (which all qualify as remixes defined above) and, therefore, has four mixers involved: Ml, M2, M3, M4 where M4 is the last/final user to mix the media. There is one clip from a record label, PVl, and one sound track from the same record label, PAl. There are three other clips and two photos each from separate users: UVl, UV2, UV3, UPl, UP2.

[0180] The durations are as follows:

UVl: 20sec UV2: lOsec

UV3: 5 sec UPl : 5sec UP2: 5 sec PVl : 15sec PAl : 60sec

[0181] The total mix duration is 60 seconds. The total media duration is 120 seconds. The user generated media duration is 120 seconds - 15 seconds - 60 seconds, which is 45 seconds. The copyright content pricing assumed as follows: PAl : $0.003/stream; and PVl : $0.003/stream.

[0182] During the course of one month the remix exhibited the following statistics: gross revenue: $2,000; net revenue: $1,000; streams: 67 Kilobytes.

[0183] Revenue compensation then can be calculated as follows:

Premium / Copyright Allocation

PVl : $201 (67,000 x $0,003) PAl : $201 (67,000 x $0,003) SubTotal: $402 Net Revenue Pool: $598 ($1,000 -$402)

System Allocation E%=25%; $149.50 (25% of $598) Net Revenue Pool: $448.50 ($598 - $149.50) Mashup Allocation M%=25%; $112.12

M4: 50% x 25% x $448.50 = $56.06 M3: 50% x $56.06 = $28.02 M2: 50% x $28.02 = $14.01 Ml: 50% x $28.02 = $14.01

User Allocation 1-M% = 75%; $336.37 UVl: 20/45 x $336.37 = $149.50 UV2: 10/45 x $336.37 = $74.75 UPl : 5/45 x $336.37 = $37.37 UP2: 5/45 x $336.37 = $37.37

[0184] In summary, the revenue in this example would be as follows: Gross Ad-Revenue: $2,000 ($30 CPM with 1 : 1 ad-2-content ratio)

Total: $1,000 (covers all system & syndication fees)

Net Revenue: $1,000

PVl : $201

PAl : $201

Platform: $149.50

M4: $56.06

M3: $28.02

M2: $14.01

Ml: $14.01

UVl: $149.50

UV2: $74.75

UV3: $37.37

UPl : $37.37

UP2: $37.37

Total: $999.96

[0185] Monetization algorithms can also take into account distribution of revenues where the media is syndicated to other websites. For example, if a mix is requested for viewing by another website, in real time the targeting module 297 can attach a targeted advertisement based on the video category, and serve the mix with the advertisement; revenue derived from the advertising can be shared between the two sites and the author of the mix. More sophisticated monetization algorithms can also take into account more complex networks of systems, such as a network consisting of a video creation/mixing site, a separate video sharing site to which the mix is published, an advertising site that attaches advertisements to mixes and distributes them to third-party sites, and the third- party sites that publish the resulting mix with attached advertising.

[0186] The above description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles described herein can be applied to other embodiments without departing from the spirit or scope of the invention. Thus, it is to be understood that the description and drawings presented herein represent a presently preferred embodiment of the invention and are therefore representative of the subject matter that is broadly contemplated by the present invention. It is further understood that the scope of the present invention fully encompasses other embodiments that may become obvious to those skilled in the art and that the scope of the present invention is accordingly limited by nothing other than the appended claims.

Claims

1. A method for creating a digital media mix designed to be used in an online environment comprising: uploading a first clip of digital media to an online platform by segmenting the first clip into a plurality of segments, compressing the plurality of segments into a plurality of upload segments, and uploading the plurality of upload segments to the online platform; associating a first metadata tag with the first clip on the online platform; obtaining a second clip of digital media on the online platform; associating a second metadata tag with the second clip on the online platform; combining the first and the second clips to form a mix on the online platform; and associating the first and the second metadata tags with the mix.

2. The method of claim 1 wherein the step of associating first metadata tags with the mix further comprises assigning a descriptive word to the mix.

3. The method of claim 1 wherein the step associating first metadata tags with the mix further comprises assigning a category to the mix.

4. The method of claim 1 wherein the step of associating first metadata tags with the first clip further comprises selecting a metadata tag that was associated with the first clip prior to uploading the first clip to the online platform.

5. The method of claim 3 further comprising performing an action with regard to the mix based on the first and the second metadata tags or the category.

6. The method of claim 5 wherein the step of performing an action further comprises applying a business rule to mix and scoring the mix with a score based on the business rule and/or the first and the second metadata tags associated with the mix where a magnitude of the score indicates a probability that the mix is likely to contain objectionable content.

7. The method of claim 5 wherein the step of performing an action further comprises applying a business rule to the mix and scoring the mix with a score based on the business rule and/or the first and the second metadata tags associated with the mix where a magnitude of the score indicates a probability that the mix is not likely to contain objectionable content.

8. The method of claim 6 further comprising flagging the mix for a later review based on the score.

9. The method of claim 6 further comprising flagging the mix for immediate review based on the score.

10. The method of claim 6 further comprising removing the mix immediately based on the score.

11. A computer implemented system for creating a mix of digital media comprising: a preprocessing application, which segments a first clip of digital media into a plurality of segments, compresses the plurality of segments into a plurality of upload segments, and uploads the plurality of upload segments; a metadata module, which associates a first metadata tag with the first clip on the online video platform and receives a second clip having an associated second metadata tag; an online video platform, which combines the first and the second clips to form a mix; and the metadata module being further configured to associate the first and the second metadata tags with the mix.

12. The system of claim 11 wherein the metadata module is further configured to receive a keyword associated with the first clip and associate the keyword with the mix.

13. The system of claim 11 wherein the metadata module is further configured to assign a category to the mix.

14. The system of claim 11 wherein the metadata module is further configured to inherit a keyword associated with the first clip or the second clip and associate the keyword with the mix.

15. The system of claim 12 further comprising a filtering module configured to perform an action in regard to the mix based on the first and the second metadata tags or the keyword.

16. The system of claim 15 wherein the filtering module is configured to apply a business rule to the mix and to automatically apply a score to the mix based on the business rule and/or on the first and second metadata tags associated with the mix, where a magnitude of the score indicates a probability that the mix contains objectionable content.

17. The system of claim 15 wherein the filtering module is configured to apply a score that is associated with the mix when the first or the second metadata tags indicate that the mix is not likely to contain objectionable content.

18. The system of claim 16 wherein the filtering module is configured to apply a flag associated with the mix based on the score, the flag configured to cause the filtering module to review the mix later.

19. The system of claim 16 wherein the filtering module is configured to apply a flag associated with the mix based on the score, the flag configured to cause the filtering module to review the mix immediately.

20. The system of claim 16 wherein the filtering module is configured to apply a flag associated with the mix based on the score, the flag configured to cause the filtering module to remove the mix immediately.

21. The system of claim 15 wherein the filtering module is configured to apply a score to the mix based on one or more of user metadata, media metadata, and inherited metadata.

22. The system of claim 15 wherein the filtering module applies a data mining technique to the first or the second metadata tag associated with the mix, the data mining technique including one or more of prediction, classification, clustering, and association.

23. The system of claim 22 wherein the mix or a user is placed into a category based upon the data mining technique.

24. The system of claim 22 wherein the mix is targeted for marketing based upon the data mining technique.

25. The system of claim 11 further comprising an application programming interface

(API) capable of supplying the first and the second metadata tags to a partner.

26. A method for distributing revenue associated with a mix of digital media, the mix including one or more media assets that require an allocation of the revenue comprising: determining whether a first media asset is subject to a per usage licensing term; determining whether a second media asset is subject to a per stream licensing term; subtracting an amount from the revenue if the one or more media assets include the first or the second media assets; allocating an amount of the revenue to a third media asset that is not subject to the per usage or the per stream licensing terms; and allocating a remaining amount of the revenue to a number of prior users who contributed to the mix.

27. The method of claim 26 further comprising determining whether the mix is eligible for the allocation of the revenue.

28. The method of claim 27 wherein the step of determining whether the mix is eligible for the allocation of the revenue further comprises determining if an entire soundtrack has changed.

29. The method of claim 27 wherein the step of determining whether the mix is eligible for the allocation of the revenue further comprises determining if more than half of a plurality of effects have been changed.

30. The method of claim 27 wherein the step of determining whether the mix is eligible for the allocation of the revenue further comprises determining if more than one fourth of a plurality of non-audio media assets have been changed.

31. The method of claim 27 wherein the step of determining whether the mix is eligible for the allocation of the revenue further comprises determining if a total duration of the mix has been changed by more than one fourth.