WO2009061047A1

WO2009061047A1 - System and method for providing secure multimedia registration and management framework for multimedia service on web portal

Info

Publication number: WO2009061047A1
Application number: PCT/KR2008/002760
Authority: WO
Inventors: Byeong Cheol Choi; Seung-Wan Han; Chi Yoon Jeong; Jeo Deok Lim; So Hee Park; Jeong Nyeo Kim
Original assignee: Electronics And Telecommunication Research Institute
Priority date: 2007-11-09
Filing date: 2008-05-16
Publication date: 2009-05-14
Also published as: KR20090048209A

Abstract

Provided are a system and method for providing secure multimedia registration and management framework for multimedia service on web portal. The method for providing secure multimedia registration and management framework includes: learning multimedia data loaded by a client, classifying a learning model value as an adult material class or a non-adult material class, and registering the classified learning model value in a content-based database; determining whether the loaded multimedia data is adult material or not by using the learning model values registered in the content-based database, and permitting the upload of the loaded multimedia data; and uploading and updating the loaded multimedia data according to the upload permission.

Description

SYSTEM AND METHOD FOR PROVIDING SECURE MULTIMEDIA REGISTRATION AND MANAGEMENT FRAMEWORK FOR MULTIMEDIA SERVICE ON WEB

PORTAL Technical Field

[1] The present disclosure relates to a system and method for providing secure multimedia registration and management framework for multimedia service on a web portal.

[2] This work was supported by the IT R&D program of MIC/IITA. [2007-S-016-01 , A

Development of Cost Effective and Large Scale Global Internet Service Solution]

[3]

Background Art

[4] In Web 2.0 based Internet era, a variety of user-centered application services such as a user created contents (UCC) moving picture service have been emerged. Such a Web 2.0 based user-centered multimedia service has great difficulty in managing data and information, as opposing to an existing provider-centered multimedia service. In particular, since the user can freely create and distribute multimedia data, the web portal should prepare for illegal user's distribution of malignant multimedia data (adult moving pictures or the like). At present, an adult verification is used when a user downloads an adult material. However, when uploading an adult material, a manual monitoring method or a hash-based comparison method is used, but such a method is very unsatisfactory.

[5] Therefore, there is an increasing need to securely registrate and manage web services by analyzing, determining and classifying harmful multimedia data (especially, adult moving pictures) automatically and actively.

[6]

Disclosure of Invention

Technical Problem

[7] Therefore, an object of the present invention is to provide a system and method for providing secure multimedia registration and management framework for multimedia service on web portal.

[8] Another object of the present invention is to provide a system and method for providing secure multimedia registration and management framework which are capable of learning multimedia data loaded by a user to register learning model values on a content basis, and determining whether the loaded multimedia data is an adult material with the aid of the registered content-based learning model values, thereby performing multimedia registration and management securely and easily.

[9]

Technical Solution

[10] To achieve these and other advantages and in accordance with the purposes of the present invention as embodied and broadly described herein, a system for providing multimedia registration and management framework in accordance with an aspect of the present invention includes: a content-based database (DB) for storing learning model values classified as an adult material class or a non-adult material class; an X multimedia analysis system (XMAS) core engine for learning a multimedia data loaded by a client, registering a learning model value in the content-based database, determining whether the loaded multimedia data is an adult material or not by using the learning model values registered in the content-based database, and permitting an upload of the loaded multimedia data; and a web portal server for uploading the loaded multimedia data according to the upload permission of the XMAS core engine.

[11] The system may further include a hash-based database for storing hash values of the multimedia data, and the XMAS engine core may communicate with the hash-based database.

[12] To achieve these and other advantages and in accordance with the purposes of the present invention, a method for providing multimedia registration and management framework in accordance with another aspect of the present invention includes: learning a multimedia data loaded by a client, classifying a learning model value as an adult material class or a non-adult material class, and registering the classified learning model value in a content-based database; determining whether the loaded multimedia data is an adult material or not by using the learning model values registered in the content-based database, and permitting the upload of the loaded multimedia data; and uploading and updating the loaded multimedia data according to the upload permission.

[13] To achieve these and other advantages and in accordance with the purposes of the present invention, an XMAS core engine of a system for providing multimedia registration and management framework in accordance with another aspect of the present invention includes: a codec analyzing part for reproducing a loaded multimedia data compressed by a client; a key frame extracting part for detecting a predetermined number of shots from the multimedia data reproduced by the codec analyzing part, and extracting key frames from the respective detected shots; a feature extracting part for extracting feature values with respect to the key frames extracted by the key frame ex- trading part; a machine learning part for learning the feature values extracted by the feature extracting part and registering a learning model value in a content-based database; a comparison value generating part for generating a comparison value by using the extracted feature values; and a multimedia determining part for comparing the comparison value with the learning model values registered in the content-based database, and determining whether the loaded multimedia data is the adult material or the non-adult material.

[14] To achieve these and other advantages and in accordance with the purposes of the present invention, a method for operating an XMAS core engine of a system for providing multimedia registration and management framework in accordance with another aspect of the present invention includes: reproducing a multimedia data loaded by a client; detecting a predetermined number of shots from the reproduced multimedia data, and extracting key frames from the respective detected shots; extracting feature values with respect to the extracted key frames; learning the extracted feature values and registering a learning model value in a content-based database; generating a comparison value by using the extracted feature values; and comparing the comparison value with the learning model values registered in the content-based database, and determining whether the loaded multimedia data is the adult material or the non- adult material.

[15] The foregoing and other objects, features, aspects and advantages of the present invention will become more apparent from the following detailed description of the present invention when taken in conjunction with the accompanying drawings.

[16]

Advantageous Effects

[17] The framework providing system and method according to the embodiments of the present invention, which learn multimedia data loaded by a user to register learning model values on a content basis and determine whether the loaded multimedia data is an adult material with the aid of the registered content-based learning model values. Therefore, the multimedia can be registered and managed securely and easily, and consequently the reliability of the multimedia service is capable of being improved.

[18]

Brief Description of the Drawings

[19] The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention.

[20] FIG. 1 illustrates the architecture of a system for providing secure adult multimedia registration and management framework for multimedia service on a web portal according to an exemplary embodiment;

[21] FIG. 2 illustrates an XML schema description of the multimedia data, an upload of which is requested by a user;

[22] FIG. 3 is a flowchart illustrating a method for providing secure adult multimedia registration and management framework for multimedia service on a web portal according to an exemplary embodiment;

[23] FIG. 4 is a block diagram of an XMAS core engine illustrated in FIG. 1; and

[24] FIGS. 5 and 6 are flowcharts of the operation of the XMAS core engine illustrated in

FIG. 4.

[25]

Best Mode for Carrying Out the Invention

[26] Hereinafter, specific embodiments will be described in detail with reference to the accompanying drawings.

[27] FIG. 1 illustrates the architecture of a system for providing secure adult multimedia registration and management framework for multimedia service on a web portal according to an exemplary embodiment.

[28] Referring to FIG. 1, the system for providing the secure adult multimedia registration and management framework for multimedia service on the web portal according to the exemplary embodiment includes a client 100, a web portal server 200, an XMAS core engine 300, a content-based DB 400, a hash-based DB 500, and a communication network 600.

[29] The client 100 is a user terminal used for uploading multimedia data to the web portal server 200. Examples of the client 100 include a computer, a mobile phone, and a personal digital assistant (PDA). The client 100 accesses the web portal server 200 through the communication network 600 according to user's instructions to thereby upload the multimedia data to the web portal server 200.

[30] The web portal server 200 compares a hash value of the multimedia data provided from the client 100 with hash values of multimedia data stored in the hash-based DB 500, and primarily determines whether the multimedia data provided from the client 100 is an adult material. When the hash-based determination result shows that the multimedia data is the adult material, the web portal server 200 performs an adult verification on the user and thereafter uploads the multimedia data provided from the client 100.

[31] When the hash-based determination result shows that the multimedia data is a non- adult material, the web portal server 200 requests the XMAS core engine 300 to determine whether the multimedia data provided from the client 100 is the adult material. At this point, when the content-based determination result of the XMAS core engine 300 shows that the multimedia data is the non-adult material, the web portal server 200 uploads the loaded multimedia data without adult verification. On the other hand, when the content-based determination result of the XMAS core engine 300 shows that the multimedia data is the adult material, the web portal server 200 performs the adult verification on the user and thereafter uploads the multimedia data loaded from the client 100.

[32] The XMAS core engine 300 reproduces the multimedia data loaded by the client 100, and detects the certain number of shots from the reproduced multimedia data. The XMAS core engine 300 extracts key frames from the respective detected shots and then extracts feature values of the extracted key frames. The XMAS core engine 300 learns the extracted feature values and registers learning model values in the content- based DB 400. In addition, the XMAS core engine 300 generates comparison values by using the extracted feature values and thereafter determines whether the loaded multimedia data is the adult material or the non-adult material by comparing the comparison values with the learning model values registered in the content-based DB 400. The shot is one scene of the successive moving picture images.

[33] The learning model value learned by the XMAS core engine 300 are registered in the content-based DB 400. Herein, the learning model value is classified as an adult material class or a non-adult material class to thereby be registered.

[34] The hash values of the multimedia data are registered in the hash-based DB 500.

[35] The upload type of the multimedia data, an upload of which is requested by the user, is expressed as Equation (1) below. An XML schema description of the multimedia data, an upload of which is requested by the user, is expressed as illustrated in FIG. 2.

[36] (Uploaded) Multimedia := Multimedia Il H(M) Il Tag(19+ or 19-) (1)

[37] where H(M) is a hash value (SHA- 1 or MD5) of the multimedia data, and Tag(+19) is tagging data indicating the adult material.

[38] In the XML schema description of FIG. 2, the hash value is used for searching and updating the DB, and the tag data is used for determining no adult verification or adult verification request and processing.

[39] FIG. 3 is a flowchart illustrating a method for providing secure adult multimedia registration and management framework for multimedia service on a web portal according to an exemplary embodiment.

[40] Referring to FIG. 3, the client 100, which is connected through the communication network 600, loads multimedia data such as UCC according to user's instructions and requests the web portal server 200 to upload the loaded multimedia datain in operation SlOl. In operation S 102, the web portal server 200 compares a hash value of the multimedia data loaded from the client 100 with hash values of multimedia data registered in the hash-based DB 500, and primarily determines whether the loaded multimedia data is an adult material.

[41] In operations S 103 and S 104, when the hash-based determination result shows that the multimedia data is the adult material, the web portal server 200 performs an adult verification on the user and then uploads the loaded multimedia data. In operation S 103, the adult verification process may be performed by allowing the user to input his/her own real name and resident registration number, but the present invention is not limited to this example. As another example, the adult verification can be performed by a public certificate.

[42] In operation S 105, when the hash-based determination result shows that the multimedia data is the non-adult material, the web portal server 200 requests the XMAS core engine 300 to again determine whether the multimedia data loaded from the client 100 is the adult material. In operations S 106 and S 107, the XMAS core engine 300 analyzes the features of the loaded multimedia data according to the request, and thereafter searches the learning model values registered in the content- based DB 400. In operations S 108 and S 109, the XMAS core engine 300 compares the analyzed features of the multimedia data with the searched content-based learning model values, and secondarily determines whether the multimedia data provided from the client 100 belongs to the learning model value of the adult material class or the learning model value of the non- adult material class, which is registered in the content- based DB 400.

[43] In operation Sl 10, when the content-based determination result shows that the multimedia data belongs to the non-adult material class, the XMAS core engine 300 permits the web portal server 200 to upload the loaded multimedia data. In operation Si l l, the web portal server 200 uploads the loaded multimedia data without adult verification according to the upload permission. In operation Si l l, the upload type of the multimedia data is "Multimedia Il H(M) Il Tag(19-)" and the XML schema description of the upload type of the multimedia data is expressed as illustrated in FIG. 2.

[44] In operation Sl 12, the XMAS core engine 300 automatically updates the hash value of the multimedia data, which is loaded from the client 100, to the hash-content DB 500. In addition, in operation Sl 13, the XMAS core engine 300 notifies the upload state of the multimedia data to the administrator, and manually updates the hash value of the multimedia data, which is provided from the client 100, to the hash-based DB 500 according to the administrator's instructions.

[45] When the content-based determination result shows that the loaded multimedia data is the non-adult material, the hash value of the multimedia data is automatically updated. Alternatively, the hash value of the multimedia data may be manually updated according to the administrator's selection. [46] In operation Sl 14, when the content-based determination result shows that the multimedia data is the adult material, the XMAS core engine 300 performs the adult verification and permits the web portal server 200 to upload the loaded multimedia data. In operations Sl 15 and Sl 16, the web portal server 200 performs the adult verification on the user and uploads the multimedia data provided from the client 100. In operation Sl 15, the adult verification process may be performed by allowing the user to input his/her real name and resident registration number, but the present invention is not limited to this example. As another example, the adult verification can be performed by a public certificate. In the upload operation Sl 16, the upload type of the multimedia data is "Multimedia Il H(M) Il Tag(19+)" and the XML schema description of the upload type of the multimedia data is expressed as illustrated in FIG. 2.

[47] In operation Sl 17, the XMAS core engine 300 automatically updates the hash value of the multimedia data, which is provided from the client 100, to the hash-based DB 500. In addition, in operation Sl 18, the XMAS core engine 300 notifies the upload state of the multimedia data to the administrator, and manually updates the hash value of the multimedia data, which is provided from the client 100, to the hash-based DB 500 according to the administrator's instructions.

[48] When the content-based determination result shows that the multimedia data is the adult material, the hash value of the multimedia data is automatically updated. Alternatively, the hash value of the multimedia data may be manually updated according to the administrator's selection.

[49] FIG. 4 is a block diagram of the XMAS core engine illustrated in FIG. 1.

[50] Referring to FIG. 4, the XMAS core engine 300 includes a codec analyzing part 310, a key frame extracting part 320, a feature extracting unit 330, a machine learning part 340, a comparison value generating part 350, and a multimedia determining part 360.

[51] The codec analyzing part 310 reproduces the multimedia data compressed and loaded by the client 100, and outputs them to the key frame extracting part 320. The codec analyzing part 310 may be implemented based on DirectShow using a unified codec, or may be implemented using an FFMPEG codec library.

[52] The key frame extracting part 320 detects the certain number of shots from the multimedia data reproduced by the codec analyzing part, and extracts key frames from the respective detected shots. The shot is one scene of the successive moving picture images. The key frame extracting part 320 may be implemented in a uniform distribution scheme of uniformly distributing the number of the key frames extracted from the respective detected shots, or a random distribution scheme of randomly distributing the number of the key frames extracted from the respectively detected shots. In addition, the key frame extracting unit 320 may be implemented to extract the key frames from a specific shot, for example, a first shot, a last shot, or a middle shot, of the detected shots.

[53] The feature extracting part 330 extracts the feature values with respect to the key frames extracted by the key frame extracting part 320, and outputs the extracted feature values to the machine learning part 340 and the comparison value generating unit 350. At this point, the feature extracting part 330 may be implemented to extract color histogram feature values or frequency feature values with respect to colors. The feature extracting part 330 may be implemented using an MPEG-7 Visual Descriptor (a Visual descriptor part of MPEG-7 XM library), or may be implemented using a Skin Color Model.

[54] The machine learning part 340 learns the feature values extracted by the feature extracting part 330 and registers the learning model value in the content-based DB 400. The learning model value is classified as the adult material class or the non-adult material class and registered in the content-based DB 400. The machine learning part 340 may be implemented using LibSVM that belongs to a Supervised Vector Machine (SVM) library.

[55] In this way, since the machine learning part 340 learns the loaded multimedia data, classifies it as the adult material or the non-adult material, and registers it in the content-based DB 400, and consequently the multimedia determining part 360 can easily determine whether the loaded multimedia data is the adult material or the non- adult material.

[56] The comparison value generating part 350 generates the comparison value by using the feature values extracted by the feature extracting part 330, and outputs the generated comparison value to the multimedia determining part 360. Since the multimedia determining part 360 cannot directly use the extracted feature values as the comparison target values, the comparison value generating part 350 is to generate the comparison value capable of being used as the determination target of the multimedia data.

[57] The multimedia determining part 360 compares the comparison value generated by the comparison value generating part 350 with the learning model values registered in the content-based DB 400, and determines whether the loaded multimedia data is the adult material or the non-adult material. More specifically, when the comparison value is identical or similar to the learning model value of the adult material class registered in the content-based DB 400, the multimedia determining part 360 determines the loaded multimedia data as the adult material. On the other hand, when the comparison value is identical or similar to the learning model value of the non-adult material class registered in the content-based DB 400, the multimedia determining part 360 determines the loaded multimedia data as the non- adult material.

[58] As described above, the loaded multimedia data is learned, and the learning model value is classified as the adult material class or the non-adult material class and registered in the content-based DB. Whether the loaded multimedia data is the adult material class or the non-adult material class is determined through the simple comparison. Therefore, the multimedia data can be registered and managed securely and easily, and consequently the reliability of the multimedia service can be improved.

[59] FIG. 5 is a flowchart of the operation of the XMAS core engine illustrated in FIG. 4.

[60] Referring to FIG. 5, when the multimedia data is loaded on the web portal server

200, the XMAS core engine 300 reproduces the multimedia data loaded by the client 100 in operation S201. In operations S202 and S203, the XMAS core engine 300 detects the certain number of shots from the reproduced multimedia data, and extracts key frames from the respective detected shots.

[61] In operation S204, after extracting the key frames, the XMAS core engine 300 extracts feature values with respect to the extracted key frames.

[62] In operation S205, the XMAS core engine 300 learns the extracted feature values and registers a learning model value in the content-based DB 400.

[63] FIG. 6 is a flowchart of the operation of the XMAS core engine illustrated in FIG. 4.

[64] Referring to FIG. 6, when the multimedia data is loaded on the web portal server

200, the XMAS core engine 300 reproduces the multimedia data loaded by the client 100 in operation S301. In operations S302 and S303, the XMAS core engine 300 detects the certain number of shots from the reproduced multimedia data, and extracts key frames from the respective detected shots.

[65] In operation S304, after extracting the key frames, the XMAS core engine 300 extracts feature values with respect to the extracted key frames.

[66] In operation S305, the XMAS core engine 300 generates a comparison value by using the extracted feature values. In operation S306, the XMAS core engine 300 compares the comparison value with the learning model values registered in the content-based DB 400, and determines whether the loaded multimedia data is the adult material or the non-adult materials. In operation S306, when the comparison value is identical or similar to the learning model value of the adult material class registered in the content-based DB 400, the XMAS core engine 300 determines the loaded multimedia data as the adult material. On the other hand, when the comparison value is identical or similar to the learning model value of the non- adult material class registered in the content-based DB 400, the XMAS core engine 300 determines the loaded multimedia data as the non-adult material.

[67] As the present invention may be embodied in several forms without departing from the spirit or essential characteristics thereof, it should also be understood that the above-described embodiments are not limited by any of the details of the foregoing description, unless otherwise specified, but rather should be construed broadly within its spirit and scope as defined in the appended claims, and therefore all changes and modifications that fall within the metes and bounds of the claims, or equivalents of such metes and bounds are therefore intended to be embraced by the appended claims. [68]

Industrial Applicability

[69] In the framework providing system and method according to the embodiments of the present invention, the multimedia data loaded by the user is learned and the learning model value is registered on the content basis. Whether the loaded multimedia data is the adult material class or the non-adult material class is determined using the registered content-based learning model values. Therefore, the multimedia can be registered and managed securely and easily. Accordingly, the system and method according to the embodiments of the present invention can be applied to various fields such as web portal supporting the multimedia service.

[70]

Claims

[1] A system for providing multimedia registration and management framework, the system comprising: a content-based database for storing learning model values classified as an adult material class or a non-adult material class; an X multimedia analysis system (XMAS) core engine for learning a multimedia data loaded by a client, registering a learning model value in the content-based database, determining whether the loaded multimedia data is an adult material or not by using the learning model values registered in the content-based database, and permitting the upload of the loaded multimedia data; and a web portal server for uploading the loaded multimedia data according to the upload permission of the XMAS core engine.

[2] The system of claim 1, further comprising a hash-based database for storing hash values of the multimedia data, wherein the XMAS engine core communicates with the hash-based database.

[3] The system of claim 2, wherein the web portal server compares a hash value of the loaded multimedia data with the hash values stored in the hash-based database, and determines whether the loaded multimedia data is the adult material.

[4] The system of claim 3, wherein when the loaded multimedia data is determined as the adult material, the web portal server performs an adult verification on a user and uploads the loaded multimedia data.

[5] The system of claim 3, wherein when the loaded multimedia data is determined as the non-adult material, the web portal server requests a content-based adult material determination to the XMAS core engine.

[6] The system of claim 5, wherein the XMAS core engine determines whether the loaded multimedia data is the adult material or not by using the learning model values registered in the content-based database according to the content-based determination request.

[7] The system of claim 6, wherein when the content-based determination result shows that the loaded multimedia data is the non-adult material, the XMAS core engine permits the web portal server to upload the loaded multimedia data without adult verification.

[8] The system of claim 7, wherein the web portal server uploads the loaded multimedia data according to the upload permission of the XMAS core engine, and updates the hash value of the loaded multimedia in the hash-based database.

[9] The system of claim 6, wherein when the content-based determination result shows that the loaded multimedia data is the adult material, the XMAS core engine permits the web portal server to upload the multimedia data after the adult verification.

[10] The system of claim 9, wherein the web portal server uploads the loaded multimedia data according to the upload permission of the XMAS core engine after the adult verification, and updates the hash value of the loaded multimedia data in the hash-based database.

[11] A method for providing multimedia registration and management framework, the method comprising: learning multimedia data loaded by a client, classifying a learning model value as an adult material class or a non-adult material class, and registering the classified learning model value in a content-based database; determining whether the loaded multimedia data is an adult material or not by using the learning model values registered in the content-based database, and permitting the upload of the loaded multimedia data; and uploading and updating the loaded multimedia data according to the upload permission.

[12] The method of claim 11 , further comprising: comparing a hash value of the loaded multimedia data with hash values stored in a hash-based database, and determining whether the loaded multimedia data is the adult material; performing an adult verification on a user and uploading the loaded multimedia data when the loaded multimedia data is determined as the adult material; and requesting a content-based adult material determination when the loaded multimedia data is determined as the non-adult material.

[13] The method of claim 12, wherein the upload permission operation comprises: determining whether the loaded multimedia data is the adult material or not by using the learning model values registered in the content-based database according to the content-based determination request; permitting the upload of the loaded multimedia data without adult verification when the content-based determination result shows that the loaded multimedia data is the non- adult material; and permitting the upload of the multimedia data after the adult verification when the content-based determination result shows that the loaded multimedia is the adult material.

[14] The method of claim 13, wherein the upload operation comprises: uploading the loaded multimedia data; and updating the hash value of the loaded multimedia in the hash-based database. [15] The method of claim 14, wherein the upload of the multimedia data is performed using the hash value of the multimedia data and tag data indicating whether the multimedia data is the adult material or not. [16] An XMAS core engine of a system for providing multimedia registration and management framework, the XMAS core engine comprising: a codec analyzing part for reproducing a loaded multimedia data compressed by a client; a key frame extracting part for detecting a predetermined number of shots from the multimedia data reproduced by the codec analyzing part, and extracting key frames from the respective detected shots; a feature extracting part for extracting feature values with respect to the key frames extracted by the key frame extracting part; a machine learning part for learning the feature values extracted by the feature extracting part and registering a learning model value in a content-based database; a comparison value generating part for generating a comparison value by using the extracted feature values; and a multimedia determining part for comparing the comparison value with the learning model values registered in the content-based database, and determining whether the loaded multimedia data is the adult material or the non-adult material. [17] The XMAS core engine of claim 16, wherein the machine learning part classifies the learning model value as an adult material class or a non- adult material class and registers the classified learning model value in the content-based database. [18] A method for operating an XMAS core engine of a system for providing multimedia registration and management framework, the method comprising: reproducing multimedia data loaded by a client; detecting a predetermined number of shots from the reproduced multimedia data, and extracting key frames from the respective detected shots; extracting feature values with respect to the extracted key frames; learning the extracted feature values and registering a learning model value in a content-based database; generating a comparison value by using the extracted feature values; and comparing the comparison value with the learning model values registered in the content-based database, and determining whether the loaded multimedia data is the adult material or the non-adult material.