WO2013116365A1

WO2013116365A1 - Extended applications of multimedia content previews in the cloud-based content management system

Info

Publication number: WO2013116365A1
Application number: PCT/US2013/023889
Authority: WO
Inventors: Kimber Lockhart; Jeffrey H. SEIBERT
Original assignee: Box, Inc
Priority date: 2012-01-30
Filing date: 2013-01-30
Publication date: 2013-08-08

Abstract

Systems and methods for providing previews of files stored in an online content management system with collaboration environment and permitting a user to annotate the previews are disclosed. The system further tracks the frequency of access of content in previewed documents and provides a visual representation of the frequency with which sections in a document was viewed.

Description

EXTENDED APPLICATIONS OF MULTIMEDIA CONTENT PREVIEWS

IN THE CLOUD-BASED CONTENT MANAGEMENT SYSTEM

CROSS-REFERENCES TO RELATED APPLICATIONS

[0001] This application claims the benefit of U.S. Application No. 13/588,356 entitled "EXTENDED APPLICATIONS OF MULTIMEDIA CONTENT PREVIEWS IN THE CLOUD-BASED CONTENT MANAGEMENT SYSTEM," (Attorney Docket No. 61599-8005. US01 ) which was filed August 17, 2012, which claims the benefit of U.S. Provisional Application No. 61 /592,394 entitled "EXTENDED APPLICATIONS OF MULTIMEDIA CONTENT PREVIEWS IN THE CLOUD-BASED CONTENT MANAGEMENT SYSTEM," (Attorney Docket No. 61599-8005.US00) which was filed January 30, 2012, the contents of which are all incorporated by reference herein.

[0002] This application is related to co-pending U.S. Application No. 13/152,982, entitled, "REAL TIME NOTIFICATION OF ACTIVITIES THAT OCCUR IN A WEB- BASED COLLABORATION ENVIRONMENT", filed June 3, 201 1 ; U.S. Application No. 13/274,268, entitled "AUTOMATIC AND SEMI-AUTOMATIC TAGGING FEATURES OF WORK ITEMS IN A SHARED WORKSPACE FOR METADATA TRACKING IN A CLOUD-BASED CONTENT MANAGEMENT SYSTEM WITH SELECTIVE OR OPTIONAL USER CONTRIBUTION", filed October 14, 201 1 ; and U.S. Application No. 13/208,615, entitled "COLLABORATION SESSIONS IN A WORKSPACE ON A CLOUD-BASED CONTENT MANAGEMENT SYSTEM, filed August 12, 201 1 , each of which is incorporated in its entirety.

BACKGROUND

[0003] Online file storage systems typically provide a way for a user to upload files to a storage server for backing up files, file access, and/or file distribution. Some online file storage systems allow the user to preview the contents of a file before or instead of downloading the file.

BRIEF DESCRIPTION OF THE DRAWINGS

[0004] Examples of a web-based content management system with a collaboration environment that provides previewing of files and permits annotating of the previewed files are illustrated in the figures. The examples and figures are illustrative rather than limiting.

[0005] FIG. 1 illustrates an example diagram of a system where a host server and conversion servers support previewing of files in an online collaboration environment.

[0006] FIG. 2 depicts an example diagram of a web-based or online collaboration platform deployed in an enterprise or other organizational setting for organizing work items and workspaces.

[0007] FIG. 3A depicts a block diagram illustrating an example of components in the host server of a web-based collaboration environment that supports previewing of files.

[0008] FIG. 3B depicts a block diagram illustrating an example of components in the host server of a web-based collaboration environment with real time activity notification capabilities.

[0009] FIG. 3C depicts a block diagram illustrating an example of components in a notification server for providing real time or near real time notifications of activities that occur in a web-based or online collaboration environment.

[0010] FIG. 4A depicts a block diagram illustrating an example of components in a conversion server for converting an input file format of a file in a web-based or online collaboration environment to a target file format suitable for providing a preview of the file.

[0011] FIG. 4B depicts a block diagram illustrating an example of components in an annotation module for supporting annotations of previews of files in a web-based or online collaboration environment.

[0012] FIG. 4C depicts a block diagram illustrating an example of components in a visual representation module for tracking user access of files and providing visual representation of user access of the files.

[0013] FIG. 5 depicts a flow chart illustrating an example process for determining if a user has permission to access a file preview.

[0014] FIG. 6 depicts a flow chart illustrating an example process for finding an optimum path using different conversion tools for converting an input file format to a target file format. [0015] FIG. 7 depicts a flow chart illustrating an example process for converting a format of a video file to a target format for previewing.

[0016] FIG. 8 depicts a flow chart illustrating an example process for generating a progress indicator for indicating the amount of time remaining until a file has been converted.

[0017] FIG. 9 shows a diagrammatic representation of a machine in the example form of a computer system within which a set of instructions, for causing the machine to perform any one or more of the methodologies discussed herein, may be executed.

[0018] FIG. 10 depicts a screenshot showing an example of a user interface of a file page with a feed stream of real time or near real time notifications of activities that occur in a web-based collaboration environment.

[0019] FIG. 1 1 depicts a screenshot showing an example of a user interface of a preview page with a feed stream of real time or near real time notifications of activities that occur in a web-based collaboration environment.

[0020] FIG. 12 depicts a flow chart illustrating an example process for generating a preview of a PDF file.

[0021] FIG. 13 depicts a flow chart illustrating an example process for receiving annotation commands for annotating a preview.

[0022] FIG. 14A depicts a flow chart illustrating an example process for acquiring data for a previewed file for providing a visual representation of user access to the previewed file.

[0023] FIG. 14B depicts a flow chart illustrating an example process for presenting a visual representation of user access for a file.

[0024] FIG. 15 depicts an example visual representation of user access for a file. DETAILED DESCRIPTION

[0025] A system is described for providing content previews of files stored in an online content management system with a collaboration environment. A content preview provided by the system permits a user to view a supported file type without spending the time to download the file or needing the software that created the file. Additionally, the system permits a previewing user to annotate the preview, and the system supports streaming of the annotation data between collaborators' browsers to allow for real-time collaboration.

[0026] Further, the system can track the frequency of access of content in previewed documents and provide a visual representation of the frequency with which sections in a document was viewed.

[0027] Various aspects and examples of the invention will now be described. The following description provides specific details for a thorough understanding and enabling description of these examples. One skilled in the art will understand, however, that the invention may be practiced without many of these details. Additionally, some well- known structures or functions may not be shown or described in detail, so as to avoid unnecessarily obscuring the relevant description.

[0028] The terminology used in the description presented below is intended to be interpreted in its broadest reasonable manner, even though it is being used in conjunction with a detailed description of certain specific examples of the technology. Certain terms may even be emphasized below; however, any terminology intended to be interpreted in any restricted manner will be overtly and specifically defined as such in this Detailed Description section.

[0029] FIG. 1 illustrates an example diagram of a system where a host server 100 and conversion servers 400 support previewing of files for users 108 in an online collaboration environment so that the users 108 do not have to download the file to access its contents. Further, the host server 100 and notification server 150 provide notifications of activities that occur in the online collaboration environment in real time or near real time to users 108. Additionally, the system includes a repository 130, annotation files database 376, and graphic representation database 378. The repository 130 stores files uploaded to the web-based collaboration environment; the annotation files database 376 is used by an annotation module 481 to store annotations made by a user to a file being previewed; and the visual representation database 378 is used by a visual representation module 391 to store tracking data of users previewing a document or video file.

[0030] The client devices 102 can be any system and/or device, and/or any combination of devices/systems that is able to establish a connection, including wired, wireless, cellular connections with another device, a server and/or other systems such as host server 100 and/or conversion servers 400. Client devices 102 will typically include a display and/or other output functionalities to present information and data exchanged between or among the devices 102 and/or the host server 100 and/or conversion servers 400. In one embodiment, this is only a single conversion server 400. In one embodiment, there are multiple conversion servers 400 working independently. In other embodiments, a distributed queuing system is used to run the conversion servers 400 in an efficient manner.

[0031] For example, the client devices 102 can include mobile, hand held or portable devices or non-portable devices and can be any of, but not limited to, a server desktop, a desktop computer, a computer cluster, or portable devices including, a notebook, a laptop computer, a handheld computer, a palmtop computer, a mobile phone, a cell phone, a smart phone, a PDA, a Blackberry device, a Treo, a handheld tablet (e.g. an iPad, a Galaxy, Xoom Tablet, etc.), a tablet PC, a thin-client, a hand held console, a hand held gaming device or console, an iPhone, and/or any other portable, mobile, hand held devices, etc. In one embodiment, the client devices 102, host server 100, and conversion servers 400 are coupled via a network 106 and/or a network 108. In some embodiments, the devices 102 and host server 100 may be directly connected to one another.

[0032] The input mechanism on client devices 102 can include touch screen keypad (including single touch, multi-touch, gesture sensing in 2D or 3D, etc.), a physical keypad, a mouse, a pointer, a track pad, motion detector (e.g., including 1 -axis, 2-axis, 3-axis accelerometer, etc.), a light sensor, capacitance sensor, resistance sensor, temperature sensor, proximity sensor, a piezoelectric device, device orientation detector (e.g., electronic compass, tilt sensor, rotation sensor, gyroscope, accelerometer), or a combination of the above.

[0033] Signals received or detected indicating user activity at client devices 102 through one or more of the above input mechanism, or others, can be used in the disclosed technology by various users or collaborators (e.g., collaborators 108) for accessing, through network 106, a web-based collaboration environment or online collaboration platform (e.g., hosted by the host server 100).

[0034] The collaboration platform or environment hosts workspaces with work items that one or more users can access (e.g., view, edit, update, revise, comment, discussions, download, preview, tag, or otherwise manipulate, etc.). A work item can generally include any type of digital or electronic content that can be viewed or accessed via an electronic device (e.g., device 102). The digital content can include .pdf (Portable Document Format) files, .doc, slides (e.g., Powerpoint slides), images, audio files, multimedia content, web pages, blogs, etc. A workspace can generally refer to any grouping of a set of digital content in the collaboration platform. The grouping can be created, identified, or specified by a user or through other means. This user may be a creator user or administrative user, for example.

[0035] In general, a workspace can be associated with a set of users or collaborators (e.g., collaborators 108) which have access to the content included therein. The levels of access (e.g., based on permissions or rules) of each user or collaborator to access the content in a given workspace may be the same or may vary among the users. Each user may have their own set of access rights to every piece of content in the workspace, or each user may have different access rights to different pieces of content. Access rights may be specified by a user associated with a work space and/or a user who created/uploaded a particular piece of content to the workspace, or any other designated user or collaborator.

[0036] In general, the collaboration platform allows multiple users or collaborators to access or collaborate efforts on work items such that each user can see, remotely, edits, revisions, comments, or annotations being made to specific work items through their own user devices. For example, a user can upload a document to a work space for other users to access (e.g., for viewing, editing, commenting, discussing, signing-off, or otherwise manipulating). The user can login to the online platform and upload the document (or any other type of work item) to an existing work space or to a new work space. The document can be shared with existing users or collaborators in a work space.

[0037] A diagrammatic illustration of the online collaboration environment and the relationships between workspaces and users/collaborators are illustrated with further reference to the example of FIG. 2.

[0038] In one embodiment, actions performed on work items or other activities that occur in a work space can be detected in real time or in near real time. In addition, users, collaborators, or select users can be notified in real time or near real-time of these actions or activities. Various mechanisms can be used to notify users or collaborators, including through the web interface to access the collaboration platform, via email, and/or SMS, for example. [0039] Functions and techniques disclosed for providing previewing of files or work items in the online platform can be performed by one or more distributed conversion servers 400 of the collaboration platform. Functions and techniques performed by the host server 100, the conversion servers 400, and the related components therein are described, respectively, in detail with further reference to the examples of FIGS. 3A and 4A.

[0040] In one embodiment, client devices 102 communicate with the host server 100 and/or conversion servers 400 over network 106. In general, network 106, over which the client devices 102, the host server 100, and/or conversion servers 400 communicate, may be a cellular network, a telephonic network, an open network, such as the Internet, or a private network, such as an intranet and/or the extranet, or any combination thereof. For example, the Internet can provide file transfer, remote log in, email, news, RSS, cloud-based services, instant messaging, visual voicemail, push mail, VoIP, and other services through any known or convenient protocol, such as, but is not limited to the TCP/IP protocol, Open System Interconnections (OSI), FTP, UPnP, iSCSI, NSF, ISDN, PDH, RS-232, SDH, SONET, etc.

[0041] The network 106 can be any collection of distinct networks operating wholly or partially in conjunction to provide connectivity to the client devices 102 and the host server 100 and may appear as one or more networks to the serviced systems and devices. In one embodiment, communications to and from the client devices 102 can be achieved by, an open network, such as the Internet, or a private network, such as an intranet and/or the extranet. In one embodiment, communications can be achieved by a secure communications protocol, such as secure sockets layer (SSL), or transport layer security (TLS).

[0042] In addition, communications can be achieved via one or more networks, such as, but are not limited to, one or more of WiMax, a Local Area Network (LAN), Wireless Local Area Network (WLAN), a Personal area network (PAN), a Campus area network (CAN), a Metropolitan area network (MAN), a Wide area network (WAN), a Wireless wide area network (WWAN), enabled with technologies such as, by way of example, Global System for Mobile Communications (GSM), Personal Communications Service (PCS), Digital Advanced Mobile Phone Service (D-Amps), Bluetooth, Wi-Fi, Fixed Wireless Data, 2G, 2.5G, 3G, 4G, IMT-Advanced, pre-4G, 3G LTE, 3GPP LTE, LTE Advanced, mobile WiMax, WiMax 2, WirelessMAN-Advanced networks, enhanced data rates for GSM evolution (EDGE), General packet radio service (GPRS), enhanced GPRS, iBurst, UMTS, HSPDA, HSUPA, HSPA, UMTS-TDD, 1 xRTT, EV-DO, messaging protocols such as, TCP/IP, SMS, MMS, extensible messaging and presence protocol (XMPP), real time messaging protocol (RTMP), instant messaging and presence protocol (IMPP), instant messaging, USSD, IRC, or any other wireless data networks or messaging protocols.

[0043] FIG. 2 depicts an example diagram of a web-based or online collaboration platform deployed in an enterprise or other organizational setting 250 for organizing work items 215, 235, 255 and workspaces 205, 225, 245.

[0044] The web-based platform for collaborating on projects or jointly working on documents can be used by individual users and shared among collaborators. In addition, the collaboration platform can be deployed in an organized setting including but not limited to, a company (e.g., an enterprise setting), a department in a company, an academic institution, a department in an academic institution, a class or course setting, or any other types of organizations or organized setting.

[0045] When deployed in an organizational setting, multiple workspaces (e.g., workspace A, B C) can be created to support different projects or a variety of work flows. Each workspace can have its own associated work items. For example, work space A 205 may be associated with work items 215, work space B 225 can be associated with work items 235, and work space N 245 can be associated with work items 255. The work items 215, 235, and 255 may be unique to each work space but need not be. For example, a particular word document can be associated with only one work space (e.g., work space A 205) or it may be associated with multiple work spaces (e.g., Work space A 205 and work space B 225, etc.).

[0046] In general, each work space has a set of users or collaborators associated with it. For example, work space A 205 is associated with multiple users or collaborators 206. In some instances, work spaces deployed in an enterprise may be department specific. For example, work space B may be associated with department 210 and some users shown as example user A 208 and workspace N 245 can be associated with departments 212 and 216 and users shown as example user B 214.

[0047] Each user associated with a work space can generally access the work items associated with the work space. The level of access will depend on permissions associated with the specific work space, and/or with a specific work item. Permissions can be set for the work space or set individually on a per work item basis. For example, the creator of a work space (e.g., one of user A 208 who creates work space B) can set one permission setting applicable to all work items 235 for other associated users and/or users associated with the affiliate department 210, for example. Creator user A 208 may also set different permission settings for each work item, which may be the same for different users, or varying for different users.

[0048] In each work space A, B... N, when an action is performed on a work item by a given user or any other activity is detected in the work space, other users in the same work space may be notified in real time or in near real time. Activities which trigger real time notifications can include, by way of example but not limitation, adding, deleting, or modifying collaborators in the work space, adding, deleting a work item in the work space, creating a discussion topic in the work space.

[0049] The activity can be performed in relation to a discussion topic in the work space, for example, adding a response to a discussion topic, deleting a response, or editing a response in the work space. In addition, the activity is performed on a work item in the work space by the user, including, by way of example but not limitation, download or upload of a work item, deletion of editing of the work item, selecting, adding, deleting, and modifying a tag in the work item, preview of the work item or comment of the work item, setting or changing permissions of the work item, sharing a work item, emailing a link to the work item, and/or embedding a link to the work item on another website.

[0050] FIG. 3A depicts a block diagram illustrating an example of components in the host server 100 of a web-based collaboration environment that supports content previewing of files. The online content management system allows a user to preview the contents of an entire file stored in the system without downloading the file to the user's computer from the content management system. A user can request a preview of the contents of a file before deciding to download the file from the online content management system to the user's computer or even instead of downloading it if the user does not have the appropriate software installed on the user's computer for opening the file.

[0051] The host server 100 of the web-based or online collaboration environment can generally be a cloud-based service. The host server 100 can include, for example, a network interface 302, a collaboration manager 315, a load balancing engine 310, a network engine 320, an analytics database 336, a progress indicator engine 330, a data collection module 340, a notification module 390, an annotation module 481 , and/or a visual representation module 491 . Additional or fewer components/modules/engines can be included in the host server 100 and each illustrated component.

[0052] The network interface 302 can be a networking module that enables the host server 100 to mediate data in a network with an entity that is external to the host server 100, through any known and/or convenient communications protocol supported by the host and the external entity. The network interface 302 can include one or more of a network adaptor card, a wireless network interface card (e.g., SMS interface, WiFi interface, interfaces for various generations of mobile communication standards including but not limited to 1 G, 2G, 3G, 3.5G, 4G, LTE, etc.,), Bluetooth, a router, an access point, a wireless router, a switch, a multilayer switch, a protocol converter, a gateway, a bridge, bridge router, a hub, a digital media receiver, and/or a repeater.

[0053] As used herein, a "module," "a manager," a "handler," a "detector," an "interface," or an "engine" includes a general purpose, dedicated or shared processor and, typically, firmware or software modules that are executed by the processor. Depending upon implementation-specific or other considerations, the module, manager, handler, or engine can be centralized or its functionality distributed. The module, manager, handler, or engine can include general or special purpose hardware, firmware, or software embodied in a computer-readable (storage) medium for execution by the processor. As used herein, a computer-readable medium or computer-readable storage medium is intended to include all mediums that are statutory (e.g., in the United States, under 35 U.S.C. 101 ), and to specifically exclude all mediums that are nonstatutory in nature to the extent that the exclusion is necessary for a claim that includes the computer-readable (storage) medium to be valid. Known statutory computer- readable mediums include hardware (e.g., registers, random access memory (RAM), non-volatile (NV) storage, to name a few), but may or may not be limited to hardware.

[0054] In one embodiment, the host server 100 includes a collaborator manager 315 that tracks the permission levels of users requesting a preview of a file. A user should be an authorized collaborator of the workspace in which the file is stored to be granted permission to view a preview of the file. Additionally, the user who uploads the file to the workspace is also permitted to view a preview of the file. In some instances the person who uploads may not be an authorized collaborator of the workspace, for example, an administrator, but that person may need to check whether the file was correctly uploaded. [0055] One embodiment of the host server 100 includes the data collection module 340 which performs real time collection of statistics on every job that runs through the conversion servers 400 during the conversion process for file previewing. Some file formats do not need to be converted to any intermediate formats before being converted to the target format. These file formats can be converted in a single step. Other file formats use one or more intermediate steps to convert to intermediate formats before finally being converted to the target format. The data collection module 340 records the timing information for every step of the file conversion process and stores the information in the analytics database 336. By collecting timing data for every job and analyzing the data to optimize the number of jobs processed per server in a given amount of time, the overall throughput of jobs can be maximized by the system.

[0056] One embodiment of the host server 100 includes the load balancing engine 310 which can evaluate the load on each of the distributed conversion servers 400 and use a distributed queuing system to determine which of the distributed conversion servers 400 to send the next job. In one embodiment, jobs are sent to a particular conversion server based upon a round-robin process where each job is sent to a different server until every server has received a job, and further jobs are sent to the servers in the same order. In one embodiment, video conversion jobs can be queued in a cluster separately from other conversion jobs because video conversion can take longer than conversion of other types of files. Then for each cluster of jobs, the round- robin process would be used among the servers. Additionally, other classes of files could also be queued in their own special clusters, as the need arises.

[0057] One embodiment of the host server 100 includes the network engine 320 which can identify and evaluate the available tools for converting files from an existing format to a target format and determine the optimum tool or tools to use for the conversion process of a given file. In one embodiment, the network engine 320 can include a network graph generator 322 and a network graph solver 324.

[0058] The network graph generator 322 evaluates information on the tools available in the system for converting files. Each tool supports conversion of one or more input formats to one or more output formats. A node is established by the network graph generator 322 for each unique pair of input-output formats for each tool available to the system. Thus, if tool X can convert two input formats, A and B, to three output formats, 1 , 2, and 3, then six separate nodes are established that use tool X. In particular, the six nodes correspond to conversion from A to 1 , A to 2, A to 3, B to 1 , B to 2, and B to 3.

[0059] The network graph generator 322 also maintains a priority-cost for each node. The priority-cost is a measure for assessing the advantages and disadvantages of using a tool. The priority-cost can be based on factors including, but not limited to, how long a particular conversion tool takes to perform a conversion, the fidelity of the output file of the conversion tool, and system preferences for conversion of a file into a particular file format. Conversion tools that convert the same input file format to the same output file format but use a different algorithm, e.g. ffmpeg versus mencoder, can be maintained as different nodes with a different priority-cost for each pair of input to output file formats. In one embodiment, specific versions of conversion tools can also be maintained as separate nodes because a particular version performs better for a desired input-output file format conversion. For example, with the SWFTools library, an older version provides higher quality results for particular conversion situations. As a result, upgrading to the latest version degrades the quality of the output file. Thus, a node can be associated with each installed version of the conversion tool with specified input file format, output file format, conversion algorithm, and priority-costs.

[0060] The network graph generator 322 generates a network graph made up of all the identified nodes. The network graph generator 322 then connects appropriate nodes so that a first node is connected to a second node only if the output format of the first node is accepted as an input format to the second node, thus generating a network graph of conversion tools.

[0061] Based upon the priority-costs associated with each node, the network graph solver 324 solves the network graph to find the optimum conversion tools needed to convert a given input file format to a target output file format. The network graph solution determined by the network graph solver 324 may result in the conversion of the input file format to one or more intermediate file formats. If during the conversion of a file a particular conversion tool fails to create a sufficient output file, the network graph generator 322 can remove the effected node or nodes, and the network graph solver 324 can re-solve the network graph to re-route the conversion of the file around the fault.

[0062] Although the network engine 320 was described above with respect to generating and solving a network graph, any other method for determining the optimum set of tools for converting one file format to another file format can be used based upon the priority-costs of the various conversion tools.

[0063] One embodiment of the host server 100 includes the progress indicator engine 330 which can generate a progress bar or other type of progress indicator for display to the user to let the user know the approximate amount of time remaining for a file conversion to complete and when a display of the desired file preview will be available. The progress indicator engine 330 can include a duration analysis module 332 and/or an indicator generator 334 and can access the analytics database 336.

[0064] The duration analysis module 332 accesses data collected and stored in the analytics database 336 about previous files that have been converted by the system. The analytics database 336 includes, but is not limited to, data such as the size of a file and the format of the file. For text files, the data includes the number of pages of the file. For image and video files, the data includes the width and height of the source image format and the target image format, for example in number of pixels. For video files, the data also includes the play length of the video. The database can also include information about the number of steps used to convert the input file format to the target file format, for example, the number of intermediate file formats that the input file format needs to be converted to before reaching the final target format. In one embodiment, the analytics database 336 can be made up of several separate databases. In one embodiment, the analytics database 336 can be external to the conversion server 400 and/or shared by the conversion servers 400.

[0065] The duration analysis module 332 can find previous files that have similar attributes as the file to be converted and calculate a predicted conversion time based on the conversion times of similar files that have been converted in the past. In one embodiment, the duration analysis module 332 takes an average of the conversion times of the similar files as a predicted conversion duration for the file to be converted. In one embodiment, the duration analysis module 332 can perform a more complex calculation on the collected analytics data, for example, a weighted average depending upon how similar a previous input file is to the present file to be converted, to predict the conversion time for the present file. In one embodiment, the predicted conversion duration can be further refined by the duration analysis module 332 based on any steps of the conversion process that operate linearly and can report its progress.

[0066] Further, the analytics database 336 can include data on the load on the conversion servers at the time the previous files were converted. In one embodiment, the conversion server can comprise a number of distributed servers. By determining the load on the conversion servers when a file is to be converted to a target format, the average of the conversion times of similar files can be adjusted to the current load of the servers.

[0067] The indicator generator module 334 generates the progress indicator for displaying an indication of the time that has elapsed since the start of the conversion of the file and the expected time remaining until the file has finished converting to the target format. The indicator generator module 334 includes a timer 335 for tracking the elapsed time from when the conversion process starts to when the conversion process has finished.

[0068] In some embodiments, a video file can start to be streamed to the user prior to completion of the conversion process. In this case, the timer 335 tracks the elapsed time from when the conversion process starts to the anticipated time when the video player can start reading data and showing video to the user before the remainder of the file has been converted to the target format.

[0069] FIG. 3B depicts a block diagram illustrating an example of components in the notification module 390 that is part of the host server 100 of a web-based collaboration environment with real time activity notification capabilities.

[0070] The notification module 390 can include, for example, an activity detector 405, a notification engine 415, a permissions manager 435 and/or a notification prioritizer 455. The notification engine 415 can include a notification server manager 416 and/or a recipient selection module, the permission manager 435 can include a workspace manager 426, a collaboration manager 438, an organization tracker 440 having an enterprise tracker 441 , and/or a work item manager 442; the notification prioritizer 455 can further include a rules module 456, a chronological ordering engine 458, and/or a relevance assessor 460. Additional or less components/modules/engines can be included in the notification module 390 and each illustrated component.

[0071] One embodiment of the notification module 390 includes the activity detector 405 which can detect an activity in the web-based collaboration environment. The activity can be a triggering activity which causes select or relevant users to be notified of the occurrence, which in one embodiment, is in real time or near real-time.

[0072] The detected activity can be performed by a user or collaborator in a work space and can be performed on a work item or relating to a work item, for example, download or upload of the work item, previewing the work item, commenting of a work item, deletion or editing of the work item, commenting on a work item, identifying, selecting, adding, deleting, saving, editing, and modifying a tag in the work item, setting or changing permissions of the work item, sharing the work item including, for example, emailing a link to the work item, embedding a link to the work item on another website.

[0073] The types of activities that can be detected can also relate to changes to a work space, such as adding, deleting, or modifying collaborators in the work space; changes to work items such as adding, deleting a work item in the work space; creating a discussion topic in the work space, adding a response to a discussion topic, deleting a response, or editing a response in the work space.

[0074] Detected activity in a work space that is performed by a user or otherwise occurring can trigger notifications to be sent out, for example, via the notification engine 415. The notification engine 415 can notify users, which can be collaborators of the user who performed the activity in the work space via one or more of many mechanisms, including but not limited to, email, SMS, voice-message, text-based message, RSS, feed, etc.

[0075] In one embodiment, the notification is depicted through a web-browser used by the other user to access the web-based collaboration environment, for access in real time or near real time to when the activity was performed by the user. When notifying a user in real time through a web-browser, the notification engine 415 can utilize a push- enabled service to ensure real time notification. In one embodiment, the notification is sent by a component or another server which implements push technology (e.g., the notification server 500 shown in the example of FIG. 3C). The push-enabled service can be implemented via long poll or HTTP streaming, for example, by the notification sever 500 or another component, device which may be internal to or external to the host server 100.

[0076] The notification module 390 can send a notification server an identification of the recipient to be notified and indicator of the activity to notify the recipient of. Use of an external push server, such as the notification server 550 is described with further reference to the example of FIG. 3C. The notification server 550 can be managed by the notification server manager 416 in the notification engine 415 which can communicate events to notify users in real-time via their browser interfaces. In one embodiment, the host server sends a notification server an identification of the recipient to be notified and indicator of the activity to notify the recipient of. [0077] In general, recipients of an activity notification are selected based on criteria, for example, by the recipient selection module 418 of the notification engine 415. The criteria may be determined, for example, based on a work space in which the activity was performed in the online collaboration platform. Specifically, the criteria, is in one embodiment, determined based on permissions configured for the workspace, as managed, tracked, updated, implemented, revised, based by the permissions manager 435.

[0078] For example, the workspace can be associated with an enterprise and in such an instance, the criteria can specify that the recipient that is selected is an employee of the enterprise. Enterprise associations and affiliations can be managed by the organization tracker 440, for example; in some embodiments, enterprises and/or enterprise accounts can specifically be managed, tracked, monitored by the enterprise tracker 441 . Permissions for the workspace can configured by a creator or administrative user of the workspace. The collaboration manager 438 can determine, track, and implement relationships, roles, and/or access levels of multiple users/collaborators. For example, users may be a general user, a creator of the work space, a creator of a work item, or an administrative user. The permissions for a work space can be configured by a user, creator, or the administrative user and is generally managed by the collaborations manager 438.

[0079] The criteria that are determined by work space permissions can be managed by the work space manager 436 in the permissions manager 435. The recipient selection module 418 can also determine the recipient selection criteria based on user affiliation with the workspace, including, one or more of, member, invited collaborator and collaborator in the workspace. Such user affiliation can be tracked and managed by, for example, the collaboration manger 438 of the permissions manager 435.

[0080] In one embodiment, the criteria are determined based on permissions associated with a work item on which the activity was performed in relation to in the workspace. Permissions associated with work items can be managed, tracked, updated, revised, or implemented, in one embodiment, by the work item manager 442. For example, the permissions associated with the work item can be set by, a creator of the work item or an administrative user of the work space. Each work space can include multiple work items where each of multiple work items has individually configurable permissions. The individually configured permissions can be determined by user roles and rights (e.g., as managed by the collaborations manager 438). The work item manager 442 can communicate with the collaboration manager 438 in setting, configuring, or re-configuring permissions associated with work items.

[0081] The notification of a triggering activity, can be presented to a selected user in the web-based or online collaboration environment such that the notification is accessible by the user in real time or near real time to when the triggering activity occurred. In one embodiment, the notification is presented via a user interface to the online collaboration platform, for example, when the recipient (e.g., selected recipient) is accessing the workspace (e.g., the same work space in which activity is detected) or when the recipient is accessing a different work space. Specifically, the real time or near real time notification can be presented to the user via the user interface if the user is online (e.g., online or otherwise logged into the web-based or online collaboration environment). Examples of real time notifications are illustrated with further reference to the examples of FIG. 10 and FIG. 1 1 . The feed stream is shown in the lower right corner of the files page in FIG. 10 and the preview page in FIG. 1 1 .

[0082] The notification engine 415 can determine the channel through which to notify selected users or recipients of activity. The channels that are used can include, indicators via a user interface to the online collaboration environment, SMS, audio message, text-based messages, email, desktop application, RSS, etc. The indicators presented via the user interface can include visual indicators (e.g., pop-up form including text and/or graphics), audio indicators, or any other types detectable by a user.

[0083] In one embodiment, the notification is presented in the user interface among other notifications in an order based a rule, which may be configurable by the recipient or another user. Such prioritization in presentation can be determined, managed, tracked, implemented, revised, or updated by the notification prioritizer 455, for example. The notification prioritizer 455 can present the notification in the user interface (e.g., as shown in the user interface of FIG. 10) among other notifications in an order based on a rule as determined by the rules module 456, for example.

[0084] The rule can indicate user preferences for notifications of activities based on one or more of, a type of activity that occurred and a user related to the activity. For example, a given user may explicitly or implicitly indicate preferences for activities or actions performed by specific other users or collaborators. A user may also indicate explicitly or implicitly preferences for types of activities that they wish to be notified of or not notified of. Users may also indicate that notifications for certain types of activities are to be prioritized other others. For example, a user may indicate that a notification for a 'comment on' activity is of a higher priority compared to a 'edit' activity.

[0085] In one embodiment, the notification is presented in the user interface among other notifications based on chronological order, for example as tracked or determined by the chronological ordering engine 458. For example, each notification can be depicted in the user interface based the time ordering when each associated triggering activity occurred. Notification of the most recently occurred activity can be depicted above or below other notifications, or in a location where most easily accessed by the recipient user.

[0086] In one embodiment, the notification is presented in the user interface among other notifications based on relevance to the recipient, for example, as determined, tracked, monitored, or implemented by the relevance assessor 460. The relevance to the recipient can be represented by, for example, contextual and temporal parameters. For example, contextual parameters provide metrics indicating the recipient's current activity in the online collaboration platform. Current activity can be any activity of the user that occurred within a certain time frame (e.g., within the last minute, within the last 5 minutes, within the last 10 minutes, for example). Activity of the user can include, a document that the user edited, viewed, downloaded, commented on, tagged, or otherwise accessed. Activity of the user can also include activities surrounding a workspace, including creation/modification of a workspace or attributes of a workspace, such as modification of collaborators, permissions, etc.

[0087] Temporal parameters can, for example, provide metrics indicating the recipient's activities in the online collaboration platform over a period of time, a frequency with which the recipient has accessed a work item with which the activity relates to, and/or a frequency with which the recipient has accessed the work space in which the activity was performed.

[0088] FIG. 3C depicts a block diagram illustrating an example of components in a notification server 550 for providing real time or near real time notifications of activities that occur in a web-based or online collaboration environment.

[0089] The notification server 550 generally includes, for example, a push server 505, an SMS notifier 515, and/or a priority module 525. In one embodiment, the push server 505 includes a long poll engine 506 and/or an HTTP streaming engine 508. Additional or less components/modules/engines can be included in the notification server 550 and each illustrated component.

[0090] The notification server 550 can support the services of a collaboration platform or environment to provide real time or near real time notifications of activities. In one embodiment, the notification server 550 is integrated within a host server of a collaboration platform (e.g., the host server 100 shown in the example of FIG. 1 ). The notification server 550 may also be externally coupled to the host server (e.g., the host server 100). In some instances, a portion of the functions implemented and performed by the notification server 550 can be implemented in part or in whole in the host server 100. For example, some of the components shown to be in the notification server 500 and associated functionalities can in part or in whole reside in the host server 100.

[0091] In one embodiment, the notification server 550 sends a notification of an activity that occurs within a collaboration platform to a recipient. The notification is sent by the server 550 such that the recipient is notified in real time or near real time to when the activity occurred or when the activity was performed. Real time notification can be performed via push technology, for example by the push server 505 through long polls (e.g., via the long poll engine 506) and/or through the HTTP streaming (e.g., via the HTTP streaming engine 506). The notification server 550 can communicate with the host server to determine a recipient to whom to notify. The notification server 550 can also determine the activity to notify the recipient of, for example through communication with the host server.

[0092] In one embodiment, the notification is presented in a feed stream among other notifications through a user interface on the user device according to relevancy to the user determined based on current or recent activity of the user in the web-based collaboration environment. The presentation priority in a user interface in a feed stream can be managed, in whole, or in part, for example, by the priority module 525 using information determined by the notification prioritizer (e.g., notification prioritizer 455 of notification module 390 shown in the example of FIG. 3B).

[0093] In one embodiment, the notification server 550 can send notifications to users via SMS (e.g., through the SMS notifier 515). In this instance, the notification server 500 can be coupled to an SMS center which forwards the SMS text message to a mobile device over a cellular network. The notification can be sent via SMS in real time or near real time, or with a delay. [0094] FIG. 4A depicts a block diagram illustrating an example of components in a conversion server 400 for converting a file stored in the web-based content management system with collaboration environment to a target file format suitable for providing a preview of the file. The conversion server 400 of the web-based or online collaboration environment can include, for example, a viewer module 475, a conversion engine 410, a streaming engine 430, a downsampling engine 444, a fast start preview module 450, a rotation module 466, and/or a libraries database 472. The conversion engine 410 can include a PDF rendering engine 470.

[0095] One embodiment of the conversion server 400 includes the viewer module 475 for displaying previews of requested files in a web page. The viewer module 475 takes as input files having one or more supported file formats and displays the file. In one embodiment, the viewer module 475 uses a Flash-based browser document viewer. The document viewer is embedded in a webpage and displays Adobe Flash small web format (SWF) files. Because files stored in the web-based content management system can have any format, the conversion server 400 includes conversion tools that can convert the format of an input file and convert the file to a target output format that is supported by the viewer module 475.

[0096] One embodiment of the conversion server 400 includes the conversion engine 410 which can run the conversion tools used to convert an input file format to another file format. Non-limiting examples of conversion tools include mencoder and ffmpeg. In one embodiment, the conversion tool manager 410 manages subsystems that are designed to handle specific style formats. For example, because Microsoft Office is ideally suited to convert MSWord and PowerPoint documents, a Windows- based subsystem can be used to handle MS Office file formats.

[0097] In one embodiment, the conversion engine 410 can perform a hard-coded conversion process where specific tools are used to convert certain input formats to a specified output format. In another embodiment, the conversion engine 410 uses a path-finding algorithm implemented by the network engine 320 described above.

[0098] In some instances, a user who uploads a file to the content management system can use the content preview feature to check whether the file was successfully uploaded. It is important to ensure the preview provided to the user appears as close as possible to the actual appearance of the document when opened using the file's native program to reassure the user that the file was not corrupted during the upload process or incompletely uploaded. One embodiment of the conversion engine 410 includes the PDF (Portable Document Format) rendering engine 470 which can provide a preview of PDF files with high fidelity. The PDF rendering engine 470 can access PDF rendering libraries database 472.

[0099] PDF is a file format standard that is used for document exchange. A PDF file captures a complete description of a document, including the text, fonts, graphics, and other information needed to display the document in a manner that is independent of the application software, hardware, or operating system. There are many different versions of the PDF specification, and the different versions support different display functionalities.

[00100] There are also several open source libraries that are available for parsing PDF files and converting PDF files to other formats. For example, the pdf2swf program is one of the tools provided by the SWFTools library that can be used to convert PDF files to an Adobe Flash SWF file. The pdf2swf program uses the open source xpdf library to parse the PDF file and then provide instructions for generating the SWF file. There is another open source PDF rendering library called Poppler. Because Poppler is more actively supported and also contains a number of improvements over the xpdf library, it is advantageous to use Poppler instead of the xpdf library for parsing PDF files. Thus, in one embodiment, the pdf2swf program in the SWFTools library is modified to use the Poppler library, instead of the xpdf library, to parse the PDF file to provide instructions for generating the corresponding SWF file.

[00101] By using the Poppler library rather than the xpdf library, more accurate previews of PDF files can be provided by the viewer module 475, especially for some of the less commonly used PDF features. Some advantages include PBM picture support, corrected transparency values, improved template support, drawing improvements, and other bug fixes that were present with use of the xpdf library.

[00102] In one embodiment, the PDF rendering libraries 472 includes the modified SWFTools library and the modified Poppler library and is part of the PDF rendering engine 470. In one embodiment, the PDF rendering libraries 472 is made up of several separate libraries. In one embodiment, the PDF rendering libraries 472 can be external to the conversion server 400 and/or shared by the conversion servers 400.

[00103] One embodiment of the conversion server 400 includes the streaming engine 430 which can take the video file as it is being converted by the conversion engine 410 and write it directly to a publicly-available location in chunks. The video player can then start reading the data showing the video to the user before the rest of the file has been converted.

[00104] One embodiment of the conversion server 400 includes the downsampling engine 444 which can downsample a video file by removing pixels from video frames to reduce the size of the file for converting to a target format for preview. Advantages of downsampling include reducing the time a preview takes to start playing for a user, preventing the preview from starting, stopping, or freezing during playback, and reducing the overall bandwidth usage for the server and the end user.

[00105] In one embodiment, a file can be downsampled if it is larger than 360 pixels in the height dimension of the frames. During the downsampling process, the aspect ratio of the frames is maintained. Thus, for sources having a 4:3 aspect ratio, the resulting frame sizes of the video file would be 480x360 pixels, and for wide-format video, the resulting size would be 640x360 pixels. Other downsampling criteria, such as different pixel dimensions of the frames, can also be selected when reducing the size of a file.

[00106] In one embodiment, a video file can be downsampled to multiple different conversion sizes, for example, a high resolution version, a medium resolution version, and a low resolution version. In one embodiment, depending upon the tier of service a user pays for, one, two, or all three of the different versions can be made available to the user. For a user paying a premium to subscribe to a higher tier of service, the user would receive more versions than a user subscribing to a lower tier of service. The benefit of receiving more versions having different sizes is that the user then has the option of selecting the most appropriate size or resolution. For example, a low resolution version can be selected for a slow network connection, and a high resolution version can be selected for sharing a video with friends who have a high bandwidth network connection. In one embodiment, if more than one size file is to be produced, the downsampling for each conversion size is performed on the original input file, rather than downsampling a stream of video data multiple times, to minimize the number of conversion errors introduced.

[00107] In one embodiment, if a video file is converted from its input format directly to a desired target format, the downsampling can take place prior to the conversion of the file format to reduce the processing time needed to convert the file format. However, if the video file is first converted as part of a multi-step conversion to an intermediate format before being converted to a target format, the unsupported input file format should be converted to the intermediate format first before downsampling for the final conversion to the target format.

[00108] With traditional video file downloading, a user is not able to access the video file before it has completely downloaded because a video player needs the metadata encoded in the video file, and the metadata is usually found at the end of the file. Thus, the video player needs to wait until the video file is fully loaded prior to playing the video. One embodiment of the conversion server 400 includes the fast start preview module 450 which can move the metadata from the end of the file to the beginning of the file. By moving the metadata to the beginning of the file, the preview player can begin playing the video before the user has fully downloaded the file. In one embodiment, the fast start preview module 450 uses the qt-faststart library to move the metadata to the beginning of the video file.

[00109] One embodiment of the conversion server 400 includes the rotation module 466 which uses a rotation identification tool to identify the embedded orientation data that a digital camera uses to display a video. Once the rotation module 466 has identified the orientation, it sends this information to the conversion tool(s) as a flag to correct the video for playback in the video preview in the online collaboration environment. It is important to identify the orientation data of a video file because video taken upside-down with a digital camera that plays in an appropriate orientation in the digital camera and most desktop video players would appear upside-down in the video preview.

[00110] FIG. 4B depicts a block diagram illustrating an example of components in an annotation module 481 for supporting annotations of previews. The annotation module 481 can include, for example, an annotation engine 482, a stroke grouping module 483, a rendering module 484, an annotation streaming module 485, an annotation notification module 486, and/or an annotation version tracking module 487. The stroke grouping module 483 can include a timer 489.

[00111] One embodiment of the annotation module 481 includes the annotation engine 482 which receives annotation commands from a user and converts each annotation command to an annotation in a suitable language for describing graphics, such as simple vector graphics (SVG) for describing graphics in XML. Annotation commands can be sent by using one or more input devices, such as a mouse and/or a keyboard. [00112] One embodiment of the annotation module 481 includes the stroke grouping module 483 which can group and store individual annotation strokes specified by a user as distinct SVG strokes. These stroke groupings can be erased as a group without affecting other annotation strokes. A timer 489 can be used with a timing algorithm to combine multiple strokes into a single stroke group to allow a user to make complex annotations, such as drawing shapes, and yet be able to easily delete them. In one embodiment, the timing algorithm starts a new stroke grouping if a first predetermined amount of time passes between the completion of a stroke (e.g., upward mouse movement) and the start of the next stroke (e.g., downward mouse movement) if the mouse does not move in between the strokes. If the mouse does move, only a second predetermined amount of time is allowed to elapse before a new stroke grouping is started. In one example, the first predetermined amount of time can be 800 ms, and the second predetermined amount of time can be 200 ms, but the selected time periods can be different from these examples.

[00113] In one embodiment, standard annotation drawing is made with a free-form pen drawing tool. When the shift key is held down, the free-form pen drawing tool changes to a line drawing tool. Additionally, lines within ten degrees of horizontal are automatically snapped to horizontal to facilitate underlining sentences. When the option key is held down, the free-from pen drawing tool changes to an ellipse drawing tool. When both the shift and option keys are held down, the free-form pen drawing tool changes to a circle drawing tool.

[00114] One embodiment of the annotation module 481 includes the rendering module 484 which can render annotations using different technologies on different platforms. In one embodiment, the rendering module 484 superimposes a transparent layer on top of the SWF files displayed in the browser by the viewer module 475. The transparent layer can be superimposed using a tool for presenting graphics on webpages, such as Flash or HTML5. All annotation commands from the user that are converted to SVG or other suitable graphics language are rendered on this layer using the appropriate graphics tool.

[00115] In one embodiment, the annotation commands converted to SVG can be delivered to user devices running the iOS operating system developed by Apple Inc. of Cupertino, CA., where the annotations are rendered using native views, rather than Flash or HTML5. [00116] One embodiment of the annotation module 481 includes the annotation streaming module 485 which streams in real time annotation data between the browsers of collaborators who are reviewing and collaborating on a document. By streaming the annotation data, the participants do not need to constantly download updated versions of the original document.

[00117] One embodiment of the annotation module 481 includes the annotation notification module 486 which notifies collaborators of the workspace in which the file being annotated is stored when annotations are made to the file. Examples of methods of notification include, but are not limited to, emailing collaborators of the annotated file when annotations are made, updating the event stream in a discussion workspace of the file when an annotation command is received, and including details of the annotations in the metadata associated with the file. Event streams are described in more detail in U.S. Application No. 13/208,615, entitled "COLLABORATION SESSIONS IN A WORKSPACE ON A CLOUD-BASED CONTENT MANAGEMENT SYSTEM, filed August 12, 201 1 and is incorporated herein in its entirety. Metadata is described in more detail in U.S. Application No. 13/274,268, entitled "AUTOMATIC AND SEMIAUTOMATIC TAGGING FEATURES OF WORK ITEMS IN A SHARED WORKSPACE FOR METADATA TRACKING IN A CLOUD-BASED CONTENT MANAGEMENT SYSTEM WITH SELECTIVE OR OPTIONAL USER CONTRIBUTION", filed October 14, 201 1 and is incorporated herein in its entirety.

[00118] One embodiment of the annotation module 481 includes the annotation version tracking module 487 which stores the annotation files and tracks the version history of the files. Annotation data files are stored separately from the document being annotated to allow the document to be displayed either with or without the annotations. The annotation files include the annotation data converted to a suitable graphics language, such as SVG. Additionally, storing the annotations separately permits the annotations to be rendered by the rendering module 484 using different technologies on different platforms. In one embodiment, the file being annotated can be stored as a PDF file that includes the annotation data. Then any of the collaborators of the file can print this PDF file to obtain an annotated version of the file. One or more versions of an annotation file can be stored for each version of the file being annotated. The annotation version tracking module 487 enables users to access, view, print, and set to current any annotation file version. [00119] When there are multiple collaborators simultaneously previewing the same file, any of the collaborators can annotate the file, and the annotations from all collaborators will be visible to all other collaborators. With multiple collaborators, a unique transparent layer is superimposed on top of the SWF file displayed to each of the collaborating users' browsers, and each of the unique transparent layers is associated with one of the collaborating users. Then annotation commands provided by each collaborator are converted to annotation data and rendered on the transparent layer associated with that particular user. Annotation data files can be stored separately for each collaborator, and a composite annotation data file of annotations made by all collaborators can also be stored. As with files annotated by a single user, a PDF file can be stored that includes all of the collaborators' annotation data.

[00120] FIG. 4C depicts a block diagram illustrating an example of components in a visual representation module 491 for tracking user access of files and providing visual representations of user access of the files. The visual representation module 491 can include, for example, a tracking module 492 and/or a visual representation display module 493.

[00121] One embodiment of the visual representation module 491 includes the tracking module 492 which tracks a user's interaction with a preview of a requested file. For non-video files, the tracking module 492 collects data pertaining to the frequency with which users access sections of the file so that content popularity of the different sections can be tracked. The tracking module 492 can track sections that are inherent in the viewed file. Alternatively or additionally, the tracking module 492 can treat each page as a section for tracking purposes. In one embodiment, the tracking module 492 can track user specified sections in the file.

[00122] For video files, because the entire content area of the video is visible at all times, the tracking module 492 tracks portions of the video that have been viewed most frequently. Tracking frequency of viewing of a portion of the video can reveal the point at which viewers become bored and close the video. Similar to non-video files, the tracking module 492 can track portions inherent in the video file, such as chapters, or use user-specified sections.

[00123] In one embodiment, the tracking module 492 does not initiate tracking until the user first engages with the preview player in the browser, for example, by clicking on the preview or scrolling the preview. Other methods of engagement can also be used to determine when to begin tracking the user's engagement with the preview, for example when a user holds the computer mouse over the preview player for a prolonged period of time.

[00124] The data collected by the tracking module 492 can be stored in the visual representation database 378. In one embodiment, the visual representation database 378 can be made up of several separate databases.

[00125] One embodiment of the visual representation module 491 includes the visual representation display module 493 which can access the data stored in the visual representation database 378 by the tracking module 492 and present the user with a visual representation of user access of a file based on the tracking data for the file. The visual representation display module 493 can display different representations for the requested visual representations that indicate the frequency with which users who previewed the file accessed certain portions of the document. In one embodiment, the visual representation display module 493 can display a map of a document with color- coded sections where different colors correspond to different frequencies of access, such as might be used in a heat map.

[00126] FIG. 5 depicts a flow chart illustrating an example process for determining if a user has permission to access preview of a file.

[00127] At block 505, the system receives user input from a user requesting a preview of a file. In one embodiment, the user can request the preview by clicking on the file in the workspace of the online collaboration environment.

[00128] Then at block 510, the system determines the permission level of the requesting user. In one embodiment, a first type of permission level, e.g. an editor, allows a user to have full read and write access to a workspace, including downloading or viewing a preview of the contents of the workspace as well as uploading new content to the workspace. A second type of permission level, e.g. a viewer, allows a user to have full read access to a workspace such that the user can download or view a preview of contents of the workspace but not upload or edit contents of the workspace. A third type of permission level, e.g. an uploader, can allow a user to have limited write access to contents of a workspace such that the user can see items in the workspace but not download or view the items, while being permitted to upload new content to the workspace.

[00129] At decision block 516, the system determines if the user has permission to preview the file. If the permission level of the user does not allow previewing (block 516 - No), at block 526 the system notifies the user that permission to view the preview is denied. If the permission level of the user allows previewing (block 516 - Yes), at block 530 the system converts the file to an appropriate format suitable for providing a preview of the file. Then at block 535 the system presents the preview to the user. In one embodiment, the file is a video file. In this case, the file does not have to finish converting before the file is presented to the user as a preview, as discussed above with respect to streaming a video file.

[00130] FIG. 6 depicts a flow chart illustrating an example process for finding an optimum path using different conversion tools for converting an input file format to a target file format.

[00131] At block 605, the system identifies the format of an input file that has been uploaded by a user. Then at block 610, the system specifies the desired output format appropriate for presenting a preview of the file. In one embodiment, output size and/or preview type is also specified. Output size can be used to limit the size of the output file. Thus, if the converted file exceeds the specified threshold, the file can be reprocessed, reduced in size, or not used. In one embodiment, output dimensions are specified because in different applications, different dimensions of previews may be used. For example, a preview can be displayed as a thumbnail on a file listing page (e.g. FIG. 10) or as a large display on a file preview page (e.g. in FIG. 1 1 ). With both of these previews, the files are the same format, but have different dimensions, thus the previews with different dimensions are different system outputs.

[00132] Next, at block 615, the system solves the network graph for the optimum conversion scenario using the available conversion tools. The system takes into account the priority-cost of using each of the tools to determine the optimum path. Then at block 620, the system converts the file using the identified conversion scenario path.

[00133] At decision block 625, the system determines whether the conversion of the input file to the targeted output format was successful. If the conversion was successful (block 625 - Yes), the process ends at block 699. If the conversion failed (block 625 - No), at block 630, the system identifies the conversion tool that failed to create a sufficient output file. If the faulty conversion tool was used to convert to an intermediate file format, subsequent conversions from the output of the faulty tool would not be successful. [00134] At block 635, the system re-solves the network graph without using the nodes associated with the faulty conversion tool to identify the next optimum path using the available conversion tools. The process returns to block 620 to convert the input file using the newly identified path.

[00135] FIG. 7 depicts a flow chart illustrating an example process for converting a format of a video file to a target format for presenting as a preview.

[00136] At block 705, the system begins to receive an uploading video file streamed from a user. Then at decision block 710, the system determines if sufficient data has been received to start converting the file to a desired target format. If sufficient data has not yet been received (block 710 - No), at block 715 the system waits and continues to receive the uploading file and returns to decision block 710.

[00137] If sufficient data has been received (block 710 - Yes), at block 720, the system runs a rotation identification tool on the input file to determine the orientation of the video data. Then at decision block 725, the system determines if the file format is compatible with the conversion tool or tools that the system has available, for example mencoder. If the file format is not compatible (block 725 - No), at block 730 the system converts the input file to an intermediate format that is compatible with the conversion tool and continues to block 735. If the file format is compatible with the conversion tool (block 725 - Yes), at block 735 the system specifies the desired output format to the conversion tool.

[00138] The system applies the qt-faststart program at block 740 to the file in order to move the metadata to the beginning of the video file. This allows the output file to be streamed to the user while the conversion of the remainder of the file occurs if the user wishes to do so.

[00139] Then at block 745, the system uses the video conversion tool to start converting the file, whether it is the input file or an intermediate format file, to the desired format. The conversion process on the file does not have to finish before streaming the converted portions to the user for previewing. The converted portions of the file is stored at block 750. The converted file can be written directly to a publicly- available location in chunks, and the video player can start reading the data and showing the video to the user before the file is finished converting.

[00140] FIG. 8 depicts a flow chart illustrating an example process for generating a progress indicator that shows the amount of time remaining for converting a file. [00141] At block 805, the system receives input from a user requesting a file preview. Next, at block 810, the system compares the attributes of the file to analytics stored in a database containing attributes of previously converted files. Attributes can include the size of the file and the file format.

[00142] At block 815, the system examines the format of the file to be previewed and determines the number of intermediate steps needed, if any, to convert the file to a target suitable for previewing.

[00143] At block 820, the system obtains the current load information on the conversion server(s).

[00144] Then at block 825, the system uses the obtained data to determine an expected conversion duration. The system sets a timer and generates a progress indicator for displaying to the user at block 830. The progress of the conversion servers in performing the file conversion can be displayed in any suitable form, such as a bar or a pie.

[00145] At block 835, the system checks the timer, and at decision block 840, the system determines if the conversion has finished. If the conversion has completed (block 840 - Yes), the process ends at block 899. If the conversion has not finished (block 840 - No), at block 845 the system updates the progress indicator and returns to block 835 to check the timer.

[00146] FIG. 12 depicts a flow chart illustrating an example process for generating a preview of a PDF file. In one embodiment, the process shown in FIG. 12 is a more detailed view of the process of block 530 in FIG. 5 for converting a PDF file format to a format suitable for providing a preview.

[00147] At block 1205, the system sends the PDF file to a modified pdf2swf program. The modified pdf2swf program uses the Poppler PDF library for parsing the PDF file. The output of the modified pdf2swf program are instructions for generating a SWF file.

[00148] Then at block 1210, the system takes the instructions from the modified pdf2swf program and generates the corresponding SWF file for the PDF file. Using the generated SWF file, the system provides a preview of the PDF file to the user.

[00149] FIG. 13 depicts a flow chart illustrating an example process for receiving annotation commands for annotating a preview. [00150] At block 1350, the system displays a preview of a requested file in the requesting user's browser. At block 1310, the system superimposes a transparent layer on top of the preview.

[00151] Then at decision block 1315, the system determines if a user has requested an existing annotation file for the requested file. If a previous annotation file has been requested (block 1315 - Yes), at block 1320 the system accesses the requested annotation file and displays the annotations from that file on the transparent layer.

[00152] If no annotation file has been requested (block 1315 - No), at block 1325 the system accepts an annotation command from the previewing user. At block 1340, the system converts the annotation command into simple vector graphics (SVG) and renders the graphics onto the transparent layer in the user's browser.

[00153] Then at block 1335, the system notifies collaborators about the annotations that the user has requested for the previewed file. Non-limiting examples of methods of notification include updating the event stream for the workspace in which the requested file resides in the web-based collaboration environment and emailing the information to collaborators of the workspace. The system also adds the annotation details from the command into the metadata for the file.

[00154] At block 1340, the system stores the annotation to an annotation file. The annotation file is a new file if the user did not request an annotation file at decision block 1315. The annotation file is a new version of the requested annotation file if the user requested an annotation file at block 1315.

[00155] Then at decision block 1345, the system determines if the user has sent another annotation command. If another annotation command has been sent (block 1345 - Yes), the process returns to block 1325 to accept the annotation command. If the user has no further annotation commands (block 1345 - No), at block 1350 the system stores a PDF version of the previewed along with all of the annotations requested by the user. The process ends at block 1399.

[00156] FIG. 14A depicts a flow chart illustrating an example process for acquiring data for a previewed file for providing a visual representation of user access to the previewed file.

[00157] At block 1405, the system displays a preview of a requested file in the requesting user's browser. Then at decision block 1410, the system determines if the user has engaged the preview. Non-limiting examples of engagement by a user of a preview include clicking on the preview or scrolling the preview of a document or clicking buttons to scroll a video forward. By waiting until the user has engaged the preview, it is possible to discount the fact that every document or video preview is presented to the user starting at the beginning of the document or video. If the user has not engaged the preview (block 1410 - No), the system remains at decision block 1410 until the user engages the preview.

[00158] If the user has engaged the preview (block 1410 - Yes), the system begins to track the user's use of the preview. For documents, the system can track the duration the user remains at each section of the document. Sections of the document can be identified by pages, headings, or any other user-specified divisions. For videos, the system can track which sections have been viewed and/or the frequency with which each section is viewed.

[00159] Then at block 1420, the tracking data for the current user previewing the file is added to the tracking data previously obtained for the file and stored.

[00160] FIG. 14B depicts a flow chart illustrating an example process for presenting a visual representation of user access for a file. At decision block 1450, the system determines if a user has requested a visual representation for a particular file. If no visual representation has been requested (block 1450 - No), the system remains at decision block 1450 until a request is made.

[00161] If a visual representation has been requested (block 1450 - Yes), at block 1455, the system accesses the tracking data for the requested file. Then at block 1460, the system displays a visual representation for the file based on the tracking data. Different visual representations can be generated. In one embodiment, the requesting user can request a particular graphical representation, such as a heat map.

[00162] FIG. 15 depicts an example graphical representation of user access for a file.

[00163] . In the example visual representation, a document is shown that highlights different portions of the document according to the frequency with which the portions have been previewed. In one embodiment, a different color can be used to correspond to viewing frequency, for example, highly viewed sections can be highlighted red, moderately viewed sections can be highlighted yellow, and sections that are viewed least frequently can be highlighted green. Other colors or mapping methods can be used for displaying the visual representation. [00164] FIG. 9 shows a diagrammatic representation of a machine 900 in the example form of a computer system within which a set of instructions, for causing the machine to perform any one or more of the methodologies discussed herein, may be executed

[00165] In alternative embodiments, the machine operates as a standalone device or may be connected (e.g., networked) to other machines. In a networked deployment, the machine may operate in the capacity of a server or a client machine in a client- server network environment, or as a peer machine in a peer-to-peer (or distributed) network environment.

[00166] The machine may be a server computer, a client computer, a personal computer (PC), a user device, a tablet PC, a laptop computer, a set-top box (STB), a personal digital assistant (PDA), a cellular telephone, an iPhone, an iPad, a Blackberry, a processor, a telephone, a web appliance, a network router, switch or bridge, a console, a hand-held console, a (hand-held) gaming device, a music player, any portable, mobile, hand-held device, or any machine capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by that machine.

[00167] While the machine-readable medium or machine-readable storage medium is shown in an exemplary embodiment to be a single medium, the term "machine- readable medium" and "machine-readable storage medium" should be taken to include a single medium or multiple media (e.g., a centralized or distributed database, and/or associated caches and servers) that store the one or more sets of instructions. The term "machine-readable medium" and "machine-readable storage medium" shall also be taken to include any medium that is capable of storing, encoding or carrying a set of instructions for execution by the machine and that cause the machine to perform any one or more of the methodologies of the presently disclosed technique and innovation.

[00168] In general, the routines executed to implement the embodiments of the disclosure, may be implemented as part of an operating system or a specific application, component, program, object, module or sequence of instructions referred to as "computer programs." The computer programs typically comprise one or more instructions set at various times in various memory and storage devices in a computer, and that, when read and executed by one or more processing units or processors in a computer, cause the computer to perform operations to execute elements involving the various aspects of the disclosure. [00169] Moreover, while embodiments have been described in the context of fully functioning computers and computer systems, those skilled in the art will appreciate that the various embodiments are capable of being distributed as a program product in a variety of forms, and that the disclosure applies equally regardless of the particular type of machine or computer-readable media used to actually effect the distribution.

[00170] Further examples of machine-readable storage media, machine-readable media, or computer-readable (storage) media include, but are not limited to, recordable type media such as volatile and non-volatile memory devices, floppy and other removable disks, hard disk drives, optical disks (e.g., Compact Disk Read-Only Memory (CD ROMS), Digital Versatile Disks, (DVDs), etc.), among others, and transmission type media such as digital and analog communication links.

[00171] The network interface device enables the machine 900 to mediate data in a network with an entity that is external to the host server, through any known and/or convenient communications protocol supported by the host and the external entity. The network interface device can include one or more of a network adaptor card, a wireless network interface card, a router, an access point, a wireless router, a switch, a multilayer switch, a protocol converter, a gateway, a bridge, bridge router, a hub, a digital media receiver, and/or a repeater.

[00172] The network interface device can include a firewall which can, in some embodiments, govern and/or manage permission to access/proxy data in a computer network, and track varying levels of trust between different machines and/or applications, he firewall can be any number of modules having any combination of hardware and/or software components able to enforce a predetermined set of access rights between a particular set of machines and applications, machines and machines, and/or applications and applications, for example, to regulate the flow of traffic and resource sharing between these varying entities. The firewall may additionally manage and/or have access to an access control list which details permissions including for example, the access and operation rights of an object by an individual, a machine, and/or an application, and the circumstances under which the permission rights stand.

[00173] Other network security functions can be performed or included in the functions of the firewall, can be, for example, but are not limited to, intrusion-prevention, intrusion detection, next-generation firewall, personal firewall, etc. without deviating from the novel art of this disclosure. Conclusion

[00174] Unless the context clearly requires otherwise, throughout the description and the claims, the words "comprise," "comprising," and the like are to be construed in an inclusive sense (i.e., to say, in the sense of "including, but not limited to"), as opposed to an exclusive or exhaustive sense. As used herein, the terms "connected," "coupled," or any variant thereof means any connection or coupling, either direct or indirect, between two or more elements. Such a coupling or connection between the elements can be physical, logical, or a combination thereof. Additionally, the words "herein," "above," "below," and words of similar import, when used in this application, refer to this application as a whole and not to any particular portions of this application. Where the context permits, words in the above Detailed Description using the singular or plural number may also include the plural or singular number respectively. The word "or," in reference to a list of two or more items, covers all of the following interpretations of the word: any of the items in the list, all of the items in the list, and any combination of the items in the list.

[00175] The above Detailed Description of examples of the invention is not intended to be exhaustive or to limit the invention to the precise form disclosed above. While specific examples for the invention are described above for illustrative purposes, various equivalent modifications are possible within the scope of the invention, as those skilled in the relevant art will recognize. While processes or blocks are presented in a given order in this application, alternative implementations may perform routines having steps performed in a different order, or employ systems having blocks in a different order. Some processes or blocks may be deleted, moved, added, subdivided, combined, and/or modified to provide alternative or subcombinations. Also, while processes or blocks are at times shown as being performed in series, these processes or blocks may instead be performed or implemented in parallel, or may be performed at different times. Further any specific numbers noted herein are only examples. It is understood that alternative implementations may employ differing values or ranges.

[00176] The various illustrations and teachings provided herein can also be applied to systems other than the system described above. The elements and acts of the various examples described above can be combined to provide further implementations of the invention.

[00177] Any patents and applications and other references noted above, including any that may be listed in accompanying filing papers, are incorporated herein by reference. Aspects of the invention can be modified, if necessary, to employ the systems, functions, and concepts included in such references to provide further implementations of the invention.

[00178] These and other changes can be made to the invention in light of the above Detailed Description. While the above description describes certain examples of the invention, and describes the best mode contemplated, no matter how detailed the above appears in text, the invention can be practiced in many ways. Details of the system may vary considerably in its specific implementation, while still being encompassed by the invention disclosed herein. As noted above, particular terminology used when describing certain features or aspects of the invention should not be taken to imply that the terminology is being redefined herein to be restricted to any specific characteristics, features, or aspects of the invention with which that terminology is associated. In general, the terms used in the following claims should not be construed to limit the invention to the specific examples disclosed in the specification, unless the above Detailed Description section explicitly defines such terms. Accordingly, the actual scope of the invention encompasses not only the disclosed examples, but also all equivalent ways of practicing or implementing the invention under the claims.

Claims

CLAIMS We claim:

1 . A computer-implemented method of providing an annotatable preview of a file, comprising:

generating for display a preview of a file in a requesting user's browser, wherein the file is stored in a workspace in a web-based collaboration environment; superimposing a transparent layer over the generated preview;

receiving a command from the requesting user to annotate the preview; and converting the command into annotation data, wherein the annotation data has a format suitable for rendering on the transparent layer.

2. The method of claim 1 , further comprising storing the annotation data in a memory.

3. The method of claim 2, wherein storing the annotation data in a memory occurs when the requesting user saves the previewed file.

4. The method of claim 1 , further comprising notifying authorized collaborators of the workspace when the command is received.

5. The method of claim 4, wherein notification occurs by one or more of the following methods: email, including annotation details in a metadata associated with the file, in an event stream associated with the file.

6. The method of claim 1 , wherein the requesting user is an authorized collaborator of the workspace.

7. The method of claim 1 , wherein the requesting user uploaded the file to the web-based collaboration environment.

8. The method of claim 1 , wherein Adobe Flash is used to superimpose the transparent layer over the displayed preview.

9. The method of claim 1 , wherein HTML5 is used to superimpose the transparent layer over the displayed preview.

10. The method of claim 1 , wherein the format of the annotation data is simple vector graphics (SVG).

1 1 . The method of claim 2, wherein the stored annotation data is stored separately from the file.

12. The method of claim 1 , wherein an authorized collaborator of the workspace can view the file with the annotation data rendered on the transparent layer in near real-time.

13. The method of claim 1 , wherein an authorized collaborator of the workspace can view the file without the annotation data while the requesting user is annotating the file.

14. The method of claim 1 , further comprising upon receiving from the requesting user a delete command for a particular annotation, deleting the particular annotation from the transparent layer.

15. The method of claim 1 , wherein the command comprises mouse and keyboard commands.

16. The method of claim 1 , wherein storing the format per file version, wherein previous versions of the file are accessible, viewable, printable, and can be set as a current version of the file.

17. The method of claim 1 , further comprising storing a printable composite version of the file in a PDF format, wherein the composite file includes the annotations.

18. A method comprising:

providing a preview of a file stored in a web-based collaboration environment to one or more requesting users; tracking the one or more requesting users' interaction with the preview of the file to determine a frequency of access of a plurality of sections of the file; and upon receiving a request by an evaluating user, providing a visual representation of how frequently the one or more requesting users access the plurality of sections of the file.

19. The method of claim 18, wherein the tracking initiates when the one or more requesting users first interact with the preview.

20. The method of claim 19, wherein interacting with the preview includes clicking on or scrolling the preview.

21 . The method of claim 18, wherein the tracking initiates when the one or more requesting users maintain a pointer over the preview for a minimum duration.

22. The method of claim 18, wherein the visual representation is a heat map, and further wherein the plurality of sections of the file are highlighted using a plurality of colors that indicate the frequency of access of each section.

23. A method comprising:

providing a preview of a video file stored in a web-based collaboration environment to one or more requesting users;

tracking the one or more requesting users' interaction with the review of the video file to determine a duration of access of a plurality of sections of the video file; and

upon receiving a request by an evaluating user, providing a duration map of how long the one or more requesting users view the plurality of sections of the video file.

24. The method of claim 23, wherein the tracking initiates when the one or more requesting users first interact with the preview.

25. The method of claim 24, wherein interacting with the preview includes clicking on or scrolling the preview.

26. The method of claim 23, wherein the tracking initiates when the one or more requesting users maintain a pointer over the preview for a minimum duration.

27. The method of claim 23, wherein the duration map shows a visual representation of the plurality of sections of the video file, and each of the sections is associated with a duration using a plurality of colors or a plurality of other kinds of distinguishing representations.

28. A system comprising:

a display module configured to display a preview of a file in a requesting user's browser; and

an annotation module configured to:

convert an annotation command to a graphics language format annotation;

superimpose a transparent layer over the preview; and

render the annotation on the transparent layer.

29. The system of claim 28, wherein the transparent layer and annotation rendering are performed using Flash.

30. The system of claim 28, wherein the transparent layer and annotation rendering are performed using HTML5.

31 . A system, comprising:

a tracking module configured to track one or more requesting users' interactions with a plurality of sections of a file, wherein the file is stored in a web- based collaboration environment; and

a presentation module configured to present a visual representation of information obtained by the tracking module.

32. The system of claim 31 , wherein the tracking module tracks a frequency of users' access of the plurality of sections of the file, and further wherein the visual representation shows how frequently the one or more requesting users accessed the plurality of sections of the file.

33. The system of claim 31 , wherein the file is a video file, and the tracking module tracks a duration of users' access of the plurality of sections of the video file, and further wherein the visual representation shows how long the one or more requesting users accessed the plurality of sections of the video file.

34. A system, comprising:

means for providing a preview of a file stored in a web-based collaboration environment to one or more requesting users;

means for tracking the one or more requesting users' interaction with the preview of the file to determine a frequency of access of a plurality of sections of the file; and

upon receiving a request by an evaluating user, means for providing a visual representation of how frequently the one or more requesting users access the plurality of sections of the file.

35. A computer-implemented method of permitting a plurality of users to collaborate by annotating a file, comprising:

superimposing a plurality of independent transparent layers over a generated preview for display on each of the plurality of users' browsers, wherein the file is stored in a workspace in a web-based collaboration environment, and further wherein a unique transparent layer of the plurality of independent transparent layers is associated with each of the plurality of users;

receiving commands from the plurality of users to annotate the preview; and converting each command into annotation data, wherein the annotation data has a format suitable for rendering on the transparent layers, and wherein the annotation data is rendered on the unique transparent layer associated with the user who provided the command.

36. The method of claim 35, wherein the annotation data rendered on each independent transparent layer is stored in a separate file.