WO2007082167A2

WO2007082167A2 - System and methods for storing, editing, and sharing digital video

Info

Publication number: WO2007082167A2
Application number: PCT/US2007/060175
Authority: WO
Inventors: David A. Dudas; James H. Kaskade; Kenneth W. O'flaherty
Original assignee: Eyespot Corporation
Priority date: 2006-01-05
Filing date: 2007-01-05
Publication date: 2007-07-19
Also published as: EP1969447A2; JP2009527135A; WO2007082167A3

Abstract

A system and related methods comprising an Internet-hosted application service for online storage, editing and sharing of digital video content and a companion client PC-based video upload application, whereby users can perform the task of editing their video material online while the same material is being uploaded and stored at the remote service, regardless of the size of the material. The Internet-hosted application service can be used on a dedicated website or its functionality can be served to different websites seeking to provide users with enhanced video editing capabilities.

Description

SYSTEM AND METHODS FOR STORING, EDITING₅ AND SHARING DIGITAL VIDEO

[0001] This application hereby incorporates by reference the following U.S.

Non-Provisional Patent Applications.

FIELD OF THE INVENTION

[0002] This invention relates in general to the use of computer technology to store, edit, and share personal digital video material.

BACKGROUND

[0003] Storing, editing, and sharing video is difficult for consumers who create video material today on various electronic devices, including digital still cameras ("DSCs"), digital video camcorders ("DVCs"), mobile phones equipped with video cameras and computer mounted web cameras ("webcams"). These devices create video files of varying sizes, resolutions and formats. Digital video recorders ("DVRs"), in particular, are capable of recording several hours of high-resolution material occupying multiple gigabytes of digital storage. Consumers who generate these video files typically wish to edit their material down to the highlights that they wish to keep, save the resulting edited material on some permanent storage medium, and then share this material with friends and family, or possibly with the public at large.

[0004] A wide variety of devices exist for viewing video material, ranging from

DVD players, TV-connected digital set-top boxes ("DSTBs") and DVRs, mobile phones, personal computers ("PCs"), and video viewing devices that download material via the PC, such as handheld devices (e.g., PalmOne), or the Apple video iPod. The video recording formats accepted by each of these viewing devices vary widely, and it is unlikely that the format that a particular delivery device accepts will match the format in which a particular video production will have been recorded. [0005] Figure 1 is a block diagram illustrating a prior art video editing platform including a creation block 199, a consumption block 198, and a media aggregation, storage, manipulation & delivery infrastructure 108. Figure 1 shows with arrows the paths that currently exist for transferring video material from a particular source, including a DSC 100, a DVC 102, a mobile phone 104, and a webcam 106 to a particular destination viewing device including a DVD player 110, a DSTB 112, a DVR 114, a mobile phone 116, a handheld 118, a video iPod 120, or a PC 122. The only destination device that supports material from all input devices is the PC 122. Otherwise, mobile phone 104 can send video material to another mobile phone 116, and a limited number of today's digital camcorders and digital cameras can create video material on DVDs that can then be viewed on the DVD player 110. In general, these paths are fractured and many of the devices in the creation block 199 have no way of interfacing with many of the devices in the consumption block 198. Beyond thehighlighted paths through the media aggregation, storage, manipulation & delivery infrastructure 108, no other practical video transfer paths exist today. [0006] Thus, consumers who create a family video on their digital camcorder face a major obstacle not just in editing their lengthy material, but also in sharing the resulting edited production with friends or family who may wish to view the production on their mobile phone 116, or on their video iPod 120, or on their TV via the DSTB 112 or DVR 114.

[0007] Various software solutions exist today to handle the first of these problems, the editing of video files. All of today's software solutions are applications that run on the PC 122 (or Apple Macintosh). Most of these applications carry a significant price tag. But regardless of cost, all of the existing PC-based or Mac- based video editing applications are complex to use, and as a result are not yet used by the majority of the video-creating consumers. Thus, most such consumers are left with major amounts of raw, unedited video footage stored on the original capture medium, which is unlikely to be ever shared with any but the most devoted of friends or family.

[0008] Consumers typically do not succeed in editing their video material into shareable productions. Consumers who do, or who are prepared to share their material in unedited from, are still faced with the issue of reaching their intended audience. The most prevalent solution today is to invite their audience into their home and show their productions on the PC 120, or by connecting their video capture device up to their TV.

[0009] Recently, several online services have emerged to support sharing of video content over the Internet, but most of these are so far limited to short video clips. Furthermore, these services do not support significant camcorder-generated productions and do not support video editing capabilities. [0010] A growing number of consumers are now using the available online video-sharing websites to share their short edited or unedited productions. Such websites (e.g.,YouTube, Phanfare, Clipshack, Vimeo, Ourmedia, or Google Video) typically allow their members to specify who may view their productions, and most of their users decide to make most of their videos viewable by the general public. Thus the video-sharing websites typically have thousands of videos available for general viewing.

[0011] In order to assist visitors to search for videos of interest, the video- sharing websites typically request their members to provide information describing each of their video productions, such as a title and one or more descriptive words that characterize the content of the video. The title and descriptions for each video production are stored as very simple metadata associated with the final video production by the website. They canbe displayed to visitors in simple ways, including sometimes organized by subject matter, and sometimes in the form of a collection of descriptions where the font size varies according to the popularity of the description, the most popular having the largest font. Clicking on a descriptive word brings up a set of thumbnail images of the videos corresponding to the description (often in the form of several successive pages of thumbnail images, one for each referenced video). Clicking on a thumbnail launches the video represented by the thumbnail._. No information or data is provided with regard to the elements that make up the video production.

[0012] Besides assisting in searches for particular videos, descriptions of the video have additional potential value when applied to online video material. For example, a description could be used to quickly access a specific section within a video production in order to view the specific section, or to reuse the section by inserting it into a new video production. Descriptive words may also be used to automatically aggregate and link together two or more video productions or video sections into a new production. However, none of these capabilities are offered in any of today's online video-sharing systems.

[0013] Adding descriptive words to video is not generally supported by the desktop video editors nor online video sharing solutions, since they were not designed with online sharing and reuse in mind. Some of the desktop editors do automatically detect and mark scene transitions (where a camcorder has been stopped and then restarted), and some allow the user to insert the equivalent of a bookmark or a chapter marker, but these markers are intended for later use in reviewing or re-editing a particular video production, rather than for online content searching, or sharing, or reuse in other productions. Some of today's online video- sharing sites support descriptions at the level of an entire video, but none allow adding descriptions to particular points within a video production. [0014] For example, consider a person who has shot a collection of 15-30 second clips of the family's vacation in Hawaii. In creating a movie out of these clips, the person would like to include some introductory material about the geography of Hawaii and showing some aerial footage of Waikiki beach. Current description schemes, however, do not support such an application. [0015] Moreover, none of today's online video sharing services supports editing. Thus there is no end-to-end solution that allows consumers to upload their raw video material to a website, edit the material into a viewable production, and then share the result with an audience of friends, family or the general public on the viewing device of their choice. In particular, there is no online solution that handles lengthy high-resolution video files created on digital camcorders. [0016] With an increasing number of amateur video productions of unknown quality becoming available for viewing online, there has also emerged an increasing need for viewers to be able to browse or preview available material before spending time watching it. Without such a browsing capability, viewers can easily become discontented with their viewing experience, after encountering a series of poor quality productions some of which they may abandon before completion. [0017] A few sites are focused on supporting bloggers and other users who wish to post video material to their blog sites or to other sites, such as social networking personal pages or individual websites. The state-of-the-art in online video publishing is therefore currently limited to uploading material for viewing, and, in some cases, posting to other websites. The methods of posting to other websites are generally cumbersome, involving copying and pasting a hypertext markup language ("HTML") snippet to a blog or a web page.

[0018] Similar requirements apply to the forwarding of video productions, where the current state-of-the-art is equally undeveloped. Most video-sharing sites today do not provide any way for viewers to forward a video production to another person or destination. The most sophisticated form of forwarding today (e.g., on the YouTube website) is to provide a "Forward" button (or "Share This Video" button) which, when clicked on, provides an email form to fill out in order to send the production as an attachment to an email message.

[0019] No other video forwarding channels exist today, other than email. In addition, cell phone users face major challenges in creating significant video productions out of the many video clips that they can now so easily record. Not only do they have no practical available video editing facilities, but most current video- sharing websites do not typically support direct input of video clips from cell phones and none allow sharing.

[0020] Many cell phone users, particularly younger users, also have a strong desire for instant communication. This desire extends into video sharing - they would like to send their just-recorded video clips to their friends. The ideal way to do this would be to spend a few minutes on their phone combining and editing the last set of recorded clips into a professional-looking production, rather than sending unedited clips individually.

[0021] There is thus a need to provide consumers with an online service that facilitates the creation, editing, and sharing of video productions, and eliminates many of the drawbacks associated with current schemes.

SUMMARY

[0022] Systems and methods are described for storing, editing, and sharing digital video. According to one example, a system and method is provided for uploading, editing and distributing video material in an online environment, whereby users can perform the task of editing their video material online while the same material is being uploaded and stored at a remote Internet-hosted service, regardless of the size of the material. [0023] One example of the system comprises an Internet-hosted application service for online storage, editing and sharing of digital video content and a companion client PC-based video upload application. The Internet-hosted application may be based on a group of technologies referred to as asynchronous JavaScript and extensible markup language ("AJAX"), which allows the online editing application to behave as if it resides on the user's local computing device, rather than across the Internet on a remote computing device, such as a server. The online editing application provides users with a "drag-and-drop" interface for creating their video productions. Similarly, the client PC-based video upload application facilitates the input of lengthy camcorder video material by automatically segmenting, compressing and uploading the material from the user's PC, while allowing users to edit their material during the upload process. The Internet-hosted application service can be used on a dedicated website or its functionality can be served to different websites seeking to provide users with enhanced video editing capabilities. [0024] Another aspect enables users to browse or preview video material in an online environment. The example includes variations on the use of thumbnail images, and the use of a virtual joystick to vary the replay speed of the video. In one example, users can select the browsing method that they find most effective in previewing video material presented to them.

[0025] Another aspect stores, edits, and distributes video material in an online environment. One aspect is automated, whereby creators or owners of online video productions may select a production and a destination target, and then publish the production to the destination target with one click. In another aspect, viewers of a video production are allowed to select a destination target and forward the production to any destination with one click. The possible destinations include websites, email recipients, Instant Messaging recipients, mobile phone users, software applications, digital set-top boxes and digital video recorders, as well as any pre-defined combination of these, for example.

[0026] Another aspect allows users to share the processes by which video productions have been created, in the form of hyper-templates. Users can designate as shareable the template they used in creating a video production, such that other users may reuse the template in their own productions. One method of invoking a template during the viewing of a video production is by clicking on button or on a watermark within the video that acts as a hyperlink into an online video editor and causes the editor to pre-load the particular template, ready for reuse. Additionally, styles can be provided which are automated templates. The styles include a template, a question list and a program which automatically applies the template to a user's media.

[0027] In another example, consumers may insert hypervideo links into their video material during the online editing process. Viewers of the video material may optionally follow an inserted hypervideo link by clicking on its visible representation during replay and selecting an alternative non-linear viewing path. [0028] Another aspect allows users to classify video material for future use

(e.g., filtering, advertising, copyright protection, and making recommendations) by associating tags with specific segments of a video file ("segment tags"), or with specific points in time within a segment ("deep tags"), such that the tags can later be used as search terms to find video productions of particular interest, or to quickly access specific parts of video material for viewing or for reuse in creating a new video production or for advertising or for filtering or for personalization. [0029] Other features and advantages of the present invention will become more readily apparent to those of ordinary skill in the art after reviewing the following detailed description and accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The details of the present invention, both as to its structure and operation, may be gleaned in part by study of the accompanying drawings, in which like reference numerals refer to like parts, and in which:

[0031] Figure 1 is a block diagram illustrating a prior art video editing platform.

[0032] Figure 2 is a block diagram illustrating the functional blocks or modules in an example architecture.

[0033] Figure 3 is a block diagram illustrating an example online video platform.

[0034] Figure 4 is a block diagram illustrating an example online video editor application.

[0035] Figure 5 is a block diagram illustrating an example video preprocessing application. [0036] Figure 6 is a diagram illustrating an example process for automatically segmenting a video file.

[0037] Figure 7 is a diagram illustrating an example process for automatically compressing a video file.

[0038] Figure 8 is a diagram illustrating an example process for automatically uploading a video file.

[0039] Figure 9 is a diagram illustrating an example process for allowing immediate online editing of video material, using thumbnails, while the material is being uploaded.

[0040] Figure 10 is a diagram illustrating an example process for browsing a video file.

[0041] Figure 11 is a diagram illustrating an example process for automatically transcoding video materials to the appropriate format for a video-receiving destination device.

[0042] Figure 12 is a block diagram illustrating an example edit sequence.

[0043] Figure 13 is a block diagram illustrating example data structures that support hyper-templates.

[0044] Figure 14 is a diagram illustrating an example process for editing video material and distributing the edited video material using a cell phone.

[0045] Figure 15 is a diagram illustrating an example process for using a hypervideo link.

[0046] Figure 16 is a diagram illustrating an example process for defining a hotspot.

[0047] Figure 17 is a diagram illustrating an example process for direct uploading and editing.

DETAILED DESCRIPTION

[0048] Certain examples as disclosed herein provide for the use of computer technology to store, edit, and share personal digital video material. Various methods, for example, as disclosed herein enable a user to handle large video files created on video recording devices; enable users to browse video material in an online environment; publish a video production or forward a viewed production to any destination with one click; view an online video and create a video using the same process that was used to create the viewed video; edit and distribute video material directly from a mobile device on a network, such as a cell phone; pursue multiple possible viewing paths within or outside a video production; use tags with specific segments of a video file; and use the tags to find video productions or portions of video productions of particular interest.

[0049] After reading this description it will become apparent to one skilled in the art how to implement the invention in various alternative examples and alternative applications. However, although various examples of the present invention are described herein, it is understood that these examples are presented by way of example only, and not limitation. As such, this detailed description of various alternative examples should not be construed to limit the scope or breadth of the present invention as set forth in the appended claims.

[0050] Those of skill will further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the embodiments disclosed herein can often be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled persons can implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the invention. In addition, the grouping of functions within a module, block, circuit or step is for ease of description. Specific functions or steps can be moved from one module, block or circuit without departing from the invention.

[0051] Referring now to the Figures, Figure 2 is a block diagram illustrating the functional blocks or modules in an example architecture. In the illustrated example, a system 200 includes an online video platform 206, an online video editor 202, a preprocessing application 204, as well as a content creation block 208 and a content consumption block 210.

[0052] The content creation block 208 may include input data from multiple sources that are provided to the online video platform 206, including personal video creation devices 212, personal photo and music repositories 214, and personally selected online video resources 216, for example.

[0053] In one example, video files may be uploaded by consumers from their personal video creation devices 212. The personal video creation devices 212 may include, for example, DSCs, DVCs, mobile devices equipped with video cameras, and webcams. In another example, input to the online video platform 206 may be obtained from other sources of digital video and non-video content selected by the user. Non-video sources include the personal photo and music repositories 214, which may be stored on the user's PC, or on the video server, or on an external server, such as a photo-sharing application service provider ("ASP"), for example. Additional video sources include websites that publish shareable video material, such as news organizations or other external video-sharing sites, which are designated as personally selected online video resources 216, for example. [0054] Video productions may be output by the online video platform 206 to the content consumption block 210. Content consumption block 210 may be utilized by a user of a variety of possible destination devices, including, but not limited to, mobile devices 218, computers 220, DVRs 222, DSTBs 224, and DVDs 226. The mobile devices 218 may be, for example, cell phones or PDAs equipped with video display capability. The computers 220 may include PCs, Apples, or other computers or video viewing devices that download material via the PC or Apple, such as handheld devices (e.g., PalmOne), or an Apple video iPod. The DVDs 226 may be used as a media to output video productions to a permanent storage location, as part of a fulfillment service for example.

[0055] The online video editor 202 (also referred to as the Internet-hosted application service) can be used on a dedicated website or its functionality can be served to different websites seeking to provide users with enhanced video editing capabilities. For example, a user may go to any number of external websites providing an enhanced video editing service. The present system may be used, for example, to enable the external websites to provide the video editing capabilities while maintaining the look and feel of the external websites. In that respect, the user of one of the external websites may not be aware that they are using the present system other than the fact that they are using functionality provided by the present system. In a transparent manner then, the system may serve the application to the external IP address of the external website and provide the needed function while at the same time running the application in a manner consistent with the graphical user interface ("GUI") that is already implemented at the external IP address. Alternatively, a user of the external website may cause the invocation of a redirection and GUI recreation module 230, which may cause the user to be redirected to one of the servers used in the present system which provides the needed functionality while at the same time recreating the look and feel of the external website. [0056] Delivery by the online video platform 206 to the mobile devices 218 may use a variety of methods, including but not limited to a multimedia messaging service ("MMS"), a wireless application protocol ("WAP"), and instant messaging ("IM"). Delivery by the online video platform 206 to the computers 220 may use a variety of methods, including but not limited to: email, IM, uniform resource locator ("URL") addresses, peer-to-peer file distribution ("P2P"), or really simple syndication ("RSS"), for example.

[0057] The functions and the operation of the online video platform 206 will now be described in more detail with reference to Figure 3. Figure 3 is a block diagram illustrating an example online video platform. In the illustrated example, the online video platform 206 includes an opt-in engine module 300, a delivery engine module 302, a presence engine module 304, a transcoding engine module 306, an analytic engine module 308, and an editing engine module 310. [0058] The online video platform 206 may be implemented on one or more servers, for example, Linux servers. The system can leverage open source applications and an open source software development environment. The system has been architected to be extremely scalable, requiring no system reconfiguration to accommodate a growing number of service users, and to support the need for high reliability.

[0059] The application suite may be based on AJAX where the online application behaves as if it resides on the user's local computing device, rather than across the Internet on a remote computing device, such as a server. The AJAX architecture allows users to manipulate data and perform "drag and drop" operations, without the need for page refreshes or other interruptions. [0060] The opt-in engine module 300 may be a server, which manages distribution relationships between content producers in the content creation block 208 and content consumers in the content consumption block 210. The delivery engine module 302 may be a server that manages the delivery of content from content producers in the content creation block 208 to content consumers in the content consumption block 210. The presence engine module 304 may be a server that determines device priority for delivery of content to each consumer, based on predefined delivery preferences and detection of consumer presence at each delivery device.

[0061] The transcoding engine module 306 may be a server that performs decoding and encoding tasks on media to achieve optimal format for delivery to target devices. The analytic engine module 308 may be a server that maintains and analyzes statistical data relating to website activity and viewer behavior. The editing engine module 310 may be a server that performs tasks associated with enabling a user to edit productions efficiently in an online environment. [0062] The functions and the operation of the online video editor 202 will now be described in more detail with reference to Figure 4. Figure 4 is a block diagram illustrating an example online video editor 202. In the illustrated example, the online video editor 202 includes an interface 400, input media 402a-h, and a template 404. A digital content aggregation and control module 406 may also be used in conjunction with the online video editor 202 and thumbnails 408 representing the actual video files may be included in the interface 400.

[0063] The online video editor 202 may be an Internet-hosted application, which provides the interface 400 for selecting video and other digital material (e.g., music, voice, photos) and incorporating the selected materials into a video production via the digital content aggregation and control module 406. The digital content aggregation and control module 406 may be software, hardware, and/or firmware that enables the modification of the video production as well as the visual representation of the user's actions in the interface 400. The input media 402a-h may include such input sources as the shutterfly website 402a, remote media 402b, local media 402c, the napster web service 402d, the real rhapsody website 402e, the garage band website 402f, the fiickr website 402g and webshots 402h. The input media 402a-h may be media that the user has selected for possible inclusion in the video production and may be represented as the thumbnails 408 in a working "palette" of available material elements, in the main window of the interface 400. The input media 402a-h may be of diverse types and formats, which may be aggregated together by the digital content aggregation and control module 406. [0064] The thumbnails 408 are used as a way to represent material and can be acted on in parallel with the upload process. The thumbnails 408 may be generated in a number of manners. For example, the thumbnails may be single still frames created from certain sections within the video, clip, or mix. Alternatively, the thumbnails 408 may include multiple selections of frames (e.g., a quadrant of four frames). In another example, the thumbnails may include an actual sample of the video in seconds (e.g., a 1 minute video could be represented by the first 5 seconds). In yet another example, the thumbnails 408 can be multiple samples of video (e.g., 4 thumbnails of 3 second videos for a total of 12 seconds). In general, the thumbnails 408 are a method of representing the media to be uploaded (and after it is uploaded), whereby the process of creating the representation and uploading it takes a significantly less amount of time than either uploading the original media or compressing and uploading the original media. [0065] The online video editor 202 allows the user to choose (or can create) the template 404 for the video production. The template 404 may represent a timeline sequence and structure for insertion of materials into the production. The template 404 may be presented in a separate window at the bottom of the screen, and the online video editor 202 via the digital content aggregation and control module 406 may allow the user to drag and drop the thumbnails 408 (representing material content) in order to insert them into the timeline to create the new video production. The online video editor 202 may also allow the user to select from a library of special effects to create transitions between scenes in the video. The work-in-progress of a particular video project may be shown in a separate window. [0066] On completion of the project, the online video editor 202 allows the user to publish the video to one or more previously defined galleries / archives 410. Any new video published to the gallery / archive 410 can be made available automatically to all subscribers 412 to the gallery. Alternatively, the user may choose to keep certain productions private or to only share the productions with certain users.

[0067] The functions and the operation of the preprocessing application 204 will now bθ described in more detail with reference to Figure 5. Figure 5 is a block diagram illustrating an example preprocessing application. In the illustrated example, the preprocessing application 204 includes a data model module 502, a control module 504, a user interface module 506, foundation classes 508, an operating system module 510, a video segmentation module 512, a video compression module 514, a video segment upload module 516, a video source 518, and video segment files 520.

[0068] In one example, the preprocessing application 204 is written in C++ and runs on a Windows PC, wherein the foundation classes 508 includes Microsoft foundation classes ("MFCs"). In this example, an object-oriented programming model is provided to the Windows APIs. In another example, the preprocessing application 204 is written, wherein the foundation classes 508 are in a format suitable for the operating system module 510 to be the Linux operating system. The video segment upload module 516 may be an application that uses a Model-View- Controller ("MVC") architecture. The MVC architecture separates the data model module 502, the user interface module 506, and the control module 504 into three distinct components. ,

[0069] In operation, the preprocessing application 204 automatically segments, compresses, and uploads video material from the user's PC, regardless of length. The preprocessing application 204 uses the video segmentation module 512, the video compression module 514, and the video segment upload module 516 respectively to perform these tasks. The uploading method works in parallel with the online video editor 202, allowing the user to begin editing the material immediately, while the material is in the process of being uploaded. The material may be uploaded to the online video platform 206 and stored as one or more video segment files 520, one file per segment, for example.

[0070] The video source 518 may be a digital video camcorder or other video source device. In one example, the preprocessing application 204 starts automatically when the video source 518 is plugged into the user's PC. Thereafter, it may automatically segment the video stream by scene transition using the video segmentation module 512, and save each of the video segment files 520 as a separate file on the PC.

[0071] From the user's perspective, a video would be captured on any number of devices at the video source block 518. Once the user captured the video (i.e., on their camcorder, cellular phone, etc.) it would be transferred to a local computing device, such as the hard drive of a client computer with Internet access. [0072] Alternatively videos can be transferred to a local computing device whereby an intelligent uploader can be deployed. In some cases, the video can be sent directly from the video source block 518 over a wireless network (not shown), then over the Internet, and finally to the online video platform 206. This alternative bypasses the need to involve a local computing device or a client computer. However, this example is most useful when the video, clip, or mix is either very short, or highly compressed, or both.

[0073] In the case that the video is not compressed or long or both, and, therefore, relatively large, it is typically transferred first to a client computer where an intelligent uploader is useful. In this example, an upload process is initiated from a local computing device using the video segment upload module 516, which facilitates the input of lengthy video material. To that end, the user would be provided with the ability to interact with the user interface module 506. Based on user input, the control module 504 controls the video segmentation module 512 and the video compression module 514, wherein the video material is segmented and compressed into the video segment files 520. For example, a lengthy production may be segmented into 100 upload segments, which are in turn compressed into 100 segmented and compressed upload segments.

[0074] Each of the compressed video segment files 520 begin to be uploaded separately via the video segment upload module 516 under the direction of the control module 504. This may occur, for example, by each of the upload segments being uploaded in parallel. Alternatively each of the upload segments may be uploaded in order, the largest segment first, the smallest segment first, or any other manner.

[0075] As the video material is being uploaded, the online video editor 202 is presented to the user. Through a user interface provided by the user interface module 506, thumbnails representing the video segments in the process of being uploaded are made available to the user. The user would proceed to edit the video material via an interaction with the thumbnails. For example, the user may be provided with the ability to drag and drop the thumbnails into and out of a timeline or a storyline, to modify the order of the segments that will appear in the final edited video material.

[0076] The system is configured to behave as if all of the video represented by the thumbnails is currently in one location (i.e., on the user's local computer) despite the fact that the material is still in the process of being uploaded by the video segment upload module 516. When the user performs an editing action on the thumbnails, for example, by dragging one of the thumbnails into a storyline, the upload process may be changed. For example, if the upload process was uploading all of the compressed upload segments in sequential order and the user dropped an upload segment representing the last sequential portion of the production into the storyline, the upload process may immediately begin to upload the last sequential portion of the production, thereby lowering the priority of the segments that were currently being uploaded prior to the user's editing action. [0077] All of the user's editing actions are saved by the online video editor

202. Once the material is uploaded completely (including the prioritized upload segments and the remaining upload segments), the saved editing actions are applied to the completely uploaded segments. In this manner, the user may have already finished the editing process and logged off or the user may still be logged on. Regardless, the process of applying the edits only when the material is finished uploading saves the user from having to wait for the upload process to finish before editing the material. Once the final edits are applied, various capabilities exist to share, forward, publish, browse, and otherwise use the uploaded video in a number of ways.

[0078] Figure 6 is a diagram illustrating an example process for automatically segmenting a video file. This process can be carried out by the preprocessing application 204 previously described with respect to Figure 2. In particular, the video segmentation module 512 of the preprocessing application 204 may be used to carry out one or more of the steps described in Figure 6. At step 600, scene transitions within the video material are automatically detected. At step 602, the material is segmented into separate files. Step 602 may include the preprocessing application 204 providing for the application of metadata tags by the user for the purpose of defining the subject matter. These additional steps may allow the user to apply one or more descriptive names to each file segment ("segment tags") at step 604, and further to preview the content of each file segment and to provide additional descriptive names ("deep tags") defining specific points-in-time within the file segment at step 606.

[0079] Both segment tags and deep tags at steps 604 and 606 can later be used as metadata references in search and retrieval operations by the user on video material stored within a remote computing device, such as a server. Thus, for example, if the segment tag "harbor" has been applied to the file segment and the deep tag "sailboat" has been applied to the one-minute mark within the segment where a sailboat appears, then any subsequent viewer searching on either of these tags will retrieve the file segment, and the segment will be positioned for viewing at the appropriate point: at the start of the segment if the search term was "harbor" or at the one-minute mark if the search term was "sailboat."

Furthermore, in any subsequent video editing process, if the user searches on the term "sailboat," the drag-and-drop editor will automatically extract the segment beginning at the sailboat scene, rather than requiring the user to manually edit or clip the segment. In the above example, the deep tags 606 can be used to dynamically serve up advertisements at appropriate times of viewing based on an association between time and the deep tags 606.

[0080] The separate files may be ready for uploading to a server at this stage, for example. At step 608, a thumbnail image is created for each file segment. Then, at step 610, the set of thumbnail images representing all of the video file segments is initially uploaded to the server. In one example, the thumbnail images may be selected by copying the first non-blank image in each video file segment, for example, and then uploading them to a remote computing device using the video segment upload module 516.

[0081] Figure 7 is a diagram illustrating an example process for automatically compressing a video file. This process can be carried out by the preprocessing application 204 previously described with respect to Figure 2. In particular, the video compression module 514 of the preprocessing application 204 may be used to carry out one or more of the steps described in Figure 7. At step 700, the format and resolution of the subject video material is automatically detected. At step 702, the appropriate decode software module to handle the detected input format is selected. At step 704, the video material is decoded from the input format using the selected decode codec. At step 706, the video material is encoded into a base format using a base codec. In one example, a DivX codec can be used as the base codec to encode the video material into the DivX format, although other base codecs can be used.

[0082] The video compression module 514 may use DivX because it is an emerging industry-standard format for digital video compression, which typically achieves a space reduction of 15:1 over raw video material. By using the DivX video compression technology, user and equipment productivity may be greatly enhanced by dramatically shortening the subsequent upload time for the video. (A typical 30- minute sequence of uncompressed digital camcorder material would take approximately 30 hours to upload over a standard DSL line, whereas the compressed form would take approximately 2 hours.) After encoding the video material to the base format, a local copy of the compressed video material is stored on the user's local PC at step 708.

[0083] Figure 8 is a diagram illustrating an example process for automatically uploading a video file. This process can be carried out by the control module 504 and the video segment upload module 516 of Figure 5, which typically resides in the preprocessing application 204 previously described with respect to Figure 2. At step 800, video segments that are subject to editing actions by the user are automatically detected. At step 802, segments that the user has requested to be deleted in their entirety are automatically detected and deleted. At step 804, the compressed video file segments are uploaded individually by the video segment upload module 516 to the remote computing device, while giving priority to those remaining segments that have been subject to user editing actions, for example. Then at step 806, the process of uploading all except deleted segments to the remote computing device is completed, without involving the user.

[0084] One aspect of the process described in Figure 8 is that the uploading of compressed video material is accomplished independently and asynchronously from the user, who can be offline from his or her computer during the remaining upload process, or can be engaged in other activities on his or her PC (including online editing of the video material prior to its arrival at the server). The resulting material is eventually uploaded to the online video editor 202. [0085] Figure 9 is a diagram illustrating an example process for allowing immediate online editing of video material, using thumbnails, while the material is being uploaded. This process can be carried out by the online video editor 202 in conjunction with the preprocessing application 204 previously described with respect to Figure 2. At step 900, the uploaded thumbnail images representing each video file segment that the user wishes to retain are saved. At step 902, the uploaded thumbnail images are visually displayed to the user as editable entities within the interface 400 (which may act as surrogate placeholders for the actual video file segments). At step 904, the user is allowed to perform editing actions on the thumbnail images segments, including, for example, dragging and dropping thumbnails into a video production timeline.

[0086] At step 906 all of the editing actions performed by the user are remembered and/or saved by the remote computing device. Then at step 908, all of the editing actions are applied to the actual video material after the material has completed the uploading process. This process may occur, for example, without the continuing involvement of the user. User productivity is thereby further enhanced by not requiring the user to be online while the actual editing actions are performed on the uploaded video material.

[0087] Thus two particular aspects of the system greatly improve user productivity: First, the system supports online editing of material in parallel with the uploading of the same material, accomplishing this by using thumbnail images representing the material, rather than requiring the presence of the actual material. And second, by autonomously uploading the video material and by applying the user's editing actions asynchronously after completion of upload, the system does not require the user to remain online after completing his or her editing actions. [0088] Many modifications and variations are possible in the light of the above teaching. For example, although the foregoing has been described with respect to its application to digital video material, the system and methods can be applied to other forms of digital media, including files of digital photographs, digital music and digital audio files. In the case of digital photographs, it can be seen that the system and methods described herein can be used to build a slideshow production by uploading a file of digital photographs and editing the photographs into a preferred sequence, removing unwanted items, and optionally adding an overlay of music or voice-over. In the case of digital music or audio, it can be seen that it can be used to insert deep tags at specific points in the music or audio, such that the users can later retrieve the specifically tagged section of the material, either for play back or for inclusion in multimedia productions.

[0089] In addition to editing video material, the online video editor 202 may be used to enable users to browse or preview video material in an online environment. In one example, the browsing and previewing function includes several variations on the use of thumbnail images, and the use of a virtual joystick to vary the replay speed of the video. In one example, users can select the browsing method that they find most effective in previewing video material presented to them. [0090] According to various examples, the online video editor 202 provides the following ways of representing video productions using thumbnail images: as a single thumbnail image taken from the beginning of the video production; as a single thumbnail image selected by the owner of the video production through an interface provided by the online video editor 202; as a quadrant of four thumbnail images taken the beginning of four equal sections of the video production; as a collection of thumbnail images taken from the start of each scene transition in the video production; as a collection of thumbnail images selected by the user through an interface provided by the online video editor 202; as a slideshow of thumbnail images taken from random points within the video production, where the owner of the video production specifies the number of points through an interface provided by the online video editor 202; or as a slideshow of thumbnail images taken at regular intervals within the video production, where the owner of the video production specifies the interval period through an interface provided by the online video editor 202.

[0091] In the latter four cases involving potentially large numbers of images, the system provides a means of representing the images in a visual hierarchy, through which the viewer can navigate in order to see further detail. In one example, the visual hierarchy is displayed in quadrant form, with the top level containing four images selected as equidistantly as possible across the entire video production. If the viewer clicks on one of the four images, the quadrant is replaced with four images selected as equidistantly as possible from the region represented by the clicked-on image. The user can click successively on individual images within quadrants until reaching the lowest level of the hierarchy, at which point the lowest- level images remain in place. The user can navigate back up the hierarchy by mechanisms such a right-clicking on the quadrant.

[0092] Figure 10 is a diagram illustrating an example process for browsing a video file. This process can be carried out by the online video editor 202 previously described with respect to Figure 2. At step 1000 a visual hierarchy is displayed in a quadrant form, the visual hierarchy including a plurality of images selected to be primarily equidistant across the video material. At step 1002, the user is provided with the ability to select one of the images and it is determined whether the user selected one of the images. If not, the process repeats until the user selects one of the images. [0093] When the user selects one of the images, a region is obtained at step

1004, the region being one that is represented by the selected one of the images. Thereafter, at step 1006 another visual hierarchy is displayed in a quadrant form including a plurality of images selected to be primarily equidistant across the region represented by the selected one of the images. The process then repeats at step 1002 wherein the user can continue to browse material by moving further down the hierarchy until such time as the user finds the material they are browsing for or reaches the lowest possible level of granularity.

[0094] According to another example, the online video editor 202 provides a method of varying the replay speed of a video production. In one example, the replay speed is adjusted by the viewer by means of a virtual joystick, which displays a speed dial ranging from very slow to very fast and allows the user to adjust the speed by using the mouse to move a virtual needle left or right from its central position, which represents normal speed.

[0095] Using the virtual joystick to replay a video production at high speed creates the effect of time-lapse photography, and provides a way for the viewer to browse the production in a short period of time, and to receive a visual summary of the content that may be more effective than thumbnails, due to its use of motion. Using the virtual joystick to replay a video production at low speed creates the effect of slow motion, and allows users to study sections of video to more accurately determine actions captured in them - actions that may have been missed when viewing at normal speed. For example, by replaying in slow motion a video of a bird flying, a viewer would be able to better study the ways in which the bird moves its wings.

[0096] A variation of the variable-speed replay method, which also applies to video material played at normal speed, is a process whereby the system partitions the video production into four equal-length segments, and plays the four segments in parallel in a quadrant format. This provides a faster means of browsing a video production in motion form.

[0097] Users of the online video editor 202 can select the browsing method that they find most effective in previewing video material presented to them. One means of selecting a browsing method is by right-clicking on the currently displayed representation, at which point a menu appears listing the available browsing options. By clicking on a browsing option, the user causes the system to switch to the appropriate representation.

[0098] In addition to editing or browsing video material, the online video platform 206 may be used to enable users to publish and forward video productions. In one example, an automated method provides an abstraction layer that shields the user from detailed concerns regarding the distribution of the video material. One automated publishing method comprises an interface whereby creators or owners of online video productions can select a production and a destination target, and then publish the production to an external location, such as an Internet site, with one click. [0099] Publishing may be accomplished by a three-step process whereby: (1) from a toolbar, users navigate through their video galleries to select the video they wish to publish; (2) users then select the distribution target via an automated address book; and then (3) users invoke the automated publishing process with one click. [00100] One automated forwarding method comprises an interface whereby viewers of a video production can select a destination target and forward the production to any destination with one click. Forwarding may be accomplished in a three-step process whereby: (1 ) the user clicks on a "Forward" button displayed with the viewed video, or available through a toolbar. In one example, video productions created by the online video editor are replayed with a TV-like encasement surrounding the video image, with several control buttons located below the image, one such control button being a button which, when clicked on, invokes forwarding of a viewed production; (2) the user selects a distribution target via an automated address book; and then (3) the user invokes the automated forwarding process with one click.

[00101] In the cases of both video publishing and video forwarding, the distribution targets may cover a variety of possible potential destinations, including websites, email recipients, Instant Messaging recipients, mobile phone users, software applications, digital set-top boxes and digital video recorders, or any combination of these. Users may pre-define destination groups, where each group may consist of any combination of possible destinations. Users may also set up any of the potential destinations or destination groups in their address books, and the system will automatically take care of all issues related to delivery of each video production to the requested destinations. Thus the one-click publishing and one- click forwarding methods enable users to automatically send their productions to multiple destinations with one click, without the need to enter individual destination targets repeatedly, each time they wish to publish or forward a video. [00102] The delivery system also includes a process whereby users can set up a prioritized list of possible delivery mechanisms for any destination (e.g., 1 = send to cell phone, 2 = notify by email, 3 = send to set-top box). In association with this, the delivery system incorporates presence detection mechanisms for target devices, whereby the current presence of the user at a device (e.g., a user active at his or her PC) is detected in real time, and the video is delivered via the most immediate channel. Thus, in the example, if the system detects that the destination user's cell phone and PC are switched off, but their TV is switched on, it will deliver the video to the destination user's set-top box. This in effect results in a "smart address book," where the system learns from its environment and takes intelligent delivery actions accordingly, rather than following a static link to a pre-determined destination. [00103] Figure 11 is a diagram illustrating an example process for automatically transcoding video materials to the appropriate format for a video-receiving destination device. In the illustrated example, possible delivery mechanisms for the destination are determined at step 1400. To this end, information about each destination device may be gathered and maintained by the system at step 1402, and may include the specific video format that each device requires and the highest priority destination device may be selected. Where this information is not available, the system may use the default format that most closely matches the device type. [00104] In order to transcode the subject video, the system may use a base decode codec in association with the encode codec required for the selected destination device at steps 1404 and 1406, and may create a copy of the subject material on the server in the destination format, prior to streaming it to the destination at step 1408. In one example, the base codec used in steps 1406 and 1408 may be the DivX codec. If the video material is not delivered successfully to a device, the system may provide a feedback mechanism whereby users are solicited to provide details about the device in question.

[00105] Distribution of video material can be accomplished both directly from an online video-sharing portal website, or indirectly from any website via a toolbar and associated browser plug-in. If a video is posted on another website (e.g., on a blogger's home page or on a Myspace user's home page, for example), the video material is not actually exported, but remains on the video-sharing website, which acts as a proxy server that retrieves and streams the video when requested. In order forward the video, the viewer interacts with the browser plug-in via the toolbar, which communicates with the portal to perform the actual forwarding. Thus the sharing controls established by the owner of the video material are still enforced, and all of the previously described delivery mechanisms still apply. [00106] The online video editor 202 also may support the construct of a "hyper- template" - a shareable definition of how a video production was created, that can be reused by others to help them create their own derivative works. Hyper-templates, therefore, are shareable versions of templates. A template defines the sequence of scenes that make up a video, and the related soundtrack, transitions, filters or special effects that are used in the production.

[00107] Figure 12 is a block diagram illustrating an example edit sequence. In the illustrated example, four video clips (a 1104, b 1106, c 1108, and d 1110) are combined into a video production 1100. In the example of Figure 12, the editing sequence occurs whereby first the individual clips are edited, then clips a 1104 and b 1106 are merged with sound added 1102, and then clips c 1108 and d 1110 are combined with the previously merged clips a and b to form the video production 1100.

[00108] Figure 13 is a block diagram illustrating example data structures that support hyper-templates. In the illustrated example, data structures 1200 include an edit tree table 1202, an edit dependencies table 1204, an edit command table 1206, a sequence table 1208, and a sequence composition map 1210. [00109] The sequence composition map 1210 provides pointers to the four video files (a 1104, b 1106, c 1108, and d 1110) previously described in Figure 12. The edit tree table 1202 identifies a sequence of six editing actions. The edit dependencies table 1204 defines dependencies between editing actions (e.g., editing action E must wait for completion of editing actions A and B). The sequence table 1208 identifies the sequence of editing actions and the root of the editing tree (where the Root Flag = "1"). The sequence composition map 1210 identifies the video clips that are used in each sequence step.

[00110] The online video editor 202 may be used to provide a growing library of community hyper-templates, based on the work of its members. When creating a video production, a user can either use one of the available hyper-templates that have been designated as "shareable," or create a video and its accompanying template from scratch. When creating a video from scratch, the user may drag and drop components from a palette of available video segments into a timeline that defines the sequence for the video production. The user also may drag and drop transitions between segments, and can optionally drag and drop special transitions on to individual segments. The user can also select still photos and add them into the timeline (e.g., from the Flickr website), and can select and add a soundtrack to the video production (e.g., from the Magnatune website). [00111] On completion of a video production, the creator has the option of defining whether the video is shareable with other users. In one example, the video can be shared at multiple levels: at the community level (by any person viewing the video), or at one or more levels within a group hierarchy (e.g., only by people identified as "family" within a "friends and family" group). The sharing hierarchy may be implemented as a system of folders within a directory structure, similar to the structure of a UNIX file system or a Windows file system, for example. Each member who creates video productions has such a directory, and a folder is created within the directory for each group or subgroup that the member defines. [00112] For each video production that the member creates, he or she has the ability to define which folders have the ability to view the video. When a member designates a person as belonging to a group, or when a person accepts a member's invitation to join a group, the person's ID is entered into the appropriate folder, and the person inherits the sharing privileges that are associated with the folder. [00113] The system also provides convenient mechanisms for creators of video productions to share their creation processes. On completion of a video production, for example, the user has the option of defining whether the hyper-template used in the production is shareable with other users, and whether the content of the video is also shareable in combination with the hyper-template. In one example, the hyper- template can be shared at multiple levels: at the community level (by any person viewing the video), or at one or more levels within a group hierarchy (e.g., only by people identified as "family" within a "friends and family" group). Sharing controls for hyper-templates and their content may be implemented using the same method outlined above, for sharing video productions.

[00114] In another example, the user can identify individual segments within the video that are shareable when reusing the hyper-template and which are not. In a further example, the user can identify which specific groups or subgroups of people can share specific video segments when reusing the hyper-template. [00115] The system provides two methods for selecting hyper-templates for reuse: browsing and hyper-linking. Using the first method, members of the video- sharing website browse among the set of hyper-templates designated as available to them for reuse. The hyper-templates may be organized in a variety of classification structures, similar to the structures by which the actual video productions are classified. These include but are not limited to classification schemes based on categories of videos (or "channels"), styles of video production, lengths of videos, tags or titles of videos, a grouping of favorite hyper-templates (based on popularity), and a set of hyper-templates recommended by the website, organized by category. [00116] The second method of selecting hyper-templates for reuse involves the use of hyperlinks, and, in particular, hypervideo links. Hyperlinks are a referencing device in hypertext documents. They are used widely on the World Wide Web to act as references that, when clicked on, link dynamically from one webpage to another. The hypervideo concept extends the use of the hyperlink device to provide a link out of a video production (rather than a text document) to another webpage, typically to another section of video.

[00117] The presently described system and methods use the hypervideo link as a method of transferring control out of a viewed video and into the online video editor 202, such that the viewer can use the template of the viewed video to create his or her own production. In this method, hyper-template linking is a special case of hypervideo linking, the special case being that the system always transfers control to the online video editor 202, rather than to a destination defined by the video-creator. Various implementation techniques exist to implement the special case of a hyper- template link, and to distinguish this from other hypervideo links (i.e., hotspots). [00118] In one example, video productions created by the online video editor 202 are replayed with a TV-like encasement surrounding the video image, with several control buttons located below the image, one such control button being a "Remix" button which, when clicked on, specifically invokes a hyper-template link into the online video editor. In another example, video productions created by the online video editor 202 are discretely watermarked with a small logo that appears in the lower left or right corner of the video, for example. At any time during a viewing of the video, the watermark acts as a hyper-template link, in the sense that, if clicked on, it triggers a hyperlink that takes the viewer seamlessly into the online video editor 202, with the hyper-template of the viewed video pre-loaded and ready to be reused in creating a new video production. This is achieved by structuring the hyperlink in

"hypertemplateidentifier" identifies the particular video that is being viewed and its hyper-template, and "websiteaddress" and "editor" identify the online editor to be linked to.

[00119] Since a watermark may also be used to identify a hypervideo hotspot, a hyper-template watermark may be distinguished in several possible ways, such as by having two separate watermarks placed in different areas of the video image, or, in the case of a shared watermark, by a passive appearance for a hyper-template hyperlink (as opposed to flashing, which indicates a hotspot), or by color-coding (e.g., blue indicates a hyper-template link, whereas red indicates a hotspot). [00120] A hyper-template hyperlink is initially generated by the online video editor 202 during construction of a video production, and is stored as metadata with the video. The data structures supporting the metadata were described earlier in this section, and shown in Figure 13. If the video is posted on another website (e.g., on a blogger's home page or on a Myspace user's home page), the hyperlink metadata remains associated with it. No matter where the video is viewed, on any website, it still retains the hyperlink that will link back to the original online editor if the hypervideo hyperlink is clicked on. This is because the video is never actually exported, but remains on the video-sharing website which acts as a proxy server that retrieves and streams the video when requested. The hyper-template thus not only provides users with a convenient way of sharing and reusing video creation processes, but also benefits the online video sharing website by generating traffic to the website and potentially enlisting new members.

[00121] Upon selecting a hyper-template via either of the methods described above, the user may be linked into the online video editor 202 and, in one example, is presented with a webpage showing the hyper-template of the selected video in the form of a timeline at the bottom of the screen, with the shareable segments of the related video displayed on the main palette in the center of the screen. In an alternative example, the timeline of the hyper-template is displayed vertically at the left or right side of the screen, with an additional vertical window alongside the timeline to allow insertion of text to be used as a commentary relating to the contents of the video timeline. The positioning of the text can be adjusted to appear alongside the particular video sequence that it relates to. The text can then serve as a teleprompter, and the commentary can then be recorded by the user in synchronization with the video sequence, as the video is played back in a separate window, and a marker moves down the timeline and its associated commentary. [00122] Upon selecting a hyper-template, users have a variety of choices regarding content that they may include in their new production. From the selected video, they can reuse any segments that the owner has designated as shareable. Users can also add or remove segments of video. They can select and include material from their own work-in-progress or their own galleries of completed productions, as well as from external sources that they have defined to be of interest and that the system has aggregated on their behalf, such as sources of photos, music, animation and other video content. Users can also change titles, credits and other text that may appear in the production, as well as any of the transitions, filters or special effects. Thus hyper-templates offer users a wide range of options regarding reuse of others' work, ranging from simple substitution of one or more video segments or other elements, to a major restructuring of the video production. [00123] Additionally, the online video editor 202 may provide a user interface that enables users of mobile devices on a network, such as cell phones to issue commands directly from their cell phones to accomplish simple editing of their video material, and to distribute the resulting edited video material to individuals or to predefined distribution groups. One example is implemented in a command line interface (the "mobile video editor") that supports all of the basic functions required to edit and distribute video material. In one example, the commands are entered on the cellular phone by the user in text form and are transmitted separately or in groups to the online editor using a short message service ("SMS") or a multimedia message service ("MMS").

[00124] SMS messages are typically available on digital global system for communications ("GSM") networks allowing text messages of up to 160 characters to be sent and received via the network operator's message center to the cell phone, or from the Internet, using a so-called "SMS gateway" website. If the phone is powered off or out of range, messages are stored in the network and are delivered at the next opportunity. MMS is a method of transmitting graphics, video clips, sound files, and text messages over wireless networks using the wireless application protocol ("WAP"). In the present example, the entire online video editing process may be accomplished using SMS or MMS messages, thereby obviating the need for any supporting application executing on the user's cell phone handset. [00125] In other examples, the user may interface with a Java-based application or a binary runtime environment for wireless ("BREW") based application residing on the cell phone handset, which then uses SMS, MMS, WAP, or some other interface to transmit the editing commands to the online editing service. In a further example, the mobile video-editing commands can also be input in command- line form from an Internet-connected PC.

[00126] Figure 14 is a diagram illustrating an example process for editing video material and distributing the edited video material using a mobile device, such as a cell phone. This process can be carried out by the online video editor 202 previously described with respect to Figure 2. At step 1300, the user sets up a work-in- progress folder to receive video clips from the cell phone, or from other sources available to the user (as used herein, the term "clips" refers to video material, audio, photographs, and other content that is useful for insertion into a project). The user may supply a name for the project, which is later used as the title for the video production. At step 1302, a project is created.

[00127] At step 1304, one or more video clips are added into the work-in- progress folder, typically from the user's cell phone input folder that contains clips that the user has just sent to the system. The system may maintain a cell phone input folder for each user who has requested the ability to use the mobile editor. In one example, the user may select a template (or "style") to be used in the video production. Templates have options to add enhancements to a production, including but not limited to: soundtracks, captions, transitions, filters and other special effects. A default template may be provided by the system.

[00128] At step 1306 the clips are combined and transformed, which may cause the editor to create a timeline/storyline for insertion of video clips, and to then insert clips into the timeline/storyline serially from the work-in-progress folder. The editor may apply a template to the production, using the last template that was selected by the user. If no template has ever been specified by the user, the system applies the default template. The command also may have an option to specify "No Template." [00129] At step 1308, the production is previewed. In one example, previewing the production includes replaying the combined set of video clips from the timeline, displaying the combined production on the user's cell phone, such that the user can preview the production before distributing it. At this point the user may optionally remove a clip from the production, for example, by specifying the sequence number of the clip within the production. Thereafter, at step 1310, the user sends the production. For example, the user may distribute the video production to the addressee of the command. The addressee may be the phone number or email address of an individual, or it may be a website, an Instant Messaging recipient, a software application, a digital set-top box or a digital video recorder, or it may be a pre-defined group consisting of any combination of these. By using the "group" function, the user avoids the need to individually enter multiple addressees. [00130] In other examples, additional functions may also be included in the mobile video editor command set. For example, the mobile video editor supports a library of templates that the user may choose from. Users may supply templates that they have created into the template library, thereby sharing their creative processes with others. The mobile video editor also supports a macro command whereby the user can create and distribute a video production by issuing just one command: "create production." The create production command references a previously created project (in a "using" clause), and causes the system to execute the set of commands that were previously entered for the referenced project. Prior to issuing the create production command, the user will have sent a set of clips to his or her input folder. By executing the commands from the referenced project, the editor will create a new production using the clips from the user's input folder, and send the production to the distribution defined in the referenced project. [00131] The mobile video editor also provides an API to its command set. The API can be used by developers of applications that reside in a cell phone handset, in order to incorporate online video editing into their feature set. This includes third- party application software providers and the cell phone handset manufacturers themselves.

[00132] In an alternative example, a new type of mobile video editor is created which is a WAP-enabled subset of the PC browser-based video-editing application. In this case, users with WAP-enabled cell phones can interface to the WAP-enabled video editor over the Internet, and are provided a simplified visual environment for editing their video material. The simplified interface compensates for the absence of mouse input for such functions as dragging and dropping, instead providing more automated forms of video production, using pre-defined templates that the user can select from the cell phone.

[00133] In one example, the online video editor 202 supports the construct of a hypervideo link - a means of allowing non-linear viewing of video material. Figure 15 is a diagram illustrating an example process for using a hypervideo link. In general, a hypervideo link allows the viewer to navigate among multiple possible viewing paths within or outside the video production he or she is currently viewing. When viewing any video created by the system, the user sees an unobtrusive mark in one area of the display. In one example, the mark is a rendered as a watermark, for example in the form of the logo of the video-sharing service or in a form selected by the video-creator, and all videos produced by the service bear such a watermark. [00134] At step 1500, it is determined whether a hypervideo link occurred in the video stream. When a temporal hypervideo link occurs in the video stream, the mark may become "active" at step 1502, making itself more noticeable to the viewer, by techniques such as glowing brighter or flashing, for example. [00135] At step 1504 it is determined whether the user selected the hypervideo mark. If the viewer does not click on the hypervideo mark, the process repeats at step 1500. When the user clicks on the active hypervideo mark at step 1504, he or she is given the option at step 1506 of switching out of the current video sequence and following one or more links to an alternative viewing destination. If the user does not switch out of the current video sequence at this step the process repeats at step 1500. Otherwise, at step 1508, the user proceeds to an alternate viewing destination.

[00136] Video targets of a hypervideo link may be within the viewed video production, or to any video material external to the production that has been tagged by the system. External material may include any material from other users that has been marked as reusable, or any material that has been aggregated by the system. [00137] Hypervideo marks may come and go during the playing of a video production. The length of time for which a mark is active on replay can be determined by the system (e.g., by a default value), or by the creator of the video production. When the viewer clicks on an active hypervideo mark, navigation options may be displayed in a menu form, listing one or more possible viewing destinations that are alternatives to continuing to view the production sequentially. If the user then clicks on one of the alternative viewing destinations, the system executes the hypervideo link associated with the destination description, thereby transferring control to the target webpage. The target webpage may be the entry into another video production, or to any tagged segment or section of a video production (all of which are examples of temporal links); alternatively, the target may be an Internet webpage or email message (both being examples of a textual link). [00138] In the case of a spatio-temporal hypervideo link, the viewer is able to click on or select a specific area on the screen where a particular activity is occurring, and thereby link out to a different section of video that pertains to the activity. The hotspot is thus not related to a mark on the screen, but to an area of the screen that makes itself noticeable to the viewer. Various techniques may be used to attract the attention of the viewer, such as temporarily brightening up the area of the hotspot, or temporarily zooming in on the area.

[00139] Rather than a section of video, the target of a spatio-temporal link may be an Internet webpage or email message (both being examples of a textual link). Thus a textual link may result from a temporal or a spatio-temporal opportunity. One special case of a textual link is a mouse over. In the case of a mouse over, clicking on a hypervideo link (temporal or spatio-temporal) results in a text-box appearing on the screen, providing commentary or information about the section of video that is currently being viewed. The text-box may appear on the screen outside the video viewing space, or it may appear in an area of the video viewing space (e.g., over a spatio-temporal hotspot area).

[00140] Various mechanisms are possible for returning control back to the original viewing point, after a hypervideo link has been executed. For temporal and spatio-temporal links, one such method is to return control at completion of the linked-to video segment, (i.e., when the first segment transition is detected in the linked-to video). In one example, each video segment is stored as a separate file, rendering straightforward the detection of the end of a video segment. An alternative return method is to return control on completion of the entire linked-to video production. A further method, which could be used in conjunction with the prior two, and is also applicable to textual links, is to provide a means for the user to initiate the return link, for example by clicking on a "Return" button that is always displayed by the system, and that is activated (e.g., by glowing brighter) on issuance of a hypervideo link.

[00141] A general return mechanism that applies to all forms of hypervideo links (textual, temporal and spatio-temporal) is for the system to superimpose or overlap the linked-to window over the linked-from window, or to show both windows beside each other, in all cases in such a manner that the user may at any time close the linked-to window and reactivate the linked-from window. Thus, in the case of a textual link to an email message, a user could compose a message within his or her email system and send it, then close the email window, and return to viewing the video.

[00142] In the case of temporal and spatio-temporal links, the target of a link is defined by the creator of the video production by referring to a tag. Tags identify whole productions, segments of productions, or (in the case of "deep tags") a point- in-time within a segment or a production. The online video editor 202 provides a convenient graphical interface for users to look up tags among their own material and among material designated as shareable by their creators. The system also syndicates publicly available video segments and makes them available with tags for videographers to include in their productions. In all cases, tagged material can either be easily embedded in the sequence of the production or easily set up as the target of a hypervideo link, using a drag-and-drop interface. Thus, through a combination of video editing, tagging, and hypervideo linking, the system implements a means of linking video material across the Internet, making this facility available to any consumer who wishes to work in the medium of video. [00143] The online video editor 202 also provides a convenient graphical interface enabling users to mark sections within their video material as hotspots carrying hypervideo links. The user can replay video material, either completed productions or work-in-progress, and stop the action at any point-in-time to define a hotspot. Figure 16 is a diagram illustrating an example process for defining a hotspot.

[00144] In the illustrated example, the user stops the action at step 1600 by clicking on a virtual "Pause" button located with other virtual controls below the replay window, for example. To set up the start point of the hotspot, the user then clicks on the mark on the video at step 1602 (which can be rendered as a watermark), and is provided a window providing various options for creating a hypervideo link. The options may include, for example, "Start Hotspot", End Hotspot", "Mark Spatial Hotspot", "Set Hotspot Duration", and "Select Hypervideo Destination". The user clicks on the "Start Hotspot" option at step 1604 to set up the temporal start of the hotspot. To define the duration of the hotspot by time (versus by a later point-in-time in the video), the user clicks on the "Set Hotspot Duration" option, and enters a time in seconds at step 1606.

[00145] At step 1608 it is determined whether the user wants to include a spatio-temporal hotspot. To identify a spatio-temporal hotspot, the user also clicks on the "Mark Spatial Hotspot" option, for example, and then uses an input device, such as a mouse, to outline the spatial area of the video to be associated with the hotspot (e.g., the upper righthand quadrant of the video replay window) at step 1610. [00146] By next clicking on the "Select Hypervideo Destination" option, the user can then select one or more destination targets from a list at step 1612 of system- supplied linkage options. These may include, for example, the user's set of available segment of deep tags (either within the current production or in other productions created by the user), a set of system-supplied tags to other video material, or a link to any Internet webpage or email message that the user then specifies. [00147] Having set up the start of the hotspot, if the user has not set up a time- based duration for the hotspot, he or she can then click on a virtual "Continue" button to continue playing the video at step 1614, and then at step 1616 is determined whether the user clicked on the "Pause" button to again stop the video and define the end point-in-time of the hotspot at step 1618. If the user does not define a time- based duration or an end point-in-time for the hotspot, the system automatically applies a user-definable default time for the duration of the hotspot (which, in one example, is initially set to ten seconds).

[00148] After marking a section of video material as a hotspot, the creator can associate multiple tags with the hotspot, each tag representing a possible choice that the viewer can later select in order to follow a hypervideo link to material outside the normal sequence of the video. Hypervideo links are also dynamic, in the sense that the creator can alter the targets of links at any time, even after publication. By re- entering the video editor, creators can change productions on the fly, changing the content both in terms of modifying the sequential material and inserting or modifying hypervideo links. This is achieved by deploying the two mechanisms of a proxy server and metadata. [00149] In one example, video productions created by the system are served dynamically by the system acting as a proxy server to the requesting service. Proxy servers cache frequently referenced material, thus improving performance for groups of users accessing similar content. When a video production is posted to another website (e.g., on a blogger's home page or on a Myspace user's home page), the video is may not be actually exported, but can remain on the video-sharing website which retrieves and streams the video when requested.

[00150] When hypervideo links are defined by users, the online video editor creates metadata pertaining to the link, including such information as the tag name and the URL address of the destination. The metadata is stored by the system and its association with the video production is maintained by the system. If the video is posted on another website, the hyperlink metadata remains associated with it. No matter where the video is viewed, on any website, it still retains all hyperlinks that have been defined for it.

[00151] When defining hypervideo links in their productions, users may include links to external video material that the system has previously aggregated. The system may have either already created a local copy of aggregated external material, or may have simply provided a link to the material. If the system has not previously stored a copy of the aggregated material locally, but has instead saved a link to the material together with the related commands for retrieving it, the system accesses the material via the API and creates copies of it in Flash and DivX formats, prior to making the material available to be referenced by hypervideo link in the user's production.

[00152] In order to accomplish this, the system first detects the format and resolution of the subject video material, then selects the appropriate decode software module to handle the detected video format, then decodes the video material from the input format using the selected decode codec, and then encodes it into Flash format using a Flash codec and into DivX format using a Divx codec. [00153] The online video editor 202 also handles uploading of video clips directly from a PC, or cell phone, without the need to use the preprocessing application 206. Figure 17 is a diagram illustrating an example process for direct uploading and editing. During the direct upload process, the online video editor 202 treats each video clip as a separate video segment, and creates a thumbnail image for each segment (based on the first non-blank image detected in the segment's data stream, for example). If the clip includes transitions, the editor detects these and splits the clip into separate segments, creating a new segment following each transition, and builds an accompanying thumbnail image for each created segment. For each segment, the editor prompts the user to supply one or more segment tags. After each segment has been uploaded, the user can review the segment and create additional deep tags defining specific points-in-time within the segment. [00154] When uploading video clips, users are provided with the ability to define a folder at step 1700, which is retrieved to receive a set of clips that they wish to associate together later in the editing process. Upon completion of the upload process, the folder will contain identification information (including tags) for each of the segments relating to the clip set. When users subsequently use the online video editor 202 to create a video production, by accessing a particular folder they retrieve the set of segments that they intended to use together, which are displayed as a set of segment thumbnails at step 1702. They can then drag and drop segment thumbnails into the editor's timeline at step 1704 to create a video sequence out of the segments they wish to include in their new production.

[00155] External content is provided for selection by tag at step 1706. The user is also provided with the ability to add transitions, special effects, as well music or voice overlays at steps 1708 and 1710 before saving the edited work as a new production at step 1712. The drag-and-drop interface provides an extremely simple method of video editing, and is designed to enable the average Internet user to easily edit his or her video material. The process of video editing is thus greatly simplified, by providing a single Internet-hosted source that automatically manages the processes of uploading, storing, organizing, editing, and subsequently sharing video material. The video-editing process is further simplified through the mechanism of hyper-templates, which allow users to reuse video-production processes and methods that they previously created, or that other users have created, or that the system supplies.

[00156] Since any new video production will have been constructed from separately defined segments, on completion it will inherently include segment tags for every separate clip included in the production, as well as for every scene transition. The new production will exist as a separate file, but the system also retains separate files for all of segments from which it is constructed. The segments can be rearranged in any manner, or combined in a variety of ways with other tagged segments, to create new productions with tags. These features in effect amount to automatic segment tagging of video material.

[00157] A further extension of the tagging concept is embodied in the ability to tag external content, such as photos, music or other external video material, and to include the tagged external content into a video production. Thus a video production can include a mixture of video segments and photos from multiple sources, plus a music overlay, and all segments, photos and music start points will be automatically tagged within the production.

[00158] The system may also automatically tag all digital content that it has aggregated on behalf of the user. Where a file name or title is supplied with a piece of aggregated material, this may be used as the tag. Where no file name or title is supplied, the system may create a tag in the form of: "Photo mm/dd/yy nnn", "Audio mm/dd/yy nnn", "Music mm/dd/yy nnn ", "Video mm/dd/yy nnn" or "Animation mm/dd/yy nnn", for example, where "mm/dd/yy" is the date when the spidering occurred, and "nnn" is a sequential number representing the sequence in which the piece of material was aggregated by the system on the date specified. The user can change any of the automatically aggregated material tags to a more meaningful tag name.

[00159] In a further variation of tagging, users can create entire video productions by aggregating together a set of tagged segments or sections of video from any source available within the system, including tagged material from external sources. It thus becomes extremely easy for users to create new video productions from existing material from multiple sources, without the need to introduce their own new material. Any such aggregated production will exist as a separate file, but the system also retains separate files for all of aggregated segments from which it is constructed.

[00160] A further extension of the tagging concept relates to the concept of hypervideo links. During viewing of a video, a hypervideo link makes its presence known by a visible change in the appearance of an area of the screen, or in the appearance of a watermark which is always present on the video. By clicking on the changed area or watermark, the viewer is given the option of switching out of the current video sequence and following one or more hypervideo links that may lead to another video, or to any tagged segment or section of a video, or to an internet webpage, or into an email message. [00161] When a user creates a hypervideo link in his or her video production, the online video editor prompts the user to supply one or more tags to be associated with the link. Hypervideo tags then become another form of segment tag, which viewers can subsequently search on, just as they can search on any other form of tag. By searching on a hypervideo tag, a viewer can gain access to any Internet- connected media source that has been referenced by a video creator. A video creator can also reuse a hypervideo link and include it in a new production, either by reusing it as a non-linear hypervideo link, or by retrieving the linked-to material and including it as one or more inline video segments.

[00162] The above description of the disclosed examples is provided to enable any person skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles described herein can be applied to other embodiments without departing from the spirit or scope of the invention. For example, references above to "clips" are not intended to be limited to video but are intended to encompass different types of digital media, including, for example, photographs, audio and multimedia. Thus, it is to be understood that the description and drawings presented herein represent a presently preferred embodiment of the invention and are therefore representative of the subject matter which is broadly contemplated by the present invention. It is further understood that the scope of the present invention fully encompasses other embodiments that may become obvious to those skilled in the art and that the scope of the present invention is accordingly limited by nothing other than the appended claims.

Claims

1. A method for editing video material comprising: segmenting the video material into a plurality of segments; compressing the plurality of segments into a plurality of upload segments; generating a thumbnail for each of the plurality of upload segments on the remote computing device; initiating an upload process on the upload segments to upload the upload segments from a local computing device to a remote computing device; displaying the thumbnails on the remote computing device as the upload process is occurring; performing one or more editing actions with the one or more thumbnails on the remote computing device as the upload process is occurring; saving the editing actions on the remote computing device as the upload process is occurring; changing an order in which the upload process uploads the upload segments based on the editing actions; and applying the editing actions to the video material on the remote computing device once the upload process has completed.

2. The method of claim 1 wherein the step of generating a thumbnail further comprises selecting a first non-blank image in one or more portions of the video material to form the one or more thumbnails.

3. The method of claim 1 further comprising: detecting scene transitions within the video material; segmenting the video material into separate files based upon the scene transitions; receiving an input of metadata tags; and associating the metadata tags with the separate files.

4. The method of claim 3, wherein the step of associating further comprises: associating a first tag to one or more of the separate files; and associating a second tag to one or more portions of the separate files.

5. The method of claim 1 wherein the step of displaying further comprises displaying a template in the form of a storyline.

6. The method of claim 5 wherein the step of performing one or more editing actions further comprises: receiving an insertion of one or more of the thumbnails into timeline; and receiving a removal of one or more of the thumbnails out of the timeline.

7. The method of claim 1 wherein the step of displaying further comprises: providing a library of special effects; and receiving one or more of the special effects for inclusion in the video material.

8. The method of claim 1 wherein the remote computing device is a server and wherein the step of initiating an upload process further comprises establishing a connection to the server.

9. The method of claim 1 further comprising publishing the video material to a previously defined gallery.

10. The method of claim 9 further comprising making the video material available to a subscriber of the previously defined gallery.

11. The method of claim 1 wherein the remote computing device includes a server and wherein the local computing device includes a client, the step of initiating further comprising: automatically segmenting the video material on the client; compressing the segmented video material on the client; establishing a connection between the client and the server; and uploading the segmented, compressed video material from the client to the server.

12. The method of claim 1 wherein the step of providing for a user to perform one or more editing actions further comprises: redirecting a user from an external website; recreating a graphical user interface ("GUI") associated with the external website.

13. The method of claim 1 wherein the step of performing one or more editing actions further comprises receiving a command via a short message service ("SMS"), a multimedia message service ("MMS"), or an instant message ("IM").

14. A method for browsing video material comprising: displaying a visual hierarchy in a quadrant form, the visual hierarchy including a plurality of images selected to be primarily equidistant across the video material; displaying a virtual joystick; and accessing the visual hierarchy or the virtual joystick.

15. The method of claim 14 wherein the step of displaying a visual hierarchy further comprises: receiving one of the images; obtaining a region represented by the selected one of the images; displaying a second visual hierarchy in a quadrant form, the second visual hierarchy including a plurality of images selected to be primarily equidistant across the region represented by the selected one of the images.

16. The method of claim 12 wherein the step of displaying a virtual joystick further comprises: displaying a speed dial including a range from relatively slow to relatively fast; adjusting a speed for playing back the video material or for sequencing through the video via the speed dial.

17. A method for viewing video material comprising: determining whether a hypervideo link occurs within a stream of the video material; making a mark associated with the hypervideo link active; determining if the mark was selected by a user; and causing a transition to an alternate viewing destination, if the mark was selected by the user.

18. The method of claim 17 wherein the step of causing further comprises: displaying a menu including a number of alternate viewing destinations; determining whether the user selected one of the number of alternate viewing destinations; transitioning to the one of the number of alternate viewing destinations, if the user has selected the one of the number of alternate viewing destinations.

19. The method of claim 17 further comprising forwarding the video material to a destination.

20. The method of claim 19 wherein the destination includes one or more of a website, an instant messaging recipient, a software application, a digital set-top box, or a digital video recorder.

21. A system comprising: a video platform which receives video material; a preprocessing application which receives the video material from the video platform, segments the video material into a plurality of segments, compresses the plurality of segments into a plurality of upload segments, and generates one or more thumbnails associated with the plurality of upload segments; an online video editor which displays the one or more thumbnails on a remote computing device as the upload process is occurring, the online video editor further configured to perform one or more editing actions with the one or more thumbnails on the remote computing device as the upload process is occurring and to apply the one or more editing actions to the video material on the remote computing device once the upload process has completed; and an upload module which transfers the plurality of upload segments from a local computing device to a remote computing device wherein an order in which the plurality of upload segments are handled by the upload module is changed based on the editing actions.

22. The system of claim 21 wherein the preprocessing application is configured to select a first non-blank image in one or more portions of the plurality of upload segments in order to generate the one or more thumbnails.

23. The system of claim 21 wherein the preprocessing application further comprises: a transition detection module which detects scene transitions within the video material; a video segmentation module which segments the video material into separate files; and a tag application module which applies metadata tags to the separate files.

24. The system of claim 23, wherein the tag application module further comprises: a first tag application module which applies descriptions to one or more of the separate files; and a second tag application module which applies descriptions to one or more portions of the separate files.

25. The system of claim 21 wherein the online video editor further comprises a template in the form of a timeline.

26. The system of claim 21 wherein the online video editor further comprises: a library of special effects, one or more of the special effects which are selected to be included in the video material as scene transitions.

27. The system of claim 21 wherein the preprocessing application further comprises: a video segmentation module which automatically segments the video material; a video compression module which compresses the segmented the video material; and a video upload module which uploads the segmented, compressed video material from the local computing device to the remote computing device.

28. The system of claim 21 wherein the online video editor further comprises a redirection and GUI recreation module which redirects a user from an external website and recreates a graphical user interface ("GUI") associated with the external website.