WO2000052551A2

WO2000052551A2 - Method and apparatus for communicating graphics data between a source and a recipient over a network

Info

Publication number: WO2000052551A2
Application number: PCT/US2000/003812
Authority: WO
Inventors: Sari Gilbert; Alexandre Barilov
Original assignee: Gomo Technologies, Inc.
Priority date: 1999-03-04
Filing date: 2000-02-14
Publication date: 2000-09-08
Also published as: WO2000052551A3; WO2000052551A9; AU3231700A

Abstract

A method for communicating graphics from a source to a recipient over a network (12) commencing with the construction of an electronic mail message. The electronic mail message is constructed to have graphics data embedded therein. The electronic mail message also has embedded therein a location identifier (e.g., a URL) that identifies a location on a server machine (10) from which a graphics application (24) is for displaying a graphics image derived from the graphics data. Following construction of the electronic mail message, the message is communicated from the source to a specified recipient over the communications network (12).

Description

METHOD AND APPARATUS FOR COMMUNICATING GRAPHICS DATA BETWEEN A SOURCE AND A RECIPIENT OVER A NETWORK

This application claims priority from provisional application serial no. 60/123_.016 filed March 4, 1999 and U.S. patent application serial no. 09. 315,899 filed May 20, 1999.

FIELD OF THE INVENTION

The present invention relates generally to the field of data communication and, more specifically to a method and apparatus for constructing and communicating a data structure representing graphics images between a source and recipient over a network.

BACKGROUND OF THE INVENTION

To date, e-mail communication has been primarily a text-based experience, in that viewers are typically limited to generating and communicating only original text-based information. Graphics images are typically communicated as file attachments to such text e-mails, the file attachments usually having been generated utilizing a dedicated graphics program.

With the advent of Hypertext Markup Language (HTML)-enabled e-mail client programs, users have been presented with the option of capturing images externally to the e-mail client, and then importing such images into an HTML-formatted e-mail document.

The process of creating graphics images, and communicating them across the Internet utilizing e-mail, currently tends to be a cumbersome and uncertain experience for both the sender and receiver. For example, where images are included within an HTML-formatted e-mail message in the manner described above (e.g., utilizing a cut-and-paste method), the incorporation of images into the e-mail document is a multi-step operation. Furthermore, such operations may be technically challenging for inexperienced computer users.

To address the absence of support for user-generated graphics within an e-mail context, two solutions have been proposed in the form of multi-media products available respectively from Media Synergy Incorporated and Softlink Incorporated. Media Synergy Incorporated has developed the Aloha^@Home product that allows a user to perform drag-and-drop operations to create a multi-media e-mail document. Softlink Incorporated has developed the PossaLink software product that similarly allows a screen capture of graphics

-l- images that may then be imported into the e-mail environment. Both of these programs are large, commercially-sold program that both the sender and recipient of an appropriately formatted image must purchase and manually install on respective computer systems in order to view the image. Further, the above discussed programs may be too large to be downloaded from a computer network, such as the Internet, on a need-to-use basis.

A number of electronic greeting card services are also accessible on the World Wide Web (WWW). For example, BlueMountainArts and 123Greetings provide a user with extensive on-line libraries of images and illustrations that a user may utilize to construct a personalized greeting card. However, to construct such a greeting card, a user must visit the web site of electronic greeting card companies to select and utilize the service. Furthermore, a recipient of an electronic greeting card is only able to view, and not edit or annotate, the received greeting card.

SUMMARY OF THE INVENTION

According to a first aspect of the present invention, there is provided a method of constructing a data structure for communicating graphics data between a source and a recipient. A graphics operation, performed on a machine executing a graphics application, is recorded as the graphics data. An electronic mail message is constructed to have embedded therein the graphics data and a location identifier for a graphics application that interprets and displays a graphics image corresponding to the graphics data, the location identifier identifying a location on a server from which the graphics application is retrievable by a client.

According to a second aspect of the present invention, there is provided a method of communicating graphics data between a source and a recipient over a network. An electronic mail message is constructed to have embedded therein first graphics data and a location identifier to identify a location on a server machine from which a graphics application, for displaying a first graphics image derived from the first graphics data is retrievable by a client machine. The electronic mail message is communicated from the source to the recipient over the network.

According to a third aspect of the present invention, there is provided a method of de-constructing a data structure for communicating graphics data between a source and a recipient. A location identifier is located within an electronic mail message received at a source from a recipient. The location identifier describes a location on a server machine from which a graphics application is retrievable by a client machine. The graphics application is automatically retrieved from the location on the server machine described by the location identifier. Graphics data, embedded from the electronic mail message, is located whereafter a graphics image, derived from the graphics data utilizing the graphics application, is displayed.

According to a fourth aspect of the present invention there is provided a method of communicating graphics data between a source and a recipient over a communications network. An electronic mail message is received at a recipient at a source. The electronic mail message is the parsed to identify a location descriptor and graphics data embedded therein. The location descriptor identifies a location on a server machine from which a graphics application is retrievable by a client machine. The graphics application is executable to display a graphics image, derived from the graphics data, on a display device.

Other features of the present invention will be apparent from the accompanying drawings and from the detailed description which follows.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention is illustrated by way of example and not limitation in the figures of the accompanying drawings, in which like references indicate similar elements and in which:

Figure 1 is a diagrammatic representation of an exemplary network environment within which the present invention may be implemented.

Figure 2 is a block diagram illustrating a page descriptor language document, according to an exemplary embodiment of the present invention, that may be constructed and communicated over a network according to the teachings of the present invention to provide multi-media content to a viewer of the document.

Figure 3 is a block diagram illustrating the architecture of a graphics application, according to an exemplary embodiment of the present invention, that is comprised by executable code residing on a server machine and that is referenced by a pointer within the document illustrated in Figure 2. Figure 4 is a flow chart illustrating a method, according to an exemplary embodiment of the present invention, of retrieving a template for a page descriptor language document from a server in a network environment.

Figure 5 is a representation of a web page, according to an exemplary embodiment of the present invention, that may be uploaded from a server machine to a client machine and the includes a number of hypertext-linked areas that are linked to various templates in the form of, for example, pre- formulated and populated HTML documents.

Figure 6 is a representation of a graphics environment, according to an exemplary embodiment of the present embodiment of the present invention¹, that may provided by a graphics application so as to enable a source user to perform various operations and actions to create a graphics image.

Figure 7 is a flow chart illustrating a method, according to an exemplary embodiment of the present invention, of providing a multi-media function with respect to an e-mail message when the multi-media data is embedded within the e-mail message.

Figure 8 is a method, according to an exemplary embodiment of the present invention, by which a source user may generate graphics data and communicate this graphics data as an embedded component of an HTML- formatted e-mail message to a target recipient.

Figure 9 is a block diagram illustrating an exemplary structure for graphics data that may be embedded within an HTML document.

Figure 10 is a block diagram illustrating details regarding the content of a drawing command, according to an exemplary embodiment of the present invention, that may be included within a sequence of drawing commands that comprises the graphics data.

Figure 11 is a block diagram providing a representation of a machine, in the exemplary form of a computer system, that may comprise a client machine, a server machine, or any one of a series of server machines within a web-based e-mail service. DETAILED DESCRIPTION

A method and apparatus for communicating graphics data between a source and a recipient over a network are described. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present invention. It will be evident, however, to one skilled in the art that the present invention may be practiced without these specific details.

For the purposes of the present invention, the term "network" shall be deemed to include, but not be limited to, a wire or wireless communications or a computer network that facilitates communications between devices coupled to the network. The term "network" shall furthermore be taken to include the Internet, an intranet, a Local Area Network (LAN), a Wide- Area Network (WAN), a Virtual Private Network (VPN), and a home network implemented utilizing infrared, radio frequency or telephone wire mediums.

Overview Modern e-mail clients, such as Microsoft Outlook Express developed by Microsoft Corporation of Redmond, Washington, or Netscape Messenger developed by Netscape Communications, Incorporated of Menlo Park, California, and many web-based e-mail services, for example, YahoolMail provided by Yahoo! Incorporated of Santa Clara, California, support HTML- formatted e-mail messages and the execution of "Java applets". The present invention specifies a format for sending e-mail messages with embedded multimedia data, and a pointer to an application (e.g., a Java applet) that is automatically downloaded over the network and that automatically executes within the context of an e-mail environment to provide a multi-media enhancement. For the purposes of the present invention, the term "e-mail environment" shall be taken to include an e-mail client application executing on a local client computer or a web-based e-mail service executing on a server machine that generates output, for example, to a browser client program executing on a local client machine.

Network Environment Figure 1 is a diagrammatic representation of a network environment 10 within which the present invention may be implemented. The network environment 10 includes a network in the form of the Internet 12 for facilitating communications between a client machine 14, a template and client code server machine 16 and a web-based e-mail service or mailing list 18. The client machine 14 is shown to host a number of applications, namely a browser 20, such as the Internet Explorer browser developed by Microsoft Corporation or the Navigator browser developed by Netscape Communications. The client machine 14 further hosts an e-mail client 22 and a graphics application 24, according to an exemplary embodiment of the present invention, which may be written in the Java program language developed by Sun Microsystems of Palo Alto, California. The graphics application 24 and e-mail client 22 are shown to access an e-mail mailbox 26, created and maintained by the e-mail client 22 for the purpose of storing e-mail messages.

The server machine 16 is shown to host a web server 30 that delivers both web content and applications to the client machine 14 over the internet 12. The web server 30 may include "servlets" that are able to communicate with the e-mail client 22 via an appropriate e-mail server (e.g., or Internet Service Provider (ISP) provided e-mail server). The web server 30 is also shown to access executable code 32 for the graphics application 24 that is uploadable from the server machine 16 to the client machine 14.

The web-based e-mail service 18, such as for example the Yahoo! Mail service, is typically implemented utilizing a large array of web-servers 34, that access a database 36 that maintains numerous mailboxes 38 for registered users of the e-mail service 18.

The server machine 16, and particularly the web server 30, is responsible for delivering HTML-formatted mail templates to the client machine 14 for display via the browser 20 for the purpose of initiating a media-enabled e-mail message, as will be described in further detail below. The server machine 16 is furthermore responsible for delivering the code 32 to the client machine 14 for execution thereon, and also for providing resources (e.g., graphics and audio resources) which may be utilized to include multi-media elements into an e- mail message, and for providing specific e-mail related services.

E-Mail Message With Embedded Application Pointer and Graphics Data The present invention proposes a method and apparatus for enabling multi-media features within an e-mail environment that supports e-mail messages formatted according to a platform-independent page descriptor language or Standard Generalized Markup Language (SGML), such as HTML or extensible Markup Language (XML). An SGML-formatted document includes tags that allow commands to be inserted into the document that specify how the document, or portion thereof, should be formatted or displayed, or that invoke specified functions or applications in support of presentation of the relevant document. As such, applications or functions may include multi-media functions, for example audio and visual functions.

Specifically, the present invention proposes the construction and communication over a network of a document, such as an e-mail message, that includes an identifier or pointer to a location accessible via the network from which a multi-media application may automatically be retrieved by an application or program that displays the relevant document. Furthermore, the present invention proposes embedding data within the document that is accessed by the retrieved application, and that is then utilized by the application to provide multi-media content to a user of a machine on which the application is executing.

Figure 2 is a block diagram illustrating an exemplary page descriptor language (e.g., HTML) document 40, according to an exemplary embodiment of the present invention, that may be constructed according to the teachings of the invention, communicated over a network, and utilized to provide multimedia content to a viewer of the HTML document 40. Specifically, the exemplary HTML document 40 is shown to include HTML page layout code 41 that includes a number of tags identifying, inter alia, content and specifications for the display of the relevant content. The document 40 further includes a pointer (or location identifier) that identifies executable code on a server machine 16 accessible via a network 12. In an exemplary embodiment, the pointer 42 is a Uniform Resource Locator (URL) that identifies a protocol to be utilized for retrieval of the executable code 32 and an Internet Protocol (IP) address or domain name at which the executable code 32 is located. The specified protocol may be the Hyper Text Transfer Protocol (HTTP) or the File Transfer Protocol (FTP). While the pointer 42 is illustrated above as comprising a URL encapsulated within an <applet code base> tag, it will be appreciated that any number of pointers that are able to identify an address at a remote server over a network may be utilized. The pointer may, for example, comprise any structure that is broadly identified as comprising a Uniform Resource Identifier (URI).

The executable code 32 to which the pointer 42 references is described below as being code that comprises a Java graphics application, such as the application 24 described above with reference to Figure 1. However, the present invention contemplates that the executable code 32 may be any code that provides multi-media functionality or enhancements to the display of the document 40. The executable code 32 may, in an alternative embodiment of the present invention, also be an ActiveX control, or any other platform- independent executable code.

The HTML document 40 further includes an encoded data portion 44 that, in the exemplary embodiment, comprises ASCII base64 encoded data 48 encapsulated in a <parameter name> tag that identifies a name 46 of the data 48. The data 48 is interpreted by the executable code 32 referenced by the pointer 42 to render the specific graphic. The encoded data portion 44 is accordingly embedded within the HTML document 40, integral therewith and does not comprise an attachment to the relevant document. Accordingly, a recipient of the HTML document 40 will not receive the encoded data 48 as an attachment, and the data 48 will be "invisible" to a viewer except as presented, for example, by a graphics application 24 embodied by the executable code 32 referenced by the pointer 42. The encoding of the data 48 as ASCII data is useful in that the line lengths for ASCII data are predetermined, and thus particularly suitable for embedding and transport within the context of an e- mail environment that utilizes the Simple Mail Transfer Protocol (SMTP).

In alternative embodiments, the data 48 may comprise other ASCII forms of serialized data, and it will be appreciated that base64 is merely one example of a useable format.

Software Architecture Figure 3 is a block diagram illustrating the architecture of a graphics application 24, according to an exemplary embodiment of the present invention, comprised by the executable code 32 residing on the server machine 16 and that is referenced by the pointer 42 within the document 40. As shown in Figure 1, the graphics application 24 may reside on a client machine, together with the e-mail client 22 and the mailbox 26 maintained by the e-mail client 22. The graphics application 24 is described with reference to Figure 2 as comprising a Java application (or applet) that is executed by a Java interpreter embodied within a Java virtual machine (JVM) 50. The Java virtual machine 50 is shown in the exemplary embodiment to be embedded within the browser 20. Further, the Java virtual machine 50 may comprise part of the operating system , such as for example Unix, Mac OS developed by Apple Computer, Incorporated of Cupertino, California or Windows 95/98 /NT operating system developed by Microsoft Corporation of Redmond, Washington. The Java virtual machine 50 implements an Application Program Interface (API) 52 to which the graphics application 24, in the role of a Java client, may make various calls as will be described in further detail below. The graphics application 24 furthermore has access to the HTML document 40 via the Java virtual machine 50.

The graphics application 24 includes a number of objects that may be implemented as Java classes. In an alternative embodiment of the present invention, these components may be implemented as ActiveX controls, Distributed Component Object Model (DCOM) objects, Object Linking and Embedding (OLE) objects, or any other type of objects.

The objects that populate the graphics application 24 include a loader object 54 that progressively loads components of the graphics application 24, displays a "splash screen" and status, checks the client environment for compatibility and shows appropriate warnings. An initializer object 56 is responsible for initializing and building reuseable resources. A message decoder object 58, as will be described in further detail below, is responsible for parsing HTML documents, extracting the encoded data 48 from the HTML document 40, and for decoding the data 48 from ASCII (or any other form of serial data) form to a binary internal format.

A viewer object 60 is responsible for rendering and showing graphics objects, images and documents on a display screen of, for example, the client machine 14. The graphics objects are derived from the binary data generated by the message decoder object 58. A composer object 62 provides a user of the client machine 14 with a graphical What-You-See-Is-What-You-Get (WYSIWYG) environment with a set of tools to create and edit graphical images to thereby create a new or modified HTML document 40.

A mail wizard object 64 implements a step-by-step process for assigning various e-mail attributes to an HTML document 40 within an e-mail environment. For example, the mail wizard 64 may assign addresses, format and date of delivery attributes to an HTML document to be delivered as an e- mail message.

A message coder /generator object 66 operates to create new HTML documents 40 to be delivered to a target recipient as an HTML-formatted e- mail message. Finally, a mailer object 68 operates to send a mail message, in the form of an HTML document 40, through an Internet mail gateway utilized by an e-mail client 22 and accessible from the client machine 14. The mailer object 68 furthermore tracks mail in progress and reports as in the mail delivery.

Further operation of the graphics application 24 shall be described below with reference to a number of flow charts. Methodology - Initial Generation of a Message Template

The present invention proposes to add multi-media functionality to e- mail messages propagated from a source to a recipient. More specifically, the present invention proposes embedding data that records multi-media -user activity associated with the e-mail message into a page descriptor language document (e.g., the HTML document 40) that comprises the e-mail message. By embedding data that records user activity and interaction with respect to the e-mail message, the data provides a "memory" or history of user activities with respect to the e-mail message. In this way, a particular e-mail message may be propagated between multiple users, each of whom may modify the data recording the multi-media interaction of successive users with respect to the relevant e-mail message. Also, an e-mail message may be propagated back and forth between a pair of users, each of whom may modify the data representative of the cumulative multi-media interaction with the e-mail message.

Where the multi-media activity represents the generation and modification of graphic images, for example, consecutive recipients of the e- mail message may edit, delete or modify graphic images associated with the e- mail message and represented by the embedded data, to provide a history of edits to the relevant e-mail message. For example, it is envisaged that a pair of users may play a "virtual" game of tic-tac-toe in which a first user may provide a mark within a graphics environment that is recorded within the data embedded within an e-mail message. The e-mail message may then be propagated to a second user, who may make a further mark in addition to the mark made by the first user. The second user may then propagate the e-mail message, with the embedded graphics data, back to the first user. The first user will then be presented with a composite graphics image derived from the embedded graphics data, that represents both the mark that the first user originally made and the mark that the second user has now added.

Figure 2 described the structure of an exemplary HTML document 40 that provides the structure for the transport of the multi-media data. While the HTML document 40 described in Figure 2 may be independently generated, it is envisiged that it would be most user friendly for an initiator of a multi-media e-mail message to be enabled to extract a "template" of the HTML document 40 having the required structure from a centralized web site on the Internet.

Figure 4 is a flow chart illustrating method 60, according to an exemplary embodiment of the present invention, of retrieving a template for the HTML document 40 from a server machine 16 in a network environment 10. At step 62, a user, utilizing a remote client machine 14 on which a browser 20 is executing, may access a web site provided by the web server 30 hosted on the server machine 16. At step 64, the user then uploads the web page (not shown) from the web server 30 to the browser 20 that includes links (e.g., URLs) to a number of "templates". Each template comprises an HTML document 40 having the structure described above with reference to Figure 2. Specifically, each of the "templates" may include a pointer 42 to a different executable code 32 that embodies a unique graphics application 24. Accordingly, the web server 30 may store a number of executable codes 32, each of which may embody a specific graphics application 24 that provides a different graphics environment within which to create graphic images. For example, the different graphics environments may provide different sets of graphics tools that are suited to particular applications. A first graphics application 24 may provide tools that are more suited to a personal communications environment, where as a second graphics application may provide tools that are more suited towards business or technical communication environments.

Each template, in the form of an HTML document 40, may also include data 48 that provides certain predefined graphics images and features for the graphics environment generated by the graphics application 24. For example, the data 48 may specify a particular background. Again, in a personal communications environment, a more family-oriented initial background may be represented by the data 48, whereas a more formal and business-like background may be provided for a template in which the pointer 42 references a graphics application 24 that generates a business graphics environment.

Figure 5 is a representation of an exemplary web page 67 that may be uploaded to the browser 20 and the client machine 14 at step 64. As illustrated, the web page 67 includes a number of hypertext-linked areas 69 - 74 that are linked to various templates in the form of pre-formatted and populated HTML documents 40. Each of the areas 69 - 74 is hypertext-linked to a HTML document 40 that includes a pointer 42 to an appropriate executable code 32 that provides tools appropriate to the character and requirements of a particular theme or environment.

Returning to Figure 4, following user selection of a template at step 64, the selected template and the executable code 32 identified by the pointer 42 of the template are uploaded from the server machine 16 to the client machine 14. The executable code 32 is then recognized as a graphics application 24 by the client machine 14, and executed, for example, utilizing the Java virtual machine 50 where the executable code 32 is a Java code, to provide a graphics environment.

At step 76, the user then composes a new graphics document utilizing the tools and environment provided by the graphics application 24. Specifically, the user performs a number of graphics operations and interactions that are recorded by the graphics application 24 as binary data, that is formatted according to an internal formatting specification.

Figure 6 is a representation of an exemplary graphics environment 78 provided by the graphics application 24 within which the source user may perform various operations and actions to create graphics images. Specifically, the graphics environment include an address area 80 within the source user may specify source and recipient e-mail address details, a text area 82 in which the source user may provide a textual message, a graphics area 84 in which graphic images may be created, a tools area 86 that may provide a number of graphics tools for the creation of graphic images within the graphics area 84 and tool bar areas 88 from which further functions may be invoked.

Returning to the flow chart shown in Figure 4, at step 90, the source user provides the e-mail address of a recipient in the address area 80 of the environment 78, and may optionally provide a textual message within the text area 82.

At step 92, the graphics application 24 contacts the server machine 16, and specifically the web server 30, and downloads the graphics data captured by the graphics application 24 to the server machine 16. At step 94, the mail server 28 composes an HTML-formatted e-mail message, for example in the form of an HTML document 40, that includes the data downloaded from the graphics application 24 at step 92 as the ASCII encoded data portion 44. The web server 30 propagates the composed e-mail to the recipient identified by the e-mail address entered by the source user into the address area 80 utilizing SMTP. The method 60 then ends at step 96.

Accordingly, a method by which an initial HTML-formatted e-mail message may be constructed to have a particular format and to include both a pointer 42 and data 48 has been described above with reference to Figure 4.

Methodology - Rendering of Graphic Images By The Graphics Application Further information regarding the rendering on a graphics image by the graphics application 24 utilizing the data embedded within an HTML-formatted message will now be described. Figure 7 is a flow chart illustrating a method 100, according to an exemplary embodiment of the present invention, of providing a multi-media function with respect to an e-mail message when the multi-media data is embedded within the e-mail message. The method 100 is described with reference to a graphics application. It will nonetheless be appreciated that the teachings of the present invention may be extended to audio, video and other multi-media applications.

The method 100 commences at step 102 with the receipt of an HTML- formatted e-mail message into the mailbox 26 of an addressed recipient. The e- mail message may have originated from the server machine 16 following the performance of the above described method 60 by a source user with respect to the server machine 16. Alternatively, the e-mail message may be received directly from the client machine 14 of a further user.

At step 104, the e-mail client 22 hosted on the client machine 14 accesses the received e-mail message now stored in the mailbox 26 for the purposes of displaying the message. To this end, the e-mail client 22 parses the HTML tags within the HTML document 40 that comprises the e-mail message. At step 106, the e-mail client 22 locates the <applet> tag that constitutes the pointer 42 to the executable code 32 stored by the server machine 16. The <applet> tag identifies the executable code 32 as a Java application to the e-mail client 22.

At decision box 108, a determination is made as to whether the e-mail client 22 supports the execution of Java applications. Early versions of many e- mail programs do not support the execution of Java applications, making this determination necessary. If the e-mail client 22 is determined not to support the execution of Java applications, a default message is displayed at step 110 on a display device (e.g., a Cathode Ray Tube (CTR)). The default message may advise the recipient user that the e-mail client 22 does not support the execution of Java applications, and include a URL that links to a copy of the e-mail message stored on the server machine 16. Utilizing the URL provided in the default message, a user may then optionally retrieve the HTML document 40 that comprised the message from the server machine 16 for display by the browser 20.

On the other hand, if the e-mail client 22 does support the execution of Java applications, the method 100 proceeds from decision box 108 to step 112 with the automatic download and launch of the graphics application 24. Specifically, at step 112 the e-mail client 22 automatically accesses the location on the server machine 16 identified within the <applet> tag, and retrieves the executable code 32 that is then interpreted by the Java virtual machine 50 to implement the Java graphics application 24. At step 114, the graphics application 24 makes a call to the Java virtual machine 50, and specifically to the API 52, that specifies a parameter name corresponding to the name 46 of a particular graphic represented by the encoded data portion 44. The parameter name included in a call corresponds to the name 46 of the encoded graphics data 48 identified by the <parameter name> tag, as shown in Figure 2.

At step 116, the Java virtual machine 50 parses the HTML document 40 to retrieve the graphics data 48 identified by the relevant parameter name communicated in the call at step 114. At step 118, the Java virtual machine 50 then returns the graphics data 48 to the graphics application 24.

As described above with reference to Figure 2, the graphics data 48 is ASCII encoded data and accordingly, at step 120, the message decoder object 58 of the graphics application 24 converts the ASCII data to an internal binary data format. At step 122, the viewer object 60 of the graphics application 24 renders a graphic image within the graphics area 84 of the graphics environment 78 utilizing the binary data generated at step 120. The method 10 then ends at step 124.

Methodology - User-Creation and Communication of Graphics Data Embedded in an E-mail Message with a Pointer to Execution Code Figure 8 is a method 130, according to an exemplary embodiment of the present invention, by which a source user may generate the graphics data 48 and communicate this graphics data 48 as an embedded component of an HTML-formatted e-mail message to a targeted recipient. The method 130 commences at step 132 with the detection by the composer object 62 of the graphics application 24, hosted on the client machine 14, of a drawing command inputted to the graphics application 24 by a source user. The detected drawing command is provided by the user utilizing, merely for example, any one of the graphics tools provided by the graphics application 24 within the graphics tools area 86 of the graphics environment 78 illustrated in Figure 6. For example, a user may select a particular drawing primitive (e.g., a square, oval, circle or other predefined graphic) provided within the tool bar, may select a particular color in which the drawing primitive is to be displayed, and then locate the drawing primitive within the graphics area 84 of the graphics environment 78.

At step 134, the composer object 62 of the graphics application 24 records the drawing command executed by the user as internally formatted binary graphics data. At step 136, the viewer object 60 of the graphics application 24 then renders a graphic image within the graphics area 84 of the graphics environment 78 utilizing the binary graphics data recorded at step 134.

Steps 132 - 136 are repeatedly performed until, at decision box 138, the source user elects to send the graphics image to a recipient. At step 140, the user provides the recipient's e-mail address in the address area 80 and a textual message within the text area 82 provided within the graphics environment 78.

At step 142, the message coder/ generator object 66 of the graphics application 24 then converts the internally formatted binary graphics data into the ASCII data 48. At step 144, the object 66 of the graphics application 24 then composes the HTML document 40 to include the ASCII base64 graphics data 48 within the data portion 44, and also to include a pointer 42 (also termed a link or a location identifier) that references the executable code 32 embodying the graphics application 24 on the server machine 16.

At step 146, the mailer object 68 of the graphics application 24 then propagates the HTML document 40 to the recipient identified by the e-mail address provided at step 140 as an e-mail message utilizing SMTP. The method 130 then terminates at step 148.

Graphics Data Format and Drawing Command Data Figure 9 is a block diagram illustrating an exemplary structure for the graphics data 48 that may be embedded within the HTML document 40 in the data portion 44 identified by the <parameter name> tag. The graphics data 48 is shown in Figure 9 to include a header 150 that provides information regarding the graphics application that is authorized to read the graphics data 48, data length information that specifies a length value for the graphics data 48, and a version number for the graphics data 48 so as to allow the graphics application 24 to identify the most current version thereof.

The graphics data 48 further includes a body 152 that constitutes a sequence of drawing commands recorded during various iterations of the steps 132 - 136 of method 130, described above with reference to Figure 8.

Figure 10 is a block diagram providing details regarding the content of a drawing command, according to an exemplary embodiment of the present invention, that may be included within the sequence of drawing commands within the body 152 of the graphics data 48. Specifically, the drawing command 160 may be the information recorded by the graphics application at step 134 of method 130 described above with reference to Figure 8. In an exemplary embodiment, each drawing command 160 comprises both type information 162 and attribute information 164. The type information 162 may identify the drawing command as being of a particular type, for example a free form stroke, an image, a sound, a text, or a drawing primitive type. The attribute information 164 may-identify various attributes pertaining to the drawing command, such as for example coordinates, time, color, tool (e.g., pen, brush or quill) and moveable or editable attributes.

Computer System

Figure 11 is a block diagram providing a representation of a machine, in an exemplary form of a computer system 170, that may comprise the client machine 14, the server machine 16, or any one of a series of server machines implementing a web-based e-mail service. A set of instructions, for causing the computer system 170 to perform any one of the methodologies discussed above, may be executed within the computer system 170. The computer system 170 includes a processor 172, a main memory 174 and a static memory 176 that communicate with each other via a bus 178. The computer system 170 is further shown to include a video display unit 180 (e.g., a liquid crystal display (LCD) or a CTR). The computer system 170 also includes an alpha-numeric input device 182 (e.g., a keyboard), a cursor control device 184 (e.g., a mouse), a disk drive unit 186, a signal generation device 188 (e.g., a speaker) and a network interface device. Graphic images and a graphics environment may be displayed on the video display unit 180 responsive to the various drawing commands. Drawing commands may be communicated to the graphics application 24 utilizing the alpha-numeric input device 182 or the cursor control device 184.

The disk drive 186 includes a machine-readable medium 192 on which is stored a set of instructions (i.e., software) 194 embodying any one or all of the methodologies for implementing the present invention. The software 194 may comprise the graphics application 24, the e-mail client 22, the browser 20, the executable code 32, the web server 30 or the mail server 28.

The software 194 is also shown to reside, completely or at least partially, within the main memory 174 and/or within the processor 172.

The software 194 may furthermore be transmitted and received by the network interface device 190.

For the purposes of the present specification, the term "machine- readable medium" shall be taken to include any medium that is capable of storing and coding a sequence of instructions for execution by a machine, that causes the machine to perform any one of the methodologies of the present invention. The term "machine-readable medium" shall accordingly be taken to include, but not be limited to, solid-state memories, optical and magnetic disks and carrier wave signals.

In summary, the present invention is advantageous in that it seeks to provide a convenient and user-friendly structure and methodology for adding multi-media functionality to e-mail messages propagated between users utilizing existing e-mail technology. The use of a platform-independent graphics application 24 is advantageous in that it allows the executable code 32 that is propagated from the server machine 16 to the client machine 14 to be relatively small and thus uploaded automatically, quickly and unobtrusively to the client machine 14.

Further, by embedding the graphics data 48 within the e-mail message itself (e.g., within the HTML document 40), the graphics data 48 may be conveniently stored and maintained within a mailbox 26 maintained on a client machine 14 by an e-mail client 22. Further, as the graphics data 48 is embedded within the e-mail message, user friendliness is enhanced in that the recipient does not perceive the graphics data as being an attachment to the e-mail message, but instead comprising an integral part of the e-mail message.

The storage of the graphics data 48 as embedded data of an e-mail message within a mailbox 26 is advantageous in that no separate file system or structure is required for storage or handling of the graphics data 48, and existing structures are leveraged for this purpose.

The inclusion of the pointer within the HTML document 40 is advantageous in that it allows the graphics application 24 to be automatically uploaded upon opening of an e-mail by the recipient. Accordingly, the recipient is spared the step of manually invoking the graphics application.

While the exemplary embodiment of the present invention discussed above has been described primarily within the context of a client machine 14, it will readily be appreciated that the teachings of the present invention may be implemented within a web-based e-mail service environment. In this case, the e-mail message in which the data graphics data 48 is embedded will be stored within a mailbox 38 maintained by a web-based server, and the user will access the relevant e-mail message utilizing a browser 20 executing on the client machine 14. In this case, the web-based e-mail service 18 will be responsible for initiating the uploading of the graphics application 24 to the client machine 14. In an alternative embodiment, the graphics application 24 may execute on a machine provided by the web-based e-mail service, and interface with the client machine 14 utilizing the browser 20.

Thus, a method and apparatus for communicating graphics data between a source and a recipient over a network have been described. Although the present invention has been described with reference to a specific exemplary embodiment, it will evident that various modifications and changes may be made to these embodiments without departing from the broader spirit and scope of the invention. Accordingly, the specification and drawings are to be regarded in an illustrative rather than a restrictive sense.

Claims

CLAIMSWhat is claimed is:

1. A method of constructing a data structure for communicating graphics data between a source and a recipient, the method including:

recording a graphics operation, performed by the source utilizing a graphics application, as the graphics data; and

constructing an electronic mail message as a markup language document having embedded therein graphics data and a location identifier for the graphics application that displays to the recipient a graphics image corresponding to the graphics data and that records a further graphics operation performed by the receiver to modify the graphics data, the location identifier identifying a location on a server from which the graphics application is retrievable by a client.

2. The method of claim 1 wherein the graphics application comprises a platform-independent executable code.

3. The method of claim 2 wherein the executable code is a Java applet or an ActiveX control.

4. The method of claim 1 wherein the recording of the graphics operation comprises recording user-inputted information pertaining to the graphics image.

5. The method of claim 4 wherein the recording of the graphics operation comprises recording a drawing command.

6. The method of claim 5 wherein the recording of the graphics operation includes recording type information concerning the drawing command.

7. The method of claim 6 wherein the type information includes information selected from the group consisting of free-form stroke, image, sound, text and drawing primitive information.

8. The method of claim 6 wherein the recording of the graphics information includes recording attribute information concerning the drawing command.

9. The method of claim 8 wherein the attribute information includes coordinate, time, color, drawing tool, and edit information concerning a graphics image produced responsive to the drawing command.

10. The method of claim 1 including modifying the graphics data to record a graphics operation performed on a further machine executing the graphics application.

11. The method of claim 1 wherein the electronic mail message comprises a page descriptor language document.

12. The method of claim 11 wherein the markup language document comprises a Hypertext Markup Language (HTML) document.

13. The method of claim 1 wherein the location identifier comprises a hypertext link to the graphics application on the server, and wherein the graphics application is automatically downloaded from the server responsive to an access operation with respect to the electronic mail message by the recipient.

14. The method of claim 1 wherein the construction of the electronic mail message includes converting the graphics data from a binary format to an ASCII format.

15. The method of claim 1 wherein the construction of the electronic mail message includes embedding the graphics data within a markup language document between a pair of parameter markup language tags.

16. The method of claim 1 including storing the constructed electronic mail message within an electronic mail box maintained by an electronic mail application program resident on the client.

17. The method of claim 1 wherein the source comprises a client machine on which an electronic mail application program is resident.

18. The method of claim 1 wherein the source comprises a server machine to which the graphics data is communicated from a client machine on which an electronic mail application is resident.

19. The method of claim 1 wherein both the recording of the graphics operation and the construction of the electronic mail message are performed on a client machine on which an electronic mail application is resident.

20. The method of claim 1 wherein the recording of the graphics operation is performed on a client machine, and the construction of the electronic mail message is performed on a server machine coupled to the client machine via a network.

21. A method of communicating graphics data between a source and a recipient over a network, the method including:

constructing an electronic mail message as a markup language document having embedded therein first graphics data at least partially generated by the source and a location identifier identifying a location on a server machine from which a graphics application, to display to the receiver a first graphics image derived from the first graphics data and to facilitate modification of the graphics data by the receiver, is retrievable by a client machine; and

communicating the electronic mail message from the source to the recipient over the network.

22. The method of claim 21 including automatically retrieving the graphics application from the location on the server machine in response to a user action with respect to the electronic mail message by the recipient.

23. The method of claim 21 including storing the electronic mail message, inclusive of the first graphics data, within an electronic mail storage structure maintained by an electronic mail application resident on a first client machine of the recipient.

24. The method of claim 21 wherein the construction of the electronic mail message includes combining the first graphics data with second graphics data, wherein a second graphics image generated prior to the first graphics image is derivable by the graphics application from the second graphics data.

25. The method of claim 24 wherein the second graphics data is communicated to the source within an earlier electronic mail message for combination with the first graphics data within the electronic mail message.

26. The method of claim 21 wherein the construction of the electronic mail message includes recording a graphics operation, performed on a client machine of the source that executes the graphics application, as the first graphics data.

27. The method of claim 26 wherein the recording of the graphics operation comprises recording user-input information pertaining to the first graphics image.

28. The method of claim 26 wherein the recording of the graphics operation comprises recording a sequence of drawing commands.

29. The method of claim 28 wherein the recording of the graphics operation includes recording type information concerning a drawing command.

30. The method of claim 29 wherein the type information includes information selected from a group consisting of free-form stroke, image, sound, text and drawing primitive information.

31. The method of claim 28 wherein the recording of the graphics information includes recording attribute information concerning the drawing command.

32. The method of claim 31 wherein the attribute information includes information selected from a group consisting of coordinate, time, color, drawing tool, and edit information concerning a graphics image produced responsive to the drawing command.

33. The method of claim 21 wherein the electronic mail message comprises a markup language document.

34. The method of claim 33 wherein the markup language document comprises a Hypertext Markup Language (HTML) document.

35. The method of claim 21 wherein the location identifier comprises a hypertext link to the graphics application on the server.

36. The method of claim 21 wherein the graphics application comprises a Java applet or an ActiveX control that is downloadable from the server machine to the client machine of the recipient.

37. The method of claim 21 wherein the construction of the electronic mail message includes converting the graphics data from a binary format to an ASCII format.

38. The method of claim 21 wherein the construction of the electronic mail message includes embedding the graphics data within a markup language document between a pair of parameter markup language tags.

39. The method of claim 21 wherein the source comprises a client machine on which an electronic mail application program is resident.

40. The method of claim 21 wherein the source comprises a server machine to which the graphics data is communicated from a client machine of the source.

41. The method of claim 21 wherein the construction of the electronic mail message is performed on a client machine on which an electronic mail application is resident.

42. The method of claim 21 wherein the construction of the electronic mail message is performed on the server machine coupled to the client machine of the source via the network.

43. The method of claim 21 wherein the communication of the electronic mail message from the source to the recipient over the communications network is performed utilizing the Simple Mail Transport Protocol (SMTP).

44. A system for constructing a data structure for communicating graphics data between a source and a recipient, the system including:

a composer component to record a graphics operation, performed by the source utilizing a graphics application, as the graphics data; and

a message generator component to construct an electronic mail message as a markup language document to have embedded therein the graphics data and a location identifier for the graphics application that displays a graphics image corresponding to the graphics data and that records a further_graphics operation performed by the receiver to modify the graphics, data, the location identifier identifying a location on a server from which the graphics application is retrievable by a client.

45. A system for communicating graphics data between a source and a recipient over a network, the system including:

a composer component to construct an electronic mail message as a markup language document having embedded.therein first graphics data at least partially generated by the source and a location identifier identifying a location on a server machine from which a graphics application, to display to the receiver a first graphics image derived from the first graphics data and to.f acilitate modification of the graphics data by the receiver, is retrievable by a client machine; and

a mailer component to communicate the electronic mail message from the source to the recipient over the communications network.

46. A method of de-constructing a data structure for communicating graphics data between a source and a recipient, the method including:

locating a location identifier within an electronic mail message received at the source from the recipient, the location identifier describing a location on a server machine from which a graphics application is retrievable by a client machine;

automatically retrieving the graphics application from the location on the server machine described by the location identifier;

locating graphics data embedded within the electronic mail message; and

displaying to the recipient a graphics image derived from the graphics data utilizing the graphics application ; and

recording a modification by the recipient_, to the_.graphics data.utilizing the graphics application.

47. A method of communicating graphics data between a source and a recipient over a communications network, the method including:

receiving an electronic mail message, in the form of a markup.language document, at the recipient from the source; and

parsing the electronic mail message to identify a location descriptor and graphics data, at least partially generated by the source utilizing_a graphics application, embedded within the electronic mail message, the location descriptor identifying a location on a server machine from which the graphics application is retrievable by a client machine, the graphics application being executable to display a graphics image, derived from the graphics data, on a display device, and to facilitate modification of the graphics data by the recipient.

48. A system for de-constructing a data structure for communicating graphics data between a source and a recipient, the system including:

a message decoder to locate a location identifier embedded within an electronic mail message, in the form of a markup language document, received at the source from the recipient, the location identifier describing a location on a server machine from which a graphics application is retrievable by a client machine, to retrieve the graphics application from the location on the server machine described by the location identifier, and to locate graphics data, at least partially generated by the source utilizing the graphics application, embedded within the electronic mail message; and a viewer to display to the receiver a graphics image derived from the graphics data utilizing the graphics application and to facilitate modification of the graphics data by the receiver utilizing the graphics application.

49. A system for communicating graphics data between a source and a recipient over a communications network, the system including:

a mailer component to receive an electronic mail message, in the form. of a markup language document, at the recipient from the source; and

a message decoder to parse the electronic mail message to identify a location descriptor and graphics data,, at least.partially generated by _.the source utilizing a graphics application, embedded within the electronic mail message, the location descriptor identifying a location on a server machine from which the graphics application is retrievable by a client machine, the graphics application being executable to display a graphics image, derived from the graphics data, on a display device and to facilitate modification of the graphics data by the receiver.