WO2003085927A1

WO2003085927A1 - Device and method for in-flight data compression

Info

Publication number: WO2003085927A1
Application number: PCT/FR2003/001074
Authority: WO
Inventors: Karel Mittig; Eric Maschio-Esposito; Amar Nezzari
Original assignee: France Telecom
Priority date: 2002-04-05
Filing date: 2003-04-04
Publication date: 2003-10-16
Also published as: EP1495619A1; AU2003260725A1; FR2838268A1

Abstract

The invention concerns an in-flight method for compressing data supplied by a server (40) of a collective network such as Internet in response to a request from a consulting terminal (30), which consists in: providing a data compression module (20), and a redirection module (10) designed to receive data supplied by the server (40) and redirect them towards the compression module (20), said redirection module (10) being also designed to receive data from the compression module (20) and redirect them towards the consulting terminal (30). The method further consists in: examining in the request an indication concerning decompression aptitude of the terminal (30) and performing in the compression module (20), on the data requested by the request and supplied by the server (40) a compression adapted to the indications concerning decompression aptitude contained in the request.

Description

DEVICE AND METHOD FOR COMPRESSING DATA ON THE FLIGHT

The invention relates to on-the-fly data compression tools used on networks, and in particular on the Internet. These tools aim to optimize the flow or “bandwidth” and thus increase the speed of information display at the level of the Internet customer.

For the customer, these tools allow a reduction in the loading time of content, that is to say a better perception of the quality of service (QoS) and also a reduction in costs when the connection is billed by volume.

To date, there are several solutions for accelerating flows on the Internet, called Booster Web, which respond to this problem, but none correspond to a real offer at the heart of the network, which can be easily applied to different types of users. .

Boosters with clients require the installation of a specific module on the client's workstation. This local module on plug'in will actually modify the structure of the messages exchanged between the client and the booster. Communication using the usual HTTP (Hypertext Transport Protocol) protocol is replaced by a proprietary protocol.

“TCP” boosters modify the structure of messages, others are content to modify the logic for resuming TCP sending by adapting it to the response time of the network such as mobile phones for example.

Clientless boosters do not modify http port 80 messages, they use standard headers. But it is then essential to declare them in the properties of browsers as we would for a classic cache. In addition, some boosters also have a "cache" function and others claim to comply with the ICP protocol (communication protocol between caches). These solutions therefore require intervention on the software installed on the user's computer and are therefore far from being transparent to the user. It is proposed according to the invention to resolve this drawback, that is to say to propose a tool and a method of compressing data on the fly which is transparent to the Internet user, that is to say which does not require intervention on the computer or the local network thereof. This object is achieved according to the invention thanks to a method of on-the-fly compression of data supplied by a server of a collective network such as the Internet in response to a request from a consultation terminal, in which method a data compression module, and a redirection module provided for receiving data supplied by the server and redirecting them to the compression module, this redirection module also being provided for receiving data from the compression module and redirecting them towards the consultation terminal, method according to which an indication concerning an aptitude for decompression of the terminal is examined in the request and a compression is performed in the compression module, on the data requested by the request and supplied by the server, which is adapted to the indications of suitability contained in the request.

A device for compressing data on the fly in a collective network such as the Internet is also proposed according to the invention, comprising a compression module and a redirection module, this redirection module being designed to receive data supplied by a server in response to a consultation request sent by a consultation terminal, and redirect this data to the compression module, this redirection module also being intended to receive data supplied back by the compression module and redirect them to the terminal of consultation, this device further including means for examining, in a request for consultation of data sent by a terminal, an indication concerning a decompression ability of the terminal considered, and means for controlling compression by the compression module which is adapted to the indication of capacity for decompression contained in the request. Other characteristics, aims and advantages of the invention will appear better on reading the detailed description which follows, made with reference to the appended figures in which:

- Figure 1 is a simplified diagram of a data compression assembly according to the invention;

- Figure 2 is another simplified diagram of an assembly according to the invention;

- Figure 3 shows in detail this same assembly according to the invention; - Figure 4 shows in detail an iCAP server according to a variant of the invention;

- Figure 5 is a flowchart representative of a method according to a variant of the invention;

- Figure 6 is a flowchart of data and instructions according to a variant of the invention.

The device described below uses a data exchange protocol intended to allow modifications of content on the Internet, a protocol called iCAP - Internet Content Adaptation Protocol -.

The device of FIG. 1 is architecture around two modules 10 and 20 of the iCAP type, namely a cache 10 of the client iCAP type and a content modification server 20, of the iCAP server type to

“Response modification” according to the terminology set out in the iCAP protocol.

When a user requests, via browser software running on his personal computer 30, to access a Web page, the following steps are implemented in the device of FIG. 1.

First an HTTP request is sent to the cache 10, also called “proxy cache 10”, configured in the browser of the user 30 (but it could just as easily be a proxy proxy operating in transparent mode in the context of an ISP - Internet Service Provider -).

The cache 10 then sends an HTTP request to an original web server 40, containing this page. The web server 40 returns the requested page to cache 10.

Cache 10 now acting as an iCAP client redirects this response to the aforementioned iCAP server 20, passing it the full page. The iCAP server 20 then processes this request by calling on a content modification service integrated into the server, then responds to cache 10 by returning the result of the processing: the modified page (or not as will be seen below).

Finally, the cache 10 returns the response from the iCAP server 10 to the end user 30 and also stores this response.

The scenario that we describe here takes place when the cache 10 does not already have the requested document in memory, otherwise it immediately responds to the user by returning the page that he previously stored (i.e. the original page, i.e. the modified page if the modification of the latter proved to be appropriate).

The cache 10 of the “iCAP client” type is currently available on the market in an industrial version.

The principle of the iCAP protocol rests on a simple basis: to each HTTP request corresponds a response coming from a Web server or from a cache: it is therefore enough to transform either the request or the response to offer a content transformation service . iCAP offers two main modes (“Request Modification” and “Response Modification”) which are based on the HTTP protocol.

The role of iCAP is not to perform the content transformation itself. It is just limited to connecting "iCAP clients" (web caches, routers, etc.) with "iCAP servers", the latter being responsible for the transformation.

The iCAP forum (http://www.i-cap.org) was created in December 1999 under the leadership of the companies Network Appliance and Akamai. Its objective is to standardize, through the iCAP protocol (Internet Content Adaptation, Protocol), the exchanges between equipment representative of Web architectures (Web servers, caches, load balancers, routers ...) and content transformation systems . The ICAP 1.25 specification is the most recent. It has already been the subject of several implementations in the context of value-added file transformation services, here are some examples:

- virus filtering - content filtering

The iCAP protocol was also submitted on November 27, 2000 to the IETF (Internet-Draft "iCAP the internet content adaptation protocol" http: //www.ietf.orq/internet-drafts/draft-elson-opes-icap-00 .txt) in the Open Proxy Extensible Services Working Group. There are not many technical implementations of services using the iCAP protocol. Only Anti-virus software publishers benefiting from their expertise in network solutions installing on FireWall (curtain of fire) or proxies (proxy) offer a port of their services on iCAP. We are also seeing the appearance of new commercial players in the field of content filtering.

But for all the other applications that may be envisaged, the initial investment remains heavy and costly. This further explains why value-added services using iCAP are slow to come out. Using the ICAP protocol here and in particular the fact of recovering flows originating from the WEB servers via an iCAP redirection module, here constituted by the cache 10, a particularly advantageous on-the-fly compression method is implemented.

The operation of this iCAP compression tool, or "ICAP Zipper", can be represented simply by the generic diagram in Figure 2.

The main steps of the compression service are as follows: a) the client 30 sends a request to a Web server 40. This request possibly passes through redirection equipment, here the cache 10. b) the server 40 transmits its response to client 30 via cache 10. c) the redirection service 10 (iCAP client) redirects the response to the iCAP server 20. This server will be referred to below by the more general expression “compression module 20” in order to qualify the main function of this server. d) the response from the server 40 is if possible compressed and / or optimized by the compression module 20. e) the compression module 20 returns the request to the redirection module 10. f) the redirection module 10 returns the response to the client 30. g) the client browser 30 automatically decompresses the document received before displaying it.

Automatic decompression of client-side data is a feature defined in the HTTP 1.1 protocol (Standard IETF defined by RFC 2616). This functionality is thus found in the majority of current browsers which integrate a gzip type decompression mechanism (Internet Explorer - registered trademark - of Microsoft, version 4 and higher, Netscape Communicator (registered trademark) of Netscape, version 4 and higher).

This compression system (without a specific client or local module - or plug-in -) offers the advantage of not intervening on the software installed on the user's computer 30 (or on the mobile phone or any other consultation terminal used).

This compression service also does not modify the network and protocol layers, which avoids possible incompatibilities with other types of services or equipment that would be located between the compression service and the end user.

Figures 3 and 4 show in more detail the preferred device described here. In these figures, the compression module 20 (or iCAP content transformation module) is broken down into an administrator module 22, a flow adapter module 23, and a service module or sub-module 25 dedicated specifically to compression tasks. Datas. The function of the flow adapter 23 is to adapt the content between the redirection module (iCAP client 10) and the service module 25.

Such a stream adapter 23 is preferably provided to adapt the data streams also for a series of other additional service modules, implementing services other than compression, these modules can for example be content filtering modules or anti-virus filtering modules.

The adapter 23 is located in the compression module 20, on the one hand, in the client 10 - server 20 direction, adapt the files so that they can be directly used for the file transformation services, and on the other hand, in the server 20 - client 10 direction, ensure the integrity of the transformed file and possibly its reconstruction.

The adapter 23 makes it possible to formalize the exchanges between the redirection module 10 (iCAP client) and the service module 25 (file transformation service) by adapting the iCAP streams to make them compatible with the compression service module. 25, as well as with other modules corresponding to file transformation services, without it being necessary to develop an iCAP layer at the level of these services. This then makes it possible to define file transformation services, modular, scalable and independent of the iCAP layers.

Thus, we recover the flows coming from Web servers via an iCAP content adapter and then convert them to Zip format, which is a usual decompression format. It is clearly seen in FIG. 4 that the service module 25 is structurally integrated into the iCAP layer, which means that the service module 25, that is to say the part specific to the compression service, is here very simple to produce , in particular by adopting the functional architecture of an iCAP flow adapter as previously described. The general service or process described here will therefore make it possible to compress http content, for example objects encoded in htm, html format, to optimize transfer rates, this at the heart of the network and in a transparent manner for the client. To implement this service, a detection of the capabilities of the browser 30 to decode compressed content is implemented.

Knowledge of these properties particularly differentiates the present iCAP compressor (or the iCAP Zipper) from other Booster systems, which are only valid for a given type of client or a particular network configuration.

The present compression set is therefore capable of dynamically taking into account the particularities of customers and thus of adapting compression on the fly. When sending an http request to a web server 40, the browser 30 announces, in the header of this request, all the characteristics qualifying this browser.

The compression module 20 (iCAP server) will then test the existence of an "Accept-encoding" line in the http request. If this line does not exist, this means that the browser 30 does not accept any compressed page. The compression module 20 must then return the pages without modifying them.

If this line exists, the server 20 must look at the value of the types of compressed elements which are accepted by browsers to choose a good encoder or submodule of compression.

The service module 25 then modifies the content of the header to be returned to the client 30, compresses the data, and returns it to the browser 30 via the redirection module 10.

The service module 25 specifically carrying out the compression of content, therefore operates here natively in “response modification” mode.

The compression procedure executed in this overall device, implemented according to a “Data Analysis” method of the class CicapServiceZip, is as follows, here detailed according to a series of steps A) to I) also represented on the diagram Figure 6.

The procedure described here is carried out using a compression module 20 conforming to the architecture shown in FIG. 4, but comprising several service modules 25 integrated in the same server iCAP 20, these different service modules 25 each corresponding to a compression of a different type of object.

We ignore here the steps relating to the preparation of the various objects passed to the compression module 20. It is simply specified that the header of the consultation request initially sent by the terminal 30 is seen transmitted to the compression module 20, by the module 10, associated with the data provided by the server 40 in response to such a request.

The objects passed are, more generally, objects corresponding to the various http headers relating to the client 10, to the server 20, and to the content supplied.

A) The compression module 20 searches in the request-client header for the string "Accept Encoding", making it possible to know whether the content can be compressed. If the value is not equal to "gzip" or "deflate" or if the chain does not exist, no compression is performed (go to point I below).

B) The compression module 20 searches in this same header for a string “User-Agent” making it possible to know the minimum version of browser used by the client. If the value extracted is not at least equal to 4 (major version number of browsers), no compression is performed (go to point I).

C) This operation fulfills a referral module function. Depending on the http response header (filled in by the original Web server) and more specifically on the "Content-Type" chain, the stream is redirected to a compression sub-module 25 most suitable for the type of declared content. For example, a “Content-Type: text.html” indication in the data returned by the server indicates that the content is of type HTML - Hypertext Markup Language - (text).

D) This operation is performed at the level of the service module 25 (compression sub-module): in the http response header, the string "Content-Encoding" is searched, making it possible to know whether the content is already compressed. Compression is not done if this chain is present and if one of these two values ("gzip" or "deflate") is present (in this case go to point I).

E) the first bytes of the content are compared with the following bytes: it may be that a poor implementation of a Web server 40 introduces coding errors and it is then necessary to ensure that the content is indeed text. If the content is not of HTML type, it is not modified (passage to point I below).

F) We call a compression function “compress” which returns the compressed content in a buffer to which it is necessary to add the following bytes: 1 F 8B 08 00 00 00 00 00, which define the start of the header gzip compression.

It is also then necessary to modify the first two bytes 00 03 to the. end of gzip compression header.

G) We replace the content in a pIcapSocketDataResponseContent object with the content of this buffer.

H) We modify and add, at the end of the response header, the line "Content-Encoding: gzip".

I) We send the response (modified or not) to the iCAP 10 client.

The class diagram in Figure 6 shows the generalization and dependency links between server classes and specific service classes.

The complete class diagram of a preferred iCAP zipper is the logical fusion of this diagram with that of an iCAP flow adapter as described above. The CicapServiceZip class is the child class of the ClcapService iCAP service class. It contains the functionality used for compressing objects.

The CicapGlobalsService class is the abstract child class of the class of definitions of constants and variables common to the service classes. It is not instantiable.

The CIcapErrorService class defines specific error constants for the service. It is not instantiable. The compression mechanisms provided by the service described here may act differently depending on the content to be processed.

These mechanisms can act to reduce the volume of data exchanged (data compression) with or without loss. They can also optimize the data exchanged by reorganizing the data.

To promote the growth of iCAP and extend the range of content transformation services in the heart of the network, we therefore offer here a compression service on the fly in the heart of the network which results in an acceleration of the downloading of web pages and usually content on http.

This service finds an immediate application for any Internet connectivity from mobile terminals (a PC - personal computer - or a PDA - pocket digital assistant - via a GPRS terminal - General Packet Radio Service - or CSD - Circuit Switched Data).

However, it is not limited to mobile operators only and may be applicable to any Internet service provider (ISP).

It therefore not only increases the quality of service perceived by the end user, but also optimizes the ISP bandwidth on the most critical portion: between the internet access point and the user.

This service also finds its effectiveness in the connections of nomadic users with their corporate Intranet. Indeed, it is through connections of PSTN (Switched Telephone Network) that they recover data from the Intranet, and, as such, compression on the fly offers a gain in relative bandwidth and a reduction in download time for new information.

Finally, from the point of view of a service provider, the iCAP or iCAP Zipper compressor described here has the primary advantage of saving bandwidth and improving the quality of service offered.

Claims

1. A method of on-the-fly compression of data supplied by a server (40) of a collective network as an Internet in response to a request from a consultation terminal (30), in which method a compression module is provided data (20), and a redirection module (10) provided for receiving data supplied by the server (40) and redirecting them to the compression module (20), this redirection module (10) also being provided for receiving data from the compression module (20) and redirect them to the consultation terminal (30), method according to which an indication concerning a decompression ability of the terminal (30) is examined in the request and carried out in the compression (20), on the data requested by the request and supplied by the server (40), a compression which is adapted to the indication of aptitude contained in the request.

2. Method according to claim 1, characterized in that no compression is implemented in the case where the indication of the request reveals an inability of the terminal (30) to decompression.

3. Method according to claim 1 or claim 2, characterized in that one searches in the request an indication of aptitude concerning any one of several types of decompression, and one implements that of the types of compression which specifically corresponds to the indication of aptitude contained in the request.

4. Method according to claim 3, characterized in that the compression module (20) comprises means for implementing said several types of compression, and in that the search for indications concerning the different types of compression is carried out by the compression module (20).

5. Method according to any one of the preceding claims, characterized in that the compression module (20) and the redirection module (10) are provided for exchanging data according to the ICAP protocol - Internet Content Adaptation Protocol.

6. Method according to any one of the preceding claims, characterized in that the redirection module (10) is a mass storage module intended to supply data stored in this module (20) several times in response to several requests for these same data, without having to interrogate, between these requests, a server (40) having initially supplied this data.

7. Method according to any one of the preceding claims, characterized in that the compression module (20) includes several sub-modules each dedicated to the compression of a different type of object, and in that one identifies in the data to compress the presence of objects having any one of these different types, then each object thus identified is directed to a module specifically dedicated to the compression of the type of the object considered.

8. Method according to any one of the preceding claims, characterized in that the consultation request is a request of the type

HTTP - HyperText Transport Protocol.

9. Device for compressing data on the fly in a collective network such as the Internet, comprising a compression module (20) and a redirection module (10), this redirection module (10) being designed to receive data supplied by a server (40) in response to a consultation request sent by a consultation terminal (30), and redirect this data to the compression module (20), this redirection module (10) also being designed to receive data supplied back by the compression module (20) and redirect them to the consultation terminal (30), this device further including means (20) for examining, in a data consultation request sent by a terminal (30 ), an indication concerning a decompression ability of the terminal in question (30), and means (20) for controlling compression by the compression module (20) which is adapted to the indication of decompression ability. contained in the request.

10. Device according to claim 9, characterized in that the means (20) provided for examining an indication of capacity for decompression in the request are provided so as to control no compression in the case where this indication reveals an inability of the terminal (30) to decompress.

11. Device according to claim 9 or claim 10, characterized in that the means (20) provided for examining an indication of capacity for decompression in the request are provided for searching in the request for indications of suitability relating to several types of decompression, and order the one of the types of compression which corresponds specifically to the indication of aptitude contained in the request.

12. Device according to claim 11, characterized in that the compression module (20) comprises means for implementing said several types of compression, and in that the search for indications concerning several types of compression is carried out by the compression module (20).

13. Device according to any one of claims 9 to 12, characterized in that the compression module (20) and the redirection module (10) are provided for exchanging data according to the ICAP protocol - Internet Content Adaptation Protocol.

14. Device according to any one of claims 9 to 13, characterized in that the redirection module (10) is a mass storage module designed to supply data stored in this module (20) several times in response to several requests requesting the same data, without having to interrogate, between these requests, a server that initially provided this data.

15. Device according to any one of claims 9 to 14, characterized in that the compression module (20) includes several sub-modules each dedicated to the compression of a different type of object, and in that it comprises means for identifying in the data to be compressed the presence of objects having any one of these different types, then for directing each object thus identified towards a module specifically dedicated to the compression of the type of object considered.

16. Device according to any one of claims 9 to 15, characterized in that it is intended to process requests of the HTTP - HyperText Transport Protocol type.