WO1992001281A1

WO1992001281A1 - Selective content synchronization of multiple image buffers

Info

Publication number: WO1992001281A1
Application number: PCT/US1991/004692
Authority: WO
Inventors: Jonathan D. Garman
Original assignee: Athenix Corporation
Priority date: 1990-07-10
Filing date: 1991-07-02
Publication date: 1992-01-23
Also published as: AU8393191A

Abstract

A system is described which provides a rendering engine (101) between the application engine (100) and multiple displays. Both the displays and the application engine are remotely connected to the rendering engine. At the rendering engine, a frame buffer (102) ("rendering frame buffer") into which the rendering engine (101) directly draws (i.e. provides the pixels represented as bits) is associated with each display. Periodically, the content of each rendering frame buffer is transferred from the rendering engine to the respective display, which has a local frame buffer ("display frame buffer") within it. To reduce the bandwidth required to transfer the pixels from the rendering frame buffer (102) to the remote display frame buffer, only the portion of the rendering frame buffer in which the rendering engine has drawn is transferred.

Description

SELECTIVE CONTENT SYNCHRONIZATION OF MULTIPLE IMAGE BUFFERS

FIELD OF THE INVENTION This invention relates to the control of multiple remote display stations; in particular, relates to the method of providing pixels ("rendering") at the remote display stations from a centralized controller.

BACKGROUND OF THE INVENTION There are several methods by which a system with a central controller ("rendering engine") controlling a remote display, may transmit rendered pixels to the display. These methods include direct transmission of video signals, pixel by pixel updating, and periodic retransmission of the entire display. All of these methods require very high information bandwidth, which is both difficult and expensive to achieve.

A common method to reduce the bandwidth requirement is to perform some rendering at the remote display itself, transmitting codes representing considerably more information than the pixels themselves. For example, a line (vector) is transmitted as a pair of end-point coordinates (or triples, for 3-dimensional lines) . The X Window System, a product of the MIT X Consortium, is an example of a standardized method for performing rendering remotely from a processor running an application program ("application engine") . The purpose of the X Window System is to allow efficient rendering at the remote display, even though the application program is executed at a location separated from the rendering location by a great distance. An^' efficient set of high level (in general, higher than pixel level) objects are defined and known by name to both the application and the rendering engines. The need to send complete objects pixel by pixel is eliminated by transmitting object names and parametric descriptions.

In order to carry out a standardized method, such as the X Window System, a complete knowledge of the predefined objects is required at the rendering engine, which is typically located either near or in the display.

Hence, considerable data storage is required for both object data (such as fonts, cursors, borders, menus, color maps) and programs in order to perform rendering and window management functions. This level of resource requirement results in a high cost associated with the rendering/display station.

It is thus desirable to have a remote rendering engine carrying out a standardized method, such as the X Window System, and which supports multiple displays stations, in order to achieve lower cost by sharing the required resources.

SUMMARY OF THE INVENTION

A system is described which provides a rendering engine between the application engine and multiple displays. Both the displays and the application engine are remotely connected to the rendering engine. At the rendering engine, a frame buffer ("rendering frame buffer") into which the rendering engine directly draws (i.e. provides the pixels represented as bits) is associated with each display. Periodically, the content of each rendering frame buffer is transferred from the rendering engine to the respective display, which has a local frame buffer ("display frame buffer") within it. To reduce the bandwidth required to transfer the pixels from the rendering frame buffer to the remote display frame buffer, only the portion of the rendering frame buffer in which the rendering engine has drawn is transferred.

These and other features of the present invention are better understood after considering the following description in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

Figure 1 is a block diagram showing a scheme by which rendering engine 101 periodically updates remote displays, in accordance with the present invention. Figure 2 shows, for one display, the contents of the rendering frame buffer and the display frame buffer at various times during the execution of an application program, in accordance with the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The embodiments of the present invention described herein are taken from a system of multiple X window display stations controlled by a single controller. The controller may be integrated with a host computer ("Z-controller" configuration) or stand-alone ("Z-node" configuration) . This system is described in copending application entitled "Workgroup Controller Architecture for Multiple Display Stations to be used with X Windows" by Robert A. Garrow, Serial No. 07/551,694, filed July 10, 1990, assigned to Athenix Corporation, and is hereby incorporated by reference in its entirety.

Figure 1 is a block diagram showing a scheme by which a rendering engine 101 of a Z-controller or Z-node supporting multiple remote display stations achieves a lower cost by sharing the required resources of the X Window System at the rendering and display locations, in accordance with the present invention. Under this scheme, the Z-controller or Z-node updates each supported display periodically or when requested, but updates only that portion of each display which has changed since the last update.

As shown in Figure 1, the rendering engine 101 receives data for central processing unit (CPU) 100. These data may be rendering commands and high level graphical object descriptions sent to the rendering engine. CPU 100 may be the application engine, or may be a network receiver of commands from a remote application engine.

The rendering engine 101, which and CPU 100 may form parts of one computer, converts high level graphical object descriptions into pixels, and draws the pixel values into frame buffer 102 at the addresses computed during the rendering operations. For example, a high level object may be as simple as a vector (straight line) specified by the coordinates of its end-points. Upon receiving the vector, the rendering engine 101 renders the vector by writing the vector value (color) into the pixel addresses located between the end-points. This is a one-to-many type operation, where the number of pixel write operations depends upon the distance between the end-points, and also whether more complicated functions such as anti-aliasing are required.

Frame buffer 102 holds the pixel values for the displayed image, with each value stored at an addressable location in frame buffer 102 corresponding to a pixel on the physical display. This correspondence must be one-to-one (unique and unambiguous) , but need not be in the same organizational form or number of dimensions or type of contiguity. Frame buffer 102 is written into by the rendering engine 101 and read by the update block refresh controller 112. When anti-aliasing is required, frame buffer 102 may also be read by the rendering engine 101.

In this embodiment shown in Figure 1 , there are four update block coordinate monitors 108-111 corresponding to four update block coordinate registers 104-107. The update block coordinate registers 104-107 contain the four pixel values Xmin, Ymin, Xmax and Ymax, being respectively the minimum X pixel address, the minimum Y pixel address, the maximum X pixel address, and the maximum Y pixel address, which collectively define the smallest area ("update block") on the display bounding all pixels written since the last update cycle. This arrangement presumes that addresses are available in 2-dimensional Cartesian coordinate form, although not necessary for the practice of the present invention. The same comparison can be accomplished with a min/max pair operating in a single dimension, though that will not allow the update block to be as small as in the 2-dimensional case.

Other coordinate systems are also possible, the choice consideration is a matter of cost versus benefit. The cost of using one coordinate system is often determined by the cost of address comparison (during writing) and address generation (during refresh) , both at the rendering engine and at the display. The benefit, on the other hand, is the possibility of having the smallest number of pixels in the update area relative to the number of updates within the area. Pixels which are in the update area but are not written will be transmitted redundantly to the remote display, and it is fundamental to the present invention that the number of redundant pixels transmitted during update be much less than the number of bits required to independently specify the address of each updated pixel. Of course, the similarity of pixels makes the redundant pixels easily compressible for efficient transmission, whereas addresses would be much less compressible.

For clarity of illustration, this description will use the 2-dimensional coordinate system.

Each update block coordinate monitor compares each pixel address as it is written by the rendering engine 101 with the value stored in the corresponding update block coordinate register, to determine whether the pixel being written lies within the currently defined update block coordinates, or whether any of the values stored in the update block coordinate registers 104-107 must be updated to include the pixel address being written (the pixel value is ignored, and only the address is used for comparison) .

Update block refresh controller 112 reads all the pixels within the update block using the values in the update block coordinate registers 104-107 and sends the pixels within the update block to the remote display. This operation can be performed after the rendering engine 101 completes a group of rendering operations, or upon receiving a command from the application program, such as a "flush buffer" command, or under control of a periodic timer, or a combination of these events, whichever occurs first. In any case, updating must be done often enough to provide the user at the display a fair degree of interactivity. A fair degree would be 5-10 times per second, for example, and preferably more frequent than that.

The pixel transmitter 113 provides the mechanism for moving the pixel data between the update refresh controller 112 and a remote display (not shown) . The transfer may be over a network, or a dedicated interface. In either case, it must support a transmission facility capable of the required bandwidth and covering the distance between the rendering engine and the display unit.

At the beginning of an update cycle, the update area is nil. The first pixel written creates an area (a rectangle in this embodiment) of size 1 (1 pixel) . The address of this pixel is stored in the update block coordinate registers 104-107 as the minimum and maximum coordinates of the update block. As each subsequent pixel is written, the pixel's address is compared to the update block (pixel values are not compared) , and if the pixel's address lies outside of the update block, the update block is increased to include that pixel address. Accordingly, the new value of the update block is written into the update block coordinate registers 104-107. This process continues until the end of the update cycle when the pixels in the updated block are transferred to the remote display by pixel transmitter 113.

When the update block is transmitted to the remote display by pixel transmitter 113, writing into the frame buffer 102 by rendering engine 101 is not allowed, and only the update block is sent to the remote display. In this embodiment, the update block coordinates (the contents of update block coordinate registers 104-107) are first sent to the remote display, followed by the pixel values within the update block. The pixel values may be sent in a compressed form in order to reduce transmission time. Both linear or 2-dimensional compression algorithms may be used (there is no 3-dimensional information remaining at this point) , and can involve the dynamic determination of subblocks which can be repeated to further reduce the transmission time, such as possible with images containing background texture "fill" patterns. Another simplification may be achieved by treating the 2-dimensional data as linear, with the conversion determined from the dimensions of the update area. For example, the area could be considered as a single vector composed of a linearly contiguous set of y vectors of length x, if the update area has dimensions of x by y. This serves to reduce the amount of location information attached to the data.

At the completion of transmission, the update block coordinate registers 104-107 are re-set to nil, and writing into the frame buffer 102 by rendering engine 101 is again enabled. The nil update block is a special condition to be recognized: while in this condition, the values in the update block coordinate registers 104-107 must not be construed as address values. One method to recognize this special condition of the nil state is to set the values of the update block coordinate registers 104-107 to values outside the display address limits, e.g. set the nil value of Xmin to a value greater than or equal to the largest possible value of X, so whatever value of X is next written will pass the A<B comparison in update block coordinate monitor 108 and get written into the update block coordinate register 104. Likewise, set Xmax to zero, or less than or equal to the smallest value of X possible to ensure the first value is written into the corresponding register. The treatment for the Y registers are exactly alike.

To further illustrate the present invention. Figure 2 is provided to illustrate the states of the update and display frame buffers during execution of an application program in a sequence of snapshots covering eight time periods (tl to t8) . For the purpose of illustrating the principles of the present invention, it is sufficient to show only the high level objects. The individual pixel operations will be difficult to resolve on the diagrams, and may cause confusion amid the borders and boxes.

As shown in Figure 2, at time t = tl, both update and the display frame buffers are at initial states. The first vector 200r is written by rendering engine 101 into the update frame buffer 102 at time t = t2. After the first vector 200r has been written into the update frame buffer 102 (that is, all the pixels resulting from rendering the vector 20Or into pixels) the update block coordinates have grown to be the smallest rectangle 250r which just encompasses the area into which the pixels were written. The rectangle 250r is aligned with the axes of the display because the 2-dimensional addressing scheme is used. After the second vector 201r is written into the update frame buffer 102 at time t = t3, the update area has grown to rectangle 251r, which is smallest rectangle encompassing all the pixels written since time t = tl. Up to this point, no pixel has been transferred to the display frame buffer.

At time t = t4, the refresh controller 112 initiates a update operation to transfer the pixels within the update block to the display frame buffer in the remote display. The pixel transmitter 113 performs compression on the pixels in rectangle 251r before transmission. (It is of course possible, in another embodiment, to have the refresh controller rather than the pixel transmitter, perform compression on the pixels before transmission) . Some of the criteria for initiating the transfer operation are discussed in the above; of course, the criteria for initiating a transfer operation are not factors affecting this invention.

During the transmission of the update block to the remote display, the refresh controller 112 must take control of the update frame buffer 102 from the rendering engine 101, and prevents further rendering into the update frame buffer 102 for the duration of the update operation. The means for exchanging control of a resource such as a frame buffer is well known in the art. If for any reason, control is lost during a refresh or update operation, the refresh or update operation must be aborted and restarted. It is critical that the addresses in the block update coordinate registers 104-107 not be re-set to their nil values until the update operation is successful, unless care is taken to incrementally reduce the update block; such care is not taken in this embodiment. Since the update block is not affected by aborting the update operation, a later update operation is made with the last update block, if no further pixels are written. In that case, the effect would be as though the last update operation had not been attempted. It can be readily seen that, even if further pixels are written which have increased the size of the update block, an aborted update operation which leaves the update block coordinate registers 104-107 intact leaves no undesirable effects. A variation on the method, which aborts completely when the update operation is interrupted, is to transmit the pixels a row at a time, decrementing the appropriate update block coordinate register as each row is successfully transmitted. This is somewhat more costly, but might be of value if there is a high incidence of interrupted update operations. This might be the case in a system where the circuit for the update operation contends with other devices for control of the frame buffer, or a system in which the rendering engine is given higher priority to the frame buffer than the refresh controller. If the CPU itself is the rendering engine, it may be appropriate to allow the CPU highest priority, at the expense of some occasional redundancy in the update operations.

The contents of the display frame buffer and the rendering frame buffer are completely identical following the successful completion of an update operation, as shown in the diagrams in Figure 2 corresponding to time t = t5. The diagrams showing the update and display frame buffers at times t = t6 and t = t7 further illustrate the expansion of the update block from nil to rectangles 252r and 253r, as new vectors 202r and 203r are drawn. Another update operation is shown at time t = t8. The diagrams corresponding to times t = t6 to t = t8 are provided here for completeness.

The embodiments described herein are intended to be illustrative of the general principles of the present invention. A skilled person in the art will be able to provide numerous modifications and variations within the scope of the present invention after consideration of the above description, in conjunction with the accompanying drawings.

Claims

new claims 5-8 added; other claims unchanged (1 page)]

4. An apparatus as in Claim 1, wherein said update controller prevents said rendering engine from writing into said update frame buffer during said transmission.

5. A method for efficient update of a display frame buffer at a remote display terminal with pixels provided by a rendering engine, comprising the steps of: storing in an update frame buffer said pixels provided by said rendering engine; maintaining an update block representing the minimum area of said update frame buffer covering all pixels written into said update frame buffer since an initialization step, and monitoring each of said pixels provided by said rendering engine, such that if a monitored pixel is not within said update block, said update block is increased to include said monitored pixel; and transmitting the pixels in said update block of said update frame buffer to said display frame buffer, and performing said initialization step.

6. A method as in Claim 5, wherein said step of maintaining an update block means comprises the step of providing register means for storing the addresses of said update frame buffer representing said update block.

7. A method as in Claim 6, wherein said step of maintaining an update block further comprises the step of providing comparator means for comparing the address of said monitored pixel with the addresses stored in said register means.

8. A method as in Claim 5, wherein said transmitting step further comprises the step of preventing said rendering engine from writing into said update frame buffer during transmission.

STATEMENT UNDER ARTICLE19

Claims 5-8 submitted herewith in the "Letter under PCT Rule 46.5" recite alternative embodiments of Applicant's invention as described in the Applicant's Specification. Since each of the added claims is supported by Applicant's Specification and drawings, the added claims have no effect upon the Specification and the drawings. Claims 5-8 more clearly recite Applicant's invention and more particularly distinguish over the cited references in the International Search Report of November 7, 1991, regarding the above Application.