WO2006095313A1

WO2006095313A1 - Method for remotely controlling a display apparatus based thereon and a portable device comprising such an apparatus

Info

Publication number: WO2006095313A1
Application number: PCT/IB2006/050714
Authority: WO
Inventors: Ewa Hekstra-Nowacka; Bart P. V. Peters; Wilhelmus J. M. Smits; Van Den Peter Hamer
Original assignee: Koninklijke Philips Electronics N.V.
Priority date: 2005-03-11
Filing date: 2006-03-08
Publication date: 2006-09-14
Also published as: TW200643851A

Abstract

Apparatus comprising a first device (22.2) with a parallel input side (14) and a serial output side (15). A second device (13) is provided that has a serial input side (15) and a parallel output side (16). An interface connects the parallel output side (16) of the second device (13) to a display panel (23.1). A serial link (15) is employed for the transmission of a serial data stream and provides for a connection between the serial output side (15) of the first device (12) and the serial input side (15) of the second device (13). The serial link (15) comprises at least one channel for the transmission of payload (RGB) comprising image signals and control signals (CLK, HS, VS, DE) for horizontal and vertical synchronizations. The control signals (HS, VS, DE) enable a number of regions in the serial data stream to be distinguished, each of said regions being characterized by a unique combination of the control signals (HS, VS, DE). An encoding unit (12.3) is employed for the generation of a characteristic sequence of bits. The encoding unit (12.3) is connected to the first device (12). A building block (13.1) may be employed for the word synchronization and the serial to parallel conversion.

Description

Method for remotely controlling a display apparatus based thereon and a portable device comprising such an apparatus

The invention relates to a display system and the serial communication between a controller and the display system. The invention relates in particular to a highspeed serial link for use in portable devices.

The present patent application claims priority of an earlier application that was filed on March 11, 2005. The following application number EP05101895 was assigned to this earlier application.

In many mobile devices there is a tendency to replace parallel bus connections by serial connections. This demand is at least to some extent driven by the trend towards appealing and lightweight designs. There are more and more devices where the display is remote from the actual host. An example is the so-called clamshell design that has become quite popular for mobile phones and other portable devices. In this case, the display typically is integrated in the cover lid (upper chassis) whereas the host (controller) resides in a base part (lower chassis) of the device. Most popular at the moment are the novel clamshell-type phones with a swiveling multidirectional hinge.

The design of these kinds of devices basically hinges on serial links, since it would be very expensive, if not impossible, to provide for a parallel link running through the hinge portion between the lower chassis and the upper chassis. Traditionally, designers have used CMOS parallel interfaces for the connection between a controller and the display. However, as the resolution of both LCDs and camera modules increases, so does the number of parallel wires running through the hinge. This situation not only complicates the design of the hinge, but also creates other problems, such as high power consumption and EMI (electro magnetic interference) emissions that can interfere with peripheral circuits.

An example of such a parallel link is illustrated in Fig. 1. The parallel link 9 in this Figure connects two interface circuits 7 and 8. The link itself has in the given example four bus lines 9.1 for control purposes (PCLK, VS, HS, DE) and eighteen bus lines for the 18 bit wide data stream for the pixel RGB data. To solve these problems associated with the parallel links, several types of serial- interlace technologies have been proposed. All of the proposed technologies emphasize optimization of electrical characteristics by restricting the transmission distance to several tens of centimeters. The new high-speed interface technologies currently under discussion are designed to be used to connect a baseband or application controller in the lower chassis with a display or image sensor (photo camera) in the upper chassis, or both.

Improved quality displays require a higher resolution and an increase of color depth. This translates into higher bit rates. Furthermore, moving pictures become more widely used. Special new serial interfaces with high speed are thus required. The Mobile Industry Processor Interface group (www.mipi.org) is seeking to standardize all interfaces around baseband and applications processors. In the context of Mobile Industry Processor Interlace (MIPI) standardization, there is a strong desire to replace the dedicated parallel interface, such as the one depicted in Fig. 1, with a standardized highspeed serial interface, such as the one depicted in Fig. 2, which allows increased composibility and interoperability between different components of the device. The bridge solution in Fig, 2 is believed to be an intermediate solution. In the future it is expected that a serial interface with no need for dedicated bridge devices will be used. This, however, will require a new definition of the future interfaces.

However, for parallel interfaces - such as e.g. a display interlace - this shift towards a serial interface requires serialization of data communicated through the parallel interlace. The control signals on the parallel link PCLK (Pixel clock), VS (vertical sync), HS (horizontal sync), and DE (data enable), are mandatory. When serializing this parallel interlace, these signals must be taken along with the pixel information. This is illustrated in Fig. 2. A so-called bridge chip 12 is situated next to the controller 11. The controller 11 sends the parallel data stream via the parallel bus 14 to this bridge 12. The bridge 12 then performs the serialization of these data. The serialized data are transmitted via a serial link 15 to a remote bridge chip 13. This bridge 13 is for example situated inside the upper chassis of a clamshell phone. The bridge 13 transforms the serial data into parallel data before sending them via another parallel bus 16 and a respective display driver or interface onto a display. The invention presented herein is concerned specifically with the display parallel interface (DPI) as promoted by the Mobile Industry Processor Interface group. The standard parallel display interface 14, as shown in Fig. 2, requires the following control signals: pixel clock (PCLK), data enable (DE), horizontal sync (HS), vertical sync (VS). It also requires between 12 and 24 bits (e.g., 18 bits RGB data with 6 bits R, 6 bits G, 6 bits B if an equal partitioning of the bits between these three colors is used). With each clock cycle a new data word is injected into the bridge 12. Such a word is defined to comprise of a header with DE, HS, VS bits, and the payload of 12 to 24 bits of RGB data for a pixel.

For MIPI purposes one has to serialize such a parallel word, so the parallel data will occur as the continuous sequence of bits on the serial link 15. This continuous sequence of bits, herein referred to as serial data stream, consists then of consecutive serialized words with a specified structure, i.e. DE, HS, and VS bits first, followed by the RGB bits, or in the reverse order with the RGB bits first followed by the DE, HS, and VS bits. The control signals DE, HS, and VS are hereinafter referred to as timing parameters or control signals, whereas the pixel data are referred to as payload.

There exist specific dependencies of signals occurring in such a display interlace 10. These dependencies are addressed in Figures 3 A and 3B. In these Figures the following abbreviations are used beyond the abbreviations that were introduced in connection with the Figs. 1 and 2:

18 bits RGB data [D17...0], HFP = Horizontal Front Porch, HS = Horizontal Synchronization, HBT = Horizontal Back Porch, VFP = Vertical Front Porch,

VS = Vertical Synchronization, VBP = Vertical Back Porch.

The pixel clock signal is provided in order to synchronize pixel data to the LCD display panel. VS is the vertical synchronization signal which in passive mode is the frame clock employed to signal the start of a new frame of pixels to the LCD display panel. In active mode, VS is the vertical synchronization signal. HS is the horizontal synchronization signal which in passive mode is the line clock that signals the end of a line of pixels to the LCD display panel. In active mode, HS is the horizontal synchronization signal. There are a number of options how such a serial interface can be organized.

For the purposes of the present invention the clock synchronous serial interface is considered with two options, namely either clock and data, or strobe and data. The respective link can also be characterized as source synchronous link, since the display uses the same clock frequency to display data as the source uses. That is, one can have as few as two wires or connections (data and clock) for the serial link 15. This is symbolically indicated in Fig. 2 by two wires present in the serial interface 15. In case of a differential signaling scheme in said serial link one would need two wire pairs. This is symbolically indicated in Fig. 9 by two twisted wire pairs present in the serial interface 25. On the serial link 15 an error can occur, which means that bit values may get inverted in case of clock and data, as illustrated in Fig. 4A. Bits can be removed from the serial data stream in case of strobe and data, as illustrated in Fig. 4B. The black boxes in these two Figures indicate the bit errors, the bit values in these are the erroneous values. As can be seen in case of clock and data, as illustrated in Fig. 4A, the word alignment is preserved even when data signal is "dirty". However, this is not the case for the strobe and data, as illustrated in Fig. 4B, where the recovered clock can be severely affected.

When replacing a standard parallel display interface with a high-speed serial interlace, serialization of data is required, as addressed above. In the presence of errors on the serial interface the line alignment at the receiving side of the interface can be lost. In the presence of long lasting errors the line following the loss, after successful re-synchronization, might be misplaced within the frame, which leads to severe artifacts on the display.

It is another problem, that while serializing data, the word/line boundaries are lost. If one starts to receive data at the display side, one sees just the bit stream and one does not know where the words are in this stream. That is, the word/line boundaries are lost on the serial link. So in the first instance one needs to find out where the word boundaries are in order to be able to retrieve these words (hereinafter referred to as word synchronization). The alignment between word and line boundaries becomes relevant when errors on the serial link occur. Such errors on the serial link can affect how the words align to each other within the bit stream.

The loss of the word/line boundaries but also the loss of the line alignment might lead to severe artifacts on the display. Therefore, a synchronization method is needed that allows the re-gaining of the word/line boundaries and then the line alignment at the receiving side of the serial interface. Such synchronization method also makes the display interlace robust against errors. In the presence of errors on the serial interface, not only the line alignment, as described above, but also the word/line alignment at the receiving side of the interface can be lost. In addition to the loss of the word/line boundaries and/or word/line alignment many other issues need to be taken into consideration when designing high-speed serial links for employment in portable devices, including power consumption, bandwidth of signals, distance that signals can be carried, implementation cost, sensitivity to noise and hardware/software complexity.

As mentioned above, the pixels can be represented by three colors R, G and B, for instance. Each of these colors can in turn be represented by different numbers of bits. This means that the RGB pixel representation can have different lengths. There can also be a different partitioning of the numbers of bits between these colors. There is thus also a need for a method that allows dealing with a dynamically changing format of the pixel representation at a line or frame level.

It is an object of the present invention to provide a synchronization method which allows reasonably quickly to provide for the word synchronization at the receiving side of a serial high-speed interface, on the line basis. It is an object of the present invention to provide a synchronization method which allows reasonably quickly to re-gain the line alignment within the video frame at the receiving side of a serial high-speed interface.

It is another object of the present invention to provide a synchronization method for re-gaining the line alignment at the receiving side of a serial high-speed interface after the word synchronization was performed.

It is another object of the present invention to provide a synchronization method allowing to cope with changing video (pixel) formats.

It is yet another object of the present invention to provide a synchronization method that allows to provide word synchronization in the active video region on the word basis.

It is another object of the present invention to provide improved implementations of such methods.

These and other objects are accomplished by a method according to claim 1, an apparatus according to claim 30, and a device, for instance a portable device, according to claim 44. Further advantageous implementations are given in the dependent claims.

The synchronization method proposed herein allows to quickly and reliably provide for a word synchronization at the receiving side of a high-speed serial link. To enable the word synchronization within the serial stream of data, an inventive synchronization method is proposed which allows a re-synchronization on a regular basis. According to the present invention horizontal synchronization patterns are constructed, which occur in the serial data stream at regular intervals, and on which one can perform a synchronization at the receiving end of a high-speed serial link.

The advantages of the inventive word synchronization are many- fold. With this synchronization method, the RGB pixel representation of the active video remains unchanged.

According to the present invention a method is proposed, which allows reasonably quickly to re-gain the line alignment within the video frame by incorporating the line numbers in the video stream. This method for line alignment is typically carried out after the word synchronization was performed. In accordance with the present invention a characteristic word is constructed, which is located in the porch part of each line together with the active video data, and which contains an appropriate line number. The advantages of the proposed method are many- fold. With the inventive method, no changes are needed for other words in the serial data stream than in the porch.

The characteristic word may instead of the line number also be a tag that defines the bit-budgeting for the different colors of a pixel.

The inventive method works for variable lengths of the RGB pixel representations and also for differently partitioned numbers of bits between the colors. There is no extra overhead required since one uses the bits already present in the serial data stream to send the line numbers and/or tags and/or the horizontal synchronization information in the serial data stream.

According to another embodiment of the invention, reliable word synchronization is also provided for the active video part. To this end, the method according to the present invention is characterized in that, if the predetermined sequence occurs in the active video part or its direct neighborhood, these data are replaced by alternative unique data. Hereby false synchronizations are prevented.

Implementations of the inventive synchronization methods in hardware and/or software are low cost.

It is another advantage that the inventive methods also work for video formats other than RGB, such as YUV, for instance. The inventive methods also work for other polarities of the control signals than those depicted in Fig. 3 A and 3B. The inventive methods can also be used for other devices dealing with serialized video, graphic streams, or still pictures, such as e.g. cameras, for example.

For a more complete description of the present invention and for further objects and advantages thereof, reference is made to the following description, taken in conjunction with the accompanying drawings, in which:

Fig. 1 is a schematic representation of a conventional parallel interface; Fig. 2 is a schematic representation of a serial high-speed interface, as for example used in a portable device according to the present invention;

Fig. 3 A is a schematic diagram illustrating specific dependencies of signals occurring in the parallel display interface (these dependencies are preserved during serialization);

Fig. 3B is a schematic diagram illustrating specific dependencies of signals occurring in the parallel interface;

Fig. 4A is a schematic diagram illustrating bit errors on the source synchronous serial link in case of a clock and data implementation;

Fig. 4B is a schematic diagram illustrating bit errors on the source synchronous serial link in case of a strobe and data implementation; Fig. 5 is a schematic diagram illustrating the losing of the word synchronization and thus also the line alignment in case of errors on the serial link;

Fig. 6 is a schematic diagram illustrating the words retrieved from the serial data stream being displayed in the line-by-line manner (from left to right);

Fig. 7 is a schematic diagram giving a pictorial view of the programmable timing parameters on a serial link;

Fig. 8 is a schematic block diagram of an embodiment of the present invention;

Fig. 9 is a schematic block diagram of another embodiment of the present invention.

As addressed in the introductory part, if one starts to receive data at the display side, one sees just the bit stream and one does not know where the words are in this stream. So in the first instance one needs to find out where the word boundaries are in order to be able to retrieve these words. This word synchronization is schematically illustrated in Fig. 5. In this Figure a situation is illustrated where the word synchronization and thus the line synchronization was lost, as indicated by the gray bits.

The invention presented herein deals with this synchronization/re-alignment problem and a synchronization method is described and claimed, which allows a word realignment at the receiving side of the serial link at regular intervals.

In order for the present method to function properly, one has to take certain measures at the transmitter side. As will be addressed later on, according to the present invention horizontal synchronization patterns are constructed, which occur in the serial data stream at regular intervals, and on which one can perform a synchronization at the receiving end of a high-speed serial link. These horizontal synchronization patterns are generated at the transmitter side and replace certain bits in the data stream.

The proposed solution to the word synchronization problem satisfies the following additional requirements: - no additional synchronization signals are used beyond the actual signals already present in the display interface; the word alignment is restored as soon as possible at the beginning of a new line following errors; implementations at the transmitter and the receiver side are of a low complexity (both in hardware and software).

As described in connection with Fig. 2, a serial data stream is transmitted via the serial link. At the receiving side, words are retrieved from the serial data stream. These words are displayed on a display in a line-by-line manner, as shown in Fig. 6. Four lines are depicted in this Figure. Assuming that the original DE, HS, and VS signals are present in serialized words of the serial data stream, it is possible to distinguish at the receiving end a number of regions in the serial data stream. These regions are characterized by a specific combination of DE, HS, and VS bits/signals. The respective regions are depicted in Fig. 7.

The diagram in Fig. 7 gives a pictorial overview of the programmable timing parameters. Each region in this diagram can be described by a three bit word, where the first digit represents DE, the second digit represents HS, and the third digit VS. With these three digits one can describe all regions inside the diagram. The gray area in the middle is the active video region.

The following table gives a better overview:

The region in the left hand corner at the top (1^st row, 1^st column) is represented by the digits 000, for example. The active video region (3^rd row, 3^rd column) is represented by the digits 111, just to give two examples.

Words outside the active video region have one of the following four headers: [000]; [001]; [010]; or [011]. The words inside the active video region have a [111] header followed by the original payload (e.g. an RGB payload).

According to the present invention, horizontal synchronization patterns (also referred to as characteristic sequence of bits) are constructed at the transmitter side and replace the not-active video payload data. The replacement is done in specific regions in Fig. 7 on which one can synchronize in order to (re-)gain the word synchronization. In the present embodiment the horizontal synchronization patterns replace bits in the regions at the beginning of each line (1^st column in Fig. 7). Such a horizontal synchronization pattern preserves the characteristic [DE HS VS] header and does not have any impact on the clock speed. To achieve this, each of these regions in the 1^st column requires a different horizontal synchronization pattern. The choice of payload of the horizontal synchronization patterns has to fulfill the following conditions: the horizontal synchronization pattern is characteristic in the sense that it has to differ from other words; - the horizontal synchronization pattern does not contain (or form with surrounding words) a sequence of three or more consecutive ones, since a sequence of three or more consecutive ones [111] represents the active video header (cf. Fig. 7); it is also essential that words coming from other regions of the serial data stream and their RGB payload are constructed in such a way that they too cannot be misinterpreted as a horizontal synchronization pattern.

In more general terms, according to the present invention one obtains the synchronization at word aligned positions only (except for errors in the communication) thus guaranteed false-sync free.

Taking all the above into account the following horizontal synchronization patterns are proposed. The proposed horizontal synchronization patterns are examples only: in the combined horizontal/vertical synchronization region (1^st row, 1^st column), i.e. the region designated by [000], the proposed horizontal synchronization pattern has the following structure: the [000] header (herein referred to as synchronization header), followed by a payload (herein referred to as synchronization payload) comprising a concatenation of Os and 1 s, and two Os at the end

^■» 000 0101....010100; or

^■» 000 0101 .... 01000

depending on the length of the RGB part; in the horizontal synchronization region (rows 2 through 4, 1^st column), i.e. the region designated by [001], the proposed horizontal synchronization pattern has the following structure: the [001] header (herein referred to as synchronization header), followed by synchronization payload (herein referred to as synchronization payload) comprising a concatenation of Os and Is

^■» 001 0101.... For all the relevant word transitions across the lines it can be shown that these horizontal synchronization patterns comprising the synchronization header and the synchronization payload do not occur in the off- word aligned positions, therefore allowing finding a unique place in the serial data stream. This unique place in the serial data stream provides a reliable reference for finding subsequent words in the serial data stream. This in turn enables the device (receiver) at the receiving end of the serial link to perform a word synchronization.

It is also conceivable that the word synchronization is performed in any of the other regions except the active video part, e.g. in the second column depicted in Fig.7. In order to do so, similar constraints as those imposed on column 1 must apply to the synchronization words in the addressed regions of column 2. In such a way the number of reference points in the bitstream for finding subsequent words increases, therefore decreasing the chance of word misalignment in case of errors as the region without horizontal synchronization is restricted only to the active video. Of course other horizontal synchronization patterns could be proposed.

However, they have to fulfill the requirement addressed before. In order to guarantee uniqueness of the synchronization patterns/words no modifications of the RGB payload in the active video are required (i.e., the active video remains intact).

The synchronization method of the present invention works for other numbers (typically varying between 12 and 24) of bits needed for the RGB representation. These colors can be represented by different numbers of bits and the RGB pixel representation can have different length. Furthermore, there can be different partitioning of the numbers of bits between the colors. The structure of the horizontal synchronization pattern remains then unchanged, i.e. the synchronization header is the same as well as the synchronization payload comprises alternating Os and Is. Only the length of the synchronization payload would have to be adapted.

One needs a synchronization header for each RGB payload to make the inventive synchronization method work. This synchronization header may have 2, 3 or more bits. This synchronization header enables an identification of words belonging to different regions of the serial data stream. However, in the case of the display or camera these 3 bits are inherent (already present) in the original parallel display interface anyhow.

The hardware implementation of the inventive synchronization method requires, for 18 bits RGB representation, 21 comparators for the total length of the serial word. Further details of possible implementations are discussed in connection with Fig. 8.

In accordance with the present invention and besides the word synchronization presented so far, there remains freedom for other aspects like line numbering, bit-budgeting by means of appropriate tags, control signal polarity coding, format and variable word length coding, and so forth.

The synchronization(re-alignment) of words can be regained by using the synchronization method proposed in connection with the above embodiment of the present invention. However, in case of long- lasting errors for lines following the newly found horizontal synchronization it is not obvious what is their line number. Or in other words, it is not clear what is their location within a frame.

If nothing is done about the line identification, the line misplacement due to long-lasting errors on the serial link can cause severe artifacts on the display. Such a line misplacement is herein called line misalignment. It is essential that in the case of such errors the proper line position will be restored as soon as possible. The invention presented herein deals with such a line misalignment problem, too.

The solution to the line alignment problem, as proposed and claimed herein, has the following advantages: no additional signals are used beyond the actual signals already present in the display interface; the line alignment can be restored as soon as possible at the beginning of a new line following errors, the implementation are of a low complexity (both in hardware and software); the horizontal synchronization method with the proposed horizontal synchronization pattern remains unaffected, that is, one can combine the method for line realignment and the method for word synchronization.

The words 30 retrieved from the serial data stream are displayed in a line-byline manner, as shown in Fig. 6. Assuming that the original DE, HS, and VS signals (herein referred to as control signals) are present in the serialized words 30 received at the receiving side, it is possible to distinguish a number of regions in the serial data stream. Each such region is characterized by a unique combination of DE, HS, and VS bits/signals. These regions are depicted in Fig. 7. As addressed above, the diagram in Fig. 7 gives a pictorial overview of the programmable timing parameters. According to the present invention, a new characteristic word is constructed, which is placed as the first word in the porch region (it could be also placed in other parts of the porch or simply repeated in the porch of each line that contains the active video). The porch region is the region in Fig. 7 being represented by the three digits [011]. In other words, the porch region is the "frame" around the active video region [111]. The new characteristic word incorporates an appropriate line number, according to the present invention.

The new characteristic word has the following structure: the header of such word (herein referred to as region header) is the porch header, i.e. [DE HS VS] = [011]. The new payload (herein referred to as characteristic payload) consists of burst of Os followed by the binary representation of the respective line number. This can be symbolically represented as:

* 0 1 1 0 0 0 ... 0 X X X X

with Xs being the binary representation of the line number.

Assuming that for the display resolution one needs up to 1024 lines, 10 bits for the binary representation of the maximal line number are required. On the other hand, the smallest RGB format requires 12 bits, which means that the smallest size of the zero-burst preceding the line number is of 2 bits. If one uses a different synchronization word, in for example column 1, it is conceivable to come up with a different format for the words which contain the line numbers.

For all the relevant word transitions across the lines with the line number indication incorporated in the words located in the porch region, it can be shown that the new characteristic word does not occur in the off-word aligned positions. Thus, the introduction of the characteristic words for line re-alignment, as proposed herein, does not have any impact on the horizontal synchronization or any other functions.

According to the present invention, the line number of the respective line can be extracted whenever a word in the porch is detected at the receiving side. This enables the bridge device 13 at the receiving side of the serial link to do a line re-alignment for each line or whenever less often.

The format of the pixel representation can change dynamically. In order to enable the receiving side to do an adjustment in case that a different pixel representation is used, according to a preferred embodiment of the invention appropriate format information is sent in the band. This enables a format detection at the receiving side.

According to the present invention, the format detection is done in two steps:

First the total length of the new pixel format (i.e. the total sum of bits used for all color representations for a pixel) is detected;

Then, the bit-budget for different colors for a pixel is detected.

Since the structure of the synchronization words is known, as described above, for each possible length of pixel representation the synchronization words can be constructed. If the requirement of at least two consecutive synchronization words is fulfilled, then for each position in the bitstream a search for two consecutive synchronization words is done, for each possible length of pixel representation, in order to determine the total length of the new pixel format for two consecutive synchronization words prevents possible misdetection of the new pixel format, for example in case when a valid short synchronization word occurs in the video payload of longer pixel format. In such a case the three ones header [111] of the following pixel of an active video will make the test for synchronization word to fail since the expected second synchronization word does not contain three consecutive ones.

According to the present invention, then the bit-budgeting for pixel colors is solved in a similar way as the line number problem discussed and described above. In a first step the possible bit-budgets are for example enumerated and a unique number/tag is assigned to each possible bit-budget. Such a tag is communicated across the serial link instead of the line number. As discussed and described above, 10 bits can be reserved for line numbers. These 10 bits gives also ample of freedom to define the different bit-budgeting for pixel colors.

The method proposed for the format detection allows a nearly instantaneous detection of the new pixel format with its length and bit-budget per color on the line basis. It is also possible to implement the format detection on the frame basis only. In such a case the format could be changed only on a frame basis, and preferably at the beginning of a frame. In case of errors on the serial display interface in the worst case this would lead to a loss of one frame. The format detection method, as discussed above, does not change the synchronization mechanisms as herein addressed.

The same format detection method can be applied also to other than RGB formats, (e.g. YUV). In this case the length detection would remain the same, but the bit- budgeting options must be extended by extra combinations specific for e.g. YUV. In the active video part one cannot synchronize using the method described so far.

In the active video area per video line there are no additional trigger events encoded so far. the word boundaries are found from the last trigger events outside the active video area (the horizontal back porch).

In the following an improved method is proposed in accordance with the present invention which provides for a synchronization even in the active video part.

The active video data is characterized by a 111 header followed by payload data. For the active video area one identifies a synchronization event as a matching of the active video header 111 -sequence. To prevent false synchronization, next one replaces all active video data words having at least a l l 1 -sequence in the concatenation of active video data word header 111 and active video data payload, not starting at the word boundary, by unique valid synchronization words from the space of unique synchronization words that satisfy the synchronization requirements, as outlined above, and that are not synchronization words already used for word synchronization in other regions.

Since the transmitter knows the word boundaries it is easy to replace a word with the 111 property by a unique valid synchronization word (forward mapping) before the serialization and transmission.

The replacement unique synchronization words are available at the receiving end of the bridge and one can do the synchronization at the receiving end on the 111 header of the active video data word or on the replacing unique valid synchronization word or on the synchronization words for the not active video regions. The receiver then has to replace the unique synchronization words used in the active video area with the original data using an inverse mapping available to the receiver. In the last sections an advanced single-lane (single channel) synchronization was described. The method described in these sections can also be expanded so that it can be used for multilane synchronization. In a multilane implementation one can either re-use the advanced single-lane concept, or one could provide for a shift of the communication of the synchronization in time so that the receiver would find headers at different position in time on the different channels. This allows to synchronize at the sub-word boundary level.

The polarity of the HS₅VS and DE signals can be different. If the polarity is different (e.g., DE is active low), one can do a mapping to the polarity that was assumed so far (e.g. DE active high). One then has to signal in the encoding of the synchronization words (by using different synchronization words) that a different polarity is used such that in the receiver the inverse mapping can take place.

In more general terms, methods are provided for remotely controlling a display, where a serial link is employed for transmitting a serial data stream from a controller to the display. In this context reference is made to an embodiment depicted in Fig. 9. The serial data stream carries payload, e.g. RGB formatted image signals, and control signals (HS, VS, DE) for the horizontal and vertical synchronization. These control signals (HS, VS, DE) enable a number of regions in the serial data stream to be distinguished, as depicted in Fig. 7, where each of these regions is characterized by a unique combination of the control signals (HS, VS, DE). According to the present invention the following steps are typically carried out. Please note that the serial bit clock is not part of the serial data stream. The time reference is used at the receiver side to sample the data.

A characteristic sequence of bits is constructed at the serializer/transmitter side

22.2 of the serial link 25 . The characteristic sequence of bits comprises a header and a payload portion.

Then the serial data stream is prepared for transmission across the serial link 25. The serial data stream comprises the control signals (CLK, HS, VS, DE), the payload (RGB), and the characteristic sequence of bits.

The serial data stream is then transmitted across the serial link 25 and at the receiving end 23.2 the words are retrieved from the serial data stream. A bridge 23.2, for instance, processes the serial data stream to detect the characteristic sequence of bits that indicate the word boundaries. The bridge can be a receiver or any other kind of device. It is also possible to include the functionality of the bridge into some other device, such as a display driver for instance. The characteristic sequence of bits that indicate the word boundaries enables the bridge 23.2 to provide for a word synchronization. In a last step the words are displayed on the display 23.1 in a line-by-line manner.

There are various advantageous embodiments of this method. These embodiments can be derived from the detailed description of the inventive methods. The present invention can be implemented in a display port bridge integrated circuit pair where one bridge 12 serves as transmitter-serializer and one bridge 13 as receiver- deserializer, as for example illustrated in Fig. 8.

According to the present invention, the bridges 12 implement building blocks

12.3 that allow the characteristic word and/or the horizontal synchronization pattern to be added to the serial data stream when transmitting data and building blocks 13.1 that allow the data to be processed at the receiving end 13 when having been received.

In the following one possible hardware implementation of the present invention is described. A corresponding block diagram is depicted in Fig. 8. The bridge 12 has a parallel input 14 and a serial output 15. Preferably, the inputs and outputs are CMOS based. The bridge 12 receives via the parallel bus 14 RGB (e.g., RGB888) and control signals (e.g., HS, VS, DE). In Fig. 8, the data lines are designated as Dat[0] through Dat[17] and the control lines 9.1 are designated as PCLK, DE, VS, and HS. The bridge 12 comprises the following building blocks: a parallel to serial converter 12.1, a PLL circuit 12.2, and an encoding unit 12.3 that implements the generation of the characteristic word and/or the horizontal synchronization patterns. Preferably, the bridge 12 is implemented so that the data are replaced by the horizontal synchronization words before the parallel- serial conversion is carried out by the parallel to serial converter 12.1. Furthermore, there are output buffers 12.4 for driving the signals across the serial link 15. In the present embodiment differential signaling is used on the serial link 14, as indicated by the CLK- / CLK+ and DO- / DO+ signal pairs.

The bridge 13 has a serial input 15 and a parallel output 16. The bridge 13 receives via the serial link 15 a serial data stream. This serial data stream comprises control signals, payload with image data, and characteristic words and/or horizontal synchronization patterns, in accordance with the present invention.

At the output side of the bridge 13 the data lines are designated as Dat[0] through Dat[17] and the lines 9.2 are designated as DE, VS, and HS. The bridge 13 comprises the following building blocks: a serial to parallel converter 13.1that implements the extraction of the characteristic word and/or the horizontal synchronization patterns. The serial to parallel converter 13.1 performs the word synchronization according to the present invention. For this purpose the serial to parallel converter 13.1 may comprise a number or comparators arranged in parallel for the total length of the serial word.

Details of another embodiment are addressed in connection with Fig. 9. A portable phone 40 is schematically depicted in this Figure. Only those elements of the phone 40 are shown that are relevant in the present context. The phone 40 comprises a first device 22.2 that has a parallel input side for connecting it via a parallel bus 24 to a controller or an application engine 22.1. The first device 22.2 further comprises a serial output for connecting it via a serial link 25 to a second device 23.2. The first devices 22.2 and the controller 22.1 may both be located in the lower chassis 41 of the phone 40. The second device 23.2 together with a display 23.1 may be located in the upper chassis 42 of the phone 40. The second device 23.2 has a serial input side 25 and a parallel output side 26. Typically there is an interface or display driver located between the second device 23.2 and the display 23.1. This element is not shown in Fig. 9. The serial link 25 is designed for the transmission of a serial data stream and it provides for a connection between the serial output side of the first device 22.2 and the serial input side of the second device 23.2. This serial link 25 may be realized by means of a flexible foil carrying the required number of conductive lines.

In the present embodiment the serial link 25 comprising just one channel for the transmission of payload (RGB) comprising the image signals and the control signals (HS, VS, DE) for horizontal and vertical synchronizations. More than one channel may be provided. One may obtain the synchronization from one channel only, but one can also obtain the synchronization from more channels.

The first device 22.2 comprises an encoding unit (e.g. an encoding unit 12.3 similar to the one depicted in Fig. 8) implementing the generation of a characteristic sequence of bits, as discussed in connection with the inventive methods. This encoding unit is connectable to the first device 22.2.

The second device 23.2 may comprise a decoding unit implementing the extraction of the characteristic sequence of bits. This decoding unit is connectable to the second device 23.2.

In Fig. 9 the position of a hinge is indicated by means of the dotted line 29. The upper chassis 42 of the phone 40 can be flipped around an axis that runs parallel to the line 29.

In the following section further details of possible embodiments are described. These details concern the embodiments of Fig. 8 and 9 in particular. The bridges 12, 22.2 and 13, 23.2 typically have separate pins for link power and ground. These pins are not depicted in the Fig. 8 and 9. The bridge 12, 22.2 accept parallel CMOS input data including color pixel data R[7:0], G[7:0], B[7:0], horizontal synchronization bit HS, vertical synchronization bit VS, data enable bit DE and pixel clock PCLK. The bridges 12, 22.2 generate the characteristic word and/or the horizontal synchronization pattern, as discussed in connection with the inventive methods and they serialize theses data.

The bridges 12, 22.2 output a high-speed serial data stream on up to three differential output channels. In Fig. 8 and in Fig. 9 implementations are shown with just one differential output D0+, DO-. The PLL 12.2 may be an integrated low-jitter PLL. It is employed to generate internally the bit clock used for the serialization of video input data Dat[0] ... Dat[17] and the controlling signals DE, HS₅VS and the bits representing the characteristic word and/or the horizontal synchronization pattern. The bridge 12, 22.2 outputs along with the serial output data DO+, DO- a differential pixel clock on a differential output pair CLK+ and CLK-.

The bridge 13, 23.2 accepts serial differential data inputs DO+, DO- and the differential input clock CLK+ and CLK- from the signaling channel (serial link 15 or 25) and deserializes the received data into parallel output data on the parallel bus 16 or 26. This bus 16, 26 may carry the following signals, for instance: R[7:0], G[7:0], B[7:0], HS, VS, DE along with pixel clock PCLK generated by logic from the bitclock (CLK) and the synchronization hardware. When a characteristic pattern is transmitted inside the serial data stream, the decoding unit 13.3 extracts the line number of the actual line whenever a characteristic word in the region header is detected. This information is sent to a display driver where it is used to rearrange the lines using the extracted the line number. When a horizontal synchronization pattern is transmitted inside the serial data stream, the decoding unit 13.3 detects this pattern to enable at the receiving side a word/line re-alignment (re-synchronization) within the serial data stream. This word/line re-alignment being done on a regular basis.

The transmitter-serializer (12 or 22.2) can be used to serialize parallel input data into 1 or more high-speed serial data channels. The receiver-deserializer bridge (13 or 23.2) may be realized so that it is able to deserialize up to 3 high-speed serial data channels into parallel data signals, for instance.

In a preferred embodiment the number of high-speed serial channels is made configurable from 1 to 3 depending on the bandwidth needed. The data link speed is determined by the PCLK (pixel clock) rate and the number of serial channels selected.

The present invention is well suited for the transmission of RGB formatted image signals, e.g. RGB888 video data, via a serial high-speed link to the display.

The present invention allows to provide a transparent interface between a baseband (e.g. located in a lower chassis 41) and a display portion (e.g. located in an upper chassis 42) of a mobile phone 40, for instance. The bridge 22.2, for instance, can be employed in the lower chassis 41 and the bridge 23.2 in the upper chassis 42. The bridges 22.2, 23.2 allow to establish a connection between a controller 22.1 in the lower chassis 41 and a display, e.g. an LCD display panel 23.1, in the upper chassis 42. The signal routing can be as follows: controller 22.1 + bridge 22.2 + serial link 25 (e.g., a flexible foil) + bridge 23.2 ^■*^■ display 23.1.

Preferably, low voltage differential signaling (LVDS), sub- LVDS or SLVS is used on the serial link. A high-speed serial link targeting a specific circuit iunction and having a tightly defined area of application is presented. The present invention allows to interconnect the system board and the display module in mobile phones and other types of handheld devices, especially those with a clamshell design.

The present invention allows reliable display interface connections to be made that can be used in clamshell phones, for instance. The display interface chips comprise a serializer and deserializer pair that is designed to replace the conventional parallel approaches. If combined with a flexible interconnect, one can realize new clamshell designs.

The invention can be used in mobile phone, e.g. high resolution mobile phones, and in portable applications with video capability, for instance. One can use the invention presented herein to allow users to watch multimedia images and still pictures displayed on the display of a handheld terminal, or to transmit images photographed by a camera phone from the camera to the remote controller or to a memory stick, for instance.

Of course other characteristic words or horizontal synchronization patterns for regions in the stream could be proposed as well as other payload structures. However, they have to fulfill the requirement of no misinterpretation in places other than prescribed word sync location.

The advantage of the inventive line alignment methods is that no extra overhead is needed in order to incorporate the respective characteristic bit sequences in the serial data stream. Instead of having meaningless information one uses the available bits. This allows at the receiving side of the serial interface to do the re-alignment.

It should be remarked that instead of original line numbers other representations, such as modulo values of a line number, could be communicated using the method presented herein. One could also use the inventive line alignment method on a few lines only to free up additional bits in the porch words for other purposes. These additional bits which are now unused can for example be used to send additional information, e.g. teletext or something comparable, across the serial link. In this case however, it is possible that artifacts on the display are not as quickly removed as desired since a line re-alignment would only be possible after the next line number was received, detected and extracted. The proposed method has also the advantage that it does not require adaptation of other words in the serial data stream. It works also for variable length of RGB payload (assuming the minimum length of 12 bits). The method can also be used for other video formats such e.g. YUV or RGBWhite. The present invention reduces reliability problems associated with the current implementations of the high-speed serial links as used in portable applications, for instance. The present invention enhances the reliability.

A synchronization method is proposed, which allows line realignment within a video frame at the receiving side of the serial link. It is an advantage of this synchronization method that the line realignment within a video frame is achieved without affecting the horizontal synchronization method proposed in connection with one of the embodiments of the present invention.

In the drawings and specification there has been set forth preferred embodiments of the invention and, although specific terms are used, the description thus given uses terminology in a generic and descriptive sense only and not for purposes of limitation.

Claims

CLAIMS:

1. A method for transmitting from a controller (11 ; 22.1) via a serial link (15; 25) to an image processing device (23.1) a serial data stream with payload (RGB) comprising image signals and control signals (HS, VS, DE) for horizontal and vertical synchronizations, said control signals (HS, VS, DE) enabling a number of regions in the serial data stream to be distinguished, each of said regions being characterized by a unique combination of the control signals (HS, VS, DE), characterized by comprising the steps of: constructing a characteristic sequence of bits, the characteristic sequence of bits comprising a header and a payload portion comprising predefined bursts, preparing the serial data stream comprising the control signals (HS, VS, DE), the payload (RGB), and said characteristic sequence of bits, transmitting the serial data stream across said serial link (15; 25), at a receiving end (13; 23.2) of said serial link (15; 25), detecting the characteristic sequence of bits, retrieving words from the serial data stream, - providing for a realignment, and providing said words to the image processing device in a line-by-line manner.

2. The method of claim 1, wherein said image processing device is a display. (23.1)

3. The method of claim 2, wherein said characteristic sequence of bits is a characteristic word for each line to be displayed on said display (23.1), the characteristic word comprising a region header for one of said regions and a characteristic payload consisting of predefined bursts followed a binary representation of the line number, comprising the steps: at the receiving end (13; 23.2) of said serial link (15; 25) extracting the line number of the line whenever a characteristic word in said region header is detected, realigning lines within a frame at the receiving side using the extracted line number to correct alignment problems that were caused by errors that can occur during the transmission of the serial data stream across said serial link (15; 25) among the plurality of bits of data in the serial data stream, said re-alignment of lines being preferably carried out by a display driver of said display (23.1).

4. The method of claim 3, wherein said region header is a porch header.

5. The method of claim 1, 2 or 3, wherein said characteristic sequence of bits is a horizontal synchronization pattern, said horizontal synchronization pattern being used to provide for a word synchronization.

6. The method of claim 1 or 2, comprising the following step: prior to a parallel to serial conversion replacing bits by a horizontal synchronization pattern

7. The method of claim 5 or 6, wherein said horizontal synchronization patterns are employed in order to enable at the receiving end (13; 23.2) a word/line re-alignment (re- synchronization) within the serial data stream, said word/line re-alignment being done on a regular basis.

8. The method of claim 5 or 6, wherein said horizontal synchronization patterns are present in said serial data stream at regular intervals such that they can be used at the receiving end (13; 23.2) for a word/line re-alignment (re-synchronization).

9. The method of claim 5 or 6, wherein said horizontal synchronization patterns are present in said serial data stream in order to enable at the receiving end (13; 23.2) a word alignment.

10. The method of claim 9, wherein said word alignment is restored at a beginning of a new line following an error.

11. The method of one of the preceding claims 3 through 10, wherein said horizontal synchronization patterns preserve the characteristic control signals in said serial data stream.

12. The method of claim 5, wherein for each of the regions a different horizontal synchronization pattern is provided.

13. The method of one of the preceding claims, wherein said serial link (15; 25) is a clock synchronous serial link (15; 25).

14. The method of one of the preceding claims, wherein the control signals for horizontal and vertical synchronization comprise DE, HS, and VS signals.

15. The method of one of the preceding claims, wherein said characteristic sequence of bits do not occur anywhere else in said serial data stream.

16. The method of claim 5, wherein said horizontal synchronization patterns do not contain or form with surrounding words in said serial data stream a sequence of three or more consecutive ones.

17. The method of claim 5, wherein each of said horizontal synchronization patterns allows the finding of a unique place in the serial data stream, which in turn serves as a reference for finding subsequent words.

18. The method of one of the preceding claims, wherein the payload (RGB) comprising the image signals remains unaffected by the insertion of said characteristic sequence of bits.

19. The method of one of the preceding claims, wherein video format information is incorporated in the serial data stream.

20. The method of claim 19, wherein the video format information is incorporated in the serial data stream on a regular basis at the beginning of a frame/line.

21. The method of claim 19, wherein a detection algorithm is employed at the receiving end (13; 23.2) of said serial link (15; 25) that prevents faulty detection of the video format information.

22. The method of claim 21, wherein a video format detection is preformed as follows:

Detection of the total length of a new pixel format; and Detecting a bit-budget for different colors for a pixel.

23. The method of claim 22, wherein the detection of the total length is done by searching for two consecutive characteristic sequences of bits in the bitstream.

24. The method of claim 22, wherein the bit-budgeting for the different colors of a pixel is done in that the possible bit-budgets are enumerated and in that each combination has a unique number/tag.

25. The method of claim 3, wherein a tag is communicated instead of the line number.

26. The method of claim 19, wherein the format detection is either done on a frame basis or on a line basis.

27. The method of claim 5, wherein the payload (RGB) comprising said horizontal synchronization pattern is replaced by a unique valid synchronization word and wherein said unique valid synchronization word is available to said receiving end (13; 23.2) for an inverse mapping.

28. The method of claim 27, wherein the synchronization at said receiving end (13; 23.2) is done on the replacing unique valid synchronization word.

29. The method of claim 27, wherein at the receiving end (13 ; 23.2) of said serial link (15; 25) a search is performed of the serialized pixel data stream (bit stream) for these unique synchronization words to obtain word realignment (synchronization).

30. Apparatus (10; 40) comprising: a first device (12; 22.2) having a parallel input side (14; 24) and a serial output side (15; 25), a second device (13; 23.2) having a serial input side (15; 25) and a parallel output side (16; 26), an interface for connecting the parallel output side (16; 26) of said second device (13; 23.2) to a display panel (23.1), a serial link (15; 25) for transmission of a serial data stream, said serial link (15; 25) providing a connection between the serial output side (15; 25) of said first device (12; 22.2) and the serial input side (15; 25) of said second device (13; 23.2), said serial link (15; 25) comprising at least one channel for the transmission of payload (RGB) comprising image signals and control signals (HS, VS, DE) for horizontal and vertical synchronizations, said control signals (HS, VS, DE) enabling a number of regions in the serial data stream to be distinguished, each of said regions being characterized by a unique combination of the control signals (HS, VS, DE), an encoding unit (12.3) implementing a generation of characteristic sequence of bits, said encoding unit (12.3) being connectable to said first device (12; 22.2), a decoding unit implementing an extraction of the characteristic sequence of bits, said decoding unit being connectable to said second device (13; 23.2).

31. The Apparatus (10; 40) of claim 30 further comprising means at the receiving end of said serial link (15; 25) that perform a realignment based on the extracted characteristic sequence of bits.

32. The Apparatus (10; 40) of claim 30 or 31, wherein said characteristic sequence of bits is a characteristic word for each line to be displayed on said display (23.1), and wherein said means perform a realignment of lines within a frame.

33. The Apparatus (10; 40) of one of the claims 30 through 32, wherein said characteristic sequence of bits is a horizontal synchronization pattern, said horizontal synchronization pattern being used by said means to correct for errors that can occur during the transmission of the serial data stream across said serial link (15; 25) among the plurality of bits of data in the serial data stream.

34. The Apparatus (10; 40) of one of the claims 30 through 33, wherein said first device (12; 22.2) and said second device (13; 23.2) are interface circuits realized as integrated circuits.

35. The Apparatus (10; 40) of one of the claims 30 through 34, wherein said serial link (15; 25) is established via a flexible foil.

36. The Apparatus (10; 40) of one of the claims 30 through 35 further comprising said display panel (23.1), said display panel (23.1) preferably being an LCD display panel.

37. The Apparatus (10; 40) of one of the claims 30 through 36, wherein said serial link (15; 25) comprises one or more high-speed serial channels.

38. The Apparatus (10; 40) of claim 37, wherein the bandwidth of the serial link

(15; 25) is determined by the number of high-speed serial channels and a pixel clock rate (PCLK).

39. The Apparatus (10; 40) of claim 30, wherein video format information is incorporated in the serial data stream.

40. The Apparatus (10; 40) of claim 39, wherein a detection algorithm is employed at the receiving end (13; 23.2) of said serial link (15; 25) that prevents faulty detection of the video format information.

41. The Apparatus (10; 40) of claim 39, wherein a tag is communicated instead of the line number.

42. The Apparatus (10; 40) of claim 33, wherein the payload (RGB) comprising said horizontal synchronization pattern is replaced by a unique valid synchronization word and wherein at said receiving end (13; 23.2) an inverse mapping is performed.

43. The Apparatus (10; 40) of claim 42, wherein the synchronization is done on the replacing unique valid synchronization word.

44. Portable device, preferably a phone, comprising an apparatus in accordance with one or more of the claims 30 through 43.