WO2007105130A2

WO2007105130A2 - Method of generating content on a display device

Info

Publication number: WO2007105130A2
Application number: PCT/IB2007/050667
Authority: WO
Inventors: Galileo J. Destura; Luc F. A. Knaapen
Original assignee: Koninklijke Philips Electronics N.V.
Priority date: 2006-03-14
Filing date: 2007-03-01
Publication date: 2007-09-20
Also published as: WO2007105130A3

Abstract

A method of generating content to be displayed on a display device is disclosed. The method receives by means of a detection device signals generated by a pointer device in a hand of a user, wherein the detection device is in a pre-determined position with respect to the display device. From said signals, a parameter having a relationship with jitter in said signals due to hand tremor is determined. Subsequently, content enabling a display device to reproduce said content with a dimension which has a relationship with said jitter is generated and supplied to a display device. The method improves user interaction with said content when the user uses a pointing device to point at said content.

Description

Method of generating content on a display device

FIELD OF THE INVENTION

The present invention relates to a method of generating content on a display device. The invention further relates to a device for generating content on a display device, a computer program product and a computing system or semiconductor device.

BACKGROUND ART

The market for large area displays is growing. The increase in viewing size enables the multitude of possibilities of multimedia contents that can be shown. WO2005/029395 discloses a display apparatus which enables the possibility for user interaction with the content on the screen by means of a laser pointer device.

Presently, the content is shown to the user in a predefined way. Specifically, the size of the on-screen buttons and controls are fixed. The size is based on an average distance between a user and the screen of the display device. The size of the content and especially the size of the control buttons shown on the screen are not optimal anymore when a user is too close or too far from the display apparatus.

US2005/0030322 discloses a display technique to determine the desired size of an object to be displayed. A distance dependent measure such as visual arc angle is used to adjust the size of the displayed object to a desired size. A distance detector such as a video device or infrared or acoustic range detector is used to determine the distance.

SUMMARY OF THE INVENTION

It is an object of the present invention to provide an improved method of generating content on a display device so as to improve user interaction with said content.

This object is achieved by a method of generating content on a display device comprising: receiving by means of a detection device, signals generated by a pointer device in a hand of a user, wherein the detection device is in a pre-determined position with respect to the display device; determining a parameter from said signals, the parameter having a relationship with jitter in said signals due to hand tremor; generating the content in dependence of said parameter, wherein the content enables a display device to reproduce said content with a dimension which has a relationship with said parameter; supplying the content to said display device.

The present invention is based on the recognition that people are not able to hold a pointing device, e.g. laser pointer, absolutely steady due to hand tremor. It is known from literature that the hand tremor is of low amplitude at around 10 - 12 Hz. Furthermore it is known that this low amplitude tremor subtends an angle of around 0.4 - 0.5 degrees, with respect to the most direct path from the laser to the surface where the laser light impinges on. If the information on a screen comprises a lot of small details, it would become difficult to select or point at a detail (content). This will become even worse if a user with Parkinson disease is using a laser pointer. A person with Parkinson disease will have a degree of hand tremor which subtends a much larger angle. The extent of jitter due to hand tremor in the signals received at the detection device is used to enable the generation of content on a screen which facilitates the user to select or to point at said content. According to the invention the ease to point clearly at the content is made independent of the extent of jitter due to hand tremor. In a further embodiment of the invention the dimension of the content on the display device is the size of the content on the screen. This feature enables to increase the size of the content on the screen if the jitter in the signal at the detection device increases. In a further embodiment of the invention the content represents a button to control an application. This feature enables to improve the ease of controlling the application as the size of the buttons is adapted to the amount of jitter, while the size of the non- controlling content on the display remains unchanged. This feature is very suitable for video applications and presentation application. All content related to the video or presentation remains visible on the screen, while the ease to control the application, i.e. to point at a specific button, is independent of the jitter in the signals received at the detection device and screen due to the hand tremor of the user controlling the application. As the size of the buttons has a relationship with the area in which the spot of the pointing device moves on the display, the ease to select a button is increased and the chance to select none or another button is decreased, as the size of the buttons increases when the area in which the spot moves increases. In a further embodiment of the invention the parameter has a relationship with a distance between the user and the display device. This feature enables to change the size of the content such that it remains constant from the user point of view. The angle wherein the pointing device moves due to the hand tremor is steady. The distance between the user and the display or input determines the extent of jitter in the signals received at the detection device and movement of the spot on display device. If the distance doubles the amplitude of the jitter in the signals received at the detection device and movement of the spot will double. Knowing the distance or change in distance enables to adapt the content such that the perceptibility of the content on the display device remains the same for each distance. This could be very helpful if characters or symbols are present on control buttons.

In a further embodiment of the invention the signals received at the detection device are generated by a light emitting pointing device, such as a laser pointer or an LED pointing device. In a further embodiment of the invention the detection device has a light- sensitive surface and the parameter has a relationship with a dimension of the area of the light-sensitive surface receiving light from the pointing device. These features enable to use commonly known image processing techniques to determine the jitter. Furthermore, a traditional camera could be used as the detection device.

In a further embodiment of the invention the parameter is the shape of the area of the light-sensitive surface receiving light from the pointing device. This feature enables to determine the position of the user with respect to the display device. For example, if the user stands in front of a display device the shape of the area will be almost circular, whereas the shape will become elliptic when the user moves to the left of to the right with respect to the display device.

According to an embodiment of the invention the signals are received via the display device. This feature enables a more simplified determination of the distance between the user and the display device. Furthermore, this feature enables to determine the distance between display and pointing device continuously as long as the user uses the pointing device to indicate content on the display or interacts with the content on the screen.

In a further embodiment of the invention the detection device has a light redirecting surface and the parameter has a relationship with a dimension of the area of the light redirecting surface receiving light from the pointing device. This feature enables to project the light on one or two array of photo detectors positioned horizontally and vertically at one or two sides of a screen, to enable measurement of the height and/or width of the spot on the screen. A further aspect of the invention relates to a device for generating content on a display device comprising: input means for receiving signals generated by a detection device arranged for detecting signals generated by a pointer device in a hand of a user, wherein the detection device is in a pre-determined position with respect to the display device; processing means for determining a parameter from said signals, the parameter having a relationship with the jitter of said signals due to hand tremor; generator means for generating the content in dependence of said parameter, wherein the content enables a display device to reproduce said content with a dimension which has a relationship with said parameter; output means for supplying the content to a display device.

In an embodiment of the device according to the present invention the device further comprises the display device wherein the input means are combined with a screen of said display device. This feature enables a more simplified determination of the distance between the user and the display device. Furthermore, this feature enables to determine the distance between display and the pointer device, e.g. a laser pointer, continuously while the user uses the pointer device to indicate content on the display or interacts with the content on the screen.

Another aspect of the invention relates to a computer program product comprising computer executable instructions which instructions when executed on a processor perform the method according to the invention.

Yet another aspect of the invention relates to a computing system or semiconductor device comprising: input means for receiving signals generated by a detection device arranged for detecting signals generated by a pointer device in a hand of a user, wherein the detection device is in a pre-determined position with respect to the display device; ignal processor for generating a signal representing content; memory comprising executable instructions which instructions when executed on said signal processor perform the method according to the invention. - utput means for supplying the signal representing the content to a display device.

The invention may be an integral part of a computer system or implemented on a stand-alone device such as an adapter card. The invention is especially suitable to make the operability of buttons independent of the extent of hand tremor or the distance between the user controlling the application and the display.

BRIEF DESCRIPTION OF THE DRAWINGS The present invention will be described in detail, by way of example and with reference made to the accompanying drawings, in which:

Fig. 1 shows schematically the effect of hand tremor on a laser beam from a hand held laser pointer device;

Fig. 2a and 2b shows an embodiment of the result of the method according to the invention if applied on buttons in an output image; and

Fig. 3 shows a simplified block diagram of an embodiment of a system for generating content on a display device according to the present invention.

DETAILED EMBODIMENTS Fig. 1 shows schematically the effect of hand tremor on a laser spot 4 from a hand held laser pointer device 2. People are not able to hold a laser pointer absolutely steady. The laser spot will move horizontally and vertically on a surface 6 due to the hand tremor. The movement of the laser spot 4 in horizontal and vertical direction is called jitter (d_h, d_v). It is known that the hand tremor is of low amplitude at around 10-12 Hz. Furthermore, it is known that this low amplitude tremor subtends, with respect to the most direct path from the laser to the surface where the laser beam impinges on, an angle of deviation (θh , θ_v) of around 0.4 - 0.5 degrees. Persons with Parkinson disease or aged person will normally have hand tremor with a much higher angel of deviation. For those persons it will be more difficult to point at a relatively small detail on a screen with a pointer device. The amount of hand tremor is person dependent. The hand tremor can be used to determine a relative change of distance between a person and a screen. If the degree of hand tremor of a person is known, the hand tremor can be used to determine the absolute distance between a person and the screen. A simplified relationship between the distance of a user and a surface on which the laser beam impinges is given by the formulas: dh= 2 -tan (θh/2) -d_us and d_v = 2 -tan (θ_v/2) -d_us wherein dh and d_v are respectively the horizontal and vertical movement of the beam on the surface, d_us is the distance between the user and the surface, and θh and θ_v respectively the horizontal and vertical angle of the hand tremor are. According to these formulas, if the jitter doubles the distance between the user and the surface has been doubled. Furthermore, it is known that the degree of hand tremor is related to the type of pointer device. The physical design of a hand held device influences the ability to point at particular detail on a screen.

If a user is walking around, for example during his presentation, the horizontal angle of the laser beam with respect to the screen will vary substantially, whereas the vertical angle will vary within small limits. Therefore, the vertical jitter will normally be a more accurate parameter to determine the distance between the user and the surface.

Fig. 2a and 2b shows an example of the result of the method according to the present invention if applied on buttons in an output image. The output image comprises a background and control buttons 202, 204, 206. In the example the control buttons comprise text A, B, C which could indicate the function of the control button, the text could also be an icon representative for the corresponding function. Fig. 2a shows an example with buttons suitable for a person with average hand tremor and at an average distance. Fig. 2b shows a corresponding example with buttons suitable for a person with Parkinson disease or for a person having a distance to the screen which is doubled. According to the invention the average of the area due to the dwelling of the laser point is determined. The size of the area is taken to determine the appropriate size of the buttons to be displayed to enable controlling the application. The size of the buttons could be chosen such that when pointing for a predefined period at a particular detail the laser spot moves for example 80% of the time in the area of the button. In this example, the chance to select a button to control the application is 80%. Associating the button having 80% of laser spot dwell time to the click action can further increase the chance of a user selecting the right button. It might be clear that the percentage may be changed based on user requirements.

Fig. 3 shows a simplified block diagram of an embodiment of a system for generating content on a display device according to the present invention. The system 300 comprises an input 302 for receiving signals supplied by a detection device 320 which derived said signals from signals generated by a pointer device 314 in a hand of a user. The pointer device 314 could be a laser pointer, infra-red pointer or a LED pointer. The signals generated by a pointer device could be received by a combined input/output device as disclosed in WO 2005/029395. Said document discloses a display monitor with a light guiding layer to enable the determination of the position of a laser spot from a laser pointer 314 on the display monitor 310. The signals could also be received by any other detection device with a light-sensitive surface, such as a camera 320 in the neighborhood of a display monitor 310. In Fig. 3 the detection device 320 is placed on top of the display monitor 310. If the display device is a combination of a projection device, such as a beamer, and a projection screen, the camera could be positioned after the projection screen. In this way the signals are received via said display device. The detection device could obtain a substantial part of the signals from a light spot and correspondingly determine the pointed position and corresponding deviation due to hand tremor, on the display monitor. Another method is that detection device could also obtain the signals by means of ultra sound sensors or radio receivers in the neighborhood of the display monitor 310. The ultra sound sensors or radio receivers could be positioned at the corners of the display monitor or projection screen. By measuring the phase difference of the sound or radio signals from the pointing device the pointed position and related deviation due to hand tremor could be measured. The algorithm to determine the deviation is not discussed in more detail as the technology to determine distances with ultra sound and radio waves is commonly known to the person skilled in the art.

The system 300 further comprises a processing unit 304 having an input connected to the input 302. The processing unit 304 is arranged for determining a parameter from said signals. The parameter has a relationship with the jitter of said signals due to hand tremor. A parameter could be the vertical or horizontal movement of the laser spot on the screen. It should be noted that the jitter due to hand tremor could be regarded to be as noise on the movement of the laser spot by a person. Therefore, a minimal movement in a predefined period of time, for example 1 second, corresponds to the jitter due to hand tremor. Therefore, the jitter due to hand tremor can not only be measured by pointing at a particular point on the screen but also by filtering of the path of the spot on the screen. If the signal comprises noise, which has a frequency which is higher then the frequency of hand tremor, a cut-off frequency of said filter should be slightly above the frequency of hand tremor, which is approximately 10-12 Hz, to remove noise in the detected signal. The filter could further comprise a cut-off frequency slightly below the frequency of hand tremor, to remove the frequencies below said cut-off frequency which correspond to the average path the spot moves over the screen. The parameter could further be the dimensions of shape, for example a square, circle or ellipse, in which a laser spot dwells for a predefined period of time. The determined parameter is supplied to a signal generator unit 306. The parameter could also be a measure of the distance between the user with the pointing device and a screen. By means of triangulation the distance can be calculated. Above, the formulas to determine the distance are given if the movement angle of the hand is known. If the parameter is the distance, the parameter is very suitable to indicate the appropriate size of characters and/or content on the screen to be readable/recognizable at any distance.

The signal generator unit 306 is arranged for generating content in dependence of said parameter. The content is in a form to enable a display device to reproduce said content with a dimension which has a relationship with said jitter. The signal generator unit 306 could be arranged to generate the content in the form of a standard video output signal. If the system is in the form of video player, the signal generator unit 306 retrieves the video information from a storage device 312, such as a DVD or solid state memory. To enable on screen command display, the generator unit 306 generates buttons to facilitate control of the video player. Subsequently the video information and the buttons are combined into a standard video signal. According to the invention the dimensions of the generated buttons have a relationship with the determined parameter. It should be noted that the generator unit 306 could obtain the appropriate buttons by scaling of a representation of a button, changing only the boundary of a button and not information in the button or by selecting an image having a size corresponding to the parameter. In the last embodiment is present in a memory an image for each size of button.

The generator unit 306 could also be arranged to generate an MPEG-4/7 signal or any other Interactive graphics applications format or Interactive multimedia format. In this case the output signal comprises different objects. In the example of a video player application the video information is an object and the respective buttons are objects in the signal. In such a signal, each object could comprise scaling and position information. The generator unit 306 in this embodiment is arranged to assign the corresponding scaling or size information to the appropriate objects. In this embodiment the content generated by the generator unit 306 is in the form of an MPEG-4/7 signal. The content generated by the signal generator unit 306 is supplied to an output unit 308 for supplying the content to a display device 310. The output unit 308 could be the display device 310. The output unit could also be a signal transformation unit to transform the output signal from the signal generator unit 306 to the appropriate signal format for the display device 310. In light of the invention, with content is meant any visual representation of an object, scene, person, abstraction or character produced on a surface. Content could also be a graphic symbol that denotes a program or a command or a data file or a concept in a graphical user interface. Furthermore, a pointer device could be a conventional small laser pointer or any other hand held device with a build in pointer device, such as a Palm Pilot with a built in laser. In stead of a laser a LED, light emitting diode, or a combination of multitudes thereof could be used. If a laser or multitude of LEDs is used, the detection device arranged to register the hand tremor is preferably an image detector with a light sensitive surface, such as a CCD camera, a display device as disclosed in WO 2005/029395, or any other suitable optical position detection device, positioned in close proximity to the display device. The display device as disclosed in WO 2005/029395 comprises a light redirecting surface. The spot which impinges on the screen is redirected to an array of photo diodes. With the output signal from the photo diodes the height and width of the area on the screen can be measured. In this way, parameters having a relationship with a dimension of the area of the light redirecting surface receiving light from the pointing device can be measured.

If the detection device is not combined with the display device, the invention could also be used when the position of both the display device and the detection device is known with respect to each other. In this case the system according to the invention determines first the distance between the user and the detection device and orientation of the user with respect to the detection device. Subsequently, the distance between the user and the screen is calculated by taking into account the position of the display device and the detection device. The method may be implemented in hardware, software or a combination of hardware and software. The method could be implemented on a PC. In that case the computing system 316 comprises input means 302 for receiving signals generated by a detection device detecting signals generated by a pointer device in a hand of a user, a signal processor 318 for generating a signal representing content, a memory 312 comprising executable instructions which instructions when executed on said signal processor to perform the method according to the invention and output means 308 for supplying a signal representing the content to a display device. Examples of input means 302 are a USB interface, IEEE 1394 interface or video input card. The signal processor 318 performs the function of both the determining of a parameter from said signals having a relationship with the extend of hand tremor and the generation of the content in dependence of said parameter, wherein the content enables a display device to reproduce said content with a dimension which has a relationship with said extend of hand tremor. Examples of the outputs means 308 are a USB interface, IEEE 1394 interface or a video output card. The adjustment of the size of the content could be performed continuously when the pointing device is used so as to maintain the optimum performance dependent on the jitter and/or distance of the user from the display device.

Though the invention is described with reference to preferred embodiments thereof, it is to be understood that these are non- limitative examples. Thus, various modifications are conceivable to those skilled in the art, without departing from the scope of the invention, as defined by the claims. In the described embodiments the size of buttons is adapted to facilitate the ease of control of an application. However, the application is also suitable to adjust the size of highlight in presentation. This enables to make the pointing at details on a screen easier, which consequently make it unambiguous for other viewers which detail is selected. Furthermore, instead of adapting both the length and width of a button to facilitate the ease of use, only the length or width could be changed to increase the area of a button.

The use of the verb "to comprise" and its conjugations does not exclude the presence of elements or steps other than those stated in a claim. Furthermore, the use of the article "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. In the claims, any reference signs placed between parentheses shall not be construed as limiting the scope of the claims. The invention may be implemented by means of hardware as well as software. Several "means" may be represented by the same item of hardware. In the event the invention is implemented as software, the corresponding program could be stored on a record carrier so as to enable a device to execute the program and to perform the method according to the invention. Furthermore, the invention resides in each and every novel feature or combination of features.

Claims

CLAIMS:

1. Method of generating content on a display device comprising: receiving by means of a detection device signals generated by a pointer device in a hand of a user, wherein the detection device is in a pre-determined position with respect to the display device; - determining a parameter from said signals, the parameter having a relationship with jitter in said signals due to hand tremor; generating the content in dependence of said parameter, wherein the content enables a display device to reproduce said content with a dimension which has a relationship with said parameter; - supplying the content to said display device.

2. Method as claimed in claim 1, wherein the dimension of the content on the display device is the size of the content on the screen.

3. Method as claimed in claim 1, wherein the content represents a menu button to control an application.

4. Method as claimed in claim 1, wherein the parameter has a relationship with a distance between the user and the display device.

5. Method as claimed in claim 1, wherein the signals are generated by a light emitting pointing device.

6. Method as claimed in claim 5, wherein the detection device has a light- sensitive surface and the parameter has a relationship with a dimension of the area of the light-sensitive surface receiving light from the pointing device.

7. Method as claimed in claim 6, wherein the parameter is the shape of the area of the light-sensitive surface receiving light from the pointing device.

8 Method as claimed in claim 5, wherein the detection device has a light redirecting surface and the parameter has a relationship with a dimension of the area of the light redirecting surface receiving light from the pointing device.

9. Method as claimed in claim 1, wherein the signals are received via said display device.

10. Device for generating content on a display device comprising: - input means (302) for receiving signals generated by a detection device arranged for detecting signals generated by a pointer device in a hand of a user, wherein the detection device is in a pre-determined position with respect to the display device; processing means (304) for determining a parameter from said signals, the parameter having a relationship with the jitter of said signals due to hand tremor; - generator means (306) for generating the content in dependence of said parameter, wherein the content enables a display device to reproduce said content with a dimension which has a relationship with said parameter; output means (308) for supplying the content to a display device.

11. Device as claimed in claim 10 wherein the device further comprises the display device wherein the input means are combined with a screen of said display device.

12. A computer program product comprising computer executable instructions which instructions when executed on a processor perform the method according to any one of the claims 1 to 9.

13. A computing system or semiconductor device for generating content on a display device comprising: input means (302) for receiving signals generated by a detection device arranged for detecting signals generated by pointer device in a hand of a user, wherein the detection device is in a pre-determined position with respect to the display device; a signal processor (318) for generating a signal representing content; a memory (312) comprising executable instructions which instructions when executed on said signal processor perform the method according to claim 1; output means (308) for supplying the signal representing the content to a display device.