WO2007049253A2

WO2007049253A2 - Display system with a haptic feedback via interaction with physical objects

Info

Publication number: WO2007049253A2
Application number: PCT/IB2006/053989
Authority: WO
Inventors: Sander B.F. Van De Wijdeven; Vincent P. Buil; Gerrit Hollemans
Original assignee: Koninklijke Philips Electronics N.V.; U.S. Philips Corporation
Priority date: 2005-10-28
Filing date: 2006-10-27
Publication date: 2007-05-03
Also published as: WO2007049253A3

Abstract

This invention discloses a user interface that includes: a display unit (12); a layer of electro magnets (14) disposed on the surface of a flat panel display, and a control module, wherein the polarity of the electro-magnetic layer is selectively changed to attract, repel, and/or vibrate a pointing device that interacts with a screen surface. Sensing of the x-y position of the pointing device is achieved my alternating magnet movement stimuli with magnetic signal readings performed by the matrix of electro-magnets.

Description

DISPLAY SYSTEM WITH A HAPTIC FEEDBACK VIA INTERACTION WITH

PHYSICAL OBJECTS

The present invention relates generally to a user interface using a pointing instrument and, more particularly, to stylus interaction on a display with an array of a plurality of electro -magnetic systems to enhance user interaction.

Recently, the computer industry has developed the era of Tablet PC, which is a PC between a PDA and a notebook. In particular, Tablet PC is similar to a portable PC usually with a handwriting recognition system. Tablet PC has various functions, such as a digital touch panel and digital pen, to perform the handwriting model with a touch pen or stylus to input data. Users can input and operate on a touch screen by using a touch pen (see http://en.wikipedia.org/wiki/Tablet_PC).

One drawback of the stylus interaction is that there is no tactile feedback on the interaction to the user other than the feeling of the pen touching the screen.

The present invention provides an improved user interface that can not be achieved by a conventional stylus.

One aspect of the present invention is to provide a user interface display with a layer of a plurality of electro magnets, which are designed to interact with a stylus or a pointing object such that the stylus interaction provides more accuracy and helps to speed up the user interaction.

In accordance with the above description, this invention discloses a user interface display having an array of a plurality of electro -magnetic systems of a thin screen display unit, such as OLED screens or e-Ink screens. The use of a matrix of electro -magnets underneath a flat surface to move an object with a permanent magnet has been demonstrated by Pangaro et al. at MIT Media Lab in The Actuated Workbench: Computer- Controlled Actuation in Tabletop Tangible Interfaces, published in the proceedings of UIST'02, ACM. The display module of the present invention comprises: a display circuit coupled to a non-cathode display; an electromagnetic system located next to the display module; and a control module coupled to the display module and the electromagnetic system, wherein at least one of the plurality of electro magnets selectively attracts or repels the pointing instrument as a flow of current applied thereto is changed. Further, at least one of the plurality of electro magnets is selectively switched so that the pointing instrument may be vibrated due to alternating the current flow.

The foregoing aspects and many of the attendant advantages of the present invention will become more readily appreciated as the same becomes better understood by reference to the following detailed description, when taken in conjunction with the accompanying drawings, wherein:

FIG. 1 shows a cross-sectional view of a user interface device with the electromagnetic module in accordance with an embodiment of the present invention;

FIG. 2 shows a simplified block diagram illustrating a user interface device in accordance with the present invention;

FIG. 3 illustrates a passive guiding mode (scroll bar operation) of the user interface in accordance with the present invention;

FIG. 4 illustrates a passive guiding mode (menu mode) of the user interface in accordance with the present invention;

FIG. 5 illustrates an exemplary operation (active guiding mode) of the user interface in accordance with the present invention;

FIG. 6 illustrates an exemplary operation (vibration mode) of the user interface in accordance with the present invention; and

FIG. 7 illustrates an exemplary operation (pawn vibration mode) of the user interface in accordance with present the invention.

FIG. 1 illustrates a user interface 10 including a display device 12 (e.g., a touchscreen, a digital pad, a flat monitor display, and a portable interactive display device, etc.) that is operatively coupled to a computer device. The computing device is operational as any one of a number of various computing devices such as a PC, a server, a thin client, a thick client, a hand-held PDA, a portable interactive display, a personal communication assistant (PCA), an electronic organizer, a digital tablet, a laptop device, a multiprocessor system, a set-top box, programmable consumer electronics, a wireless phone, an application-specific integrated circuit (ASIC), and so on. The user interface 10 further includes a grid of small electro magnets 14 and a stylus or pointing instrument 16. The pointing instrument 16 may be a conventional stylus with a magnetic tip for use in a pen- based data entry system and can be made of a ferromagnetic material, a permanent magnet, or an electro magnet depending on the desired interaction.

In operation, each magnet 14 can be switched on individually as a magnetic north pole or south pole, or it can be switched off. If the electro magnet 14 is switched on, the attracting/repelling force will continue until the magnet is switched off. If the magnet is switched on/off fast, then a vibration can be sensed in the stylus 16 by the user. Thus, the magnets 14 are used to attract, repel or vibrate the stylus or other objects that interact with the screen or surface.

FIG. 2 illustrates a schematic diagram of the basic components of the user interface device 10 of the present invention. As shown, the user interface 10 includes a flat screen 20, an electro -magnet module 22, a polarity module 24, a stylus module 26, a touch driver 28, a system memory 30, and a controller 32. Note that if a stylus is used, its position can be sensed by the magnets sensing the presence of the stylus tip.

The screen 20 may be a conventional thin screen, such as OLED screens, e-Ink screens, and any LED-based backlighting device to display characters and/or images by adjusting the amount of transmitted portions of light from a light source (not shown). Hence, OLED, or e-ink displays are most suitable as they are not affected by magnets and magnetic fields. An array of a plurality of electro magnet module 22 is disposed under the screen 20 of display. The electro -magnetic module 22 further comprises a polarity control circuit 24 to selectively control the polarity of any one of the electro magnets provided under the screen 20 and to turn on/off the electro magnets.

To provide a user interface, the stylus module 26 displays the user inputs and invokes other tasks or modes of operation. When the stylus 16 with the magnetic tip moves over the grid of electro magnets 14, it will induce small currents to determine the x, y position of the tip of the stylus on or over the screen 12. Thus, the stylus module 26 provides a quick and simple way for the user to interact with an underlying computer program's UI (e.g., provide keyboard-like input and/or mouse-like input). To achieve various functionalities, the stylus module 26 intercepts, analyzes, and directs screen 20 events (e.g., stylus press and movement events). Therefore, the stylus module 26 provides a UI for stylus-based input for a user to easily implement: (a) keyboard-like input and/or mouse-like functionality with a stylus; (b) selection of another program's underlying UI controls; (c) right-mouse clicks (e.g., to obtain context-sensitive help or menus); (d) cursor or "pointer" movement (i.e., cursor hovering) over another program's underlying UI components; (e) simple and quick access to software applications, operational modes, etc.

The touch driver module 28 serves to detect and translate user input from the display device 12 based on the magnetic movement, then translates and communicates the user input to a program such as the stylus module 26. When the touch driver 28 receives stylus or pen -based user input from the display device 12 (e.g., the touch screen 20), the touch driver 28 communicates the corresponding event(s) to the stylus module 26. Such touch driver 28 events include interactions between a stylus 16 and the display 12 (e.g., stylus/screen touch-events, stylus lifting events, stylus drag events, and so on).

The controller 32 is coupled to the system memory 30. The system memory 30 includes any combination of volatile and non-volatile computer-readable media for reading and writing. Volatile computer-readable media includes, for example, random access memory (RAM). Non-volatile computer-readable media includes, for example, read-only memory (ROM), magnetic media such as a hard-disk, an optical disk drive, a floppy diskette, a flash memory card, a CD-ROM, and so on. The controller 32 is configured to fetch and execute computer -program instructions from a program module (not shown) and configured to fetch data while executing one or more of the program modules. Program modules typically include routines, programs, objects, components, data structures, and so on, for performing particular tasks or implementing abstract data types.

Hereinafter, the present invention will be described by way of examples, with reference to FIGs. 3 through 7. FIG. 3 is an example of a passive guiding that helps the user during operation by guiding the stylus 16 along a downward direction (e.g., over a scrollbar).

FIG. 4 shows another passive guiding that helps the user in interaction by guiding the stylus over a certain area (e.g., a menu tree). Since the user feels a force that attracts the stylus 16 to the area with which he or she interacts at that time, the attraction force assists the user to keep the stylus 16 on the screen surface during dragging or scrolling actions, thus reducing errors and speeding up interaction.

FIG. 5 shows the stylus 16 guided over the screen by moving the attraction point for the stylus 16 over the grid. As is shown in Steps 1 to 4, the stylus has moved up one position by altering the electro magnet's magnetic force at specified times. To instantiate a program or a given operation mode represented by an icon on the screen 12, the user can move the stylus 16 from location towards an icon (indicating a desired task) without removing the stylus 16 from the screen 12. Upon reaching a point on the screen 12 that overlies or intersects one of the icons, the user lifts the stylus 16 to perform the action or enter the mode of operation indicated by the underlying icon. Again, in this mode of operation, the attracting and repelling forces between the stylus 16 and its nearby electro magnets assists the user to steer the stylus 16 in the desired direction.

FIG. 7 shows a method to provide a haptic feedback in the form of vibrations. The polarity of electro magnets is continuously switched to attract the stylus 16 and repel the stylus 16 in such a way that the user senses a vibration when holding the stylus 16. This can be used to provide a feedback in case of a button click, an illegal versus correct selection, etc. Note that these forms of haptic feedback or feed forward can be specifically useful for people with a visual disorder or under bad visual conditions, e.g., little or no light, or glare.

Although the above description uses language that is specific to structural features and/or methodological acts, it is to be understood that the described arrangements and procedures defined in the appended claims are not limited to the specific features or acts described. Rather, the specific features and acts are disclosed as exemplary forms of implementing the described arrangements and procedures.

Claims

CLAIMS:

1. A user interface system comprising: a pointing instrument (16) having a magnetic tip that may be held in a hand of an operator; a display (12) for performing a display function; and an array of a plurality of electro magnets (14) disposed underneath the display (12), each of a polarity of the plurality of electro magnets (14) being controlled individually.

2. The user interface system of claim 1, wherein at least one of the plurality of electro magnets (14) is selectively attracted or repelled to the pointing instrument (16) as a flow of current applied thereto is changed.

3. The user interface system of claim 1, wherein at least one of the plurality of electro magnets (14) is selectively vibrated against the pointing instrument (16) by alternating a flow of current applied thereto.

4. The user interface system of claim 1, a control module coupled to the display (12) and the plurality of electro magnets(14) to individually control the operation of the display and the electro magnets.

5. The user interface system of claim 1, wherein the pointing instrument (16) performs a task of (a) generating handwritten data; (b) generating keyboard input; (c) moving a cursor over the display; and (d) accessing a software application or operation.

6. A method for providing a user interface for use with a stylus (16), the method comprising the steps of: providing an array of a plurality of electro magnets (14) under a display(12); detecting the one or more electro magnets (14) corresponding to one or more subsequent stylus-based user inputs; and selectively attracting or repelling the one or more electro magnets(14) corresponding to the detected stylus-based user inputs to the stylus.

7. The method of claim 6, wherein responsive to detecting the stylus-based user input, selectively switching at least one of the plurality of electro magnets (14) to vibrate the stylus (16).

8. The method of claim 6, further comprising the step of selectively turning on and off at least one of the plurality of electro magnets (14).

9. The method of claim 6, wherein the stylus(16) performs a task of (a) generating handwritten data; (b) generating keyboard input; (c) moving a cursor over the display(12); and (d) accessing a software application or operation.

10. An apparatus for providing a user interface, comprising: a stylus-based user input device(16) with a magnetic tip; a display unit(12) for projecting images; an array of a plurality of electro magnets(14) disposed under the display unit(12); and a polarity module(24) for changing the polarity of at least one of the plurality of electro magnets(14) to selectively attract and repel to the user input device(16).

11. The apparatus of claim 10, wherein at least one of the plurality of electro magnets (14) is vibrated against the input device(16) by alternating a flow of current applied thereto.

12. The apparatus of claim 10, further comprising a light source for illuminating the display unit (12).

13. The apparatus of claim 10, further comprising a control module (32) coupled to the display unit(12) and the plurality of electro magnets (14) to individually control the operation of the display unit (12) and the plurality of electro magnets(14).