WO2013044938A1

WO2013044938A1 - Method and system for providing a three-dimensional graphical user interface for display on a handheld device

Info

Publication number: WO2013044938A1
Application number: PCT/EP2011/004904
Authority: WO
Inventors: Jacob Johannes BOSMAM
Original assignee: Houthoff Buruma Cooperatief U.A.
Priority date: 2011-09-30
Filing date: 2011-09-30
Publication date: 2013-04-04

Abstract

A method is described for providing a graphical user interface for use in a touch-sensitive display area of a handheld device configured to receive user input via said display area, said handheld device being configured to generate a graphical user interface (GUI) on the basis of a curving function and GUI metadata defining a matrix of tiles. A tile may be being associated with a user-selectable item for display on said display area and configured to execute an associated action upon user-interaction. The matrix of tiles may define a first area which is substantially larger than said display area, said first area lying within a predetermined second area.

Description

METHOD AND SYSTEM FOR PROVIDING A THREE-DIMENSIONAL GRAPHICAL USER INTERFACE FOR DISPLAY ON A HANDHELD DEVICE

FIELD OF INVENTION

The disclosure generally relates to methods and systems for providing a graphical user interface having a three-dimensional appearance. In particular, the disclosure relates to methods and systems for generating a graphical user interface displayed having a positively curved appearance.

BACKGROUND

Handheld devices are often used for obtaining information, e.g., through an application to access media content. These handheld devices are often equipped with a touch-sensitive display. A user of the handheld device may use a finger as a pointer to interact with a graphical user interface (GUI) displayed on said touch-sensitive display to navigate such an application. Often, GUIs for these

applications are designed with the index finger in mind. For example, when browsing through a flat, vertical list of selectable items (e.g., a contact list), an index finger may be used to easily scroll through the list of selectable items using a swiping gesture from the top of the screen to the bottom of the screen. The index finger may also be used to easily tap and select one of the selectable items located anywhere on the screen. This type of GUI tends to work well when a user is holding the handheld device with one hand and using an index finger from the other, free hand to interact with the GUI. However, this type of GUI does not work well when a user would like to interact with the GUI using only a thumb of a hand that is holding the handheld device.

The range of motion of a thumb, especially of a thumb of a hand that is holding a handheld device, is limited when compared to the range of motion of an index finger, especially of an index finger of a free hand. Traditional graphical user interfaces such as a flat, vertical list of scrollable items does not work well with user input from a thumb. The anatomy of a thumb limits the thumb to making gestures generally associated with an arc. Therefore, the thumb of a hand holding a handheld device cannot easily provide a precise and straight swiping gesture to scroll through a list of

selectable items. Furthermore, the hand anchoring the thumb prevents the thumb from easily tapping anywhere on the touch- sensitive display of the handheld device, e.g., areas outside of the middle region of the display.

Accordingly, there is a need to provide an improved graphical user interface that at least alleviates some of these problems.

SUMMARY

Hereinafter, embodiments of the invention aiming to alleviate the problem(s) described above will be described in further detail. It should be appreciated, however, that these embodiments may not be construed as limiting the scope of protection for the present invention. For instance,

combinations of any of the embodiments and limitations are envisioned by the disclosure.

This disclosure relates to methods and systems for providing a graphical user interface having a plurality of tiles, wherein some of the tiles are part of a viewable region .

Tiles may be associated with a selectable item or a button for invoking a particular action. A tile is generally being modelled as a three-dimensional object having an

associated position and rotational angle (also known as pose) in a three-dimensional virtual space. Furthermore, a tile has associated content that is rendered for display on the basis of the associated position and rotational angle. As such, tiles are generally displayed with a suitable graphic, icon or text associated with that item or action. Those tiles within a viewable area defined over a matrix of the tiles are then rendered and displayed on the touch-sensitive display such that those displayed tiles have a positively curved, three- dimensional appearance.

Upon receiving user input, the user input having two spatial components (e.g., x-component, y-component) is then used to slide the tiles while maintaining the positively curved appearance for the tiles within the viewing area. As such, the graphical user interface gives a user experience that feels like a finger manipulating a rolling trackball. The position and angle of rotation of the tiles may be calculated on the basis of a curving/curved function to enable the rendering of tiles having the positively-curved appearance.

The graphical user interface provides an easy to use interface for a thumb of a hand holding the handheld device to navigate a plurality of tiles configured to slide based on the thumb user input. A graphical user interface reactive to user input having two components makes an arc-like gesture, which is a two-dimensional input gesture, visually and physically effective for a rolling graphical user interface. The

graphical user interface is able to appear to rotate along the arc that is made by the thumb, making the user interaction precise and effective, yet simple and intuitive to use.

A method for providing a graphical user interface for use in a touch-sensitive display area of a handheld device configured to receive user input via said display area is disclosed. Said handheld device being configured to generate a graphical user interface (GUI) on the basis of a curving function and GUI metadata defining a matrix of tiles. A tile may be associated with a user-selectable item for display on said display area and configured to execute an action upon user-interaction. Said matrix of tiles may define a first area which is substantially larger than said display area, said first area lying within a predetermined second area. A predetermined viewing area positioned within said first area is provided. A positively-curved GUI may be rendered on the basis of tiles located within said viewing area and said curved function. A dragging gesture from a user touching said display area is detected. The positions of the tiles in said matrix are moved in accordance with said dragging gesture. If a predetermined part of a tile associated with a first

peripheral column and/or row lies outside the second area, moving the position of the tiles in said peripheral first column and/or row to the opposite side of said matrix so to form a new peripheral column and/or row positioned within second area.

The disclosed embodiments will be further be

illustrated with reference to the attached drawings, which schematically show embodiments according to the invention. It will be understood that the invention is not in any way restricted to these specific embodiments.

BRIEF DESCRIPTION OF THE DRAWINGS

Aspects of the invention will be explained in greater detail by reference to exemplary embodiments shown in the drawings, in which:

FIG. 1 shows an exemplary system having a user device connected to a service provider over a network, according to one embodiment of the disclosure;

FIG. 2 shows an exemplary system having an application configured to implement some of the methods

described herein, according to one embodiment of the

disclosure;

FIG. 3A-3B show a flat surface and a positively curved surface;

FIG. 4 shows a cross section of an exemplary surface deformed by a round object, according to one embodiment of the disclosure ;

FIG. 5 shows an exemplary Cartesian coordinate system having three axes perpendicular to each other.

FIG. 6 shows an exemplary grid of tiles having a positively curved appearance, according to one embodiment of the disclosure;

FIG. 7A-7B show exemplary grids of tiles shifted to based on user input, according to some embodiments of the disclosure ;

FIG. 8A-8B show exemplary grids of tiles illustrating the repeating of tiles, according to one embodiment of the disclosure ;

FIG. 9 shows an exemplary repeating pattern for a grid of tiles, according to one embodiment of the disclosure;

FIG. 10A-10B shows grids of tiles having different positions, according to one embodiment of the disclosure;

FIG. 11 shows an exemplary data structure for a tile, according to one embodiment of the disclosure;

FIG. 12 shows an exemplary grid of tiles having tiles that repeat within the grid of tiles;

FIG. 13A-13B show exemplary screenshots illustrating the movement of tiles in response to user input, according to one embodiment of the disclosure;

FIG. 14A-14B show exemplary illustrations of an index finger and a thumb using the graphical user interface,

according to one embodiment of the disclosure;

FIG. 15 shows an illustrative method for providing the graphical user interface, according to one embodiment of the disclosure; and

FIG. 16 shows an illustrative method for providing the graphical user interface, according to one embodiment of the disclosure.

DETAILED DESCRIPTION

With some modifications, one skilled in the art may extend the embodiments described herein to other

architectures, networks, or technologies.

FIG. 1 shows an exemplary system having a user device connected to a service provider over a network, according to one embodiment of the disclosure. In one case, a user may use device 101 to access locally provided services.

Alternatively, a user may use device 102 to access, over network 104, a service that is provided by (remote) service provider 102. In general, device 101 includes several

components communicably connected via a bus, those components including input part 106, output part 108, processor 110,

storage 112, application 116, communication 114, operating system 118. Input part 116 may include at least one of: a touch-sensitive display, camera, microphone, touchpad,

keyboard, etc. Output part 108 may include any sensory output such as a display (e.g., a touch-sensitive display), a

speaker, a haptic output device, etc. Processor 110 and

operating system 118 configured to carry out processes for application 116, such as carrying out computations, accesses to (digital data) storage 112, etc. Communication 114 may include a communication device configured to allow device 101 to communicate with other devices or systems, e.g., via a Wifi or mobile network.

FIG. 2 shows an exemplary system having an graphical user interface (GUI) application configured to implement some of the methods described herein, according to one embodiment of the disclosure. The GUI application 116 comprises several components configured to display part of a matrix of so-called tiles on the screen of a user device. As will be described hereunder in more detail, a tile may to a user-selectable item (a button) for display on a screen of a user device and configured to invoke an action upon a user selecting the item. Each tile is associated with a data structure which may define the size and layout of the selectable item, coordinates associated with the position of the tile, the action to be invoked when a user interacts with the tile, etc.

Graphics generating function 210 in the user device may use tile data structures for generating the tile on the display. Tile data structures may be stored in a memory 204, e.g. a cache. Preferably, the tile data structures may be arranged in a two dimensional matrix, a GUI metadata matrix, wherein each matrix entry is associated with a tile data structure .

This way, GUI metadata matrix represents a matrix or grid of tiles which is used by graphics generating function 210 for rendering a tile-based GUI on the screen of the user device. The use of the GUI metadata matrix in generating a GUI will be described hereunder in more detail.

In one embodiment, a GUI metadata matrix represents a matrix (grid) of tiles T ,_j arranged in a N x M matrix wherein i,j ≥ 1 and i≤N and j≤M. Each entry Gi,_j in the GUI metadata matrix contains data associated with at least one tile _itj. The entry in the matrix may determine the position of a tile in a specific coordinate system.

Based on input received from input part 106, the user input is detected by input event listener 202. Data

corresponding to the user input may be filtered and provided to controller 206. At controller 206, a determination may be made with regards to what type of user input is received, and what action to execute in response to that type of user action. For instance, input event listener 202 may be

configured to detect a user-dragging gesture and a user- tapping gesture on a touch-sensitive display and to generate data associated with the respective gestures (e.g., a change in the x direction and/or the y direction in screen

coordinates) . These generated data which may be provided to controller 206 which controls the generation of the GUI on the basis of this data.

Based on the data associated with the detected gesture, GUI metadata matrix 204 may be updated. For example, if a user-dragging gesture representing a change in the x and/or y direction is detected, tile coordinates stored in the GUI metadata matrix may be updated.

On the basis of the updated GUI metadata matrix, graphics generating function 210 may render a GUI on the screen of a display of the user device on the basis of part of the tiles defined in the GUI metadata matrix. In particular, tiles in the GUI metadata matrix may be selected using a viewing window defined over the matrix of tiles.

Preferably, in one embodiment, selected tiles are used to generate a GUI which has a three-dimensional

appearance. To that end, the selected tiles are rendered and displayed on the basis of function defining a predetermined positively-curved surface (e.g. part of a sphere or spheroid). This process will be described hereunder in more detail with reference to Fig. 3-7. Accordingly, the associated content for the tiles is scaled, transformed and rendered to output 108 to achieve the positively curved appearance.

Furthermore, controller 206 may have ongoing control processes that enables tile(s) in the matrix to be updated with different content or properties. For example, controller 206 runs a process to allow new content to be used for a tile of the grid by requesting content manager 212 to provide different content for a particular tile. In another example, controller 206 may rearrange the tiles managed by grid 204 based on the properties of the tiles of the grid. In yet another example, controller 206 may change the property of the grid based on user inputs or other factors (e.g., position, flags, values, user profile) . Furthermore, other application 208 may be executed by controller 206, e.g., in response to user input. For instance, a user tapping on one of the displayed tiles may execute a web browser (other application 208) to view a particular web page associated with the tile.

Traditional GUIs typically have a substantially flat 2D appearance (e.g., interface for displaying a map, a list of selectable items) . Some three-dimensional GUIs have been proposed however the amount of information directly accessible by such GUI is limited to the surface of the 3D shape of the GUI. The present disclosure describes an improved graphical user interface which solves as least part of the problems associated with these known GUIs.

As described above, the GUI metadata matrix comprising tile data for generating a tile on a screen of a display represents a two dimensional arrangement of tiles.

FIG. 3A depicts a schematic representation a flat matrix of tiles. Part of this flat matrix may be selected and displayed on the basis of a positively curved surface. Fig. 3B depicts such positively curved surface 320.

Such surface may be generated on the basis of part of a curved object, e.g. a sphere or spheroid as illustrated in Fig. 4. This figure illustrates a cross section of a surface deformed on the basis of part of a spheroid function,

according to one embodiment of the disclosure. Hence, on the basis of a curving function (sphere or spheroid function) tiles appearing to have a position along a positively-curved surface may be generated. For instance, tiles of a grid lying on the positively curved surface having x, y, z coordinates representing their position in three-dimensional space may be transformed on the basis of a curved function.

The curving function may be applied to the z coordinate position, for example, to update the z coordinate position forwards or backwards along the curved function. As a result, the grid of tiles are positioned in such a way to provide the positively-curved appearance. A suitable

Cartesian coordinate system having three axes perpendicular to each other such that x, y, z coordinates as depicted in Fig. 5 may be used for the position of a particular tile in three- dimensional space (other suitable coordinate systems such as polar coordinates may also be used) .

FIG. 6 depicts the formation of a viewing window having a positively curved appearance on the basis of a matrix of tiles, according to one embodiment of the disclosure. In particular, Fig. 6 depicts a matrix of tiles T_irj 602 covering a two dimensional area S 604 of certain dimensions determined by the individual size of the selectable items. As the matrix typically comprises many tiles, the dimensions of area S is (much) larger than the dimensions of the display screen of the user device. Thus, when displaying tiles on the basis of the data in matrix T±,j, only a part of the tiles are selected, processed and displayed in a predetermined spatial arrangement in the screen of the display.

Preferably, the tiles are displayed in accordance with a positively curved surface, preferably a dome-shaped surface. Hence, tiles close to the center point of viewable window 606 (which would appear closest to the user having a normal vector substantially pointing straight towards the user and substantially in parallel with the normal vector of the display) . Tiles farther away from the center point of the display would appear to be farther away and have a rotational angle pointing farther away from the user. This way, the tiles presented within the viewing window provide the

appearance of a trackball. The rotational angle of a tile may be defined by the angle between the normal vector of the display screen and the normal vector of the tile. The angle affects the way how the tile is rendered (e.g., affecting how the content associated with the tile should be transformed based on the rotational angle) .

The curvature transforming the tiles in this embodiment is positioned at the center of the grid, and remains stationary as the matrix of tiles slides or moves in the left/right or up/down direction (e.g., in the direction of the x-axis and y-axis respectively) . Although in this figure the tiles are appear stretched as if it is part of a surface of a sphere, the tiles are preferably modeled and rendered without distortions, simply as flat tiles having a position along the curved surface.

As already briefly described with refence to Fig. 2, a graphic generation function uses the GUI metadata matrix to render tiles on the basis of viewable area 606. As such, tiles that are finally rendered for display are tiles within the view of viewable area 606. These tiles are arranged to have the positively curved appearance using a curved function, preferably a function representing at least part of a sphere or a spheroid. Tiles closer to the center of the display would appear closest to the user or display. Those tiles would also have a normal vector closer (more parallel) to the normal vector of the display such that those tiles faces substantially straight towards the user.

Hence, the method as described above provides the illusion of a rolling trackball interaction, without actually using a model of a three-dimensional trackball (having a round surface on all sides) . In some embodiments, the viewing area may be enlarged or reduced for a zooming feature or effect.

FIG . 7A-7B depicts a user interacting with the show exemplary grids of tiles shifted to based on user input, according to some embodiments of the disclosure.

In order for a user to navigate through all tiles defined by matrix T_irj, the touch-sensitive screen of the user device (as defined by the viewable area 704) may be arranged to receive user-input. The user input may relate to a dragging action have having at least one spatial components (e.g. a x and/or y component) . Upon receiving such user input, the graphics processing function may update the position (i.e. the coordinates) of the tiles in the matrix and display part of the updated tiles in the screen. Hence, the user-input

effectively moves the two dimensional arrangement of tiles in a certain direction so that new tiles in the display screen appear and other tiles disappear from the display screen.

In one embodiment input on a touch-sensitive display may be provided (per frame when refreshing the display) in screen coordinates. A user input event listener may be used to decipher whether the user input is part of a dragging gesture or is a tapping gesture on the touch screen. When a dragging gesture is detected, a delta x and delta y may be calculated based on a previous recorded x, y user input

position and a newly recorded x' , y' user input position. The delta x and delta y may be calculated by the event listener or controller. For instance, delta x = x' -x and delta y = y'-y. In some embodiments, the user input coordinates are scaled by a certain factor such that the user input (screen) coordinates are transformed into the coordinate system of the three- dimensional space used for modeling the grid of tiles.

The delta x and delta y is then applied to the position of each tile, e.g., through an iterative process in controller 206, such that each tile is shifted in the same direction as the delta x and delta y. For instance, for a tile with an x and y position, xl and yl, the shifted tile would have a new x and y position, xl' = xl + delta x, and yl' = yl + delta y. Accordingly, the movement of the matrix of tiles is responsive to user input in two directions.

Grid depicted in FIG. 7A has been shifted upwards and leftwards. Grid depicted in FIG. 7B has been shifted

downwards and rightwards. In practice, the refresh rate is high enough that there is not a lot of time between each frame. As such, delta x and delta y are likely to be

relatively small. Updating the x, and y coordinates of the tiles of a grid on the basis of delta x and delta y is related to method part 1620 of FIG. 16.

As the user is sliding the tiles as if using a virtual trackball, the grid preferably populates a viewing area of the virtual trackball infinitely with tiles looping on and on (i.e., a virtual track ball can roll forever) .

However, the grid as seen in these figures are limited in size. A grid could potentially move far enough such the viewing area runs out of tiles to display.

To avoid this problem, the position calculated for each tile based on delta x and delta y is used as a test position to test and see if it has fallen out of a predetermined boundary. If the position falls out of the

boundary, an update/looping process is initiated wherein a tile is moved onto the opposite side of the grid to ensure that the tile is fed back into the grid. This ensures that tiles always presented in the viewing area (or that there are always tile within the viewing area defined over the matrix of tiles ) .

Hence, in order to deal with the finite extent of the two dimensional arrangement of tiles, an update function is implemented. The update function defines boundaries Bi-₄

defining the spatial extend of the area associated with the matrix of tiles. Boundaries Bi_₄ may be defined on the basis of the coordinates of the tiles at the periphery of the tile matrix. In a normal position, the coordinates of the

peripheral tiles lie within the area defined by boundaries. Then, when user input changes the coordinates of the tiles, including the peripheral tiles, the update function checks whether a predetermined part of a peripheral tile has crossed the boundary. If this is the case, the update function, updates the arrangement of tiles by moving a first peripheral row of matrix Ti,_j to a second peripheral row or moving a first peripheral column of matrix T_i#j to a second peripheral column. This update process is repeated for each user input so that that the coordinates of all tiles stay within boundaries B.

FIG. 8A-8B depicts the process of testing and updating a matrix of tiles according to one embodiment of the disclosure. In the example depicted in the figures a matrix 804 of 24 tiles (4x6 configuration) is positioned within boundaries Bi_₄ 802i_₄. Peripheral columns 806i,₂ of the matrix are bordering boundaries 802_1>2 and peripheral rows 808i,₂ of the matrix are bordering boundaries 802₃,₄. One skilled in the art understands that the principle described with reference to these figures may be easily extended to the general situation of a matrix of NxM tiles wherein i,j ≥ 1 and i N and j≤M..

A boundary 802i_₄ may be defined by a range (e.g., pair of coordinates) corresponding to the x axis and another range (e.g., pair of coordinates) corresponding to the y axis. Further, boundaries 802i_ may be of substantially the same size as the size of area S defined by the matrix of tiles. After a test position has been calculated for a tile in a peripheral row and/or column, a check may be made to determine whether the tile has a position beyond one of the boundaries.

For instance, a check on the x-component (x-axis points left/right) for the test position of tile 6 in

peripheral column 8Ο6₂ of the matrix of tiles reveals that tile 6 is beyond the x boundary of boundary it has been determined that tile 6 has moved beyond boundary 802. Accordingly, tile 6 is moved by the width (the length going along the x-axis) leftwards in the opposite direction as the direction of the delta x by the whole width of the matrix of tiles or the width of boundary 802 to a location in the opposite peripheral column 8O6₁. This process is repeated for other tiles, as illustrated in FIG. 9, to provide virtually an infinite grid of tiles.

An exemplary implementation for providing a virtually infinite grid of tiles based on a grid of a fixed size is shown below:

//tx and ty are the actual coordinates in 3d space after adding the movement of the grid, also known as test x

tx = x + dx //x of a tile + delta x

ty = y + dy; //y of a tile + delta y

//flip the tile to the other side if the boundaries are crossed

if (tx < -boundary .boundX) {

x += 2 * boundar .boundX; //move tile the entire width of the grid to the right

} else if (tx > boundar .boundX) {

x -= 2 * boundary.boundX; //move tile the entire width of the grid to the left

}

/ /do the same for y as for x

if (ty < -boundary.boundY) {

y += 2 * boundary.boundY;

} else if (ty > boundary.boundY) {

y -= 2 * boundary.boundY;

}

FIG. 10A-10B shows matrices of tiles having different positions, according to one embodiment of the disclosure. In FIG. 10A, grid 1020 defines a first matrix of tiles with viewing area 1010 comprising tiles 2, 3, 8 and 9. Then, a movement of the grid results in a second grid 1040 having viewing area 1030 comprising tiles 7, 18, 13 and 14. Upon receiving user input (e.g., moving the grid of tiles upwards and rightward as indicated by arrow 1025) , test positions are calculated, e.g., using the method described above, and if appropriate, tiles are moved by the width or length of the grid to ensure proper looping of tiles. For instance, y

positions of tiles 20, 21 and 22 have been moved by the height of the tiles downwards. The x position of tile 23 has been moved leftward by the width of the grid and the y position of tile 23 has been moved downward by the height of the grid.

In addition to moving the x and y coordinates of the tiles, the z coordinate and rotational angle of at least tiles in the viewing area, e.g., viewing area 1030, are updated such that the tiles are displayed properly with the positively curved appearance centered on the display (as part of method part 1630 of FIG. 16) . For instance, the z coordinate of those tiles in the viewing area are updated in accordance with a curving function (in the example below a calculation for a point on the surface of a sphere is used to affect the z coordinate of those tiles in viewing area 1030.

As such, a tile that is closest to the center of the screen would be moved to a position closer to the screen/user than a tile that is farther away from the center of the screen. A similar method is used to update the rotational angle of the tile, based on the geometry of the sphere.

Accordingly, the tiles rendered using the x, y, z coordinates, the rotational angle and the content for the tile (may

generally be known as the skin), e.g., by graphics generation 210 of FIG. 2, would have a positively curved appearance (as part of method part 1540 and 1640 of FIG. 15 and FIG. 16, respectively) . Note that to achieve readability, tiles are preferably rotated around the x and y axis and never the z axis. Rotation around the z axis results in tiles that are flipped up side down, making it hard for someone to read or recognize, are avoided. An exemplary implementation of the updating of z-component and the rotational angle for a tile is shown below.

//SPHERE SPECIFIC CODE, this is to morph the position according to the "shape underneath"

//calculate distance to center of the galaxy;

CGFloat distance = sqrtf ( powf (tx, 2. Of) + powf (ty,

2. Of) ) ;

CGFloat r2 = sqrt ( powf (self .galaxy . adius , 2. Of) - powf(ty, 2. Of) ); //at what Z coordinate should this tile render z = sqrtf( po f(r2, 2. Of) - powf(tx, 2. Of) ) ;

//place according to the Z location of the grid (this is static but could potentially move)

z = self . galaxy . centerZ + z ;

//calculate the rotations

if(z != O.Of) {

rotY = a anf ( (tx - self . galaxy . centerX) / (z - self .galaxy. centerZ) ) * (180 / M_PI)

rotX = -1 * atanf ( (ty - self . galaxy . centerY) / (z - self .galaxy. centerZ) ) * (180 / M_PI) ;

}else{

rotY = O.Of; rotX = O.Of;

}

Besides moving tiles around, users may also select a particular tile such that an associated action is executed. In one embodiment, there are two methods for enabling the selection of a tile that advantageously works well when a user is using a thumb of a hand holding the handheld device to select items. Upon detecting a tapping gesture on the screen, a user input event listener (e.g., listener 202 of FIG. 2) provides the x and y screen coordinates of the tap to

controller 206 of FIG. 2 for processing. In one embodiment, controller 206 iterates through at least tiles of the viewable area to determine which tile is the closest to the tap

coordinates. In one embodiment, scaling and application of the Pythagoras theorem may be used to transform the user input coordinates in x and y screen coordinates to coordinates of the three-dimensional model. The distance (closeness) of a tile to a tap may be determined by the shortest distance between the coordinates of the tile and the coordinates of a tap .

If the tap is close to the center of the screen, the tap automatically activates the tile that is the closest to that tap (and likely closest to the middle of the screen) . To determine whether the tap is close to the center of the 16 04904 screen, each coordinate of the user input tap may be compared with a threshold. The threshold generally defines a circular region around the center of the screen based on a radius, dependent on the size of the screen.

If the tap is inside the circular region, then the closest tile to the tap is selected and an action associated with the tile is executed. Preferably, a flag or property of the tile is updated to reflect the status of being selected. Also, the alpha channel (i.e., related to the brightness of the tile when rendered) or other visual properties related to that selected tile may be modified to allow the tile to appear selected or different from other non-selected tiles. If the tap is not close to the center of the screen, then an

animation is produced to create an appearance of rotation of the interface such that the tapped tile would move to a center position of the screen (e.g., graphic generation 210 of FIG. 2) .

If the user taps the tile again (e.g., via a second tapping gesture) once it has been animated to the center, the action associated with the tile is executed. Animation to have the appearance of the grid rotation to move the tapped tile to the center of the display may be performed by graphics generation 210. Other properties related to the selected status of the tile may also be updated accordingly. This feature advantageously enables a user to use the interface easily without accidentally selecting tiles that were not intended by the user. As described earlier on in this

disclosure, the thumb is not very precise and the range of pointing motion is limited. Tapping on the screen may be accidental, and such a feature of moving a side tile to the middle to await for further confirmation of selection

alleviates this problem.

FIG. 11 shows an exemplary data structure for a tile, according to one embodiment of the disclosure. The list of properties shown for, e.g., tile 1110 is merely illustrative and not exhaustive, and is not intended to be limiting because not all the properties are required. A tile's position within the grid may be identified by an index (1 value) or by its row and column value (2 values) . The tile may have an associated size (e.g., in pixels, "Width (px) " and "Height (px)") within the three-dimensional model and/or an associated size in screen resolution coordinate system. A tile has a position that defines the tile' s position within three-dimensional model ("Space Position X (px)", "Space Position Y (px) " and "Space Position Y (px)") . Optionally, the data structure may also store the screen x and y positions of the tile (which may be in a different coordinate system than the coordinate system used for the three-dimensional model of the grid) .

A tile may have associated text (e.g., text-string) or metadata that describes the tile. The associated text may be rendered for display as part of the tile. A tile may also have pointers to other sources such as a path for the image or icon to be displayed for that tile, a link for the web page to be shown when a tile has been selected, and/or a path to an action to be executed upon selection (e.g., activate other application 208 of PIG. 2) .

Another exemplary property of a tile may indicate whether the tile is active. An active tile may be selected by a user and such a flag may influence the rendering of the tile to reflect an active/selected status. Another property may be whether the tile is centered on the screen. Similarly, this property may affect the rendering or interaction of the tile if it is centered. For instance, once a tile has been

determined that it is near or at the center of the screen, a tapping gesture on that tile would (directly) execute the associated action with the tile. Some tiles may have a property indicating whether it is a home tile. An interactive element (e.g., graphical, audio, etc.) may be provided to the user to enable a quick return to a home tile. Accordingly, when a user input indicating a desire to return to the home tile is detected, the grid of tiles may be rendered with an animation (appearing like a rolling trackball) such that the home tile is moved towards a centered position on screen.

Some tiles may have a property to indicate whether a tile is a favorite tile. Similar to the home tile, a different

interactive element may be provided to the user to enable the user to quickly locate their favorite tile. Accordingly, when a user input indicating a desire to head to a favorite tile, the grid of tiles may be moved and rendered with an animation (appearing like a rolling trackball) such that the tile with the favorite flag is moved to the center of the screen.

Other properties may include fields for recording usage statistics of the tiles. For instance, the number of times visited (or selected) , the time stamp of the last visit (or selected) , the time stamp of the first visit, and

expiration time. These characteristics are indicative of usage history and habits of the user.

A tile of matrix of tile represented by GUI metadata matrix may be manipulated (e.g., by controller 206) such that the tiles of the grid are dynamic in their content and

arrangement. In other words, values of the properties of a tile may be updated in the GUI metadata matrix on the basis of factors such as: profile of the user, time, state of the graphical user interface, usage history of the tiles, and a property of the tile. These factors may affect the

arrangement of the tiles in the grid or properties of the tiles themselves. The content associated with the tiles of a grid or arrangement of the tiles in a grid may also be

randomized or dependent on a probabilistic function.

In one embodiment, user profile of the user may affect the text, media or action associated with the tile.

For instance, if a user has a user profile attribute

indicating that he/she is a French user, the text may be displayed in French. In another embodiment, the time (e.g., time of day, day of the week, date, season, year) may affect the text, media or action associated with the tile. For instance, the time of day such as morning time may change some tiles from being associated with entertainment programs to news programs. Frequently visited tiles on the basis of the usage statistics may be grouped or arranged closely together in the grid to provide easier and quicker access. Those tiles may also be configured to have a different visual appearance when displayed (e.g., color settings, bold type in fonts).

FIG. 12 shows an exemplary matrix of tiles having tiles that repeat within the matrix. In some embodiments, to improve access to tiles, tiles within a matrix may be repeated to provide a way for users to quickly find the tile he/she wants without having to search a far distance for a tile that he/she just missed. As illustrated, the matrix shown in FIG. 12 where tile 8 is repeated at index 11, 20 and 23. Tile 3 is repeated at index 6, 15 and 18, and so on. A strict pattern of repeating the tiles in a grid is not required.

FIG. 13A-13B show exemplary graphical user interfaces according an embodiment of the disclosure. In particular, Fig. 13A-13B depict a track ball graphical user interface adapted for efficient navigation on the basis of one-hand thumb

gestures and illustrate the movement of tiles in response to user input, according to one embodiment of the disclosure. In this particular embodiment, the graphical user interface is generated by a GUI API running on a mobile device. The mobile device comprises a touch-screen graphical display defining a viewing window displaying at least part of a grid of tiles located on a positively curved surface, in this particular example a sphere.

The tiles of the viewing window may be displayed with a tile centered in the middle of the display screen. As illustrated, the tiles are rendered with a positively curved appearance. The tile in the center of the display screen appears the closest and has a normal vector pointing directly to the user. Tiles farther away from the center of the

display screen are scaled, transformed and rotated on the basis of the curving function such as a sphere.

FIG. 13B depicts the graphical user interface, which is rendered in response to user input dragging the tiles in the x and y direction and accordingly the x, y position of the tiles are updated. Based on the updated x, y position, the angle of rotation as well as the z position is updated

according to the curved function. As a result, the graphical user interface having the grid of tiles appears to have a rolling behavior of a trackball.

FIG. 14A-14B show exemplary illustrations of user interactions, in particular an index finger and a thumb, with the graphical user interface according to one embodiment of the disclosure. One of the advantages of the disclosed

embodiments is that the graphical user interface's response to two-dimensional user input (i.e., the ability to rotate in both x and y direction) provides an intuitive way to navigate a menu of selectable items using a thumb of a hand that is holding the user device. As seen in FIG. 14A, an index finger of a free hand has a wide range of motion that allows various gestures to be made with ease. For example, an index finger can draw straight strokes as gestures without strain and with precision. However, as seen in FIG. 15B, when holding the device in one hand, the thumb cannot reach to a far corner of a screen to select an item easily or with precision (e.g., hand is not usually capable of reaching the thumb as well as holding the user device firmly) . Also, the range of motion for making gestures on the touch-screen of the user device is limited to an arc or small motions. The improved graphical user interface alleviates these problems by providing a trackball interaction a menu of selectable items/tiles using the thumb of the hand that holds the device ("one hand, one thumb navigation"). In this particular embodiment, the tiles are of a size that allows easy navigation with the thumb.

If a user moves its thumb in a x and y direction of the touch screen, the graphical user interface reacts by moving the tiles in the x and y direction, thereby providing an intuitive interface even for arc-like movements of the thumb. Furthermore, the feature of first tapping a tile outside a center region of the display and in response moving the grid of tiles such that the tapped tile is centered prevents users from accidentally selecting a tile and invoking the associated action with the tile. A second tap is then used to confirm a selection if desired, and the associated action is then finally executed.

As shown in Fig. 14B, the graphical user interface may also comprise one or more tab buttons. These buttons may be used to alter the state of the displayed grid of tiles. In one embodiment, a first (home) button may be configured to return the grid of tiles to a home state, e.g. the state wherein a home tile is displayed as the center tile.

In another embodiment, a second (capture) button may be configured to generate a tile data structure, which is added to the grid of tiles or replacing a tile in the grid of tiles. For example, a user may decide enter a certain website into the grid of tile. In that case, it may tap the second button while the selected website is displayed and in return the GUI API may generate a tile data structure as described with reference to Fig. 11, and add the generated tile to the grid of tiles. Hence, the capture button allows the user to dynamically update the tiles in the grid.

FIG. 15 shows an illustrative method for providing the graphical user interface, according to one embodiment of the disclosure. At method part 1510, user input is received, e.g., through input 106 of FIG. 1. User input may be

represented in a two spatial coordinates (e.g., x, y screen coordinates) . The user input is preferably provided to input event listener 202 of FIG. 2 for detecting a user gesture.

For example, a dragging gesture of a thumb touching and continuing to touch a touch-sensitive display (e.g., input 106 and output 108) in a dragging motion.

In response to receiving the user input or detection of the user gesture, the matrix or grid of tiles is updated (method part 1520) . The process of updating the matrix or grid may include updating the position (e.g., x and y

component for position of the tile in the metadata associated with a particular tile) of the tiles based on the received user input or gesture. The updated position of the tile is tested accordingly to determine whether it is within the boundary of the space. Tiles beyond the boundary of the space is moved to the opposite side of the matrix/grid such that tiles of the grid loop from one edge to the other edge of the matrix/grid of tiles.

Furthermore, the updating process may include

updating the position (e.g., z-position) and angle of rotation based on a curving function to create a positively-curved appearance once tile within a viewing window is rendered.

Optionally, the metadata associated with tiles in the matrix may be updated (method part 1530) depending on at least one factor. For instance, media content associated with the tile may be updated on the basis of a randomized function.

Once the metadata of the tiles (e.g., position, rotational angle, media content) is updated, at least part of the metadata is provided to a graphic generation function to generate a graphical user interface having the tiles within a particular viewing window.

FIG. 16 shows an illustrative method for providing the graphical user interface, according to one embodiment of the disclosure. Method part 1610 is implemented in a similar fashion as method part 1510 of FIG. 15. Upon receiving user input, two method parts 1620 and 1630 may be implemented (in parallel or in any order) . Using mechanisms described herein, the x and y component of a tile position is updated on the basis of the user input at method part 1620. Tiles beyond the boundary of the space is moved to the opposite side of the matrix/grid such that tiles of the grid loop from one edge to the other edge of the matrix/grid of tiles. Furthermore, the z component of the tile and the rotational angle of the tile are updated on the basis of a curving function (method part 1630) .

Depending on a viewing area that is defined over the matrix, certain tiles are viewable to the user and is rendered for display (method part 1640) . As such, those tiles in the viewing area are rendered on the basis of the position, rotational angle, associated content, or any other suitable metadata for display (e.g., on output 108).

One embodiment of the disclosure may be implemented as a program product for use with a computer system. The program (s) of the program product define functions of the embodiments (including the methods described herein) and can be contained on a variety of computer-readable storage media. The computer-readable storage media can be a non-transitory storage medium. Illustrative computer-readable storage media include, but are not limited to: (i) non-writable storage media (e.g., read-only memory devices within a computer such as CD-ROM disks readable by a CD-ROM drive, ROM chips or any type of solid-state non-volatile semiconductor memory) on which information is permanently stored; and (ii) writable storage media (e.g., floppy disks within a diskette drive or hard-disk drive or any type of solid-state random-access semiconductor memory, flash memory) on which alterable

information is stored.

It is to be understood that any feature described in relation to any one embodiment may be used alone, or in

combination with other features described, and may also be used in combination with one or more features of any other of the embodiments, or any combination of any other of the

embodiments. Moreover, the invention is not limited to the embodiments described above, which may be varied within the scope of the accompanying claims.

Claims

1. A method for providing a graphical user interface for use in a touch-sensitive display area of a handheld device configured to receive user input via said display area, said handheld device being configured to generate a graphical user interface (GUI) on the basis of a curving function and GUI metadata defining a matrix of tiles, a tile being associated with a user-selectable item for display on said display area and configured to execute an associated action upon user-interaction,

wherein said matrix of tiles define a first area which is substantially larger than said display area, said first area lying within a predetermined second area, said method comprising:

providing a predetermined viewing area positioned within said first area;

rendering a positively-curved GUI on the basis of tiles located within said viewing area and said curved function;

detecting a dragging gesture from a user touching said display area;

moving the positions of the tiles in said matrix in accordance with said dragging gesture;

if a predetermined part of a tile associated with a first peripheral column and/or row lies outside the second area, moving the position of the tiles in said peripheral first column and/or row to the opposite side of said matrix so to form a new peripheral column and/or row positioned within second area.

2. The method according to claim 1, further comprising :

in response to said moving of said tiles in said matrix, determining the tiles within said viewing area;

updating said positively-curved GUI on the basis of said tiles located within said viewing area and said curved function .

3. The method according to claim 1, wherein said curving function defines a positively-curved surface having at least part of a sphere or spheroid.

4. The method according to any of claims 1-3, wherein rendering the positively-curved GUI includes:

scaling, transforming and rotating a selectable item in accordance with said curving function.

5. The method according to any of claims 1-4, wherein at least one tile substantially close the center of said viewing area is displayed such that that normal vector associated with said tile is pointing substantially to said user .

6. The method according to any of claims 1-5, wherein a tile is associated with at least one of a text string, an image, and rendering the positively curved GUI on the basis of the image and/or the text string.

7. The method according to claim 6, said method further comprises:

updating at least one of the text string, the image, and the action associated with the first tile on the basis of a function configured to determine a different text string, a different image and/or a different action for the first tile, said function determining the different text string, the different image and/or the different action on the basis of at least one of: profile of the user, time, state of the

graphical user interface, usage history of the tiles, and property of the tile.

8. The method according to claims 6 or 7, wherein a first tile and a second tile is associated with the same text string, the same image and/or same action.

9. The method according to any of claims 1-8, said method further comprising:

receiving a tapping gesture through said touch- sensitive display area;

determining a closest tile in said viewing area having a shortest distance between the location of the tile and the location of the tapping gesture;

comparing a distance between the location of the tapping gesture and the center of the touch-sensitive display with a threshold value;

if the distance is less than the threshold value, executing the action associated with the closest tile;

if the distance is more than the threshold value, moving the grid of tiles based on the distance between the closest tile and the center of the touch-sensitive display, such that an animation of the closest tile moving towards the center of the touch-sensitive screen is rendered.

10. The method according to claim 9, wherein the threshold is a pixel value defining the radius of a circular region around the center point of the touch-sensitive display area, said pixel value is determined on the basis of the width and/or height of the touch-sensitive screen.

11. The method according to any one of the above claims, wherein detecting the dragging gesture comprises:

recording a first user input value corresponding to a first axis and a second user input value corresponding to a second axis perpendicular to the first axis, wherein said first user input value and said second user input value are coordinates associated with a first user input received

through the touch-sensitive display;

recording a third user input value corresponding to the first axis and a fourth user input value corresponding to the third axis, wherein said third user input value and said fourth user input value are coordinates associated with a second user input received through the touch-sensitive

display;

determining a first delta value on the basis of the first user input value and the third user input value;

determining a second delta value on the basis of the second user input value and the fourth user input value.

12. The method according to claim 11, wherein moving the position of the tiles comprises:

determining a first test location value by adding the first delta value to the first location value associated with the first tile corresponding to the first axis;

determining an second test location value by adding the first delta value to the second location value associated with the first tile corresponding to the second axis.

13. The method according to claim 12, wherein:

the second area comprises a first and second boundary values corresponding to the first axis and a third and fourth boundary values corresponding to the second axis, determining whether the test location is beyond the boundary of the space comprises :

comparing the first test location value with the first boundary value and the second boundary value;

comparing the second test location value with the third boundary value and the fourth boundary value.

14. The method according to claim 13, wherein, if the first test location value is beyond the first boundary value or the second boundary value, modifying the first location value by a first length of the first area corresponding to the first axis, in the direction that is opposite of the direction associated with the change in user input ;

if the second test location value is beyond the third boundary value or the fourth boundary value, modifying the second location value by a second length of the first area corresponding to the second axis, in the direction that is opposite of the direction associated with the change in user input .

15. The method according to any of the above claims, wherein the alpha brightness properties of each tile is

determined based on the distance of the each tile from the center of the round object.

16. A graphical user interface provided by the method according to any one of the above claims.

17. A user device running an application, said application configured to implement the method according any one of the above claims .

18. A computer program product, implemented on computer-readable non-transitory storage medium, the computer program product configured for, when run on a computer,

executing the method according to any one of claims 1-15.