WO2005026940A1 - Portable electronic device with man/machine interface integrating device movements, corresponding method and programme - Google Patents

Portable electronic device with man/machine interface integrating device movements, corresponding method and programme Download PDF

Info

Publication number
WO2005026940A1
WO2005026940A1 PCT/FR2004/002313 FR2004002313W WO2005026940A1 WO 2005026940 A1 WO2005026940 A1 WO 2005026940A1 FR 2004002313 W FR2004002313 W FR 2004002313W WO 2005026940 A1 WO2005026940 A1 WO 2005026940A1
Authority
WO
WIPO (PCT)
Prior art keywords
electronic device
portable electronic
camera
movement
movements
Prior art date
Application number
PCT/FR2004/002313
Other languages
French (fr)
Inventor
Vincent Brient
Original Assignee
Wavecom
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Wavecom filed Critical Wavecom
Publication of WO2005026940A1 publication Critical patent/WO2005026940A1/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/0304—Detection arrangements using opto-electronic means
    • G06F3/0317—Detection arrangements using opto-electronic means in co-operation with a patterned surface, e.g. absolute position or relative movement detection for an optical mouse or pen positioned with respect to a coded surface
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
    • G06F1/16—Constructional details or arrangements
    • G06F1/1613—Constructional details or arrangements for portable computers
    • G06F1/1626—Constructional details or arrangements for portable computers with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
    • G06F1/16—Constructional details or arrangements
    • G06F1/1613—Constructional details or arrangements for portable computers
    • G06F1/1633—Constructional details or arrangements of portable computers not specific to the type of enclosures covered by groups G06F1/1615 - G06F1/1626
    • G06F1/1684—Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675
    • G06F1/1686—Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675 the I/O peripheral being an integrated camera
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/63—Control of cameras or camera modules by using electronic viewfinders
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2200/00—Indexing scheme relating to G06F1/04 - G06F1/32
    • G06F2200/16—Indexing scheme relating to G06F1/16 - G06F1/18
    • G06F2200/163—Indexing scheme relating to constructional details of the computer
    • G06F2200/1637—Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer

Definitions

  • Portable electronic device with man / machine interface taking account of movements of the device, corresponding process and computer program.
  • the field of the invention is that of portable electronic devices, such as radiotelephones or personal assistants ("PDAs"). More specifically, the invention relates to the man / machine interface (HMI) for such devices.
  • the invention applies to electronic devices of small size, or which are not very easy to control with conventional interface means, at least for certain applications or commands.
  • the man / machine interface of such devices includes a keyboard and a screen. To simplify or optimize the input, we have sometimes proposed extensions to these keyboards in various forms (balls, "joysticks", ).
  • an objective of the invention is to provide an original human / machine interface technique, effective, ergonomic and / or fun.
  • the invention also aims to provide such a technique, which allows the development of new applications or functions, or the improvement of known applications or functions.
  • Another objective of the invention is to provide such a technique, which does not require complex means or treatments, and more precisely which exploits those available.
  • the invention also aims to provide new devices, such as radiotelephones, with new functions, at reasonable cost and complexity.
  • a portable electronic device equipped with a camera comprising means for detecting at least one movement in function of an analysis of at least two successive images taken by said camera, and man-machine interface means associating at least one event with at least one of said movements.
  • events, commands or functions are associated with relative movements of the device. These movements are detected simply by analyzing successive images taken by a camera.
  • a camera is generally available on multimedia radiotelephones, as are motion detection means. Note that the movements taken into account are relative, in the sense that they can correspond to the movements of the device and / or of the filmed object (the user being able to move relative to the objective).
  • said detection means can determine a main motion vector between two successive images, said interface means analyzing said motion vector main and associating it with at least one event if it meets at least one predetermined criterion.
  • the one or more predetermined criteria preferentially belong to the group comprising the direction, the direction and the amplitude of said main motion vector.
  • said interface means compare at least one of said criteria to at least one threshold (for example a threshold, making it possible to eliminate small untimely movements), to decide whether said main movement vector corresponds or not to one of said events.
  • said interface means associate distinct events with said movements, according to distinct application contexts in which said device can be found.
  • Said application contexts can in particular belong to the group comprising: - different applications; - different hierarchical levels, in a given application; - specific sub-applications.
  • said device comprises at least one screen and means for displaying on said screen at least information representative of an event which will be carried out if a particular movement is carried out.
  • the device comprises at least one screen and means for displaying on said screen at least one item of information representative of an event carried out following a movement.
  • the approach of the invention makes it possible in particular to select an event proposed on the screen, and / or to interact with the screen, to modify what is displayed there.
  • said detection means implement a spatial correlation, according to which a first image is shifted in at least one direction, then compared to a second image.
  • said detection means advantageously calculate the sum of the squares of the differences of a predetermined magnitude for each pixel of said images.
  • said detection means comprise means for resizing images, to reduce the complexity of the images analyzed.
  • said detection means can take into account motion vectors calculated by an MPEG-4 encoder.
  • said detection means take into account movements determined by image stabilization means. The detected movements can be relative or absolute.
  • detection means implementing shape recognition means, to identify a main shape (for example the silhouette or the outline of the user's face), and means for analyzing the movements of said main shape.
  • the invention can in particular be implemented in devices comprising radiocommunication means, in particular radio telephones or PDAs.
  • the invention also relates to a man-machine interface method for a portable electronic device equipped with a camera, associating at least one event with at least one movement detected by analysis of at least two successive images taken by said camera.
  • the invention also relates to computer programs for a portable electronic device equipped with a camera, comprising instructions for associating at least one event with at least one movement detected by analysis of at least two successive images taken by said camera.
  • Such programs can be integrated from the start (for example for navigation in menus) and / or be developed by third parties, and for example downloaded (for example for games). Such programs do not necessarily include. the method of the invention (also implemented in the device), but take advantage of the novelty of the invention to provide new functionalities.
  • Figure 1 schematically illustrates a radiotelephone implementing the invention
  • - Figure 2 shows a user holding in hand the radiotelephone of Figure 1
  • - Figures 3A, 3B and 3C are three schematic views of shots taken with the device of Figure 2, when the user implements the principle of the invention
  • - Figure 4 is a simplified block diagram of the processing carried out in the context of the invention.
  • the embodiment described below relates to a multimedia type radiotelephone, equipped with a camera (or a camera).
  • FIG. 1 An example of such a radiotelephone is illustrated diagrammatically in FIG. 1.
  • the housing 11 conventionally carries a keyboard 12, comprising a reduced number of keys (generally a numeric keypad), a screen 13 and a lens 14, allowing the taking of still pictures and / or animated.
  • this objective is mounted mobile relative to the housing 11, and for example mobile in rotation.
  • the general principle of the invention is to detect certain movements applied by the user to his telephone, and for example four movements of base, up, down, left or right (see arrows H, B, G, D shown in Figure 1).
  • images taken by the objective 14 are used to detect these movements. More precisely, the relative inclination of the telephone is deduced from the displacement of the image captured by the camera. We can then deduce relative movements of the telephone.
  • the user 21 holds his phone facing him, and looks at the screen 13.
  • the camera 14 films it, and the corresponding images are analyzed, to detect movements therein.
  • This approach is illustrated in Figures 3A to 3C.
  • FIG. 3A the telephone being directed towards the user, there is a first image. If the user moves or tilts their phone slightly to the left, the image will move to the right, as shown in Figure 3B. Likewise, if the user points their phone up, the resulting image will move down, as shown in Figure 3C.
  • the processing is therefore very simple, as illustrated diagrammatically in FIG. 4, and implements simple means and treatments, and often available, such as a camera and means for detecting movement. More specifically, the processing comprises: a step 41 for taking pictures, for example at a rate of 5 to 10 images per second.
  • these images can be resized to a reduced size, and / or subsampled; a movement detection step 42, consisting for example of calculating a main movement vector in the image.
  • a spatial correlation consisting of shifting two images in two dimensions (or more generally in the directions associated with a possible event), and detecting the position which comes closest to the superposition.
  • the correlation calculation for each displacement of the image consists for example in calculating the sum of the squares of the differences of each pixel; - An association of the detected movement with a predetermined command 43, which can for example use a simple table.
  • This step can take into account one or more thresholds, for example to avoid taking into account small untimely displacements, or to graduate the displacements according to their amplitude.
  • the detection can be digital, or logical: existence or not of a movement in a direction, or analog, that is to say progressive, or proportional. It is then possible to obtain also proportional commands, for example for the movement of an object on the screen: the movement of the latter can take account of the amplitude and / or the speed of movement of the radiotelephone; a step of carrying out the command 44, in the same way as it would be done with an event obtained using another interface.
  • the power necessary to perform these operations, and in particular the detection of movement, is a priori acceptable, in particular for multimedia processors of mobile telephony.
  • the amount of calculations is of the order of 180 Mips (5 images / sx (30x30) displacements x ( 100x100) pixels x 4 operations / pixel).
  • the cameras generally having a resolution of at least 640 x 480 pixels.
  • image stabilizer integrated in the multimedia processor such an image stabilizer is for example available in the processors produced by the company Emblaze (registered trademark)).
  • This stabilizer detects the overall movement of the image, to compensate for the movements of the camera.
  • These means can easily be modified to obtain an overall movement of the image, which can be used in the algorithm illustrated in FIG. 4. It is also possible to envisage using an optimized algorithm, such as an MPEG4 encoder, already present. and already in the interface of multimedia phones. The latter offers motion detection in macroblocks (squares of 8 x 8 pixels), which could be used to determine a global movement. According to the invention, only sufficiently large movements should preferably be taken into account, to avoid any untimely control, due to an involuntary movement.
  • the detected movement can be absolute or relative.
  • An upward movement can be associated with a control of validation, and a movement down to a cancellation command, or back to the previous step.
  • the interface can also be used for games. For example, in the context of a racing game, movements to the left and to the right make it possible to move a vehicle to the left or to the right, a movement to the right forwards makes it possible to obtain a acceleration, and a backward motion slowing down.

Abstract

The invention concerns a portable electronic device equipped with a camera (14) comprising means for detecting at least one movement (H, B, G, D) based on the analysis of at least two successive images captured by said camera, and man/machine interface means associating at least one event with at least one of said movements.

Description

Dispositif électronique portable à interface homme/machine tenant compte de mouvements du dispositif, procédé et programme informatique correspondants. Le domaine de l'invention est celui des dispositifs portables électroniques, tels que les radiotéléphones ou les assistants personnels (« PDA »). Plus précisément, l'invention concerne l'interface homme/machine (IHM) pour de tels dispositifs. Notamment, l'invention s'applique aux dispositifs électroniques de petite taille, ou qui sont peu aisés à contrôler avec les moyens d'interface classiques, au , moins pour certaines applications ou commandes. Généralement, l'interface homme/machine de tels dispositifs comprend un clavier et un écran. Pour simplifier ou optimiser la saisie, on a proposé parfois des extensions à ces claviers sous des formes diverses (boules, « joysticks », ...). On a également développé des moyens complémentaires et/ou alternatifs, tels que des commandes vocales, des écrans tactiles, etc. Dans certains cas, ces moyens connus sont cependant insuffisants. C'est par exemple le cas lorsque la surface disponible est réduite, et que le clavier est de petite taille, et peu aisé à manipuler. Le fait que le nombre de touches soit réduit peut également poser problème. Dans d'autres cas, une interface telle qu'un clavier ne permet pas de transmettre des informations rapides, de façon ergonomique. C'est par exemple le cas de commandes d'un jeu, pour interagir avec des images sur l'écran. Il existe encore d'autres situations où il est difficile de manipuler un clavier, par exemple lorsque l'appareil est tenu d'une seule main. Il y a alors un besoin d'une solution simple et efficace pour effectuer un certain nombre de commandes. Plus généralement, il existe un besoin de nouvelles approches d'interface homme/machine pour des raisons pratiques, ludiques ou ergonomiques. Cela permet notamment de définir de nouveaux produits, ou de nouvelles applications, et donc d'offrir des prestations nouvelles au public. L'invention a donc pour objectif de pallier ces inconvénients de l'art antérieur, et de fournir une nouvelle approche d'interface homme/machine. Plus précisément, un objectif de l'invention est de fournir une technique d'interface homme/machine originale, efficace, ergonomique et/ou ludique. L'invention a également pour objectif de fournir une telle technique, qui permette le développement de nouvelles applications ou fonctions, ou l'amélioration d'applications ou fonctions connues. Un autre objectif de l'invention est de fournir une telle technique, qui ne nécessite pas de moyens ou de traitements complexes, et plus précisément qui exploite ceux disponibles. L'invention a également pour objectif de fournir de nouveaux appareils, tels que des radiotéléphones, avec des fonctions nouvelles, à coût et complexité raisonnables. Ces objectifs, ainsi que d'autres qui apparaîtront plus clairement par la suite, sont atteints selon l'invention à l'aide d'un dispositif électronique portable équipé d'une caméra, comprenant des moyens de détection d'au moins un mouvement en fonction d'une analyse d'au moins deux images successives prises par ladite caméra, et des moyens d'interface homme-machine associant au moins un événement à au moins un desdits mouvements. Ainsi, selon l'invention, on associe des événements, des commandes ou fonctions à des mouvements relatifs du dispositif. Ces mouvements sont détectés simplement, en analysant des images successives prises par une caméra. Une telle caméra est généralement disponible sur les radiotéléphones multimédia, de même des moyens de détection de mouvement. On notera que les mouvements pris en compte sont relatifs, dans le sens où ils peuvent correspondre aux mouvements du dispositif et/ou de l'objet filmé (l'utilisateur pouvant se déplacer par rapport à l'objectif). Selon un mode de réalisation avantageux, lesdits moyens de détection peuvent déterminer un vecteur mouvement principal entre deux images successives, lesdits moyens d'interface analysant ledit vecteur mouvement principal et lui associant au moins un événement s'il répond à au moins un critère prédéterminé. Le ou lesdits critères prédéterminés appartiennent préférentiellement au groupe comprenant la direction, le sens et l'amplitude dudit vecteur mouvement principal. De façon avantageuse, lesdits moyens d'interface comparent au moins un desdits critères à au moins un seuil (par exemple un seuil, permettant d'éliminer de petits mouvements intempestifs), pour décider si ledit vecteur mouvement principal correspond ou non à un desdits événements. Préférentiellement, lesdits moyens d'interface associent des événements distincts auxdits mouvements, en fonction de contextes applicatifs distincts dans lesquels peut se trouver ledit dispositif. Lesdits contextes applicatifs peuvent notamment appartenir au groupe comprenant : - des applications différentes ; - des niveaux hiérarchiques différents, dans une application donnée ; - des sous-applications spécifiques. Avantageusement, ledit dispositif comprend au moins un écran et des moyens pour afficher sur ledit écran au moins une information représentative d'un événement qui sera réalisé si un mouvement particulier est effectué. Selon un autre aspect avantageux, le dispositif comprend au moins un écran et des moyens pour afficher sur ledit écran au moins une information représentative d'un événement réalisé suite à un mouvement. En d'autres termes, l'approche de l'invention permet notamment de sélectionner un événement proposé à l'écran, et/ou d'interagir avec l'écran, pour modifier ce qui y est affiché. Selon un mode de réalisation avantageux de l'invention, lesdits moyens de détection mettent en oeuvre une corrélation spatiale, selon laquelle une première image est décalée dans au moins une direction, puis comparée à une seconde image. Dans ce cas, lesdits moyens de détection calculent avantageusement la somme des carrés des différences d'une grandeur prédéterminée pour chaque pixel desdites images. Selon un autre aspect avantageux, lesdits moyens de détection comprennent des moyens de re-dimensionnement d'images, pour réduire la complexité des images analysées. Selon une approche avantageuse de l'invention, lesdits moyens de détection peuvent prendre en compte des vecteurs mouvement calculés par un encodeur MPEG-4. Selon encore une autre approche avantgeuse, lesdits moyens de détection prennent en compte des mouvements déterminés par des moyens de stabilisation d'images. Les mouvements détectés peuvent être relatifs ou absolus. Dans le cas d'un repérage absolu notamment, on peut avantageusement utiliser des moyens de détection mettant en œuvre des moyens de reconnaissance de forme, pour identifier une forme principale (par exemple la silhouette ou le contour du visage de l'utilisateur), et des moyens d'analyse des mouvements de ladite forme principale. L'invention peut notamment être mise en œuvre dans les dispositifs comprenant des moyens de radiocommunication, notamment des radio-téléphones ou des PDA. L'invention concerne également un procédé d'interface homme-machine pour dispositif électronique portable équipé d'une caméra, associant au moins un événement à au moins un mouvement détecté par analyse d'au moins deux images successives prises par ladite caméra. L'invention concerne encore les programmes informatiques pour dispositif électronique portable équipé d'une caméra, comprenant des instructions pour associer au moins un événement à au moins un mouvement détecté par analyse d'au moins deux images successives prises par ladite caméra. De tels programmes peuvent être intégrés dès l'origine (par exemple pour la navigation dans des menus) et/ou être développés par des tiers, et par exemple téléchargés (par exemple pour des jeux). De tels programmes n'incluent pas forcément. le procédé de l'invention (implanté par ailleurs dans le dispositif), mais tirent parti de la nouveauté de l'invention pour fournir des fonctionnalités nouvelles. D'autres caractéristiques et avantages de l'invention apparaîtront à la lecture de la description suivante d'un mode de réalisation préférentiel de l'invention, donné à titre de simple exemple illustratif et non limitatif, et des figures annexées parmi lesquelles : - la figure 1 illustre schématiquement un radiotéléphone mettant en œuvre l'invention ; - la figure 2 représente un utilisateur tenant en main le radiotéléphone de la figure 1 ; - les figures 3A, 3B et 3C sont trois vues schématiques de prises de vue effectuées avec le dispositif de la figure 2, lorsque l'utilisateur met en œuvre le principe de l'invention ; - la figure 4 est un schéma synoptique simplifié du traitement effectué dans le cadre de l'invention. Le mode de réalisation décrit par la suite concerne un radiotéléphone de type multimédia, équipé d'une caméra (ou d'un appareil photo). Bien entendu, l'approche de l'invention peut également s'appliquer à d'autres dispositifs portables, comme les assistants personnels, les dispositifs de stockage de fichiers audio et/ou vidéo, les consoles de jeux portables, ... Un exemple d'un tel radiotéléphone est illustré schématiquement en figure 1. Le boîtier 11 porte classiquement un clavier 12, comportant un nombre réduit de touches (généralement un pavé numérique), un écran 13 et un objectif 14, permettant la prise de vues fixes et/ou animées. Souvent, cet objectif est monté mobile par rapport au boîtier 11 , et par exemple mobile en rotation. Le principe général de l'invention est de détecter certains mouvements appliqués par l'utilisateur à son téléphone, et par exemple quatre mouvements de base, vers le haut, vers le bas, à gauche ou à droite (voir flèches H, B, G, D représentées sur la figure 1). Selon l'invention, on utilise des images prises par l'objectif 14 pour détecter ces mouvements. Plus précisément, l'inclinaison relative du téléphone est déduite du déplacement de l'image saisie par la caméra. On peut alors en déduire des mouvements relatifs du téléphone. Dans le mode de réalisation illustré schématiquement en figure 2, l'utilisateur 21 tient son téléphone face à lui, et regarde l'écran 13. La caméra 14 le filme, et les images correspondantes sont analysées, pour y détecter des mouvements. Cette approche est illustrée par les figures 3A à 3C. Sur la figure 3 A, le téléphone étant dirigé vers l'utilisateur, on dispose d'une première image. Si l'utilisateur déplace ou incline légèrement son téléphone vers la gauche, l'image se déplacera vers la droite, comme illustré en figure 3B. De même, si l'utilisateur oriente son téléphone vers le haut, l'image résultante se déplacera vers le bas, comme illustré en figure 3C. Il suffit donc de comparer les images 3A et 3B pour déduire que l'utilisateur a effectué un déplacement relatif du téléphone vers la gauche, et les images 3A et 3C pour détecter un mouvement vers le haut. Il suffit alors d'associer ces mouvements à des commandes, des actions ou événements prédéfinis pour définir une nouvelle interface homme/machine selon l'invention, consistant à associer un mouvement relatif à un événement. Le traitement est donc très simple, comme illustré schématiquement par la figure 4, et met en œuvre des moyens et des traitements simples, et souvent disponibles, comme une caméra et des moyens de détection du mouvement. Plus précisément, le traitement comprend : - une étape de prises de vues 41, par exemple à un rythme de 5 à 10 images par seconde. Pour simplifier les calculs, ces images peuvent être redimensionnées à une taille réduite, et/ou sous-échantillonnées ; - une étape de détection du mouvement 42, consistant par exemple à calculer un vecteur de mouvement principal dans l'image. On peut notamment mettre en œuvre une corrélation spatiale consistant à décaler deux images dans les deux dimensions (ou plus généralement dans les directions associées à un événement possible), et détecter la position qui se rapproche le plus de la superposition. Le calcul de corrélation pour chaque déplacement de l'image consiste par exemple à calculer la somme des carrés des différences de chaque pixel ; - une association du mouvement détecté à une commande prédéterminée 43, pouvant par exemple utiliser une simple table. Cette étape peut prendre en compte un ou plusieurs seuils, par exemple pour éviter la prise en compte de petits déplacements intempestifs, ou pour graduer les déplacements selon leur amplitude. La détection peut être de type numérique, ou logique : existence ou non d'un mouvement dans une direction, ou analogique, c'est-à-dire progressive, ou proportionnelle. On peut alors obtenir des commandes également proportionnelles, par exemple pour le déplacement d'un objet sur l'écran : le mouvement de ce dernier peut tenir compte de l'amplitude et/ou de la vitesse du déplacement du radiotéléphone ; - une étape de réalisation de la commande 44, de la même façon que cela serait fait avec un événement obtenu à l'aide d'une autre interface. La puissance nécessaire pour effectuer ces opérations, et notamment la détection du mouvement, est a priori acceptable, notamment pour des processeurs multimédias de téléphonie mobile. Par exemple, pour des images redimensionnées à 100 x 100 pixels, et des déplacements de l'objet correspondant à 30 pixels au maximum, la quantité de calculs est de l'ordre de 180 Mips (5 images/s x (30x30) déplacements x (100x100) pixels x 4 opérations/pixel). A ce besoin approximatif de puissance, il faut également ajouter un redimensionnement de l'image, les caméras ayant en général une résolution d'au moins 640 x 480 pixels. Il existe de nombreuses autres façons connues, plus simples et plus efficaces, de déterminer des mouvements dans l'image. En outre, avantageusement, on peut utiliser des moyens disponibles pour d'autres applications. Ainsi, on peut utiliser un stabilisateur d'image intégré dans le processeur multimédia (un tel stabilisateur d'image est par exemple disponible dans les processeurs produits par la société Emblaze (marque déposée)). Ce stabilisateur détecte le mouvement global de l'image, pour compenser les mouvements de la caméra. On pourra aisément modifier ces moyens, pour obtenir un mouvement global de l'image, qui pourra être exploité dans l'algorithme illustré en figure 4. On peut également envisager d'utiliser un algorithme optimisé, comme un encodeur MPEG4, présent d'ores et déjà dans l'interface des téléphones multimédia. Ce dernier propose une détection de mouvement dans des macroblocs (carrés de 8 x 8 pixels), que l'on pourrait utiliser pour déterminer un mouvement global. Selon l'invention, seuls des mouvements suffisamment importants devront préférentiellement être pris en compte, pour éviter toute commande intempestive, due à un mouvement involontaire. Le mouvement détecté peut être absolu ou relatif. Dans le cas d'un repérage absolu, on peut également prévoir une étape de détection de la silhouette (par reconnaissance de forme) de l'utilisateur (ou d'un autre objet unique), puis de détecter son mouvement. Dans le mode de réalisation décrit, notamment avec la figure 2, le mouvement est analysé par observation du visage de l'utilisateur. On comprend bien sûr que la même approche peut être mise en œuvre de façon similaire, si la caméra est orientée vers l'avant, par rapport à l'utilisateur. Cette nouvelle interface selon l'invention peut être utilisée de nombreuses manières, les développeurs imaginant facilement des applications. A titre d'exemple, elle peut permettre la navigation dans des menus, les mouvements vers la gauche ou vers la droite permettant la sélection d'une commande parmi deux, présentées sur l'écran, respectivement du côté gauche et du côté droit. Un mouvement vers le haut peut être associé à une commande de validation, et un mouvement vers le bas à une commande d'annulation, ou de retour à l'étape précédente. L'interface peut également être utilisée pour des jeux. Par exemple, dans le cadre d'un jeu relatif à une course automobile, les mouvements vers la gauche et vers la droite permettent de déplacer un véhicule à gauche ou à droite, un mouvement vers la droite vers l'avant permet d'obtenir une accélération, et un mouvement vers l'arrière un ralentissement. Portable electronic device with man / machine interface taking account of movements of the device, corresponding process and computer program. The field of the invention is that of portable electronic devices, such as radiotelephones or personal assistants ("PDAs"). More specifically, the invention relates to the man / machine interface (HMI) for such devices. In particular, the invention applies to electronic devices of small size, or which are not very easy to control with conventional interface means, at least for certain applications or commands. Generally, the man / machine interface of such devices includes a keyboard and a screen. To simplify or optimize the input, we have sometimes proposed extensions to these keyboards in various forms (balls, "joysticks", ...). We have also developed complementary and / or alternative means, such as voice commands, touch screens, etc. In certain cases, these known means are however insufficient. This is for example the case when the available surface is reduced, and the keyboard is small, and not very easy to handle. Reducing the number of keys can also be a problem. In other cases, an interface such as a keyboard does not allow fast, ergonomic information to be transmitted. This is for example the case of game commands, to interact with images on the screen. There are still other situations where it is difficult to handle a keyboard, for example when the device is held with one hand. There is then a need for a simple and effective solution for placing a certain number of orders. More generally, there is a need for new man / machine interface approaches for practical, fun or ergonomic reasons. This allows in particular to define new products, or new applications, and therefore to offer new services to the public. The invention therefore aims to overcome these drawbacks of the prior art, and to provide a new approach to man / machine interface. More specifically, an objective of the invention is to provide an original human / machine interface technique, effective, ergonomic and / or fun. The invention also aims to provide such a technique, which allows the development of new applications or functions, or the improvement of known applications or functions. Another objective of the invention is to provide such a technique, which does not require complex means or treatments, and more precisely which exploits those available. The invention also aims to provide new devices, such as radiotelephones, with new functions, at reasonable cost and complexity. These objectives, as well as others which will appear more clearly below, are achieved according to the invention using a portable electronic device equipped with a camera, comprising means for detecting at least one movement in function of an analysis of at least two successive images taken by said camera, and man-machine interface means associating at least one event with at least one of said movements. Thus, according to the invention, events, commands or functions are associated with relative movements of the device. These movements are detected simply by analyzing successive images taken by a camera. Such a camera is generally available on multimedia radiotelephones, as are motion detection means. Note that the movements taken into account are relative, in the sense that they can correspond to the movements of the device and / or of the filmed object (the user being able to move relative to the objective). According to an advantageous embodiment, said detection means can determine a main motion vector between two successive images, said interface means analyzing said motion vector main and associating it with at least one event if it meets at least one predetermined criterion. The one or more predetermined criteria preferentially belong to the group comprising the direction, the direction and the amplitude of said main motion vector. Advantageously, said interface means compare at least one of said criteria to at least one threshold (for example a threshold, making it possible to eliminate small untimely movements), to decide whether said main movement vector corresponds or not to one of said events. . Preferably, said interface means associate distinct events with said movements, according to distinct application contexts in which said device can be found. Said application contexts can in particular belong to the group comprising: - different applications; - different hierarchical levels, in a given application; - specific sub-applications. Advantageously, said device comprises at least one screen and means for displaying on said screen at least information representative of an event which will be carried out if a particular movement is carried out. According to another advantageous aspect, the device comprises at least one screen and means for displaying on said screen at least one item of information representative of an event carried out following a movement. In other words, the approach of the invention makes it possible in particular to select an event proposed on the screen, and / or to interact with the screen, to modify what is displayed there. According to an advantageous embodiment of the invention, said detection means implement a spatial correlation, according to which a first image is shifted in at least one direction, then compared to a second image. In this case, said detection means advantageously calculate the sum of the squares of the differences of a predetermined magnitude for each pixel of said images. According to another advantageous aspect, said detection means comprise means for resizing images, to reduce the complexity of the images analyzed. According to an advantageous approach of the invention, said detection means can take into account motion vectors calculated by an MPEG-4 encoder. According to yet another advantageous approach, said detection means take into account movements determined by image stabilization means. The detected movements can be relative or absolute. In the case of absolute tracking in particular, it is advantageously possible to use detection means implementing shape recognition means, to identify a main shape (for example the silhouette or the outline of the user's face), and means for analyzing the movements of said main shape. The invention can in particular be implemented in devices comprising radiocommunication means, in particular radio telephones or PDAs. The invention also relates to a man-machine interface method for a portable electronic device equipped with a camera, associating at least one event with at least one movement detected by analysis of at least two successive images taken by said camera. The invention also relates to computer programs for a portable electronic device equipped with a camera, comprising instructions for associating at least one event with at least one movement detected by analysis of at least two successive images taken by said camera. Such programs can be integrated from the start (for example for navigation in menus) and / or be developed by third parties, and for example downloaded (for example for games). Such programs do not necessarily include. the method of the invention (also implemented in the device), but take advantage of the novelty of the invention to provide new functionalities. Other characteristics and advantages of the invention will appear on reading the following description of a preferred embodiment of the invention, given by way of simple illustrative and nonlimiting example, and of the appended figures among which: Figure 1 schematically illustrates a radiotelephone implementing the invention; - Figure 2 shows a user holding in hand the radiotelephone of Figure 1; - Figures 3A, 3B and 3C are three schematic views of shots taken with the device of Figure 2, when the user implements the principle of the invention; - Figure 4 is a simplified block diagram of the processing carried out in the context of the invention. The embodiment described below relates to a multimedia type radiotelephone, equipped with a camera (or a camera). Of course, the approach of the invention can also be applied to other portable devices, such as personal assistants, audio and / or video file storage devices, portable game consoles, etc. An example of such a radiotelephone is illustrated diagrammatically in FIG. 1. The housing 11 conventionally carries a keyboard 12, comprising a reduced number of keys (generally a numeric keypad), a screen 13 and a lens 14, allowing the taking of still pictures and / or animated. Often, this objective is mounted mobile relative to the housing 11, and for example mobile in rotation. The general principle of the invention is to detect certain movements applied by the user to his telephone, and for example four movements of base, up, down, left or right (see arrows H, B, G, D shown in Figure 1). According to the invention, images taken by the objective 14 are used to detect these movements. More precisely, the relative inclination of the telephone is deduced from the displacement of the image captured by the camera. We can then deduce relative movements of the telephone. In the embodiment illustrated diagrammatically in FIG. 2, the user 21 holds his phone facing him, and looks at the screen 13. The camera 14 films it, and the corresponding images are analyzed, to detect movements therein. This approach is illustrated in Figures 3A to 3C. In FIG. 3A, the telephone being directed towards the user, there is a first image. If the user moves or tilts their phone slightly to the left, the image will move to the right, as shown in Figure 3B. Likewise, if the user points their phone up, the resulting image will move down, as shown in Figure 3C. It is therefore sufficient to compare the images 3A and 3B to deduce that the user has made a relative movement of the phone to the left, and the images 3A and 3C to detect an upward movement. It then suffices to associate these movements with predefined commands, actions or events to define a new man / machine interface according to the invention, consisting in associating a movement relating to an event. The processing is therefore very simple, as illustrated diagrammatically in FIG. 4, and implements simple means and treatments, and often available, such as a camera and means for detecting movement. More specifically, the processing comprises: a step 41 for taking pictures, for example at a rate of 5 to 10 images per second. To simplify the calculations, these images can be resized to a reduced size, and / or subsampled; a movement detection step 42, consisting for example of calculating a main movement vector in the image. We can in particular, to implement a spatial correlation consisting of shifting two images in two dimensions (or more generally in the directions associated with a possible event), and detecting the position which comes closest to the superposition. The correlation calculation for each displacement of the image consists for example in calculating the sum of the squares of the differences of each pixel; - An association of the detected movement with a predetermined command 43, which can for example use a simple table. This step can take into account one or more thresholds, for example to avoid taking into account small untimely displacements, or to graduate the displacements according to their amplitude. The detection can be digital, or logical: existence or not of a movement in a direction, or analog, that is to say progressive, or proportional. It is then possible to obtain also proportional commands, for example for the movement of an object on the screen: the movement of the latter can take account of the amplitude and / or the speed of movement of the radiotelephone; a step of carrying out the command 44, in the same way as it would be done with an event obtained using another interface. The power necessary to perform these operations, and in particular the detection of movement, is a priori acceptable, in particular for multimedia processors of mobile telephony. For example, for images resized to 100 x 100 pixels, and object displacements corresponding to a maximum of 30 pixels, the amount of calculations is of the order of 180 Mips (5 images / sx (30x30) displacements x ( 100x100) pixels x 4 operations / pixel). To this approximate power requirement, it is also necessary to add a resizing of the image, the cameras generally having a resolution of at least 640 x 480 pixels. There are many other known, simpler and more effective ways of determining movements in the image. In addition, advantageously, means available for other applications can be used. Thus, one can use an image stabilizer integrated in the multimedia processor (such an image stabilizer is for example available in the processors produced by the company Emblaze (registered trademark)). This stabilizer detects the overall movement of the image, to compensate for the movements of the camera. These means can easily be modified to obtain an overall movement of the image, which can be used in the algorithm illustrated in FIG. 4. It is also possible to envisage using an optimized algorithm, such as an MPEG4 encoder, already present. and already in the interface of multimedia phones. The latter offers motion detection in macroblocks (squares of 8 x 8 pixels), which could be used to determine a global movement. According to the invention, only sufficiently large movements should preferably be taken into account, to avoid any untimely control, due to an involuntary movement. The detected movement can be absolute or relative. In the case of absolute tracking, it is also possible to provide a step of detecting the silhouette (by pattern recognition) of the user (or of another unique object), then of detecting his movement. In the embodiment described, in particular with FIG. 2, the movement is analyzed by observation of the face of the user. It is understood of course that the same approach can be implemented in a similar manner, if the camera is oriented forward, relative to the user. This new interface according to the invention can be used in many ways, developers easily imagine applications. For example, it can allow navigation in menus, movements to the left or to the right allowing the selection of one command from among two, presented on the screen, respectively on the left and right side. An upward movement can be associated with a control of validation, and a movement down to a cancellation command, or back to the previous step. The interface can also be used for games. For example, in the context of a racing game, movements to the left and to the right make it possible to move a vehicle to the left or to the right, a movement to the right forwards makes it possible to obtain a acceleration, and a backward motion slowing down.

Claims

REVENDICATIONS
1. Dispositif électronique portable équipé d'une caméra, caractérisé en ce qu'il comprend des moyens de détection d'au moins un mouvement en fonction d'une analyse d'au moins deux images successives prises par ladite caméra, et des moyens d'interface homme-machine associant au moins un événement à au moins un desdits mouvements.1. Portable electronic device equipped with a camera, characterized in that it comprises means for detecting at least one movement as a function of an analysis of at least two successive images taken by said camera, and means for man-machine interface associating at least one event with at least one of said movements.
2. Dispositif électronique portable selon la revendication 1 , caractérisé en ce que lesdits moyens de détection déterminent un vecteur mouvement principal entre deux images successives, lesdits moyens d'interface analysant ledit vecteur mouvement principal et lui associant au moins un événement s'il répond à au moins un critère prédéterminé.2. portable electronic device according to claim 1, characterized in that said detection means determine a main motion vector between two successive images, said interface means analyzing said main motion vector and associating with it at least one event if it responds to at least one predetermined criterion.
3. Dispositif électronique portable selon la revendication 2, caractérisé en ce que le ou lesdits critères prédéterminés appartiennent au groupe comprenant la direction, le sens et l'amplitude dudit vecteur mouvement principal.3. Portable electronic device according to claim 2, characterized in that the said predetermined criterion or criteria belong to the group comprising the direction, the direction and the amplitude of said main movement vector.
4. Dispositif électronique portable selon l'une quelconque des revendications 2 et 3, caractérisé en ce que lesdits moyens d'interface comparent au moins un desdits critères à au moins un seuil, pour décider si ledit vecteur mouvement principal correspond ou non à un desdits événements. 4. Portable electronic device according to any one of claims 2 and 3, characterized in that said interface means compare at least one of said criteria to at least one threshold, to decide whether said main motion vector corresponds or not to one of said events.
5. Dispositif électronique portable selon l'une quelconque des revendications5. Portable electronic device according to any one of the claims
1 à 4, caractérisé en ce que lesdits moyens d'interface associent des événements distincts auxdits mouvements, en fonction de contextes applicatifs distincts dans lesquels peut se trouver ledit dispositif.1 to 4, characterized in that said interface means associate distinct events with said movements, as a function of distinct application contexts in which said device can be found.
6. Dispositif électronique portable selon l'une quelconque des revendications 1 à 5, caractérisé en ce qu'il comprend au moins un écran et des moyens pour afficher sur ledit écran au moins une information représentative d'un événement qui sera réalisé si un mouvement particulier est effectué.6. Portable electronic device according to any one of claims 1 to 5, characterized in that it comprises at least one screen and means for displaying on said screen at least information representative of an event which will be carried out if a movement particular is performed.
7. Dispositif électronique portable selon l'une quelconque des revendications 1 à 6, caractérisé en ce qu'il comprend au moins un écran et des moyens pour afficher sur ledit écran au moins une information représentative d'un événement réalisé suite à un mouvement.7. portable electronic device according to any one of claims 1 to 6, characterized in that it comprises at least one screen and means for displaying on said screen at least information representative of an event made following a movement.
8. Dispositif électronique portable selon l'une quelconque des revendications 1 à 7, caractérisé en ce que lesdits moyens de détection mettent en œuvre une corrélation spatiale, selon laquelle une première image est décalée dans au moins une direction, puis comparée à une seconde image.8. portable electronic device according to any one of claims 1 to 7, characterized in that said detection means implement a spatial correlation, according to which a first image is shifted in at least one direction, then compared to a second image .
9. Dispositif électronique portable selon la revendication 8, caractérisé en ce que lesdits moyens de détection calculent la somme des carrés des différences d'une grandeur prédéterminée pour chaque pixel desdites images.9. Portable electronic device according to claim 8, characterized in that said detection means calculate the sum of the squares of the differences of a predetermined magnitude for each pixel of said images.
10. Dispositif électronique portable selon l'une quelconque des revendications 1 à 9, caractérisé en ce que lesdits moyens de détection comprennent des moyens de re-dimensionnement d'images, pour réduire la complexité des images analysées.10. Portable electronic device according to any one of claims 1 to 9, characterized in that said detection means comprise means for resizing images, to reduce the complexity of the images analyzed.
11. Dispositif électronique portable selon l'une quelconque des revendications 1 à 10, caractérisé en ce que lesdits moyens de détection prennent en compte des vecteurs mouvement calculés par un encodeur MPEG-4.11. Portable electronic device according to any one of claims 1 to 10, characterized in that said detection means take into account motion vectors calculated by an MPEG-4 encoder.
12. Dispositif électronique portable selon l'une quelconque des revendications 1 à 11 , caractérisé en ce que lesdits moyens de détection prennent en compte des mouvements déterminés par des moyens de stabilisation d'images.12. Portable electronic device according to any one of claims 1 to 11, characterized in that said detection means take into account movements determined by image stabilization means.
13. Dispositif électronique portable selon l'une quelconque des revendications 1 à 12, caractérisé en ce que lesdits moyens de détection mettent en œuvre des moyens de reconnaissance de forme, pour identifier une forme principale, et des moyens d'analyse des mouvements de ladite forme principale.13. portable electronic device according to any one of claims 1 to 12, characterized in that said detection means implement shape recognition means, to identify a main shape, and means for analyzing the movements of said main form.
14. Dispositif électronique portable selon l'une quelconque des revendications 1 à 13, caractérisé en ce qu'il comprend des moyens de radiocommunication. 14. Portable electronic device according to any one of claims 1 to 13, characterized in that it comprises radio communication means.
15. Procédé d'interface homme-machine pour dispositif électronique portable équipé d'une caméra, caractérisé en ce qu'il associe au moins un événement à au moins un mouvement détecté par analyse d'au moins deux images successives prises par ladite caméra. 15. Man-machine interface method for a portable electronic device equipped with a camera, characterized in that it associates at least one event with at least one movement detected by analysis of at least two successive images taken by said camera.
16. Programme informatique pour dispositif électronique portable équipé d'une caméra, caractérisé en ce qu'il comprend des instructions pour associer au moins un événement à au moins un mouvement détecté par analyse d'au moins deux images successives prises par ladite caméra. 16. Computer program for portable electronic device equipped with a camera, characterized in that it includes instructions for associating with the at least one event with at least one movement detected by analysis of at least two successive images taken by said camera.
PCT/FR2004/002313 2003-09-12 2004-09-10 Portable electronic device with man/machine interface integrating device movements, corresponding method and programme WO2005026940A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
FR0310775 2003-09-12
FR0310775A FR2859800B1 (en) 2003-09-12 2003-09-12 PORTABLE ELECTRONIC DEVICE WITH MAN-MACHINE INTERFACE TAKING ACCOUNT OF DEVICE MOVEMENTS, CORRESPONDING METHOD AND COMPUTER PROGRAM

Publications (1)

Publication Number Publication Date
WO2005026940A1 true WO2005026940A1 (en) 2005-03-24

Family

ID=34203424

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/FR2004/002313 WO2005026940A1 (en) 2003-09-12 2004-09-10 Portable electronic device with man/machine interface integrating device movements, corresponding method and programme

Country Status (2)

Country Link
FR (1) FR2859800B1 (en)
WO (1) WO2005026940A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11153472B2 (en) 2005-10-17 2021-10-19 Cutting Edge Vision, LLC Automatic upload of pictures from a camera

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB0503253D0 (en) * 2005-02-17 2005-03-23 Univ Northumbria Newcastle User control of a hand-held device
FR2889324B1 (en) * 2005-07-29 2007-11-23 Realeyes3D Sa METHOD FOR CONTROLLING A GRAPHICAL INTERFACE USING A CAMERA EQUIPPING A COMMUNICATION TERMINAL
FR2889323B1 (en) * 2005-07-29 2008-12-12 Realeyes3D Sa METHOD FOR CONTROLLING AN INTERFACE USING A CAMERA COMPRISING A COMMUNICATION TERMINAL
FR2898007B1 (en) * 2006-02-27 2008-09-19 Sagem Comm METHOD FOR ADAPTING THE MAN / MACHINE INTERFACE OF A TELEPHONE AND TELEPHONE ACCORDING TO THE METHOD
KR100764698B1 (en) 2006-08-07 2007-10-08 삼성전자주식회사 Mobile device and user interface control method therefor based on pattern cognition and analysis of image captured by camera
US20080134102A1 (en) * 2006-12-05 2008-06-05 Sony Ericsson Mobile Communications Ab Method and system for detecting movement of an object
GB2448319A (en) * 2007-04-10 2008-10-15 Univ Northumbria Newcastle User Control of a Hand-Held Device
US8384718B2 (en) 2008-01-10 2013-02-26 Sony Corporation System and method for navigating a 3D graphical user interface

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0745866A1 (en) * 1995-06-01 1996-12-04 Aerospatiale Societe Nationale Industrielle Method and apparatus for movement detection of a target and its applications
JPH10240436A (en) * 1996-12-26 1998-09-11 Nikon Corp Information processor and recording medium
US5808678A (en) * 1993-11-10 1998-09-15 Canon Kabushiki Kaisha Method and apparatus for designing a position on a view finder based on motion detection
US5994710A (en) * 1998-04-30 1999-11-30 Hewlett-Packard Company Scanning mouse for a computer system
US6269174B1 (en) * 1997-10-28 2001-07-31 Ligos Corporation Apparatus and method for fast motion estimation
EP1130906A2 (en) * 2000-01-18 2001-09-05 Hewlett-Packard Company, A Delaware Corporation Pointing device for digital camera display
US20020027547A1 (en) * 2000-07-11 2002-03-07 Noboru Kamijo Wristwatch type device and method for moving pointer
EP1220143A2 (en) * 2000-12-25 2002-07-03 Hitachi, Ltd. Electronics device applying an image sensor

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5808678A (en) * 1993-11-10 1998-09-15 Canon Kabushiki Kaisha Method and apparatus for designing a position on a view finder based on motion detection
EP0745866A1 (en) * 1995-06-01 1996-12-04 Aerospatiale Societe Nationale Industrielle Method and apparatus for movement detection of a target and its applications
JPH10240436A (en) * 1996-12-26 1998-09-11 Nikon Corp Information processor and recording medium
US6269174B1 (en) * 1997-10-28 2001-07-31 Ligos Corporation Apparatus and method for fast motion estimation
US5994710A (en) * 1998-04-30 1999-11-30 Hewlett-Packard Company Scanning mouse for a computer system
EP1130906A2 (en) * 2000-01-18 2001-09-05 Hewlett-Packard Company, A Delaware Corporation Pointing device for digital camera display
US20020027547A1 (en) * 2000-07-11 2002-03-07 Noboru Kamijo Wristwatch type device and method for moving pointer
EP1220143A2 (en) * 2000-12-25 2002-07-03 Hitachi, Ltd. Electronics device applying an image sensor

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
PATENT ABSTRACTS OF JAPAN vol. 1998, no. 14 31 December 1998 (1998-12-31) *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11153472B2 (en) 2005-10-17 2021-10-19 Cutting Edge Vision, LLC Automatic upload of pictures from a camera
US11818458B2 (en) 2005-10-17 2023-11-14 Cutting Edge Vision, LLC Camera touchpad

Also Published As

Publication number Publication date
FR2859800B1 (en) 2008-07-04
FR2859800A1 (en) 2005-03-18

Similar Documents

Publication Publication Date Title
JP6907366B2 (en) Touchscreen hover detection in enhanced and / or virtual reality environments
US11102398B2 (en) Distributing processing for imaging processing
TWI688879B (en) Method, virtual reality system, and computer-readable recording medium for real-world interaction in virtual reality environment
EP3158418B1 (en) Architechture for managing input data
KR100899610B1 (en) Electronic device and a method for controlling the functions of the electronic device as well as a program product for implementing the method
FR2878641A1 (en) METHOD OF AUTOMATICALLY AUTOMATIC NAVIGATION TOWARDS REGIONS OF INTEREST OF AN IMAGE
US11150794B2 (en) Electronic device control in response to finger rotation upon fingerprint sensor and corresponding methods
US20120105589A1 (en) Real time three-dimensional menu/icon shading
EP3489811B1 (en) Method of mixed reality display of at least one virtual object, terminal and system thereof
WO2005026940A1 (en) Portable electronic device with man/machine interface integrating device movements, corresponding method and programme
CN113676655B (en) Shooting method and device, mobile terminal and chip system
TWI267061B (en) Method for processing multi-layered images
US9350918B1 (en) Gesture control for managing an image view display
EP1910915A2 (en) Method for controlling an interface using a camera equipping a communication terminal
US20240080543A1 (en) User interfaces for camera management
López et al. Multimodal sensing-based camera applications
Hannuksela et al. Camera assisted multimodal user interaction
FR2889324A1 (en) Graphical user interface controlling method for communication terminal, involves calculating apparent image by using pre-processed images and image correspondence technique, and filtering movement in amplitude and/or temporal manner
EP3038354B1 (en) A method for displaying images or videos
CN115619665A (en) Video generation method and device, electronic equipment and storage medium
EP3942390A1 (en) Virtual reality data-processing device, system and method
FR3058607A1 (en) VIDEO CAPTURE DEVICE WITH CAPTURE OF USER EYE MOVEMENT
FR3015092A1 (en) CONTROLLING A POINTER ON A DIGITAL SCREEN

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BW BY BZ CA CH CN CO CR CU CZ DK DM DZ EC EE EG ES FI GB GD GE GM HR HU ID IL IN IS JP KE KG KP KZ LC LK LR LS LT LU LV MA MD MK MN MW MX MZ NA NI NO NZ PG PH PL PT RO RU SC SD SE SG SK SY TJ TM TN TR TT TZ UA UG US UZ VN YU ZA ZM

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): BW GH GM KE LS MW MZ NA SD SZ TZ UG ZM ZW AM AZ BY KG MD RU TJ TM AT BE BG CH CY DE DK EE ES FI FR GB GR HU IE IT MC NL PL PT RO SE SI SK TR BF CF CG CI CM GA GN GQ GW ML MR SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
122 Ep: pct application non-entry in european phase