WO2011086466A1 - Apparatus and method for detecting and classifying so as to be able to evaluate and self-evaluate exercises - Google Patents

Apparatus and method for detecting and classifying so as to be able to evaluate and self-evaluate exercises Download PDF

Info

Publication number
WO2011086466A1
WO2011086466A1 PCT/IB2011/000077 IB2011000077W WO2011086466A1 WO 2011086466 A1 WO2011086466 A1 WO 2011086466A1 IB 2011000077 W IB2011000077 W IB 2011000077W WO 2011086466 A1 WO2011086466 A1 WO 2011086466A1
Authority
WO
WIPO (PCT)
Prior art keywords
exercise
carried out
user
dimensional video
movement
Prior art date
Application number
PCT/IB2011/000077
Other languages
French (fr)
Inventor
Giovanni Saggio
Manfredo Giuseppe Mario Ferrari
Roberto Mugavero
Original Assignee
Giovanni Saggio
Manfredo Giuseppe Mario Ferrari
Roberto Mugavero
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Giovanni Saggio, Manfredo Giuseppe Mario Ferrari, Roberto Mugavero filed Critical Giovanni Saggio
Priority to EP11710262A priority Critical patent/EP2525880A1/en
Publication of WO2011086466A1 publication Critical patent/WO2011086466A1/en

Links

Classifications

    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B71/00—Games or sports accessories not covered in groups A63B1/00 - A63B69/00
    • A63B71/06—Indicating or scoring devices for games or players, or for other sports activities
    • A63B71/0619—Displays, user interfaces and indicating devices, specially adapted for sport equipment, e.g. display mounted on treadmills
    • A63B71/0622—Visual, audio or audio-visual systems for entertaining, instructing or motivating the user
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/0002—Remote monitoring of patients using telemetry, e.g. transmission of vital signals via a communication network
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
    • A61B5/1121—Determining geometric values, e.g. centre of rotation or angular range of movement
    • A61B5/1122—Determining geometric values, e.g. centre of rotation or angular range of movement of movement trajectories
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/22—Ergometry; Measuring muscular strength or the force of a muscular blow
    • A61B5/221—Ergometry, e.g. by using bicycle type apparatus
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/68—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient
    • A61B5/6801—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient specially adapted to be attached to or worn on the body surface
    • A61B5/6802—Sensor mounted on worn items
    • A61B5/6804—Garments; Clothes
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/68—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient
    • A61B5/6801—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient specially adapted to be attached to or worn on the body surface
    • A61B5/6802—Sensor mounted on worn items
    • A61B5/6804—Garments; Clothes
    • A61B5/6806—Gloves
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235—Details of waveform analysis
    • A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B24/00—Electric or electronic controls for exercising apparatus of preceding groups; Controlling or monitoring of exercises, sportive games, training or athletic performances
    • A63B24/0003—Analysing the course of a movement or motion sequences during an exercise or trainings sequence, e.g. swing for golf or tennis
    • A63B24/0006—Computerised comparison for qualitative assessment of motion sequences or the course of a movement
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/014—Hand-worn input/output arrangements, e.g. data gloves
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/0346—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—3D [Three Dimensional] image rendering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B19/00—Teaching not covered by other main groups of this subclass
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B19/00—Teaching not covered by other main groups of this subclass
    • G09B19/003—Repetitive work cycles; Sequence of movements
    • G09B19/0038—Sports
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B5/00—Electrically-operated educational appliances
    • G09B5/02—Electrically-operated educational appliances with visual presentation of the material to be studied, e.g. using film strip
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B24/00—Electric or electronic controls for exercising apparatus of preceding groups; Controlling or monitoring of exercises, sportive games, training or athletic performances
    • A63B24/0003—Analysing the course of a movement or motion sequences during an exercise or trainings sequence, e.g. swing for golf or tennis
    • A63B24/0006—Computerised comparison for qualitative assessment of motion sequences or the course of a movement
    • A63B2024/0012—Comparing movements or motion sequences with a registered reference
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B71/00—Games or sports accessories not covered in groups A63B1/00 - A63B69/00
    • A63B71/06—Indicating or scoring devices for games or players, or for other sports activities
    • A63B71/0619—Displays, user interfaces and indicating devices, specially adapted for sport equipment, e.g. display mounted on treadmills
    • A63B71/0622—Visual, audio or audio-visual systems for entertaining, instructing or motivating the user
    • A63B2071/0636—3D visualisation
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B71/00—Games or sports accessories not covered in groups A63B1/00 - A63B69/00
    • A63B71/06—Indicating or scoring devices for games or players, or for other sports activities
    • A63B71/0619—Displays, user interfaces and indicating devices, specially adapted for sport equipment, e.g. display mounted on treadmills
    • A63B2071/0647—Visualisation of executed movements
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B71/00—Games or sports accessories not covered in groups A63B1/00 - A63B69/00
    • A63B71/06—Indicating or scoring devices for games or players, or for other sports activities
    • A63B71/0619—Displays, user interfaces and indicating devices, specially adapted for sport equipment, e.g. display mounted on treadmills
    • A63B2071/065—Visualisation of specific exercise parameters
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B2220/00—Measuring of physical parameters relating to sporting activity
    • A63B2220/10—Positions
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B2220/00—Measuring of physical parameters relating to sporting activity
    • A63B2220/80—Special sensors, transducers or devices therefor
    • A63B2220/803—Motion sensors
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B2220/00—Measuring of physical parameters relating to sporting activity
    • A63B2220/80—Special sensors, transducers or devices therefor
    • A63B2220/806—Video cameras
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B2220/00—Measuring of physical parameters relating to sporting activity
    • A63B2220/80—Special sensors, transducers or devices therefor
    • A63B2220/83—Special sensors, transducers or devices therefor characterised by the position of the sensor
    • A63B2220/836—Sensors arranged on the body of the user
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B2225/00—Miscellaneous features of sport apparatus, devices or equipment
    • A63B2225/50—Wireless data transmission, e.g. by radio transmitters or telemetry
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B23/00—Exercising apparatus specially adapted for particular parts of the body
    • A63B23/035—Exercising apparatus specially adapted for particular parts of the body for limbs, i.e. upper or lower limbs, e.g. simultaneously
    • A63B23/12—Exercising apparatus specially adapted for particular parts of the body for limbs, i.e. upper or lower limbs, e.g. simultaneously for upper limbs or related muscles, e.g. chest, upper back or shoulder muscles
    • A63B23/16—Exercising apparatus specially adapted for particular parts of the body for limbs, i.e. upper or lower limbs, e.g. simultaneously for upper limbs or related muscles, e.g. chest, upper back or shoulder muscles for hands or fingers
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems

Definitions

  • the present invention refers to an apparatus and to a method for detecting and classifying the movement, useful for an evaluation and self-evaluation of exercises.
  • purposes there can be teaching, training, rehabilitation, but also the classification of gestures, of postures, of movement so as to identify whether it belongs to a predefined range like that dictated by standards, for example in aeronautics, shipping, in the military, in construction sites, etc., which associate posture/movements of the body to specific meanings.
  • the evaluation of the gesture, posture or movement of the person involved can be made by others or by expert personnel, or the individual can self-evaluate himself having qualitative and quantitative feedback provided by the entire system object of the present patent.
  • the exercise can be relative to a single person, or to a group of people, and the system can provide an overall quantitative evaluation or a quantitative evaluation of the exercise of each person "user" of the present invention.
  • a coach can in advance give each player actions that they are required to carry out; the actions can be displayed by as many avatar-guides as there are players; each player is asked to copy the gestures of the avatar- guide he is assigned; the system that measures the movements by means of "wearable” sensors then provides quantitative values to each player evaluating their performance, and/or the overall quantitative values for the performance relating to the team as a whole.
  • Each player can see his own movements copied through an avatar-user that, by superimposing itself on the avatar-guide, for example transparently, provides useful information, even visual, to the player who thus self-evaluates his own actions.
  • the system is capable of determining whether the movements carried out by the individual player or by many players, or by the whole team belong to a very specific classification, for example losing a marker, "zonal defence", catenaccio, Cambridge pyramid, free kick routines, wing play, etc... If the movements carried out do not correspond to the predefined classification, the system can signal it, advising the variants necessary to reach the predetermined purpose.
  • a very specific classification for example losing a marker, "zonal defence", catenaccio, Cambridge pyramid, free kick routines, wing play, etc.
  • fields involved in such an aspect can be mentioned both the field of sports, such as dancing, athletics or sport in general, etc., and the field of work, such as surgery, music performances, training astronauts and pilots or simple amateurs, etc.
  • exercise we mean both the movement of a single part of the body, such as a limb, and any movement that involves the entirety, as well as maintaining a particular posture over time.
  • this apparatus comprises video cameras capable of acquiring movement data of the user connected to a processor.
  • Such a processor returns the information of whether movements that exceed a virtual space limit, pre-set in the processor, have been carried out during the exercise to the user.
  • Such an apparatus has numerous drawbacks, amongst which the most considerable one is that of not providing the user with any quantitative or classification information of the useful movement about the movements he has carried out inside the aforementioned virtual space limit previously set in the processor.
  • the apparatus described in US2004/0219964 is capable of providing only information of the "qualitative” and not “quantitative” type, since it does not per se process the movement carried out by the user and it does not classify it in any way.
  • Another obvious drawback is how such an apparatus requires a long set-up time, with particular attention to the position and to the orientation with which the video cameras are positioned.
  • Such an apparatus comprises a work station that is capable of virtually simulating situations in which the user is required to carry out particular movements.
  • this work station there are sensors that are capable of acquiring movement data of the user connected to a processor.
  • the information returned to the user is of the "qualitative" type based upon whether or not a predetermined virtual objective has been reached.
  • the movements of the user are not forced, no "force feedback” or “tactile feedback” is indeed foreseen, no “virtual interactions” or presence of actuators are foreseen, the recording is not carried out by means of an optical detection system, for example video cameras, nor is there the limitation of considering just the movement relative to the hand.
  • the purpose of the present invention is that of making a device that is capable of solving the aforementioned drawbacks of the prior art in an extremely simple, cost-effective and particularly functional manner.
  • the present patent proposal foresees an avatar “guide” and an avatar of the "user” that "are superimposed” with the system that provides both visual and numerical information relative to "how much” they are superimposed .
  • Another purpose is that of making an apparatus and a method for detecting, classifying, teaching and training exercises that is capable of providing information back to the user, or the users, according to the movement carried out without the necessary reference to space limits and/or predetermined virtual obj ectives .
  • a further purpose is that of obtaining an apparatus and a method for detecting, classifying, teaching and training exercises that is not limited to a work station.
  • Yet another purpose is that of obtaining an apparatus for detecting, classifying, teaching and training exercises that is capable of simultaneously displaying both predetermined reference movements for example of a teacher, a doctor, a professional, a coach, a trainer, etc., the numerical quantitative data of which are known, and the movements carried out by the user or users and providing both numerical quantitative results on the correctness of the exercises carried out (by the user or by the users with respect to the reference) , and a classification of the movement/s, and advice on how to carry out the activity correctly.
  • a further purpose is that of obtaining an apparatus for detecting, classifying, teaching and training exercises that by means of suitable algorithms, analysis, classifying systems, realized for example through hardware and/or firmware and/or software, "recognize” the postures taken up by the user so as to be able to classify them with respect to a predetermined "library” .
  • the predefined "library” of movements and/or gestures can thus be referred to one or more standards or to a set of movements and/or gestures and/or routines that have suitably been "catalogued” by a coach, trainer, doctor that describes the rehabilitation movements, etc..
  • figure 1 is a schematic representation of an apparatus for detecting, classifying, teaching and training exercises according to the present invention.
  • figure 2 is a schematic representation of a second apparatus for detecting, classifying, teaching and training exercises according to the present invention
  • figure 3 is a schematic representation of a third apparatus for detecting, classifying, teaching, training exercises according to the present invention
  • figure 4 is a schematic view of an example of application of the apparatus according to the present invention.
  • FIGS 5 and 6 are schematic views of further examples of applications of the apparatus according to the present invention.
  • an apparatus for detecting, classifying, teaching and training exercises according to the present invention is shown with reference numeral 10.
  • Such an apparatus 10 comprises:
  • Such images can be relative to movements advised by a doctor, or taken from a standard, or advised by a professional, for example movements related to sport activities ;
  • the means 11 for acquiring data related to the exercise being carried out comprise at least a garment 14 provided with sensors 15 for detecting the movement;
  • the means 12 for processing the acquired and processed parameters for generating information relating to the exercise being carried out comprise software both for generating a three-dimensional video signal reproducing the exercise being carried out, and for generating a numerical signal relating to the exercise being carried out .
  • the classification of movements is based upon the recent progress, amongst other things, concerning algorithms of neural networks and/or Fuzzy Logic, and/or Support Vector Machine, and/or Markov models, and/or Principal Component Analysis, etc..
  • the means 13 for returning the information to the user comprise means 16 for displaying the three-dimensional video signal reproducing the exercise being carried out and/or the numerical signal.
  • exercise we mean both the movement of a single part of the body, such as a limb, and any movement that involves the entire body, as well as maintaining a particular posture over time.
  • the exercise can relate to a single person, or to a group of people, and the system can provide an overall quantitative evaluation or a quantitative evaluation of the exercise of each person "user" of the present invention.
  • FIG. 5 One example of such an embodiment is schematically shown in figure 5 in which reference numerals 30, 30', 30'' indicate the various users and 31, 31', 31'' indicate the relative avatars, with the broken line, and 32, 32', 32'' indicate the so called avatar- teachers that show the correct exercises to be carried out, which are different from one user to the next.
  • the garment 14 consists of a glove, a shirt, a shoe, therefore parts of the body but even up to a complete bodysuit, equipped with sensors capable of acquiring data relative to positions and movements of the body part involved or of the entire body.
  • the analysed movements of a hand can be of the flexing-extending type, or relative to the adduction of the interphalangeal , proximal interphalangeal , metacarpo- interphalangeal joints and to the movements of the wrist.
  • the aforementioned data is measured through suitable sensors and/or transducers that may or may not be fixedly attached to the body where the sensors and/or transducers can be of the bending, piezoresistive types, accelerometers , gyroscopes, magnetoresistances , IMUs or others .
  • the sensors/transducers are connected through an electronic circuit 20 with a cable and/or wirelessly to the processor 12 used for reading the data, for storing it, managing it, processing it, converting it and transmitting it to the means for returning and/or classifying 13 the information to the user.
  • the processor 12 through a preselected routine, generates a signal of the visual qualitative type 21, and of the numerical quantitative type, and a classification, when required and/or possible, of the static and/or dynamic postures that the subject must carry out.
  • the signal of the visual type 21 represents in its entirety the portion associated to the means for acquiring data 11.
  • the numerical quantitative information can be provided, for example, through simple numerical indicators 22 or by grey or chromatic scale charts 23 that can provide the user with an objective evaluation of his exercise.
  • the data received by the processor 12 are then translated into movements and displayed through three- dimensional representations of the portion itself in movement, so that the user has an immediate and easy control of his actions.
  • a value added is given by the fact that the movements carried out by the user can be displayed through an avatar-user which, transparently, is superimposed over an avatar-teacher, so that the user can immediately verify whether his movement follows the preset one that he is supposed to carry out.
  • This information, of the visual type can be accompanied by quantitative type information so that the system, which analyses both the avatars, is capable of giving a quantitative evaluation of how closely the movement of the user corresponds to that of the teacher.
  • FIG. 4 Such an embodiment just described is schematically shown in figure 4 in which the user during exercise is indicated with reference numeral 30, the avatar-user is indicated with reference numeral 31, in the broken line, and reference numeral 32 indicates the avatar- teacher .
  • the means 13 for returning the information to the user comprise a monitor 16 that is capable of displaying both the three-dimensional signal reproducing the exercise being carried out processed by the processor 12, and numerical quantitative information relative to the analysed posture.
  • the display can occur in real time, or the user, or users, can see it at a later time since the system is capable of storing the recorded information in a specific memory.
  • FIG. 6 An example of such a last embodiment is shown in figure 6 in which the memory in which the information is recorded is indicated with reference numeral 33.
  • the apparatus can comprise a common monitor 16 and eyeglasses with polarized lenses, or equivalent means, for the three-dimensional focusing of said three- dimensional video signal of the stereoscopic type, or the display can be carried out by means of a monitor that provides the three-dimensional display even without the help of special eyeglasses.
  • the signal can be of the auto- stereoscopic and/or holographic type, and it is possible to foresee a monitor 16 comprising means for automatically focusing said three-dimensional video signal of the stereoscopic type.
  • the apparatus can comprise projectors 16' for reproducing - not acquiring- and for the automatic three-dimensional focusing of said three-dimensional video signal of the holographic type .
  • the apparatus can also be capable of providing useful advice to the user on how to modify and/or improve his exercises, as well as analysing them with the modalities of play, pause, fast/slow rewind, freeze frame, zoom, etc.
  • the apparatus for detecting, classifying, teaching, training exercises of the present invention is capable of providing information back to the user according to the movement carried out through the three-dimensional reconstruction of the moving limb without the necessary reference to space limits and/or predetermined virtual objectives. It is thus clear how, differently from apparatuses known today, the present invention makes it possible to obtain greater information, such as the continuous progress over time of the posture during the exercise, the classification of the gesture/exercise, etc... Moreover, it allows the user, or the users, to self- evaluate their own movements, since it can be capable of comparing what has been carried out with respect to predetermined movements of a guide (teacher, professional, doctor, etc.).
  • the user can carry out (at his own home or elsewhere) the rehabilitation movements advised by a doctor, self-evaluating the results since the system provides him with the information on how much his movements differ from those that he is to carry out.
  • the self-evaluation can be carried out both in "realtime” during the actual carrying out of the movements and/or gestures, and "off-line” since the user is capable of re-watching any phase in any subsequent moment, since the postures, the movements and/or gestures are simultaneously recorded in a memory.
  • the evaluation can thus be carried out personally, or carried out immediately or subsequently by the teacher, professional, doctor, .. who evaluates also based upon the system according to the present patent.
  • the apparatus for detecting, classifying, teaching, training exercises according to the present invention does not in any way restrict the user to a particular work station, but allows free movement in the space.
  • the displayed three-dimensional images of the exercise irrespective of whether they are represented through holography, stereoscopy, auto-stereoscopy, or through simple images on the monitor, in addition to displaying the movements carried out by the user also the movements that should be carried out can be generated.
  • the apparatus of the present invention can advise the user on the movement to be carried out displaying it beforehand.
  • the apparatus can also provide the user with information about how his movement is or is not close to the correct set movement.
  • the apparatus can indeed display both the proposed exercise and simultaneously the exercise actually carried out by the user evaluating both numerically and visually when the exercise itself is carried out correctly.
  • the apparatus is therefore capable of displaying, stereoscopically, auto-stereoscopically, three- dimensionally, on a screen etc., both the pre-set movements for which it is known the numerical quantitative data, since it has already been measured earlier, and the movements carried out by the user (or by the users) and of providing both numerical quantitative results on the correctness of the exercise carried out and advices on how to carry out the exercise correctly.
  • Such pre-recorded movements can be, for example, those recorded by a professional athlete, footballer, skier, cyclist, etc., or by yoga, martial art instructors, etc., as also a dance instructor or music teacher, or by a driving instructor, or by a doctor who indicates which exercises are necessary to carry out in order to achieve the rehabilitation goals, etc.
  • the user in such a case, is thus asked to repeat the same movements and can see his own avatar possibly superimposed over the image of the avatar provided by the professional, teacher etc...
  • the movement carried out by the user is (are) free and not forced by any external agent .
  • the apparatus thus provides, based upon measured parameters, numerical quantitative results on the correctness of the exercise carried out.
  • the system is capable of providing evaluation on which were the incorrect movements and indication on which ones to correct because they differ from those indicated by more than a certain quantitative limit.
  • the apparatus of the present invention can comprise an "intelligent" processor equipped with self- evaluation capabilities even referring to software that may be introduced on the market and/or already present through which dedicated industries and/or professionals propose exercises of physical training, of correct posture maintenance, more in general, for learning sport, gymnastics and rehabilitation exercises such as for example exercises of yoga, dancing, of free-style gymnastics or gymnastics with equipment and teaching all sport activities.
  • the intelligent system is capable of generating special software that focuses not only on the optimal carrying out of the exercises proposed but also on the opportunity of reducing the learning time of each exercise according to the exercise movements displayed by the user. It is very easy to understand how the apparatus for detecting, teaching and training exercises object of the present invention operates.
  • the method implemented by the aforementioned apparatus for detecting, teaching and training exercises comprises the simple phases of:
  • the acquiring phase of the position and movement data related to the exercise being carried out is realized by wearing a garment provided with movement detection sensors;
  • the processing phase comprises the phases of generating both a three-dimensional video signal reproducing the exercise being carried out, and a numerical signal relating to the exercise being carried out, of classifying the actions/movement highlighting whether and how it belongs to a predefined category, and of advising how to modify actions/movements in order to make them belong to a desired category;
  • the returning phase of the information to the user is realized through the displaying of the three-dimensional video signal reproducing the exercise being carried out and/or of the numerical signal and/or the simultaneous displaying both of the signal recorded by the user, and, by comparison, of a pre-set movement, for example by a professional .
  • the method is extremely advantageous in the case in which the phase of generating the three-dimensional video signal is realized through a three-dimensional image on a screen or again when generating a three-dimensional video signal of the stereoscopic, auto-stereoscopic or holographic type or other of the part of body or of the entire body that is moving.
  • the apparatus for detecting, classifying, teaching, training exercises of the present invention thus conceived can undergo numerous modifications and variants, all covered by the same inventive concept; moreover, all the details can be replaced by technically equivalent elements.
  • the materials used, as well as their sizes, can be of any type according to the technical requirements.

Abstract

Method for detecting, classifying, teaching and training exercises comprising the phases of : - acquiring position and movement data relating to an exercise being carried out; - processing said acquired data to generate information relating to the exercise being carried out; - returning said information to the user/s so as to help teaching and training with the exercise being carried out; in which - said acquiring phase of said movement data relating to the exercise being carried out is realized by wearing one or more garments provided with movement detection sensors,- said exercise being carried out in the absence of external forcing; said processing phase comprises the phases of generating both a three-dimensional signal reproducing the exercise being carried out, and a numeric signal relating to the exercise being carried out; - the returning phase of the information to the user/s, being realized through the simultaneous displaying of the three-dimensional video signal reproducing the exercise being carried out, of a three-dimensional signal reproducing the exercise to be carried out and of a numerical signal comparing the three-dimensional video signals.

Description

APPARATUS AND METHOD FOR DETECTING AND CLASSIFYING SO AS TO BE ABLE TO EVALUATE AND SELF-EVALUATE EXERCISES
The present invention refers to an apparatus and to a method for detecting and classifying the movement, useful for an evaluation and self-evaluation of exercises. Amongst the purposes there can be teaching, training, rehabilitation, but also the classification of gestures, of postures, of movement so as to identify whether it belongs to a predefined range like that dictated by standards, for example in aeronautics, shipping, in the military, in construction sites, etc., which associate posture/movements of the body to specific meanings.
By providing the system with quantitative data, the evaluation of the gesture, posture or movement of the person involved can be made by others or by expert personnel, or the individual can self-evaluate himself having qualitative and quantitative feedback provided by the entire system object of the present patent.
The exercise can be relative to a single person, or to a group of people, and the system can provide an overall quantitative evaluation or a quantitative evaluation of the exercise of each person "user" of the present invention.
We can refer to the exercise of a football team: a coach can in advance give each player actions that they are required to carry out; the actions can be displayed by as many avatar-guides as there are players; each player is asked to copy the gestures of the avatar- guide he is assigned; the system that measures the movements by means of "wearable" sensors then provides quantitative values to each player evaluating their performance, and/or the overall quantitative values for the performance relating to the team as a whole. Each player can see his own movements copied through an avatar-user that, by superimposing itself on the avatar-guide, for example transparently, provides useful information, even visual, to the player who thus self-evaluates his own actions.
The system is capable of determining whether the movements carried out by the individual player or by many players, or by the whole team belong to a very specific classification, for example losing a marker, "zonal defence", catenaccio, Cambridge pyramid, free kick routines, wing play, etc... If the movements carried out do not correspond to the predefined classification, the system can signal it, advising the variants necessary to reach the predetermined purpose. Nowadays, it is considerably important in various fields to be able to carry out particular movements in the correct manner.
As an example of fields involved in such an aspect can be mentioned both the field of sports, such as dancing, athletics or sport in general, etc., and the field of work, such as surgery, music performances, training astronauts and pilots or simple amateurs, etc.
Another field in which it is considerably important to carry out movements in the most correct manner possible is that of medical-rehabilitation therapies.
In general, in all the aforementioned activities the fact that there is the possibility of monitoring, by measuring and classifying the exercise that is either being carried out, or carried out in a next moment, provides a fundamental support to improve both the teaching/learning phase, and the training/repeating of the desired exercise, so as to improve one's self, so as to be able to carry out a certain movement/action in the best way possible, to achieve even a predetermined purpose .
Of course, with the term "exercise" we mean both the movement of a single part of the body, such as a limb, and any movement that involves the entirety, as well as maintaining a particular posture over time.
Today some apparatuses are known that are capable of acquiring movement data relative to an exercise being carried out, processing such data to generate information relating to the exercise being carried out and returning this information to the user to promote, as described above, the teaching and the training of the particular exercise being carried out.
A first example of such known apparatuses is described in patent application US2004/0219964.
In particular, this apparatus comprises video cameras capable of acquiring movement data of the user connected to a processor.
Such a processor returns the information of whether movements that exceed a virtual space limit, pre-set in the processor, have been carried out during the exercise to the user.
Such an apparatus has numerous drawbacks, amongst which the most considerable one is that of not providing the user with any quantitative or classification information of the useful movement about the movements he has carried out inside the aforementioned virtual space limit previously set in the processor.
In other words, the apparatus described in US2004/0219964 is capable of providing only information of the "qualitative" and not "quantitative" type, since it does not per se process the movement carried out by the user and it does not classify it in any way.
It should indeed be noted that the information provided to the user is only related to the set particular virtual space limit, which can vary according to the requirements .
Another obvious drawback is how such an apparatus requires a long set-up time, with particular attention to the position and to the orientation with which the video cameras are positioned.
Indeed, an incorrect positioning of the video cameras will necessarily lead to an incorrect processing of the movement of the user with respect to the predetermined virtual space limit. The images, precisely because they are recorded by video cameras, can moreover be affected by blocking out, or rather the fact that if an element is hidden from the video camera the movement cannot be recorded.
A second apparatus known today is described in patent application US2008/00387702.
Such an apparatus, of the simulator type, comprises a work station that is capable of virtually simulating situations in which the user is required to carry out particular movements.
In this work station there are sensors that are capable of acquiring movement data of the user connected to a processor.
Differently from the previous case, in which the movement of the user was not processed but only recorded, in such an apparatus the exercise is processed with the purpose of representing the effect produced by such a movement on the virtually generated situation.
Therefore, even in such a case, like in the previous case, the information provided back to the user is not focused on the particular movement carried out, but on the effect it generates with reference to whether or not a specific goal has been achieved.
In other words, in such simulators the correctness of a given movement is not tested, but rather it is tested whether or not the overall movements carried out have made it possible to achieve the predetermined goal of the simulation.
As an example flight simulators can be mentioned in which the aspiring pilots practice their first flight training.
Even in this apparatus therefore, the information returned to the user is of the "qualitative" type based upon whether or not a predetermined virtual objective has been reached.
As a further obvious drawback of such technology is that the exercise of the user is necessarily limited in the area around the work station in which the movement sensors are integrated.
Further devices for teaching and training exercises are described in the prior art documents US2006/137511 , US2004/046777, US2004/174337 , US7084884, US2002/146672 , US5846086 and US2009/237763.
However, even such devices have some drawbacks which shall be solved by the present invention.
In relation to the apparatus according to the patent application US2006/137511, a hand is described the movement of which is not free but "guided" by a signal that "forces" the hand itself to carry out the movement so as to press the right key on a piano.
Therefore, in such a device, the motion of the user is not free but is forced to carry out the relative exercise possibly interacting with virtual objects.
With reference to the apparatus according to patent application US2004/046777 , a system is described entirely based upon "tactile feedback" . On a monitor it is foreseen for there to be shown the movements of the user and the surrounding objects, but there is no avatar "guide" showing the correct movement to be carried out .
For the apparatus according to the patent application US2004/174337 , a system exclusively focused on the movements of the hand is described, and exclusively based upon "force feedback", as for the apparatus according to patent application US2006/137511.
For the apparatus according to patent application US7084884, a system focused only on the hand is described. It foresees "virtual interaction": a hand- avatar, moved by the movements of a real hand, interacts with virtual objects. Moreover, it foresees "actuators" .
For the apparatus according to patent application US2002/146672 , a system focused only on the hand is described, which foresees the interaction with virtual objects and "force feedback".
For the apparatus according to patent application US5846086, a system is described for which a sequence of the movements of the "teacher" is recorded by video cameras. The system digitizes the video images and saves them in a computer memory.
For the apparatus according to patent application US2009/237763 , systems and methods are described that allow the user to interact with virtual objects through an interface given by a glove with sensors .
Differently from all the patents mentioned, according to the present invention the movements of the user are not forced, no "force feedback" or "tactile feedback" is indeed foreseen, no "virtual interactions" or presence of actuators are foreseen, the recording is not carried out by means of an optical detection system, for example video cameras, nor is there the limitation of considering just the movement relative to the hand.
Moreover, differently from other patents, in the present invention there is a measurement of the movements that is not of the "qualitative" type but of the "quantitative" type foreseeing, in addition, a classifying system that has never been mentioned in the aforementioned prior art patents.
The purpose of the present invention is that of making a device that is capable of solving the aforementioned drawbacks of the prior art in an extremely simple, cost-effective and particularly functional manner.
Differently from the previous systems, the present patent proposal foresees an avatar "guide" and an avatar of the "user" that "are superimposed" with the system that provides both visual and numerical information relative to "how much" they are superimposed .
Another purpose is that of making an apparatus and a method for detecting, classifying, teaching and training exercises that is capable of providing information back to the user, or the users, according to the movement carried out without the necessary reference to space limits and/or predetermined virtual obj ectives .
A further purpose is that of obtaining an apparatus and a method for detecting, classifying, teaching and training exercises that is not limited to a work station.
Yet another purpose is that of obtaining an apparatus for detecting, classifying, teaching and training exercises that is capable of simultaneously displaying both predetermined reference movements for example of a teacher, a doctor, a professional, a coach, a trainer, etc., the numerical quantitative data of which are known, and the movements carried out by the user or users and providing both numerical quantitative results on the correctness of the exercises carried out (by the user or by the users with respect to the reference) , and a classification of the movement/s, and advice on how to carry out the activity correctly.
A further purpose is that of obtaining an apparatus for detecting, classifying, teaching and training exercises that by means of suitable algorithms, analysis, classifying systems, realized for example through hardware and/or firmware and/or software, "recognize" the postures taken up by the user so as to be able to classify them with respect to a predetermined "library" .
For example, reference is made to standards that consider the movements of the body with which specific meanings are associated, like in the case of the standard STANAG 3379 FS "IN-FLIGHT VISUAL SIGNALS" of the NATO Standardization Agency, the standard STANAG 3117 FS "AIRCRAFT MARSHALLING SIGNALS" of the MILITARY AGENCY FOR STANDARDIZATION (MAS) and BUREAU MILITAIRE DE STANDARDISATION (BMS) , the FM2160 "Visual signals" of the HEADQUARTERS DEPARTMENT OF THE ARMY of Washington, the "guidelines for gestural signals" according to the attachment XXXII in the Ordinary Supplement to Official Gazette Ufficiale general series no. 101 of 30-04-2008, etc. The predefined "library" of movements and/or gestures can thus be referred to one or more standards or to a set of movements and/or gestures and/or routines that have suitably been "catalogued" by a coach, trainer, doctor that describes the rehabilitation movements, etc..
These purposes, according to the present invention, are achieved by making a method and an apparatus for detecting, classifying movement, teaching and training exercises as outlined respectively in claims 1 and 5. Further characteristics of the invention are highlighted in the dependent claims.
The characteristics and the advantages of an apparatus and of a method for detecting, classifying, teaching and training exercises according to the present invention shall become clearer from the following description, given as an example and not for limiting purposes, with reference to the attached schematic drawings, in which:
figure 1 is a schematic representation of an apparatus for detecting, classifying, teaching and training exercises according to the present invention; and
figure 2 is a schematic representation of a second apparatus for detecting, classifying, teaching and training exercises according to the present invention; figure 3 is a schematic representation of a third apparatus for detecting, classifying, teaching, training exercises according to the present invention; figure 4 is a schematic view of an example of application of the apparatus according to the present invention; and
figures 5 and 6 are schematic views of further examples of applications of the apparatus according to the present invention.
With reference to the figures, an apparatus for detecting, classifying, teaching and training exercises according to the present invention is shown with reference numeral 10.
Such an apparatus 10 comprises:
- a pre-set database of guiding images that show the user, or users, the movements that they are to carry out. Such images can be relative to movements advised by a doctor, or taken from a standard, or advised by a professional, for example movements related to sport activities ;
- means 11 for acquiring data of position and of movement relating to an exercise being carried out;
- means 12 for processing the acquired data to generate information relating to the exercise being carried out;
- means for classifying the acquired data to generate information relating to the exercise being carried out.
The classification "catalogues" the movement carried out and verifies whether it corresponds to a reference of the type mentioned previously;
- means 13 for returning the information to the user so as to promote teaching and training of the exercise being carried out.
In particular, according to the invention, the means 11 for acquiring data related to the exercise being carried out comprise at least a garment 14 provided with sensors 15 for detecting the movement; the means 12 for processing the acquired and processed parameters for generating information relating to the exercise being carried out comprise software both for generating a three-dimensional video signal reproducing the exercise being carried out, and for generating a numerical signal relating to the exercise being carried out .
The classification of movements, is based upon the recent progress, amongst other things, concerning algorithms of neural networks and/or Fuzzy Logic, and/or Support Vector Machine, and/or Markov models, and/or Principal Component Analysis, etc..
The means 13 for returning the information to the user comprise means 16 for displaying the three-dimensional video signal reproducing the exercise being carried out and/or the numerical signal.
By the term exercise, we mean both the movement of a single part of the body, such as a limb, and any movement that involves the entire body, as well as maintaining a particular posture over time.
As mentioned previously, according to the invention the exercise can relate to a single person, or to a group of people, and the system can provide an overall quantitative evaluation or a quantitative evaluation of the exercise of each person "user" of the present invention.
One example of such an embodiment is schematically shown in figure 5 in which reference numerals 30, 30', 30'' indicate the various users and 31, 31', 31'' indicate the relative avatars, with the broken line, and 32, 32', 32'' indicate the so called avatar- teachers that show the correct exercises to be carried out, which are different from one user to the next.
All the considerations that will be specified in the following description relative to a single user are of course valid also for an embodiment in which the exercise is carried out in a group.
As visible in the example shown in the figures, the garment 14 consists of a glove, a shirt, a shoe, therefore parts of the body but even up to a complete bodysuit, equipped with sensors capable of acquiring data relative to positions and movements of the body part involved or of the entire body.
For example, the analysed movements of a hand can be of the flexing-extending type, or relative to the adduction of the interphalangeal , proximal interphalangeal , metacarpo- interphalangeal joints and to the movements of the wrist.
The aforementioned data is measured through suitable sensors and/or transducers that may or may not be fixedly attached to the body where the sensors and/or transducers can be of the bending, piezoresistive types, accelerometers , gyroscopes, magnetoresistances , IMUs or others .
The sensors/transducers are connected through an electronic circuit 20 with a cable and/or wirelessly to the processor 12 used for reading the data, for storing it, managing it, processing it, converting it and transmitting it to the means for returning and/or classifying 13 the information to the user.
According to the invention, the processor 12, through a preselected routine, generates a signal of the visual qualitative type 21, and of the numerical quantitative type, and a classification, when required and/or possible, of the static and/or dynamic postures that the subject must carry out.
As can be seen from the figures, the signal of the visual type 21 represents in its entirety the portion associated to the means for acquiring data 11.
The numerical quantitative information can be provided, for example, through simple numerical indicators 22 or by grey or chromatic scale charts 23 that can provide the user with an objective evaluation of his exercise. The data received by the processor 12 are then translated into movements and displayed through three- dimensional representations of the portion itself in movement, so that the user has an immediate and easy control of his actions.
A value added is given by the fact that the movements carried out by the user can be displayed through an avatar-user which, transparently, is superimposed over an avatar-teacher, so that the user can immediately verify whether his movement follows the preset one that he is supposed to carry out. This information, of the visual type, can be accompanied by quantitative type information so that the system, which analyses both the avatars, is capable of giving a quantitative evaluation of how closely the movement of the user corresponds to that of the teacher.
Such an embodiment just described is schematically shown in figure 4 in which the user during exercise is indicated with reference numeral 30, the avatar-user is indicated with reference numeral 31, in the broken line, and reference numeral 32 indicates the avatar- teacher .
According to the example shown in figure 1, the means 13 for returning the information to the user comprise a monitor 16 that is capable of displaying both the three-dimensional signal reproducing the exercise being carried out processed by the processor 12, and numerical quantitative information relative to the analysed posture.
The display can occur in real time, or the user, or users, can see it at a later time since the system is capable of storing the recorded information in a specific memory.
An example of such a last embodiment is shown in figure 6 in which the memory in which the information is recorded is indicated with reference numeral 33.
In figures 2 and 3, alternative embodiments are shown in which the three-dimensional video signal reproducing the exercise being carried out is respectively of the stereoscopic, auto-stereoscopic and/or holographic type.
In the case of three-dimensional video signals of the stereoscopic type, i.e. in general a three-dimensional signal "outside" the relative projection monitor, the apparatus can comprise a common monitor 16 and eyeglasses with polarized lenses, or equivalent means, for the three-dimensional focusing of said three- dimensional video signal of the stereoscopic type, or the display can be carried out by means of a monitor that provides the three-dimensional display even without the help of special eyeglasses.
Alternatively, the signal can be of the auto- stereoscopic and/or holographic type, and it is possible to foresee a monitor 16 comprising means for automatically focusing said three-dimensional video signal of the stereoscopic type.
In the case in which the three-dimensional video signal is of the holographic type, i.e. in general a three- dimensional signal without a projection monitor, the apparatus can comprise projectors 16' for reproducing - not acquiring- and for the automatic three-dimensional focusing of said three-dimensional video signal of the holographic type .
According to preferred embodiments, the apparatus can also be capable of providing useful advice to the user on how to modify and/or improve his exercises, as well as analysing them with the modalities of play, pause, fast/slow rewind, freeze frame, zoom, etc.
In general, therefore, the apparatus for detecting, classifying, teaching, training exercises of the present invention is capable of providing information back to the user according to the movement carried out through the three-dimensional reconstruction of the moving limb without the necessary reference to space limits and/or predetermined virtual objectives. It is thus clear how, differently from apparatuses known today, the present invention makes it possible to obtain greater information, such as the continuous progress over time of the posture during the exercise, the classification of the gesture/exercise, etc... Moreover, it allows the user, or the users, to self- evaluate their own movements, since it can be capable of comparing what has been carried out with respect to predetermined movements of a guide (teacher, professional, doctor, etc.). In the case of necessary movement rehabilitation, for example following a trauma, the user can carry out (at his own home or elsewhere) the rehabilitation movements advised by a doctor, self-evaluating the results since the system provides him with the information on how much his movements differ from those that he is to carry out. The self-evaluation can be carried out both in "realtime" during the actual carrying out of the movements and/or gestures, and "off-line" since the user is capable of re-watching any phase in any subsequent moment, since the postures, the movements and/or gestures are simultaneously recorded in a memory.
The evaluation can thus be carried out personally, or carried out immediately or subsequently by the teacher, professional, doctor, .. who evaluates also based upon the system according to the present patent.
Furthermore, thanks to the use of three-dimensional holographic or stereoscopic/auto-stereoscopic video signals, or of other nature, it is possible to obtain information on all the portion of body involved in the movement without necessarily artificially rotating or tilting the portion itself.
As a further advantage, it should be highlighted that the apparatus for detecting, classifying, teaching, training exercises according to the present invention does not in any way restrict the user to a particular work station, but allows free movement in the space. Finally, the displayed three-dimensional images of the exercise, irrespective of whether they are represented through holography, stereoscopy, auto-stereoscopy, or through simple images on the monitor, in addition to displaying the movements carried out by the user also the movements that should be carried out can be generated.
In other words the apparatus of the present invention can advise the user on the movement to be carried out displaying it beforehand.
In such a case, extremely useful in the field of rehabilitation, the apparatus can also provide the user with information about how his movement is or is not close to the correct set movement.
Preferably the apparatus can indeed display both the proposed exercise and simultaneously the exercise actually carried out by the user evaluating both numerically and visually when the exercise itself is carried out correctly. The apparatus is therefore capable of displaying, stereoscopically, auto-stereoscopically, three- dimensionally, on a screen etc., both the pre-set movements for which it is known the numerical quantitative data, since it has already been measured earlier, and the movements carried out by the user (or by the users) and of providing both numerical quantitative results on the correctness of the exercise carried out and advices on how to carry out the exercise correctly.
Such pre-recorded movements can be, for example, those recorded by a professional athlete, footballer, skier, cyclist, etc., or by yoga, martial art instructors, etc., as also a dance instructor or music teacher, or by a driving instructor, or by a doctor who indicates which exercises are necessary to carry out in order to achieve the rehabilitation goals, etc.
The user, in such a case, is thus asked to repeat the same movements and can see his own avatar possibly superimposed over the image of the avatar provided by the professional, teacher etc...
In particular, the movement carried out by the user (or by the users) is (are) free and not forced by any external agent .
The apparatus thus provides, based upon measured parameters, numerical quantitative results on the correctness of the exercise carried out.
Based upon this, it provides a numerical quantitative evaluation, for example on a scale from 0 to 100, of whether or not the exercise carried out corresponds to that indicated by the professional.
Moreover, the system is capable of providing evaluation on which were the incorrect movements and indication on which ones to correct because they differ from those indicated by more than a certain quantitative limit. Concerning the ability of the apparatus of processing the data in input, the latter can have self-evaluation capability in terms of both numerical quantitative performance and display qualitative performance.
Indeed the apparatus of the present invention can comprise an "intelligent" processor equipped with self- evaluation capabilities even referring to software that may be introduced on the market and/or already present through which dedicated industries and/or professionals propose exercises of physical training, of correct posture maintenance, more in general, for learning sport, gymnastics and rehabilitation exercises such as for example exercises of yoga, dancing, of free-style gymnastics or gymnastics with equipment and teaching all sport activities.
For each of these possible applications the intelligent system is capable of generating special software that focuses not only on the optimal carrying out of the exercises proposed but also on the opportunity of reducing the learning time of each exercise according to the exercise movements displayed by the user. It is very easy to understand how the apparatus for detecting, teaching and training exercises object of the present invention operates.
Indeed, the method implemented by the aforementioned apparatus for detecting, teaching and training exercises comprises the simple phases of:
- acquiring position and movement data relating to an exercise being carried out;
- processing said acquired data to generate information relating to the exercise being carried out;
returning to the user said information so as to promote teaching and training with the exercise being carried out.
In particular, the acquiring phase of the position and movement data related to the exercise being carried out is realized by wearing a garment provided with movement detection sensors; the processing phase comprises the phases of generating both a three-dimensional video signal reproducing the exercise being carried out, and a numerical signal relating to the exercise being carried out, of classifying the actions/movement highlighting whether and how it belongs to a predefined category, and of advising how to modify actions/movements in order to make them belong to a desired category; finally, the returning phase of the information to the user is realized through the displaying of the three-dimensional video signal reproducing the exercise being carried out and/or of the numerical signal and/or the simultaneous displaying both of the signal recorded by the user, and, by comparison, of a pre-set movement, for example by a professional .
As described earlier, the method is extremely advantageous in the case in which the phase of generating the three-dimensional video signal is realized through a three-dimensional image on a screen or again when generating a three-dimensional video signal of the stereoscopic, auto-stereoscopic or holographic type or other of the part of body or of the entire body that is moving.
As a preliminary phase it is possible to foresee the phase of showing data and figures of positions relative to physical/postural exercises to be carried out based upon quantitative parameters recorded from movements, for example, of a professional, whereas as final phases it is possible to foresee those of returning to the user information about the comparison between his activity and that advised by the professional as well as comparing and quantitatively evaluating whether the exercise carried out is correct or not.
It has thus been seen that the apparatus and the method for detecting, classifying, teaching and training exercises according to the present invention achieve the previously highlighted purposes.
The apparatus for detecting, classifying, teaching, training exercises of the present invention thus conceived can undergo numerous modifications and variants, all covered by the same inventive concept; moreover, all the details can be replaced by technically equivalent elements. In practice, the materials used, as well as their sizes, can be of any type according to the technical requirements.

Claims

1) Method for detecting, classifying, teaching and training exercises of individual users or groups comprising the phases of:
- acquiring position and movement data relating to an exercise being carried out;
- processing said acquired data to generate information relating to the exercise being carried out;
- returning said information to the user/s so as to help teaching and training with the exercise being carried out;
wherein
- said acquiring phase of said movement data relating to the exercise being carried out is realized by wearing one or more garments provided with movement detection sensors; said exercise being carried out in the absence of external forcing;
said processing phase comprises the phases of generating both a three-dimensional signal reproducing the exercise being carried out and a numerical signal relating to the exercise being carried out, of classifying the action/movement highlighting whether and how it belongs to a pre-defined category and of advising how to modify actions/movements in order to make them belong to a desired category;
- said returning phase of said information to the user, or to the users, being realized through the simultaneous displaying of said three-dimensional video signal reproducing the exercise being carried out, of a three-dimensional signal reproducing the exercise to be carried out and of a numerical signal comparing said three-dimensional video signals.
2) Method according to claim 1 characterized in that it comprises the phases of :
- classifying the exercise being carried out; and
- comparing said classified exercise with a group of movements and/or gestures catalogued in a pre-defined library.
3) Method according to claim 1 characterized in that said three-dimensional video signals are of the stereoscopic and/or auto-stereoscopic type, or of any other possible nature.
4) Method according to claim 1 characterized in that said three-dimensional video signals are of the holographic type .
5) Apparatus for the implementation of the method according to any of the preceding claims comprising: - a pre-set database containing guiding images that show the user, or the users, the movements to carry out ;
means for acquiring and classifying position and movement data relating to an exercise being carried out ;
- means for processing said acquired data to generate information relating to the exercise being carried out;
- means for returning said information to the user, or to the users, helping teaching and training of the exercise being carried out;
wherein
- said means for acquiring said movement data relating to the exercise being carried out comprising at least a garment provided with movement detection sensors;
- said means for processing said acquired parameters to generate information relating to the exercise being carried out comprising hardware and/or firmware and/or software both to generate a three-dimensional video signal reproducing the exercise being carried out and to generate a numeric and/or classifying signal relating to the exercise being carried out;
said means for returning said information to the user, or to the users, comprising means for the simultaneous displaying of said three-dimensional video signal reproducing the exercise being carried out, of a three-dimensional video signal reproducing the exercise to carry out and of a numerical signal comparing said three-dimensional video signals.
6) Apparatus according to claim 5 characterized in that said video signals are of the stereoscopic, or auto-stereoscopic or holographic type.
7) Apparatus according to claim 6 characterized in that said means for displaying said three-dimensional signals comprise a monitor.
8) Apparatus according to claim 7 characterized in that it comprises eyeglasses with polarized lenses for 3D-focusing of said three-dimensional video signals of the stereoscopic type.
9) Apparatus according to claim 7 characterized in that said monitor comprises means for automatic 3D- focusing of said three-dimensional video signals of the auto-stereoscopic type.
10) Apparatus according to claim 6 characterized in that said means for displaying said three-dimensional video signals of the holographic type comprise projectors for the automatic 3D-focusing of said three- dimensional video signals of the holographic type.
PCT/IB2011/000077 2010-01-18 2011-01-17 Apparatus and method for detecting and classifying so as to be able to evaluate and self-evaluate exercises WO2011086466A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP11710262A EP2525880A1 (en) 2010-01-18 2011-01-17 Apparatus and method for detecting and classifying so as to be able to evaluate and self-evaluate exercises

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
ITMI2010A000044 2010-01-18
ITMI2010A000044A IT1397737B1 (en) 2010-01-18 2010-01-18 EQUIPMENT AND METHOD OF DETECTION, TRAINING AND TRAINING OF MOTOR ACTIVITIES
ITMI2010A002130 2010-11-17
ITMI2010A002130A IT1402873B1 (en) 2010-01-18 2010-11-17 EQUIPMENT AND METHOD OF QUANTITATIVE MEASUREMENT AND CLASSIFICATION BY EVALUATION AND SELF-ASSESSMENT OF MOTOR ACTIVITIES

Publications (1)

Publication Number Publication Date
WO2011086466A1 true WO2011086466A1 (en) 2011-07-21

Family

ID=42617466

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2011/000077 WO2011086466A1 (en) 2010-01-18 2011-01-17 Apparatus and method for detecting and classifying so as to be able to evaluate and self-evaluate exercises

Country Status (3)

Country Link
EP (1) EP2525880A1 (en)
IT (2) IT1397737B1 (en)
WO (1) WO2011086466A1 (en)

Cited By (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2016112126A1 (en) * 2015-01-06 2016-07-14 Asensei, Inc. Movement based fitness and fitness product management
WO2016146834A1 (en) * 2015-03-18 2016-09-22 Lifesymb Holding Ab Optimizing recommendations in a system for assessing mobility or stability of a person
CN106075856A (en) * 2016-07-24 2016-11-09 鲁辰超 Self-service trainer aircraft
US9649052B2 (en) 2014-09-05 2017-05-16 Vision Service Plan Systems, apparatus, and methods for using eyewear, or other wearable item, to confirm the identity of an individual
CN106730648A (en) * 2017-01-03 2017-05-31 江苏德长医疗科技有限公司 Artificial scene rehabilitation platform
WO2017096436A1 (en) * 2015-12-10 2017-06-15 Guided Knowledge Ip Pty Ltd Frameworks and methodologies configured to enable real-time adaptive delivery of skills training data based on monitoring of user performance via performance monitoring hardware
US9910298B1 (en) 2017-04-17 2018-03-06 Vision Service Plan Systems and methods for a computerized temple for use with eyewear
US10215568B2 (en) 2015-01-30 2019-02-26 Vision Service Plan Systems and methods for tracking motion, performance, and other data for an individual such as a winter sports athlete
US10617342B2 (en) 2014-09-05 2020-04-14 Vision Service Plan Systems, apparatus, and methods for using a wearable device to monitor operator alertness
US10722128B2 (en) 2018-08-01 2020-07-28 Vision Service Plan Heart rate detection system and method
US10806982B2 (en) 2015-02-02 2020-10-20 Rlt Ip Ltd Frameworks, devices and methodologies configured to provide of interactive skills training content, including delivery of adaptive training programs based on analysis of performance sensor data
US10942968B2 (en) 2015-05-08 2021-03-09 Rlt Ip Ltd Frameworks, devices and methodologies configured to enable automated categorisation and/or searching of media data based on user performance attributes derived from performance sensor units
US20210125148A1 (en) * 2019-10-24 2021-04-29 Accenture Global Solutions Limited Artificial intelligence based project implementation
CN113160984A (en) * 2021-04-20 2021-07-23 郑州大学第一附属医院 Nasal nursing sensitivity quality evaluation index system
US11918375B2 (en) 2014-09-05 2024-03-05 Beijing Zitiao Network Technology Co., Ltd. Wearable environmental pollution monitor computer apparatus, systems, and related methods

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5846086A (en) 1994-07-01 1998-12-08 Massachusetts Institute Of Technology System for human trajectory learning in virtual environments
US20020146672A1 (en) 2000-11-16 2002-10-10 Burdea Grigore C. Method and apparatus for rehabilitation of neuromotor disorders
US20040046777A1 (en) 1995-11-30 2004-03-11 Virtual Technologies, Inc. Tactile feedback man-machine interface device
US20040174337A1 (en) 2002-12-27 2004-09-09 Akira Kubota Force-feedback supply apparatus and image correcting method
US20040219964A1 (en) 2002-03-01 2004-11-04 Delmar Bleckley Ergonomic motion and athletic activity monitoring and training system and method
US20060137511A1 (en) 2003-06-06 2006-06-29 Mcgregor Rob Musical teaching device and method
US7084884B1 (en) 1998-11-03 2006-08-01 Immersion Corporation Graphical object interactions
US20080038702A1 (en) 2004-09-27 2008-02-14 Claude Choquet Body Motion Training and Qualification System and Method
US20090237763A1 (en) 2008-03-18 2009-09-24 Kramer Kwindla H User Interaction with Holographic Images

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5846086A (en) 1994-07-01 1998-12-08 Massachusetts Institute Of Technology System for human trajectory learning in virtual environments
US20040046777A1 (en) 1995-11-30 2004-03-11 Virtual Technologies, Inc. Tactile feedback man-machine interface device
US7084884B1 (en) 1998-11-03 2006-08-01 Immersion Corporation Graphical object interactions
US20020146672A1 (en) 2000-11-16 2002-10-10 Burdea Grigore C. Method and apparatus for rehabilitation of neuromotor disorders
US20040219964A1 (en) 2002-03-01 2004-11-04 Delmar Bleckley Ergonomic motion and athletic activity monitoring and training system and method
US20040174337A1 (en) 2002-12-27 2004-09-09 Akira Kubota Force-feedback supply apparatus and image correcting method
US20060137511A1 (en) 2003-06-06 2006-06-29 Mcgregor Rob Musical teaching device and method
US20080038702A1 (en) 2004-09-27 2008-02-14 Claude Choquet Body Motion Training and Qualification System and Method
US20090237763A1 (en) 2008-03-18 2009-09-24 Kramer Kwindla H User Interaction with Holographic Images

Cited By (29)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10188323B2 (en) 2014-09-05 2019-01-29 Vision Service Plan Systems, apparatus, and methods for using eyewear, or other wearable item, to confirm the identity of an individual
US11918375B2 (en) 2014-09-05 2024-03-05 Beijing Zitiao Network Technology Co., Ltd. Wearable environmental pollution monitor computer apparatus, systems, and related methods
US10694981B2 (en) 2014-09-05 2020-06-30 Vision Service Plan Wearable physiology monitor computer apparatus, systems, and related methods
US9649052B2 (en) 2014-09-05 2017-05-16 Vision Service Plan Systems, apparatus, and methods for using eyewear, or other wearable item, to confirm the identity of an individual
US10617342B2 (en) 2014-09-05 2020-04-14 Vision Service Plan Systems, apparatus, and methods for using a wearable device to monitor operator alertness
US10542915B2 (en) 2014-09-05 2020-01-28 Vision Service Plan Systems, apparatus, and methods for using a wearable device to confirm the identity of an individual
US9795324B2 (en) 2014-09-05 2017-10-24 Vision Service Plan System for monitoring individuals as they age in place
US10448867B2 (en) 2014-09-05 2019-10-22 Vision Service Plan Wearable gait monitoring apparatus, systems, and related methods
US10307085B2 (en) 2014-09-05 2019-06-04 Vision Service Plan Wearable physiology monitor computer apparatus, systems, and related methods
GB2551062B (en) * 2015-01-06 2019-11-13 Asensei Inc Movement based fitness and fitness product management
US10360811B2 (en) 2015-01-06 2019-07-23 Asensei, Inc. Movement based fitness and fitness product management
GB2551062A (en) * 2015-01-06 2017-12-06 Asensei Inc Movement based fitness and fitness product management
WO2016112126A1 (en) * 2015-01-06 2016-07-14 Asensei, Inc. Movement based fitness and fitness product management
US11302214B2 (en) 2015-01-06 2022-04-12 Asensei, Inc. Movement based fitness and fitness product management
US10533855B2 (en) 2015-01-30 2020-01-14 Vision Service Plan Systems and methods for tracking motion, performance, and other data for an individual such as a winter sports athlete
US10215568B2 (en) 2015-01-30 2019-02-26 Vision Service Plan Systems and methods for tracking motion, performance, and other data for an individual such as a winter sports athlete
US10806982B2 (en) 2015-02-02 2020-10-20 Rlt Ip Ltd Frameworks, devices and methodologies configured to provide of interactive skills training content, including delivery of adaptive training programs based on analysis of performance sensor data
US10918924B2 (en) 2015-02-02 2021-02-16 RLT IP Ltd. Frameworks, devices and methodologies configured to enable delivery of interactive skills training content, including content with multiple selectable expert knowledge variations
WO2016146834A1 (en) * 2015-03-18 2016-09-22 Lifesymb Holding Ab Optimizing recommendations in a system for assessing mobility or stability of a person
US10942968B2 (en) 2015-05-08 2021-03-09 Rlt Ip Ltd Frameworks, devices and methodologies configured to enable automated categorisation and/or searching of media data based on user performance attributes derived from performance sensor units
US11074826B2 (en) 2015-12-10 2021-07-27 Rlt Ip Ltd Frameworks and methodologies configured to enable real-time adaptive delivery of skills training data based on monitoring of user performance via performance monitoring hardware
WO2017096436A1 (en) * 2015-12-10 2017-06-15 Guided Knowledge Ip Pty Ltd Frameworks and methodologies configured to enable real-time adaptive delivery of skills training data based on monitoring of user performance via performance monitoring hardware
CN106075856A (en) * 2016-07-24 2016-11-09 鲁辰超 Self-service trainer aircraft
CN106730648A (en) * 2017-01-03 2017-05-31 江苏德长医疗科技有限公司 Artificial scene rehabilitation platform
US9910298B1 (en) 2017-04-17 2018-03-06 Vision Service Plan Systems and methods for a computerized temple for use with eyewear
US10722128B2 (en) 2018-08-01 2020-07-28 Vision Service Plan Heart rate detection system and method
US20210125148A1 (en) * 2019-10-24 2021-04-29 Accenture Global Solutions Limited Artificial intelligence based project implementation
US11755999B2 (en) * 2019-10-24 2023-09-12 Accenture Global Solutions Limited Artificial intelligence based project implementation
CN113160984A (en) * 2021-04-20 2021-07-23 郑州大学第一附属医院 Nasal nursing sensitivity quality evaluation index system

Also Published As

Publication number Publication date
ITMI20100044A1 (en) 2011-07-19
IT1402873B1 (en) 2013-09-27
ITMI20102130A1 (en) 2011-07-19
EP2525880A1 (en) 2012-11-28
IT1397737B1 (en) 2013-01-24

Similar Documents

Publication Publication Date Title
WO2011086466A1 (en) Apparatus and method for detecting and classifying so as to be able to evaluate and self-evaluate exercises
US20210272376A1 (en) Virtual or augmented reality rehabilitation
Velloso et al. Qualitative activity recognition of weight lifting exercises
EP2451339B1 (en) Performance testing and/or training
US9754512B2 (en) Real-time feedback of task performance
KR101520113B1 (en) Unitary vision and neuro-processing testing center
Bleser et al. A personalized exercise trainer for the elderly
Melero et al. Upbeat: augmented reality-guided dancing for prosthetic rehabilitation of upper limb amputees
US20140148931A1 (en) Running form diagnosis system and method for scoring running form
JP5575652B2 (en) Method and system for selecting display settings for rendered images
O'Reilly et al. Mobile app to streamline the development of wearable sensor-based exercise biofeedback systems: system development and evaluation
Nunes et al. Human motion analysis and simulation tools: a survey
Echeverria et al. KUMITRON: Artificial intelligence system to monitor karate fights that synchronize aerial images with physiological and inertial signals
JP2005111178A (en) Motion training display system
Patil et al. Body posture detection and motion tracking using ai for medical exercises and recommendation system
Fasko et al. Towards human activity recognition and objective performance assessment in human patient simulation: A case study
US20230169880A1 (en) System and method for evaluating simulation-based medical training
Richter et al. Assisted Motion Control in Therapy Environments Using Smart Sensor Technology: Challenges and Opportunities
Beacon et al. Assessing the suitability of Kinect for measuring the impact of a week-long Feldenkrais method workshop on pianists’ posture and movement
Echeverria et al. KUMITRON: A Multimodal Psychomotor Intelligent Learning System to Provide Personalized Support when Training Karate Combats.
Ukita et al. A user-centered design approach to physical motion coaching systems for pervasive health
Durve et al. Machine learning approach for physiotherapy assessment
Noh et al. Smart exercise application to improve LEG function and short-term memory through game-like lunge exercises: development and evaluation
Kahtan et al. Motion analysis-based application for enhancing physical education
Kutafina et al. Learning Manual Skills with Smart Wearables

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 11710262

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

WWE Wipo information: entry into national phase

Ref document number: 2011710262

Country of ref document: EP