WO2003015056A2 - Automated behavioral and cognitive profiling for training and marketing segmentation - Google Patents

Description

Automated Behavioral and Cognitive Profiling for Training and Marketing Segmentation

Inventors:

Eberhard Schmidt

WulfTeiwes

Assignee:

Visual Interaction GmbH

Priority Date:

August 9, 2001

References Cited:

Background of the invention

Plenty background art is available that describes intelligent training and tutoring methodologies and systems as well as systems and methodologies to profile users for marketing segmentation. Prior art that comes closest to the invention are cited in the references above. However none of the background art attempts to automate the whole process of non-intrusively observing the user, profiling the user and/or adjusting the user interface according to the user profile thereafter. Also users prior to the invention have been profiled with questionnaires along social-demographic differentiators. The invention however targets to profile the behavior of the user tracking natural human factors such as gesture, eye movements, head movements, mimics, gaze point on the content, voice, keyboard and mouse input in addition to the social-demographic factors and potential other psychological methods based e.g. on questionnaires. Under normal environmental conditions behavioral profiles tend to remain relatively stable. Thus a comprehensive and normative database of behavioral profiles also results in standardized behavioral segmentation.

US6338628 proposes a personal training and development system. A user logs on a computer and is led through some questionnaires (four or five lists of 18 statements each) to fill in manually. The application matches the answers to a Kinsel-Hartman Profile to unveil strengths and weaknesses of the user and propose training accordingly. The profile test is repeated periodically to benchmark training success and alter the training style. Users are profiled manually with questionnaires and profiles are not updated continuously. US5551880 proposes a sophisticated employee success system. Behavioral information is derived from the individuals through questionnaires. The system then analyzes the answers and compares these against standards for behavior and values previously calculated for a specific job. The output is a one-off benchmark report only and does not imply any customized training thereafter.

US5326270 describes a system and method for assessing an individual's task-processing style. This time the user is put in a case study with a simulated situation and the individual's responses and time for resolving the situation are recorded and statistically analyzed to benchmark the individual against others. Again, the method is mainly used to decide whether an individual is suitable for a job and only manually implies to propose some training to improve the user's ability to handle tasks.

US 6164975 describes an interactive instructional system using adaptive cognitive profiling. The system derives at' a cognitive profile of the user via presenting different multimedia presentations of the same content. Via a utility function the system is testing the comprehension of content by the user for the different representations and establishes a cognitive profile and continuously updates the utility function. The neither claim an adaptive training session nor automatically tracks the user's cognitive behavior. US5597312 discloses an intelligent tutoring method and system comprising a computer system for selecting adjustable teaching parameters and a student model which is monitored and updated. Again the system lacks to automatically sense the user's behavior as the performance is watched along manually provided teaching parameters. US6298328 discloses an apparatus, method, and system for sizing markets. The system utilizes a general purpose computer and simulates market and submarket evolutions by taking weighted coefficients for parameters such as products, geographic area, market segment, provider, time period, regional market data, demographic, psychographic and/or firmagraphic data and profiles. It requires the operator to weight the different coefficient and does not observe the behavior of all individuals to derive at different socio-demographic, behavioral and cognitive profiles for market segmentation as the invention does.

US6236975 discloses a system and method for profiling customers for targeted marketing. The marketing system presents selected questions to an individual, receives responses and stores the data. The data is then analyzed in comparison to a selected peer group to the individual. A report is provided to give immediate graphical feedback of the individual's standing in respect to the peer group. The system however requires the user to respond to several questions and does not observe the individual non-intrusively. It also lacks to observe a behavioral profile and concentrates on socio-demographic information. US5848396 and a second filing US5991735 of the same inventor disclose a method, apparatus and computer program for determining behavioral profile of a computer user. The system presents via a network several visual displays, such as market or stock data, theatre or television schedules, weather and/or travel information and advertisements, to a targeted user group in order to obtain psychographic profiles. The user is tracked while processing the information, especially the individual's way of formatting graphically presented information such as color schemes, text sizes and shapes, is recoded. All together with demographic data, a psychographic profile is established and via regression analysis constantly updated. With the profile a customized displaying is possible. However the system lacks to record non- intrusively human-factor input and behavior such as gesture, mimics, head movements or eye movements on the content and the system only profiles via pre-defined artificial images presented to the individual. The system also foregoes an opportunity to use profile information for marketing segmentation.

Summary of the invention

A methodology and system is disclosed to utilize multi-variant analysis techniques or other statistical or heuristic methods to cluster users based on behavior and socio-demographic data. Behavioral and socio-demographic data is retrieved while observing users via sensors and the recording of user inputs.

One aspect of the disclosed methodology allows automating an objective training process. Trainees are observed during training sessions and user profile is recorded that includes behavioral factors, socio-demographic factors and potential other psychological methods based e.g. on questionnaires. The observation system comprises multi-modal user input devices and user sensors that sense the behaviors of the trainees extracted from the trainees head and eye movements, mimics and gestures, voice, physical stress conditions as inputs coming from mouse, joystick or any other haptic device. The trainee socio-demographic and behavioral profile is automatically matches to expert profiles solving the same task and thereby to target success strategy for the trainee to solve the task. The system is able to monitors training performance in multiple sessions, adapts content, the training process or the user interface to best suit the individual trainee and thereby trains the trainee fully automatically towards the target success strategies for solving tasks on hand.

Another aspect of the disclosed methodology allows automating an objective segmentation process of users in marketing and advertising. The method of market segmentation does not match expert behavior, performance and target success strategies to trainee behavior. It however matches different tasks, stimuli or marketing messages to segmented clusters of users. Again the observation system comprises multi-modal user input devices and user sensors that sense the behaviors of the users extracted from the users head and eye movements, mimics and gestures, voice, physical stress conditions as inputs coming from mouse, joystick or any other haptic device. Socio-demographic profiles and the behavior of the users are assessed and clustered for the messages or stimuli exposed to the user. The users are therefore objectively and automatically segmented based on socio-demographic and behavioral information while watching marketing messages or content, or other stimuli or performing tasks. The process allows also building a strong database of behavioral reference profiles that automatically links the best suited message, content or stimuli to users observed. The segmentation process copes with single user observation but also with the observation of selected focus groups and panels.

Under normal environmental conditions behavioral profiles tend to remain relatively stable. Thus a comprehensive and normative database of behavioral profiles will also result in standardized behavioral segmentation.

Referring the background art cited, the invention allows monitoring and adjusting a training process as well as segmenting users and adjusting content, messages or stimuli in marketing or advertising. Outstanding criteria of the invention are:

♦ Objective, automatic and adaptive methods for training and market segmentation

♦ User profiles are based on socio-demographic and behavioral data

♦ Behavioral data is captured non-intrusively from user via user sensors and recording of user inputs

♦ Profiles are continuously updated

Brief description of the drawings

FIG. 1 illustrates the tutoring and training method according to the preferred embodiment of the present invention.

FIG. 2 illustrates the tutoring and training system according to the preferred embodiment of the tutoring and training method as of FIG. 1.

FIG. 3 illustrates the method for market segmentation according to the preferred embodiment of the present invention.

FIG. 4 illustrates the system for market segmentation according to the preferred embodiment of the market segmentation method as of FIG. 3.

Detailed description of the preferred embodiment

A methodology and system is disclosed to utilize multi-variant analysis techniques or other statistical or heuristic methods to cluster users based on behavior and socio-demographic data. Behavioral and socio-demographic data is retrieved while observing users via sensors and the recording of user inputs.

FIG. 1 explains the disclosed method for tutoring and training subjects. In preparation of the training, tasks and goals for the training are defined and entered into the system (©). Training goals and the tasks are qualified via performance measures, such as time to results, quality of the anticipated results, negative side effects, costs, but not limited to these. Upon definition of the performance measures, experts are observed while solving the tasks recording a variety of user inputs to the training system and sensing the behaviors of the experts ((D). Inputs could come from mouse, joystick or any other haptic device, keyboard. User passive behavior such as eye movements on the task, head movements, mimics, gestures, speech and physical stress could be recorded by variety of user sensors directed on or mounted to the experts. Behavioral aspects include among others e.g. duration of fixations, speed of eye movements, facial expression, certain activity sequences. All recorded data from the expert users are used to cluster behavior and performance analytically or manually and thereby define behavioral success strategies for solving the tasks. The preparation process can iterated to observe users of very different levels of expertise for an adaptive training process. Expert behavior, performance and success strategies are stored in a reference database of expert profiles. The profiling of experts is optional and can be skipped if a reference database of expert profiles is in place already or target success strategies are manually defined.

With a reference database in place, trainees are observed in the same way while solving the tasks (®). Again behavior and performance of the trainees are recorded via different user input devices and sensors. Both the trainees and the training sessions are profiles and assessed (©). The profile of the trainee is matched to a corresponding behavioral and/or socio- demographic expert profile with an accompanying target success strategy for the task. Once the matching was successful, training sessions can be modified and adjusted to train target success strategies (©). The user interface of the training program but also the solution process can be modified to improve the trainee's performance.

The training results can be fed back into the training methodology. One feedback loop according to FIG. 1 can adapt training goals and task definition for establishing a modified reference database. Iterative training sessions for the trainee are possible to monitor training improvements. Training sessions can be adjusted to reflect trainees growing expertise in iterative training sessions.

The automated multi-modal and objective user behavior observation is directed to create a comprehensive and normative database behavioral profiles that allows predicting online the trainee's behavior while acting on new tasks. This enables the trainer to adjust the training to the trainee proactively as the behavior of the trainee is predictive.

FIG. 2 exhibits the system best suited to tutoring and training methodology. Expert user and trainees 1 are recoded via user sensors 3 and multi-modal user input devices 4. One possible sensor would be at least one imaging sensor that records eye movements, head movements and gestures of the user. A voice recorder can record the user's speech, medical diagnosis system could record stress symptoms of the user. Also different multi-modal user input devices can be observed such as keyboard, mouse, joystick or any other haptic device but not limited to these. While the user sensors and input devices are mainly observing the behavior of the user, additional socio-demographic data can be entered, e.g. via the use of questionnaires. User sensors 3, multi-modal user input devices 4 and other user data is fed into the central processing unit 6, which could be part of the training system 5 but do not have to. A recorder 7 captures the stream of available user data, which could be clicks and moves of mouse and joystick or any other haptic device, eye movement coordinates on the user screen of the training system 5, eye saccades, head movements, gestures and facial marks, voice commands and many more and are synchronized with one single time code.

Is the user represented by an expert user as explained above, all recorded data is stored in the expert profile and success strategy database 8. The recorded data can also be filtered to store just relevant data in database 8 according to performance measures previously defined. The database differentiates expert behavior and performance for a task and establishes success strategies for resolving the tasks, which a stored in the database as well.

Is the user represented by a trainee user as explained above, all recorded data is stored in the trainee profile database 9. Again the recorded data can be filtered to store just relevant data in database 9 according to performance measures previously defined. The database 9 differentiates trainee behavior and performance for a task. Part of the system can be an interpreter 11 to that matches the expert profiles and success strategies of the expert profile and success strategy database 8 with the trainee behavior and performance logged in the trainee profile database 9. The interpreter 11 can both work online and offline, i.e. online while the training is performed for instantaneous guidance, user interface adaptation or training process adaptation or offline once a training session is performed to monitor results and later discussion.

The interpreter 11 thereby also can have access to a database of different training programs 10 best suited for the target success strategies solving the training task. That database 10 could be held separate as drawn or be integrated in the other databases 8 or 9. The different training programs are used to modify the training sessions via adapting the process or user interface.

FIG. 3 exhibits a methodology for market segmentation that differs from the method of tutoring and training. The method of market segmentation does not match expert behavior, performance and target success strategies to trainee behavior. It however matches different stimuli to segmented clusters of users. With the method, goals, e.g. messages, tasks, stimuli target user segment, and performance measures are defined (©). Different standard profile segmentations can optionally be generated e.g. a standard socio-demographic segmentation using age, income and education variables, or a standard segmentation using innovation adoption types etc. to pre-cluster users according to standard market research profile segments.

Upon definition of the goals and performance measures, users are observed and tested recording a variety of user inputs to the stimuli presentation system and sensing the behaviors of the users (©). Inputs could come from mouse, joystick or any other haptic device, keyboard. User passive behavior such as eye movements on the task, head movements, mimics, gestures, speech and physical stress could be recorded by variety of user sensors directed on or mounted to the users. Behavioral aspects include among others e.g. duration of fixations, speed of eye movements, facial expression, certain activity sequences. Testing can be performed for single users, focus groups but also panels. The user testing is optional and can be skipped if a central database of abstracted reference profiles of user behavior, e.g. per type of message, stimulus, etc., is in place. All recorded data from the users are used to assess behavior and performance analytically or manually and thereby profile user behavior and build an abstracted central reference database of user behavioral profiles. Users observed are clustered across segments, e.g. socio-demographic or behavioral segmentation (®), while the segmentation information could either come from profiles defined in the dedicated test or directly from the central reference database, if in place.

Performance, e.g. marketing, user interface, stimuli, etc, is analyzed per user or per segment (©). An abstraction of created user behavioral profiles, e.g. per type of marketing message, stimuli, etc. is stored in a central reference database, which builds up over time (©).

The method also comprises three feedback loops, one to vary user segments and test more users before even clustering the users to find more behavioral profiles that can be stored in the central reference database, on to cluster a varied users segment based on the central reference database. The third loop allows adapting messages, stimuli, and goals or tasks globally or per segment before extracting new behavioral profiles or clustering more users based on the central reference database.

The process results in a central database of user behaviors related to the type of message, task, stimuli or target user segment and allow adapting messages and stimuli preventive to the observed user behavior. The methodology can be used in a lab based environment with a small number of pre-selected subjects but also in panel research.

FIG. 4 exhibits the system adapted to best suit market segmentation. Upon definition of goals, e.g. messages, tasks, stimuli and target user segment, and performance measures users 12 are recoded via user sensors 3 and multi-modal user input devices 4. One possible sensor would be at least one imaging sensor that records eye movements, head movements and gestures of the user. A voice recorder can record the user's speech, medical diagnosis systems could record stress symptoms of the users. Also different multi-modal user input devices can be observed such as keyboard, mouse, joystick or any other haptic device but not limited to these. While the user sensors and input devices are mainly observing the behavior of the user, additional socio-demographic data can be entered, e.g. via the use of questionnaires. User sensors 3, multi-modal user input devices 4 and other user data is fed into the central processing unit 6, which could be part of the stimuli presentation system 16 but do not have to. A recorder 7 captures the stream of available user data, which could be clicks and moves of mouse and joystick or any other haptic device, eye movement coordinates on the user screen of the training system 5, eye saccades, head movements, gestures and facial marks, voice commands and many more and are synchronized with one single time code.

All recorded user data is stored in a user database 13. The recorded data can also be filtered to store just relevant data in database 13 according to performance measures previously defined.

The interpreter 11 extracts abstracted user behavioral profiles depending on message, stimuli and task and stores these in the central reference database 14. All related messages and stimuli are logged in a stimuli database 15. This stimuli database 15 could well be part of the cluster profile database 13 but do not have. All databases 13, 14 and 15 could be held separated or could be combined.

While the central reference database 14 gains momentum an interpreter 11 could observe user behavior and matches best suited messages or stimuli in the stimuli database 15 to the user profile on hand, and adapts stimuli offerings preventively to the users either online or offline.

Results are provided through a multi-modal behavior monitor:

In the preferred embodiment of tutoring and training performance and improvements of the trainees are monitored. The multi-modal monitor can be used to supervise trainees and alert the trainer to specific deficits in performance that the trainer should watch more carefully. Alternatively, training can be automated to close the identified gap by adapting content and the user interface best suited to the trainees' proficiency and the task on hand. Comparisons between different training rounds and stages for the same user and task can be used to analyze individual progress and to indicate the most valuable areas of future training for the individual trainee in order to approach the target performance and target profile with a personal training program.

In the preferred embodiment of marketing segmentation the multi-modal behavior monitor is used to present the different user profiles found. User segments are presented combining both behavioral and socio-demographic data. An additional database matches user segments with content representations that worked and that did not work for the relevant segment using quantitative behavioral feedback and/or qualitative feedback from the users. List of Drawing Objects

1 User

2 Other user data, e.g. questionnaires

3 User sensors

4 Multi-modal User Input Devices

5 Training System, e.g. PC, Flight Simulator, Car, software on PC

6 Central processing unit

7 Recorder

8 Expert Profile & Success Strategy Database

9 Trainee Profile Database

10 Training Programs Database

11 Interpreter

12 User Cluster

13 User Database

14 Central Reference Database

15 Database of stimuli, e.g. content representations

16 Stimuli Presentation

17 User Profiles

18 Reference Profiles

Claims

ClaimsWe claim:

1. A methodology for tutoring and training a trainee comprising the steps of: a. Definition of target success strategy or strategies including behavioral components for solving a task; b. Having the trainee perform said task; c. Profiling and assessing trainee's behavior patterns and the training session while performing said task; d. Comparing trainee's behavior patterns to identify the target success strategy or strategies for training; e. Adjusting training based on said comparison to best fit trainee's personal target success strategy or strategies; f. Comparing said trainee's behavior patterns of each step c. in order to analyze progress and to indicate the most valuable areas of future training for the trainee; and g. Repeating steps b. and c, optionally d. and e..

2. Methodology for tutoring and training a trainee as claimed in claim 1 comprising the further step a.l of defining target success strategy or strategies by observing experts' behavior and performance and clustering the results.

3. Methodology for tutoring and training a trainee as claimed in claim 1 comprising the further step c.l of highlighting performance gaps relatively to the target success strategy or strategies.

4. Methodology for tutoring and training a trainee as claimed in claim 1 comprising the further step c.2 of supervising trainee and alerting the trainer to specific deficits in performance that the trainer should watch more carefully.

5. Methodology for tutoring and training a trainee as claimed in claim 1 comprising the further step e.l of adapting training goals and task definitions based on the comparison of user behavioral patters and performance to said personal target success strategy or strategies.

6. Methodology for tutoring and training a trainee as claimed in claim 1, wherein training is automated to close the identified gap by adapting content, the training process and the user interface best suited to the trainees' proficiency, personal target success strategy or strategies and said task.

7. Methodology for tutoring and training a trainee as claimed in claim 1, wherein results are provided through a multi-modal behavior monitor.

8. Methodology as claimed in claim 7, wherein the multi-modal behavior monitor comprises a monitor to track performance and improvements of the trainee, to alert the trainer about deficits or automate the training process closing the identified gap by adapting content, training process and the user interface best suited to the trainee's proficiency, personal target success strategy or strategies and said task.

9. Methodology as claimed in claim 8, wherein the monitor also compares different training rounds and stages of the same trainee and task to analyze individual progress and to indicate the most valuable areas of future training for the individual trainee in order to approach the target performance and target success strategy or strategies with a personal training program.

10. A methodology for marketing segmentation comprising the steps of: a. Observing subjects while exposed to stimuli; b. Clustering users based on user behavioral and socio-demographic profiles; c. Analyzing performance using quantitative behavioral feedback and/or qualitative feedback from the subjects; d. Abstracting user behavior profiles per type of stimuli; e. Repeating steps a. through d. varying user segments from which subjects are selected; f. Comparing marketing performance of each user segment;

11. A methodology for marketing segmentation comprising the steps of: a. Observing subjects while exposed to stimuli; b. Clustering users based on user behavioral and socio-demographic profiles; c. Analyzing performance using quantitative behavioral feedback and/or qualitative feedback from the subjects; d. Abstracting user behavior profiles per type of stimuli; e. Repeating steps a. through d. with different stimuli; f. Comparing marketing performance for each stimulus.

12. Methodology as claimed in claim 10 or 11, wherein stimuli can be text, graphics, film, web pages, tasks, marketing messages or content, software and other textual and graphical representations.

13. Methodology as claimed in claim 10 or 11 comprising the further step d.l of building a reference database of abstracted user behavioral and socio-demographic profiles per type of stimuli.

14. Methodology as claimed in claim 10 or 11 comprising the further step b.l of clustering users based on socio-demographic and behavioral profiles extracted during the test or stored in a reference database.

15. Methodology as claimed in claim 10 or 11, wherein results are provided through a multi- modal behavior monitor.

16. Methodology as claimed in claim 15, wherein the multi-modal behavior monitor comprises a monitor to present the different user profiles found in the marketing segmentation.

17. Methodology as claimed in claim 16, wherein the monitor presents user segments combining both behavioral and socio-demographic data together with content representations that worked and that did not work for the relevant segment using quantitative behavioral feedback and/or qualitative feedback from the users.

18. An observation system for behavioral profiling comprising: a. a central processing unit; b. at least one user sensor connected to the central processing unit; c. a recorder to collect all data coming from the at least one user sensor and stimuli information; d. an interpreter to extract target reference profiles and to extract user behavioral profiles; and e. A matching function that matches extracted user behavioral patterns to target reference profiles.

19. An observation system as claimed in claim 18, wherein the observation system comprises at least one input device which is connected to the central processing unit.

20. An observation system as claimed in claim 18, wherein all events, target reference profiles and user behavioral patterns are stored in a database.

21. An observation system as claimed in claim 18, wherein the user sensor is an eyetracker, head tracker, face and gesture tracker, medical diagnosis system to measure physical stress.

22. An observation system as claimed in claim 19, wherein the at least one input device is a microphone, keyboard, mouse, or haptic device.

23. An observation system as claimed in claim 18, wherein said at least user sensor is a imaging sensor for imaging a user's eye movements, head movements, facial marks or gestures and for interfacing video information to the central processing unit; an audio input device for interfacing audio information to the central processing unit or a medical diagnosis system.

24. An observation system as claimed in claim 18, wherein said at least user sensor further comprises a voice interpreter for interpreting the voice and speech of a user transferred via an audio input device to the central processing unit in form of digitized events and forwards them to the event logger.

25. An observation system as claimed in claim 18, wherein the recorder captures all events coming from human factor input devices, such events including mouse clicks and moves, joystick clicks and moves, keyboard strokes, numbers and letters, eye coordinates, fixations and saccades, head coordinates and movements, a discrete number of facial marks and expressions, a discrete number of voice commands, and wherein all events are synchronized with a single time code.

26. An observation system as claimed in claim 18, wherein the interpreter utilizes multi- variant analysis techniques or other statistical or heuristic methods to create user profiles based on behavior and socio-demographic data, wherein the interpreter matches expert profiles and target success strategies to the trainee behavior and performance and links different training programs to the training system.

27. An observation system as claimed in claim 18, wherein the interpreter utilizes multi- variant analysis techniques or other statistical or heuristic methods to create user profiles based on behavior and socio-demographic data; wherein the interpreter extracts abstracted user behavioral profiles depending on message, stimuli and task to cluster users across segments and adjust messages, task and/or stimuli for the user.

28. An observation system as claimed in claim 24, wherein the interpreter could operate offline and/or online.

29. An observation system as claimed in claim 26, wherein the database is further defined by: a. Storing distinctive user profiles based on behavior and socio-demographic data. b. Linking the user profiles to distinctive event logs. c. Linking the user profiles to different tasks. d. Linking the user profiles to different content representations. e. Linking the user profiles to different performance criteria. f. Linking the user profiles to different levels of expertise. g. Linking the user profiles to adaptive content, processes and user interface presentations.

30. An observation system as claimed in claim 24, wherein the matching function is further defined by: a. A task specific quality target function that matches observed user behavior profiles to target reference profiles; b. Highlighting performance gaps relatively to a targeted expert profiles; c. Supervising the trainee and alert the trainer to specific deficits in performance that the trainer should watch more carefully; d. Automating training to close the identified gap by adapting content and the user interface best suited to the trainee's proficiency and said task; e. Comparing different training rounds and stages for the same trainee and task to analyze individual progress and to indicate the most valuable areas of future training for the individual trainee in order to approach the target performance and target profile with a personal training program.