WO2001016882A1

WO2001016882A1 - An interaction process

Info

Publication number: WO2001016882A1
Application number: PCT/AU2000/001013
Authority: WO
Inventors: Alan Leslie Liddle; Martin John Stannard; Roy Charles Stannard
Original assignee: Bioadaptive Systems Pty Ltd
Priority date: 1999-08-27
Filing date: 2000-08-25
Publication date: 2001-03-08
Also published as: AUPQ249799A0

Abstract

An interaction process executed on a computer system, including generating an output based on at least one string of a population, the at least one string representing elements of the output, receiving a response to the output caused by a user of the system, and executing a genetic algorithm (GA) on the population to generate a subsequent generation of the population using the response.

Description

AN INTERACTION PROCESS

The present invention relates to an interaction process and a system and software for executing the process.

Interaction between humans and machines has been the subject of considerable research, with recent work being conducted on the way humans work and interact with computers and the processes computers execute. For software developers, in particular game developers, one aim is to provide an interface and process which reacts and develops in response to a user's communication with the software, similar to that which would occur if the user was interacting or communicating with a human.

For example, in the context of a video game, developers seek to provide a challenging and rewarding game for users with a variety of skills and capabilities. To try and achieve this, developers provide games with different levels, normally having a hierarchical basis with the initial levels being relatively simple and then increasing in difficulty as each level is completed. This hierarchical structure however tends to be inflexible and is the same for all players, which can be tedious for skilled players and off putting for novices. An alternative is to allow users to enter profile or preference data which the software can use to adjust the game process based on the user's personal characteristics. This is a relatively limited solution as the adjustments are restricted to the information fields on which the software can operate and the variations on the entries which are permitted. Also any further changes to the process is predicated on users adjusting the personal profile data.

It is desired to allow a user to have a more direct role in the development or evolution of a process executed by a computer system, or at least provide a useful alternative. The present invention provides an interaction process executed on a computer system, including: generating an output based on at least one string of a population, said at least one string representing elements of said output; receiving a response to said output caused by a user of said system; and executing a genetic algorithm (GA) on said population to generate a subsequent generation of said population using said response.

The present invention also provides a computer system, including: means for generating an output based on at least one string of a population, said at least one string representing elements of said output; means for receiving a response to said output caused by a user of said system; and means for executing a genetic algorithm (GA) on said population to generate a subsequent generation of said population using said response.

The present invention also provides software stored on at least one storage medium, including: code for generating an output based on at least one string of a population, said at least one string representing elements of said output; code for receiving a response to said output caused by a user; and code for executing a genetic algorithm (GA) on said population to generate a subsequent generation of said population using said response.

A preferred embodiment of the present invention is hereinafter described, by way of example only, with reference to the accompanying drawings, wherein: Figure 1 is a schematic diagram of a preferred embodiment of a computer system executing an interaction process;

Figure 2 is a block diagram of the format of a pattern string of the system; and Figure 3 is a block diagram of the software modules of the system. An interaction process is executed by a standard computer system 2, as shown in the Figures, which includes software 4 stored thereon that allows a human user to interact directly with the process, and in particular with a genetic algorithm (GA) that is executed by the software 4. The computer system 2 may take various forms, such as a single Pentium™ -class personal computer or a communications system which allows the user remote access using a display terminal, such as a computer or mobile telephone. The software 4 may also be stored in the one location on the system 2 or distributed over various storage locations which are logically connected by a messaging system maintained by the system 2.

GAs are known for their use in providing solutions for complex problems where the solution space is large and complex. The solutions are represented by gene strings or chromosomes which each contain a set of parameters or genes, being the string's genotype, that describe the attributes, or phenotype, of a string. The strings form a population which is operated on by the GA after initialisation. The GA effectively allows the population to evolve to a set of possible solutions in the solution space. The quality of strings in the population is evaluated using a fitness function. The fitness function is used to assign fitness values to the strings and the assigned values can be used by the GA to select strings for generating or breeding a subsequent generation of strings of the population. The strings selected may be considered to be parent strings, whereas those of the subsequent generation derived from the parent strings can be considered to be child strings. The GA generates the child strings from the parent strings using selected breeding operators, such as crossover operators and mutation operators. The GA also allows the rates of crossover and mutation to be adjusted to introduce different genotypes into the population if the population is converging to a local minimum or maximum. On generating a subsequent generation, the extent to which that generation replaces the parents can be selected using a number of criteria based on evaluation of the strings using the fitness function. If desired, as for the GA described below, all parent strings can be replaced by a new generation of child strings. GAs are used as approximation algorithms for solving solutions to problems using a closed loop procedure, which is executed over a large number of iterations, thereby allowing the population to evolve without any outside influence. For a more detailed discussion of GAs reference may be had to David E. Goldberg, Genetic Algorithms in Search, Optimization, and Machine Learning, 1989 Addison- Wesley, Reading, Massachusetts.

Contrary to the normal use of GAs, the software 4 of the system 2 executes an interaction process which involves allowing a user to interact with a GA that reacts to decisions made by a user. Factors relevant to the decision making process are linked to a population of strings that the GA operates upon. The strings are used to represent output data presented to the user on the system and the response the user gives to the output is used to evaluate the strings and thereby affect how the GA generates a subsequent generation of the population from the strings. This allows the output data represented to the user to evolve based on the user's responses, and accordingly based on the user's capabilities and skills in responding to the data presented to the user. One implementation is described below where the strings are used to represent display elements presented to the user in the context of a computer video game and the GA allows a player of the game to evolve or adjust the game process in a manner which is not structured on a hierarchical or on any other basis, but is more akin to biological evolution.

The computer game executed by the software 4 generates visual patterns by the application of spatial rules such as translation and rotation to various two or three dimensional objects. These patterns are the basis of procedures to develop thinking skills including abstracting ability, problem solving and creativity. The patterns are stored respectively as gene strings that are coded for the number of objects to be displayed and rules relating to their shape, colour, position and other characteristics. The two or three dimensional patterns are each displayed in a 3x3 matrix 6, as shown in Figure 1, with one cell 8 of the matrix empty. The empty cell 8 is filled by the user from a range of alternative images forming a response set 10 displayed below the matrix. The task for the user is to complete the overall visual pattern of the string by selecting a particular image from the response set 10. The patterns are displayed iteratively. The user and the game interact adaptively with each other through the use of the GA, as described in detail below. The game adapts to changes in the user's level of proficiency, by varying how difficult the patterns are to solve. This is achieved by the GA producing "child patterns" from selected "parental patterns". The "parental patterns" are selected probabilistically for reproduction by a fitness function for the GA based whether the user's choice is correct or not and on the time taken by the user to choose an image from the alternatives. To optimise a learning rate for the user, the user-game interaction is designed to tend to a low error-rate equilibrium which is under the control of a fitness function negative feedback loop, as described below.

Visual patterns which depict progressive changes in several dimensions, have been used extensively to assess individual intelligence and have been found by practical usage to be highly accurate in assessing the general and powerful abstracting skills which underlie intelligence and creativity. They are also highly predictive of scholastic performance. The game extends this use into the development of intellectual skills by providing for their extensive practice by a user in an optimised training environment. The computer program 4 of the game displays progressively changing visual patterns for developing mind or intellectual skills, abstracting ability, problem solving and creativity.

In the stimulus set 6 that comprises the 3x3 matrix, the upper left hand cell of the matrix is the starting point of a series of changes to the individual image in each matrix cell. The changes proceed sequentially along each row and down each column of the matrix and are generated algorithmically by the application of rules which vary such factors as the position, number, colour and orientation of elements within each pattern. As a method of varying the overall difficulty level of a particular pattern set, thus allowing users of different levels of ability to use the same pattern set, the number of alternatives in the response set 10 can vary from two to eight. The correct response is randomly placed among incorrect alternatives in the row 10. The incorrect alternatives are generated algorithmically in the same manner as the elements of the stimulus set 6. The pattern sets are functionally separate from the program code, as described below, enabling the user to change pattern sets without difficulty.

An adaptive engine 12 of the game program 4 interactively modifies the visual patterns in response to changes in the user's performance to produce visual patterns which vary in difficulty level as the user's skills vary. For example, an improvement in user performance generates a difficulty level increase in the patterns generated by the adaptive engine 12. This matching or tracking process tends to optimize the learning environment, ensuring that the visual patterns displayed have difficulty levels appropriate to the user's current capability. The engine 12 adapts to the individual user's performance using the functionality of the GA. Changes in the user's skill level induce adaptive changes in the structures of the GA which in turn influence further skill development.

On startup or on selection of a new pattern set, the adaptive engine 12, generates an initial population of binary strings. Figure 2 shows the structure of a typical pattern string, which is based on the principle that application relevant data is represented and stored in a chromosome-like structure. The initial population of binary strings can be user-selected with respect to difficulty level by the number of rules of the string which are actuated. The properties of each pattern which includes the number and characteristics of the objects, their position, colour, opacity and orientation and which are part of the functionality of the pattern set code, are uniquely specified by locations on the binary string. Variations in a population of binary strings determine variations in the appearance of the patterns. The format of the binary string shown in Figure 2 includes data fields which specify the shape of the objects of a pattern, and rules which selectively designate the spatial location, colour, orientations and translations which each object is subject to, together with rules also indicating whether particular features or characteristics or shapes are to be displayed or not. For example, a binary string may indicate that the shapes to be displayed are a cone and a square which are rotated in a particular fashion on the x, y and z axes, whereas other shapes, such as a sphere, may be hidden. The binary string is decoded, as described below, into the specified graphic elements and the operations are executed on the elements prior to display. The string format allows for a large variety of pattern sets to be displayed.

The adaptive engine 12, as described below, iterates through the population

(current generation) of binary strings in the following steps: a) A binary string is decoded by a graphics engine module 14 and displayed. b) The time taken by the user to select an alternative from the response set 10 and the correctness of the choice are recorded for each string for use with the fitness function. As a measure of individual user performance, the program records which patterns are solved and which are not solved, and how long it takes a user to select a response from the alternatives. A pattern fitness value is assigned to the string based on the user's response and is the mechanism by which patterns are selected for reproduction. Pairs of strings are selected as parents probabilistically in terms of their fitness values. The strings are subject to crossover to produce child strings as members of the new generation. Each new generation member is then subjected to a function which reverses the value of randomly selected locations on the string, i.e. from 1 to 0 or vice versa in the string at relatively infrequent, randomly determined locations in the string. This function is, in effect, a mutation operator whereby aspects of the specific visual patterns are changed randomly from one generation to another. Mutation has the desirable effect of increasing the variability in the population of strings and their visual counterparts, the patterns. It expands the area of search that the GA is operating in, allowing it to explore new aspects of pattern variability and for the user, new areas of skill development.

Skill optimisation requires tasks that are, for the user, neither trivially easy nor excessively difficult. To maximise skill development, the task difficulty level of the tasks must remain relatively high for the individual and near the failure point; and must extend the user's capability. The adaptive engine 12 establishes a low error rate learning paradigm or environment which requires that the user experience tasks which are near or on borderline between the solvable and the unsolvable. Significant learning occurs within that region of difficulty.

A negative feedback loop of the GA maintains the low error rate learning environment, with the engine 12 executing the following: a) The solved strings are assigned a fitness value proportional to their reaction time. b) The unsolved strings are assigned a single fitness value which is inversely proportional to the proportion of patterns which were unsolved during the presentation of that generation of patterns. c) If a user is solving all or most patterns without error, the fitness function is adjusted so that the likelihood of any unsolved patterns and solved patterns of high fitness value producing offspring patterns in the next generation will be increased. d) If a user is not solving many patterns, the fitness function is adjusted so that the likelihood of any solved patterns of low fitness value producing offspring patterns in the next generation will be increased. e) To maintain sufficient variation in the new population of patterns being generated by the GA so that the game 4 can explore new areas of user skill development, the mutation rate is varied in the following manner. If the user is solving all or most of the patterns without error or the user is not solving many patterns, the mutation rate is increased. If the user has solved patterns from the previous generation on or near a low error rate criterion (usually set within the range of 10% to 20% incorrect for all patterns displayed), then the mutation rate is not changed from an initial preset value.

The game program 4 stored on the system 2 includes the adaptive engine 12, and modules 14 and 16, and associated data 18 for executing the game and generating the interface for the user on the system, as shown in Figure 3. The adaptive engine 12 includes a pattern set population module 20 which maintains the current population of patterns that includes a set of pattern strings having the format as shown in Figure 2 with individual fitness values. The engine 12 also includes a population generation module 22 which executes the GA for the game.

Pattern strings are forwarded by the pattern set population module 20 to a graphics engine module 14 of the program 4. The module 14 decodes the pattern string into graphic elements which are used to render three or two dimensional graphic objects for display.

Strings are decoded on the basis of object rule and graphic user interface data held in a pattern set repository 18, which is user selectable and replaceable.

Operation of the game is controlled by a core program module 16 which provides sequencing instructions for the graphics engine module 14 and handles all application specific input/output data communicated between the module 16 and the operating system of the computer system 2. The core program module 16 therefore coordinates delivery of the graphic display data from the graphics engine module 14 to the user interface, and handles all operating system messages and events, which include messages and data passed between the adaptive engine 12 and the user. The core program module 16 controls the graphics engine module 14 to generate the interface for the user which provides the visual stimulus that the user is able to respond to using standard input devices of the system 12, such as a mouse and keyboard. The user response data is forwarded by the core program module 16 back to the population module 20 for allocation of an individual fitness value to the displayed string.

After all patterns (strings) have been displayed, the population generation module

22 executes the fitness function to adjust the fitness values for the individual pattern strings of the population and normalise the values. The fitness function assigns to each individual pattern binary string a fitness value which is used to determine the likelihood of the particular string being selected by the GA to become a parent (i.e. to combine with another string to produce for the next generation a child string). This selection process is a probabilistic (stochastic) one in which the likelihood of a string becoming a parent depends on its fitness value, with higher fitness values associated with higher probabilities and lower fitness values with lower probabilities. As described previously, solved strings are assigned fitness values by the function based on the reaction time of the user, with lower reaction time strings having a lower fitness value. Unsolved strings are assigned a fitness value based on the percentage of strings in a population which were unsolved, with low percentages giving rise to high fitness values. The high fitness values produced by the fitness function represent strings which are more likely to survive in the population and produce child pattern strings. The parents are selected probabilistically in a "roulette wheel" procedure whereby the strings occupy intervals in fitness scale proportional to their fitnesses and are chosen randomly from this interval. This ensures that the likelihood of selection varies with fitness. The roulette wheel procedure is executed by first obtaining a total sum of the string fitnesses of a population as a value sumfitness. A random number is generated in the range of 0 to 1 and multiplied by sumfitness to obtain a value randsumfit. The selection procedure then iterates through the population of strings in order of presentation, although any order could be selected, and executes the following: i) The fitness value of a string is added to a summing variable, partialsumfit which is initialised at 0. ii) If randsumfit is equal or less than partialsumfit then a string is selected, if not, the iteration goes through another loop. The above steps are executed to obtain a pair of strings to act as parents.

The module 22 executes the following procedures until a new generation of strings for the population has been produced: a) A mutation rate for the strings is determined as a function of the fitness values and the number of unsolved patterns, as described previously. b) Pairs of parent patterns are selected probabilistically as a function of the fitness values for the generation of child pattern strings, as described above. c) The number of string crossover points during a combination of parents is determined as a function of the fitness values of the parents. For parents with a high combined fitness value, a low number of crossover points are used, whereas for parents with a low joint fitness value, a high number of crossover points are used.

The pattern generation module 22 then selectively uses the new generation of child strings to replace strings in the population maintained by the population module 20.

The population of chromosome-like pattern strings in the game interacts with a range of individual skills possessed by a user, and both the GA population and the user's skills are modified by this interaction. This adaptation modifies the GA population over successive generations, and the pattern strings which are evolved each generation are records of this modification. These changes will be apparent in the changes in the frequencies of genes from one generation to another. Since the individual locations (genes) on the pattern strings (chromosomes) represent specific features of the displayed patterns, modification of the composition of the pattern strings as a result of the user-program interaction reflects changes in the user's skills. Analysis of the changes occurring in the strings over successive generations will indicate those aspects of the visual patterns which the user is responding to and the time-based sequence of user skill development. This analytic procedure provides highly detailed information, unavailable by any contemporaneous technique, regarding the development of the skills required to solve pattern sets.

The interaction process executed by the software 4 can be applied in a number of areas, such as advanced games technology, special education and neurological diagnosis and rehabilitation.

Many modifications will be apparent to those skilled in the art without departing from the scope of the present invention as herein described with reference to the accompanying drawings.

Claims

CLAIMS:

1. An interaction process executed on a computer system, including: generating an output based on at least one string of a population, said at least one string representing elements of said output;

receiving a response to said output caused by a user of said system; and executing a genetic algorithm (GA) on said population to generate a subsequent generation of said population using said response.

2. An interaction process as claimed in claim 1, wherein executing said GA includes applying a fitness function to strings of said population on the basis of said response.

3. An interaction process as claimed in claim 2, wherein said applying includes allocating fitness values to said strings, respectively, said fitness values being determined on the basis of said response.

4. An interaction process as claimed in claim 3, wherein executing said GA includes executing crossover and mutation on said population and the mutation rate is determined on the basis of said response.

5. An interaction process as claimed in claim 4, wherein the strings represent tasks to be solved by said user and said response represents correct or incorrect answers with respective answer times for the strings, and the solved strings are allocated a fitness value proportional to the answer time for the solved string, and the unsolved strings are allocated a fitness value inversely proportional to the percentage of unsolved strings in said population.

6. An interaction process as claimed in claim 5, wherein the mutation rate is determined on the basis of the percentage of unsolved strings in said population.

7. An interaction process as claimed in claim 6, wherein executing said crossover includes determining the number of string crossover points in proportion to the fitness values of the strings used for said crossover.

8. An interaction process as claimed in claim 5, including adjusting said fitness function on the basis of said response to control selection of said strings for said crossover so that said response for said subsequent generation may have a low percentage of incorrect answers.

9. An interaction process as claimed in claim 1, wherein generating said output includes decoding said string into graphic elements and rendering, a display for said user on the basis of said elements.

10. An interaction process as claimed in claim 9, wherein said receiving said response includes recording at least one event as an input after said rendering and storing said input against said at least one string of said population.

11. A computer system, including: means for generating an output based on at least one string of a population, said at least one string representing elements of said output; means for receiving a response to said output caused by a user of said system; and means for executing a genetic algorithm (GA) on said population to generate a subsequent generation of said population using said response.

12. A computer system as claimed in claim 11, wherein said executing means applies a fitness function to strings of said population on the basis of said response.

13. A computer system as claimed in claim 12, wherein said executing means allocates fitness values to said strings, respectively, said fitness values being determined on the basis of said response.

14. A computer system as claimed in claim 13, wherein said executing means executes crossover and mutation on said population and the mutation rate is determined on the basis of said response.

15. A computer system as claimed in claim 14, wherein the strings represent tasks to be solved by said user and said response represents correct or incorrect answers with respective answer times for the strings, and the solved strings are allocated a fitness value proportional to the answer time for the solved string, and the unsolved strings are allocated a fitness value inversely proportional to the percentage of unsolved strings in said population.

16. A computer system as claimed in claim 15, wherein the mutation rate is determined on the basis of the percentage of unsolved strings in said population.

17. A computer system as claimed in claim 16, wherein in executing said crossover the number of string crossover points is proportional to the fitness values of the strings used for said crossover.

18. A computer system as claimed in claim 15, wherein said executing means adjusts said fitness function on the basis of said response to control selection of said strings for said crossover so that said response for said subsequent generation may have a low percentage of incorrect answers.

19. A computer system as claimed in claim 11, wherein said generating means decodes said string into graphic elements and renders a display for said user on the basis of said elements.

20. A computer system as claimed in claim 19, wherein said receiving means records at least one event as an input after said rendering, and stores said input against said at least one string of said population.

21. Software stored on at least one storage medium, including: code for generating an output based on at least one string of a population, said at least one string representing elements of said output; code for receiving a response to said output caused by a user; and code for executing a genetic algorithm (GA) on said population to generate a subsequent generation of said population using said response.

22. Software stored on at least one storage medium as claimed in claim 21, wherein said executing code applies a fitness function to strings of said population on the basis of said response.

23. Software stored on at least one storage medium as claimed in claim 22, wherein said executing code allocates fitness values to said strings, respectively, said fitness values being determined on the basis of said response.

24. Software stored on at least one storage medium as claimed in claim 23, wherein said executing code executes crossover and mutation on said population and the mutation rate is determined on the basis of said response.

25. Software stored on at least one storage medium as claimed in claim 24, wherein the strings represent tasks to be solved by said user and said response represents correct or incorrect answers with respective answer times for the strings, and the solved strings are allocated a fitness value proportional to the answer time for the solved string, and the unsolved strings are allocated a fitness value inversely proportional to the percentage of unsolved strings in said population.

26. Software stored on at least one storage medium as claimed in claim 25, wherein the mutation rate is determined on the basis of the percentage of unsolved strings in said population.

27. Software stored on at least one storage medium as claimed in claim 26, wherein in executing said crossover the number of string crossover points is proportional to the fitness values of the strings used for said crossover.

28. Software stored on at least one storage medium as claimed in claim 25, wherein said executing code adjusts said fitness function on the basis of said response to control selection of said strings for said crossover so that said response for said subsequent generation may have a low percentage of incorrect answers.

29. Software stored on at least one storage medium as claimed in claim 21, wherein said generating code decodes said string into graphic elements and renders a display for said user on the basis of said elements.

30. Software stored on at least one storage medium as claimed in claim 29, wherein said receiving code records at least one event as an input after said rendering, and stores said input against said at least one string of said population.

31. Adaptive software stored on at least one storage medium, including: a population module for maintaining strings of a population, the strings representing elements of a computer generated output; a generation module for generating said output; a processing module for handling a response to the output caused by a user; and a population generation module for executing a genetic algorithm (GA) on said population to generate a subsequent generation of said population using the response.