WO2007069118A2

WO2007069118A2 - Context aware food intake logging

Info

Publication number: WO2007069118A2
Application number: PCT/IB2006/054567
Authority: WO
Inventors: Maarten P. Bodlaender; Arvid R. Nicolaas; Mariana Simons-Nikolova
Original assignee: Koninklijke Philips Electronics N.V.
Priority date: 2005-12-14
Filing date: 2006-12-04
Publication date: 2007-06-21
Also published as: WO2007069118A3

Abstract

A context aware food intake logging system (10) is provided. The system comprises an input device (11) for enabling a user to log food intake information, a storage means (12) for storing the food intake information, a food items database (13) comprising a plurality of food items and corresponding food context information, a context identifier (14) for obtaining at least one environmental parameter, a processor (15) for generating food log suggestions and an output device (16) for providing the food log suggestion to the user. The processor (15) is coupled to the food items database (13) and the context identifier (14) and calculates a selection probability of at least one food item of the plurality of food items, based on the corresponding food context information and the at least one environmental parameter. Then the processor (15) generates a food log suggestion, based on the selection probability of the at least one food item.

Description

Context aware food intake logging

The invention relates to a food intake logging system, comprising an input device for enabling a user to log food intake information, a storage means for storing the food intake information and a food items database comprising a plurality of food items to be selected for logging. The invention further relates to a mobile device for logging food intake.

The invention further relates to a method for logging food intake.

The invention further relates to a computer program product for performing said method.

Such a food intake logging system is described in European patent application EP 1179799. The method described therein comprises using video photographic means for obtaining pictures of food items. The obtained pictures are compared to electronically stored pictures of food items for identifying the food item on the obtained picture. Based on nutritional values of the identified food item a display message is generated for warning or instructing the user regarding the food item and pre-programmed dietary limitations. The method of EP 1179799 relieves the user of part of the task of data entry, but also has some disadvantages. First, a user of the method has to make pictures of all consumed food items for which a video photographic means are required. Furthermore, often the pictures of the food items will not be recognized or are misinterpreted as other food items. When the food item is not correctly identified conventional labor-intensive methods for logging food intake have to be performed.

It is an object of the invention to provide a system for logging food intake, which is more easy to use. According to the invention this object is achieved by providing a system as described in the opening paragraph, wherein the food items database further comprises corresponding food context information for the plurality of food items and wherein the system further comprises a context identifier for obtaining at least one environmental parameter, a processor and an output device for providing a food log suggestion to the user, the processor being operatively coupled to the food items database and the context identifier for calculating a selection probability of at least one food item of the plurality of food items, based on the corresponding food context information and the at least one environmental parameter and generating the food log suggestion, based on the selection probability of the at least one food item.

The context identifier obtains environmental parameters when the food logging is going to be performed. The environmental parameters may relate to temperature, time of day, day of the year, purchase history, geographical position and many more, as will be elucidated below. By correlating the obtained parameters to the context information in the food items database, a probability is calculated for food items in the database of being selected for logging. The food item, which is most likely to be selected for logging, is suggested to the user via the output device. The invention takes advantage of the fact that the probability of a food item being selected by a user depends on many contextual factors. With the system according to the invention, those items, which are most likely to be selected, are presented to the user and consequently, the user can easily select food items for logging the food intake.

In a preferred embodiment, the food context information is adjustable by the user. When the user is able to adjust the food context information, the suggestions can be personalized. The user may enter some user specific food context information into the database. For example, the user may indicate to like or dislike certain food items, to like or dislike certain combinations of food items or to like certain food items especially at specific times of the day. Lots of further examples will be provided below.

In a further embodiment, the processor is configured for, based on the at least one environmental parameter, adjusting the food context information upon logging of the food intake information by the user. In this embodiment logged food items are associated with the environmental parameters at the moment of logging. By adjusting the food context information in the food items database, this association is stored for later use. If, for example, a user selects a specific food item at a specific place and time of day, this selected food item may be suggested again, the next time when the user is at said place at said time of day. By using this advantageous embodiment of the system the food context information will become more and more personalized and consequently its suggestions will become better and better.

In a further embodiment, the food log suggestion is a list of multiple food items, which food items are ordered according to their corresponding selection probability. Because the user will not always select the most probable food item for logging, it is advantageous that more than one suggestion is provided.

Preferably, the input device, the context identifier, the processor and the output device are comprised in a mobile device, the mobile device further comprising means for coupling to the storage means and/or the food items database. The mobile device may be a cellular phone, PDA or portable computer running an application that provides the functions of the system according to the invention. The context identifier may comprise known technical units, such as a clock/calendar, GPS or a digital photo/video camera. The food items database and the storage means for storing the food intake information may be located at a central server which can be connected to via, e.g., Bluetooth, GSM, WAP, USB, infrared, and/or Internet. Preferably, a user of a cellular phone subscribes to a food logging service and thereby gets permission to contact the server. It is an advantage of using a central server that new food items and food context information can be added to the database for later use by all subscribers to the service. If the food intake information is also stored centrally, it can be accessed by various weight management application clients. For example, the mobile device may provide for short summaries of the food intake of the last week, while a computer program used at home or at the fitness club may be used for providing other possibly more extensive overviews of the food intake history. Alternatively, the food items database is stored on a memory of the mobile device. The food items database may then be updated using known communication techniques. The food intake information may also be stored on the memory of the mobile device and may then be downloaded to other devices for backup or other purposes.

These and other aspects of the invention are apparent from and will be elucidated with reference to the embodiments described hereinafter.

BRIEF DESCRIPTION OF THE DRAWINGS In the drawings:

Figure 1 schematically shows an embodiment of the food intake logging system according to the invention, Figure 2, schematically shows an embodiment of the food intake logging system according to the invention, comprising a mobile device, and

Figure 3 shows a flow diagram of a method according to the invention. Figure 1 schematically shows an embodiment of the food intake logging system 10 according to the invention. The system comprises an input device 11, such as a keyboard, a touch screen, a mouse, a track ball or another type of pointing device for enabling a user to log food intake information. Alternatively, the system 10 may use microphone and speech recognition software for receiving user input. The user uses the input device 11, to provide information about his or hers food intake to, for example, a weight management system. It is to be noted that food intake also includes drinking.

The system 10 further comprises storage means 12 for storing the logged food intake information. A weight management system preferably combines the stored food intake information with, for example, performed activities and the user's weight for determining and providing surveys and tips regarding health aspects of the user's behavior. The storage means 12 may be part of a device to which the input device 11 is directly coupled. Alternatively, the food intake information may be sent to a remote storage means 12, via wired or wireless connections, optionally via the Internet. The system 10 further comprises a food items database 13. The food items database 13 comprises a plurality of food items and corresponding food context information. The food context information defines the environmental parameters that are likely to be present when the corresponding food item is consumed. For example, ice cream is generally consumed in the summer months, when the outside temperature is above 20° Celsius. At an Italian restaurant, pizza is often chosen to be eaten. The food items database 13 may be stored on the storage means 12 or on a separate memory.

A context identifier 14 is provided for obtaining the environmental parameters. The context identifier may, for example, comprise a clock for obtaining a current time, a calendar for obtaining a day in the year, location measurement means, such as GPS, for obtaining a location of the user. Other environmental parameters that may be obtained by the context identifier 14 are, for example, an outside temperature (hot chocolate when temperatures are below 0° Celsius), an opened refrigerator, heated oven or used microwave, smoke levels (peanuts and beer in areas with much smoke) and much more.

The processor 15 compares the obtained environmental parameters to the food context information in the food items database 13. Based on the food context information and the environmental parameter, the processor 15 calculates a selection probability of a number of food items. Some environmental parameters increase the selection probability of a food item, while other environmental parameters decrease the selection probability. When a user is at the beach, the selection probability of an ice cream is increased, when it's rainy and cold, the selection probability is decreased. Different environmental parameters may have different amounts of effect on the selection probability. Normally, at 6.00 PM the selection probability of spaghetti will be slightly higher than at 4.00 PM, however when a user is in a Chinese restaurant at 6.00 PM the selection probability of spaghetti will be far less than when the user is at home at 4.00 PM.

Based on the calculated probabilities for the food items, the processor 15 generates food log suggestions. The food log suggestions are presented to the user via an output device, such as a display screen 16, speakers, or a printer. By suggesting the food items with the highest selection probability, the food intake logging is made very easy. Instead of searching through long lists of food items or using expensive and error prone image recognition techniques, the user may choose from a small set of food items with a high selection probability. In a preferred embodiment, the food items are ordered by their selection probability and a list is provided with the most probably selected food items. The user may then scroll through the list to select a food item to be logged. Preferably, together with the food log suggestion, information is provided about the nutritional values of the suggested food item, thereby enabling the user to decide whether it would be wise to consume the food item. In an advanced embodiment, the processor 15 may even calculate an expected effect of the consumption of the food item on the user's diet. The selection probability of a food item may depend on a large variety of environmental parameters and other factors, such as personal preferences, recently bought food items, usual combinations of two or more food items. Below a preferred embodiment is described, which takes into account a large, but not exhaustive, number of such parameters and factors. The preferred system 10 uses available context information to reduce the food item selection space for the user by showing the food items from the food items database 13 starting with the food items that have the highest calculated selection probability. All food items initially have an equal probability of selection, which are adapted using the available context information as explained in the following strategies: The preferred system 10 uses the current time of day to estimate the type of meal when logging food, and gives food items typically belonging to the meal type a higher probability. To this end, all food items are categorized according to type-of-day. In addition, the system 10 learns over time the typical time of day in which a user consumes a certain time of food. For example, if a user enters food intake in the morning, the preferred system 10 can present to the user only selected part of the food items database 13, namely items related to breakfast such as bread, milk, and items that have been consumed by a user in the morning, such as a candy bar and a piece of fruit.

The preferred system 10 uses the type of meal when logging food to give the food items typical for that meal a higher probability, as discussed in the previous strategy.

The preferred system 10 uses the user's geographical location information to give types of food available at that location a higher probability. For example, the preferred system 10 can detect that the user is inside a certain restaurant, and give the menus of that restaurant a higher probability in the selection list. The preferred system 10 uses season and recorded or predicted air temperature information to give certain types of food typically consumed at certain temperatures a higher probability. For example, when the temperature is higher than 20° Celsius, an ice cream for a snack has a higher probability than a glass of hot chocolate. In addition, during the asparagus season, asparagus have a higher probability than outside the asparagus season. The preferred system 10 uses ethnic and cultural knowledge to attribute certain types of food with higher or lower probability. For example, dairy products are more popular in some cultures than others.

The preferred system 10 uses known food preferences of the user to give certain types of food items less probability. For example, if a user is a vegetarian then all meat items can obtain a very low probability.

The preferred system 10 uses food allergy information to give certain types of food lower probabilities. For example, if a user has a food allergy, e.g. celiac (gluten- free) allergy that is caused by gluten - a protein in wheat, rye, barley and oats, then all food items having gluten are irrelevant for the user. The preferred system 10 uses time between meal information to give either snack or meal food a higher probability. For example, when a user wants to log food, and has had a meal 1 hour before, the food intake is more likely a snack than a meal.

The preferred system 10 uses statistical analysis of the food intake from an online community to give popular foods a higher probability than less popular foods. For example, if most users eat low- fat butter on their bread, this choice is also more likely for other users.

The preferred system 10 uses common food type combination knowledge to give certain food types a higher probability. For example, when the user has already chosen 'bread' in the breakfast food intake log, the system 10 can give types of food that are usually eaten on bread in the morning, e.g. jam, cheese, salami, etc. a higher priority.

The preferred system 10 uses knowledge about previously purchased food items to give certain food types a higher probability. For example, if a user has digitally paid for grocery shopping items, this information can be submitted to the system 10. Until the purchased food has been reported eaten or until their fresh-dates have expired this food types will have a higher selection probability.

The preferred system 10 takes a user's food logging history into account and gives frequently chosen food items higher selection probabilities. Also specific combinations of food intake logging and environmental parameters at the moment of logging may be stored and used for future recommendations. Preferably, the system initially has a predefined probability distribution that is common to most users. When being used, the system adapts to the user's habits.

In an alternative embodiment, the system 10 detects food items that are logged regularly and automatically places these food items in the food log by extending the detected regular pattern. Users can remove these items if they did not consume it. For example, if the system detects that the user drinks a glass of milk every morning for a working day, the system can automatically place milk in the food log for each working day. Another example of complete meal - the food intake logging history reveals a user's preferences towards a bowl of cereals and milk for breakfast. Then, this meal can be automatically placed in the breakfast log and the user will need to remove this item if it was not consumed.

In a further embodiment of the system 10 according to the invention, the system 10 compares the daily total nutrient amounts of a person's food intake to the daily nutrient targets of one or more selected diets, resulting in a list showing the person's distance to the nutrient targets and to the diet(s). The distance to a diet is calculated by taking the average of the distances for each nutrient between the food intake value and the diet target. The distance for a nutrient value to a nutrient target is calculated by taking the deviation of the actual value from the target value. The distance to a diet is indicative for the eating profile changes required and therefore for the lifestyle changes needed. This distance can be used as a feasibility measure for changes.

In addition, the system 10 can add a "best matching and worst matching food" to the feedback. This gives feedback in terms that the user understands (food types) and can help a user in changing his eating behavior more effectively than just numbers. An example of feedback is given in the following table:

The system 10 may calculate a distance of the food intake to several popular diets, resulting in a list showing how well the pattern matches with the diets. For example, if the user's food intake logging shows tendency to approximately 12% fat, 60% carb and 28% pro per day, then the best fitting diet from those mentioned in the background section might be Diabetic Like Diet (20% fat, 60% carb, 20% pro). In this way, the user can see which diet best fits his eating habits. An example of the given feedback is given in the following table:

The system 10 may monitor the distance of the food intake to a specific diet over a sequence of days, resulting in a list showing the user's progress in the distance to the selected diet. In this way, the user can assess whether the diet still suits, or otherwise if a switch may be a better choice. When presented as a chart, a user can see trends and crossover points to other diet-options. The system can provide information on diets that a user is moving towards. Optionally, the system 10 provides feedback over larger periods. An example of feedback is given in the following table:

The system 10 may assess how a certain food item influences the distance to a diet by looking at the distance to a diet before and after the food item is added to the menu. In this way, the user can get advice on whether or not to eat the food item. Optionally, the system provides feedback on the most harmful and/or beneficial ingredients. An example of feedback is given in the following table:

The system 10 may suggest food items or complete meals that reduce the user's distance to a diet, and create a list of foods that minimizes the distance to a diet. This helps the user in making a food choice, and avoids the user from drifting away from the diet. An example of feedback is given in the following table:

Figure 2, schematically shows an embodiment of the food intake logging system 10 according to the invention, comprising a mobile device 21. The mobile device 21 may, for example, be a cellular phone, PDA, or portable computer with communication means (wired or wireless). The mobile device may comprise all elements of the system 10. It is however preferable that the mobile device only comprises the input device 11 , context identifier 14, processor 15 and output device 16. The food items database 13 and/or the storage means 12 for storing the user's food intake information are then provided at a central server. When the food item database 13 is at a central server, new food items and new food context information may be added regularly and will directly be available to all users of the food items database 13. Furthermore, the food items database 13 may be too large to be stored on the mobile device 21 itself. When the user's food intake information is stored centrally, the user can easily access the information using various devices on various locations, by only logging in to the server. Personal information, such as food preferences, purchase history and regularly consumed combinations of food items are stored either on the mobile device 21 or remote. Storage of the personal information may be realized on the storage means 12 for storing the user's food intake information. The calculation of the selection probabilities and the generation of the food log suggestions are preferably performed by a processor 15 in the mobile device, but may also are performed at a remote server. Figure 3 shows a flow diagram of a method according to the invention. The method starts with a detection step 31 for obtaining environmental parameters. The parameters may be obtained using, e.g., a camera, a microphone, a clock, a calendar, a thermometer or a barometer from or may be obtained from external sources, for example, via the Internet. Then, in a calculation step 32, the selection probability of the food items is calculated, based on the food context information and the environmental parameters obtained in the detection step 31. Thereafter, a suggestion step 33 for generating food log suggestions is performed. The food log suggestions are based on the selection probabilities as calculated in the calculation step 32. Then, the food log suggestions are provided to the user in display step 34. Optionally, the method also includes a selection step 35 for selecting a food item of the plurality of food items for storing as food intake information in the storage means 12. Preferably, thereafter the food context information corresponding to the selected food item is updated in update step 36. The update results in the environmental parameters, as obtained at the moment of logging, being stored as food context information in the food items database 13. The next suggestion provided by the system 10 will then be based on information from the updated food items database 13. The next time when similar environmental parameters are detected, the probability of the previously selected items will be higher. It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. Use of the verb "comprise" and its conjugations does not exclude the presence of elements or steps other than those stated in a claim. The article "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the device claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

Claims

CLAIMS:

1. A food intake logging system (10), comprising: an input device (11) for enabling a user to log food intake information, a storage means (12) for storing the food intake information, a food items database (13) comprising a plurality of food items and corresponding food context information a context identifier (14) for obtaining at least one environmental parameter, a processor (15), operatively coupled to the food items database (13) and the context identifier (14) for, calculating a selection probability of at least one food item of the plurality of food items, based on the corresponding food context information and the at least one environmental parameter, generating a food log suggestion, based on the selection probability of the at least one food item, and an output device (16) for providing the food log suggestion to the user.

2. A food intake logging system (10) according to claim 1, wherein the food context information is adjustable by the user.

3. A food intake logging system (10) according to claim 1, wherein the processor (15) is configured for, based on the at least one environmental parameter, adjusting the food context information upon logging of the food intake information by the user.

4. A food intake logging system (10) according to claim 1, wherein the food log suggestion is a list of multiple food items, which food items are ordered according to their corresponding selection probability.

5. A food intake logging system (10) according to claim 1, wherein the at least one environmental parameter comprises a time of day and wherein the context identifier (14) is operative to determine the time of day.

6. A food intake logging system (10) according to claim 1, wherein the at least one environmental parameter comprises a location of the user and wherein the context identifier (14) is operative to determine the location of the user.

7. A food intake logging system (10) according to claim 1, wherein the at least one environmental parameter comprises an outside temperature and wherein the context identifier (14) is operative to obtain the outside temperature.

8. A food intake logging system (10) according to claim 1, wherein the processor

(15) is operative to calculate the selection probability further based on personal characteristics of the user, the personal characteristics, for example, comprising an ethnic or cultural background, known food preferences and/or food allergy information.

9. A food intake logging system (10) according to claim 1, wherein the input device, the context identifier (14), the processor (15) and the output device (16) are comprised in a mobile device (21), the mobile device (21) further comprising means for coupling to the storage means (12) and/or the food items database (13).

10. A mobile device (21) for use in the system according to claim 9.

11. A method of logging food intake, the method comprising the steps of: obtaining (31) at least one environmental parameter, calculating (32) a selection probability of at least one food item of a plurality of food items, based on corresponding food context information and on the at least one environmental parameter, generating (33) a food log suggestion, based on the selection probability of the at least one food item, and providing (34) the food log suggestion to the user.

12. A method as claimed in claim 11, further comprising the steps of: selecting (35) a food item of the plurality of food items for storing as food intake information, and updating (36) the food context information corresponding to the selected food item, based on the at least one environmental parameter.

13. A computer program product, which program is operative to cause a processor (15) to perform a method as claimed in claim 11.