WO2016013702A1

WO2016013702A1 - Wearable device for determining whether packaged food is opened and providing information related to time at which packaged food has been opened, and control method therefor

Info

Publication number: WO2016013702A1
Application number: PCT/KR2014/006809
Authority: WO
Inventors: 박시화; 김지환
Original assignee: 엘지전자 주식회사
Priority date: 2014-07-25
Filing date: 2014-07-25
Publication date: 2016-01-28

Abstract

Disclosed are a wearable device for determining whether packaged food is opened and providing information related to the time at which the packed food has been opened, and a control method therefor. The wearable device, according to the present specification, can comprise: a memory unit for storing data; a camera for sensing an image; and a control unit for distinguishing hand motions of a user and the packaged food in a first image received through the camera, storing, in the memory unit, data on the food of which the package is opened (hereinafter, referred to as "food data") and data on the time at which the package is opened (hereinafter, referred to as "opening time") when it is determined that the package of the food distinguished in the first image received through the camera is opened by means of the hand motions of the user distinguished in the first image received through the camera, and outputting an information signal providing, to the user, the information related to the opening time when the food distinguished from a second image received through the camera matches the food data after the food data and the opening time are stored.

Description

Wearable device and control method for providing information related to the determination of whether the packaged food is opened

The present specification relates to a wearable device and a control method for providing information related to whether a packaged food is opened and opened.

Augmented reality technology, which is a mixture of real and virtual objects, allows the user to view the reality image along with the virtual image, providing additional information along with the reality. For example, when the camera is illuminated with a smartphone, augmented reality images such as locations and phone numbers of nearby stores are displayed as stereoscopic images. Augmented reality technology can also be applied to wearable devices. In particular, a head-type display such as a head mounted display provides augmented reality to a user by overlapping a virtual image or a character in real time as seen by the user in real time.

The wearable device may provide various conveniences to the user in combination with the aforementioned augmented reality technology, N screen technology, and the like beyond the simple display function.

In addition, the wearable device may be used in connection with various external devices. The wearable device may communicate with an external digital device to receive a user input for the corresponding external device or to perform a task of interworking with the corresponding external device.

1A to 1D show examples of a wearable device worn on a head or a face. As shown in (a) to (d) of Figure 1, the wearable device has a variety of forms, any shape that can be worn on the head or face is possible. For example, glasses form as shown in Fig. 1 (a), sunglasses form as shown in Fig. 1 (b), hair band form as shown in Figs. 1 (c), (d) is also possible.

The wearable device of FIGS. 1A through 1D provides an image and / or voice to a user through a display and / or speaker. In particular, a wearable device is generally provided with a small display device such as a liquid crystal at a position proximate to at least one of both eyes to project an image.

On the other hand, as the modern society develops, processed foods that can be eaten with shorter time are also developing. For example, there are retort foods and frozen foods that can be eaten simply by heating in a microwave oven, and various instant, cans or bottled beverages. The advantage of these processed foods is the advantage of simplicity. On the other hand, processed foods that have exceeded the shelf life may also cause various diseases such as food poisoning. In particular, in the case of a packaged food, the food should be consumed within a preservable period after the time of opening.

Various food control programs are being studied to remedy this problem. As an example of a food control program, when a user opens a package of food, there may be a method in which the food management program calculates the shelf life of the remaining food by inputting the opening time to the DB.

However, when opening a package of food, there is an inconvenience that a user must input directly to a DB of a food management program. The food may not be properly managed by the user. Therefore, there is a need for a method capable of recognizing and managing a change in the packaging state of a food, such as when the package is opened.

The present specification relates to a wearable device and a control method capable of determining whether a packaged food is opened.

The present disclosure also relates to a wearable device and a control method capable of storing the time the package of food is opened and providing information related thereto.

Therefore, the present specification aims to substantially solve the limitations and drawbacks of the prior art by providing a user with information related to whether or not to open a packaged food and opening time.

Wearable device for achieving the object of the present specification, the memory unit for storing data; A camera for sensing an image; And distinguishing the user's hand motion and the packaged food from the first image received through the camera, and the first image received through the camera by the user's hand motion distinguished from the first image received through the camera. When it is determined that the package of food distinguished from is opened, the memory unit stores data about the food in which the package is opened (hereinafter 'food data') and data on the time when the package is opened (hereinafter 'opening time'). And storing the food data and the opening time and outputting an information signal for providing the user with information related to the opening time when the food distinguished from the second image received through the camera matches the food data. The control unit may include.

According to one embodiment of the present specification, the memory unit may further store an opening image data table including image data (hereinafter 'opening image data') in a state in which a package is opened according to various packaging characteristics of the food, and the controller Determines the packaging characteristics of the food distinguished from the first image received through the camera, and according to the packaging characteristics of the food from the food image distinguished from the first image received through the camera and the packaging image data table. By comparing the mapped opening image data, it may be determined whether a food package is opened.

The opening image data may be an image of a state in which a crack is formed in a portion of a package of food, an image of a state in which a lid included in the package of the food is removed, or an image of a state in which some of the joints formed in the package of the food are separated. have.

The wearable device according to the present specification may further include a microphone for sensing a sound. In this case, the memory unit further stores an opening sound data table including sound data (hereinafter 'opening sound data') generated when the package is opened according to the packaging characteristics of various foods, and the control unit received through the camera. Determine whether the packaging of the food is opened by determining the packaging characteristics of the food distinguished from the first image, and comparing the opening sound data mapped according to the packaging characteristics of the food in the opening sound data table with the sound received through the microphone. Can be determined.

The opening sound data may be generated when a crack is formed in a part of a package of food, a sound generated when the lid included in the package of the food is removed, or a part of the joint formed on the package of the food falls. It can be sound.

According to an embodiment of the present disclosure, the controller may output time information that has elapsed from the opening time to the present as an information signal.

According to another exemplary embodiment of the present specification, the memory unit further stores an elapsed period data table including an elapsed period according to various food characteristics, and the controller controls the characteristics of the food distinguished from the first image received through the camera. The information related to the remaining time or the elapsed time of the elapsed time calculated by mapping the characteristics of the determined food in the elapsed time data table from the opening time may be output as an information signal.

According to another embodiment of the present specification, the memory unit further stores an elapsed time input by the user for the food in which the package is opened, and the controller controls the remaining time of the elapsed period input by the user from the opening time or Information related to the transition time can be output as an information signal.

The wearable device according to the present disclosure may further include a warning unit configured to output a visual signal or an audio signal. In this case, the controller may output the information signal to the warning unit.

According to an embodiment of the present disclosure, the warning unit may be a display unit for outputting an image. In this case, the controller may output the information signal to display the time information associated with the opening time stored in the memory unit on the display unit.

Furthermore, the wearable device according to the present disclosure may further include an image processing unit for arranging one or more virtual objects on one or more layers and superimposing the images on the image of the real world captured by the camera. In this case, the controller may control the time information related to the opening time to be displayed as the virtual object, and control the virtual object to be positioned in a portion adjacent to the food image of the photographed real world image.

According to one embodiment of the present specification, the control unit may determine whether the food is opened or unpacked when receiving a user input.

According to another embodiment of the present disclosure, the control unit may determine whether the food is unpacked when reading the information displayed on the packaging of the food distinguished from the first image received through the camera. At this time, the information displayed on the packaging of the food, may be a brand name or barcode or QR code of the food.

A control method of a wearable device for achieving the object of the present specification includes the steps of: (a) distinguishing a user's hand motion and packaged food from a first image received through a camera; (b) determining whether the package of the food distinguished in the image received through the camera is opened by a user's hand gesture distinguished in the first image received through the camera; (c) when the package of the food is opened, storing data about the food in which the package is opened (hereinafter 'food data') and data on the time when the package is opened (hereinafter 'opening time') step; And (d) outputting an information signal for providing the user with information related to the opening time when the food distinguished from the second image received through the camera matches the food data.

According to an aspect of the present disclosure, whether the packaged food is opened may be determined through image comparison or sound comparison. Therefore, the user can manage the opened food without the user inputting separate information about the food.

According to another aspect of the present disclosure, it is possible to provide information related to the time when the package of the food is opened, so that the user can determine whether to eat, manage, preserve, discard or the like of the food.

BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings, which are provided to provide a further understanding of the specification and which are incorporated in and constitute a part of the present application, illustrate embodiments of the present specification in conjunction with the detailed description describing the principles of the present specification.

1 is an illustration of a wearable device.

2 is a block diagram schematically illustrating a configuration of a wearable device according to an exemplary embodiment of the present specification.

3 is a reference diagram for understanding the appearance of the wearable device according to the present specification.

4 is an exemplary view for determining whether a package of food is opened by a user's hand operation.

5 is an exemplary diagram of a data table that may be used to determine whether or not to open.

6 is an exemplary view of the food data and the opening time stored in the memory unit.

7 is a flowchart illustrating a control method of a wearable device according to an exemplary embodiment of the present specification.

Hereinafter, exemplary embodiments disclosed herein will be described in detail with reference to the accompanying drawings. At this time, the configuration and operation of the embodiments shown in the drawings and described by this description will be described as at least one embodiment, by which the technical spirit of the present specification and its core configuration and operation is not limited. .

2 is a block diagram schematically illustrating a configuration of the wearable device 100 according to an exemplary embodiment of the present specification.

Referring to FIG. 2, the wearable device 100 according to the present specification includes a control unit 101, a memory unit 102, and a camera 103. In addition, the wearable device 100 may further include a microphone 104, a warning unit 105, an image processing unit 106, or a communication unit 107.

The controller 101 may execute an application (or program) and process data in the wearable device 100. In addition, the control unit 101 may control the memory unit 102, the camera 103, the microphone 104, the warning unit 105, the image processing unit 106, and the communication unit 107. Control data transmission and reception between them.

The controller 101 is a processor, an application-specific integrated circuit (ASIC), other chipsets, logic circuits, registers, communication modems, data processing apparatuses known in the art to execute control logic, which will be described in detail below. And the like. In addition, when the control logic is implemented in software, the control unit 101 may be implemented as a set of program modules. In this case, the program module may be stored in the memory unit 102 and executed by a processor.

The memory unit 102 may store various data. In addition, an application (or a program) for the operation of the controller 101 may be stored, or an image obtained through the camera 103 or a sound obtained through the microphone 104 may be stored.

The memory unit 102 may be a random access memory (RAM), a static random access memory (SRAM), a read only memory (ROM), an electrically erasable programmable read only memory (EPEROM), a programmable read only memory (PROM), or the like. have. In addition, the wearable device 100 may operate in association with a web storage that performs a storage function of the memory unit 102 on the Internet.

In addition, the memory unit 102 may further include an external storage medium detachable from the wearable device 100. The external storage medium may be in the form of a slot such as Secure Digital (SD) or Compact Flash (CF) memory, a memory stick, or a universal serial bus (USB). That is, the external storage medium may be detachable from the wearable device 200, and any storage medium capable of providing data such as audio, a photo, a video, an application, and the like to the wearable device 100 may be used.

The camera 103 senses an environment image of the wearable device 100 and converts the image into an electrical signal. To this end, the camera 103 may include an image sensor, and the image sensor may convert an optical signal into an electrical signal. The image sensed by the camera 103 and converted into an electrical signal may be stored in the memory unit 102 and then output to the controller 101 or directly to the controller 101 without being stored. The image sensed by the camera 103 may be a still image or a video.

The microphone 104 detects the ambient sound of the wearable device 100 and converts it into an electrical signal. In the present specification, the microphone 104 may be always in an ON state to sense sound. In addition, even when the microphone 104 is in an OFF state, the microphone 104 may be turned on by a command of the controller 101.

The warning unit 105 serves to output a visual signal or an audio signal to the user by the signal output from the control unit 101.

According to one embodiment of the present specification, the warning unit 105 may be a display unit. The display unit includes a liquid crystal display, a thin film transistor liquid crystal display, a light emitting diode, an organic light emitting diode, a flexible display, It may be a 3D display or the like. In addition, the display unit may be a hollow or transparent glass for displaying a virtual screen. That is, anything that can visually deliver a video signal to a human can be a display unit.

According to another embodiment of the present specification, the warning unit 105 may include an LED, an LCD, an alarm alarm, or a combination thereof. In this case, the warning unit 105 may output a visual signal or an audio signal to the user by blinking the LED, outputting a warning message on the LCD, or generating an alarm or buzzer sound. However, the present specification is not limited thereto. In addition, the LED, LCD and alarm alarm is only one example of the warning unit, it will be apparent to those skilled in the art that various modified forms of visual or audio alarm device can be employed as the warning unit 105.

The image processor 106 arranges one or more virtual objects on one or more layers and superimposes them on the image of the real world sensed by the camera 103. An augmented reality image in which virtual objects of the one or more layers are overlapped may be displayed through the display unit. That is, the camera 103 senses an image of the real world, stores it in the memory 102, and outputs the image to the image processor 106, or directly to the image processor 106 without storing the image. The image processor 106 creates an augmented reality image by overlaying one or more virtual objects in a layered structure on an image of the real world sensed by the camera 103.

The wearable device 100 may be connected to one or more external devices 200 that provide data through wired or wireless. In this case, the wearable device 100 may transmit / receive data with the external device 200 through the communication unit 107.

The external device 200 may be any device capable of transmitting or receiving data to the wearable device 100. For example, the external device 200 may be a mobile terminal or a fixed terminal. The mobile terminal may be a mobile phone, a smart phone, a tablet PC (Personal Computer), a smart pad, a notebook, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), a digital camera, a navigation device, or the like. The fixed terminal may be a desktop, a DVD (Digital Video Disc or Digital Versatile Disc) player, a TV, or the like.

The communication unit 107 and the external device 200 may transmit / receive data via wired or wirelessly using various protocols. For example, when wired, an interface such as HDMI (High Definition Multimedia Interface) or DVI (Digital Visual Interface) may be supported. As another example, in the case of wireless, 2G, 3G, 4G mobile communication methods such as Global System for Mobile Communications (GSM), Code Division Multiple Access (CDMA), Wibro, other High Speed Packet Access (HSPA), High Speed Downlink (HSDPA) Mobile communication such as Packet Access (LTE), Long Term Evolution (LTE), or Bluetooth, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, Wireless LAN A short range communication interface such as Wi-Fi may be supported.

Herein, the wired / wireless interface schemes are embodiments for better understanding of the present specification, and the interface scheme for transmitting / receiving information may be easily changed by those skilled in the art. Will not.

Referring to FIG. 3A, as an example of the wearable device 100, a user wearing a head mounted display (HMD) may be checked. And the user can confirm that the hand holding the packaged food. An example of a packaged food is shown a snack packaged in a poly bag.

Referring to FIG. 3B, an image sensed through the camera 103 included in the wearable device 100 may be checked. The operation of the controller 101 is performed based on the image received through the camera 103. Therefore, the image received through the camera 103 to be described in detail herein below and the operation of the controller 101 are easy to understand in consideration of the situation shown in the reference diagram of FIG. 3.

On the other hand, in the present specification, as a wearable device 100, but shown as a device in the form of wearing on the head like glasses, the example is not intended to limit the scope of the present disclosure. The wearable device 100 refers to any device that can be attached to the user's body to perform a computing action. At the time of filing the present specification, various types of wearable devices such as glasses, hair bands, watches, arm bands, shoes, neckbands, rings, and contact lens types have been disclosed, but the wearable device 100 according to the present disclosure It is not restricted depending on the form.

The control unit 101 according to the present specification may distinguish the user's hand motion and the packaged food from the first image received through the camera 103. Techniques for distinguishing the user's hand motion from the image received by the controller 101 through the camera 103 are well known to those skilled in the art at the time of filing the present specification, and thus detailed descriptions thereof will be omitted. In addition, the controller 101 may distinguish the packaged food through a method of recognizing a brand name printed on a wrapping paper or a barcode printed on the wrapping paper from an image received through the camera 103. How the controller 101 distinguishes the user's hand motion and the packaged food from the image received through the camera 103 may be applied to all techniques that can be easily used by those skilled in the art not only at the time of the present application but also after the application. have. The first image refers to an image received through the camera 103 in time with a second image to be described later.

The controller 101 determines whether the package of the food distinguished from the first image received through the camera 103 is opened by a user's hand gesture distinguished from the first image received through the camera 103. To judge.

Referring to FIG. 4A, a user's hand operation of opening a polybag package may be confirmed. In the case of the polybag, both hands holding the package may be opened through an operation of spreading from side to side. In addition, in the case of the poly bag, it can be opened through the operation of holding the end of the package with one hand to tear down. In addition, referring to Figure 4 (b), it is possible to check the hand operation of the user opening the bottle. In the case of a bottle, you can open it by turning the lid by hand.

Accordingly, the controller 101 determines whether the user's hand gesture distinguished from the first image received through the camera 103 is an operation of opening the package according to the packaging characteristics of the food such as tearing, cutting and turning. do. Then, it is determined whether the food package is opened by the user's hand operation.

According to an embodiment of the present invention, the controller 101 may determine whether the package of the food is opened by comparing the images.

In order to compare the images, the memory unit 102 may further store an opening image data table including image data of a state in which a package is opened (hereinafter, referred to as “opening image data”) according to packaging characteristics of various foods.

Referring to FIG. 5, a table including unsealed image data may be confirmed. For example, the opening image data may be an image of a state in which a crack is formed in a part of a package of food, such as an image in which a poly bag is torn. As another example, the opening image data may be an image of a state in which a lid included in a package of food is removed, such as a lid of a soda bottle is opened. As another example, the opening image data may be an image of a state in which some of the joints formed on the package of the food are separated, such as the opening of the milk carton is opened.

In this case, the controller 101 may determine packaging characteristics of the food distinguished from the first image received through the camera 103. In addition, the control unit 101 may compare the package image of the food distinguished from the first image received through the camera 103 with the image data mapped to the package image of the food in the opening image data table. It may be determined whether the package has been opened. For example, when it is determined that the packaging property of the food distinguished from the first image is a poly bag, the controller 101 may display the unsealed image data corresponding to the poly bag in the unsealed image data table stored in the memory 102. Read it. In addition, when it is determined that the package image of the food distinguished from the first image and the unsealing image data are matched or similar, the controller 101 determines that the package of the food is opened.

According to another embodiment of the present invention, the control unit 101 may determine whether the package of the food is opened by comparing the sound.

For sound comparison, the memory unit 102 may further store an opening sound data table including sound data generated when the package is opened according to various packaging properties of the food (hereinafter, 'opening sound data').

Referring to FIG. 5, a table including opening sound data may be confirmed. For example, the opening sound data may be a sound generated when a crack is formed in a part of a package of food. In the case of the polybag, the opening sound data may be a unique sound generated when the polybag is torn. As another example, the opening sound data may be a sound generated when the lid included in the package of food is removed. In the case of a bottle containing a carbonated beverage, the opening sound data may be a unique sound of carbonic acid leaking out when the lid is opened. As another example, the opening sound data may be a sound generated when some of the joints formed on the package of food drops. In the case of a milk carton, the opening sound data may be a unique sound generated when the opening of the carton is opened. 5 of the present specification shows a 'speaker image' to indicate that it is sound data, and describes 'onomatopia' with a similar voice. There are many limitations to expressing sounds in text. Therefore, the 'speaker image' and the 'ontology' are only one expression method for representing the opening sound data according to the present specification, and should be interpreted as not limiting the scope of the present specification.

The controller 101 determines the packaging characteristics of the food distinguished from the first image received through the camera 103, and the packaging characteristics of the food in the sound and the opening sound data table received through the microphone 104. It is possible to determine whether the package of the food is opened by comparing the opening sound data mapped according to. For example, when it is determined that the packaging characteristics of the food distinguished from the first image are polybags, the controller 101 may read the opening sound data corresponding to the polybags from the opening sound data table stored in the memory unit 102. Read it. In addition, when it is determined that the sound received through the microphone 104 and the opening sound data are matched or similar, the package of the food is determined to be opened.

On the other hand, if it is determined that the package of the food distinguished from the first received image is opened, the control unit 101 includes data about the opened food (hereinafter 'food data') and the time when the package is opened. Data (hereinafter, referred to as 'opening time') may be stored in the memory unit 102.

As illustrated in FIG. 6, food data and opening time may be stored in the memory unit 102. The food data is data for distinguishing the food in which the package is opened. Accordingly, the controller 101 may allocate identification information (for example, a number) to each food and store the same in the memory 102 to distinguish a plurality of foods. In addition, the opening time is time data when the package is opened for each food. Therefore, the time information is stored in the memory unit 102 in association with the food data.

The controller 101 stores information regarding the opening time when the food distinguished from the second image received through the camera 103 matches the food data after storing the food data and the opening time. It can output the provided information signal. The second image is an image received through the camera 103 after the control unit 101 stores the food data and the opening time in the memory unit 102, that is, later in time than the first image. Means an image received through the camera 103. For example, when the package of the snack is opened, the image received through the camera 103 is the first image, and when the user views the snack again a few days after the user has left the snack, the camera 103 The image received through may be referred to as a second image.

The information signal is a signal including information related to the opening time. The information signal may include the opening time itself, and may include a time elapsed from the opening time, a period in which the food can be ingested from the day of opening the food, or a remaining time of ingesting the food. For example, suppose that the food distinguished from the second image received through the camera 103 corresponds to 'snack 001' of FIG. 5. In this case, the controller 101 may check the memory unit 102 for '2014.07.01', which is time information corresponding to 'snack 001'. The time information '2014.07.01' may be output as an information signal to the user.

According to one embodiment of the present specification, the controller 101 may output time information that has elapsed from the opening time to the present as an information signal. Assuming the current time is 30 July 2014, 'Snack 001' has been 30 days since the package was opened. Therefore, the controller 101 can output '30 days' as an information signal.

According to another exemplary embodiment of the present specification, the controller 101 may output a content of comparing the elapsed time and the opening time previously stored as the information signal. To this end, the memory unit 102 may further store an elapsed period data table including an elapsed period according to various food characteristics. The elapsed period refers to a period in which food can be consumed after the food package is opened apart from the distribution period of the food. In this case, the controller 101 may determine the characteristics of the food distinguished from the first image received through the camera 103. The controller 101 may output information related to the remaining time or the elapsed time of the elapsed time calculated by mapping the characteristics of the determined food in the elapsed time data table from the opening time as an information signal.

For example, in the case of a snack, it is assumed that '45 days' is stored in the memory unit 102 as the elapsed period data. If the opening date of the snack is July 1, 2014 and the current time is July 30, 2014, the controller 101 outputs the remaining 15 days as the information signal. As another example, it is assumed that '10 days' is stored as elapsed period data in the memory unit 102. If the opening date of the soda is July 15, 2014 and the current time is July 30, 2014, the control unit 101 outputs '5 days of the transition time' as the information signal.

According to another embodiment of the present specification, the controller 101 may output information related to the remaining time or the elapsed time of the elapsed period input by the user from the opening time as an information signal. To this end, the memory unit 102 may further store the elapsed time input by the user for the packaged food.

According to the present specification, the controller 101 may output the information signal to the warning unit 105.

As described above, according to the exemplary embodiment of the present specification, the warning unit 105 may be a display unit for outputting an image. The controller 101 may output the information signal to display time information related to the opening time stored in the memory unit 102 on the display unit.

Preferably, the controller 101 may control to display the time information related to the opening time as a virtual object, and to control the virtual object to be located in a portion adjacent to the food image of the image of the real world sensed by the camera. have. In this case, the wearable device 100 according to the present specification may further include the image processor 106 described above.

On the other hand, the control unit 101 may determine whether to open the package of food when receiving a user input. In other words, the controller 101 may perform the process of determining whether the above-mentioned food package is opened and storing the opening time and outputting an information signal based on this, only when the user desires.

On the other hand, the control unit 101 may determine whether to open the package of food when reading the information displayed on the package of food distinguished from the first image received through the camera 103. Even if there is no separate input by the user, the controller 101 can automatically perform a process of determining whether the package of the food is opened and opening time and outputting an information signal based on the opening time. At this time, the information displayed on the packaging of the food may be a brand name of the food, may be a barcode or QR code.

Hereinafter, the control method of the wearable device 100 described above will be described. However, since each configuration and role of the wearable device 100 have already been described in detail, repeated description thereof will be omitted.

Referring to FIG. 7, first, in step S300, the controller 101 distinguishes a user's hand motion from a first image received through the camera 103. Then, the controller 101 finishes step S300 and proceeds to step S310.

In step S310, the controller 101 distinguishes the packaged food from the first image received through the camera 103. The controller 101 completes step S310 and proceeds to step S320.

A method of distinguishing the user's hand motion and the packaged food from the first image received through the camera 103 has been described with reference to FIGS. 3 and 4. In addition, since the first image has already been described, repeated description thereof will be omitted. In the meantime, the order of steps S300 and S310 may be changed.

In a next step S320, the control unit 101 opens the package of food distinguished from the image received through the camera 103 by a user's hand gesture distinguished from the first image received through the camera 103. Determine whether or not.

According to one embodiment of the present specification, the controller 101 determines whether the package of food is opened through image comparison.

According to another embodiment of the present specification, the control unit 101 determines whether or not the package of food is opened through sound comparison.

Since the above embodiment has been described with reference to FIG. 5, repeated description thereof will be omitted.

If the package of the food is opened (YES in step S330), the controller 101 proceeds to step S330. On the other hand, if the package of food is not opened (NO in step S330), the controller 101 proceeds to step S300, and repeatedly performs steps S300 to S320.

In step S330, the controller 101 stores the data about the packaged food (hereinafter 'food data') and the data about the time when the package is opened (hereinafter 'opening time') of the memory unit 102. Store in The controller 101 finishes this step and proceeds to step S340. Since the food data and the opening time have been described with reference to FIG. 6, repeated description thereof will be omitted.

In operation 340, the controller 101 determines whether the food distinguished from the second image received through the camera 103 matches the food data. If the food data matches (YES in step S350), the control unit 101 proceeds to step S360. On the other hand, if the food data does not match (NO in step S350), the control unit 101 proceeds to step S340 to repeatedly perform steps S340 and S350. On the other hand, since the description of the second image is repeated, it will be omitted.

In step S360, the controller 101 outputs an information signal for providing a user with information related to the opening time corresponding to the food data.

According to one embodiment of the present specification, the information signal may be time information that has elapsed from the opening time to the present.

According to another exemplary embodiment of the present specification, the memory unit 102 may further store an elapsed period data table including an elapsed period according to various food characteristics. The controller 101 may output information related to the remaining time or the elapsed time of the elapsed time calculated by mapping the characteristics of the determined food in the elapsed time data table from the opening time as the information signal.

According to another embodiment of the present specification, the memory unit 102 may further store an elapsed time input by the user for the food in which the package is opened. The controller 101 may output information related to the remaining time or the elapsed time of the elapsed period input by the user from the opening time as the information signal.

The terminology used herein is a general term that has been widely used as far as possible while considering the functions of the embodiments disclosed herein, but may vary according to the intention or custom of a person skilled in the art or the emergence of a new technology. In addition, in certain cases, there is also a term arbitrarily selected by the applicant, in which case the meaning will be described in detail in the description of the specification. Therefore, it is to be understood that the terminology used herein is to be defined based on the meaning of the term and the contents throughout the specification, rather than on the name of the term.

In addition, specific structural to functional descriptions of the embodiments disclosed herein are illustrated for the purpose of describing the embodiments only, and the embodiments may be embodied in various forms. It should not be construed as limiting the scope of rights.

Embodiments according to the concepts of the present disclosure may be variously modified and may have various forms, and specific embodiments are illustrated in the drawings and described in detail herein. However, this is not intended to limit the embodiments in accordance with the concepts herein to the specific form disclosed, it should be understood to include all changes, equivalents, and substitutes included in the spirit and scope of the present specification.

In addition, throughout the specification, when a part is said to "include" a certain component, it means that it can further include other components, without excluding the other components unless otherwise stated. In addition, the term "~ part" described in the specification means a unit for processing at least one function or operation, which may be implemented by hardware or software, or a combination of hardware and software.

In the above description, the present specification has been described through specific embodiments, but those skilled in the art may make modifications and changes without departing from the spirit and scope of the present specification. Therefore, what can be easily inferred by the person belonging to the field to which the technology disclosed by this specification belongs from the detailed description and the Example of this specification is interpreted as belonging to the scope of the present specification.

In addition, in the description of the embodiments of the present disclosure, for convenience of understanding, the dependent configuration has been described based on the example of adding only one, but an embodiment in which two or more dependent configurations are added in combination is also possible. Accordingly, the scope of the present specification is not limited to the embodiments disclosed herein.

Various embodiments have been described in the best mode for carrying out the present specification.

It will be apparent to those skilled in the art that various modifications and variations can be made herein without departing from the spirit and scope of this specification. Accordingly, this specification is intended to cover the modifications and variations of this specification provided within the scope of the appended claims and their equivalents.

This disclosure may apply to electronic devices in whole or in part.

Claims

A memory unit for storing data;

A camera for sensing an image; And

Distinguish the user's hand motion from the first image received through the camera and the packaged food, and in the first image received through the camera by the user's hand motion distinguished from the first image received through the camera. When it is determined that the package of the distinguished food is opened, data about the food in which the package is opened (hereinafter 'food data') and data on the time when the package is opened (hereinafter 'opening time') are stored in the memory unit. Storing the food data and the opening time, and outputting an information signal for providing the user with information related to the opening time when the food distinguished from the second image received through the camera matches the food data. Wearable device comprising a control unit.
The method of claim 1,

The memory unit further stores an opening image data table including image data (hereinafter, referred to as 'opening image data') in a state in which a package is opened according to various packaging characteristics of food.

The control unit may determine packaging characteristics of the food distinguished from the first image received through the camera, and package the food from the package image of the food distinguished from the first image received through the camera and the unsealed image data table. Wearable device, characterized in that to determine whether the package of the food is opened by comparing the image image data mapped according to the characteristics.
The method of claim 2,

The opening image data is a wearable device, characterized in that the image of a state in which a crack is formed on a portion of the package of food.
The method of claim 2,

The opening image data, wearable device, characterized in that the image of the lid is removed in the package of food.
The method of claim 2,

The opening image data is a wearable device, characterized in that the image of a part of the bonding portion formed on the packaging of food apart.
The method of claim 1,

Further comprising a microphone for sensing the sound,

The memory unit further stores an opening sound data table including sound data generated when the package is opened according to various packaging properties of the food (hereinafter, 'opening sound data'),

The control unit may determine packaging characteristics of the food distinguished from the first image received through the camera, and the sound of the opening sound data mapped according to the packaging characteristics of the food in the sound and the opening sound data table received through the microphone. A wearable device, characterized in that determining whether the package of food is opened by comparison.
The method of claim 6,

The opening sound data is a wearable device, characterized in that a sound generated when a crack is formed in a part of the package of food.
The method of claim 6,

The opening sound data is a wearable device, characterized in that the sound generated when the lid included in the packaging of the food is removed.
The method of claim 6,

The opening sound data is a wearable device, characterized in that a sound generated when some of the joints formed on the packaging of the food falls.
The method of claim 1,

The control unit is a wearable device, characterized in that for outputting time information elapsed from the opening time to the present as an information signal.
The method of claim 1,

The memory unit further stores an elapsed period data table including an elapsed period according to various food characteristics,

The controller may determine a characteristic of the food distinguished from the first image received through the camera, and the remaining time of the elapsed period calculated by mapping the characteristic of the determined food in the elapsed period data table from the opening time or Wearable device, characterized in that for outputting information related to the time of the transition as an information signal.
The method of claim 1,

The memory unit further stores the elapsed time input by the user for the food, the package is opened,

The control unit, wherein the wearable device, characterized in that for outputting information relating to the remaining time or the elapsed time of the elapsed period input by the user from the opening time as an information signal.
The method of claim 1,

And a warning unit for outputting a visual signal or an audio signal.

The control unit outputs the information signal to the warning unit.
The method of claim 13,

The warning unit is a display unit for outputting an image,

The control unit, wherein the wearable device, characterized in that for outputting the information signal to display the time information associated with the opening time stored in the memory unit on the display unit.
The method of claim 14,

And an image processor for disposing one or more virtual objects on one or more layers and superimposing them on the image of the real world captured by the camera.

The controller may be configured to control the time information related to the opening time to be displayed as the virtual object, and to control the virtual object to be positioned in a portion adjacent to the food image of the photographed real world image. .
The method of claim 1,

The control unit, the wearable device, characterized in that for determining whether to open the food packaging when receiving a user input.
The method of claim 1,

The control unit, the wearable device, characterized in that for determining whether to open the package of food when reading the information displayed on the packaging of the food distinguished from the first image received through the camera.
The method of claim 17,

Wearable device, characterized in that the information displayed on the packaging of the food is a brand name of the food.
The method of claim 17,

Wearable device, characterized in that the information displayed on the packaging of the food is a barcode or QR code.
(a) distinguishing the user's hand gesture and the packaged food from the first image received through the camera;

(b) determining whether the package of the food distinguished in the image received through the camera is opened by a user's hand gesture distinguished in the first image received through the camera;

(c) when the package of the food is opened, storing data about the food in which the package is opened (hereinafter 'food data') and data on the time when the package is opened (hereinafter 'opening time') step; And

and (d) outputting an information signal for providing the user with information related to the opening time when the food distinguished from the second image received through the camera matches the food data. Type control method.