EP2843337A1

EP2843337A1 - Refrigerator and refrigerator system for monitoring food

Info

Publication number: EP2843337A1
Application number: EP13182845.1A
Authority: EP
Inventors: Toyoshi Kamisako; Masashi Nakagawa; Motoki Nakamura
Original assignee: Panasonic Corp
Current assignee: Panasonic Corp
Priority date: 2013-09-03
Filing date: 2013-09-03
Publication date: 2015-03-04

Abstract

The present invention provides a refrigerator and a refrigeration system capable of easily discriminating food in the storage chamber. The refrigerator includes communication means (53) which are connectable to the internet, and a camera (35) which captures the food in the storage chamber. The communication means (53) transmit image data captured with the camera (35) to a server connected to the internet. The image data is visually confirmed through a display terminal (15) connected to the server. In-refrigerator illumination lamps (34) that illuminate the inside of the storage chamber at the time of the capturing with the camera (35) are provided in the storage chamber. The cameras (35) and the in-refrigerator illumination lamps (34) are positioned on each sidewall of the storage chamber. Therefore, the food in the storage chamber can be captured brightly and clearly. Further, the image processing for facilitating the discrimination of the food is not necessary, and the refrigerator can be put to practical use at low cost.

Description

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a refrigerator and a refrigerator system of the refrigerator capable of capturing and monitoring food in a storage chamber and confirming a stored state of the food by a display terminal or the like.

2. Description of the Related Art

Generally, a large number of this kind of refrigerators and refrigeration systems have been proposed. For example, a refrigerator and a refrigeration system described in Japanese Patent Application Laid-Open No. 2002-236798 are configured such that a compact camera is provided in a storage chamber, and a state of the food stored in the storage chamber and captured with the compact camera can be confirmed through a display terminal such as a cell-phone via the internet line.
Further, a refrigerator and a refrigeration system similar to those described in Japanese Patent Application Laid-Open No. 2002-236798 are also disclosed in Japanese Patent Application Laid-Open No. 2007-46833 . The refrigerator and the refrigeration system described in Japanese Patent Application Laid-Open No. 2007-46833 are configured such that when a door of the refrigerator is closed, a state of the food in the storage chamber is captured with a compact camera.
Further, a refrigerator and a refrigeration system described in Japanese Patent Application Laid-Open No. 2002-81818 are configured such that a display terminal is further provided on the surface of the refrigerator in addition to the configuration described in Japanese Patent Application Laid-Open No. 2002-236798 . The display terminal is configured by a fixed terminal unit built into the surface of the refrigerator, and a display terminal such as a cell-phone that can be detached from the surface of the refrigerator and carried around. Both of the terminals can display in their respective display units a state of the food stored in the storage chamber and captured with a compact camera, via the internet line, and a user can know the state of the food stored in the storage chamber without opening the door of the refrigerator both when the user is at the outside and when the user is at home.
As described above, there have been proposed a large number of this kind of refrigerator and refrigerator systems. That is, refrigerators and refrigerator systems that are configured to be able to confirm a state of the food stored in the storage chamber by using the display terminal by capturing the food in the storage chamber. However, none of these refrigerators and refrigerator systems of the refrigerators has been put to practical use.
The present inventors have studied for practical use of the refrigerators and the refrigerator systems, and have carried out tests. As a result, the present inventors have found many problems to be solved to achieve the practical use of the refrigerators and the refrigerator systems.
One of the problems is that the quality of the image of a state of the food stored in the storage chamber and captured with the compact camera is poor and it is difficult to discriminate the type of the food.
Japanese Patent Application Laid-Open No. 2007-46833 discloses that in capturing with a compact camera, the capturing is performed by turning on an in-refrigerator illumination lamp in the storage chamber. However, when the food illuminated by the in-refrigerator illumination lamp positioned on a storage chamber back wall is captured with the compact camera provided on a door inner surface, the image is dark and the type of the food cannot be easily discriminated. Alternatively, it is necessary to facilitate the discrimination of the type of the food by performing a high level correction process, and this results in substantial cost increase and becomes a large barrier for the practical use.

SUMMARY OF THE INVENTION

The present invention has been made in view of the above points, and an objective thereof is to provide at low cost a refrigerator and a refrigeration system capable of satisfactorily capturing the food in a storage chamber and capable of easily discriminating the food.
To achieve the above objective, the present invention provides a refrigerator including communication means connectable to the internet, and cameras for capturing food in a storage chamber, the communication means to transmit image data captured with the cameras to a server connected to the internet, and the image data being visually confirmed through a display terminal connected to the server. The cameras and in-refrigerator illumination lamps that illuminate the inside of the storage chamber when capturing with the cameras are provided in the storage chamber. The cameras and the in-refrigerator illumination lamps are disposed respectively in the front of the opening sides of the storage chamber.
With this configuration, the food in the storage chamber is illuminated by the in-refrigerator illumination lamp, and the inside of the storage chamber is captured with the camera next to the in-refrigerator illumination lamp. Therefore, the food in the storage chamber can be captured brightly and clearly. Accordingly, the type of the food in the storage chamber can be easily discriminated, and the refrigerator can be put to practical use at low cost without requiring the image processing to facilitate discrimination of the food.
According to the present invention, it is possible to provide at low cost a refrigerator and a refrigeration system capable of satisfactorily capturing the food in the storage chamber and easily discriminating the food in the storage chamber.

BRIEF DESCRIPTION OF DRAWINGS

Fig. 1 is a front view of a refrigerator according to an embodiment of the present invention;
Fig. 2 is a schematic cross-sectional view of the refrigerator;
Fig. 3 is a perspective view of the inside of the refrigerator in a state that a door of a storage chamber is opened;
Fig. 4A is a perspective view of the refrigerator in which a display terminal is integrated, and Fig. 4B is a perspective view of the refrigerator in a state that the display terminal is detachably mounted on the refrigerator;
Figs. 5A and 5B show examples of a configuration of mounting the display terminal used in the refrigerator, Fig. 5A is a cross-sectional view showing integrally fitting of the display terminal, and Fig. 5B is a cross-sectional view showing detachable mounting of the display terminal;
Figs. 6A, 6B, and 6C show examples of a visual confirmation configuration of a display of the display terminal used in the refrigerator, Figs. 6A and 6B are schematic cross-sectional views of the display terminal when an outer panel of the door is made of glass, and Fig, 6C is a schematic cross-sectional view of the display terminal when the outer panel of the door is configured by a magic mirror;
Fig. 7 is a vertical cross-sectional view of the refrigerator showing an example of fitting of cameras to the refrigerator;
Figs. 8A and 8B show examples of the fitting of cameras to the refrigerator, Fig. 8A is a lateral cross-sectional view of a refrigerator door portion in a state that the door is closed, and Fig. 8B is a front view of a door inner surface side when the door is opened;
Figs. 9A, 9B, and 9C are vertical cross-sectional views of the refrigerator showing another example of the fitting of cameras to the refrigerator;
Fig. 10 is a vertical cross-sectional view of the refrigerator showing an example of the fitting of cameras when the refrigerator has a single door;
Figs. 11A and 11B show examples of the fitting of cameras to the refrigerator, Fig. 11A is a lateral cross-sectional view of the refrigerator door portion in a state that the door is closed, and Fig. 11B is a front view of the door inner surface side when the door is opened;
Fig. 12A and B are vertical cross-sectional views of the refrigerator showing another example of the fitting of cameras when the refrigerator has a single door;
Fig. 13 is an explanatory view showing a capturing state in the example of the fitting of the camera in Fig. 12;
Fig. 14 is an explanatory view showing an example of cameras fitted to the refrigerator;
Fig. 15 is a system configuration diagram of a refrigerator system according to the embodiment;
Fig. 16 is a block diagram showing a refrigerator configuration in the refrigerator system;
Fig. 17 is a block diagram showing a server configuration in the refrigerator system;
Fig. 18 is a flowchart showing the capturing to data storing operation when the door of the refrigerator system is closed;
Fig. 19 is a flowchart showing the capturing to data storing operation when a cooling operation of the refrigerator system is stopped;
Fig. 20 is a flowchart showing the capturing to data storing operation when the door of the refrigerator system is opened;
Fig. 21 is a flowchart showing another operation example of the capturing operation in Fig. 18 to Fig. 20;
Fig. 22 is a flowchart showing the operation of data storing and combining in the refrigerator system when it is performed by a server unit;
Fig. 23 is a flowchart showing the operation of data storing and combining in the refrigerator system when it is performed at a refrigerator side;
Fig. 24 is a flowchart showing the operation of data storing and combining in the refrigerator system when it is performed at a display terminal side;
Fig. 25 is a flowchart showing ordinary transmission and reception of image data in the refrigerator system;
Fig. 26 is a flowchart showing transmission and reception of image data when image data is stored in the display terminal in the refrigerator system;
Fig. 27 is a flowchart showing transmission and reception of an individual image and a combined image by changing over between these images in the refrigerator system;
Fig. 28 is a flowchart showing transmission and reception of image data at the time of making a request for prior transmission in the refrigerator system;
Fig. 29 is a front view of the display terminal for describing changeover between transmission and reception paths in the refrigerator system; and
Fig. 30 is a view for describing the image combining process in the refrigerator system.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

Hereinafter, before describing in detail a concrete embodiment of the present invention, an outline of the embodiment of the present invention is described.
A refrigerator according to the configuration of the present invention includes communication means connectable to the internet and a camera for capturing the food in the storage chamber, the communication means to transmit image data captured with the camera to a server connected to the internet, and the image data able to be visually confirmed through a display terminal connected to the server. The camera and an in-refrigerator illumination lamp that illuminates the inside of the storage chamber at the time of capturing with the camera are mounted in the storage chamber. The cameras and the in-refrigerator illumination lamps are positioned at the front opening sides of the storage chamber.
With this configuration, the food in the storage chamber is illuminated by the in-refrigerator illumination lamps, and the inside of the storage chamber is captured with the cameras at the side of illuminating by the in-refrigerator illumination lamps. Therefore, the food in the storage chamber can be captured brightly and clearly. Accordingly, the food in the storage chamber can be easily identified, and the refrigerator can be put to practical use at low cost without requiring image processing to facilitate discrimination of the food.
The storage chamber includes different shelf boards, and the cameras and the in-refrigerator illumination lamps are positioned in front of the front ends of the shelf boards.
With this configuration, images of the inside of the storage chamber captured with the cameras become to when the user actually looks at the inside by opening the door of the refrigerator. When the state of the food in the storage chamber is displayed in the display terminal, the user can confirm the state of the food quite naturally. Further, because the in-refrigerator illumination lamps illuminate the food on the shelf boards at positions in front of the front ends of the shelf boards, the food placed on the front end part of each shelf board can be also captured brightly and clearly.
The cameras are mounted on the inner surface of the door, and the in-refrigerator illumination lamps are positioned at both sides of the sidewalls of the storage chamber.
Accordingly, the cameras are positioned at approximately the center of the storage chamber. Images captured with the cameras are similar to when the user actually looks at the inside by opening the door of the refrigerator. A wide area of the inside of the storage chamber can be captured at a large distance from the food on the shelf boards, and the user can confirm the state of the food stored in the wide range in the display terminal.
Further, the cameras are positioned at approximately the center position in a front-and-back direction of each door pocket provided on the door inner surface part. The food such as a milk pack stored in the door pocket can be also captured with the cameras. The cameras can capture a wider range of the inside of the storage chamber and are very convenient.
The in-refrigerator illumination lamps are mounted at positions where light of the in-refrigerator illumination lamps does not directly enter the cameras, or are configured such that light does not directly enter the cameras.
With this configuration, it is possible to prevent poor quality of the image of each camera caused by the entrance of light of the in-refrigerator illumination lamps into the cameras, and it is always possible to obtain a satisfactory image.
The refrigerator system according to the implementation of the present invention includes any one of the refrigerators described in the above examples, a display terminal that can display the in-refrigerator image of the refrigerator, and a server that stores image information transmitted via the internet from communication means of the refrigerator, and that receives an image request signal transmitted from the display terminal via the internet and transmits the image information to the display terminal.
With this configuration, the user can clearly confirm the state of the food in the refrigerator at any time and at any place, which could help to prevent forgetting to purchase articles of food, and can conveniently use the refrigerator system.
Preferably, the server is configured to include a function of transmitting image data of the inside of the refrigerator in advance to the display terminal in response to a request for prior transmission of the image data from the display terminal. With this configuration, the user can immediately confirm the image data of the inside of the refrigerator by operating the display terminal even when a response from the server is late due to lag from the server. Consequently, ease of use further improves.
Hereinafter, an embodiment of the present invention is described with reference to the drawings. The present invention is not limited to the following embodiment.

EMBODIMENT

<1. Configuration>

1-1. Configuration of refrigerator

1-1-1. Configuration of French-door type refrigerator

A configuration example of a French-door type refrigerator is first described with reference to Figs. 1 to 9.
In Figs. 1 to 3, refrigerator body 1 includes outer box 2 made of a metal (for example, a metal plate) opened to the front, inner box 3 made of a hard resin (for example, ABS), and heat insulation material 4 made of hard foamed urethane or the like that is foamed and filled in between outer box 2 and inner box 3. Refrigerator body 1 has a storage chamber partitioned into a plurality of compartments inside refrigerator body 1. The storage chamber includes from the top in refrigerator body 1, cold compartment 5, change compartment 6 which is positioned below cold compartment 5, ice compartment 7 which is positioned in parallel with change compartment 6, freezing compartment 8 which is positioned at a lower part of change compartment 6 and ice compartment 7, and vegetable compartment 9 which is positioned at a lower part of freezing compartment 8. The front of cold compartment 5 is openably closed by double doors 10. Front parts of change compartment 6, ice compartment 7, freezing compartment 8, and vegetable compartment 9 are openably closed by drawer doors 11, 12, 13, and 14 (hereinafter, drawer doors).
Cooling compartment 16 is present on a back surface of refrigerator body 1, and is provided with cooler 17 for generating cold air, blast fan 18 for supplying cold air to each compartment, and damper (air volume adjusting means) 22 for adjusting the air volume to the cold compartment. Compressor 19 is present at a body ceiling back part of refrigerator body 1, and is configured to perform a cooling operation by sealing a cooling medium in a freezing cycle formed by sequentially connecting in a circular shape a condenser (not shown), radiation pipe 20 for releasing heat, capillary tube 21, and cooler 17.
Doors 10 to 14 have heat insulation properties by being filled in with hard foamed urethane similar to that filled in refrigerator body 1. In the refrigerator, display terminal 15 is provided at the front of one of doors 10 of storage chamber 5. Display terminal 15 is configured by a display unit such as a liquid crystal panel and a control unit that controls the display unit and is integrally fitted to door 10 as shown in Fig. 4A, or is formed by a smartphone or a tablet PC (Personal Computer) and is detachably provided on door 10 as shown in Fig. 4B. Display terminal 15 may be provided in one of these methods.
Figs. 5A and 5B show examples of a configuration of mounting display terminal 15 on door 10. Fig. 5A shows integrally fitting of display terminal 15, and Fig. 5B shows detachable mounting of display terminal 15.
In Figs. 5A and 5B, door 10 is mainly configured by outer panel 10a that faces the front side of the refrigerator, inside plate 10b that faces the in-refrigerator side of the refrigerator, and heat insulation material 4 filled in between outer panel 10a and inside plate 10b. Outer panel 10a is configured by a metal member such as a color steel plate or a transparent member such as a glass plate. Inside plate 10b is a synthetic resin member formed by vacuum molding. Protruded wall part 10b-1 that is protruded to the in-refrigerator side in a D shape viewed from the in-refrigerator side is provided at the peripheral portion of inside plate 10b. Hard foamed urethane or the like, for example, is used for heat insulation material 4, as described above.
Door 10 to which display terminal 15 is integrated as shown in Fig. 5A is provided with storage frame body 23 made of a resin which continuously covers heat insulation material 4 with outer panel 10a, at the front side of door 10 (when outer panel 10a is a glass plate, the storage frame body is integrally adhered to the glass plate). Display terminal 15 is fixed to storage frame body 23 via terminal holder 24. Touch panel 25 is provided at the front side of display terminal 15. Gap 26 is provided between display terminal 15 and touch panel 25 so that touch panel 25 does not generate an erroneous operation due to the noise from display terminal 15. In display terminal 15 and touch panel 25, frame body 27 that covers the outer peripheral edges of display terminal 15 and touch panel 25 is fitted and fixed to the surface of door 10. With the provision of storage frame body 23, heat insulation material 4 does not directly touch display terminal 15. Therefore, at the time of foaming heat insulation material 4, storage frame body 23 can prevent such inconvenience as exposure of display terminal 15 to a high temperature or deformation of display terminal 15. On the other hand, in door 10 shown in Fig. 5B in which display terminal 15 is set detachable, storage frame body 23 is provided at the front part of door 10 so that concave part 28 is formed, and display terminal 15 is configured to be set to or detached from concave part 28.
Figs. 6A, 6B, and 6C show examples of a see-through configuration of a display of door 10 and display terminal 15. Figs. 6A and 6B show that outer panel 10a of door 10 is glass plate 10a-1, and Fig. 6C shows that outer panel 10a of door 10 is configured by magic mirror 10a-2.
In Fig. 6A, smoked sheet 29 that is transparent at only a portion facing display terminal 15 is provided between display terminal 15 which is buried in door 10 and glass plate 10a-1. Even when display terminal 15 looks black when display terminal 15 is off, the portion of glass plate 10a-1 other than display terminal 15 also appears black due to smoked sheet 29. Because a boundary between glass plate 10a-1 and display terminal 15 is not noticeable, satisfactory appearance can be maintained. In Fig. 6B, a transparent liquid crystal display unit is used for the display unit of display terminal 15, and smoked sheet 29a which is black as a whole is provided on the rear surface of the transparent liquid crystal display unit. Satisfactory appearance can be also maintained in this form. Further, in Fig. 6C, glass plate 10a-1 in Figs. 6A and 6B is set as magic mirror 10a-2, and smoked sheet 29 is not required. Satisfactory appearance can be also maintained in this case. In all cases, display terminal 15 is integrally fitted to door 10. Instead of smoked sheet 29, a black coating may be provided on the rear surface side of glass plate 10a-1.
Fig. 7 and Figs. 8A and 8B show examples of the fitting of cameras and in-refrigerator illumination lamps which are the principal parts of the present invention. Fig. 7 is a vertical cross-sectional view of the refrigerator in a state that the door is closed, Fig. 8A is a lateral cross-sectional view of the door inner surface part of the refrigerator in a state that the door is closed, and Fig. 8B is a front view of the door inner surface part when the door is opened.
In Fig. 7, shelf boards 30-1 to 30-4 can be fastened at different heights in storage chamber 5 of the refrigerator. Shelf boards 30-1 to 30-4 are formed by a transparent material such as a transparent synthetic resin material and glass, except the surrounding frames.
Reference numeral 31 denotes a storage container that is drawably provided below lowest shelf board 30-4. Lowest shelf board 30-4 is fixed because this shelf board configures an upper surface of storage container 31, but shelf boards 30-1 to 30-3 are mounted respectively on sidewall convex parts 32-1 to 32-3 that are provided at both sidewall parts of storage chamber 5 and are set detachable.
Particularly, center shelf board 30-2 can be placed at different heights, since multiple sidewall convex parts 32-2 are provided.
In Figs. 7, 8A and 8B, reference numerals 33-1 to 33-3 denote door pockets that are provided at different heights on the inner surface part of door 10. Door pockets 33-1 to 33-3 are also formed by a transparent material.
Reference numerals 34-1 and 34-2 denote in-refrigerator illumination lamps, and these in-refrigerator illumination lamps are positioned on the sidewall and upper wall of storage chamber 5 in front of front ends of shelf boards 30-1 to 30-4. In-refrigerator illumination lamps 34-1 and 34-2 are provided such that substrates having a plurality of LEDs are provided in a vertical direction, for example, and are covered with a transparent material for the purpose of energy saving.
Reference numerals 35-1 and 35-2 denote cameras positioned at the front side of storage chamber 5 in a similar manner to that of in-refrigerator illumination lamps 34-1 and 34-2, and these cameras could be CCD cameras and CMOS cameras, for example. Although the kind of the cameras is not limited, it is preferred that the cameras are resistant to dew formation and the like.
In the present set-up, two cameras are provided. Camera 35-1 is fitted to the lower surface of the upper door pocket 33-1 at approximately the center in a front-back direction, and camera 35-2 is fitted close to the upper part of door pocket 33-3.
Further, upper camera 35-1 is provided at a position approximately in front of top shelf board 30-1, and lower camera 35-2 is provided at a position in front of third-stage shelf board 30-3 in the drawings.
Further, as is apparent from Fig. 7, cameras 35-1 and 35-2 are provided at positions in front of the front ends of shelf boards 30-1 to 30-4, and also near the open-end-side-end part Y opposite to the axial support X of door 10 as shown in Fig. 8A, that is, at approximately the center part in the left and right direction of refrigerator 51. With this configuration, two cameras 35-1 and 35-2 can capture almost the whole region inside the refrigerator. In the case of the example of the French door, cameras 35-1 and 35-2 are provided at the door 10 side in a large area, and are set to be positioned at approximately the center part of refrigerator 51. Further, in this example, cameras 35-1 and 35-2 are positioned at the door 10 side where insulation flap 36 (see Fig. 3) is not mounted, and are set to be positioned at approximately the center part of refrigerator 51. Insulation flap 36 is fastened to the side surfaces of one of the two doors 10, and is provided to prevent leakage of cold air from storage chamber 5 through the gap between the two doors 10.
Cameras 35-1 and 35-2 may be flexibly mounted according to the configuration of the refrigerator. For example, if camera 35-3 for exclusive use in the storage container is also provided as shown in Fig. 9A, lower camera 35-2 may be vertically adjustable as shown in Fig. 9B, or lower camera 35-2 may be vertically rotatable as shown in Fig. 9C.
As can be seen in Fig. 7 and 8, the in-refrigerator illumination lamps 34-1 and 34-2 are placed in a way such that they do not directly enter the capturing sections of cameras 35-1 and 35-2.
More specifically, as shown in Fig. 8A, the capturing sections of cameras 35-1 and 35-2 are placed such that the captured illuminosity is less than half of the maximum illuminosity of the lamps 34-1 and 34-2. In the present example, the capturing sections of cameras 35-1 and 35-2 are outside of a 60 degrees cone of the lamps.
Although not shown, a light shielding unit may be provided at the sides of in-refrigerator illumination lamps 34-1 and 34-2 which face the cameras, or the light sources of the LEDs or the like that constitute in-refrigerator illumination lamps 34-1 and 34-2 may be inclined to a diagonally backward direction. With such a configuration, the light of in-refrigerator illumination lamps 34-1 and 34-2 may be prevented from directly entering the capturing sections of cameras 35-1 and 35-2. The configuration may be suitably selected according to camera setting conditions and the like. This is similarly applied to a single-door type refrigerator described next.

1-1-2. Configuration of single-door type refrigerator

Figs. 10 to 13 show examples of fitting cameras and in-refrigerator illumination lamps when door 10 is a single door. In the single-door type refrigerator, portions which are the same as those of the French door type refrigerator are attached with the same reference numerals, their descriptions are omitted, and only different portions are described.
First, an example of the single-door type refrigerator is described with reference to Fig. 10 and Figs. 11A and 11B.
In this case, the upper camera 35-1 is mounted on protruded wall part 10b-2 that is placed between door pockets 33-4, 33-5, and 33-6 . Lower camera 35-2 is placed on a door inner surface part between door pockets 33-7 and 33-8. Cameras 35-1 and 35-2 are placed at positions approximately in front of shelf board 30-1 and 30-3 respectively that are detachable but cannot be adjusted in height, in a similar manner to that of the French door type refrigerator. Further, cameras 35-1 and 35-2 are provided at positions in front of front ends of shelf boards 30-1 to 30-4 as shown in Fig. 10, and near the center portion of door 10 as can be understood from Figs. 11A and 11B, that is, at positions of approximately center parts in the left-and-right direction in refrigerator 51. These two cameras 35-1 and 35-2 are arranged to be able to capture almost the whole region inside the refrigerator.
In-refrigerator illumination lamps 34-1 and 34-2 are placed in a similar manner to that of the French door type refrigerator, that is, on the sidewalls and on the ceiling of the storage chamber in front of front ends of shelf boards 30-1 to 30-4. The capturing sections of cameras 35-1 and 35-2 are provided at positions where the light of in-refrigerator illumination lamps 34-1 and 34-2 does not directly enter the capturing sections. That is, the capturing sections of cameras 35-1 and 35-2 are placed such that the captured illuminosity is less than half of the maximum illuminosity of the lamps 34-1 and 34-2. In the present example, the capturing sections of cameras 35-1 and 35-2 are placed outside of a 60 degrees angle cone.
Figs. 12A and 12B, and Fig. 13 show another example of the single-door type refrigerator.
In this example, cameras 35-1 and 35-2 are placed near the door open-end part Y opposite to the axial support X of door 10 as shown in Figs. 12A and 13. Height positions of cameras 35-1 and 35-2 are the same as those of a single-door refrigerator. When cameras 35-1 and 35-2 are set near the door open-end part Y, capturing is performed at the time of opening door 10 by a predetermined angle, for example, 30 degrees. With this configuration, the cameras can capture a wide range including a part near the opening portion of storage chamber 5 as shown by the dotted line in Fig. 13. Further, the angle is similar to when the user were to look inside the refrigerator, giving him a similar impression. 1-1-3. Camera configuration
Fig. 14 shows an example of cameras that are used to capture the inside of the refrigerator. In capturing the inside of the refrigerator, it is most preferable to provide the cameras at upper and lower positions at a distance as described in the above examples. However, it has been confirmed as a result of a test that the whole region inside of the refrigerator can be also captured even when upper and lower cameras 35-1 and 35-2 are set close to each other as shown in Fig. 14.
Accordingly, in such a case, cameras 35-1 and 35-2 can be considered to be set as camera module 41 by integrating the cameras. By this modularization, installation can be facilitated and cost reduction becomes possible. Modularization also becomes possible by using a wider-angle lens such as a fisheye lens.

1-2. Total configuration of refrigerator system

Next, a total refrigerator system that uses the refrigerator as described above is described with reference to Fig. 15 to Fig. 30.
Fig. 15 shows the overall configuration of the refrigerator system 50 that uses the refrigerator described above. As shown in Fig. 15, this refrigerator system 50 displays the state of the food in refrigerator 51 by display terminal 15 and the like.
This refrigerator system 50 includes refrigerator 51 and display terminal 15, and further includes radio adapter (communicating unit) 53, gateway unit (relay unit) 54, router unit 55, internet 56, and server unit 57.
Refrigerator 51 is arranged in a user's house A, for example, in the kitchen room in the house A.
Radio adapter (communicating unit) 53 is configured to be electronically connected to the control unit (described below) of refrigerator 51 and to communicate with router unit 55 via gateway unit (relay unit) 54. Radio adapter (communicating unit) 53 receives a signal transmitted from router unit 55, and outputs the received signal to the control unit of refrigerator 51. Refrigerator 51 is configured to perform a corresponding operation based on the received signal.
Radio adapter 53 is configured to obtain identification information (for example, a manufacturing number and a model number) of refrigerator 51 and in-refrigerator image data described later, from a storage unit of refrigerator 51, and transmit the received information and data to router unit 55 via gateway unit (relay unit) 54. Radio adapter 53 may be provided integrally with refrigerator 51, or may be detachably attached to refrigerator 51.
Radio adapter 53 includes "connect" button 53a (Fig. 16). "Connect" button 53a is used to newly connect radio adapter 53 to gateway unit (relay unit) 54. Radio adapter 53 is configured to obtain identification information (for example, a manufacturing number and a model number) of refrigerator 51 from the storage unit of refrigerator 51 when "connect" button 53a is operated by the user.
Display terminal 15 is a general-purpose portable terminal such as a smartphone and a tablet PC, for example, as described above. In the present configuration, display terminal 15 can be detachably attached to door 10 of refrigerator 51 as described above, and has the ability to connect to internet 56, and a communicate with gateway unit 54 as described later.
Display terminal 15 (for example, a smartphone) is configured to be connected to internet 56 via a telephone line network (for example, a 3G line network) and /or public radio LAN (by Wi-Fi communication, for example).
By connecting display terminal 15 to internet 56, display terminal 15 can obtain a program for obtaining image data of the food in the refrigerator that becomes the subject-matter of the present invention, and data (for example, the number of times of opening and closing the door) for performing the energy-saving operation, from the homepage of the manufacturer of refrigerator 51. By installing the obtained program in display terminal 15 and by starting the installed program, display terminal 15 can obtain image data and the like of the food in the refrigerator. That is, display terminal 15 can generate an operation signal for obtaining the image data of refrigerator 51 and can transmit the signal, by operating the operation screen of display 16a. This is described in detail later.
Display terminal 15 is also configured to be connected to router unit 55 by general-purpose communication (not via internet 56) such as by Wi-Fi communication, Bluetooth (registered trademark) communication, and infrared communication, and is configured to be connected for communication to gateway unit 54 via router unit 55. A device for this purpose (for example, a Wi-Fi antenna) is built in display terminal 15.
Gateway unit (relay unit) 54 is a device that relays the communication between radio adapter 53 and display terminal 15. For example, gateway unit 54 is configured to communicate with radio adapter 53, by using a signal of a specific low power radio special compact frequency band (924.0 MHz to 928.0 MHz). A frequency band for communication between gateway unit 54 and radio adapter 53 is preferably a low frequency band that reaches a far distance. Gateway unit 54 is set in the user's house A in which refrigerator 51 is set together with router unit 55.
Gateway unit 54 is also configured to be connected for communication with display terminal 15 via router unit 55. That is, gateway unit 54 is configured to be able to perform communication (first communication) with display terminal 15 via only router unit 55 without going through internet 56.
Gateway unit 54 is further configured to be able to be connected to internet 56 via router unit 55. With this configuration, gateway unit 54 can communicate (second communication) with display terminal 15 and can also communicate with server unit 57, via internet 56 and router unit 55.
Therefore, gateway unit 54 is configured to be able to perform communication with display terminal 15, both communication without internet 56 (that is, first communication via only router unit 55) and communication via internet 56 (that is, second communication via internet 56 and router unit 55). A reason for the configuration is described later.
Server unit 57 is provided by the manufacturer of refrigerator 51, for example. Server unit 57 manages information and the like necessary for access and authentication, concerning display terminal 15, refrigerator 51, and radio adapter 53. For example, server unit 57 authenticates whether the identification information of display terminal 15 that is contained in a communication request signal matches the identification information of display terminal 15 that is stored and registered in relation to radio adapter 53. When these pieces of identification information match each other, server unit 57 authenticates that accessed display terminal 15 is a qualified display terminal, and permits communication with refrigerator 51 positioned in the house A.
Server unit 57 stores the image data of the food in the refrigerator transmitted from refrigerator 51 and the data for performing the energy-saving operation described above. This data is transmitted to display terminal 15 based on a data-call signal from display terminal 15. That is, the user can confirm the image data of the food in the refrigerator by displaying the data in the display, by operating display terminal 15.

1-3. Configuration of system part of refrigerator

Fig. 16 shows a configuration of a system part of the refrigerator that is used in the refrigeration system.
As shown in Fig. 16, this refrigerator 51 includes interface (I/F) 58, controller (control unit) 59, storage unit 60, door open-close detecting unit 61, in-refrigerator illumination lamps 34-1 and 34-2, cameras 35-1 and 35-2, fan drive circuit 62, compressor drive circuit 63, blast fan 18, and compressor 19. Refrigerator 51 receives power supply from AC power source 64 such as a commercial power source in the house A, for example.
Interface (I/F) 58 handles exchange of data and the like between radio adapter 53 and controller 59 of refrigerator 51.
Controller (control unit) 59 controls the operation of the refrigerator. Controller 59 causes the image of the food in the refrigerator to be captured by controlling cameras 35-1 and 35-2 and in-refrigerator illumination lamps 34-1 and 34-2, temporarily stores the image data in storage unit 60, and transmits the image data to server unit 57 of internet 56. Further, in response to an image-data prior-transmission request signal from display terminal 15 via server unit 57, controller 59 instructs the refrigerator to perform transmission of the data to display terminal 15 via server unit 57 immediately after capturing the food. This is described in detail later.
Storage unit 60 rewrites and stores the image data of the food in the storage chamber that is captured with cameras 35-1 and 35-2 and the data for performing the energy-saving operation, in addition to identification signals and control programs.
In-refrigerator illumination lamps 34-1 and 34-2 are controlled to be turned on and off by controller 59 based on the signal from door open-close detecting unit 61. When door 10 is opened, in-refrigerator illumination lamps 34-1 and 34-2 are turned on, and when door 10 is closed, in-refrigerator illumination lamps 34-1 and 34-2 are turned off with a delay time of a few seconds.
Cameras 35-1 and 35-2 are controlled by controller 59 based on the signal from door open-close detecting unit 61, and capture the food in the refrigerator when door 10 is closed. This is described in detail later.
Fan drive circuit 62 receives a control signal from controller 59, and controls drive of rotation of blast fan 18.
Compressor drive circuit 63 receives a control signal from controller 59, and controls performance of compressor 19.
Blast fan 18 operates following the control of fan drive circuit 62, and generates airflow to circulate cold air.
Compressor 19 compresses a cooling medium (not shown) that is circulated in refrigerator 51, following the control of compressor drive circuit 63.

1-4. Configuration of server unit

Next, a configuration example of the server unit of the refrigerator system according to the embodiment is described with reference to Fig. 17. One example of the configuration of server unit 57 in Fig. 15 is described.
As shown in Fig. 17, this server unit 57 includes server interface I/F 65, server control unit 66, and data storage unit 67.
Server interface 65 is a communicating unit that handles data exchanges and the like between display terminal 15, router unit 55, and server unit 57 via internet 56, following the control of server control unit 66.
Server control unit 66 controls the complete operation of server unit 57. Server control unit 66 combines the image data of the food transmitted from refrigerator 51, and stores the combined image data in data storage unit 67. In response to an image-data prior-transmission request signal from display terminal 15, server control unit 66 transmits the latest image data of the image data stored in data storage unit 67 to display terminal 15, each time when the image data is transmitted from refrigerator 51, and causes display terminal 15 to rewrite and update the image data. Needless to mention, when a data call signal is transmitted from display terminal 15 in the state that there is no image-data prior-transmission request signal, server control unit 66 transmits the latest image data stored in data storage unit 67 to display terminal 15.
Data storage unit 67 stores necessary data such as a management program and an application program for executing the refrigerator system. Further, data storage unit 67 rewrites and stores the latest data of the food image data transmitted from refrigerator 51, following the control of server control unit 66. Data storage unit 67 is configured by an HDD (Hard Disc Drive) or an SSD (Solid State Drive), for example.
Server device 57 can be structured by either a server configured by a single large computer or a cloud server configured by many computers.

<2. Operations>

Next, operations of the refrigerator and the refrigeration system configured as described above are described. In the following description, the same operation is described by attaching the same step number.

2-1. Food capturing operation

2-1-1. Capturing when door is closed

Fig. 18 is a flowchart showing the operation of capturing the food in the storage chamber when door 10 of refrigerator 51 is closed.
When door 10 is opened to move food into or out of the refrigerator (Y in step 1), in-refrigerator illumination lamps 34-1 and 34-2 are turned on (step 2). When door 10 is closed (Y in step 3), illuminance of in-refrigerator illumination lamps 34-1 and 34-2 is decreased (step 4), and cameras 35-1 and 35-2 are started after lapse of a few seconds, for example, and the food in the refrigerator is captured (step 5). Thereafter, the captured image data is stored in storage unit 60 of refrigerator 51 (step 6), and in-refrigerator illumination lamps 34-1 and 34-2 are turned off (step 7).
When capturing is performed by decreasing the illuminance of in-refrigerator illumination lamps 34-1 and 34-2 at the capturing time, the food can be clearly and satisfactorily captured. That is, when the food is captured by lighting in-refrigerator illumination lamps 34-1 and 34-2, the inner surface of the refrigerator strongly reflects light to the food from a whole peripheral surface excluding the opening portion of the refrigerator because the inner surface of the refrigerator is a white color system (the light is strongly reflected to the food in high illuminance without decreasing the illuminance). Therefore, the food looks white, and it is difficult to perform clear capturing. However, when the illuminance of in-refrigerator illumination lamps 34-1 and 34-2 is decreased like the above example, the influence of the reflection light from the inner surface of the storage chamber can be substantially decreased, and it is possible to perform clear capturing.

2-1-2. Capturing when cold compartment is cooled

Fig. 19 is the flowchart showing the operation of capturing the food in the storage chamber during the cooling of the cold compartment after moving the food into or out of the refrigerator. When door 10 is opened to take the food into and out of the refrigerator (Y in step 1), in-refrigerator illumination lamps 34-1 and 34-2 are turned on (step 2). After the door 10 has been closed (Y in step 3), it is being determined whether the cold compartment is in the cooling operation state (step 8). When blast fan 18 is ON, and damper 22 (see Fig. 22) is opened, and the cold compartment is in the cooling operation (step 8), the illuminance of in-refrigerator illumination lamps 34-1 and 34-2 is decreased (step 4), cameras 35-1 and 35-2 are started, and the food in the refrigerator is captured (step 5). Thereafter, the captured image data is stored in storage unit 60 of refrigerator 51 (step 6), and in-refrigerator illumination lamps 34-1 and 34-2 are turned off (step 7).
By performing the capturing during the cooling operation of the cold compartment in this way, even when dew is formed on the surface of the food newly placed in the storage chamber due to the difference of temperatures between the outside and the inside of the storage chamber, for example, this dew formation is cancelled and disappears by the cooling operation. As a result, cameras 35-1 and 35-2 can clearly and satisfactorily capture images to be able to read characters described on the food packaging container.
In another example of the above operation (not shown in Fig.19), when door 10 is closed (Y in step 3), in-refrigerator illumination lamps 34-1 and 34-2 are once turned off. In-refrigerator illumination lamps 34-1 and 34-2 are turned on again before capturing is performed, and the illuminance is decreased. With this configuration, wasteful power consumption due to continuous lighting of in-refrigerator illumination lamps 34-1 and 34-2 can be suppressed.

2-1-3. Capturing when door is opened

Fig. 20 is a flowchart of the case where the refrigerator has a single door, showing the operation of capturing the food in the storage chamber at the time of opening door 10 by a predetermined angle.
When door 10 is opened to take the food into and out of the refrigerator (Y in step 1), in-refrigerator illumination lamps 34-1 and 34-2 are turned on (step 2). When the opening angle of the door becomes a predetermined angle, for example 30 degrees as shown in Fig. 13 (Y in step 10), the illuminance of in-refrigerator illumination lamps 34-1 and 34-2 is decreased (step 4), cameras 35-1 and 35-2 are started, and the food in the refrigerator is captured (step 5). Thereafter, the obtained image data is stored in storage unit 60 of refrigerator 51 (step 6), and when the door is closed (Y in step 11), in-refrigerator illumination lamps 34-1 and 34-2 are turned off (step 7). When door 10 is closed, the same steps 1-7 described above are processed.
By performing the capturing when door 10 is opened at a predetermined angle, the inside of the storage chamber can be captured in a wide range as described above.
In case of performing the capturing when the door is opened at a predetermined angle, the capturing is performed at least twice at the door opening time and at the door closing time. Therefore, by storing in storage unit 60 the last image data (when closing the door) the most recent state of the food in the storage chamber can be captured.
Needless to mention, after step 10 of detecting whether the open angle of door 10 reaches a predetermined angle is performed for a predetermined number of times, in-refrigerator illumination lamps 34-1 and 34-2 are turned off when door 10 is closed before reaching the predetermined angle, and this lamp-off state is maintained until door 10 is opened again.
Representative examples of the capturing operation are described above. Alternative capturing operation can be conceived such as capturing at a constant time or after lapse of a predetermined time since the last capturing, and capturing when there is a capturing instruction from display terminal 15.

2-1-4. Illumination at capturing time

As described above with reference to Figs. 18, 19, and 20, illumination of the food at the capturing time is performed such that the illuminance of in-refrigerator illumination lamps 34-1 and 34-2 is decreased after turning on in-refrigerator illumination lamps 34-1 and 34-2, and then capturing is performed. Depending on the situation, it is conceivable that illumination is performed such that the illuminance is decreased after turning on either only in-refrigerator illumination lamps 34-1 on both sides of the storage chamber or only in-refrigerator illumination lamp 34-2 on the ceiling of the storage chamber and then capturing is performed. A reason for the capturing by decreasing the illuminance is described later.
Further, another example of the illumination of the food at the capturing time can be also considered. Specifically, Fig. 21 is a flowchart showing the turning on and off of in-refrigerator illumination lamps 34-1 and 34-2 at the time of capturing with cameras 35-1 and 35-2. The operation corresponds to those indicated by dotted lines in Figs. 18, 19, and 20.
The capturing in the examples of Figs. 18, 19, and 20 is performed by decreasing the illuminance after in-refrigerator illumination lamps 34-1 and 34-2 are turned on. However, in the present example, the food is illuminated and captured in the state that one of the in-refrigerator illumination lamps 34-1 while the other is turned on. In this case, in-refrigerator illumination lamp 34-2 on the upper surface of the storage chamber may be in the turned-on state or in the turned-off state, and either one of these states may be suitably selected to optimize the illuminance in the refrigerator for the capturing.
In Fig. 21, before starting the capturing with cameras 35-1 and 35-2, both of the in-refrigerator illumination lamps 34-1 are already lighted, then, for example, the right-side illumination lamp in the refrigerator is turned off (step 12), and the food that is in the state of being illuminated by the left-side illumination lamp in the refrigerator is captured (step 13) and stored (step 14). Thereafter, the right-side illumination lamp in the refrigerator is turned on (step 15), next, the left-side illumination lamp in the refrigerator is turned off (step 16), and the food that is in the state of being illuminated by the right-side illumination lamp in the refrigerator is captured (step 17). This image data is stored (step 18), and the right-side illumination lamp in the refrigerator is turned off (step 19). The image data stored by illuminating the food from each side in this way is combined to form one image, and the combined one image is stored (step 29).
A reason for capturing the food by alternately turning on each one of in-refrigerator illumination lamps 34-1 and 34-2 and combining the images in this way is as follows. As described above in step 4 of Figs. 18 to 20, at the capturing time, the capturing is performed by decreasing the illuminance of in-refrigerator illumination lamps 34-1 and 34-2. When reduction of the illuminance is too large, the images become dark and not clear, and therefore, there is a limit to the decrease of the illuminance. Accordingly, a certain level of illuminance is maintained. Even in this illuminance, when the volume in the storage chamber is small, there is an influence of reflection light in some cases. However, when the food is captured by alternately turning on each one of the left and right lamps of in-refrigerator illumination lamps 34-1 in the manner as described in this example, the influence of the reflection light can be decreased, and the side of the food lighted by in-refrigerator illumination lamp 34-1 can be captured in proper brightness. Consequently, when the combining process of combining the left and right images of the food captured in proper brightness is performed, the whole image becomes clear and satisfactory.

2-2. Storing and combining of images

Next, storing and combining of image data obtained by each operation described above is described.

2-2-1. Storing and combining by server

Fig. 22 shows storing and combining of image data by the server unit. When the inside of the refrigerator has been captured by the operations shown in Figs. 18 to 20 including Fig. 21, refrigerator 51 is connected to internet 56 from radio adapter 53 via gateway unit 54 and router unit 55 (step 20). Refrigerator 51 transmits individual image data captured with each camera and stored, to server unit 57 connected to internet 56 (step 21). Server control unit 66 of server unit 57 performs the combining process of combining the individual image data transmitted via internet 56 (step 22). The combined image data (including the time information) is stored in data storage unit 67 of server unit 57 (step 23).
When the images are combined by server unit 57 and are stored like this example, the load of controller 59 and storage unit 60 at the refrigerator site can be decreased, and a refrigerator adapted to the internet can be provided at low cost.

2-2-2. Storing and combining at refrigerator side

Fig. 23 shows storing and combining of image data at the refrigerator site. When the inside of the refrigerator has been captured by the operations shown in Figs. 18 to 20 including Fig. 21, controller (control unit) 59 of refrigerator 51 fetches individual image data from each camera stored in storage unit 60, connects and combines the image data (step 24). Controller 59 stores the combined image data (step 25), and refrigerator 51 is connected to internet 56 from radio adapter 53 via gateway unit 54 and router unit 55 (step 20), and transmits the stored image data to server unit 57 connected to internet 56 (step 21). The combined image data transmitted via internet 56 is stored in data storage unit 67 of server unit 57 (step 23).
In this example, the load of controller 59 and storage unit 60 at the refrigerator site becomes large. However, refrigerator 51 can transmit and receive image data to and from display terminal 15 without via data storage unit 67 of server unit 57, and system operation cost can be reduced.

2-2-3. Storing and combining at display terminal side

Fig. 24 shows storing and combining of image data at the display terminal 15 site. When the inside of the refrigerator has been captured by the operations shown in Figs. 18 to 20 including Fig. 21, refrigerator 51 is connected to internet 56 from radio adapter 53 via gateway unit 54 and router unit 55 (step 20), and transmits image data to display terminal 15, not to the server (step 26). Display terminal 15 combines the image data transmitted via internet 56 (step 27). The combined image data is stored in the data storage unit of display terminal 15 (step 28).
The load of display terminal 15 also increases in the case of this example. However, because the load of controller 59 and storage unit 60 at the refrigerator site can be decreased, a refrigerator adapted to the internet can be provided at low cost.
Representative examples of transmission and combining operations of image data are described above. The storage and combining are not necessarily required to be performed at the same position such as server unit 57, for example, but may be flexibly set according to the system. For example, the image data may be combined by refrigerator 51, and the image data may be stored in server unit 57.

2-3. Ordinary transmission and reception of image data

2-3-1. Transmission and reception of image data stored in server and refrigerator

Fig. 25 is a flowchart showing transmission and reception between server unit 57 or refrigerator 51 and display terminal 15, that is, the flowchart shows transmission and reception of image data stored in server unit 57 or refrigerator 51.
First, the user presses a server connection button (not shown) displayed in the display of display terminal 15 at a shopping place or the like, and requests the reading of the image data stored in data storage unit 67 of server unit 57 (step 31). Whether the image data has been fetched from server unit 57 is detected (step 32), and display terminal 15 is connected to internet 56 if the image data has not been transferred (step 33). The image data stored in data storage unit 67 of server unit 57 or storage unit 60 of the refrigerator is fetched (step 34), and the image data is transferred to display terminal 15 of a smartphone or the like of the data calling source, via internet 56 (step 35). When the image data has been fetched from server unit 57 (Y in step 32), and the image data is output to the display of display terminal 15 when the image data has been fetched (step 36). Thereafter, whether an enlargement button (not shown) displayed in the display of display terminal 15 has been pressed is detected (step 37). When pressing of the enlargement button has been detected, the image data of the inside of the storage chamber output to the display is digitally converted, and image data that is enlarged in at least 1.1 magnification is generated. The enlarged image data is output in the display of display terminal 15 (step 36).
Alternatively thereafter, whether a ten-key button (not shown) displayed in the display has been pressed is detected (step 38). When pressing of the ten-key button has been detected, the enlarged image data of the inside of the storage chamber output to the display is output to the display as the image data at the position designated by the ten-key button (step 36). Whether the ten-key button has been further pressed is detected (step 38). When the pressing is not detected, the process ends.
The enlargement button and the ten-key button for displaying the image data at a predetermined position may be arranged as a system by which the user slides the surface of the display with a finger. By arranging this configuration, convenience of the system is improved.
As described above, according to the refrigerator system, the user can confirm the inside of refrigerator 51 at a glance even when the user is at a shopping place. The refrigerator system can assist the user from forgetting to purchase food and can also assist the user from purchasing more foodstuff than is necessary. For a married couple both of whom work at the outside of their house, the person who purchases foodstuff on his or her way back home can confirm the inside of storage chamber 5 of refrigerator 51 after the work, and can save time.
Further, by causing display terminal 15 to perform an enlarged display and a display of a predetermined position, the user can reliably confirm the image of the foodstuff, which it is usually difficult to achieve, with progressively advanced compact display terminal 15.

2-3-2. Transmission and reception of image data stored in display terminal

Fig. 26 is a flowchart showing transmission and reception of image data when image data is already stored in the display terminal 15. In this case, the image data is already transmitted to and stored in display terminal 15 as described with reference to Fig. 24. When a request for reading the image data is performed (step 31), the image data is fetched from the storage unit of display terminal 15, and is output to the display of display terminal 15 (step 36). Subsequent enlarged display and local display operations are as described with reference to Fig. 25.

2-3-3. Switched reception of image data

Images captured with cameras 35-1 and 35-2 are stored as individual images (images of an upper half or a lower half of the inside of the storage chamber) that are captured with cameras 35-1 and 35-2, and as combined images (combined images of upper and lower parts of the inside of the storage chamber) that are obtained by combining the individual images. A description is given of an example of display of these images in display terminal 15 by switching these images and transmitting and receiving these images.

Fig. 27 is a flowchart showing this operation.

First, the user presses an individual image/combined image call button (not shown) that is displayed in the display of display terminal 15 (step 41), and calls either an individual image or a combined image (step 42). Thereafter, image data of either the individual image or the combined image is fetched following the flowchart shown in Fig. 25 or Fig. 26, and is displayed in the display of display terminal 15.
By making it possible to switch between the individual image and the combined image for the image to be displayed in display terminal 15 like this example, usually, the combined image can be called and displayed to confirm at glance the state of the food in the storage chamber. When it is difficult to discriminate the state of the food because the food is at the corner in the storage chamber, for example, the displayed image can be switched to the individual image. With this configuration, the food can be reliably confirmed, and convenience of the system is improved.

2-4. Transmission and reception of image data at image-data prior-transmission request time

Fig. 28 shows a flowchart of an operation in response to a request for prior transmission of an image from display terminal 15.
First, the user operates display terminal 15, and performs a transmission operation of a request for prior transmission of an image (step 51). Server unit 57 or refrigerator 51 receives this request signal (step 52), and reads and transmits the image data (hereinafter, meaning individual image data or combined image data) stored in data storage unit 67 or storage unit 60 (step 53). Display terminal 15 detects whether the image data has been fetched from server unit 57 or refrigerator 51 (step 54). If the image data has not been transmitted, display terminal 15 is connected to internet 56 (step 55). Server unit 57 or the like fetches the image data stored in refrigerator 51 (step 56), and transmits the image data to display terminal 15 via internet 56 (step 57). When the image data has been fetched from server unit 57 (Y in step 54) - display terminal 15 waits until the image data is called (N in step 58). When the image data has been called (Y in step 58), the image data is output to the display of display terminal 15 (step 59).
With the above configuration, at the request of the image data from display terminal 15, the image data can be displayed in the display in real time simultaneously with the request for the image data so that the image can be confirmed, even when transmission of the image data from server unit 57 is late because of concentration of access to server unit 57. Therefore, the user can avoid a situation of waiting for the display of the image data at the shopping place, and can shorten the shopping time.
When sever unit 57 is set to transmit image data to display terminal 15 each time when door 10 of refrigerator 51 is closed at the request of an image prior-transmission request from display terminal 15, and also when the user having display terminal 15 leaves the transmission of the image data, the user can know the opening and closing of the door each time. Accordingly, when the door is likely to be opened and closed at many times to have a risk of increasing power consumption, the user can give a phone call to the home to caution children not to use refrigerator 51, for example. Consequently, convenience of energy saving also improves.

2-5. Switching of transmission and reception paths of image data

Fig. 29 shows a display of selection of a communication path of display terminal 15. The flow of transmission and reception of image data is as described above. It is also possible to speed up the obtaining of image data by changing the communication path.
Specifically, the user can select whether to connect display terminal 15 directly to gateway unit 54 via router unit 55, or to connect display terminal 15 to server unit 57 via internet 56.
In Fig. 29, reference numerals 70 and 71 denote selection buttons of a communication path that is displayed in the display of display terminal 15. The reference numeral 70 denotes a "direct" selection button for directly connecting display terminal 15 to refrigerator 51 via router unit 55 and gateway unit 54, and 71 denotes a "server" selection button for connecting display terminal 15 to server unit 57 or refrigerator 51 via internet 56. Reference numerals 72 and 73 denote data selection buttons of data to be called, 72 denotes image data of food, 73 denotes an energy-saving-data selection button of a number of door opening/closing times and the like, and 74 denotes a transmission button.
When "direct" selection button 70 is selected, display terminal 15 is set to "direct" communication. When the user touches transmission button 74 in the state of the "direct" communication, an image data request signal of display terminal 15 is directly transmitted from display terminal 15 to gateway unit (relay unit) 54 via router unit 55. Gateway terminal 54 transmits the received data request signal to radio adapter 53 of refrigerator 51. Upon receiving the image data request signal, radio adapter 53 transmits this signal to controller 59 of refrigerator 51. Upon receiving the image data request signal, controller 59 of refrigerator 51 reads the image data that is stored in storage unit 60, and transmits the image data to display terminal 15 via radio adapter 53 and router unit 55.
Therefore, when the user is inside of the house A, that is, in a broad sense, when the user is in a position where display terminal 15 can directly communicate with router unit 55, display terminal 15 can directly obtain image data from storage unit 60 of refrigerator 51 by communicating with gateway unit 54 via router unit 55.
Accordingly, when a delay occurs in the transmission of the image data to display terminal 15 to respond to the user operation due to congestion of lines or the like because of limitation of access to internet 56, the user can switch display terminal 15 to the "direct" communication so that the user can obtain the image data without receiving the influence of the congestion in lines of internet 56. Consequently, the user can confirm the state of the food in storage chamber 5 by displaying the image in the display of display terminal 15 in real time without waiting time.
On the other hand, when the user is away from house A, , that is, in a broad sense, when the user is at a position where display terminal 15 cannot directly communicate with router unit 55, the user selects "server" selection button 71 so that display terminal 15 can communicate with server unit 57 via internet 56 and read the image data of refrigerator 51. That is, when the user is outside of the house A, the user can obtain the image data of the refrigerator and know the stored state of the food in display terminal 15 even at a distant place, when the user sets the "server" communication by selecting "server" selection button 71.
As described above, in the refrigerator system according to the present embodiment, the user can select either one of the "direct" communication and the "server" communication. When the user is inside the house A, the user can look at the image in a comfortable way without being influenced by congestion of internet lines, by selecting the "direct" communication.
With reference to a detailed configuration example, the refrigerator and the refrigerator system of the present invention are described above. However, the present invention is not limited to the above configuration.
For example, in the above set-up, gateway unit 54 is present in the communication path between display terminal 15 and radio adapter 53 of refrigerator 51 and internet 56. However, this gateway unit 54 is not necessarily required, and it may be configured such that display terminal 15 directly communicates with radio adapter 53 of refrigerator 51 and internet 56 via router unit 55. Accordingly, a system configuration can be simplified. Further, in the above set-up, when the user is at th outside of the house A, the image data is obtained from server unit 57. Alternatively, it may be configured such that display terminal 15 communicates with internet 56, router unit 55, and gateway unit 54, without via server unit 57, and directly fetches from refrigerator 51 the image data stored in storage unit 60 of refrigerator 51. In this case, and when performing the "direct" communication, preferably, storage unit 60 of refrigerator 51 combines images, and transmits the image data after the combining process.
Further, in the present configuration, "direct" selection button 70 and "server" selection button 71 are displayed in the display of display terminal 15, and the user selects either the "direct" communication or the "server" communication. Alternatively, display terminal 15 may have a function of automatically switching communication to the "direct" communication when display terminal 15 catches the wave of router unit 55 so that the "direct" communication is automatically selected. Consequently, convenience of handling can be improved.

<3. Work effects>

The refrigerator and the refrigerator system according to the set-up of the present invention achieve various work effects as described with reference to a detailed configuration. According to the installation configuration of the cameras and the in-refrigerator illumination lamps as the subject-matter of the present invention, it is possible to provide at low cost a refrigerator or a refrigerator system capable of satisfactorily capturing the food in the storage chamber and capable of easily discriminating the food.
That is, in the refrigerator according to the present configuration, cameras 35-1 and 35-2 capture the food in storage chamber 5 inform the front of storage chamber 5. In this case, in-refrigerator illumination lamps 34-1 and 34-2 are also installed in the front of storage chamber 5, and illuminate the food in storage chamber 5 in the same direction of capturing with cameras 35-1 and 35-2. Therefore, images captured with the cameras are clear without darkness or difficulty of looking at, unlike if in-refrigerator illumination lamps 34-1 and 34-2 were positioned on the back wall of storage chamber 5. Further, it is not necessary to ask server unit 57 to perform image processing to make bright and clear images.
Further, because cameras 35-1 and 35-2 and in-refrigerator illumination lamps 34-1 and 34-2 are positioned in the front of the front ends of shelf boards 30-1 and 30-4, the images in storage chamber 5 captured with cameras 35-1 and 35-2 resemble the view of the user actually looking in by opening door 10 of refrigerator 51.
Further, because cameras 35-1 and 35-2 are positioned in the center (regarding left-right) of door 10 of refrigerator 51, the user will recveive images captured with cameras 35-1 and 35-2, which resemble the state that the user actually gets by looking through opened door 10 of refrigerator 51. Further, a wide range of the inside of storage chamber 5 can be captured with a large distance between cameras 35-1 and 35-2 and the food on the shelf boards. Therefore, the user can confirm in display terminal 15 the state of the food stored in a wide range. Further, the most difficult region to capture with cameras 35-1 and 35-2 is taken up by compressor 19 in this example. Therefore the top back part can also be reliably captured. While improving the storage volume at a lower part of the body by providing compressor 19 at an upper part of the body, the top back part can be satisfactorily captured with cameras 35-1 and 35-2.
Further, because cameras 35-1 and 35-2 are positioned at approximately a center portion of each of door pockets 33-1 and 33-2 on the door inner surface, the food stored in door pockets 33-1 and 33-2 as well as the food in storage chamber 5, such as a milk pack, for example, can be also partly captured with the cameras. Because the user has stored the food by himself or herself, the user can recognize the food when it is partly captured. Accordingly, almost all of the food in the storage chamber can be captured.
Further, when camera 35-3 for exclusive use in the storage container is used as shown in Fig. 9A, the camera can clearly capture the food in storage container 31. Further, when camera 35-2 is set vertically moveable as shown in Fig. 9B, or when this camera is set vertically rotatable as shown in Fig. 9C, the food in storage container 31 can be efficiently captured with two cameras instead of three.
On the other hand, because in-refrigerator illumination lamps 34-1 and 34-2 and 35-1 and 35-2 have been placed so that the light of in-refrigerator illumination lamps 34-1 and 34-2 does not directly enter the capturing sections of cameras 35-1 and 35-2 (as is clear from Figs. 7 and 10), cameras 35-1 and 35-2 are prevented from taking poor images due to the entrance of the light of in-refrigerator illumination lamps 34-1 and 34-2 into the cameras. Therefore, always satisfactory images can be obtained. A similar effect can be also obtained by employing such a configuration as provision of a light shielding unit at a camera side of in-refrigerator illumination lamps 34-1 and 34-2 or provision of the light sources of the LEDs constituting in-refrigerator illumination lamps 34-1 and 34-2 arranged by inclination to face away from the cameras.
The illuminance of in-refrigerator illumination lamps 34-1 and 34-2 in the storage chamber is between 150 lux and 200 lux. When capturing is performed in this range of lux, the food looks white and it becomes difficult to discriminate the food as described in the above capturing operation with reference to Fig. 18. Therefore, in the present configuration, capturing is performed by decreasing the illuminance. With this configuration, a CCD camera and a CMOS camera can capture clear images in proper brightness. The illuminance of the light of in-refrigerator illumination lamps 34-1 and 34-2 can be controlled by server control unit 66 such that the illuminance becomes lower than during normal time, only at the time of capturing by opening or closing door 10. By performing this control, usual illuminance is maintained to keep the inside of the refrigerator easy to look at during the normal time when capturing is not performed, and it is possible to perform clear capturing. The illuminance can be decreased by turning off either one of in-refrigerator illumination lamps 34-1 and 34-2, or by turning off a part of both in-refrigerator illumination lamps 34-1 and 34-2, or by decreasing the illuminance of both in-refrigerator illumination lamps 34-1 and 34-2, for example. A method can be suitably selected.
The images of the inside of storage chamber 5 captured with cameras 35-1 and 35-2 are transmitted to server unit 57, and are combined by server unit 57, or refrigerator 51, or display terminal 15. In the present configuration, the combining process of the images is facilitated.
Because cameras 35-1 and 35-2 are placed at the door inner surface corresponding to each front end of shelf boards 30-1 to 30-4, that is, on the door inner surface part corresponding to front ends of upper shelf board 30-1 and third shelf board 30-3 in this example, cameras 35-1 and 35-2 capture around shelf boards 30-1 and 30-3 with a line Z for a border line in Fig. 30.
Therefore, the image data of the inside of storage chamber 5 captured with two cameras 35-1 and 35-2 are combined together at a portion of shelf board 30-2 that becomes the border line Z. That is, shelf board 30-2 is commonly included in upper and lower images as shown in "captured raw images" in Fig. 30. By correcting curves of the "captured raw images", the "captured raw images" are set to "curve-corrected images". By matching common shelf board 30-2 that is included in upper and lower images, a captured image of the whole inside of the storage chamber is obtained. As a result, it becomes unnecessary to perform an extremely hard correction process of combining the images by positioning each food placed on each shelf board at the image combining time. Consequently, the image combining process becomes easy. Accordingly, a processing program for the image combining also becomes simple. Capacity of a control unit of server unit 57, or refrigerator 51, or display terminal 15 for the image combining process is not required to be large, and this contributes to increasing the processing speed.
Further, two cameras 35-1 and 35-2 are provided at positions approximately in front of shelf board 30-1 and shelf board 30-3 of which height cannot be adjusted. Therefore, even when the height of height-adjustable shelf board 30-2 is changed, images can be easily combined by adjusting the range of combining the images according to the height of shelf board 30-2 at the time of combining the images.
The present invention can provide at low cost a refrigerator and a refrigeration system capable of satisfactorily capturing the food in the storage chamber and capable of easily discriminating the food. The present invention can be applied to not only a refrigerator at home but also for business. For example, by applying the present invention to a refrigerator (including a chest freezer in the present invention) at each chain store of a convenience store, the headquarter of the convenience store can automatically deliver the food by confirming the state of the food in the refrigerator in each convenience store. The material department of a company can automatically deliver the food by confirming the food and foodstuff or other materials and chemicals in the refrigerator set in each facility such as a plant and a research laboratory.

Claims

A refrigerator comprising:
communication means connectable to the internet; and

a camera for capturing food in a storage chamber,

wherein

the communication means transmits image data captured with the cameras to a server connected to the internet,

the image data is visually confirmed through a display terminal connected to the server,

the storage chamber has an in-refrigerator illumination lamp that illuminates an inside of the storage chamber when captured with the camera, and

the camera and the in-refrigerator illumination lamp are placed at front opening sides of the storage chamber.
The refrigerator according to claim 1, wherein
the storage chamber includes a plurality of shelf boards, and
the camera and the in-refrigerator illumination lamp are positioned in front of front ends of the shelf boards.
The refrigerator according to claim 1 or 2, wherein
the camera is provided at an interior of a door, and
the in-refrigerator illumination lamp is placed at both sidewalls of the storage chamber.
The refrigerator according to claim 3, wherein
the camera is provided at approximately a center position in a front-and-back direction of a door pocket provided on the interior of the door.
The refrigerator according to any one of claims 1 to 4, wherein the in-refrigerator illumination lamp is placed where light of the in-refrigerator illumination lamp does not directly enter the camera, or is configured such that light does not directly enter the camera.
A refrigerator system comprising:
the refrigerator according to any one of claims 1 to 5,

a display terminal that is configured to request an in-refrigerator image of the refrigerator, and

a server that stores image information transmitted from communication means of the refrigerator via the internet, and that is configured to receive an image request signal transmitted from the display terminal via the internet and transmits the image information to the display terminal.
The refrigerator system according to claim 6,
wherein the server has a function of transmitting to the display terminal in advance image data representing the inside of the refrigerator in response to a request for prior transmission of the image data from the display terminal.