WO2001073603A1

WO2001073603A1 - Information retrieval method, information retrieval server and personal digital assistant

Info

Publication number: WO2001073603A1
Application number: PCT/JP2001/002582
Authority: WO
Inventors: Tomonori Fujisawa; Susumu Hibi
Original assignee: Kabushiki Kaisha Eighting
Priority date: 2000-03-29
Filing date: 2001-03-28
Publication date: 2001-10-04
Also published as: JP2001282825A; AU4459501A

Abstract

An information retrieval method using image recognition is provided that is suitable for personal or home use. When a user transmits image data produced using a camera (31) to a mail server (12) through a personal digital assistant (30), a reception folder (13a) of the mail server (12) stores electronic mail with an appended search image. A search server (10) separates the image data from the electronic mail to form an image key. The image key is compared with the standard pattern stored in a recording medium (11b) of the search server (10), and database registration information stored in a recording medium (11a) associated with the standard pattern is extracted and send to the personal digital assistant (30).

Description

Description Information search method, information search server and mobile terminal

The present invention relates to an information search method using a portable terminal, and more particularly to an information search method that allows a search server to perform an information search based on image recognition and obtain the search result from a network using the mobile terminal. Things. Background art

Conventionally, characters and symbols have been used as search keys for searching electronic databases, but in recent years images have been used as search keys. Such image databases based on image recognition have achieved great results in various industries, logistics and medical fields. It is also becoming popular in fields such as personal authentication, crime prevention, and maps.

However, most of these information searches are performed by large-scale systems or expensive systems, and are not configured as personal or home-friendly systems.

For example, when acquiring or identifying specialty items such as sake and wine, if there is a spot, use character information as a search key or directly from the image information of the spot, rather than using a search code. It is very convenient because you can get information about the wine.

In particular, wine is produced all over the world, and the etiquette (label) affixed to bottles is not only written in the language of the country of origin, but also written according to the laws and regulations of each country or region, It is a well-known fact that there are many things with the same name but different contents, and that the evaluation differs depending on the age. Therefore, in-depth knowledge is required to distinguish wines, and it is difficult to obtain accurate information simply by using the characters displayed on etiquette as search keys. Disclosure of the invention

An object of the present invention is to easily and easily obtain necessary database information by using the Internet and mobile terminals typified by mobile phones. It is intended to provide an information retrieval method by image recognition suitable for home use.

According to the information search method of the present invention, image data as a search key is transmitted from a mobile terminal to a search server. The database registration information associated therewith is transmitted to the portable terminal.

The information search server of the present invention uses image data obtained from a mobile terminal as an input pattern, and performs image recognition for comparison with a standard pattern stored therein to obtain database registration information previously associated with the standard pattern. And transmitting the registration information to the portable terminal.

It is preferable that the information search server has a recording medium for storing the database registration information and the standard pattern. Preferably, the image data transmitted from the portable terminal is obtained via a mail server.

A portable terminal according to the present invention includes an electronic camera, and includes means for transmitting image data captured by the electronic camera to an information search server, and means for receiving an e-mail from the information search server. Is what you do. BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 is an overall explanatory diagram of an information search system using a mobile terminal according to the present invention.

FIG. 2 is an explanatory diagram showing a method of creating an image key as an input pattern.

FIG. 3 is a block diagram showing the configuration of the search server 10.

FIG. 4 shows an example of wine information displayed on the display screen of the mobile terminal 30. BEST MODE FOR CARRYING OUT THE INVENTION

Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is an overall explanatory diagram of an information search system using a mobile terminal according to the present invention. The upper part of the figure shows a search server 10 for searching for information by image recognition, and a mail server 12 having an electronic mail transmission / reception function between the search server 10 and the outside. On the right side of the search server 10, the recording medium 11a storing the database registration information, which is the management and control target of the search server 10 as a feature of the information search system, and the "standard pattern" in image recognition And the recording medium 1 1b in which is stored is shown.

In addition, the recording medium 13 taken out and shown on the left side of the mail server 12 contains a receiving folder 13 a for receiving the search image attached mail sent from the external network, and a standard pattern attached mail. A receiving folder 13b for receiving is provided. Note that the mail server 12 may be provided as a unit with the search server 10.

Reference numerals 30, 30, ′ denote a mobile terminal equipped with a camera 31, and a connection server 4 for converting the signals of the mobile terminals 30, 30, ■ ■ to Internet signals via a radio wave 42. Connected to one. Of course, the camera 31 does not have to be configured integrally with the portable terminal 30.

The fixed terminal 50 is a personal computer or the like used for registering a standard pattern on the recording medium 11 b of the search server 10 from a network, and includes a camera 51 as an image input device. The search server 10, mail server 12, connection server 41, and fixed terminal 50 are connected by a network 60 such as the Internet.

In the present invention, a user of the mobile terminal 30 (hereinafter referred to as an inquirer) can obtain the database registration information stored in the search server 10 via the connection server 41.

FIG. 2 is an explanatory diagram showing a method for creating an image key as an input pattern. In FIG. 2, an object 20 to be surveyed, an identification label 21 attached thereto, and a camera 31 attached thereto are shown. A portable terminal 30 and an image 23 of data transmitted as an image key (input pattern) are drawn.

First, the inquirer takes an image of the area of the dotted line 22 of the object 20 with the camera 31 and comprises a front view 20 a of the object 20 and a front view 21 a of the identification label 21. You get 23 for one night. The image data 23 is transmitted to the mail server 12 as an attached file by the transmission function of the mobile terminal 30 and stored in the receiving folder 13a for the attached mail with the search image. This image data is separated from the electronic mail by the function of the search server 110 described later, and becomes an image key. This image key is compared with the standard pattern stored in the recording medium 11b, and the database registration information stored in the recording medium 11a associated with the standard pattern is extracted. Is returned to the mobile terminal 30.

In addition, image data that becomes a standard pattern in image recognition is usually created and stored in the search server 10. However, it is auxiliaryly stored in the search server 10 via an external network 60. You may. In this case, the standard pattern attached mail created by the fixed terminal 50 equipped with the camera 51 etc. is stored in the receiving folder 13 b of the mail server 12 via the network 60, and the image data It is separated from the e-mail by the function of the search server 10 described later and stored in the recording medium 11b for storing the standard pattern. The above-mentioned external standard pattern accumulation work is particularly useful when the accumulation of the standard patterns by the search server 110 alone is limited.

FIG. 3 is a block diagram showing the configuration of the search server 110. The search server 10 includes, as components that should normally be provided for processing, input / output, and transmission / reception of various data, a control unit 320 that controls the operation of the search server 10 as a whole, and a processing unit 3 that performs data processing. 30; an input / output interface 310 connected to various input / output devices and the network 60; an input unit 350 receiving data from the input / output interface 310; An output section 360 for outputting, a storage section 340 for temporarily storing data during data processing, a receiving section 370 for receiving various data, and a transmitting section 380 for transmitting various data. And The search server 100 constantly monitors the incoming mail to the receiving folder 13a of the mail server 12 and the mailer unit 300 having an e-mail transmission / reception transfer function in addition to the above-mentioned components that should be provided normally. 1 3 a File monitoring section 3001, input data separation section 3002 for separating image attachment mail into image data and other data, and unique identification ID and mail address of mobile terminal An ID storage unit 3003 that stores registered data, an ID determination unit 3004 that determines whether the search server 110 itself can respond based on the registered data, and image data input as a search key. Input pattern correction unit 3005 that performs correction processing on the image, feature extraction unit 3006 that extracts feature parameters or feature vectors from the corrected image data, and standard patterns for image recognition. Standard putter After comparing the extracted feature amount with the standard pattern, an identification unit 30 that extracts database information associated with the selected standard pattern from the database information storage unit 301. 08, a response information generation unit 3009 that generates response information to the inquiry, a mail generation unit 3101 that generates an e-mail notifying the creation of the response information, and reception of the mail server 12 It is provided with a 13b file monitoring unit 3103 that constantly monitors incoming calls to the folder 13b, and a link information generating unit 31015.

Next, the operation of the search server 110 will be described.

The image attached mail sent from the mobile terminal 30 is received by the receiving folder 13 a of the mail server 12, and the received mail is constantly monitored for incoming calls to the receiving folder 13 a 13 a file monitoring unit Detected by the controller 301 and reported to the controller 320. Upon receiving the report, the control unit 320 instructs the processing unit 330 to send an e-mail with an image attached to it via the mail server — input / output from the reception folder 13 a of the mail server 12 using the mail server. Interface: I: Input to the input unit 350 via the source 310, and further transfer to the storage unit 340 according to the instruction of the processing unit 330.

The transferred image attached mail is divided into an e-mail address, a unique identification ID of the portable terminal, image data, and other data by the input data dividing unit 3002. Next, it is determined whether or not the search server 10 can respond based on the unique identification ID of the mobile terminal among the separated data. That is, the unique identification I Whether or not D matches the registered data stored in the ID storage section 303 beforehand is collated by the ID determination section 304 and the result is reported to the processing section 330.

When reporting response permission to the processing unit 3 3 0 is performed, the separation image data is _c subjected to correction processing by the input pattern correction unit 3 0 0 5 receives the instruction of the processing unit 3 3 0 This Before matching the input pattern with the standard pattern, it removes information unnecessary for identification as much as possible from the information contained in the image data as the input pattern, improving the recognition accuracy and processing speed in image recognition. This is a process for causing From the image data corrected by the input pattern correction unit 3005, a feature parameter or a feature vector is obtained by a feature extraction unit 3006 instructed by the processing unit 330. The feature parameters and the feature vector are data extracted from the input pattern as the properties of the pattern that are effective for identification.

The feature quantity extracted in this way is then standardized by a discriminating unit 3008 into a standard pattern accumulating unit 3007 (corresponding to the recording medium 11b of the search server 10 described above). Based on the pattern (image data), it is identified which standard pattern matches or is closest, and the database information storage unit 3102 associated with this standard pattern (recorded by the search server 10 described above) This corresponds to medium 11a.) Is reported to the processing unit 330.

Upon receiving the report, the processing section 330 instructs the response information generation section 310 to generate data for a response, and at the same time, creates an email notifying the inquirer that the response data has been created. Instructs the e-mail generation unit 301 to create an e-mail that combines the prepared message and the URL that describes the location of the generated response file, and sends it to the transmission unit 3. Transfer to 80. The transferred mail data is sent as an e-mail to the mail server 12 via the input / output interface 310, and the mail server 12 sends the e-mail to the portable terminal 30 that has referred the e-mail.

The referrer who received the above-mentioned mail by the mobile terminal 30 reads the data for response to the URL specified in the received mail, and for example, reads the data of the mobile terminal 30 shown in the left half of FIG. The display screen 401 shows the wine information 40 shown in the right half of FIG. 2 can be obtained.

In the above description, the mail server 12 transmits the URL relating to the response data as an e-mail, but it is also possible to directly return the e-mail attached with the response data in the database to the inquirer.

As described above, the image data serving as the standard pattern is created and accumulated in the search server 10 according to the usual method. The external standard pattern accumulation operation as an auxiliary operation will be described.

A person who intends to register a standard pattern in the search server 10 (hereinafter referred to as “registrant c”) enters the URL screen specified by the information search system from the screen of the fixed terminal 50 and receives the specified password. After inputting the standard pattern information to be transmitted on the instruction screen of the link information generating unit 301 after receiving the check, the file name of the standard pattern is issued.

Next, the registrant sends the image data with the issued file name to a predetermined mail address. The e-mail attached with the image is stored in the receiving folder 13 b for the e-mail attached with the standard pattern on the mail server 12. The reception of the image-attached e-mail is detected by the 13b file monitoring section 310, which constantly monitors incoming calls to the receiving folder 13b, and reported to the control section 320. Upon receiving the report, the control unit 320 instructs the processing unit 330 to input the e-mail from the receiving folder 13 b of the mail server 12 by using the mailer unit 3000 to send the image attached mail. Output The data is input to the input unit 350 via the interface 310, and further transferred to the storage unit 340 according to the instruction of the processing unit 330.

The transferred image attached mail is separated into a mail address, image data, and other data by the input data separation unit 3002. Next, it is determined whether or not the search server 10 can accept the mail address in each of the separated data. That is, whether or not the e-mail address matches the registered data previously stored in the ID storage unit 3003 is collated by the ID determination unit 304 and reported to the processing unit 330. Only when the acceptance is reported to the processing unit 330, the separated image data is stored in the standard pattern storage unit 3007 under the instruction of the processing unit 330. You.

According to the present invention, necessary database information can be obtained easily and easily by using a mobile terminal represented by a mobile phone. Therefore, in addition to specialty items such as sake and wine, various information on animals and plants, minerals, electrical equipment parts, daily necessities parts, civil engineering structures, buildings, constellations, and other roughly shaped objects can be searched. Can be widely applied. Further, according to the present invention, at any time and at any place, by accessing the search server from the mobile terminal at hand, the user can immediately obtain the desired information. It has an excellent effect that it can be done.

Claims

Scope

1. The mobile terminal transmits image data as a search key to a search server, and the search server identifies a standard pattern from the image data by image recognition, and stores database registration information previously associated with the standard pattern. An information retrieval method that involves responding to mobile terminals.

2. Image data obtained from the mobile terminal is used as an input pattern, and database registration information preliminarily associated with the standard pattern is extracted by image recognition by comparing the input pattern with the standard pattern stored therein. An information search server that responds to the mobile terminal.

3. The information retrieval server according to claim 2, further comprising a recording medium for storing the database registration information and the standard pattern.

4. The information retrieval server according to claim 2, wherein the image data transmitted from the portable terminal is obtained via a mail server.

5. A mobile terminal equipped with an electronic camera, the mobile terminal having means for transmitting an image captured by the electronic camera to an information search server and means for receiving an e-mail from the information search server. .