WO2007141095A1

WO2007141095A1 - Method and apparatus for repelling spurious multimodal messages

Info

Publication number: WO2007141095A1
Application number: PCT/EP2007/054247
Authority: WO
Inventors: Martina Kauffmann; Klaus Lukas
Original assignee: Nokia Siemens Networks Gmbh & Co. Kg
Priority date: 2006-06-09
Filing date: 2007-05-02
Publication date: 2007-12-13
Also published as: DE102006026923A1

Abstract

The invention provides a method and an apparatus for repelling spurious multimodal messages, particularly multimodal spam messages. In this case, the user or a service provider sets message categories which are to be blocked. The modalities or information transmission formats of incoming or outgoing multimodal messages are identified by a modality identification unit and then filter features which are associated with the set message categories and the identified modalities are read from a memory. An incoming multimodal message or a multimodal message which is to be sent is subsequently filtered if the multimodal message has at least one of the filter features which have been read.

Description

description

Method and device for warding off disturbing multimodal messages

The invention relates to a method and a device for warding off disturbing multimodal messages, in particular for the defense against so-called spam messages, which are transmitted by means of multimodal services.

Spam messages, also known as junk mail, are increasingly being sent over the Internet to distribute unsolicited advertising via email, news and mailing lists. To block spam messages are spam filter loading known who voted for example, the content of a message to be ^¬ search terms or keywords that are specified in a blacklist, and weed out those messages that contain such terms or keywords , This blacklist or keyword lists are created manually in general and are therefore expensive to exercise ver ^¬. In addition, such spam filters are easy to circumvent. Therefore, self-learning Bayesian spam filters are also used. The defense against spam messages is handled so far service-specific. For e-mail messages, filtering is done, for example, by searching for some customizable keywords. In the field of Internet telephony (Voice over IP), the defense of so-called SPIT messages (Spam over Internet Telefony) takes place, ie the defense against unwanted calls by monitoring the voice content or a cached voice signal. The speech signal is thereby identifier for example by means Spracher ^¬ converted to text and searches then after ten Schlüsselwor ^¬.

While in the past messages transmitted over data networks were generally unimodal, ie transmitted in a single information transmission format such as text, image, audio, video, multimodal data are increasingly being used. transmit information that summarizes information in different transmission formats in a message. Examples of multimodal messages are MMS, a website posted on the Internet and the transmission of an email with a graphic attachment.

Figure 1 shows an example of a multimodal message in which information in the modalities text, image and audio are transmitted. The multimodal message, as shown in FIG. 1, is an MMS in which, for example, a greeting in text form is transmitted together with an image of the person who is greeting, together with an audio clip. Illustrated in Figure 1 multimodal message thus comprises three Modalitä ^¬ on th, namely text, image and audio. One modality is an information transmission format which can be perceived by a human sensory organ and which, in addition to text, image, audio and video, also includes, for example, haptic information transmission ^formats .

A user can select the most optimal modality for him when interacting with his terminal. For example, to obtain information in a noisy environment, such as a factory floor, a user may have an optical display, i. H. Select the modality "image." If the user is currently sitting at the wheel of his car, the user can

Receive messages acoustically via loudspeaker, the user that selects the modality "Audio". A multimodal Interak ^¬ tion with a terminal allows the user simultaneous input from input devices, the different modalities support, such as microphone, stylus or keyboard. In addition, permits multimodal interaction the output of ^information as synthesized speech, graphics, text or haptic feedbacks.

As multimodal messages increase, the disruption of users through multimodal spam messages also increases. It is therefore the object of the present invention, a method and apparatus that allow ward off multimodal spam messages modalitätsübergreifend ^¬ ren.

This object is achieved by a method having the features specified in claim 1.

The invention provides a method for preventing disturbing multimodal messages with the following steps, namely setting of message categories to be blocked, recognition of modalities contained in a multimodal message,

Reading all the filter features associated with the detected modalities and the set message categories, and filtering the multimodal message if the multimodal message has at least one of the read filter features.

The filter features are preferably read from a memory or a database.

In one embodiment of the method according to the invention, the filtering of the multimodal message takes place by blocking or blocking the entire multimodal message.

In an alternative embodiment of the method according to the invention, the filtering of the multimodal message takes place by filtering out message contents which have a read-out filter feature from the multimodal message, while the remaining message contents pass unfiltered.

In a preferred embodiment of the method according to the invention, the modality has a text, audio, image or video information transmission format. In a preferred embodiment of the method according to the invention the multimodal message is transmitted over a data network ^¬ factory.

The data network is, for example, the Internet.

In a further embodiment of the method according to the invention, the multimodal message is transmitted via at least one Internet link of an Internet website.

In this case, the filtering of the multimodal messages is carried out in one embodiment of the method according to the invention by blocking the Internet links.

In one embodiment of the method according to the invention, the setting of the message ^categories to be blocked takes place by a user.

In an alternative embodiment of the method according to the invention, the setting of the message categories to be blocked is carried out by a service provider.

The invention further provides a device for preventing interfering multimodal messages with a recognition unit which recognizes modalities contained in the multimodal messages and reads out filter characteristics associated with the recognized modalities and adjustable message categories, and with a filter unit for filtering those multimodal messages which have at least one of the read filter characteristics.

In the following, preferred embodiments of the method according to the invention as well as the device according to the invention will be described with reference to the attached figures to explain features essential to the invention. Show it :

FIG. 1 shows an example of a multimodal message;

FIG. 2 shows a flow diagram of a possible embodiment of the method according to the invention;

Figure 3 is an abstract representation of a multimodal message comprising several modalities;

FIG. 4 shows an example of a memory-stored table of filter characteristics used in the method according to the invention for filtering multimodal messages;

FIG. 5 shows an example for explaining the setting of message categories to be blocked according to an embodiment of the method according to the invention;

Figure 6: a table of filter features in abstract

Form used in one embodiment of the method of filtering multimodal messages according to the invention;

Figures 7A, 7B are diagrams for explaining the application of the inventive method for the defense against annoying multimodal messages transmitted via linked websites of the Internet;

FIG. 8 shows an example of an interactive multimodal ^terminal in which the method ^according to the invention can be used to ward off disturbing multimodal messages;

FIG. 9 shows a block diagram of a possible embodiment of the device according to the invention for filtering multimodal messages. FIG. 2 shows a simple flow diagram of a possible embodiment of the method according to the invention for warding off disturbing multimodal messages.

After a starting step SO ver ^¬ first be set to different blocking message categories K in step Sl. The setting of the messages takes place in one embodiment by a user at his terminal. In an alternative embodiment, the setting of the message categories K to be blocked is carried out by a service provider. A message category can be formed on any topic, such as lottery, football, investment, car, etc. The categories are, for example, hierarchically ordered. Figure 5 shows an example of the hierarchical order of different message categories. The news category "Sports" in this case has the Unterkatego ^¬ rien "football", "Handball", "Sumo", "Chess" in. A user can set under menu guidance, for example me- to lock News Categories K by example clicks to lock categories. Conversely can not be selected categories to be blocked by the user, so that the remaining categories are set as the Nachrichtenka ^¬ categories to be blocked.

The setting can, for example, by entering an adjustment command by the user is done, "hear ge ^¬ except messages to category" example ^¬ as "Lock all messages belonging to the category" Sport Chess "belonging ren".

After setting the message categories to be blocked K by the user or the service provider he ^¬ follows in step S2, the recognition of the decision in a message preserved modalities M. The M modalities include, for example a text, audio, image or video information transfer format. In addition, Mo ^¬ are modalities in for haptic or vibrating or Braille provided further embodiments of the method according to the invention.

Figure 3 shows a multi-modal message schematic having various information contents, which are transmitted in _k ver ^¬ different modalities M _1, M ,, M. The multimodal message may be, for example, an MMS containing a text, an image, or an audio clip, or an email with a graphics attachment, or a web site that includes speech, graphics, and text.

After all the modalities contained in the received multimodal message are recognized, in one embodiment of the inventive method, filter characteristics are read from a memory or a database, which are assigned to the recognized modalities M and the set message categories K to be blocked. The readout of the filter characteristics takes place in step S3.

Figure 4 shows an example of the memory contents of the ^¬-like filter characteristic memory for the news category "gambling." Want for example, a user who Nachrich ^¬ tenkategorie "gambling" lock, that he would not hold multi-modal message to that item type ER, he locks the News Category Each modality, for example text, image, audio has different filter characteristics In the example shown in FIG. 4, the keywords "Lotto" and "are used as filter characteristics or keywords for the" text "mode. Profit "filed in the filter table. For example, hash values x, y are stored for the next modality "image" for corresponding images that represent lotto balls, for example, for the next modality "audio" for the category "gambling" keywords such as "lotto" or "super" in Filing feature memory stored.

Contains a multimodal message at least one of Fil ^¬ termerkmale to be oppressive category K, it is in Step S4 filtered. The filtering of the multimodal message in one embodiment is done by disabling or discarding the entire message. In an alternative embodiment, the filtering of the multimodal demand occurs Judges in step S4, by only those message content having a readout filter characteristic, are removed from the multi-modal message ^¬ and the other message contents are forwarded. Contains in the dargestell ^¬ th example, the received multimodal message in text content, for example, the keyword "lottery" or "profit", the received message is either completely blocked in step S4 ^¬ constantly or it is only the text content ge ^¬ locked. In the same way, a multimodal message is ^filtered if an image contained in the message has the hash value x or the hash value y. In an alternative embodiment, fingerprints (OCR), watermarks or the like are also used as graphics-related spam features. If in the example shown in FIG. 4 an audio clip contains the speech signal "Lotto" or the speech signal "Super", this multimodal message is likewise ^filtered in step S4. The method according to the invention ends in step S5.

The method illustrated in FIG. 2 is performed in one embodiment of a user's terminal. In an alternative embodiment, the method according to the invention for warding off disturbing multimodal messages is carried out in a network node to which the message passes during the transmission over a data network. In this case, the filtering takes place, for example, by a service provider or network operator. The setting of the message categories K to be blocked can also be done either by the user or the service provider.

On the one hand, the filtering of the multimodal message can be carried out for messages received by a device or, on the other hand, for the purpose of increasing security, also for filtering messages sent by a device. For example, at a Sending mails via multimodal web interfaces Spam content is detected and appropriate filtering is carried out.

FIG. 6 shows the table ^{contents of} a filter ^feature memory in abstract form. The table comprises Z columns for Z message categories K and N rows for N different modalities M. In each field of the table different Fil ^¬ termerkmale are stored. For example, two filter features Fl, F2 are stored for the message category p and the modality i. For the message category q and the modality j, j three filter features are stored in the illustrated example. Contains a multi-modal message an After ^¬ set content in the modality j and is the category q is set as to lock message category, the mul ^¬ is filtered timodale message when the j in the modality ge ^¬ passed message content of the three Darge in Figure 6 ^¬ presented Filter features Fl, F2, F3 has.

The inventive method can also be used to defend against multimodal messages that are transmitted via linked websites. FIG. 7A graphically shows the linkage of a website B with a website A. Both websites A, B have a multimodal message content. When a user clicks on the website B's website A view, the multimodal message content is transmitted from the site B server B via the site A server A and the Internet to the user. The filtering of the multimodal information or the multimodal message takes place in a first embodiment of the terminal of the user or in a second embodiment in addition to the server A. The service providers can ensure in this way by means of the inventive method that linked content is free of spam, and thus their Protect customers from being redirected to unwanted interfering content. About the categorization of Merkmalsidentifikatoren or the filter characteristics capitalization, for example, young people of all modalities are out ^¬ protected away from certain content. For example Categorizations regarding protection of minors, product lines or paid numbers can be applied and customized. If suspicious contents are found, for example on further-linked multimodal websites, the forwarding links can be blocked by the provider.

As a description language for the notation of the filter or spam features, for example, EMA (Extensible Multimodal Anotation) can be used, in which various Modalitä ^¬ th will be described.

Figure 8 shows an example of a mobile terminal having a multimodal interaction with the user. In the illustrated example in Figure 8 a multimodal Filterda ^¬ tenbank is provided to the operator of the network. The service provider or service provider also performs a spam check of linked services. At the terminal received multimodal messages are filtered out by means of the inventive method using a local database.

Figure 9 shows a possible embodiment of a device 1 according to Inventive ^¬ be blocked or filtered when th interfering multimodal Nachrich-. The device 1 transmits or receives multimodal messages via a communication channel 2, which is hardwired or formed by a radio interface.

A modality recognition unit 3 recognizes the modalities M contained in the multi ^¬ modal messages and outputs them to a filter feature memory 4. In the filter feature memory 4, one or more message categories K to be blocked are set, for example via an interface device. From the feature memory 4 those filter features are read out, which are assigned to the detected modalities M and the set message categories K. These filter features F ₁ are sent to a filter unit 5 of Device 1 delivered. The filter unit 5 filters those multimodal messages which have at least one of the read-out filter features F ₁ . The filtering is carried out at ^¬ example by locking the to be sent or received multi-modal message. The unfiltered messages are processed by any data processing unit 6.

With the device according to the invention spam messages are repudiated across modalities and there can be a consistent shielding. With the method according to the invention or the device 1 according to the invention, as shown in FIG. 9, spam features of various services can be assigned a modality-related classification via a common classification, so that multimodal spam messages can be combated across services. The user is equally protected against spam attacks, regardless of the modality used. An advantage for the operator is that spam characteristics of individual services can also be used for the same modalities in other services. It can therefore consistent, easy to manage, since ^¬ tenbasis provide. Service providers can also ensure that linked content is spam-free and thereby protect ih ^¬ re customers such before forwarding content. The method according to the invention is also suitable for use in various mobile environments.

Claims

claims

A method for preventing annoying multimodal messages, comprising the steps of:

(a) setting (Sl) of message categories to be blocked

(K);

(b) detecting (S2) modalities (M) contained in a multimodal message;

(c) reading out (S3) all filter features associated with the detected modalities (M) and the set message categories (K); and

(d) filtering (S4) when the multi-modal message comprises the multimodal message of at least one of the read Fil ^¬ termerkmale.

The method of claim 1, wherein the filtering (S4) of the multimodal message is performed by disabling the multimodal message.

The method of claim 1, wherein the filtering (S4) of the multimodal message is performed by filtering out message contents having a read filter feature from the multimodal message.

The method of claim 1, wherein the modality (M) comprises a text, audio, image, video information transmission format.

5. The method of claim 1, wherein the multi-modal message is discharged via a data network via ^¬.

6. The method of claim 5, wherein the data network is formed by the Internet.

7. The method of claim 6, wherein the multimodal message is transmitted via at least one Internet link of an Internet website.

8. The method of claim 7, wherein the filtering (S4) of the multimodal message is performed by blocking the Internet link.

9. The method according to claim 1, wherein the setting (Sl) of the message categories (K) to be blocked is carried out by a user.

10. The method of claim 1, wherein setting (Sl) is carried to the blocking messages Kate ^¬ Categories list (K) by a service provider.

11. Device for warding off disturbing multimodal messages with:

(A) a recognition unit (3), the modalities (M) contained in each of the multimodal messages recognizes and reads out filter features that are associated with the detected Modalitä ^¬ th (M) and adjustable message categories (K), and with

(b) a filter unit (5) for filtering those multimoda ^¬ len messages that have at least one of the read-out filter characteristics.