US8725736B2 - Computer-implemented system and method for clustering similar documents - Google Patents

Computer-implemented system and method for clustering similar documents Download PDF

Info

Publication number
US8725736B2
US8725736B2 US13/767,849 US201313767849A US8725736B2 US 8725736 B2 US8725736 B2 US 8725736B2 US 201313767849 A US201313767849 A US 201313767849A US 8725736 B2 US8725736 B2 US 8725736B2
Authority
US
United States
Prior art keywords
documents
concepts
concept
document
occurrence
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Lifetime
Application number
US13/767,849
Other versions
US20130159300A1 (en
Inventor
Dan Gallivan
Kenji Kawai
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nuix North America Inc
Original Assignee
FTI Technology LLC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by FTI Technology LLC filed Critical FTI Technology LLC
Priority to US13/767,849 priority Critical patent/US8725736B2/en
Publication of US20130159300A1 publication Critical patent/US20130159300A1/en
Priority to US14/275,802 priority patent/US9195399B2/en
Application granted granted Critical
Publication of US8725736B2 publication Critical patent/US8725736B2/en
Assigned to BANK OF AMERICA, N.A., AS ADMINISTRATIVE AGENT reassignment BANK OF AMERICA, N.A., AS ADMINISTRATIVE AGENT NOTICE OF GRANT OF SECURITY INTEREST IN PATENTS Assignors: FTI CONSULTING TECHNOLOGY LLC, FTI CONSULTING TECHNOLOGY SOFTWARE CORP, FTI CONSULTING, INC.
Priority to US14/949,829 priority patent/US9619551B2/en
Priority to US15/484,122 priority patent/US20170262455A1/en
Assigned to FTI CONSULTING TECHNOLOGY LLC reassignment FTI CONSULTING TECHNOLOGY LLC CHANGE OF NAME (SEE DOCUMENT FOR DETAILS). Assignors: FTI TECHNOLOGY LLC
Assigned to FTI CONSULTING TECHNOLOGY LLC reassignment FTI CONSULTING TECHNOLOGY LLC RELEASE OF SECURITY INTEREST IN PATENT RIGHTS AT REEL/FRAME 036031/0637 Assignors: BANK OF AMERICA, N.A., AS ADMINISTRATIVE AGENT
Assigned to NUIX NORTH AMERICA INC. reassignment NUIX NORTH AMERICA INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: FTI CONSULTING TECHNOLOGY LLC
Assigned to FTI TECHNOLOGY LLC reassignment FTI TECHNOLOGY LLC ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: ATTENEX CORPORATION
Assigned to ATTENEX CORPORATION reassignment ATTENEX CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: GALLIVAN, DAN, KAWAI, KENJI
Anticipated expiration legal-status Critical
Expired - Lifetime legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/35Clustering; Classification
    • G06F16/355Class or cluster creation or modification
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23Updating
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • G06F16/2457Query processing with adaptation to user needs
    • G06F16/24575Query processing with adaptation to user needs using context
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/28Databases characterised by their database models, e.g. relational or object models
    • G06F16/284Relational databases
    • G06F16/285Clustering or classification
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/31Indexing; Data structures therefor; Storage structures
    • G06F16/313Selection or weighting of terms for indexing
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/35Clustering; Classification
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/93Document management systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/95Retrieval from the web
    • G06F16/955Retrieval from the web using information identifiers, e.g. uniform resource locators [URL]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638Organizing or formatting or addressing of data
    • G06F3/064Management of blocks
    • G06F3/0641De-duplication techniques
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10TECHNICAL SUBJECTS COVERED BY FORMER USPC
    • Y10STECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10S707/00Data processing: database and file management or data structures
    • Y10S707/99931Database or file accessing
    • Y10S707/99932Access augmentation or optimizing
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10TECHNICAL SUBJECTS COVERED BY FORMER USPC
    • Y10STECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10S707/00Data processing: database and file management or data structures
    • Y10S707/99931Database or file accessing
    • Y10S707/99933Query processing, i.e. searching
    • Y10S707/99936Pattern matching access
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10TECHNICAL SUBJECTS COVERED BY FORMER USPC
    • Y10STECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10S707/00Data processing: database and file management or data structures
    • Y10S707/99941Database schema or data structure
    • Y10S707/99943Generating database or data structure, e.g. via user interface
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10TECHNICAL SUBJECTS COVERED BY FORMER USPC
    • Y10STECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10S707/00Data processing: database and file management or data structures
    • Y10S707/99941Database schema or data structure
    • Y10S707/99944Object-oriented database structure
    • Y10S707/99945Object-oriented database structure processing

Definitions

  • the present invention relates in general to text mining and, in particular, to a system and method for clustering similar documents.
  • Document warehousing extends data warehousing to content mining and retrieval. Document warehousing attempts to extract semantic information from collections of unstructured documents to provide conceptual information with a high degree of precision and recall. Documents in a document warehouse share several properties. First, the documents lack a common structure or shared type. Second, semantically-related documents are integrated through text mining. Third, essential document features are extracted and explicitly stored as part of the document warehouse. Finally, documents are often retrieved from multiple and disparate sources, such as over the Internet or as electronic messages.
  • Document warehouses are built in stages to deal with a wide range of information sources.
  • document sources are identified and documents are retrieved into a repository.
  • the document sources could be electronic messaging folders or Web content retrieved over the Internet.
  • the documents are pre-processed to format and regularize the information into a consistent manner.
  • text mining is performed to extract semantic content, including identifying dominant themes, extracting key features and summarizing the content.
  • metadata is compiled from the semantic context to explicate essential attributes.
  • the metadata is provided in a format amenable to normalized queries, such as database management tools.
  • Document warehousing is described in D. Sullivan, “Document Warehousing and Text Mining, Techniques for Improving Business Operations, Marketing, and Sales,” Chs. 1-3, Wiley Computer Publishing (2001), the disclosure of which is incorporated by reference.
  • Text mining is at the core of the data warehousing process. Text mining involves the compiling, organizing and analyzing of document collections to support the delivery of targeted types of information and to discover relationships between relevant facts.
  • identifying relevant content can be difficult.
  • Second, analyzing an unstructured document collection without the benefit of a priori knowledge in the form of keywords and indices can present a potentially intractable problem space.
  • synonymy and polysemy can cloud and confuse extracted content. Synonymy refers to multiple words having the same meaning and polysemy refers to a single word with multiple meanings. Fine-grained text mining must reconcile synonymy and polysemy to yield meaningful results.
  • syntactic searching provides a brute force approach to analyzing and extracting content based on literal textual attributes found in each document.
  • Syntactic searching includes keyword and proximate keyword searching as well as rule-based searching through Boolean relationships.
  • Syntactic searching relies on predefined indices of keywords and stop words to locate relevant information.
  • syntactic searching can fail to yield satisfactory results due to incomplete indices and poorly structured search criteria.
  • a more advanced prior art approach uses a vector space model to search for underlying meanings in a document collection.
  • the vector space model employs a geometric representation of documents using word vectors. Individual keywords are mapped into vectors in multi-dimensional space along axes representative of query search terms. Significant terms are assigned a relative weight and semantic content is extracted based on threshold filters.
  • the present invention provides a system and method for indexing and evaluating unstructured documents through analysis of dynamically extracted concepts.
  • a set of unstructured documents is identified and retrieved into a document warehouse repository.
  • Individual concepts are extracted from the documents and mapped as normalized data into a database. The frequencies of occurrence of each concept within each document and over all documents are determined and mapped.
  • a corpus graph is generated to display a minimized set of concepts whereby each concept references at least two documents and no document in the corpus is unreferenced.
  • a subset of documents occurring within predefined edge conditions of a median value are selected.
  • Clusters of concepts are grouped into themes. Inner products of document concept frequency occurrences and cluster concept weightings are mapped into a multi-dimensional concept space for each theme and iteratively generated until the clusters settle. The resultant data minima indicates those documents having the most pertinence to the identified concepts.
  • An embodiment provides a computer-implemented system and method for clustering similar documents.
  • Concepts are identified for a set of documents and occurrence frequencies are determined for each concept in the documents set.
  • a distance quantifying a similarity for each of the documents in the set with one or more clusters of documents is calculated.
  • Each document is mapped to at least one of the one or more document clusters.
  • the present invention semantically evaluates terms and phrases with the goal of creating meaningful themes.
  • Document frequencies and co-occurrences of terms and phrases are used to select a minimal set of highly correlated terms and phrases that reference all documents in a corpus.
  • FIG. 1 is a block diagram showing a system for dynamically evaluating latent concepts in unstructured documents, in accordance with the present invention.
  • FIG. 2 is a block diagram showing the software modules implementing the document analyzer of FIG. 1 .
  • FIG. 3 is a process flow diagram showing the stages of text analysis performed by the document analyzer of FIG. 1 .
  • FIG. 4 is a flow diagram showing a method for dynamically evaluating latent concepts in unstructured documents, in accordance with the present invention.
  • FIG. 5 is a flow diagram showing the routine for performing text analysis for use in the method of FIG. 4 .
  • FIG. 6 is a flow diagram showing the routine for creating a histogram for use in the routine of FIG. 5 .
  • FIG. 7 is a data structure diagram showing a database record for a concept stored in the database 30 of FIG. 1 .
  • FIG. 8 is a data structure diagram showing, by way of example, a database table containing a lexicon of extracted concepts stored in the database 30 of FIG. 1 .
  • FIG. 9 is a graph showing, by way of example, a histogram of the frequencies of concept occurrences generated by the routine of FIG. 6 .
  • FIG. 10 is a table showing, by way of example, concept occurrence frequencies generated by the routine of FIG. 6 .
  • FIG. 11 is a graph showing, by way of example, a corpus graph of the frequency of concept occurrences generated by the routine of FIG. 5 .
  • FIG. 12 is a flow diagram showing a routine for creating a matrix for use in the routine of FIG. 5 .
  • FIG. 13 is a table showing, by way of example, the matrix of themes generated by the routine of FIG. 12 .
  • FIG. 14 is a flow diagram showing a routine for determining results for use in the routine of FIG. 5 .
  • FIG. 1 is a block diagram showing a system 11 for dynamically evaluating latent concepts in unstructured documents, in accordance with the present invention.
  • the system 11 operates in a distributed computing environment 10 which includes a plurality of heterogeneous systems and document sources.
  • the system 11 implements a document analyzer 12 , as further described below beginning with reference to FIG. 2 , for evaluating latent concepts in unstructured documents.
  • the system 11 is coupled to a storage device 13 which stores a document warehouse 14 for maintaining a repository of documents and a database 30 for maintaining document information.
  • the document analyzer 12 analyzes documents retrieved from a plurality of local sources.
  • the local sources include documents 17 maintained in a storage device 16 coupled to a local server 15 and documents 20 maintained in a storage device 19 coupled to a local client 18 .
  • the local server 15 and local client 18 are interconnected to the system 11 over an intranetwork 21 .
  • the document analyzer 12 can identify and retrieve documents from remote sources over an internetwork 22 , including the Internet, through a gateway 23 interfaced to the intranetwork 21 .
  • the remote sources include documents 26 maintained in a storage device 25 coupled to a remote server 24 and documents 29 maintained in a storage device 28 coupled to a remote client 27 .
  • the individual documents 17 , 20 , 26 , 29 include all forms and types of unstructured data, including electronic message stores, such as electronic mail (email) folders, word processing documents or Hypertext documents, and could also include graphical or multimedia data. Notwithstanding, the documents could be in the form of structured data, such as stored in a spreadsheet or database. Content mined from these types of documents does not require preprocessing, as described below.
  • the individual documents 17 , 20 , 26 , 29 include electronic message folders, such as maintained by the Outlook and Outlook Express products, licensed by Microsoft Corporation, Redmond, Wash.
  • the database is an SQL-based relational database, such as the Oracle database management system, release 8, licensed by Oracle Corporation, Redwood Shores, Calif.
  • the individual computer systems including system 11 , server 15 , client 18 , remote server 24 and remote client 27 , are general purpose, programmed digital computing devices consisting of a central processing unit (CPU), random access memory (RAM), non-volatile secondary storage, such as a hard drive or CD ROM drive, network interfaces, and peripheral devices, including user interfacing means, such as a keyboard and display.
  • Program code including software programs, and data are loaded into the RAM for execution and processing by the CPU and results are generated for display, output, transmittal, or storage.
  • FIG. 2 is a block diagram showing the software modules 40 implementing the document analyzer 12 of FIG. 1 .
  • the document analyzer 12 includes three modules: storage and retrieval manager 41 , text analyzer 42 , and display and visualization 43 .
  • the storage and retrieval manager 41 identifies and retrieves documents 44 into the document warehouse 14 (shown in FIG. 1 ).
  • the documents 44 are retrieved from various sources, including both local and remote clients and server stores.
  • the text analyzer 42 performs the bulk of the text mining processing.
  • the display and visualization 43 complements the operations performed by the text analyzer 42 by presenting visual representations of the information extracted from the documents 44 .
  • the display and visualization 43 can also generate a graphical representation which preserves independent variable relationships, such as described in common-assigned U.S. Pat. No. 6,888,548, issued May 3, 2005, the disclosure of which is incorporated by reference.
  • the text analyzer 42 identifies terms and phrases and extracts concepts in the form of noun phrases that are stored in a lexicon 18 maintained in the database 30 . After normalizing the extracted concepts, the text analyzer 42 generates a frequency table 46 of concept occurrences, as further described below with reference to FIG. 6 , and a matrix 47 of summations of the products of pair-wise terms, as further described below with reference to FIG. 10 . Similarly, the display and visualization 43 generates a histogram 47 of concept occurrences per document, as further described below with reference to FIG. 6 , and a corpus graph 48 of concept occurrences over all documents, as further described below with reference to FIG. 8 .
  • Each module is a computer program, procedure or module written as source code in a conventional programming language, such as the C++ programming language, and is presented for execution by the CPU as object or byte code, as is known in the art.
  • the various implementations of the source code and object and byte codes can be held on a computer-readable storage medium or embodied on a transmission medium in a carrier wave.
  • the document analyzer 12 operates in accordance with a sequence of process steps, as further described below with reference to FIG. 5 .
  • FIG. 3 is a process flow diagram showing the stages 60 of text analysis performed by the document analyzer 12 of FIG. 1 .
  • the individual documents 44 are preprocessed and noun phrases are extracted as concepts (transition 61 ) into a lexicon 45 .
  • the noun phrases are normalized and queried (transition 62 ) to generate a frequency table 46 .
  • the frequency table 46 identifies individual concepts and their respective frequency of occurrence within each document 44 .
  • the frequencies of concept occurrences are visualized (transition 63 ) into a frequency of concepts histogram 48 .
  • the histogram 48 graphically displays the frequencies of occurrence of each concept on a per-document basis.
  • FIG. 4 is a flow diagram showing a method 70 for dynamically evaluating latent concepts in unstructured documents 44 (shown in FIG. 2 ), in accordance with the present invention.
  • the set of documents 44 to be analyzed is identified (block 71 ) and retrieved into the document warehouse 14 (shown in FIG. 1 ) (block 72 ).
  • the documents 44 are unstructured data and lack a common format or shared type.
  • the documents 44 include electronic messages stored in messaging folders, word processing documents, hypertext documents, and the like.
  • the set of documents 44 is analyzed (block 73 ), as further described below with reference to FIG. 5 .
  • a matrix 47 shown in FIG. 2 ) of term-document association data is constructed to summarize the semantic content inherent in the structure of the documents 44 .
  • the frequency of individual terms or phrases extracted from the documents 44 are displayed and the results are optionally visualized (block 74 ). The routine then terminates.
  • FIG. 5 is a flow diagram showing the routine 80 for performing text analysis for use in the method 70 of FIG. 4 .
  • the purpose of this routine is to extract and index terms or phrases for the set of documents 44 (shown in FIG. 2 ).
  • each document in the documents set 44 is preprocessed (block 81 ) to remove stop words.
  • stop words include commonly occurring words, such as indefinite articles (“a” and “an”), definite articles (“the”), pronouns (“I”, “he” and “she”), connectors (“and” and “or”), and similar non-substantive words.
  • a histogram 48 of the frequency of terms (shown in FIG. 2 ) is logically created for each document 44 (block 82 ), as further described below with reference to FIG. 6 .
  • Each histogram 48 maps the relative frequency of occurrence of each extracted term on a per-document basis.
  • a document reference frequency (corpus) graph 49 is created for all documents 44 (block 83 ).
  • the corpus graph 49 graphically maps the semantically-related concepts for the entire documents set 44 based on terms and phrases.
  • a subset of the corpus is selected by removing those terms and phrases falling outside either edge of predefined thresholds (block 84 ). For shorter documents, such as email, having less semantically-rich content, the thresholds are set from about 1% to about 15%, inclusive. Larger documents may require tighter threshold values.
  • the selected set of terms and phrases falling within the thresholds are used to generate themes (and concepts) (block 85 ) based on correlations between normalized terms and phrases in the documents set.
  • themes are primarily used, rather than individual concepts, as a single co-occurrence of terms or phrases carries less semantic meaning than multiple co-occurrences.
  • any reference to a “theme” or “concept” will be understood to include the other term, except as specifically indicated otherwise.
  • clusters are created (block 86 ) from groups of highly-correlated concepts and themes. Individual concepts and themes are categorized based on, for example, Euclidean distances calculated between each pair of concepts and themes and defined within a pre-specified range of variance, such as described in commonly-assigned U.S. Pat. No. 6,778,995, issued Aug. 17, 2004, the disclosure of which is incorporated by reference.
  • a matrix 47 of the documents 44 is created (block 87 ), as further described below with reference to FIG. 13 .
  • the matrix 47 contains the inner products of document concept frequency occurrences and cluster concept weightings mapped into a multi-dimensional concept space for each theme.
  • the results of the text analysis operations are determined (block 88 ), as further described below with reference to FIG. 14 , after which the routine returns.
  • FIG. 6 is a flow diagram showing the routine 90 for creating a histogram 48 (shown in FIG. 2 ) for use in the routine of FIG. 5 .
  • the purpose of this routine is to extract noun phrases representing individual concepts and to create a normalized representation of the occurrences of the concepts on a per-document basis.
  • the histogram represents the logical union of the terms and phrases extracted from each document.
  • the histogram 48 need not be expressly visualized, but is generated internally as part of the text analysis process.
  • noun phrases are extracted (block 91 ) from each document 44 .
  • concepts are defined on the basis of the extracted noun phrases, although individual nouns or tri-grams (word triples) could be used in lieu of noun phrases.
  • the noun phrases are extracted using the LinguistX product licensed by Inxight Software, Inc., Santa Clara, Calif.
  • the individual terms or phrases are loaded into records stored in the database 30 (shown in FIG. 1 ) (block 92 ).
  • the terms stored in the database 30 are normalized (block 93 ) such that each concept appears as a record only once.
  • the records are normalized into third normal form, although other normalization schemas could be used.
  • FIG. 7 is a data structure diagram showing a database record 100 for a concept stored in the database 30 of FIG. 1 .
  • Each database record 100 includes fields for storing an identifier 101 , string 102 and frequency 103 .
  • the identifier 101 is a monotonically increasing integer value that uniquely identifies each term or phrase stored as the string 102 in each record 100 .
  • the frequency of occurrence of each term or phrase is tallied in the frequency 103 .
  • FIG. 8 is a data structure diagram showing, by way of example, a database table 110 containing a lexicon 111 of extracted concepts stored in the database 30 of FIG. 1 .
  • the lexicon 111 maps out the individual occurrences of identified terms 113 extracted for any given document 112 .
  • the document 112 includes three terms numbered 1 , 3 and 5 .
  • Concept 1 occurs once in document 112
  • concept 3 occurs twice
  • concept 5 occurs once.
  • the lexicon tallies and represents the occurrences of frequency of the concepts 1 , 3 and 5 across all documents 44 .
  • a frequency table is created from the lexicon 111 for each given document 44 (block 94 ).
  • the frequency table is sorted in order of decreasing frequencies of occurrence for each concept 113 found in a given document 44 . In the described embodiment, all terms and phrases occurring just once in a given document are removed as not relevant to semantic content.
  • the frequency table is then used to generate a histogram 48 (shown in FIG. 2 ) (block 95 ) which visualizes the frequencies of occurrence of extracted concepts in each document. The routine then returns.
  • FIG. 9 is a graph showing, by way of example, a histogram 48 of the frequencies of concept occurrences generated by the routine of FIG. 6 .
  • the x-axis defines the individual concepts 121 for each document and the y-axis defines the frequencies of occurrence of each concept 122 .
  • the concepts are mapped in order of decreasing frequency 123 to generate a curve 124 representing the semantic content of the document 44 . Accordingly, terms or phrases appearing on the increasing end of the curve 124 have a high frequency of occurrence while concepts appearing on the descending end of the curve 124 have a low frequency of occurrence.
  • FIG. 10 is a table 130 showing, by way of example, concept occurrence frequencies generated by the routine of FIG. 6 .
  • Each concept 131 is mapped against the total frequency occurrence 132 for the entire set of documents 44 .
  • a cumulative frequency 134 is tallied.
  • the corpus table 130 is used to generate the document concept frequency reference (corpus) graph 49 .
  • FIG. 11 is a graph 140 showing, by way of example, a corpus graph of the frequency of concept occurrences generated by the routine of FIG. 5 .
  • the graph 140 visualizes the extracted concepts as tallied in the corpus table 130 (shown in FIG. 10 ).
  • the x-axis defines the individual concepts 141 for all documents and the y-axis defines the number of documents 44 referencing each concept 142 .
  • the individual concepts are mapped in order of descending frequency of occurrence 143 to generate a curve 144 representing the latent semantics of the set of documents 44 .
  • a median value 145 is selected and edge conditions 146 a - b are established to discriminate between concepts which occur too frequently versus concepts which occur too infrequently. Those documents falling within the edge conditions 146 a - b form a subset of documents containing latent concepts.
  • the median value 145 is document-type dependent.
  • the upper edge condition 146 b is set to 70% and the 64 concepts immediately preceding the upper edge condition 146 b are selected, although other forms of threshold discrimination could also be used.
  • FIG. 12 is a flow diagram showing the routine 150 for creating a matrix 47 (shown in FIG. 2 ) for use in the routine of FIG. 5 .
  • those documents 44 having zero values for frequency counts are removed through filtering (block 151 ).
  • the inner products of document concept frequency occurrences and cluster concept weightings mapped into a multi-dimensional concept space for each theme are calculated and used to populate the matrix (block 152 ).
  • the individual cluster weightings are iteratively updated (block 153 ) to determine best fit. Those documents having the smallest inner products are deemed most relevant to a given theme and are identified (block 154 ).
  • the routine then returns.
  • FIG. 13 is a table 170 showing the matrix 47 generated by the routine of FIG. 12 .
  • the matrix 47 maps a cluster 171 to documents 172 based on a calculated inner product.
  • Each inner product quantifies similarities between documents, as represented by a distance.
  • the distance is mapped into a multi-dimensional concept space for a given document, as measured by the magnitude of a vector for a given term drawn relative to an angle ⁇ , held constant for the given cluster.
  • the distance d cluster is calculated by taking the sum of products (inner product) by terms between document concept frequency occurrences and cluster concept weightings, using the following equation:
  • d cluster ⁇ i ⁇ n ⁇ doc term i ⁇ cluster term i
  • doc term represents the frequency of occurrence for a given term i in the selected document
  • cluster term represents the weight of a given cluster for a given term i.
  • the weights of the individual inner products are iteratively updated until the clusters settle.
  • the goal is to calculate the minimum distances between as few clusters as possible until the rate of change goes constant.
  • the rate of change can be calculated, for example, by taking the first derivative of the inner products over successive iterations.
  • FIG. 14 is a flow diagram showing the routine 180 for determining results for use in the routine of FIG. 5 .
  • Duplicate documents 44 are removed from the results (block 181 ).
  • the results are re-run (block 182 ), as necessary by repeating the text analysis operations (block 183 ), beginning with creating the corpus graph 49 (block 84 in FIG. 5 ). After satisfactory results have been obtained (block 182 ), the routine returns.
  • results are re-run (block 182 ).
  • One reason to re-run the results set would be to re-center the median value 145 of the corpus graph 140 (shown in FIG. 11 ) following the filtering of further documents 44 .
  • the filtering of edge condition concept frequency occurrences will cause the curve 144 to be redefined, thereby requiring further processing.

Abstract

A computer-implemented system and method for clustering similar documents is provided. Concepts are identified for a set of documents and occurrence frequencies are determined for each concept in the documents set. A distance quantifying a similarity for each of the documents in the set with one or more clusters of documents is calculated. Each document is mapped to at least one of the one or more document clusters.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS
This patent application is a continuation of U.S. patent application Ser. No. 13/225,325, filed Sep. 2, 2011, pending; which is a continuation of U.S. Pat. No. 8,015,188, issued Sep. 6, 2011; which is a continuation of U.S. Pat. No. 7,809,727, issued Oct. 5, 2010; which is a continuation of U.S. Pat. No. 7,313,556, issued Dec. 25, 2007; which is a continuation of U.S. Pat. No. 6,978,274, issued Dec. 20, 2005, the priority filing dates of which are claimed and the disclosures of which are incorporated by reference.
FIELD
The present invention relates in general to text mining and, in particular, to a system and method for clustering similar documents.
BACKGROUND
Document warehousing extends data warehousing to content mining and retrieval. Document warehousing attempts to extract semantic information from collections of unstructured documents to provide conceptual information with a high degree of precision and recall. Documents in a document warehouse share several properties. First, the documents lack a common structure or shared type. Second, semantically-related documents are integrated through text mining. Third, essential document features are extracted and explicitly stored as part of the document warehouse. Finally, documents are often retrieved from multiple and disparate sources, such as over the Internet or as electronic messages.
Document warehouses are built in stages to deal with a wide range of information sources. First, document sources are identified and documents are retrieved into a repository. For example, the document sources could be electronic messaging folders or Web content retrieved over the Internet. Once retrieved, the documents are pre-processed to format and regularize the information into a consistent manner. Next, during text analysis, text mining is performed to extract semantic content, including identifying dominant themes, extracting key features and summarizing the content. Finally, metadata is compiled from the semantic context to explicate essential attributes. Preferably, the metadata is provided in a format amenable to normalized queries, such as database management tools. Document warehousing is described in D. Sullivan, “Document Warehousing and Text Mining, Techniques for Improving Business Operations, Marketing, and Sales,” Chs. 1-3, Wiley Computer Publishing (2001), the disclosure of which is incorporated by reference.
Text mining is at the core of the data warehousing process. Text mining involves the compiling, organizing and analyzing of document collections to support the delivery of targeted types of information and to discover relationships between relevant facts. However, identifying relevant content can be difficult. First, extracting relevant content requires a high degree of precision and recall. Precision is the measure of how well the documents returned in response to a query actually address the query criteria. Recall is the measure of what should have been returned by the query. Typically, the broader and less structured the documents, the lower the degree of precision and recall. Second, analyzing an unstructured document collection without the benefit of a priori knowledge in the form of keywords and indices can present a potentially intractable problem space. Finally, synonymy and polysemy can cloud and confuse extracted content. Synonymy refers to multiple words having the same meaning and polysemy refers to a single word with multiple meanings. Fine-grained text mining must reconcile synonymy and polysemy to yield meaningful results.
In the prior art, text mining is performed in two ways. First, syntactic searching provides a brute force approach to analyzing and extracting content based on literal textual attributes found in each document. Syntactic searching includes keyword and proximate keyword searching as well as rule-based searching through Boolean relationships. Syntactic searching relies on predefined indices of keywords and stop words to locate relevant information. However, there are several ways to express any given concept. Accordingly, syntactic searching can fail to yield satisfactory results due to incomplete indices and poorly structured search criteria.
A more advanced prior art approach uses a vector space model to search for underlying meanings in a document collection. The vector space model employs a geometric representation of documents using word vectors. Individual keywords are mapped into vectors in multi-dimensional space along axes representative of query search terms. Significant terms are assigned a relative weight and semantic content is extracted based on threshold filters. Although substantially overcoming the shortcomings of syntactic searching, the multivariant and multidimensional nature of the vector space model can lead to a computationally intractable problem space. As well, the vector space model fails to resolve the problems of synonymy and polysemy.
Therefore, there is a need for an approach to dynamically evaluating concepts inherent in a collection of documents. Such an approach would preferably dynamically discover the latent meanings without the use of a priori knowledge or indices. Rather, the approach would discover semantic relationships between individual terms given the presence of another item.
There is a further need for an approach to providing a graphical visualization of concepts extracted from a document set through semantic indexing. Preferably, such an approach would extract the underlying meanings of documents through statistics and linear algebraic techniques to find clusters of terms and phrases representative of the concepts.
SUMMARY
The present invention provides a system and method for indexing and evaluating unstructured documents through analysis of dynamically extracted concepts. A set of unstructured documents is identified and retrieved into a document warehouse repository. Individual concepts are extracted from the documents and mapped as normalized data into a database. The frequencies of occurrence of each concept within each document and over all documents are determined and mapped. A corpus graph is generated to display a minimized set of concepts whereby each concept references at least two documents and no document in the corpus is unreferenced. A subset of documents occurring within predefined edge conditions of a median value are selected. Clusters of concepts are grouped into themes. Inner products of document concept frequency occurrences and cluster concept weightings are mapped into a multi-dimensional concept space for each theme and iteratively generated until the clusters settle. The resultant data minima indicates those documents having the most pertinence to the identified concepts.
An embodiment provides a computer-implemented system and method for clustering similar documents. Concepts are identified for a set of documents and occurrence frequencies are determined for each concept in the documents set. A distance quantifying a similarity for each of the documents in the set with one or more clusters of documents is calculated. Each document is mapped to at least one of the one or more document clusters.
In summary, the present invention semantically evaluates terms and phrases with the goal of creating meaningful themes. Document frequencies and co-occurrences of terms and phrases are used to select a minimal set of highly correlated terms and phrases that reference all documents in a corpus.
Still other embodiments of the present invention will become readily apparent to those skilled in the art from the following detailed description, wherein is described embodiments of the invention by way of illustrating the best mode contemplated for carrying out the invention. As will be realized, the invention is capable of other and different embodiments and its several details are capable of modifications in various obvious respects, all without departing from the spirit and the scope of the present invention. Accordingly, the drawings and detailed description are to be regarded as illustrative in nature and not as restrictive.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 is a block diagram showing a system for dynamically evaluating latent concepts in unstructured documents, in accordance with the present invention.
FIG. 2 is a block diagram showing the software modules implementing the document analyzer of FIG. 1.
FIG. 3 is a process flow diagram showing the stages of text analysis performed by the document analyzer of FIG. 1.
FIG. 4 is a flow diagram showing a method for dynamically evaluating latent concepts in unstructured documents, in accordance with the present invention.
FIG. 5 is a flow diagram showing the routine for performing text analysis for use in the method of FIG. 4.
FIG. 6 is a flow diagram showing the routine for creating a histogram for use in the routine of FIG. 5.
FIG. 7 is a data structure diagram showing a database record for a concept stored in the database 30 of FIG. 1.
FIG. 8 is a data structure diagram showing, by way of example, a database table containing a lexicon of extracted concepts stored in the database 30 of FIG. 1.
FIG. 9 is a graph showing, by way of example, a histogram of the frequencies of concept occurrences generated by the routine of FIG. 6.
FIG. 10 is a table showing, by way of example, concept occurrence frequencies generated by the routine of FIG. 6.
FIG. 11 is a graph showing, by way of example, a corpus graph of the frequency of concept occurrences generated by the routine of FIG. 5.
FIG. 12 is a flow diagram showing a routine for creating a matrix for use in the routine of FIG. 5.
FIG. 13 is a table showing, by way of example, the matrix of themes generated by the routine of FIG. 12.
FIG. 14 is a flow diagram showing a routine for determining results for use in the routine of FIG. 5.
DETAILED DESCRIPTION Glossary
    • Keyword: A literal search term which is either present or absent from a document. Keywords are not used in the evaluation of documents as described herein.
    • Term: A root stem of a single word appearing in the body of at least one document.
    • Phrase: Two or more words co-occurring in the body of a document.
    • A phrase can include stop words.
    • Concept: A collection of terms or phrases with common semantic meanings.
    • Theme: Two or more concepts with a common semantic meaning.
    • Cluster: All documents for a given concept or theme.
      The foregoing terms are used throughout this document and, unless indicated otherwise, are assigned the meanings presented above.
FIG. 1 is a block diagram showing a system 11 for dynamically evaluating latent concepts in unstructured documents, in accordance with the present invention. By way of illustration, the system 11 operates in a distributed computing environment 10 which includes a plurality of heterogeneous systems and document sources. The system 11 implements a document analyzer 12, as further described below beginning with reference to FIG. 2, for evaluating latent concepts in unstructured documents. The system 11 is coupled to a storage device 13 which stores a document warehouse 14 for maintaining a repository of documents and a database 30 for maintaining document information.
The document analyzer 12 analyzes documents retrieved from a plurality of local sources. The local sources include documents 17 maintained in a storage device 16 coupled to a local server 15 and documents 20 maintained in a storage device 19 coupled to a local client 18. The local server 15 and local client 18 are interconnected to the system 11 over an intranetwork 21. In addition, the document analyzer 12 can identify and retrieve documents from remote sources over an internetwork 22, including the Internet, through a gateway 23 interfaced to the intranetwork 21. The remote sources include documents 26 maintained in a storage device 25 coupled to a remote server 24 and documents 29 maintained in a storage device 28 coupled to a remote client 27.
The individual documents 17, 20, 26, 29 include all forms and types of unstructured data, including electronic message stores, such as electronic mail (email) folders, word processing documents or Hypertext documents, and could also include graphical or multimedia data. Notwithstanding, the documents could be in the form of structured data, such as stored in a spreadsheet or database. Content mined from these types of documents does not require preprocessing, as described below.
In the described embodiment, the individual documents 17, 20, 26, 29 include electronic message folders, such as maintained by the Outlook and Outlook Express products, licensed by Microsoft Corporation, Redmond, Wash. The database is an SQL-based relational database, such as the Oracle database management system, release 8, licensed by Oracle Corporation, Redwood Shores, Calif.
The individual computer systems, including system 11, server 15, client 18, remote server 24 and remote client 27, are general purpose, programmed digital computing devices consisting of a central processing unit (CPU), random access memory (RAM), non-volatile secondary storage, such as a hard drive or CD ROM drive, network interfaces, and peripheral devices, including user interfacing means, such as a keyboard and display. Program code, including software programs, and data are loaded into the RAM for execution and processing by the CPU and results are generated for display, output, transmittal, or storage.
FIG. 2 is a block diagram showing the software modules 40 implementing the document analyzer 12 of FIG. 1. The document analyzer 12 includes three modules: storage and retrieval manager 41, text analyzer 42, and display and visualization 43. The storage and retrieval manager 41 identifies and retrieves documents 44 into the document warehouse 14 (shown in FIG. 1). The documents 44 are retrieved from various sources, including both local and remote clients and server stores. The text analyzer 42 performs the bulk of the text mining processing. The display and visualization 43 complements the operations performed by the text analyzer 42 by presenting visual representations of the information extracted from the documents 44. The display and visualization 43 can also generate a graphical representation which preserves independent variable relationships, such as described in common-assigned U.S. Pat. No. 6,888,548, issued May 3, 2005, the disclosure of which is incorporated by reference.
During text analysis, the text analyzer 42 identifies terms and phrases and extracts concepts in the form of noun phrases that are stored in a lexicon 18 maintained in the database 30. After normalizing the extracted concepts, the text analyzer 42 generates a frequency table 46 of concept occurrences, as further described below with reference to FIG. 6, and a matrix 47 of summations of the products of pair-wise terms, as further described below with reference to FIG. 10. Similarly, the display and visualization 43 generates a histogram 47 of concept occurrences per document, as further described below with reference to FIG. 6, and a corpus graph 48 of concept occurrences over all documents, as further described below with reference to FIG. 8.
Each module is a computer program, procedure or module written as source code in a conventional programming language, such as the C++ programming language, and is presented for execution by the CPU as object or byte code, as is known in the art. The various implementations of the source code and object and byte codes can be held on a computer-readable storage medium or embodied on a transmission medium in a carrier wave. The document analyzer 12 operates in accordance with a sequence of process steps, as further described below with reference to FIG. 5.
FIG. 3 is a process flow diagram showing the stages 60 of text analysis performed by the document analyzer 12 of FIG. 1. The individual documents 44 are preprocessed and noun phrases are extracted as concepts (transition 61) into a lexicon 45. The noun phrases are normalized and queried (transition 62) to generate a frequency table 46. The frequency table 46 identifies individual concepts and their respective frequency of occurrence within each document 44. The frequencies of concept occurrences are visualized (transition 63) into a frequency of concepts histogram 48. The histogram 48 graphically displays the frequencies of occurrence of each concept on a per-document basis. Next, the frequencies of concept occurrences for all the documents 44 are assimilated (transition 64) into a corpus graph 49 that displays the overall counts of documents containing each of the extracted concepts. Finally, the most relevant concepts are summarized (transition 65) into a matrix 46 that presents the results as summations of the products of pair-wise terms.
FIG. 4 is a flow diagram showing a method 70 for dynamically evaluating latent concepts in unstructured documents 44 (shown in FIG. 2), in accordance with the present invention. As a preliminary step, the set of documents 44 to be analyzed is identified (block 71) and retrieved into the document warehouse 14 (shown in FIG. 1) (block 72). The documents 44 are unstructured data and lack a common format or shared type. The documents 44 include electronic messages stored in messaging folders, word processing documents, hypertext documents, and the like.
Once identified and retrieved, the set of documents 44 is analyzed (block 73), as further described below with reference to FIG. 5. During text analysis, a matrix 47 (shown in FIG. 2) of term-document association data is constructed to summarize the semantic content inherent in the structure of the documents 44. As well, the frequency of individual terms or phrases extracted from the documents 44 are displayed and the results are optionally visualized (block 74). The routine then terminates.
FIG. 5 is a flow diagram showing the routine 80 for performing text analysis for use in the method 70 of FIG. 4. The purpose of this routine is to extract and index terms or phrases for the set of documents 44 (shown in FIG. 2). Preliminarily, each document in the documents set 44 is preprocessed (block 81) to remove stop words. These include commonly occurring words, such as indefinite articles (“a” and “an”), definite articles (“the”), pronouns (“I”, “he” and “she”), connectors (“and” and “or”), and similar non-substantive words.
Following preprocessing, a histogram 48 of the frequency of terms (shown in FIG. 2) is logically created for each document 44 (block 82), as further described below with reference to FIG. 6. Each histogram 48, as further described below with reference to FIG. 9, maps the relative frequency of occurrence of each extracted term on a per-document basis.
Next, a document reference frequency (corpus) graph 49, as further described below with reference to FIG. 10, is created for all documents 44 (block 83). The corpus graph 49 graphically maps the semantically-related concepts for the entire documents set 44 based on terms and phrases. A subset of the corpus is selected by removing those terms and phrases falling outside either edge of predefined thresholds (block 84). For shorter documents, such as email, having less semantically-rich content, the thresholds are set from about 1% to about 15%, inclusive. Larger documents may require tighter threshold values.
The selected set of terms and phrases falling within the thresholds are used to generate themes (and concepts) (block 85) based on correlations between normalized terms and phrases in the documents set. In the described embodiment, themes are primarily used, rather than individual concepts, as a single co-occurrence of terms or phrases carries less semantic meaning than multiple co-occurrences. As used herein, any reference to a “theme” or “concept” will be understood to include the other term, except as specifically indicated otherwise.
Next, clusters are created (block 86) from groups of highly-correlated concepts and themes. Individual concepts and themes are categorized based on, for example, Euclidean distances calculated between each pair of concepts and themes and defined within a pre-specified range of variance, such as described in commonly-assigned U.S. Pat. No. 6,778,995, issued Aug. 17, 2004, the disclosure of which is incorporated by reference.
A matrix 47 of the documents 44 is created (block 87), as further described below with reference to FIG. 13. The matrix 47 contains the inner products of document concept frequency occurrences and cluster concept weightings mapped into a multi-dimensional concept space for each theme. Finally, the results of the text analysis operations are determined (block 88), as further described below with reference to FIG. 14, after which the routine returns.
FIG. 6 is a flow diagram showing the routine 90 for creating a histogram 48 (shown in FIG. 2) for use in the routine of FIG. 5. The purpose of this routine is to extract noun phrases representing individual concepts and to create a normalized representation of the occurrences of the concepts on a per-document basis. The histogram represents the logical union of the terms and phrases extracted from each document. In the described embodiment, the histogram 48 need not be expressly visualized, but is generated internally as part of the text analysis process.
Initially, noun phrases are extracted (block 91) from each document 44. In the described embodiment, concepts are defined on the basis of the extracted noun phrases, although individual nouns or tri-grams (word triples) could be used in lieu of noun phrases. In the described embodiment, the noun phrases are extracted using the LinguistX product licensed by Inxight Software, Inc., Santa Clara, Calif.
Once extracted, the individual terms or phrases are loaded into records stored in the database 30 (shown in FIG. 1) (block 92). The terms stored in the database 30 are normalized (block 93) such that each concept appears as a record only once. In the described embodiment, the records are normalized into third normal form, although other normalization schemas could be used.
FIG. 7 is a data structure diagram showing a database record 100 for a concept stored in the database 30 of FIG. 1. Each database record 100 includes fields for storing an identifier 101, string 102 and frequency 103. The identifier 101 is a monotonically increasing integer value that uniquely identifies each term or phrase stored as the string 102 in each record 100. The frequency of occurrence of each term or phrase is tallied in the frequency 103.
FIG. 8 is a data structure diagram showing, by way of example, a database table 110 containing a lexicon 111 of extracted concepts stored in the database 30 of FIG. 1. The lexicon 111 maps out the individual occurrences of identified terms 113 extracted for any given document 112. By way of example, the document 112 includes three terms numbered 1, 3 and 5. Concept 1 occurs once in document 112, concept 3 occurs twice, and concept 5 occurs once. The lexicon tallies and represents the occurrences of frequency of the concepts 1, 3 and 5 across all documents 44.
Referring back to FIG. 6, a frequency table is created from the lexicon 111 for each given document 44 (block 94). The frequency table is sorted in order of decreasing frequencies of occurrence for each concept 113 found in a given document 44. In the described embodiment, all terms and phrases occurring just once in a given document are removed as not relevant to semantic content. The frequency table is then used to generate a histogram 48 (shown in FIG. 2) (block 95) which visualizes the frequencies of occurrence of extracted concepts in each document. The routine then returns.
FIG. 9 is a graph showing, by way of example, a histogram 48 of the frequencies of concept occurrences generated by the routine of FIG. 6. The x-axis defines the individual concepts 121 for each document and the y-axis defines the frequencies of occurrence of each concept 122. The concepts are mapped in order of decreasing frequency 123 to generate a curve 124 representing the semantic content of the document 44. Accordingly, terms or phrases appearing on the increasing end of the curve 124 have a high frequency of occurrence while concepts appearing on the descending end of the curve 124 have a low frequency of occurrence.
FIG. 10 is a table 130 showing, by way of example, concept occurrence frequencies generated by the routine of FIG. 6. Each concept 131 is mapped against the total frequency occurrence 132 for the entire set of documents 44. Thus, for each of the concepts 133, a cumulative frequency 134 is tallied. The corpus table 130 is used to generate the document concept frequency reference (corpus) graph 49.
FIG. 11 is a graph 140 showing, by way of example, a corpus graph of the frequency of concept occurrences generated by the routine of FIG. 5. The graph 140 visualizes the extracted concepts as tallied in the corpus table 130 (shown in FIG. 10). The x-axis defines the individual concepts 141 for all documents and the y-axis defines the number of documents 44 referencing each concept 142. The individual concepts are mapped in order of descending frequency of occurrence 143 to generate a curve 144 representing the latent semantics of the set of documents 44.
A median value 145 is selected and edge conditions 146 a-b are established to discriminate between concepts which occur too frequently versus concepts which occur too infrequently. Those documents falling within the edge conditions 146 a-b form a subset of documents containing latent concepts. In the described embodiment, the median value 145 is document-type dependent. For efficiency, the upper edge condition 146 b is set to 70% and the 64 concepts immediately preceding the upper edge condition 146 b are selected, although other forms of threshold discrimination could also be used.
FIG. 12 is a flow diagram showing the routine 150 for creating a matrix 47 (shown in FIG. 2) for use in the routine of FIG. 5. Initially, those documents 44 having zero values for frequency counts are removed through filtering (block 151). The inner products of document concept frequency occurrences and cluster concept weightings mapped into a multi-dimensional concept space for each theme are calculated and used to populate the matrix (block 152). The individual cluster weightings are iteratively updated (block 153) to determine best fit. Those documents having the smallest inner products are deemed most relevant to a given theme and are identified (block 154). The routine then returns.
FIG. 13 is a table 170 showing the matrix 47 generated by the routine of FIG. 12. The matrix 47 maps a cluster 171 to documents 172 based on a calculated inner product. Each inner product quantifies similarities between documents, as represented by a distance. The distance is mapped into a multi-dimensional concept space for a given document, as measured by the magnitude of a vector for a given term drawn relative to an angle θ, held constant for the given cluster.
For a set of n documents, the distance dcluster is calculated by taking the sum of products (inner product) by terms between document concept frequency occurrences and cluster concept weightings, using the following equation:
d cluster = i n doc term i · cluster term i
where docterm represents the frequency of occurrence for a given term i in the selected document and clusterterm represents the weight of a given cluster for a given term i. The weights of the individual inner products are iteratively updated until the clusters settle. The goal is to calculate the minimum distances between as few clusters as possible until the rate of change goes constant. The rate of change can be calculated, for example, by taking the first derivative of the inner products over successive iterations.
FIG. 14 is a flow diagram showing the routine 180 for determining results for use in the routine of FIG. 5. Duplicate documents 44 are removed from the results (block 181). The results are re-run (block 182), as necessary by repeating the text analysis operations (block 183), beginning with creating the corpus graph 49 (block 84 in FIG. 5). After satisfactory results have been obtained (block 182), the routine returns.
Satisfactory results are shown when a meaningful cluster of documents is found. Objectively, each document within a given theme will have an inner product falling within a pre-defined variance of other related documents, thereby reflecting a set amount of similarity. The cluster itself represents a larger grouping of document sets based on related, but not identical, themes.
If necessary, the results are re-run (block 182). One reason to re-run the results set would be to re-center the median value 145 of the corpus graph 140 (shown in FIG. 11) following the filtering of further documents 44. The filtering of edge condition concept frequency occurrences will cause the curve 144 to be redefined, thereby requiring further processing.
While the invention has been particularly shown and described as referenced to the embodiments thereof, those skilled in the art will understand that the foregoing and other changes in form and detail may be made therein without departing from the spirit and scope of the invention.

Claims (20)

What is claimed is:
1. A computer-implemented system for clustering similar documents, comprising:
concepts for a set of documents;
an occurrence module to determine occurrence frequencies of each concept in the document set;
a distance module to calculate an inner product quantifying a similarity for each of the documents in the set with one or more clusters of documents based on the occurrence frequencies of the concepts;
a map module to map each document to each of the document clusters based on the inner product, to identify those documents with the smallest inner products as most relevant to a theme, and to generate a matrix as a representation of the document and cluster mappings; and
a processor to execute the modules.
2. A system according to claim 1, further comprising:
an extraction module to extract terms from the documents;
a concept determination module to generate the concepts from a subset of the extracted terms that satisfy threshold conditions for occurrences.
3. A system according to claim 1, further comprising:
a database record for each concept; and
a database to store the database records.
4. A system according to claim 1, further comprising:
a relevance module to determine those concepts that are most relevant; and
a summary module to summarize the most relevant concepts in a matrix that maps the concepts to clusters of documents.
5. A system according to claim 1, further comprising:
a lexicon generation module to build a lexicon of the concepts by mapping individual occurrences of each concept within one or more of the documents.
6. A system according to claim 5, further comprising:
a frequency table generation module to generate a frequency table from the lexicon by removing terms that occur only once in the documents and ordering the concepts by decreasing order.
7. A system according to claim 6, further comprising:
a histogram generation module to build from the frequency table, a histogram comprising a visualization of the frequencies of occurrences of the concepts extracted from each document.
8. A system according to claim 1, further comprising:
an occurrence module to determine a total frequency occurrence for the concepts by mapping each of the concepts across all the documents in the set.
9. A system according to claim 8, further comprising:
a concept graph generation module to generate a concept graph based on the total frequency occurrence for each concept by mapping the concepts in order of descending frequency of occurrence within a number of documents that reference that concept.
10. A system according to claim 1, further comprising:
a similarity module to quantify the similarity by comparing frequency occurrences for each of the concepts with concept weightings for each cluster.
11. A method for clustering similar documents, comprising the steps of:
identifying concepts for a set of documents;
determining occurrence frequencies of each concept in the document set;
calculating an inner product quantifying a similarity for each of the documents in the set with one or more clusters of documents based on the occurrence frequencies of the concepts;
mapping each document to each of the document clusters based on the inner product;
identifying those documents with the smallest inner products as most relevant to a theme; and
generating a matrix as a representation of the document and cluster mappings, wherein the steps are executed by a suitably-programmed computer.
12. A method according to claim 11, further comprising:
extracting terms from the documents;
generating the concepts from a subset of the extracted terms that satisfy threshold conditions for occurrences.
13. A method according to claim 11, further comprising:
generating a database record for each concept; and
storing the database records.
14. A method according to claim 11, further comprising:
determining those concepts that are most relevant; and
summarizing the most relevant concepts in a matrix that maps the concepts to clusters of documents.
15. A method according to claim 11, further comprising:
building a lexicon of the concepts by mapping individual occurrences of each concept within one or more of the documents.
16. A method according to claim 15, further comprising:
generating a frequency table from the lexicon by removing terms that occur only once in the documents and ordering the concepts by decreasing order.
17. A method according to claim 16, further comprising:
building from the frequency table, a histogram comprising a visualization of the frequencies of occurrences of the concepts extracted from each document.
18. A method according to claim 11, further comprising:
determining a total frequency occurrence for the concepts by mapping each of the concepts across all the documents in the set.
19. A method according to claim 18, further comprising:
generating a concept graph based on the total frequency occurrence for each concept by mapping the concepts in order of descending frequency of occurrence within a number of documents that reference that concept.
20. A method according to claim 11, further comprising:
quantifying the similarity by comparing frequency occurrences for each concept with concept weightings for each of the clusters.
US13/767,849 2001-08-31 2013-02-14 Computer-implemented system and method for clustering similar documents Expired - Lifetime US8725736B2 (en)

Priority Applications (4)

Application Number Priority Date Filing Date Title
US13/767,849 US8725736B2 (en) 2001-08-31 2013-02-14 Computer-implemented system and method for clustering similar documents
US14/275,802 US9195399B2 (en) 2001-08-31 2014-05-12 Computer-implemented system and method for identifying relevant documents for display
US14/949,829 US9619551B2 (en) 2001-08-31 2015-11-23 Computer-implemented system and method for generating document groupings for display
US15/484,122 US20170262455A1 (en) 2001-08-31 2017-04-10 Computer-Implemented System And Method For Identifying Relevant Documents

Applications Claiming Priority (6)

Application Number Priority Date Filing Date Title
US09/944,474 US6978274B1 (en) 2001-08-31 2001-08-31 System and method for dynamically evaluating latent concepts in unstructured documents
US11/304,406 US7313556B2 (en) 2001-08-31 2005-12-14 System and method for dynamically evaluating latent concepts in unstructured documents
US11/964,000 US7809727B2 (en) 2001-08-31 2007-12-24 System and method for clustering unstructured documents
US12/897,710 US8015188B2 (en) 2001-08-31 2010-10-04 System and method for thematically grouping documents into clusters
US13/225,325 US8380718B2 (en) 2001-08-31 2011-09-02 System and method for grouping similar documents
US13/767,849 US8725736B2 (en) 2001-08-31 2013-02-14 Computer-implemented system and method for clustering similar documents

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
US13/225,325 Continuation US8380718B2 (en) 2001-08-31 2011-09-02 System and method for grouping similar documents

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US14/275,802 Continuation US9195399B2 (en) 2001-08-31 2014-05-12 Computer-implemented system and method for identifying relevant documents for display

Publications (2)

Publication Number Publication Date
US20130159300A1 US20130159300A1 (en) 2013-06-20
US8725736B2 true US8725736B2 (en) 2014-05-13

Family

ID=35465698

Family Applications (9)

Application Number Title Priority Date Filing Date
US09/944,474 Expired - Lifetime US6978274B1 (en) 2001-08-31 2001-08-31 System and method for dynamically evaluating latent concepts in unstructured documents
US11/304,406 Expired - Lifetime US7313556B2 (en) 2001-08-31 2005-12-14 System and method for dynamically evaluating latent concepts in unstructured documents
US11/964,000 Expired - Lifetime US7809727B2 (en) 2001-08-31 2007-12-24 System and method for clustering unstructured documents
US12/897,710 Expired - Lifetime US8015188B2 (en) 2001-08-31 2010-10-04 System and method for thematically grouping documents into clusters
US13/225,325 Expired - Lifetime US8380718B2 (en) 2001-08-31 2011-09-02 System and method for grouping similar documents
US13/767,849 Expired - Lifetime US8725736B2 (en) 2001-08-31 2013-02-14 Computer-implemented system and method for clustering similar documents
US14/275,802 Expired - Lifetime US9195399B2 (en) 2001-08-31 2014-05-12 Computer-implemented system and method for identifying relevant documents for display
US14/949,829 Expired - Lifetime US9619551B2 (en) 2001-08-31 2015-11-23 Computer-implemented system and method for generating document groupings for display
US15/484,122 Abandoned US20170262455A1 (en) 2001-08-31 2017-04-10 Computer-Implemented System And Method For Identifying Relevant Documents

Family Applications Before (5)

Application Number Title Priority Date Filing Date
US09/944,474 Expired - Lifetime US6978274B1 (en) 2001-08-31 2001-08-31 System and method for dynamically evaluating latent concepts in unstructured documents
US11/304,406 Expired - Lifetime US7313556B2 (en) 2001-08-31 2005-12-14 System and method for dynamically evaluating latent concepts in unstructured documents
US11/964,000 Expired - Lifetime US7809727B2 (en) 2001-08-31 2007-12-24 System and method for clustering unstructured documents
US12/897,710 Expired - Lifetime US8015188B2 (en) 2001-08-31 2010-10-04 System and method for thematically grouping documents into clusters
US13/225,325 Expired - Lifetime US8380718B2 (en) 2001-08-31 2011-09-02 System and method for grouping similar documents

Family Applications After (3)

Application Number Title Priority Date Filing Date
US14/275,802 Expired - Lifetime US9195399B2 (en) 2001-08-31 2014-05-12 Computer-implemented system and method for identifying relevant documents for display
US14/949,829 Expired - Lifetime US9619551B2 (en) 2001-08-31 2015-11-23 Computer-implemented system and method for generating document groupings for display
US15/484,122 Abandoned US20170262455A1 (en) 2001-08-31 2017-04-10 Computer-Implemented System And Method For Identifying Relevant Documents

Country Status (1)

Country Link
US (9) US6978274B1 (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9323721B1 (en) * 2013-02-27 2016-04-26 Google Inc. Quotation identification
US20170242879A1 (en) * 2007-07-26 2017-08-24 Hamid Hatami-Hanza Ontological Subjects Of A Universe And Knowledge Representations Thereof
US11269812B2 (en) 2019-05-10 2022-03-08 International Business Machines Corporation Derived relationship for collaboration documents

Families Citing this family (185)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7035864B1 (en) * 2000-05-18 2006-04-25 Endeca Technologies, Inc. Hierarchical data-driven navigation system and method for information retrieval
US7617184B2 (en) * 2000-05-18 2009-11-10 Endeca Technologies, Inc. Scalable hierarchical data-driven navigation system and method for information retrieval
USRE46973E1 (en) * 2001-05-07 2018-07-31 Ureveal, Inc. Method, system, and computer program product for concept-based multi-dimensional analysis of unstructured information
US7469246B1 (en) 2001-05-18 2008-12-23 Stratify, Inc. Method and system for classifying or clustering one item into multiple categories
US7308451B1 (en) 2001-09-04 2007-12-11 Stratify, Inc. Method and system for guided cluster based processing on prototypes
US7945600B1 (en) 2001-05-18 2011-05-17 Stratify, Inc. Techniques for organizing data to support efficient review and analysis
US20030004996A1 (en) * 2001-06-29 2003-01-02 International Business Machines Corporation Method and system for spatial information retrieval for hyperlinked documents
US7487544B2 (en) * 2001-07-30 2009-02-03 The Trustees Of Columbia University In The City Of New York System and methods for detection of new malicious executables
US7657935B2 (en) 2001-08-16 2010-02-02 The Trustees Of Columbia University In The City Of New York System and methods for detecting malicious email transmission
US6978274B1 (en) * 2001-08-31 2005-12-20 Attenex Corporation System and method for dynamically evaluating latent concepts in unstructured documents
US7818797B1 (en) 2001-10-11 2010-10-19 The Trustees Of Columbia University In The City Of New York Methods for cost-sensitive modeling for intrusion detection and response
US8544087B1 (en) 2001-12-14 2013-09-24 The Trustess Of Columbia University In The City Of New York Methods of unsupervised anomaly detection using a geometric framework
US9306966B2 (en) 2001-12-14 2016-04-05 The Trustees Of Columbia University In The City Of New York Methods of unsupervised anomaly detection using a geometric framework
US20030115191A1 (en) * 2001-12-17 2003-06-19 Max Copperman Efficient and cost-effective content provider for customer relationship management (CRM) or other applications
US7225343B1 (en) 2002-01-25 2007-05-29 The Trustees Of Columbia University In The City Of New York System and methods for adaptive model generation for detecting intrusions in computer systems
US7448084B1 (en) 2002-01-25 2008-11-04 The Trustees Of Columbia University In The City Of New York System and methods for detecting intrusions in a computer system by monitoring operating system registry accesses
US8015143B2 (en) * 2002-05-22 2011-09-06 Estes Timothy W Knowledge discovery agent system and method
US7568148B1 (en) 2002-09-20 2009-07-28 Google Inc. Methods and apparatus for clustering news content
US8090717B1 (en) 2002-09-20 2012-01-03 Google Inc. Methods and apparatus for ranking documents
US7359930B2 (en) * 2002-11-21 2008-04-15 Arbor Networks System and method for managing computer networks
US20040117366A1 (en) * 2002-12-12 2004-06-17 Ferrari Adam J. Method and system for interpreting multiple-term queries
US20050038781A1 (en) * 2002-12-12 2005-02-17 Endeca Technologies, Inc. Method and system for interpreting multiple-term queries
US8943024B1 (en) 2003-01-17 2015-01-27 Daniel John Gardner System and method for data de-duplication
US8630984B1 (en) 2003-01-17 2014-01-14 Renew Data Corp. System and method for data extraction from email files
US8375008B1 (en) 2003-01-17 2013-02-12 Robert Gomes Method and system for enterprise-wide retention of digital or electronic data
US8065277B1 (en) 2003-01-17 2011-11-22 Daniel John Gardner System and method for a data extraction and backup database
JP2004355069A (en) 2003-05-27 2004-12-16 Sony Corp Information processor, information processing method, program, and recording medium
US7610313B2 (en) * 2003-07-25 2009-10-27 Attenex Corporation System and method for performing efficient document scoring and clustering
US7363214B2 (en) * 2003-08-08 2008-04-22 Cnet Networks, Inc. System and method for determining quality of written product reviews in an automated manner
US7333997B2 (en) * 2003-08-12 2008-02-19 Viziant Corporation Knowledge discovery method with utility functions and feedback loops
US20050278362A1 (en) * 2003-08-12 2005-12-15 Maren Alianna J Knowledge discovery system
US7577655B2 (en) 2003-09-16 2009-08-18 Google Inc. Systems and methods for improving the ranking of news articles
US7503070B1 (en) * 2003-09-19 2009-03-10 Marshall Van Alstyne Methods and systems for enabling analysis of communication content while preserving confidentiality
CN1704957A (en) * 2004-05-28 2005-12-07 国际商业机器公司 Apparatus and method for dynamically assembling operation flow model
US20050283357A1 (en) * 2004-06-22 2005-12-22 Microsoft Corporation Text mining method
US7580921B2 (en) * 2004-07-26 2009-08-25 Google Inc. Phrase identification in an information retrieval system
US7584175B2 (en) * 2004-07-26 2009-09-01 Google Inc. Phrase-based generation of document descriptions
US7599914B2 (en) * 2004-07-26 2009-10-06 Google Inc. Phrase-based searching in an information retrieval system
US7567959B2 (en) 2004-07-26 2009-07-28 Google Inc. Multiple index based information retrieval system
US7426507B1 (en) * 2004-07-26 2008-09-16 Google, Inc. Automatic taxonomy generation in search results using phrases
US7580929B2 (en) 2004-07-26 2009-08-25 Google Inc. Phrase-based personalization of searches in an information retrieval system
US7711679B2 (en) 2004-07-26 2010-05-04 Google Inc. Phrase-based detection of duplicate documents in an information retrieval system
US7702618B1 (en) 2004-07-26 2010-04-20 Google Inc. Information retrieval system for archiving multiple document versions
US7536408B2 (en) * 2004-07-26 2009-05-19 Google Inc. Phrase-based indexing in an information retrieval system
EP1812898A2 (en) * 2004-11-02 2007-08-01 Eagleforce Associates System and method for predictive analysis and predictive analysis markup language
US8069151B1 (en) 2004-12-08 2011-11-29 Chris Crafford System and method for detecting incongruous or incorrect media in a data recovery process
US8527468B1 (en) 2005-02-08 2013-09-03 Renew Data Corp. System and method for management of retention periods for content in a computing system
US7421419B2 (en) * 2005-04-12 2008-09-02 Viziant Corporation System and method for evidence accumulation and hypothesis generation
AU2006239734B2 (en) * 2005-04-27 2011-09-15 The University Of Queensland Automatic concept clustering
US20090327259A1 (en) * 2005-04-27 2009-12-31 The University Of Queensland Automatic concept clustering
US20070006129A1 (en) * 2005-06-01 2007-01-04 Opasmedia Oy Forming of a data retrieval, searching from a data retrieval system, and a data retrieval system
US7937344B2 (en) 2005-07-25 2011-05-03 Splunk Inc. Machine data web
US7587395B2 (en) * 2005-07-27 2009-09-08 John Harney System and method for providing profile matching with an unstructured document
US20070156720A1 (en) * 2005-08-31 2007-07-05 Eagleforce Associates System for hypothesis generation
US8019752B2 (en) * 2005-11-10 2011-09-13 Endeca Technologies, Inc. System and method for information retrieval from object collections with complex interrelationships
US7756855B2 (en) * 2006-10-11 2010-07-13 Collarity, Inc. Search phrase refinement by search term replacement
US8903810B2 (en) 2005-12-05 2014-12-02 Collarity, Inc. Techniques for ranking search results
US8429184B2 (en) 2005-12-05 2013-04-23 Collarity Inc. Generation of refinement terms for search queries
US20070174296A1 (en) * 2006-01-17 2007-07-26 Andrew Gibbs Method and system for distributing a database and computer program within a network
US20070198470A1 (en) * 2006-01-27 2007-08-23 Gordon Freedman Method of reducing search space complexity using suggested search terms with display of an associated reduction factor
US8150827B2 (en) * 2006-06-07 2012-04-03 Renew Data Corp. Methods for enhancing efficiency and cost effectiveness of first pass review of documents
WO2007146100A2 (en) * 2006-06-07 2007-12-21 Cnet Networks, Inc. Evaluative information system and method
GB0611303D0 (en) * 2006-06-08 2006-07-19 Ibm A method, apparatus and software for selecting terms for a glossary in a document processing system
US7529740B2 (en) * 2006-08-14 2009-05-05 International Business Machines Corporation Method and apparatus for organizing data sources
US8442972B2 (en) 2006-10-11 2013-05-14 Collarity, Inc. Negative associations for search results ranking and refinement
US8600922B2 (en) * 2006-10-13 2013-12-03 Elsevier Inc. Methods and systems for knowledge discovery
WO2008055034A2 (en) * 2006-10-30 2008-05-08 Noblis, Inc. Method and system for personal information extraction and modeling with fully generalized extraction contexts
WO2008061002A2 (en) * 2006-11-14 2008-05-22 Networked Insights, Inc. Method and system for automatically identifying users to participate in an electronic conversation
US8060390B1 (en) 2006-11-24 2011-11-15 Voices Heard Media, Inc. Computer based method for generating representative questions from an audience
US8676802B2 (en) 2006-11-30 2014-03-18 Oracle Otc Subsidiary Llc Method and system for information retrieval with clustering
US8280877B2 (en) * 2007-02-22 2012-10-02 Microsoft Corporation Diverse topic phrase extraction
US7925655B1 (en) 2007-03-30 2011-04-12 Google Inc. Query scheduling using hierarchical tiers of index servers
US7693813B1 (en) 2007-03-30 2010-04-06 Google Inc. Index server architecture using tiered and sharded phrase posting lists
US8166021B1 (en) 2007-03-30 2012-04-24 Google Inc. Query phrasification
US8166045B1 (en) 2007-03-30 2012-04-24 Google Inc. Phrase extraction using subphrase scoring
US8086594B1 (en) 2007-03-30 2011-12-27 Google Inc. Bifurcated document relevance scoring
US7702614B1 (en) 2007-03-30 2010-04-20 Google Inc. Index updating using segment swapping
US8504553B2 (en) * 2007-04-19 2013-08-06 Barnesandnoble.Com Llc Unstructured and semistructured document processing and searching
US20080255925A1 (en) * 2007-04-16 2008-10-16 Aditya Vailaya Systems and methods for generating value-based information
WO2008130501A1 (en) * 2007-04-16 2008-10-30 Retrevo, Inc. Unstructured and semistructured document processing and searching and generation of value-based information
US7917493B2 (en) 2007-04-19 2011-03-29 Retrevo Inc. Indexing and searching product identifiers
US8290967B2 (en) 2007-04-19 2012-10-16 Barnesandnoble.Com Llc Indexing and search query processing
US7853589B2 (en) * 2007-04-30 2010-12-14 Microsoft Corporation Web spam page classification using query-dependent data
US8793264B2 (en) * 2007-07-18 2014-07-29 Hewlett-Packard Development Company, L. P. Determining a subset of documents from which a particular document was derived
WO2009026140A2 (en) 2007-08-16 2009-02-26 Hollingsworth William A Automatic text skimming using lexical chains
US8117223B2 (en) 2007-09-07 2012-02-14 Google Inc. Integrating external related phrase information into a phrase-based indexing information retrieval system
US7856434B2 (en) 2007-11-12 2010-12-21 Endeca Technologies, Inc. System and method for filtering rules for manipulating search results in a hierarchical search and navigation system
US8615490B1 (en) 2008-01-31 2013-12-24 Renew Data Corp. Method and system for restoring information from backup storage media
US7925743B2 (en) * 2008-02-29 2011-04-12 Networked Insights, Llc Method and system for qualifying user engagement with a website
US7904445B2 (en) * 2008-03-26 2011-03-08 The Go Daddy Group, Inc. Displaying concept-based search results
US8069187B2 (en) * 2008-03-26 2011-11-29 The Go Daddy Group, Inc. Suggesting concept-based top-level domain names
US20090248736A1 (en) * 2008-03-26 2009-10-01 The Go Daddy Group, Inc. Displaying concept-based targeted advertising
US7962438B2 (en) * 2008-03-26 2011-06-14 The Go Daddy Group, Inc. Suggesting concept-based domain names
GB2463515A (en) * 2008-04-23 2010-03-24 British Telecomm Classification of online posts using keyword clusters derived from existing posts
GB2459476A (en) * 2008-04-23 2009-10-28 British Telecomm Classification of posts for prioritizing or grouping comments.
US8577884B2 (en) * 2008-05-13 2013-11-05 The Boeing Company Automated analysis and summarization of comments in survey response data
US20090313363A1 (en) * 2008-06-17 2009-12-17 The Go Daddy Group, Inc. Hosting a remote computer in a hosting data center
US8438178B2 (en) 2008-06-26 2013-05-07 Collarity Inc. Interactions among online digital identities
EP2374099A1 (en) * 2008-12-30 2011-10-12 Telecom Italia S.p.A. Method and system of content recommendation
US20100235314A1 (en) * 2009-02-12 2010-09-16 Decisive Analytics Corporation Method and apparatus for analyzing and interrelating video data
US8458105B2 (en) * 2009-02-12 2013-06-04 Decisive Analytics Corporation Method and apparatus for analyzing and interrelating data
US8527500B2 (en) * 2009-02-27 2013-09-03 Red Hat, Inc. Preprocessing text to enhance statistical features
US8396850B2 (en) * 2009-02-27 2013-03-12 Red Hat, Inc. Discriminating search results by phrase analysis
US20100274807A1 (en) * 2009-04-23 2010-10-28 Frank John Williams Method and system for representing information
US10891659B2 (en) 2009-05-29 2021-01-12 Red Hat, Inc. Placing resources in displayed web pages via context modeling
US8312029B2 (en) * 2009-05-29 2012-11-13 Peter Snell System and related method for digital attitude mapping
US8972424B2 (en) 2009-05-29 2015-03-03 Peter Snell Subjective linguistic analysis
US8478749B2 (en) * 2009-07-20 2013-07-02 Lexisnexis, A Division Of Reed Elsevier Inc. Method and apparatus for determining relevant search results using a matrix framework
US8386410B2 (en) * 2009-07-22 2013-02-26 International Business Machines Corporation System and method for semantic information extraction framework for integrated systems management
BR112012006743A2 (en) * 2009-09-26 2019-09-24 Ogilvy Hamish method for indexing a plurality of documents, system for indexing a plurality of documents, method for parsing a portion of text and retrieving documents relevant to the text portion, method for refining search results, system for refining search results , system for parsing a portion of input text and retrieving documents relevant to the portion of text, and computer readable media
WO2011075610A1 (en) 2009-12-16 2011-06-23 Renew Data Corp. System and method for creating a de-duplicated data set
US8875038B2 (en) 2010-01-19 2014-10-28 Collarity, Inc. Anchoring for content synchronization
CN102141978A (en) 2010-02-02 2011-08-03 阿里巴巴集团控股有限公司 Method and system for classifying texts
US8490056B2 (en) * 2010-04-28 2013-07-16 International Business Machines Corporation Automatic identification of subroutines from test scripts
JP5894149B2 (en) * 2010-06-03 2016-03-23 トムソン ライセンシングThomson Licensing Enhancement of meaning using TOP-K processing
US8375061B2 (en) * 2010-06-08 2013-02-12 International Business Machines Corporation Graphical models for representing text documents for computer analysis
US20120047172A1 (en) * 2010-08-23 2012-02-23 Google Inc. Parallel document mining
US9652551B2 (en) * 2010-08-31 2017-05-16 Disney Enterprises, Inc. Automated effort judgement of user generated content
US8346772B2 (en) 2010-09-16 2013-01-01 International Business Machines Corporation Systems and methods for interactive clustering
JP5512489B2 (en) * 2010-10-27 2014-06-04 株式会社日立ソリューションズ File management apparatus and file management method
WO2012127968A1 (en) * 2011-03-23 2012-09-27 日本電気株式会社 Event analysis device, event analysis method, and computer-readable recording medium
US9870376B2 (en) * 2011-04-01 2018-01-16 Excalibur Ip, Llc Method and system for concept summarization
US9454528B2 (en) 2011-10-17 2016-09-27 Xerox Corporation Method and system for creating ordered reading lists from unstructured document sets
US20150012263A1 (en) * 2011-12-07 2015-01-08 Dw Associates, Llc System and method for semantic analysis of candidate information to determine compatibility
US8949111B2 (en) * 2011-12-14 2015-02-03 Brainspace Corporation System and method for identifying phrases in text
US9015160B2 (en) * 2011-12-14 2015-04-21 Brainspace Corporation Multi-concept latent semantic analysis queries
US8886651B1 (en) 2011-12-22 2014-11-11 Reputation.Com, Inc. Thematic clustering
US9075498B1 (en) * 2011-12-22 2015-07-07 Symantec Corporation User interface for finding similar documents
US9501566B1 (en) 2012-01-17 2016-11-22 Veritas Technologies Llc User interface for transparent concept search
US8909648B2 (en) * 2012-01-18 2014-12-09 Technion Research & Development Foundation Limited Methods and systems of supervised learning of semantic relatedness
US20130212093A1 (en) 2012-02-15 2013-08-15 International Business Machines Corporation Generating visualizations of a display group of tags representing content instances in objects satisfying a search criteria
US9360982B2 (en) 2012-05-01 2016-06-07 International Business Machines Corporation Generating visualizations of facet values for facets defined over a collection of objects
US9064491B2 (en) * 2012-05-29 2015-06-23 Nuance Communications, Inc. Methods and apparatus for performing transformation techniques for data clustering and/or classification
US9412366B2 (en) 2012-09-18 2016-08-09 Adobe Systems Incorporated Natural language image spatial and tonal localization
US9588964B2 (en) * 2012-09-18 2017-03-07 Adobe Systems Incorporated Natural language vocabulary generation and usage
US10656808B2 (en) 2012-09-18 2020-05-19 Adobe Inc. Natural language and user interface controls
US9104710B2 (en) 2013-03-15 2015-08-11 Src, Inc. Method for cross-domain feature correlation
US9805018B1 (en) 2013-03-15 2017-10-31 Steven E. Richfield Natural language processing for analyzing internet content and finding solutions to needs expressed in text
US9501506B1 (en) 2013-03-15 2016-11-22 Google Inc. Indexing system
US9122681B2 (en) 2013-03-15 2015-09-01 Gordon Villy Cormack Systems and methods for classifying electronic information using advanced active learning techniques
US8788263B1 (en) * 2013-03-15 2014-07-22 Steven E. Richfield Natural language processing for analyzing internet content and finding solutions to needs expressed in text
WO2014176754A1 (en) * 2013-04-30 2014-11-06 Hewlett-Packard Development Company, L.P. Histogram construction for string data
US9483568B1 (en) 2013-06-05 2016-11-01 Google Inc. Indexing system
IN2013MU02217A (en) 2013-07-01 2015-06-12 Tata Consultancy Services Ltd
US9058374B2 (en) * 2013-09-26 2015-06-16 International Business Machines Corporation Concept driven automatic section identification
US20150142679A1 (en) * 2013-11-15 2015-05-21 Adobe Systems Incorporated Provisioning rules to manage user entitlements
CN104679778B (en) * 2013-11-29 2019-03-26 腾讯科技(深圳)有限公司 A kind of generation method and device of search result
US10356032B2 (en) 2013-12-26 2019-07-16 Palantir Technologies Inc. System and method for detecting confidential information emails
US8837835B1 (en) 2014-01-20 2014-09-16 Array Technology, LLC Document grouping system
WO2015117074A1 (en) * 2014-01-31 2015-08-06 Global Security Information Analysts, LLC Document relationship analysis system
US11803561B1 (en) * 2014-03-31 2023-10-31 Amazon Technologies, Inc. Approximation query
US9660933B2 (en) 2014-04-17 2017-05-23 Go Daddy Operating Company, LLC Allocating and accessing hosting server resources via continuous resource availability updates
US9501211B2 (en) 2014-04-17 2016-11-22 GoDaddy Operating Company, LLC User input processing for allocation of hosting server resources
US10127229B2 (en) 2014-04-23 2018-11-13 Elsevier B.V. Methods and computer-program products for organizing electronic documents
US9535974B1 (en) 2014-06-30 2017-01-03 Palantir Technologies Inc. Systems and methods for identifying key phrase clusters within documents
US9619557B2 (en) 2014-06-30 2017-04-11 Palantir Technologies, Inc. Systems and methods for key phrase characterization of documents
US9256664B2 (en) 2014-07-03 2016-02-09 Palantir Technologies Inc. System and method for news events detection and visualization
US10013467B1 (en) * 2014-07-10 2018-07-03 Purdue Pharma L.P. System and method for evaluating chemical entities using and applying a virtual landscape
US10296616B2 (en) * 2014-07-31 2019-05-21 Splunk Inc. Generation of a search query to approximate replication of a cluster of events
WO2016028770A1 (en) 2014-08-18 2016-02-25 HavenLock Inc. Improved locking apparatus, locking member, and method of use
US10353961B2 (en) 2015-06-19 2019-07-16 Gordon V. Cormack Systems and methods for conducting and terminating a technology-assisted review
US10254934B2 (en) 2015-08-01 2019-04-09 Splunk Inc. Network security investigation workflow logging
US9516052B1 (en) 2015-08-01 2016-12-06 Splunk Inc. Timeline displays of network security investigation events
US9363149B1 (en) 2015-08-01 2016-06-07 Splunk Inc. Management console for network security investigations
US10474672B2 (en) * 2015-08-25 2019-11-12 Schlafender Hase GmbH Software & Communications Method for comparing text files with differently arranged text sections in documents
US10275426B1 (en) * 2015-09-22 2019-04-30 Amazon Technologies, Inc. Dynamic kerning pair reduction for digital font rendering
US10095747B1 (en) 2016-06-06 2018-10-09 @Legal Discovery LLC Similar document identification using artificial intelligence
US10776399B1 (en) 2016-06-06 2020-09-15 Casepoint LLC Document classification prediction and content analytics using artificial intelligence
US10372743B2 (en) * 2016-07-20 2019-08-06 Baidu Usa Llc Systems and methods for homogeneous entity grouping
US10444945B1 (en) * 2016-10-10 2019-10-15 United Services Automobile Association Systems and methods for ingesting and parsing datasets generated from disparate data sources
US10733224B2 (en) * 2017-02-07 2020-08-04 International Business Machines Corporation Automatic corpus selection and halting condition detection for semantic asset expansion
US10204100B2 (en) 2017-03-28 2019-02-12 International Business Machines Corporation Lexicon extraction from non-parallel data
US11113308B1 (en) * 2017-07-13 2021-09-07 Groupon, Inc. Method, apparatus, and computer program product for improving network database functionalities
CN107402994B (en) * 2017-07-17 2021-01-19 云润大数据服务有限公司 Method and device for classifying multi-group hierarchical division
CN107562843B (en) * 2017-08-25 2020-07-14 贵州耕云科技有限公司 News hot phrase extraction method based on title high-frequency segmentation
US10657145B2 (en) * 2017-12-18 2020-05-19 International Business Machines Corporation Clustering facets on a two-dimensional facet cube for text mining
JP7006403B2 (en) * 2018-03-14 2022-01-24 富士通株式会社 Clustering program, clustering method and clustering device
WO2019204086A1 (en) * 2018-04-18 2019-10-24 HelpShift, Inc. System and methods for processing and interpreting text messages
US11163952B2 (en) 2018-07-11 2021-11-02 International Business Machines Corporation Linked data seeded multi-lingual lexicon extraction
US10831793B2 (en) 2018-10-23 2020-11-10 International Business Machines Corporation Learning thematic similarity metric from article text units
US10909320B2 (en) * 2019-02-07 2021-02-02 International Business Machines Corporation Ontology-based document analysis and annotation generation
US11829386B2 (en) 2020-01-30 2023-11-28 HG Insights, Inc. Identifying anonymized resume corpus data pertaining to the same individual
CN111898366B (en) * 2020-07-29 2022-08-09 平安科技(深圳)有限公司 Document subject word aggregation method and device, computer equipment and readable storage medium
US11797486B2 (en) 2022-01-03 2023-10-24 Bank Of America Corporation File de-duplication for a distributed database
US20230252067A1 (en) * 2022-02-10 2023-08-10 International Business Machines Corporation Ranking explanatory variables in multivariate analysis

Citations (187)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US3416150A (en) 1965-08-02 1968-12-10 Data Disc Inc Magnetic recording disc cartridge
US3426210A (en) 1965-12-22 1969-02-04 Rca Corp Control circuit for automatically quantizing signals at desired levels
US3668658A (en) 1969-12-22 1972-06-06 Ibm Magnetic record disk cover
US4893253A (en) 1988-03-10 1990-01-09 Indiana University Foundation Method for analyzing intact capsules and tablets by near-infrared reflectance spectrometry
US5056021A (en) 1989-06-08 1991-10-08 Carolyn Ausborn Method and apparatus for abstracting concepts from natural language
US5121338A (en) 1988-03-10 1992-06-09 Indiana University Foundation Method for detecting subpopulations in spectral analysis
US5133067A (en) 1985-10-09 1992-07-21 Hitachi, Ltd. Method and apparatus for system for selectively extracting display data within a specified proximity of a displayed character string using a range table
US5278980A (en) 1991-08-16 1994-01-11 Xerox Corporation Iterative technique for phrase query formation and an information retrieval system employing same
US5371673A (en) 1987-04-06 1994-12-06 Fan; David P. Information processing analysis system for sorting and scoring text
US5442778A (en) 1991-11-12 1995-08-15 Xerox Corporation Scatter-gather: a cluster-based method and apparatus for browsing large document collections
US5477451A (en) 1991-07-25 1995-12-19 International Business Machines Corp. Method and system for natural language translation
US5488725A (en) 1991-10-08 1996-01-30 West Publishing Company System of document representation retrieval by successive iterated probability sampling
US5524177A (en) 1992-07-03 1996-06-04 Kabushiki Kaisha Toshiba Learning of associative memory in form of neural network suitable for connectionist model
US5528735A (en) 1993-03-23 1996-06-18 Silicon Graphics Inc. Method and apparatus for displaying data within a three-dimensional information landscape
US5619632A (en) 1994-09-14 1997-04-08 Xerox Corporation Displaying node-link structure with region of greater spacings and peripheral branches
US5619709A (en) 1993-09-20 1997-04-08 Hnc, Inc. System and method of context vector generation and retrieval
US5635929A (en) 1995-02-13 1997-06-03 Hughes Aircraft Company Low bit rate video encoder and decoder
US5649193A (en) 1993-03-12 1997-07-15 Kabushiki Kaisha Toshiba Document detection system using detection result presentation for facilitating user's comprehension
US5675819A (en) 1994-06-16 1997-10-07 Xerox Corporation Document information retrieval using global word co-occurrence patterns
US5696962A (en) 1993-06-24 1997-12-09 Xerox Corporation Method for computerized information retrieval using shallow linguistic analysis
US5737734A (en) 1995-09-15 1998-04-07 Infonautics Corporation Query word relevance adjustment in a search of an information retrieval system
US5754938A (en) 1994-11-29 1998-05-19 Herz; Frederick S. M. Pseudonymous server for system for customized electronic identification of desirable objects
US5787422A (en) * 1996-01-11 1998-07-28 Xerox Corporation Method and apparatus for information accesss employing overlapping clusters
US5794236A (en) 1996-05-29 1998-08-11 Lexis-Nexis Computer-based system for classifying documents into a hierarchy and linking the classifications to the hierarchy
US5799276A (en) 1995-11-07 1998-08-25 Accent Incorporated Knowledge-based speech recognition system and methods having frame length computed based upon estimated pitch period of vocalic intervals
US5819258A (en) 1997-03-07 1998-10-06 Digital Equipment Corporation Method and apparatus for automatically generating hierarchical categories from large document collections
US5842203A (en) 1995-12-01 1998-11-24 International Business Machines Corporation Method and system for performing non-boolean search queries in a graphical user interface
US5844991A (en) 1995-08-07 1998-12-01 The Regents Of The University Of California Script identification from images using cluster-based templates
US5857179A (en) 1996-09-09 1999-01-05 Digital Equipment Corporation Computer method and apparatus for clustering documents and automatic generation of cluster keywords
US5860136A (en) 1989-06-16 1999-01-12 Fenner; Peter R. Method and apparatus for use of associated memory with large key spaces
US5862325A (en) 1996-02-29 1999-01-19 Intermind Corporation Computer-based communication system and method using metadata defining a control structure
US5864871A (en) 1996-06-04 1999-01-26 Multex Systems Information delivery system and method including on-line entitlements
US5864846A (en) 1996-06-28 1999-01-26 Siemens Corporate Research, Inc. Method for facilitating world wide web searches utilizing a document distribution fusion strategy
US5867799A (en) 1996-04-04 1999-02-02 Lang; Andrew K. Information system and method for filtering a massive flow of information entities to meet user information classification needs
US5870740A (en) 1996-09-30 1999-02-09 Apple Computer, Inc. System and method for improving the ranking of information retrieval results for short queries
US5909677A (en) 1996-06-18 1999-06-01 Digital Equipment Corporation Method for determining the resemblance of documents
US5915024A (en) 1996-06-18 1999-06-22 Kabushiki Kaisha Toshiba Electronic signature addition method, electronic signature verification method, and system and computer program product using these methods
US5920854A (en) 1996-08-14 1999-07-06 Infoseek Corporation Real-time document collection search engine with phrase indexing
US5924105A (en) 1997-01-27 1999-07-13 Michigan State University Method and product for determining salient features for use in information searching
US5940821A (en) 1997-05-21 1999-08-17 Oracle Corporation Information presentation in a knowledge base search and retrieval system
US5950189A (en) 1997-01-02 1999-09-07 At&T Corp Retrieval system and method
US5950146A (en) 1996-10-04 1999-09-07 At & T Corp. Support vector method for function estimation
US5966126A (en) 1996-12-23 1999-10-12 Szabo; Andrew J. Graphic user interface for database system
US5987446A (en) 1996-11-12 1999-11-16 U.S. West, Inc. Searching large collections of text using multiple search engines concurrently
US6006221A (en) 1995-08-16 1999-12-21 Syracuse University Multilingual document retrieval system and method using semantic vector matching
US6012053A (en) 1997-06-23 2000-01-04 Lycos, Inc. Computer system with user-controlled relevance ranking of search results
US6026397A (en) 1996-05-22 2000-02-15 Electronic Data Systems Corporation Data analysis system and method
US6038574A (en) 1998-03-18 2000-03-14 Xerox Corporation Method and apparatus for clustering a collection of linked documents using co-citation analysis
US6070133A (en) 1997-07-21 2000-05-30 Battelle Memorial Institute Information retrieval system utilizing wavelet transform
US6089742A (en) 1989-11-01 2000-07-18 Warmerdam; Thomas P. H. Method and apparatus for controlling robots and the like using a bubble data hierarchy placed along a medial axis
US6092059A (en) 1996-12-27 2000-07-18 Cognex Corporation Automatic classifier for real time inspection and classification
US6092091A (en) * 1996-09-13 2000-07-18 Kabushiki Kaisha Toshiba Device and method for filtering information, device and method for monitoring updated document information and information storage medium used in same devices
US6094649A (en) 1997-12-22 2000-07-25 Partnet, Inc. Keyword searches of structured databases
US6100901A (en) 1998-06-22 2000-08-08 International Business Machines Corporation Method and apparatus for cluster exploration and visualization
US6119124A (en) 1998-03-26 2000-09-12 Digital Equipment Corporation Method for clustering closely resembling data objects
US6122628A (en) 1997-10-31 2000-09-19 International Business Machines Corporation Multidimensional data clustering and dimension reduction for indexing and searching
US6137911A (en) 1997-06-16 2000-10-24 The Dialog Corporation Plc Test classification system and method
US6137499A (en) 1997-03-07 2000-10-24 Silicon Graphics, Inc. Method, system, and computer program product for visualizing data using partial hierarchies
US6137545A (en) 1994-06-28 2000-10-24 Samsung Electronics Co., Ltd. Reception mode control in radio receivers for receiving both VSB and QAM digital television signals
EP1049030A1 (en) 1999-04-28 2000-11-02 SER Systeme AG Produkte und Anwendungen der Datenverarbeitung Classification method and apparatus
US6148102A (en) 1997-05-29 2000-11-14 Adobe Systems Incorporated Recognizing text in a multicolor image
US6154219A (en) 1997-12-01 2000-11-28 Microsoft Corporation System and method for optimally placing labels on a map
US6167368A (en) 1998-08-14 2000-12-26 The Trustees Of Columbia University In The City Of New York Method and system for indentifying significant topics of a document
US6173275B1 (en) 1993-09-20 2001-01-09 Hnc Software, Inc. Representation and retrieval of images using context vectors derived from image information elements
US6202064B1 (en) 1997-06-20 2001-03-13 Xerox Corporation Linguistic search system
US6216123B1 (en) 1998-06-24 2001-04-10 Novell, Inc. Method and system for rapid retrieval in a full text indexing system
US6243713B1 (en) 1998-08-24 2001-06-05 Excalibur Technologies Corp. Multimedia document retrieval by application of multimedia queries to a unified index of multimedia data for a plurality of multimedia data types
US6243724B1 (en) 1992-04-30 2001-06-05 Apple Computer, Inc. Method and apparatus for organizing information in a computer system
US6260038B1 (en) 1999-09-13 2001-07-10 International Businemss Machines Corporation Clustering mixed attribute patterns
US6338062B1 (en) 1998-09-28 2002-01-08 Fuji Xerox Co., Ltd. Retrieval system, retrieval method and computer readable recording medium that records retrieval program
US6345243B1 (en) 1998-05-27 2002-02-05 Lionbridge Technologies, Inc. System, method, and product for dynamically propagating translations in a translation-memory system
US6349307B1 (en) 1998-12-28 2002-02-19 U.S. Philips Corporation Cooperative topical servers with automatic prefiltering and routing
US20020032735A1 (en) 2000-08-25 2002-03-14 Daniel Burnstein Apparatus, means and methods for automatic community formation for phones and computer networks
US6360227B1 (en) 1999-01-29 2002-03-19 International Business Machines Corporation System and method for generating taxonomies with applications to content-based recommendations
US6377287B1 (en) 1999-04-19 2002-04-23 Hewlett-Packard Company Technique for visualizing large web-based hierarchical hyperbolic space with multi-paths
US6381601B1 (en) 1998-12-22 2002-04-30 Hitachi, Ltd. Grouping and duplicate removal method in a database
US6389436B1 (en) 1997-12-15 2002-05-14 International Business Machines Corporation Enhanced hypertext categorization using hyperlinks
US6389433B1 (en) 1999-07-16 2002-05-14 Microsoft Corporation Method and system for automatically merging files into a single instance store
US20020065912A1 (en) 2000-11-30 2002-05-30 Catchpole Lawrence W. Web session collaboration
US6408294B1 (en) 1999-03-31 2002-06-18 Verizon Laboratories Inc. Common term optimization
US20020078090A1 (en) 2000-06-30 2002-06-20 Hwang Chung Hee Ontological concept-based, user-centric text summarization
US6414677B1 (en) 1998-09-14 2002-07-02 Microsoft Corporation Methods, apparatus and data structures for providing a user interface, which exploits spatial memory in three-dimensions, to objects and which visually groups proximally located objects
US6415283B1 (en) 1998-10-13 2002-07-02 Orack Corporation Methods and apparatus for determining focal points of clusters in a tree structure
US6418431B1 (en) 1998-03-30 2002-07-09 Microsoft Corporation Information retrieval and speech recognition based on language models
US6421709B1 (en) 1997-12-22 2002-07-16 Accepted Marketing, Inc. E-mail filter and method thereof
US6438564B1 (en) 1998-06-17 2002-08-20 Microsoft Corporation Method for associating a discussion with a document
US6438537B1 (en) 1999-06-22 2002-08-20 Microsoft Corporation Usage based aggregation optimization
US6442592B1 (en) 1998-12-11 2002-08-27 Micro Computer Systems, Inc. Message center system
US6446061B1 (en) 1998-07-31 2002-09-03 International Business Machines Corporation Taxonomy generation for document collections
US20020122543A1 (en) 2001-02-12 2002-09-05 Rowen Chris E. System and method of indexing unique electronic mail messages and uses for the same
US6449612B1 (en) 1998-03-17 2002-09-10 Microsoft Corporation Varying cluster number in a scalable clustering system for use with large databases
US6453327B1 (en) 1996-06-10 2002-09-17 Sun Microsystems, Inc. Method and apparatus for identifying and discarding junk electronic mail
US6460034B1 (en) 1997-05-21 2002-10-01 Oracle Corporation Document knowledge base research and retrieval system
US6470307B1 (en) 1997-06-23 2002-10-22 National Research Council Of Canada Method and apparatus for automatically identifying keywords within a document
US6480843B2 (en) 1998-11-03 2002-11-12 Nec Usa, Inc. Supporting web-query expansion efficiently using multi-granularity indexing and query processing
US6480885B1 (en) 1998-09-15 2002-11-12 Michael Olivier Dynamically matching users for group communications based on a threshold degree of matching of sender and recipient predetermined acceptance criteria
US6484168B1 (en) 1996-09-13 2002-11-19 Battelle Memorial Institute System for information discovery
US6484196B1 (en) 1998-03-20 2002-11-19 Advanced Web Solutions Internet messaging system and method for use in computer networks
US20020184193A1 (en) 2001-05-30 2002-12-05 Meir Cohen Method and system for performing a similarity search using a dissimilarity based indexing structure
US6493703B1 (en) 1999-05-11 2002-12-10 Prophet Financial Systems System and method for implementing intelligent online community message board
US6496822B2 (en) 1999-04-12 2002-12-17 Micron Technology, Inc. Methods of providing computer systems with bundled access to restricted-access databases
US6502081B1 (en) 1999-08-06 2002-12-31 Lexis Nexis System and method for classifying legal concepts using legal topic scheme
US6507847B1 (en) 1999-12-17 2003-01-14 Openwave Systems Inc. History database structure for Usenet
US6510406B1 (en) 1999-03-23 2003-01-21 Mathsoft, Inc. Inverse inference engine for high performance web search
US6519580B1 (en) 2000-06-08 2003-02-11 International Business Machines Corporation Decision-tree-based symbolic rule induction system for text categorization
US6523026B1 (en) 1999-02-08 2003-02-18 Huntsman International Llc Method for retrieving semantically distant analogies
US6523063B1 (en) 1999-08-30 2003-02-18 Zaplet, Inc. Method system and program product for accessing a file using values from a redirect message string for each change of the link identifier
US6542889B1 (en) 2000-01-28 2003-04-01 International Business Machines Corporation Methods and apparatus for similarity text search based on conceptual indexing
US6544123B1 (en) 1999-10-29 2003-04-08 Square Co., Ltd. Game apparatus, command input method for video game and computer-readable recording medium recording programs for realizing the same
US6549957B1 (en) 1998-12-22 2003-04-15 International Business Machines Corporation Apparatus for preventing automatic generation of a chain reaction of messages if a prior extracted message is similar to current processed message
US6560597B1 (en) 2000-03-21 2003-05-06 International Business Machines Corporation Concept decomposition using clustering
US6571225B1 (en) 2000-02-11 2003-05-27 International Business Machines Corporation Text categorizers based on regularizing adaptations of the problem of computing linear separators
US6584564B2 (en) 2000-04-25 2003-06-24 Sigaba Corporation Secure e-mail system
US20030130991A1 (en) 2001-03-28 2003-07-10 Fidel Reijerse Knowledge discovery from data sets
US6594658B2 (en) 1995-07-07 2003-07-15 Sun Microsystems, Inc. Method and apparatus for generating query responses in a computer-based document retrieval system
US6598054B2 (en) 1999-01-26 2003-07-22 Xerox Corporation System and method for clustering data objects in a collection
US6606625B1 (en) 1999-06-03 2003-08-12 University Of Southern California Wrapper induction by hierarchical data analysis
US6611825B1 (en) 1999-06-09 2003-08-26 The Boeing Company Method and system for text mining using multidimensional subspaces
US6629097B1 (en) 1999-04-28 2003-09-30 Douglas K. Keith Displaying implicit associations among items in loosely-structured data sets
US6628304B2 (en) 1998-12-09 2003-09-30 Cisco Technology, Inc. Method and apparatus providing a graphical user interface for representing and navigating hierarchical networks
EP0886227B1 (en) 1997-06-16 2003-10-01 Compaq Computer Corporation Full-text indexed mail repository
US6651057B1 (en) 1999-09-03 2003-11-18 Bbnt Solutions Llc Method and apparatus for score normalization for information retrieval applications
US6654739B1 (en) 2000-01-31 2003-11-25 International Business Machines Corporation Lightweight document clustering
US6658423B1 (en) 2001-01-24 2003-12-02 Google, Inc. Detecting duplicate and near-duplicate files
US6675164B2 (en) 2001-06-08 2004-01-06 The Regents Of The University Of California Parallel object-oriented data mining system
US6675159B1 (en) 2000-07-27 2004-01-06 Science Applic Int Corp Concept-based search and retrieval system
US6678705B1 (en) 1998-11-16 2004-01-13 At&T Corp. System for archiving electronic documents using messaging groupware
US6684205B1 (en) 2000-10-18 2004-01-27 International Business Machines Corporation Clustering hypertext with applications to web searching
US6697998B1 (en) 2000-06-12 2004-02-24 International Business Machines Corporation Automatic labeling of unlabeled text data
US6701305B1 (en) 1999-06-09 2004-03-02 The Boeing Company Methods, apparatus and computer program products for information retrieval and document classification utilizing a multidimensional subspace
US6711585B1 (en) 1999-06-15 2004-03-23 Kanisa Inc. System and method for implementing a knowledge management system
US6714929B1 (en) 2001-04-13 2004-03-30 Auguri Corporation Weighted preference data search system and method
US6735578B2 (en) 2001-05-10 2004-05-11 Honeywell International Inc. Indexing of knowledge base in multilayer self-organizing maps with hessian and perturbation induced fast learning
US6738759B1 (en) 2000-07-07 2004-05-18 Infoglide Corporation, Inc. System and method for performing similarity searching using pointer optimization
US6747646B2 (en) 1999-07-16 2004-06-08 International Business Machines Corporation System and method for fusing three-dimensional shape data on distorted images without correcting for distortion
US6751628B2 (en) 2001-01-11 2004-06-15 Dolphin Search Process and system for sparse vector and matrix representation of document indexing and retrieval
US6757646B2 (en) 2000-03-22 2004-06-29 Insightful Corporation Extended functionality for an inverse inference engine based web search
US6785679B1 (en) 2000-03-29 2004-08-31 Brassring, Llc Method and apparatus for sending and tracking resume data sent via URL
US6789230B2 (en) * 1998-10-09 2004-09-07 Microsoft Corporation Creating a summary having sentences with the highest weight, and lowest length
US6804665B2 (en) 2001-04-18 2004-10-12 International Business Machines Corporation Method and apparatus for discovering knowledge gaps between problems and solutions in text databases
US6816175B1 (en) 1998-12-19 2004-11-09 International Business Machines Corporation Orthogonal browsing in object hierarchies
US6819344B2 (en) 2001-03-12 2004-11-16 Microsoft Corporation Visualization of multi-dimensional data having an unbounded dimension
US6823333B2 (en) 2001-03-02 2004-11-23 The United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration System, method and apparatus for conducting a keyterm search
US6841321B2 (en) 2001-01-26 2005-01-11 Hitachi, Ltd. Method and system for processing a semi-conductor device
US6862710B1 (en) 1999-03-23 2005-03-01 Insightful Corporation Internet navigation using soft hyperlinks
US6879332B2 (en) 2000-05-16 2005-04-12 Groxis, Inc. User interface for displaying and exploring hierarchical information
US6883001B2 (en) 2000-05-26 2005-04-19 Fujitsu Limited Document information search apparatus and method and recording medium storing document information search program therein
US6888584B2 (en) 1998-06-29 2005-05-03 Hitachi, Ltd. Liquid crystal display device
US6915308B1 (en) 2000-04-06 2005-07-05 Claritech Corporation Method and apparatus for information mining and filtering
US6922699B2 (en) 1999-01-26 2005-07-26 Xerox Corporation System and method for quantitatively representing data objects in vector space
US6941325B1 (en) 1999-02-01 2005-09-06 The Trustees Of Columbia University Multimedia archive description scheme
US6970881B1 (en) 2001-05-07 2005-11-29 Intelligenxia, Inc. Concept-based method and system for dynamically analyzing unstructured information
US6978419B1 (en) 2000-11-15 2005-12-20 Justsystem Corporation Method and apparatus for efficient identification of duplicate and near-duplicate documents and text spans using high-discriminability text fragments
US6990238B1 (en) 1999-09-30 2006-01-24 Battelle Memorial Institute Data processing, analysis, and visualization system for use with disparate data types
US7003551B2 (en) 2000-11-30 2006-02-21 Bellsouth Intellectual Property Corp. Method and apparatus for minimizing storage of common attachment files in an e-mail communications server
US7013435B2 (en) 2000-03-17 2006-03-14 Vizible.Com Inc. Three dimensional spatial user interface
US7020645B2 (en) 2001-04-19 2006-03-28 Eoriginal, Inc. Systems and methods for state-less authentication
US7054870B2 (en) 2000-11-15 2006-05-30 Kooltorch, Llc Apparatus and methods for organizing and/or presenting data
US7080320B2 (en) 2001-01-29 2006-07-18 Kabushiki Kaisha Toshiba Translation apparatus and method
US7096431B2 (en) 2001-08-31 2006-08-22 Sony Corporation Menu display apparatus and menu display method
US7099819B2 (en) 2000-07-25 2006-08-29 Kabushiki Kaisha Toshiba Text information analysis apparatus and method
US7117246B2 (en) 2000-02-22 2006-10-03 Sendmail, Inc. Electronic mail system with methodology providing distributed message store
US7130807B1 (en) 1999-11-22 2006-10-31 Accenture Llp Technology sharing during demand and supply planning in a network-based supply chain environment
US7137075B2 (en) 1998-08-24 2006-11-14 Hitachi, Ltd. Method of displaying, a method of processing, an apparatus for processing, and a system for processing multimedia information
US7155668B2 (en) 2001-04-19 2006-12-26 International Business Machines Corporation Method and system for identifying relationships between text documents and structured variables pertaining to the text documents
US7194483B1 (en) 2001-05-07 2007-03-20 Intelligenxia, Inc. Method, system, and computer program product for concept-based multi-dimensional analysis of unstructured information
US7209949B2 (en) 1998-05-29 2007-04-24 Research In Motion Limited System and method for synchronizing information between a host system and a mobile data communication device
US7233886B2 (en) 2001-01-19 2007-06-19 Smartsignal Corporation Adaptive modeling of changed states in predictive condition monitoring
US7240199B2 (en) 2000-12-06 2007-07-03 Rpost International Limited System and method for verifying delivery and integrity of electronic messages
US7251637B1 (en) 1993-09-20 2007-07-31 Fair Isaac Corporation Context vector generation and retrieval
US7266365B2 (en) 1998-05-29 2007-09-04 Research In Motion Limited System and method for delayed transmission of bundled command messages
US7266545B2 (en) 2001-08-07 2007-09-04 International Business Machines Corporation Methods and apparatus for indexing in a database and for retrieving data from a database in accordance with queries using example sets
US7277919B1 (en) 1999-03-19 2007-10-02 Bigfix, Inc. Relevance clause for computed relevance messaging
US7325127B2 (en) 2000-04-25 2008-01-29 Secure Data In Motion, Inc. Security server system
US7353204B2 (en) 2001-04-03 2008-04-01 Zix Corporation Certified transmission system
US7363243B2 (en) 2000-10-11 2008-04-22 Buzzmetrics, Ltd. System and method for predicting external events from electronic posting activity
US7366759B2 (en) 2001-02-22 2008-04-29 Parity Communications, Inc. Method and system for characterizing relationships in social networks
US7379913B2 (en) 2000-11-27 2008-05-27 Nextworth, Inc. Anonymous transaction system
US7383282B2 (en) 2000-10-19 2008-06-03 Anthony David Whitehead Method and device for classifying internet objects and objects stored on computer-readable media
US7412462B2 (en) 2000-02-18 2008-08-12 Burnside Acquisition, Llc Data repository and method for promoting network storage of data
US7418397B2 (en) 1998-04-30 2008-08-26 Fujitsu Limited Witness system
US7444356B2 (en) 2000-01-27 2008-10-28 Manning & Napier Information Services, Llc. Construction of trainable semantic vectors and clustering, classification, and searching using trainable semantic vectors
US7457948B1 (en) 2000-09-29 2008-11-25 Lucent Technologies Inc. Automated authentication handling system
US7490092B2 (en) 2000-07-06 2009-02-10 Streamsage, Inc. Method and system for indexing and searching timed media information based upon relevance intervals
US7571177B2 (en) 2001-02-08 2009-08-04 2028, Inc. Methods and systems for automated semantic knowledge leveraging graph theoretic analysis and the inherent structure of communication
US7698167B2 (en) 2000-04-28 2010-04-13 Computer Pundits, Inc. Catalog building method and system
EP1024437B1 (en) 1999-01-26 2010-04-21 Xerox Corporation Multi-modal information access

Family Cites Families (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB8719572D0 (en) * 1987-08-19 1987-09-23 Krebs M S Sigscan text retrieval system
US5182773A (en) * 1991-03-22 1993-01-26 International Business Machines Corporation Speaker-independent label coding apparatus
US5371807A (en) * 1992-03-20 1994-12-06 Digital Equipment Corporation Method and apparatus for text classification
US5706497A (en) * 1994-08-15 1998-01-06 Nec Research Institute, Inc. Document retrieval using fuzzy-logic inference
US5819260A (en) * 1996-01-22 1998-10-06 Lexis-Nexis Phrase recognition method and apparatus
US5915249A (en) * 1996-06-14 1999-06-22 Excite, Inc. System and method for accelerated query evaluation of very large full-text databases
US5987457A (en) * 1997-11-25 1999-11-16 Acceleration Software International Corporation Query refinement method for searching documents
NZ515293A (en) 1999-05-05 2004-04-30 West Publishing Company D Document-classification system, method and software
EP1156430A2 (en) * 2000-05-17 2001-11-21 Matsushita Electric Industrial Co., Ltd. Information retrieval system
US6820081B1 (en) * 2001-03-19 2004-11-16 Attenex Corporation System and method for evaluating a structured message store for message redundancy
US7039638B2 (en) * 2001-04-27 2006-05-02 Hewlett-Packard Development Company, L.P. Distributed data clustering system and method
US6978274B1 (en) * 2001-08-31 2005-12-20 Attenex Corporation System and method for dynamically evaluating latent concepts in unstructured documents
US6778995B1 (en) * 2001-08-31 2004-08-17 Attenex Corporation System and method for efficiently generating cluster groupings in a multi-dimensional concept space
US7610313B2 (en) * 2003-07-25 2009-10-27 Attenex Corporation System and method for performing efficient document scoring and clustering

Patent Citations (194)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US3416150A (en) 1965-08-02 1968-12-10 Data Disc Inc Magnetic recording disc cartridge
US3426210A (en) 1965-12-22 1969-02-04 Rca Corp Control circuit for automatically quantizing signals at desired levels
US3668658A (en) 1969-12-22 1972-06-06 Ibm Magnetic record disk cover
US5133067A (en) 1985-10-09 1992-07-21 Hitachi, Ltd. Method and apparatus for system for selectively extracting display data within a specified proximity of a displayed character string using a range table
US5371673A (en) 1987-04-06 1994-12-06 Fan; David P. Information processing analysis system for sorting and scoring text
US4893253A (en) 1988-03-10 1990-01-09 Indiana University Foundation Method for analyzing intact capsules and tablets by near-infrared reflectance spectrometry
US5121338A (en) 1988-03-10 1992-06-09 Indiana University Foundation Method for detecting subpopulations in spectral analysis
US5056021A (en) 1989-06-08 1991-10-08 Carolyn Ausborn Method and apparatus for abstracting concepts from natural language
US5860136A (en) 1989-06-16 1999-01-12 Fenner; Peter R. Method and apparatus for use of associated memory with large key spaces
US6089742A (en) 1989-11-01 2000-07-18 Warmerdam; Thomas P. H. Method and apparatus for controlling robots and the like using a bubble data hierarchy placed along a medial axis
US5477451A (en) 1991-07-25 1995-12-19 International Business Machines Corp. Method and system for natural language translation
US5278980A (en) 1991-08-16 1994-01-11 Xerox Corporation Iterative technique for phrase query formation and an information retrieval system employing same
US5488725A (en) 1991-10-08 1996-01-30 West Publishing Company System of document representation retrieval by successive iterated probability sampling
US5442778A (en) 1991-11-12 1995-08-15 Xerox Corporation Scatter-gather: a cluster-based method and apparatus for browsing large document collections
US6243724B1 (en) 1992-04-30 2001-06-05 Apple Computer, Inc. Method and apparatus for organizing information in a computer system
US5524177A (en) 1992-07-03 1996-06-04 Kabushiki Kaisha Toshiba Learning of associative memory in form of neural network suitable for connectionist model
US5649193A (en) 1993-03-12 1997-07-15 Kabushiki Kaisha Toshiba Document detection system using detection result presentation for facilitating user's comprehension
US5528735A (en) 1993-03-23 1996-06-18 Silicon Graphics Inc. Method and apparatus for displaying data within a three-dimensional information landscape
US5696962A (en) 1993-06-24 1997-12-09 Xerox Corporation Method for computerized information retrieval using shallow linguistic analysis
US7251637B1 (en) 1993-09-20 2007-07-31 Fair Isaac Corporation Context vector generation and retrieval
US5619709A (en) 1993-09-20 1997-04-08 Hnc, Inc. System and method of context vector generation and retrieval
US6173275B1 (en) 1993-09-20 2001-01-09 Hnc Software, Inc. Representation and retrieval of images using context vectors derived from image information elements
US5675819A (en) 1994-06-16 1997-10-07 Xerox Corporation Document information retrieval using global word co-occurrence patterns
US6137545A (en) 1994-06-28 2000-10-24 Samsung Electronics Co., Ltd. Reception mode control in radio receivers for receiving both VSB and QAM digital television signals
US5619632A (en) 1994-09-14 1997-04-08 Xerox Corporation Displaying node-link structure with region of greater spacings and peripheral branches
US5754938A (en) 1994-11-29 1998-05-19 Herz; Frederick S. M. Pseudonymous server for system for customized electronic identification of desirable objects
US5754939A (en) * 1994-11-29 1998-05-19 Herz; Frederick S. M. System for generation of user profiles for a system for customized electronic identification of desirable objects
US5635929A (en) 1995-02-13 1997-06-03 Hughes Aircraft Company Low bit rate video encoder and decoder
US6594658B2 (en) 1995-07-07 2003-07-15 Sun Microsystems, Inc. Method and apparatus for generating query responses in a computer-based document retrieval system
US5844991A (en) 1995-08-07 1998-12-01 The Regents Of The University Of California Script identification from images using cluster-based templates
US6006221A (en) 1995-08-16 1999-12-21 Syracuse University Multilingual document retrieval system and method using semantic vector matching
US5737734A (en) 1995-09-15 1998-04-07 Infonautics Corporation Query word relevance adjustment in a search of an information retrieval system
US5799276A (en) 1995-11-07 1998-08-25 Accent Incorporated Knowledge-based speech recognition system and methods having frame length computed based upon estimated pitch period of vocalic intervals
US5842203A (en) 1995-12-01 1998-11-24 International Business Machines Corporation Method and system for performing non-boolean search queries in a graphical user interface
US5787422A (en) * 1996-01-11 1998-07-28 Xerox Corporation Method and apparatus for information accesss employing overlapping clusters
US5862325A (en) 1996-02-29 1999-01-19 Intermind Corporation Computer-based communication system and method using metadata defining a control structure
US5867799A (en) 1996-04-04 1999-02-02 Lang; Andrew K. Information system and method for filtering a massive flow of information entities to meet user information classification needs
US6026397A (en) 1996-05-22 2000-02-15 Electronic Data Systems Corporation Data analysis system and method
US5794236A (en) 1996-05-29 1998-08-11 Lexis-Nexis Computer-based system for classifying documents into a hierarchy and linking the classifications to the hierarchy
US5864871A (en) 1996-06-04 1999-01-26 Multex Systems Information delivery system and method including on-line entitlements
US6453327B1 (en) 1996-06-10 2002-09-17 Sun Microsystems, Inc. Method and apparatus for identifying and discarding junk electronic mail
US5915024A (en) 1996-06-18 1999-06-22 Kabushiki Kaisha Toshiba Electronic signature addition method, electronic signature verification method, and system and computer program product using these methods
US5909677A (en) 1996-06-18 1999-06-01 Digital Equipment Corporation Method for determining the resemblance of documents
US5864846A (en) 1996-06-28 1999-01-26 Siemens Corporate Research, Inc. Method for facilitating world wide web searches utilizing a document distribution fusion strategy
US5920854A (en) 1996-08-14 1999-07-06 Infoseek Corporation Real-time document collection search engine with phrase indexing
US5857179A (en) 1996-09-09 1999-01-05 Digital Equipment Corporation Computer method and apparatus for clustering documents and automatic generation of cluster keywords
US6484168B1 (en) 1996-09-13 2002-11-19 Battelle Memorial Institute System for information discovery
US6092091A (en) * 1996-09-13 2000-07-18 Kabushiki Kaisha Toshiba Device and method for filtering information, device and method for monitoring updated document information and information storage medium used in same devices
US5870740A (en) 1996-09-30 1999-02-09 Apple Computer, Inc. System and method for improving the ranking of information retrieval results for short queries
US5950146A (en) 1996-10-04 1999-09-07 At & T Corp. Support vector method for function estimation
US5987446A (en) 1996-11-12 1999-11-16 U.S. West, Inc. Searching large collections of text using multiple search engines concurrently
US6326962B1 (en) 1996-12-23 2001-12-04 Doubleagent Llc Graphic user interface for database system
US5966126A (en) 1996-12-23 1999-10-12 Szabo; Andrew J. Graphic user interface for database system
US6092059A (en) 1996-12-27 2000-07-18 Cognex Corporation Automatic classifier for real time inspection and classification
US5950189A (en) 1997-01-02 1999-09-07 At&T Corp Retrieval system and method
US5924105A (en) 1997-01-27 1999-07-13 Michigan State University Method and product for determining salient features for use in information searching
US6137499A (en) 1997-03-07 2000-10-24 Silicon Graphics, Inc. Method, system, and computer program product for visualizing data using partial hierarchies
US5819258A (en) 1997-03-07 1998-10-06 Digital Equipment Corporation Method and apparatus for automatically generating hierarchical categories from large document collections
US5940821A (en) 1997-05-21 1999-08-17 Oracle Corporation Information presentation in a knowledge base search and retrieval system
US6460034B1 (en) 1997-05-21 2002-10-01 Oracle Corporation Document knowledge base research and retrieval system
US6148102A (en) 1997-05-29 2000-11-14 Adobe Systems Incorporated Recognizing text in a multicolor image
US6137911A (en) 1997-06-16 2000-10-24 The Dialog Corporation Plc Test classification system and method
EP0886227B1 (en) 1997-06-16 2003-10-01 Compaq Computer Corporation Full-text indexed mail repository
US6202064B1 (en) 1997-06-20 2001-03-13 Xerox Corporation Linguistic search system
US6470307B1 (en) 1997-06-23 2002-10-22 National Research Council Of Canada Method and apparatus for automatically identifying keywords within a document
US6012053A (en) 1997-06-23 2000-01-04 Lycos, Inc. Computer system with user-controlled relevance ranking of search results
US6070133A (en) 1997-07-21 2000-05-30 Battelle Memorial Institute Information retrieval system utilizing wavelet transform
US6122628A (en) 1997-10-31 2000-09-19 International Business Machines Corporation Multidimensional data clustering and dimension reduction for indexing and searching
US6154219A (en) 1997-12-01 2000-11-28 Microsoft Corporation System and method for optimally placing labels on a map
US6389436B1 (en) 1997-12-15 2002-05-14 International Business Machines Corporation Enhanced hypertext categorization using hyperlinks
US6421709B1 (en) 1997-12-22 2002-07-16 Accepted Marketing, Inc. E-mail filter and method thereof
US6094649A (en) 1997-12-22 2000-07-25 Partnet, Inc. Keyword searches of structured databases
US6449612B1 (en) 1998-03-17 2002-09-10 Microsoft Corporation Varying cluster number in a scalable clustering system for use with large databases
US6038574A (en) 1998-03-18 2000-03-14 Xerox Corporation Method and apparatus for clustering a collection of linked documents using co-citation analysis
US6484196B1 (en) 1998-03-20 2002-11-19 Advanced Web Solutions Internet messaging system and method for use in computer networks
US6119124A (en) 1998-03-26 2000-09-12 Digital Equipment Corporation Method for clustering closely resembling data objects
US6349296B1 (en) 1998-03-26 2002-02-19 Altavista Company Method for clustering closely resembling data objects
US6418431B1 (en) 1998-03-30 2002-07-09 Microsoft Corporation Information retrieval and speech recognition based on language models
US7418397B2 (en) 1998-04-30 2008-08-26 Fujitsu Limited Witness system
US6345243B1 (en) 1998-05-27 2002-02-05 Lionbridge Technologies, Inc. System, method, and product for dynamically propagating translations in a translation-memory system
US7209949B2 (en) 1998-05-29 2007-04-24 Research In Motion Limited System and method for synchronizing information between a host system and a mobile data communication device
US7266365B2 (en) 1998-05-29 2007-09-04 Research In Motion Limited System and method for delayed transmission of bundled command messages
US6438564B1 (en) 1998-06-17 2002-08-20 Microsoft Corporation Method for associating a discussion with a document
US6100901A (en) 1998-06-22 2000-08-08 International Business Machines Corporation Method and apparatus for cluster exploration and visualization
US6216123B1 (en) 1998-06-24 2001-04-10 Novell, Inc. Method and system for rapid retrieval in a full text indexing system
US6888584B2 (en) 1998-06-29 2005-05-03 Hitachi, Ltd. Liquid crystal display device
US6446061B1 (en) 1998-07-31 2002-09-03 International Business Machines Corporation Taxonomy generation for document collections
US6167368A (en) 1998-08-14 2000-12-26 The Trustees Of Columbia University In The City Of New York Method and system for indentifying significant topics of a document
US6243713B1 (en) 1998-08-24 2001-06-05 Excalibur Technologies Corp. Multimedia document retrieval by application of multimedia queries to a unified index of multimedia data for a plurality of multimedia data types
US7137075B2 (en) 1998-08-24 2006-11-14 Hitachi, Ltd. Method of displaying, a method of processing, an apparatus for processing, and a system for processing multimedia information
US6414677B1 (en) 1998-09-14 2002-07-02 Microsoft Corporation Methods, apparatus and data structures for providing a user interface, which exploits spatial memory in three-dimensions, to objects and which visually groups proximally located objects
US6480885B1 (en) 1998-09-15 2002-11-12 Michael Olivier Dynamically matching users for group communications based on a threshold degree of matching of sender and recipient predetermined acceptance criteria
US6338062B1 (en) 1998-09-28 2002-01-08 Fuji Xerox Co., Ltd. Retrieval system, retrieval method and computer readable recording medium that records retrieval program
US6789230B2 (en) * 1998-10-09 2004-09-07 Microsoft Corporation Creating a summary having sentences with the highest weight, and lowest length
US6415283B1 (en) 1998-10-13 2002-07-02 Orack Corporation Methods and apparatus for determining focal points of clusters in a tree structure
US6480843B2 (en) 1998-11-03 2002-11-12 Nec Usa, Inc. Supporting web-query expansion efficiently using multi-granularity indexing and query processing
US6678705B1 (en) 1998-11-16 2004-01-13 At&T Corp. System for archiving electronic documents using messaging groupware
US6628304B2 (en) 1998-12-09 2003-09-30 Cisco Technology, Inc. Method and apparatus providing a graphical user interface for representing and navigating hierarchical networks
US6442592B1 (en) 1998-12-11 2002-08-27 Micro Computer Systems, Inc. Message center system
US6816175B1 (en) 1998-12-19 2004-11-09 International Business Machines Corporation Orthogonal browsing in object hierarchies
US6549957B1 (en) 1998-12-22 2003-04-15 International Business Machines Corporation Apparatus for preventing automatic generation of a chain reaction of messages if a prior extracted message is similar to current processed message
US6381601B1 (en) 1998-12-22 2002-04-30 Hitachi, Ltd. Grouping and duplicate removal method in a database
US6349307B1 (en) 1998-12-28 2002-02-19 U.S. Philips Corporation Cooperative topical servers with automatic prefiltering and routing
US6922699B2 (en) 1999-01-26 2005-07-26 Xerox Corporation System and method for quantitatively representing data objects in vector space
US6598054B2 (en) 1999-01-26 2003-07-22 Xerox Corporation System and method for clustering data objects in a collection
EP1024437B1 (en) 1999-01-26 2010-04-21 Xerox Corporation Multi-modal information access
US6360227B1 (en) 1999-01-29 2002-03-19 International Business Machines Corporation System and method for generating taxonomies with applications to content-based recommendations
US6941325B1 (en) 1999-02-01 2005-09-06 The Trustees Of Columbia University Multimedia archive description scheme
US6523026B1 (en) 1999-02-08 2003-02-18 Huntsman International Llc Method for retrieving semantically distant analogies
US7277919B1 (en) 1999-03-19 2007-10-02 Bigfix, Inc. Relevance clause for computed relevance messaging
US6510406B1 (en) 1999-03-23 2003-01-21 Mathsoft, Inc. Inverse inference engine for high performance web search
US7051017B2 (en) 1999-03-23 2006-05-23 Insightful Corporation Inverse inference engine for high performance web search
US6862710B1 (en) 1999-03-23 2005-03-01 Insightful Corporation Internet navigation using soft hyperlinks
US6408294B1 (en) 1999-03-31 2002-06-18 Verizon Laboratories Inc. Common term optimization
US6496822B2 (en) 1999-04-12 2002-12-17 Micron Technology, Inc. Methods of providing computer systems with bundled access to restricted-access databases
US6377287B1 (en) 1999-04-19 2002-04-23 Hewlett-Packard Company Technique for visualizing large web-based hierarchical hyperbolic space with multi-paths
EP1049030A1 (en) 1999-04-28 2000-11-02 SER Systeme AG Produkte und Anwendungen der Datenverarbeitung Classification method and apparatus
US6629097B1 (en) 1999-04-28 2003-09-30 Douglas K. Keith Displaying implicit associations among items in loosely-structured data sets
US6493703B1 (en) 1999-05-11 2002-12-10 Prophet Financial Systems System and method for implementing intelligent online community message board
US6606625B1 (en) 1999-06-03 2003-08-12 University Of Southern California Wrapper induction by hierarchical data analysis
US6701305B1 (en) 1999-06-09 2004-03-02 The Boeing Company Methods, apparatus and computer program products for information retrieval and document classification utilizing a multidimensional subspace
US6611825B1 (en) 1999-06-09 2003-08-26 The Boeing Company Method and system for text mining using multidimensional subspaces
US6711585B1 (en) 1999-06-15 2004-03-23 Kanisa Inc. System and method for implementing a knowledge management system
US7401087B2 (en) 1999-06-15 2008-07-15 Consona Crm, Inc. System and method for implementing a knowledge management system
US6438537B1 (en) 1999-06-22 2002-08-20 Microsoft Corporation Usage based aggregation optimization
US6389433B1 (en) 1999-07-16 2002-05-14 Microsoft Corporation Method and system for automatically merging files into a single instance store
US6747646B2 (en) 1999-07-16 2004-06-08 International Business Machines Corporation System and method for fusing three-dimensional shape data on distorted images without correcting for distortion
US6502081B1 (en) 1999-08-06 2002-12-31 Lexis Nexis System and method for classifying legal concepts using legal topic scheme
US6523063B1 (en) 1999-08-30 2003-02-18 Zaplet, Inc. Method system and program product for accessing a file using values from a redirect message string for each change of the link identifier
US6651057B1 (en) 1999-09-03 2003-11-18 Bbnt Solutions Llc Method and apparatus for score normalization for information retrieval applications
US6260038B1 (en) 1999-09-13 2001-07-10 International Businemss Machines Corporation Clustering mixed attribute patterns
US6990238B1 (en) 1999-09-30 2006-01-24 Battelle Memorial Institute Data processing, analysis, and visualization system for use with disparate data types
US6544123B1 (en) 1999-10-29 2003-04-08 Square Co., Ltd. Game apparatus, command input method for video game and computer-readable recording medium recording programs for realizing the same
US7130807B1 (en) 1999-11-22 2006-10-31 Accenture Llp Technology sharing during demand and supply planning in a network-based supply chain environment
US6507847B1 (en) 1999-12-17 2003-01-14 Openwave Systems Inc. History database structure for Usenet
US7444356B2 (en) 2000-01-27 2008-10-28 Manning & Napier Information Services, Llc. Construction of trainable semantic vectors and clustering, classification, and searching using trainable semantic vectors
US6542889B1 (en) 2000-01-28 2003-04-01 International Business Machines Corporation Methods and apparatus for similarity text search based on conceptual indexing
US6654739B1 (en) 2000-01-31 2003-11-25 International Business Machines Corporation Lightweight document clustering
US6571225B1 (en) 2000-02-11 2003-05-27 International Business Machines Corporation Text categorizers based on regularizing adaptations of the problem of computing linear separators
US7412462B2 (en) 2000-02-18 2008-08-12 Burnside Acquisition, Llc Data repository and method for promoting network storage of data
US7117246B2 (en) 2000-02-22 2006-10-03 Sendmail, Inc. Electronic mail system with methodology providing distributed message store
US7013435B2 (en) 2000-03-17 2006-03-14 Vizible.Com Inc. Three dimensional spatial user interface
US6560597B1 (en) 2000-03-21 2003-05-06 International Business Machines Corporation Concept decomposition using clustering
US6757646B2 (en) 2000-03-22 2004-06-29 Insightful Corporation Extended functionality for an inverse inference engine based web search
US7269598B2 (en) 2000-03-22 2007-09-11 Insightful Corporation Extended functionality for an inverse inference engine based web search
US6785679B1 (en) 2000-03-29 2004-08-31 Brassring, Llc Method and apparatus for sending and tracking resume data sent via URL
US6915308B1 (en) 2000-04-06 2005-07-05 Claritech Corporation Method and apparatus for information mining and filtering
US7325127B2 (en) 2000-04-25 2008-01-29 Secure Data In Motion, Inc. Security server system
US6584564B2 (en) 2000-04-25 2003-06-24 Sigaba Corporation Secure e-mail system
US7698167B2 (en) 2000-04-28 2010-04-13 Computer Pundits, Inc. Catalog building method and system
US6879332B2 (en) 2000-05-16 2005-04-12 Groxis, Inc. User interface for displaying and exploring hierarchical information
US6883001B2 (en) 2000-05-26 2005-04-19 Fujitsu Limited Document information search apparatus and method and recording medium storing document information search program therein
US6519580B1 (en) 2000-06-08 2003-02-11 International Business Machines Corporation Decision-tree-based symbolic rule induction system for text categorization
US6697998B1 (en) 2000-06-12 2004-02-24 International Business Machines Corporation Automatic labeling of unlabeled text data
US20020078090A1 (en) 2000-06-30 2002-06-20 Hwang Chung Hee Ontological concept-based, user-centric text summarization
US7490092B2 (en) 2000-07-06 2009-02-10 Streamsage, Inc. Method and system for indexing and searching timed media information based upon relevance intervals
US6738759B1 (en) 2000-07-07 2004-05-18 Infoglide Corporation, Inc. System and method for performing similarity searching using pointer optimization
US7099819B2 (en) 2000-07-25 2006-08-29 Kabushiki Kaisha Toshiba Text information analysis apparatus and method
US6675159B1 (en) 2000-07-27 2004-01-06 Science Applic Int Corp Concept-based search and retrieval system
US20020032735A1 (en) 2000-08-25 2002-03-14 Daniel Burnstein Apparatus, means and methods for automatic community formation for phones and computer networks
US7457948B1 (en) 2000-09-29 2008-11-25 Lucent Technologies Inc. Automated authentication handling system
US7363243B2 (en) 2000-10-11 2008-04-22 Buzzmetrics, Ltd. System and method for predicting external events from electronic posting activity
US6684205B1 (en) 2000-10-18 2004-01-27 International Business Machines Corporation Clustering hypertext with applications to web searching
US7383282B2 (en) 2000-10-19 2008-06-03 Anthony David Whitehead Method and device for classifying internet objects and objects stored on computer-readable media
US7054870B2 (en) 2000-11-15 2006-05-30 Kooltorch, Llc Apparatus and methods for organizing and/or presenting data
US6978419B1 (en) 2000-11-15 2005-12-20 Justsystem Corporation Method and apparatus for efficient identification of duplicate and near-duplicate documents and text spans using high-discriminability text fragments
US7379913B2 (en) 2000-11-27 2008-05-27 Nextworth, Inc. Anonymous transaction system
US7003551B2 (en) 2000-11-30 2006-02-21 Bellsouth Intellectual Property Corp. Method and apparatus for minimizing storage of common attachment files in an e-mail communications server
US20020065912A1 (en) 2000-11-30 2002-05-30 Catchpole Lawrence W. Web session collaboration
US7240199B2 (en) 2000-12-06 2007-07-03 Rpost International Limited System and method for verifying delivery and integrity of electronic messages
US6751628B2 (en) 2001-01-11 2004-06-15 Dolphin Search Process and system for sparse vector and matrix representation of document indexing and retrieval
US7233886B2 (en) 2001-01-19 2007-06-19 Smartsignal Corporation Adaptive modeling of changed states in predictive condition monitoring
US6658423B1 (en) 2001-01-24 2003-12-02 Google, Inc. Detecting duplicate and near-duplicate files
US6841321B2 (en) 2001-01-26 2005-01-11 Hitachi, Ltd. Method and system for processing a semi-conductor device
US7080320B2 (en) 2001-01-29 2006-07-18 Kabushiki Kaisha Toshiba Translation apparatus and method
US7571177B2 (en) 2001-02-08 2009-08-04 2028, Inc. Methods and systems for automated semantic knowledge leveraging graph theoretic analysis and the inherent structure of communication
US20020122543A1 (en) 2001-02-12 2002-09-05 Rowen Chris E. System and method of indexing unique electronic mail messages and uses for the same
US7366759B2 (en) 2001-02-22 2008-04-29 Parity Communications, Inc. Method and system for characterizing relationships in social networks
US6823333B2 (en) 2001-03-02 2004-11-23 The United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration System, method and apparatus for conducting a keyterm search
US6819344B2 (en) 2001-03-12 2004-11-16 Microsoft Corporation Visualization of multi-dimensional data having an unbounded dimension
US20030130991A1 (en) 2001-03-28 2003-07-10 Fidel Reijerse Knowledge discovery from data sets
US7353204B2 (en) 2001-04-03 2008-04-01 Zix Corporation Certified transmission system
US6714929B1 (en) 2001-04-13 2004-03-30 Auguri Corporation Weighted preference data search system and method
US7194458B1 (en) 2001-04-13 2007-03-20 Auguri Corporation Weighted preference data search system and method
US6804665B2 (en) 2001-04-18 2004-10-12 International Business Machines Corporation Method and apparatus for discovering knowledge gaps between problems and solutions in text databases
US7155668B2 (en) 2001-04-19 2006-12-26 International Business Machines Corporation Method and system for identifying relationships between text documents and structured variables pertaining to the text documents
US7020645B2 (en) 2001-04-19 2006-03-28 Eoriginal, Inc. Systems and methods for state-less authentication
US7194483B1 (en) 2001-05-07 2007-03-20 Intelligenxia, Inc. Method, system, and computer program product for concept-based multi-dimensional analysis of unstructured information
US6970881B1 (en) 2001-05-07 2005-11-29 Intelligenxia, Inc. Concept-based method and system for dynamically analyzing unstructured information
US6735578B2 (en) 2001-05-10 2004-05-11 Honeywell International Inc. Indexing of knowledge base in multilayer self-organizing maps with hessian and perturbation induced fast learning
US20020184193A1 (en) 2001-05-30 2002-12-05 Meir Cohen Method and system for performing a similarity search using a dissimilarity based indexing structure
US6675164B2 (en) 2001-06-08 2004-01-06 The Regents Of The University Of California Parallel object-oriented data mining system
US7266545B2 (en) 2001-08-07 2007-09-04 International Business Machines Corporation Methods and apparatus for indexing in a database and for retrieving data from a database in accordance with queries using example sets
US7096431B2 (en) 2001-08-31 2006-08-22 Sony Corporation Menu display apparatus and menu display method

Non-Patent Citations (47)

* Cited by examiner, † Cited by third party
Title
Artero, Almir Olivette., et al., Viz3D: Effective Exploratory Visualization of Lage Multidimensional Datasets, Oct. 20, 2004, IEEE, Computer Graphics and Image Processing 2004, pp. 340-347.
Baeza-Yates et al., "Modern Information Retrieval," Ch. 2 "Modeling," Modern Information Retrieval, Harlow: Addison-Wesley, Great Britain 1999, pp. 18-71 (1999).
Bernard, M. et al.: "Labeled Radial Drawing of Data Structures" Proceedings of the Seventh International Conference on Information Visualization, Infovis. IEEE Symposium, Jul. 16-18, 2003, Piscataway, NJ, USA, IEEE, Jul. 16, 2003, pp. 479-484, XP010648809, ISBN: 0-7695-1988-1.
Bier et al. "Toolglass and Magic Lenses: The See-Through Interface", Computer Graphics Proceedings, Proceedings of Siggraph Annual International Conference on Computer Graphics and Interactive Techniques, pp. 73-80, XP000879378 (Aug. 1993).
Bier, E.A. et al. "Toolglass and Magic Lenses: The See-Through Interface", Computer Graphics Proceedings, Proceedings of Siggraph Annual International Conference on Computer Graphics and Interactive Techniques, pp. 73-80, XP000879378 (Aug. 1993).
Boukhelifa et al., "A Model and Software System for Coordinated and Multiple Views in Exploratory Visualization," Information Visualization, vol. 2003, No. 2, pp. 258-269, GB.
Can, F., Incremental Clustering for Dynamic Information Processing: ACM Transactions on Information Systems, ACM, New York, NY, US, vol. 11, No. 2, PD. 143-164, XP-002308022 (Apr. 1993).
Chen An et al., "Fuzzy Concept Graph and Application in Web Document Clustering," 2001, pp. 101-106, IEEE.
Chen An et al., "Fuzzy Concept Graph and Application in Web Document Clustering," IEEE, pp. 101-106 (2001).
D. Sullivan, "Document Warehousing and Text Mining: Techniques for Improving Business Operations, Marketing and Sales," Ch. 1-3, John Wiley & Sons, New York, NY (2001).
Davison et al., "Brute Force Estimation of the Number of Human Genes Using EST Clustering as a Measure," IBM Journal of Research & Development, vol. 45, pp. 439-447 (May 2001).
Eades et al. "Multilevel Visualization of Clustered Graphs," Department of Computer Science and Software Engineering, University if Newcastle, Australia, Proceedings of Graph Drawing '96, Lecture Notes in Computer Science, NR. 1190, Sep. 18-20, 1996, pp. 101-112, Berkeley, CA, USA, ISBN: 3-540-62495-3 (Sep. 18, 1996).
Eades, P. et al. "Multilevel Visualization of Clustered Graphs" Proceedings of Graph Drawing '96, Lecture Notes in Computer Science, NR. 1190, Sep. 18, 1996, Sep. 20, 1996 pp. 101-112, Berkeley, CA, USA, ISBN: 3-540-62495-3.
Eades, P. et al., "Orthogonal grid drawing of clustered graphs" Technical Report 96-04, [Online] 1996, University of Newcastle, AU Retrieved from the internet: URL:http://citeseer.ist.psu.edu/eades96ort hogonal.html>[retrieved on May 29, 2006].
Estivill-Castro, V. et al. "AMOEBA; Hierarchical Clustering Based on Spatial Proximity using Delaunay Triangulation", 1999 ACM Sigmod International Conference on Management of Data, vol. 28, No. 2, Jun. 1, 1999, Jun. 3, 1999 pp. 49-60, Philadelphia, PA.
F. Can, Incremental Clustering for Dynamic Information Processing: ACM Transactions on Information Systems, ACM, New York, NY, US, vol. 11, No. 2, pp. 143-164, XP-002308022 (Apr. 1993).
Faber, Vance., "Clustering and the Continuous K-Means Algorithm," Los Alamos Science, The Laboratory, Los Alamos, NM, US, No. 22, Jan. 1, 1994, pp. 138-144.
Fekete, J. D. et al., "Excentric Labeling: Dynamic Neighborhood Labeling for Data Visualization," CHI 1999 Conference Proceedings Human Factors in Computing Systems, Pittsburgh, PA, pp. 512-519 (May 15-20, 1999).
Inxight Vizserver, "Speeds and Simplifies the Exploration and Sharing of Information", www.inxight.com/products/vizserver, copyright 2005.
Jain et al., "Data Clustering: A Review," ACM Computing Surveys, vol. 31, No. 3, Sep. 1999, pp. 264-323, New York, NY, USA.
James Osborn et al., "JUSTICE: A Jidicial Search Tool Using Intelligent Cencept Extraction," Department of Computer Science and Software Engineering, University of Melbourne, Australia, ICAIL-99, 1999, pp. 173-181, ACM (1999).
Jiang Linhui, "K-Mean Algorithm: Iterative Partitioning Clustering Algorithm," http://www.cs.regina.ca/-linhui/K.sub.--mean.sub.--algorithm.html, (2001) Computer Science Department, University of Regina, Saskatchewan, Canada (2001).
Kanugo et al., "The Analysis of a Simple K-Means Clustering Algorithm," pp. 100-109, Proc 16th annual symposium of computational geometry (May 2000).
Kawano, Hiroyuki., "Overview of Mondou Web Search Engine Using Text Mining and Information Visualizing Technologies," IEEE, 2001, pp. 234-241.
Kazumasa Ozawa, "A Stratificational Overlapping Cluster Scheme," Information Science Center, Osaka Electro-Communication University, Neyagawa-shi, Osaka 572, Japan, Pattern Recognition, vol. 18, pp. 279-286 (1985).
Kohonen, T., "Self-Organizing Maps," Ch. 1.2, Springer-Verlag (3d ed. 2001).
Kurimo M., "Fast latent semantic indexing of spoken documents by using self-organizing maps" IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 6, Jun. 5, 2000, pp. 2425-2428, XP010504794.
Lam et al., "A Sliding Window Technique for Word Recognition," SPIE, vol. 2422, pp. 38-46, Center of Excellence for Document Analysis and Recognition, State University of New Yrok at Baffalo, NY, USA (1995).
Linhui, Jiang, "K-Mean Algorithm: Iterative Partitioning Clustering Algorithm," http://www.cs.regina.ca/-linhui/K.sub.--mean.sub.--algorithm.html, (2001) Computer Science Department, University of Regina, Saskatchewan, Canada.
Magarshak, Greg., Theory & Practice. Issue 01. May 2000. http://www.flipcode.com/articles/tp.sub.--issue01-pf.shtml.
Maria Cristin Maria Cristin Ferreira de Oliveira and Haim Levkowitz, From Visual Data Exploration to Visual Data Mining: A Survey, Jul.-Sep. 2003, IEEE,IEEE Transactions onVisualization and Computer Graphics vol. 9, No. 3, pp. 378-394.
McNee, Sean M., "Meeting User Information Needs in Recommended System." Ph.D. Dissertation, University of Minnesota-Twin Cities, Jun. 2006.
Miller et al. Proceedings of the IEEE Visualization Conference. 1998, pp. 189-196. Topic Islands: A Wavelet Based Text Visualization System.
Osborn, James. et al., "Justice: A Judicial Search Tool Using Intelligent Concept Extraction," ICAIL, 1999, pp. 173-181, ACM, Oslo, Norway.
Ozawa. A Stratificational Overlapping Cluster Scheme. Pattern Recogn. vol. 18. 1985.
Parsons Software, Complete Legal Collection 2001, a division of Mattel Interactive, Inc., Fremont, California.
Pelleg et al., "Accelerating Exact K-Means Algorithms With Geometric Reasoning," pp. 277-281, Conf on Knowledge Discovery in Data, Proc fifth ACM SIGKDD (1999).
Rauber, Andreas., et al. Text Mining in the SOMLib Digital Library System: The Representation of Topics and Genres, 2003 Kluwer Academic Publishers, Applied Intelligence 18, pp. 271-293.
Sachinopoulou, Anna., Multidimensional Visualization, 2001 VTT Publishers, VTT Research Notes 2114, pp. 1-37.
Shneiderman, Ben., et al. "A Taxonomy of Multiple Window Coordinations" Technical Report, University of Maryland, Computer Science Department [Online] No. CS-TR-3854, Maryland, USA, URL:http://www.lib.umd.edu/drum/bitstream/1903/927/2/CS-TR-3854.p.
Slaney, M., et al., "Multimedia Edges: Finding Hierarchy in all Dimensions" Proc. 9-th ACM Intl. Conf. on Multimedia, ISBN.1-58113-394-4, Sep. 30, 2001, pp. 29-40, XP002295016 Ottawa.
Slaton G. et al., "Extended Boolean Information retrieval" Communications of the Association for Computing Machinery, ACM, New York, NY, US., vol. 26, o. 12, Nov. 1, 1983, pig 1022-1036, XP000670417.
Strehl, A. et al., "Cluster Ensembles-A Knowledge Reuse Framework for Combining Partitioning," Journal of Machine Learning Research, MIT Press, Cambridge, MA, US, ISSN: 1533-7928, vol. 3, No. 12, pp. 583-617, XP002390603 (Dec. 2002).
Sullivan, Dan., "Document Warehousing and Text Mining: Techniques for Improving Business Operations, Marketing and Sales," Ch. 1-3, John Wiley & Sons, New York, NY (2001).
Wang et al., "Learning text classifier using the domain concept hierarchy," Communications, Circuits and Systems and West Sino Expositions, IEEE 2002 International Conference on Jun. 29-Jul. 1, 2002, Piscataway, NJ, USA, IEEE, vol. 2, pp. 1230-1234.
Whiting et al., Image Quantization: Statistics and Modeling. SPIE, vol. 3336, 1998, pp. 260-271.
Yip Chung, Christina., et al, "Thematic Mapping-From Unstructured Documents to Taxonomies," CIKM'02, Nov. 4-9, 2002, pp. 608-610, ACM, McLean, Virginia, USA.

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170242879A1 (en) * 2007-07-26 2017-08-24 Hamid Hatami-Hanza Ontological Subjects Of A Universe And Knowledge Representations Thereof
US10846274B2 (en) * 2007-07-26 2020-11-24 Hamid Hatami-Hanza Ontological subjects of a universe and knowledge representations thereof
US9323721B1 (en) * 2013-02-27 2016-04-26 Google Inc. Quotation identification
US11269812B2 (en) 2019-05-10 2022-03-08 International Business Machines Corporation Derived relationship for collaboration documents

Also Published As

Publication number Publication date
US6978274B1 (en) 2005-12-20
US20170262455A1 (en) 2017-09-14
US7809727B2 (en) 2010-10-05
US20160078126A1 (en) 2016-03-17
US9619551B2 (en) 2017-04-11
US8015188B2 (en) 2011-09-06
US20080104063A1 (en) 2008-05-01
US20140250087A1 (en) 2014-09-04
US20130159300A1 (en) 2013-06-20
US9195399B2 (en) 2015-11-24
US7313556B2 (en) 2007-12-25
US20060089947A1 (en) 2006-04-27
US8380718B2 (en) 2013-02-19
US20110022597A1 (en) 2011-01-27
US20110320453A1 (en) 2011-12-29

Similar Documents

Publication Publication Date Title
US9619551B2 (en) Computer-implemented system and method for generating document groupings for display
US9558259B2 (en) Computer-implemented system and method for generating clusters for placement into a display
US7610313B2 (en) System and method for performing efficient document scoring and clustering
EP1573660B1 (en) Identifying critical features in ordered scale space
US7085771B2 (en) System and method for automatically discovering a hierarchy of concepts from a corpus of documents
US20020016800A1 (en) Method and apparatus for generating metadata for a document
Nevill-Manning et al. Browsing in digital libraries: a phrase-based approach
Changala et al. A SURVEY ON DEVELOPMENT OF PATTERN EVOLVING MODEL FOR DISCOVERY OF PATTERNS IN TEXT MINING USING DATA MINING TECHNIQUES.
DEOGUN INFORMATION RETRIEVAL, RESEARCH: STRATEGIES AND USER IMPLICATIONS
Kourik Performance of classification tools on unstructured text
Muvva WikiSearch: An Information Retrieval System
Hossain et al. Sense based organization of descriptive data
Yang et al. A Text Processing System for Mining Association Rules.

Legal Events

Date Code Title Description
STCF Information on status: patent grant

Free format text: PATENTED CASE

AS Assignment

Owner name: BANK OF AMERICA, N.A., AS ADMINISTRATIVE AGENT, TE

Free format text: NOTICE OF GRANT OF SECURITY INTEREST IN PATENTS;ASSIGNORS:FTI CONSULTING, INC.;FTI CONSULTING TECHNOLOGY LLC;FTI CONSULTING TECHNOLOGY SOFTWARE CORP;REEL/FRAME:036031/0637

Effective date: 20150626

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551)

Year of fee payment: 4

AS Assignment

Owner name: FTI CONSULTING TECHNOLOGY LLC, MARYLAND

Free format text: CHANGE OF NAME;ASSIGNOR:FTI TECHNOLOGY LLC;REEL/FRAME:045785/0645

Effective date: 20110926

AS Assignment

Owner name: FTI CONSULTING TECHNOLOGY LLC, MARYLAND

Free format text: RELEASE OF SECURITY INTEREST IN PATENT RIGHTS AT REEL/FRAME 036031/0637;ASSIGNOR:BANK OF AMERICA, N.A., AS ADMINISTRATIVE AGENT;REEL/FRAME:047060/0107

Effective date: 20180910

AS Assignment

Owner name: NUIX NORTH AMERICA INC., VIRGINIA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:FTI CONSULTING TECHNOLOGY LLC;REEL/FRAME:047237/0019

Effective date: 20180910

AS Assignment

Owner name: ATTENEX CORPORATION, WASHINGTON

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:GALLIVAN, DAN;KAWAI, KENJI;REEL/FRAME:051679/0205

Effective date: 20010831

Owner name: FTI TECHNOLOGY LLC, MARYLAND

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:ATTENEX CORPORATION;REEL/FRAME:051679/0232

Effective date: 20091229

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 8TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1552); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Year of fee payment: 8