WO1994015310A1

WO1994015310A1 - Information storage and retrieval system

Info

Publication number: WO1994015310A1
Application number: PCT/GB1993/002626
Authority: WO
Inventors: Roy Jack Bernard Assersohn
Original assignee: Roy Jack Bernard Assersohn
Priority date: 1992-12-22
Filing date: 1993-12-22
Publication date: 1994-07-07
Also published as: GB9226660D0

Abstract

A data storage and retrieval system includes means (223) for applying a unique document identifier header tag to files stored in a storage medium (218) incorporating an hierarchical stratum representing empirical filing operations, a flat-file stratum representing subject areas, a date-range stratum for a date-based retrieval and a free text stratum for keyword-based retrieval.

Description

Information storage and retrieval system This invention relates to information storage and retrieval systems for the indexing and retrieval of digitised files and, in particular, to computer systems for the retrieval of compressed data images. It includes search and select methods for the subsequent identification, location and retrieval of those images for their subsequent retrieval and processing.

There is a variety of data image processing, or scanning, systems comprising hardware and facilitating software for the compression and de-compression of scanned images. They transform hard copy material into compressed digitised files for storage on computer disks, optical disks or magnetic tape. The digitised files can be subsequently de-compressed, viewed, printed or converted through optical character recognition and intelligent character recognition processes into text files. The digitised files may be accessed direct from the computer or from optical disk drives which in turn draw on the files stored in optical disks which in turn be accessed from their multiple storage in so-called juke boxes. It is common for data image processing systems to rely on limited indexing for the subsequent retrieval of those scanned and stored documents. This has been confined to two or three indices, usually consisting of the name of the owner of the material, an identifying number or code, and the date of its creation.

This approach has severely restricted the use to which the images can be put and has also determined that data image processing systems have been confined to producing electronic archive files with limited use and value to the owners. There are four main reasons: 1. the marketing and technological thrust has been towards providing more efficient methods of converting hard copy documents to images and storing them in increasing quantities in magnetic media. 2. the high cost of inputting sufficiently substantive indices to take the systems out of the archiving era. 3. the difficulties posed by the subjective nature of the indexing of documents. 4. the absence of methodology and techniques to produce customised indexing and retrieval services. The prior art complications involving any in substantive indexing of hundreds of thousands and in many cases millions of documents have been compounded by the problems that would have been involved in the easy retrieval of those documents. These difficulties are evidenced by the number of users who have turned to optical character recognition and intelligent character recognition systems which convert image files into text files. These systems have a number of shortcomings, the most notable of which are the cost of conversion, the uncertainty of dropped or unrecognised characters, the cost of correcting conversion errors, and the uncertainty about accurate retrieval .

A further important factor contributing to the shortcomings of data image processing systems is the prior art incompatibility between such systems and established database techniques and technologies. This has been compounded by the consequential heavy demands on electronic storage capacity when dealing with digitised images in large numbers.

It has been common to make use of relational databases, hierarchical databases, flatfile databases and text databases for the storage of computer files which have clearly defined and specific functional characteristics and parameters, within which the user has limited flexibility.

One of the major barriers to any development of a multi-discipline database system for digitised images has been the difficulties that would be involved in the creation of detailed and extensive indices.

One of the most important of these is the fact that the process would be heavily labour intensive and the compounding factor that there would be a consequential dependence on operators making decisions and applying subjective decisions about indexing a wide variety of other organisation's material across the spectrum of data image processing users.

We have devised a computerised database system which facilitates the storage and retrieval of data files. According to the present invention there is provided an information storage and retrieval system including a plurality of database management systems, said plurality including flatfile storage means for the storage of specific subject categories of data, date range storage means for selection and retrieval of date-tagged data and hierarchical storage means for empirical filing operations.

The invention embodies data management and information organisation techniques, database and indexing expertise into an application utilising its own proprietary software programs and drivers in association, where necessary, with other generic facilitating software. The invention combines the individual strengths and characteristics of different database techniques to satisfy the multiple and highly variable requirements for the comprehensive and detailed indexing and retrieval of digitised data utilising a wide variety of computer systems. It enables the electronic filing system to be customised according the practical needs and the practice of the user, replicating, if necessary, the format, structure and style of the user's current hard-copy filing system. A number of related and symbiotic processes are involved in the invention. The first involves indexing which is carried out by setting up pre-defined fields for data entry. Each field is accompanied by a Look Up Table containing previously entered data. Much of this data constitutes common denominators running through different documents and material relating to an individual user. Thus the Look Up Table minimises keystrokes at the data entry stage, enabling rapid, accurate and cost effective indexing involving just a single key stroke in the majority of instances. Preferably, the invention embodies four different types of database. Each is used to control different aspects of electronic filing. Advantageously, they are utilised and accessed in background mode through the customised data entry and select and search user screens of the invention. An hierarchical stratum deals with category databases, representing empirical filing operations of the user. A flat file stratum represents specific subject areas of the user's filing system.

A date range stratum is provided for selection and retrieval within set periods of time. Finally, a free text stratum provides a notepad function. The keywords contained in the notepad allow documents to be identified and retrieved according to annotations made at the time of indexing or subsequently.

Each of the four database strata can be searched separately or in combination with any or all of the others. This provides considerable flexibility and power to refine searches and retrieve digitised images according matching criteria which can be as general or as narrowly defined as the user may require for display, referral, printing or onward transmission. A database system in accordance with the present invention can be used on most types of computers either on a stand-alone or a network basis. It can be used in conjunction with a wide range of ancillary equipment including leading scanners, optical disks, optical disk drives and juke boxes for accessing multiple optical disks, printers, and modems.

Embodiments of the invention will now be particularly described with reference to the accompanying drawings, in which:-

Fig. 1 is a diagram showing the range of computers and the other hardware which may be used with the invention;

Fig. 2. is a flow chart representing the processes involved from the setting up of Header Indices to the delivery of the fully indexed electronic files; Fig. 3. is a diagrammatic representation of the file levels and the application of different types of generic database structures;

Fig. 4. is a diagram illustrating how Customer Index titles for the subsequent creation of the Header

Index are set up, in accordance with one embodiment; Fig. 5. shows an example of how the Headings for the Header index are set up on a customised basis to meet the operation needs of the user;

Fig. 6. is a representation of the Batch Status report which is generated after the indexing and scanning operations have been checked;

Fig. 7. is a flow chart showing the process by which a user creates search macros, retrieves the stored material, views it, prints it and if required saves the search macros for future use; Fig. 8. is a diagram showing the process by which search criteria are entered from look up tables to form a search macro;

Fig. 9. is a flow chart illustrating the process by which a unique document identifier is created for the retrieval of the required digitised images, and Fig. 10. is a diagram representing the processing sequence available for different types of computer embodied in the invention.

Referring now to the drawings, Fig.l., the overall process shown in overview in Fig 2. embodies either a desktop computer, a work station, a mini computer or a main frame computer, including the relevant CPUs, disk storage, keyboards and monitors together with a digital scanner, optical disk drive, optical disk storage media, printer and modem with the appropriate generic software and operating systems. One embodiment of the invention employs two separate but inter-dependent modules. The first module consists of the methodology, techniques and programs and drivers embodied in a computer based system for creating the Header Index 100 for attachment to material transformed by digital scanning into compressed images. The second module consists of the methodology, techniques and programs and drivers embodied in a computer based system for identifying, locating and retrieving the relevant indices together with the stored Images as shown in Search and Select 200. Both parts are menu driven and may be accessed by means of Microsoft Windows application and may reside on most types of desktop PCs, workstations, mini-computers and mainframes and may be accessed on a stand-alone or a network basis as illustrated in Fig. 10. It may also be associated with most leading makes of other equipment used in connection with data image processing as set out in Fig. 1.

Embodiments of the invention may be applied as a complete data image processing operation complete with scanning or they may be used solely for creating the Header Indices to be applied to documents which have been scanned by a third party. In either case the creation of the Header Index is identical. It may involve one indexer 13 or a second indexer 13A for double entry. In either case the process is identical but the two files are kept unique until the system runs a check 16 to compare one with the other. When there is an exact match the

Header Index is passed for attached to the relevant scanned image. Corrections can be carried out via the Admin, menu 18.

The first step in setting up a Header Index is to create a New Customer record 19 by entering the name in a response frame via the Edit Box 22. A customer password is 19 is created by the same means. The New Header item 21 is invoked so that the Customer Indexing Titles illustrated in FIG. 4. can be set up. The relevant titles 101 to 112, excepting the Date which is entered as 106 at the indexing stage, are entered by means of an Edit Box as shown in 22. The titles are written to disk by activating the Save Titles command 113.

The Header Index as illustrated in FIG. 5. is created against the Customer Indexing Titles. Each document requiring indexing is given a Batch No.24 which is automatically and sequentially generated. The number of pages is entered 25 and this is information which is used in the audit trail is carried through into the Header Index screen 114. Index information supplied by the user is entered by means of Look Up Tables 117 which contained previously entered data, much of which is common to successive and different documents. This indexing information is entered by a single key stroke. If the index data is new it is entered in a response text editing box 22. This automatically records it in the current Header Index and automatically updates the Look Up Table for future use 118.

Annotations to the document may be entered at this stage 115 using a response frame text editor. The Header Index is saved to disk by means of Save Details 116. This indexing procedure is repeated by the second 13A if double entry indexing is required in which case Check 15 is used to verify that the data entry is correct by matching the first Header Index 13 with the second 13A. If the two match the Header Index is deemed ready to be attached to the scanned image which is either carried out within the invention or by a third party. If the two indices do not match the Correct Indices 18 option is activated. At any state the Batch Status option 17 can be invoked to check on the progress of each Batch as illustrated in Fig.6. At this stage the correct Header Indices may be output 16 to the customer to optical disk 6 by means of an optical disk drive 6 for multiple storage in a juke box 8 or to a computer hard disk on desktop 1, work station 2, mini-computer 3 or mainframe 4. If scanning is to be carried Scan 14 is invoked and the relevant selections governing resolution, size of document, shape of document and others are set from within the present system and the scanning is completed. Check 15 is invoked to verify that the correct indices have been attached to the digitised files before the output is activated. Files 16 also allows in-house files to be saved and loaded. Once the Header Indices have been attached to the digitised files they can be loaded onto various computers as an electronic filing system. The files are then accessed through the Search and Select module 200 which is illustrated in the flow chart given in Fig. 7. This module consists of the Header Indices and the present system screens and facilitating programs to enable the required files to be identified, located and retrieved by means of the Unique Document Identifier 223. This is created by means of the process illustrated in the flow chart shown in FIG. 9. Using the Search and Select module the user's name is selected name 215 and the password entered 216. The Search menu 201 is invoked and the various search criteria ranging from Category 217 to Subject 219, Keyword 220 and Date Range 221 are selected from Look Up Tables 213 as shown in Fig. 8.showing the entries for each of those levels. The criteria are entered either by scrolling through the list or by entering letters to build up a unique Alpha identifier to jump to the relevant section of the Look Up Table. Two or three ore more or all the criteria may be entered in any required combination. Having selected the required criteria Search 214 the macro is sent to the computer database of Indices 207 to identify and locate those files which match. The files which match the Unique Document Identifier 223 are stored in the document list file 208.

All the files in the document list or those selected in the view menu 202 may be viewed or notepads created or edited before the files are retrieved from the database of images 206. Generic software decompresses the images which are displayed 211. The Print menu 203 allows selected files to be de-compressed and printed 212. The Save menu 204 enables the search results created in 201 to be saved and stored in new files with unique user specified names for further refinements to the search criteria or for future viewing or processing. The saved documents list 210 can be loaded through the Admin, menu 205.

Background audit trails monitor the process of indexing through to scanning and output of fully indexed and digitised files.

Search and select macros may be saved in unique files so that the same files can be retrieved at a later stage either for further refined searching or for processing.

Text notepads may be attached to digitised files to record required annotations which may be searched on a keyword basis.

System programs enable the generations and/or storage in memory or on hard disk the Header Index for each digitised document, the search macros, subset files of search macros and associated header indices, updates to system and customer files and customised Header Index screens.

Front end menus facilitate the inputting of the relevant index data, including text notepads to be attached to the digitised material. Front end menus also enable the easy setting up of search macros to identify relevant documents for their subsequent retrieval and processing.

There is also a facility for searching within data ranges and free text notepads, attached where required, to the indices.

The retrieval software provides considerable flexibility and the capability for very detailed and comprehensive searching of files, either single or multiple, which have been scanned into a digitised format. Searches are conducted according to a permutation of search criteria involving one or more of the hierarchical fields in combination with one or more of the flat file fields. Further refinement of the search results can be achieved by free text or Boolean text searching of text note pads which are also associated with and attached to the scanned documents. These text searches can be carried out simultaneously with the hierarchical and/or flat file searches or in isolation of one or both of them. The results of any search pattern may be stored in new files which are given unique identifying file names at the time of storage. These stored files of documents can be searched according to additional criteria at a later stage. The indices and the computer programs which are used to create them can be stored on a PC, mini or mainframe or in magnetic media, either tape or optical disk, both WORM (write once read many) disks or re-writable optical disks.

The retrieval programs enable multiple cross-database searches to be carried out. The searches can be conducted on any permutation of the hierarchical (or category) databases in combination with the flatfile indices (specific subjects and/or references) and in conjunction with keyword searches of the notepads. A date range search can be introduced at any level of interrogation of the database. The documents which are identified can be saved in the user's own computer files. They can be viewed on screen, printed or transmitted. They can also be recalled at a later date for further searching, viewing etc.

The system will run under the following environments: DOS, Windows, Unix and bespoke and other mainframe programs.

The programs may be stored in a variety of permanent media including floppy disks, hard disks, WORM optical disks, magnetic tapes.

The system can be customised to provide Header Index formats relating to any size of document, press cutting, drawing, painting, picture, paper record, data record, photograph, microfiche, microfilm or computer file.

Claims

- 11 - Claims

1. An information storage and retrieval system including a plurality of database management systems characterised in that said plurality including flatfile storage means for the storage

5 of specific subject categories of data, date range storage means for selection and retrieval of date-tagged data and hierarchical storage means for empirical filing operations.

2. An information storage and retrieval system including a plurality of database management systems as claimed in claim 1

10 characterised in that it Includes compression and decompression means ⁽224⁾ to permit the storage of optical images in compressed format.

3. An information storage and retrieval system including a plurality of database management systems as claimed in claim 1

15 characterised in that it includes a pair of indexing means (13.13A) for applying a retrieval header tag to files to be stored on said storage and retireival system and comparator means for verifying the output of one of said pair of indexing means against that of the other.

204. An information storage and retrieval system including a plurality of database management systems as claimed in claim 3 characterised in that it includes lookup table means (213) to supply previously stored data for inclusion with said header tag. 5. An information storage and retrieval system including a

25 plurality of database management systems as claimed in claim 1 characterised in that it includes comparator means for identifying stored files to be retrieved in response to a key-word-based search request.