WO2002082277A1

WO2002082277A1 - Computer system performance improvements by relocation of files on devices

Info

Publication number: WO2002082277A1
Application number: PCT/AU2002/000434
Authority: WO
Inventors: Adrian Sheedy; Colin Lillywhite; Andrew Trigg; James Ross; James O'reilly
Original assignee: Platypus Technology Holdings, Inc.
Priority date: 2001-04-06
Filing date: 2002-04-05
Publication date: 2002-10-17
Also published as: AUPR428701A0

Abstract

The present invention can be used to improve the performance of a computer system. Monitoring software is introduced into computer system so that the monitoring software can identify or assist to identify files on a drive which place a substantial load on the computer system. Identification of files can be performed automatically by software, manually, or by data being sent to a web-site. By monitoring factors for a file such as seek times, read times, write times, etc. the monitoring software can build a list of information which contains data on each file on the drive accessed by the computer system's operating system. Files are then identified that may provide an improvement in system performance if these files were relocated to a faster or improved performance drive. Calculations or estimations may also be made to quantify the expected improvements should files (or a file) be moved to a faster drive. Files loading the computer system are moved to a faster drive. The files may be manually moved, or automatically moved by software provided to perform such a task.

Description

Computer System Performance Improvements By Relocation

Of Files On Devices

Technical Field The present invention relates to the performance of computer systems, and in particular, to obtaining performance improvements of a computer system by first identifying and then relocating certain files on an electronic device which then becomes part of the computer system.

The present invention seeks to provide a method for achieving improved performance of a computer system, and, a computer readable medium of instructions for achieving improved performance of a computer system, or identifying files which if relocated may contribute to improved performance of the computer system.

Background Art

It has become quite evident that system administrators are unaware of which files are accessed by various processes, and, system administrators are unaware as to which files may be generating bottlenecks in a computer system. Presently, it is exceedingly difficult or impossible for an organisation to understand the nature and scope of the Input/Output (I/O) activity occurring within the organisation's computer systems.

In the Applicant's experience, system administrators do not usually know which files on their computer systems are the most heavily utilised (i.e. those which make up the majority of disk or drive I/O activity). In the majority of cases, around 5% of the files/data stored on a computer system's drive constitutes up to 90% of the I/O activity on the computer system.

There currently exists a need for organisations to be able to identify and place selected files, data and/or databases on an improved performance device, such as, for example, a fast solid-state storage device as opposed to a slower storage device. Relocating a file, data, database, program, digital information or the like (these terms being simply referred to hereinafter as a file) on an improved performance device can generate tremendous improvements in computer system performance.

There is also a need for software which can automatically identify such high use files, allowing a system administrator to then relocate the file(s) onto a different device/system as desired.

In order to understand the impact that an improved performance drive can have on the speed of I/O activity one needs to understand the nature of the 110, which can in part be categorized by the volume of transactions and whether the transactions were read or write operations.

This further identifies a need to provide a means to obtain a specific or global view of disk I/O speed. Once this has been established in a normal system, it is then possible to anticipate the impact an improved performance drive can have if certain files are loaded onto an improved performance device, system or drive.

This identifies a need for a new type of method or computer readable medium of instructions for identifying or achieving improved performance of a computer system which overcomes or at least ameliorates the problems inherent in the prior art.

Disclosure Of Invention

The present invention provides a method for improving the performance of a computer system, the computer system including a drive and an improved performance drive, the method including the steps of: obtaining I/O information of the drive of the computer system for files on the drive which are accessed on the computer system, the I/O information obtained by monitoring software; identifying which of the files substantially affect the performance of the computer system by analysing the I/O information; and relocating selected files to the improved performance drive based on the analysis of the I/O information, thereby effecting an improvement in the performance of the computer system without requiring that all files in the computer system be relocated to the improved performance drive. The present invention also seeks to provide a method of identifying to a user possible performance improvements in a computer system by relocating identified files in the computer system, the computer system including a drive and the possibility of attaching an improved performance drive, the method including the steps of: installing monitoring software in the computer system; obtaining I/O information of the drive of the computer system for files on the drive which are accessed on the computer system, the I/O information obtained by the monitoring software; transmitting the I/O information to an analysis web-site; and the analysis web-site presenting the I/O information to the user in a manner that facilitates the user to readily identify which of the monitored files substantially affect the performance of the computer system.

The present invention also seeks to provide a method of estimating an expected performance improvement in a computer system if selected files were to be relocated from a first drive to a second drive, the second drive being an improved performance drive, the method including the steps of: obtaining I/O information of the first drive for a selected file on the first drive; calculating a first Read/Write Performance value as a function of Read/Write operations, Read/Write speeds, file seek time, the volume of data addressed, and the data read stream speed for the selected file on the first drive; obtaining I/O information of the second drive for a selected file on the second drive; calculating a second Read/Write Performance value as a function of Read/Write operations, Read/Write speeds, file seek time, the volume of data addressed, and the data read stream speed for the selected file on the second drive; using the calculated first and second Read/Write Performance values to estimate the expected performance improvement of the computer system if selected files are relocated to the second drive.

The present invention also seeks to provide a method of providing an analysis report to a user of a computer system which identifies a possible performance improvement in the computer system if identified files in the computer system were to be relocated from a drive of the computer system to an improved performance drive, the method including the steps of: obtaining I/O information of the drive of the computer system for files on the drive which are accessed on the computer system, the I/O information obtained by monitoring software resident on the computer system, the I/O information including Drive Read Time, Average Drive Read Queue Length, Drive Write Time, Average Drive Write Queue Length, Drive Read Bytes/sec, Drive Write Bytes/sec, and Processor Time; amalgamating the I/O information into fixed time period snapshots; transmitting the I/O information to an analysis means; and the analysis means generating the analysis report.

The I O information can include seek times, read times and write times. Furthermore, the selected files can be relocated manually by a user, or automatically by the monitoring software or other software as a result of the analysis of the I/O information. Also, I/O information relating to a particular drive, a particular directory on the drive, or a particular file type can be selectively obtained. According to another embodiment reports can be generated or calculations can be performed at an analysis web-site, and I/O information can be transmitted to the web-site as a XML file. Furthermore, I O information can be automatically transmitted to the analysis web-site by the monitoring software.

In a further form, the present invention also provides a computer readable medium of instructions for obtaining I/O information of a drive of a computer system for all files on the drive which are accessed during operation of the computer system, the computer readable medium of instructions including procedures adapted to: monitor the I/O activity of a drive of the computer system; store I/O information for each file accessed on the drive; provide a user interface allowing the user to control aspects of the monitoring of drive I/O activity via the user interface; and output the I/O information for subsequent analysis.

The present invention also provides a computer readable medium of instructions for assisting to achieve improved performance of a computer system, the computer system including a first drive and a second drive, the second drive being an improved performance drive, and the computer readable medium of instructions including procedures adapted to: monitor the I/O activity of at least the first drive of the computer system; store I/O information for each file accessed on at least the first drive; provide a user interface allowing the user to control aspects of the monitoring of at least the first drive I/O activity via the user interface; calculate an expected performance improvement in the computer system if files were to be relocated from the first drive to the second drive; and relocate selected files to the second drive based on the expected performance improvement calculations. The present invention furthermore provides a software tool used in analysing the performance of a drive in a computer system, the software tool residing on the computer system and allowing a user to: start or stop logging I/O activity on the drive; log computer system performance data; log I/O data for a file on the drive; and close the software tool and cease logging.

In a further embodiment, the software tool and the analysis software are integrated. Furthermore, the computer system performance data can include drive, processor, memory and process data. In another form of the invention there is provided a computer readable medium of instructions and/or a method which dynamically relocates selected files on an improved performance drive. In another form of the invention the method and/or computer readable medium of instructions can monitor any or all drives in a computer system. In another form of the invention there is provided a computer readable medium of instructions and/or a method which records the I/O of databases on remote database servers. According to a further embodiment, analysis or reporting software can reside on a web-site.

Brief Description Of Figures

The present invention will become apparent from the following description, which is given by way of example only, of a preferred but non-limiting embodiment thereof, described in connection with the accompanying figures, wherein:

Figure 1 illustrates an embodiment of the present invention wherein, the figure shows a method for achieving improved performance of a computer system;

Figure 2 illustrates an example of a user interface for the monitoring software.

Modes For Carrying Out The Invention

The following modes are described in order to provide a more precise understanding of the subject matter of the present invention. In one embodiment, the present invention provides a method for achieving improved performance of a computer system. It should be noted that in the specification reference is made to a drive of a computer system, however, this is taken to be a reference to any, some or all of the drives in a computer system. That is, the present invention may equally be applied to all drives in a computer system whether those drives are similar devices or not. In accordance with a further particular embodiment of the present invention software is provided which is able to simultaneously or selectively monitor one, some or all drives of a computer system.

Reference to the term drive should also be considered to relate to local or non-local drives, that is, I/O file information may also be monitored where the file is transferred over some form of network - the drive may be remote to some parts of the computer system. As mentioned previously, this may refer to numerous remote drives.

Furthermore, the term drive should be interpreted broadly, a drive is used to refer to any form of memory device which can be incorporated into a computer system. Such devices include, for example, disks, solid state storage devices, magnetic storage devices, optical storage devices, devices that store data using quantum physics phenomenon, etc..

It should also be noted that reference to the term file should be interpreted broadly and be taken to be a reference to any type of computer file, data, information, database, program, record or the like. Moreover, the file itself may contain any type or format of program, data or information, for example, text, ASCII, binary, executable, etc..

I. Preferred embodiment

A preferred, but non-limiting, embodiment of the present invention is shown in figure 1. The method 10 can be used to improve the performance of a computer system. The step 20 involves placing monitoring software in a computer system so that the monitoring software can identify or be used to assist in identifying files on a drive which place a substantial load on the computer system. By monitoring factors such as seek times, read times, write times, etc. (i.e. I/O information) for a file the monitoring software can build a list of information which contains data on each file on the drive accessed by the computer system's operating system.

The step 30 then involves either manually or automatically searching through the data obtained by the monitoring software to identify which files may provide an improvement in system performance if these files were relocated to a faster or improved performance drive. In analysis step 40 calculations, graphs, tables, estimations, etc. can be used to quantify the expected improvements should files (or a file) be moved to a faster drive. It should also be noted that in a particular embodiment actual calculations need not be performed as by simply relocating a number of identified files which substantially load a computer system to a faster drive an improvement in computer system performance should naturally follow. However, it is preferred that a quantitative analysis is made prior to relocation of files.

In step 50 the files substantially loading the computer system according to some criteria, which may or may not be preselected, are moved to a faster drive. The files may be manually moved, or automatically moved by software provided to perform such a task which looks at step 40 calculations, estimations or other related information and moves files accordingly.

A. Software to monitor I/O activity on a drive

In a further embodiment of the present invention there is provided software to monitor I/O activity on a drive (herein referred to as monitoring software or program). The monitoring software can be used to identify files which impact the system, allowing an administrator or other person to move certain identified files onto an improved performance drive such as a fast solid-state drive system. The identification/movement or files may also occur automatically.

Preferably, emphasis is placed on simplicity in the Graphical User interface (GUI) for the monitoring software. In a particular form of the invention, the monitoring software is a single executable file (Operating System (OS) permitting) which generally does not require an installation program. When executed, the monitoring software automatically logs system file events and can display a real-time list of the all files with drive I/O activity. Users can be allowed to customise what they wish to have monitored by selecting which drives, directories or file types they wish to have monitored or analysed.

A Windows (NT, 2000, etc.) GUI interface is presented in figure 2. Naturally, this GUI may be significantly varied but is presented to assist the reader understand the scope of the present invention and the invention's embodiments. Similar command-line versions of the monitoring software can also be built for OS's that do not always have a graphical user interface. The monitoring software can also collect pre-defined statistics from the OS with regard to "system availability" and "system usage". This allows analysis at a later stage of how drive I/O activity impacts other components of the computer system, and whether there is "room" for growth in the speed of drive I/O activity.

The drive I/O activity is amalgamated in fixed-time "snapshots" where each transaction line can be recorded or summarised over the time interval. The performance statistics of the system can continue to be collected in fixed-time intervals. All data is recorded in a file in a directory, for example where the monitoring software is located. Various snapshot time periods, recorded file types or stored data locations could be used. In this embodiment, users would receive the actual monitoring software that collects data of drive I/O activity. If required, this data could then be sent to a specialist for analysis and the preparation of a customer report advising how the user might improve their system.

It should also be noted that in one embodiment the monitoring program can recognise the existence of a database server, for example SQL server, Oracle, or DB2, and log statistics or I/O information specific to these database servers.

B. Analysis of results from monitoring software

In a further embodiment of the present invention there is provided a method or software to analyse the data/results from the monitoring software. Analysis may be provided as a method, a software program (on the host system or a remote system), a series of algorithms embedded in a spreadsheet, manually performed, or a combination of these means. Analysis of results from the monitoring software could be effected by the monitoring software itself or a user transmitting results to a web-site. The web-site may contain analysis software, or the web-site may pass the results to a analysis person or further analysis software.

In order to understand the impact that an improved performance drive can have on the speed of drive I/O activity one needs to understand the nature of the I/O, which can broadly be categorized by the volume of transactions and whether the transactions were read or write operations. This allows one to formulate or estimate a global disk I/O speed, if required. Once this has been established in the system the impact the improved performance drive can have if certain identified files are relocated to the improved performance drive can be estimated.

In one embodiment, the results of the analysis can be presented to a customer (or user) in a report. The first step in preparing an analysis report is to take the data from the monitoring software and then analyse this data. This can provide a performance impact assessment, if all or some of the files having disk I/O activity during the monitoring period are placed on the improved performance drive.

Once this analysis has been completed it can presented to a customer (or any other person) in a manner that allows, for example: Ease of comprehending the performance benefits; Which files are impacting disk I/O performance; The most economical allocation of files to the improved performance drive to meet certain target performance criteria.

Part of the analysis report may show how much a particular process will be impacted by improving I/O performance (remembering that the system processes are not made up exclusively of disk I/O activity). A further part of the analysis report may show which files have the greatest impact on performance if allocated to an improved performance drive such as a solid-state system. A further part of the analysis report may show the performance gains to be made, by the allocation of files to the improved performance drive of a particular size, thereby possibly showing an optimum or acceptable price per performance for allocation of files.

C. Integration of software to monitor I/O activity on a drive and analysis of results from the monitoring software

In a further embodiment the two previous embodiments described separately in sections A and B may be combined to allow monitoring and analysis to be performed on a particular computer system. Analysis software (integrated with monitoring software) can be provided to analyse file I/O information and recommend which files be relocated to an improved performance drive and/or the expected performance improvement if certain files are relocated to the improved performance drive. This combination monitoring and analysis software can be provided with the ability to automatically (or dynamically) relocate files with or without user approval, or may simply inform a user of statistics or recommendations thereby allowing the user to manually relocate files.

In a further embodiment the combined monitoring and analysis methods can be incorporated into a firmware application that can perform the analysis and allocation of files to a faster device (or drive) on the fly, that is by dynamic allocation. In this embodiment the method/software would identify the files best suited for a faster storage device and automatically relocate these files, possibly subject to user selected options.

II. Various embodiments

The following examples provide a more detailed outline of embodiments of the present invention. These examples are intended to be merely illustrative and not limiting of the scope of the present invention.

A. Example improved performance drives

An example of an improved performance drive/device is now presented. In contrast to, for example, a hard disk drive (HDD), a solid state drive (SSD) is generally considered to be an improved performance drive. Examples of SSDs include the drives sold under the brand names QrkDATA and QikDRlNE by the applicant. These SSD drives can be grouped, for example, as 32 bit (QikDRlNE32), 64 bit (QikDRlNE64), QikDATA (including all performance variants), etc.. It should be reinforced to the reader that the present invention applies to any type or form of improved performance drive/device which can become part of a computer system and on which files can be relocated, and should not be considered to be limited to only SSDs.

B. Software to monitor I/O activity on a drive

General: A further, more detailed, description of the monitoring software follows. The monitoring software is preferably a single executable file (OS permitting) that does not require an installation program. When executed, the monitoring software can begin automatically logging system file events and show a real-time list of the all files with drive I/O activity. There can be provided an option to save the list so that the list can be viewed elsewhere or -l ie-mailed to an organisation or person for analysis. Users are able to filter the drives and files to be monitored. Users can select which drive(s) will be monitored, which directory or directories, and even which files or file types.

In a particular form, the monitoring software program has been developed for Windows NT, 2000 or XP, however, other platforms and applications can be supported. In one form, the monitoring software consists of an executable and a kernel-mode file system filter driver. The filter driver is actually contained within the executable. It is written to the disk and loaded at runtime, then deleted when the monitoring software exits. Installation can simply require putting the executable onto a local drive. During initialisation, the monitoring software collects the list of performance counters on the system and then informs the file system to notify the system driver every time a program reads or writes data. This way the monitoring software can log all read and write activity to all files.

After the monitoring software has been closed, the filter driver is no longer logging file system activity, but it is not deactivated and deleted until the system is rebooted. This is because data loss may occur if the driver is forcibly removed from the device list during normal system activity. Inserting filter drivers during system operation is safe. It is acceptable to delete the actual system file, as its image is normally loaded into the system. The monitoring software logs all data to a .xml file in the same directory as the monitoring software. As the file extension suggests, the format is XML. Logging happens at a fixed time period, for example every five seconds. An option can be provided for the time period to be set by a user. A summary of performance data and file I/O can be generated and written to the log file. A command-line version can also be built for OS's that do not always have a graphical user interface.

Data Collection:

The drive or disk I/O activity can be accumulated in two ways. Firstly it can be amalgamated in fixed-time "snapshots" or each transaction line can be recorded. The performance statistics of the system can continue to be collected in fixed time intervals, for example five-second intervals. All data is recorded in a file in a particular directory. Architecture:

The fundamental architecture is not required to differ between operating systems, but the application would. The monitoring software integrates into the file system of the OS and watches all activity. In order to collect realistic data, it is important that the monitoring software has a low impact on overall system performance. It should have a relatively small memory profile, and have minimal impact on the CPU.

The Business Process:

In one particular, non-limiting, embodiment users would only receive a monitoring software program that collects data and sends it to a specialist organisation for analysis. The identification of the most-used files and any drive I/O graphs are not performed or provided on the users system. The organisation can generate reports from the data sent to the organisation and send the reports back to the user.

Operating Safety Components:

By its very nature the monitoring software may build up a sizeable database of transactions over time that could lead to an adverse impact on the computer system's overall performance. To prevent such system degradation features are provided that can shut down the monitoring software, or possibly only log when under load. The shut down features can include a time limit, a monitoring software log file size limit, a disk free space limit and a monitoring software CPU usage limit.

A possible GUI that a user sees is visually simple and is illustrated in figure 2.

Output:

The output .xml file typically has two components as its output. The first is logged when recording starts and details which performance statistics will be monitored and which processes are currently running on the system. An example is shown in Table I.

Table I

</counter>

</counter>

</counter>

</counter>

</counter>

</counter>

</counter>

</counter>

<instance number="4" name="WINLOGON'7> instance number="5" name="SERVICES"/>

</counter>

</perfobjects>

The second component of the output is the fixed-period time interval snapshots, for example five-second periods, and summations of all performance statistics and disk I/O activity. The performance statistics are shown in Table II using a menu system.

Table II

The disk I/O activity statistics collected can include:

Path The full path and file-name accessed for disk I/O activity during the interval Transactions The count of reads and writes that have been performed on the file during the interval Bytes Read The bytes read from the file during the interval

Bytes Written The bytes written to the file during the interval

Time The time taken to complete the disk I/O activity Process The process (or program) that has been performing the reads and writes to and from the file These statistics are shown in the example in Table III.

Table III

<fιleactivity>

<fιle name="G:\SQL_DATA\DELOITTE_data01.mdf ' trans="937" read="97738752" write="0" time="6406236" proc="sqlservr.exe:896"/>

<fιle name="G:\SQL_DATA\DELOITTE_data02.ndf ' trans="920" read="93044736" write="0" time=" 12366608" proc="sqlservr.exe:896"/>

<fιle name="F:\SQL_LOG\DELOITTE_log01.1df ' trans="7" read="0" write="6656" time="318368" proc="sqlservr.exe:896"/>

<fιle name="C:\$LogFile" trans="3" read="0" write=" 12288" time="28559643" proc="System:8"/>

<fιle name="C:\$LogFile" trans="2" read="0" write="8192" time=" 14271336" proc="VxSvc.exe:10327>

<fιle name="G:\$LogFile" trans="2" read="0" write="8192" time="49832882" proc="VxSvc.exe:1032"/> <file name="F:\$LogFile" trans="2" read="0" write="8192" time=" 1634577" proc="VxSvc.exe: 1032"/>

The monitoring software can also collect pre-defined statistics from the operating system with regard to "system availability" and "system usage". The disk I/O activity is amalgamated in five-second "snapshots" where each transaction line can be recorded or summarised over the time interval. The performance statistics of the system can continue to be collected in five-second intervals. It should be noted that data can be recorded in an ASCII file (eg. .txt, .xml, ect.), or any other desirable type of file, in any location, for example the root directory of the monitoring software.

General operation:

Using the monitoring software is relatively simple. Referring to the embodiment as illustrated in figure 2, a user sets the number of minutes to log file I/O and clicks Start Logging. The program will stop after the selected time has elapsed. All data is logged to the .xml file in the same directory as the program.

Instead of setting the number of minutes, the user may simply click Start and Stop during the times that logging is desired. Preferably, an .xml file is appended to earlier files rather than files being overwritten. For improved clarity during analysis, it is recommended that a meaningful description of what is being logged be entered into the Comment field prior to clicking Start. After logging data is completed, the .xml file is ready for analysis. Possibilities for proceeding with analysis of logged data of I/O information include, emailing to an address for analysis or uploading to a web-site for analysis, where the analysis could be performed manually or automatically by appropriate software, or having further software resident on the user's system perform the analysis.

The monitoring software measures the successful requests from programs running on the system as they are sent to and from a storage device driver(s). So when a program wants to read the 100^th byte from a file, the monitoring software logs a 1-byte read. The storage device driver often turns this into a read of an entire 4KB block. But it will only pass back the contents of the requested byte. The time taken to return this single byte is also measured, so the monitoring software knows how long each transaction takes. The monitoring software's concept of a transaction is also defined as a single request for an arbitrary length of data from (or to) a file. So a transaction could be any length, 1 byte or longer.

Typically, there are five aspects to I/O collected by the monitoring software. These are relevant to each transaction (or request): (1) Is it a read or a write request?; (2) What file is the request for?; (3) How much data is requested?; (4) How long did the request take to complete?; (5) What process initiated the request?

Functionality: The following function references refer to figure 2. If desirable, only selected elements of the following functionality can be provided to a user. The following functionality merely indicates possible functionality.

Start Logging: The 'Start Logging' button is illustrated in figure 2 and begins the process of storing all file I/O and performance data to the .xml file. A date/time stamp is written to the .xml file every time this button is clicked. For example:

Stop Logging: This button ends the current logging activity and readies the monitoring software to continue logging.

About: Displays a simple form to take a user to an analysis organisation's home page or email technical support.

Exit: This button stops any logging that is happening and closes the monitoring software.

Options: Options can be provided to the user by the GUI. Examples could include: Stop Logging after (minutes):

60

If this checkbox is selected, then the monitoring software will only log for the selected number of minutes.

This checkbox will force a check of the Logfile size before it is written to at each data collection point. If the selected size is exceeded, then logging will cease.

Stop logging when less than 5MB free disk space:

This checkbox forces a check of the free disk space and stops logging if there is less than 5MB free. Naturally, other quantities of free space could be provided for.

Stop logging when monitoring software CPU is over 10%:

This checkbox forces a check of the CPU overhead that the monitoring software is causing. If it is too high then logging will cease. Naturally, other quantities of CPU % could be provided for.

Log System Performance Data:

This checkbox will cause logging of some performance counters at the same time the file I/O is logged. The counters that can be logged include (non-exhaustive list):

PhysicalDisk: % Disk Read Time PhysicalDisk: Avg. Disk Read Queue Length PhysicalDisk: % Disk Write Time PhysicalDisk: Avg. Disk Write Queue Length PhysicalDisk: Disk Read Bytes/sec PhysicalDisk: Disk Write Bytes/sec Processor: % User Time Processor: % Privileged Time Memory: Pages/sec Memory: Available KBytes Process: % Processor Time Log only under I/O load:

This checkbox causes logging to take place only when the disk queue length is over, for example, 1.5/sec, or the bytes read/written /sec is over 500,000.

Log File I/O Activity:

This checkbox selects whether I/O activity will be logged into the output .xml file. The 'Summary' radio button causes the I/O transactions to be summarised for each file in the sampling period. The 'All Transactions' radio button logs every transaction as it happens.

The user can also be provided with the option of entering a comment describing what is being logged in a comment field of the GUI.

An upload data option, for example and upload XML data option, can also be provided by the GUI. Such a feature can enable a user to directly upload the log file produced by the program to an analysis web-site.

C. Analysis of results from monitoring software

Introduction:

An analysis procedure can be used to analyse file I/O information, system activity and drive I/O activity. Such analysis may include: Predicting a performance improvement on a system by installing an improved performance drive, possibly showing a graph of predicted performance increases as a function of relocating certain files to the improved performance dπve; or a view of the data collected can display a graph of the disk access over time. This will enable users to see a difference between I/O on, for example, a standard HDD and on an improved performance drive.

In one non-limiting embodiment, the monitoπng software can send I/O information (logged data or information forming log files) to an analysis web-site. The analysis web- site is designed to permit uploading of the monitoring software log files from anywhere via the internet, and then display various reports on the data. The purpose of the analysis web-site is to quickly decide if there is really an I/O bottleneck that a high-performance storage system can fix, and then to identify which files or directories should benefit the most. As a specific example, the analysis web-site can be built on Java servlets and XSL. In this embodiment, a user would preferably be required to login to the web-site. After a successful login, the user can choose to upload log files from a local hard drive. Uploading is simply a matter of browsing for the log file, entering a description, and clicking an Upload button. It can also be provided that the web-site would recognise any file compression and automatically uncompress.

After choosing which log file to analyse, an overview of the data can be displayed, including time started, finished, and any comments that were entered into the monitoring software. The available report types can also be listed.

Reports:

Once the analysis has been completed it can be presented to a customer/user in a manner that can, for example, allow for: Ease of comprehending the performance benefits; Which files are impacting disk I/O performance; The most economical allocation of files to the improved performance drive to meet certain target performance criteria.

In one particular non-limiting embodiment, the reports can be presented on the analysis web-site after the analysis has been automatically performed by the web-site. Types of reports can include the following:

File I/O Statistics (Top 20): This report is actually three-in-one: Top 20 files by bytes read; Top 20 files by bytes written; Top 20 files by the number of transactions. If there were less than 20 files in any category then only those are listed. Naturally, a number of files other than 20 may be chosen.

File I/O Statistics (All Files): This report is a list of all reads and writes on all drives over time. Each five second snapshot is listed and the reads, writes, and transaction count are shown. The initial view shows only the data, but clicking on links in the report can show rudimentary bar graphs of the scaled activity over time. This allows a user to scroll down the page and see the trend in I/O over time. Clicking on the date and time link shows the detail of all I/O data during that snapshot.

Performance Counter Statistics: The perfmon-type data that the monitoring software collects can be presented with optional graphs to assist in identifying important events. Such graphs also assist a user to spot trends in system performance.

Processor Usage Statistics: Each process running on the system is recorded and the average amount of CPU time the process used in the last five seconds (or other set time period) is recorded. The graphs assist in quickly identifying the CPU drains and the trends throughout the time data was collected.

Total Directory Activity: This report displays the cumulative amount of data read and written to every directory where I/O was recorded. This allows a user to instantly view which directories could be moved to high-performance storage. In situations where there is a small amount of I/O to many files this report can calculate the totals and possibly present a very different view to the Top 20 report.

Total Disk Activity: This report shows the I/O to each logical disk or drive in the system. By using this report a user can see the drives that had the most I/O activity.

Total Process Activity: This report lists the processes in the system that created I/O, and the amount of I/O they performed. This allows a user to identify the processes that are causing the most significant I/O activity on the system.

Snapshot Details Report: Clicking on a snapshot link in any of the previous reports brings up details about that five second snapshot.

A full or further analysis request can also be provided for on the analysis web-site by a user clicking an icon. Typically, such a full or further analysis request would be performed manually, although not necessarily.

The analysis website could also provide: The ability to select portions of a log file to analyse, for example if most of the I/O activity was in a short identifiable period;

Automatic correction of malformed XML in an uploaded file; A knowledge-base of analysis information; and Analysis of SQL Server, Oracle, or DB2 database data.

Naturally, the presentation of the analysis report can be significantly varied or customised according to a particular situation or customer. Furthermore, the aforementioned or other analysis reports could be provided in paper form, by email, as a computer file, etc.. The reports or the analysis itself can be produced or performed by software, mathematical algorithms, manually, etc. or any combination thereof, and thus may be automatically or manually generated, in whole or in part.

Analysis Methodology: Generally, there are two desired outputs from analysing I/O information log files: (1) to establish if there is an I/O bottleneck, and ascertain the cause; and (2) to produce a report that can be passed back to the customer/user.

There are a number of steps that assist in the analysis process: Note the start and end times. The amount of time that the monitoring software was collecting data will come to bear on the overall I/O amounts. This time should be related to the figures in the rest of the analysis.

View the Total Disk Activity report. This list of logical drives and the I/O to each drive gives an overview of the I/O to each drive. If the I/O is relatively small over time, then this system can immediately be ruled out as one that does not have an I/O bottleneck. View the File I O Statistics (Top 20) report. The three tables usually show an immediate trend of large I/O to particular files. Also note the top 20 by transaction table.

Files may have relatively low I/O but that I/O could be made up of a large number of transactions. View the Processor Usage Statistics report. Viewing the graphs of this report can show high CPU usage processes and trends over time.

View the Performance Counter Statistics report. Viewing the graphs can provide useful information. Several counters can be listed that may or may not provide useful information: Processor utilisation - both system and user time for each processor; Read and write bytes per disk; Disk queue lengths; Memory utilisation; Hard page faults.

View the File I/O (All Files) report with graphs. Scroll and view trends in I/O over time. Drill into useful snapshot details if required.

View the Total Process Activity report. This shows each process on the system and how much I/O the processes performed. This report can be very useful in identifying the high I/O programs running on the server.

View the Total Directory Activity report. Noting the directories that contained the most I/O, indicates those that should benefit the most from a faster storage device.

Following is a list of notable aspects of the data that are often relevant to the analysis process:

Qdmon process: If this process is running then an improved performance drive, such as a QikDATA drive, was once or still is installed in the system.

Low amount of I/O: The report showing the amount of I/O per drive shows if there is enough I/O to warrant further investigation. These figures should be weighed against the amount of time the monitoring software was running. For example, 500MB of I/O is significant if the time is only 1 minute, but insignificant if the time is, say, 4 hours.

High CPU Usage: It seems that many administrators believe that their servers are I/O bound when there is clearly a serious CPU bottleneck. The CPU usage graphs should be checked for servers that have no idle CPU. In these cases, higher performing storage devices will not generally help improve overall system performance.

High CPU Usage and High Transaction Count: Sometimes, the system overhead of managing a huge transaction load can be the reason for high CPU usage. This can mean that fast storage may actually reduce the CPU time that the system uses, allowing more processing for other processes or even more I/O. Transaction Count: As there are system and seek overheads in each request that a program makes for I/O, a very high number of transactions can signify that the I/O is being spread over random parts of a disk - perfect for a storage system that can handle high transaction loads. On the other hand, a relatively low transaction count with high volume can signify a streaming application that can benefit from fast I/O. Single Threaded Applications: Many applications do not divide their operation into multiple threads that can take advantage of more than one processor. These applications never utilize more than one processor, but such applications usually do not have any processor affinity set. This translates to (for example) the process using no more than 25% of each of the four CPU's in a quad-processor system. A single-threaded application can be switched to run on any processor, but can only be running on one processor at a time. These applications can often cause the bottleneck, not I/O. Remedies may include: Re-develop the application to be multithreaded Run multiple instances of the application Purchase a single-processor system with a much faster CPU

Upgrade the application to a version that is multi-threaded. The NTVDM (NT Virtual DOS Machine) Process: This process is a container for 16-bit DOS and Windows applications. If it is a 16-bit Windows application then the WOW (Windows-on-Windows) process will also be running. Processes: It is reasonably common to find a process that is not recognized.

Internet searches can usually help. A basic understanding of what the program is and does can help to appreciate the I/O that the program is performing. It is also important to be able to recognize the various uses that a server may have, as this may lead to the identification of other opportunities.

What Defines High I/O?: The answer to this question can be complex. However, Table IN shows possible guidelines. The information in Table IN is provided as estimates only and could reasonably be expected to change according to particular circumstances.

Table IV

Type of I/O Low Medium High

Read bytes per second < 100,000 100,000 to 300,000 > 300,000

Write bytes per second < 100,000 100,000 to 300,000 > 300,000

Read bytes per 5 < 400,000 400,000 to 1,000,000 > 1,000,000 seconds

Write bytes per 5 < 300,000 300,000 to 800,000 > 8,000,000 seconds

Transactions per second < 10 10 to 100 > 100

Transactions per 5 < 50 50 to 500 > 500 seconds Thus, there has been provided in accordance with the present invention, a method for achieving or assisting to achieve improved performance of a computer system and a computer readable medium of instructions for achieving or assisting to achieve improved performance of a computer system which satisfies the advantages set forth above.

The invention may also be said broadly to consist in the parts, elements and features refeπed to or indicated in the specification of the application, individually or collectively, in any or all combinations of two or more of said parts, elements or features, and where specific integers are mentioned herein which have known equivalents in the art to which the invention relates, such known equivalents are deemed to be incorporated herein as if individually set forth.

Although the preferred embodiment has been described in detail, it should be understood that various changes, substitutions, and alterations can be made herein by one of ordinary skill in the art without departing from the scope of the present invention as hereinbefore described and as hereinafter claimed.

Claims

The claims defining the invention are as follows:

1. A method for improving the performance of a computer system, the computer system including a drive and an improved performance drive, the method including the steps of: obtaining I/O information of the drive of the computer system for files on the drive which are accessed on the computer system, the I/O information obtained by monitoring software; identifying which of the files substantially affect the performance of the computer system by analysing the I/O information; and relocating selected files to the improved performance drive based on the analysis of the I/O information, thereby effecting an improvement in the performance of the computer system without requiring that all files in the computer system be relocated to the improved performance drive.

2. The method as claimed in claim 1, wherein the I/O information includes seek times, read times and write times.

3. The method as claimed in either claim 1 or claim 2, wherein the selected files are relocated manually by a user, or automatically by the monitoring software or other software as a result of the analysis of the I/O information.

4. The method as claimed in any one of the claims 1 to 3, wherein the drive I/O information is amalgamated in preselected time period snapshots.

5. The method as claimed in any one of the claims 1 to 4, wherein the analysis of the I/O information is performed by a specialist person.

6. The method as claimed in claim 5, wherein an analysis report is prepared after the analysis.

7. The method as claimed in any one of the claims 1 to 6, wherein the I/O information is transmitted to a web-site for analysis.

8. The method as claimed in any one of the claims 1 to 7, wherein the improved performance drive is a SSD.

9. The method as claimed in any one of the claims 1 to 8, wherein I/O information relating to a particular drive, a particular directory on the drive, or a particular file type can be selectively obtained.

10. A method of identifying to a user possible performance improvements in a computer system by relocating identified files in the computer system, the computer system including a drive and the possibility of attaching an improved performance drive, the method including the steps of: installing monitoring software in the computer system; obtaining I/O information of the drive of the computer system for files on the drive which are accessed on the computer system, the I/O information obtained by the monitoring software; transmitting the I/O information to an analysis web-site; and the analysis web-site presenting the I/O information to the user in a manner that facilitates the user to readily identify which of the monitored files substantially affect the performance of the computer system.

11. The method as claimed in claim 10, wherein calculations of an expected performance improvement are performed at the analysis web-site.

12. The method as claimed in either claim 10 or 11, wherein the I/O information is transmitted to the web-site as a XML file.

13. The method as claimed in any one of the claims 10 to 12, wherein the I/O information is automatically transmitted to the analysis web-site by the monitoring software.

14. The method as claimed in any one of the claims 10 to 13, wherein the I/O information is transmitted to the analysis web-site by the monitoring software in response to the user activating a transmit function of the monitoring software.

15. The method as claimed in any one of the claims 10 to 14, wherein the I/O information is transmitted to the analysis web-site by uploading a file to the analysis website.

16. The method as claimed in any one of the claims 10 to 15, wherein the analysis web-site presents at least one analysis report to the user based on the I/O information.

17. The method as claimed in claim 16, wherein the at least one analysis report is a File I/O Statistics (Top 20) report; File I/O Statistics (All files) report; Performance

Counter Statistics report; Processor Usage Statistics report; Total Directory Activity report; Total Disk Activity report; Total Process Activity report; or Snapshot Details report.

18. A method of estimating an expected performance improvement in a computer system if selected files were to be relocated from a first drive to a second drive, the second drive being an improved performance drive, the method including the steps of: obtaining I/O information of the first drive for a selected file on the first drive; calculating a first Read/Write Performance value as a function of Read/Write operations, Read/Write speeds, file seek time, the volume of data addressed, and the data read stream speed for the selected file on the first drive; obtaining I/O information of the second drive for a selected file on the second drive; calculating a second Read/Write Performance value as a function of Read/Write operations, Read/Write speeds, file seek time, the volume of data addressed, and the data read stream speed for the selected file on the second drive; using the calculated first and second Read/Write Performance values to estimate the expected performance improvement of the computer system if selected files are relocated to the second drive.

19. The method as claimed in claim 18, wherein the I/O information is obtained by computer system monitoring software.

20. A method of providing an analysis report to a user of a computer system which identifies a possible performance improvement in the computer system if identified files in the computer system were to be relocated from a drive of the computer system to an improved performance drive, the method including the steps of: obtaining I/O information of the drive of the computer system for files on the drive which are accessed on the computer system, the I/O information obtained by monitoring software resident on the computer system, the I/O information including Drive Read Time, Average Drive Read Queue Length, Drive Write Time, Average Drive Write Queue Length, Drive Read Bytes/sec, Drive Write Bytes/sec, and Processor Time; amalgamating the I/O information into fixed time period snapshots; transmitting the I/O information to an analysis means; and the analysis means generating the analysis report.

21. The method as claimed in claim 20, wherein the analysis means is a web-site.

22. The method as claimed in claim 20, wherein the analysis means is analysis software resident on the computer system.

23. The method as claimed in claim 20, wherein the analysis means is a person.

24. A computer readable medium of instructions for obtaining I/O information of a drive of a computer system for all files on the drive which are accessed during operation of the computer system, the computer readable medium of instructions including procedures adapted to: monitor the I/O activity of a drive of the computer system; store I O information for each file accessed on the drive; provide a user interface allowing the user to control aspects of the monitoring of drive I/O activity via the user interface; and output the I/O information for subsequent analysis.

25. The claim according to claim 24, wherein the computer readable medium of instructions is adapted to simultaneously monitor more than one drive of the computer system.

26. The claim according to claim 24, wherein the computer readable medium of instructions is adapted to selectively monitor more than one drive of the computer system.

27. The claim according to any one of the claims 24 to 26, wherein the drive is a non- local drive.

28. The claim according to any one of the claims 24 to 27, wherein the drive is a disk, HDD, SSD, magnetic storage device or optical storage device.

29. The claim according to any one of the claims 24 to 28, wherein the user controlled aspects include: monitoring time limit, output file size limit, free drive space limit, CPU overhead limit, drive queue length, or output file type.

30. The claim according to any one of the claims 24 to 29, wherein the I/O information includes seek times, read times and write times.

31. The claim according to any one of the claims 24 to 30, wherein I/O information relating to a particular drive, a particular directory on the drive, or a particular file type can be selectively obtained.

32. The claim according to any one of the claims 24 to 31, wherein the I/O information is transmitted by the computer readable medium of instructions to a web-site for analysis.

33. A computer readable medium of instructions for assisting to achieve improved performance of a computer system, the computer system including a first drive and a second drive, the second drive being an improved performance drive, and the computer readable medium of instructions including procedures adapted to: monitor the I/O activity of at least the first drive of the computer system; store I/O information for each file accessed on at least the first drive; provide a user interface allowing the user to control aspects of the monitoring of at least the first drive I/O activity via the user interface; calculate an expected performance improvement in the computer system if files were to be relocated from the first drive to the second drive; and relocate selected files to the second drive based on the expected performance improvement calculations.

34. The claim according to claim 33, wherein the files are automatically relocated to the second drive by the computer readable medium of instructions.

35. The claim according to either of the claims 33 or 34, wherein the files are relocated to the second drive by the computer readable medium of instructions after acceptance of the file relocation by a user.

36. The claim according to any one of the claims 33 to 35, wherein the files are dynamically relocated to the second drive.

37. The claim according to any one of the claims 33 to 36, wherein the expected performance improvement is calculated as a function of the Read/Write operations, the

Read Write speeds, the file seek times, the volume of data addressed, or the data read stream speed for the selected file.

38. A software tool used in analysing the performance of a drive in a computer system, the software tool residing on the computer system and allowing a user to: start or stop logging I/O activity on the drive; log computer system performance data; log I/O data for a file on the drive; and close the software tool and cease logging.

39. The software tool as claimed in claim 38, wherein the software tool is a single executable file plus additional support files.

40. The software tool as claimed in either claim 38 or claim 39, wherein the software tool obtains a real-time list of all files associated with drive I/O activity.

41. The software tool as claimed in any one of the claims 38 to 40, wherein the I/O activity is amalgamated into preselected time period snapshots.

42. The software tool as claimed in any one of the claims 38 to 41, wherein the I/O data for the drive is transmitted to analysis software adapted to analyse the performance of the drive.

43. The software tool as claimed in claim 42, wherein the software tool and the analysis software are integrated.

44. The software tool as claimed in either of the claims 42 or 43, wherein the analysis software resides on a web-site.

45. The software tool as claimed in any one of the claims 38 to 44, wherein the software tool includes shut-down features.

46. The software tool as claimed in claim 45, wherein the shut down features include a time limit, a software log file size limit, a disk free space limit, and/or a software CPU usage limit.

47. The software tool as claimed in any one of the claims 38 to 46, wherein the computer system performance data includes drive, processor, memory and process data.

48. The claim according to any one of the claims 24 to 47, wherein the drive I/O activity includes path, transaction, bytes read, bytes written, time and process details.