WO2000029993A2

WO2000029993A2 - Method and system for determining a selection of significant quantities among a number of quantities describing a technical system

Info

Publication number: WO2000029993A2
Application number: PCT/DE1999/003497
Authority: WO
Inventors: Peter Liggesmeyer; Michael Rettelbach
Original assignee: Siemens Aktiengesellschaft
Priority date: 1998-11-13
Filing date: 1999-11-02
Publication date: 2000-05-25
Also published as: WO2000029993A3

Abstract

During the description of a technical system a number of quantities exist among which those providing the greatest amount of information for a modelling of the system are to be determined. To this end a simple-evaluation method and a holdout method are combined so that for said quantities a number of quantities which best describe the system is determined analytically.

Description

description

Method and arrangement for determining a selection of significant quantities from a quantity of quantities describing a technical system

The invention relates to a method and an arrangement for determining a selection of significant variables from a set of variables describing a technical system.

Progress in the implementation of a technical system is often assessed on the basis of certain variables that describe the technical system. The sizes usually have different qualitative expressions. If an expert uses his or her expertise and experience to determine significant sizes from a number of different sizes, this selection is usually not optimal. It could be that a different selection would lead to a better assessment, in particular of the progress of the implementation of the technical system.

Another approach is followed in [1]. An algorithm for determining significant quantities is given there. However, [1] is based on the assumption that the values of the sizes are normally distributed, which is often not the case in practice.

A holdout method is understood below to use a number of n-i data records to determine a model or a model description (n is the number of data records available), the i remaining data records being used to evaluate the determined model. In the method described here, i = 1 is set in particular.

The object of the invention is a method and an arrangement for automatically determining a selection to determine significant sizes, in particular not having to start from normally distributed values of the sizes.

This task is carried out according to the characteristics of the independent

Claims resolved. Further developments of the invention also result from the dependent claims.

To solve the problem, a method for determining a selection of significant quantities from a quantity describing a technical system is specified, in which a number of n data records is specified, on the basis of which models for the technical system are determined in the context of a first model calculation. The models are classified based on the data sets, with a first one each

Classification measure for the quality of the classification is determined. A holdout procedure is carried out on the data records in the following steps:

a) for the model on which the best first classification measure is based from the first

Model calculations are carried out using the holdout method, a second model calculation and a second classification;

b) if a second classification measure determined by the classification is better than, equal to or worse than the first classification measure by a predetermined threshold value, the quantities belonging to the second classification dimension are determined as significant quantities and the method is ended;

c) if the second classification measure determined by the second classification is worse than the first by more than the predetermined threshold value

Classification measure is the next best first classification measure underlying model of the first model calculation, the second model calculation and the second classification carried out according to the holdout method and branched to step b).

The loop in the described method ensures in particular that the same is properly scheduled.

A particular advantage is that those quantities are automatically determined whose information content is the greatest for describing the technical system, in particular with regard to a forecast during implementation.

If, for example, a prediction regarding timely completion is to be made for a system that is well ahead of its completion in a specific implementation phase, then those variables that were determined according to the method described above are used as meaningful. In particular, each individual size can be weighted.

A further development is that the significant sizes are used for the design of the technical system. The design can represent a new design, an adaptation or a control of the technical system.

In a further development, in particular by determining current values for the respective significant quantities, the significant quantities can be used to forecast the timing and / or quality of the completion of the technical system.

It is also a further development that the technical system is a plant (process engineering), a circuit or a software cycle, that is, the process of software generation from Specifying the requirements for the software until the integration test has been completed. In the software cycle in particular there are individual phases on the basis of which a forecast, for example the time of completion, decides on a further procedure in the software cycle. With early detection of a possibly delayed completion, measures can be taken in time to counteract this. For example, the number of employees working on a project can be increased in order to guarantee the timeliness mentioned.

It is also a further development that the significant quantities are each weighted by a factor. This shows the influence of each size on the result. Different sizes, in the example of the forecast, have different effects on the time of completion.

In particular, the quantities mentioned can be measured values or metrics. In the software cycle there are different methods for measuring the progress or for validating a quality of the software. These methods are reflected in metrics, which can influence quantities in the process described here.

One embodiment consists in that a model is determined as part of the first model calculation by using a predetermined part of the n data records or all n data records for the calculation. In particular, the model can be calculated by performing a discriminant analysis or a regression analysis.

It is also a further development that the sizes are weighted with factors and those sizes are determined as significant sizes whose values of the factors are greater than a predetermined threshold value. As part of additional training, the model is assessed by determining a measure that indicates how well the respective data set is described by the model.

To solve the problem, an arrangement for

Determination of a selection of significant quantities from a set of quantities describing a technical system, in which a processor unit is provided which is set up in such a way that a) a number of n data records is specified, on the basis of which models for the technical system can be determined in the context of a first model calculation are; b) the models can be classified on the basis of the data sets, a classification measure for the quality of the classification being determined in each case; c) a holdout process can be carried out on the data records in the following steps: aa) for the model on which the best classification measure is based from the first model calculation, the holdout process is used to carry out a separate model calculation and classification; bb) if a classification measure determined by the classification is better, equal or worse than that by a predetermined threshold value

Classification measure of the first model calculation is branched to step d); cc) if the classification measure determined by the classification of the holdout procedure is worse than the classification measure of the first model calculation by more than the predetermined threshold value, the model calculation and the classification according to the holdout procedure become the model for the next best classification measure carried out and branched to step bb); d) the quantities belonging to the determined classification measure can be determined as significant quantities.

This arrangement is particularly suitable for carrying out the method according to the invention or one of its developments explained above.

Embodiments of the invention are illustrated and explained below with reference to the drawing.

Show it

Fig.l is a flow chart illustrating steps of a method for automatically determining significant quantities describing a technical system;

2 shows a processor unit;

3 shows a sketch which represents a system, a circuit or a software cycle as possible variants of the technical system.

DATA AND METRICS

In the following, 329 projects are assumed. There is a data set for each individual project, which in particular has the following metrics (also: quantities, measured quantities):

- the planned duration of the project in months (Ml); - the estimated hours worked in hours (M2);

- the total cost of the project (M3);

These sizes Ml, M2 and M3 are known at the beginning of the project. After a certain time, the following sizes are still determined:

- Percentage of hours actually worked (M4);

- share of costs already spent (M5); - The square of the proportionate difference between the two variables M4 and M5, in order to determine whether a planning disproportionately deviates from a target value (M6).

In addition, depending on the project, other different sizes (metrics) can be recorded.

STATISTICAL PROCEDURES

To determine a model that describes the underlying technical system, methods of discriminant analysis (or alternatively: regression analysis) are applied to the quantities. A common goal is to predict an overrun of the planned total costs at the earliest possible stage for a project that is currently running. In [2] the result is a vector

1 = (lχ, 1 ₂ ,. • •, l _m ) (1) with an index m for the number of quantities (metrics)

and a scalar size c, so that a linear combination

liMl + l2 2 + ... + l _m M (2)

follows. The scalar quantity c is used to classify a project in one of two classes. This is done by comparing the threshold value c with whether the costs for the project to be assessed are met or not (this corresponds to two states as Result of the evaluation). CALCULATION OF THE CLASSIFICATION

To determine the quality of the classification, two relationships in particular are taken into account:

An error rate (AER) includes all misclassifications compared to a total number of classifications, whereas the expected misclassifications (ECM) represent costly misclassifications compared to the total classifications. It is assumed that n data records are subdivided into n ^ data records that identify projects that were not successfully completed in the planned time with the planned costs and n ₂ data records that could be successfully completed in the planned period. n [and n ₂ are based on the available data sets n, which were recorded for different cases from previous projects. These results can be predicted within the framework of the classification by models, the prediction by the model being rated as correct or incorrect in each case

(Classification). The following illustration illustrates this connection:

The following relationships apply in particular:

n = n ^ _ + n ₂ (3)

ⁿ l = ⁿ ll + ⁿ 12 (4)

ⁿ 2 = ⁿ 21 + ⁿ 22 (5) This means that a number n] _ι and a number n ₂₂ observations are correctly predicted (classified). A number n] _ ₂ observations and a number n ₂ χ observations are each incorrectly predicted. The error rate value is determined as follows:

n ₁₂ + n ₂₁

AER = (6] n

The expected errors in the incorrect classifications are determined taking into account the costs cχ ₂ and c ₂ ι for each type of error classification:

E _C M = ⁿ 12 ^{• c} 12 + "21 ^• c _{21 (?)} N

In order to calculate both values for the quantities AER and ECM, all data records are included in the model formation in a first model calculation, and all data records are used to classify the model. This procedure leads to a "good-natured" recognition of the data records, since exactly the data records used for the model determination are to be recognized again. Such a detection method is also referred to below as "simple evaluation".

Another possible approach is to perform a model calculation using only part of the

Data sets for determining the model and the remaining part of the data sets for classification (in particular: fits / does not fit) are used. This has the disadvantage that not all data records are used for model determination in the case of small amounts of data records, and information is therefore dispensed with in the model formation. Now, especially with small amounts of data records, it is desirable to use as much information as possible in the data records for the modeling. A special form of this so-called holdout procedure, in the version selected here, uses all data records apart from one data record for model formation and verifies the model using the remaining data record. This process is carried out for all data records, so that a number of

"• (2" -!) (8)

Calculations is necessary. As a result, the available computing capacity is quickly exceeded in the case of a large number of data records.

Assuming the example with 329 data records, for which values were determined on the basis of 10 quantities, according to equation (8) there are a total of approximately 336,000 calculations for determining the models. To reduce this effort, the procedure is as follows: First, using the "simple evaluation" method, classification measures are calculated, which are sorted according to size. The best classification measure points to a selection of metrics. This selection of metrics is given in a holdout process, which in turn carries out a model calculation and results in a subsequent classification with a classification measure. The classification measure of the holdout procedure is compared with the previous classification measure (from the "simple evaluation" procedure) and, if the classification measure of the holdout procedure is greater than or equal to the classification measure of the "simple evaluation" procedure, the selection of metrics as significant selection determined. In the other case, the next best classification measure of the "simple evaluation" classification is determined and, as described above, introduced into the holdout process. This process is iterated until a result of the holdout process reaches a sufficiently high classification quality. This relationship is explained again with reference to Fig.l.

FIG. 1 shows a flowchart which shows steps of a method for the automatic determination of significant quantities describing a technical system.

In a step 101, a first model calculation MB1 is carried out with n data records. The "simple evaluation" method described above is suitable for this. In a step 102, the first model calculation MB1 is classified, with classification measures KMI being determined. In particular, a separate classification measure is determined for each model of the model calculation (each selection of metrics stands for a separate model). The classification measures are ordered according to quality. In step 103, the best classification measure is determined and the underlying selection of metrics is used in a holdout process. For this purpose, a second model calculation MB2 and a second classification with associated classification measures KM2 are determined in the holdout method (see step 104).

The aim is to reduce the number of arithmetic steps so that, if possible, no complete holdout procedure has to be carried out. However, it must now be ensured that the largest possible classification measure can be determined early in the holdout process, so that the entire process is terminated as early as possible. To do this, constants for the selected metrics are newly determined in the model calculation using the holdout method. These newly determined constants result (in a way) in a new one, after the

Holdout method determined model that is evaluated with the data set not taken into account in the holdout method for determining the model. This results in a classification measure that is used in direct comparison to the classification measure according to the "simple evaluation" method. In a step 105, the first classification measure is compared with the second classification measure. In particular, if the second classification measure is greater than or equal to the first classification measure, the selection of metrics is determined as being characteristic of the technical system and the method is ended in a step 107. Otherwise, the next best classification measure of the first classification measure determination (according to the si ple evaluation method) is determined in a step 108 and a branch is made to step 104.

A processor unit PRZE is shown in FIG. The processor unit PRZE comprises a processor CPU, a memory SPE and an input / output interface IOS, which is used in different ways via an interface IFC: output is visible on a monitor MON and / or on a printer via a graphic interface PRT issued. An entry is made using a mouse MAS or a keyboard TAST. The processor unit PRZE also has a data bus BUS, which ensures the connection of a memory MEM, the processor CPU and the input / output interface IOS. Furthermore, additional components can be connected to the data bus BUS, e.g. additional memory, data storage (hard disk) or scanner.

A sketch is shown in FIG. 3, which shows the characteristics of a technical system 301. The technical system can thus be a system 302 (for example from process engineering), a circuit 303 (in particular an electrical circuit) or a software cycle 304. Bibliography :

[1] Ludwig Fahrmeir, Alfred Hamerle, Gerhard Tutz:

"Mulitvariate statistical methods," deGruyter, Berlin, 2 ^nd edition., 1996

[2] Richard A. Johnson, Dean W. Wichern: "Applied

Multivariate Statistical Analysis ", Prentice Hall, 1982.

Claims

claims

1. Method for determining a selection of significant quantities from a set of quantities describing a technical system,

a) in which a number of n data records is specified, on the basis of which models for the technical system are determined as part of a first model calculation;

b) in which the models are classified on the basis of the data sets, a classification measure for the quality of the classification being determined in each case;

c) in which a holdout procedure is carried out on the data records in the following steps:

aa) for the model on which the best classification measure is based from the first model calculation, a separate model calculation and classification are carried out using the holdout method;

bb) if a classification measure determined by the classification is better, equal or worse than the classification measure of the first model calculation by a predetermined threshold value, a branch is made to step d);

cc) if the classification measure determined by the classification of the holdout procedure is worse than the classification measure of the first model calculation by more than the predetermined threshold value, the model calculation and the classification according to the holdout procedure become the model for the next best classification measure carried out and branched to step bb);

d) in which the variables belonging to the determined classification measure are determined as significant variables.

2. The method of claim 1, wherein the significant sizes are used to design the technical system.

3. The method of claim 2, wherein the design represents a redesign, an adaptation or control of the technical system.

4. The method according to any one of claims 1 to 3, in which, based on the significant quantities, a forecast is made with regard to the time and / or quality of the completion of the technical system.

5. The method according to any one of claims 1 to 4, wherein the technical system is a system, a circuit or a software cycle.

6. The method according to any one of the preceding claims, wherein the significant sizes are weighted.

7. The method according to any one of the preceding claims, in which a model is determined in the context of the first model calculation by using a predetermined part of the n or all n data records for the calculation.

8. The method according to any one of the preceding claims, wherein the model is calculated by performing a discriminant analysis or a regression analysis.

9. The method according to any one of the preceding claims, in which the sizes are weighted with factors and those sizes are determined as significant sizes whose values of the factors are greater than a predetermined threshold value.

10. The method as claimed in one of the preceding claims, in which the model is classified by determining a classification measure for a predetermined part of the n or for all n data records, which indicates how well the model is described by the respective data record.

11. The method according to any one of the preceding claims, in which the currently determined values of the variables are inserted into the model linked to the significant variables and a current forecast is carried out.

12. Arrangement for determining a selection of significant quantities from a quantity describing a technical system, in which a processor unit is provided which is set up in such a way that

a) a number of n data records is specified, on the basis of which models for the technical system can be determined as part of a first model calculation;

b) the models can be classified on the basis of the data sets, a classification measure for the quality of the classification being determined in each case;

c) A holdout procedure can be carried out on the data records in the following steps:

aa) for the model on which the best classification measure is based from the first Model calculations are carried out with the holdout procedure, a separate model calculation and a separate classification;

cc) if the classification measure determined by the classification of the holdout method is worse than the classification measure of the first model calculation by more than the predetermined threshold value, the next best for that

Model on which the classification measure is based carried out the model calculation and the classification according to the holdout method from the first model calculation and branched to step bb);

the quantities belonging to the determined classification measure can be determined as significant quantities.