CN101807046A - Online modeling method based on extreme learning machine with adjustable structure - Google Patents

Abstract

The invention discloses an online modeling method based on an extreme learning machine with adjustable structure, belongs to the fields of automatic control, information technology and advanced manufacturing, in particular to a method for adjusting the structure and parameters of an extreme learning machine to hold newly acquired data in the online learning process of the extreme learning machine. The method is characterized by comprising the following steps: defining the concept of a category ball; judging whether the newly acquired data is out of the category ball and leads to reduction of modeling accuracy or not in every learning process; if yes, adding a new hidden node; if not, only adjusting the center and the radius of the category ball, and updating the weight of the output layer of the extreme learning machine at last. The method firstly introduces the concept of the category ball for holding the used data in a previous training process, when determining the parameters of the newly added hidden node, the output of the node at a point nearest to the category ball is made to small enough so as to guarantee the output value of the node on the used data is zero, and a formula for updating the weight of the output layer is given. The method can enhance the online modeling accuracy by adding the hidden node.

Description

A kind of line modeling method based on extreme learning machine with adjustable structure

Technical field

The invention belongs to automatic control, infotech and advanced manufacturing field, be specifically related in the on-line study process of extreme learning machine, adjust its structure and parameter to hold the method for new acquisition data.

Background technology

In the many modeling environments that detect, control and optimize towards actual industrial process, the modeling desired data usually has the characteristics that arrive successively.At These characteristics, academia and industry member have proposed line modeling method (or on-line study method), as RAN, RANEKF, MRAN, GAP-RBF, GGAP-RBF, these class methods can be according to the online data adjustment model structure and parameter of new generation, holding new data information, and need not at the modeling again of acquired all data.Wait to regulate that parameter is many, training speed waits deficiency slowly but said method mostly exists, have a strong impact on its practical application effect.Though the OS-ELM method that occurs is reduced to one with parameter to be regulated recently, it lacks adjustability of structure, and the ability that makes it hold fresh information is relatively limited, and model accuracy can't further improve.

Summary of the invention

For solving an above-mentioned line modeling difficult problem, the present invention proposes a kind of line modeling method based on extreme learning machine with adjustable structure (being called for short SAO-ELM).In SAO-ELM, (Extreme Learning Machine: extreme learning machine) network is identical, but its number of hidden nodes can be regulated in the line modeling process with ELM for the basic structure of network.The main difficult point that increases hidden node in modeling process is, the training objective of SAO-ELM is to make adjusted model reach minimum with respect to the sum of errors of all training datas, but in each on-line study process, the training data of before using must abandon, and this makes and increases hidden node output the unknown on those data that abandoned newly.For this reason, the present invention has defined classification ball notion, in order to surrounding the training data that all had been used, and record and according to the centre of sphere and the radius of newly-increased this ball of Data Update at any time.When increasing hidden node, its excitation function is elected Gaussian function as, select the center and the width of suitable excitation function then, make this node enough little, thereby can be considered as 0 increasing the output of hidden node on the data that abandoned newly in the output of distance classification ball closest approach.Under these conditions, can derive the iterative output layer right value update formula when increasing hidden node, thereby realize line modeling based on extreme learning machine with adjustable structure.

A kind of line modeling method based on extreme learning machine with adjustable structure is characterized in that this method realizes according to following steps:

Step (1): Model Selection and parameter initialization

The number of hidden nodes M of single hidden layer extreme learning machine is set, and the input layer number is identical with the training sample dimension n, and the output node number is identical with the dimension m of training objective;

Excitation function G (a of hidden node _i, b _i, x) adopt Gaussian function, determine the center a of each hidden node at random _iWith width b _i, i=1,2 ... M;

According to an initial N sample

The training extreme learning machine obtains initial hidden layer output matrix H ₀With output layer connection matrix β ₀, wherein,

β ₀＝(H ₀ ^TH ₀) ^-1H ₀T ₀

${\mathrm{<figure-callout\; id="0"\; label="T"\; filenames="HSA00000031561100022.png"\; state="\{\{state\}\}">T</figure-callout>}}_0=\left[\begin{array}{c}{t}_{11}\&CenterDot;\&CenterDot;\&CenterDot;t_{1m}\\ \&CenterDot;\&CenterDot;\&CenterDot;\\ {\mathrm{<figure-callout\; id="1"\; label="t\; N"\; filenames="HSA00000031561100022.png,HSA00000031561100023.png"\; state="\{\{state\}\}">t</figure-callout>}}_N\&CenterDot;\&CenterDot;\&CenterDot;t_{\mathrm{Nm}}\end{array}\right]={\left[\begin{array}{c}{t}_1^T\\ \&CenterDot;\&CenterDot;\&CenterDot;\\ t_N^T\end{array}\right]}_{N\&times;m}$

The initialization matrix K makes

Calculate Preserve β ₀, K ₀And P ₀

Surround initial training sample set X with a classification ball O ₀, make this ball just with X ₀In all sample points be enclosed in its inside, determine the centre of sphere C of this ball ₀And radius R ₀

Step (2): on-line study process

Newly-increased training data x ₁=(x _N+1, t _N+1) when arriving, according to following steps training ELM, make it can preserve X ₀Knowledge, can hold x again ₁The new knowledge that is comprised

Step (2.1): it is constant to keep network structure, only according to x ₁Adjust output layer connection matrix β ₀, the output layer weights connection matrix after the renewal is β ₁, upgrade matrix K simultaneously ₀And P ₀Be K ₁And P ₁,

${\mathrm{\&beta;}}_1={\mathrm{\&beta;}}_0+P_1{H}_1^T(T_1-{\mathrm{<figure-callout\; id="1"\; label="H"\; filenames="HSA00000031561100022.png,HSA00000031561100023.png"\; state="\{\{state\}\}">H</figure-callout>}}_1{\mathrm{\&beta;}}_0)$

${\mathrm{<figure-callout\; id="1"\; label="P"\; filenames="HSA00000031561100022.png,HSA00000031561100023.png"\; state="\{\{state\}\}">P</figure-callout>}}_1=K_1^{-1}=P_0-P_0{H}_1^T{(I+H_1{P}_0{H}_1^T)}^{-1}{H}_1{\mathrm{<figure-callout\; id="0"\; label="P"\; filenames="HSA00000031561100022.png"\; state="\{\{state\}\}">P</figure-callout>}}_0$

${\mathrm{<figure-callout\; id="1"\; label="K"\; filenames="HSA00000031561100022.png,HSA00000031561100023.png"\; state="\{\{state\}\}">K</figure-callout>}}_1=K_0+H_1^T{H}_1$

Wherein, H ₁And T ₁Be respectively ELM to new samples x ₁Hidden layer output matrix and training objective matrix, promptly

H ₁＝[G(a ₁，b ₁，x _N+1)…G(a _M，b _M，x _N+1)] _1×M

${\mathrm{<figure-callout\; id="1"\; label="T"\; filenames="HSA00000031561100022.png,HSA00000031561100023.png"\; state="\{\{state\}\}">T</figure-callout>}}_1=[t_{(N+1)1}\&CenterDot;\&CenterDot;\&CenterDot;t_{(N+1)m}]={\left[t_{N+1}^T\right]}_{1\&times;m}$

Step (2.2): the ELM after the calculating adjustment parameter is to newly-increased sample x ₁Training error e, judge new samples x ₁Whether outside ball O, if outside ball O and e greater than setting threshold, abandon above-mentioned all adjustment, turn to step (2.3), otherwise, turn to step (3);

Step (2.3): increase a hidden node, setting its center a is x ₁, width b is definite by following formula, promptly

$b\&le;-\frac{\left|\right|x_c-a\left|\right|}{\ln \mathrm{\&epsiv;}}$

Wherein, ε is the threshold value that presets, x _cFor the last coordinate of classification ball O, can determine by following formula apart from new sample point x1 closest approach

x _c＝x _o+λ ₁(x _a-x _o)

Wherein, x _oBe the sphere centre coordinate of ball O, λ ₁Can determine by following formula

${\mathrm{\&lambda;}}_1=\frac{\left|\right|x_c-x_o\left|\right|}{\left|\right|x_a-x_o\left|\right|}=\frac{R}{\left|\right|x_a-x_o\left|\right|}$

X in the following formula _aBe new sample point x ₁Coordinate; Readjust output layer connection matrix β ₀Be β ₁, and correspondingly upgrade matrix K ₀And P ₀Be K ₁And P ₁, make

${\mathrm{\&beta;}}_1=P_1\left[\begin{array}{c}{\mathrm{<figure-callout\; id="0"\; label="K"\; filenames="HSA00000031561100022.png"\; state="\{\{state\}\}">K</figure-callout>}}_0{\mathrm{\&beta;}}_0+H_1^T{T}_1\\ H_{11}^{\mathrm{<figure-callout\; id="1"\; label="T\; T"\; filenames="HSA00000031561100022.png,HSA00000031561100023.png"\; state="\{\{state\}\}">T</figure-callout>}}{T}_{}{}_1\end{array}\right],K_1=\left[\begin{array}{cc}{K}_0+H_1^T{H}_1& H_1^T{H}_{11}\\ H_{11}^T{H}_1& H_{11}^T{H}_{11}\end{array}\right],{\mathrm{<figure-callout\; id="1"\; label="P"\; filenames="HSA00000031561100022.png,HSA00000031561100023.png"\; state="\{\{state\}\}">P</figure-callout>}}_1=K_1^{-1}=\left[\begin{array}{cc}{A}_{11}& A_{12}\\ A_{21}& A_{22}\end{array}\right]$

$A_{11}={\mathrm{<figure-callout\; id="1"\; label="P"\; filenames="HSA00000031561100022.png,HSA00000031561100023.png"\; state="\{\{state\}\}">P</figure-callout>}}_1^{\&prime;}+{\mathrm{<figure-callout\; id="1"\; label="P"\; filenames="HSA00000031561100022.png,HSA00000031561100023.png"\; state="\{\{state\}\}">P</figure-callout>}}_1^{\&prime;}\left(H_1^T{H}_{11}\right)R^{-1}\left(H_{11}^T{H}_1\right){\mathrm{<figure-callout\; id="1"\; label="P"\; filenames="HSA00000031561100022.png,HSA00000031561100023.png"\; state="\{\{state\}\}">P</figure-callout>}}_1^{\&prime;},A_{12}=-{\mathrm{<figure-callout\; id="1"\; label="P"\; filenames="HSA00000031561100022.png,HSA00000031561100023.png"\; state="\{\{state\}\}">P</figure-callout>}}_1^{\&prime;}\left(H_1^T{H}_{11}\right)R^{-1},A_{21}=A_{12}^T,A_{22}=R^{-1}$

$R=H_{11}^T{H}_{11}-\left(H_{11}^T{H}_1\right){\mathrm{<figure-callout\; id="1"\; label="P"\; filenames="HSA00000031561100022.png,HSA00000031561100023.png"\; state="\{\{state\}\}">P</figure-callout>}}_1^{\&prime;}\left(H_1^T{H}_{11}\right)$

${\mathrm{<figure-callout\; id="1"\; label="P"\; filenames="HSA00000031561100022.png,HSA00000031561100023.png"\; state="\{\{state\}\}">P</figure-callout>}}_1^{\&prime;}={(K_0+H_1^T{H}_1)}^{-1}=P_0-P_0{H}_1^T{(I+H_1{P}_0{H}_1^T)}^{-1}{H}_1{\mathrm{<figure-callout\; id="0"\; label="P"\; filenames="HSA00000031561100022.png"\; state="\{\{state\}\}">P</figure-callout>}}_0,{\mathrm{<figure-callout\; id="0"\; label="P"\; filenames="HSA00000031561100022.png"\; state="\{\{state\}\}">P</figure-callout>}}_0=K_0^{-1}$

Wherein, H ₀₁And H ₁₁Be respectively newly-increased hidden node to former sample set X ₀With new sample point x ₁The hidden layer output matrix, promptly

N ₁Be respectively x with L ₁In sample point number and newly-increased hidden node number, this method is only considered the situation that new sample point arrives one by one, so N ₁=L=1;

Step (3): the parameter of upgrading classification ball O

Upgrade the parameter of classification ball O, promptly upgrade its sphere centre coordinate and radius, make new ball O ₁Just with X ₀And x ₁In all sample points all be enclosed in wherein, its more new formula be:

$R_{\mathrm{new}}=\frac{\left|\right|x_a-x_b\left|\right|}{2}$

$x_{o\_\mathrm{new}}=\frac{x_a+x_{\mathrm{<figure-callout\; id="2"\; label="b"\; filenames="HSA00000031561100022.png"\; state="\{\{state\}\}">b</figure-callout>}}}{2}$

Wherein, x _aAnd x _bBe respectively new sample point x ₁Coordinate and ball O go up apart from x ₁The coordinate in solstics, x _bCan calculate by following formula,

x _b＝x _o+λ ₂(x _o-x _a)

Wherein, x _oBe the coordinate of centre of sphere O, λ ₂Can calculate by following formula,

${\mathrm{\&lambda;}}_2=\frac{\left|\right|x_b-x_o\left|\right|}{\left|\right|x_o-x_a\left|\right|}=\frac{R}{\left|\right|x_o-x_a\left|\right|}$

According to above-mentioned line modeling method, the present invention has done a large amount of l-G simulation tests, can find out that from simulation result the line modeling method that the present invention proposes has higher learning accuracy than other line modeling methods, the model of setting up with the method also has better extensive performance.

Description of drawings

Fig. 1: algorithm flow chart is each step based on the line modeling method specific implementation of extreme learning machine with adjustable structure that the present invention proposes.

Fig. 2: the synoptic diagram of the classification ball of all data that the encirclement training process had been used, wherein, the other ball O of group is that encirclement all except that newly-increased training data had been used the classification ball of data, big classification ball O ₁Be to surround all to have used the classification ball of data and newly-increased data.

Fig. 3: the Gaussian function synoptic diagram, wherein, middle peak value is its output in the Gaussian function center, the less output valve of edge is that it is in the output of distant place from the Gaussian function center.

Fig. 4: the graph of a relation that training precision and checking precision change with the number of hidden nodes in the emulation experiment, wherein, red curve is the variation relation of training precision with the number of hidden nodes, green curve is the variation relation of checking precision with the number of hidden nodes.

Fig. 5: line modeling process synoptic diagram in the emulation experiment, wherein, the variation relation that Fig. 5 .1 increases with training data for the checking precision, Fig. 5 .2 are the variation relation that the number of hidden nodes increases with training data.

Embodiment

The line modeling method based on extreme learning machine with adjustable structure that the present invention proposes, its main advantage is can adjust network structure as required in the line modeling process.According to the characteristics of line modeling, if just learn when having newly-increased training data to arrive, have model now and predict, and along with the increase of training data, precision of prediction can progressively be improved otherwise just use.

Below to being elaborated that the present invention proposes based on the related step of the line modeling method of extreme learning machine with adjustable structure:

The first step, Model Selection

For the method that the present invention proposes, Model Selection only relates to determines initial ELM the number of hidden nodes M.The present invention adopts the method for cross validation to determine initial hidden node number: the initial training data are divided into two parts, and a part is used for training, and another part is used for checking; Since a less latent node number, train ELM with training data earlier, obtain the checking error with verification msg then, progressively increase the hidden node number again, and repeat above-mentioned training and verification step, last, selecting to make the number of hidden nodes of checking error minimum is initial the number of hidden nodes.

Second step, the initialization of model

The initialization of model is the initialization of model parameter.The method that the present invention proposes adopts the ELM network structure, and the excitation function of hidden node adopts Gaussian function, so at first want initialized parameter that the center a of Gaussian function is arranged _iWith width b _i, i=1,2 ... M, a _iAnd b _iFrom the random number that meets specific distribution, choose respectively.Secondly, determine the initial number of samples of participating in training.In the present invention, to being used for the model of classification problem, initial number of training is elected M+100 as; For the model that is used for regression problem, then elect M+50 as.The principle of determining the initial training sample number is to make H ₀The row full rank.At last, with the classification ball S of a radius minimum ₀Surround the initial training data, remember that its centre of sphere and radius are respectively C ₀And R ₀

The 3rd goes on foot, and determines the initial value of training process data

So-called training process primary data comprises hidden layer output matrix H, output layer connection matrix β, calculates the intermediary matrix K and the inverse matrix K thereof of output layer connection matrix needs ^-1

If the initial training data are

Then corresponding hidden layer output matrix is

Risk minimization (ERM) principle rule of thumb, the objective function of asking for output layer connection matrix β is min (‖ F-T ₀‖)=min (‖ H _{0 (N * M)}β _{(M * m)}-T _{0 (N * m)}‖), wherein, T ₀Be training objective, promptly

${\mathrm{<figure-callout\; id="0"\; label="T"\; filenames="HSA00000031561100022.png"\; state="\{\{state\}\}">T</figure-callout>}}_0=\left[\begin{array}{c}{t}_{11}\&CenterDot;\&CenterDot;\&CenterDot;t_{1m}\\ \&CenterDot;\&CenterDot;\&CenterDot;\\ {\mathrm{<figure-callout\; id="1"\; label="t\; N"\; filenames="HSA00000031561100022.png,HSA00000031561100023.png"\; state="\{\{state\}\}">t</figure-callout>}}_N\&CenterDot;\&CenterDot;\&CenterDot;t_{\mathrm{Nm}}\end{array}\right]={\left[\begin{array}{c}{t}_1^T\\ \&CenterDot;\&CenterDot;\&CenterDot;\\ t_N^T\end{array}\right]}_{N\&times;m}$

According to the knowledge of matrix, find the solution easily above-mentioned optimization problem separate for

Wherein

Be H ₀The pseudoinverse of matrix.Work as matrix H ₀During the row full rank, i.e. rank (H ₀During)=M, have

Be convenient and derive, introduce intermediary matrix K, make K=H ^TH, then

Note P=K ^-1, then

The 4th step, do not adjust the ELM structure, only come match to increase data newly by adjusting ELM output layer weights

If newly-increased data are

(N in fact, in the method ₁Elect 1 as; And N ₁Situation greater than 1 then all can be converted into N ₁Equal 1 situation processing), then corresponding therewith hidden layer output matrix and training objective are respectively:

${\mathrm{<figure-callout\; id="1"\; label="T"\; filenames="HSA00000031561100022.png,HSA00000031561100023.png"\; state="\{\{state\}\}">T</figure-callout>}}_1=\left[\begin{array}{c}{t}_{(N+1)1}\&CenterDot;\&CenterDot;\&CenterDot;t_{(N+1)m}\\ \&CenterDot;\&CenterDot;\&CenterDot;\\ t_{(N+N_1)1}\&CenterDot;\&CenterDot;\&CenterDot;t_{(N+N_1)m}\end{array}\right]={\left[\begin{array}{c}{t}_{N+1}^T\\ \&CenterDot;\&CenterDot;\&CenterDot;\\ t_{N+N_1}^{\mathrm{<figure-callout\; id="1"\; label="T\; N"\; filenames="HSA00000031561100022.png,HSA00000031561100023.png"\; state="\{\{state\}\}">T</figure-callout>}}& \\ \end{array},\right]}_{N_{}}$

At this moment, the objective function of asking for new output layer connection matrix β is

$\min (\left[\begin{array}{c}{H}_0\\ {\mathrm{<figure-callout\; id="1"\; label="H"\; filenames="HSA00000031561100022.png,HSA00000031561100023.png"\; state="\{\{state\}\}">H</figure-callout>}}_1\end{array}\right]\mathrm{\&beta;}-\left[\begin{array}{c}{T}_0\\ T_1\end{array}\right])$

, obtain above-mentioned optimization problem separate for

${\mathrm{\&beta;}}_1={\left(K_1\right)}^{-1}{\left[\begin{array}{c}{H}_0\\ H_1\end{array}\right]}^T\left[\begin{array}{c}{T}_0\\ {\mathrm{<figure-callout\; id="1"\; label="T"\; filenames="HSA00000031561100022.png,HSA00000031561100023.png"\; state="\{\{state\}\}">T</figure-callout>}}_1\end{array}\right],{\mathrm{<figure-callout\; id="1"\; label="K"\; filenames="HSA00000031561100022.png,HSA00000031561100023.png"\; state="\{\{state\}\}">K</figure-callout>}}_1={\left[\begin{array}{c}{H}_0\\ H_1\end{array}\right]}^T\left[\begin{array}{c}{H}_0\\ {\mathrm{<figure-callout\; id="1"\; label="H"\; filenames="HSA00000031561100022.png,HSA00000031561100023.png"\; state="\{\{state\}\}">H</figure-callout>}}_1\end{array}\right]$

For realizing the target of line modeling, β ₁Must and H ₀, T ₀Irrelevant, and only be about P ₀, K ₀, β ₀Function.Derive and can get by simple mathematical,

${\mathrm{\&beta;}}_1={\mathrm{\&beta;}}_0+K_1^{-1}{H}_1^T(T_1-{\mathrm{<figure-callout\; id="1"\; label="H"\; filenames="HSA00000031561100022.png,HSA00000031561100023.png"\; state="\{\{state\}\}">H</figure-callout>}}_1{\mathrm{\&beta;}}_0)$

${\mathrm{<figure-callout\; id="1"\; label="K"\; filenames="HSA00000031561100022.png,HSA00000031561100023.png"\; state="\{\{state\}\}">K</figure-callout>}}_1={\mathrm{<figure-callout\; id="0"\; label="K"\; filenames="HSA00000031561100022.png"\; state="\{\{state\}\}">K</figure-callout>}}_0+H_1^{\mathrm{<figure-callout\; id="1"\; label="T\; H"\; filenames="HSA00000031561100022.png,HSA00000031561100023.png"\; state="\{\{state\}\}">T</figure-callout>}}{H}_{}{}_1$

Above-mentioned two formula are not to be adjusted the ELM structure and only adjusts ELM training algorithm under its parameter situation.

In the 5th step, judge that the ELM that obtains with the 4th training method that goes on foot is at newly-increased data X ₁On training error whether meet the requirements, also need judgment data X simultaneously ₁Whether at classification ball S ₀Outside.If training error does not meet the demands, and X ₁At S ₀Outside, then turned to for the 6th step, otherwise, turned to for the 7th step.

The 6th step increased a hidden node, adjusted output layer then and connected weights

Under the situation that increases a hidden node, the hidden layer output matrix becomes

$\left[\begin{array}{cc}{H}_0& H_{01}\\ H_1& H_{11}\end{array}\right]$

Wherein, H ₀₁For increasing the data set X that latent node had been used with respect to training process newly ₀Output, H ₁₁For increasing hidden node newly with respect to increasing newly and unworn data set X ₁Output.Correspondingly, the objective function of asking for new output layer connection matrix β becomes

$\min (\left[\begin{array}{cc}{H}_0& H_{01}\\ H_1& H_{11}\end{array}\right]\mathrm{\&beta;}-\left[\begin{array}{c}{T}_0\\ T_1\end{array}\right])$

But because the data set X that had used ₀Be dropped before latent node increases, this causes H ₀₁The unknown must be handled H earlier well so will ask for β ₀₁Unknown problem.

From the synoptic diagram (Fig. 3) of Gaussian function as can be seen, latent node can be considered 0 in the output away from the center.If data set X ₀And X ₁Synoptic diagram as shown in Figure 2, among the figure, S ₀The data of having used for training process of surrounding, the A point is newly-increased data present position.So if the A point is elected at the center that will increase hidden node newly as, then as long as its output of ordering at C is enough little, also i.e. requirement

$e^{\frac{-\left|\right|x_c-a\left|\right|}{b}}\&le;\mathrm{\&epsiv;}\&DoubleRightArrow;b\&le;-\frac{\left|\right|x_c-a\left|\right|}{\ln \mathrm{\&epsiv;}}$

Wherein ε is previously selected threshold value, x _cBe the C point coordinate, can determine by following two formulas

x _c＝x _o+λ ₁(x _a-x _o)

${\mathrm{\&lambda;}}_1=\frac{\left|\right|x_c-x_o\left|\right|}{\left|\right|x_a-x_o\left|\right|}=\frac{R}{\left|\right|x_a-x_o\left|\right|}$

Wherein, x _oBe classification ball S ₀Sphere centre coordinate, x _aBe the coordinate that A is ordered, R is classification ball S ₀Radius.

Select according to said method after the center a and width b of newly-increased hidden node, the output that newly-increased hidden node is ordered at C is just less than a very little real number ε, and it is at classification ball S ₀Within output then littler so that can be considered 0, so H ₀₁Can be considered 0 matrix.At this moment, the hidden layer output matrix behind the increase hidden node is

$\min (\left[\begin{array}{cc}{\mathrm{<figure-callout\; id="0"\; label="H"\; filenames="HSA00000031561100022.png"\; state="\{\{state\}\}">H</figure-callout>}}_0& 0\\ H_1& H_{11}\end{array}\right]\mathrm{\&beta;}-\left[\begin{array}{c}{T}_0\\ T_1\end{array}\right])$

Find the solution above-mentioned optimization problem, can get

${\mathrm{\&beta;}}_1=({{\left[\begin{array}{cc}{\mathrm{<figure-callout\; id="0"\; label="H"\; filenames="HSA00000031561100022.png"\; state="\{\{state\}\}">H</figure-callout>}}_0& 0\\ H_1& H_{11}\end{array}\right]}^{\mathrm{<figure-callout\; id="0"\; label="T\; H"\; filenames="HSA00000031561100022.png"\; state="\{\{state\}\}">T</figure-callout>}}\left[\begin{array}{c}{H}_{}\end{array}\right]\left[\begin{array}{cc}{}_0& 0\\ H_1& H_{11}\end{array}\right])}^{-1}{\left[\begin{array}{cc}{\mathrm{<figure-callout\; id="0"\; label="H"\; filenames="HSA00000031561100022.png"\; state="\{\{state\}\}">H</figure-callout>}}_0& 0\\ H_1& H_{11}\end{array}\right]}^T\left[\begin{array}{c}{T}_0\\ {\mathrm{<figure-callout\; id="1"\; label="T"\; filenames="HSA00000031561100022.png,HSA00000031561100023.png"\; state="\{\{state\}\}">T</figure-callout>}}_1\end{array}\right]$

If order

$H=\left[\begin{array}{c}{H}_0\\ {\mathrm{<figure-callout\; id="1"\; label="H"\; filenames="HSA00000031561100022.png,HSA00000031561100023.png"\; state="\{\{state\}\}">H</figure-callout>}}_1\end{array}\right],\mathrm{\&delta;H}=\left[\begin{array}{c}0\\ H_{11}\end{array}\right]$

Then

${\mathrm{\&beta;}}_1={\left({\left[\begin{array}{cc}H& \mathrm{\&delta;H}\end{array}\right]}^T\left[\begin{array}{cc}H& \mathrm{\&delta;H}\end{array}\right]\right)}^{-1}{\left[\begin{array}{cc}H& \mathrm{\&delta;H}\end{array}\right]}^T\left[\begin{array}{c}{T}_0\\ {\mathrm{<figure-callout\; id="1"\; label="T"\; filenames="HSA00000031561100022.png,HSA00000031561100023.png"\; state="\{\{state\}\}">T</figure-callout>}}_1\end{array}\right]$

At this moment,

${\mathrm{<figure-callout\; id="1"\; label="K"\; filenames="HSA00000031561100022.png,HSA00000031561100023.png"\; state="\{\{state\}\}">K</figure-callout>}}_1={\left[\begin{array}{cc}H& \mathrm{\&delta;H}\end{array}\right]}^T\left[\begin{array}{cc}H& \mathrm{\&delta;H}\end{array}\right]=\left[\begin{array}{c}{H}^T\\ {\mathrm{\&delta;H}}^T\end{array}\right]\left[\begin{array}{cc}H& \mathrm{\&delta;H}\end{array}\right]$

Order

$K_1^{-1}=A=\left[\begin{array}{cc}{A}_{11}& A_{12}\\ A_{21}& A_{22}\end{array}\right]=({\left[\begin{array}{c}{H}^T\\ {\mathrm{\&delta;H}}^T\end{array}\right]\left[\begin{array}{cc}H& \mathrm{\&delta;H}\end{array}\right])}^{-1}$

Wherein, each element of matrix A is

A ₁₁＝(H ^TH) ^-1+(H ^TH) ^-1(H ^TδH)×R ^-1(δH ^TH)(H ^TH) ^-1

A ₁₂＝-(H ^TH) ^-1(H ^TδH)R ^-1，

A ₂₂＝R ^-1

Wherein, R=δ H ^Tδ H-(δ H ^TH) (H ^TH) ^-1(H ^Tδ H), the expression formula substitution with H and δ H gets:

$A_{11}={({\mathrm{<figure-callout\; id="0"\; label="K"\; filenames="HSA00000031561100022.png"\; state="\{\{state\}\}">K</figure-callout>}}_0+H_1^T{H}_1)}^{-1}+{({\mathrm{<figure-callout\; id="0"\; label="K"\; filenames="HSA00000031561100022.png"\; state="\{\{state\}\}">K</figure-callout>}}_0+H_1^T{H}_1)}^{-1}\left(H_1^T{H}_{11}\right)R^{-1}\left(H_{11}^T{H}_1\right){({\mathrm{<figure-callout\; id="0"\; label="K"\; filenames="HSA00000031561100022.png"\; state="\{\{state\}\}">K</figure-callout>}}_0+H_1^T{H}_1)}^{-1}$

$A_{12}=-{({\mathrm{<figure-callout\; id="0"\; label="K"\; filenames="HSA00000031561100022.png"\; state="\{\{state\}\}">K</figure-callout>}}_0+H_1^T{H}_1)}^{-1}\left(H_1^T{H}_{11}\right)R^{-1},$ $A_{21}=A_{12}^T,$ A ₂₂＝R ^-1

$R=H_{11}^T{H}_{11}-\left(H_{11}^T{H}_1\right){({\mathrm{<figure-callout\; id="0"\; label="K"\; filenames="HSA00000031561100022.png"\; state="\{\{state\}\}">K</figure-callout>}}_0+H_1^T{H}_1)}^{-1}\left(H_1^T{H}_{11}\right)$

Comprehensive above various can be under the situation that increases hidden node, the more new formula of K, P and β is:

${\mathrm{\&beta;}}_1=P_1\left[\begin{array}{c}{\mathrm{<figure-callout\; id="0"\; label="K"\; filenames="HSA00000031561100022.png"\; state="\{\{state\}\}">K</figure-callout>}}_0{\mathrm{\&beta;}}_0+H_1^T{T}_1\\ H_{11}^{\mathrm{<figure-callout\; id="1"\; label="T\; T"\; filenames="HSA00000031561100022.png,HSA00000031561100023.png"\; state="\{\{state\}\}">T</figure-callout>}}{T}_{}{}_1\end{array}\right],$ ${\mathrm{<figure-callout\; id="1"\; label="K"\; filenames="HSA00000031561100022.png,HSA00000031561100023.png"\; state="\{\{state\}\}">K</figure-callout>}}_1=\left[\begin{array}{cc}{\mathrm{<figure-callout\; id="0"\; label="K"\; filenames="HSA00000031561100022.png"\; state="\{\{state\}\}">K</figure-callout>}}_0+H_1^T{H}_1& H_1^T{H}_{11}\\ H_{11}^T{H}_1& H_{11}^T{H}_{11}\end{array}\right],$ ${\mathrm{<figure-callout\; id="1"\; label="P"\; filenames="HSA00000031561100022.png,HSA00000031561100023.png"\; state="\{\{state\}\}">P</figure-callout>}}_1=K_1^{-1}=\left[\begin{array}{cc}{A}_{11}& A_{12}\\ A_{21}& A_{22}\end{array}\right]$

$A_{11}={\mathrm{<figure-callout\; id="1"\; label="P"\; filenames="HSA00000031561100022.png,HSA00000031561100023.png"\; state="\{\{state\}\}">P</figure-callout>}}_1^{\&prime;}+{\mathrm{<figure-callout\; id="1"\; label="P"\; filenames="HSA00000031561100022.png,HSA00000031561100023.png"\; state="\{\{state\}\}">P</figure-callout>}}_1^{\&prime;}\left(H_1^T{H}_{11}\right)R^{-1}\left(H_{11}^T{H}_1\right){\mathrm{<figure-callout\; id="1"\; label="P"\; filenames="HSA00000031561100022.png,HSA00000031561100023.png"\; state="\{\{state\}\}">P</figure-callout>}}_1^{\&prime;},$ $A_{12}=-{\mathrm{<figure-callout\; id="1"\; label="P"\; filenames="HSA00000031561100022.png,HSA00000031561100023.png"\; state="\{\{state\}\}">P</figure-callout>}}_1^{\&prime;}\left(H_1^T{H}_{11}\right)R^{-1},$ $A_{21}=A_{12}^T,$ A ₂₂＝R ^-1

$R=H_{11}^T{H}_{11}-\left(H_{11}^T{H}_1\right){\mathrm{<figure-callout\; id="1"\; label="P"\; filenames="HSA00000031561100022.png,HSA00000031561100023.png"\; state="\{\{state\}\}">P</figure-callout>}}_1^{\&prime;}\left(H_1^T{H}_{11}\right)$

${\mathrm{<figure-callout\; id="1"\; label="P"\; filenames="HSA00000031561100022.png,HSA00000031561100023.png"\; state="\{\{state\}\}">P</figure-callout>}}_1^{\&prime;}={({\mathrm{<figure-callout\; id="0"\; label="K"\; filenames="HSA00000031561100022.png"\; state="\{\{state\}\}">K</figure-callout>}}_0+H_1^T{H}_1)}^{-1}=P_0-P_0{H}_1^T{(I+H_1{P}_0{H}_1^T)}^{-1}{H}_1{\mathrm{<figure-callout\; id="0"\; label="P"\; filenames="HSA00000031561100022.png"\; state="\{\{state\}\}">P</figure-callout>}}_0,$ ${\mathrm{<figure-callout\; id="0"\; label="P"\; filenames="HSA00000031561100022.png"\; state="\{\{state\}\}">P</figure-callout>}}_0=K_0^{-1}$

In the 7th step, upgrade classification ball S ₀Be S ₁, make S ₁Comprised S ₀In data X ₀With newly-increased data X ₁

As can be seen from Figure 2, S ₁The centre of sphere should be at the mid point of A and B, radius should be half of line segment AB.Wherein, the A point is the new data position, and the B point is ball S ₀Go up apart from A point solstics.So new classification ball center and radius is respectively

$R_{\mathrm{new}}=\frac{\left|\right|x_a-x_b\left|\right|}{2}$

$x_{o\_\mathrm{new}}=\frac{x_a<figure-callout\; id="2"\; label="+\; x\; b"\; filenames="HSA00000031561100022.png"\; state="\{\{state\}\}">+</figure-callout>x_b{}_{}}{2}$

x _bBe the coordinate that B is ordered, it can be determined by following two formulas

x _b＝x _o+λ ₂(x _o-x _a)

${\mathrm{\&lambda;}}_2=\frac{\left|\right|x_b-x_o\left|\right|}{\left|\right|x_o-x_a\left|\right|}=\frac{R}{\left|\right|x_o-x_a\left|\right|}$

The method flow diagram that the present invention proposes as shown in Figure 1.

According to above-mentioned line modeling method based on extreme learning machine with adjustable structure, the present invention has done a large amount of emulation experiments, as space is limited, only provides the effect of said method in actual steel-making continuous casting quality forecast data here.This data set derives from industry spot, and the input dimension is 84, and the output dimension is 1; Number of training is 1056, and the test specimens given figure is 508.

The present invention has compared above-mentioned SAO-ELM method and batch processing formula algorithm---the effect of BP neural network algorithm and OS-ELM method.The BP neural network algorithm is classical neural metwork training method, but it is not an on-line learning algorithm, the OS-ELM method is a kind of on-line learning algorithm, and the difference of the algorithm that itself and the present invention propose is that it does not have adjustability of structure, promptly lacks the step of the 5th among the present invention, the 6th step and the 7th step.Comparative result sees Table 1:

The performance of table 1SAO-ELM and other algorithms relatively

As can be seen from Table 1, it all is best that two kinds of modeling methods of the training precision of SAO-ELM and measuring accuracy and other are compared, and its training time has then been lacked order of magnitude nearly than the BP algorithm.Fig. 5 has provided the line modeling process, from its variation tendency more as can be known, along with the propelling of modeling process, the number of hidden nodes is in continuous increase, and learning accuracy is also in continuous improvement, and both variation tendencies are consistent, and this also shows the validity of the SAO-ELM that the present invention proposes.

Claims (1)

1. the line modeling method based on extreme learning machine with adjustable structure (ELM) is characterized in that, described method is to realize as follows successively on computers:

Step (1): Model Selection and parameter initialization

The number of hidden nodes M of single hidden layer extreme learning machine is set, and the input layer number is identical with the training sample dimension n, and the output node number is identical with the dimension m of training objective;

Excitation function G (a of hidden node _i, b _i, x) adopt Gaussian function, determine the center a of each hidden node at random _iWith width b _i, i=1,2 ... M;

According to an initial N sample The training extreme learning machine obtains initial hidden layer output matrix H ₀With output layer connection matrix β ₀, wherein,

β ₀＝(H ₀ ^TH ₀) ^-1H ₀T ₀

$T_0=\left[\begin{array}{c}{t}_{11}...t_{1m}\\ ...\\ t_{N1}...t_{\mathrm{Nm}}\end{array}\right]={\left[\begin{array}{c}{t}_1^T\\ ...\\ t_N^T\end{array}\right]}_{N\&times;m}$

The initialization matrix K makes Calculate Preserve β ₀, K ₀And P ₀

Surround initial training sample set X with a classification ball O ₀, make this ball just with X ₀In all sample points be enclosed in its inside, determine the centre of sphere C of this ball ₀And radius R ₀

Step (2): on-line study process

Newly-increased training data x ₁=(x _N+1, t _N+1) when arriving, according to following steps training ELM, make it can preserve X ₀Knowledge, can hold x again ₁The new knowledge that is comprised

Step (2.1): it is constant to keep network structure, only according to x ₁Adjust output layer connection matrix β ₀, the output layer weights connection matrix after the renewal is β ₁, upgrade matrix K simultaneously ₀And P ₀Be K ₁And P ₁,

${\mathrm{\&beta;}}_1={\mathrm{\&beta;}}_0+P_1{H}_1^T(T_1-H_1{\mathrm{\&beta;}}_0)$

$P_1=K_1^{-1}=P_0-P_0{H}_1^T{(I+H_1{P}_0{H}_1^T)}^{-1}{H}_1{P}_0$

$K_1=K_0+H_1^T{H}_1$

Wherein, H ₁And T ₁Be respectively ELM to new samples x ₁Hidden layer output matrix and training objective matrix, promptly

H ₁＝[G(a ₁，b ₁，x _N+1)…G(a _M，b _M，x _N+1)] _1×M

$T_1=[t_{(N+1)1}...t_{(N+1)m}]={\left[t_{N+1}^T\right]}_{1\&times;m}$

Step (2.2): the ELM after the calculating adjustment parameter is to newly-increased sample x ₁Training error e, judge new samples x ₁Whether outside ball O, if outside ball O and e greater than setting threshold, abandon above-mentioned all adjustment, turn to step (2.3), otherwise, turn to step (3);

Step (2.3): increase a hidden node, setting its center a is x ₁, width b is definite by following formula, promptly

$b\&le;-\frac{\left|\right|x_c-a\left|\right|}{\ln \mathrm{\&epsiv;}}$

Wherein, ε is the threshold value that presets, x _cFor classification ball O goes up apart from new sample point x ₁The coordinate of closest approach can be determined by following formula

x _c＝x _o+λ ₁(x _a-x _o)

Wherein, x _oBe the sphere centre coordinate of ball O, λ ₁Can determine by following formula

${\mathrm{\&lambda;}}_1=\frac{\left|\right|x_c-x_o\left|\right|}{\left|\right|x_a-x_o\left|\right|}=\frac{R}{\left|\right|x_a-x_o\left|\right|}$

X in the following formula _aBe new sample point x ₁Coordinate; Readjust output layer connection matrix β ₀Be β ₁, and correspondingly upgrade matrix K ₀And P ₀Be K ₁And P ₁, make

${\mathrm{\&beta;}}_1=P_1\left[\begin{array}{c}{K}_0{\mathrm{\&beta;}}_0+H_1^T{T}_1\\ H_{11}^T{T}_1\end{array}\right],$ $K_1=\left[\begin{array}{cc}{K}_0+H_1^T{H}_1& H_1^T{H}_{11}\\ H_{11}^T{H}_1& H_{11}^T{H}_{11}\end{array}\right],$ $P_1=K_1^{-1}=\left[\begin{array}{cc}{A}_{11}& A_{12}\\ A_{21}& A_{22}\end{array}\right]$

$A_{11}=P_1^{\&prime;}+P_1^{\&prime;}\left(H_1^T{H}_{11}\right)R^{-1}\left(H_{11}^T{H}_1\right)P_1^{\&prime;},$ $A_{12}=-P_1^{\&prime;}\left(H_1^T{H}_{11}\right)R^{-1},$ $A_{21}=A_{12}^T,$ A ₂₂＝R ^-1

$R=H_{11}^T{H}_{11}-\left(H_{11}^T{H}_1\right)P_1^{\&prime;}\left(H_1^T{H}_{11}\right)$

$P_1^{\&prime;}={(K_0+H_1^T{H}_1)}^{-1}=P_0-P_0{H}_1^T{(I+H_1{P}_0{H}_1^T)}^{-1}{H}_1{P}_0,$ $P_0=K_0^{-1}$

Wherein, H ₀₁And H ₁₁Be respectively newly-increased hidden node to former sample set X ₀With new sample point x ₁The hidden layer output matrix, promptly

N ₁Be respectively x with L ₁In sample point number and newly-increased hidden node number, this method is only considered the situation that new sample point arrives one by one, so N ₁=L=1;

Step (3): the parameter of upgrading classification ball O

Upgrade the parameter of classification ball O, promptly upgrade its sphere centre coordinate and radius, make new ball O ₁Just with X ₀And x ₁In all sample points all be enclosed in wherein, its more new formula be:

$R_{\mathrm{new}}=\frac{\left|\right|x_a-x_b\left|\right|}{2}$

$x_{o\_\mathrm{new}}=\frac{x_a+x_b}{2}$

Wherein, x _aAnd x _bBe respectively new sample point x ₁Coordinate and ball O go up apart from x ₁The coordinate in solstics, x _bCan calculate by following formula,

x _b＝x _o+λ ₂(x _o-x _a)

Wherein, x _oBe the coordinate of centre of sphere O, λ ₂Can calculate by following formula,

${\mathrm{\&lambda;}}_2=\frac{\left|\right|x_b-x_o\left|\right|}{\left|\right|x_o-x_a\left|\right|}=\frac{R}{\left|\right|x_o-x_a\left|\right|}.$

Images