Research Papers

Nonlinear Model Identification From Multiple Data Sets Using an Orthogonal Forward Search Algorithm

[+] Author and Article Information
Ping Li

e-mail: p.li@sheffield.ac.uk

Hua-Liang Wei

e-mail: w.hualiang@sheffield.ac.uk

Stephen A. Billings

e-mail: s.billings@sheffield.ac.uk

Michael A. Balikhin

e-mail: m.balikhin@sheffield.ac.uk

Richard Boynton

e-mail: r.boynton@sheffield.ac.uk
Department of Automatic Control and Systems Engineering,
The University of Sheffield Sheffield S10 2TN,

Contributed by the Design Engineering Division of ASME for publication in the Journal of Computational and Nonlinear Dynamics. Manuscript received October 4, 2011; final manuscript received February 22, 2013; published online March 26, 2013. Assoc. Editor: Jozsef Kovecses.

J. Comput. Nonlinear Dynam 8(4), 041001 (Mar 26, 2013) (10 pages) Paper No: CND-11-1168; doi: 10.1115/1.4023864 History: Received October 04, 2011; Revised January 21, 2013

A basic assumption on the data used for nonlinear dynamic model identification is that the data points are continuously collected in chronological order. However, there are situations in practice where this assumption does not hold and we end up with an identification problem from multiple data sets. The problem is addressed in this paper and a new cross-validation-based orthogonal search algorithm for NARMAX model identification from multiple data sets is proposed. The algorithm aims at identifying a single model from multiple data sets so as to extend the applicability of the standard method in the cases, such as the data sets for identification are obtained from multiple tests or a series of experiments, or the data set is discontinuous because of missing data points. The proposed method can also be viewed as a way to improve the performance of the standard orthogonal search method for model identification by making full use of all the available data segments in hand. Simulated and real data are used in this paper to illustrate the operation and to demonstrate the effectiveness of the proposed method.

Copyright © 2013 by ASME
Your Session has timed out. Please sign back in to continue.


Leontaritis, I. J., and Billings, S. A., 1985, “Input-Output Parametric Models for Nonlinear Systems, Part I: Deterministic Nonlinear Systems; Part II: Stochastic Nonlinear Systems,” Int. J. Control, 41(1), pp. 303–344. [CrossRef]
Sjöberg, J., Zhang, Q., Ljung, L., Benveniste, A., Delyon, B., Glorennec, P., Hjalmarsson, H., and Juditsky, A., 1995, “Nonlinear Black-Box Modeling in System Identification: A Unified Overview,” Automatica, 31(12), pp. 1691–1724. [CrossRef]
Billings, S. A., and Chen, S., 1998, “The Determination of Multivariable Nonlinear Models for Dynamic Systems,” Control Dynamic Systems, Neural Network Systems Techniques and Applications, Vol. 7, C. T.Leondes, ed., Academic, San Diego, pp. 231–278.
Billings, S. A., Korenberg, M. J., and Chen, S., 1988, “Identification of Non-Linear Output Affine Systems Using an Orthogonal Least-Squares Algorithm,” Int. J. Syst. Sci., 19, pp. 1559–1568. [CrossRef]
Billings, S. A., Chen, S., and Korenberg, M. J., 1989, “Identification of MIMO Non-Linear Systems Using a Forward Regression Orthogonal Estimator,” Int. J. Control, 49, pp. 2157–2189.
Chen, S., Cowan, C. F. N., and Grant, P. M., 1991, “Orthogonal Least Squares Learning Algorithm for Radial Basis Function Networks,” IEEE Trans. Neural Netw., 2(2), pp. 302–309. [CrossRef] [PubMed]
Billings, S. A., and Wei, H. L., 2005, “A New Class of Wavelet Networks for Nonlinear System Identification,” IEEE Trans. Neural Netw., 16(4), pp. 862–874. [CrossRef] [PubMed]
Hong, X., Chen, S., and Harris, C. J., 2008, “A Fast Linear-in-the-Parameters Classifier Construction Algorithm Using Orthogonal Forward Selection to Minimize Leave-One-Out Misclassification Rate,” Int. J. Syst. Sci., 39(2), pp. 119–125. [CrossRef]
Chen, S., Hong, X., Harris, C. J., and Sharkey, P. M., 2004, “Sparse Kernel Density Construction Using Orthogonal Forward Regression With Leave-One-Out Test Score and Local Regularization,” IEEE Trans. Syst. Man Cyber. Part B, 34(4), pp. 1708–1717. [CrossRef]
Chen, S., Hong, X., Luk, B. L., and Harris, C. J., 2009, “Orthogonal-Least-Squares Regression: A Unified Approach for Data Modeling,” Neurocomputing, 72(10–12), pp. 2670–2681. [CrossRef]
Piroddi, L., and Spinelli, W., 2003, “An Identification Algorithm for Polynomial NARX Models Based on Simulation Error Minimization,” Int. J. Control, 76(17), pp. 1767–1781. [CrossRef]
Tibshirani, R. J., 1996, “Regression Shrinkage and Selection via the LASSO,” J. R. Stat. Soc. B, 58(1), pp. 267–288.
Kukreja, S. L., 2009, “Application of a Least Absolute Shrinkage and Selection Operator to Aeroelastic Flight Test Data,” Int. J. Control, 82(12), pp. 2284–2292. [CrossRef]
Kukreja, S. L., Galiana, H. L., and Kearney, R. E., 2004, “A Bootstrap Method For Structure Detection of NARMAX Models,” Int. J. Control, 77, pp. 132–143. [CrossRef]
Lind, I., and Ljung, L., 2005, “Regressor Selection With the Analysis of Variance Method,” Automatica, 41(4), pp. 693–700. [CrossRef]
Lind, I., and Ljung, L., 2008, “Regressor and Structure Selection in NARX Models Using a Structured ANOVA Approach,” Automatica, 44(2), pp. 383–395. [CrossRef]
Chen, S., Billings, S. A., and Luo, W., 1989, “Orthogonal Least Squares Methods and Their Application to Non-Linear System Identification,” Int. J. Control, 50(5), pp. 1873–1896. [CrossRef]
Guo, L., Billings, S. A., and Zhu, D. Q., 2008, “An Extended Orthogonal Forward Regression Algorithm for System Identification Using Entropy,” Int. J. Control, 81(4), pp. 690–699. [CrossRef]
Billings, S. A., and Wei, H. L., 2007, “Sparse Model Identification Using a Forward Orthogonal Regression Algorithm Aided by Mutual Information,” IEEE Trans. Neural Netw., 18(1), pp. 306–310. [CrossRef] [PubMed]
Zhu, Q. M., and Billings, S. A., 1996, “Fast Orthogonal Identification of Nonlinear Stochastic Models and Radial Basis Function Neural Networks,” Int. J. Control, 64, pp. 871–886. [CrossRef]
Chen, S., Chng, E. S., and Alkadhimi, W., 1996, “Regularised Orthogonal Least Squares Algorithm for Constructing Radial Basis Function Networks,” Int. J. Control, 64(5), pp. 829–837. [CrossRef]
Hong, X., and Harris, C. J., 2002, “Nonlinear Model Structure Design and Construction Using Orthogonal Least Squares and D-Optimality Design,” IEEE Trans. Neural Netw., 13(9), pp. 1245–1250. [CrossRef] [PubMed]
Hong, X., and Harris, C. J., 2003, “Experimental Design and Model Construction Algorithms for Radial Basis Function Networks,” Int. J. Syst. Sci., 34(14–15), pp. 733–745. [CrossRef]
Chen, S., Hong, X., and Harris, C. J., 2003, “Sparse Kernel Regression Modeling Using Combined Locally Regularized Orthogonal Least Squares and D-Optimality Experimental Design,” IEEE Trans. Auto. Control, 48(6), pp. 1029–1036. [CrossRef]
Guo, L., and Billings, S. A., 2007, “A Modified Orthogonal Forward Regression Least-Squares Algorithm for System Modelling From Noisy Regressors,” Int. J. Control, 80(3), pp. 340–348. [CrossRef]
Efron, B., and Tibshirani, R. J., 1993, An Introduction to the Bootstrap, Chapman and Hall, London.
Myers, R. H., 1990, Classical and Modern Regression With Applications, 2nd ed., PWS-KENT, Boston.
Wang, L., and Cluett, W. R., 1996, “Use of PRESS Residuals in Dynamic System Identification,” Automatica, 32(5), pp. 781–784. [CrossRef]
Hong, X., Sharkey, P. M., and Warwick, K., 2003, “Automatic Nonlinear Predictive Model-Construction Algorithm Using Forward Regression and the PRESS Statistic,” IEEE Proc. Control Theory Appl., 150(3), pp. 245–254. [CrossRef]
Chen, S., Hong, X., Harris, C. J., and Sharkey, P. M., 2004, “Sparse Modeling Using Orthogonal Forward Regression With PRESS Statistic and Regularization,” IEEE Trans. Systems Man Cyber. Part B, 34(2), pp. 898–911. [CrossRef]
Wei, H. L., and Billings, S. A., 2009, “Improved Model Identification for Non-Linear System Using a Random Subsampling and Multifold Modelling (RSMM) Approach,” Int. J. Control, 82(1), pp. 27–42. [CrossRef]
Shao, J., 1993, “Linear Model Selection by Cross-Validation,” J. Am. Stat. Assoc., 88(422), pp. 486–494. [CrossRef]
Klimas, A. J., Vassiliadis, D., Baker, D. N., and Roberts, D. A., 1996, “The Organized Nonliear Dynamics of the Magnetosphere,” J. Geophys. Res., 101(A6), 13089–13113. [CrossRef]
Boaghe, O. M., Balikhin, M. A., Billings, S. A., and Alleyne, H., 2001, “Identification of Nonlinear Processes in the Magnetospheric Dynamics and Forecasting of Dst Index,” J. Geophys. Res., 106(A12), 30047–30066, 10.1029/2000JA900162. [CrossRef]


Grahic Jump Location
Fig. 1

The two input-output data sets for modeling

Grahic Jump Location
Fig. 2

Comparison between the measured Dst and the model predicted D∧st obtained with D1 only for two events

Grahic Jump Location
Fig. 3

Comparison between the measured Dst and the model predicted D∧st obtained with D2 only for two events

Grahic Jump Location
Fig. 4

Comparison between the measured Dst and the model predicted D∧st obtained with D1 and D2 for two events




Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In