Abstract

Surrogate models are efficient tools which have been successfully applied in structural reliability analysis, as an attempt to keep the computational costs acceptable. Among the surrogate models available in the literature, artificial neural networks (ANNs) have been attracting research interest for many years. However, the ANNs used in structural reliability analysis are usually the shallow ones, based on an architecture consisting of neurons organized in three layers, the so-called input, hidden, and output layers. On the other hand, with the advent of deep learning, ANNs with one input, one output, and several hidden layers, known as deep neural networks, have been increasingly applied in engineering and other areas. Considering that many recent publications have shown advantages of deep over shallow ANNs, the present paper aims at comparing these types of neural networks in the context of structural reliability. By applying shallow and deep ANNs in the solution of four benchmark structural reliability problems from the literature, employing Monte Carlo simulation (MCS) and adaptive experimental designs (EDs), it is shown that, although good results are obtained for both types of ANNs, deep ANNs usually outperform the shallow ones.

Issue Section:
Research Papers
Keywords:
structural reliability, metamodels, surrogate models, artificial neural networks, deep neural networks
Topics:
Artificial neural networks, Reliability

References

1.
Soares
,
R. C.
,
Mohamed
,
A.
,
Venturini
,
W. S.
, and
Lemaire
,
M.
,
2002
, “
Reliability Analysis of Non-Linear Reinforced Concrete Frames Using the Response Surface Method
,”
Reliab. Eng. Syst. Saf.
,
75
(
1
), pp.
1
–
16
.10.1016/S0951-8320(01)00043-6
2.
Dubourg
,
V.
,
Sudret
,
B.
, and
Deheeger
,
F.
,
2013
, “
Metamodel-Based Importance Sampling for Structural Reliability Analysis
,”
Probabilist. Eng. Mech.
,
33
, pp.
47
–
57
.10.1016/j.probengmech.2013.02.002
3.
Blatman
,
G.
, and
Sudret
,
B.
,
2010
, “
An Adaptive Algorithm to Build Up Sparse Polynomial Chaos Expansions for Stochastic Finite Element Analysis
,”
Probabilist. Eng. Mech.
,
25
(
2
), pp.
183
–
197
.10.1016/j.probengmech.2009.10.003
4.
Gomes
,
W. J. S.
, and
Beck
,
A. T.
,
2013
, “
Global Structural Optimization Considering Expected Consequences of Failure and Using ANN Surrogates
,”
Comput. Struct.
,
126
, pp.
56
–
68
.10.1016/j.compstruc.2012.10.013
5.
Gomes
,
W.
,
2019
, “
Structural Reliability Analysis Using Adaptive Artificial Neural Networks
,”
ASME J. Risk Uncertainty Part B
,
5
(
4
), p.
041004
.10.1115/1.4044040
6.
Chojaczyk
,
A. A.
,
Teixeira
,
A. P.
,
Neves
,
L. C.
,
Cardoso
,
J. B.
, and
Soares
,
C. G.
,
2015
, “
Review and Application of Artificial Neural Networks Models in Reliability Analysis of Steel Structures
,”
Struct. Saf.
,
52
, pp.
78
–
89
.10.1016/j.strusafe.2014.09.002
7.
Gomes
,
H. M.
, and
Awruch
,
A. M.
,
2004
, “
Comparison of Response Surface and Neural Network With Other Methods for Structural Reliability Analysis
,”
Struct. Saf.
,
26
(
1
), pp.
49
–
67
.10.1016/S0167-4730(03)00022-5
8.
Bucher
,
C.
, and
Most
,
T.
,
2008
, “
A Comparison of Approximate Response Functions in Structural Reliability Analysis
,”
Probabilist. Eng. Mech.
,
23
(
2–3
), pp.
154
–
163
.10.1016/j.probengmech.2007.12.022
9.
Kroetz
,
H. M.
,
Tessari
,
R. K.
, and
Beck
,
A. T.
,
2017
, “
Performance of Global Metamodeling Techniques in Solution of Structural Reliability Problems
,”
Adv. Eng. Software
,
114
, pp.
394
–
404
.10.1016/j.advengsoft.2017.08.001
10.
Kim
,
T.
,
Kwon
,
O.-S.
, and
Song
,
J.
,
2019
, “
Response Prediction of Nonlinear Hysteretic Systems by Deep Neural Networks
,”
Neural Networks
,
111
, pp.
1
–
10
.10.1016/j.neunet.2018.12.005
11.
Kulkarni
,
P. A.
,
Dhoble
,
A. S.
, and
Padole
,
P. M.
,
2019
, “
Deep Neural Network-Based Wind Speed Forecasting and Fatigue Analysis of a Large Composite Wind Turbine Blade
,”
Proc. Inst. Mech. Eng. C
,
233
(
8
), pp.
2794
–
2812
.10.1177/0954406218797972
12.
Schmidhuber
,
J.
,
2015
, “
Deep Learning in Neural Networks: An Overview
,”
Neural Networks
,
61
, pp.
85
–
117
.10.1016/j.neunet.2014.09.003
13.
Bengio
,
Y.
,
2009
, “
Learning Deep Architectures for AI
,”
Foundations Trends Mach. Learn.
,
2
(
1
), pp.
1
–
127
.10.1561/2200000006
14.
Nielsen
,
M. A.
,
2015
,
Neural Networks and Deep Learning
,
Determination Press
.
15.
Mhaskar
,
H.
,
Liao
,
Q.
, and
Poggio
,
T.
,
2017
, “
When and Why Are Deep Networks Better Than Shallow Ones?
,”
Thirty-First AAAI Conference on Artificial Intelligence (AAAI-17)
, pp. 2343–2349
.
16.
Poggio
,
T.
,
Mhaskar
,
H.
,
Rosasco
,
L.
,
Miranda
,
B.
, and
Liao
,
Q.
,
2017
, “
Why and When Can Deep - but Not Shallow - Networks Avoid the Curse of Dimensionality: A Review
,”
Int. J. Autom. Comput.
,
14
(
5
), pp.
503
–
519
.10.1007/s11633-017-1054-2
17.
Friedman
,
J. H.
,
1994
, “
An Overview of Prediction Learning and Function Approximation
,”
From Statistics to Neural Networks: Theory and Pattern Recognition Applications
,
V.
Cherkassky
,
J. H.
Friedman
, and
H.
Wechsler
, eds.,
Springer-Verlag
,
New York
.
18.
Haykin
,
S.
,
2009
,
Neural Networks and Learning Machines
, 3rd ed.,
Prentice Hall
,
Upper Saddle River, NJ
.
19.
Lataniotis
,
C.
,
Marelli
,
S.
, and
Sudret
,
B.
,
2018
,
Extending Classical Surrogate Modelling to Ultrahigh Dimensional Problems Through Supervised Dimensionality Reduction: A Data-Driven Approach
, Research Report,
ETH Zurich
,
Zurich, Switzerland
.
20.
Poggio
,
T.
, and
Liao
,
Q.
,
2018
, “
Theory I: Deep Networks and the Curse of Dimensionality
,”
Bull. Pol. Acad. Tech. Sci.
,
66
(
6
), pp.
761
–
773
.10.24425/bpas.2018.125924
21.
Hutzenthaler
,
M.
,
Jentzen
,
A.
,
Kruse
,
T.
, and
Nguyen
,
T. A.
,
2020
, “
A Proof That Rectified Deep Neural Networks Overcome the Curse of Dimensionality in the Numerical Approximation of Semilinear Heat Equations
,” SN Partial Differ. Equ. Appl., 1, Paper No. 10. 10.1007/s42985-019-0006-9
22.
Montavon
,
G.
,
Samek
,
W.
, and
Müller
,
K.-R.
,
2018
, “
Methods for Interpreting and Understanding Deep Neural Networks
,”
Digital Signal Process.
,
73
, pp.
1
–
15
.10.1016/j.dsp.2017.10.011
23.
Gopalakrishnan
,
K.
,
Gholami
,
H.
,
Vidyadharan
,
A.
,
Choudhary
,
A.
, and
Agrawal
,
A.
,
2018
, “
Crack Damage Detection in Unmanned Aerial Vehicle Images of Civil Infrastructure Using Pre-Trained Deep Learning Model
,”
Int. J. Traffic Transp. Eng.
,
8
(
1
), pp.
1
–
14
.10.7708/ijtte.2018.8(1).01
24.
Ditlevsen
,
O.
, and
Madsen
,
H. O.
,
2007
,
Structural Reliability Methods
,
Technical University of Denmark
,
Kongens Lyngby, Copenhagen, Denmark
.
25.
Melchers
,
R. E.
, and
Beck
,
A. T.
,
2018
,
Structural Reliability Analysis and Prediction
, 3rd ed.,
Wiley
,
New York
.
26.
Engelund
,
S.
, and
Rackwitz
,
R.
,
1993
, “
A Benchmark Study on Importance Sampling Techniques in Structural Reliability
,”
Struct. Saf.
,
12
(
4
), pp.
255
–
276
.10.1016/0167-4730(93)90056-7
27.
Maes
,
M. A.
,
Breitung
,
K.
, and
Dupuis
,
D. J.
,
1993
, “
Asymptotic Importance Sampling
,”
Struct. Saf.
,
12
(
3
), pp.
167
–
186
.10.1016/0167-4730(93)90001-H
28.
Au
,
S.-K.
, and
Beck
,
J. L.
,
2001
, “
Estimation of Small Failure Probabilities in High Dimensions by Subset Simulation
,”
Probabilist. Eng. Mech.
,
16
(
4
), pp.
263
–
277
.10.1016/S0266-8920(01)00019-4
29.
Echard
,
B.
,
Gayton
,
N.
, and
Lemaire
,
M.
,
2011
, “
AK-MCS: An Active Learning Reliability Method Combining Kriging and Monte Carlo Simulation
,”
Struct. Saf.
,
33
(
2
), pp.
145
–
154
.10.1016/j.strusafe.2011.01.002
30.
Schöbi
,
R.
,
Sudret
,
B.
, and
Marelli
,
S.
,
2017
, “
Rare Event Estimation Using Polynomial-Chaos Kriging
,”
ASME J. Risk Uncertain. Eng. Syst. A Civ. Eng.
,
3
(
2
), p.
D4016002
.10.1061/AJRUA6.0000870
31.
Marelli
,
S.
, and
Sudret
,
B.
,
2018
, “
An Active-Learning Algorithm That Combines Sparse Polynomial Chaos Expansions and Bootstrap for Structural Reliability Analysis
,”
Struct. Saf.
,
75
, pp.
67
–
74
.10.1016/j.strusafe.2018.06.003
32.
Marelli
,
S.
, and
Sudret
,
B.
,
2016
, “
Bootstrap-Polynomial Chaos Expansions and Adaptive Designs for Reliability Analysis
,”
Proceedings of the Sixth Asian-Pacific Symposium on Structural Reliability and Its Applications (APSSRA6)
,
Shangai, China
, May 28–30, pp.
217
–
224
.10.3929/ethz-a-010671460
33.
McCulloch
,
W.
, and
Pitts
,
W.
,
1943
, “
A Logical Calculus of the Ideas Immanent in Nervous Activity
,”
Bull. Math. Biophys.
,
5
(
4
), pp.
115
–1
33
.10.1007/BF02478259
34.
Hagan
,
M. T.
, and
Menhaj
,
M. B.
,
1994
, “
Training Feedforward Networks With the Marquardt Algorithm
,”
IEEE Trans. Neural Networks
,
5
(
6
), pp.
989
–
993
.10.1109/72.329697
35.
Kingma
,
D. P.
, and
Ba
,
J. L.
,
2015
, “
ADAM: A Method for Stochastic Optimization
,”
Proceedings of the Third International Conference on Learning Representations
,
San Diego, CA
, May 7–9.
36.
Beale
,
M. H.
,
Hagan
,
M. T.
, and
Demuth
,
H. B.
,
2011
,
Neural Network Toolbox: User's Guide
,
Mathworks
,
Natick, MA
, p. 404.
37.
Nguyen
,
D.
, and
Widrow
,
B.
,
1990
, “
Improving the Learning Speed of 2-Layer Neural Networks by Choosing Initial Values of the Adaptive Weights
,”
Proceedings of the International Joint Conference on Neural Networks
, San Diego, CA, June 17–21.10.1109/IJCNN.1990.137819
38.
Haykin
,
S.
,
1996
,
Adaptive Filter Theory
, 3rd ed.,
Prentice Hall
,
Upper Saddle River, NJ
.
39.
Waarts
,
P.-H.
,
2000
, “
Structural Reliability Using Finite Element Methods: An Appraisal of DARS—Directional Adaptive Response Surface Sampling
,” Ph.D. thesis,
Technical University of Delft
, Delft, The Netherlands.
40.
Rackwitz
,
R.
,
2001
, “
Reliability Analysis — A Review and Some Perspective
,”
Struct. Saf.
,
23
(
4
), pp.
365
–
395
.10.1016/S0167-4730(02)00009-7
41.
Lee
,
S. H.
, and
Kwak
,
B. M.
,
2006
, “
Response Surface Augmented Moment Method for Efficient Reliability Analysis
,”
Struct. Saf.
,
28
(
3
), pp.
261
–
272
.10.1016/j.strusafe.2005.08.003
Copyright © 2020 by ASME
You do not currently have access to this content.