Theory Manual Open Source MAPOD Framework

User Manual:

Open the PDF directly: View PDF .
Page Count: 15

Download
Open PDF In Browser	View PDF

Theory Manual
Open-Source pyMAPOD Framework – v.beta
Developed by Computational Design Laboratory (CODE Lab)
Department of Aerospace Engineering, Iowa State University
Leading Developers:

Leifur Thor Leifsson
Xiaosong Du

We gratefully acknowledge the support of the Center for Nondestructive Evaluation
Industry/University Cooperative Research Program at Iowa State University

Code and updates are available at the link: https://github.com/CODE-Lab-IASTATE/MAPOD

Content
1. Introduction
2. Operation system
2.1 Python on Linux / Windows systems
2.2 Suggested compiler and necessary modules
3. Mathematical theories
3.1 Linear regression
3.1.1 Introduction
3.1.2 Least squares method
3.1.3 Maximum likelihood method
3.2 POD calculation
3.2.1 Background
3.2.2 “ahat vs. a” regression based POD
3.3 Confidence interval
3.3.1 Bootstrap
3.3.2 Wald method
A. Fisher information
B. Wald method
C. Application on “ahat vs. a” regression and POD curves
4. Polynomial Chaos Expansions
4.1 Generalized format
4.2 Solving for coefficients
References

i

1. Introduction
Model-assisted probability of detection (MAPOD) [1, 2] is an important terminology in nondestructive
testing (NDT) area [3], not only because it describes the reliability of detecting system, but also it can
greatly reduce the required experimental information.
MAPOD calculation has been widely applied to varieties of NDT areas, such as ultrasonic testing, eddy
current testing, and x-ray based testing. Researchers, such as J. Aldrin [4, 5], J. Knopp [6], and R. Miorreli
[7], have made great progress and improvements. However, as far as the authors know, most of the MAPOD
calculations have to rely on commercial software, such as CIVA. Due to these reasons, the authors decide
to develop this open-source MAPOD framework, aiming at providing convenient tools to the NDT
researchers.
This open-source MAPOD framework is developed using python, which makes it cross-platform,
although some necessary python modules are still needed. More details on prerequisites are shown in
Chapter 1. Basic mathematical theories, such as linear regression, “ahat vs. a” plots, and probability of
detection (POD) calculation, are given in Chapter 2. Chapter 3 has some test cases to demonstrate the
process, and validate the results with MIL-HDBK-1823 [8, 9], officially verified software by Department
of Defense, United States. Researchers, who have sufficient background in POD calculation or only want
to make practical application to their research, can feel free to go through the associated document, User
Guide, directly.
The Computational Design (CODE) lab would like to thank the center of nondestructive evaluation
(CNDE) for funding this program. In addition, we also want to say thanks to all the colleagues, Dr. Jiming
Song, Dr. William Meeker, Dr. Ronald Roberts, Dr. Leonard Bond, etc. for providing physics-based NDT
simulation models, valuable ideas and suggestions.
2. Operation environment
2.1 Python on Linux / Windows systems
This open-source framework is constructed using python, which is well known for its cross-platform
capability. Therefore, the users can run it on any systems. The authors wrote and tested the code on python
v2.7, Window x64 local desktop and laptop machine.
2.2 Suggested compiler and necessary modules
The code is written within Enthought Canopy compiler, which is very powerful and convenient for
integrating python toolboxes. Users simply select Package Manager under the toolbar Tools, to add any
necessary modules. In this work, authors are trying to avoid using additional modules as much as possible.
The necessary modules and corresponding utilizations are:
collections: the module ‘OrderedDict’ is used to specify random inputs with statistical distributions
pyDOE: randomly generate sample points, using latin hypercube sampling (LHS) sheme
numpy: various numerical operations, such as numpy.array
pandas: read data frame from excel file
matplotlib.pyplot: view imported data, generate “ahat vs. a” plots, and POD curves
sys: sys.exit() used when incorrect data format is provided
mlab: link python with Matlab, making Matlab a callable module for python
sklearn: linear_model of sklearn is implemented, in particular, the linear_model.LinearRegression and
linear_model.LassoLarsCV are utilized for linear regression
math: used for calculating factorial of integral values
scipy.stats: used for generating normally distributed sample points
1

3. Mathematical theories
This chapter will talk about least squares method and maximum likelihood method for linear regression
in Section 3.1. In Section 3.2 linear regression will be applied to “ahat vs. a” regression, followed by POD
calculation. Section 3.3 has the details on the calculation of confidence interval.
3.1 Linear regression
3.1.1 Introduction
Linear regression [10] is a statistical method that allows us to summarize and study relationships
between two continuous or quantitative variables: one variable, denoted x, is named as independent variable
or predictor, while the other one, denoted y, is named as dependent variable or response. The linear
relationship between x and y is usually given as:
y =  0 + 1  x +  ,

(1)

where ꞵ0 is the intercept of the regression line, ꞵ1 is the slope of the regression line, ɛ is random error
following a zero-mean and constant-standard deviation Normal distribution:

 ~ Normal (0,  2 ).

(2)

Due to this random error, which may come from operation system or other noises, the observed value
varies slightly each time even at the same x value. And the prediction using estimated parameters usually
represents the expected value of y at that x location (Fig. 1).

Figure 1. Linear regression line on random observations.

As shown above, the only unknowns in linear regression problems are ꞵ0, ꞵ1, and σɛ, which can be solved
by various methods. Among all these methods, least squares (LS) method and maximum likelihood (ML)
method are the most commonly used. We will discuss both of LS and ML methods, and also their
relationship, in the following sections.
2

3.1.2 Least squares method
As shown in Eqn. 1, the linear relation is constructed using β0, β1, and σε. It can be converted as:
 = y − (  0 + 1  x),

(3)

which is the residual between observed values and estimation from linear regression line. A widely used
method for estimating the coefficients is least squares method [11], which minimizes the sum of the squared
of residual at each observed value:
min( S ) = min  i =1 (  i ) = min  i =1 ( yi − (  0 + 1  xi ) ) .
n

n

2

2

(4)

Taking the first-order derivative of Eqn. 4 about the coefficients β0 and β1 as 0, it is straightforward to
obtain

 ( x − x )( y − y ) ,
 (x − x)
n

ˆ1 =

i

i =1

i

n

(5)

2

i =1

i

and

垐
0 = y − 1 x ,

(6)

where x and y are mean values of x and y, respectively, and ̂ 0 and ˆ1 are the estimates of β0 and β1,
respectively.
Since the expected value of ε

E   = E Y − (  0 + 1 X )  = 0.

(7)

From Eqn. 4, it is straightforward to obtain

E  2  =  i =1 ( yi − (  0 + 1  xi ) ) / n.
n

2

(8)

Therefore, the standard deviation of ε is

  = E  2  − ( E  ) =
2

 ( y − (
n

i =1

i

+ 1  xi ) ) / n ,
2

0

(9)

which is actually of the format of the root mean squared error (RMSE).
The Eqn. 4 can be solved by many optimization method, such as Newton method, pattern search or OLS
method within python. However, in this code we aim at avoiding prerequisite modules as much as possible,
so Eqn. 5, 6, and 9 are packed. And please remember the expression in these three equations, because we
will compare them with those estimated from maximum likelihood method, to prove that they are the same
under this situation.
3

3.1.3 Maximum likelihood method
Now let’s review the assumption made in linear regression:
1. The distribution of X is arbitrary.
2. If X = x, the Y = β0 + β1x + ε, for some coefficients, β0 and β1 and some random noise variable ε.
3. ε ~ N(0, σε2), and is independent of X.
4. ε is independent across observations.
Based on these assumptions, the response Y is independent across observations, conditional on the
predictor X. Besides, the noise variable ε has zero mean and constant variance, and follows the Normal
distribution. Therefore, the conditional probability density function of Y for each x, given arbitrary number
of data sets, (x1, y1), (x2, y2), …, (xn, yn), can be written as
n

n

i

i

0

1

i =1

( yi −( 0 + 1xi ))2

1

 p ( y | x ;  ,  ,   ) =
2

2 2

i =1

2  2

e

.

(10)

,

(11)

For any estimates on unknown parameters, β0, β1, and σε, the pdf becomes

 p ( y | x ; 垐,  ,   ) =
n

n

2

i

i =1

i

0

1

( yi −( 垐0 +1xi ))

1
2ˆ 2

i =1

2

2 ˆ2

e

which is called likelihood, a function of the parameter values. For the convenience of calculation, usually
it is taken as log-likelihood,

(

)

(

n

)

2
2
垐
垐
L 垐
0 , 1 ,   = log  p yi | xi ;  0 , 1 ,   ,
i =1

(12)

and so,

(

)

n
1
2
垐
L 垐
log 2 − n log   − 2
0 , 1 ,   = −
2
2ˆ 

 ( y − (垐+  x ) ) .
n

i =1

2

i

0

1 i

(13)

We can maximize Eqn. 13 to get the best estimates on unknowns. This method is called maximum
likelihood [12]. Any optimization methods can be used to maximize Eqn. 13, like what was mentioned
above for least squares method. We can still use the same method, taking first-order derivative of Eqn. 13
and setting it as 0, to obtain

 ( x − x )( y − y ) ,
 (x − x)
n

ˆ1 =

i

i =1

i

n

i =1

2

垐
0 = y − 1 x ,

4

(14)

i

(15)

 =

 ( y − (
n

i =1

i

+ 1  xi ) ) / n .
2

0

(16)

It is obvious that the estimation equations above are the same as Eqn. 5, 6, and 9. Therefore, in the
framework, we code up the numerical expression of unknown parameters, directly, without having to claim
which method it comes from. The authors prefer to maximum likelihood method, because it is more
convenient and straightforward to compute Fisher information matrix and apply Wald method for
confidence intervals on “ahat vs. a” plots and POD curves. More details on this type of confidence interval
calculation are given in Section 2.3.2.
3.2 POD calculation
3.2.1 Background
The concept of POD was initially developed to quantitatively describe the detection capabilities of NDT
systems, starting from pioneering work since the late 1960’s for the aerospace industry. POD curves have
been widely generated through various NDT equipment, such as ultrasound, eddy currents, magnetic
particle inspection and radiography, focusing on different quantities of interests. A commonly used term is
“90% POD” and “90% POD with 95% confidence interval”, which are written as a90 and a90/95, respectively.
POD curves were initially only based on experiments, however, to save computational budgets it can be
enhanced by utilizing physics-based computational models, which is known as the MAPOD methodology.
Here in current work, POD is performed using linear regression method.
3.2.2 “ahat vs. a” regression based POD
For signal response data, much more information is supplied in the signal for analysis than is in hit/miss
data. Here, POD function is generated from the correlation of “ahat vs. a” data [8, 9]. And through reviews
on experiments data, it shows a log-log scale between ahat and a:
ln aˆ =  0 + 1 ln a +  ,

(17)

where the coefficients ꞵ0 and ꞵ1 can be determined by the maximum likelihood method, and the ε has a
Normal distribution with zero mean and standard deviation σε, N(0, σε). This standard deviation can be
determined by the residuals of the observed data, as shown in Section 2.1.
The POD can be obtained as the probability that the obtained signal lies above arbitrary user-defined
threshold:

 ln aˆthreshold − ( 0 + 1 ln a) 
POD(a) = 1 −  
,




(18)

where Φ is the standard normal distribution function.
From Eqn. 18, it is straightforward to obtain:

ln aˆthreshold −  0

 ln a −
1
POD(a) =  



1
5



,



(19)

which is a cumulative log-normal distribution function with mean μ and standard deviation σ given by:

=

ln aˆthreshold −  0

1

=

,


,
1

(20)

(21)

where the parameters ꞵ0, ꞵ1, and σε can be obtained by least squares method, maximum likelihood method
or the numerical expression discussed in Section 2.1.
3.3 Confidence interval
In statistics, especially when uncertainty exists in NDT system, it is impossible to specify a value to a
variable with 100% certainty. POD requires two numerical values, alower and aupper, depending on the sample
set, and varying for each random set. The interval within alower and aupper is called a “confidence interval”
[8, 9], which is usually expressed as:
P (alower  a  aupper ) = const ,

(22)

where the const is the “confidence level”, alower is called the “lower confidence limit” and aupper is called the
“upper confidence limit”. In POD calculation, only lower confidence limit is used. If confidence level is set
as 95%, the interpretation is based on repeated sampling, meaning if samples of the same size are drawn
repeatedly from a population and a confidence interval is calculated from each sample, then we can expect
95% of these different intervals to contain the true value.
In POD calculation, the defect size a90 meaning the 90% probability to be detected is always considered
within application of POD or framework of design. To take the uncertainty into account, the upper bounds
of 95% confidence interval is considered, written as a90/95. Note that these two values are not characteristic
properties of an NDT system, but rather are calculated from the particular random results.
There are various methods, such as bootstrap, Wald method, and likelihood ratio method, existing in the
area of confidence interval calculation. In this work, we will talk about bootstrap due to its simplicity, and
Wald method due to efficiency. In current version of the framework, we select to pack up the Wald method.
Bootstrap method is introduced here as a comparison on the POD results in Section 3.
3.3.1 Bootstrap
Brad Efron invented a revolutionary new statistical procedure called the bootstrap [13, 14], in 1979.
This is a computer-intensive procedure that substitutes fast computation for theoretical mathematics. The
main benefit of the bootstrap is the confidence intervals on parameters without having to make unreasonable
assumptions.
The idea of bootstrap is simple:
(1) Gather the sample data set (x1, y1), (x2, y2), … (xn, yn), and use it to estimate the unknown parameters, θ.
(2) Draw a new random sample of size n, with replacement, from the sample data set, and estimate the
unknowns.
(3) Repeat step (2) as many times as necessary or user-defined arbitrary number of times, e.g. 1,000.
(4) Put the 1000 additional estimates on unknown parameters into an ascending order, separately.

6

(5) Confidence intervals can be obtained, based on the ordered sets of estimates. For example, 95% lower
*
*
confidence interval is the 975th value, ˆ975
while 95% upper confidence interval is the 25th value, ˆ25
.

(6) This step is optional. We can calculate pivot confidence interval, following the formula
*
CI lower = 2垐
− 975
,

(23)

*
CI upper = 2垐
−  25
,

(24)

where ˆ is the estimate on θ from the original set, CIlower and CIupper are lower and upper confidence
interval, respectively.
The samples obtained from step 2 and 3, are called bootstrap samples. And each of newly generated
bootstrap samples approximates the original set of data, making the new estimates on θ approximate the
results from original set. This approximating distribution is used to set confidence interval.
3.3.2 Wald method
Wald method [15, 16] is a well-known likelihood-based procedure for calculating confidence interval,
and usually performs well in large samples. For a location-scale distribution or for a distribution which can
be transformed to a location-scale distribution, the Wald confidence interval is easy to compute for
quantiles. Therefore, it is suitable for exponential, Weilbull, and lognormal distributions. The MIL-HDBK1823, the officially used POD software from the department of defense (DOD), United State, also utilizes
this method for calculation of confidence interval.
A. Fisher information
In mathematical statistics, Fisher information [17] is used for measuring the amount of information that
an observable random variable X carries about an unknown parameter θ of a distribution that models X.
Formally, it is the expected value of the observed information. Observed information is the negative of the
second derivative, the Hessian matrix, of the “log-likelihood” (the logarithm of the likelihood function).
Suppose that we observe random variables, X1, X2, …, Xn, independently and identically distributed with
density f(X; θ), where θ is assumed to be a n-dimensional vector. The log-likelihood of the parameters θ
given the data X1, X2, …, Xn is

l (θ | X 1 , X 2 ,..., X n ) =  i =1 log f ( X i | θ).
n

(25)

Then the observed information matrix can be obtained as

 2
 21
 
J (θ) =  2 1
 ...
  2
  p 1
2

2
1 2
2
12

...
2
 p  2



2
... 2n 
 l (θ).
... ... 
2

...  2 
n

...

2
1 n

With observed information matrix ready, the Fisher information can be obtained
7

(26)

I (θ) = E ( J ( θ ) ) .

(27)

B. Wald method
As mentioned above, Wald method is widely used for the calculation of confidence interval, and is
simple to apply. Based on the Lawless’ general procedure for a location-scale distribution, computational
details for confidence interval using Wald method is given as follows.
Let xp be the quantile of a location-scale distribution with parameter u and b respectively, while wp is
the quantile of the same distribution with u = 0, and b = 1. Then, xp = u + wpb which we can estimate by

ˆ
x垐
p = u + w p b,

(28)

using maximum likelihood (ML) estimates û and b̂ . The pivotal quantity is

Zp =

xˆ p − x p
,
se( xˆ p )

(29)

where

(

2
垐
?
se( x垐
p ) = var(u ) + w p var(b) + 2 w p cov(u , b )

)

1/2

,

(30)

where the variance and covariance terms come from the asymptotic covariance matrix for (uˆ , bˆ) , which is
the inverse of Fisher’s observed information matrix, I (uˆ , bˆ) , evaluated at (uˆ , bˆ) . The diagonal elements of

( I (uˆ, bˆ)) −1 give the variances and the off-diagonal elements give the covariance.
Due to assumption of the asymptotic normality of maximum likelihood estimates, Zp is approximately
N(0, 1). Let Z be a N(0, 1) random variable and zα be the value such that

P ( Z  z ) =  .

(31)

A Wald 100(1 - α)% CI for xp is given by

( x垐+ z se( x ), x垐− z se( x )).
p



2

p

p

8



2

p

(32)

Figure 2. “ahat vs. a” regression, within bounded lines.

C. Application on “ahat vs. a” regression and POD curves
As mentioned in [8, 9], “ahat vs. a” regression is constructed with a regression line (the solid),
surrounded by two sets of nearly parallel bounds (dotted lines), as shown in Fig. 2.
The innermost set is the 95% confidence bounds on the line itself. The outer set of dotted lines is called
the 95% prediction bounds. A new response value is expected to be contained by these bounds in 95 of 100
similar situations. Usually these sets of lines go further at both ends, meaning less confidence in the solid
line as we get further from the centroid of the data. For linear regression problem, we obtain the estimates
on the intercept and slope of the solid line, also with uncertainties on those estimates. Near the centroid the
uncertainty in the slope has little influence, but becomes increasingly influential away from the centroid,
resulting in this “dog-bone” confidence bounds.
The estimated response, ŷ , is given by the regression equation

yˆ = 垐
0 + 1 x.

(33)

Based on the variance of a sum, the variance on ŷ can be expressed as

垐
垐 2
var( yˆ ) = var( 垐
0 + 1 x ) = var(  0 ) + 2 x cov(  0 , 1 ) + x var( 1 ),

(34)

from which the 95% Wald confidence bounds on ŷ can be constructed as

垐
?
y垐
 = 0.95 = y  1.645se yˆ =  0 + 1 x  1.645 var( y ).

(35)

The 95% prediction bounds can be constructed following the same process, except that the variance of
the random residual also needs to be included
9

vartotal ( y垐
) = var( y ) +  2 .

(36)

The variances and covariance terms in Eqn. 34 can be obtained from the inverse matrix of Fisher
information. The distribution of model response, y, has the same format of Eqn. 10, and the corresponding
log-likelihood follows Eqn. 13. Thus, the resulted Fisher information matrix is

(

ˆ
I 垐
0 , 1 ,  

)

 n
 垐2
 nX
i
=  i垐=21
 
 0



 i=1 X i
n

 2

 i=1 X i 2
n

 2

0


0

0 ,

2n 
ˆ



(37)

then the covariance matrix has the format of

 V00 V01 V02 


垐
Var  0 , 1 , ˆ  =  V10 V11 V12  .
V

 20 V21 V22 

(

)

(38)

When applying the Wald method to POD curves, the covariance matrix on ̂ and ˆ is can be calculated
from Eqn. 38
1

V + 2 垐
V01 +  2V11 
ˆ1  00

Var ( 垐
, ) =
 12 垐
垐V11 
V01 − V20 − V12 + 
 垐1

1

ˆ12

垐V11  
垐V01 − V20 − V12 + 
1

12

V22 − 2垐
V21 +  V11 
2

.



(39)

With these information ready, it is still not that straightforward because POD curve is actually a
curriculum density function of a log-normal distribution, which is not location-scale. However, the log
format of the random variable follows the normal distribution. Therefore, we can generate the lower
confidence interval on this corresponding normal distribution, N ( 垐
,  ) , then take the exponential value of
the results, due to the monotone characteristics of log-normal distribution [18, 19].
4. Polynomial Chaos Expansions
4.1 Generalized format
The polynomial chaos expansions (PCE) [20] method has the generalized format of:


Y = M ( X) =   i Ψi ( X),

(40)

i =1

where, X∈ℝM is a vector with random independent components, described by a probability density function
fX, Y ≡ M(X) is a map of X, i is the index of ith polynomial term, Ψ is multivariate polynomial basis, and α
is corresponding coefficient of basis function. In practice, the total number of sample points needed does
not have to be infinite, instead, a truncated form of the PCE is used
10

P

M ( X)  M PC ( X) =   i Ψi ( X),

(41)

i =1

where, MPC(X) is the approximate truncated PCE model, P is the total number of sample points needed,
which can be calculated as
P=

( p + n)!
,
p !n !

(42)

where, p is the required order of PCE, and n is the total number of random variables.
4.2 Solving for coefficients
Since a polynomial basis has the characteristics of orthonormality, the equation can be solved by taking
the expectation of Eqn. 40 multiplied by Ψj,

 i = E[ i ( X)  M ( X)],

(43)

which is called quadrature method [21]. This method works well for low-dimensional problems, but suffers
the “curse of dimensionality”.
Another method is to treat the model response as a summation of PCE prediction and corresponding
residual
P

M ( X) = M PC ( X) +  P =   i Ψi ( X) +  P  αT Ψ( X) +  P ,

(44)

i =1

where, εp is the residual between M(X) and MPC(X), which is to be minimized in least-squares methods.
Then the initial problem can be converted to a least-squares minimization problem

αˆ = arg min E[αT ( X ) − M ( X )].

(45)

The first method, used for solving this problem above and applied in this work, is called ordinary leastsquares (OLS) [22], with the coefficients obtained by solving

αˆ = ( AT A ) −1 AT Y,

(46)

where Y is vector of model response, Aji = Ψi(xj), j = 1, …, n, i = 1, …, P.
The second method used for solving Eqn. 45, is the least-angle regression sparse (LARS) [23, 24],
adding one more regularization term to favor low-rank solution [25]

αˆ = arg min E[αT ψ( x) − M ( x)] +  || α ||1,

(47)

where λ is a penalty factor, ||α||1 is L1 norm of the coefficients of PCE.
The LARS [26] algorithm is based on the sparsity-of-effects principle, meaning that only low-order
relationship among inputs are important. These two types of methods solving for least-squares minimization
problem are very efficient in calculation, and can accept an arbitrary number of sample points.
The mean value of PCE is

11

 PC = E[ M PC ( X)] = 1 ,

(48)

where α1 is the coefficient of the constant basis term Ψ1 = 1. The standard deviation of PCE is
P

 PC = E[( M PC ( X) −  PC )2 ] =   i2 ,

(49)

i =2

where it is the summation on coefficients of non-constant basis terms only.
References
[1] Thompson, R., Brasche, L., Forsyth, D., Lindgren, E. and Swindell, P., “Recent Advances in Model-Assisted
Probability of Detection”, 4th European-American Workshop on Reliability of NDE, Berlin, Germany, June 2426, 2009.
[2] Aldrin, J., Knopp, J., Lindgren, E., and Jata, K., “Model-Assisted Probability of Detection Evaluation for Eddy
Current Inspection of Fastener Sites,” Review of Quantitative Nondestructive Evaluation, Vol. 28, 2009, pp.
1784-1791.
[3] Blitz, J., Simpson, G., “Ultrasonic Methods of Non-destructive Testing,” London Chapman & Hall, 1996.
[4] Aldrin, J., Medina, E., Lindgren, E., Buynak, C., and Knopp, J., “Case Studies for Model-Assisted Probabilistic
Reliability Assessment for Structural Health Monitoring Systems,” Review of Progress in Nondestructive
Evaluation, Vol. 30, 2011, pp. 1589-1596.
[5] Aldrin, J., Medina, E., Lindgren, E., Buynak, C., Steffes, G., and Derriso, M., “Model-Assisted Probabilistic
Reliability Assessment for Structure Health Monitoring Systems,” Review of Quantitative Nondestructive
Evaluation, Vol. 29, 2010, pp. 1965-1972.
[6] Knopp, J., Blodgett, M., Aldrin, J., “Efficient propagation of uncertainty simulations via the probabilistic
collocation method”, Studies in Applied Electromagnetic and Mechanics; Electromagnetic Nondestructive
Evaluation Proceedings, Vol. 35, 2011.
[7] Miorelli, R., Artusi, X., Abdessalem, A., and Reboud, C., “Database Generation and Exploitation for Efficient and
Intensive Simulation Studies,” 42nd Annual Review of Progress in Quantitative Nondestructive Evaluation,
2016, pp. 180002-1 – 180002-8.
[8] “Nondestructive Evaluation System Reliability Assessment,” MIL-HDBK-1823, Department of Defense
Handbook, April 2009.
[9] “Nondestructive Evaluation System Reliability Assessment,” MIL-HDBK-1823, Department of Defense
Handbook, April 1999.
[10] Kutner, M., Nachtsheim, C., Neter, J., and Li, W., “Applied Linear Statistical Models,” McGraw-Hill Irwin,
ISBN 0-07-238688-6.
[11] Abdi, H., “The method of least squares," Encyclopedia of Measurement and Statistics, 2007.
[12] Shalizi, C., “The Method of Maximum Likelihood for Simple Linear Regression,” Online course, Carnegie
Mellon University.
[13] Efron, B., “Bootstrap Methods: Another Look at the Jackknife,” The Annals of Statistics, Vol. 7, 1979, pp. 1-26.
[14] “Bootstrap Confidence Intervals,” Online course, Duke University.
[15] Dean, N., and Pagano, M., “Evaluating Confidence Interval Methods for Binomial Proportions in Clustered
Surveys,” Journal of Survey Statistics and Methodology, Vol. 3, No. 4, 2015, pp. 484-503.
[16] Aho, K., and Bowyer, R., “Confidence Intervals for Ratios of Proportions: Implications for Selection Ratios,”
Methods in Ecology and Evolution, Vol. 6, 2015, pp. 121-132.
[17] Godo, B., and Nagy, A., “Fisher Information and Topological Pressure,” Journal of Mathematical Physics, Vol.
58, 2017.
[18] Cheng, R., and Iles, T., “Confidence Bands for Cumulative Distribution Functions of Continuous Random
Variables,” Technometrics, Vol. 25, No. 1, 1983.
[19] Cheng, R., and Iles, T, “One-Sided Confidence Bands for Cumulative Distribution Functions,” Technometrics,
Vol. 30, No. 2, 1988.
[20] Wiener, N., “The Homogeneous Chaos,” American Journal of Mathematics, Vol. 60, 1938, pp. 897-936.

12

[21] Zhang, Z., El-Moselhy, T., Elfadel, I, and Daniel, L., “Calculation of Generalized Polynomial-Chaos Basis
Functions and Gauss Quadrature Rules in Hierarchical Uncertainty Quantification,” IEEE Transactions on
Computer-Aided Design of Integrated Circuits and Systems, Vol. 33, No. 5, May 2014, pp. 728 – 740.
[22] Blatman, G., “Adaptive sparse polynomial chaos expansion for uncertainty propagation and sensitivity analysis”.
Ph.D. thesis, Blaise Pascal University - Clermont II. 3, 8, 9, 2009.
[23] Blatman, G., and Sudret, B., “An adaptive algorithm to build up sparse polynomial chaos expansions for
stochastic finite element analysis,” Probabilistic Engineering Mechanics, Vol. 25, No. 2, 2010, pp. 183–197.
[24] Blatman, G., and Sudret, B., “Adaptive sparse polynomial chaos expansion based on Least Angle Regression,”
Journal of Computational Physics, Vol. 230, 2011, pp. 2345–2367.
[25] Udell, M., Horn, C., Zadeh, R., and Boyd, S., “Generalized Low Rank Models,” Foundations and Trends in
Machine Learning, Vol. 9, No. 1, 2016, pp. 1-118.
[26] Efron, B., Hastie, T., Johnstone, I., and Tibshirani, R., “Least Angle Regression,” The Annals of Statistics, Vol.
32, No. 2, 2004, pp. 407-499.

13



---

Source Exif Data:

File Type                       : PDF
File Type Extension             : pdf
MIME Type                       : application/pdf
PDF Version                     : 1.7
Linearized                      : No
Page Count                      : 15
Language                        : en-US
Tagged PDF                      : Yes
XMP Toolkit                     : 3.1-701
Producer                        : Microsoft® Word 2016
Creator                         : Du, Xiaosong [AER E]
Creator Tool                    : Microsoft® Word 2016
Create Date                     : 2018:07:12 21:53:25-05:00
Modify Date                     : 2018:07:12 21:53:25-05:00
Document ID                     : uuid:CB9D4406-3C0C-4DEA-BFA1-8FE73F8CAC6A
Instance ID                     : uuid:CB9D4406-3C0C-4DEA-BFA1-8FE73F8CAC6A
Part                            : 3
Conformance                     : B
Author                          : Du, Xiaosong [AER E]

EXIF Metadata provided by EXIF.tools