Ranadu Manual

RanaduManual

RanaduManual

User Manual:

Open the PDF directly: View PDF .
Page Count: 49

Download
Open PDF In Browser	View PDF

Earth Observing Laboratory
Memorandum:
8 November 2018
To:
From:
Subject:
Version:

1

Ranadu documentation
Al Cooper
Ranadu manual and tutorial
Ranadu_2.5-18-11-02

Introduction

This manual describes an R package named “Ranadu” (R-based analysis for aircraft data users),
which is intended to facilitate use of the data archives produced by the data systems on the
NSF/NCAR/EOL/RAF research aircraft. (NSF=National Science Foundation; NCAR=National
Center for Atmospheric Research; EOL=Earth Observing Laboratory; RAF=Research Aviation
Facility) All the recent data files are in netCDF format, so that is the format emphasized here.
Those files contain measurements made in field campaigns that use the NSF research aircraft operated by NCAR, presently consisting of a C-130 and a Gulfstream V. A list of recent projects is
available at this EOL web page, and data requests can be made via links on that page.
The standard data archives consist of netCDF files that contain a set of measurements each second
during a flight. Special archives may be available at higher rate, usually 25 Hz. The Ranadu functions produce R data.frames containing subsets of those measurements and then facilitate plotting
and analyses using those data.frames. The table in the Appendix contains an alphabetical list of
the functions, brief descriptions, and references where additional information can be found.
There are several groups of functions that serve different purposes. Some examples are listed
below. For a complete list, see the Appendix.
1. Functions for data access:
(a) getNetCDF() for reading netCDF files to R data.frames
(b) DataFileInfo() for a summary of the content of a netCDF archive file
(c) standardVariables() to define common variables for (a)
(d) setVariableList() to provide a GUI interface to variable selection
(e) TimeRoutines() including setRange(), getIndex(), getStartEnd()
2. Functions that implement calculation algorithms for derived measurements: [partial list]
(a) TrueAirspeed()
(b) WindProcessor() to calculate wind from basic measurements
(c) AirTemperature() to calculate ambient temperature
(d) DPfromE() to get the dew point temperature from the water vapor pressure
(e) PCorFunction() to correct pressure measurements for the static defect

Memo to: Ranadu documentation
8 November 2018
Page 2

3. Convenience functions for plots:
(a) plotTrack() for a simple flight track.
(b) plotWAC() and lineWAC() to generate plots.
(c) plotSD() for size-distribution plots.
(d) contourPlot() for colored-area plots substituting for scattergrams.
(e) OpenInProgram() to transfer variables in a data.frame to another analysis program
like ncplot
4. Utilities for handling and examining data:
(a) makeNetCDF() to make a new netCDF file from a data.frame
(b) Rsubset() to create subset data.frames while preserving global and variable attributes
(c) TellAbout() to list characteristics of variables
(d) ValueOf() to look up values from the data.frame
(e) getAttributes() to see the original netCDF attributes
(f) DataFileInfo() to list variables and other information in a netCDF file or corresponding Rdata file
5. Miscellaneous functions for analysis:
(a) DemingFit() to minimize the perpendicular distance of points from a line
(b) VSpec() to generate plots of variance spectra.
(c) skewT() to construct a skew-T thermodynamic diagram and superimposing measurements
(d) LCL() to calculate the lifted condensation level
(e) CAPE() to calculate the convective available potential energy as well as adiabatic profiles for ascent
(f) VSpec() to generate plots of spectral variance
(g) LagrangeInterpolation()
(h) binStats() to aid in constructing error-bar or bar-and-whisker plots
6. Functions to improve measurements:
(a) CorrectPitch(), CorrectRoll() to improve measurements of pitch and roll
(b) CorrectHeading() to improve measurements of heading

Memo to: Ranadu documentation
8 November 2018
Page 3

When data.frames are created, global attributes in the netCDF file become attributes of the generated data.frame and variable attributes in the netCDF file are associated with column variables
in the data.frame. Other information in the netCDF file is also stored with the data.frame, so the
data.frame becomes a full representation of the netCDF file and complete netCDF files can be
re-generated from the data.frames. However, usually data.frames are generated with only subsets
of the data needed for particular analysis routines. They therefore can be small and suitable for
archiving with analysis projects to document the data as used for the project.
Because these functions are assembled into an R package, they each have associated usage information. This wiki is intended to provide additional information regarding the basis for the
algorithms used, the limitations of the functions, and more general documentation. Some of the
functions (will) have their own pages for this information.
This manual includes an introductory set of sections that discuss R and how to use it for data
analysis. This is intended to be tutorial information, focused on the applications to aircraft data,
and some of the information is also specific to uses on NCAR/EOL computers, so Sections 2–4
can be skipped by those who already are comfortable with R. Section 5 is the central discussion of
Ranadu. Section 6 is mostly advocacy for using these routines and other capabilities of R to make
analyses reproducible by others, and Sect. 7 presents a brief discussion of how these routines can
be used for data-quality review during field projects.

2

Introduction to R

The very limited intent here is to provide a brief context in R for using the Ranadu package. There
are many books, tutorials, and web resources that provide complete information on R, including
the manuals available via R itself (e.g., via the ’help’ tab in RStudio). Therefore, this and the
following few chapters are provided only to get a reader started with R and to illustrate some of
the analysis tasks that are possible.

2.1

Getting access: Some recommended set-up work

This section will guide you through obtaining R (if necessary), RStudio (which I recommend but
which isn’t essential), and the Ranadu package itself. I’m going to use barolo.eol.ucar.edu, a linux
machine, for these illustrations. R and RStudio are available on barolo, so if you have access to
that machine these are already installed and maintained. However, you may also want them on
your own machines or laptop.
The central repository for R is CRAN (http://cran.r-project.org/), the Comprehensive R Archive
Network. That page has links for downloading versions for Windows, Mac, and linux. For RStudio, you can download from http://www.rstudio.com/products/RStudio/. To run R (without RStudio) on barolo or any linux machine where it is installed, just type “R” into a terminal window1
1 To

exit, type “quit()”; reply ’n’ to ’save workspace’ for now.

Memo to: Ranadu documentation
8 November 2018
Page 4

and you get the classic command-line interface that has been in use since at least 1998. RStudio is
no longer available on EOL computers, so to use it you need to install it on your own computer.2
My suggestion for getting started is this:
The best way to install Ranadu, if you already have R and RStudio installed, is to enter
devtools::install_github(“WilliamCooper/Ranadu”)
into the R console, from your directory “~/RStudio”, once you have installed R and RStudio. This
provides the functions in the package. If you want to install the R scripts from the package or
otherwise set up your system to work with this package, you may also want to do the following:
1. Make a new directory “RStudio” in your workspace on the machine you will use, named
’~/RStudio’. You can do this using a terminal or using the RStudio interface; for the latter,
(a) click the ’Files’ tab in the lower right pane
(b) use the ’New Folder’ item just below that tab to create the new folder named ’RStudio’
under your home directory.
2. Subsection 2.3 below describes how to download your own copy of the libraries that will be
needed so that you can maintain and update your own copy.
3. Type “rstudio”.
4. Go to Tools -> Global Options -> R General
(a) enter ’~/RStudio’ (without quotes) in the line that says “Default Working Directory”
5. Go to Sweave under the same window
(a) in the top where it says “Weave Rnw files using:”, select ’knitr’. You may get a message
saying knitr is not available; in that case, knitr will be added to the available packages
when you follow the steps in subsection 2.3.
6. Go to “Git/SVN” under the same window
(a) check “Enable version control”. This is very useful and will be the best way to stay
current with the Ranadu package, which is installed and maintained there.
7. For other options, the defaults are probably reasonable to start, so click “OK” to save the
options.
2 It

is also possible to run RStudio and R on Amazon Web Services, for example in a
linux AMI machine used as a web server for RStudio.
To do this, follow instructions here:
https://blogs.aws.amazon.com/bigdata/post/Tx3IJSB6BMHWZE5/Running-R-on-AWS . You will also have to
install the netcdf library (netcdf and netcdf-devel), and for that you will first need hdf5 and hdf5-devel. Download
the AWS version of Ranadu, which omits one routine dependent on tcltk and omits that package dependence, if the
version of R you use doesn’t support tcltk.

Memo to: Ranadu documentation
8 November 2018
Page 5

8. Optionally, make a new project called RSessions for the purpose of a tutorial having this
name:
(a) Go to the ’File’ menu and select ’New Project’
(b) Select ’Version Control’
(c) Select ’Git”
(d) Use this repository name: https://github.com/WilliamCooper/RSessions.git .
(e) Give the project the name ’RSessions’ and create it as a subdirectory of ’~/RStudio’
(f) At this point, RStudio will switch to windows that apply to this new project and copy
from the repository into your new project.
9. If you are preparing your personal machine, you may eventually want LATEX, and you may
need to install git, although it seems to be standard on linux and Mac machines. If you are installing on another machine that doesn’t have them try these links: For LATEX, http://miktex.org/
and for git http://git-scm.com.

2.2

A quick tour of the RStudio window panes:

If you haven’t already done this, start RStudio on your machine so you can see the windows, panes,
and functions described here. You have seen the ’Files’ window before if you followed the steps
in the preceding section; click on that to see the files in RStudio/RSessions. Then, if you have
followed step 8 above, click on ’RSessionsDataFrame.Rnw’ to have something to look at. It will
appear in the top left window. Don’t worry about the details of what is there just yet; we’ll get
back to that. First, notice that there are four panes visible. At start-up, you will usually see these
panes:
1. Top left: The source editor, where a working file (or many working files in tabs) are displayed.
2. Bottom left: The console, where you can enter R commands just as you would in the
command-line mode that you would use without RStudio. Try it now: Click where the ’>’
prompt is displayed, type 2+2 [enter]. I use this frequently as a very capable calculator, because the math functions and R functions are all available here. Try ’pi’, ’rnorm(5, 3.5, 0.5)’,
plot(0:360, sin(0:360 * pi / 180), type=’l’), and ’Ranadu::WetEquivalentPotentialTemperature
(P=800., AT=25., E=Ranadu::MurphyKoop (DP=20., P=800)).
3. Top right: This pane has several functions.
(a) The ’History’ tab displays the history of what you have entered in the console, along
with options to transfer lines either to the source editor or to the console for re-execution,
possibly with editing (although that function is available more conveniently from the
console using the up and down arrows to move through the history).

Memo to: Ranadu documentation
8 November 2018
Page 6

(b) Another tab will show the environment, where defined variables for your workspace
are shown. [Try X = 5 in the console and see what appears here.] There is also a useful
tab for importing a data set from either a web location or a text file.
(c) This pane also has a ’git’ tab if you have set up the project as recommended above,
where you can monitor the status of what has changed since you last saved to your
local repository and where you can commit back to your local repository. You can also
“pull” from the github repository where appropriate (e.g., to get Ranadu updates).
4. Bottom right: Again, there are several functions.
(a) Plot: If you tried the examples in item 2 above, you will have noticed this pane switch
to ’Plots’ and display the latest plot. RStudio also maintains a history of the plots that
have been generated, and you can skip back through that history, which seems endless
in my experience so far. The ’export’ function in the ’Plot’ tab allows you to save the
plot externally to either png or pdf format.
(b) Files: This displays the files in the working directory, which for now should be the
project directory. There are a few functions for dealing with those files.
(c) Help: Requests for help are displayed in this window. If you click on the ’Home’
button, you are taken to a page that shows links to some of the basic manuals and also
a definitive definition of the R language, along with other more specialized manuals. If
you want to know how a particular function works, you can enter your search phrase
into the console as, for example, ?atan2 to see information on the arctangent function,
or you can enter this into the entry line on the help window, which has the advantage
that it displays auto-completion options as you type. [Try ’plot’ to see this in action.]
There are many cross-references among the help pages that are quite useful. If you
click on the left drop-down menu you will see a history of the help topics you have
consulted.
(d) Viewer: I’m going to postpone discussion of this until later; you won’t need the viewer
for a while.
(e) Packages: This displays the packages that are available for your use, either in the
system-wide repository or in your personal repository. To test if you have access to
Ranadu, look for ’Ranadu’ in the list of packages. If it is there, let your mouse linger
over it and its location will be displayed. You could try this for other packages also;
check, for example, ggplot2, a very useful and standard plotting package not yet part of
the EOL repository. The check boxes indicate which packages will be available in your
session without special loading; if unchecked, you can either check the box or include
a statement similar to ’require(Ranadu)’ in your session (or when you use these functions, use the form Ranadu::routine_you_want). The ’install’ and ’update’ functions at
the top of the ’Packages’ pane provide easy access to the ’CRAN’ archive of packages
or to packages you may have from other sources that you want to load for use. There
is a very extensive set of packages available via CRAN. If you need something new,
check there to see if someone has already provided a package for the task you need.

Memo to: Ranadu documentation
8 November 2018
Page 7

2.3

Getting the ’Ranadu’ package

You may first need to install some packages on which Ranadu depends. Ranadu uses these packages: R (version >= 2.10), ncdf4, maps, ggplot2, ggthemes, grid, tcltk, nleqslv,
reshape2, zoo, fields, stats, reshape2, bspec, utils, scales, plyr, and signal.
You will also want knitr if you want to combine text and R code in files that generate documents,
as described in a later section. If any of these are missing, they can be installed by clicking ’Install’
at the top of the ’Packages’ pane of RStudio, selecting ’repository (CRAN)’ in the first drop-down
menu, and entering a list of packages that you are missing in the second line labeled ’Packages’.
The default location for local libraries on the last line should be kept, and the check-box labelled
’Install dependencies’ should be checked. Click ’Install” and those packages should be downloaded from the CRAN repository, compiled where necessary, and added to your list of packages.
Then to get the package ’Ranadu’, follow these steps:
Preferred Method:
1. In the R or RStudio console, type devtools::install_github(“WilliamCooper/Ranadu”).
If it is not already present, you may need to install devtools,
2. Change to the “Ranadu” project.
3. Go to the “Build” window and click “Build & Reload”.
An advantage of this method is that, if you create Shiny apps that use Ranadu, the installer will be
able to upload the needed parts of Ranadu with the Shiny app.
Alternate Method:
1. Within RStudio, select ’File’ -> ’New Project’ -> ’Version Control’ -> ’Git’ (clone a repository)
2. For the package URL, enter ’https://github.com/WilliamCooper/Ranadu.git’ (without the
quote marks). Use the package name ’Ranadu’ and make the directory for it a subdirectory under ’RStudio’.
3. Click ’Create Project’ and RStudio will download the files and change the working project
to Ranadu.
4. Go to the ’Build” window in the top-right pane and click ’Build & Reload’. You may get
an error listing missing dependencies if any required packages are unavailable. In that case,
return to the above description of how to install packages and install the missing dependencies.
5. The package ’Ranadu’ should appear in your list of packages.

Memo to: Ranadu documentation
8 November 2018
Page 8

3

R objects, with focus on the data.frame

3.1

Some basic objects

vector
The fundamental objects we will deal with the most is the vector. This is an indexed set of quantities, which can be numeric (double, integer), logical, character, or some other less common types.
Vectors are ’atomic’ if they consist all of the same type; ’generic’ vectors can have mixed types,
but I won’t deal much with them. Vectors can have only one element, which would be called a
scalar in other languages, but scalars in R are just vectors with one element. Vectors have a length
property [try length(x)] but not a dimension property: Try the following:
x <- 2
print (x[1])
length(x)
dim(x)
The ’<-’ operator above is the R assignment operator. It is mostly equivalent to the ’=’ operator
from other languages, and ’=’ can usually be used in place of this operator, but it does have the
clarity of indicating that the variable on the left is assigned the value of the quantity on the right.
The operator ’->’ can also be used, but usually will result in rather confusing code.
list
A list resembles an atomic vector but can have mixed types. Lists also have additional ways of
selecting elements, e.g., by character name. Lists are generic vectors, but when I talk about vectors
I’ll almost always be talking about atomic vectors.
array
A special vector-like object is the array or matrix. These are like vectors but with an assigned
dimension so that, in the two-dimensional or matrix case, the vector is a vector of columns in a
matrix. The dimension can be single-valued, but more often there are at least two dimensions.
R convention is column-major format (like FORTRAN but opposite to C); i.e., the row index
represents adjacent values in the underlying vector (items that lie vertically in the conventional
display of a matrix) so that for example z[1,1] and z[2,1] are adjacent in the underlying vector
and aligned vertically in a display of the matrix. The first index in a two-dimensional array can
be thought of as specifying the row and the second the column; all rows in a particular column
appear together. You can construct an array from a vector by assigning it a dimension via ’dim(x)

Memo to: Ranadu documentation
8 November 2018
Page 9

<- c(dim1, dim2)’ as in the following example,3 which also illustrates the column-major structure
and the effect of transposing the array using t():
a <- 1:12
print (a)
##

[1]

1

# generate a vector containing the sequence 1 to 12

2

3

dim (a) <- c(3,4)
print (a)

4

5

6

7

8

9 10 11 12

# c() constructs a vector from its elements

##
[,1] [,2] [,3] [,4]
## [1,]
1
4
7
10
## [2,]
2
5
8
11
## [3,]
3
6
9
12
print (t(a))
##
##
##
##
##

[1,]
[2,]
[3,]
[4,]

[,1] [,2] [,3]
1
2
3
4
5
6
7
8
9
10
11
12

(Note that ’#’ indicates the the remainder of the line is a comment.) This works well for arrays
constructed from our netCDF files because the time index can be thought of as the row index and
each column represents a vector of measurements of a particular variable.
data.frame
A data.frame is a named list of vectors and resembles a matrix or a spreadsheet. For example,
it may consists of columns each representing a variable and rows that are the time sequence of
observations of that variable. Applied to RAF data files, it may have a structure like this:
3 Want

your original vector back? Use x <- as.vector(x)

Memo to: Ranadu documentation
8 November 2018
Page 10

Time
15:00:00
15:00:01
15:00:02
15:00:03
15:00:04
...

ATX
-25.1
-25.1
-25.2
-25.1
-25.0
...

PSXC
410.8
410.9
411.1
411.1
411.2
...

WDC
275.4
275.2
275.4
275.1
275.3
...

WSC
25.4
25.6
25.5
25.7
25.3
...

...
...
...
...
...
...
...

All columns must be of the same length, but they may contain different types of variables (Time,
character, numeric, logical). Like a spreadsheet, the columns can be assigned names like the header
in this table. Rows can also be assigned names, but in the absence of special assignment they will
default to the character names ’1’, ’2’, ’3’, ’4’, ... and can be addressed by an index that is the first
in a two-index specification; i.e., D[500, 10] represents the 10th variable as measured at the 500-th
time in the dataframe.
Let’s get an example. Included with Ranadu is a sample data.frame named RAFdata that contains
a short sample of measurements from a research flight.
require(Ranadu, quietly = TRUE, warn.conflicts = FALSE)
names (RAFdata)
## [1] "Time"
## [8] "VNS"
## [15] "QCXC"
## [22] "GGALT"

"ATTACK"
"ATX"
"EWX"
"LATC"

"SSLIP"
"DPXC"
"RTH1"
"LONC"

"GGVEW"
"THDG"
"RTH2"
"PLWC"

"GGVNS"
"ROLL"
"ADIFR"
"CONCD"

"GGVSPD"
"PITCH"
"BDIFR"
"WDC"

"VEW"
"PSXC"
"TASX"
"WSC"

Ranadu::getStartEnd (RAFdata)
## [1] 201000 201500
names() is a standard R function that lists the names associated with an object, so here it lists the
names of the variables in the data.frame. getStartEnd() is a Ranadu function that returns the
start and end times in the data.frame. The leading ’Ranadu::’ is not necessary here because
the package is loaded, but these notes will tend to include that superfluous header to highlight
which parts of the code refer to the Ranadu package. The lines above starting with ’##’ are the
output from the R code in the listing. The first few rows and first few columns in the data.frame
’RAFdata’ look like this:
##
Time
ATTACK
SSLIP
GGVEW
GGVNS
## 9721 2013-10-01 20:10:00 1.999224 -0.1875578 53.44360 226.5322
## 9722 2013-10-01 20:10:01 1.943987 -0.1734225 53.57179 226.9628
## 9723 2013-10-01 20:10:02 1.981951 -0.1649992 53.69322 227.4117

Memo to: Ranadu documentation
8 November 2018
Page 11

##
##
##
##
##
##
##

9724
9725
9726
9727
9728
9729
9730

2013-10-01
2013-10-01
2013-10-01
2013-10-01
2013-10-01
2013-10-01
2013-10-01

20:10:03
20:10:04
20:10:05
20:10:06
20:10:07
20:10:08
20:10:09

1.993126
1.996357
1.987496
1.996506
1.989084
1.994024
1.989765

-0.1854287
-0.1891418
-0.1899927
-0.1594946
-0.1842116
-0.1646231
-0.1765334

53.81462
53.94920
54.07005
54.18315
54.26129
54.33908
54.41174

227.8630
228.2795
228.7055
229.1384
229.5390
229.9423
230.3561

The ’Time’ variable is a special-format variable in POSIX format, discussed later when the
Ranadu access functions are described. The first item in each row is the row name, seldom used
but here a remnant from the index in the original data.frame from which this subset was extracted.
Having the time variable in this format is particularly convenient when examining time series of
measurements, as will be described later.

3.2

Working with data.frames

Addressing elements of a data.frame
You can address particular elements in many ways, using syntax like the following:
RAFdata$ATX[5]
## [1] -36.71588
RAFdata[5, 2]

# note the [row,column] syntax

## [1] 1.996357
RAFdata[5, ]
##
##
##
##
##
##
##
##

9725
9725
9725
9725

Time
ATTACK
SSLIP
GGVEW
GGVNS
GGVSPD
2013-10-01 20:10:04 1.996357 -0.1891418 53.9492 228.2795 -0.58602
VEW
VNS
ATX
DPXC
THDG
ROLL
PITCH
54.11258 228.2388 -36.71588 -50.19225 2.978105 0.3509116 1.760113
PSXC
QCXC
EWX
RTH1
RTH2
ADIFR
BDIFR
301.8117 126.1786 0.06180029 -12.34667 -12.4678 -13.82708 -0.7944382
TASX
GGALT
LATC
LONC
PLWC CONCD
WDC
WSC
223.3128 9424.435 44.99973 -101.271 8.453611
0 262.9346 42.98902

RAFdata[5, "ATX"]

# using " or ' to surround text is usually equivalent

Memo to: Ranadu documentation
8 November 2018
Page 12

## [1] -36.71588
RAFdata$ATX[1:10]
##
##

[1] -36.77266 -36.77094 -36.75904 -36.73860 -36.71588 -36.70886 -36.69385
[8] -36.68116 -36.71080 -36.72738

RAFdata$ATX[Ranadu::getIndex(RAFdata, 201105)]
## [1] -36.58235
RAFdata$ATX[RAFdata$Time == as.POSIXct("2013-10-01 20:11:05", tz='UTC')]
## [1] -36.58235
(’getIndex ()’ is part of Ranadu and provides a convenient way to determine the data.frame index
that corresponds to a specified time.)
Creating subsets of a data.frame
New data.frames that contain subsets of original data.frames can be created using logical vectors.
For example:
RAFdata[RAFdata$TASX < 213, "TASX"]
## [1] 212.9421 212.9741 212.9477
RAFdata[Ranadu::setRange(RAFdata, 201210, 201215), c(1, 6:10)]
##
##
##
##
##
##
##

9851
9852
9853
9854
9855
9856

2013-10-01
2013-10-01
2013-10-01
2013-10-01
2013-10-01
2013-10-01

Time
20:12:10
20:12:11
20:12:12
20:12:13
20:12:14
20:12:15

GGVSPD
-10.78104
-10.80652
-10.88169
-10.89062
-10.89485
-10.81118

VEW
57.81869
57.80622
57.78197
57.74649
57.71845
57.69278

VNS
243.1800
243.0562
242.9372
242.8225
242.7063
242.5950

ATX
-33.63138
-33.50258
-33.42321
-33.37087
-33.28124
-33.20294

Another useful subset is that omitting all missing-variable rows from the data.frame:

DPXC
-54.88298
-54.90860
-54.71121
-55.01767
-54.75344
-54.59950

Memo to: Ranadu documentation
8 November 2018
Page 13

RAFdataNoNA <- na.omit(RAFdata)
However, be careful using this and other subsetting commands because if the time sequence has
gaps some functions like ’setRange()’ won’t work, although plots will just skip the missing values.
When it is necessary to omit missing values (e.g., for filtering), it is best to do other subsetting
before using the na.omit() function.
Adding or changing variables in a data.frame
You can operate on variables in the data.frame, changing values, and you can add new variables to
the data.frame as follows:
# wind component from the east:
RAFdata$UEW <- RAFdata$WSC * sin (RAFdata$WDC * pi / 180)
summary (RAFdata$UEW)
##
##

Min. 1st Qu.
-44.64 -43.67

Median
-42.66

Mean 3rd Qu.
-41.73 -38.79

Max.
-38.01

A few comments on this example may be useful. The function summary() provides a quick indication of the range of a variable and also includes an indication of the number of missing elements
(having the R value NA). The assignment equivalently could have been made to RAFdata[’UEW’].
The standard math functions in R include the ’sin()’ function used here, but it takes an argument
in radians so the wind direction is multiplied by pi/180 to convert from the wind direction WDC
in degrees. “pi” is always available, and R makes no distinction between the numerical values
“180.’ and “180” so the decimal point that would be used in many other languages is superfluous
here.
Simple plots
Let’s plot something, using ’plotWAC()’ from Ranadu for convenience, although you can try the
standard ’plot ()’ function from R base routines to see what changes I have made:
Ranadu::plotWAC(RAFdata$Time, RAFdata$GGVSPD, ylab='rate of climb [m/s]')

−4
−6
−8
−10

rate of climb [m/s]

−2

0

Memo to: Ranadu documentation
8 November 2018
Page 14

20:10

20:11

20:12

20:13

20:14

20:15

TIME [UTC]
It is also useful to define special data.frames for constructing plots, especially when using the more
advanced plotting capabilities provided by ’ggplot2’. To see a simple scatterplot, you can use the
following:
D <- RAFdata[, c("ATX", "GGALT")]
plot(D, pch=20)
# pch=20 plots small solid dots
## equivalent to:
plot (RAFdata[, c("ATX", "GGALT")], pch=20)

8500
8000
7500

GGALT

9000

9500

Memo to: Ranadu documentation
8 November 2018
Page 15

●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●●●
●
●●●
●●●
●
●●●
●●●
●
●●
●
●
●
●
●
●
●
●●●
●●●
●●●
●●
●
●●
●
●●●
●●●
●●●
●●●
●●●
●●●
●
●●●
●●●
●●●
●●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●●
●●
●
●●
●●●
●●
●
●
●●●
●●
●●●
●●
●
●
●
●
●
●
●●●
●●●
●
●●●
●●
●●
●
●
●●
●
●●●
●●●
●●●
●●●
●●●
●●●
●
●
●●
●
●
●

−35

−30

−25

ATX
## or to
plot (subset (RAFdata, , c(ATX, GGALT)), pch=20)
You may find the last form attractive because it avoids the need to type the quote marks in the
second form, and those quote marks can seem tiresome in extended strings. Exercise: See what
happens if you instead include three variables in the preceding plot, e.g., by adding DPXC.
Exporting to Excel
If you have the xlsx package available, you can create an Excel spreadsheet with the data:
# require(xlsx)
xlsx::write.xlsx (RAFdata, file="RAFdata.xlsx")
# system("libreoffice RAFdata.xlsx") ## to see the resulting spreadsheet
This can be very convenient for those who are comfortable working with the features of Excel
spreadsheets, and this is also a convenient way to examine data.frames.

Memo to: Ranadu documentation
8 November 2018
Page 16

Exporting to ncplot
Many users of these aircraft data archives will be familiar with ncplot, a program provided by
NCAR/EOL. Use the OpenInProgram() to start ncplot with the variables in an R data.frame, as
follows:
# OpenInProgram (RAFdata) ## not executed here
## try ?OpenInProgram to see the options for this function
New variables added to ’Data’, perhaps representing alternate calculations of standard variables,
can then be plotted along with the original variables. 4

4

Some basic operations in R

4.1

Using the RStudio console as a calculator

The standard R interface or the console pane in RStudio (the lower left pane) can be used as a
convenient calculator, and it is possible to do some simple programming interactively. As a simple
example, let’s find the roll angle for a 3-min turn (i.e., a turn through 360◦ in 3 minutes). If the
airspeed is v, the radius of the turn is r, the acceleration of gravity is g, the time taken for a turn
through 2π radians is T , and the roll angle is φ , then
v2
= g tan φ
r
is the required horizontal acceleration and the distance flown around the turn is
2πr = vT
so


2πv
φ = arctan
gT



Here is an evaluation in R:
4 This

also works with ’Xanadu’, a legacy program not included with this package and no longer supported that
nevertheless provides some additional capabilities including the possibility to transfer the data to still other analysis
programs including IDL, NCL, and python and to take a variety of approaches to spectral analysis.

Memo to: Ranadu documentation
8 November 2018
Page 17

v <- 220
## do the calculation for a flight speed of 220 m/s
g <- 9.8
## acceleration of gravity in m/s^2
T <- 3 * 60
phi <- atan (2 * pi * v / (g * T)) * (180 / pi) ## convert to degrees
print (sprintf ("required roll angle is %.1f degrees", phi))
## [1] "required roll angle is 38.1 degrees"
The console is often very useful for exploring aspects of data. For example, after running R
scripts, the R objects remain available in the working environment and can be examined, plotted,
or otherwise used in exploratory analysis.

4.2

Some math conventions

Here the focus is on what might seem different to those experienced in other programming languages.
Operator precedence
• :: $ [ ] PEU:MDAS
:: references a package and has highest precedence
$ references a column of a data.frame (for example)
[] are used for subsetting operations
P denotes parentheses
E denotes exponentiation (see below)
U denotes the unary operators ’-’ and ’+’ when used to indicate signs
: is an operator that generates sequences
%% operators include the modulus operator and some others like %/%
M,D denote multiplication and division and have equal precedence so are evaluated in order
A,S denote additional and subtraction and are equal in precedence.
• Here are some special cases to note:
In “1:5 * 2”, : has precedence so you get c(2,4,6,8,10) not c(1,2,3,4,5,6,7,8,9,10)
The precedence of logical operators is ! (& &&) (| ||) xor . Note that & has precedence over
|, so for example the result of TRUE | FALSE & FALSE is TRUE.
Operators to note:
• The exponentiation operator is ’^’, although ’**’ is also accepted.
• & and | are vectorized while && and || return a single logical result.

Memo to: Ranadu documentation
8 November 2018
Page 18

• %% is the modulus operator; i.e., 8 %% 3 is 2. %/% gives integer division; e.g., 16 %/% 3
gives 5.
• %in% tests if an element is present in a vector; e.g., “TASX” %in% names(RAFdata)
• The function c() combines its arguments, usually to form a vector; e.g., A <- c(1, 3, 6)
• The test for equality is ’==’, not ’=’; the latter will result in assignment and is usually an
acceptable substitute for ’<-’
• R does not have operators like ’+=”, “++”, etc, so you have to write out statements like ’i  MinTAS) & (abs (Data$ROLL) < 4) & (Data$GGVSPD > 2)
Data[!Valid, c(“WDC”, “WSC”)] <- NA
(c) Consider also the ’subset’ command, for which the second argument is a logical test
and the third specifies the variables to include in the subset (see ?subset for more information):
D <- subset (RAFdata, RAFdata$GGVSPD > 0, c(Time, ATX, GGVSPD))
str (D)
## 'data.frame': 29 obs. of 3 variables:
## $ Time : POSIXct, format: "2013-10-01 20:10:10" "2013-10-01 20:10:11" ...
## $ ATX
: num -36.7 -36.7 -36.6 -36.7 -36.6 ...
## $ GGVSPD: num 0.0557 0.0586 0.0251 0.0359 0.0864 ...

4.4

R packages

A strength of R is the extensive set of packages available to help with analysis tasks. The standard
repository is called “CRAN”, although there are many packages available on GitHub also. To see
the packages available on CRAN, check this URL. Most of what has been discussed to this point
is in the ’base’ package, including the math functions. Some additional frequently used functions,
like ’plot()’, are in other standard packages (in this case, ’graphics’) and are usually available
without special actions. However, others require special action like loading via the ’require()’
or ’library()’ functions, or access via statements like ’packageName::function()’ (e.g.,
Ranadu::getNetCDF()). A few of the packages used by Ranadu routines, and some others that
have proven especially useful, are discussed in this subsection to provide some context for what
capabilities they provide.
These are some of the packages referenced by Ranadu:
1. ncdf: basic netCDF functions. This package is used for access to the netCDF data files
archived by NCAR/EOL/RAF.

Memo to: Ranadu documentation
8 November 2018
Page 20

2. ggplot2 and ggthemes: These are alternatives to the standard graphics package that provide
particularly aesthetic plots.
3. stats and signal: used for interpolation and filtering of signals
4. nleqslv: solve non-linear equations (used e.g. by DPfromE() and LCL())
5. maps: provides state or national boundaries for maps
6. fields: used by the skewT-generating program to average measurements in bins
7. zoo: used to replace missing values by interpolation
8. grid and reshape2: used to customize the ggplot theme
Two other packages that are particularly useful for the purposes of this manual are knitr and devtools. The former provides the ability to intermix text-generating code with R code, and so provides an important route to making analyses reproducible, as discussed later. ’devtools’ is useful
for checking and building packages and may be useful if you want to add components to a package
like Ranadu.

5
5.1

Using the Ranadu package
Getting help

All functions will provide information regarding their usage in response to a ?FunctionName
query in the console or the RStudio help window. (Example: ’?getNetCDF’) In addition, in most
cases that help window provides an example that uses the ’RAFdata’ data.frame and shows how
results should look. These help displays should be considered the definitive information because
they are more likely to remain up-to-date than is this manual.

5.2

The data.frame containing measurements

Using getNetCDF
Ranadu is based on data.frames that contain subsets of data representing measurements in the
NCAR/EOL/RAF netCDF data archives. The starting point therefore is to construct those data.frames.
These routines assist in that task:
• Data <- getNetCDF ( ): loads a data.frame named Data with requested variables
• standardVariables ( ): defines a common set of variables for input to getNetCDF().
This list is the default if no variables are specified in the call to getNetCDF.

Memo to: Ranadu documentation
8 November 2018
Page 21

• DataDirectory ( ): This is system dependent and is intended to aid in portability by
providing the usual location of the netCDF data files. It tries a set of likely names, but may
need redefinition for some file systems.
• getIndex ( ): finds the index for a specified time
• setRange ( ): sets a range of indices covering a specified time interval
• TellAbout (V): lists some characteristics of variable V
• Z <- getAttributes (V): lists the original netCDF attributes associated with V (either a
data.frame or a variable in a data.frame)
• FI <- DataFileInfo ( ): produces a list containing properties of a specified netCDF file or the
corresponding Rdata file.
The starting point will usually be getNetCDF(). Here is an example of how this might be used:
require (Ranadu, quietly=TRUE)
## try ?getNetCDF
Data <- getNetCDF (fnm <- sprintf ("%s/extdata/RAFdata.nc",
path.package ("Ranadu")), "ALL")
dim (Data)
## the length in seconds and the number of variables
## [1] 301

28

names (Data)
## [1] "Time"
## [8] "VNS"
## [15] "QCXC"
## [22] "GGALT"

## see the variables in Data
"ATTACK"
"ATX"
"EWX"
"LATC"

getStartEnd (Data)

"SSLIP"
"DPXC"
"RTH1"
"LONC"

"GGVEW"
"THDG"
"RTH2"
"PLWC"

"GGVNS"
"ROLL"
"ADIFR"
"CONCD"

## the start and end times

## [1] 201000 201500
## try TellAbout (Data)
Z <- getAttributes (Data$ATX)

"GGVSPD"
"PITCH"
"BDIFR"
"WDC"

"VEW"
"PSXC"
"TASX"
"WSC"

Memo to: Ranadu documentation
8 November 2018
Page 22

##
##
##
##
##
##
##
##

[1]
[1]
[1]
[1]
[1]
[1]
[1]
[1]

"attributes for variable"
"units: deg_C"
"_FillValue: -32767"
"long_name: Ambient Temperature, Reference"
"standard_name: air_temperature"
"Category: Atmos. State"
"DataQuality: Preliminary"
"Dependencies: 1 ATHR1"

FI <- DataFileInfo (fnm)
## try str (FI)
## try str (Data)
If you have Ranadu installed, these and other segments of R commands should work as they do in
these examples. Where there are comments that start with ## try, you should type the command
that follows, but the output is usually too large to include here.
These comment elaborate on the code in the preceding code block:
1. Using Ranadu::DataDirectory(): In the case shown above the path to the example
netCDF data file provided with Ranadu is special and path.package() is used to find the file.
Usually you will not load the data from this path. The function DataDirectory() is provided to
help make routines portable among systems where the data files might be loaded in different
locations. That function tries a set of possibilities and sets the returned directory to the first
found. The possibilities are: $DATA_DIR, “/scr/raf_data/”, “/home/data/”, “/home/Data/”,
“/Data/”, and “~/Data/”. You may want to set your environment variable $DATA_DIR appropriately if one of these doesn’t match your system. A typical use of DataDirectory() is
shown in the code block that follows.
2. Using Ranadu::standardVariables(): In the code block shown above, “ALL” is used to
load all the variables in the data file. Usually this will result in data.frames that are very large
and will slow processing, and if this is done for a high-rate file you may exceed available
memory, so it is better to list only the variables that you want. “standardVariables()” provides
a basic set, and an argument to standardVariables like c(“VAR1”, “VAR2”) can be used to
add more variables. Alternately, you can just provide a list of variables that you want, as
illustrated in the code block that follows.
3. Ranadu::getAttributes(): The data.frames produced by getNetCDF() have assigned
attributes that match those in the original netCDF file, and this can be retrieved via getAttributes().5 The result of that call is a printed listing of attributes and creation of a list (here
in Z) that contains the attributes, so it is usually preferable to assign the returned value to
something to avoid the double listing that would be produced otherwise.
5 Subsetting

will often lose some attributes so special steps are needed to preserve them if that is desired.

Memo to: Ranadu documentation
8 November 2018
Page 23

4. Ranadu::TellAbout() and str() provide some details about the characteristics of variables.
5. Ranadu::DataFileInfo(): Returns a list with properties of the netCDF file including the
Project, Flight number, the flight date and times, the data rate, the range of latitudes and
longitudes included in the flight, and a list of the variable names in the file.
A more typical starting block of code would look like the following, listed here but not executed:
require (Ranadu)
Project <- "DEEPWAVE"
Flight <- "rf15"
fname <- sprintf ("%s%s/%s%s.nc", DataDirectory(), Project, Project, Flight)
## or you can provide the name via
## fname <- "/scr/raf_data/DEEPWAVE/DEEPWAVErf15.nc"
##
##
##
##
##

try this option for selecting fname:
WD <- getwd () ## save working directory to return to it
setwd(DataDirectory ()) ## set usual data directory
fname <- file.choose () ## use the R gui file chooser
setwd (WD)}
## return to original directory

VarList <- standardVariables (c("ADIFR", "BDIFR", "CONCU_"))
## try 'standardVariables()' to see the standard list
Data <- getNetCDF (fname, VarList)
## try ?getNetCDF to see additional optional arguments to getNetCDF()
Some additional comments on getNetCDF:
1. The variable name “CONCD_” is not a standard name, but this structure is provided to
avoid the need to know the full names. Variables from probes like the UHSAS or CDP have
suffixes that indicate where the probe was located on the aircraft (e.g., _RWO for right wing
outboard pylon). If a variable name is provided that ends in ’_’, a matching variable with
a location index will be provided. The full name (e.g., “CONCU_RWO”) can be provided
if there is possibility of ambiguity, but usually this avoids the need to lookup where a probe
was installed for a particular experiment.
2. The “Time” variable needs special discussion. In NCAR/EOL/RAF data files, time (except
for some very old projects) is provided as an integer representing seconds after a reference
time, usually 0:0:0 UTC on the day the flight started. The “units” attribute is listed as,
for example, “seconds since 2014-06-06 00:00:00 +0000”. For files at rates higher than
1 Hz, the time remains integer seconds but variables are provided with additional dimensions
that represent the measurements within that second. The Ranadu function getNetCDF()

Memo to: Ranadu documentation
8 November 2018
Page 24

always provides “Time” as the first variable, but this is converted into POSIXct format (see
?POSIXct in R) which is the number of seconds since the start of 1970 (UTC). Fractional
values are used in cases of data rates higher than 1 Hz, giving a “flat” file. This format has
the advantage that plot routines interpret times appropriately and produce appropriate labels
in terms of time. To see the date and time in conventional format, use “as.POSIXlt ()’ with
a POSIXct-format time. This also returns a named list with names like “hour”, “min”, “sec”
that you can use to create different-format representations of the time or date.
3. The optional getNetCDF argument “F” can be used to preserve the flight number, by for
example providing F=15 for flight 15. This results in the addition of a new variable called
“RF” to the data.frame. This is provided for cases where multiple data.frames are combined
in order to facilitate analysis tasks using data that span more than one flight. If data.frames
have exactly the same variables, then can be combined using the ’rbind()’ function:
DataM <- rbind (Data1, Data2)
If flight numbers have been added, the individual flights can be retrieved by subsetting; e.g.,
D1 <- DataM[RF == 1, ]
This may be useful, for example, when finding radome sensitivity coefficients for an entire
project, to check if individual flights provide outlier measurements.
4. In the netCDF files, missing values are usually indicated by the value -32767. getNetCDF
converts such values to NA, the R value for missing values. is.na() will test for such values.
Examining the measurements
Ranadu provides some functions intended for examination of the measurements. Some, “getStartEnd()”,
“TellAbout()” and “getAttributes()”, were discussed above. Others include:
1. Ranadu::getIndex(Data, HHMMSS): For data.frame Data, getIndex() returns the index
corresponding to the time provided by HHMMSS, which should be a time like 201213 (without : separators). Data[i, ] will then contain all the measurements at the time corresponding
to HHMMSS. See the examples below.
2. Ranadu::setRange(Data, Start, End): This function returns a sequence of inclusive
indices covering the period from “Start” to “End” (both provided in HHMMSS format).
Data[setRange(Data, Start, End), ] will then be a subset data.frame containing all the measurements in Data for the specified period.
3. Ranadu::ValueOf(Data, V, HHMMSS): This will return the value of variable V in data.frame
Data at time HHMMSS.

Memo to: Ranadu documentation
8 November 2018
Page 25

4. Ranadu::ValueOfAll(Data, HHMMSS): This is like “ValueOf()” but returns the values
of all the measurements at the specified time.
RAFdata[getIndex (RAFdata, 201213), "ATX"]
## [1] -33.37087
RAFdata[getIndex (RAFdata, 201213), ]
##
##
##
##
##
##
##
##

9854
9854
9854
9854

Time
ATTACK
SSLIP
GGVEW
GGVNS
GGVSPD
2013-10-01 20:12:13 1.698893 -0.1679749 57.76046 242.881 -10.89062
VEW
VNS
ATX
DPXC
THDG
ROLL
PITCH
57.74649 242.8225 -33.37087 -55.01767 3.345159 -0.2978565 -1.002424
PSXC
QCXC
EWX
RTH1
RTH2
ADIFR
BDIFR
320.1704 149.8021 0.03480845 -6.05707 -6.13966 -18.06383 -0.7949235
TASX
GGALT
LATC
LONC
PLWC CONCD
WDC
WSC
236.3266 9000.719 45.28072 -101.1768 8.39129
0 260.7146 44.72694

RAFdata[setRange (RAFdata, 201200, 201210), "TASX"]
##
##

[1] 239.9797 239.5875 239.3123 239.0751 238.8636 238.4475 238.0735
[8] 237.7671 237.5088 237.1420 237.0540

ValueOf (RAFdata, GGALT, 201200)
## [1] 9140.305
Z <- ValueOfAll (RAFdata, 201200)
##
##
##
##
##
##
##
##

9841
9841
9841
9841

Time
ATTACK
SSLIP
GGVEW
GGVNS
GGVSPD
2013-10-01 20:12:00 1.666395 -0.1723428 58.18249 244.6779 -10.72564
VEW
VNS
ATX
DPXC
THDG
ROLL
PITCH
58.17815 244.6206 -34.50745 -54.7943 3.396143 -0.1813508 -0.9280989
PSXC
QCXC
EWX
RTH1
RTH2
ADIFR
BDIFR
313.9382 152.9558 0.03578181 -6.334847 -6.429331 -18.57253 -0.8421018
TASX
GGALT
LATC
LONC
PLWC CONCD
WDC
WSC
239.9797 9140.305 45.25227 -101.1864 8.307867
0 263.08 44.49925

Memo to: Ranadu documentation
8 November 2018
Page 26

5.3

Constructing plots

Ranadu standard plot routines
Plots can always be constructed using the base graphics routines in R rather than Ranadu routines.
Try the following example, which compares a standard plot to those generated by Ranadu::plotWAC()
and Ranadu::ggplotWAC(). The second simply calls plot() with some changes in plot parameters: inward tics, tics duplicated on opposing axes, different time labels, etc. The third uses the
“grammar of graphics” package ggplot2 with a special theme. You can tailor any of these to meet
your needs. The follow-up function Ranadu::lineWAC() will add lines to a previously constructed
plot, but it does not modify the axis limits or add a legend so there are advantages to constructing
plots using single calls to plotWAC() or ggplotWAC().

8500
7500

GGALT

9500

plot(RAFdata[, c("Time", "GGALT")], type='l')

10:00

11:00

12:00

13:00
Time

plotWAC(RAFdata[, c("Time", "GGALT")])

14:00

15:00

8500
7500

GGALT

9500

Memo to: Ranadu documentation
8 November 2018
Page 27

20:10

20:11

20:12

20:13

20:14

20:15

TIME [UTC]

## try plotWAC (subset (RAFdata,, c(Time, GGALT))) ##note absence of " marks
ggplotWAC (RAFdata[, c("Time", "GGALT")])

Memo to: Ranadu documentation
8 November 2018
Page 28

9500
GGALT

GGALT

9000
8500
8000
7500
20:10

20:12
Time [UTC]

20:14

Other manipulations for plots
1. The Ranadu::setRange() function can be used to plot a limited range in time.
2. Legends are added automatically by Ranadu::plotWAC() and Ranadu::ggplotWAC().
See ?legend for information on how to add or modify or suppress legends.
3. Many additional parameters can be used as arguments to these plotting routines, for example
“lwd=2” or “lty=2” to set the line width or type. You can explore the possibilities by starting
with ?par
4. Missing values are handled in special ways in plots, as illustrated in the next code block:
(a) If a measurement is missing in the data.frame, plotted lines will show a gap.
(b) If the index itself is missing, plotted lines will connect across the gap. A subset constructed as is R2 below omits the entire row of values when R1$ATX is NA, so the
plotted blue line just connects the remaining points and shows no indication of the gap.
R1 <- RAFdata
R1$ATX[setRange (R1, 201100,201120)] <- NA
R2 <- RAFdata[!is.na (R1$ATX), ]

Memo to: Ranadu documentation
8 November 2018
Page 29

## note 0.5 offset

−30
−35

ATX

−25

plotWAC (R2[, c("Time", "ATX")])
lineWAC (R1$Time, R1$ATX+0.5, col='red')

20:10

20:11

20:12

20:13

20:14

20:15

TIME [UTC]

Flight tracks
The routine Ranadu::plotTrack() is provided to plot flight tracks on a background that shows
geographic boundaries. In its simplest form it is called with a single argument of a data.frame that
contains at least latitude and longitude in variables named LATC and LONC. This routine also has
an option to plot the track in a reference frame that drifts with the wind. See ?plotTrack for more
information on options.

5.4

Implementing processing algorithms

Many of the algorithms used for calculation of the variables in the netCDF data archives are available as Ranadu functions. These may be useful during data analysis for checking how values might
change with different algorithms or to apply the algorithms to new measurements. The processing algorithms are documented in a Technical Note available at this URL. The Appendix lists the
Ranadu functions and, under references, notes thos that represent descriptions in that technical
note.

Memo to: Ranadu documentation
8 November 2018
Page 30

A few will benefit from additional description, but these are only a select set of the algorithm
routines:
1. Two functions, CorrectPitch() and CorrectHeading(), provide corrections to the measurements of pitch, roll, and heading. They are based on detection of the errors in INS
measurements that result in discrepancies between those measurements and measurements
from GPS. These are the basis for a manuscript that is accessible from this GitHub repository. For example, an error in pitch results in the acceleration of gravity being rotated
into a false component of horizontal acceleration, and this causes a false derivative in the
INS-provided ground speed components. This can be detected by comparing to the GPSprovided measurements, so a correction is possible. Similarly, an error in heading results
in the accelerations measured in the aircraft reference frame being transformed erroneously
into Earth-relative accelerations, so again comparison to GPS can provide an estimate of this
error. These two Ranadu functions provide estimates of those errors and can be used to improve the measurements. These are not part of standard NCAR/EOL/RAF data processing.
2. The function MurphyKoop() provides the water vapor pressure in equilibrium at a specified
temperature. MurphyKoopIce() provides the equilibrium water vapor pressure relative to
a plane ice surface. References are provided in the technical note.
3. Equivalent potential temperature can be calculated via three different formulas:
(a) RossbyEquivalentPotentialTemperature():

The original Rossby formulation

(b) BoltonEquivalentPotentialTemperature(): The Bolton revision, used in many
earlier data archives and still retained as the variable THETAE
(c) EquivalentPotentialTemperature(): The Davies-Jones formula, leading to the
variable THETAP (for pseudo-adiabatic equivalent potential temperature) now in standard archives. This is now the favored calculation. It uses the Murphy-Koop values
for equilibrium water vapor pressure and otherwise represents the favored equation for
analysis, as discussed in the technical note.
4. WindProcessor() replicates the code for calculation of wind that is represented in RAF
Bulletin 23 (except for the omission of a small term representing the motion of the INS
relative to the aircraft during turns). It can be used to calculate wind from new sets of
measurements, as for example is needed for new sensors of the relative wind or for corrected
attitude angles resulting from the first item above.
5. XformLA() transforms a vector as measured in the aircraft coordinate system to one in an
Earth-relative coordinate system, or vice versa.
6. KingProbe() calculates the liquid water content measured by a CSIRO (King) sensor with
drift removed by reference to a cloud-droplet spectrometer like a Cloud Droplet Probe. This
duplicates the algorithm used for data processing. It is useful because it may be preferable
to change the characteristics of the updating and the relationship between Nusselt number
and Reynolds number used in this processing.

Memo to: Ranadu documentation
8 November 2018
Page 31

7. PCorFunction() provides the correction for static defect developed in this reference. That
has led to significant improvement in the measurements of pressure, airspeed, and temperature, as described in that reference.
8. StandardConstant(), Gravity() and SpecificHeats() return the values of various
constants used in processing, as listed in the section on defined constants in the technical
note. For example, some of the constants available are the molecular weights of water and of
air, the universal gas constant, the angular rotation rate of the Earth, and Avogadro’s constant,
among others.

5.5

Some Ranadu utilities

Some of the Ranadu functions that are provided to implement tasks that may be useful are described
here:
1. binStats() is intended to bin measurements and then compute averages and standard deviations in bins, as might be needed for constructing error-bar plots or soundings vs. height
or pressure. (For most uses, equivalent functionality is provided in more standard ways via
the ’fields’ package for R.)
2. SmoothInterp() interpolates to fill regions with missing values and then filters the result
to obtain a smoothed variable.
3. ShiftInTime() moves variables forward or backward in time to support shifting variables that
are sampled at different times.
4. makeNetCDF() makes a netCDF file from a data.frame, provided it has the required characteristics (dimensions, attributes, etc.) If a data.frame is constructed by getNetCDF(), it
is usually possible to construct the corresponding netCDF file that will have the originally
selected subset of variables from the original file.
5. LagrangeInterpolate() implements standard Lagrange interpolation.
6. DemingFit() provides a fit that minimizes the perpendicular distance from points to a line,
as is appropriate when comparing measurements when neither is known to be a standard.

5.6

Miscellaneous analysis tools

Skew-T thermodynamic diagram
Ranadu::SkewTsounding() provides the capability of plotting measurements on a skew-T diagram. The diagram itself has been constructed using modern reference values for the water vapor

0
−4

0
−5

0
−6

0
−7

−8

0

0

0
−9

100

−1
0

−1
1

0

Memo to: Ranadu documentation
8 November 2018
Page 32

−3

0

Measurement
T

−2

0

DP

0

300
400

10

Pressure [hPa]

−1

0

200

20

500
600

30

700

30

20

10

0

0
−1

0
−2

−3

0

800
900
1000

Temperature or dew point [°C]

pressure, adiabatic ascent, and air and water properties and represents a project in itself (documented in this repository. The usual usage is to average measurements of temperature and dewpoint in pressure intervals and then superimpose those measurements on the previously generated
background. It is also possible to add multiple soundings to a single plot. See ?SkewTSounding
for more information. An example using the short flight segment “RAFdata” is shown below:
SkewTSounding (RAFdata[, c('PSXC', 'ATX', 'DPXC')])

LCL ( ):
The LCL function determines the lifted condensation level given measurements of pressure, temperature, and mixing ratio. The function uses iteration to find the point where the lifted parcel will
have a temperature corresponding to the dewpoint. An approximate equation for the LCL is that
published by Bolton:

Memo to: Ranadu documentation
8 November 2018
Page 33

2840
+ 55
3.5 ln θ − ln(1000e/p) − 4.805
The Bolton equation usually provides answers differing from this routine by only a few 0.01◦ C,
but this routine uses moist-air properties, the Murphy-Koop vapor pressure relationship, and convergence to near-machine-precision.
TBolton =

CAPE ( ):
This function performs a set of calculations on a supplied sounding and returns information useful to evaluate the potential for severe weather and other aspects of instability. In addition to
estimating the convective available potential energy, it returns the convective inhibition, level of
free convection, LCL determined by averaging a lowest-level layer, and profiles of temperature
for pseudo-adiabatic ascent or reversible adiabatic ascent from the LCL. These are in data.frame
variables suitable for plotting on the skew-T diagram discussed above. The returned data.frame
also includes a column giving the liquid water content for adiabatic ascent and the virtual temperatures for the environment and for the adiabatic parcels, as are needed for calculating buoyancy. If
there is no region of positive buoyancy, the function returns only the sounding variables (pressure,
temperature, and dew point) and does not add the data.frame attributes discussed below.
Here are some details:
1. The routine begins by some adjustments to the input data, to remove measurements where
any are missing at a particular level and to enforce a ceiling on the dewpoint at the corresponding temperature. The latter is often needed because, on descent, the dewpoint instruments on the aircraft sometimes overshoot and produce erroneous measurements that
otherwise would contaminate the sounding.
2. The lowest ’player’ increment of the sounding (by default, 50 hPa) is then averaged to determine mean values of the lowest-level potential temperature and mixing ratio. These are
used, via the ’LCL ( )’ function, to find the pressure and temperature at the LCL.
3. Next the measurements are binned according to pressure to give a set of average profiles.
If the supplied sounding contains more than one sounding, all are averaged together by this
procedure.
4. A new data.frame is then constructed from these binned values of pressure, temperature, and
dewpoint, and the corresponding profile of virtual temperature is added to the data.frame as
variable TV.
5. Values of the pseudo-adiabatic equivalent potential temperature (using the Davies-Jones
equation, and returned as THP’) and of the wet-equivalent potential temperature THQ are
then calculated at the LCL.
6. Next a set of profiles are added to the data.frame:

Memo to: Ranadu documentation
8 November 2018
Page 34

(a) TP, representing the temperature for pseudoadiabatic ascent from the LCL or moistair descent from the LCL. This profile is calculated at each pressure level above the
LCL by finding the temperature that, at the given pressure in the sounding and for
vapor pressure corresponding to equilibrium at that temperature, would give the same
value of pseudoadiabatic equivalent potential temperature as the value at the LCL, THP.
Below the LCL, the same procedure is followed to find the temperature that would give
the mean value of potential temperature as foundd in step 2 above. Contrary to standard
practice, this is calculated using moist-air, not dry-air, properties like specific heats and
the gas constant.
(b) TQ, the temperature analogous to TP but for reversible ascent above the LCL. Below
the LCL, the values are the same as those for TP.
(c) LWC, the liquid water content (in units of g/m3 ) for ascent from the LCL.
(d) TPV and TQV, the virtual temperatures corresponding to TP and TQ. These are needed
for the calculation of buoyancy. In the case of TQV, the virtual temperature is adjusted
for the liquid water content contained in the adiabatic parcel.
7. An integration is then performed from the first positively buoyant point above the LCL (the
level of free convection or LFC) to the last positively buoyant point (or the top of the sounding) to find the CAPE. The starting and ending points are both interpolated between bounding levels to refine the integration limits, and the integration is performed by constructing, at
each level, variables Rd (Tpv − Tv ) and ln p where Rd is the dry-air gas constant6 (287.0653),
Tpv the virtual temperature for pseudoadiabatic ascent, Tv the virtual temperature of the environment., and p the pressure. A 7-term Lagrangian intorpolation function is used to produce
a smoothR representation of the sounding, and the R routine ’integrate’ is then used to find the
integral Rd (Tpv − Tv ) d ln p between the specified limits. It was found that a 7th-order polynomial produced better convergence than lower-orders. The integration uses the method of
quadrature documented in R. Piessens, E. deDoncker–Kapenga, C. Uberhuber, D. Kahaner
(1983) Quadpack: a Subroutine Package for Automatic Integration; Springer Verlag. It was
tested by Euler-method integration, as documented in the code.
8. A similar integration is also performed using TQV, the virtual temperature for reversible
adiabatic ascent after adjustment for the retained liquid water content.
9. An analogous integration is performed using TPV from the LCL to the LFC, to determine
the convective inhibition.
10. A set of attributes are then associated with the new data.frame to support passing these
values as returned arguments from the function. The addributes are: LCLp (pressure at the
LCL), LCLt (temperature at the LCL), THP, THQ, CAPE, CAPEW (the CAPE for reversible
ascent), CIN (the convective inhibition), MaxLWC (the peak liquid water content for the reversible ascent), and pMaxLWC (the pressure at which this maximum occurs). Note that this
6 not

the moist-air gas constant, as a consequence of the definition of virtual temperature as the temperature corresponding to the density of dry air having the same density as the moist air

Memo to: Ranadu documentation
8 November 2018
Page 35

is often not at the top of the sounding because at some point dilution from expansion reduces
the liquid water content (expressed in mass per volume) more than continuing condensation
increases it.
11. The new data.frame is then returned from the function. It has the original measurements
of pressure, temperature, and dewpoint but now in a set of bins rather than the original
data.frame. The returned data.frame can be used as input to SkewT( ) and the columns
added to the data.frame can be plotted on that diagram to show the ascenting trajectories, as
in the following example:
Data <- getNetCDF ('/Data/CONTRAST/CONTRASTrf01.nc', c('PSXC', 'ATX', 'DPXC'),
Start=250400, End=260100)
suppressWarnings(NSND <- CAPE (Data, nbins=50, player=50))

S <- SkewTSounding (Data, AverageInterval=2)
S2 <- SkewTSounding (data.frame (Pressure=NSND$Pressure,
Temperature=NSND$TP, DewPoint=-120), ADD=TRUE)
S3 <- SkewTSounding (data.frame (Pressure=NSND$Pressure,
Temperature=NSND$TQ, DewPoint=-120), ADD=TRUE)
S <- S + geom_path (data=S2, aes (x=AT, y=P), colour='red', lwd=1.0)
S <- S + geom_path (data=S3, aes (x=AT, y=P), colour='green', lwd=1.0)
plcl <- attr (NSND, 'LCLp')
tlcl <- attr (NSND, 'LCLt')
maxLWC <- attr (NSND, 'MaxLWC')
pmaxLWC <- attr (NSND, 'pMaxLWC')
SP <- SkewTSounding (data.frame(Pressure=plcl, Temperature=tlcl, DewPoint=-120), ADD=TRU
S <- S + geom_point (data=SP, aes(x=AT, y=P), pch=19, colour='darkorange', size=4)
labelText <- paste(sprintf('orange dot: LCL %.1f hPa %.2f degC', plcl, tlcl),
'red line: pseudo-adiabatic ascent',
'bright green line: wet-adiabatic ascent',
sprintf ('max LWC: %.2f g/m3 at %.1f hPa', maxLWC, pmaxLWC),
sprintf ('cape=%.0f J/kg (adiabatic cape=%.0f)',
attr(NSND, 'CAPE'), attr (NSND, 'CAPEW')),
sprintf ('conv. inh. %.0f J/kg, LFC=%.0f hPa',
attr(NSND, 'CIN'), attr(NSND, 'LFC')), sep='\n')
S <- S + geom_label (aes(x=0, y=2.85, label=labelText), size=3,
fill='ivory', hjust='left')
print(S)

0
−4

0
−5

0
−6

0
−7

−8

0

0

0
−9

100

−1
0

−1
1

0

Memo to: Ranadu documentation
8 November 2018
Page 36

−3

0

Measurement
T

−2

0

DP

0

300
400

10

Pressure [hPa]

−1

0

200

20

500
orange dot: LCL 913.8 hPa 16.34 degC

600 red line: pseudo−adiabatic ascent
30

700 bright green line: wet−adiabatic ascent
max LWC: 6.07 g/m3 at 425.8 hPa

30

20

10

●
0

0
−1

0
−2

−3

0

800 cape=1095 J/kg (adiabatic cape=541)
900 conv. inh. 27 J/kg, LFC=810 hPa
1000

Temperature or dew point [°C]

VSpec():
The function VSpec() generates plots of spectral variance. The help info generated by ?Ranadu::VSpec
is fairly extensive, and the function has many arguments, but using the defaults is usually a good
starting point. A Shiny-app tutorial on plots of spectral variance is available here, although that
tutorial is focussed on the underlying functions used by VSpec(). It explains the various methods
of calculating the variance spectrum, shows ways of reducing the uncertainty in the estimates, and
discusses the effect of random error or white noise on the spectrum.
g <- VSpec(Data, 'WIC', smoothBins=50, ylim=c(0.0001,20))
g <- VSpec(Data, 'VYC', smoothBins=50, ADD=g)
VSpec(Data, 'UXC', smoothBins=50, ADD=g) + theme_WAC()
The following plot (Fig. 1 illustrates a typical plot of spectral variance that can be generated by
this function.

variance spectrum fP(f) for WIC

Memo to: Ranadu documentation
8 November 2018
Page 37

UXC

VYC

WIC

100

10−2

10−4
10−3

10−2

10−1
100
frequency [Hz]

101

Figure 1: Variance spectra from DEEPWAVE flight 16, 8:30:00 – 8:40:00 UTC. WIC is the vertical wind, UXC the longitudinal wind along the axis of the aircraft, and VYC the lateral wind
perpendicular to the other two components.

Memo to: Ranadu documentation
8 November 2018
Page 38

2015−07−17 17:30:00 −− 17:35:00

dN/dlog10D [cm−3]

100

100

10−3

10−3

10−6

10−6

10−9
1e−02

1e+00

1e+02

exceedance fraction

CCDP
C1DC
CUHSAS
exceedance

103

1e+04

D [µm]

Figure 2: Size distribution measured during the CSET project, research flight 6, 17:30:00 –
17:35:00 UTC.
plotSD():
Plots of size distributions of aerosol particles or hydrometeors can be generated using this function.
An example is shown in Fig. ??. A memo that discusses these size distributions and some of the
options available is available here. That memo has some information on VSpec() as well, as does
the help information generated by ?Ranadu::plotSD .
D <- getNetCDF('/Data/CSET/CSETrf06.nc',
c('CCDP_', 'C1DC_', 'CUHSAS_'), 173000, 173500)
T <- plotSD(D[,1:4], logAxis='xy', CDF=TRUE,
FirstCell=c(1,3,1), legend.position='topright')

Memo to: Ranadu documentation
8 November 2018
Page 39

contourPlot():
Scatterplots with large numbers of points often are hard to interpret because the density of points
is hard to perceive. The R routine “filled.contour()” provides an alternate way of displaying
the density of points in such a plot, and the Ranadu function “contourPlot()” provides an interface to that function that facilitates generation of a two-dimensional plot of event density. See
?Ranadu::contourPlot for more information.

6

Using R for reproducible data analysis

6.1

Key steps toward reproducibility

There is a growing and beneficial trend toward documentation of research and analysis in ways
that permit others to check, duplicate, and extend the work. In R, the package ’knitr’ has made
a substantial contribution toward that goal by making it possible to blend the text of a document
with the R code that generates the results, as discussed in the next subsection. A minimum set of
actions toward reproducibility might include these steps:
1. Use of data from public repositories where the data files are assigned DOI numbers and
where there is a commitment to long-term preservation and public access.
2. Use of programs, similarly archived and associated with a DOI number where possible.
When these programs generate the final document while containing the R code and accessing
the original data, they will be reproducible by others.
3. A workflow and/or provenance document describing the steps taken in the analysis project.
This goes beyond what is in the code, and it should help those trying to understand the
code by providing context and intent for various steps. For example, how were the data
collected and by whom, how were they processed before being used in this analysis, what
approaches were tried and rejected or abandoned, how does control flow through the program
and through any subprograms, how was the final document reviewed and archived, and who
are the appropriate contacts for the data, analysis, and interpretation? A good workflow
document can be of great aid to someone who wants to repeat the analysis, because snippets
of code can be difficult to understand and the reasons for the steps in a program are often
obscure.
As an example of one attempt, see the work-in-progress that is contained in this GitHub repository:
https://github.com/WilliamCooper/AMTD-AAC.
These principles can also be applied to smaller analysis projects that might need to be repeated.
For an example of the latter, the following will describe one use of R and Ranadu to determine
appropriate empirical coefficients relating the pressure difference measured between radome ports

Memo to: Ranadu documentation
8 November 2018
Page 40

to the flow angle perpendicular to those ports. This is a common task needed for the measurement
of vertical or horizontal wind, and the empirical coefficients may change, for example when the
radome is replaced or even re-installed after removal or when sensors are placed in locations that
might affect the airflow around the aircraft. The document is this memo giving the recommended
empirical coefficients, and it was generated by this program file which combines the R code and
LATEX commands that produced the memo. At the end of the memo there is an appended section
called “Reproducibility” that contains these entries:
Reproducibility:
Project:
Archive package:
Contains:
Program:
This document:
Workflow::
Original Data:
Data archive:
Git:
Attachments:

WI-HIPPO1
WI-HIPPO1.zip
attachment list below
AKRDforHIPPO1.Rnw
AKRDforHIPPO1.pdf
WorkflowFindAKRDcal.pdf
/scr/raf_data/HIPPO/HIPPO-1rf01.nc etc
NCAR HPSS (not github)
https://github.com/WilliamCooper/Reprocessing.git

AKRDforHIPPO1.Rnw
AKRDforCSET.pdf
WorkflowFindAKRDcal.pdf
SessionInfo

The content of some of these items is as follows:
• Archive package: The content of this package is given by the attachment list, and it is located
in the repository listed as the last item. The actual address is this URL.
• Program: The listed program produces the document and contains the R code. It uses ’knitr’
as discussed in the next subsection.
• Workflow: In this case the workflow description is a general description of the task, with
some discussion of the application to the HIPPO-1 project.
• Original data and Data archive: In this case the original data are in temporary files on EOL
computer systems. The reprocessing task to which this document contributes was underway
at the time this was written, and those files may change or change location. Therefore a
separate archive was constructed that consists of the data.frames used in this analysis. Those
are saved on the NCAR HPSS. (They tend to be larger than seems appropriate to place on
github.) At this time, these datasets have not been assigned DOI numbers, but the underlying
data archive eventually will meet this requirement.
• Attachments: A “SessionInfo” section is included. This is the result of the R command ’sessionInfo()’ and includes information like the R version, platform, operating system, packages
used, etc.

Memo to: Ranadu documentation
8 November 2018
Page 41

6.2

Using knitr

As for R itself, this manual will only give very sketchy information regarding the use of knitr,
but this may help readers of the ’Rnw’ files. This manual itself is constructed from the file
’RanaduManual.Rnw’ in which the R examples and the descriptive code are merged. The file
is basically a LATEX file into which segments of R code called ’chunks’ have been inserted. There
is also an alternate format, ’Rmd’, in which the text is html (markdown) instead of LATEX. In that
format, the chunks have a different but similar format. In the Rnw format, chunks are set apart
from other LATEX prose by headers and trailers that have this format:
<>=
[R code]
@
Results contained in R variables can be referenced in the LATEX part of the document using the
’\Sexpr{}’ command, and plots can be referenced by a label corresponding to the chunk-name and
will be placed in the document at an appropriate location. In the case of this manual, the first chunk
has this content:
<‌‌>=
library(knitr)
opts_chunk$set(echo=FALSE, include=FALSE, fig.lp="fig:")
opts_chunk$set(fig.width=6, fig.height=5, fig.align="center",
digits=4)
thisFileName <- "RanaduManual"
require(Ranadu, quietly = TRUE, warn.conflicts=FALSE)
@
This loads the knitr library, sets some options, and loads the Ranadu package. Here is a small
example of how R calculations and text can be merged:
<‌‌>=
meanTemperature <- mean (RAFdata$ATX)
sdTemperature <- sd (RAFdata$ATX)
@
The mean and standard deviation of temperature is -31.33±4.98, placed here using the command
\Sexpr{meanTemperature} where the first number appears and \Sexpr{sdTemperature} for the second.
For more information on using knitr, see the resources at the end of this manual.

Memo to: Ranadu documentation
8 November 2018
Page 42

7

An example: Data quality review

7.1

Standard ’Review.R’ approach

During field projects, monitoring data quality is essential so that problems can be found and corrected to minimize data loss. One approach to this has been to set up an R script that constructs
plots of key measurements that can be reviewed quickly after flights. These plots are constructed
from the preliminary netCDF files produced in the field, and in recent projects they have been
stored in the field catalog for projects where they are available to all personnel in the project as
well as for documentation after the project.
A flowchart for the Review() program, constructed using the ’DiagrammeR’ package for R, is
shown below:

1. Project preparation: This only needs to be done once at the start of a project:

Memo to: Ranadu documentation
8 November 2018
Page 43

(a) Data files by convention are in a directory with the project name and have names beginning with the project name, so “Project” should be set appropriately.
(b) There is a ’Configuration.R’ file that is read by Review(), and it has sections for each
project. A new section should be added that provides the variables used by the various
plots. A template from recent projects can be a good starting point. If runs terminate
with messages that variables are not present in the data files, the project configuration
should be modified to omit those variables. There is a list of what is plotted by each
plot function in the program, so that can help determine what variables are appropriate
for the different entries in ’Configuration.R’.
(c) The plot functions used by Review() are located in the directory ’PlotFunctions” and
have names like RPlot15.R. There is a model in that directory, named RPlotModel.R,
that can be used as a starting point for adding new plots, and the existing plots also can
be used as starting points for new plots. Usually no new plots will be needed, and it
may be best to get started with the available plots and add others as needed.
(d) The default list of plots for the project also should be defined. It may be useful to skip
some plots, e.g., to skip the display of CDP size distributions in cases where no CDP is
used in the project.
2. Initialization: This may involve interactive entry of information or entry via the ’Rscript’
method of running Review(). See the comments near the start of the program for information
on optional methods for running and providing the needed information.
(a) Each run processes an individual flight, so the flight number must be provided.
(b) An individual-run set of plots can be provided, or the default set can be accepted.
(c) The program will read the appropriate section of ’Configuration.R’ for the project and
construct a list of the variables needed.
(d) The data file is then read and the needed variables are returned in a data.frame.
3. pdf-format and html-format plots: The program produces plots in two formats, a PDF format
that can be magnified without loss of resolution and an html format that is much smaller and
faster to read and transfer. For each, it loops through the plot routines, loading them from
’PlotFunctions’ if they are needed then running them with appropriate variables. The result
will be two files named PROJECTrfxxPlots.pdf and PROJECTrfxxPlots.html where
PROJECT is replaced by, e.g., DEEPWAVE, the name of the project.

7.2

’Shiny’ approach

A ’shiny’ app that provides more interactive control is also available. It provides the same plots
and can construct a set of PDF-format plots similar to those from ’Review.R’,
but also supports more costomization and interactive displays.
See the repository at https://github.com/WilliamCooper/DataReview.git.

Memo to: Ranadu documentation
8 November 2018
Page 44

8

Additional resources

These are some resources that have proven useful in constructing and using the Ranadu package. Almost all of the analyses reported here were performed using R (R Core Team [2013]),
with RStudio (RStudio [2009]) and knitr (Xie [2013, 2014]). An example of a good general introductory book on R is Lander [2013]. The "ggplot2" package (Wickham [2009]) was used for
some figures. The effort to make these results reproducible benefited greatly from the work represented in these analysis tools, especially those provided by Y. Xie. The book by C. Gandrud
(Gandrud [2014]) and material and presentations related to the “Geoscience Paper of the Future”
(http://www.ontosoft.org/gpf/) also had a strong influence on the approach to this work. For those
wanting more advanced information about R, consider Wickham [2014].

References
C. Gandrud. Reproducible Research with R and RStudio. CRC Press, Boca Raton, FL, USA, 2014.
ISBN 13:978-1-4665-7284-3.
Jared P. Lander. R for Everyone: Advanced Analytics and Graphics. Addison-Wesley Professional,
1st edition, 2013. ISBN 0321888030, 9780321888037.
R Core Team. R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria, 2013. URL http://www.R-project.org.
RStudio. RStudio: Integrated development environment for R (Version 0.98.879), 2009. URL
http://www.rstudio.org.
H. Wickham. ggplot2: elegant graphics for data analysis. Springer New York, 2009. ISBN
978-0-387-98140-6. URL http://had.co.nz/ggplot2/book.
H. Wickham. Advanced R. Chapman & Hall/CRC The R Series. Taylor & Francis, 2014. ISBN
9781466586963. URL https://books.google.com/books?id=PFHFNAEACAAJ.
Y. Xie. Dynamic Documents with R and knitr. Chapman and Hall/CRC, Boca Raton, Florida,
2013. URL http://yihui.name/knitr/. ISBN 978-1482203530.
Y. Xie. knitr: A general-purpose package for dynamic report generation in R, 2014. URL http:
//yihui.name/knitr/. R package version 1.6.

– End of Memo –

Memo to: Ranadu documentation
8 November 2018
Page 45

Appendix
name
AdiabaticTandLWC
AirTemperature

purpose
For adiabatic ascent, calculates the temperature and
liquid water content at specified levels
Implements the algorithm for calculating ambient air
temperature from measurements, with inclusion of the
correction for dynamic heating
AOAfromRadome
Calculates angle of attack from the differential
pressure measurements from the radome, applying the
standard empirical coefficients.
binStats
A convenience function for binning measurements of
one variable in bins of another, then calculating means
and standard deviations for the bins. Useful for
constructing error-bar plots.
BoltonEquivalentCalculates equivalent potential temperature using the
PotentialTemperature Bolton equation
calcAttack
A function that returns the angle of attack that would
be present in the absence of vertical wind. Useful for
determining sensitivity coefficients for the angle of
attack
CAPE
Find convective available potential energy and
adiabatic trajectories, given a sounding
ComplementaryFilter A duplication of the filter used in original processing
that combined low-pass measurements from GPS and
high-pass measurements from an inertial system to
provide a representation of wind that has the
long-term stability of GPS combined with the fast
response of an inertial system
contourPlot
A replacement for a scattergram that shows the
density of points, possibly in logarithmic intervals, as
colored regions.
CorrectHeading
Estimate the error in measured heading based on
comparison of accelerations in the body frame and
Earth reference frame, the latter measured by GPS.
CorrectPitch
Estimates the error in measured pitch based on the
Schuler oscillation as represented by the difference
between ground speeds measured by GPS and by the
inertial system providing the measurement of pitch.

reference7

[a] Section 4.4

[a] Sect. 4.6
[b] Sect.

[a] Sect. 3.4;
[b] Sect.

[c] Sect.

cf. this
presentation,;
[b] Sect.;
[c] Sect.

Memo to: Ranadu documentation
8 November 2018
Page 46

name
DPfromE

purpose
Given the water vapor pressure, calculates the dew
point. Uses the Murphy-Koop representation of
equilibrium water vapor pressure as a function of
temperature.
DataDirectory
A convenience function providing the directory in
which data reside on different systems, so that code
can be written that is portable among systems. (May
need modification for new systems.)
DataFileInfo
Describes the contents (data/time, variables, project,
etc) of a netCDF data file
DemingFit
Implements a fit that minimizes the distance between
points and a fitted line, treating pairs of measurements
equally in the fit (in contrast to standard regression,
treating one as a function of the other)
EquivalentAn implementation of the Davies-Jones (2009)
PotentialTemperature formula for pseudo-adiabatic equivalent potential
temperature, the THETAP variable in RAF netCDF
files
getAttributes
List the attributes of a data.frame or a column in a
data.frame. These duplicate the attributes present in
the original netCDF file.
getIndex
Determine the index (row number) in the data.frame
that corresponds to a specific time
getNetCDF
The starting point for all these functions. Provides a
data.frame consisting of specified variables and times
from the original netCDF file.
getRAFData
A specialized routine for transfer of data using
NCAR/EOL storage and ftp; not useful except for
those with an NCAR/EOL computer account.
getStartEnd
Provides the start and end times for a data.frame
ggplotWAC
An incomplete early try at using ggplot for plots in
ways similar to those provided for plotWAC below.
CAVEAT: work in progress.
GV_AOACalculates the angle of attack on the GV aircraft given
fromRadome
the pressure measurements on the radome. Same as
AOAfromRadome
GV_YawCalculates the sideslip angle on the GV aircraft given
FromRadome
the pressure measurements on the radome
Gravity
A representation of the acceleration of gravity as a
function of latitude and altitude

reference8
[a] Sect. 4.5

special ref.

[a] Sect.4.6

[special use]

[special use]

[a] Sect. 4.7.1

[a] Sect. 4.7.1
[a] Section 1.3

Memo to: Ranadu documentation
8 November 2018
Page 47

name
KingProbe

LCL
LagrangeInterpolate
lineWAC
MachNumber

makeNetCDF

MixingRatio
MurphyKoop

MurphyKoopIce
ncsubset
OpenInProgram
PCorFunction

plotSD()

plotTrack

plotWAC

purpose
Calculates the liquid water content sensed by a CSIRO
(King) sensor, with correction for the power required
in clear air
Given pressure, temperature, and mixing ratio, finds
the lifted condensation level
Standard implementation of Lagrange interpolation
Following plotWAC(), lineWAC() can be used to add
additional lines to the plot.
From pressure, dynamic pressure, and humidity,
calculate the Mach number representing the flight
speed
From a data.frame generated by getNetCDF, construct
a netCDF file containing the measurements in that
data.frame
Calculation of water vapor mixing ratio as
implemented in the original processing
A function that provides the equilibrium water vapor
pressure as a function of temperature, following the
formulation of Murphy and Koop (2005), with
optional "enhancement factor" arising from the
ambient pressure
Similar to MurphyKoop but for equilibrium over ice
Simple subsetting routine for netCDF files. Uses ncks,
and ncks could be used directly to do the same thing.
Start ’ncplot’ with data from an R data.frame
The pressure-correction functions applied to NCAR
aircraft to account for the static defect in measured
pressure and dynamic pressure
Plot size distributions from hydrometeor instruments
like the CDP or aerosol-particle sensors like the
UHSAS.
Plot a flight track on a simple representation of
geographic boundaries, with an option to plot the
track in a coordinate frame drifting with the wind.
(Better looking flight tracks can be constructed using
the .kml files provided by EOL/RAF.)
A convenience function that calls base-R plot() with
some tailoring of the parameters of the call. See also
lineWAC.

reference8
[a] Sect. 5.1

[a] Sect. 4.7.1

[a] Sect. 4.5
[a] Sect. 4.5

[a] Sect. 4.5
[special use]

[a] Sect. 4.3

Memo to: Ranadu documentation
8 November 2018
Page 48

name
purpose
PotentialTemperature Standard calculation of potential temperature, with an
option to apply a correction for moist air properties
PressureAltitude
Standard formula for pressure altitude, International
Standard Atmosphere
RecoveryFactor
The formulas used to correct temperature sensors for
the effect of dynamic heating.
Rsubset
Create subsets of data.frames while preserving
attributes of the data.frame and its columns
RossbyEquivalent- Implementation of the original Rossby formula for
PotentialTemperature equivalent potential temperature. See also
BoltonEquivalentPotentialTemperature and
EquivalentPotentialTemperature for other
representations.
setRange
Generate a sequence of indices that spans a specified
time range, for constructing plots that cover specified
periods
setVariableList
Displays a GUI with variables from a netCDF file,
displaying those selected for inclusion in a data.frame
and allowing interactive addition and subtraction of
variables
ShiftInTime
Shifts a variable forward or backward in time to
improve timing matches among measurements
SkewTSounding
Plot measurements of temperature and humidity on a
skew-T background. The background was generated
by code outside this package, and plots are
superimposed on that background. It was generated
with modern representation of equilibrium water
vapor pressure (Murphy and Koop, 2005) and using
full entropy-preserving trajectories with specific heats
and gas constant dependent on humidity, and was
checked by comparison to the formula of
Davies-Jones (2009), so the plot may differ in minor
ways from standard plots. A separate github
repository, WilliamCooper/SkewT.git, contains the
code that generated the background.
SmoothInterp
Interpolate to fill missing values and smooth via a
Savitzky-Golay filter
SpecificHeats
A representation of the specific heats and gas constant
for moist air
StandardConstant
Returns a value or set of values as used in processing
data from research aircraft

reference8
[a] Sect. 4.6
[a] Sect. 3.3
[a] Sect. 4.4

[a] Sect. 4.6

[special use]

[a] Sect. 4.4,
pp. 37–38
[a] Sect. 1.3

Memo to: Ranadu documentation
8 November 2018
Page 49

name
standardVariables

purpose
A standard set of variables that can be included in a
call to getNetCDF, with optional addition of more
variables
TellAbout
Simple functions to describe R objects and attributes
of measurements.
theme_WAC
A ggplot2 theme used by WAC; others may prefer the
standard theme for most uses.
TrueAirspeed
The standard equation for true airspeed in terms of
Mach number and temperature
ValueOf
Quick look-up of values in the data.frame
ValueOfAll
Like ValueOf but for all measurements in the
data.frame at a specified time
VirtualTemperature Standard calculation of virtual temperature
VirtualStandard calculation of virtual potential temperature,
PotentialTemperature with optional adjustment of the specific heats and gas
constant for moist air.
VSpec
Plot variance spectra for measurements.
WetEquivalentThe wet-equivalent potential temperature as defined
PotentialTemperature by Paluch (1978); see also Emanual (1994).
WindProcessor
An implementation of the calculation of wind from
basic measurements, incorporating transformations
among reference frames and combination of the
relative wind and the ground track to obtain wind with
reference to the Earth coordinate system
XformLA
Transform a vector from the aircraft reference frame
to the local Earth-relative (ENU) frame, or v.v.

Reproducibility:
Project:
Archive package:
Contains:
Program:
Original Data:
Workflow:
Git:
Attachments:

RanaduManual
RanaduManual.zip
attachment list below
RanaduManual.Rnw
/scr/raf_data/
WorkflowRanaduManual.pdf
https://github.com/WilliamCooper/Ranadu.git

RanaduManual.Rnw
RanaduManual.pdf
SessionInfo

reference8

[special use]
[a] Sect. 4.7

[a] Sect. 4.6
[a] Sect. 4.6

[a] Sect. 4.6
RAF Bulletin
23



File Type                       : PDF
File Type Extension             : pdf
MIME Type                       : application/pdf
PDF Version                     : 1.5
Linearized                      : No
Page Count                      : 49
Page Mode                       : UseOutlines
Author                          : 
Title                           : 
Subject                         : 
Creator                         : LaTeX with hyperref package
Producer                        : pdfTeX-1.40.18
Create Date                     : 2018:11:08 17:12:05-07:00
Modify Date                     : 2018:11:08 17:12:05-07:00
Trapped                         : False
PTEX Fullbanner                 : This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017) kpathsea version 6.2.3