SPECCHIO User Guide
SPECCHIO_UserGuide
User Manual:
Open the PDF directly: View PDF 
.
Page Count: 161 [warning: Documents this large are best viewed by clicking the View PDF Link!]
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 1 of 161
User Guide
Version: 3.3.0.0
Date: 20/02/2018
Authors: L. Chisholm (University of Wollongong), P. Roberts (Intersect), A. Hueni,
D. Kuekenbrink, S. Bertschi (Remote Sensing Laboratories, University of
Zurich)
File: SPECCHIO_UserGuide.pdf
Pages: 161
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 2 of 161
Table of Contents
Table of Contents ...................................................................................... 2!
1!Introduction ........................................................................................ 7!
1.1!Document scope ............................................................................................... 7!
1.2!Intended audience ............................................................................................ 7!
1.3!SPECCHIO ownership and access ....................................................................... 8!
1.4!Copyright and licensing ...................................................................................... 8!
1.5!For Further Information ..................................................................................... 8!
2!Overview of the SPECCHIO Application .............................................. 9!
2.1!SPECCHIO Basic Operation ................................................................................ 9!
2.1.1!macOS and Windows Operation ................................................................... 9!
2.1.2!Unix Operation ........................................................................................... 9!
2.2!SPECCHIO User Interface ................................................................................. 10!
2.3!Main Functions and Processes .......................................................................... 10!
2.3.1!Installing and getting access to the database .............................................. 11!
2.3.2!Data Loading Workflow ............................................................................. 12!
3!Installing SPECCHIO ......................................................................... 13!
3.1!Before you install ............................................................................................ 13!
3.2!Installing Java Runtime Environment ................................................................ 13!
3.3!Installing SPECCHIO ........................................................................................ 13!
3.3.1!Installing SPECCHIO for the First Time ....................................................... 13!
3.3.2!Upgrading an Existing Installation .............................................................. 14!
3.4!Launching SPECCHIO ...................................................................................... 14!
3.4.1!Windows .................................................................................................. 14!
3.4.2!Mac OS X ................................................................................................. 14!
3.4.3!Unix and Linux ......................................................................................... 15!
4!Get Access to the SPECCHIO Database ............................................. 19!
4.1!Create a new user account ............................................................................... 19!
4.2!Administrator Access ....................................................................................... 19!
4.3!Connect to Database ....................................................................................... 20!
4.4!Logging Out .................................................................................................... 21!
4.5!Edit user information ....................................................................................... 22!
4.6!Edit db_config file ........................................................................................... 22!
4.7!Preferences ..................................................................................................... 23!
5!Data structure and concepts ............................................................. 24!
5.1!Campaign Hierarchy Structure .......................................................................... 24!
5.2!Spaces and the Space Factory .......................................................................... 26!
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 3 of 161
6!Operational Dataflow ........................................................................ 28!
7!File Formats ....................................................................................... 29!
7.1!Supported Input Spectrum File Formats ............................................................ 29!
7.1.1!ASD Binary Files ....................................................................................... 29!
7.1.2!ASD Indico Version 7 Files ......................................................................... 29!
7.1.3!GER Signature Files .................................................................................. 30!
7.1.4!MFR OUT Files .......................................................................................... 30!
7.1.5!Microtops Sun photometer Files ................................................................. 31!
7.1.6!Bruker FTIR Files ...................................................................................... 32!
7.1.7!SVC HR-1024 Files .................................................................................... 32!
7.1.8!Apogee Files ............................................................................................. 34!
7.1.9!ENVI Spectral Library Files ........................................................................ 35!
7.1.10!Ocean Optics SpectraSuite Data Files ......................................................... 35!
7.1.11!Ocean Optics Ocean View Data Files .......................................................... 36!
7.1.12!HDF5 Files containing FGI goniometer measurements ................................. 36!
7.1.13!UniSpec Single Channel ............................................................................. 37!
7.1.14!UniSpec Double Channel SPU .................................................................... 37!
7.1.15!SPECPR .................................................................................................... 37!
7.1.16!Modtran Albedo File .................................................................................. 38!
7.1.17!Excel files ................................................................................................. 38!
7.1.18!TXT Space Formatted Text Files ................................................................. 39!
7.2!Supported Output Spectrum File Formats .......................................................... 40!
8!Metadata ........................................................................................... 41!
8.1!Research Groups and Accessing SPECCHIO Campaigns ...................................... 41!
8.2!Non-Redundant Storage .................................................................................. 41!
8.3!Metadata Storage Levels .................................................................................. 42!
8.4!Time Data ....................................................................................................... 42!
8.5!Data Links ...................................................................................................... 42!
8.6!Manufacturers, Sensors, Instruments and Calibrations ....................................... 44!
8.7!Campaign-related Metadata ............................................................................. 45!
8.8!Metadata Attributes ......................................................................................... 46!
8.8.1!Associated Campaigns Group ..................................................................... 46!
8.8.2!Campaign Details Group ............................................................................ 46!
8.8.3!Data Link Group ....................................................................................... 47!
8.8.4!Data Portal Group ..................................................................................... 47!
8.8.5!Environmental Conditions Group ................................................................ 48!
8.8.6!General Group .......................................................................................... 49!
8.8.7!Generic Target Properties Group ................................................................ 50!
8.8.8!Geochemistry ........................................................................................... 52!
8.8.9!Illumination Group .................................................................................... 52!
8.8.10!Instrument Group ..................................................................................... 52!
8.8.11!Instrument Settings Group ........................................................................ 53!
8.8.12!Instrumentation Group .............................................................................. 54!
8.8.13!Keywords Group ....................................................................................... 55!
8.8.14!Location Group ......................................................................................... 55!
8.8.15!Names Group ........................................................................................... 56!
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 4 of 161
8.8.16!Optics Group ............................................................................................ 57!
8.8.17!PDFs Group .............................................................................................. 57!
8.8.18!Personnel Group ....................................................................................... 57!
8.8.19!Pictures Group .......................................................................................... 57!
8.8.20!Processing Group ...................................................................................... 58!
8.8.21!Sampling .................................................................................................. 60!
8.8.22!Sampling Design Group ............................................................................. 62!
8.8.23!Scientific References Group ....................................................................... 62!
8.8.24!Soil Parameters Group .............................................................................. 63!
8.8.25!Vegetation Biophysical Variables ................................................................ 64!
9!Data Input ......................................................................................... 66!
9.1!Overview of SPECCHIO Data Loading ................................................................ 66!
9.2!Creating a new Campaign ................................................................................ 66!
9.3!Loading Campaign Spectral Data ...................................................................... 67!
9.3.1!Instrument and Sensor selection ................................................................ 69!
9.3.2!Reference Panel selection .......................................................................... 69!
9.3.3!Handling Spectrum upload errors ............................................................... 69!
9.4!Loading Additional Spectral Data ...................................................................... 69!
9.4.1!Uploading Additional Spectral Data from the Same Computer ....................... 70!
9.4.2!Uploading Additional Spectral Data from a Second Computer ....................... 70!
9.4.3!Uploading Additional Spectral Data in a New Data Hierarchy ........................ 71!
9.5!Edit Metadata ................................................................................................. 73!
9.5.1!Metadata Editor window ............................................................................ 73!
9.5.2!Drag & Drop Options ................................................................................ 75!
9.5.3!Displaying and Editing Campaign Metadata ................................................. 75!
9.5.4!Displaying and Editing Spectrum Metadata ................................................. 75!
9.6!Using the Metadata Editor functions ................................................................. 80!
9.6.1!Compute measurement support ................................................................. 80!
9.6.2!Augment Altitude ...................................................................................... 81!
9.6.3!Switch Longitude E-W ............................................................................... 81!
9.6.4!Visualisations ............................................................................................ 81!
9.7!Uploading Metadata from XLS (Excel) files ........................................................ 81!
9.8!Import campaign from XML file ........................................................................ 88!
10!Data Selection & Output ................................................................. 90!
10.1!The Spectrum Browser ................................................................................. 90!
10.1.1!Visualisations ............................................................................................ 92!
10.2!Query Builder .............................................................................................. 93!
10.2.1!SQL Matching Strings ................................................................................ 95!
10.2.2!Entering Dates and Times ......................................................................... 96!
10.3!Show Report ................................................................................................ 96!
10.4!Spectral plot ................................................................................................ 99!
10.5!Process ....................................................................................................... 99!
10.6!Refl. calc ................................................................................................... 100!
10.7!File Export ................................................................................................. 100!
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 5 of 161
10.7.1!CSV Spectrum Export Format ................................................................... 102!
10.8!Export campaign ........................................................................................ 103!
10.9!Publish Collection ....................................................................................... 104!
11!Data Processing ............................................................................ 105!
11.1!Functions to generate additional metadata ................................................... 105!
11.1.1!Compute UTC ......................................................................................... 105!
11.1.2!Calculation of Sun Angles ........................................................................ 106!
11.1.3!Calculation of Goniometer Angles ............................................................. 107!
11.2!Managing Target-Reference Links ............................................................... 109!
11.2.1!Viewing or deleting existing Target-Reference links ................................... 109!
11.2.2!Deleting Existing Target-Reference Links .................................................. 111!
11.2.3!Adding new Target-Reference links .......................................................... 112!
11.2.4!Creating Data Links in the Metadata Editor ............................................... 114!
12!Data Maintenance ......................................................................... 116!
12.1!Remove data ............................................................................................. 116!
12.2!Load sensor definition (Obsolete) ................................................................ 117!
12.3!Instrument Administration .......................................................................... 119!
12.3.1!Instrument Calibrations ........................................................................... 122!
12.4!Reference Panel Administration ................................................................... 123!
12.4.1!Reference Panel Calibrations ................................................................... 125!
13!Help Functions .............................................................................. 127!
13.1!List available Metadata Elements ................................................................. 127!
13.2!Get SPECCHIO user contacts ....................................................................... 127!
13.3!About ........................................................................................................ 127!
14!Publishing Data to ANDS .............................................................. 128!
15!Interactive Processing using Space Networks ............................. 133!
15.1!Graphical Representations of Spaces and Modules ........................................ 134!
15.2!Adding Modules and linking with Spaces ...................................................... 135!
15.3!Configuration of Modules ............................................................................ 136!
15.4!Processing Module Descriptions ................................................................... 136!
15.4.1!Radiance to Reflectance Transformation ................................................... 136!
15.4.2!Reference Panel Correction Factors .......................................................... 136!
15.4.3!Correct for Reference Panel Non-Idealness ............................................... 137!
15.4.4!Delta ..................................................................................................... 138!
15.4.5!Waveband Filter ..................................................................................... 139!
15.4.6!Broadband and Narrowband Filters .......................................................... 140!
15.5!Visualisation Modules ................................................................................. 141!
15.5.1!Spectral Line Plot .................................................................................... 142!
15.5.2!Spectral Scatter Plot ............................................................................... 142!
15.5.3!Gonio Sampling Points Plot ...................................................................... 143!
15.5.4!Gonio Hemisphere Explorer ..................................................................... 143!
15.5.5!Time Line Plot ........................................................................................ 145!
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 6 of 161
15.5.6!Time Line Explorer .................................................................................. 146!
15.6!File Export Module ..................................................................................... 147!
16!Matlab Integration ....................................................................... 148!
17!R Integration ................................................................................ 150!
18!Python Integration ....................................................................... 151!
19!Glossary ........................................................................................ 152!
20!References .................................................................................... 155!
21!Document History ......................................................................... 156!
Appendix A:!Regular Expressions Tutorial .......................................... 158!
Appendix B:!Predefined Manufacturer Table ...................................... 160!
Appendix C:!Predefined Sensor Table ................................................. 161!
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 7 of 161
1 Introduction
SPECCHIO is a database for storing spectral data acquired by spectroradiometers and
their associated metadata, combined with a user-friendly interface.
SPECCHIO was first developed at the Remote Sensing Labs at the Geography
Department, University of Zurich to support long term usability and data sharing
between researchers.
SPECCHIO was then further enhanced through a project run by the University of
Wollongong in 2012/2013. The alias of “AUS-SPECCHIO” represents this instance.
This project was supported by the Australian National Data Service (ANDS). ANDS is
supported by the Australian Government through the National Collaborative Research
Infrastructure Strategy Program and the Education Investment Fund (EIF) Super
Science Initiative.
“Specchio” is an Italian word meaning “mirror” or “looking glass”. It can also be used
to refer to a table of data or a scoreboard.
1.1 Document scope
SPECCHIO uses a Client-Server architecture. This User Guide details operation of the
Client component of SPECCHIO.
This document focuses on the Windows operation of SPECCHIO for most of its
descriptions, screen shots and examples. The operation of the SPECCHIO on Mac
and Linux is equivalent and can be inferred from the Windows descriptions.
1.2 Intended audience
This document is intended for users who are:
• installing SPECCHIO on a desktop
• using SPECCHIO to input or browse data
• administrating the data within SPECCHIO
It also assumes that readers are familiar with:
• remote sensing and the disciplines and processes related to it
• the general operation of their own computer
• the general concept of a client-server architecture
•
• The Administration Guide is a separate document and is intended for users
performing the following tasks:
• installing the Server side of SPECCHIO
• understanding the architecture
• understanding system requirements
• release notes
•
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 8 of 161
1.3 SPECCHIO ownership and access
SPECCHIO was originally built by the Remote Sensing Laboratories at the University
of Zurich, and extended by Intersect for the School of Earth and Environmental
Sciences at the University of Wollongong.
1.4 Copyright and licensing
This program is free software: you can redistribute it and/or modify it under the
terms of the GNU General Public License as published by the Free Software
Foundation, either version 3 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY
WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this
program. If not, see <http://www.gnu.org/licenses/>.
1.5 For Further Information
Please refer to the following documents for more information about SPECCHIO.
Unless otherwise stated, they can be found in the SPECCHIO Installation kit.
SPECCHIO_ReleaseNotes.pdf can be found in each Installation Kit and provides
installation instructions for the SPECCHIO Client.
SPECCHIO_Tutorial.pdf provides instruction in the operation of key areas of the
SPECCHIO Client.
SPECCHIO_ServerInstallation.pdf provides system administrators with
information to assist in managing and maintaining a SPECCHIO
Server System.
SPECCHIO_VM.pdf provides information about the SPECCHIO Virtual Machine,
basing on a VirtualBox CentOS virtual machine. The SPECCHIO VM
is an easy solution to running a complete SPECCHIO server without
the complex installation procedure of a complete native server
installation.
SPECCHIO Web Site (www.specchio.ch) General information about SPECCHIO.
Some of this information may be related to other non-UOW
versions of SPECCHIO.
SPECCHIO UoW (https://specchio.uow.edu.au) Installation kits for University of
Wollongong version of the SPECCHIO Client and documentation for
that version.
SPECCHIO GitHub (https://github.com/ahueni/SPECCHIO) Source code of
SPECCHIO.
See also
Section 16 References
for a list of academic articles related to SPECCHIO
and its use.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 9 of 161
2 Overview of the SPECCHIO Application
SPECCHIO is a database for storing spectral data acquired by spectroradiometers and
their associated metadata, combined with a user-friendly interface.
SPECCHIO organises the Spectra for each new sampling experiment into Campaigns.
Users can view the Spectra for each Campaign by using the browser or by building a
query.
SPECCHIO provides an API to access all system functionality from other
programming languages via a Java bridge.
2.1 SPECCHIO Basic Operation
SPECCHIO has been setup to comply as much as possible with the look and feel of
the operating system on which it is running.
The screenshots provided in the following sections have been taken on a macOS
system. Depending on your operating system, the windows and widgets may look a
little different.
2.1.1 macOS and Windows Operation
SPECCHIO uses the following mouse click combinations on Windows. This document
describes SPECCHIO operation in terms of these Windows mouse click patterns.
Left Click The normal selection or activation click.
Right Click Context menu display.
Control+Left Click Adding a single item to the selection. Control+Click adds
the item clicked to any already selected items.
Shift+Left Click Adding a range of items to the selection. Shift+Click adds
the item clicked, plus any items between the last selected
item and the currently selected item, to any already
selected items.
Under macOS the Apple command key is equivalent to the Windows Control key.
There are a variety of Mac mouse types which can be used, and also a variety of
settings for configuring the Mac mouse. The Mac user is advised to consider their
mouse button settings prior to starting SPECCHIO operation so all of the above
functions are accessible.
2.1.2 Unix Operation
As described under Mac Operation above, Unix users should also consider how to
access on their computer the various mouse click combinations that are described in
this document.
The functionality of the GUI is generally consistent across all tested operating
systems.
Depending on the UNIX windowing system, some of the dialogs are not properly
displayed unless they are maximized.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 10 of 161
2.2 SPECCHIO User Interface
SPECCHIO’s Main Window opens when the application is started. It is used to launch
all of SPECCHIO’s functions.
Figure 1: Main Window
Main menu Select the SPECCHIO functions you wish to run from here.
Progress Report Panel Displays the database connection information and
information about the progress of many of SPECCHIO’s processes
(e.g. file loading).
Database Connection Status This information is displayed in the Progress
Report Panel and shows the User Name you are logged in as, and
the database you are logged in to.
The panel on the left side of this window is not used in this version of SPECCHIO.
2.3 Main Functions and Processes
The following diagram illustrates the functions and tasks performed by SPECCHIO.
Each section relates to a different menu item in the application, and is a generally
described in a different section in the User Guide. Information about campaigns is in
a separate section due to its size.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 11 of 161
2.3.1 Installing and getting access to the database
This is a brief guide for getting started. Further information on installing and getting
access to the database are contained in sections
3 (Installing SPECCHIO)
and 4 (Get
Access to the SPECCHIO Database
)
respectively.
Browsing and querying the SPECCHIO database is documented in section
10 Data
Selection & Output.
First of all, you need to install the SPECCHIO
client.
In order to get access to a SPECCHIO
database, a User Account must be created.
Connect to a database by selecting any of
your existing user account details.
SPECCHIO allows users to browse all data
available in the database via the Campaign
hierarchy.
The Query Builder provides a way to select
one or more Spectra based on their
Metadata.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 12 of 161
2.3.2 Data Loading Workflow
The process of loading data into SPECCHIO involves a number of steps. Depending
on your acquisition device, some of these steps may not be needed because the
information is recorded by your device and uploaded with the Spectral data.
Detailed information on how to import, update, process and maintain Spectral
data and related Metadata can be found later in the document in section
9 (Data
Input)
, in section
11 (Data Processing)
and in section
12 (Data Maintenance)
.
The basic operational flow in order to load data into SPECCHIO is as follows:
This establishes a name for your Campaign
and sets up structures in the SPECCHIO
database to receive the Campaign Spectra
and related data.
This examines the format of the spectral
data on your hard disk, parses its content
from your hard disk and inserts into the
SPECCHIO database.
The Spectral Metadata that was not included
in the Spectral files must be uploaded, either
from an Excel file or manually using
SPECCHIO’s Metadata Editor.
Built-in functions in SPECCHIO can be used
to generate additional metadata.
After these steps, your Campaign’s Spectral data is complete and ready for use,
exporting or publishing to ANDS (Australian National Data Service) if your SPECCHIO
server supports the ANDS portal.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 13 of 161
3 Installing SPECCHIO
3.1 Before you install
Check the following software requirements before installing SPECCHIO. You should
consult your Systems Administrator if you have any queries.
SPECCHIO CLIENT (i.e., the desktop application) requires the following software.
These should be installed in the following order:
● Java Runtime Environment v. 1.8
● SPECCHIO V3
3.2 Installing Java Runtime Environment
SPECCHIO requires that Java Runtime Environment (JRE) version 1.8 is already
installed on your computer before installing SPECCHIO itself. To check the Java
version on your system, open a command window under Windows, or a terminal for
Macintosh or UNIX systems, and type:
java -version
The output will be similar to:
java version "1.8.0_152"
Java(TM) SE Runtime Environment (build 1.8.0_152-b16)
Java HotSpot(TM) 64-Bit Server VM (build 25.152-b16, mixed mode)
If you do not have Java installed, or the version number is less than 1.8, you should
install an appropriate version of the Java Runtime Environment (JRE) from the
internet at http://www.oracle.com/technetwork/java/javase/downloads/index.html.
Note: at the time of writing, Java 9 is not yet supported by the SPECCHIO client.
3.3 Installing SPECCHIO
3.3.1 Installing SPECCHIO for the First Time
The SPECCHIO application plus the libraries it uses are supplied as an application
bundle in JAR file format. The installation bundle is usable on Windows, Mac and
Unix systems. For Mac OS X, a special installation bundle is additionally provided,
packaging the SPECCHIO application as Mac OS application.
The installation bundle is available from www.specchio.ch or via Git from
https://github.com/ahueni/SPECCHIO
1. If getting the package from GitHub, navigate to the ready-to-install packages stored
in the pkg directory. If downloading from the SPECCHIO web page, just click the
required version in the Application Download.
2. Click on specchio-client-installer.jar. On the next page click on the link to
view the
full file
. When asked, choose to keep or save the file – this will start the installation
process.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 14 of 161
3. Alternatively you may execute java -jar specchio-client-installer.jar from the
command line or simply double-click the specchio-client-installer.jar in your file
system.
4. Follow the prompts to install the software into a directory of your choice. You may
need to create the directory folder using Windows Explorer before trying to install
SPECCHIO, i.e., to install SPECCHIO to C:\SPECCHIO, first create the SPECCHIO
folder using Windows Explorer before selecting the directory in the SPECCHIO
installer. You may need Admin rights if you wish to install under C:\Program
Files\Specchio.
Note SPECCHIO does not currently support multi-user installations. SPECCHIO should
usually be installed within the personal folder of each user wanting to use the
software. This is the folder called C:\Users\UserName in Windows;
/Users/username in Mac OS X; or /home/username in Unix and Linux.
3.3.2 Upgrading an Existing Installation
You can upgrade an existing installation by following the same procedure as for a
new installation. Just install the new version of SPECCHIO into the same folder as
your existing installation. You may wish to create a backup of your existing
installation first.
Older versions of SPECCHIO store the database account information in a file named
db_config.txt. New versions store this information in system preferences.
Installations that still contain a db_config.txt file will continue to use it.
Upgrading to a new version of SPECCHIO will not change the content of an existing
db_config.txt file.
3.4 Launching SPECCHIO
3.4.1 Windows
SPECCHIO should be available in a folder under the “Start” menu. This folder
contains options for launching SPECCHIO on both 32-bit versions and 64-bit versions
of Windows. Select the option that matches your version of Windows.
The SPECCHIO shortcut may also have been added to the Accessories or Startup
folder, depending on your choices when the application was installed. Alternatively,
you may search for the SPECCHIO folder using
Search programs and files
under the
“Start” menu.
3.4.2 Mac OS X
In case you installed using the MacOS packaged installer open Finder, and navigate
to the folder into which you installed SPECCHIO. Double click the SPECCHIO
application icon. You may also drag the icon to your dock to make it accessible like
any other packaged MacOS application.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 15 of 161
In case you installed the standard bundle (i.e. non Mac OS specific bundle), open
Finder, and navigate to the folder into which you installed SPECCHIO. Navigate to
the macosx folder and double-click on specchio.
3.4.3 Unix and Linux
Navigate to the folder into which you installed SPECCHIO. If using a graphical
interface that supports it, you may be able to start the application by double-clicking
on specchio-client.jar. Otherwise, execute java -jar specchio-client.jar from the
command line.
• From the Main menu, select
Database – Create a new user account
to display the following dialog box:
Figure 2: Create a new user account dialog
• Enter the
Web Application Server
,
Port
and
Application Path
.
• To use the public ANDS-enabled database hosted by the University of
Wollongong, use the following details:
Web Application Server: specchio.uow.edu.au
Port: 443
Application Path: /specchio_service
• Data Source: jdbc/specchio
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 16 of 161
• To use the Worldwide SPECCHIO Online System, hosted by University of
Zurich, use the following details:
Web Application Server: v473.vanager.de
Port: 443
• Application Path: /specchio_service
• Data Source: Choose between productive and test servers:
jdbc/specchio_prod and jdbc/specchio_test
• If you wish to use a different database, please ask the administrator of that
database for the correct values.
Figure 3: Server connection details (example for University of Zurich in-house
SPECCHIO system)
• Click Connect.
The remaining fields on the screen will become active. If ANDS Publishing is
supported for your database, ANDS related fields will appear in the dialog
box as shown below. See
Section 12 Publishing Data to ANDS
for more
information about the ANDS service and its operation.
Figure 4: Create a user account dialog for ANDS users
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 17 of 161
Figure 5: Create User Account dialog for non-ANDS users with specific server
license agreement
• Select your
Title
and enter your
First name
and
Last Name
. These will
be used to identify the data which you upload to SPECCHIO to other
SPECCHIO users.
• Select the name of your
Institute
from the dropdown list.
If the name of your Institute is not present in the list, click on
Add new institute…. This will cause the following dialog box to appear.
Figure 6: Create new institute dialog
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 18 of 161
Fill out all the details of your Institute. After checking it is complete and
correct, click on Create.
This will create an entry for your Institute on the
SPECCHIO server.
• Enter your email address which other SPECCHIO users can use to
contact you. This is a mandatory field.
• Enter a URL which other SPECCHIO users can use to access
information about you. The URL is an optional field.
• Include a description of your role. This is a mandatory field.
• If your server’s database supports ANDS Publishing and you already
have an
ANDS Party Identifier
, set the checkbox and enter your
ANDS
Party Identifier
in the relevant field. Take care when entering this field.
You cannot change it later. If you do not have an ANDS Party
Identifier, leave the checkbox unchecked and SPECCHIO will create
one for you.
• Click on Create. The dialog box will close.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 19 of 161
4 Get Access to the SPECCHIO Database
This section describes the functionalities of SPECCHIO’s Main Menu item
Database
.
You will find information on how to:
This simplified workflow shows the first steps to get access to
the database. Once you have created your user account and
edited some general information, simply connect to your
database and use the required functions such as browsing,
importing, updating or exporting data.
4.1 Create a new user account
Warning Take care to enter the details for a new Institute correctly. There is no function to
change an Institute record nor to remove an Institute record from the SPECCHIO
server. If there is a need to change or remove an Institute record, you will need to
request your database administrator to do it for you. It is also possible to create a
totally blank institute record.
4.2 Administrator Access
Certain SPECCHIO functions, such as uploading Instrument definitions and
Calibrations, require administrator permission. For normal users, these menu
operations are greyed out.
When the SPECCHIO database is created, it has an Administrator account defined.
The User Name and Password for this account are not generally advertised. See your
SPECCHIO Administrator if you require access to these administration functions.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 20 of 161
4.3 Connect to Database
Because the user account details are kept in the database, connecting to the
SPECCHIO database and logging in are performed as one operation.
Preconfigured information for your database and the log in accounts you have
created on your computer is stored in the db-config.txt file on your computer. See
Section 4.1
for instructions on creating a User Account.
To connect to database from your computer:
• Start the SPECCHIO Application. On Windows it may be available under
the Start menu; this may depend on your user rights and windows
version.
• Once the SPECCHIO Main Window opens, select
Database - Connect to
database
to display the connections dialog box:
Figure 7: Connect to database dialog
• If you are accessing SPECCHIO from the same machine you created your
account from, the dialogue box should already be populated with your
username and password.
• From the
Known connections
dropdown list, select the connection with
your User Name following the https:// characters. The User Name will be
a contraction of your own First and Last Names.
• Inspect the values which are updated in the dialog box. It should not be
necessary to change anything unless specifically requested by your
database administrator.
• Click Connect. The dialog box will close and there will be short delay
while the database is read. The details of your database and log in
account will be displayed in the Progress Report Panel (the right hand
panel of the Main Window) and the sub-menu items for SPECCHIO
functions will now be active.
You can repeat this process at any time and log in as a different User, or even log in
again as the same User.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 21 of 161
To connect to database from a different computer:
• If you are accessing SPECCHIO from a different machine, the configuration file
will not have your connection details, you may see the following message:
Figure 8: Connection warning
• You will need to manually enter the required information into the
Connect to
database
dialog box or copy the full connection string from your original
configuration file. You will need to know your Username and Password, as well
as the connection details, before you can connect to the database. Refer to the
connection details in section 4.1 (Create a new user account), or contact your
administrator. Alternatively, you may copy the db_config.txt file. This file
contains all connection details.
Figure 9: Connect to database dialog
• Fill in the connection details, as well as your Username and Password then click
Connect. The dialog box will close and there will be short delay while the
database is read. The details of your database and log in account will be
displayed in the Progress Report Panel (the right-hand panel of the Main
Window) and the sub-menu items for SPECCHIO functions will now be active.
4.4 Logging Out
There is no specific log out function for SPECCHIO. Closing the Main Window will
close your database session. If you restart SPECCHIO, you will need to log in again.
If you select
Database
-
Connect to database
from the SPECCHIO Main Window
menus while you are logged in, SPECCHIO will log you out before logging in again as
the new User.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 22 of 161
4.5 Edit user information
After logging in and connecting to the database, you can edit your User Information.
To Edit user information:
• From the Main menu, select
Database - Edit user information
. The
following dialog box will be displayed showing your current User
information:
Figure 10: User Information update dialog
• Update the information as required. You cannot change the ANDS Party
Identifier.
• Click Update to cause the changes to be written back into the database.
This process does not change your
User Name
or
Password
, so there is no
requirement to change any of the account configuration stored on your own
computer.
4.6 Edit db_config file
Older versions of SPECCHIO stored account details in a configuration file
(db_config.txt) by default. If you have upgraded from previous versions then
SPECCHIO will continue to read account data from the configuration file. Newer
versions can be enforced to write data into the configuration file and load account
details from it.
Using the db_config file can simplify working from different machines. However, care
is required to maintain the file structure when editing it manually. Furthermore,
installations in a multi-user environment should not use the db_config file based
storage, as this will expose the connection information to other users having
accounts on the same machine.
To create and/or edit the db_config file:
• From the Main menu, select
Database - Edit db_config file
. In case the
menu item is greyed out, enable the editing of the db_config file in the
SPECCHIO Preferences by setting the according switch (see
section 4.7).
• The db_config file will be displayed in the default text editor of your
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 23 of 161
operating system.
4.7 Preferences
Preferences are stored in the system settings of your computer and define some
aspects of how SPECCHIO works.
When uploading old ASD file formats, an optional creation of measurement unit
folders can be chosen by activating the relevant box.
If ASD DN files are to be inserted, they will be inserted in addition to the calibrated
spectrum (e.g. Radiance).
Input and output directories define the default directories that are used for file input
and output, such as during campaign export/import operations (see 9.8).
To Change Preferences:
• From the Main menu, select
Database – Preferences
. The following dialog
box will be displayed showing your current preferences:
Figure 11: User preferences dialog
• Update the information as required. The new settings are available
immediately within SPECCHIO.
• Close the window.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 24 of 161
5 Data structure and concepts
In this section you will find information about Campaigns and how they are
structured. Please see
Section 7 Data Input
for information on how to set up a
campaign and load metadata.
SPECCHIO organises the Spectra for each new sampling experiment into Campaigns.
The Spectral data in each Campaign can be viewed and operated on together.
Campaigns can contain Spectra measured with different instruments and sensors,
including instruments of different Sensor types and measurements done at different
times using an Instrument which has been calibrated one or more times between
these measurements.
Spectral data handling is based on the space concept; an understanding this
paradigm is providing the background to understand how spectral data are handled
by the software. This is of particular importance if higher-level languages (e.g.
Matlab or R) are used to access SPECCHIO.
For details on the storage of metadata please refer to the following sections: 8.2
(Non-Redundant Storage), (Metadata Storage Levels) and ().
5.1 Campaign Hierarchy Structure
On your computer, the Spectral data files for a Campaign must be placed into a sub-
folder structure which organises the Spectra according to their context or sampling
design. SPECCHIO does not impose any restrictions on the complexity of Campaign
and sub-folder hierarchies. SPECCHIO does not enforce any particular way of using
this hierarchical structure. The hierarchy tree structure can have any number of
levels and can be unbalanced, if desired. This sub-folder structure will be replicated
in the SPECCHIO database when the Campaign data are uploaded.
The concept of a hierarchical data structure that is used in SPECCHIO has been
adapted from SpectraProc (Hueni and Tuohy, 2006), which was built on a fixed
hierarchy of three levels. However, SPECCHIO supports hierarchies of arbitrary shape
and number of levels.
Preferably the design of sampling experiments should include hierarchical data
structuring from the beginning. However, existing data can be re-arranged to meet
these requirements.
There are some specific restrictions on the Campaign hierarchy tree structure.
• No other files should be in the sub-folder tree except Spectral files. Keep images,
PDF files and other files in a separate folder, probably next to the top-level folder
of your tree hierarchy structure. While unknown files are generally ignored by the
file loading process, it is recommended to not store them in the spectral data
hierarchy operated on by SPECCHIO.
• To conveniently use SPECCHIO’s goniometer angle calculation, all of the Spectra
measured at the different angles should be put in one folder and observed in the
defined sequence. There should be no other Spectra in that folder. The
goniometer angle calculation has been designed for the use of the FIGOS and
LAGOS goniometers system (RSL, University of Zurich).
New Spectra can be added to ongoing Campaigns by first adding the files for those
Spectra into the existing Campaign sub-folder data structure on your computer’s disk
and then invoking the SPECCHIO load operation for the entire Campaign again. Only
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 25 of 161
new Spectra discovered in the sub-folder hierarchy will be added to the database.
Existing Spectra will not be duplicated. New data can also be added to an existing
campaign by adding a new loading path.
Example – Multiple sites for each species
In this example, each species is sampled at a number of sample sites. The sites are
numbered in the order of sampling. At each site, several readings are taken to
capture the variation exhibited by the specimen in question. A site therefore contains
a number of Spectra. This leads to the following hierarchical folder structure.
Figure 12: Hierarchical folder structure
Although the term “species” is used, it essentially represents the different classes
found in a study. These classes can either be assigned according to already existing
classification systems, e.g. for plants or minerals. In other cases a hypothesis might
exist that a number of objects can be separated into classes. If so the setup of the
experiment should mirror this hypothesis. If no such assumption exists all objects
can be put into the same class (i.e. species) and the identification of classes could
then be carried out by a technique such as cluster analysis.
Figure 13 shows an example of a folder structure containing ASD Spectral files. The
main folder Vegetation_example holds all species sub-folders for the study. This main
folder is the folder that needs to be specified in the
Main directory:
in the Campaign
Creation dialog. See
Section 7.2 Creating a new Campaign.
The three species folders contain their related site sub-folders.
The site sub-folders contain all Spectral files collected at these sites for that species.
The Spectral files are auto-numbered by the ASD capturing software within each site
folder.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 26 of 161
Figure 13: Example of a folder structures holding Spectral files
Example – Reference and Target Spectra
If a spectroradiometer is configured to acquire radiances, the irradiance will
commonly be characterised indirectly by measuring the radiance reflected by a
reference panel. This may be preferable to the direct acquisition of reflectance data
as information about the irradiance can be retrieved from the dataset (Hueni et al.,
2017, Milton et al., 2006).
The structure must therefore hold target and the related reference radiance Spectra
in adjacent folders.
Figure 14: A possible structure for the storage of target and reference radiance
Spectra
In this example, the Reference sub-folder holds the reference Spectra which relate to
the Spectra in the Plant A, Plant B and Plant C sub-folders.
Note: it is also possible to deal with target and reference spectra by giving them
according metaparameters.
5.2 Spaces and the Space Factory
SPECCHIO offers interactive, configurable data processing. The concept is based on
the feature spaces (Landgrebe, 1997) and complex process flows can be realised by
building networks consisting of spaces and processing modules. For detailed
information please refer to: (Hueni et al., 2009)
Spaces are used throughout the system for processing, visualisation and file output.
In all these cases, vector data must be related to Spectral dimensions and this
information is held by the space. Moreover, a space can hold only Spectra that are of
the same dimension.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 27 of 161
The Space Factory is a conceptual, central component of the SPECCHIO system. It
creates new spaces based on given inputs and contains the logic to form ‘non-mixed’
spaces.
Assume the use case of displaying Spectral plots of a number of Spectra. In a first
step, the user will select the Spectra to be plotted by effecting a subspace projection
(Hüni et al., 2007). Internally, this will yield a number of record IDs that are
matching the user’s selection. These IDs are now handed to the Space Factory.
Internally, spaces are created for all existing combinations of the respective sensors,
instruments, calibrations and measurement units associated with the Spectra (see
Figure 15).
Figure 15: Building spaces based on user defined subspace projections
The Space Factory returns a list of the created spaces. Each space can now in turn
be used as an input argument of a plotting class instance. Utilizing the Space Factory
ensures that all Spectra contained by a space have a common wavelength per band
and the same measurement unit, i.e. the following processing modules do not need
to carry out uniformity checks but can apply their algorithms directly, e.g. plotting of
Spectral vectors against the common wavelengths of the space.
User search conditions
– Query builder or Data
Hierarchy Browser
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 28 of 161
6 Operational Dataflow
A typical dataflow is illustrated in Figure 16. A spectroradiometer is used to capture
the radiance of field objects. Optionally a GPS connected to the field laptop (where
supported by the capturing software) records the spatial position of the field object.
This is a recommended setup as the spatial position can later be automatically loaded
into the database. Spectra and Metadata are saved on the field laptop.
These files are transferred to a laboratory computer where they are read by the
SPECCHIO application and stored in the relevant tables in the Spectral database.
Figure 16: Dataflow and involved hardware
Typically, the operations carried out for each Campaign are:
• Creation of a new Campaign in SPECCHIO and linking it to the sub-folder of
Spectra on your computer’s disk
• Uploading of Spectra data from your local disk to the SPECCHIO database
• Entering of Metadata into the SPECCHIO database for the uploaded Spectra
• Repeated data query, visualisation, processing and file output based on the
uploaded data
Field laptopSpectroradiometerField object Lab computer
Radiance
Reflectance /
Radiance / DN /
Absorbance
Data file
Spectral data
& metadata
Spatial position
GPS device
SPECCHIO DB
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 29 of 161
7 File Formats
7.1 Supported Input Spectrum File Formats
The following Spectrum file types are supported for loading into SPECCHIO
Campaigns.
Each of the 14 file categories listed below is read by a separate file loader. Spectral
files of different file types can be freely mixed in the input directories if required.
7.1.1 ASD Binary Files
Standard or
Format Owner
Analytical Spectral Devices
Devices
ASD FieldSpecPro/FS3 spectroradiometers
Supported
Formats
Old file format – ASD FS3, ASD FS PRO and ASD FSVNIR binary files
Comments
To make the hierarchies compatible with data loaded from ASD
Indico Version 7 Files, the SPECCHIO Preferences (see
section 4.7
)
can be set to generate sub-folders for the old file format as well,
e.g., for a radiance file:
ASD Calibration files (*.raw) as stored on ASD laptops can also be
loaded using SPECCHIO’s Spectrum load functions. This permits
advanced users to explore the information content of the
Calibrations they describe. (In the current release, loading *.ILL,
*.REF files will cause an error.)
7.1.2 ASD Indico Version 7 Files
Standard or
Format Owner
Analytical Spectral Devices
Devices
Indico Pro Software
Supported
Formats
New file format – Indico Version 7
Comments
Subfolders will be created automatically during loading, depending
on the selected unit during data acquisition. DN folders holding the
raw digital numbers will always be created. Metadata read from the
input file and common to DN spectra and processed spectra is
shared automatically.
Loading of ASD Calibration files of this version using the Spectrum
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 30 of 161
load functions has not been tested and may not work.
7.1.3 GER Signature Files
Standard or
Format Owner
Spectra Vista Corporation
Devices
GER 3700
Supported
Formats
GER 3700 signature text files
(Files produced by other GER instruments are untested)
Comments
These files hold two Spectral measurements at once: the Target
Spectrum and the white Reference Spectrum. When read, two sub-
folders named
targets
and
references
are created. The input file
name is used as both the Target Spectrum’s name in the
targets
folder and the Reference Spectrum’s name in the
references
folder.
A data link is created linking the Target Spectrum to the Reference
Spectrum.
Figure 17: Automatically created hierarchies for GER files
7.1.4 MFR OUT Files
Standard or
Format Owner
Devices
MFR Sun Photometer
Supported
Formats
MFR Photometer OUT files
MFR 7
Comments
These files contain the capture time, the sun zenith angle
and the Spectral data for total, diffuse and direct irradiance.
The sun angle and the direct irradiance data are discarded
and only the total and diffuse Spectra are stored.
When read, two sub-folders named
total
and
diffuse
are created.
The input file name is used as both the name of the total
Spectrum’s name in the
total
folder and the diffuse Spectrum’s
name in the
diffuse
folder.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 31 of 161
Figure 18: Automatically created hierarchies for MFR data
7.1.5 Microtops Sun photometer Files
Standard or
Format Owner
Solar Systems
Devices
Microtops Sun Photometer
Supported
Formats
Microtops text output files (TXT file extension)
Comments
When read, three sub-folders named
AOT
,
Spectrum
and
Spectrum
StdDev
are created. The input file name is used as the name of the
Spectrum’s name in the generated sub-folders.
Each spectrum is augmented with its spatial position, sun zenith
angle, air pressure and estimated atmospheric water content.
Figure 19: Automatically created hierarchies for MFR data
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 32 of 161
7.1.6 Bruker FTIR Files
Standard or
Format Owner
Bruker
Devices
Bruker FTIR
Supported
Formats
Bruker FTIR text files with dpt extension
Comments
Example files supplied by CSIRO Perth.
Example files
3996.35288,7.62443
3994.42414,7.64596
3992.49539,7.65170
3990.56665,7.65495
3988.63791,7.67457
3986.70917,7.69236
3984.78043,7.69543
3982.85169,7.69582
3980.92294,7.70848
3978.99420,7.72674
…
7.1.7 SVC HR-1024 Files
Standard or
Format Owner
Spectra Vista Corporation
Devices
SVC HR-1024
Supported
Formats
SVC HR-1024 files
Comments
The HR-1024 stores radiances of reflectance panel, target and the
computed target reflectance.
During the data loading, SPECCHIO generates a hierarchical
structure to store these files, setting up data links that connect the
reflectance to the target radiance and connecting the target
radiance to the reference radiance. If the Instrument was set to
acquire radiances only (i.e., no white reference taken), then no
special structure will be created.
The acquisition time may be wrongly read as it appears that the
date format depends on the settings of the computer used to
operate the SVC.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 33 of 161
Figure 20: Automatically generated hierarchy for HR-1024 files
Note For SPECCCHIO version pre 3.1: It is highly recommended to enter the Instrument
into the database and load a wavelengths calibration. The wavelengths between
instruments differ quite a lot and the standard Sensor definition pre-stored in
SPECCHIO is only a poor representation of the real wavelengths.
Example files
/*** Spectra Vista HR-1024 ***/
name= \My Documents\HR1024_Data\HR.071710.0017.sig
instrument= HR: 0761008
integration= 2, 22, 20, 12, 50, 30
scan time=1, 1
optic= LENS14, LENS14
temp= 31.29, 0.41, -5.71, 31.53, 0.41, -5.77
battery= 7.7, 7.7
error= 0, 0
units= Radiance, Radiance
time= 7/18/10 9:47:09 AM, 7/18/10 9:47:31 AM
longitude= 11121.2335,W, 11121.2324,W
latitude= 5330.5955,N, 5330.5964,N
gpstime= 154336.000, 154356.000
comm=
memory slot= 0, 0
factors=
data=
344.2 56023.55 4381.86 7.82
345.8 54418.00 4186.00 7.69
347.3 56037.65 4365.57 7.79
348.9 59474.63 4568.75 7.68
…
/*** Spectra Vista SIG Data ***/
name= \My Documents\HR1024_Data\HR.080910.0010.sig
instrument= HR: 0971030
integration= 20, 19.2, 30, 200, 60, 30
scan time= 3, 3
scan settings= AD, AI, AD, AI
external data set1= 8224, 8224, 8224, 8224, 8224, 8224, 8224, 8224, 8224, 8224, 8224,
8224, 8224, 8224, 8224, 8224
external data set2= 8224, 8224, 8224, 8224, 8224, 8224, 8224, 8224, 8224, 8224, 8224,
8224, 8224, 8224, 8224, 8224
external data dark= 0,0,0,0,0,0,0,0
external data mask= 0
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 34 of 161
optic= LENS 4, LENS 4
temp= 36.21, 9.08, -5.30, 36.45, 9.08, -5.37
battery= 7.5, 7.4
error= 0, 0
units= Radiance, Radiance
time= 8/9/10 10:49:47 AM, 8/9/10 10:52:36 AM
longitude= 01116.9879,E, 01116.9933,E
latitude= 4806.4988,N, 4806.5075,N
gpstime= 084748.000, 085034.000
comm=
memory slot= 0, 0
factors=
data=
350.6 127369.73 4658.17 3.66
352.1 130962.30 4815.39 3.68
353.7 132239.65 4916.35 3.72
355.3 128726.40 4836.48 3.76
356.8 124030.10 4677.99 3.77
358.4 123421.08 4677.13 3.79
359.9 130810.10 4966.42 3.80
…
7.1.8 Apogee Files
Standard or
Format Owner
Apogee Instruments
Devices
Supported
Formats
Apogee text files (restricted to a few tested files only – see example
below)
Comments
The SPECCHIO Apogee file loader was added to SPECCHIO for one
special case. That is the only Apogee file example that has been
available for testing. Therefore, this file reader has not been well
tested. It is provided “as is” and should be used with caution.
Example of supported Apogee format text file
" File: cachimbalito2\p5_2.TRM
" TRANS-> Wave:733.53nm Pix:1050 Val: 25.767 Time:23ms Avg:7 Sm:0 Sg:0 Tc:on
Xt:1 Ch:1
400.00 1.102E+000
400.50 1.131E+000
401.00 1.205E+000
401.50 1.257E+000
402.00 1.260E+000
402.50 1.336E+000
403.00 1.341E+000
403.50 1.418E+000
404.00 1.451E+000
404.50 1.444E+000
405.00 1.488E+000
…
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 35 of 161
7.1.9 ENVI Spectral Library Files
Standard or
Format Owner
Exelis Visual Information Systems
Devices
ENVI Software Library
Supported
Formats
ENVI Spectral library files (SLB and SLI)
Comments
The import routine reads each .hdr file in the folder. Each .hdr
file refers to either an .slb or .sli file for the body of the
Spectrum data. That is, Spectrum data are stored in pairs of files.
For example:
my_spectra.hdr
my_spectra.slb
or
my_spectra.hdr
my_spectra.sli
Both .slb and .sli files can be uploaded in the same sub-folder.
Each file pair can contain more than one Spectrum. If present,
Spectrum names are read from the header file and stored in the
database in the Filename Metadata Attribute.
The File Type Metadata Attribute is set to
ENVI Hdr
.
The Sensor definition (i.e. central wavelengths) is not read from the
ENVI header file. You should define the correct Sensor using the
Data Maintenance - Load sensor definition
menu functions. See
Section
12.2
.
7.1.10 Ocean Optics SpectraSuite Data Files
Standard or
Format Owner
Ocean Optics
Devices
Ocean Optics Spectra Suite
Supported
Formats
Ocean Optics Spectra Suite text files
Comments
The required file extension is .csv, although the files are not comma
separated.
Example file
SpectraSuite Data File
++++++++++++++++++++++++++++++++++++
Date: Wed Sep 15 19:14:15 CEST 2010
User: telerilevamento
Dark Spectrum Present: No
Reference Spectrum Present: No
Number of Sampled Component Spectra: 1
Spectrometers: HR4C1076
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 36 of 161
Integration Time (usec): 1000000 (HR4C1076)
Spectra Averaged: 10 (HR4C1076)
Boxcar Smoothing: 0 (HR4C1076)
Correct for Electrical Dark: No (HR4C1076)
Strobe/Lamp Enabled: No (HR4C1076)
Correct for Detector Non-linearity: No (HR4C1076)
Correct for Stray Light: No (HR4C1076)
Number of Pixels in Processed Spectrum: 3648
>>>>>Begin Processed Spectral Data<<<<<
Wavelength(nm); radiance(W*m-2*sr-1*nm-1)
717.00000;0.17904775
717.02000;0.17878146
717.04000;0.17849983
717.06000;0.17820124
…
7.1.11 Ocean Optics Ocean View Data Files
Standard or
Format Owner
Ocean Optics
Devices
Ocean Optics Ocean View Software
Supported
Formats
Ocean Optics Ocean View text files
Comments
The required file extension is .txt.
Example file
Data from Tree1_Day1_am_MAYP1114952_10-08-17-448.txt Node
Date: Wed Mar 25 10:08:17 GMT 2015
User: amacarth
Spectrometer: MAYP111495
Autoset integration time: false
Trigger mode: 0
Integration Time (sec): 8.500000E-1
Scans to average: 1
Electric dark correction enabled: true
Nonlinearity correction enabled: true
Boxcar width: 0
XAxis mode: Wavelengths
Stop averaging: false
Number of Pixels in Spectrum: 2068
>>>>>Begin Spectral Data<<<<<
649.693 838.02
649.795 -180.19
649.896 -91.96
649.997 -111.79
…
7.1.12 HDF5 Files containing FGI goniometer measurements
Standard or
Format Owner
Finnish Geodetic Institute proprietary format
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 37 of 161
Devices
ASD in combination with the FIGIFIGO goniometer
Supported
Formats
HDF5 files and XML files containing measurement data from FGI
(Finish Geodetic Institute) goniometer measurements
The newest data structure version (with a coupled xml-file,
containing all the Metadata) is still experimental and is not yet not
fully supported.
Comments
As there is more than one Spectrum per file, each Spectrum is
saved separately. The name of each Spectrum is constructed from
the Campaign name, the beam geometry (if HDRF or BRF), a data
structure identifier and an auto numbered identifier (e.g.
Snow7.HDRF.lib2.14).
7.1.13 UniSpec Single Channel
Standard or
Format Owner
PP Systems
Devices
UniSpec SC
Supported
Formats
UniSpec Single Beam text files
Comments
7.1.14 UniSpec Double Channel SPU
Standard or
Format Owner
PP Systems
Devices
UniSpec DC
Supported
Formats
UniSpec Dual Beam text files
Comments
7.1.15 SPECPR
Standard or
Format Owner
U.S. Geological Survey
Devices
USGS PRISM software
Supported
Formats
SPECPR
Comments
Search for more information about PRISM software and the SPECPR
file format from the U.S. Geological web site.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 38 of 161
7.1.16 Modtran Albedo File
Standard or
Format Owner
Spectral Sciences Inc. and U.S. Air Force Research Laboratory
Devices
Modtran5 atmospheric radiative transfer model software
Supported
Formats
Modtran Albedo files
Comments
More information can be found on the Modtran web site at
http://www.Modtran5.com.
7.1.17 Excel files
Standard or
Format Owner
Microsoft Excel Spreadsheet files
Devices
N/A
Supported
Formats
.XLS files only - .CSV, .XLSX etc are not supported
(.CSV files may be erroneously read as Ocean Optics SpectraSuite
Data Files. See Section
7.1.10
.)
Comments
The first column of the Excel file must contain the Wavelength
values in increasing order, with a heading in the first row.
Each of the second and subsequent columns contains the data for a
single Spectrum. The first row is the name of the Spectrum.
Subsequent rows in the column contain the Spectrum values for the
wavelengths specified in the first column.
The Spectra’s File name Metadata Attribute values are constructed
from the Excel file name and the Spectrum name in the first row of
the spreadsheet, separated with a “_” character.
There can be no other data in the spreadsheet and the table of data
must be rectangular with no gaps. Excel formatting is ignored.
Only the first datasheet is read.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 39 of 161
Example file
Figure 21: Top-left corner of example Spectrum spreadsheet
7.1.18 TXT Space Formatted Text Files
Note TXT files are deprecated, except in the case where Reference Panel calibrations are
loaded as Spectra. Please use the Excel file format instead.
Standard or
Format Owner
N/A
Devices
IDL (Interactive Data Language) or ENVI text file output
Supported
Formats
White space separated text files (as produced by IDL programs or
ENVI when writing Spectral data to text files)
Comments
TXT Spectrum files can be created or viewed with a text editor.
• Each file can hold one or more Spectra.
• Values in each line are white space separated. Tabs are OK.
• Starting each line with whitespace is optional.
• The first line of the file is a heading line. The first value on this
heading line must be “wvl”. Its subsequent values are used as the
File name Metadata Attribute values for the Spectra in SPECCHIO.
• Spectrum names cannot include spaces.
• All values on subsequent lines must be numeric. The first column is
the wavelength in nanometres. Subsequent columns are the
Spectrum values for each Spectrum at that wavelength.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 40 of 161
Example
The following TXT Spectrum file example defines two Spectra named mean_090499
and mean_020599. They each have observations at wavelengths from 350nm to
361nm.
wvl mean_090499 mean_020599
350.000 0.0246756 0.0229771
351.000 0.0246917 0.0228430
352.000 0.0247316 0.0229652
353.000 0.0248502 0.0231014
354.000 0.0250081 0.0232272
355.000 0.0250736 0.0232273
356.000 0.0249883 0.0233005
357.000 0.0249174 0.0233962
358.000 0.0250481 0.0234734
359.000 0.0252141 0.0235376
360.000 0.0253346 0.0236057
361.000 0.0253806 0.0236832
7.2 Supported Output Spectrum File Formats
Spectrum data can be written in two data formats.
• CSV (Comma Separated Values) for subsequent import into various 3rd party
applications like spreadsheets or statistical packages
• ENVI Spectral Library (SLB)
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 41 of 161
8 Metadata
8.1 Research Groups and Accessing SPECCHIO Campaigns
When you upload data for a Campaign to SPECCHIO, all Users of your SPECCHIO
database can access your Campaign. However, only you can modify or delete your
Campaign unless you specifically allow others to have write access to it. This access
is controlled by setting a Research Group of Users for each Campaign that you
upload. Just as you can add other Users to the Campaign’s Research Group, so you
can remove them too.
The membership of each Research Group is stored as Metadata for the Campaign.
Each Research Group applies only to one Campaign. (See
Section
6.9 Campaign-
related Metadata
for more information about Metadata and
Section
7.5.2
Drag &
Drop Options
Drag
& Drop is currently implemented for spatial metadata only. KML files containing
points, polylines or polygons can be drag and dropped onto spatial fields to populate
them.
Figure 32: Dragging of a KML file onto a Spatial Position metadata field
Displaying and Editing Campaign Metadata for instructions to update it.)
When you add another SPECCHIO User to the Research Group for your Campaign,
that User gets all the rights that you have for that Campaign. That is, they can
modify the data, add to it or remove it. They can also change the membership of the
Research Group.
It is not possible to remove yourself from the list of Users in the Research group.
8.2 Non-Redundant Storage
Many metaparameters constituting the spectral metadata are highly redundant, in
particular as often measurements are being replicated for statistical reasons.
SPECCHIO supports non-redundant storage of metadata by creating links between
spectra and values, allowing unique values to be stored only once for a group of
spectra.
There are two storage methods within SPECCHIO:
Non-EAV
Metadata
The following attributes are stored for all Spectra and are
always displayed – Measurement unit, File format, Sensor,
Instrument and Reference. If they are not relevant or no data
is available, set them to
Nil
.
EAV Metadata
So that space is optimised, for all other Metadata Attributes
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 42 of 161
SPECCHIO stores only those values which are set. They are
stored in an Entity/Attribute/Value table, where one Metadata
value is stored in each row of the table as an Attribute=Value
pair (such as Ambient Temp=23.5). A cross-relation table
between the Entity (i.e. the spectrum or the hierarchy (for
details see 8.3)) and the Value stores a list of Spectra which
are associated with this particular Attribute and Value. Details
may be found in (Hueni et al., 2012).
8.3 Metadata Storage Levels
Starting with V3.3.0, SPECCHIO supports two different storage levels:
• Spectrum Level: metadata values are linked to individual spectra
• Hierarchy Level: metadata values are linked to hierarchies
Storage at hierarchy level results in a virtual inheritance of metaparamters set for a
hierarchy to all spectra beneath this hierarchy. The significance of this can be
appreciated by looking at an example. Consider a large campaign comprising 100’000
spectra. Some metadata, like and experimental design description, will apply to all
spectra. If
8.4 Time Data
SPECCHIO expects acquisition times for Spectra acquisition to be in UTC
(Coordinated Universal Time). For practical purposes UTC is considered equivalent to
GMT. However, the term UTC is recommended for technical contexts (NOAA, 2003).
The above implies that all computers capturing data for use in SPECCHIO should
have their time zone set to GMT and their system time adjusted to UTC. However,
this is not usual, so the Spectrum acquisition date/time can be modified in the
database if a time different from UTC was used. To deal with this general problem in
a generic and traceable way, a metaparameter named Acquisition Time (UTC) has
been added to enable the storage of both computer system time and UTC time for a
spectrum.
Other times, such as Spectra upload times and calibration times, are usually stored in
local time, that is, the time of the SPECCHIO server.
8.5 Data Links
For some calculations on Spectra, such as the conversion of a Radiance Spectrum to
a Reflectance Factor Spectrum, SPECCHIO needs to know the related White
Reference Spectrum for each Target Spectrum. Data links are used to define these
relationships.
A Reference Data Link is a reference from a Target Spectrum to its related Reference
Spectrum.
A Target Data Link is a reference from a Reference Spectrum to its related Target
Spectrum or from a Target Spectrum to another Target Spectrum when an object
was sampled with different instruments.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 43 of 161
Manufacturer
A manufacturer of spectroradiometric instruments.
The list of possible Manufacturers is a fixed table in
SPECCHIO. Users cannot add to it, change it or even view it.
If you require a new Manufacturer to be added, you must
contact your System Administrator. The table as released
with this version of SPECCHIO can be found in
Appendix B:
Predefined Manufacturer Table
of this document.
Each Manufacturer is assigned an integer Manufacturer ID.
Sensor
A description of a physical setup of a Sensor or a model of a
spectroradiometric Instrument. A Sensor is usually identified
by a Sensor type designator (also referred to as an
“Instrument type” or “Instrument model”) which is generally
defined by the manufacturer.
Each Sensor has a fixed number of bands, average
wavelengths and FWHM (Full Width at Half Maximum) per
channel, and a Sensor type number (usually a numeric code
given by the manufacturer). Each Sensor Type is defined only
once in the database. As released, SPECCHIO has a standard
set of Sensors already defined in its database. The table as
released with this version of SPECCHIO can be found in
Appendix C Predefined Sensor Table
of this document.
However, new Sensors can be added to this list. Refer to
Section 9.4 Load sensor definition.
As of V3.1 of SPECCHIO, new sensors are automatically
generated if no appropriate sensor is found during the
loading of spectral input files.
Each Sensor is assigned a unique integer Sensor ID by
SPECCHIO.
Instrument
An instance of a certain Sensor Type. There can be several
different Instruments that are all of one Sensor Type. Each
Instrument has a defined owner. Each Instrument record in
the Instrument Table refers to its related entry in the Sensor
table so that its standard Sensor technical details can be
accessed.
As of V3.1 of SPECCHIO, new instruments are automatically
added to the database when no matching instrument can be
identified during data loading.
Each Instrument is assigned a unique integer Instrument
Number.
Calibration
Instruments optionally have a history of Calibrations. There
are generally multiple Calibrations in that history, each
Calibration being identified by either a date, and/or a
Calibration Number which is a sequence number incremented
each time the Instrument is sent for calibration. Calibration
Numbers are unique for an Instrument. An Instrument
Calibration defines an updated set of average wavelengths
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 44 of 161
per band. When a Calibration is available, its information
overrides the information in the Sensor record for the
Instrument. As of V3.1 of SPECCHIO, new wavelength
calibrations are automatically added to known instruments if
differences in centre wavelengths are encountered during
data loading.
A Provenance Data Link is a reference from a higher-level product to the spectral
input data used in its production. Provenance Data Links are e.g. generated
automatically when a spectral input contains target and reference radiances as well
as derived reflectance factors, e.g. for SVC or Spectral Evolution instruments.
There are two types of calculation for which SPECCHIO uses Reference Spectra. The
type is selected automatically based on Reference Spectrum’s Metadata Attributes.
White
Reference
The linked Spectrum is the Radiance Spectrum of a white
Reference Panel, such as a Spectralon panel, in the same
illumination conditions.
Cosine
The linked Spectrum is an Irradiance Spectrum, usually acquired
with a cosine receptor.
In some cases data links are set up automatically during Campaign data loading. For
example, GER signature files include the Spectra of both target and white reference.
Note Data links do not make sense for Target Spectra with Measurement Units other than
Radiance and for Reference Spectra other than Radiance or Irradiance. In other
cases the results are undefined and they should not be set.
Note The reflectance factor computation implemented in SPECCHIO is a simplified version,
as each target spectrum needs to be linked to a reference spectrum. More scientific
calculation can be achieved by using external code (e.g. Matlab or R).
8.6 Manufacturers, Sensors, Instruments and Calibrations
SPECCHIO defines Manufacturers, Sensors, Instruments and Calibrations, which each
have their own table in the database.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 45 of 161
Example
Figure 22: Relationship between an Instrument, its Calibrations, Sensor and
Manufacturer
Consider the example above of Joe’s GER 3700 Instrument. This Instrument is an
instance of a GER 3700 Sensor. The Sensor defines the average wavelength per
band. In turn, the Sensor has a reference to the Manufacturer record for that
Sensor’s Manufacturer.
As long as no Calibration for the Instrument has been entered into the database the
channels defined in the Sensor will be used for plotting and exporting Spectral data.
When Calibrations are entered for Instruments, the measured wavelengths override
the Sensor specifications. For further information please refer to (Hueni &
Kneubuehler, SPECCHIO: A System for Storing and Sharing Spectroradiometer Data,
2007).
8.7 Campaign-related Metadata
SPECCHIO allows Users to store Metadata about the Spectral data they have
uploaded into SPECCHIO Campaigns. It is stored at two levels: Campaign-related
Metadata and Spectrum-related Metadata.
The following Metadata Attributes can be set independently for each Campaign.
Campaign name
The name under which the Campaign is stored in the SPECCHIO
database.
Description
A free format description of the Campaign.
Investigator
The person in charge of this Campaign (i.e. the User defining and
loading the Campaign). This field is filled automatically and
cannot be changed.
Path
The file system path pointing to the main folder from which this
Campaign was uploaded.
Multiple paths can be defined to support uploading of subsequent
Spectral data from multiple computers. This is intended to assist
when further data is uploaded for this Campaign.
Calibration 1
Calibration 2
Instrument
“Joe’s GER
3700”
Instrument Number,
Name,
Owning Institute,
Serial Number,
Sensor ID,
Picture
Sensor
“GER 3700”
Sensor ID,
Name,
Description,
Manufacturer ID,
Type No,
No of Bands,
Wavelength at
each band
Calibration 3
Date,
Calibration Number,
Comments,
Measured wavelength
at each band
Calibration History
Manufacturer
“GER”
Manufacturer ID,
Name,
URL,
Short Name
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 46 of 161
Research Group
Members
A list of Users of this SPECCHIO database who have permission
to modify this Campaign. Each of these Users must be registered
with a SPECCHIO User account.
8.8 Metadata Attributes
For more information on the Metadata parameters supported by SPECCHIO please
refer to (Hüni and Kneubühler, 2007).
When Spectra are uploaded to a Campaign, the SPECCHIO upload process identifies
Metadata found in the Spectrum files and copies it into SPECCHIO Spectrum
Metadata fields for each Spectrum it uploads. Once uploaded, this Metadata can be
viewed, edited, removed or added to.
Similar Metadata Attributes are grouped into named sections to facilitate
management.
A few Metadata Attributes are permitted to take multiple values, such as Keywords or
Processing Group Attributes. The SPECCHIO’s Metadata Editor does not support
addition of more than a single value, but does support viewing them. Multiple values
for non-image Attributes can be added from Excel files. (See
Section 9.6)
The Metadata Attribute tables below indicate those Attributes which are permitted to
take multiple values. All others are limited to a single value.
Users cannot enter Metadata except into these already defined fields. If new
Metadata Attributes are required, they can generally be defined easily. Contact your
System Administrator with a detailed description of the Metadata field you need
added and a specification for the data that it can contain.
Also see
Section 11.1 List available Metadata Elements
for instructions for producing
a machine readable list of Spectrum-related Metadata Attributes.
A number of the Metadata Attributes below take an element of a dropdown list for
their value. The definition of these lists is either taken from user-managed SPECCHIO
tables, such as Sensor or Instrument lists, or the definition is parameterised within
the SPECCHIO database. Such parameterised lists can generally be changed. See
your SPECCHIO system administrator if you require a change in any of these lists.
The source of the dropdown list is explained for each relevant Metadata Attribute.
8.8.1 Associated Campaigns Group
Airborne Mission
ID
Multiple
permitted
[Alpha string] Mission ID of an airborne mission that is associated
with this spectrum
8.8.2 Campaign Details Group
Some formats for Spectrum files contain Campaign-related information, particularly
HDF formatted files. When Spectrum files of these formats are loaded, the
Campaign-related Metadata that they contain is loaded into these Attributes for each
Spectrum. This allows you to review it and, if necessary, manually transfer it to the
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 47 of 161
Campaign-related Metadata fields described in
Section
6.9 Campaign-related
Metadata.
Note The fields in this Group can store different values for each Spectrum in a Campaign.
It is not recommended to enter Campaign Metadata into these Spectrum-related
Metadata fields. Instead, use the Campaign Metadata fields described in
Section
6.9
Campaign-related Metadata
.
Agency
Code
[Alpha string] Agency identifier of entity involved in the sample
collection
Copied from the Agency Code field if it is set in the related input
Spectrum file when it is loaded.
Campaign
Name
[Alpha string] Further specification of a particular Campaign. Mainly
used where a SPECCHIO Campaign comprises several original
sampling campaigns
Copied from the Campaign Name field if it is set in the related
Spectrum file (particularly HDF files) when it is loaded.
Project ID
[Alpha string] Any project identification applying to this Campaign
Copied from the Project ID field if it is set in the related input
Spectrum file when it is loaded.
8.8.3 Data Link Group
These Metadata Attributes relate to links between spectra, establishing relationships
that help data processing, provenance and exploration.
Provenance
Data Link
Multiple
permitted
[Integer] Points to a spectrum used in calculating the current
spectrum
Reference
Data Link
Multiple
permitted
[Integer] Points to a reference spectrum taken by the same
instrument.
Target Data
Link
Multiple
permitted
[Integer] Points to an other spectrum taken by a different
instrument but of the same target or to a spectrum of the target if
the referencing spectrum is a reference spectrum
8.8.4 Data Portal Group
These Metadata Attributes relate to the Publishing of the Spectrum on the Australian
National Data Service (ANDS). See
Section 12 Publishing Data to ANDS
for more
information about the use of the Attributes in this Group.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 48 of 161
ANDS
Collection Key
Multiple
permitted
[Alpha string] The ANDS Collection Keys under which this
Spectrum has been Published.
This Metadata Attribute is only available when accessing a
SPECCHIO Server which is ANDS-enabled.
This Attribute supports multiple values, with a new additional value
written each time the Spectrum is part of a new Collection which is
Published to ANDS.
It is possible to edit this Metadata Attribute, but it is strongly
recommended not to.
ANDS
Collection
Name
Multiple
permitted
[Alpha string] The names of the ANDS collections in which this
Spectrum has been published.
This Metadata Attribute is only available when accessing a
SPECCHIO Server which is ANDS-enabled.
This Attribute supports multiple values, with a new additional value
written each time the Spectrum is part of a new Collection which is
Published to ANDS.
ANDS
Collection
Description
Multiple
permitted
[Alpha string] The descriptions of the ANDS collections in which
this Spectrum has been published.
This Metadata Attribute is only available when accessing a
SPECCHIO Server which is ANDS-enabled.
This Attribute supports multiple values, with a new additional value
written each time the Spectrum is part of a new Collection which is
Published to ANDS.
Data Usage
Policy
[Alpha string] String that defines the usage policy, for example
“Please cite XXX et al” or “Inform the principal investigator”
The value of this Attribute is copied to the Published Collection
when this Spectrum is published to ANDS.
FOR Code
Multiple
permitted
[Dropdown List] SPECCHIO supports selection from a small
relevant subset of the Field of Research codes defined by the
Australian Bureau of Statistics (Australian Research Council, 2008).
This Metadata Attribute is only available when accessing a
SPECCHIO Server which is ANDS-enabled.
The value of this Attribute is copied to the Published Collection
when this Spectrum is published to ANDS.
Digital Object
Identifier
[Alpha string] A digital object identifier (DOI). See
http://www.doi.org/ for further information about DOIs.
The value of this Attribute is copied to the Published Collection
when this Spectrum is published to ANDS.
8.8.5 Environmental Conditions Group
These Metadata describe the environmental conditions at the time the Spectrum
measurements were taken.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 49 of 161
Air Pressure
[hPa] Ambient air pressure
Ambient Temperature
[Degrees Celsius] Ambient air temperature
Atmospheric Water
Content
[Centimeter] Amount of water in the air column
Cloud Cover
[Percentage] Cloud cover as a percentage of the hemisphere
Relative Humidity
[Percentage] Relative air humidity
Sampling
Environment
[Dropdown list] Describes where data were spectrally
acquired, i.e. Field or laboratory
Weather Conditions
[Alpha string] Description of weather conditions
Wind Direction
[Degrees] Direction where wind is coming from
Enter a bearing, that is, clockwise from North.
Wind Speed
[Metres/second] Speed of the wind
8.8.6 General Group
This Metadata Group collects together information about the Spectrum’s file.
Acquisition Time
[Date/Time 24h] Time when Spectrum was measured.
Acquisition Time
(UTC)
[Date/Time 24h] UTC time when Spectrum was measured.
If the acquisition system’s time was not set to UTC, then set the
Acquisition Time (UTC)
using the
Compute UTC
function
described in section 11.1.1.
Acquisition Time
Approximated
[Boolean]
Check this box if the Acquisition time is not precisely known and
has been approximated.
Acquisition Time
(UTC)
Approximated
[Boolean]
Check this box if the computed time shift between the local
Acquisition time and UTC is not precisely known and has been
approximated.
Application
Domain
Defines application domains, which are used to control the
default metaparameter categories per application.
File Format
[Dropdown list] The format of the file from which this Spectrum
was uploaded.
This Attribute is always present and is generally set by
SPECCHIO when the Spectrum is read. If no data is available, it
should be set to
Nil
.
Although this value can be changed, it is not advised.
(All Spectra are stored internally in the database in SPECCHIO’s
internal Spectrum format.)
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 50 of 161
Raw Data Format
[Alpha string] Information on where data originated
File Name
[Alpha string] The name of the file from which this Spectrum
was uploaded. In the case of input file formats which support
multiple Spectra per file, such as XLS or TXT, the name of the
Spectrum within the file is appended to this file name.
Although this value can be changed, it is not advised. Changing
it may result in duplicate Spectra when a subsequent Spectrum
upload is performed for this Campaign.
File Version
[Alpha string] Version of the input file format
Loading Time
[Date/Time 24h] The system time when the Spectrum was
loaded into database
This value is automatically set to the SPECCHIO server’s system
time when the Spectrum is uploaded to SPECCHIO. Note that
this may not be the same as the client’s system time.
The
Special functions/Correct local time to UTC
function does
not operate on this time.
Measurement
Unit
[Dropdown list] Describes the nature of the measurement -
select from:
Reflectance
,
Radiance
,
Transmission
,
Absorbance
,
DNs
,
Irradiance
,
Mueller10
,
Mueller20
and
Wavelength
.
This Attribute is mandatory and cannot be deleted.
This Attribute is set automatically depending on the file format
being read. However, for some formats it is not known and is
therefore set to
Nil
. If it is not set, or is set incorrectly, plotting
may not be correct.
Spectrum Number
[Integer] Some recording devices and file formats apply a
number which is recorded in the Spectrum file. If a Spectrum is
loaded from such a file format, the number will be placed into
this Metadata field.
If the file does not define a Spectrum Number, this Metadata
Attribute remains unset. (Some file loaders may generate an
artificial number. This behaviour is deprecated and should not be
relied upon.)
Note that this number is not the Spectrum ID and usually does
not hold the same value as the Spectrum ID. The integer
numbers which are displayed when Spectra are selected in the
Query Builder or when a Spectrum is highlighted in the Hierarchy
Viewer are Spectrum IDs.
Spectrum Numbers can be changed in the Metadata Editor (but
probably shouldn’t be). Spectrum IDs cannot be changed.
8.8.7 Generic Target Properties Group
The Generic Target Properties Group holds data related to a Target or Sample.
In this context, a Sample is regarded as a sample taken from the Target and
measured. Multiple Samples can be taken from any Target and measured
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 51 of 161
independently. They may have been taken from the Target in the field, or the Target
may have been transported to a separate measuring facility and the Sample taken
from the Target at that facility.
Typically, for each Campaign, there should be PDF uploaded to the Experimental
Design Metadata item in the PDFs Metadata Group to explain the Target and Sample
structure and the reasons for that structure.
Basic Target Type
Multiple permitted
[Dropdown list]
The basic target types are defined by the
Basic Target Type
taxonomy. These target definitions have been designed for
small mapping units covering the typical pixel size of airborne
imaging sensors, typically a few square metres. It is used
generally where the CORINE system is not precise enough. This
taxonomy is based on various land classification systems and
has been refined within the APEX airborne imaging
spectrometer spectral ground control work. Required additions
to the taxonomy should be communicated to the SPECCHIO
development team.
CORINE Land
Cover
[Dropdown list]
To define basic target types, either the taxonomy from the
above list or from this dropdown list can be used as a
reference.
Sample Collection
Date
[Date/Time 24h] Time when the original sample was collected
in the field. This will generally be a different from the time it
was measured. The time zone is not specified.
Note that the
Special functions/Correct local time to UTC
function does not operate on this time.
Sample Number
[Alpha string] Identification of the sample as collected
This will not in general match the Spectrum Number or
Spectrum ID. A Sample Number may be common to a variety of
Spectra.
Site ID
[Alpha string] Identification of the site from which the sample
was collected
Target Description
[Alpha string] Description of the Target
Target ID
[Alpha string] Identification of the Target that was collected
Target/Reference
Designator
Defines if the measured object was a reference surface (e.g.
white reference) or a target that is then compared to a
reference
Tram Run
[alpha string] Refers to the number of a tram run within a
sequence of runs
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 52 of 161
8.8.8 Geochemistry
Refer to section
13.1 List available Metadata Elements
to get an up-to-date list of this large
collection of metaparameters.
8.8.9 Illumination Group
Polarization
[Alpha string] Description of illumination polarisation, typically
“Horizontal” or “Vertical”, but other comments may be used as
well.
Polarization
Direction
[Degrees] Polarisation direction, usually measured clockwise from
vertical
8.8.10 Instrument Group
This Group of Attributes describes the specific spectroradiometric Instrument
which was used to measure this Spectrum.
Instrument
[Dropdown list] The name of the specific Instrument used to take
these measurements
The drop down list is a list of Instruments which is created and
maintained using the
Data maintenance/Instrument admin.
function.
This Attribute is always present and cannot be deleted. It should be
set if known. If no data is available, set it to
Nil
.
This Attribute is generally set when Spectra are loaded, but should
be checked to ensure that SPECCHIO determined it correctly.
Sensor
[Dropdown list] The name of the Sensor used in the specific
Instrument used to take these measurements
The dropdown list is a list of Sensors which is created and
maintained using the
Data maintenance/Load sensor definition
function.
If the Instrument Attribute above is set, this Sensor Attribute is
ignored and the value stored in the Instrument record is used
instead.
This Attribute is always present and cannot be deleted. If not
required, it can be set to
Nil
.
This Attribute is generally set when Spectra are loaded, but should
be checked to ensure that SPECCHIO has determined it correctly.
Calibration
Number
[Integer] Calibration number of the most recent calibration for the
Instrument at the time the measurements were taken
Generally, the Acquisition Time is compared against the Calibration
Time in the Instrument Calibration List to select the most
appropriate calibration to use. This value is used if there is
insufficient information available to use the dates.
For some Spectrum file formats, this value is set automatically on
Spectrum loading. Otherwise, it can be set manually.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 53 of 161
Calibration
(internal ID)
[Dropdown list]
This ID refers to an internal instrument calibration (typically
wavelength) of the instrument. It is set by SPECCHIO automatically
and should not be changed manually.
Extended
Instrument
Name
[Alpha string] Instrument name
This field can be used to further describe the Instrument used to
capture this Spectrum, especially if the Instrument is not shown in
the dropdown list of Instruments.
Instrument
Serial Number
[Alpha string] Serial number of the Instrument
As for Extended Instrument Name above, enter this value if the
specific Instrument is not shown in the dropdown lists.
8.8.11 Instrument Settings Group
These attributes describe the settings of the Instrument at the time the Spectrum
was acquired.
For some Instruments and file formats, the following Instrument settings are read
from the Spectrum input file and set into the Spectrum Metadata. For others, these
values must be recorded manually.
Automatic Dark
Current Correction
[Alpha string] Indicates if Dark current has been
compensated for by the Instrument
If SPECCHIO loads this value from the input Spectrum file,
it will set one of the values “ON” or “OFF”.
Capturing Software
Name
[Alpha string] Name of the Instrument operation software
or data capture software
Capturing Software
Version
[Alpha string] Version of the Instrument operation
software or data capture software
Gain SWIR1
[Integer] ASD specific gain
Gain SWIR2
[Integer] ASD specific gain
Instrument Channel
[Alpha string] Channel designation for multi-channel
instruments, e.g. irradiance and reflected radiance
channels
Instrument
Temperature
[Degrees Celsius] Internal temperature of the instrument
Integration Time
[ms] Integration time
Number of Internal
Scans
[Integer] Number of Spectra recorded internally and
averaged over by the Instrument
Offset SWIR1
[Integer] ASD specific offset
Offset SWIR2
[Integer] ASD specific offset
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 54 of 161
Time since last DC
[Integer] Seconds since last dark current measurement
Not currently filled, even if it’s in the input file.
UniSpec Spectral
Resampling
[Alpha string] UniSpec specific setting; indicates if the
resampling was ON or OFF
If SPECCHIO loads this value from the input Spectrum file,
it will set one of the values “ON” or “OFF”.
8.8.12 Instrumentation Group
These attributes describe the Instrumentation used at the time the Spectrum was
acquired.
For some file formats (particularly HDF files), the following Instrumentation settings
are read from the Spectrum input file and set into the Spectrum Metadata. For other
formats, it must be set manually.
Azimuth
Sensor Type
[Alpha string] Sensor name used to measure the azimuth on a
goniometric facility
Contact Probe
[Dropdown list] Contact probe instrumentation name, usually
includes a light source and provides optical input to the
spectrometer.
Goniometer
[Dropdown list] Goniometer name
Illumination
Sources
[Dropdown list] Illumination source name, e.g. Sun or Hg lamp
Integrating
Sphere
[Dropdown list] Integrating sphere name
Light Source
Parameters
[Alpha string] Settings of artificial light source
Reference
[Dropdown list] Selected from the list of References defined in the
SPECCHIO database.
This Attribute is mandatory and cannot be deleted. If it is not
relevant or no data is available, set it to
Nil
.
Reference
Panel Levelled
[Boolean] Indicates if the reference panel was mounted on a stable
support and carefully levelled
- If the reference panel was not levelled, add this attribute
without checking the box
- If the reference panel was levelled, add this attribute and
check the box
The implications of not levelling the reference panel are described
in (Hueni et al., 2017).
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 55 of 161
White
Reference
Panel Name
[Alpha string] Name of the white Reference Panel.
Users do not have permission to add to SPECCHIO’s Reference
Panel table. When a Spectrum is being loaded and the Reference
Panel it refers to is not in SPECCHIO’s Reference Panel table, then
the name of the Reference Panel will be written into this field. This
provides the information needed for the user to resolve the
Reference Panel link information after the Spectrum load is
completed.
8.8.13 Keywords Group
Keywords can be used to search for particular Spectra in the SPECCHIO’s Spectrum
Query function.
Keyword
Multiple
permitted
[Alpha string] Freely chosen keyword that describes a Spectrum or a
Spectral collection
Take care with spelling and avoid leading or trailing spaces to avoid
confusion when searching. Keyword searching is not case sensitive.
8.8.14 Location Group
This Group collects information about the capturing location. WGS84 (World Geodetic
System of 1984) should be used as the reference Geoid for these measurements.
With Release 3.3.0.0, spatial geometries support based on the MySQL Spatial
Extension was introduced, enabling spatial queries and new geometry types: point,
polyline and polygon. Latitude and Longitude of the spatial position can be inserted
for each measurement point. Depending on the selected Spatial Sampling Scheme
(see section
8.8.22
), start and end point of e.g. a transect can be defined.
Altitude
[Metres] Altitude of spatial position
This attribute can be generated by the function “Augment altitude”
from the “Compute” menu, see section
9.6.2
. Mandatory attribute
information is latitude and longitude.
Depth
[Metres] Distance from the Earth surface where sample was taken
Latitude
(deprecated)
[Degrees] Latitude of spatial position
Northern hemisphere coordinates are positive, southern
hemisphere coordinates are negative. Coordinates are entered as
floating point numbers for degrees and fractions of degrees.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 56 of 161
Longitude
(deprecated)
[Degrees] Longitude of spatial position
Locations west of Greenwich are negative, east of Greenwich are
positive. (That is, longitudes in Switzerland or Australia are
positive.) Coordinates are entered as floating point numbers for
degrees and fractions of degrees.
If the longitude values are inserted as positive values instead of as
negative values, the function “Switch longitude E-W” from the
“Compute” menu can be used to convert all values of a selected
node. This function can save considerable time if multiple
coordinate values were inserted automatically from XLS (
see
section 7.6
for
this option).
Note: older versions of SPECCHIO did not follow this common
definition, but the used to define West of Greenwich as positive.
Position
Approximated
[Boolean] Indicates if the spatial position was not properly
recorded and had to be approximated by other means.
This attribute is only to be used (with checking the box) if the
spatial position (latitude, longitude) had to be approximated.
Location Name
[Alpha string] Usual name of the spatial position where the
sampling took place
State
[Alpha string] Identifier of the State in which the observations
were made.
To avoid confusion in searching later, it is suggested to use the
standard abbreviation for the state – for example, in Australia use
NSW, Qld, Vic. Do not use periods, leading, embedded or trailing
spaces in the field. Searching is not case sensitive.
Spatial Extent
Spatial extent of a spectrum in 2D space as a polygon defined by
vertices given as latitude and longitude. Stored as spatial field
using the MySQL Spatial Extension.
Spatial Position
Spatial location of a spectrum in 2D space as latitude and
longitude. Stored as spatial field using the MySQL Spatial
Extension.
Spatial
Transect
Spatial extent of a spectrum in 2D space as a polyline defined
by vertices given as latitude and longitude. Stored as spatial field
using the MySQL Spatial Extension.
Waypoint ID
[Alpha string] Name or ID of a waypoint
8.8.15 Names Group
This Group collects the possible names for the species or objects in the Spectrum
sample.
Common
[Alpha string] Common name given to target (applies particularly to
plants)
ENVI
[Alpha string] Name extracted from ENVI header
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 57 of 161
Header
When ENVI Files are loaded (see
Section
7.1.9 (ENVI Spectral Library
Files), the content of the Spectrum Names tag is copied here if it is set
in the input file.
Latin
[Alpha string] Latin (or scientific) name given to target
8.8.16 Optics Group
This Group permits describing specific optics which are attached to the recording
Instrument.
FOV
[Integer degrees] Field of view of the Instrument used for the sampling.
This may vary as a result of using a special foreoptic
Optics
Name
[Alpha string] Designator given to the optics attached to the Instrument
8.8.17 PDFs Group
SPECCHIO supports uploading of up to two PDF files for each Spectrum. To optimise
storage in the database, select all Spectra that a PDF file applies to and upload just
once. Depending on your internet access speed, uploading large PDF files may be
slow.
Experimental
Design
[Attached PDF] Document describing the experimental design
and the scientific reasons for doing so
Field Protocol
[Attached PDF] Copy of the field protocol filled in during data
capture
8.8.18 Personnel Group
Investigator
Multiple
permitted
[Alpha string] Investigator’s name
It is not necessarily related to any SPECCHIO User’s name and is
not checked against the list of Users’ names. Nor is this field
checked against the Investigator field provided in the Campaign
Metadata.
This Metadata Attribute is supported to receive the Investigator
name which is provided in some Spectrum file formats – for
example, some HDF files.
Users are generally encouraged to rely on the Campaign Metadata
(either the Investigator, Research Group Members or Description
fields) for this function.
8.8.19 Pictures Group
SPECCHIO supports uploading of up to four image types for each Spectrum. To
optimise storage in the database, select all Spectra, ideally by selecting a storage
hierarchy, to which an image applies to and upload it just once. Images are
automatically rescaled to conserve storage space and improve data loading speeds.
For such improved storage and performance, pictures are automatically resampled to
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 58 of 161
a size of 400 pixels (in width or height) while the aspect of width and height is
retained.
Warning Do not use SPECCHIO as a photo repository; the automatic downscaling will degrade
your photo resolution; it is only meant to document the spectral sampling conditions
and target state.
The supported uploadable image formats may vary between PC, Linux and Mac.
JPEG is supported on all operating systems. GIF, PNG or TIFF may also be supported
on some users’ computers. When uploaded, all image formats are converted and
stored as JPEG images. These images can be viewed directly on the Metadata
Editor screen.
Sampling
Environment Picture
Multiple permitted
[Uploaded image file] Picture showing the general sampling
environment in vicinity of the target
Sampling Setup
Picture
Multiple permitted
[Uploaded image file] Picture showing the positions of
Sensor and illumination in relation to the target
Sky Picture
Multiple permitted
[Uploaded image file] Picture of the sky, ideally taken
hemispherically
Target Picture
Multiple permitted
[Uploaded image file] Picture showing the target
8.8.20 Processing Group
These Attributes describe processing which has been performed on the Spectrum.
These Attributes are added or set when that processing is done. This processing may
be done by SPECCHIO, or it may be processing done by some other function
connecting to the SPECCHIO database. This may be a Matlab program, a Java
program, or a program written using any other tool or language.
While it is possible, generally these Attributes should not be edited manually.
Data Ingestion
Notes
Multiple
permitted
[Alpha string] Notes produced by the data ingestion module during
data loading into SPECCHIO
Multiple Data Ingestion Notes may be added to each Spectrum.
DC Flag
[Integer] Designates this Spectrum as a dark current Spectrum
The Metadata Attribute’s value should be either 0 indicating False,
or 1 indicating True. If the Metadata Attribute does not exist, this
implies False.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 59 of 161
Garbage Flag
[Integer] Designates this Spectrum is not intended for inclusion in
the main experimental evaluation
The Metadata Attribute’s value should be either 0 indicating False,
or 1 indicating True. If the Metadata Attribute does not exist, this
implies False.
If you create a node named “Garbage”, all Spectra in the sub-tree
below this node will have this flag set to 1 when they are loaded.
The test for the node name is case sensitive.
This flag can be set to exclude specific Spectra from processing.
However, negative tests cannot be performed by the Query
Builder, so this is intended for automated processing control. For
manual management, Spectra which are not intended for
processing should be placed in specific sub-folders.
There are currently no SPECCHIO processes which test this flag. It
is generally intended for use by non-SPECCHIO processes, such as
processes written using Matlab.
Processing
Algorithm
Multiple
permitted
[Alpha string] Description of processing algorithm applied to
Spectrum
Multiple Processing Algorithm fields may be added to each
Spectrum.
Each process which modifies a Spectrum should set an informative
message into a new Metadata Attribute of this type for each
Spectrum it modifies.
Processing
Algorithm Note
[Alpha string] Processing notes are produced by the algorithm
Processing
Level
[Floating point] Numeric designator of a specific processing level.
This Metadata Attribute is provided to permit non-SPECCHIO
programmatic processes to keep track of the level or amount of
processing which they have performed on the related Spectrum.
Generally, the levels are defined as:
0.0 DN: Digital numbers
1.0 Radiance Spectra, as provided by the instrument
1.x Radiance Spectra, corrected
2.0 Reflectance Spectra, as provided by the instrument
2.x Reflectance Spectra, corrected
Processing
Module
Multiple
permitted
[Alpha string] Name of processing module applied to Spectrum
Multiple Processing Module Attribute values may be added to each
Spectrum.
This Metadata Attribute is provided to permit non-SPECCHIO
programmatic processes to keep track of the level or amount of
processing which they have performed on the related Spectrum.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 60 of 161
Source File
Multiple
permitted
[Alpha string] Name of file that provided the original data
This applies if data is processed outside of SPECCHIO.
This Metadata Attribute is provided to permit non-SPECCHIO
programmatic processes to keep track of the level or amount of
processing which they have performed on the related Spectrum.
SPECCHIO does not use this value.
Solar Angle
Computation
Notes produced by the SPECCHIO solar angle computation routine.
Time Shift
Multiple
permitted
[Alpha string] This records the time shift processing that was
applied to the Spectrum using the Special functions/Correct local
time to UTC operation. See section
11.1.1
for more information on
changing the times.
The Time Shift field will be overwritten if the Time Shift function is
applied multiple times.
UTC Time
Computation
[Alpha string] Note is produced by the SPECCHIO UTC
computation routine
8.8.21 Sampling
Numbers in this Group can all be entered as floating point numbers.
Beam
Geometry
[Dropdown list] SPECCHIO supports the nine fundamental beam
geometry types describing the incoming and reflected beams.
For further information about the beam geometry types, please
refer to (Schaepman-Strub et al., 2006, Schaepman-Strub et al.,
2009) and Figure 23.
Illumination
Azimuth
[Degrees] Absolute illumination source azimuth angle measured
clockwise from geographic North.
This attribute is generated by the special function “Sun Angle
Calculation”, based on location attributes (longitude, latitude,
altitude) and UTC acquisition time.
Illumination
Distance
[Metres] Distance between the illumination source and target (for
artificial illumination)
Illumination
Zenith
[Degrees] Illumination source zenith
angle measured from nadir.
This angle will always be between 0
and 90 degrees. If the Illumination is
directly over the target, the value
will be zero.
θ
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 61 of 161
This attribute is generated by the special function “Sun Angle
Calculation”, based on location attributes (longitude, latitude,
altitude) and UTC acquisition time.
Measurement
Support Area
[Square metres] Area of the measurement support, i.e. the areas of
the projected field of view.
This attribute is generated by the function “Compute measurement
support” from the “Compute” menu, see
section 9.6.1
.
Measurement
Support
Radius
[Metres] Radius of the measurement support, i.e. the radius of the
projected field of view.
This attribute is generated by the function “Compute measurement
support” from the “Compute” menu.
Measurement
Support
Major Axis
[Metres] Major axis of the measurement support, i.e. the major axis
of the elliptical projected field of view.
This attribute is generated by the function “Compute measurement
support” from the “Compute” menu.
Measurement
Support
Minor Axis
[Metres] Minor axis of the measurement support, i.e. the minor axis
of the elliptical projected field of view.
This attribute is generated by the function “Compute measurement
support” from the “Compute” menu.
Sensor
Azimuth
[Degrees] Sensor azimuth angle
relative to the illumination angle
The angle is measured clockwise
from the Illumination Azimuth. A
negative angle implies an anti-
clockwise measurement direction.
In this example, the Sensor Azimuth
is approximately 300°.
Sensor
Distance
[Metres] Distance of Sensor from the target
Sensor Zenith
[Degrees] Sensor zenith angle
measured from nadir
This angle will always be between 0
and 90 degrees. If the Sensor is
directly over the target, the value
will be zero.
θ
θ
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 62 of 161
Figure 23: The nine beam geometry cases
8.8.22 Sampling Design Group
Spatial Sampling
Scheme
[Dropdown list] SPECCHIO supports these Spatial Sampling
Schemes:
- Cross
- Square
- Transect
- Sweep
Vegetation
Sampling Scale
[Dropdown list] SPECCHIO supports these Sampling Scales:
Leaf: Measurements at single leaf scale
Plant: Measurements at single plant scale
Stand: Measurements at stand scale, i.e. encompassing several
plants
8.8.23 Scientific References Group
Citation
Multiple
permitted
[Alpha string] Publication to be cited when using these Spectral
data
Publication
[Alpha string] Publication relevant to these Spectral data
The above reflectance and reflectance factor definitions lead
to the following special cases:
•ω
i
or ω
r
are omitted when either is zero (directional quantities).
•If 0 b(ω
i
or ω
r
)b2π, then θ,ϕdescribe the direction of the
center axis of the cone (e.g. the line from a sensor to the
center of its ground field of view—conical quantities).
•If ω
i
=2π, the angles θ
i
,ϕ
i
indicate the direction of the
incoming direct radiation (e.g., the position of the sun). For
remote sensing applications, it is often useful to separate the
natural incoming radiation into a direct (neglecting the sun's
size) and hemispherical diffuse part. One may also include a
terrain reflected diffuse component that is calculated with a
topographic radiation model such as TOPORAD (Dozier,
1980). Consequently, the preferred notation for the geometry
of the incoming radiation under ambient illumination
conditions is θ
i
,ϕ
i
,2π. Note that in this case, θ
i
,ϕ
i
describe
the position of the sun and not the center of the cone (2π),
except if the sun's position is at nadir. In the case of an
isotropic diffuse irradiance field, without any direct
irradiance component (closest approximated in the case of
an optically thick cloud deck), θ
i
,ϕ
i
are omitted.
•If ω
r
=2π,θ
r
and ϕ
r
are omitted.
Finally, according to Nicodemus et al. (1977), the angular
characteristics of the incoming radiance are named first in the
term, followed by the angular characteristics of the reflected
radiance. This leads to the attributes of radiance and reflectance
quantities as illustrated in Table 2.
It should be noted that the nine standard reflectance terms
defined by Nicodemus et al. (1977) “are applicable only to
situations with uniform and isotropic radiation throughout the
incident beam of radiation”. They then state that “If this is not
true, then one must refer to the more general expressions”. This
implies that any significant change to the nine reflectance
concepts when the incident radiance is anisotropic lies in the
mathematical expression used in their definition. Based on this
implication, Martonchik et al. (2000) adapted the terminology
and reflectance names to the remote sensing case, which
involves direct and diffuse sky illumination. In the following,
we give the mathematical description of the most commonly
used quantities in remote sensing, thus the general expressions
for non-isotropic incident radiation. When applicable, we
simplify the expression for the special case of isotropic incident
radiation. The nine possible combinations of beam geometries
of the incident and reflected radiant fluxes are indicated as
Cases 1 to 9, corresponding to the illustrations in Table 2.
2.2.1. The bidirectional reflectance distribution function
(BRDF)—Case 1
The bidirectional reflectance distribution function
(BRDF) describes the scattering of a parallel beam of incident
light from one direction in the hemisphere into another direction
in the hemisphere. The term BRDF was first used in the
literature in the early 1960s (Nicodemus, 1965). Being
expressed as the ratio of infinitesimal quantities, it cannot be
directly measured (Nicodemus et al., 1977). The BRDF
describes the intrinsic reflectance properties of a surface and
Table 2
Relation of incoming and reflected radiance terminology used to describe reflectance quantities
I
ncoming/Reflected Directional Conical
D
irectional Bidirectional
CASE 1
Directional–conical
CASE 2
Conical
H
emispherical
Conical–directional
CASE 4
Hemispherical–directional
CASE 7
Hemispherical–conical
CASE 8
Biconical
CASE 5
Hemispherical
Directional–hemispherical
CASE 3
Conical–hemispherical
CASE 6
Bihemispherical
CASE 9
The labeling with ‘Case’corresponds to the nomenclature of Nicodemus et al. (1977). Grey fields correspond to measurable quantities (Cases 5, 8), the others
(Cases 1–4, 6, 7, 9) denote conceptual quantities. Please refer to the text for the explanation on measurable and conceptual quantities.
30 G. Schaepman-Strub et al. / Remote Sensing of Environment 103 (2006) 27–42
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 63 of 161
Multiple
permitted
8.8.24 Soil Parameters Group
The Attributes in this Group have been provided to allow specification of the soil type
according to the Australian Soil Classification.
The table below shows the list of Metadata Attributes provided by SPECCHIO and the
type of data they hold. Please refer to the documents defining that classification
system to see a detailed explanation of the meaning of each item.
At the time of writing this document, an explanation of this classification system can
be found at (CSIRO, 2011).
Australian Soil
Classification
parameters
ASC Order
[Alpha string]
ASC Sub-Order
[Alpha string]
Sampling Lower Depth
[Metres]
Sampling Upper Depth
[Metres]
Horizon Desig. Master
[Alpha string]
Horizon Desig. Sub.
Division
[Alpha string]
Horizon Name
[Alpha string]
Horizon Number
[Integer]
Measurements
Each of the following measurements has two Metadata
Attributes – the measurement itself and a description
field which holds an alpha string describing the method
used to determine that measurement. The table below
lists the expected measurement unit, and gives a brief
explanation where the name is not self-explanatory.
Air Dry Water Content
[g/g] Water content of air
dried soil
Available P
[%]!Available phosphorus
Bulk Density
[g/cm²]
Carbon
[%]
CEC
[cmol/Kg]
Clay
[%]
Coarse Sand
[%]
Electrical Conductivity
[Deci-Siemens/Metre]
Exchangeable Acidity
[cmol/Kg]
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 64 of 161
Exchangeable Ca
[cmol/Kg]
Exchangeable K
[cmol/Kg]
Exchangeable Mg
[cmol/Kg]
Exchangeable Na
[cmol/Kg]
Extractable Fe
[mg/Kg]
Fine Sand
[%]
Geothite
[%]
Gibsite
[%]
Hematite
[%]
Illite
[%]
Kaolinite
[%]
Montmorillonite
[%]
pH CaCl2
[pH]
pH Water
[pH]
Silt
[%]
Smectite
[%]
Total K
[%]
Total N
[%]
Total P
[%]
Water Content 0_1 Bar
[m3 / m3]!
Water Holding Capacity 15
Bar
[%]
8.8.25 Vegetation Biophysical Variables
% Crown Cover
[Percentage] Tree Crown Cover Percentage
Antheraxanthin
[ugrams/cm2] Carotenoid and xanthophyll pigment
Approx. Crown
Diameter
[Metres] Estimated tree crown diameter
B-Carotene
[ugrams/cm2] Carotenoid pigment
Chlorophyll Content
[ugrams/ cm²] Chlorophyll content of the collected
vegetation sample
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 65 of 161
Crown Relative
Position
[Dropdown list] FPMRIS (DELWP) SOP 13 Measuring a
Large Tree Plot
DBH
[Metres] Diameter at breast height
Dry Weight
[Grams] Dry weight of the collected vegetation sample
EPS
Epoxidation state of xanthophyll cycle pigments:
(V+0.5*A)/(V+A+Z)
Height
[Metres] Height of vegetation
Leaf Area
[cm²] Leaf area of the sampled vegetation
Lutein
[ugrams/cm2] Carotenoid and xanthophyll pigment
Neoxanthin
[ugrams/cm2] Carotenoid and xanthophyll pigment
Specific Leaf Area
[cm²/gram] Calculated by: LeafArea[cm2]/DryMass[g]
Violaxanthin
[ugrams/cm2] Carotenoid and xanthophyll pigment
Water Content
[Grams/ cm²] Water content of the collected vegetation
sample
Wet Weight
[Grams] Wet weight of the collected vegetation sample
Zeaxanthin
[ugrams/cm2] Carotenoid and xanthophyll pigment
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 66 of 161
9 Data Input
This section describes the functionalities of SPECCHIO’s Main Menu
Data Input
. You
will find information on how to load and edit data:
9.1 Overview of SPECCHIO Data Loading
The process of loading data into SPECCHIO involves a
number of steps. Depending on your acquisition
device, some of these steps may not be needed
because the information is recorded by your device
and uploaded with the Spectral data.
Create a Campaign
This establishes a name for your Campaign and sets up structures in
the SPECCHIO database to receive the Campaign Spectra and other
information. See Section 7.2 Creating a new Campaign.
Load Campaign
Data
This examines the format of the Spectral data on your hard disk,
parses its content from your hard disk and inserts into the SPECCHIO
database. See Section 7.3 Loading Campaign Spectral Data.
Calculate
Acquisition Times
(UTC)
SPECCHIO expects Acquisition Times to be UTC, but many devices
record a local time. SPECCHIO provides a function to calculate UTC
based on local times. See Section 11.1.1 (Compute UTC).
Add Target-
Reference Links
For acquisition devices which do not put this information into the
Spectral files, you will need to enter this information manually. See
section 10.1 and 10.2.
Complete Metadata
upload
The Spectral Metadata that was not included in the Spectral files
must be uploaded, either from an Excel file or manually entered
using SPECCHIO’s Metadata Editor. Refer to sections 9.5.4 (inserting
metadata manually) and 9.7 (uploading from Excel files).
Calculate additional
Metadata
Some Metadata values can be computed using functions of the
Metadata Editor and SPECCHIO special functions. Refer to sections
9.6 and 11.1 respectively for further information on data
requirements of these options.
After these steps, your Campaign’s Spectral data is complete and ready for use,
exporting or publishing to ANDS.
9.2 Creating a new Campaign
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 67 of 161
• From the Main menu, select
Data Input
-
Create new campaign
. This
brings up the new Campaign dialog:
Figure 24: New Campaign dialog
• Enter the name you have selected for this Campaign. The maximum
length is 45 characters.
• To set the
Main directory
field, click Browse to display a folder tree.
The
Main directory
path is a file system pathname pointing to the folder that
contains all hierarchies and Spectra for this Campaign. When you load Spectral
data, it will be loaded from this folder. Navigate to the required path on your
hard disk.
• Click Create to create the new empty Campaign in the database.
• A message box will appear once the Campaign has been successfully
created. Click OK to close it.
• Click Cancel to close the Campaign creation dialog.
Note It is not advisable to have two Campaigns with the same name. There is no check for
the existence of a Campaign of the same name, so please check the names of the
existing Campaigns before you create your own in order to avoid multiple Campaigns
with identical or confusing names.
The Campaign is now ready to receive uploads of Spectral data.
It is also possible to use the Create and Load button to combine the operations of
creating the Campaign and uploading the data. Read the next section before using
this combined operation.
9.3 Loading Campaign Spectral Data
All Spectral data in the sub-folder tree at the disk location you specify will be loaded
from your computer to the database as a single operation.
On your local disk drive, prepare a sub-folder structure which reflects the hierarchy
you require for your data in SPECCHIO and put your Spectrum data files into this
structure. See
Section 6.1 Campaign Hierarchy Structure
for more information on
designing this structure.
The entire sub-folder tree must contain only Spectrum files, and each sub-folder
within the tree must contain only files of one Spectrum file format.
When you start the Spectrum load process, SPECCHIO will process all sub-folders
and files within this sub-folder tree and create a matching structure for your
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 68 of 161
Campaign in the database. For each sub-folder, it will use the various Spectrum File
loaders listed in
Section 6.7 Supported Input Spectrum File Formats
until one reports
that it can read a file in the sub-folder. It then processes all files in that sub-folder
using that file loader. If any file cannot be read, it will report an error giving the
name of the file that failed. Correct the offending file (either by removing it if it is not
a Spectrum file, sorting it into a sub-folder with similarly formatted files, or finding an
uncorrupted version of the file). Then restart the upload process. Those Spectra
which have already been successfully uploaded will not be processed a second time.
To upload Campaign Spectral data:
• From the Main menu, select
Data Input - Load campaign data
to display
the following dialog:
Figure 25: Load Spectral Data dialog
• From the list of
Campaign names
in the dropdown box, select the name of
the Campaign into which you want to load data.
• Highlight the
Path
name which contains the Campaign data sub-folder
tree on your computer. If the path name is not in the list, click New
Path, navigate to the path on your computer and select it. It will now be
highlighted.
• Click Load. The dialog box closes and the loading progress is shown in
the right panel of the Main Window:
• A message box will appear once all data has been loaded to the system.
Do not perform other operations on this Campaign until the load has
completed.
• Click OK to clear the message box.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 69 of 161
Note After uploading your Spectral data to SPECCHIO, do not change the sub-folder
structure on the hard disk that you set up and used. You will need this again if you
wish to upload further Spectral data to this Campaign in the future.
9.3.1 Instrument and Sensor selection
SPECCHIO searches the existing Sensor and Instrument definitions in the database
to find a match for the Spectrum data loaded. It sets the Instrument and Sensor
Metadata Attributes for the Spectrum to the one it chooses.
If SPECCHIO does not find a match in its Instrument and Sensor tables, it will create
an Instrument record and set the Spectrum’s Instrument Metadata Attribute to refer
to it. The Sensor Metadata Attribute will be filled with automatically generated sensor
information if no matching sensor is found.
Note After uploading Spectra, always use the Metadata Editor to check the settings of the
Instrument and Sensor Type which SPECCHIO has tried to determine from the input
Spectra characteristics. If they are not set or not correct, use the Metadata Editor to
correct them.
See
Section 9.5.1 Metadata
for more information on the Metadata Editor.
9.3.2 Reference Panel selection
SPECCHIO searches its Reference Panel table and attempts to find a match for any
Reference Panel specified in the input Spectrum.
If SPECCHIO does not find a match for the Reference Panel in its tables, it will set
the Reference Panel Metadata Attribute value to
Nil
, and attempt to store the
Reference Panel name in the
White Reference Panel Name
Metadata Attribute.
Note After uploading Spectra, use the Metadata Editor to check the setting of the
Reference Panel which SPECCHIO has tried to determine from the input Spectra. If it
is not set or not correct, use the Metadata Editor to correct it.
See
9.5.1 Metadata
for more information on the Metadata Editor.
9.3.3 Handling Spectrum upload errors
Warning If the message box advising successful completion does not appear, the upload did
not complete normally. Check the count of uploaded Spectra to determine how much
was completed. See below for information on recovering from upload errors.
If a folder contains unknown file types, mixed file types or files with invalid data, the
loading process will try and skip these files. In the case of errors within files of
known file format, a dialog box will generally appear, indicating the offending file and
the cause of the problem.
You should check the cause of the problem, rectify it and attempt the upload again.
Spectra which successfully uploaded the first time will not be uploaded again.
9.4 Loading Additional Spectral Data
If further Spectral data becomes available for your Campaign, it is possible to
conveniently add it to your existing SPECCHIO Campaign.
There are a number of different scenarios which may apply.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 70 of 161
9.4.1 Uploading Additional Spectral Data from the Same Computer
In this case, add the new Spectral data to the existing sub-folder structure on your
computer’s disk from which you uploaded the original Spectral data.
• Do not add any files other than Spectral data and ensure that all files within each
sub-folder have the same file format.
• Do not change the file names of any of the original files.
• Do not change or rename the sub-folders, but you may add new sub-folders.
To upload the new Spectral data, follow the process described in
Section 7.39.3
Loading Campaign Spectral Data
Only the new data will be loaded.
Note Additional Spectra can be added to XLS files and will be added when re-invoking the
loading process.
After uploading new data you may need to set the Metadata for the new data.
Depending on your Campaign Hierarchy design, the new data may be interspersed
with your existing data. Take care not to delete or corrupt the Metadata on your
existing data by ensuring that you select only your new Spectra when you update the
Metadata.
Note Existing metadata stored at hierarchy level is automatically inherited by spectra
added beneath these hierarchies.
Warning This particularly applies to UTC time changes, where applying the same time change
a second time will add or subtract the time difference again, yielding an incorrect
result.
9.4.2 Uploading Additional Spectral Data from a Second Computer
SPECCHIO permits multiple Paths to be stored against each Campaign. This permits
the same data to be managed and augmented from multiple computers.
To upload from a second computer:
• On the second computer, make an exact copy of the entire Campaign
folder tree as it was uploaded from the first computer. It does not have to
be in the same location as it was on the first computer, but the sub-folder
structure, sub-folder names and file names must be identical.
• Add the new hierarchy sub-folders and files into the existing sub-folder
structure.
• From the Main menu, select
Data Input - Load campaign data
.
• From the
Campaign name
dropdown list, select the name of the
Campaign you are uploading to. The list of paths in the
Path
box will
change to be those defined for the selected Campaign.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 71 of 161
Figure 26: File upload dialog showing no paths
SPECCHIO stores every path name that has been used to upload data to this
Campaign. However, this box will only display those paths which point to
locations which exist on your computer. In this scenario, it is likely that no
existing paths will be valid on your computer and so no paths will be shown in
this dialog box for the Campaign in question.
• Click New Path. A file selection dialog will open.
• Navigate to the folder that you copied to your computer and select it. This
new path will be added to the list of paths in the
Path
box. Be sure to
select the same level folder as the one selected when the Campaign was
first loaded.
• Click Load. The dialog box will close and progress messages will be
shown in the right hand part of the Main Window indicating the total
number of files that have been found, and the number of new Spectra
which have been loaded.
Warning If you select the folder above or below the one selected when the Campaign was
first loaded, it will not be recognised as the same folder and will cause all of the
existing data to be duplicated in your SPECCHIO Campaign.
9.4.3 Uploading Additional Spectral Data in a New Data Hierarchy
There are often cases when a new set of Spectra is required to be uploaded, and this
new set does not relate to the existing Spectra in the Campaign. In this case, a new
folder structure can be created at the top level of the Campaign.
Figure 27: Data Hierarchy of GER example after initial load
In the above example, there is one sub-folder in the GER Example. If the user has
performed a second largely independent part of the experiment, its results may be
loaded as a new top level folder in the Campaign.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 72 of 161
Figure 28: Data Hierarchy of GER example after loading second part
To upload a new data hierarchy to an existing Campaign:
• Prepare the second upload folder tree of Spectra on your computer and
ensure it has a different name from the first one. It does not need to be
in the same folder as the first one, owned by the same user as the first
one, or even on the same computer as the first one.
• From the Main menu, select
Data Input
-
Load campaign data
.
• From the
Campaign name
dropdown list, select the name of the
Campaign you are uploading to. The list of paths in the
Path
box will
change to be those defined for the selected Campaign.
Figure 29: File upload dialog showing multiple paths
SPECCHIO stores every path name that has been used to upload data to this
Campaign. However, in this box it will only display those paths which point to
locations which exist on your computer. If the original data upload was done
from this computer, it is likely that the original data upload path will be shown.
• Click New Path. A file selection dialog will open.
• Navigate to the folder that you created with the second data hierarchy
and select it. This new path will be added to the list of paths in the
Path
box. Be sure to select the correct folder level.
• Click Load. The dialog box will close and progress messages will be
shown in the right hand part of the Main Window indicating the total
number of files that have been found, and the number of new Spectra
which have been loaded.
Original GER Example Campaign
upload top-level folder which relates to
original upload path
Second GER Example Campaign
upload top-level folder which relates to
a new upload path
Main GER
Example
Campaign
folder
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 73 of 161
9.5 Edit Metadata
9.5.1 Metadata Editor window
The SPECCHIO Metadata Editor allows you to enter and modify both Campaign and
Spectrum Metadata. Refer to
Section
6.9 Campaign-related Metadata
and
Section 8.8
Metadata
for detailed information about Metadata stored by SPECCHIO.
To open the Metadata Editor select
Data Input
and
Edit metadata
from SPECCHIO’s
Main Menu. The following Metadata Editor window is displayed.
Figure 30: Metadata editor dialog
Hierarchy Tree
Browser
Shows the Campaigns present in the database and allows
selecting of Campaigns, sub-trees and Spectra.
Metadata selection
tabs
Select either Campaign, to display and edit Campaign-related
Metadata, or Metadata to display and edit Spectrum-related
Metadata.
Metadata display
area
This will show the Metadata selected by the Metadata
selection tab.
Spectrum Metadata
Group selection area
Click on these checkboxes to display or suppress the various
Groups of Spectrum Metadata. Select All displays them all,
and Select None hides them all.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 74 of 161
9.5.1.1
Display options
Figure 31: Display options in the Campaign navigation window of the Metadata editor
Metadata is most easily displayed using SPECCHIO’s Metadata editor.
Order By
dropdown
Select the sort order to use for Spectra in the Campaign navigation
tree display.
The sorting order of single spectra within a target folder can be
defined here, however, the order of subfolders within a campaign as
well as the order of the inserted campaign remains alphabetically
ascending per default.
Do conflict
detection
Switch off this option to speed up data selection.
However, it is important to have this option switched on when
verifying metadata values for several files. If a metadata value is the
same for all selected spectra, this value will be shown. Otherwise,
either the information “-- multiple values -- “ or, in case of numeric
data, the relevant value range is shown.
Uploaded documents such as target pictures and PDF’s can be opened in an external
viewer or copied to the clipboard.
9.5.1.2
Update options
Refresh
button
Click this button to update the Hierarchy Tree Browser after making
changes which affect it.
Update
button
Click this button to write all changed Metadata back into the
database.
This button is highlighted in red when changes are pending.
Reset
button
Click this button to ignore all changes made since the last time the
Update button was clicked and redisplay the Metadata panel from the
database.
In this editor, whenever changes are made to any Metadata entries, they are saved
in an action list. They are not written immediately to the database. When the
Update button is clicked, the action list is processed and the database is updated.
When the Reset button is clicked, the action list is discarded without being
processed and the content of the Metadata display area is redisplayed from the
database.
Warning Click on Update after every change to a Metadata item. If you make multiple
changes before clicking Update, these changes are written to the database, but not
necessarily in the order in which you made them. This can yield unexpected results,
as the last change you made may not be the last one to be written.
Metadata entry fields are of various types, including integer, floating point, alpha
string, date/times or special lists. Generally, the editor restricts you to entering
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 75 of 161
values which are consistent with the data type. However, the editor does not
perform validation checks on entered data. For example, while you must enter a
valid number for the Relative Humidity, the editor will not reject negative values or
values greater than 100.
Warning Take care to enter numbers which are within the valid range for the Metadata
Attribute you are entering. The editor will permit you to store values which are
outside the valid range, which can be misleading or lead to unpredictable results
later.
9.5.2 Drag & Drop Options
Drag & Drop is currently implemented for spatial metadata only. KML files containing points,
polylines or polygons can be drag and dropped onto spatial fields to populate them.
Figure 32: Dragging of a KML file onto a Spatial Position metadata field
9.5.3 Displaying and Editing Campaign Metadata
See
Section 6.9
Campaign-related Metadata
for detailed information about individual
Campaign Metadata fields, their meanings and operation.
To edit a Campaign’s Campaign-related Metadata:
• Click on the
Campaign
tab to highlight it and display Campaign-related
Metadata.
• In the Campaign Tree Navigator, click on the Campaign for which you
wish to edit the Metadata, or click on anything in that Campaign’s sub-
tree. The Metadata for that Campaign will be read from the database and
displayed in the Metadata display area.
• Modify the Metadata Attribute values as you require.
• Click Update to cause these changes to be written to the database.
You can now click on another Campaign name and repeat this procedure.
Clicking on Metadata Group names in the Spectrum Metadata selection area does not
affect the Campaign editing tab.
If you change the
Campaign Name
Attribute, click on the Refresh button to cause
that change to be shown in the Campaign Tree Navigator.
9.5.4 Displaying and Editing Spectrum Metadata
See
Section 8.8 Metadata
for detailed information about individual Campaign
Metadata fields, the way they are organised into Metadata groups, their meanings
and operation.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 76 of 161
In order to understand the operation of this Editor, it is helpful to understand how
Spectrum Metadata are stored by SPECCHIO. Please refer to section 8.2 and 8.3.
Note SPECCHIO allows changing of most Metadata fields, but for some Metadata
Attributes it does not make sense. It could often be confusing and should not be
done, except in unusual circumstances. For example, changing the
File Name
or
File
Format
for a Spectrum does not make sense and may cause unexpected results in
subsequent Spectrum upload operations. Changing the
Spectrum Number
is also
rarely meaningful and could often be confusing. Changing the automatically set
Acquisition Time
and
Loading Time
values will generally be confusing. There are
other cases too.
9.5.4.1
Displaying a Single Spectrum’s Metadata
Displaying Metadata values is done using the Metadata Editor window.
To display a single Spectrum’s Metadata:
• Using the hierarchy browser, select the Spectrum for which you wish to
display the Metadata values.
• Click on the
Metadata
tab to display Spectrum Metadata.
Figure 33: Metadata editor dialog
• Select or de-select the check boxes for the Metadata Groups. The check
boxes are displayed on the right in the Spectrum Metadata Group
selection area.
• Scroll to the required Metadata Group to see the values of the Metadata
Attribute in that Group.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 77 of 161
Note Only Metadata Attributes which have values for the selected Spectrum are displayed.
9.5.4.2
Editing Metadata for a Single Spectrum
In each of the processes in this section, after you change or delete any Metadata
value click Update to write your change back into the database. If that Metadata
value is shared by another unselected Spectrum, the following dialog will be
displayed.
Figure 34: Shared data operation selector dialog
The second sentence in the dialog box will change to “You can
delete
metadata for
all records...” if you have deleted the Metadata value rather than changed it.
This dialog allows you to select how you want the Entity/Attribute/Value table to be
updated. The exact choice of options will depend on whether you performed a
change or a delete.
Apply to all spectra
The shared record in the Entity/Attribute/Value table will be
updated or deleted and therefore all Spectra which share this
Metadata value will be affected by this change. Understand
how widely the shared record is referenced when you use this
option. It is possible for the value to be used for any other
Spectrum in the entire database, although it is usually just
shared by other Spectra in the same Campaign.
Selected spectra
o
n
l
y
Only the selected Spectrum will be affected. To achieve this,
SPECCHIO will create a new entry in the
Entity/Attribute/Value table and link only the selected
Spectrum to it. For a delete operation, only the reference to
the current Spectrum will be removed from the Entity list. No
other Spectrum will be affected.
Cancel
Your change or delete request will be discarded and no
changes for this Metadata value will be made. You should
then click on Reset to cause the original database values to
be redisplayed.
To modify a Metadata value:
• Display the Metadata Attribute values for the required Spectrum as
described in the previous sub-section.
• Ensure that the check box for the required Metadata Group is ticked and
scroll to that Metadata Group.
• Click in the field and type in the required value. When the field contains a
valid value the Update button will become valid.
• Click Update to write the modified value back to the database. If the
Metadata is shared by more than one Spectrum, the dialog in Figure 34:
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 78 of 161
will be displayed. Select your desired action and click OK.
To delete a Metadata value:
• Display the Metadata Attribute values for the required Spectrum as
described in the previous sub-section.
• Ensure that the check box for the required Metadata Group is ticked and
scroll to that Metadata Group.
• Right click on the name of the field you wish to delete. A Delete button
will appear.
• Click on this Delete button. The Metadata field will disappear from the
display and the Update button will become active.
• Click Update to write the change back to the database. If the Metadata
is shared by other Spectra, the dialog in Figure 34: will be displayed.
Select your desired action and click OK.
To add a new Metadata value:
• Display the Metadata Attribute values for the required Spectrum as
described in the previous sub-section.
• Ensure that the check box for the required Metadata Group is ticked and
scroll to that Metadata Group.
• Right click on any blank space within the Metadata Group’s display box. A
menu of Metadata items that can be added for that Group will be
displayed.
• Click on the menu entry for the Metadata item you wish to add. A data
entry field for the Metadata entry will be displayed.
• Enter the value you require for this Metadata field.
• Click Update to write this change back to the database.
9.5.4.3
Editing Metadata for All Spectra in a Sub-tree of the Campaign tree
SPECCHIO allows for simultaneous updating of Metadata for multiple Spectra. This
feature is typically used when multiple Spectra have just been uploaded and a
selection of their Metadata items shares the same values. This is a common scenario,
as Metadata of Spectra are often highly redundant, especially when multiple
measurements of the same object were acquired.
When multiple Spectra are selected (either by selecting a node in the Campaign Tree
or by selecting multiple Spectra individually), the Metadata values that are common
across all selected Spectra are displayed. For any Metadata Attribute for which even
one Spectrum has a different value, or is missing that Attribute, the string
-- Multiple
Values --
is displayed. For Metadata Attributes which are selected from dropdown
lists, the value
Nil
is shown if the values are not the same for all Spectra.
To change a specific Metadata item for all Spectra under a node:
This method requires that the Metadata Attribute has the same value for all Spectra.
See the next procedure if that is not the case.
• Navigate to the correct Campaign hierarchy node and click on it to
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 79 of 161
highlight it.
• Ensure that the check box for the required Metadata Group is ticked and
scroll to that Metadata Group.
• Click in the field to be changed and type in the required new value. When
the field contains a valid value the Update button will become valid.
• Click on the Update button to write the change back to the database for
all Spectra for this node.
• If the Metadata value is shared with other Spectra not under the scope of
this node, the dialog in Figure 34: will be displayed. Click on either Apply
to all spectra if all Spectra which share this value are to be changed.
Click on Selected spectra only if only the Spectra under this node are
to have the Metadata Attribute value changed.
To delete a Metadata value from all Spectra under a node:
• Navigate to the correct Campaign hierarchy node and click on it to
highlight it.
• Ensure that the check box for the required Metadata Group is ticked and
scroll to that Metadata Group.
• Right click on the name of the field you wish to delete. A Delete button
will appear.
• Click on this Delete button. The Metadata field will disappear from the
display and the Update button will become valid.
• Click Update to write the change back to the database.
• If the Metadata value is shared with other Spectra not under the scope of
this node, the dialog in Figure 34: will be displayed. There is a choice to
either Apply to all spectra or Selected spectra only. Click Apply to all
spectra if all Spectra which share this value are to have it deleted. Click
Selected spectra only if only the Spectra under this node are to have
the Metadata Attribute value deleted.
Warning If you delete a Metadata Attribute which shows
-- multiple values –
in the current
view,
all
values of this Metadata Attribute will be deleted from
all
selected Spectra,
even if some Spectra have multiple values of the Attribute.
To reset a specific Metadata item for all Spectra under a node to have the
same value:
• Navigate to the correct Campaign hierarchy node and click on it to
highlight it.
• Ensure that the check box for the required Metadata Group is ticked and
scroll to that Metadata Group. The value displayed in the Metadata field
should be
-- Multiple Values --
to indicate that the values are not the
same for all Spectra.
• Right click on the Metadata field name to show a Delete button. Click on
the Delete button. The Metadata field will disappear from the screen.
• Click Update to write this change into the database. This step is
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 80 of 161
critical.
• Right click within the Metadata Group box to display a menu of Metadata
Attributes that can be added to this Metadata Group. Click on the
Metadata Attribute to be added back.
• Enter the value you require for this Metadata Attribute.
• Click Update to write this change back to the database.
9.5.4.4
Editing Metadata for Multiple Unrelated Spectra
The operations in the preceding section can be applied to multiple unrelated Spectra.
It is possible to select multiple Spectra by holding the Control key while clicking on
each additional Spectrum to be selected. It is also possible to select a range of
Spectra by selecting the first Spectrum and then holding the Shift key while you click
on the last Spectrum in the range. (This is consistent with common Windows usage
of the Control and Shift keys.) The operations in the preceding section will then apply
to all selected Spectra.
Similarly, it is possible to select multiple nodes in the Campaign hierarchy and
operate on all Spectra in those nodes.
Warning Do not select multiple Spectra or nodes across different Campaigns and then update
all selected Spectra to share the same Metadata values. If subsequent users select
the Apply to all spectra button when editing Metadata, it will change the Metadata
across both Campaigns, which is a very unexpected and confusing result.
9.6 Using the Metadata Editor functions
New functionalities have been added to the Metadata Editor for Release 3.3 (2017).
They can be found in the
Compute
tab once the Metadata Editor was selected from
the Main Menu.
9.6.1 Compute measurement support
This function computes two metadata attribute values: Measurement Support Area
[m2] and Measurement Support Radius [m] or Major/Minor Axes for elliptical,
projected field of views respectively.
The computation is based on the following metadata attributes – make sure to have
them inserted/calculated before using this function:
• Sensor Zenith and Sensor Distance
• Optics FOV (should automatically have been uploaded with the ASD data import,
but may need manual definition for other sensors)
After applying this function, the user is informed by an info panel when the
calculations are completed. Click on OK to remove this panel. The calculated
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 81 of 161
metadata values can be seen in the Sampling Geometry category:
Figure 35: Example results of the measurement support calculation
9.6.2 Augment Altitude
In case the Altitude information is missing, there is a built-in function which inserts
the altitude information, based on latitude and longitude, by using the web service
“geonames” (api.geonames.org/astergdem).
This service uses NASA’s ASTER GDEM data which has a precision of 30 meters.
Thus, this function should be used only if more accurate elevation data are not
available.
Note: this function is only applicable for single spatial position coordinates. Other spatial
positions such as transects are not (yet) supported by this function. An Internet
connection is required to use this function.
9.6.3 Switch Longitude E-W
Note that Longitudes East of Greenwich (e.g. Switzerland and Australia) need to be
a positive value. If your metadata was inserted as negative values, this can be
changed by using the switch function from the
Compute
menu
Switch longitude E-W:
9.6.4 Visualisations
Under the
Visualisations
tab, all visualisation options that are available in the
Spectrum Browser are also provided for selected Spectra from the Metadata Editor.
Please refer to section
10.1 The Spectrum Browser
for details.
9.7 Uploading Metadata from XLS (Excel) files
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 82 of 161
SPECCHIO supports uploading Metadata to your Campaign from Excel spreadsheet
files saved as an .XLS file. It does not support loading form.CSV, .XLSX or other
Excel file formats.
Figure 36: Uploading Metadata from XLS process
The operation of this process can be tailored as described below.
Input .XLS file format
Metadata Excel .XLS files must conform to the following conventions in order to be
loaded correctly:
• Only the first data sheet is loaded.
• The first row must contain column headings. It will be displayed and referenced
during the upload process, but it will not be uploaded. The column headings do
not need to match Metadata Attribute names.
• Each row holds data which is uploaded to one Spectrum.
• The rows do not need to be sorted.
the row data are
copied to that
Spectrum’s Metadata
Nominated
Matching
Column
Data for
one
Spectrum
When the
Matching
Column
matches,
Matches
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 83 of 161
• One column, identified at upload time, is used as the Matching Column. It
identifies which Spectrum will receive the data from each row.
• If a Spectrum is selected in the Matching Column in more than one row, all of
these rows are ignored.
• Other columns contain the Metadata that will be uploaded.
• You can select to ignore some columns during the upload process.
Column selection operation
After you have selected the .XLS file to upload, the following panel is displayed as
part of a larger dialog.
Figure 37: Matching & Element Assignment Control
This panel shows the contents of the .XLS file you have opened and allows you to
select how it will be used.
The three rows of dropdown boxes are used as follows:
First
row
Selecting the Matching Column
In one of the columns only, select Matching Column
from the drop down list. This indicates the column
that will be used to search for the Spectrum to receive
the Metadata for each row.
Second
Row
Selecting the Metadata Group
In all columns that you want to use, select the
Metadata Group that contains the Metadata Attribute
which is listed in that column.
Third
Row
Selecting the Metadata Attribute
In all columns that you want to use, select the
Metadata Attribute which is listed in that column.
After you have set up these three rows of selection parameters, the Assignment
Details and Matching Details panels will show how the .XLS data will be uploaded.
Use these panels to check that the upload is set up correctly before proceeding.
1st Row – Select
one Matching
Column
2nd Row – Select
Metadata Group
3rd Row – Select
Metadata
Attribute
Entire contents
of selected XLS
file displayed for
reference
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 84 of 161
Number of assignable columns:
The
number of columns in
the .XLS file which
remain unused
Number of assigned columns:
The
number of columns in
the .XLS file which will
be copied into
Metadata Attributes
Number of spectra:
The number of
Spectra selected in
the hierarchy tree
browser (see below)
Number of matches:
The number of
selected Spectra
which match data in
the
Matching Column
of
the .XLS file
Spectrum
(First column) The
Spectrum IDs of all
selected Spectra
DB Value
(Second column) The
value from the
SPECCHIO database
of the Metadata
Attribute which you
have identified in the
Matching Column
Table Value
(Third column) The
matched value from
the
Matching Column
of
the .XLS file. The data
in this row will be
loaded into the
Metadata for this
Spectrum.
Figure 34: Assignment & Matching Details dialog
To upload Metadata:
You must already have an Excel .XLS file containing the Metadata to upload.
If SPECCHIO finds a Metadata Attribute already exists for any Spectrum, SPECCHIO
will ask how you wish to proceed by displaying the following dialog box, which lists
the Metadata Attribute name that already exists (in this example,
File Comments
).
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 85 of 161
Select one of these three options from the dropdown list:
Skip this parameter
No data from this column of the XLS file will be loaded into
the SPECCHIO database.
Insert anyway
Add another value for this Metadata Attribute. This option is
not supported for some Metadata Attributes. For example,
adding a second File Name or Spectrum Number is not
permitted.
Delete existing and
insert new values
The Metadata values in the XLS file will replace those in the
Spectra.
Click OK to continue processing using the action you have selected.
Click Cancel to skip processing of this column. This has the same result as selecting
Skip this parameter
.
Adjusting the matching strings
If the data in the
Matching Column
of the XLS file is a partial match to the Metadata
Attribute values in the database, you can use the REGEX box to assist in the match.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 86 of 161
Figure 38: Using Regular Expressions for selecting Spectra
For each row of the XLS file, the data in the Matching Column is concatenated with
the
REGEX start
and
REGEX end
contents to form a Regular Expression. The
matching process is then performed using Regular Expressions. (See
Appendix A:
Regular Expressions Tutorial
.) In the above example: the ^ forces the matching to
start at the beginning of the string; . is a special character so \ must be used to
escape it, so \. matches the . in the file name; 0* matches any number of 0
characters; 5 matches 5; and the $ forces the match to complete at the end of the
string.
Note If a value in the
DB Value
column matches more than one row in the XLS file, no
matching row will be shown in the
Table Value
column, and no Spectrum will be
updated.
Using preconfigured column information
If you often load Metadata from XLS files, you can pre-configure the XLS column
mappings you want to use. This will be more efficient and reduce your chance of
making a mistake.
Create an extra XLS spreadsheet which lists the Column Names you’ve used in your
spreadsheet along with the matching names SPECCHIO uses for these fields.
triticaa\.0*
5
^triticaa\.0*5$
$
^
The regular expression
matches this DB Value,
so this Spectrum is the
one that will receive the
new Metadata values
Start and end markers
always used
Start and end markers
always used
Concatenated to form a
regular expression
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 87 of 161
Figure 39: Setup up pre-defined column allocations
• The table must be in the first datasheet of the spreadsheet.
• Each SPECCHIO Metadata Attribute name must be on the same row as its
matching XLS column name.
• The names in both columns are case sensitive.
• Leading and trailing spaces must be avoided.
• There must be a header row, but it is skipped over.
• The rows do not need to be sorted into any sequence.
• The file must be saved as an .XLS file. .CSV, XLSX or other Excel file formats are
not supported.
To use a pre-configured column matching spreadsheet:
• Select
Data input
-
Get metadata from XLS
from the menus on the
SPECCHIO Main Window. The following dialog will be displayed. Only
those Campaigns to which you have write permission will be displayed.
• Select the Campaign, hierarchy node or Spectra to which you want the
Metadata to be uploaded.
• Select
File
-
Open file...
from the menu bar on this screen. Browse to the
.XLS Metadata file you have previously prepared and Open it.
• Select
File
-
Open Column-Element Matching File…
from the menu bar on
this screen. Browse to the column mappings XLS file and open it. The
name of file you have opened will be shown in the
Element – Column
Auto-Matching
control.
• Click on the Apply Auto-Matching button. The information from the
column matching XLS file will be read and the drop down boxes in the
Put XLS Column
heading names in
first column
Put corresponding
SPECCHIO
Attribute names in
second column
A header row must
be present, but is
not used
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 88 of 161
second two rows of the
Matching & Element Assignment Control
box will
be updated.
• In the column that you wish to use as the Matching Column, in the top
row dropdown box select
Matching Column
.
• Carefully review the information shown in the
Assignment Details
and
Matching Details
boxes and ensure it is correct.
• Click on the Insert Selected Metadata button.
9.8 Import campaign from XML file
Note In order to use this option, you must be logged in as an Administrator.
The import function reads an XML file that was created by the SPECCHIO
Export
campaign
function. The import will create an exact copy of the exported Campaign.
It will also add any Sensor, Instrument or Calibration information that is referenced
by this Campaign if it does not already appear in this database.
To import a Campaign:
• From the Main menu select
Data Maintenance
-
Import campaign
.
Figure 40: Campaign Import Dialog
• Click Browse to open a file dialog box.
• Select the XML file that you want to import into your current SPECCHIO
database and close the dialog box.
• In the
Campaign Owner:
box, select the name of the Researcher who will
be the imported Campaign’s owner.
• Click Import. When the import is complete, the Campaign Import dialog
will close.
To import a Campaign by drag and drop:
• Select the XML file that you want to import in the computer’s file system,
then drag and drop it onto panel holding the SPECCHIO icon in the main
window (the panel borders will be highlighted blue):
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 89 of 161
Figure 41: Drag and drop of a SPECCHIO XML file
• In the
Campaign Owner:
box, select the name of the Researcher who will
be the imported Campaign’s owner.
• Click Import. When the import is complete, the Campaign Import dialog
will close.
To import a Campaign from a file on the SPECCHIO server:
• Depending on the internet connection between server and client it may be beneficial to
copy the XML file onto filesystem of the SPECCHIO server and load it from there. To do
this, supply the file path of the server as shown in the below example.
Figure 42: Specification of a file source on a SPECCHIO server
Notes The Import database version must identical to the Export database version where
the XML file was created. SPECCHIO does not check this. If they are not the same,
the new Campaign will not be readable and the database may give unpredictable
results.
The name of the imported Campaign will be identical to the name of the exported
Campaign. If you import into the same database you exported from, you will have
two identically named Campaigns. To avoid confusion, it would be advisable to
rename the newly imported one using the Metadata Editor to change the Campaign
Metadata’s Campaign Name value.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 90 of 161
10 Data Selection & Output
This section describes the functionalities of SPECCHIO’s Main Menu tab
Data
Selection & Output
. Here you will find information on how to:
10.1 The Spectrum Browser
The SPECCHIO Spectrum Browser allows users to browse the Campaign hierarchy to
locate Spectral data, and then to launch operations on the selected data. Those
operations are selected by clicking the one of the row of six buttons in the right hand
panel. The operation of these buttons is explained in sections with those names in
this section.
From the Main menu, select
Data Processing & Output - Browse data
hierarchy
to see the following window:
Figure 43: The main Browse data hierarchy window
• Browse options: In the hierarchy browser, all campaigns are shown when using
the Browser menu. If you wish to browse only within the campaign data you
have uploaded or for which you have been added to the Research Group, check
the box
Show only my data
.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 91 of 161
Matching
Spectrum
Identifiers
The Spectrum IDs of the Spectra which are highlighted in the
browse panel are listed in this box.
Number of
results
The number of Spectra IDs listed in the Matching Spectra box is
shown in this box.
6 action
buttons
Of these six action buttons, Show Report, File Export, Process,
Spectral plot and Refl. calc will become active when you have
selected one or more Spectra. The Publish Collection button is only
present if you are accessing an ANDS-enabled SPECCHIO Server
and is only active if you have selected Show only my data. The
operations these buttons initiate will be performed on the selected
Spectra. Each of these buttons has a section in this chapter.
Splitting
rules for file
export and
plotting:
Click on the radio buttons to select one of the options: Split spaces
by sensor, Split spaces by sensor and unit or Split spaces by sensor,
instrument, calibration_no and units. This selection will affect the
operation if you select the Process, Spectral plot or File export
buttons.
There are many places in the SPECCHIO operation when a Campaign Hierarchy
browse control, such as the one below, is displayed as part of an operation dialog.
Windows Mac
Figure 44: Campaign Hierarchy browse controls for Windows and Mac
This browse control operates much like a normal folder and file browser.
• Click on the and icons (or or for Mac) to cause the hierarchy levels to
be expanded or collapsed.
• Select an individual Spectrum by clicking on it.
• Select a node in the hierarchy by clicking on it. This effectively selects all Spectra
under this node.
• Select a second node or Spectrum by holding the Control key and clicking on it
(or equivalent Mac mouse action).
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 92 of 161
• Select a range of Spectra, by clicking on the first Spectrum to select it, holding
the Shift key and clicking on the last Spectrum (or equivalent Mac mouse action).
• Change your selection by clicking on the new item you want to select (without
Control or Shift keys).
In addition, there are two options:
Order by:
Use the drop-down menu to choose the sort order for Spectra within
each node. Often these three orders are the same, or at least similar, so
you may not notice any change of sequence.
The options are:
Sampling
date
The Spectra are sorted in increasing order of the date
and time in the Acquisition Time Metadata Attribute.
Filename
The Spectra are sorted in increasing alphabetical order
of the string in the Filename Metadata Attribute.
Insert order
The Spectra are sorted in increasing order of the date
and time in the Loading Time Metadata Attribute.
Refresh
Click this button to cause the Hierarchy Tree to be reloaded from the
database. Use this button if the operation you have just performed,
some background operation, or some other user has changed the
database since this dialog box was opened.
10.1.1 Visualisations
This menu item displays a drop down list of graphic visualisation options which you
can apply to the Spectra listed in the Matching Spectra box. These options match
those listed in
Section
15.5 Visualisation Modules
. Depending on the option and
Spectra selected, some of these Visualisation options may take several minutes to
process.
Figure 45: Visualization examples: spectral scattter plot (left) and spectral line plot
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 93 of 161
Figure 46: Visualization example: Time line explorer of a series of 25 spectra.
In the above example, the first 5 and the last 5 measurements are Spectralon
reference data. Note that per default, always the first spectral band of the first
selected spectrum is displayed in the spectral plot.
10.2 Query Builder
SPECCHIO’s Query Builder provides a way to select one or more Spectra based on
their Metadata, and then perform operations on all of these selected Spectra. The
operations which can be performed on the selected Spectra are the same as for the
Spectrum Browser, and are explained in the later sections of this Section.
To Launch the Query Builder select
Data Processing & Output - Build query
from the
menu on SPECCHIO’s Main Window.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 94 of 161
Figure 47: The main Query Builder window
The parts of this window are:
Visualisations
This menu item displays a drop down list of graphic visualisation
options which you can apply to the Spectra listed in the Matching
Spectra box. These options match those listed in Section 13.5
Visualisation Modules. Depending on the option and Spectra
selected, some of these Visualisation options may take several
minutes to process.
Left panel
There is one box for each SPECCHIO Spectrum-related Metadata
Group and within each box, all of the Metadata Attributes for that
Group are listed with a data entry box.
Matching
Spectrum
Identifiers
The Spectrum IDs of the matching Spectra are listed in this box.
Number of
results
The number of Spectra IDs listed in the Matching Spectra box is
show in this box.
Six action
buttons
The six action buttons, Show Report, File export, Process, Spectral
plot, Refl. calc and Publish Collection will become active when you
selected one or more Spectra. The operations they imply will be
performed on the selected Spectra. Each of these buttons has a
section in this Chapter.
Splitting rules
for file export
and plotting:
Click on the radio buttons to select one of the options: Split spaces
by sensor, Split spaces by sensor and unit or Split spaces by sensor,
instrument, calibration_no and units. This selection will affect the
operation if you select the Process, Spectral plot or File export
buttons. See the relevant option description section in this chapter
for more information,
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 95 of 161
Queries are created by entering selection criteria into the Metadata Attribute controls
in the left panel. A new query is performed every time any change is made in any
Metadata entry box, after clicking on the Run Query button. Initially, no query is
active so the Matching Spectra box is empty.
The searching follows these rules.
• A search condition is created by making the search conditions for a Metadata
Attribute non-empty. Empty fields do not trigger any search conditions.
• Initially all Metadata query fields are empty. Even the data/time fields are empty,
but the date/time dialog displays the current date and time in this case.
• Once a date/time selection field is modified, it cannot be made empty again.
Therefore, searching will be done on this date/time until the dialog box is closed.
This applies independently to the start and end date/times.
• A new query is performed every time any field is changed. Even if only one
character is typed or removed, a new query is performed. For date/time fields, a
new query is performed each time the date/time picker is closed.
• To be selected, a Spectrum must satisfy
all
search criteria entered.
• The minima and maxima search conditions for a numeric or date/time Metadata
Attribute are independent. That is, it is valid to enter just a minimum or just a
maximum value.
• Numeric and date/time search ranges are inclusive. That is, if you enter a
minimum Spectrum Number of “20”, Spectra with Spectrum Number greater or
equal to 20 will be selected.
• If a search condition for a Metadata Attribute is present, Spectra without that
Metadata Attribute will never be selected.
• Strings can be wildcarded to enable more complex search behaviours; see 10.2.1
for details.
• With Release 3.3.0.0, spatial geometry support was introduced. When defining
spatial query conditions, it is important to choose only the relevant spatial
geometry type, depending on the spatial sampling scheme.
•
10.2.1 SQL Matching Strings
Matching strings are required to control the way string values entered in the Query
Builder are matched versus metadata entries in the database.
When matching text strings, the SPECCHIO server uses a MySQL Query LIKE clause.
Therefore, the following are supported.
% Matches zero or more characters.
_ Matches exactly one character.
\ Causes the next character to be matched. That is, \% will match a % sign and
\_ will match an underscore character.
For dropdown lists, the test is not performed if
Nil
is set. Otherwise, only those
Spectra with exactly that value will be selected.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 96 of 161
Notes Text Metadata Attributes which are present, but empty, can be selected by using %,
but not by _.
Text Metadata Attributes which contain only a single space can be selected by using
%, _ or a space.
It is common for some Spectrum file formats to have a File Comment field which
contains only a space. This is not so for other Metadata Attributes.
Warning There is no option to search by Campaign Name, except for the Campaign Name in
the Spectrum-related Metadata Group, Campaign Details. This is NOT the Campaign’s
name and will generally not be present. See
Section 6.10 Metadata
for more
information on the Spectrum-related Campaign Details Group of Metadata Attributes.
After you have entered search criteria for the Spectra you wish to operate on, click
the Run Query button, followed by one of the six action buttons. See the following
sections for details of the operation of these buttons.
10.2.2 Entering Dates and Times
SPECCHIO uses a Date and Time selection dialog which is similar to many others.
Here are a few tips for using it.
Figure 48: Date and Time picker dialog
• Select the Time first. (Because selecting the Date closes the dialog.)
• To change the Hours, click on the hour number in the Date and Time Picker (it
will not be highlighted and the cursor will not move, but trust me) and then click
on the time selector up and down triangles to adjust the number. (You cannot
type in the number directly.)
• To change Minutes and Seconds, do as for Hours.
• Clicking on the required Date closes the dialog box, but only if the date is not
already highlighted.
• If you do not want to change the date, click on the down triangle in the data
input field to close the date picker.
•
10.3 Show Report
Reports are generated by clicking on the Show report button in the Query Builder
or Spectrum Browser. It operates on all Spectra which are selected at the time this
button is clicked.
Reports are shown in a window in the following form.
Down triangular
arrow in the data
input field
Time selector up
and down
triangular arrows
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 97 of 161
Figure 49: Spectrum report frame
Just below the Spectrum graph, the number of Spectra graphs in the report is shown
with a control to allow you to select which Spectrum graph you wish to display.
Figure 50: Spectrum Graph selection control
Click on the up and down arrows to cause the Spectrum graph display to cycle
through the available Spectra. The selected graph is displayed after a short
processing delay.
Click on the Metadata Group check boxes to enable or disable display of the
respective Metadata Group.
For a larger display, documents such as target pictures and PDF’s can be opened in
an external viewer or copied to the clipboard.
Click on the Close button to close this window.
There are four action buttons at the top of this window.
Print
Click on this button to Print the graph. The Metadata are not
printed. It is recommended that you set the plot parameters
first using the button to add a title and identifying
information to your printed graph.
Reset
Click on this button to reset the graph to its originally
displayed form.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 98 of 161
Set plot
parameters
Click on this button to change the display parameters for the
graph. The following Dialog is displayed.
Figure 51: Plotting dialog
Title:
Enter the Title you want to appear at the top of the
printed page.
Caption:
Enter text that you want to appear under the
graph.
X/Y Label:
Enter the labels that you want to use to label
the X and Y axes.
X/Y Range:
Enter the lowest and highest values that you
want to appear on the X and Y axes. The two values in each
question must be separated by a comma.
Marks:
Select the way you want the data points
represented on the graph.
X/Y Ticks:
Place specific labels along the X and Y axes. For
example, entering “Threshold 450.3, Maximum 670.23” will
place the string “Threshold” at the point 450.3 along the axis
and “Maximum” at 670.23.
Grid:
If checked, the graph will include a grid.
Stems:
If checked, each data point will be joined to the
X axis by a straight line.
Connect:
If checked, the data points will be joined by
straight lines. If you check this option, it is probably best to
select the option
None
for the
Marks:
question in order to
show a thin clean line.
Use Color:
If checked, the various data lines on the graph
will be given different colours. If not checked, they will all be
black.
Use Line Styles:
If checked, the data points will be
connected with dashed lines. This option has no effect unless
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 99 of 161
Connect:
is checked.
These details are not remembered once you move away from
viewing this graph.
Fit data
Click on this button to set the graph axis parameters to
values which fit the displayed axes to the data.
SPECCHIO scales the graph’s Y Axis for Spectra with
Measurement unit
set to
Reflectance
to display correctly even when there is strong atmospheric noise in the
usual regions of 1350-1440nm, 1790-1980nm and 2360-2500nm. For these graphs,
the maximum Y value is set by calculating the mean and standard deviation in the
wavelength region 300-1200nm and then setting...
Max Y Axis value = Mean + 3 * StdDev
When Spectra having no Sensor definition are plotted, the X-axis will represent bands
and not wavelengths.
For sensors comprising broad and narrow band elements (e.g. MFR) only the narrow
bands will be plotted.
10.4 Spectral plot
Using this option, plots of all currently selected Spectra are displayed.
Separate plots are created based on the setting of the
Splitting rules for file export
and plotting:
question in the
Query Builder
or
Spectrum Browser
dialogs
.
Figure 52: Spectral plot examples: target and reference panel radiance spectra (left),
selected grass reflectance (right) plotted in separate graphs per measurement unit
Note: The range of the Y-Axis is dynamically adjusted for the range of the selected spectra.
Reflectance plots use a statistical approach including the VNIR range but excluding
water vapour bands to avoid plot Y-Axis scale being dominated by noise.
The four plot options that are described in
Section 10.3
Show Report
are also
available for the Spectral Plot option. They relate to the four buttons shown at the
top right of the above plotting windows.
10.5 Process
A separate chapter of this Use Guide is devoted to this topic. Please see
Section 12
Interactive Processing using Space Networks
.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 100 of 161
10.6 Refl. calc
This function is the same as Process, except that if you have a set of Spectra
selected which all have Data Links to white Reference Spectra, it will set up a
transformation from Radiance to Reflectance for those Spectra as default operations
in the Process window.
Refer to
Section 12
Interactive Processing using Space Networks for further
information.
Note that the reflectance calculation implemented in SPECCHIO is rather simplistic.
For a more scientific and elaborated workflow please refer to the SPECCHIO
Reflectance Calculator written in Matlab.
10.7 File Export
Files are exported by clicking on the File export button in the Query Builder or
Spectrum Browser. It operates on all Spectra which are selected at the time this
button is clicked. The following window will open:
Figure 53: File output dialog
Controls are as follows:
File format:
CSV:
ENVI SLB:
Select from the drop down list.
Data is written to a text file in comma separated value format.
This file type can be read into Excel. A full description of the way
CSV files are used can be found below at
Section 8.6.1
CSV
Spectrum Export Format
.
ENVI Spectral library file consisting of two files: a header file
(<file_name>.hdr) and a body file (<file_name>.slb). Spectral
data are written as floating point values, either 32 bit or 64 bit
depending on the architecture of the machine. The datatype
field in the header file is set accordingly (datatype 4 or 5
respectively).
Target folder:
Click Browse to select the folder into which the output files will
be written.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 101 of 161
Base name:
Enter a character string which will be prefixed to the file name.
This name should contain only characters that are allowed in
filenames (depends on your operating system). The base name
is used to build the real filenames according to one of the
following patterns:
<base name>_<sensor_name>.<file extension>
<base name>_<instrument_name>[cal<calibration
sequence number>].<file extension>
The actual choice of naming pattern depends on the selection
made in the
Splitting rules for exporting and plotting:
question in
the previous
Query builder
dialog or
Data browser
dialog.
If no Instrument has been defined the Sensor name will be
used.
Split header
a
n
d
b
o
d
y
Applies to CSV files only. The Spectra data will be split into two
separate CSV files. The body file will have the string _BODY
appended to the file name and _HDR is appended to the file
name for the header file. (ENVI SLB files are always split into
header and body.) The data are also split according to the
selection made on the
Splitting rules for file export and plotting:
question in the
Query Builder
or
Spectrum Browser
dialogs
.
See
comment below about file naming.
Time format:
Select your desired time format. Selecting milliseconds makes it
easier to use the timestamp as an X axis value in Excel plotting.
The data file names are constructed using the following components.
Campaign name
The Campaign name is always used in the output file name.
It is not recommended to export data from multiple
Campaigns at one time. The results are undefined.
Sensor name
The Sensor name is always used in the output file name.
There will be separate files for each Sensor.
Unit type
If you select
Split spaces by sensor and unit
or
Split spaces
by sensor, instrument, calibration_no and units
, there will
be separate files for each value of the Unit Metadata
Attribute.
Calibration
number
If you select
Split spaces by sensor, instrument,
calibration_no and units
, there will also be separate files for
each Calibration for each Sensor and Unit combination.
Example
Consider a Campaign containing Spectra captured with the following three
Instruments:
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 102 of 161
• An ASD Instrument belonging to the Institute of Natural Resources, Massey
University (named INR ASD) with calibration sequence number 1
• An ASD Instrument belonging to the Remote Sensing Laboratories, University of
Zurich (named RSL ASD 1) with no calibrations entered in the database
• A GER 3700 Instrument belonging to the Remote Sensing Laboratories, University
of Zurich (named RSL GER 3700) with no calibrations entered in the database
A CSV file output of this Campaign with the option
Split header and body
results in
the following files:
10.7.1 CSV Spectrum Export Format
Multiple Spectra are written into a single CSV file, with the data for each Spectrum in
a separate column.
If the
Split header and body
option is selected, then the header and body are written
into separate CSV files. Otherwise, they are written into a single CSV file header first.
The header contains as many lines as there are Metadata Attributes defined in all of
the selected Spectra. The first column of each row is the Metadata Attribute name.
The body contains one row for each wavelength. The first column is the wavelength
in nanometres and the subsequent columns are the values at those wavelengths for
each selected Spectrum.
The following figure shows an Excel view of a CSV file created with three Spectra and
the header and body written into the same file.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 103 of 161
Figure 54: CSV file example (loaded into Excel)
10.8 Export campaign
The Campaign export functionality writes all data of a Campaign to a single XML file
which retains the hierarchy and Metadata structure of the Campaign. The information
from the Sensors, Instruments and Calibrations used by the Campaign is also copied
into the XML file.
These XML files can be used to import the Campaign into another SPECCHIO
database instance.
The exported XML file will be named with the Campaign name, date and time of
export.
To export a Campaign:
• From the Main menu select
Data Maintenance
-
Export campaign
.
Figure 55: Campaign Export Dialog
• Select the Campaign you wish to export from the
Campaign name:
dropdown list.
• Click Browse to open a file dialog box.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 104 of 161
• Select the folder into which you want the XML file to be written and close
the dialog box.
• Click Export. The Campaign Export dialog box will close and a progress
box will appear. When the export is complete, it will close.
10.9 Publish Collection
A separate chapter of this User Guide is devoted to this topic. Please see
Section 11
Publishing Data to ANDS
.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 105 of 161
11 Data Processing
This section describes the functionalities of SPECCHIO’s Main Menu tab
Special
Functions.
Here you will find information on how to process data using SPECCHIO’s
special functions:
11.1 Functions to generate additional metadata
11.1.1 Compute UTC
This tool is provided to compute the capture time in UTC. Starting from SPECCHIO
DB version 3.2.1, a new attribute called ‘Acquisition Time (UTC)’ is supported. The
UTC time is computed from the ‘Acquisition Date’ Metadata Attribute by applying a
time shift of a number of hours entered by the user.
To compute Acquisition Times in UTC:
• From the Main menu select
Special Functions
-
Compute UTC
.
Figure 56: UTC Computation dialog
• Select the hierarchy node containing the Spectra that need UTC to be
calculated, or select the individual Spectra files by holding down the Ctrl
key. The first selected hierarchy name and the number of Spectra are
shown on the right in read only fields.
• Enter the number of hours to adjust in the Hours East of GMT field. Use
positive numbers for longitudes East of GMT (that is, the number for
Australia will be positive).
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 106 of 161
• Click Apply.
• Close the dialog using the Close button.
Note: if some error was made during the computation, e.g., the wrong number of
hours was entered, reapplying the function will simply overwrite the already existing
UTC values in the database.
As date/time fields are stored in milliseconds, the time shift is implemented as:
Date_time_in_milliseconds = Date_time_in_milliseconds –
hours_east_of_gmt*milliseconds_per_hour.
Therefore, if a time shift happens across midnight, the calendar date of the capture
time will also be shifted back or forth, depending on whether hours_east_of_gmt is
positive or negative. For example, for a local capture time around 10am in Oceania a
time shift of around 12 hours East of GMT is needed. The UTC time will then be
10pm the previous day.
A UTC Time Computation Attribute value in the Processing Metadata Group is added
to each Spectrum that has its UTC Acquisition Time computed by this function.
Figure 57: UTC Time Computation Metadata Attribute after applying a UTC Time
computation
11.1.2 Calculation of Sun Angles
SPECCHIO provides a function to calculate sun angles based on the UTC date and
time of the Spectrum acquisition and the spatial position in Latitude and Longitude of
the acquisition location.
The sun angles are then automatically stored for each Spectrum as Metadata in the
Illumination Group as
Illumination Zenith
and
Illumination Azimuth
.
To Calculate Spectra’s Sun Angles:
• Ensure that the Acquisition Time (UTC) has been correctly set for all
Spectra (see 11.1.1 on how to compute UTC).
• Ensure that the Spectra’s locations are correctly set in the Location
Group’s Spatial Position Metadata Attributes.
• Select
Special Functions
and
Sun angle calculation
from the menu in the
Main Window. A Hierarchy Tree Browser is displayed.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 107 of 161
Figure 58: Sun Angle Calculation dialog
• Select the Spectra to be processed. A single Spectrum or multiple Spectra
can be selected. The
Number of Selected Spectra:
field will be updated.
• Click Apply to calculate the sun angles and add them to the Spectra’s
Metadata. After the calculation is completed, the following
Update Info
box is displayed:
•
• Click OK and Close to close this dialog box.
Automatically, a new processing attribute value in the Processing Metadata Group is
added to each Spectrum that has its illumination angle attributes computed by this
function.
Applying this function twice is safe, as the calculated sun angles will always be the
same.
Spectra, which do not have Latitude and Longitude and UTC Acquisition Time values,
will not have sun angles calculated for them. If the Latitude or Longitude is outside
of the normal ranges (for example, a Latitude value of 115 degrees), sun angles will
be calculated, but they will not be meaningful. If UTC times are during the night for
the spatial position, sun angles below the horizon will be calculated.
For Spectra with more than one location position (e.g. transects), the central position
will be calculated and used as reference position for the sun angle calculation.
11.1.3 Calculation of Goniometer Angles
If a set of Spectra has been captured in a standard way using a supported
goniometer, the Sensor orientation parameters (Sensor Zenith and Sensor Azimuth)
Metadata Attributes for these Spectra can be computed and assigned using this
function.
The supported goniometers are those that use the following measurement pattern.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 108 of 161
Figure 59: Data capture sequence of the RSL FIGOS/LAGOS system
A full directional set consists of 66 Spectra. These Spectra must be grouped under a
single node in the Data Hierarchy Tree.
To calculate goniometer angles for a set of Spectra:
• From the Main menu select
Special Functions
and
Gonio angle calculation
.
Figure 60: Goniometer angle calculation dialog
• In the data hierarchy browser of the Metadata Editor select the node that
contains the spectrodirectional measurement Spectra for which you wish
to calculate the Sensor directions.
If you have selected a valid node, SPECCHIO will scan the Spectra to extract the
required information. It will take a few seconds. Valid nodes have 66 or fewer
Methodik
33
Abbildung 26: Messreihenfolge einer Hemisphäre: Position 1 entspricht 0°/75° (Azimut/Zenit)
relativ zur "solar principle plane", Position 11 180°/75°. Auf diese Weise werden für die
"reflected radiance" und die "incoming diffuse radiance" je 66 Spektren pro Hemisphäre
gemessen, bei Winkelinkrementen von 30° in Azimut und 15° in Zenit. Abbildung modifiziert
nach Schopfer [67].
Die Halterung der beiden Voroptiken kann so gedreht werden, dass beide exakt im Lot nach
unten ausgerichtet werden können. Indem im Abstand von ca. 30 cm ein Spectralon
Referenzpanel unter die Voroptiken gehalten wird, kann für beide Sensoren eine
Referenzmessung durchgeführt werden. Referenzmessungen dienen dazu, die zum
Messzeitpunkt vorherrschenden atmosphärischen Bedingungen abzubilden und für das
Berechnen von Reflektanzen aus den gemessenen "reflected radiance" Daten die nötigen
Grundlagen zu erheben (vgl. Kapitel 2.1.2). Im weiteren Verlauf der Hemisphärenmessung
werden für den Sensor ASD 1 noch drei weitere Referenzmessungen durchgeführt, jeweils an
den Positionen 21, 43 und 65. Beim verwendeten Spectralon Referenzpanel handelt es sich
um ein Produkt der Firma Labsphere Inc.4, das aus Bariumsulfat gefertigt ist und annähernd
vollständig und isotrop reflektiert. Die Referenzplatte wird wie in Abbildung 27 ersichtlich ist
manuell positioniert. Kritisch zu beurteilen ist dabei die unmittelbare Nähe von Zenitbogen
und Messperson: Die diffusen Einstrahlungsverhältnisse werden dadurch beeinflusst.
4 Labsphere Inc.: http://www.labsphere.com.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 109 of 161
Spectra. You cannot select the Spectra directly.
• If there are fewer than 66 Spectra, enter the sequence numbers of the
missing Spectra, separated by commas, in the
Gaps
field.
• Click Calculate . This will fill the
Spectra + dummies:
box with the total
number of Spectra after the gaps are inserted. It should be exactly 66. It
will also fill the calculation box with the Spectra names, sequence
numbers and angles. For example,
1: 0/75 triticaa.001
where the gonio sequence of triticaa.001 is “1”, its Sensor Azimuth is “0” and its
Sensor Zenith is “75”.
• Review the results in these boxes carefully to see that they are correct. If
not, you can repeat any of the steps above and click again on Calculate.
• When the results in this box are correct, click Apply and the results will
be written to the database. After the calculation is completed, the
following
Update Info
box is displayed:
•
• Click OK and Close to close this dialog box.
Automatically, a new processing attribute value in the Processing Metadata Group is
added to each Spectrum that has its goniometer angle attributes computed by this
function.
It is OK to repeat this process. It will overwrite any previous results.
11.2 Managing Target-Reference Links
You will need to add links from your target Spectra to the related reference Spectra if
you want to use some of the Processing options of SPECCHIO.
Note The dialog boxes for viewing existing links and adding new links appear similar, but
there are important differences. Please compare the two dialog boxes carefully in the
following sections.
In these dialogs, users with Administrator permission can view and change any data.
Other users can view and operate on only the Campaigns they uploaded or for which
they are in the Research Group.
11.2.1 Viewing or deleting existing Target-Reference links
From the Main menu, select
Special Functions - Target-reference links
menu items.
The following dialog will be displayed. Ensure that the
Show existing links
tab is
selected in the centre panel.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 110 of 161
Figure 61: Viewing Target-Reference links
Selecting and expanding nodes within these data hierarchy browsers requires many
database accesses, and so can take a little time to complete.
There are two halves to this dialog, which function completely independently.
Left hand side
Use this side to select Target Spectra and see their related Reference Spectra.
Figure 62: Viewing Target Spectra and their related Reference Spectra
Use this hierarchy tree browser to select one or more Spectra. Enter an SQL
matching string in the
Filename restriction
field to restrict the Target Spectra
selected. (See
Section 10.2.1
.) This does not affect the display of the Spectra in the
hierarchy tree browser display. It only reduces the effectively selected Spectra to
those that also match the SQL matching string.
The File Names of all Reference Spectra linked to the selected Spectra will be shown
in the
Linked references
box. If you select the Campaign node and clear the
Filename restriction
field, all linked Reference Spectra in the Campaign will be
displayed.
If any Target Spectrum is displayed in the
Linked References
box, a link has been set
up in the wrong direction.
Target Spectra
link to
these Reference Spectra
(duplicates are removed
from this list)
To see the Reference
Spectrum for just one Target
Spectrum, click on the
Target Spectrum.
To see the Reference
Spectrum for a selection of
Target Spectra, enter an SQL
search string to select them
in this box.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 111 of 161
Because each Target Spectrum should not be linked to more than one Reference
Spectrum, the number of Reference Spectra displayed should be less than the
number of Target Spectra selected.
Right-hand side
Use this side of this dialog to select Reference Spectra and show their related Target
Spectra. It operates in the same way as the left-hand side, but views the links in the
reverse direction.
Figure 63: Viewing Reference Spectra and their related Target Spectra
If you select the Campaign node on the right hand side and clear the
Filename
restriction
field, all Target Spectra with Reference Spectra links will be displayed.
If any Reference Spectrum is displayed in the
Linked targets
box, a link has been set
up in the wrong direction.
Because each Reference Spectrum can be linked from many Target Spectra, the
number of Spectra shown in the
Linked targets
box can be many more than the
number of selected Spectra.
11.2.2 Deleting Existing Target-Reference Links
The option to delete existing links works simultaneously, but independently, on the
two halves of the
Show existing links
dialog.
When the Delete selected links button is clicked, all links to and from the Spectra
highlighted in the
Linked references
and
Linked targets
boxes will be deleted. It is
possible to highlight Spectra in one of these boxes only.
After deleting, the
Linked references
and
Linked targets
boxes will be updated. This
may take a little time.
Target Spectra link to these
Reference Spectra
To see the Target Spectra for
just one Reference Spectrum,
click on that Spectrum. There
can be more than one linked
Target Spectrum.
To see the Target Spectra for a
selection of Reference Spectra,
enter an SQL search string to
select them in this box.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 112 of 161
Figure 64: Deleting Target-Reference Spectra links
11.2.3 Adding new Target-Reference links
From the Main menu, select
Special Functions
-
Target-references links
menu items.
Click on the
Create new links
tab in the centre panel to display the following dialog.
(You can switch back and forth between the
Create new links
and
Show existing links
tabs at any time.)
Figure 65: Creating a single new Target-Reference Spectra link
Notes The left and right sides of this dialog do not operate independently, unlike the dialog
which displays links in the previous section.
The positions of the Target Spectra and Reference Spectra boxes in the central panel
are reversed when compared with the position of these boxes in the previous
section.
To link a single Target and Reference Spectrum:
• Using the left-hand side
Targets
hierarchy tree browser, locate and select
the Target Spectrum you wish to link. Its name will be displayed in the
Target Spectra box.
When the Delete button is
pressed, all links to and
from these highlighted
Spectra will be deleted.
The two halves of this dialog
still operate independently.
In this example, the two
selected Campaigns are not
even the same.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 113 of 161
• Using the right-hand side
References
hierarchy tree browser, locate and
select the Reference Spectrum you wish to link. Its name will be displayed
in the Reference Spectra box.
• Verify that the correct Target and Reference Spectra are displayed in the
central
Target Spectra
and
Reference Spectra
boxes respectively. Ensure
that they are also each in the correct box and not reversed.
• Click Link to create the link.
Warnings Do not click the Link button twice. This will add the link twice, even though it is still
displayed only once. Duplicated links generally do not cause problems in later
processing, but may cause confusion if you later attempt to delete a duplicated link.
Before creating any link, check that the link does not already exist so you do not
duplicate it.
It is also possible to create multiple links simultaneously by selecting multiple Target
Spectra.
In addition, if multiple Reference Spectra are highlighted, as each Target Spectrum is
processed, the Reference Spectrum with the closest Acquisition time Metadata
Attribute is selected for linking.
Figure 66: Referencing of reference panel Spectra by target Spectra based on
timeline information
This function is useful for campaigns where Target and Reference Spectra are
collected in separate Spectral files, for example, when using the ASD
spectroradiometer in radiance mode.
Notes The target-reference links assigned in this manner are a simplification. A more
scientific approach to calculating reflectance factors from target and reference
radiances is implemented in the Matlab based Reflectance Calculator.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 114 of 161
Figure 67: Creating multiple new Target-Reference Spectra links
To create links between multiple targets and references:
• Using the left-hand side
Targets
hierarchy tree browser, locate and select
the Target Spectra you wish to link. You can do this by highlighting
multiple Spectra or by highlighting a hierarchy node (as shown in Figure
67 above).
• If required, refine your selection by entering an SQL matching string in
the related
Filename restriction
field. (See
Section 10.2.1
.)
• Using the right-hand side
References
hierarchy tree browser, locate and
select the Reference Spectra you wish to link. You can do this by
highlighting multiple Spectra or by highlighting a hierarchy node (as
shown in Figure 67 above)
• If required, refine your selection by entering an SQL matching string in
the related
Filename restriction
field. (See
Section
10.2.1
.)
• Verify that the correct Target and Reference Spectra are displayed in the
central
Target Spectra
and
Reference Spectra
boxes respectively. Ensure
that they are also each in the correct box and not reversed.
• Click Link to create the links according to the method described above.
11.2.4 Creating Data Links in the Metadata Editor
The dialog the link Targets and Reference as introduced in the above sections is set
up to deal with single links between target and reference spectra.
For a more flexible approach Data Links can also be created in the Metadata Editor
using a basic number input field to enter spectrum ids (Figure 68). Multiple entries
are possible.
The easiest way to populate the field with a spectrum id is to open a Data Browser in
a parallel window and just copy/paste spectrum ids as required. Information on
referenced spectra can be shown as spectrum report by clicking the ‘Show linked
spectrum’ button.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 115 of 161
Figure 68: Editing of Data Links in the Metadata Editor
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 116 of 161
12 Data Maintenance
This section describes the functionalities of SPECCHIO’s Main Menu tab
Data
Maintenance
. You will find information on how to use these functions. Note that most
of these functions require specific user permissions:
12.1 Remove data
Spectra, hierarchies or entire Campaigns can be removed from the database using
the Data Remover.
Users can only remove data from the database if they are a member of the
Campaign’s Research Group (see
Section 6.3 Research Groups and Accessing
SPECCHIO Campaigns
. A user with Administrator permissions can remove all
datasets irrespective of their owner.
To remove data:
• From the Main menu select
Data Maintenance
-
Remove data
. The
following window is displayed. Only the Campaigns which you have
permission to remove will be displayed:
Figure 69: Data Remover dialog
• Use the Spectral data browser to select the nodes or Spectra that you
wish to remove. Multiple Spectra and nodes can be selected by using the
Shift and Control keys. You can select any node except the
Specchio
database root node. Selecting a Campaign node is valid.
• Click Remove. All data that is below the selected node or nodes will be
deleted. For example, if a Campaign is selected then all hierarchies and
Data$Maintenance
•Remove!data
•Load!sensor!definition
•Instrumentation!admin
•Reference!panel!admin
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 117 of 161
Spectra belonging to this Campaign will be deleted. In addition, all
Metadata that has been entered for the deleted objects will be removed
along with the objects.
• Close the dialog box by clicking Close.
Warning There is no further prompt to warn you. The data are deleted as soon as you click
the Remove button. This action cannot be undone.
Removing data can take a long time. A progress bar shows which hierarchy is
currently being removed.
12.2 Load sensor definition (Obsolete)
A new Sensor can be defined by loading a Sensor definition file. Once defined, it will
appear in the dropdown list of Sensor names which appears at several places within
SPECCHIO.
Note This function is essentially obsolete as new sensors are automatically generated upon
loading. The only information that usually may be a bit sparse is the exact sensor
name or type designator as these are taken from information generated by the file
readers in turn. Editing sensor information must be done directly on the database by
an administrator, e.g. by using MySQL Workbench.
Warning Take care when defining new sensors. There is no method of editing or inspecting
Sensor information using the SPECCHIO Client. Therefore, there is no method of
confirming that your data was entered correctly, so prepare your input Sensor
definition file very carefully.
To load a Sensor Definition file:
• From the Main menu select
Data Maintenance
-
Load sensor definition
.
The following dialog is displayed:
Figure 70: Read Sensor Definition File dialog
• Click Browse, navigate to the Sensor definition file and open it.
• Click OK to read the file and insert the new Sensor into the database.
Sensor definition files are a proprietary format tab separated text file that can be
edited in a text editor or in Excel. The file format is as follows (the tab positions have
been lined up for clarity):
Name Description Company Type no no of channels
<sensor name> <sensor descr> <company name> <type number> <no of channels>
Band Average Wavelength(nm) FWHM (nm)
<band number> <wavelength> <fwhm>
<band number> <wavelength> <fwhm>
<band number> <wavelength> <fwhm>
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 118 of 161
:
:
The first two rows define basic parameters of this Sensor, the first row being the
names of the five parameters which are required and the second row being their
values. These parameters must be provided in the given sequence. The names in the
first row are required and must be correct.
<sensor name>
The name by which this new Sensor will be known to
SPECCHIO. It will appear in the Sensor Metadata Attribute’s
dropdown list.
<sensor descr>
A short description of this Sensor
<company
name>
The name of the company which manufactures this Sensor. It
must exactly match the value in the short_name column for
one of the manufacturers in SPECCHIO’s predefined
manufacturer’s table. See Appendix B: Predefined Manufacturer
Table for the list of manufacturers and their short_name
values.
<type number>
SPECCHIO will attempt to match this integer number to the
Instrument Type number in Spectrum files when they are read.
This is used to automatically set the Sensor in the Spectrum’s
Metadata.
<no of channels>
The number of frequency channels measured by this Sensor.
This must be the same as the number of lines in the Band
Table in this file.
The third row must have the headings of the parameters for each band.
Rows 4 to 3 + <no of channels> must have three numeric values on each line.
<band number> An integer number, starting at 1 and incrementing for each band in
the Band Table.
<wavelength> The central wavelength of this band in nanometers.
<fwhm> The Full Width at Half the Maximum value for this band. The value
is difference of the upper and lower frequencies in nanometers.
This value is presently ignored. (It is included for possible later new
features.)
Notes on the file format.
• There must be exactly one tab character between each field.
• The lines must not start with a tab character.
Example
An Excel view of a Sensor Definition File.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 119 of 161
Figure 71: Part of a Sensor definition file being edited in Excel
A text file view of the same file. Note that the tab positions do not appear to line up
when displayed this way.
Name Description Company Type no no of channels
ASD FSFR ASD Fieldspec FR ASD 4 2151
Band Average Wavelength (nm) Full Width at Half the Maximum FWHM (nm)
1 350 2
2 351 2
3 352 2
4 353 2
5 354 2
6 355 2
7 356 2
8 357 2
9 358 2
10 359 2
11 360 2
12 361 2
13 362 2
14 363 2
15 364 2
16 365 2
17 366 2
18 367 2
19 368 2
20 369 2
21 370 2
12.3 Instrument Administration
Note Any user can open the Instrument Administration dialogs but only Administrators can
save changes to the database.
For each Instrument in the SPECCHIO database, the following fields are stored.
Field name
Format
Comments
Instrument
name
Text string
This name will appear in the
dropdown list of Instruments.
Instrument
owner
Selected from dropdown list of
Institutes
This Institute list is created in the
Create new institute dialog. See
Section 3.4.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 120 of 161
Serial
number
Text string
The Instrument’s physical serial
number.
Sensor
Selected from dropdown list of
Sensors defined in database
Defines the Instrument type or
model for this particular
Instrument.
Pictures
JPEG image (on some users’
computers, PNG, GIF or TIFF
may also be supported.)
Multiple pictures of the
Instrument may be uploaded to
each Instrument definition.
Calibrations
See
Section
12.3.1 Instrument
Calibrations
.
Multiple Calibrations make up the
Calibration history for each
Instrument.
To open the Instrument Administration dialog:
• From the Main menu select
Data Maintenance
and
Instrumentation
Admin.
.
• Click in the
Instrument Data
tab.
Figure 72: Instrument Data Editor
The various parts of this dialog and their operation are explained in the following
table.
Instrument
Selector
This dropdown list includes all Instruments in the SPECCHIO
Instrument Table. Use this to select which Instrument you wish to
view or edit.
Instrument
Data
There are two related boxes with this title. One shows the data for
the Instrument Selected by the Instrument Selector. You can
modify the data displayed in this box. The other has two buttons.
Reset
Click this button to reset the display in the Instrument Data box
with information re-read from the database.
Instrument
Selector
Calibration information and
graphs will be shown here
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 121 of 161
Update
Click this button to write any updated data entered into the fields
in the Instrument Data box back to the database. This button is
only active for users with Administration permission.
Pictures
Any picture of the Instrument uploaded to the database is
displayed in this box.
Calibrations
This box shows all Calibrations which have been uploaded for this
Instrument.
Insert new
instrument
Click this button to create a new Instrument in the database. Set
the correct details for this new Instrument in the Instrument Data
fields, and then click Update to write this new data back into the
database. This button is only active if you have Administrator
permissions.
Delete
instrument
Click this button to delete the currently displayed Instrument. You
must click OK in a confirmation dialog box which is displayed
before the Instrument is deleted from the database. This button is
only active if you have Administrator permissions.
An Instrument cannot be deleted while still referenced by one or
more Spectra in the database.
Each Instrument can have one or more pictures loaded. JPEG images are supported
on all platforms. On some computer systems, GIF, PNG and TIFF may also be
supported. It is recommended not to load large images to avoid lengthy upload
times. Images should generally be re-sampled to be about 400 to 500 pixels high or
wide.
To add a picture to the currently displayed Instrument definition:
• Right click anywhere within the
Pictures
box. A menu will be displayed.
• Click
Add picture
from this menu. A file selection dialog box is displayed.
• Select the image file you wish to upload. The image will appear in the
Pictures
box with a space for a caption below it.
• Click within the caption entry control and type in your required image
caption.
• Click the Update button to load the image and caption to the database.
(Larger images may take a short while to upload.)
Note If you decide against uploading your new selected image, click the Reset button. Do
not right click and select
Remove Image
. This may not yield the desired result when
Update is clicked.
To remove a picture from the currently displayed Instrument definition:
• Right click on the image you wish to remove. A menu will be displayed.
• Click
Remove picture
from this menu. The picture and its caption will
disappear from the
Pictures
box.
• Click Update to write this change to the database.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 122 of 161
12.3.1 Instrument Calibrations
Note SPECCHIO versions 3.1 and higher automatically insert wavelength calibration data
upon spectral file loading. The manual handling of instrument calibrations as
described here is essentially no longer required.
Each Instrument has an optional Calibration History with one or more Calibrations.
When Spectra from an Instrument are processed, the Instrument’s Calibration
History is searched for the most recent Calibration at the time each Spectrum was
acquired. If there is no Calibration History, or the acquisition time precedes the date
of the first Calibration, the Instrument’s Sensor definition is used instead.
For each Calibration, the following information can be stored.
Field name
Format
Comment
Calibration
date
Date/time
selected by
date picker
The date on which this Calibration was performed.
Where possible, SPECCHIO will use the Calibration
with the most recent date and time which is prior to
the Acquisition date/time of the Spectrum being
processed.
Calibration
number
Integer
The sequence number of this Calibration. If there is
insufficient date information, and a Calibration
Number is given in the Spectrum’s Metadata,
SPECCHIO will attempt to select a Calibration based
on this Calibration sequence number. It usually starts
at 0 and is incremented for each new Calibration.
However, this is not a SPECCHIO requirement.
Comments
Text String
A comment about this Calibration.
Calibration
wavelengths
Floating
point array
Table of wavelengths of the centre of each band. The
number of entries in this table must match the
number of entries in the Sensor definition for this
Instrument.
Calibrations can be added from a file or manually entered. If added from a file, they
are complete Calibrations. If added manually, they are place markers only and do not
contain enough information to be used as a Calibration. They should be replaced
with a Calibration uploaded from a file when that information is available.
SPECCHIO supports the following formats for loading Instrument Calibrations.
• Unispec SPU files
• SpectraVista SIG files (Only HR-1024 has been tested)
Note ASD calibration files cannot be loaded as SPECCHIO calibration files. The Spectra
from these Instruments are adjusted according to the calibration within the ASD
Instrument itself and so there is no requirement to load them here.
SPECCHIO reads the centre wavelengths of the Instrument bands from these files,
but ignores the Spectrum values when reading these files as Calibration files. In
normal operations, the same file which is used to load an Instrument Calibration
would also be read by the general Campaign Spectra load process (see
9.1 Overview
of SPECCHIO Data Loading
).
Note Other file formats which also store centre wavelengths of bands (such as Ocean
Optics, UniSpec single beam, GER 3700 or APOGEE) are not yet supported by
SPECCHIO for Calibration file reading. Attempting to use these file formats may lead
to unpredictable results.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 123 of 161
To add a new place marker Calibration:
• Select the correct Instrument to display its data.
• Right click anywhere within the
Calibrations
box. A menu is displayed.
• Select
Add calibration
from this menu. A new Calibration with empty
information is immediately loaded to the database and displayed.
• Enter the required Calibration identification parameters. You cannot enter
the actual calibrated values.
• Click Update to write the new Calibration information back to the
database.
To add a new complete Calibration from a file:
• Select the correct Instrument to display its data.
• Right click anywhere within the
Calibrations
box. A menu is displayed.
• Select
Add calibration from file
from this menu. A file selection dialog is
displayed.
• Select the Calibration file (Unispec SPU file or SpectraVista SIG file) you
wish to upload.
To remove a Calibration:
• Right click anywhere over the Calibration you wish to remove from this
Instrument’s information. A menu is displayed.
• Select
Remove calibration
from this menu. The Calibration information is
immediately removed from the database and remove from the display in
the
Calibrations
box.
Warning Calibration removal is immediate and cannot be undone. Please take care not to
select this option for Calibrations you do not wish to remove.
12.4 Reference Panel Administration
Note In order to change or remove Reference Panel information, you must be logged in as
an Administrator.
SPECCHIO stores a table of Reference Panel information. The Reference Panels in
this list appear in the dropdown list for the Reference Metadata Attribute in the
Instrumentation Group. This Metadata Attribute is used in some functions of the
Space Processing Network described in
Section 13 Interactive Processing using Space
Networks
, and in particular during the reference panel correction procedure as
described in
Section 13.4.2 Reference Panel Correction Factors
and
Section 13.4.3
Correct for Reference Panel Non-Idealness
.
Each Reference Panel record holds the following information.
Field name
Format
Comment
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 124 of 161
Reference
name
Text string
This name will appear in the dropdown
list of Reference Panels, which is used to
set the Reference Metadata Attribute in
the Instrumentation Group.
Reference
owner
Selected from dropdown
list of Institutes
This Institute list is created in the Main
Window. See
Section 4.1
.
Serial
number
Text string
The Reference Panel’s physical serial
number
Reference
brand
Selected from predefined
dropdown list
There is presently only one brand
(Spectralon) defined in SPECCHIO. If you
require further brands to be added, speak
to your System Administrator.
Pictures
JPEG image (on some
users’ computers, PNG,
GIF or TIFF may also be
supported.)
Multiple pictures of the Reference Panel
may be uploaded to each Reference Panel
definition.
Calibrations
See section
12.4.1
Reference Panel
Calibrations
.
Multiple Calibrations make up the
Calibration history for each Reference
Panel.
When the SPECCHIO database is installed, the table of Reference Panels is empty.
To open the Reference Panel Administration dialog:
• Select
Data Maintenance
and
Instrumentation Admin.
from the menu on
the Main Window.
• Click in the
Reference Data
tab.
Figure 73: Reference Data Editor
Instrument
Selector
Calibration information and
graphs will be shown here
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 125 of 161
The operation of the Reference Panel dialog is analogous to that for Instruments.
Refer to
Section Instrument
and use the instructions in that section and its sub-
section on
Instrument Calibrations
to understand the operation of Reference Panel
viewing and maintenance.
12.4.1 Reference Panel Calibrations
Each Reference Panel can have one or more Calibrations defined. For each
Calibration, the following information can be stored.
Field name
Format
Comment
Calibration
date
Date/time
selected by
date picker
The date on which this Calibration was performed.
Where possible, SPECCHIO will use the Calibration
with the most recent date and time which is prior to
the Acquisition date/time of the Spectrum being
processed.
Calibration
number
Integer
The sequence number of this Calibration. If there is
insufficient date information, and a Calibration
Number is given in the Spectrum’s Metadata,
SPECCHIO will attempt to select a Calibration based
on this Calibration sequence number. It usually starts
at 0 and is incremented for each new Calibration.
However, this is not a SPECCHIO requirement.
Comments
Text String
A comment about this Calibration.
Calibration
table
Table
A table of calibration values at various wavelengths
with the following fields.
Wavelength The wavelength in nanometres at
which the Calibration values apply
Rho The reflectance factor of the panel at this
wavelength, in the range 0 to 1
Sigma The uncertainty estimate of the Rho value,
expressed as a standard deviation
Calibrations can be added from a file or manually entered. If added from a file, they
are complete Calibrations. If added manually, they are place markers only and do not
contain enough information to be used as a Calibration. They should be replaced
with a Calibration uploaded from a file when that information is available.
Calibrations do not need to be loaded in chronological order, but they will be used
according to their dates.
The Calibration file supplies the Wavelength/Rho/Sigma table. The Calibration date,
Calibration number and comments must be manually updated after the table is
loaded. This file is formatted as follows…
• A tab separated text file, usually with TXT extension.
• The first line must be a heading line. Its first value must be exactly “wvl” and
there must be a heading for the Rho column.
• The entry for each wavelength is on a new line.
• The values on each line are in the sequence Wavelength, Rho and Sigma.
• The Sigma value is optional. However, if it is present, it must be present on all
lines, and there must also be a heading for it in the first line of the file.
Note For Spectralon calibration files as provided by LabSphere, if required, uncertainty
information must be added by the user to the file before loading.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 126 of 161
Example
wvl rho sigma
250 0.98 0.02
251 0.98 0.02
252 0.981 0.02
253 0.98 0.02
254 0.98 0.02
255 0.981 0.02
256 0.98 0.02
257 0.98 0.02
258 0.98 0.02
259 0.981 0.02
260 0.98 0.02
261 0.98 0.02
262 0.978 0.02
263 0.979 0.02
:
:
Before loading a calibration file, a Sensor definition fitting the wavelengths of the
calibration must be present in the database. In the case of Spectralon reference
panels, the Sensor definition is the Perkin-Elmer Lambda 19 Sensor.
To add a new place marker Calibration:
• Select the correct Reference Panel to display its data.
• Right click anywhere within the
Calibrations
box. A menu is displayed.
• Select
Add calibration
from this menu. A new Calibration with empty
information is immediately loaded to the database and displayed.
• Enter the required Calibration identification parameters. You cannot enter
the actual calibrated values.
• Click Update to write the new Calibration information back to the
database.
To add a new complete Calibration from a file:
• Select the correct Reference Panel to display its data.
• Right click anywhere within the
Calibrations
box. A menu is displayed.
• Select
Add calibration from file
from this menu. A file selection dialog is
displayed.
• Select the Calibration file you wish to upload.
To remove a Calibration:
• Right click anywhere over the Calibration you wish to remove from this
Instrument’s information. A menu is displayed.
• Select
Remove calibration
from this menu. The Calibration information is
immediately removed from the database and remove from the display in
the
Calibrations
box.
Warning Calibration removal is immediate and cannot be undone. Please take care not to
select this option for Calibrations you do not wish to remove.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 127 of 161
13 Help Functions
The Help functions do not affect your data or the SPECCHIO Database. They are
selected from the menu on the Main Window.
13.1 List available Metadata Elements
This function will launch your computer’s default text editor and open a text file
which lists all of the Metadata Attributes which are supported by the connected
database.
It shows the Metadata Groups, the Attributes in each Group, and the data type for
each Attribute.
The data type values and their meanings are:
binary_val
The value is a binary stream of data representing an
image, such as a JPEG image file.
boolean_val
A Boolean value. True if box is checked, False otherwise.
datetime_val
The value is selected from a date and time picker.
double_val
A floating point value
int_val
An integer value
string_val
An alphanumeric string or a PDF file
taxonomy_id
The value is chosen from a drop down list.
spectrum_id
An integer that refers to the id of a spectrum in the
database.
13.2 Get SPECCHIO user contacts
This option is available to SPECCHIO administrators only.
It returns a list of user emails to enable the distribution of information about the SPECCHIO
database, such as upgrade instructions or maintenance windows.
13.3 About
This function shows the version of the SPECCHIO Client that you are running and
provides a link to the SPECCHIO web site.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 128 of 161
14 Publishing Data to ANDS
Data from SPECCHIO can be published to the Research Data Australia (RDA) service
of the Australian National Data Service (ANDS). From the ANDS website...
Research Data Australia, the flagship service of the Australian National Data Service
(ANDS), provides a comprehensive window into the Australian Research Data Commons.
Research Data Australia is an Internet-based discovery service designed to provide rich
connections between data, projects, researchers and institutions, and promote visibility
of Australian research data collections in search engines.
ANDS is partnering with research institutions and data producing agencies to bring about
four transformations to data - unmanaged to managed, disconnected to connected,
invisible to findable and single use to reusable - that will enable Australia's research data
to become a national strategic resource to support better, more efficient and defensible
research, and improved policy input.
ANDS is funded by the Australian Government through the National Collaborative
Research Infrastructure Strategy (NCRIS) and the Education Investment Fund (EIF)
Super Science Initiative.
See http://www.ands.org.au/ and https://researchdata.ands.org.au/ for more
information on ANDS and this service.
In SPECCHIO, you can only Publish data for which you are the owner or a member of
the Research Group.
Publication from SPECCHIO involves these key steps.
1. Decide on the Spectra to be grouped together and published as a Collection. The
Spectra can be all of a Campaign, part of a Campaign, or even be selected from
multiple Campaigns.
2. Ensure that the Spectra in the Collection are complete and meet the minimum
Metadata requirements for Publishing. See the following page for a description of
Metadata used for Published Collections.
3. Decide on the Principal Investigator, and ensure their SPECCHIO User
information is correct and up-to-date.
4. Publish the Collection, indicating the Collection name, Collection description and
Principal Investigator.
5. SPECCHIO generates an ANDS Collection Key for this Collection and writes it to
all Spectra in the Collection key Metadata Attribute in the Data Portal Group.
The ANDS Collection Key can be used to identify the Spectra in this Collection by
searching for it using SPECCHIO’s Query Builder. Each Spectrum will have one ANDS
Collection Key Metadata value for each Collection in which it has been included.
Warning Once Published, there is no way to retract the Publication using the SPECCHIO client.
You will need to approach your System Administrator if you have Published in error.
When data are published, SPECCHIO reformats the Metadata from the selected
Spectra into the RIF-CS format, which is an XML format involving two files. It then
places those files in a discoverable location on the server. On a regular basis
(currently daily), the RDA service inspects this location. If there are any RIF-CS files
present, it copies them, validates them and, if they pass validation, publishes the
Collection description on RDA for general access. Information about the RIF-CS
format can be found on the ANDS website referred to at the start of this Chapter.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 129 of 161
Note The RIF-CS file is a logical reference to your Spectral data, so it’s important to ensure
that this data remains valid and accessible in the SPECCHIO database. This will
ensure that it can be provided at a later time to those that request it based on the
Published ANDS information.
In the Publication step, the SPECCHIO server fills RIF-CS fields using the following
Metadata. Most Metadata is either automatically generated by SPECCHIO itself, or
drawn from data set during the creation of user accounts and/or the publication
process. Some other Metadata, however, must first be set in the metadata editor if
you wish it to be sent to ANDS.
RIF-CS Field
SPECCHIO Field
Notes
Collection Key
The collection key generated by the
SPECCHIO server. The University of
Wollongong’s server generates keys of the
form uow.edu.au/SL/COL/nnnn, where
nnnn is a serial number generated by the
SPECCHIO server. This value will be as ANDS
Collection Key in SPECCHIO’s metadata.
Generated by
SPECCHIO.
Collection
Name
The collection name value specified in the
ANDS Collection Key dialog. This value will
be saved as ANDS Collection Name in
SPECCHIO’s metadata.
Mandatory, filled
during publication.
Collection
Description
The collection description specified in the
ANDS Collection Key dialog. This value will
be saved as ANDS Collection Description in
SPECCHIO’s metadata.
Mandatory, filled
during publication.
Group
The institution of the principal investigator.
Mandatory, filled
during account
creation.
Date
The earliest Acquisition Time of any
Spectrum in the Collection.
Mandatory.
Identifier
(local)
An identifier generated by the SPECCHIO
server. The University of Wollongong’s server
generates local identifiers of the form
Collection~nnnn where nnnn is a serial
number.
Generated by
SPECCHIO.
Identifier (doi)
The Digital Object Identifiers associated with
every Spectrum in the Collection.
Optional.
Location
(electronic)
The e-mail address of the Principal
Investigator.
Mandatory, filled
during account
creation.
Location
(physical)
The Department and Institute of the Principal
Investigator.
Mandatory, filled
during account
creation.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 130 of 161
Coverage
(temporal)
The range of Acquisition Times present
amongst Spectra in the Collection.
Optional.
Coverage
(spatial)
The Location Name of the last Spectrum in
the Collection.
Optional.
RelatedObjects
The ANDS Party Identifier of the Principal
Investigator.
Filled by
SPECCHIO.
Subjects
The FOR Codes of all Spectra in the
Collection.
Mandatory.
Rights
The Data Usage Policies of all Spectra in the
Collection, or a standard-form text generated
by the SPECCHIO server if no Spectra have
policies.
Mandatory.
In addition, User Information for the nominated Principal Investigator is copied to the
published files as follows (The User Information for the User doing the SPECCHIO
publication operation is not used, unless of course, it’s the same User.)
RIF-CS
Field
SPECCHIO Field
Key
ANDS Party Identifier.
Group
Department and Institute.
Name
First Name and Last Name.
Description
Description
Relation
The ANDS Collection Key of the Collection just published.
Subjects
The FOR Codes of the Collection just published.
Refer to 4.1 for more information, especially information on the ANDS Party
Identifier.
To Publish a Collection to ANDS:
• Ensure that all Spectra to be included in the Collection have their
Metadata set correctly as described above.
• From the Main menu select
Data Processing & Output -
Browse data
hierarchy
.
• Set the
Show only my data
option.
• Select the Spectra to be included in the Collection in the Data hierarchy
browser.
• Review carefully that you have selected the correct Spectra. (Important
– do not skip this step, because you cannot retract once you have
Published.)
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 131 of 161
• Click Publish. The following dialog, which shows all members of the
Research Group, is displayed:
• Fill in the collection name and description. If the selected set of Spectra
have already been published as a Collection, the dialog should be pre-
filled with the existing name and description.
• Select the principal investigator from the list by clicking on the name.
• Click Submit. If the Collection Metadata is successfully validated, the
Publication will proceed and a dialog box will be displayed showing the
ANDS Collection Key that was assigned to this Publication. (If validation is
not successful, an error dialog will be displayed.)
• Click Close to close the ANDS Collection Key dialog.
Alternatively, you can select
Build query
instead of
Browse data hierarchy
in the
above process, and build a query to select the Spectra you want included in your
Collection.
If any mistake in the Metadata is detected for the Spectra in your Collection,
SPECCHIO will display an error dialog similar to the following:
Press Details… to display the error message returned from the server. This error
message should explain the problem with the proposed Collection, as shown below.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 132 of 161
Press Dismiss to remove the dialog. Correct the error and retry the Publication
process.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 133 of 161
15 Interactive Processing using Space Networks
For an introduction to the concept of the Spaces and the Space Factory please refer
to
Section 5.2
Spaces and the Space Factory
.
Interactive, flexible and configurable data processing is based on the concept of the
Space Network. Such networks consist of processing modules and data
sinks/sources, connected by directed edges.
Figure 74: Transformation into a new space by a processing module
Processing modules are effecting a transformation on a space, i.e. the Spectral data
vectors of the input space are transformed to an output space. The algorithm of the
processing module defines the dimensionality of the resulting space. This is
illustrated in Figure 74 with an input space of dimensionality N being transformed
into another discrete space of dimensionality M. Although processing modules tend
to have singular input/output in most cases, they may have multiple inputs and
generate multiple outputs.
Pressing the Process button in the Data Hierarchy Browser or Query Builder starts
this processing tool. The window below appears.
Figure 75: Space Network Processor interface with the Query Builder in the
background
A number of spaces are placed on the processing plane of the Space Network
Processor, depending on the query built in Query Builder.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 134 of 161
Figure 75 shows a space containing 66 Spectra, created based on a selection in the
Query Builder.
Figure 76 shows the common elements of the Space Network Processor: processing
plane, spaces, edges, context sensitive menu of the processing plane and the Run
button.
Figure 76: Elements of the Space Network Processor
15.1 Graphical Representations of Spaces and Modules
Spaces are depicted as rectangular boxes as shown here.
Figure 77: Examples of the graphical representation of a space (left) and a
processing module (right)
Each space displays the following information:
[Integer] A unique number (zero in the example below)
Type A space type, which can be ‘Sensor/Instrument Space’ if loaded
from the database or ‘Spectral Space’ if generated by a processing
module.
D A dimension D, which is equal to the number of Spectral bands of
the Spectra held by this space
!"#$%&&'()*+,-(%*.+-$%* /0)%*
1#(2%32*&%(&'45%*
6%(7*#8*29%*
!"#$%&&'()*+,-(%*
!"#$%&&'()*:#07,%* .+-$%* ;7(*<7=#(*
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 135 of 161
# Vectors A number of vectors, which is equal to the number of Spectra held
by this space
Modules are also depicted as rectangular boxes, but their contents are quite
different.
Figure 78: Examples of the graphical representation of a space (left) and a
processing module (right)
Proc. module
A module name
Settings
Information about the configuration (if applicable)
Status
The current processing status (text and progress bar). During
processing, the progress is shown by the progress bar.
Spaces and modules can be rearranged by dragging with the mouse. The edges are
updated automatically.
A group of elements can be moved as single block. Select the elements by dragging
a box around them using the mouse, then drag the selection (Figure 79).
Figure 79: Multiple-selection of elements after dragging a box around them
15.2 Adding Modules and linking with Spaces
To add a module, click the mouse menu button over some empty area of the
processing plane. This brings up the context sensitive menu shown in Figure 76.
Select ‘Add Module’ and a selection of the available modules will appear (Figure 80).
Select one of the modules, click ‘OK’ and a new module will be added to the
processing plane.
To connect the module with an input space, click the menu button over the module.
This displays the popup menu of the module (Figure 81). Select ‘Set Input Spaces’
and in the ‘Input Space Selection’ dialog select the number of the space to connect
and click ‘OK’ (Figure 81). Note that all spaces are given a unique number, which is
located on the top left of each space. Connecting an input space automatically
generates an output space, which is added to the processing plane and connected
with the module.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 136 of 161
Figure 80: Module selection dialog
Figure 81: Popup menu of a module (left) and the input space selection dialog (right)
15.3 Configuration of Modules
Some modules may need configuration. To display the configuration dialog of a
module click menu over the module and select ‘Configure’.
The possible configuration settings of each module are detailed in the respective
module description.
15.4 Processing Module Descriptions
15.4.1 Radiance to Reflectance Transformation
The module ‘Radiance to Reflectance’ is building ratios of target and reference panel
radiances:
As a prerequisite, datalinks of the type ‘Spectralon’ must exist between the target
Spectra and the corresponding reference Spectra. For an explanation of how such
links are created please refer to 8.5 and 11.2.
15.4.2 Reference Panel Correction Factors
This module retrieves the correction factors for all reference panels that were used
during data acquisition of the Spectra contained by the input space.
As prerequisites, a reference panel must be set for the input Spectra using the
Metadata Editor (Figure 82) and calibration data for the panel must have been
loaded to the database. Calibration data loading is a task of the system
administrator. For details on the reference panel administration see
Section
Instrument
.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 137 of 161
Figure 82: Reference panel setting in the Metadata Editor
If no calibration data can be found, a warning is displayed and the module creates
no output space.
Figure 83 shows a processing chain that selects the panel correction factors and plots
using a Spectral line plot (Figure 84).
Figure 83: Processing chain selecting the correction factors and plotting them as a
Spectrum.
Figure 84: Reference panel correction factors
15.4.3 Correct for Reference Panel Non-Idealness
This module applies calibration factors to correct Spectra for the reference panel
non-idealness. These correction factors are selected from the database using the
‘Reference Panel Correction Factors’ module.
The module requires two inputs: the Spectra to be corrected and the correction
factors. The input space selection dialog offers the according choices (Figure 85).
Figure 85: Input space selection for the ‘Correct for Panel’ module
Note that the dimensions of the spaces holding the Spectra and correction factors
must be identical (a check on the wavelengths is not carried out). In the example
shown in Figure 869 a waveband filtering is applied to the correction factors as these
were measured with a larger wavelength range.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 138 of 161
Figure 86: Space Processing Network for reference panel correction
The panel correction can be applied to radiance or reflectance values. However,
correcting radiances for the panel will only make sense if followed by a radiance to
reflectance conversion such that:
The correction procedure can handle multiple reference panels and multiple
calibration coefficients and will apply the correct coefficients to the appropriate
Spectra.
15.4.4 Delta
This module calculates a delta value, i.e. the difference between two inputs A and B:
Delta = A – B
where:
A, B = MxN matrix of M Spectra of dimensionality N
Thus, a delta vector is calculated by:
where m = {1…M}
This implies that the input spaces must have identical number of vectors and
dimensionalities.
The delta module takes two input spaces (Figure 87):
Figure 87: Input space selection for the Delta module
€
d
m=
a
m−
b
m
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 139 of 161
A possible use is the calculation of the difference between recorded reflectance and
reference panel corrected reflectance. Figure 88 shows a processing network for this
purpose and Figure 89 shows the according Spectral plots generated by the network.
Figure 88: Space Processing Network for the panel correction and delta calculation
Figure 89: Spectral plots of: input Spectrum (left), panel corrected Spectrum (middle)
and delta Spectrum (left)
15.4.5 Waveband Filter
Waveband filtering is used to cut wavelengths regions from Spectra. A typical
example is the removal of noisy wavebands caused by water vapour absorption.
The wavebands are freely configurable. The ‘Configure’ menu brings up the ‘Filter
configuration’ dialog (Figure 90).
Figure 90: Filter configuration window
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 140 of 161
To add a new filter region, click ‘New’ and enter the upper and lower wavelengths in
nanometres in the Filter Definition dialog (Figure 91).
Figure 91: Filter definition dialog
To remove an existing filter region, select the region in the list and click ‘Remove’.
15.4.6 Broadband and Narrowband Filters
The filters act on the element type of Sensor elements. They are useful to process
data of sensors that contain bands widely differing Spectral resolutions.
An example is the MFR sunphotometer, which features one panchromatic band and 6
narrower bands. The value ranges of the broad and narrow bands are very different
and even a simple plot cannot be created satisfactorily (see Figure 92). The
broadband channel has been placed at the likely centre wavelength of 673nm. As the
bands of the MFR are defined in the order of the broadband followed by the
narrowbands in the database, the plot shows two Spectral lines. Mixing broad- and
narrowbands should not be done from a physical point of view.
Figure 92: Spectral plot of broad- and narrowband MFR channels
Figure 93 shows a space processing network that illustrates the function of broad-
and narrowband filters.
Figure 93: A space processing network demonstrating the function of broad- and
narrowband filters
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 141 of 161
The plot shown in Figure 92 was generated by the ‘Spectral Line Plot’ of space
number 0.
Figure 94 shows the plots of space 2 (Broadband value as a scatter plot) and space 3
(Narrowband channels as a Spectral curve).
Figure 94: Plots of the narrowband channels (left) and of the broadband channel
(right)
15.5 Visualisation Modules
Visualisation modules can be attached to any space and do not generate an output
space but display a new window containing the respective plot or data explorer. A
visualisation module can be configured to produce a number of plots/explorers as
listed in the following sections. Data explorers are graphical components that allow
interactive data exploration.
The titles of the plot windows do contain information about the plot type and the
space the data was read from (Figure 95).
Figure 95: Information displayed in the window title
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 142 of 161
15.5.1 Spectral Line Plot
Use this plot to display one or more Spectra as continuous curves. Spectra are
plotted with an automatic colour shift from red to blue for easier interpretation.
Figure 96: Spectral line plot of snow avalanche reflectance Spectra
15.5.2 Spectral Scatter Plot
Use this plot to display values per Spectral band as singular points. This is useful to
plot e.g. data of sensors with just one band, e.g. MFR broadband, as single values
cannot be visualized as line plots. Scatter plots are also useful to show the variation
of the values per band. Figure 970 shows an example of the variation per channel for
several MFR sunphotometer readings.
Note: the generation of scatter plots of several Spectra with high dimensionality
tends to be quite slow.
Figure 97: Scatterplot showing the variation per channel for several MFR
sunphotometer readings
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 143 of 161
15.5.3 Gonio Sampling Points Plot
Note This function is currently unavailable due to licensing issues.
Use this plot to visualise the sampling positions of directional measurements acquired
by a goniometer system. The plot is not expecting any specific number of points or
particular geometry.
The sampling points are projected from their angular definition (zenith/azimuth) onto
a 2d Cartesian coordinate system (Figure 98).
Figure 98: Goniometer sampling point positions
15.5.4 Gonio Hemisphere Explorer
Note This function is currently unavailable due to licensing issues.
The Hemisphere Explorer allows the interactive exploration of a spectrodirectional
dataset, typically acquired by a goniometer system. Figure 99 shows an explorer
window displaying a LAGOS (Laboratory goniometer system) dataset (Schopfer
2008).
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 144 of 161
Figure 99: Gonio Hemisphere Explorer window
The explorer window comprises a number of components interacting with each other
as described hereafter.
The hemispherical plot (Figure 100) displays an interpolated surface of a specific
wavelength. Clicking the icon in the top-left of the plot brings up a control panel for
plot adjustments (rotations).
Figure 100: Interpolated 3D plot for a chosen wavelength (left) and rotation toolbox
to manipulate the 3D plot
Use the band selection slider to select a Spectral band for display (Figure 101). The
wavelength text field below is reflecting the wavelength of the chosen band.
Selecting a band triggers a re-plotting of both hemispherical plot and Spectral plot.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 145 of 161
Figure 101: Band selection slider and wavelength text field
The sampling points plot shows the sampling point positions projected onto a 2d
Cartesian coordinate system (Figure 102). One of these points is always selected
(indicated by the little square around it). Information about the selected point is
shown in the text fields on the right of the plot: filename of the respective Spectrum,
azimuth and zenith angles of the observation geometry. Changing the selected
sampling point (by clicking the mouse on another point) changes the Spectrum
displayed in the Spectrum plot automatically.
The azimuth angle is measured relative to the solar principal plane, i.e. 0° = principal
plane opposite of the illumination source.
Figure 102: Sampling point position plot and information about the selected sampling
point
The Spectral plot component displays the Spectrum of the selected sampling point
(Figure 103). A red, vertical line indicates the current wavelength as selected by the
band selection slider. The text fields on the right of the plot display Spectral statistics
of the current Spectrum.
Figure 103: Spectral plot component with wavelength indicator and Spectrum
statistic information
15.5.5 Time Line Plot
Use a time line plot to plot a Spectral band versus time. Figure 104 shows showing
the direct irradiance over time for an MFR sunphotometer band with centre
wavelength 496.4nm. The channel to be plotted can be chosen in the list below the
plot. Time information is retrieved from the sampling time of the Spectra.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 146 of 161
Figure 104: Time Line Plot showing the direct irradiance over time for centre
wavelength 496.4nm
15.5.6 Time Line Explorer
The time line explorer consists of a time line plot and a Spectral plot (Figure 105).
The red bar in the time plot indicates what Spectrum is plotted in the Spectral plot,
i.e. the Spectral plot shows the Spectrum taken at a certain time. The red bar in the
Spectral plot shows the currently selected Instrument channel, which is plotted
versus time in the time line plot.
Figure 105: Time Line Explorer window
The example given in Figure 105 is using MFR sunphotometer data. A removal of the
broadband channel is needed for the Spectral plot to work properly. The according
processing chain is shown in Figure 106.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 147 of 161
Figure 106: Processing chain for the exploration of the narrowband MFR channels in
the Time Line Explorer
15.6 File Export Module
File export modules can be attached to any space and do not generate an output
space but write the data to a file.
A file export module must be configured using its configuration dialog (Figure 107).
The dialog is identical to the one described in
Section 10.7.
Figure 107: File export configuration
The Spectral data written to the file reflects the data content of the space. The Metadata is
read from the database where possible.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 148 of 161
16 Matlab Integration
The Matlab environment is a well-established tool in engineering, research and
science. Matlab includes a Java Virtual Machine and thus allows the use of Java
classes within Matlab code.
To extract Java code for a Matlab query from the SPECCHIO client:
• From the Main menu select
Data Processing & Output
and
Browse data
hierarchy
. The Data Browser window will open.
• Using the Hierarchy Browser, select a Hierarchy node or a selection of
Spectra. The Spectra IDs will appear in the
Matching Spectra
box.
• Right click in the
Matching Spectra
box. A menu will appear.
• Select
Copy Matlab-ready query to clipboard
from the menu.
• Switch to the Matlab application (or other application) and paste the
query from the clipboard as you require.
It is also possible to use
Build query
instead of
Browse data hierarchy
in the process
above. See
Section 8 Data Selection & Output
for more information about using
these tools.
For more details on using Matlab with SPECCHIO data, please refer to the SPECCHIO
MATLAB Guide SPECCHIO_MatlabGuide.pdf.
Example Matlab-ready query from Data Hierarchy Browser
The following is an example of what is placed on the clipboard by the above process
when the data hierarchy browser is used.
query = Query('spectrum');
query.setQueryType(Query.SELECT_QUERY);
query.addColumn('spectrum_id')
cond = QueryConditionObject('spectrum', 'spectrum_id');
id_array = [65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84];
ids_list = java.util.ArrayList();
for i=1:size(id_array,2) ids_list.add(id_array(i)); end;
cond.setValue(ids_list);
cond.setOperator('in');
query.add_condition(cond);
ids = specchio_client.getSpectrumIdsMatchingQuery(query);
Example Matlab-ready query from Query Builder
The following is an example of what is placed on the clipboard by the above process
when the query builder is used.
query = Query('spectrum');
query.setQueryType(Query.SELECT_QUERY);
query.addColumn('spectrum_id')
cond = SpectrumQueryCondition('spectrum', 'measurement_unit_id');
cond.setValue('0');
cond.setOperator('=');
query.add_condition(cond);
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 149 of 161
cond = EAVQueryConditionObject('eav', 'spectrum_x_eav', 'File Name', 'string_val');
cond.setValue('triti%%');
cond.setOperator('like');
query.add_condition(cond);
cond = SpectrumQueryCondition('spectrum', 'sensor_id');
cond.setValue('0');
cond.setOperator('=');
query.add_condition(cond);
cond = SpectrumQueryCondition('spectrum', 'instrument_id');
cond.setValue('0');
cond.setOperator('=');
query.add_condition(cond);
ids = specchio_client.getSpectrumIdsMatchingQuery(query);
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 150 of 161
17 R Integration
R integration uses the rJava package. For more information please see www.specchio.ch.
To extract R / Java code for a R query from the SPECCHIO client...
• Select
Data Processing & Output
and
Browse data hierarchy
from the
menu on the Main Window. The Data Browser window will open.
• Using the Hierarchy Browser, select a Hierarchy node or a selection of
Spectra. The Spectra IDs will appear in the
Matching Spectra
box.
• Right click in the
Matching Spectra
box. A menu will appear.
• Select
Copy R-ready query to clipboard
from the menu.
• Switch to the R application and paste the query from the clipboard as you
require.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 151 of 161
18 Python Integration
Python integration uses the JPype package. For more information please see
www.specchio.ch.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 152 of 161
19 Glossary
The explanations of words and terms in this glossary include both those terms which
may not be familiar to some readers and those which have a specific meaning in the
context of SPECCHIO or this document.
ANDS
Australian National Data Service
This body hosts the Research Data Australia (RDA) service. SPECCHIO
supports publishing research data to this service. See
http://www.ands.org.au/ and
Chapter
14
Publishing Data to ANDS
.
Calibration
The process of linking an Instrument's response to a defined physical
input, for example, assigning the correct wavelengths to a band, or
assigning factors that describe the relationship between radiance and
recorded digital numbers.
Campaign
Refers to an activity during which Spectral samples were acquired. A
Campaign is defined in a contextual sense. It is not restricted by any
temporal or spatial properties.
Client-server
A commonly used networked computer architecture which involves a
“server”, which hosts a main computing facility such as a database or
computation facility, and a number of “clients”, which access the server
using a network, such as an intranet or the internet. SPECCHIO uses
this architecture.
FOR Codes
Field of Research Codes
This coding system was designed and developed by the Australian
Research Council (http://www.arc.gov.au/) to categorise areas of
research. FOR Codes are widely used across Australian research
bodies, such as universities and the CSIRO, and government
departments, such as the Bureau of Census and Statistics. Lists of FOR
Codes can be found at sites such as
http://www.abs.gov.au/ausstats/abs@.nsf/Products/6BB427AB9696C22
5CA2574180004463E?opendocument.
HTTP/HTTPS
Hypertext Transfer Protocol
HTTP is a very widely used network protocol, which is also used by
SPECCHIO Clients to communicate with the SPECCHIO Server. HTTPS
is the secure version of this protocol.
Instrument
In the context of this document, an individual spectroradiometric
Instrument. Such an Instrument will have a serial number and an
owner.
Instrument
Type
A model or type of spectroradiometric Instrument. It will be described
by a model number and manufacturer. In this document and the
SPECCHIO client, this term is used interchangeably with the terms
“Instrument Model”, “Sensor” and “Sensor Type”.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 153 of 161
MySQL
The world’s most widely used relational database management system
(RDBMS) (according to Wikipedia). It is the RDMS used by the
SPECCHIO Server. (See SQL.)
RDA
Research Data Australia
Australian National Data Service’s (ANDS) flagship service, which
provides a comprehensive window into the Australian Research Data
Commons. It is an Internet-based discovery service designed to
provide rich connections between data, projects, researchers and
institutions, and promote visibility of Australian research data
collections in search engines. See http://researchdata.ands.org.au/.
Reference
Panel
A calibrated reflective panel with known reflectance properties. It is
usually white, but may be grey or even close to black. A Reference
Panel can be measured under the same lighting conditions as a target
sample in order to get an approximation of the Spectrum of the solar
irradiance or to collect data for the conversion of a target radiance to
reflectance factors.
Reference
Spectrum
A Spectrum acquired over a Reference Panel.
Regular
Expression
In computing, a regular expression (often abbreviated “regex” or
“regexp”) is a sequence of text characters, some of which are
understood to be metacharacters with symbolic meaning, and some of
which have their literal meaning, that together can be used by a
regular expression processor to identify textual material of a given
pattern. See (Oracle Technology Network, 2013, Wikipedia, 2013) for
more information.
Sample
A Sample taken from a Target and measured, often at some later time,
under controlled conditions. A sample will generally have a collection
date/time and a measurement date/time, which will be different.
Sensor
See Instrument Type.
Spectralon
A particular brand of reflective reference panel, produced by
LabSphere. Spectralon panels are available in various configurations,
ranging from pure white to greyish and almost black. See Reference
Panel.
SQL
Structured Query Language (sometimes pronounced “sequel”)
A special-purpose programming language designed for managing data
held in a relational database management system (RDBMS).
Target
Spectrum
Spectrum acquired by measuring the target, where the target is the
object of interest.
TCP/IP
Transmission Control Protocol/Internet Protocol
The common designator to refer to the protocols used for internet
communication.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 154 of 161
White
Reference
Spectrum
A Spectrum acquired over a reference panel that has a reflectance of
close to 100%.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 155 of 161
20 References
Australian Research Council, 2008. Australian and New Zealand Standard Research
Classification (ANZSRC), http://www.abs.gov.au/ausstats/abs@.nsf/mf/1297.0
CSIRO, 2011. The Australian Soil Classification,
http://www.clw.csiro.au/aclep/asc_re_on_line/soilhome.htm
Hueni, A., Damm, A., Kneubuehler, M., Schläpfer, D., Schaepman, M., 2017. Field and
Airborne Spectroscopy Cross-Validation - Some Considerations. IEEE Journal of Selected
Topics in Applied Earth Observations and Remote Sensing
10(3), 1117 - 1135.
Hueni, A., Nieke, J., Schopfer, J., Kneubühler, M., Itten, K., 2009. The spectral database
SPECCHIO for improved long term usability and data sharing. Computers & Geosciences
35(3), 557-565.
Hueni, A., Tuohy, M., 2006. Spectroradiometer Data Structuring, Pre-Processing and
Analysis - An IT Based Approach. Journal of Spatial Science
51(2), 93-102.
Hüni, A., Kneubühler, M., 2007. SPECCHIO: a system for storing and sharing
spectroradiometer data. SPIE Newsroom(December 2007), 1-2, http://spie.org/x18220.xml.
Hüni, A., Nieke, J., Schopfer, J., Kneubühler, M., Itten, K., 2007. 2nd Generation of RSL's
Spectrum Database "SPECCHIO"
.
In: Proceedings Schaepman, M. E., Liang, S., Groot, N. E.,
Kneubühler, M. (Eds.), 10th Intl. Symposium on Physical Measurements and Spectral
Signatures in Remote Sensing, Davos (CH)
,
XXXVI, Part 7/C50, pp. 505–510.
Landgrebe, D., 1997. On Information Extraction Principles for Hyperspectral Data, Purdue
University, West Lafayette, IN, 34 pp.
Milton, E. J., Fox, N. P., Schaepman, M., 2006. Progress in Field Spectroscopy
.
In:
Proceedings Geoscience and Remote Sensing Symposium, Denver, CO, US, pp. 1966-1968.
NOAA, 2003. What is UTC or GMT Time?, http://www.nhc.noaa.gov/aboututc.shtml
Oracle Technology Network, 2013. Java Regular Expressions Tutorial,
http://docs.oracle.com/javase/tutorial/essential/regex/index.html
Wikipedia, 2013. Regular Expression Description,
http://en.wikipedia.org/wiki/Regular_expression
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 156 of 161
21 Document History
Version
Date
Author
Remark
0.1
15.12.2006
A. Hueni
First document version
1.0
14.02.2007
A. Hueni
Bug fixes and added features
1.1
18.10.2007
A. Hueni
Added txt file reader, multi-user capabilities,
db_config.txt file, Admin functions.
1.2
29.05.2008
A. Hueni
Added Spectral plot, Campaign export,
Metadata editor enhancements.
1.3
15.06.2008
A. Hueni
Added Campaign import function.
2.0
12.03.2009
A. Hueni
Major update: Added reference panel handling,
processing extension, Instrument and reference
panel tools. Updated export/import.
2.1
02.09.2010
A. Hueni
Added Instrument settings, Metadata
enhancements, time selection, SVC-HR1024
Instrument, Instrument calibration, Apogee
support.
2.2.0
02.05.2012
A. Hueni
D. Kuekenbrink
Added FGI HDF file support, generic Metadata,
target-reference link improvements, Metadata
editor enhancements, ASD binary file format,
Metadata date/time selection, beam
geometries.
2.2.1
A. Hueni
D. Kuekenbrink
Upgrade direct irradiance calculation for MFR-7,
query builder/hierarchy browser split, space
processing improvements, mixed folder data
loading, UniSpec file loader, SPECPR file loader.
3.0
21.06.2013
Peter Roberts
(Intersect)
UOW/Intersect overhaul: Database now
referenced via HTTP front end, ANDS Collection
export.
3.0.2
30.06.2014
Nicholas
Sheppard
(Intersect)
Updated the description of publishing to ANDS
and added a mapping of SPECCHIO fields to
RIF-CS output.
3.0.3
06/03/2015
Sandra
Carruthers
(UOW SEES)
Restructured the document, amended the
installation and setup instructions and added
diagrams for clarity.
3.1
05.03.2014
A. Hueni
Added calibration support and automatic
sensor/instrument definition.
3.2
13.05.2015
A. Hueni
L. Chisholm
Added information on new metaparamters for
geochemistry and others. Updates due to Data
Link upgrade. New input file formats added.
Document restructuring and editing to render it
more user friendly.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 157 of 161
3.2.1.5
02.11.2016
A. Hueni
Added information about the db_config file, the
SPECCHIO Preferences and the UTC Acquisition
Time, which is now used to calculate sun
angles.
Mentioned Python support, details are on
www.specchio.ch
3.3
20.02.2018
A. Hueni
S. Bertschi
General update of user guide. Added
information about new functions: spatial
support, hierarchy metadata storage, metadata
processing.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 158 of 161
Appendix A: Regular Expressions Tutorial
Regular expressions are widely used across the computer industry when text string
matching is required. They permit a rich array of sophisticated matching
functionality.
A complete explanation of regular expressions is beyond the scope of this tutorial.
Readers with a desire to see a complete list of all options should see the Java tutorial
at http://docs.oracle.com/javase/tutorial/essential/regex/index.html.
The following is a quick summary of the most useful regular expression functions for
SPECCHIO Users.
Simple string
A simple string containing no
special characters will match
those characters in a target
string, regardless of where
within the target string they
occurs.
mpl will match the string
example at the fourth
character.
Start of string
^
The character ^ will match
the beginning of the target
string.
^exa
will match example,
but ^xa will not.
End of string
$
The character $ will match
the end of the text string.
le$ will match example, but
pl$ will not.
Any character
.
The period character will
match any character.
a.c will match abc, aac,
adc, a7c and a-c but it will
not match ac or abbc.
Repeated
character
*
Asterisk causes the matching
to zero or more repetitions of
the preceding character.
ab*c will match ac, abc,
abbc or abbbc, but will not
match a7c or ahc. It will
match aac at the second
character and acc at the first
character, because there are
zero characters between the
a and c in those strings.
Repeated
characters
+
The plus sign causes the
matching to one or more
repetitions of the previous
character.
ab+c will match abc, abbc
or abbbc but will not match
ac.
Alternate
characters
[ ]
Strings enclosed within square
brackets will match any one
of the characters within the
brackets.
a[123]b will match a1b,
a2b or a3b only. It will not
match ab or any other
substring.
Character
ranges
[-]
Use – between [] to match
one of range of characters.
[0-9] matches any digit.
[a-z] matches any letter.
[a-z0-9] matches any digit
or letter.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 159 of 161
Escape
character
\
In order to match a special
character, precede it with the
backslash character.
Special characters are
[]\^$.|?*+(){}
Putting \ before other
characters often has a special
meaning, so should be
avoided.
\\ will match \
\. will match .
\* will match *
\[ will match [
Combinations
Any of the above search
methods can be combined.
^.c will match any string
with c as its second
character.
^abc$ will match the string
abc only. abcd or aabc will
not be matched.
1[abcd]+2 will match any
combination of the characters
a, b, c or d which occurs
between the digits 1 and 2.
[\[\]] will match either [ or
].
\.+ will match any run of
periods.
[0-9]+ will match any
integer number.
[0-9]+\.[0-9]* will match
any number with a decimal
point.
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 160 of 161
Appendix B: Predefined Manufacturer Table
SPECCHIO User Guide
SPECCHIO_UserGuide.pdf
Version 3.3.0.0 20/02/2018 Page 161 of 161
Appendix C: Predefined Sensor Table
