Scope A2 SDC Localization Guide
User Manual:
Open the PDF directly: View PDF 
.
Page Count: 99
| Download | |
| Open PDF In Browser | View PDF |
AWIPS II Site Data Configuration & Localization: Step-By-Step Guide Build 11.9 Prepared in Support of AWIPS Software Continuous Technology Refresh Re-Architecture Document No. AWP.DOC.TOSCAT.LOC-06.00 17 October 2011 Prepared Under Contract DG133W-05-CQ-1067 Advanced Weather Interactive Processing System (AWIPS) Operations and Maintenance T0013 AWIPS II SCAT Platform Task Order, Work Assignment 002 Submitted to: Mr. Walter Scott Contracting Officer’s Technical Representative U.S. Department of Commerce NOAA NWS Office of Science and Technology Programs and Plans Division, Program Management Branch SSMC2, OST11, Room 15130 1325 East-West Highway Silver Spring, MD 20910 By: Raytheon Technical Services Company LLC 8401 Colesville Road, Suite 800 Silver Spring, MD 20910 This document includes data that shall not be duplicated, used, or disclosed – in whole or in part – outside the Government for any purpose other than to the extent provided in contract DG133W-05-CQ-1067. However, the Government shall have the right to duplicate, use, or disclose the data to the extent provided in the contract. This restriction does not limit the Government’s right to use information contained in this data if it is obtained from another source without restriction. The data subject to this restriction are contained in all sheets. AWIPS II Site Data Configuration & Localization: Step-by-Step Guide (Build 11.9) Change History Document No. AWP.DOC.TOSCAT.LOC-01.00DFT AWP.DOC.TOSCAT.LOC-01.00 AWP.DOC.TOSCAT.LOC-02.00 Publication Date 1 December 2010 25 February 2011 25 March 2011 Section(s) Affected All All 1, 2, 3, 4, 11, App. A Description of Change(s) Original Draft Final document Modified Section 1.0 (para. 6, verification statement) Modified Section 1.2, Document Variable Conventions Added new Section 1.3, CAVEOnly Configuration Modified Table 2.1, AWIPS II Baseline Database Listing (title only) Modified Section 2.1, Configuring the PostgreSQL Database Engine Modified Section 2.2, Create and Import Database Files Modified Table 3.1-1, LDM Feedtypes Modified Table 3.1-4, Radar Patterns Modified Section 4.1, Create the Directory Structure Added new Section 4.2, Create FFMP Run Configuration File Modified Section 11.1, Configure AvnFPS GUI and Product Files Modified App. A, LDM Ingest Checklist (step 2 only) Modified App. C, Creating Custom Map Scales Contract DG133W-05-CQ-1067 / DCN AWP.DOC.TOSCAT.LOC-06.00 / 17October 2011 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this document. ii AWIPS II Site Data Configuration & Localization: Step-by-Step Guide (Build 11.9) Document No. AWP.DOC.TOSCAT.LOC-03.00 Publication Date 9 May 2011 Section(s) Affected 1, 2, 3, 4, 5, 9 Description of Change(s) Modified Section 1.0 (para. 6, verification statement) Modified Section 2.3, Updated FFMP ShapeFiles [WFO Only] Modified Section 2.4, Update afos2awiips With Local Changes Modified Table 3.1-3, Hydrology Patterns Modified Section 3.2, Plugin Ingest Filters Modified Section 4.3 Progressive Disclosure Files (including Table 4.3-1) Modified Section 4.4, Create / Copy Common Hydrology Files Into Place Modified Section 4.5.1, Create / Edit hydroSitesInUse.txt File Modified Example 5.2-1, Text of AWIPS II OAX siteConfig.py Modified Table 5.2.2-1, Necessary localConfig.py Changes From AWIPS I to AWIPS II Modified Example 5.2.2-1 Modified Section 5.3, Create / Edit Subgridding Files AWP.DOC.TOSCAT.LOC-04.00 9 June 2011 1 – 5, 9 – 11, App. A Modified Section 9.0, Create / Edit CAVE Map Displays Modified Section 1.0, Introduction Modified Section 2.3 and retitled as Update Shape Files Modified Section 2.4, Update afos2awips With Local Changes Modified Section 3.0, SBN Ingest Configuration (including changes to 3.1, 3.2, and 3.3) Modified Table 4.3-1, SPI File Inputs Modified Section 4.4, Contract DG133W-05-CQ-1067 / DCN AWP.DOC.TOSCAT.LOC-06.00 / 17October 2011 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this document. iii AWIPS II Site Data Configuration & Localization: Step-by-Step Guide (Build 11.9) Document No. Publication Date Section(s) Affected Description of Change(s) Create/Copy Common Hydrology Files Into Place Modified Section 5.2.2, Copy/Edit localConfig.py (including Table 5.2.2-1, Necessary localConfig.py Changes From AWIPS I to AWIPS II) Modified Table 5.2.3-1, Smartinit Changes Modified Section 5.4, Create MPE.xml Localized Grid Identification File Added new Section 5.4 Create MPE.xml Localized Grid Identification File Renumbered Section titled Fix Ownership and Permissions (from 5.4 to 5.5) Modified Section 9.0, Create/EDIT CAVE Map Displays (para. 3, bullet 4 only) Modified Section 10.1 Create/Edit Satellite Menu Modified Section 11.1, Configure AvnFPS GUI and Product Files (bullet 6 only) Modified Section 11.2, Copy AvnFPS Climate Files (bullet 5 only) Modified App. A, LDM Ingest Checklist (step 2 only) AWP.DOC.TOSCAT.LOC-05.00 25 July 2011 1 – 6, 12, App. B, App. C Modified Section 1.0, Introduction Modified Section 2.1, Configuring the PostgreSQL Database Engine Modified Section 2.3, Update Shape Files Modified 3.1, Ingest Filter Configuration Modified 3.2, Plugin Ingest Filters Modified 4.2, Create FFMP Run Contract DG133W-05-CQ-1067 / DCN AWP.DOC.TOSCAT.LOC-06.00 / 17October 2011 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this document. iv AWIPS II Site Data Configuration & Localization: Step-by-Step Guide (Build 11.9) Document No. Publication Date Section(s) Affected Description of Change(s) Configuration File Modified 4.3, Progressive Disclosure Files Modified 4.4, Create/Copy Common Hydrology Files Into Place Modified 5.3, Create/Edit Subgridding Files Modified 6.0, Correctset_hydro_env Modified 11.2, Copy AvnFPS Climate Files Modified 12.0, Restart EDEX and Activate MPE Scripts Modified App. B, WarnGen Templates AWP.DOC.TOSCAT.LOC-06.00 Xx October 2011 1, 5, 13, App. B, App. D Modified App. C, Creating Custom Map Scales Modified 1.0, Introduction (document number only) Modified 5.2.2, Copy/Edit localConfig.py (numbered examples) Modified 5.4, Create MPE.xml Localized Grid Identified File (to add references DHR-MOSAIC Added Section 13, “Add Text Triggers” Modified Appendix B, WarnGen Templates Added Appendix D, Adding Grid Information Contract DG133W-05-CQ-1067 / DCN AWP.DOC.TOSCAT.LOC-06.00 / 17October 2011 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this document. v AWIPS II Site Data Configuration & Localization: Step-by-Step Guide (Build 11.9) Table of Contents 1.0 2.0 3.0 4.0 5.0 Page Introduction ............................................................................................................................ 1 1.1 Document Conventions ................................................................................................. 2 1.2 Document Variable Conventions .................................................................................. 2 1.3 CAVE-Only Configuration ........................................................................................... 3 1.4 Notes About Use on ADAM Platform .......................................................................... 4 Database Configuration ......................................................................................................... 5 2.1 Configuring the PostgreSQL Database Engine............................................................. 5 2.2 Create and Import Database Files ................................................................................. 7 2.2.1 hmdb Database [ WFO Only ] ........................................................................... 7 2.2.2 IHFS Database .................................................................................................... 7 2.2.3 DamCrest Database............................................................................................. 8 2.2.4 lsrdata Database [ WFO Only ] ........................................................................... 8 2.2.5 Additional Databases [ RFC Only ] .................................................................... 9 2.3 Update Shape Files ....................................................................................................... 9 2.4 Update afos2awips With Local Changes .................................................................... 12 SBN Ingest Configuration.................................................................................................... 13 3.1 Ingest Filter Configuration.......................................................................................... 13 3.2 Plugin Ingest Filters .................................................................................................... 17 3.3 Distribution Filters ...................................................................................................... 21 Common Utility Configuration Files ................................................................................... 23 4.1 Create the Directory Structure .................................................................................... 23 4.2 Create FFMP Run Configuration File......................................................................... 23 4.3 Progressive Disclosure Files ....................................................................................... 25 4.4 Create / Copy Common Hydrology Files Into Place .................................................. 26 4.4.1 Create / Edit Site Apps_defaults File ................................................................ 27 4.5 Create Radar, NCEP/Hydro and UpperAir Menu Files .............................................. 29 4.5.1 Create/Edit radarsInUse.txt File........................................................................ 29 4.5.2 Create/Edit the eavConfigTable.txt .................................................................. 32 4.5.3 Create/Edit raobSitesInUse.txt File For Upper Air Menus ............................... 32 4.6 Copy / Edit WarnGen Templates [ WFO Only ] ........................................................ 34 4.7 Disable Fog Monitor [ RFC Only ] ............................................................................. 34 4.8 Correct Permissions and Ownership ........................................................................... 34 EDEX Utility Configuration Files ....................................................................................... 35 5.1 Create the Directory Structure .................................................................................... 35 5.2 Create/Edit GFE Configuration Files.......................................................................... 35 5.2.1 Copy / Edit siteConfig.py ................................................................................. 35 Contract DG133W-05-CQ-1067 / DCN AWP.DOC.TOSCAT.LOC-06.00 / 17October 2011 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this document. vi AWIPS II Site Data Configuration & Localization: Step-by-Step Guide (Build 11.9) 5.2.2 Copy/Edit localConfig.py ................................................................................. 36 5.2.3 Copy / Edit SmartInit Files ............................................................................... 38 5.3 Create / Edit Subgridding Files ................................................................................... 40 5.4 Create MPE.xml and DHR-MOSAIC.xml Localized Grid Identification File .......... 43 5.5 Fix Ownership and Permissions.................................................................................. 46 6.0 Correct set_hydro_env ......................................................................................................... 47 7.0 Edit setup.env....................................................................................................................... 48 8.0 Create / Edit WarnGen GUI Setup [ WFO Only ] ............................................................... 49 9.0 Create/Edit CAVE Map Displays ........................................................................................ 51 9.1 Create / Edit CAVE Station/Location Plot Information File(s) .................................. 51 10.0 Create / Edit Other CAVE Menus ....................................................................................... 53 10.1 Create / Edit Satellite Menu ........................................................................................ 53 10.2 Configure Local and Regional Warnings Display Menu ............................................ 54 11.0 Configure AvnFPS [ WFO Only ] ....................................................................................... 56 11.1 Configure AvnFPS GUI and Product Files ................................................................. 56 11.2 Copy AvnFPS Climate Files ....................................................................................... 57 12.0 Restart EDEX and Activate MPE Scripts ............................................................................ 58 13.0 Add Text Triggers ................................................................................................................ 59 Appendix A. LDM Ingest Checklist ........................................................................................... A-1 A.1 Deciphering a pqact.conf Entry ................................................................................ A-2 Appendix B. WarnGen Templates .............................................................................................. A-1 B.1 How to Access the Data Contained in a Value Inside a Template ......................... A-16 B.2 Modifying / Editing Templates ............................................................................... A-16 B.2.1 Time Ranges ................................................................................................. A-16 B.2.2 Locking Text ................................................................................................ A-16 B.2.3 Suppress Counties ........................................................................................ A-17 B.2.4 Wrapping Text.............................................................................................. A-17 B.2.5 Adding Bullets.............................................................................................. A-17 B.2.6 Adding “Call To Actions” ............................................................................ A-18 B.2.7 Examples ...................................................................................................... A-18 Appendix C. Creating Custom Map Scales ................................................................................ C-1 Appendix D. Adding Grid Information ..................................................................................... D-1 Part 1. Grid Parameter Definition ...................................................................................... D-1 Part 2. Geospatial Information ........................................................................................... D-2 Part 3. Grid Definition ....................................................................................................... D-3 Contract DG133W-05-CQ-1067 / DCN AWP.DOC.TOSCAT.LOC-06.00 / 17October 2011 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this document. vii AWIPS II Site Data Configuration & Localization: Step-by-Step Guide (Build 11.9) List of Tables Page Table 1.2-1. Document Variable Conventions ............................................................................... 3 Table 2-1. AWIPS II Baseline Database Listing ............................................................................ 5 Table 2.3-1. AWIPS II Shape File-To-Table Mapping ................................................................ 10 Table 3.1-1. LDM Feedtypes ........................................................................................................ 13 Table 3.1-2. LDM Action Options ................................................................................................ 15 Table 3.1-3. Hydrology Patterns ................................................................................................... 16 Table 3.2-1. Plugin Filter Filtering Elements ............................................................................... 19 Table 3.2-2. List of Base Plugin Filters ........................................................................................ 19 Table 3.2-3. Plugins That Support Filtering ................................................................................. 20 Table 3.2-4. Fields for Use Within a Plugin Filter XML File ...................................................... 20 Table 4.2-1. FFMP Run Configuration Fields .............................................................................. 23 Table 4.3-1. SPI File Inputs .......................................................................................................... 25 Table 4.5-1. radarsInUse.txt title Lines and Descriptions ............................................................ 30 Table 5.2.2-1. Necessary localConfig.py Changes From AWIPS I to AWIPS II ........................ 36 Table 5.2.3-1. Smartinit Changes ................................................................................................. 38 Table 5-3-1. Base Model Names and Descriptions ...................................................................... 40 Table 8-1. WarnGen UI config.xml XML Tag Descriptions........................................................ 49 Table 9-1. Delivered AWIPS-II Base Scale XML Files ............................................................... 51 Table 10.1-1. Satellite Sector Names ............................................................................................ 54 Table B-1. Baseline WarnGen Templates .................................................................................. B-1 Table B-2. Velocity .xml Template Configuration Values ......................................................... B-2 Table B-3. Variables Passed to WarnGen Templates ............................................................... B-10 Table B.2.1-1. Start and End Times for Actions....................................................................... B-16 Table D-1. Important XML Definitions in a grib model definition .xml ................................... D-3 List of Figures Page Figure 1-1. Localization in AWIPS II............................................................................................. 1 Figure 3.2-1. Plugin Filter, When Used to Filter Output from a Process ..................................... 18 Figure 3.2-2. Plugin Filter, When Used to Filter the Input of a Process ...................................... 18 Contract DG133W-05-CQ-1067 / DCN AWP.DOC.TOSCAT.LOC-06.00 / 17October 2011 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this document. viii AWIPS II Site Data Configuration & Localization: Step-by-Step Guide (Build 11.9) Acronyms and Abbreviations Used in This Document ACARS ADAM AFOS ASR AvnFPS AWIPS CAVE CONUS CPSBN CWA CWSU D2D DR DVB DX EAV EDEX FFMP FNMOC GFE GMS GOES GUI ICAO ID IHFS IP km LAN LDM LWX MPE NCDC NCEP NEXRAD NIDS NOAA NOGAPS NWS Aircraft Communications Addressing and Reporting System AWIPS Data and Application Migration Automation of Field Operations and Services Airport Surveillance Radar Aviation Forecast Preparation System Advanced Weather Interactive Processing System Common AWIPS Visualization Environment Continental United States; Contiguous United States Communications Processor Satellite Broadcast Network County Warning Area Center Weather Service Unit Display 2-Dimensional Discrepancy Report Digital Video Broadcaster Data Server Estimated Actual Velocity Enterprise Data EXchange Flash Flood Monitoring and Prediction Fleet Numerical Meteorology and Oceanography Center Graphical Forecast Editor Geostationary Meteorological Satellite Geostationary Operational Environmental Satellite Graphical User Interface International Civil Aviation Organization Identifier; Identification Integrated Hydrologic Forecast System Internet Protocol kilometer Local Area Network Local Data Manager WFO Sterling, Virginia Multi-sensor Precipitation Estimates National Climatic Data Center National Centers for Environmental Prediction Next Generation Weather Radar NEXRAD Information Dissemination Service National Oceanic and Atmospheric Administration Navy Operational Global Atmospheric Prediction System National Weather Service Contract DG133W-05-CQ-1067 / DCN AWP.DOC.TOSCAT.LOC-06.00 / 17October 2011 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this document. ix AWIPS II Site Data Configuration & Localization: Step-by-Step Guide (Build 11.9) OAX OCONUS POI PostgreSQL PX RAOB RFC SBN SCAT SQL TAF TCP UGC VTEC VTL WAN WFO WMO XML WFO Omaha, Nebraska Outside Continental United States; Outside Contiguous United States Point-of-Interest Postgres Structured Query Language Linux Preprocessor RAwinsonde OBservation River Forecast Center Satellite Broadcast Network Standard Configuration for AWIPS II Testing Structured Query Language Terminal Aerodrome Forecast Transmission Control Protocol Universal Geographic Code Valid Time Event Code Velocity Template Language Wide Area Network Weather Forecast Office World Meteorological Organization EXtensible Markup Language Contract DG133W-05-CQ-1067 / DCN AWP.DOC.TOSCAT.LOC-06.00 / 17October 2011 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this document. x AWIPS II Site Data Configuration & Localization: Step-by-Step Guide (Build 11.9) 1.0 Introduction Localization adapts (i.e., configures) the AWIPS national baseline software to the unique data and display requirements of the site. AWIPS II performs localization dynamically at system startup using data from the localization data environment. This document is designed as a guide to changing specific files in the localization environment to configure an EDEX installation to a specific site, and as a guide to adding localization files needed for launching and running client applications such as CAVE and AlertViz. It supersedes Raytheon document AWP.DOC.TOSCAT.LOC-05.00, dated 25 July 2011, and all previous versions of this document. This document focuses on a SERVER-CLIENT localization paradigm, much like that which exists on AWIPS I. The steps taken should be sufficient to guide a person familiar with AWIPS I systems through the creation of a usable AWIPS II localization for an EDEX and CAVE instance. Localization in AWIPS II is done by the EDEX process itself, as opposed to a script (mainScript.csh) from AWIPS I. Based on specific configuration files, the EDEX process produces output files, which are then transferred, or downloaded, to the CAVE client when CAVE starts and then whenever a change to a file has been detected. Figure 1-1 attempts to describe the flow of configuration on the left, the localization engine (EDEX) in yellow (center), the output of that localization in orange (top right), and the usage of those localization files via download to a user’s caveData directory on a workstation in red (bottom right). Figure 1-1. Localization in AWIPS II Contract DG133W-05-CQ-1067 / DCN AWP.DOC.TOSCAT.LOC-06.00 / 17 October 2011 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this document. 1 AWIPS II Site Data Configuration & Localization: Step-by-Step Guide (Build 11.9) This document focuses on creating the configuration files necessary for EDEX to correctly create the output files needed for proper baseline operations. General Note: You should never edit files that are in a base directory tree. Although there are a few exceptions as of the date of this document's creation, anything beneath a directory named base in the configuration tree should never be edited. This is analogous to the nationalData directory in AWIPS I. Assistance Note: A tool developed under Task Order 0013 is available for all sites that have installed an ADAM (AWIPS II Data and Application Migration) platform to help collect the files referenced in this document for moving to a platform which normally may not have access to those files. The script is collect_files.sh and would be located in the /data/fxa/INSTALL/ADAM directory. For more information on this script, please see the document titled “AWIPS II Site Data Configuration Automation Tool” (AWP.DOC.TOSCAT.AUTO-04.00). This document has been verified to work for AWIPS II Releases Build 11.5 - Build 11.7. 1.1 Document Conventions When specific commands to be typed are being displayed, the following will be used as the convention. The user should type on the screen everything following the # symbol. For example: # ssh dx1 This indicates that the user should secure shell into dx1 (ssh dx1) At points in the document, it is necessary to reference a command that could be used for multiple iterations to achieve the same effect. The double bracket will enclose a description of what the user is to input in place of the notated reference. For example: [[ database name ]] This indicates that the user should run the command for as many iterations of the database name as needed to complete the desired configuration. When a substitution needs to be made in a command that is to be interpreted by the user, the following syntax will be used: << Site ID >> This indicates that the user should interpret the correct site identifier to put into the command. When specific paths are shown, they will be given a background shade, as shown in the following example: /awips2/edex 1.2 Document Variable Conventions Table 1.2-1 defines a list of variables referenced in this document, provides a description of each variable, and identifies the current location of each variable as of the date of this document. Wherever a variable occurs in this document, the actual value should be substituted when typing the commands on an AWIPS system. These are not environment variables at this point in time, Contract DG133W-05-CQ-1067 / DCN AWP.DOC.TOSCAT.LOC-06.00 / 17 October 2011 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this document. 2 AWIPS II Site Data Configuration & Localization: Step-by-Step Guide (Build 11.9) so manual substitution should be made when typing commands that contain references to these variables. Note: As of the creation of this document there is no plan to have these be system environment variables on an AWIPS or ADAM system. The location should be known to the user following the steps in the document. Normally the value under the expected location is that which an AWIPS site or ADAM platform would adhere with the exception of the server and cave platforms. Those, for an ADAM platform only, would all be adam1. Table 1.2-1. Document Variable Conventions Environment Variable $CAVE $DB_SERVER $EDEX1 $EDEX_CONFIG $EDEX_DB_HOME $EDEX_HOME $LDM_DOWNSTREAM $LDM_HOST $RCM_INSTALL_DIR $RCM_SERVER 1.3 Description The hostname of a CAVE workstation The hostname of the server running the Postgres database engine The primary EDEX servers The location in which the configuration files on the EDEX servers reside The location in which the database files reside that control the functionality of the PostgreSQL engine, as well as the physical storage for the data held within the tables The home location for the EDEX JVM The hostname of the server running the downstream LDM client The hostname of the server running the upstream/feeder LDM host The location in which the AWIPS II RadarServer is installed The hostname of the server running the AWIPS II RadarServer Expected Location lx1, lx2, lx3, etc…. dx1f dx3 /awips2/edex/data/utility /awips2/data [DX1/DX2] /awips2/edex dx2f cpsbn1/cpsbn2 /awips2/rcm dx1f CAVE-Only Configuration The instructions in this document are designed to configure an AWIPS II installation to a site other than OAX. If only CAVE configuration is desired, follow the instructions in these sections: • • • • • • • Section 3 Section 4 [Do all except most of section 4.4. The steps in section 4.4.1 need to be followed in this section without exception.] Section 8 Section 9 Section 10 Section 11 Section 12. Contract DG133W-05-CQ-1067 / DCN AWP.DOC.TOSCAT.LOC-06.00 / 17 October 2011 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this document. 3 AWIPS II Site Data Configuration & Localization: Step-by-Step Guide (Build 11.9) 1.4 Notes About Use on ADAM Platform When using this document to localize an AWIPS II installation on the ADAM platform, it is important to note that all references to a server, workstation, or other host instances via the document conventions defined in this section will be adam1. This is due to the fact that not only do the AWIPS II server instances run on the ADAM platform, but also the visualization software. Contract DG133W-05-CQ-1067 / DCN AWP.DOC.TOSCAT.LOC-06.00 / 17 October 2011 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this document. 4 AWIPS II Site Data Configuration & Localization: Step-by-Step Guide (Build 11.9) 2.0 Database Configuration An integral part of each system setup is the database, which continues to be housed within a PostgreSQL Database Engine (www.postgresql.org). Table 2-1 lists the functional database table and indicates whether they are WFO centric, RFC centric, or both. It also identifies the method to be used to create each table. Table 2-1. AWIPS II Baseline Database Listing Database Name WFO / RFC / Both Creation Method dc_ob7xxx* Both Post installation from backup fxatext Both On first start of EDEX hd_ob92xxx* Both Post installation from backup hmdb WFO Post installation from backup lsrdata WFO Post installation from backup maps Both During installation metadata Both On first start of EDEX ob7_histdata RFC Post installation from backup postgres Both N/A (Systems Database) template0 Both N/A (Systems Database) template1 Both N/A (Systems Database) * Sites will not see xxx in their output; “xxx” shown in this table will be replaced by the 3-letter AWIPS Site ID. Those databases not created or populated during software installation or during software startup should be restored from the latest backup file. Note: The need to import databases exists if configuration is being performed in order for EDEX to run as a site other than OAX. If configuration is being done merely to run CAVE in order to back up an adjacent WFO, there is no need to acquire and import the databases for that site. In order to proceed, the EDEX server must not be running. • Stop the EDEX server, first by logging into all EDEX servers issuing the stop command. # ssh $EDEX1 # service edex_camel stop • Repeat the previous step for all EDEX client machines. 2.1 Configuring the PostgreSQL Database Engine Due to the dynamic names of the hydrology (hd_) and damcrest (dc_) databases, and due to the ability of RFCs to create local databases in the dblocalrfc tablespace, the PostgreSQL database engine needs to be configured to allow access to these tables. The file that controls access is $EDEX_DB_HOME/pg_hba.conf. For each database, two lines must be present in this file: one for local access and one for remote access through tcp protocol. The lines resemble the following example: Contract DG133W-05-CQ-1067 / DCN AWP.DOC.TOSCAT.LOC-06.00 / 17 October 2011 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this document. 5 AWIPS II Site Data Configuration & Localization: Step-by-Step Guide (Build 11.9) local host fxatext fxatext all all trust 165.92.XXX.0/24 trust In the preceding example, XXX would be substituted for the site’s AWIPS LAN third octet. To configure the database to allow access from all of a site’s CAVE clients, a line for each database matching the above should be entered. Note: If a CWSU is hooked to your system, and has an IP outside the local site’s LAN, additional entries will be necessary to allow connections from these workstations. Follow these steps to configure the PostgreSQL access for an AWIPS site: • Log into the database server as user root and change to the $EDEX_DB_HOME directory: # cd $EDEX_DB_HOME • Use a text editor (suggested: vi) to edit the pg_hba.conf file and input the lines that will allow clients on the AWIPS site to access the databases: # vi pg_hba.conf • Using table 2.1 to determine which databases you will have installed, insert the lines, or modify the existing lines to reflect your site’s subnet of the AWIPS WAN, in the pg_hba.conf file. See the following example, which includes all necessary edits for configuring site LWX: local local local host host host hd_ob92lwx dc_ob7lwx lsrdata hd_ob92lwx dc_ob7lwx lsrdata all all all all all all 165.92.52.0/24 165.92.52.0/24 165.92.52.0/24 trust trust trust trust trust trust RFC Note: Because a River Forecast Center has the capability now to create local databases, you will need to reflect each of these local databases in the AWIPS II pg_hba.conf file as well, following the example above. • Once you are finished, save and quit: # :wq! • If the database was not currently running while these changes were being made, it should be restarted at this point. If the engine was running, then a control command can be sent to the postmaster process to force it to reload its configuration files. # ps aux | grep postgres If a postmaster process returns, then reload the configuration data: # su awips –lc “pg_ctl reload –D $EDEX_DB_HOME” Otherwise, start the database server: # service edex_postgres start Contract DG133W-05-CQ-1067 / DCN AWP.DOC.TOSCAT.LOC-06.00 / 17 October 2011 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this document. 6 AWIPS II Site Data Configuration & Localization: Step-by-Step Guide (Build 11.9) 2.2 Create and Import Database Files Once the database has been configured to allow connections from your site’s AWIPS LAN, any databases that are not delivered should be created and populated (if they have not been already). Find a list of databases currently loaded on the AWIPS system by logging into an AWIPS machine with the AWIPS II software and environment successfully installed, and following these steps: • • Open a terminal window, if one is not already open. Issue the following command to get a list of currently loaded databases: # psql –U awips –h $DB_SERVER –l Note: If prompted for a password, the database user password is ‘awips’ • • Compare this list to Table 2-1. AWIPS II Baseline Database Listing. Log into the AWIPS II Database Server: # ssh –l root $DB_SERVER Logs into the database server as user root 2.2.1 hmdb Database [ WFO Only ] • Drop and then re-create the hmdb database by issuing the following commands: Note: Because the database already exists, it must first be dropped before it can be imported from a backup with the following command: # dropdb –h $DB_SERVER –U awips hmdb ** Only run this if the database already exists ** Note: If prompted for a password, the database user password is ‘awips’ # createdb –h $DB_SERVER –U awips hmdb Runs the createdb command giving ownership to database user awips for each database needed. Note: If prompted for a password, the database user password is ‘awips’ • Import the most current backups for the databases. AWIPS backup_pgdb cron runs daily at 0522 GMT and puts a backup for each database into /data/fxa/DAILY_BACKUP/postgres/. These files can be used to import into the newly created databases: # pg_restore –h $DB_SERVER –U awips –d hmdb –Fc /data/fxa/DAILY_BACKUP/postgres/<< Day Of Week>>/hmdb Note: The previous command is all on one line, and not two lines. 2.2.2 IHFS Database • Create the hd_ob92xxx database by issuing the following commands: Note: Substitute the AWIPS Site ID of the desired site, in lowercase letters, for xxx in the database name: Contract DG133W-05-CQ-1067 / DCN AWP.DOC.TOSCAT.LOC-06.00 / 17 October 2011 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this document. 7 AWIPS II Site Data Configuration & Localization: Step-by-Step Guide (Build 11.9) # createdb –h $DB_SERVER –E SQL_ASCII –U awips hd_ob92xxx Runs the createdb command giving ownership to database user awips for each database needed. Note: If prompted for a password, the database user password is ‘awips’ • Import the most current backups for the databases. AWIPS backup_pgdb cron runs daily at 0522 GMT and puts a backup for each database into /data/fxa/DAILY_BACKUP/postgres/ . These files can be used to import into the newly created databases: # pg_restore –h $DB_SERVER –U awips –d hd_ob92xxx –Fc /data/fxa/DAILY_BACKUP/postgres/<< Day Of Week>>/hd_ob92xxx Note: The previous command is all on one line, and not two lines. 2.2.3 DamCrest Database • Create the dc_ob7xxx database by issuing the following commands: Note: Substitute the AWIPS Site ID of the desired site, in lowercase letters, for xxx in the database name: # createdb –h $DB_SERVER –U awips dc_ob7xxx Runs the createdb command giving ownership to database user awips for each database needed. Note: If prompted for a password, the database user password is ‘awips’ • Import the most current backups for the databases. AWIPS backup_pgdb cron runs daily at 0522 GMT and puts a backup for each database into /data/fxa/DAILY_BACKUP/postgres/ . These files can be used to import into the newly created databases: # pg_restore –h $DB_SERVER –U awips –d dc_ob7xxx –Fc /data/fxa/DAILY_BACKUP/postgres/<< Day Of Week>>/dc_ob7xxx Note: The previous command is all on one line, and not two lines. 2.2.4 lsrdata Database [ WFO Only ] • Create the lsrdata database by issuing the following commands: # createdb –h $DB_SERVER –U awips lsrdata Runs the createdb command giving ownership to database user awips for each database needed. Note: If prompted for a password, the database user password is ‘awips’ • Import the most current backups for the databases. AWIPS backup_pgdb cron runs daily at 0522 GMT and puts a backup for each database into /data/fxa/DAILY_BACKUP/postgres/ . These files can be used to import into the newly created databases: Contract DG133W-05-CQ-1067 / DCN AWP.DOC.TOSCAT.LOC-06.00 / 17 October 2011 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this document. 8 AWIPS II Site Data Configuration & Localization: Step-by-Step Guide (Build 11.9) # pg_restore –h $DB_SERVER –U awips –d lsrdata –Fc /data/fxa/DAILY_BACKUP/postgres/<< Day Of Week>>/lsrdata Note: The previous command is all on one line, and not two lines. 2.2.5 Additional Databases [ RFC Only ] River Forecast Centers have the ability to create local databases for use with RFC applications. These databases will need to be imported into the AWIPS II database engine. To retrieve the names of these databases, on the AWIPS I database server (dx1f) issue the following command: # psql –l | grep –vE “(fxatext|dc_ob7|hd_ob|hmdb|ob7_his|postgres|template)” If the AWIPS I database engine has already been shut down, issue this command on dx1 in order to find those files that were backed up by the nightly cron (all on one line): # find /data/fxa/DAILY_BACKUP/postgres/ -regextype posix-egrep mtime -1 -type f ! -regex '/.*(fxatext|dc_ob7...|hd_ob.....|hmdb|ob7_histdata|postgres|temp late[01])' The following example shows one possible output: [root@dx1-krf ~]# find /data/fxa/DAILY_BACKUP/postgres/ -regextype posix-egrep -mtime -1 -type f ! -regex '/.*(fxatext|dc_ob7...|hd_ob.....|hmdb|ob7_histdata|postgres|globals |template[01])' /data/fxa/DAILY_BACKUP/postgres/Thursday/fastetc /data/fxa/DAILY_BACKUP/postgres/Thursday/gisrs /data/fxa/DAILY_BACKUP/postgres/Thursday/hdkrf_testdb Note the databases below: ________________________________________________________________________ For each database returned, run the createdb and pg_restore commands when the AWIPS II database engine is installed and running: # createdb –h $DB_SERVER –U awips [[ dbname ]] # pg_restore –h $DB_SERVER –U awips –d [[ dbname ]] –Fc /data/fxa/DAILY_BACKUP/postgres/<< Day Of Week>>/[[ dbname ]] 2.3 Update Shape Files In AWIPS II, a number of shape files reside within the PostgreSQL database named maps. A utility named importShapeFiles.sh is provided to help import the files into the database. Table 2.3-1 (taken from mapdata.map_version) lists those shape files imported into the database on installation and their AWIPS I equivalent file names. Use this table when updating shape files. Contract DG133W-05-CQ-1067 / DCN AWP.DOC.TOSCAT.LOC-06.00 / 17 October 2011 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this document. 9 AWIPS II Site Data Configuration & Localization: Step-by-Step Guide (Build 11.9) Table 2.3-1. AWIPS II Shape File-To-Table Mapping Table Name basins cwa offshore zone timezones firewxaor latlon10 majorrivers allrivers lake world states customlocations highsea interstate railroad county hsa File Name ba23de09.shp w_15jl09.shp oz02jn09.shp z_11my10.shp tz04nv07.shp cf17jl07.shp LatLon10_Ocean.shp rs14fe02.shp rv14fe02.shp lk24ap09.shp world_adm0.shp s_01au07.shp customlocations.shp hz30mr09.shp in30ap08.shp railrdl020.shp c_15jl09.shp hs07my09.shp Table Name highway canada specialuse lowaltitude navaid artcc airport highaltitude fix rfc isc marinezones mexico firewxzones city marinesites ffmp_basins ffmp_streams File Name rd30ap08.shp province.shp sua.shp vAirways.shp navaid.shp artcc.shp airport.shp jAirways.shp fix.shp rf12ja05.shp cm28fe06.shp mz01jn10.shp st_mx.shp fz01my10.shp ci16ap10.shp MarineInfo_OB9.2.txt FFMP_aggr_basins.shp FFMP_ref_sl.shp The FFMP streams and basins shape files specifically must be localized at a WFO for each site using the created shapefiles in /data/fxa/nationalData. WFOs only update these files as follows: • Log into the server hosting the AWIPS II database (DX1 in normal operations) as user root # ssh –q $DB_SERVER –l root • Unzip the gzipped versions of the FFMP shape files # cd /data/fxa/nationalData # gunzip FFMP_*.shp.gz • Switch system users to user awips # su - awips • Change directories to the directory containing the importShapeFile.sh script # cd /awips2/static/database.maps (For OB11.4 & OB11.5) # cd /awips2/database/sqlScripts/share/sql/maps/ OB11.7) • (For OB11.6 & Update the ffmp_basins shape files using the site-specific version # ./importShapeFile.sh /data/fxa/nationalData/FFMP_aggr_basins.shp mapdata ffmp_basins 0.064,0.016,0.004,0.001 awips Note: If prompted for a password, the database user password is ‘awips’ Contract DG133W-05-CQ-1067 / DCN AWP.DOC.TOSCAT.LOC-06.00 / 17 October 2011 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this document. 10 AWIPS II Site Data Configuration & Localization: Step-by-Step Guide (Build 11.9) Example 2.3-1 shows typical output from this script: Example 2.3-1 Importing FFMP_aggr_basins.shp into mapdata.ffmp_basins ... Shapefile type: Polygon Postgis type: MULTIPOLYGON[2] psql: :29: NOTICE: CREATE TABLE will create implicit sequence "ffmp_basins_gid_seq" for serial column "ffmp_basins.gid" psql: :29: NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "ffmp_basins_pkey" for table "ffmp_basins" addgeometrycolumn ----------------------------------------------------------------mapdata.ffmp_basins.the_geom SRID:4326 TYPE:MULTIPOLYGON DIMS:2 (1 row) Creating Creating Creating Creating Creating • simplification levels 0.064,0.016,0.004,0.001... simplified geometry level 0.064 ... simplified geometry level 0.016 ... simplified geometry level 0.004 ... simplified geometry level 0.001 ... Update the ffmp_streams shape files using the site-specific version # ./importShapeFile.sh /data/fxa/nationalData/FFMP_ref_sl.shp mapdata ffmp_streams 0.064,0.016,0.004,0.001 awips Note: If prompted for a password, the database user password is ‘awips’ Example 2.3-2 shows typical output from this script: Example 2.3-2 Importing FFMP_ref_sl.shp into mapdata.ffmp_streams ... Shapefile type: Arc Postgis type: MULTILINESTRING[2] psql: :27: NOTICE: CREATE TABLE will create implicit sequence "ffmp_streams_gid_seq" for serial column "ffmp_streams.gid" psql: :27: NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "ffmp_streams_pkey" for table "ffmp_streams" addgeometrycolumn --------------------------------------------------------------------mapdata.ffmp_streams.the_geom SRID:4326 TYPE:MULTILINESTRING DIMS:2 (1 row) Creating Creating Creating Creating Creating • simplification levels 0.064,0.016,0.004,0.001... simplified geometry level 0.064 ... simplified geometry level 0.016 ... simplified geometry level 0.004 ... simplified geometry level 0.001 ... Exit the user awips shell back to root. # exit Contract DG133W-05-CQ-1067 / DCN AWP.DOC.TOSCAT.LOC-06.00 / 17 October 2011 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this document. 11 AWIPS II Site Data Configuration & Localization: Step-by-Step Guide (Build 11.9) • GZIP the FFMP Shape files that were unzipped before # cd /data/fxa/nationalData # gzip FFMP_*.shp 2.4 Update afos2awips With Local Changes If local changes have been made to the site’s afos2awips.txt file, they should be imported into the AWIPS II system. The functionality of this file has now been moved into the fxatext database (into a table named afos_to_awips). To make it easy to import a site’s localized version of this file easily, the site should make edits to the same file it maintained in AWIPS I . When completed, the file is placed in a national data management directory within the EDEX installation tree. From there EDEX uses the file to update its database automatically when there are changes. These steps should be completed for all sites performing an initial site data configuration. • Log into the server hosting an AWIPS II EDEX (DX3 in normal operations) as user root: # ssh –q $EDEX1 –l root • Switch users to system user awips: # su – awips • Copy the file into the ndm directory of the EDEX installation tree: # cp /awips/fxa/data/afos2awips.txt $EDEX_HOME/data/ndm/ • Ensure the proper ownership and permission: # chown awips:fxalpha $EDEX_HOME/data/ndm/afos2awips.txt • Exit from the system user awips shell, and from the ssh connection: # exit # exit Contract DG133W-05-CQ-1067 / DCN AWP.DOC.TOSCAT.LOC-06.00 / 17 October 2011 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this document. 12 AWIPS II Site Data Configuration & Localization: Step-by-Step Guide (Build 11.9) 3.0 SBN Ingest Configuration AWIPS II introduces the Unisys Local Data Manager (LDM) as the ingest tool for SBN data. It replaces the legacy acq_send / acqServer processes from AWIPS I. The upstream LDM host runs on the CPSBN machines, and listens directly to the SBN stream from the DVB receivers. The downstream LDM host runs in conjunction with the DX2F package, and requests products from the upstream host. Based on a filtering file, only certain products will be sent for ingest into the system. This is where the ingest filtering takes place now. 3.1 Ingest Filter Configuration Ingest filtering is controlled through the pqact.conf file, which resides in the /usr/local/ldm/etc directory. The general syntax of an entry in the file is as follows: feedtype TAB Regex Pattern TAB action TAB [arg] Where TAB is an actual TAB. Lines can, and are, split between new lines at a TAB. Table 3.1-1 is a list of feedtypes that can be used (reference http://www.unidata.ucar.edu/software/ldm/ldm-6.7.1/basics/feedtypes/index.html ). Table 3.1-1. LDM Feedtypes Primary Name Alternate Names PPS DDS HDS IDS SPARE UNIWISC FT0 FT1, DOMESTIC FT2, HRS FT3, INTNL FT4 FT5, MCIDAS PCWS FSL2 FSL3 FSL4 FSL5 GPSSRC CONDUIT FNEXRAD LIGHTNING WSI FT6, ACARS FT7, PROFILER FT8 FT9 FT10 FT11, NMC1, AFOS FT12, NMC2, NCEPH FT13, NMC3 FT14,NLDN FT15 DIFAX FT16 Description Public Products Service Domestic Data Service High resolution Data Service International Data Service Reserved for IDD use Satellite imagery and derived products from the Unidata/Wisconsin Broadcast ACARS data from commercial aircraft Wind profiler data Reserved for NOAA/GSD use Reserved for NOAA/GSD use Reserved for NOAA/GSD use SuomiNet GPS data gathering NCEP high-resolution model output NEXRAD Level-III composites Lightning data NEXRAD Level-III (NIDS) radar products and composites from WSI Corporation Unidata community-generated replacement for defunct DIFAX feed Contract DG133W-05-CQ-1067 / DCN AWP.DOC.TOSCAT.LOC-06.00 / 17 October 2011 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this document. 13 AWIPS II Site Data Configuration & Localization: Step-by-Step Guide (Build 11.9) Primary Name Alternate Names FAA604 FT17, FAA, 604 GPS FNMOC NONE FT18 FT19, SEISMIC, NOGAPS FT20, CMC FT21, IMAGE FT22, TEXT FT23, GRID FT24, POINT, NBUFR, BUFR FT25, GRAPH FT26, OTHER FT27, NNEXRAD, NEXRAD FT28, CRAFT, NEXRD2 FT29 FT30 FT0 | FT1 | FT2 | ... | FT31 -- DDPLUS WMO FT0 | FT1 FT0 | FT1 | FT2 | FT3 UNIDATA FT0 | FT1 | FT2 | FT3 | FT5 FT6 | FT7 | FT8 | FT9 | FT10 FT11 | FT12 | FT13 GEM NIMAGE NTEXT NGRID NPOINT NGRAPH NOTHER NEXRAD3 NEXRAD2 NXRDSRC EXP ANY FSL NMC NPORT FT22 | FT23 | FT24 | FT25 | FT26 Description FAA604 products for NWS use (private network), but available for IDD use SuomiNet GPS data NOGAPS and COAMP model output from FNMOC Canadian Meteorological Center GEM model output NOAAport satellite imagery NOAAport textual products (for future use) NOAAport high-resolution model output NOAAport point products (for future use) NOAAport Redbook Graphics (for future use) NOAAport miscellaneous products (for future use) NEXRAD Level-III products NEXRAD Level-II radar data NCDC NEXRAD Level-II data archiving For experiments, testing, etc. Predefined feed set name for any feed type Predefined feed set name for no feed types (will not match anything) Predefined feed set name for PPS or DDS Predefined feed set name for PPS, DDS, HDS, or IDS Predefined feed set name for PPS, DDS, HDS, IDS, or UNIWISC Predefined feed set name for PCWS, FSL2, FSL3, FSL4, or FSL5 Predefined feed set name for AFOS, NMC2, or NMC3 Predefined feed set name for NTEXT, NGRID, NPOINT, NGRAPH, or NOTHER Regex Pattern is a regular expression used in pattern matching the LDM stream for ingest. Action is the action to take on the product once it arrives. Table 3.1-2 describes the possibilities for action and can be found on LDM’s Web page here: http://www.unidata.ucar.edu/software/ldm/ldm-current/basics/pqact.conf.html Contract DG133W-05-CQ-1067 / DCN AWP.DOC.TOSCAT.LOC-06.00 / 17 October 2011 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this document. 14 AWIPS II Site Data Configuration & Localization: Step-by-Step Guide (Build 11.9) Table 3.1-2. LDM Action Options Action NOOP FILE STDIOFILE DBFILE EXEC PIPE Description Don’t do anything with the product. Write the data product to a file using the write() function. Write the data product to a file using the fwrite() function. Write the data product to a database. Execute a program. Write the product to a program’s standard input. Arg is any optional arguments for the Action and is generally one of the following: In general, the syntax for the ARG portion of the line in the pqact.conf file is as seen below: –overwrite –close –log –edex /store/path Where: indicates hitting the tab key on the keyboard /store/path is the path the file should be written to disk. -overwrite indicates overwrite the file (if exists) -close indicates to close the file after fwrite() function -log indicates log the product to the ldmd.log -edex indicates the sending of data over edexBridge for processing In order to determine the correct FEEDTYPE above, the following command can be run from a downstream LDM host. # ./notifyme –h $LDM_HOST –v –l- -p ‘$REGEX’ Where $REGEX is a regular expression that will match against the type of data which is of interest. Once a match is found, it is displayed on the screen as seen in Example 3.1-1. Example 3.1-1 Dec 21 15:52:21 notifyme[32196] INFO: KWBC 211549 Dec 21 15:52:21 notifyme[32196] INFO: 211545 1522 20101221155220.618 IDS|DDPLUS 11247393 408 20101221155220.618 HDS 11247394 SOAK45 SFUS41 KWBC The example shows the FEEDTYPE in bold type. AWIPS I customizes only specific patterns based on the localized site. Tables 3.1-3 and 3.1-4 depict information on each, and where to find the corresponding information. Contract DG133W-05-CQ-1067 / DCN AWP.DOC.TOSCAT.LOC-06.00 / 17 October 2011 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this document. 15 AWIPS II Site Data Configuration & Localization: Step-by-Step Guide (Build 11.9) Table 3.1-3. Hydrology Patterns AWIPS I Pattern Information Location AWIPS II LDM Pattern Wxxx from XXX-hydroSiteConfig.txt file in /awips/fxa/data/localization/XXX RegCode from XXX-hydroSiteConfig.txt file SRUSRegCode.KWBC in /awips/fxa/data/localization/XXX Rxxx from XXX-hydroSiteConfig.txt file in FOUS...(Rxxx) /awips/fxa/data/localization/XXX Rxxx from XXX-hydroSiteConfig.txt file in FGUS[57]..(Rxxx) /awips/fxa/data/localization/XXX Wxxx from XXX-hydroSiteConfig.txt file in [AF][BS]US...(Wxxx) /awips/fxa/data/localization/XXX Rxxx from XXX-hydroSiteConfig.txt file in AGUS5..(Rxxx) /awips/fxa/data/localization/XXX Wxxx from XXX-hydroSiteConfig.txt file in AGUS4..(Wxxx) /awips/fxa/data/localization/XXX Rxxx from XXX-hydroSiteConfig.txt file in [YZ]EI.98.(Rxxx) /awips/fxa/data/localization/XXX Note: An example of the first row in the pqact.conf would look like this: ^SRU[EMSW][1-9]..(Wxxx) IDS|DDPLUS FILE IDS|DDPLUS (Wxxx) IDS|DDPLUS (KWBC) IDS|DDPLUS ^(SRU[EMSW][1-9].) IDS|DDPLUS (Rxxx) IDS|DDPLUS (Wxxx) IDS|DDPLUS ^(FGUS[567].) IDS|DDPLUS ^(AGUS4.) (Wxxx) ANY ^(SRUSRegCode) ^(FOUS..) (Rxxx) ^([AF][BS]US..) ^(AGUS5.) (Rxxx) ^([YZ]EI.98) (Rxxx) ^(SRU[EMSW][1-9].) (Wxxx) (..)(..)(..) -overwrite -log -close -edex /data_store/shef/\4/\1_\2_\3\4\5_(seq).txt Remember that there are TABs between the different sections of the entry and that Wxxx would be substituted based on the rules above. T able 3.1-4. R adar Patter ns AWIPS I Pattern Information Reference CODE 17 INCLUDE SDUS2??KZZZ Substitute KZZZ for the reporting site for each radar listed in: /awips/fxa/data/localizationDataSets/XXX/dialRadars.txt CODE 17 INCLUDE SDUS3??KZZZ Reporting sites can be found by issuing the following command: CODE 17 INCLUDE SDUS4??KZZZ grep –i xxxy /awips/fxa/data/wmoSiteInfo.txt CODE 17 INCLUDE SDUS5??KZZZ Where xxx is the radar ID without the preceding K, P or T. CODE 17 INCLUDE SDUS7??KZZZ The following file on DX1 also contains the information needed: CODE 17 INCLUDE SDUS8??KZZZ /awips/fxa/data/localizationDataSets/XXX/acq_wmo_parms.sbn.radar CODE 17 INCLUDE NXUS6??KZZZ AWIPS II has only one pattern in the pqact.conf file, which lists all KXXX: NNEXRAD ^(SDUS[234578].|NXUS6.) (K|P|T)(XXX|XXX|XXX|XXX|XXX) Change the XXX to match each of the reporting sites for the radars listed in dialRadars.txt file. An example of a full NNEXRAD localized line for site LWX follows. Please note there are only two lines, so word wrap applies (see Appendix A for more detailed information on LDM acquisition patterns): NNEXRAD ^(SDUS[234578].|NXUS6.) (K|P|T)(LWX|BGM|CHS|RLX|ILN|CLE|AKQ|JKL|CTP|MHX|MRX|OKX|PHI) (..)(..)(..) /p(...)(...) FILE -overwrite –close –log –edex /data_store/radar/\2\8/\7/\5\6_\2\8_\7_(seq).rad Contract DG133W-05-CQ-1067 / DCN AWP.DOC.TOSCAT.LOC-06.00 / 17 October 2011 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this document. 16 AWIPS II Site Data Configuration & Localization: Step-by-Step Guide (Build 11.9) Additional patterns can also be found in the acqPatternsAddOn.txt file, which can reside in /awips/fxa/data/localization/XXX or /data/fxa/customFiles. These patterns should also be added to the pqact.conf.xxx file in the proper syntax (where xxx is your site ID). Delivered with the AWIPS II LDM rpm is a baselined pqact.conf.template file which should be used as a basis for the active pqact.conf file. It should not be edited; rather, it should be copied into the active pqact.conf file. From there, custom patterns can be concatenated onto the active file. Note: ADAM platform’s active file is adam-pqact.conf To manually edit or add a new filter rule, follow these steps. Utilize Appendix A, LDM Ingest Checklist, as a tool to help ensure all the correct information is known before proceeding: • Log into the downstream LDM client host (normally PX2) as user root: # ssh root@$LDM_DOWNSTREAM • Change directories to the pqact.conf location on the server: # cd /usr/local/ldm/etc • Edit the pqact.conf using a text editor (shown here is vi): # vi pqact.conf.xxx • Once you are finished making changes, save the file and exit the editor: # :wq! • Check to ensure that the edited file still has the proper syntax using the ldmadmin command: # su ldm –lc “ldmadmin pqactcheck –f /usr/local/ldm/etc/pqact.conf.xxx” Expect the phrase “syntactically correct” for each file you have configured in ldmadmin-pl.conf • Concatenate this file with the pqact.conf.template file to create the active pqact.conf file: # cat pqact.conf.template pqact.conf.xxx > pqact.conf • Ensure the proper ownership and permissions: # chown ldm:fxalpha /usr/local/ldm/etc/pqact.conf • Signal the LDM server to re-read the configuration files: # su ldm –lc “ldmadmin pqactHUP” 3.2 Plugin Ingest Filters The purpose of plugin filters is to provide a data filtering capability that is base or site configurable without recompiling baseline code. This allows a subset of a national data set to be filtered based upon an area of interest, for example, filtering out the products that are put through shef decoding to only those in the local CWA of interest. Plugin filters may be used to filter output from a process, as shown in Figure 3.2-1. They may also be used to filter the input of a process, as shown in Figure 3.2-2. Contract DG133W-05-CQ-1067 / DCN AWP.DOC.TOSCAT.LOC-06.00 / 17 October 2011 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this document. 17 AWIPS II Site Data Configuration & Localization: Step-by-Step Guide (Build 11.9) BufrAScatPlugin BufrAscatDecoder Downstream Processing PointAscat xxxx Exclude SouthernHemisphere Figure 3.2-1. Plugin Filter, When Used to Filter Output from a Process ObsPlugin METARDecoder ShefPlugin MetarToShefFilter ShefDecoder Filter Element Figure 3.2-2. Plugin Filter, When Used to Filter the Input of a Process Filtering can be done through EXCLUSION of data, INCLUSION of data, or a combination of both. When using INCLUSIVE filtering, the output of the filtering operation returns those records that match the filtering criteria. The following shows an INCLUSIVE filter that will pass only those observations within a geographical rectangle which encompasses the OAX CWA. When using EXCLUSIVE filtering, the output of the filtering operation returns those records that are not matched by the filtering criteria. The following example shows an EXCLUSIVE filter that will stop all Southern Hemisphere data from being processed by the QuikScat plugin: OAX.WFO INCLUDE 44 Contract DG133W-05-CQ-1067 / DCN AWP.DOC.TOSCAT.LOC-06.00 / 17 October 2011 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this document. 18 AWIPS II Site Data Configuration & Localization: Step-by-Step Guide (Build 11.9)-100 40.0 -94.90 OAX.MetarToShef Table 3.2-1 describes filtering elements that can be used as of the date on this document (the xsi: type=). Table 3.2-1. Plugin Filter Filtering Elements Filter Element RectFilterElement RadiusFilterElement StationIdFilterElement Description Select based upon reports contained within a rectangular bounding. Select based upon reports within a specified distance from a given latitude/longitude. Select based upon reports matching a specified station identifier. Regular expressions may be used. Table 3.2-2 describes a list of present files in the base plugin-filters directory that filter out all Southern Hemisphere data. Finer-grain filtering may be done within a site-specific plugin-filters directory. Table 3.2-2. List of Base Plugin Filters File Name ascat_filters.xml modelsounding_filters.xml pointssmi_filters.xml quikscat_filters.xml Description Allows only northern hemisphere ASCAT data to be stored. Allows only northern hemisphere model sounding data to be stored. Allows only northern hemisphere SSMI data to be stored. Allows only northern hemisphere QUIKSCAT data to be stored. Contract DG133W-05-CQ-1067 / DCN AWP.DOC.TOSCAT.LOC-06.00 / 17 October 2011 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this document. 19 AWIPS II Site Data Configuration & Localization: Step-by-Step Guide (Build 11.9) Table 3.2-3 describes a list of plugins which, as of the date on this document, support plugin filtering. Table 3.2-3. Plugins That Support Filtering Plugin Filter File Name mesowest_filters.xml metarToShefFilter.xml modelSounding_filters.xml synopticMetarToShef.xml Description As delivered in the OAX localization, this filters out all data except that within the site's CWA for the mesowest plugin by latitude / longitude. As delivered in the OAX localization, filters out all data except that within the site's CWA by latitude/longitude. -As delivered in the OAX localization, filters out all data except that within the site's CWA by latitude/longitude. Table 3.2-4 describes a list of fields that can be used within a plugin filter xml file. Table 3.2-4. Fields for Use Within a Plugin Filter XML File Filter Tag Tag Effect filterElementName Global filterName Global filterType upperLeftLat upperLeftLon lowerRightLat lowerRightLon pointLat pointLon radius patterns Global RectFilterElement RectFilterElement RectFilterElement RectFilterElement RadiusFilterElement RadiusFilterElement RadiusFilterElement StationIdFilterElement Description For information purposes only. Appears in EDEX log for tracing loading and functionality of filters. For information purposes only. Appears in EDEX log for tracing loading and functionality of filters. INCLUDE or EXCLUDE Latitude measure for upper left corner point for filter area. Longitude measure for upper left corner point for filter area. Latitude measure for lower right corner point for filter area. Longitude measure for lower right corner point for filter area. Latitude of center point for radius filtering Longitude of center point for radius filtering Radius around center point to either include or exclude data. Station ID or regular expression to glob multiple station IDs into station ID filtering element. Note: Latitude values may range from -90 degrees (south) to 90 degrees (north); longitude values may range from -180 degrees (west) to 180 degrees (east). If a box must cross the date line (180), two filter boxes must be created to handle the filtering. To create, or edit, a plugin filter xml, follow these steps utilizing the delivered filters in the OAX subdirectory as a guide: • Log into an EDEX server, if not already logged in: # ssh root@${EDEX1} • If the directory does not exist, create the site level plugin-filters edex static configuration directory and change into the directory: # mkdir –p ${EDEX_CONFIG}/edex_static/site/ Exclude SouthernHemisphere EXCLUDE -0.1 -180.0 -90.0 180.0 Point.QuikScat /plugin-filters Contract DG133W-05-CQ-1067 / DCN AWP.DOC.TOSCAT.LOC-06.00 / 17 October 2011 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this document. 20 AWIPS II Site Data Configuration & Localization: Step-by-Step Guide (Build 11.9) # cd ${EDEX_CONFIG}/edex_static/site/ /plugin-filters • Copy the file from the OAX directory and edit using the vi editor: # cp –a ../../OAX/plugin-filters/[[ filter file ]].xml . # vi [[ filter file ]].xml Note: The name of the file should be one listed in Table 3-2.3. • Enter / edit the fields, ensuring the file looks like the example shown earlier in this section. A general rule of thumb is to define the area of responsibility for the desired site starting with its center point. To retrieve a site’s center point, issue the following command: For WFO sites: # psql –h $DB_SERVER –U awips –d maps –c “select lat,lon from mapdata.cwa where wfo=’$SITEID’;” RFC sites should issue the same command, using their co-located WFO to substitute for $SITEID Use the center point, and add and subtract 3 degrees from the latitude to get the upper and lower latitude. Add and subtract 4 degrees to get the west and east longitude. Adjust as necessary. • When finished editing, save and quit: # :wq! • Repeat the previous steps for all plugin filter files. • Ensure proper ownership and permissions: # chown –R awips:awips ${EDEX_CONFIG}/edex_static/site/ /plugin-filters # chmod –R 755 ${EDEX_CONFIG}/edex_static/site/ /pluginfilters 3.3 Distribution Filters AWIPS II introduces another layer of ingest filtering. Before processing a piece of data, EDEX will determine by its WMO header passed via JMS from QPID which decoder plugins it should awaken to process the data. It uses XML files full of regular expressions to match the patterns against the WMO header or file name of the raw data. A piece of data might awaken zero, one or more plugins which are interested in decoding the data. Base distribution XML Location: $EDEX_CONFIG/edex_static/distribution/ Site distribution XML Location: $EDEX_CONFIG/edex_static/site/XXX/distribution Note: Base distribution files should never be edited. To localize as any site, a distribution xml must be created in the site localization for the following: shef.xml Contract DG133W-05-CQ-1067 / DCN AWP.DOC.TOSCAT.LOC-06.00 / 17 October 2011 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this document. 21 AWIPS II Site Data Configuration & Localization: Step-by-Step Guide (Build 11.9) Follow these steps to copy the base file into a site directory, and then edit to match the LDM pqact pattern for hydro data set up earlier: • Log into an EDEX server, if not already logged in: # ssh root@$EDEX1 • If the directory does not exist, create the site-level plugin-filters edex static configuration directory: # mkdir –p ${EDEX_CONFIG}/edex_static/site/ /distribution • Change directories into the edex_static localization tree and copy the base file into the sitelevel directory: # cd $EDEX_CONFIG/edex_static # cp –a base/distribution/shef.xml site/XXX/distribution • Edit/create a file using vi: # vi shef.xml • Change the patterns to match the changes you made based on the hydro patterns in section 2.1; then save the file and quit the editor: # :wq! • Ensure proper ownership and permissions: # chown –R awips:awips ${EDEX_CONFIG}/edex_static/site/ /distribution # chmod –R 755 ${EDEX_CONFIG}/edex_static/site/ /distribution Contract DG133W-05-CQ-1067 / DCN AWP.DOC.TOSCAT.LOC-06.00 / 17 October 2011 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this document. 22 AWIPS II Site Data Configuration & Localization: Step-by-Step Guide (Build 11.9) 4.0 Common Utility Configuration Files Files configured in this section are common to EDEX and VIZ applications. 4.1 Create the Directory Structure Create the directory, replacing XXX with the site ID selected for localization (e.g., LWX) using a command similar to the following on the EDEX server: # ssh root@${EDEX1} # cd ${EDEX_CONFIG}/common_static/site # mkdir Create, also, the list of subdirectories needed for subsequent steps: # cd XXX # mkdir ffmp # mkdir goessndg # mkdir hydro # mkdir modelsndg # mkdir poessndg # mkdir radar # mkdir shef # mkdir upperair # mkdir warngen 4.2 Create FFMP Run Configuration File FFMP user and server side processes depend on a configuration file that defines which data is to be used for FFMP product creation. The file that controls this setup is the ffmp/FFMPRunConfig.xml file. Table 4.2-1 addresses the fields and what properties the entries in this file control. Table 4.2-1. FFMP Run Configuration Fields XML Tag Note: At least one product minimum is necessary for proper FFMP functionality. Description Defines an RFC whose flash flood guidance is to be used in the operation of FFMP. Defines a product and source to be used in the operational of FFMP. Required fields inside the tag (see example 4.2-1): name: Corresponds to the primary source of one of the products listed in the FFMPSourceConfig.xml. key: Corresponds to the site template for this product. Generally the ICAO of the radar sites in the case of DHR data, and either “hpe” or “bhpe” for HPN/E. Note: Be as descriptive as possible with the key, as this will be the name of the template created by FFMP. Contract DG133W-05-CQ-1067 / DCN AWP.DOC.TOSCAT.LOC-06.00 / 17 October 2011 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this document. 23 AWIPS II Site Data Configuration & Localization: Step-by-Step Guide (Build 11.9) XML Tag Description Creates a domain to define the area of interest for FFMP. Required fields inside the tag (see example 4.2-1): primary: Sets as primary domain. Valid values are true or false. Only one true value per file. cwa: The three-letter AWIPS ID of the CWA of interest. Example 4.2-1 shows an example file for site OAX. Use this example as a base template for creating an FFMPRunConfig.xml file for a new site. Example 4.2-1 Note: If no FFMPRunConfig.xml is generated manually, EDEX will attempt to create a minimal configuration automatically. This feature is not fully operational, so it is recommended that a manual setup be created. To create an FFMP Configuration file for a new site, follow these steps: • If not already logged in, log into an EDEX server and change to the site-level common utility configuration directory.:# ssh root@${EDEX1}: # cd ${EDEX_CONFIG}/common_static/site/XXX • Copy the OAX version of the FFMPRunConfig.xml into the new site’s directory: # cp –a ../OAX/ffmp/FFMPRunConfig.xml ffmp/ • Using the vi editor, enter editing mode for the file FFMPRunConfig.xml: # vi ffmp/FFMPRunConfig.xml • • Change the values for each XML tag to be specific to the new site. At least one KKRF KMSR , and field is necessary. Delete any multiple tags that are not necessary. Save and exit the vi text editor: # :wq! Contract DG133W-05-CQ-1067 / DCN AWP.DOC.TOSCAT.LOC-06.00 / 17 October 2011 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this document. 24 AWIPS II Site Data Configuration & Localization: Step-by-Step Guide (Build 11.9) 4.3 Progressive Disclosure Files The use of pre-created AWIPS I progressive disclosure files is necessary for certain portions of the AWIPS II software. However, AWIPS II Build 11.4 introduced the /awips2/edex/data/ndm directory, which is used as an input for placing files that create certain spi files. Use Table 4.3-1 to identify which files are necessary to create certain spi files. Table 4.3-1. SPI File Inputs Input File /awips/fxa/data/modelBufrStationInfo.txt /awips/fxa/data/MTR.goodness /awips/fxa/data/MTR.primary /awips/fxa/data/maritimeStationInfo.txt /awips/fxa/data/goesBufrStationInfo.txt /awips/fxa/data/poesBufrStationInfo.txt /data/fxa/nationalData/raob.goodness /data/fxa/nationalData/raob.primary Created spi File modelBufr.spi MTR.spi BUOY.spi goesBufr.spi poesBufr.spi raob.spi All spi files are created in the following location: ${EDEX_CONFIG}/common_static/site/XXX/basemaps Where XXX is the AWIPS site ID as configured by the setup.env file (see Section 7.0) Any other spi file not listed above as being created will have to be put into the basemaps directory pre-created. To put all of these files in place, firs, log into an EDEX server and change directories into the ndm directory # ssh root@${EDEX1} # cd ${EDEX_HOME}/data/ndm Next, copy the proper files into place from their original locations: # scp /awips/fxa/data/modelBufrStationInfo.txt . # scp /awips/fxa/data/MTR.goodness . # scp /awips/fxa/data/MTR.primary . # scp /awips/fxa/data/maritimeStationInfo.txt . # scp /awips/fxa/data/goesBufrStationInfo.txt . # scp /awips/fxa/data/poesBufrStationInfo.txt . # scp /data/fxa/nationalData/raob.primary . # scp /data/fxa/nationalData/raob.goodness . Contract DG133W-05-CQ-1067 / DCN AWP.DOC.TOSCAT.LOC-06.00 / 17 October 2011 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this document. 25 AWIPS II Site Data Configuration & Localization: Step-by-Step Guide (Build 11.9) 4.4 Create / Copy Common Hydrology Files Into Place The use of previously created hydrology files is necessary for certain parts of the AWIPS II software. Copy these files from their respective AWIPS I locations: • First, log into an EDEX server and change to the common utility configuration directory for the desired site: # ssh root@${EDEX1} # cd ${EDEX_CONFIG}/common_static/site/ Where XXX is the AWIPS ID for the site for which configuration is being performed. • Save and exit the vi editor # :wq! • Create the directory structure necessary for, and then copy the AWIPS I host ascii and binary file(s) into the new directory with the new file name syntax: If configuring for a WFO, use these commands: # mkdir –p hydro/hydroapps/geo_data/host #cp –r /awips/hydroapps/geo_data/host/ascii hydro/hydroapps/geo_data/host/ # cp –r /awips/hydroapps/geo_data/host/binary hydro/hydroapps/geo_data/host/ If configuring for an RFC, use this command: # mkdir –p hydro/hydroapps/geo_data/xxxxx # cp –r /awips/hydroapps/geo_data/xxxxx/ascii hydro/hydroapps/geo_data/xxxxx/ # cp –r /awips/hydroapps/geo_data/host/binary hydro/hydroapps/geo_data/xxxxx/ Where xxxxx is your 5-letter RFC identifier. For example, RHA is “marfc” • If the directory does not exist, create the site-level hydro common configuration directory for geo data: # mkdir –p hydro/hydroapps/whfs/local/data/geo • Copy the AWIPS I local geo data into the new directory with the new file name syntax: # cp –ar /awips/hydroapps/whfs/local/data/geo/* hydro/hydroapps/local/data/geo/ • Copy the AWIPS I local MPE-related files into the new directory with the new file name syntax: # mkdir –p hydro/hydroapps/precip_proc/local/data/app/mpe Contract DG133W-05-CQ-1067 / DCN AWP.DOC.TOSCAT.LOC-06.00 / 17 October 2011 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this document. 26 AWIPS II Site Data Configuration & Localization: Step-by-Step Guide (Build 11.9) # cp –r /awips/hydroapps/precip_proc/local/data/app/mpe/prism hydro/hydroapps/precip_proc/local/data/app/mpe/ # cp –r /awips/hydroapps/precip_proc/local/data/app/mpe/misbin hydro/hydroapps/precip_proc/local/data/app/mpe/ # cp –r /awips/hydroapps/precip_proc/local/data/app/mpe/utiltriangles hydro/hydroapps/precip_proc/local/data/app/mpe/ • Copy in XDAT related files [RFC Only]: # mkdir -p hydro/hydroapps/rfc/xdat/parameters/groups # cp –r /awips/hydroapps/rfc/xdat/parameters/groups/* hydro/hydroapps/rfc/xdat/parameters/groups • Copy the AWIPS I group_definition.cfg file into place for the Time Series application: # cp /awips/hydroapps/whfs/local/data/app/timeseries/group_definition. cfg hydro/ • Ensure proper ownership and permission of the common hydro configuration directory tree: # chown –R awips:fxalpha hydro # chmod –R 775 hydro • Copy the AWIPS I metar.cfg file into the new directory with the new file name syntax: # cp /awips/hydroapps/whfs/local/data/app/metar2shef/metar.cfg shef/ • Ensure proper ownership and permission of the common shef configuration directory tree: # chown –R awips:fxalpha shef # chmod –R 775 shef 4.4.1 Create / Edit Site Apps_defaults File A site can use its existing AWIPS I .Apps_defaults_site file for use in the AWIPS II environment. However, not all of the tokens from the AWIPS I file are necessary for AWIPS II. As of the date on this document, the following tokens are required for AWIPS II to operate. This token list will be dynamic in nature and change as more tokens are discovered. • Copy the AWIPS I .Apps_defaults_site file into the new directory with the new file name syntax: # cp /awips/hydroapps/.Apps_defaults_site hydro/Apps_defaults • Verify that the following tokens ARE NOT defined in the Apps_defaults file, as they need to remain their AWIPS II baseline definition. If these tokens are present, use the vi editor to edit the file and remove them.: apps_dir mpe_dir pproc_dir Contract DG133W-05-CQ-1067 / DCN AWP.DOC.TOSCAT.LOC-06.00 / 17 October 2011 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this document. 27 AWIPS II Site Data Configuration & Localization: Step-by-Step Guide (Build 11.9) • Verify that the following tokens ARE defined in the Apps_defaults file for proper EDEX Hydro functionality. If they are not defined the edit the file to add these tokens in: # vi hydro/Apps_defaults If configuring for a WFO: mpe_site_id : xxx (where xxx is the lower-case 3 letter identifier of your WFO) mpe_center_lat : ##.### (where ##.### is the latitude coordinate of your site’s center point) mpe_center_lon : ##.### (where ##.### is the longitude coordinate of your site’s center point) fxa_local_site : XXX (where XXX is the upper-case 3 letter identifier of your WFO) If configuring for an RFC: mpe_site_id : xxx (where xxx is the lower-case 3 letter identifier of your RFC) mpe_center_lat : ##.### (where ##.### is the latitude coordinate of your site’s center point) mpe_center_lon : ##.### (where ##.### is the longitude coordinate of your site’s center point) fxa_local_site : XXX (where XXX is the upper-case 3 letter identifier of your RFC) rfctb_site_id : xxx (where xxx is the lower-case 3 letter identifier of your RFC) st3_rfc : xxxxx (where xxxxx is the lower-case 5 letter identifier of your RFC) ifp_rfc : xxxxx (where xxxxx is the lower-case 5 letter identifier of your RFC) rfcw_rfcname : xxxxx (where xxxxx is the lower-case 5 letter identifier of your RFC) • Ensure proper ownership and permission of the common hydro configuration directory tree: # chown –R awips:fxalpha hydro/Apps_defaults Contract DG133W-05-CQ-1067 / DCN AWP.DOC.TOSCAT.LOC-06.00 / 17 October 2011 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this document. 28 AWIPS II Site Data Configuration & Localization: Step-by-Step Guide (Build 11.9) # chmod –R 775 hydro/Apps_defaults 4.5 Create Radar, NCEP/Hydro and UpperAir Menu Files EDEX dynamically creates the files necessary for the CAVE Radar, NCEP/Hydro and UpperAir menus based on specific input files in the common utility configuration directory tree. Create these files, using those delivered with the OAX localization as guides: 4.5.1 Create/Edit radarsInUse.txt File Section Note: As of the date on this document, there is a DR created against the software’s ability to successfully create the radar menus in CAVE when a site has no ASR or ARSR radars listed in the radarsInUse.txt configuration file. For this reason, if the desired site does not ingest an ASR or ARSR radar, please leave the default entries from the OAX version of this file. Radar menu configuration is done dynamically by the EDEX server when launched based on a single input file: $EDEX_CONFIG/common_static/site/ /radar/radarsInUse.txt The file is split into six main sections which are used by the software to build the menus on the main CAVE toolbar, and under the Radar menu. To configure the radar menus in CAVE, do the following: • If not already logged in, log into an EDEX server and change to the common utility configuration directory for the desired site: # ssh root@${EDEX1} # cd ${EDEX_CONFIG}/common_static/site/ • Copy the base version of radarsInUse.txt into the desired site’s directory: # cp ../../base/radar/radarsInUse.txt radar/ • Change directories to the local site’s common radar configuration directory: # cd radar • Edit the radarsInUse.txt file to configure radar menus: # vi radarsInUse.txt Table 4.5-1 describes the five main sections used for radar menu configuration. It includes the title for each section (which is a necessary line in the radarsInUse.txt file) and a description of what should follow the title line in the file. Contract DG133W-05-CQ-1067 / DCN AWP.DOC.TOSCAT.LOC-06.00 / 17 October 2011 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this document. 29 AWIPS II Site Data Configuration & Localization: Step-by-Step Guide (Build 11.9) Table 4.5-1. radarsInUse.txt title Lines and Descriptions radarsInUse.txt Title # LOCAL_RADARS (including terminal) MUST HAVE THIS LINE Description of What Follows Includes a list of four-letter radar IDs to list in the main tool bar of CAVE. Each entry will be a full menu of dedicated radar products and will be created between the Satellite and Radar menus. Each ID is separated by a new line character. In AWIPS I, this is every radar ID listed with a + sign in one of the following files: # DIAL_RADARS - MUST HAVE THIS LINE /data/fxa/customFiles/radarsOnMenu.txt OR /data/fxa/customFiles/XXX-radarsOnMenu.txt OR /awips/fxa/data/localization/XXX/XXX-radarsOnMenu.txt Includes a list of four-letter radar IDs to list under the Dial Radars submenu of the Radars menu in CAVE. Each ID is separated by a new line character. In AWIPS I, this is every radar ID not listed with a + sign in one of the following files: # ASR_RADARS - MUST HAVE THIS LINE # ARSR_RADARS - MUST HAVE THIS LINE # MOSAIC_RADARS - MUST HAVE THIS LINE /data/fxa/customFiles/radarsOnMenu.txt OR /data/fxa/customFiles/XXX-radarsOnMenu.txt OR /awips/fxa/data/localization/XXX/XXX-radarsOnMenu.txt Includes a list of four-letter ASR radar (e.g., eeri) IDs to list in the main tool bar of CAVE. Each entry will get its own menu created and should be separated by a new line character. DR 8004 workaround requires you add a radar ID here. Use eeri. Includes a list of four-letter ARSR radar (e.g., fqwa) IDs to list in the main tool bar of CAVE. Each entry will get its own menu created and should be separated by a new line character. DR 8004 workaround requires you add a radar ID here. Use fqwa. Includes a list of four-letter radar IDs to load when selecting the default Mosaic products under the Radar menu. Each ID is separated by a new line character. In AWIPS I you can find a list of radars in the mosaicRadarList.txt file located in the /awips/fxa/data/localizationDataSets/XXX directory. An example of a localized radarsInUse.txt file follows. Contract DG133W-05-CQ-1067 / DCN AWP.DOC.TOSCAT.LOC-06.00 / 17 October 2011 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this document. 30 AWIPS II Site Data Configuration & Localization: Step-by-Step Guide (Build 11.9) # DO NOT EDIT LINES BEGINNING WITH '#' # LOCAL_RADARS (including terminal) - MUST HAVE THIS LINE koax tmsp ktlx # DIAL_RADARS - MUST HAVE THIS LINE kabr kcys kdmx kgld kuex keax karx kmpx klnx kdvn kudx kfsd ktwx tmsp # ASR_RADARS - MUST HAVE THIS LINE eeri # ARSR_RADARS - MUST HAVE THIS LINE fqwa # MOSAIC_RADARS - MUST HAVE THIS LINE koax kabr kcys kdmx kgld kuex keax karx kmpx klnx kdvn kudx kfsd ktwx • Save and quit the text editor # :wq! Note: The # CWA entry in the OAX version of this file is no longer used by the software, so a newly created file for another site does not need to include the line. Contract DG133W-05-CQ-1067 / DCN AWP.DOC.TOSCAT.LOC-06.00 / 17 October 2011 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this document. 31 AWIPS II Site Data Configuration & Localization: Step-by-Step Guide (Build 11.9) 4.5.2 Create/Edit the eavConfigTable.txt The EAV display and sample strings are done through the same file as in AWIPS I, the eavConfigTable.txt. Configure this file by doing the following: • If not already logged in, log into an EDEX server and change to the common utility configuration directory for the desired site: # ssh root@${EDEX1} # cd ${EDEX_CONFIG}/common_static/site/ • Copy the AWIPS I version of eavConfigTable.txt into the desired site’s directory: # cp /data/fxa/nationalData/eavConfigTable.txt radar/ 4.5.3 Create/Edit raobSitesInUse.txt File For Upper Air Menus The Upper Air RAOB menus are created dynamically with the input of a single common utility configuration file through the EDEX menu builder. File Location: ${EDEX_CONFIG}/common_static/site/ /upperair The file has two necessary menu lines: # UPPER_AIR menu # LOCAL_UPPER_AIR submenu The EDEX menu builder will look for these lines, and parse the entries that follow to create the RAOBs that sit on the Upper Air Menu itself, and in the Local submenu of the Upper Air CAVE dropdown. Each entry under the template line consists of three pieces of information: • • • RAOB icao. This is the four-letter identifier for which the RAOB is best known. WMO ID. The WMO ID can be found as the stationed returned from the SQL statement below, or within the existing AWIPS file /awips/fxa/data/station_table.dat. Location Description. The Location Description should match the RAOB icao. This can be found with the following command on a device that has access to the database server. (Please note that PostgreSQL must be running for this to work: # psql –h $DB_SERVER –U awips –d metadata –c “select name,stationid from common_obs_spatial where catalogtype=22 and icao=’XXXX’;” Note: Substitute the RAOB icao for XXXX in the above command. Note: This information can also be found in the AWIPS I files /data/fxa/nationalData/raobProductButtons.txt, /awips/fxa/data/ICAODICT.TBL, or /awips/fxa/data/raobStationInfo.txt An example of the file configured for site OAX follows. Contract DG133W-05-CQ-1067 / DCN AWP.DOC.TOSCAT.LOC-06.00 / 17 October 2011 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this document. 32 AWIPS II Site Data Configuration & Localization: Step-by-Step Guide (Build 11.9) # DO NOT EDIT LINES BEGINNING WITH '#' # FULLY DEFINES ALL SITES USED FOR THE LOCALIZED UPPER AIR MENU IN CAVE # Format is as follows : # icao city name, state # KXXX SiteID Anywhere, US # UPPER_AIR menu KOAX 72558 Omaha, NE # LOCAL_UPPER_AIR submenu KABR 72659 Aberdeen, SD KBIS 72764 Bismarck, ND KLBF 72562 North Platte, NE KMPX 72649 Minneapolis, MN KOAX 72558 Omaha, NE KUNR 72662 Rapid City, SD KTOP 72456 Topeka, KS To configure the Upper Air Menu, follow these steps: • Collect the needed RAOB icaos, WMO IDs, and Descriptions from an AWIPS I D2D or the AWIPS I localization file /awips/fxa/data/localizationDataSets/ /raobLocalMenus.txt • If not already logged in, log into an EDEX server and change to the common utility configuration directory for the desired site: # ssh root@${EDEX1} # cd ${EDEX_CONFIG}/common_static/site/ • Copy the base version of hydroSitesInUse.txt into the desired site’s directory: # cp ../../base/upperair/raobSitesInUse.txt upperair/ • Edit the hydroSitesInUse.txt file and, using the example above, change the entries to the correct entries for the desired site: # vi upperair/raobSitesInUse.txt • If not already present, create a line containing this exact text (including the ‘#’ symbol): # UPPER_AIR menu • Type in the RAOB icao, WMOID and Description – all separated by a space for each RAOB desired to be seen under the Upper Air menu of CAVE. • If not already present, create a line containing this exact text (including the ‘#’ symbol): # LOCAL_UPPER_AIR submenu • Type in the RAOB icao, WMOID, and Description – all separated by a space for each RAOB desired to be seen under the Local submenu of the Upper Air menu of CAVE. • Save the file and exit the text editor: # :wq! Contract DG133W-05-CQ-1067 / DCN AWP.DOC.TOSCAT.LOC-06.00 / 17 October 2011 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this document. 33 AWIPS II Site Data Configuration & Localization: Step-by-Step Guide (Build 11.9) 4.6 Copy / Edit WarnGen Templates [ WFO Only ] WarnGen templates are common to both EDEX and VIZ applications. For more information on editing the templates, please see Appendix B. 4.7 Disable Fog Monitor [ RFC Only ] River Forecast Centers do not need to run the fog monitor plugin within EDEX. Sites configuring EDEX to run as an RFC should disable this plugin by following these steps: • If not already logged in, log into an EDEX server and change to the base level common utility configuration directory for monitoring plugins: # ssh root@${EDEX1} # cd ${EDEX_CONFIG}/common_static/base/monitoring • Change the token for the fog monitor plugin from true, to false: # sed –i ‘/fog/s/true/false/1’ MonitorPluginState.xml 4.8 Correct Permissions and Ownership After making the correct files with proper entries, ownership must be set on the files in the directory tree. • If not already logged in, log into an EDEX server and change to the site-level common utility configuration directory” # ssh root@${EDEX1} # cd ${EDEX_CONFIG}/common_static/site/ • Recursively change the ownership and permissions of the site directory created in the previous steps: # chown –R awips:fxalpha # chmod –R 755 Contract DG133W-05-CQ-1067 / DCN AWP.DOC.TOSCAT.LOC-06.00 / 17 October 2011 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this document. 34 AWIPS II Site Data Configuration & Localization: Step-by-Step Guide (Build 11.9) 5.0 EDEX Utility Configuration Files Files configured in this section are specific to EDEX functionality. 5.1 Create the Directory Structure Create the directory, replacing XXX with the site ID selected for localization (e.g., LWX) using a command similar to the following on the EDEX server: # ssh root@${EDEX1} # cd ${EDEX_CONFIG}/edex_static/site # mkdir Create, also, the list of subdirectories needed for subsequent steps: # # # # 5.2 cd XXX mkdir –p config/gfe mkdir –p grib/subgrids mkdir smartinit Create/Edit GFE Configuration Files GFE’s siteConfig.py and localConfig.py files are placed within the config/gfe directory for use by EDEX in its GFE functions. 5.2.1 Copy / Edit siteConfig.py Copy the siteConfig.py from the delivered OAX localization and use it as a template for creating the siteConfig.py for the desired site: • If not already logged in, log into an EDEX server and change to the site-level edex utility configuration directory: # ssh root@${EDEX1} # cd ${EDEX_CONFIG}/edex_static/site/ • Copy the delivered OAX siteConfig.py into the desired site’s directory: # cp –a OAX/config/gfe/siteConfig.py /config/gfe • Edit the file using a text editor such as vi and replace instances of OAX with the desired site’s ID. Example 5.2-1 shows the OAX siteConfig.py with the needed changes highlighted in red:: # vi /config/gfe/siteConfig.py Example 5.2-1: Text of AWIPS II OAX siteConfig.py import os, socket GFESUITE_HOME = GFESUITE_SERVER GFESUITE_PORT GFESUITE_SITEID GFESUITE_PRDDIR GFESUITE_MHSID os.environ['EDEX_HOME']+"/../GFESuite" = "localhost" = '98000000' = 'OAX' = GFESUITE_HOME+"/products" = 'OAX' Contract DG133W-05-CQ-1067 / DCN AWP.DOC.TOSCAT.LOC-06.00 / 17 October 2011 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this document. 35 AWIPS II Site Data Configuration & Localization: Step-by-Step Guide (Build 11.9) Note: For non-operational sites, the GFESUITE_MHSID should match the ID of your system. For example, Southern Region Headquarters would put EHU instead of whichever site is being configured by following this document’s procedures. The GFESUITE_SITEID must match the AWIPS ID of the site for which the software is being configured. • Save and exit the text editor: # :wq! 5.2.2 Copy/Edit localConfig.py Copy the localConfig.py file from the AWIPS I instance of the file. Certain tokens must be changed in order to use the file successfully in an AWIPS II environment. Table 5.2.2-1 lists all known changes needed. Table 5.2.2-1. Necessary localConfig.py Changes From AWIPS I to AWIPS II AWIPS I Token serverConfig.D2DDIRS AWIPS II Equivalent Description serverConfig.D2DMODELS Change all instances of D2DDIRS to D2DMODELS and change the arguments passed from the AWIPS I netCDF directory to the AWIPS II dataURI model name. Example: CHANGE serverConfig.D2DDIRS.append((‘/data/fxa/Grid/net CDF/CONUS212/SREF/’,’SREF’)); TO serverConfig.D2DMODELS.append(('SREF21 2', 'SREF')); You can retrieve the model name from the grib_models table in the metadata database with the following SQL statement: Any local model coming into AWIPS I Model name may change based on addition of product into AWIPS II select distinct(modelname) from grib_models; or from the AWIPS II file /awips2/edex/data/utility/common_static/base/grid/g ribModels.xml If any local model is sent into GFE in AWIPS I, this model will have to first be added into AWIPS II in order to utilize in GFE’s localConfig.py. Immediately after copying this file, comment out any entries pertaining to local model data until these data sets are added into the system. See Table 5.3-1 for baseline model name reference. • If not already logged in, log into an EDEX server and change to the site level edex utility configuration directory: # ssh root@${EDEX1} Contract DG133W-05-CQ-1067 / DCN AWP.DOC.TOSCAT.LOC-06.00 / 17 October 2011 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this document. 36 AWIPS II Site Data Configuration & Localization: Step-by-Step Guide (Build 11.9) # cd ${EDEX_CONFIG}/edex_static/site/ • Copy the AWIPS I GFE localConfig.py into the correct location in the AWIPS II directory structure: # cp /awips/GFESuite/primary/etc/SITE/localConfig.py /config/gfe • Edit the file, making necessary changes to allow for successful AWIPS II EDEX start: # vi /config/gfe/localConfig.py Example 5.2.2-1 shows an example of converting a line from AWIPS I to AWIPS II syntax. Example 5.2.2-1 1. AWIPS I Entry: ############################# ### Add RUC40 database ### serverConfig.D2DDIRS.append(('/data/fxa/Grid/SBN/netCDF/Grid236/RUC2/', ‘RUC40’)) serverConfig.INITMODULES["myRUC40"] = ["RUC40"] RUC40 = ('RUC40', GRID, '', NO, NO, 2, 0) RUC40_Parms = [([QPF, SnowAmt], LT6NG), ([Haines], LT3NG), ([Temp, Td, Wind, MixHgt, TransWind, FzLevel, RH, Sky], TC1), ([MaxT], MaxTTC), ([MinT], MinTTC), ([MaxRH],MaxRHTC),([MinRH],MinRHTC)] 2. Information Gathering For AWIPS-II: # grep -B1 RUC236 $EDEX_CONFIG/common_static/base/grid/gribModels.xml RUC40 RUC236 3. AWIPS II Entry: ############################# ### Add RUC40 database ### serverConfig.D2DMODELS.append(('RUC236','RUC40')) serverConfig.INITMODULES["myRUC40"] = ["RUC40"] RUC40 = ('RUC40', GRID, '', NO, NO, 2, 0) RUC40_Parms = [([QPF, SnowAmt], LT6NG), ([Haines], LT3NG), ([Temp, Td, Wind, MixHgt, TransWind, FzLevel, RH, Sky], TC1), ([MaxT], MaxTTC), ([MinT], MinTTC), ([MaxRH],MaxRHTC),([MinRH],MinRHTC)] Contract DG133W-05-CQ-1067 / DCN AWP.DOC.TOSCAT.LOC-06.00 / 17 October 2011 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this document. 37 AWIPS II Site Data Configuration & Localization: Step-by-Step Guide (Build 11.9) Note: If EDEX is having problems starting after making all the previous changes, just rename the file to something other than localConfig.py. This file is not necessary for proper functionality. A restart of EDEX is necessary after renaming. • When finished, save the file and exit the text editor: # :wq! 5.2.3 Copy / Edit SmartInit Files Copy the local SmartInit files from the AWIPS I store location to the AWIPS II location for use within EDEX. • If not already logged in, log into an EDEX server and change to the site-level edex utility configuration directory: # ssh root@${EDEX1} # cd ${EDEX_CONFIG}/edex_static/site/ • Copy the AWIPS I GFE smartInit scripts into the correct location in the AWIPS II directory structure: # scp /awips/GFESuite/primary/etc/SITE/