Instructions For Bulk Loading Data
User Manual:
Open the PDF directly: View PDF 
.
Page Count: 5
| Download | |
| Open PDF In Browser | View PDF |
Instructions for bulk loading data The BDRS allows Users with Administrator privileges to bulk load data into the system using TM an Excel spreadsheet. The process can be temperamental so it important to follow these instructions exactly. The basic process is as follows: TIP: It is important that you use the downloaded XLS template as it will contain all of the relevant fields with established mappings to the database (important when you come to upload the completed spreadsheet). Pre-requisites: o All surveys which include species observations must have all of the species in the dataset loaded into the BDRS taxonomy, otherwise the load will fail. (see Creating taxon groups and Creating taxon pages for how to do this) o All of the data to be loaded must have a corresponding field included in the BDRS survey form before downloading the form template. (See Creating a BDRS survey) o Locations must have already been created in the BDRS and associated with the relevant survey (see notes on how to create locations). NB. If the locations in the dataset are just a collection of point locations defined by latitude and longitude values, establishing pre-defined locations is not necessary. (Note: Lat/Long values must be in WGS84 decimal format). How To: 1. Create Survey Form 1. See Creating a survey. TIPS: a. Minimise the number of mandatory fields in the form. If you try to submit the data spreadsheet with null (blank) values in a mandatory field, it will fail to load. b. Ensure that all locations (where they are pre-defined) are created and associated with the survey. These will appear in a Locations tab in the data spreadsheet template. If you have locations in your data which are not in the Locations tab of the data spreadsheet, it will fail to load. c. Ensure that all species recorded in the data have species pages in the appropriate taxon groups before downloading the data template. These will appear in a Taxonomy tab in the data spreadsheet. If you have species in your spreadsheet which are not in the Taxonomy tab of the data spreadsheet, it will fail to load. 2. Download the Form Template 2. In the BDRS, goto the Admin > Manage Data tab and select “Bulk Data Loader”. 3. Select the relevant project (survey) and click the Download template button. You may be prompted to select a program for the spreadsheet to open into. This will then open the spreadsheet with each form field shown as a column heading on the Data sheet (Observations tab). Note the tabs on the spreadsheet (see Tips above) 3. Populate the Form Template 4. You can populate the form by either manually entering a row for each record, or by copying and pasting columns of data from an existing spreadsheet. TIPS: 2. If you have an existing spreadsheet with data to import into the BDRS, copy and paste data in columns from this sheet into the appropriate fields in the form template. 3. All mandatory fields in the form template must have a valid value or the upload will fail. 4. Ignore columns A-D in the Observations tab. 5. Date fields must be entered in or converted to the correct format – ie. dd/mm/yyyy 6. Time fields must be entered in or converted to the correct format – ie. hh:mm:ss AM 7. The location in the Observations tab can be either a named location (if the coordinate has a named location associated with it), or a WGS84 decimal Latitude/Longitude value. 8. The species name entered in the Observations tab can be either a scientific name or a common name if the scientific name in the Taxonomy tab has a common name associated with it, otherwise it must be the scientific name only. 9. For taxonomy and locations, spelling and numerical values must exactly match between these tabs and the Observations tab. NB. • You cannot add, remove or move columns in the spreadsheet. If you need to modify these at all you should return to the survey form in the BDRS and edit that, then repeat from Step 2 above. • You should not change the name of the spreadsheet other than to remove the “Copy of” prefix text that is often appended to the front of the file name on save by TM Microsoft Excel . 5. Check the following: o All mandatory fields in the form template have valid values for all records o All dates and times are in the correct format. o All locations in the Observations tab have a corresponding named location and WGS84 decimal Latitude/Longitude value in the Location tab. Spelling and numerical values must exactly match between the two tabs. o All species listed in the Observations tab have a corresponding species name and, if applicable, common name in the Taxonomy tab. Spelling must also exactly match between the two tabs. o All species listed in the Taxonomy tab must have a taxon group associated with them. o All taxon groups and all species listed on the Taxonomy tab must exist in the BDRS. 6. Save the completed form template to your local drive (where you can find it again), ensuring that the file name is exactly the same as it was when downloaded. TM [NB. Sometimes Microsoft Excel will append “Copy of ..” as a prefix to the file name]. 4. Upload the Form Template 7. In the BDRS, goto the Admin > Manage Data tab and select “Bulk Data Loader”. 8. Select the relevant project (survey) 9. Browse to find the completed form template file and click “Upload spreadsheet”. 10. The upload process may take a while, but when completed should indicate a successful load. NB. (VIP) • Do not reload the same records in either the same spreadsheet or a different one without first checking the data load (see below) to see whether they loaded successfully, regardless whether an error message displays or not. Sometimes you will get an error message on the load, but all or most of the records will have actually loaded successfully. If you try to reload, you will likely get duplicate records. These cannot be easily deleted. 5. Check Data Load 11. Count the number of records (rows) in your upload spreadsheet. 12. In the BDRS, goto menu item “View Records” and select either “My Sightings” or “Advanced Review”. 13. If “My Sightings”, select the appropriate survey, owner, and date range for the records of interest. 14. If “Advanced Review”, filter the data using the facets to show the relevant survey, contributor, and dates applicable to the survey records in your upload spreadsheet. 15. For both steps 13 and 14, click on the “Table” tab. At the bottom of the page will be a record count for that dataset. This should match the record count in the upload spreadsheet. 16. If the record counts differ, then further investigation of the data is required – see Troubleshooting below. Troubleshooting - What can go wrong? If you are experiencing problems with data loading, please work your way progressively down the following list of potential issues. Problem Cause Remedy The required survey is not available in the drop-down list The survey has not been made public The Administrator must check the “Publish” flag on the survey. The spreadsheet template does not have all of the fields that I need for my data Required fields have not been added to the online survey form. The administrator must insert the required fields into the survey form. See Developing a survey. Error loading the spreadsheet A scientific name or common name in the data tab spreadsheet is not in the Taxonomy tab. Species must be recorded in the BDRS for it to be able to accept a species name. See: Importing Taxa NB. Spelling of names must be identical between the Data and Taxonomy tabs Error loading the spreadsheet A named location or coordinate in the data tab spreadsheet is not in the Location tab. Named locations and corresponding coordinates must be in the BDRS for it to be able to accept a named location. NB. The character strings for both named locations and coordinates must be identical between the Data and Location tabs. Error loading the spreadsheet Data loading process times out. Break the data into multiple workbooks. NB. The file name should remain the same, so on each successful load, save the file as version 1, then replace the data with the next batch, load, and save as version 2, etc. Mismatch in record count between spreadsheet and loaded records Some records load and some fail to load. Thoroughly check the data to determine which records in the dataset did load. Remove these from the upload spreadsheet and try to reload the subset that did not load successfully. NB. If it fails a second time, contact the system administrator.
Source Exif Data:
File Type : PDF File Type Extension : pdf MIME Type : application/pdf PDF Version : 1.4 Linearized : No Page Count : 5 XMP Toolkit : XMP toolkit 2.9.1-13, framework 1.6 About : 8bf0856b-277b-11e1-0000-d9a01d2a4ddd Producer : GPL Ghostscript 8.64 Keywords : () Modify Date : 2011:12:13 11:18:29+11:00 Create Date : 2011:12:13 11:18:29+11:00 Creator Tool : \376\377\000P\000D\000F\000C\000r\000e\000a\000t\000o\000r\000 \000V\000e\000r\000s\000i\000o\000n\000 \0000\000.\0009\000.\0008 Document ID : 8bf0856b-277b-11e1-0000-d9a01d2a4ddd Format : application/pdf Title : \376\377\000I\000n\000s\000t\000r\000u\000c\000t\000i\000o\000n\000s\000 \000f\000o\000r\000 \000b\000u\000l\000k\000 \000l\000o\000a\000d\000i\000n\000g\000 \000d\000a\000t\000a Creator : \376\377\000b\000r\000e\0002\0001\0003 Description : () Author : bre213 Subject :EXIF Metadata provided by EXIF.tools