Ibm High Performance Storage System Hpss Users Manual

2015-02-02

• Chapter 1. Release 6.2
  • 1.1.New Features
    • 1.1.1.DCE Replacement
    • 1.1.2.Linux Support
    • 1.1.3.Security
    • 1.1.4.SCSI PVR
    • 1.1.5.HPSS VFS Interface
    • 1.1.6.GridFTP Interface
    • 1.1.7.SAN3P/PIO Support
    • 1.1.8.Additional hpssadm Configuration Options
    • 1.1.9.Additional hpssadm operations
    • 1.1.10.Additional Library and Device Support
    • 1.1.11.SAN Virtual Volume ID Mapping
    • 1.1.12.Drive Pools
    • 1.1.13.FTP Enhancement
    • 1.1.14.mkhpss Enhancement
    • 1.1.15.DB2 Monitoring
    • 1.1.16.File Family enhancements
    • 1.1.17.SSM Configuration Files Consolidation
    • 1.1.18. Mover Enhancement
  • 1.2.Retired Features
  • 1.3.Deferred Features
  • 1.4.HPSS Changes
    • 1.4.1.Documentation Organization Changes
    • 1.4.2.Metadata Changes
    • 1.4.3.DMAP Gateway Changes
      • 1.4.3.1.Creating an XDSM Fileset
      • 1.4.3.2.Viewing DMAP Gateway XDSM Fileset Information
      • 1.4.3.3.Viewing Core Server XDSM Fileset Information
    • 1.4.4.SSM Changes
      • 1.4.4.1.Changes Affecting Sites Upgrading Directly from 4.5
      • 1.4.4.2.Changes Affecting Sites Upgrading from 5.1
• Chapter 2. HPSS Basics
  • 2.1.Introduction
  • 2.2.HPSS Capabilities
    • 2.2.1.Network-centered Architecture
    • 2.2.2.High Data Transfer Rate
    • 2.2.3.Parallel Operation
    • 2.2.4.Based on Standard Components
    • 2.2.5.Data Integrity Through Transaction Management
    • 2.2.6.Multiple Hierarchies and Classes of Services
    • 2.2.7.Storage Subsystems
  • 2.3.HPSS Components
    • 2.3.1.HPSS Files, Filesets, Volumes, Storage Segments and Related Metadata
    • 2.3.2.HPSS Servers
    • 2.3.3.HPSS Storage Subsystems
    • 2.3.4.HPSS Infrastructure
    • 2.3.5.HPSS User Interfaces
    • 2.3.6.HPSS Management Interfaces
    • 2.3.7.HPSS Policy Modules
  • 2.4.HPSS Hardware Platforms
    • 2.4.1.Server Platforms
    • 2.4.2.Client Platforms
    • 2.4.3.Mover Platforms
• Chapter 3. HPSS Planning
  • 3.1.Overview
    • 3.1.1.HPSS System Architecture
    • 3.1.2.HPSS Configuration Planning
    • 3.1.3.Purchasing Hardware and Software
    • 3.1.4.HPSS Operational Planning
    • 3.1.5.HPSS Deployment Planning
  • 3.2.Requirements and Intended Uses for HPSS
    • 3.2.1.Storage System Capacity
    • 3.2.2.Required Throughputs
    • 3.2.3.Load Characterization
    • 3.2.4.Usage Trends
    • 3.2.5.Duplicate File Policy
    • 3.2.6.Charging Policy
    • 3.2.7.Security
      • 3.2.7.1.Cross Realm Access
    • 3.2.8.High Availability Option
  • 3.3.Prerequisite Software Considerations
    • 3.3.1.Prerequisite Software Overview
      • 3.3.1.1.DB2
      • 3.3.1.2.Kerberos
      • 3.3.1.3.LDAP and IBM Kerberos
      • 3.3.1.4.Java
    • 3.3.2.Prerequisite Summary By HPSS Node Type
      • 3.3.2.1.HPSS Server Nodes
        • 3.3.2.1.1.AIX Requirements
        • 3.3.2.1.2.Linux Requirements
      • 3.3.2.2.HPSS Mover Nodes 
        • 3.3.2.2.1.AIX Requirements
        • 3.3.2.2.2.Linux Requirements
        • 3.3.2.2.3.Solaris Requirements
        • 3.3.2.2.4.IRIX Requirements
      • 3.3.2.3.HPSS Client Nodes
        • 3.3.2.3.1.SSM Client Requirements
        • 3.3.2.3.2.Client API Requirements
        • 3.3.2.3.3.FTP/PFTP Client Requirements
      • 3.3.2.4.HPSS HDM Nodes (Linux only)
  • 3.4.Hardware Considerations
    • 3.4.1.Network Considerations
    • 3.4.2.Robotically Mounted Tape
      • 3.4.2.1.IBM 3494
      • 3.4.2.2.Drive-Controlled LTO Libraries (IBM 3582, IBM 3583, IBM 3584, Spectralogic T120)
      • 3.4.2.3.STK L40, STK SL500, STK SL8500
      • 3.4.2.4.STK
      • 3.4.2.5.ADIC AML
    • 3.4.3.Manually Mounted Tape
    • 3.4.4.Tape Devices
      • 3.4.4.1.Multiple Media Support
    • 3.4.5.Disk Devices
    • 3.4.6.Special Bid Considerations
  • 3.5.HPSS Sizing Considerations
    • 3.5.1.HPSS User Storage Space
    • 3.5.2.HPSS Infrastructure Storage Space
    • 3.5.3.HPSS Filesystems
      • 3.5.3.1./opt/hpss
      • 3.5.3.2./var/hpss
      • 3.5.3.3./var/hpss/adm/core
      • 3.5.3.4./var/hpss/hpssdb
      • 3.5.3.5./var/hpss/hpssdb/subsys1 & subsysX
      • 3.5.3.6./db2/backups/cfg
      • 3.5.3.7./db2/backups/subsys1 & subsysX
      • 3.5.3.8./db2/log/cfg
      • 3.5.3.9./db2/log/subsys1 & subsysX
      • 3.5.3.10./db2/mirror-log/cfg
      • 3.5.3.11./db2/mirror-log/subsys1 & subsysX
      • 3.5.3.12./db2/mirror-backup/cfg
      • 3.5.3.13./db2/mirror-backup/subsys1 & subsysX
      • 3.5.3.14.SUBSYS1 Database Allocation
    • 3.5.4.HPSS Metadata Space
      • 3.5.4.1.SMS versus DMS Space
      • 3.5.4.2.'CFG' Database Allocation
      • 3.5.4.3.'SUBSYS' Database Allocation
      • 3.5.4.4.DB2 Disk Space
    • 3.5.5.System Memory and Disk Space
      • 3.5.5.1.Operating System Disk Spaces
      • 3.5.5.2.System Disk Space Requirements for Running SSM
      • 3.5.5.3.System Memory and Paging Space Requirements
  • 3.6.HPSS Interface Considerations
    • 3.6.1.Client API
    • 3.6.2.FTP
    • 3.6.3.Parallel FTP
    • 3.6.4.XFS
  • 3.7.HPSS Server Considerations
    • 3.7.1.Core Server
    • 3.7.2.Migration/Purge Server
    • 3.7.3.Gatekeeper
    • 3.7.4.Location Server
    • 3.7.5.PVL
    • 3.7.6.PVR
      • 3.7.6.1.STK PVR
      • 3.7.6.2.LTO PVR
      • 3.7.6.3.3494 PVR
      • 3.7.6.4.AML PVR
      • 3.7.6.5.Operator PVR
      • 3.7.6.6.SCSI PVR
    • 3.7.7.Mover
      • 3.7.7.1.AIX Asynchronous I/O
      • 3.7.7.2.Tape Devices
        • 3.7.7.2.1.AIX
        • 3.7.7.2.2.Solaris
        • 3.7.7.2.3.IRIX
        • 3.7.7.2.4.Linux
      • 3.7.7.3.Disk Devices
      • 3.7.7.4.Performance
    • 3.7.8.Logging Service
    • 3.7.9.Startup Daemon
    • 3.7.10.Storage System Management
  • 3.8.Storage Subsystem Considerations
  • 3.9.Storage Policy Considerations
    • 3.9.1.Migration Policy
      • 3.9.1.1.Migration Policy for Disk
      • 3.9.1.2.Migration Policy for Tape
    • 3.9.2.Purge Policy
    • 3.9.3.Accounting Policy and Validation
    • 3.9.4.Security Policy
      • 3.9.4.1.Client API
      • 3.9.4.2.FTP/PFTP
      • 3.9.4.3.XFS
      • 3.9.4.4.Name Space
      • 3.9.4.5.Security Audit
    • 3.9.5.Logging Policy
    • 3.9.6.Location Policy
    • 3.9.7.Gatekeeping
  • 3.10.Storage Characteristics Considerations
    • 3.10.1.Storage Class
      • 3.10.1.1.Media Block Size Selection
      • 3.10.1.2.Virtual Volume Block Size Selection (disk)
      • 3.10.1.3.Virtual Volume Block Size Selection (tape)
      • 3.10.1.4.Stripe Width Selection
      • 3.10.1.5.Blocks Between Tape Marks Selection (tape only)
      • 3.10.1.6.Minimum Storage Segment Size Selection (disk only)
      • 3.10.1.7.Maximum Storage Segment Size Selection (disk only)
      • 3.10.1.8.Maximum VVs to Write (tape only)
      • 3.10.1.9.Average Number of Storage Segments (disk only)
      • 3.10.1.10.PV Estimated Size / PV Size Selection
      • 3.10.1.11.Optimum Access Size Selection
      • 3.10.1.12.Some Recommended Parameter Values for Supported Storage Media
        • 3.10.1.12.1.Disk Media Parameters
        • 3.10.1.12.2.Tape Media Parameters
    • 3.10.2.Storage Hierarchy
    • 3.10.3.Class of Service
      • 3.10.3.1.Selecting Minimum File Size
      • 3.10.3.2.Selecting Maximum File Size
      • 3.10.3.3.Selecting Stage Code
      • 3.10.3.4.Selecting Optimum Access Size
      • 3.10.3.5.Selecting Average Latency
      • 3.10.3.6.Selecting Transfer Rate
      • 3.10.3.7.StripeLength and StripeWidth Hints
    • 3.10.4.File Families
  • 3.11.HPSS Performance Considerations
    • 3.11.1.DB2
    • 3.11.2.Bypassing Potential Bottlenecks
    • 3.11.3.Configuration
    • 3.11.4.FTP/PFTP
    • 3.11.5.Client API
    • 3.11.6.Core Server
    • 3.11.7.Location Server
    • 3.11.8.Logging
    • 3.11.9.Cross Realm Trust
    • 3.11.10.Gatekeeping
    • 3.11.11.XFS
    • 3.11.12.HPSS VFS Interface
  • 3.12.HPSS Metadata Backup Considerations
  • 3.13.HPSS Security Considerations
• Chapter 4. System Preparation
  • 4.1.General Setup
  • 4.2.Setup Filesystems
    • 4.2.1.DB2 Filesystem
    • 4.2.2.HPSS Filesystem
  • 4.3.Setup Tape Libraries
    • 4.3.1.Special LTO Considerations
    • 4.3.2.IBM 3584
    • 4.3.3.3494
    • 4.3.4.STK
    • 4.3.5.AML
  • 4.4.Verify Tape Drives
    • 4.4.1.AIX
    • 4.4.2.Solaris
    • 4.4.3.IRIX
    • 4.4.4.Linux
  • 4.5.Setup Disk Drives
    • 4.5.1.AIX
    • 4.5.2.Linux
    • 4.5.3.IRIX
  • 4.6.Setup Network Parameters
    • 4.6.1.HPSS.conf Configuration File
    • 4.6.2.SP/x Switch Device Buffer Driver Buffer Pools
• Chapter 5. HPSS Installation and Infrastructure Configuration
  • 5.1.Prepare for Installation
    • 5.1.1.Distribution Media
    • 5.1.2.Software Installation Packages
    • 5.1.3.Create Owner Account for HPSS Files
    • 5.1.4.Installation Target Directory Preparation
  • 5.2.Install Prerequisite Software
    • 5.2.1.Install Java
    • 5.2.2.Install MIT Kerberos (If Using Kerberos Authentication)
    • 5.2.3.Install LDAP (If Using LDAP Authorization)
    • 5.2.4.Install Prerequisite Software for XFS HDM
  • 5.3.Install HPSS/DB2 and Configure HPSS Infrastructure
    • 5.3.1.Install and Configure HPSS - Root Subsystem Machine
      • 5.3.1.1.Pre-Installation Configuration
      • 5.3.1.2.Install HPSS Documentation and DB2 Software
      • 5.3.1.3.Set Up DB2 Permanent License
      • 5.3.1.4.Configure HPSS Security Services
        • 5.3.1.4.1.Configure UNIX Authentication and UNIX Authorization
        • 5.3.1.4.2.Configure Kerberos Authentication and UNIX Authorization
        • 5.3.1.4.3.Configure Kerberos Authentication and LDAP Authorization
      • 5.3.1.5.Configure DB2 Services
        • 5.3.1.5.1.Remote DB2 Client Access & Fileset Creation/Deletion
      • 5.3.1.6.Configure Other Services
      • 5.3.1.7.Create Configuration Bundle
    • 5.3.2.Install and Configure HPSS – Secondary Subsystem Machine
      • 5.3.2.1.Pre-Installation Configuration
      • 5.3.2.2.Install HPSS Documentation and DB2 Software on a subsystem
      • 5.3.2.3.Set Up DB2 Permanent License
      • 5.3.2.4.Install Configuration Bundle
      • 5.3.2.5.Configure HPSS Security Services
      • 5.3.2.6.Configure DB2 Services
      • 5.3.2.7.Configure Other Services
    • 5.3.3.Install and Configure HPSS – Mover/Client Machine
      • 5.3.3.1.Install Mover/Client source code
      • 5.3.3.2.Install Configuration Bundle
      • 5.3.3.3.Create /var/hpss subdirectories
      • 5.3.3.4.Modify Kerberos Configuration File, If Necessary
      • 5.3.3.5.Check Time and IP Address
  • 5.4.Post Installation Procedures
  • 5.5.HPSS Documentation & Manual Page Setup
    • 5.5.1.Documentation and SSM Help Package
    • 5.5.2.Manual Page Setup
  • 5.6.Define HPSS Environment Variables
  • 5.7.Tune DB2
  • 5.8.Install and Build HPSS Source Code
    • 5.8.1.Construct and Build the HPSS Base Source Tree
      • 5.8.1.1.Construct the HPSS Source Tree
      • 5.8.1.2.Build the HPSS Base Source Tree
      • 5.8.1.3.Generate and Bind the DB2 Helper Program
    • 5.8.2.Construct and Build the HPSS Mover/Client Source Tree
      • 5.8.2.1.Construct the HPSS Mover/Client Source Tree
      • 5.8.2.2.Build the HPSS Mover/Client Source Tree
    • 5.8.3.Construct and Build the HPSS HDM Source Tree
      • 5.8.3.1.Construct the HPSS HDM Source Tree
      • 5.8.3.2.Build the HPSS HDM Source Tree
  • 5.9.Supporting Both Unix and Kerberos Authentication for SSM
• Chapter 6. Upgrading to HPSS Release 6.2
  • 6.1.Special Instructions for Upgrading to HPSS 6.2.2
  • 6.2.Planning for the HPSS 6.2 Upgrade
    • 6.2.1.Metadata changes in HPSS 6.2
    • 6.2.2.Upgrade Requirements and Limitations
    • 6.2.3.New Authentication and Authorization Mechanisms
      • 6.2.3.1.Authentication Mechanisms
      • 6.2.3.2.Authorization Mechanisms
    • 6.2.4.New HPSS 6.2 System Files
    • 6.2.5.Testing the Metadata Conversion
    • 6.2.6.Estimating the Metadata Conversion Time (for 4.5 upgrades only)
      • 6.2.6.1.Running Time for the Long Running Metadata Conversion Utilities (for 4.5 upgrades only)
    • 6.2.7.Capturing the Metadata Conversion Output
    • 6.2.8.DB2 Configuration and Tuning (for 4.5 upgrades only)
    • 6.2.9.Overview of the Upgrade Utilities
      • 6.2.9.1.HPSS 4.5 and HPSS 5.1 Upgrade Utilities
      • 6.2.9.2.HPSS 4.5 Upgrade Utilities
      • 6.2.9.3.HPSS 5.1 Upgrade Utilities
  • 6.3.HPSS 6.2 Upgrade Procedures
    • 6.3.1.Verify Prerequisites
    • 6.3.2.Acquire Software
    • 6.3.3.Install Authentication and Authorization Mechanisms
    • 6.3.4.Install or Upgrade DB2
    • 6.3.5.Upgrade AIX
    • 6.3.6.Install or Upgrade Java
    • 6.3.7.Save Current HPSS Code and Configuration Files
    • 6.3.8.Prepare HPSS 6.2 Code
      • 6.3.8.1.Install HPSS 6.2 Distribution Image
      • 6.3.8.2. Compile HPSS 6.2 Source Code (if necessary)
      • 6.3.8.3.Disable Binaries, temporarily
    • 6.3.9.Set Environment Variables
    • 6.3.10.Setup Authentication and Authorization
    • 6.3.11.Pre-Conversion System Check
    • 6.3.12.Take a full backup of SFS or DB2
    • 6.3.13.Upgrade from HPSS 4.5 to HPSS 6.2
      • 6.3.13.1.Prepare for the Conversion
      • 6.3.13.2.Run db_convert_collect_info to Collect Metadata Information
      • 6.3.13.3.Convert Configuration Metadata
      • 6.3.13.4.Convert Subsystem Metadata
      • 6.3.13.5.Run the Long Running Utilities
      • 6.3.13.6.Create Core Server ACLs
      • 6.3.13.7.Terminate the Scripting Session
      • 6.3.13.8.Modify Permissions on Devices
    • 6.3.14.Verify HPSS 4.5 Conversion Results
      • 6.3.14.1.Capture Session Output
      • 6.3.14.2.Run db_convert_size_check
      • 6.3.14.3.Run db_convert_ns_check
      • 6.3.14.4.Run db_convert_address_check
      • 6.3.14.5.Terminate Scripting Session
    • 6.3.15.Upgrade from HPSS 5.1 to HPSS 6.2
    • 6.3.16.Enable DB2 Backup
    • 6.3.17.Perform the DCE Export: hpss_dce_export
    • 6.3.18.Perform the Unix, LDAP or Kerberos Import
    • 6.3.19.Prepare the 6.2 System
      • 6.3.19.1.Tune DB2 for normal operations
      • 6.3.19.2.Modify Accounting, if applicable
      • 6.3.19.3.Update FTP Configuration Files
      • 6.3.19.4.Populate the HPSS.conf files
      • 6.3.19.5.Copy the rc.hpss Script to /etc
      • 6.3.19.6.Run the bind Script
      • 6.3.19.7.Create Default Server Security ACLs
      • 6.3.19.8.Create SSM User Ids
      • 6.3.19.9.Create Location Server Endpoints
      • 6.3.19.10.Perform Additional Remote Mover Configuration
    • 6.3.20.Bring up the HPSS 6.2 Servers
      • 6.3.20.1.Invoke the SSM System Manager, Startup Daemon and prerequisite software
      • 6.3.20.2.Update HPSS Configurations
      • 6.3.20.3.Dump Accounting Metadata, if applicable
      • 6.3.20.4.Start HPSS 6.2 Servers
    • 6.3.21.Verify 6.2 System
    • 6.3.22.Clean Up After a 4.5 to 6.2 Upgrade
    • 6.3.23.Clean Up After a 5.1 to 6.2 Upgrade
    • 6.3.24.Revert HPSS 6.2 System to Prior Release
      • 6.3.24.1.Revert the HPSS 6.2 System to Version 4.5
      • 6.3.24.2.Revert the HPSS 6.2 System to Version 5.1
  • 6.4.Metadata Conversion Troubleshooting Procedures
    • 6.4.1.HPSS 4.5 to 6.2 Conversion Utility Errors and Warnings
      • 6.4.1.1.db_convert_collect_info Errors
      • 6.4.1.2.db_config_convert, db_subsys_convert, and db_lr_convert Errors and Warnings
    • 6.4.2.HPSS 5.1 to 6.2 Conversion Utility Errors
      • 6.4.2.1.hpss_md_convert_51 Errors
      • 6.4.2.2.hpss_init_server_acls Errors
  • 6.5.HPSS 4.5 Conversion Utilities Output
    • 6.5.1.Interpreting Output from the 4.5 Conversion Utility
    • 6.5.2.Examples of HPSS 4.5 Conversion Utility Output
      • 6.5.2.1.db_convert_collect_info Output
      • 6.5.2.2.db_config_convert Output
      • 6.5.2.3.db_subsys_convert Output
      • 6.5.2.4.Long Running Conversion Utilities Output
        • Appendix A. Glossary of Terms and Acronyms
        • Appendix B. References
        • Appendix C. Developer Acknowledgments
        • Appendix D. HPSS.conf Configuration File
          • D.1. PFTP Client Stanza
          • D.2. PFTP Client Interfaces Stanza
          • D.3. Multinode Table Stanza
          • D.4. Network Option Stanza
          • D.5. PFTP Daemon Stanza
          • D.6. Transfer Agent Stanza
          • D.7. Stanzas Reserved for Future Use
        • Appendix E. hpss_env_defs.h
        • Appendix F. /var/hpss files