Dell EMC PowerStore: Clustering and High Availability
Document Type: Technical White Paper
Publication Date: September 2020
Executive Summary
This white paper discusses the redundant hardware and high-availability features of Dell EMC™ PowerStore™. PowerStore features fully redundant hardware and several high-availability features designed to withstand component failures within the system and its environment (e.g., network or power failures). If a component fails, the storage system remains online and continues to serve data. The system can also withstand multiple failures in separate component sets. After an alert, the failed component can be ordered and replaced without impact.
Audience
This document is intended for IT administrators, storage architects, partners, and Dell Technologies™ employees. It also includes individuals evaluating, acquiring, managing, operating, or designing a Dell EMC networked storage environment using PowerStore systems.
1. Introduction
Continuous access to data is critical for modern business. Inaccessible data can impact operations and cause revenue loss. IT administrators ensure that every component in the data center has no single point of failure. This white paper details how to cluster PowerStore systems and describes the high availability features of the PowerStore system, designed for 99.9999% availability.
1.1 PowerStore Product Overview
PowerStore offers enhanced value, flexibility, and simplicity in storage. It utilizes a container-based microservices architecture, advanced storage technologies, and integrated machine learning to unlock data potential. PowerStore is a versatile platform with a performance-centric design, offering scale-up and scale-out capabilities, always-on data reduction, and support for next-generation media. It brings public cloud simplicity to on-premises infrastructure, streamlining operations with an integrated machine learning engine and seamless automation. PowerStore also provides predictive analytics for monitoring, analysis, and troubleshooting. Its adaptability allows hosting specialized workloads directly on the appliance and modernizing infrastructure without disruption, while offering investment protection through flexible payment solutions.
1.2 Terminology
Key terms used with PowerStore include:
- Appliance: A solution containing a base enclosure and any attached expansion enclosures.
- Base enclosure: Contains both Nodes (Node A and Node B) and 25x NVMe drive slots.
- Cluster: Multiple appliances in a single grouping, consisting of one or more appliances. PowerStore T model appliances can be expanded by adding more appliances (up to 4 total).
- Expansion enclosure: Enclosures attached to a base enclosure for additional storage via SAS drives.
- Fibre Channel (FC) protocol: A protocol for Internet Protocol (IP) and Small Computer Systems Interface (SCSI) commands over a Fibre Channel network.
- File system: Storage accessible via file-sharing protocols like SMB or NFS.
- Internet SCSI (iSCSI): Provides access to block-level data storage over network connections.
- Intracluster Management (ICM) Network: An internal network for continuous management connectivity between appliances in a PowerStore cluster.
- Intracluster Data (ICD) Network: An internal network for continuous storage connectivity between appliances in a PowerStore cluster.
- Network-attached storage (NAS) server: A file-level storage server hosting file systems, required for SMB or NFS shares, and VMware NFS datastores and VMware® vSphere® Virtual Volumes™™ (vVols).
- Network File System (NFS): An access protocol for data access from Linux®/UNIX® hosts.
- Node: A storage node providing processing resources for storage operations and servicing I/O between storage and hosts.
- PowerStore Manager: The web-based user interface (UI) for storage management.
- PowerStore T model: Container-based storage system on purpose-built hardware, supporting unified (block and file) or block-optimized workloads.
- PowerStore X model: Container-based storage system within a virtual machine on a VMware hypervisor, supporting block-optimized workloads and direct application deployment.
- REpresentational State Transfer (REST) API: Resources, operations, and attributes for interactive, scripted, and programmatic management of the PowerStore cluster.
- Server Message Block (SMB): A network file sharing protocol (also known as CIFS) used by Microsoft® Windows® environments for file and folder access.
- Snapshot: A point-in-time view of data, allowing file recovery, storage resource restoration, or cloning.
- Storage Policy Based Management (SPBM): Policies controlling storage capabilities for a VM throughout its lifecycle.
- PowerStore Command Line Interface (PSTCLI): An interface for performing tasks via commands.
- Virtual Volumes (vVols): A VMware storage framework for VM data storage, enabling VM-granularity data services with SPBM.
- vSphere API for Array Integration (VAAI): A VMware API offloading storage-related tasks to the storage system.
- vSphere API for Storage Awareness (VASA): A VMware API providing insight into vSphere storage capabilities.
- Volume: A block-level storage device shared via iSCSI or Fibre Channel.
2. Clustering
Every PowerStore appliance is deployed into a PowerStore cluster, with a minimum of one and a maximum of four appliances. Clusters can be configured during initial setup or by adding appliances to an existing cluster. Clusters can also be scaled down by removing appliances. Clustering is supported only on PowerStore T models.
Clustering PowerStore appliances offers several benefits:
- Easy scale-out for CPU, memory, storage capacity, and front-end connectivity.
- Independent scaling of storage and compute resources.
- Centralized management for multi-appliance clusters.
- Automated orchestration for host connectivity.
- Increased resiliency and fault tolerance.
Appliances can be scaled out with different model numbers to create a cluster. Appliances can also scale up with different numbers of expansion enclosures. Appliances within a cluster can utilize different media types (e.g., NVMe SCM drives or NVMe SSD drives). This flexible deployment allows customers to grow clusters without dependence on model number, drive count, or drive type.
Figure 1: Multi-appliance cluster
Illustrates a multi-appliance cluster with PowerStore 1000T, 3000T, 5000T, and 7000T models, each with varying expansion enclosures, connected via a dual-node architecture and clustered for scale-out.
2.1 Requirements
Ensure PowerStore systems are cabled correctly per the Dell EMC PowerStore Quick Start Guide. Nodes in each appliance must communicate via bonded ports. The network for node communication is the Intracluster Management (ICM) and Intracluster Data (ICD) networks, using untagged VLAN network packets with auto-generated IPv6 addresses. All appliances should reside in the same rack or multiple racks within the same data center. If PowerStore appliances span multiple switches, the untagged network (or native VLAN) must be configured on the switch ports and shared across switches. Stretched clusters across different buildings on the same campus are not supported.
Figure 2: Back view of PowerStore appliance
Diagram showing the back of a PowerStore appliance with various ports and components labeled.
2.2 Primary appliance
Each cluster has a primary appliance, selected during initial configuration. A primary appliance can be configured in unified or block-optimized mode during initial configuration.
2.3 Initial configuration
The Initial Configuration Wizard scans for other appliances on the network. If multiple PowerStore T model or PowerStore X model appliances exist, only PowerStore T models are displayed. To add appliances, click 'Select Additional Appliances'. The view shows eligible appliances for the cluster.
Figure 3: Unified storage configuration
Screenshot of the PowerStore Initial Configuration wizard, showing the 'Cluster Details' tab with options for Cluster Name, Required Appliance, Storage Configuration (Unified/Block Optimized), and Additional Appliances.
Figure 3 shows an appliance configured in unified mode. File services run only on the primary appliance in a unified cluster. Ensure 'Storage Configuration' is set to 'Unified' for file services, and 'Block Optimized' for block workloads.
Note: An appliance's configuration type (Unified or Block Optimized) cannot be changed after the cluster is configured.
Figure 4: Multi-appliance selection
Screenshot of the PowerStore Initial Configuration wizard, showing the 'Cluster Configuration' tab where multiple appliances (PowerStore 7000T, 1000T, 5000T) can be selected for a multi-appliance cluster.
General guidelines for creating a multi-appliance cluster:
- The cluster size cannot exceed four appliances.
- Only healthy and uninitialized appliances can be selected.
- Appliances are configured synchronously, starting with the primary.
- Only the primary appliance can be in unified mode.
- Additional appliances are automatically rebooted into a block-optimized configuration.
- Appliances must communicate using untagged VLAN and IPv6 addresses on system bonds.
- System bonds of all appliances should be on the same native VLAN.
2.3.1 Unified cluster
For initial cluster setup, if file services will be used, deploying a unified cluster offers the most future flexibility. As noted in section 2.2, file services run only on the primary appliance in a unified cluster. Other appliances in the cluster are configured in block-optimized mode.
2.3.2 Block-optimized cluster
If file services are not used, a cluster can be configured as block-optimized for maximum block performance. Figure 6 shows an example of a multi-appliance cluster configured as block-optimized, with the primary appliance set to block-optimized. All selected appliances are automatically rebooted into a block-optimized configuration.
Note: A cluster's configuration type (unified or block optimized) cannot be changed after it has been configured.
Figure 5: Reboot non-primary appliances to block optimized configuration
Screenshot of the PowerStore Initial Configuration wizard showing an 'Appliance Reboot Required' dialog, indicating that selected appliances need to reboot to become block-optimized.
Figure 6: Reboot all appliances in a block-optimized cluster
Screenshot of the PowerStore Initial Configuration wizard showing an 'Appliance Reboot Required' dialog for enabling 'Block Optimized' configuration.
2.4 Add appliance
PowerStore allows independent scaling of clusters, scaling up, and scaling out. Add expansion enclosures for capacity, and appliances for compute, memory, and connectivity. Adding an appliance is supported via PowerStore Manager only; REST API or PSTCLI are not supported.
General guidelines for adding appliances to an existing cluster:
- Final cluster size cannot exceed four appliances.
- Only uninitialized appliances can be added.
- Only one appliance can be added at a time.
- The new appliance and the existing cluster must be in a healthy state.
- Extra appliances are automatically rebooted into a block-optimized configuration.
- All appliances must communicate using untagged VLAN and IPv6 addresses on system bonds.
- System bonds of all appliances should be on the same native VLAN.
2.4.1 Selecting the appliance
PowerStore Manager simplifies adding an appliance to an existing cluster. Figure 7 shows adding an appliance. Navigate to the Hardware page and click 'Add'. This displays available unconfigured appliances. Once selected, the appliance reboots into block-optimized configuration.
Figure 7: How to add an appliance
Screenshot of the PowerStore Manager 'Hardware' page, showing the 'APPLIANCES' tab with an 'Add' button and a list of appliances, including 'DemoCluster-appliance-1'.
2.4.2 Configuring IP addresses
After selecting an appliance, additional IP addresses must be added to the cluster if none are available. Figure 8 and Figure 9 illustrate adding IP addresses during the Add appliance workflow. For cluster expansion, three additional management IP addresses and two storage IP addresses are required. Users can click 'Add Network IPs' to supply new IP addresses for management or storage networks.
Figure 8: How to add IPs from the Add Appliance Wizard
Screenshot of the PowerStore 'Add Appliance' wizard, showing the 'Networks' tab where Management Network and Storage Network IPs can be configured.
Figure 9: Adding Management IPs
Screenshot of the PowerStore 'Add Appliance' wizard, showing the 'Management IPs' section with added IP addresses for Core OS Node and Management Cluster.
The alternative method involves going to Settings > Networking > Network IPs. Figure 10 shows this page, where users can reserve a range of IP addresses.
Figure 10: How to add IPs from the Settings Page
Screenshot of the PowerStore Manager 'Settings' page, showing the 'Network IPs' section for both Management and Storage networks, with options to add and reconfigure IPs.
2.4.2.1 Validate network configuration
In the Add Appliance > Summary page, perform a network validation check before starting the Add Appliance job. Warnings or errors may appear. Address warnings before cluster creation; warnings can be bypassed, but errors must be corrected to start the job. Figure 11 shows the Summary page and network validation.
Figure 11: Add appliance network validation
Screenshot of the PowerStore 'Add Appliance' wizard, showing the 'Summary' page with a network validation prompt.
2.4.3 Review the job
After the Add Appliance wizard is complete, a new job is created. View job details by clicking the 'Jobs' icon in the top-right of PowerStore Manager. Figure 12 shows an example of an Add Appliance job running in the background, allowing other tasks to be performed concurrently.
Figure 12: Add appliance job
Screenshot of the PowerStore Manager interface showing the 'RECENT JOBS' section with an 'Add appliance to cluster' job at 40% completion.
2.5 Remove appliance
Clusters can be scaled down by removing appliances. This requires careful planning, including moving all storage resources off the appliance before removal. After removal, the appliance resets to factory settings. The service script `svc_remove_appliance` is used for removal.
Guidelines for using the service script:
- Notify users to avoid creating storage resources or VMs during the operation.
- Run `svc_remove_appliance`, noting storage resources and workloads.
- Review prompts, stop the scheduler, and allow active jobs to complete.
- Migrate storage resources or VMs to another appliance.
- Review prompts to ensure no active workloads remain.
- Confirm removal.
Note: The `svc_remove_appliance` script does not automatically migrate storage resources.
2.6 VMware integration
PowerStore T and X models integrate deeply with VMware vSphere, supporting VAAI, VASA, event notifications, snapshot management, storage containers for vVols, and VM discovery/monitoring. By default, a storage container is created upon appliance initialization. Storage containers present vVol storage to vSphere, which mounts them as vVol datastores for VM storage, accessible by internal or external ESXi nodes. In a multi-appliance cluster, a storage container's capacity spans all appliances. For PowerStore X models, cluster Distributed Resource Scheduler (DRS) is set to partially automated mode; changing this is not supported.
For more information on VMware integration, see Dell EMC PowerStore: Virtualization Integration on Dell.com/StorageResources.
2.7 Resource balancer
PowerStore's intelligent analytical engine aids administrators in making data placement decisions, volume migrations, and storage expansion based on analytics and capacity forecasting.
2.7.1 Initial placement of data
When creating storage resources, the resource balancer offers benefits. In a multi-appliance cluster, it intelligently places newly created volumes on different appliances based on available capacity. Manual placement is also an option.
Figure 13 shows automatic volume placement on a specific appliance. Volumes and volume groups are placed on a single appliance and do not span multiple appliances. They remain on the initial appliance until manually migrated.
Figure 13: Automatic Placement of volumes or manual assignment
Screenshot of the 'Create Volumes' interface, showing options for Name, Quantity, Size, Placement (Auto/Manual), and Associated Volume Group.
2.7.2 Volume groups
When volume groups are created, all members are placed on the same appliance. If existing volumes were placed on different appliances, they must be manually migrated to a single appliance before being placed in a volume group.
2.7.3 Migration overview
After creation, capacity and demand may change. PowerStore supports non-disruptive migration of volumes, volume groups, or vVols to another appliance within the cluster. vVol migration requires powering off the vVol-based virtual machine first. Migration can be performed manually or assisted via PowerStore Manager, REST API, or PSTCLI.
Before starting a migration job, verify:
- Zoning on FC switches between host, source, and destination appliances (if applicable).
- Host connectivity to source and destination appliances via iSCSI or FC.
- Host object in PowerStore Manager shows initiators to source and destination appliances.
- Perform a rescan of the storage object from the host.
After verification, a migration job can start. Storage resources are transferred non-disruptively using asynchronous replication. Associated storage objects (snapshots, clones) are also moved. The host then switches paths to the destination appliance.
Figure 14 shows a multi-appliance cluster (four appliances) where a storage resource is moved from a PowerStore 1000T to a PowerStore 5000T. Host connectivity via iSCSI or FC is shown for some appliances, but not others, impacting migration feasibility.
Figure 14: Volume migration example
Diagram illustrating a multi-appliance cluster with four PowerStore appliances, showing storage network connections and host connectivity for volume migration.
2.7.4 Manual migration
PowerStore supports moving storage resources (volume, volume group, vVol) to another appliance. Ensure paths are visible in PowerStore Manager and a host rescan is performed. Manual migration depends on the storage resource:
- Volume: Storage > Volumes > Select Volume > More Actions > Migrate.
- Volume Group: Storage > Volume Group > Select Volume Group > More Actions > Migrate.
- vVol: Storage > Storage Containers > Select Container > Virtual Volumes tab > Select vVol > Migrate.
Figure 15 shows migrating 'Test-Volume-001' from appliance 2 to appliance 1 in a two-appliance cluster.
Figure 15: Start volume migration job
Screenshot of the 'Migrate Volume' wizard, showing 'Test-Volume-001' selected for migration to 'DemoCluster-appliance-1'.
2.7.5 Assisted migration
The system monitors storage resource utilization and generates migration recommendations when an appliance approaches maximum capacity. Recommendations are based on drive wear, capacity, and health. Administrators can view these via alerts or Migration > Migration Actions in PowerStore Manager. Accepting a recommendation creates an automatic migration session. Snapshots and clones associated with the original storage resources are also migrated.
Figure 16: Assisted Migration Recommendations
Screenshot of an alert indicating 'Appliance2 out of space' and an 'Assisted Migration Recommendation' table listing resources to be migrated.
2.7.6 Migration states
Migration sessions can be paused, resumed, or deleted. Table 1 lists possible migration session states:
| State | Definition |
|---|---|
| Initializing | Migration session starts and stays in this state until the session initialization completes |
| Initialized | Migration session transitions to this state as soon as session initialization completes |
| Synchronizing | Represents that a background copy is in progress |
| Idle | Migration session transitions to this state as soon as initial background copy completes |
| Cutting_Over | A final portion of the copy is performed in this state, and the ownership of the storage resource is transferred to the new appliance. |
| Deleting | Represents a migration session being deleted |
| Completed | Migration session is completed and it is safe to delete the session |
| Pausing | Migration session transitions to this state as soon as pause command is issued |
| Paused | Migration session is Paused, user intervention is required to resume the session |
| System_Paused | Migration session transitions to this state if it encounters any error, user may resume or delete the migration after resolving the error |
| Resuming | Migration session background copy is resumed |
| Failed | Migration session encountered an error |
2.7.7 Capacity forecasting
PowerStore provides details on cluster capacity and helps plan scaling based on business needs. It maintains historical storage consumption statistics to predict future usage. After 15 days, PowerStore Manager displays a forecast of when the system may run out of space. Figure 17 shows an example of capacity forecasting.
Figure 17: Capacity forecasting
Screenshot of PowerStore Manager showing 'OVERVIEW' and 'CAPACITY' sections, including physical capacity, historical usage, and a forecast indicating '5 weeks until cluster is full'.
3. High availability
PowerStore appliances support multi-appliance configurations for redundancy and fault tolerance. Each appliance has two nodes forming an HA pair. PowerStore features fully redundant hardware and high availability features designed to withstand component and environmental failures. If a component fails, the system remains online. Multiple failures in separate component sets can also be tolerated. Failed components can be replaced without impact.
3.1 Hardware redundancy
PowerStore has a dual-node architecture with active/active controller configuration for simultaneous I/O servicing, increasing hardware efficiency. The base enclosure includes these nodes and up to twenty-five 2.5-inch drives.
Components include:
- Power supply
- Battery backup unit (BBU)
- Intel CPU
- Intel® QAT Hardware Offload Engine
- Motherboard
- Cooling fans
- 2x M.2 solid-state drives
- Memory DIMMs
- I/O modules (optional)
Figure 18: Internal components of node (top view)
Diagram showing the internal components of a PowerStore node, including CPU, DIMMs, M.2 drives, BBU, and cooling fans.
3.1.1 Management software
The management software (PowerStore Manager) runs on one node at a time, designated as the primary node. Figure 20 shows the primary node highlighted in the Hardware > Rear View.
If the primary node reboots, panics, or loses management connection, the management software automatically fails over to the peer node. Services may take several minutes to restart. Users logged in during failover might see a connection lost message. After failover, refresh the browser to restore access. Host access to storage is prioritized and available before PowerStore Manager. The management software continues on the new node even if the original node becomes healthy.
Figure 20: Primary node
Screenshot of PowerStore Manager showing the 'Hardware' details for 'DemoCluster-appliance-1', highlighting 'NodeA(primary)' and its details.
3.1.2 System bond
PowerStore ensures no single point of failure. For PowerStore T models, the first two ports of the 4-port card are recommended to be cabled to a top-of-rack switch, forming the system bond. This ensures data services and production data are always available.
The system bond handles various traffic types:
- Management traffic (for PowerStore X models)
- Cluster communication (for PowerStore T models)
- iSCSI host traffic
- Replication traffic
- NAS traffic (for PowerStore T models)
Figure 21: First two ports of the 4-port card (system bond)
Diagram showing the first two ports of a 4-port card connected to two top-of-rack switches, illustrating the system bond configuration.
For PowerStore T models, ports in the system bond can operate in active/active or active/passive mode, depending on Link Aggregation Control Protocol (LACP) configuration on network switches. Enabling LACP ensures optimal resiliency and performance. PowerStore X models handle link aggregation and failover via virtual switches within the VMware hypervisor.
Additional ports can be added for bandwidth or fault tolerance. Administrators can extend iSCSI traffic by mapping additional ports. Replication traffic can be untagged from system bond ports if tagged to other ports. Refer to section 3.3.1 for adding ports for host connectivity and Dell EMC PowerStore: Replication Technologies for replication scaling.
Starting with PowerStoreOS version 1.0.2, Unified PowerStore T model appliances route Intracluster Management (ICM) traffic within the appliance instead of the Top-of-Rack (ToR) switch.
Note: NAS interfaces are permanently fixed to system bond interfaces.
3.2 Dynamic Resiliency Engine (DRE)
Enterprise storage systems require high reliability and protection against data loss. Unlike traditional RAID, DRE uses proprietary algorithms to partition drives into virtual chunks and create redundancy extents across drives. This improves performance and allows scaling as more drives are added. Data can be spread across any number of drives, and re-balancing occurs automatically as new drives are added.
Figure 22: Data placement in PowerStore
Diagram illustrating data placement in PowerStore, showing virtual chunks distributed across drives for Node 1 and Node 2.
3.2.1 Overview
PowerStore implements proprietary algorithms where each drive is partitioned into multiple virtual chunks, creating redundancy extents across several drives. It automatically consumes drives within an appliance, creating redundancy using all drives. This enhances performance and allows scaling as more drives are added. Data written to a volume can be spread across any number of drives within an appliance, with automatic re-balancing as new drives are added.
3.2.2 Distributed Sparing
Unlike traditional RAID, PowerStore does not require dedicated spare drives. Spare space is distributed across the appliance. A small chunk of space from each drive is reserved for sparing in case of drive failure. One drive worth of spare space is reserved for every 25 drives.
When a drive fails, only the portion with data written is rebuilt. This efficiently manages spare capacity and shortens rebuild time.
Figure 23: Distributed Sparing
Diagram showing distributed sparing across multiple drives, with 'Spare' sections indicated.
3.2.3 Resiliency Sets
PowerStore implements resiliency sets to improve reliability while minimizing spare overhead. Multiple failure domains (resiliency sets) increase system reliability, allowing tolerance for drive failures within each set, even if they occur simultaneously.
The appliance can tolerate multiple drive failures within the same resiliency set if failures occur at different times. Resiliency sets can have up to 25 drives, dynamically increasing with added drives. For example, a 26th drive splits the resiliency set into two. Resiliency sets can span physical enclosures and accommodate mixed drive sizes.
Key Benefits:
- Enterprise Class Availability
- Faster rebuild times with distributed sparing.
- Rebuild smaller chunks of the drive simultaneously to multiple drives.
Figure 25: Parallel rebuild of single drive to distributed spare space
Diagram illustrating parallel rebuild of data chunks from a failed drive to distributed spare space.
Intelligent Infrastructure:
- Automatically allocates unused user space to replenish spare space for multiple failures. DRE dynamically transfers unused capacity to replenish spare capacity if sufficient unused capacity is available.
- Intelligently varies rebuild speed based on incoming IO traffic while maintaining availability. PowerStore uses machine learning to adjust rebuild rates, prioritizing host IO during drive failures for optimal performance and reliability.
Figure 26: Rebuild data chunks to spare space after drive failure and replenish spare space with unused user space
Diagram illustrating data chunk rebuilding and spare space replenishment.
Flexible Configurations:
- Lower TCO by expanding storage with single drives. Appliances can have 6 to 96 drives, allowing non-disruptive capacity expansion.
- Flexible options for adding different drive sizes based on storage needs. PowerStore algorithms optimize redundancy extent distribution across drives.
Figure 27: Single Drive Expansion
Diagram illustrating single drive expansion in PowerStore.
3.3 Block storage
PowerStore's active/active thinly provisioned mapping subsystem commits IOs from any path, ensuring consistency without redirection. This reduces time to service IO requests when paths become unavailable, contrasting with architectures that use redirection and single-node locking, which can cause long trespass times and data unavailability during failover.
PowerStore uses a dual-node, shared storage architecture with Asymmetric Logical Unit Access (ALUA) for host access. IO requests are committed locally and consistently with the peer node. IO is typically sent via an active/optimized path; however, if all active/optimized paths fail, IO is processed on the local node via active/non-optimized paths without sending data to the peer node. Data is written to the shared NVRAM write cache, where it is deduplicated and compressed before being written to drives.
ALUA multipathing ensures high availability. The symmetric thin provisioning architecture simplifies complexity and overhead. Multipathing software like Dell EMC PowerPath™ must be installed on the host. Configure multipathing software to use optimized paths first, then non-optimized paths. Use separate network interface cards (NICs) or Fibre Channel host bus adapters (HBAs) on the host to avoid single points of failure.
Physical ports must match on both Nodes for host access during failover. Connect the same port number on both Nodes to multiple switches for multi-pathing and redundancy.
3.3.1 iSCSI configuration
iSCSI interfaces are deployed in mirrored pairs to both PowerStore nodes, as they do not fail over. This ensures continuous host access to block-level storage if a node becomes unavailable. iSCSI interfaces can be created during the Initial Configuration Wizard (ICW) or after cluster creation. For robust HA, create additional interfaces on other ports after cluster creation.
If iSCSI is enabled during ICW, enter iSCSI information on the Storage Network page (Figure 28). IP addresses are assigned to a virtual bonded interface linked to the first two ports on the 4-port card of nodes A and B. To remove single points of failure at the host and switch level, ensure the first two ports of the 4-port card are cabled to switches (Figure 29).
Figure 28: Storage network
Screenshot of the PowerStore Initial Configuration wizard, showing the 'Networks' tab with options for Management Network and Storage Network, including iSCSI Networking configuration.
Figure 29: First two ports of the 4-port card cabled to top-of-rack switches
Diagram showing the first two ports of a 4-port card connected to two physical Ethernet switches.
3.3.2 Fibre Channel configuration
To achieve high availability with Fibre Channel (FC), configure at least one connection to each node. This enables continuous host access to block-level storage if a node becomes unavailable. Zoning must be configured on the switch for host-appliance communication. For multi-appliance clusters using FC, ensure each appliance has zones configured for host communication. Create a zone for each host HBA port to each node FC port on each appliance. For robust HA, more FC ports can be zoned for additional paths.
Port World Wide Names (WWNs) can be found in PowerStore Manager on the Hardware > Ports page (Figure 32) and Hardware > Rear view page (Figure 33). For host configuration details, see the Dell EMC PowerStore Host Configuration Guide on the PowerStore Info Hub.
Figure 32: PowerStore Manager hardware ports page
Screenshot of PowerStore Manager showing the 'Hardware' page with 'PORTS' tab, listing appliances and their port details, including WWN and speed.
Figure 33: PowerStore Manager hardware back view
Screenshot of PowerStore Manager showing the 'Hardware' details for 'DemoCluster-appliance-2', highlighting 'FEPort3' and its details.
3.3.3 Block example
Designing a highly available infrastructure requires redundant components, removing single points of failure at the host and switch level to avoid data unavailability. Figure 34 shows a highly available configuration for a PowerStore T model system with no single point of failure. Refer to PowerStore Network Planning Guide and PowerStore Network Configuration for PowerSwitch Series Guide for cabling and network configuration details.
Figure 34: Highly available block configuration
Diagram illustrating a highly available block configuration for a PowerStore T model system, showing network connections, switches, and appliances.
3.4 File storage
Before sharing file-level resources from a PowerStore T model system, NAS server interfaces are automatically created on the first two bonded ports of the 4-port card. NAS interfaces cannot be assigned to specific ports. An administrator creates a NAS server for SMB, NFS, FTP, or SFTP access to file systems. New NAS servers are assigned on a round-robin basis across available nodes. The preferred node indicates the node where the NAS server should run. Once provisioned, the preferred node never changes. The current node shows where the NAS server is running. Moving the NAS server to a different node can be used for load balancing. When a NAS server moves, all its file systems move with it.
Figure 35 shows the current and preferred node columns in PowerStore Manager.
Figure 35: Current Node and Preferred Node
Screenshot of PowerStore Manager showing the 'NAS Servers' list with 'Current Node' and 'Preferred Node' columns.
During a PowerStore node failure, NAS servers automatically fail over to the surviving node within 30 seconds to avoid host timeouts. After the failed node recovers, a manual process is required to fail back NAS servers for a balanced configuration.
NAS servers are automatically moved to the peer node and back during upgrades. After the upgrade, they return to their original assigned node.
3.5 VMware integration
The following subsections describe HA capabilities with the PowerStore X model. For PowerStore integration with VMware details, reference Dell EMC PowerStore: Virtualization Integration on Dell.com/StorageResources.
3.5.1 Controller VMs
The PowerStore X model appliance is fully redundant with controller VMs that provide data services. Each controller VM is deployed into private datastores representing the physical M.2 device. Since these are not shared datastores, if a node fails, the controller VM does not restart on another host. However, the peer node running the other controller VM continues to service I/O. The two controller VMs act as the HA pair. Controller VMs should not be moved or have their settings changed.
3.5.2 vSphere HA
When leveraging virtual machines on PowerStore T or X models, enabling vSphere HA on the cluster is recommended. In case of power loss for a node or host, virtual machines are restarted on a surviving host or node.
3.5.3 vCenter connection
Both PowerStore T and X models integrate with VMware environments via vCenter. For PowerStore T models, a vCenter connection is optional after cluster setup. For PowerStore X models, a vCenter connection is required during the initial configuration wizard. If a vCenter connection is lost, management tasks via vCenter may be temporarily unavailable, but I/O continues to be processed for all storage objects.
3.6 Replication
To protect against system or data-center outages, replication to a remote site is recommended for planned maintenance, power outages, or natural disasters. PowerStore supports replication for disaster recovery. For setup and usage details, see Dell EMC PowerStore: Replication Technologies on Dell.com/StorageResources.
3.7 Platform high availability
Each storage resource is assigned to node A or B for load balancing and redundancy. Each node runs PowerStore OS components. If a node becomes unavailable, its resources (storage and containers) automatically fail over to the surviving node. Failover time depends on system utilization and resource count. The peer node assumes ownership and continues servicing I/O.
Failovers occur due to:
- Node reboot (system or user).
- Hardware or software fault.
- Service mode (automatic or manual via service script).
- Powered off (user).
Note: Manually placing a node into service mode is only available via a service script.
While a node is unavailable, the peer node services all resources. After the node is back online or the fault is corrected, block-storage resources automatically fail back. File-storage resources must be failed back manually.
During code upgrades, both nodes reboot in a coordinated manner, failing over resources to the peer node. After the peer returns online, resources fail back to their original owner. A pre-upgrade health check is recommended for a smooth upgrade.
3.8 Cluster high availability
PowerStore appliances use a Quorum management feature for cluster resource management and split-brain handling, ensuring data consistency during cluster changes (adding/removing appliances, failures). Management services remain available if quorum is met during an appliance failure.
3.8.1 Cluster quorum
A quorum is defined as N/2+1 appliances in active communication. If quorum is lost, management operations are temporarily unavailable, but data continues to be serviced if available. Figure 36 shows a two-appliance cluster servicing I/O via FC, but with lost management access due to no quorum between Appliance 1 and Appliance 2. Host I/O is unaffected, but management services like PowerStore Manager, scheduled snapshots, and replication syncing are temporarily suspended.
Figure 36: Two-appliance cluster with no quorum
Diagram showing two appliances connected via IP and FC networks, with a red 'X' indicating no quorum.
Figure 37 shows a three-appliance cluster where Appliance 3 crashes. Quorum is met because Appliances 1 and 2 are active. I/O is accessible from Appliances 1 and 2, but inaccessible for Appliance 3.
Figure 37: Three-appliance cluster with quorum
Diagram showing three appliances connected via IP and FC networks, with a red 'X' indicating Appliance 3 has failed.
3.8.2 Cluster IP
The cluster IP address is a required field for initial cluster configuration, providing highly available access to PowerStore Manager. It is typically on the primary node of the primary appliance. In case of a dual-node failure on the primary appliance, the cluster IP fails over to another appliance if quorum exists. Regaining access to PowerStore Manager may take several minutes after cluster IP failover.
3.8.3 Global storage IP
The Global Storage IP (GSIP) address is an optional field for cluster configuration. It is a global, floating storage-discovery IP. iSCSI hosts require only one GSIP to discover all storage paths. Otherwise, iSCSI hosts need a list of storage IPs for redundancy.
4. Conclusion
Designing infrastructure with high availability ensures continuous access to business-critical data. Unavailable data impacts operations, productivity, and revenue. PowerStore systems offer full redundancy across hardware and software components, enabling 99.9999% availability. By utilizing PowerStore's clustering and high availability features, organizations can minimize the risk of data unavailability.
A. Technical support and resources
Dell.com/support offers services and support. Storage technical documents and videos provide expertise for Dell EMC storage platforms. The PowerStore Info Hub provides detailed documentation for installing, configuring, and managing Dell EMC PowerStore systems.
1 Based on Dell Technologies specification for Dell EMC PowerStore, April 2020. Actual system availability may vary.
