Aspects of Remote Replication Performance in Symmetrix DMX and Symmetrix V-Max Series with Enginuity

V-Max Baseline

  • FC Performance improved by up to 36%
    • Write latency can be as low as 1 ms for small block size and zero distance
    • Single round trip continue to deliver significant savings in long distance
    • Maximal FC IO rate is 5000 IOPS
    • Maximal throughput up to 260 MB/s per RA
  • GigE Performance
    • Write latency can be as low as 1.25ms
    • Maximal IO is 4000 IOPS
    • Maximal throughput is about 90 MB/s per link (1 Gb link limit)
  • Note that these are maximum utilization limits. Don’t plan to reach director limits as part of the design.

SRDF Copy over FC Improved by 36%

  • Up to 520 MB/s from 380 MB/s with two FC links

FC Protocol

  • Two-phase FCP protocol for every write
    • Send command, ready receive
    • Send data, status acknowledge
  • This is fine in short distance but becomes prohibitive in longer distances. This could make FC worse than GigE if nothing is done about it.
  • Sound-round trip was introduced in DMX3/4 in 5772
    • Send command, send data unsolicited
    • Target buffers the data, send status
  • On average can save 1ms per 100km of distance
  • New to Symmetrix V-Max, improved buffering enables Single Round Trip for any I/O size (only done on small I/O for DMX)
  • This mode is configured as on and will start working in this mode when the latency reaches a certain threshold.


  • Support for 250 SRDF groups per array
  • Up to 64 groups on a single port
  • Support for up to 32 SRDF ports in a fully loaded array
  • Concurrent writes in OS can run in parallel on all SRDF ports
  • qoS can set priority among competing SRDF/S volumes
  • DCP (dynamic cache partitioning) and SRDF/A cache limits can be used to set priorities among competing SRDF/A groups
  • SRDF performance scales linearly. For example, four RA directors can do about four times as much as one RA director in both DMX and V-Max
  • Same response time between 1, 32, 64, 250 groups (4KB write cache hit, 8 FC links, SRDF/S)

SRDF/A Architecture

  • Write-folding to repeat writes into a cycle are sent once
  • Response times are at 0.3ms. V-Max will allow you to go farther in terms of total IOPS (50k IOPS vs 42k IOPS) before response times begin to rapidly increase
  • Small impact even with smaller cycle times (30 seconds, 5 seconds, 2 seconds), but this is dependent on link bandwidth

SRDF/A Performance Considerations

  • Correct planning is key, cache, bandwidth, future workload increase, resiliency features (DSE, transmit idle)
  • Long distance networks (see SRDF/A Network Performance Analysis and Troubleshooting)

Three-site Solutions

  • Concurrent SRDF
    • R11 — S –> R2
    • R11 — A –> R2
  • Cascaded SRDF
    • R1 — S –> R21 — A –> R2
  • SRDF/Star for differential from B to C in the concurrent model
  • SRDF/EDP (Extended Distance Protection) through Cascaded SRDF
    • R1 — S –> R21 — A –> R2
    • Production site > Pass-thru Site > Out-of-region Site
    • Here the R21 is diskless, and the site is a pass-thru site
    • No data loss at an out-of-region site at a lower cost, if only the production site goes down.
    • DSE volumes can be put behind the R21
    • The middle site needs to be V-Max
    • SRDF/S has the best response time.
    • SRDF/A gets higher response times as the cycle times shorten.

Virtual Provisioning with SRDF

  • Storage is allocated in extents of 768KB
  • SRDF operations continue to be supported with the granularity of blocks (512 bytes)
  • SRDF supports connecting a VP device to another VP device (thin to thin only)
  • Indirection penalty with every write
    • Noticable at high I/O rates
    • Consequently more SRDF CPU power is needed to achieve equivalent performance
    • Maximal 2500 IOPS (roughly, still doing testing and engineering)

Control Operations Speed-up

  • New architecture for SRDF meta-data in V-Maxx
    • Dynamically allocated
    • Continuous in memory
    • Support for bulk operations
    • Therefore all control operations can complete faster
  • Increased scalability
    • Commands are broken into multiple extents, executed in parallel
  • As a result:
    • Faster dynamic SRDF operations
    • Faster failover and failback operations
    • Faster SRDF re-synchronization
    • Therefore lower RTO
  • Speed ups
    • Need V-Max to V-Max, 5874
    • Create a new srdf pair (symrdf createpair -invalidate) from 54 to 7 seconds
    • Full sRDF establish (symrdf establish -full) from 53 to 4 seconds
    • Incremental establish after split (symrdf establish) from 6 to 3 seconds
    • Failback (symrdf failback) from 47 to 19 seconds


Leave a Reply

Please log in using one of these methods to post your comment: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s