I currently have two EMC Symmetrix systems talking over enrange to each other. The systems are about 200 miles apart. When we tried to do synchronous replication the production boxes were frozen. A few quick calls around suggested that there are distance limitations of around 100 Kilometers for the amount of bandwidth we are trying to do this over (T3). Is this true?
Is it true -- no.
Until recently, Fibre Channel fabrics connected by dark fiber, T3, and/or DWDM (dense wave division multiplexing) could only span metropolitan distances, typically well under 100 miles. Today, new IP SAN technologies have abolished those limits, with demonstrated connectivity of thousands of miles.
Keep in mind however that DR applications such as synchronous data replication are sensitive to latency, which is tied directly to distance. Speed of light propagation dictates about one millisecond of latency for every 100 miles. A thousand mile span between primary and DR sites, for example, would inject roughly 10 milliseconds of latency each way, or 20 milliseconds latency round trip. For synchronous applications, vendors typically recommend a maximum of 150-200 miles between sites as a maximum based on the performance impact incurred. In practice, I have customers who have pushed synchronous data replication more than twice that distance.
Why not use asynchronous data replication. Asynchronous data replication, in contrast, is highly tolerant of latency and can be driven across thousands of miles. It also will significantly reduce your bandwidth requirements versus traditional synchronous solutions.