Azure Cloud Storage has more than 58 regions spanning 140 countries (you can find a map of them here). While these wide-ranging storage locations provide geographically distributed businesses with many options, data transfer latency within and across Azure regions can be extreme. This translates to slow and unpredictable transfers and failed transfers.
But the distance between one region to another isn’t the only factor contributing to latency. How you replicate data across Azure regions and how your network bandwidth is used matters, too.
Azure offers some solutions to help reduce latency such as Accelerated Networking. But depending on your industry and use case, these solutions are expensive and may not actually solve or overcome the latency issues inherent in transferring or synchronizing files across regions. Moreover, Azure’s replication architecture is point-to-point. Distributing large updates (GB-sized files) across hundreds of web/app servers via point-to-point replication will take a very long time — even with accelerated networking.
If latency between Azure regions is a big deal for your business, Resilio Connect could be the solution. Resilio’s engineers have tested and validated 100+ Gbps per server replication speeds within and across Azure cloud regions. Cloud scenarios tested include transferring a 1 TB data set between Microsoft Azure regions in 90 seconds and transferring a 500 GB data set in Google Cloud from London to Australia in 50 seconds.
In this guide, we’ll discuss how to measure and reduce Azure latency, including how our WAN-optimized, P2P (peer-to-peer) replication solution — Resilio Connect — provides fast, reliable data transfer and reduces latency for transfers across Azure regions.
More specifically, we’ll discuss how Resilio:
- Enables every device in your environment to take part in replication (n-way mesh) to achieve blazing fast, cross-region transfer speeds.
- Efficiently transfers data across any network, including poor-quality networks and areas with little network coverage, thanks to a proprietary WAN acceleration protocol.
- Simplifies management via a centralized console where you can manage all devices and cloud environments running Resilio.
- Works with any device, operating system, and cloud storage service.
- Keeps your data secure with built-in security features like AES 256-bit encryption.
Organizations such as Skywalker Sound, Cisco, Mixhits Radio, 2K Games, and Deutsche Aircraft use Resilio Connect to sync data across cloud servers and on-premises devices for remote work, web/app server sync, software update distribution, disaster recovery, and more. To learn more about how Resilio Connect can reduce latency and enhance replication speeds, schedule a demo.
How to Measure and Monitor Azure Region Latency
Latency is the time it takes a data packet to travel from one point to another. Azure cross-region latency can be impacted by the distance between your Azure regions and Azure availability zones, network quality, network congestion, and data transfer methods.
The distance between Azure regions has a large impact on latency. As you might expect, latency is greater between geographically dispersed regions, such as Qatar central, Korea central, India, and Southeast Asia, than it is for regions that are close to each other, such as East US 2 (e.g., Georgia), North Central US (e.g., Illinois), and South Central US (e.g., Texas).
To monitor and measure average latency, use one of these three tools to run speed tests on your Azure storage endpoints:
- Azurespeed.com: Azurespeed.com is a free, Azure-specific tool for testing latency between Azure cloud regions.
- Ping: Ping is a simple latency test utility available on all operating systems that measures the RTT (Round Trip Time) between a device and a target destination.
- TWAMP: TWAMP tests latency bidirectionally by establishing a connection between the source device and the target with TCP and using UDP packets to monitor the latency.
How to Reduce Latency Issues between Azure Regions
If high latency is disrupting your applications and user/employee experience, try these methods for reducing Azure cross-region latency.
Use Proximity Placement Groups
Azure’s storage regions range from West Europe (e.g., Netherlands), South India (e.g., Chennai), North Europe (e.g., Ireland), Brazil South (e.g., Sao Paulo), Southeast Asia (e.g., Singapore), and more. No matter how good network performance is on any region in the Azure network, the further your data must travel, the more latency you’ll incur.
Proximity placement is a strategy that involves reducing latency by reducing the distance your data must travel by:
- Keeping your data in Azure regions that are close to each other.
- Keeping your data in Azure data centers that are near your end-users — i.e., the same country or region or in nearby regions.
While these may not be possible for all businesses depending on legal regulations and costs, the key is to research your options and aim to reduce traveling distance as much as possible.
How Resilio Connect Enables Proximity Placement
Resilio Connect provides granular control over how synchronization occurs in your environment. It enables you to control which files are stored at which endpoints.
In other words, you can store frequently accessed files at the cloud or on-premises endpoints that are closest to your end-users, so you can minimize distance and latency.
Resilio’s multidirectional synchronization capabilities (more on this later) also enable you to sync data across all of your endpoints simultaneously. This enables you to keep data synchronized across all of your Azure cloud regions, so end-users can always access data from the endpoint closest to them.
Efficient Routing of Network Traffic
Managing how data travels across your network is another effective way to reduce latency.
Efficient routing involves minimizing distance and congestion by:
- Transferring data along the shortest path to the target endpoint.
- Controlling network traffic to minimize congestion.
When an application needs access to data, it should be able to retrieve that data from the endpoint closest to it. To transfer data along the shortest path between Azure cloud regions, set up Accelerated Networking in Azure (which you can learn how to do here).
An easy way to minimize network traffic when transferring data across Azure regions is to enable Receive Side Scaling. Per their documentation, “Receive Side Scaling is a network driver technology that enables the efficient distribution of network receive processing across multiple CPUs in multiprocessor systems.”
Basically, it helps control and spread traffic across your network to minimize congestion on any single network by reducing processing delays, spinlock overhead, reloading of caches, and more.
How Resilio Connect Efficiently Routes Network Traffic
Resilio Connect efficiently routes network traffic and reduces latency through two features (more information on both will be provided in the next section):
1. P2P replication architecture
With Resilio Connect, Resilio agents are installed on every endpoint (servers, physical locations, cloud servers, etc.) and data is replicated via a peer-to-peer transfer architecture.
Unlike most replication solutions that use point-to-point transfer, P2P replication enables every device in your environment to share data with and receive data from every other device.
Because every device can communicate with every other device, Resilio can provide the benefits of:
- Accelerated networking: Your endpoints/Azure cloud regions can receive data from the regions/endpoints closest to them, thus minimizing distance and latency.
- Network traffic distribution: Every endpoint with a Resilio agent installed on it can take part in replication simultaneously. When replicating data across your environment, you can also distribute the network workload across your environment.
2. WAN acceleration technology
Resilio Connect utilizes a proprietary WAN acceleration protocol known as Zero Gravity Transport™ (ZGT) to enhance transfer over any network by efficiently managing network traffic. For example, ZGT uses a congestion control algorithm that constantly probes the RTT of a network to identify and maintain an ideal data packet send rate.
Feel free to read Resilio’s WAN Optimization White Paper.
More Ways to Optimize Cross-Region Transfer in Azure with Resilio
Resilio Connect is a real-time file synchronization software that provides the fastest data transfer speeds in the industry, enabling users to reduce network latency, improve application performance, and optimize workflows.
Below, we provide a deeper exploration of how Resilio reduces latency and speeds up access to data.
Sync Azure Regions Quickly with P2P Replication & WAN Optimization
Resilio Connect reduces latency by increasing the speed at which data is transferred. The two primary features driving Resilio’s blazing fast replication speeds are Resilio’s P2P replication architecture and proprietary WAN optimization protocol.
High-Performance P2P Replication for Fast, Scalable, Multidirectional Replication
Most replication solutions use a point-to-point replication architecture. In point-to-point replication, data is transferred in one of two models:
1. Hub-and-spoke replication: In hub-and-spoke replication, one device is designated as a hub server and the other devices are remote servers. The remote servers can’t communicate with each other; all file transfers must go through the hub server. So, if Device 1 wants to share data with the other devices in your environment, it must first transfer the data to the hub server, which will then share the data with the other devices one by one.
2. Follow-the-sun replication: In follow-the-sun replication, data is transferred from one device to another sequentially. So Device 1 shares data with Device 2, which then shares the data with Device 3, and so forth.
Both of these replication models are slowed by the fact that data can only transfer between two devices at a time. Replication speed is also limited by the network on each device.
In Resilio’s P2P replication model, every device in your environment can communicate with every other device. Resilio also utilizes a process known as file chunking, which breaks files down into multiple chunks that can transfer independently of each other. The combination of P2P transfer and file chunking enables all of your devices to take part in synchronization simultaneously.
Imagine you want to sync a file across five devices. Resilio can split that file into five separate chunks. Device 1 can share the first chunk with Device 2. As soon as it receives the first chunk, Device 2 can share it with Device 3. With every device working together, you can achieve full synchronization 3-10x faster than with traditional replication solutions.
This process enables you to reduce network latency by distributing file synchronization across every device in your environment and reducing the load on any single network.
For all data transfers, Resilio Connect engages in smart routing — i.e., the process of using the fastest, most optimal path to deliver data from a source to a destination.
Resilio’s P2P replication architecture also provides:
Organic Scalability
With traditional replication solutions, every new endpoint you add to your system adds more strain on your environment and reduces sync speed. Since data can only transfer between two devices at a time, having more devices means longer sync times.
With Resilio, every device can participate in synchronization simultaneously. So adding new devices only increases the resources and bandwidth available for sync. In other words, more demand creates more supply.
Resilio can sync hundreds of endpoints in roughly the same time that most solutions take to sync just two endpoints.
Higher Resilience
Point-to-point replication creates multiple single points of failure. In both point-to-point sync models, if any device goes down, it can delay synchronization for all the other devices awaiting data transfers. And, in hub-and-spoke replication, if the hub server goes down, replication fails entirely.
But Resilio’s P2P replication architecture makes it highly resilient with no single point of failure. If any device or network goes down, the necessary files or services can be retrieved from any other device in your environment.
This makes Resilio Connect a great solution for disaster recovery — it can provide Active-Active High Availability, meet sub-five second RPOs (Recovery Point Objective), and achieve RTOs (Recovery Time Objective) within minutes of an outage.
Multidirectional Synchronization
Resilio Connect can sync data in any direction — one-to-one, bidirectional, one-to-many, many-to-one, and N-Way sync.
The ability to replicate data in any direction enables you to quickly sync data across your entire system, reduce traffic on any single network (i.e., load balancing), and minimize latency.
N-way sync, in particular, is very effective for remote and distributed workforce scenarios. Employees in any office located anywhere in the world can make changes to files and have those file changes distributed to every other office instantly.
Case Study: VoiceBase
VoiceBase is a software that provides speech-to-text to audio and video transcription, knowledge extraction, speech analytics, and predictive analytics. They use Resilio Connect to quickly distribute speech model files of 50+ GB to over 400 production servers every 2-4 weeks. Resilio’s P2P replication enables them to synchronize these updates much faster than their previous solutions (Rsync and NFS).
“Resilio Connect enables us to reliably distribute our code, specifically new language models in a fraction of the time. These copy jobs now take an hour, down from eight. Best of all, once Resilio Connect was installed, it just works: We never need to manually intervene in any way.”
Learn more about how Resilio helped VoiceBase reduce software distribution time by 88%.
WAN Acceleration for Optimizing Transfers over Any Network
The quality of your network and how your replication solution utilizes that network can also impact latency.
Resilio Connect’s proprietary WAN acceleration protocol, ZGT, efficiently utilizes any network and optimizes transfer via:
- A congestion control algorithm: ZGT uses a congestion control algorithm that regularly probes the RTT of the network. It uses this data to identify and maintain the ideal data packet send rate.
- Bulk data transfer: ZGT maintains an ideal data packet send rate over time.
- Interval acknowledgments and delayed retransmission: ZGT reduces unnecessary transmissions over the network by using interval acknowledgments (i.e., periodically sending acknowledgments for received packets in groups) and delayed retransmission (i.e., retransmitting lost packets once per RTT).
With ZGT, no matter which Azure region your data is stored in or what the quality of the network is, you can be sure that file transfers will be as fast and efficient as possible.
ZGT also enables Resilio to utilize low-quality networks and sync data from the edge. Resilio can transfer data over any connection, including VSAT, cell (3G, 4G, 5G), Wi-Fi, and any IP connection.
The combination of P2P replication and WAN optimization enables Resilio to:
- Transfer, replicate, and sync data 10-20x faster over WANs and poor-quality networks 10-20x faster than traditional solutions.
- Achieve speeds of 100+ Gbps per server through horizontal scale-out replication (assuming adequate bandwidth on a low-latency network; if you’re using slower/lossy networks such as a 4G cell signal, sync will be reliable but not as speedy).
Granular, Flexible Control over Endpoints and Replication
Resilio Connect is designed to be a highly versatile solution that provides granular, centralized control over your entire replication environment.
Versatile Deployment
Resilio Connect can be installed on:
- Any device: You can install Resilio agents on desktops, laptops, mobile devices, servers, NAS devices, DAS devices, SAN devices, and cloud endpoints.
- Any cloud provider: Resilio supports any S3-compatible cloud service, including Microsoft Azure, AWS, Google Cloud Platform, MinIO, Backblaze, Wasabi, and more.
- Any OS: Resilio supports popular operating systems such as Windows, Linux, Mac OS, Unix, FreeBSD, Open BSD, and more.
- Virtual machines: Resilio supports virtual machines, such as VMware, Citrix, and hypervisors.
Because of Resilio’s flexibility, you can sync data across Azure cloud regions as well as between Azure and other cloud storage services (in multi-cloud scenarios).
Centralized Management and Granular Control
Resilio Connect’s Management Console provides a graphic interface that you can use to control all of your data across all on-prem and cloud endpoints from one unified location.
The Management Console enables you to control:
- Bandwidth allocation: Control how much bandwidth is allocated to each endpoint. You can even create profiles that govern how much bandwidth each endpoint receives at certain times of the day and on certain days of the week.
- Replication parameters: You can adjust replication parameters as desired, such as buffer size, disk IO, and more.
- Sync jobs: You can create, manage, pause, and stop all sync jobs.
- Selective Sync: You can control which specific files and folders sync to which endpoints.
- Review transfer reports: You can review transfer logs and a history of all executed jobs.
- Automate jobs: You can use Resilio’s REST API to configure any type of scripting or functionality you require.
Access Files through Resilio’s Object Storage Gateway
Resilio Connect can also act as an object storage gateway, providing efficient, low-latency access to files stored in cloud object storage.
Resilio’s object storage gateway includes features and capabilities that minimize costs, reduce latency, and boost productivity for remote work scenarios such as:
- Transparent Selective Sync (TSS): With TSS, you get full control over how files sync across your environment. You can automate synchronization so employees don’t need to manually sync files and can focus on their projects. And you can control which files sync to which endpoints, so relevant employees and data centers always have access to the files they need.
- Local caching: You get granular control over which files are stored on local devices vs. cloud storage. You can store frequently accessed files locally and store infrequently accessed files in long-term cloud storage, freeing up space on local devices, providing faster access to critical files, and reducing data egress costs.
- Partial downloads: You can perform full or partial downloads, so employees get faster access to the portions of files they need while reducing traffic across the network (and reducing data egress costs).
Case Study: Deutsche Aircraft
Deutsche Aircraft is a manufacturer of commercial aircraft. They use Resilio Connect to quickly synchronize 20+ million files across their Microsoft DFS namespace. Switching from DFSR to Resilio made the process much easier to manage, increased efficiency, and gave them more control over their data infrastructure.
“We have a 10Gbps network but prefer to use under 1Gbps for data transfer and replication. With Resilio, we’re able to keep that down to 250Mbps during the day and at night move back up to 1GBps… Resilio Connect is much easier to manage than DFSR. Using the Resilio Connect management console, you can see everything you need to know. Everything is visible.”
Keep Data Secure with Native Security Features
While security features aren’t directly related to reducing latency, they are critical to storing and transferring data in the cloud. While cloud storage enables businesses to more-easily scale their storage environment, it also presents additional risks of data being intercepted or corrupted.
Many traditional replication solutions don’t include native security features, forcing you to invest in 3rd party security tools and VPNs.
But Resilio includes built-in security features that protect your data from interception and corruption, such as:
- End-to-end encryption: Resilio encrypts data using AES 256-bit encryption.
- Mutual authentication: Each endpoint must provide an authentication key before initiating a transfer, ensuring your data is only delivered to authorized endpoints.
- Cryptographic data integrity validation: Resilio uses cryptographic integrity validation to ensure files always arrive at their destination intact and uncorrupted.
- Access control: You can control who can access specific files and folders, so you can limit access to only privileged employees.
Use Resilio Connect for Faster Data Transfer
Whether you’re sending TBs of files from Japan East to UK South or just going from Canada Central to West US 2, Resilio Connect can help you reduce latency between Azure regions by providing faster data transfers and more efficient network utilization through:
- P2P replication: Resilio’s P2P replication architecture enables every endpoint to take part in replication, increasing replication speed and balancing the load across your entire network.
- WAN optimization: No matter what network you’re using, Resilio’s WAN acceleration protocol can efficiently utilize the network to provide the fastest transfer speeds possible.
- Granular control over endpoints and replication: Resilio enables you to manage all of your cloud and on-premises endpoints from one unified location. You get granular control over bandwidth and replication parameters, so you can optimize how data transfers across your environment.
- Object storage gateway: Resilio’s object storage gateway enables you to achieve low-latency, cost-efficient access to files by selectively caching files on local devices, automating how synchronization occurs in your environment, and performing partial downloads.
Organizations such as Skywalker Sound, Cisco, Mixhits Radio, 2K Games, and Deutsche Aircraft use Resilio Connect to sync data across cloud servers and on-premises devices. To learn more about how Resilio Connect can reduce latency and enhance transfer speeds, schedule a demo.
