Bidirectional Sync (or two-way sync) is the ability to reliably sync files across two or more locations concurrently — and preferably in real-time. Each location could have a few to many systems. Some use cases for bidirectional sync include:
- Real-time sync of geographically distributed servers and storage.
- Data protection across sites with built-in high availability.
- Global cross-site collaboration by remote workers.
- VDI (virtual desktop infrastructure) sync.
Most conventional approaches can only sync one way between 2 systems. However, our synchronization software (Resilio Connect) offers fast, reliable, and efficient bidirectional synchronization across as many systems and sites as needed, including the cloud.
In this article, we’ll discuss traditional approaches to one- and two-way synchronization. Then, we’ll explore why businesses need bidirectional sync capabilities and how Resilio Connect delivers them.
Resilio Connect can sync file changes in any direction across your entire environment quickly and efficiently, regardless of physical distance, file size, or file numbers (we’ve successfully synced 450+ million files in a single job).
Resilio Connect uses P2P (peer-to-peer) replication to sync environments 3–10x faster than traditional solutions. It also scales organically, eliminates single points of failure, and has built-in security features, making it an incredibly reliable sync solution.
Organizations in gaming (Bungie, 2K Games), media (Sunrise, Warner Brothers, CBS), technology (Microsoft, Match), retail (McDonald’s, Mercedes-Benz), and more use Resilio Connect to sync their data in real-time and any direction. To see Resilio in action, schedule a demo.
One-Way Sync
Most conventional replication technologies offer one-way synchronization from a single data source (system 1) to a single target (system 2). Tools like Rsync, for example, can efficiently (depending on the payload, network, and other factors) sync one way between 2 systems.
One-way sync is popular and serves a purpose.
Storage replication products (like Isilon SyncIQ and NetApp SnapMirror) offer one-way sync based on point-in-time snapshots initiated in their file systems. So, one-way sync may suffice to protect your data for use cases like Disaster Recovery (depending on your RPO or RTO) or basic server-to-server or storage-to-storage backup. But if the customers’ DR goals are for faster RTOs and lower RPOs — and synchronizing files across more than two sites — these solutions may not be a good fit.
Basic Point-to-Point Two-Way Sync
Other tools that support bidirectional sync are Unison, Aspera Sync, and Microsoft DFSR. These are limited by their point-to-point (client-server) architecture, among other caveats like the prospect of file corruption or data loss.
These implementations limit the two-way sync to at most two systems. Thus, only those users and apps accessing those two systems will benefit from two-way sync. Worse, each system is a performance bottleneck and a single point of failure.
At best, point-to-point two-way sync enables a classic hub-and-spoke: the hub is the source, and each spoke is the target. The benefit of the hub and spoke is that updates can be relayed from the hub (source) to the spokes (targets) and vice versa. This obviously does not scale. Worse, each spoke is land-locked from the other spokes; all traffic must go through the hub. So implementers, beware: the hub is a performance bottleneck and a single point of failure.
Point-to-point architectures are generally helpful for certain use cases but greatly limit how you can move and sync data. (Here’s a good comparison of point-to-point vs. peer-to-peer.)
Other potential problems with point-to-point two-way sync tools include poor reliability, poor management and monitoring, and lack of conflict resolution and diagnostics tools.
High-Level Business Needs for Bidirectional Sync
Businesses and organizations of all types and sizes need their file-based data sets protected, current, and readily available when and where they are needed. Your business processes and workflows should determine when bidirectional sync is needed.
Generally, the most current files should be continuously available to users, applications, and workflows. For many shops, when data stops, business stops. Organizations use Resilio Connect’s bidirectional sync capabilities to keep their files current and accessible across multiple locations.
Here are a few real-life examples and use cases:
- Global collaboration across sites. Blockhead Studios uses Resilio Connect’s two-way sync to keep large media files in sync between Australia and New Zealand.
- Real-time sync across any system. Turner Sports uses Resilio to edit and sync thousands of files during live events. Their workflow includes a remote team connected via a shared server, a render farm, and a live broadcast, with small windows for set-up and breakdown adding even more pressure.
- Server-to-server real-time data synchronization. Music streaming service MixHits Radio uses Resilio to deliver curated playlists to their clients and sync the metadata necessary for a high-quality listener experience. They also drastically reduced their troubleshooting times thanks to Resilio’s simple configuration and centralized management.
One thing to note is that Resilio is designed to sync files, not different databases (such as CRM, Salesforce, mySQL, etc.). We have had customers sync their entire database files when virtualized in a file — but that is not a topic of discussion in this blog.
Reliable Bidirectional Sync at Scale
At Resilio, we believe that bidirectional synchronization is a differentiated replication capability that should “just work” and not be limited to only 2 servers. That’s why our solution for bidirectional sync is built on a P2P architecture. This highly reliable design enables multiple servers (or other endpoints running Resilio agents) to participate in the sync process.
Resilio’s P2P architecture results in three key benefits:
1. High-Speed Sync
In a P2P scenario, every device in your environment can take part in data synchronization. This removes the need for cloud hopping — i.e., sending the data to a central server first before syncing with the other servers — and enables you to utilize the full bandwidth of your entire environment to achieve faster sync.
In addition, Resilio uses file chunking to split files into smaller pieces that can be transferred independently. This process results in transfer and sync speeds that are 3–10x faster than traditional solutions.
Resilio’s P2P architecture and file chunking capabilities enable any-directional sync — you can sync files one-way, two-way, one-to-many, many-to-one, and N-way.
2. No Single Point of Failure
One of the biggest issues with traditional transfer and sync solutions is that they create single points of failure. However, Resilio’s P2P architecture eliminates single points of failure, making our solution ideal for disaster recovery.
If one device in your environment fails, Resilio can always access data from the others.
When recovering from a server failure, power outage, or another disaster, Resilio can also use all your servers and help you achieve sub-five-second RPOs (Recovery Point Objectives) and RTOs (Recovery Time Objectives) within minutes of an outage.
These capabilities enable you to use Resilio for a variety of disaster recovery scenarios, including:
- Hot-site DR
- Warm-site DR
- Offsite copy
- Cold DR
3. Organic Scalability
Resilio Connect’s P2P architecture makes it an organically scalable solution.
Because every device in your environment acts as a hub, adding more demand automatically creates more supply. For example, Resilio can synchronize data 50% faster than point-to-point solutions in a 1:2 scenario and 500% faster in a 1:10 scenario.
This makes our software incredibly valuable for applications that rely on serving the right data at the right time to many geographically dispersed users.
Efficient Real-Time Sync
Polling the file system or listening for file change notifications is a key enabler of real-time sync. All systems should be able to efficiently capture and replicate only the changed portion of the file during the two-way sync.
That’s why Resilio automatically detects and only syncs the changed portion of your files across the network, making it a very efficient sync solution. Plus, Resilio lets you reduce unnecessary data movements and improve efficiency even more by:
- Storing frequently accessed files on-premises, so they don’t have to be downloaded from the cloud every time.
- Using Smart Routing to select the most optimal network for your traffic. For example, you can keep all your traffic on a cloud provider’s network or move parts of it to a remote edge network. That way, you’re not relying on expensive WAN connections all the time but can also utilize cheaper LANs.
- Selectively downloading and synchronizing files with Transparent Selective Sync (TSS). TSS lets you browse objects as files, select individual files, and download, partially download, or sync them according to your needs. This granular control helps you ensure that no unnecessary data gets moved across the network.
Reliable Sync Across any Network with Built-In WAN Optimization
Resilio Connect can synchronize your files and deltas across any type of network and is resilient to failures. This is possible thanks to Zero Gravity Transport™ (ZGT) — our proprietary WAN acceleration protocol.
ZGT is:
- Designed to efficiently transfer files across any network and overcome latency and packet loss. ZGT does this by maintaining a uniform rate of packet distribution over time, reducing unnecessary retransmissions, and sending out interval acknowledgements.
- Optimized for transferring files in predictable timeframes over unreliable networks. You can rely on sync, replicate, and ingest data from the edge of a network to a centralized location using any device and network (VSAT, broadband, Wi-Fi, cell, etc.).
For example, our client Shifo provides health data solutions to over 3000 communities across eight countries. Their teams often have to transfer data across networks with low bandwidth and poor connectivity. Thanks to Resilio, they now reliably sync patient data from remote locations with their headquarters and reduce their file delivery times from months to days.
If you want a more detailed look into the topic of WAN optimization, check out:
- Our transfer speed calculator. The calculator can help you estimate how much time Resilio’s technology can save your organization, depending on your use case.
Built-in Security and Data Integrity
There should never be data loss or data corruption during the two-way data synchronization, no matter how large the data set. If there’s a conflict accessing a file, the system should preserve data integrity. When files or deltas are in transit across the network, data integrity should also be preserved during the sync process.
As we mentioned, Resilio’s P2P architecture and proprietary WAN optimization technology play a big role here. However, our solution also comes with built-in security features, like:
- AES 256 encryption, which encrypts your data at rest and in transit.
- Mutually authenticate endpoints, which guarantees data only arrives at designated endpoints.
- Cryptographic data integrity validation, which ensures your data always arrives at its destination uncorrupted.
- Data immutability, which means that our solution stores copies of your data in the public cloud to protect you from data loss and ransomware.
These and all other Resilio data protection features have been verified by 3rd-party security experts.
Centralized Management and Flexibility
Resilio offers an easy-to-use management console that lets you create and manage data sync jobs and gives you full visibility into their progress. At the same time, you can also use Resilio’s powerful REST API to configure any type of scripting or functionality your sync job requires.
The console enables automation for endpoint bandwidth automation and lets you adjust key parameters like:
- Buffer size
- Packet size
- File priorities
- Disk I/O threads
- And more
You can receive notifications (e.g., via email or a webhook connector) to stay up-to-date with the sync process.
Besides making it easy to set up and control sync jobs, Resilio Connect drastically simplifies file access. Our solution can work as a file gateway that overcomes data silos by enabling simple, low-latency access to all files (regardless of where they’re stored) from an interface that operates much like Microsoft OneDrive.
Finally, our solution works with:
- Desktops, laptops, mobile devices, and (most) NAS devices.
- Virtual machines like Citrix, VMware, and hypervisors.
- Any S3-compatible cloud storage, such as AWS S3, Google Object Storage, and Azure.
- Popular operating systems, including Windows, Mac, Linux, FreeBSD, and OpenBSD.
This, combined with the ease of use and centralized management, makes it painless to incorporate Resilio into your existing workflows. Instead of managing new hardware, you can just install Resilio on your existing infrastructure — including servers, desktops, edge devices, and more — and start syncing data in as little as two hours.
Fast, Reliable Bidirectional Sync and More with Resilio
Resilio Connect gives you the flexibility to enable bidirectional synchronization across as many servers and sites as needed. This opens up opportunities for global file collaboration, real-time server sync, and active-active high availability across sites in support of HA and DR, and much more.
With Resilio Connect, you can:
- Sync servers 3–10x faster than traditional solutions.
- Sync in any direction, such as one-way, two-way, one-to-many, many-to-one, and N-way sync.
- Quickly recover from sync errors with minimal human intervention and eliminate single points of failure.
- Reliably transfer and sync files over any network, thanks to Resilio’s proprietary WAN optimization technology.
- Manage your entire sync environment (across on-prem devices, cloud providers, and hybrid cloud setups) from one unified interface.
- Keep your data secure with native AES 256 encryption and other security features.
- And much more!
To learn more about how Resilio Connect can help your business, schedule a demo with our sales team.
