Robocopy (Robust File Copy) is a command line tool that can be used to transfer and synchronize files between computers. But it’s designed primarily for small, simple, one-way sync jobs. There are limitations to what kind of sync environments you can create, especially for business users. And if you want to perform file synchronization (especially of large files) over a network, you’ll need to know how to set up such an operation and what to expect.
In this article, we’ll discuss 5 issues that hinder Robocopy synchronization and how to sync large files with Robocopy.
We’ll also discuss our own, superior file synchronization solution: Resilio Connect. Resilio Connect is a fast, reliable file synchronization solution that uses a P2P transfer architecture and WAN optimization to synchronize files of any size over any network. It has been tested and is able to sync 250+ million files in a single sync job at speeds of 10+ Gbps per server.
Resilio Connect is used by organizations across multiple industries (gaming, tech, media, logistics, retail, and more) to quickly synchronize, distribute, and consolidate files across any network. And its organic scalability makes it particularly well-suited for large, enterprise deployments.
To see if Resilio Connect could be a fit for you, schedule a demo.
5 Issues That Hinder Robocopy Sync
If you plan on using robocopy to sync files on your computers, there are a few limitations you should be aware of. While some can be overcome, doing so involves implementing complex coding and sync set-ups.
1. Robocopy is really a one-way file sync solution
Robocopy is really a one-way file transfer solution. It’s designed to transfer files from one computer to another on an individual case basis, or for syncing folders from one computer to another (i.e., syncing from a source folder to a destination folder).
While Robocopy can be used to perform two-way synchronization, setting up such a sync job requires more complex coding and management.
You can learn how to create a two-way folder sync with Robocopy here.
2. Robocopy is best for simple sync jobs
As we just stated, Robocopy is primarily designed for simple, one-way file transfer. As your file replication needs grow (i.e., syncing larger files, syncing larger numbers of files, or syncing to many endpoints), Robocopy will become unusable — it will either break down often or simply won’t be able to perform the kind of replication jobs you need it to.
For more complex sync jobs, you’ll need to use other sync solutions, DFSR, or (best of all) Resilio Connect.
3. Robocopy syncs poorly over WANs
Robocopy takes a very long time when syncing over WAN connections, particularly when transferring large files over long (high latency) or lossy connections. So it should only be used to sync large files over LAN connections.
4. Robocopy doesn’t perform real-time synchronization
In order to sync your files, Robocopy must scan your entire file directory to identify file changes to replicate. The larger your files and/or the more files you have in your directory, the longer it will take to scan and sync.
While you can program Robocopy to scan file directories periodically (more on this later), Robocopy can’t perform real-time synchronization ( i.e., detecting and syncing file changes as soon as they occur). Organizations that rely on having the most up-to-date versions of files as quickly as possible will need a real-time synchronization solution.
5. Robocopy only works on Windows computers
As a Microsoft product, Robocopy only works on Windows servers. If you plan on using a different operating system (such as Linux, Mac, Ubuntu, or others) or syncing cross-platform, you’ll need another tool.
It’s also important to note that Robocopy doesn’t mirror properly on systems older than Windows Vista, as it ignores changed security attributes of previously mirrored files.
How to Sync Large Files with Robocopy
Performing any kind of sync with Robocopy will be difficult for novices. If you’re inexperienced with Robocopy syntax and need to learn the basics, check out this ultimate guide to Robocopy to get up to speed.
Once you know the basics of creating Robocopy commands, you can use the following command prompts to sync large files.
Robocopy Sync Basics
To sync files from your source directory to your destination directory, use the following command:
robocopy C:\src C:\dst /MIRIn the event that a file sync fails in the middle of a transfer, you can program Robocopy to retry the transfer using one of the two following commands:
robocopy C:\src \\SRV1\share /Zrobocopy C:\src \\SRV1\share /FFTSome users have reported that the /Z command reduces replication performance. The /FFT option uses FAT file system timestamps, a method that more accurately preserves file timestamps.
Speeding Up Large File Transfer with Robocopy
If Robocopy is transferring your large files too slowly, you can try using the /J switch. This programs Robocopy to use unbuffered I/O for the file transfer, which may speed things up.
Other Robocopy Sync Commands
Depending on the parameters of your sync job, you may consider implementing other commands to govern various aspects of how synchronization occurs in your setup. Use the following commands as needed:
| /s | This command programs Robocopy to copy subdirectories (but not empty directories). |
| /e | This command programs Robocopy to copy empty directories. |
| /b | This command programs Robocopy to copy files in backup mode, which enables you to backup files without worrying about administrative permissions. |
| /zb | This command programs Robocopy to use restartable mode. |
| /efsraw | This command programs Robocopy to copy all encrypted files in EFS RAW mode. |
| /mt | This command enables multithreaded copying. The default is 8 threads, but you can specify the number of threads you want. |
| /copy:DAT | This command will copy the source data, attributes, and time stamps of the source files. |
| /copyall or copy: DATSOU | This command programs Robocopy to copy all file info: data, attributes, time stamps, security, owner info, and auditing info. |
| /xc | This command will exclude changed files. |
| /xo | This command programs Robocopy to exclude older files — i.e., if a file is older than the existing copy in the source directory, this command will exclude it from the sync. |
| /xn | This command programs Robocopy to exclude newer files — i.e., if a file is newer than the existing copy in the source directory, this command will exclude it from the sync. |
| /mot: | This command will program Robocopy to perform real-time sync by monitoring for file changes (in minutes). You specify the frequency of file scans — e.g., /mot:3 to scan for file changes every 3 minutes. |
Use Resilio Connect for Superior Large File Synchronization
While your file replication needs are small, you can use Robocopy to adequately sync and transfer files across your environment. But as your file sizes and replication needs grow, you’ll need a better, faster, more reliable solution.
Fast, Reliable, and Scalable File Synchronization with P2P Transfer
To transfer files, Robocopy uses a “follow-the-sun” transfer architecture. In this setup, files are transferred from one computer to the next sequentially (i.e., from Computer 1 to Computer 2, then from Computer 2 to Computer 3, and so forth).
This method of file transfer is both slow and unreliable. Each computer must wait for replication to complete on the computer that precedes it before it can receive files. And if one of those computers is on a slow network, then replication will be delayed for all other computers awaiting files.
Resilio Connect, however, uses a P2P transfer architecture to replicate files. In a P2P topology, every computer can share files with every other computer at any time. Connect also uses a process known as file chunking, where files are broken up into small chunks that can transfer independently. This enables all of the computers in your replication environment to share files concurrently, dramatically enhancing replication speed.
For example, if Computer 1 wants to share a file with the other 9 computers in your environment, it can divide that file into 5 chunks. It can share the first file chunk with Computer 2. Once Computer 2 receives that file chunk, it can share that chunk with any other computer in your network, even before it receives the other 4 file chunks.
Resilio can also perform real-time replication. It uses optimized checksum calculations and real-time notifications from the host OS to detect and replicate only the changed portions of files.
Resilio’s P2P architecture means that there is no single point of failure in your system. If one computer goes down, the other computers in your network can provide the files and services needed. It also enables Resilio to leverage the internet channels across all devices in your environment in order to increase replication speed. So if one computer is on a slow network, files can be transferred from a computer on a faster network.
P2P transfer also makes Resilio organically scalable. As more computers are added to your environment, both speed and reliability increase. Connect will perform 50% faster than most one-to-one solutions in a 1:2 transfer scenario, and 500% faster in a 1:10 scenario.
Case Study: An Employee Benefits Administration Company
The IT manager for one Resilio client, an employee benefits administration company, said this about switching to Resilio from their previous file replication solution: “At the end of the day it’s really about peace of mind and knowing that replication and sync just works all the time; no matter what: Resilio really put us in a good spot. I can trust the file is getting replicated.”
Optimized WAN Transfer Technology
Robocopy has no means of optimizing transfer over WANs, making it so slow as to be unusable in any WAN transfer scenario.
But Resilio Connect uses a proprietary WAN transfer protocol known as Zero Gravity Transport™ (ZGT™). Unlike traditional transfer protocols (which are hindered by high packet transmission time and packet loss), ZGT™ maximizes transfer speed across WANs by minimizing the impact of packet loss and high latency on any network. It accomplishes this using:
- Bulk data transfer: ZGT sends packets periodically with a fixed packet delay to create a uniform distribution.
- Interval acknowledgement and delayed retransmission: ZGT only sends acknowledgements for groups of packets (with additional info about packet loss), rather than for each individual packet. Lost packets get retransmitted once per RTT to decrease unnecessary retransmissions.
- Congestion control algorithm: ZGT uses a congestion control algorithm to calculate and maintain an optimal sending rate.
Further reading: For more information on how Resilio optimizes WAN transfer, check out our WAN optimization whitepaper.
Easy to Use and Manage
Because Robocopy is a command line replication solution, it requires a degree of expertise to use. It’s also difficult to gain insight into your replication environment and fix problems that occur.
But Resilio was designed to be easy to use. You can deploy it on your existing infrastructure and begin replicating in as little as 2 hours. It also works on a large number of operating systems (such as Windows, Mac, Linux, Ubuntu, and more) and can be configured cross-platform.
Resilio’s user-friendly Management Console gives you an easy way to gain insight into and control over your replication environment. You can obtain real-time notifications and detailed logs on the Resilio dashboard. And you can optimize performance and resource usages by adjusting replication parameters, such as:
- Network parameters: Control packet size, buffer size, and more.
- Storage parameters: Control disk io threads, data hashing, file priorities, and more.
- Bandwidth parameters: Create rules governing the amount of bandwidth each device can use.
- Functionality: Use Resilio’s REST API to manage agents, script any functionality, control jobs, and more.
Case Study: Sunrise Productions
Sunrise Productions Head of Technology stated: “There isn’t really another cost-effective way for us to do what we need to do. If we didn’t have Resilio Connect, it would take a lot of time. It would take a lot of energy to make sure things are in parity. It would take a lot of IT overhead to make sure things don’t break. It would negatively change the viability of our business.”
Learn more about how Resilio provides Sunrise Productions with fast and cost-effective file sync.
Use Resilio Connect for Optimized File Sync
While Robocopy can satisfy basic synchronization requirements for small and large files, you’ll need a better file sync tool to handle your replication needs as your file sizes and replication environment grow. Resilio Connect provides the fastest, most-reliable file synchronization due to its:
- P2P transfer architecture: Resilio can perform real-time synchronization and sync up to 10x faster than conventional tools with transfer speeds of up to 10+Gbps per endpoint.
- Organic scalability: The more endpoints you add, the better Resilio performs. And with the ability to transfer peer-to-peer, you can sync in any direction: one-to-one, bidirectional, one-to-many, many-to-one, and N-way.
- Optimized WAN transfer: Resilio can optimize WAN transfer to rapidly sync over any network, utilize all of your network bandwidth, and dynamically route around outages.
- Ease-of-use: Resilio’s Management Dashboard makes it easy to monitor and control your replication environment in order to optimize performance and minimize costs. And you can get set up and begin replicating in as little as 2 hours.
To see if Resilio Connect could be a fit for you, schedule a demo.
