Azure Storage Mover (ASM) is a fully managed migration service recently released by Microsoft. The Azure Storage Mover service is a V.1.0 product with serious limitations — no bandwidth throttling, poor performance over WANs, and others we’ll address.
ASM lets users migrate data once or repeatedly from an on-premise source to an Azure cloud target (in limited scenarios) while minimizing downtime.
ASM also enables fast replication of small to mid-sized datasets (less than 50 TB) for a few basic hybrid-cloud replication use cases — such as migrating file shares from an on-premise server into the Azure cloud, merging data from multiple servers into the Azure cloud, and periodic backups of on-prem data into Azure. Users can manage the migration of all globally distributed file shares from a single solution.
While Microsoft plans to expand the features of ASM over time, it’s a brand-new tool with limited functionality. For example:
- ASM supports limited sources and targets. It can only transfer files from an NFS mount on a NAS or server to an Azure blob container or from an SMB mount to an Azure file share.
- ASM is only useful for the three use cases mentioned above; it’s not effective for disaster recovery or synchronizing multiple cloud OR on-prem endpoints.
- ASM can’t adequately handle large enterprise workloads (greater than 50 TB).
Some small to midsize businesses may still find ASM useful, as it’s well-suited for migrating smaller data sets one-way into Azure. In this article, we’ll discuss how it works, what scenarios you can use it in, and what its limitations are.
We’ll also contrast ASM with Resilio Connect — our file replication solution designed to move or sync files and objects of any size and type, and in any direction — for enterprises with more demanding data movement and replication needs.
Would you like to see a live demo or speak with one or our engineers? Please schedule a demo to learn more.
Resilio Connect is a highly reliable, software-only file replication and synchronization solution. Unlike ASM, which is point-to-point, Resilio is P2P (peer-to-peer). This means all agents can work together to replicate data.
Resilio also provides built-in WAN optimization for fast and efficient replication across hybrid and multi-cloud environments.
Overall, Resilio is a superior alternative to ASM and other replication solutions because it is:
- Flexible: You can use Resilio Connect with any device, OS, or cloud storage provider such as Azure, AWS, Google Cloud, and many more.
- Easily managed: You can manage and automate replication across all of your on-prem and cloud endpoints from a single, unified location. And Resilio gives you granular control over how replication occurs in your environment so you can maximize performance and minimize costs.
- Fast: Resilio’s P2P replication and WAN optimization enable it to achieve replication speeds of 100+ Gbps per server and sync endpoints 3-10x faster than traditional solutions.
- Scalable: Resilio Connect scales organically to support environments of any size. You can sync hundreds of endpoints in the same time that most sync solutions take to sync two. And Resilio can handle files of any size or number.
- Reliable: Resilio fully utilizes any network, eliminates single points of failure, dynamically reroutes around outages, and retries transfers until completion.
- Versatile: Resilio supports a variety of cloud, hybrid, and multi-cloud use cases, such as web/app server sync, software update distribution, remote work, data backup, disaster recovery, and more.
- Secure: Resilio keeps data secure end-to-end with built-in security features.
Organizations in tech (Cisco, Match), media (Turner Sports, Skywalker Sound), gaming (2K Games, Bungie), retail (KFC, McDonald’s), and more use Resilio Connect to sync and transfer data across cloud and hybrid/multi-cloud environments. To learn more about how Resilio Connect can help you replicate data into and across any cloud storage provider, schedule a demo.
How Azure Storage Mover Works
While Microsoft has plans to expand ASM’s functionality, the current iteration (public preview) provides simple replication for basic migration scenarios.
How Azure Storage Mover Works
ASM is a virtual machine-based migration appliance that runs on a virtualization host. It works by deploying agents known as Storage Mover Resources on a device near the source storage (ideally, as close as possible). You can deploy ASM agents on NFS mounts (on a NAS or server) and SMB mounts for just two migration configurations:
- Agents on NFS mounts can only migrate data to an Azure Blob container.
- Agents on SMB mounts can only migrate data to an Azure file share.
Currently, those are the only two supported scenarios. You can’t, for example, migrate data from an NFS share to an Azure file share.
Unlike AzCopy, ASM supports full file fidelity when migrating data using the SMB protocol — i.e., it maintains folder structures and metadata (such as timestamps, file attributes, and ACLs) on the target destination.
When migrating from an NFS source, ASM preserves the metadata of the source folder in the custom metadata field of the blob.
Azure Storage Mover Deployment: How to Set It Up
ASM is available in your Azure portal. To use it, you must deploy a Storage Mover migration agent VM on a device near the source storage (learn more about deploying agents here). The migration agent sends your data directly to the target storage destination (blob storage container or file shares) in Azure.
To set up ASM, first create a new VM to host the agent. Then download the zip file for the ASM agent here, and extract the virtual hard disk (VHD) image to your host.
You should calculate the available resources (compute, memory, network, and storage space) your agent will require, which is usually determined by the overall size of your data as well as the number of files and folders. Agents will need a minimum of 20 GiB of local storage.
You can deploy agents on multiple on-prem endpoints and use AMS to:
- Manage data migration for all of your globally distributed file shares from a single solution.
- Create migration projects for every workload.
- Define the source, target, and migration settings for each source share.
- Control when migrations start and track their progress in ASM.
Resources: Learn more about ASM and how best to use it with these Azure resources. Learn more about incorporating ASM into your migration plan here.
Azure Storage Mover Limitations
If you’re new to cloud storage or have recently started an Azure storage account, ASM is a useful tool for getting your data on the cloud.
In its current form, it only supports two use cases:
- Lift-and-shift: You can perform a one-time migration of data stored on premises to the Azure cloud (blobs or file shares).
- Periodic merges and backups: You can use AMS to periodically backup data to the Azure cloud for business continuity. And you can merge data from multiple on-prem sources into the Azure cloud for data protection.
Beyond these two basic use cases, ASM is not a suitable solution because it suffers from many limitations, such as:
1. Limited Source-Target Pairings
As discussed earlier, ASM can only move files from NFS mounts on a NAS device or a server to an Azure Blob or from an SMB mount to Azure file shares.
If you’re using a different kind of on-prem device, want to replicate to different Azure services, or want to replicate across cloud regions, you’ll need another solution. ASM also can’t be used with other cloud storage providers, such as AWS.
2. Management Difficulty
ASM is managed through the Azure portal, PowerShell, or CLI. Each requires you to learn and use Azure command-line coding.
3. Can’t Handle Large Data
ASM is designed for workloads of 50 TB or less. Larger enterprise businesses will need a solution capable of handling larger workloads.
4. Slow Replication, Especially Over WANs
Azure Storage Mover uses traditional point-to-point replication which is slow. And, although it uses the Azure REST protocol (which provides fairly fast transfer), it doesn’t use any WAN optimization technology to enhance WAN transfers. If you’re replicating over a WAN (especially if merging data from multiple devices to the Azure cloud), ASM will perform poorly.
5. Limited Use Cases
ASM is suitable for simple migration jobs. However, it can’t sync data across two or more endpoints.
And while it can perform periodic backups, it’s not designed to backup data as frequently as needed for effective DR plans, doesn’t provide real-time backup, and can’t provide Active-Active High Availability. Using ASM as a backup tool for DR would result in long RPOs (Recovery Point Objectives) and poor RTOs (Recovery Time Objectives).
6. No Bandwidth Throttling
ASM agents utilize all of the available bandwidth on the source and target. It doesn’t currently support the creation of bandwidth throttling schedules.
How Resilio Connect Overcomes Azure Storage Mover’s Limitations
Resilio Connect is a real-time file replication and synchronization software system that can perform the same functions as ASM (and much more), but better:
- It’s more flexible and easier to manage
- Provides high-performance replication through its P2P architecture
- Fully utilizes any network through proprietary WAN acceleration technology
- Supports multiple cloud and hybrid cloud use cases
- Secures your data end-to-end
Resilio Provides Flexible Deployment & Easy Management
As discussed in this article, ASM is limited to NFS mounts, SMB shares, and the Azure cloud.
Resilio Connect is a much more flexible solution that you can deploy on:
- Any device: Resilio agents can be installed on NAS devices, SAN devices, DAS devices, servers, desktops, laptops, mobile devices, and virtual machines.
- Any cloud storage provider: In addition to Microsoft Azure, Resilio supports almost any cloud storage provider, such as AWS, Google Cloud Platform, Wasabi, MinIO, and more.
- Any operating system: Resilio works with Windows, MacOS, Linux, Unix, FreeBSD, OpenBSD, and more.
You can deploy Resilio on your existing IT infrastructure and begin replicating data in as little as two hours. Because of Resilio’s flexibility, you can use it to replicate across on-prem devices, cloud endpoints, cloud regions, and different cloud providers.
You can manage all of your cloud and on-prem endpoints from one centralized location. Resilio’s Management Console gives you granular control over how replication occurs in your environment, and you can use it to:
- Create and control replication jobs across your entire environment.
- Adjust bandwidth allocation for every endpoint.
- Create bandwidth profiles that govern how much bandwidth is allocated to each endpoint at certain times of the day and on certain days of the week.
- Adjust replication parameters on each job, such as data hashing, disk IO, and more.
- Review replication logs and a history of all executed jobs.
- Get real-time reports on replication, and configure updates to be automatically delivered via email or Webhooks.
- Establish file priorities.
- Use Resilio’s powerful REST API to script any type of functionality.
Resilio Connect provides a degree of flexibility and control that makes it the ideal solution for managing and replicating data in hybrid and multi-cloud environments.
Beyond simply migrating data into an Azure storage account, Resilio enables you to synchronize all of your cloud and on-prem endpoints (regardless of what type of cloud provider or device you use) in real-time, and optimize sync parameters in order to minimize costs and maximize performance.
Case Study: Optimized Resource Utilization via Granular Control
Deutsche Aircraft is a manufacturer of commercial aircraft. They use Resilio Connect to automatically and efficiently sync millions of files in real-time, as well as enable Active-Active High Availability for fast failover across all of their Microsoft DFS servers.
“We have a 10Gbps network but prefer to use under 1Gbps for data transfer and replication. With Resilio, we’re able to keep that down to 250Mbps during the day and at night move back up to 1GBps… Resilio Connect is much easier to manage than DFSR. Using the Resilio Connect management console, you can see everything you need to know. Everything is visible.”
Resilio Provides High-Performance Replication and Sync
ASM provides simple, one-way replication, where the target must always be an Azure cloud endpoint (blob containers or file shares). You can use it to transfer large amounts of data into Azure, merge files from multiple on-prem endpoints into Azure, and periodically backup files to the cloud. But it doesn’t sync bidirectionally, doesn’t sync data in real-time, and can’t keep both on-premise and cloud endpoints synchronized.
Resilio’s superiority as a replication solution is due in large part to its P2P replication architecture. In a P2P architecture, every endpoint with a Resilio agent installed on it can share data with and receive data from every other endpoint with a Resilio agent. And each endpoint can replicate data simultaneously.
This P2P dynamic enables Resilio to provide many capabilities that ASM can’t, such as:
Blazing Fast Replication and Sync Speeds
Resilio Connect uses a process known as file chunking, which splits files into several chunks that can transfer independently of each other. And, because every endpoint in a P2P architecture can share files with every other endpoint simultaneously, Resilio can fully utilize your entire environment to quickly sync files across your entire system.
Imagine you want to sync a 10 TB file across five endpoints. Resilio can split that file into five 2 TB chunks. Device 1 can share the first 2 TB chunk with Device 2. Device 2 can immediately share that first chunk with Device 3, even before it receives the other 8 TB of the file. Soon, all five devices will be sharing file chunks at the same time, enabling you to sync your entire environment 3-10x faster than traditional sync solutions.
Resilio can also perform real-time replication, and only replicates the changed portions of files. While other replication solutions must scan file systems in order to detect file changes, Resilio uses optimized checksum calculations (identification markers assigned to each file that change when the file changes) and notification events from the host OS to detect and immediately replicate file changes (Resilio can also perform scheduled and manual syncs).
So anytime a change is made to a file on any of your cloud or on-prem endpoints, that change is immediately propagated across your entire environment (or you can use Resilio’s Selective Sync capability to control which endpoints files are synced to). Resilio’s engineers were able to replicate a 1 TB data set between Azure regions in 90 seconds.
Multi-Directional Replication
Resilio’s P2P architecture allows it to replicate files in any direction, such as:
- One-way: Backup data to the cloud or a physical data center in real-time.
- Bidirectional: Keep files synchronized across two devices in both directions.
- One-to-many: Sync data from one source to multiple cloud, on-prem, or hybrid backup sites for a more resilient DR strategy. Distribute software updates to all of your geographically distributed servers.
- Many-to-one: Merge data from multiple endpoints to one location for applications and DR.
- N-way (many-to-many): Keep all of your endpoints synchronized for remote work, DR, and web/app server sync.
Case Study: MixHits Radio Syncs Web Servers in Real-Time
MixHits Radio is a music streaming service that provides music for businesses, such as McDonald’s, Dunkin Donuts, and more. They use Resilio Connect to sync their music and metadata across all of their US web servers in real-time.
“If a music program updates in one location, file changes are detected and propagated across servers within 2 seconds. That rapid update and real-time synchronization has been a saving grace for us.”
Learn more about how Resilio helps MixHits Radio quickly and easily sync their servers.
Organic Scalability
P2P environments scale organically. Since every endpoint can share files, every new endpoint added to the application only increases replication speed and resources. In fact, Resilio can sync hundreds of endpoints in roughly the same time it takes most solutions to sync two.
Resilio’s P2P architecture and organic scalability enable it to perform horizontal scale-out replication and reach replication speeds of 100+ Gbps per server.
Resilio can also handle files of any size or number. Resilio’s engineers were able to successfully sync 450+ million files in a single job.
Bulletproof Reliability and Resiliency
Resilio Connect is the most reliable replication solution available, so you can be sure your files will always reach their destination no matter what.
Other replication and sync solutions use point-to-point replication (i.e., from one device to another), which creates single points of failure. If any device or network in your environment goes down, it can stop or delay replication for every other device.
But Resilio’s P2P architecture eliminates single points of failure. If any device goes down, any other device can supply the necessary files or services.
This allows Resilio to provide Active-Active High Availability for disaster recovery scenarios. Resilio effectively turns every endpoint in your system into a backup device. In the event of a disaster, your application can retrieve data and services from any endpoint in your environment. And every endpoint can work together to bring your system back online — enabling Resilio to achieve sub-five second RPOs and RTOs within minutes of an outage.
Resilio is also incredibly resilient. If a transfer is interrupted, Resilio can perform a checksum restart to resume the transfer where it left off and will retry transfers until they are complete. And Resilio can dynamically reroute around downed networks and devices to ensure files always reach their destination.
Resilio is designed to operate with minimal human intervention. You can program rules such as file conflict resolution to automate troubleshooting in the event of a replication error.
Resilio Fully Utilizes Any Network with WAN Optimization
Replication performance is closely tied to the quality of your network. Many organizations operate with limited bandwidth and in areas with poor connections. And replicating data across cloud regions requires utilizing high-latency, loss-prone WANs.
Resilio Connect uses a proprietary WAN acceleration protocol known as Zero Gravity Transport™ (ZGT). ZGT manages network traffic in order to fully utilize any network connection and ensure transfers occur as quickly as possible.
ZGT uses a congestion control algorithm that constantly probes the RTT of a network to identify and maintain the ideal packet send rate and a uniform packet distribution over time. Rather than sending an acknowledgment for each packet receipt, ZGT sends acknowledgments in groups. Each acknowledgment contains info about lost packets, and ZGT retransmits lost packets once per RTT in order to reduce unnecessary retransmission.
ZGT enables Resilio to use and optimize any network connection, no matter how low-quality it is. Resilio works with any type of network, such as VSATs, Wi-Fi, cell (3G, 4G, and 5G), and any IP connection.
It also allows you to use Resilio in remote locations with little/no network connectivity, such as at the edge of networks, at sea, and in countries with underdeveloped connectivity. You can continuously sync data from data centers at the edge to the cloud, then sync those files across all of your cloud and on-prem endpoints.
Further reading: To learn more about Resilio’s WAN acceleration capabilities, read our WAN optimization whitepaper.
Resilio Supports More Cloud and Hybrid-Cloud Use Cases
As discussed in this article, ASM is limited to cloud migration, simple backup, and merging use cases.
Resilio Connect is the best file replication and sync solution for a variety of use cases, such as:
Remote, Hybrid, and Distributed Workforces
Resilio Connect’s N-way sync capabilities make it an ideal replication solution for remote and distributed workforces. When collaborating on files, employees located anywhere in the world can make changes to files and have those changes immediately synced to every other office.
Every office/endpoint with a Resilio agent will have the power of P2P replication and WAN acceleration, so files are reliably updated and synchronized regardless of network quality or outages. Everyone in your organization will always have the most up-to-date versions of files with little/no file conflicts.
You can also use Resilio as an efficient object storage gateway to get low-latency access to files stored in any cloud storage platform. You can manage all files stored on-prem and in the cloud from one location. And employees can access files from a unified interface that operates much like Microsoft OneDrive.
Resilio’s object storage gateway includes several features that allow you to enhance efficiency and productivity better than any other object storage gateway solution, such as:
- Selective Sync: You can control which specific files and folders sync to which specific endpoints, which enables you to share files only with relevant/privileged employees and reduce transfers over your network (as well as data egress fees).
- Selective Caching: Other object storage gateways cache recently accessed files locally. But Resilio enables you to choose which files get cached locally. You can store frequently used files on local devices and store infrequently used files in long-term cloud storage — freeing up space on your on-prem storage.
- Partial downloads: Resilio enables end-users to download files or partially download only the portions of files that they need. Employees get faster access to the files they need and you can reduce data egress costs.
- Automated syncs: You can automate synchronization so employees don’t need to manually sync files and can spend more time on their tasks.
Disaster Recovery and Active Active HA
Resilio is an excellent replication solution for hot-site disaster recovery and cloud disaster recovery.
With one-to-many replication, you can back up your on-prem servers in real time to multiple cloud-based backup sites. Or you can use many-to-many replication to sync data across your entire environment and effectively turn every server into a backup site and configure Active-Active High Availability.
In the event of an outage, every endpoint in your environment can replicate files simultaneously in order to quickly bring your system back online. With Resilio’s real-time, P2P replication and WAN optimization, you can achieve sub-five-second RPOs and RTOs within minutes of an outage.
Server Sync and Software Deployment
Organizations that run websites and applications can use Resilio to keep geographically distributed servers in sync. With Resilio’s blazing-fast replication speeds, updates are immediately synced and available to all users. Resilio’s checkpoint restarts, dynamic rerouting, high availability, and elimination of single points of failure enable websites and apps to meet uptime SLAs and improve the experience for their users.
Software companies can use Resilio’s one-to-many replication to distribute software updates across their servers. Resilio’s P2P architecture and WAN optimization protocol enable you to reduce latency issues in Azure (or any other cloud) and reliably distribute large updates much faster than traditional solutions.
Case Study: VoiceBase Reduces Software Distribution Time by 88%
VoiceBase is a software that performs speech-to-text transcriptions for audio and video. VoiceBase uses Resilio Connect to quickly and reliably distribute software updates (speech model files) of 50+ GB to over 400 production servers.
“Resilio Connect enables us to reliably distribute our code… in a fraction of time. These copy jobs now take an hour, down from eight. Best of all, once Resilio Connect was installed, it just works: We never need to manually intervene in any way.”
Resilio Secures Your Data End-to-End
As a new tool, ASM’s security features are limited. You can secure your Azure resources through RBAC (role-based access control), which lets you manage who has access to your resources and what they can do with them. And you can protect your SMB endpoints by integrating with Azure Key Vault in order to securely store and control access to user credentials and other secrets.
But Resilio Connect includes many built-in features that provide superior protection and keep your data secure end-to-end, such as:
- Cryptographic data integrity validation: Resilio uses cryptographic validation to ensure files arrive at their destination intact.
- Permission controls: You can control who can access specific files and folders, so you can limit access to only privileged employees.
- Mutual authentication: Before initiating transfers, every endpoint must provide an authentication key. This ensures files are only delivered to designated endpoints.
- Forward secrecy: Resilio uses one-time session encryption keys to protect sessions.
- Immutable file copies: Resilio stores immutable copies of files in the cloud to protect against ransomware.
Use Resilio Connect for Hybrid and Multi-Cloud Replication
While ASM is a suitable tool for moving data into Azure in simple scenarios, Resilio Connect is a much better solution for replicating and syncing files across on-premise endpoints and cloud providers, endpoints, and regions. This is because Resilio:
- Can be deployed flexibly on any device, OS, or cloud storage provider.
- Enables centralized management of all cloud and on-prem endpoints from one unified location.
- Provides granular control over every aspect of replication in your environment.
- Uses a P2P replication architecture to provide blazing-fast replication in any direction, while eliminating single points of failure and enabling you to scale organically.
- Utilizes proprietary WAN acceleration technology to optimize transfers over any network (including high-latency WANs), any type of connection (VSATs, cell, Wi-Fi, and any IP connection), and at the edge of networks.
- Keeps data secure end-to-end.
- Supports multiple hybrid and multi-cloud use cases, such as disaster recovery, remote/distributed workforces, web/app server sync, and software update distribution.
Organizations in tech (Cisco, Match), media (Turner Sports, Skywalker Sound), gaming (2K Games, Bungie), retail (KFC, McDonald’s), and more use Resilio Connect to sync and transfer data across cloud and hybrid/multi-cloud environments. To learn more about how Resilio Connect can help you replicate data into and across any cloud storage provider, schedule a demo.
