Early in my career, when I was still running BitTorrent, I would frequently give talks highlighting the incredible scaling challenges that exist at the edge of the network.
To make the point, I liked to riff on a popular report published by Cisco that was called the ExaByte Era. It documented the exponential growth in data expected on the network a few years hence (today the same bit of Cisco marketing is called the Zettabyte Era).
Let’s put it in perspective
I would start each talk with a little thought exercise to put the challenge in perspective. For kicks, I’ll share the exercise below, but as you read the questions, try to keep in mind that this was waaaaaaay back in 2007—a Precambrian era of the Internet so distant that at the time, the only network capable of delivering Netflix was the Post Office. In each talk, I’d start by dryly informing the audience that “…an ExaByte was over a billion GigaBytes or 2^60 bytes…” and smile inwardly as the eyes staring back at me invariably start to glaze over.
And that’s when I’d hit ‘em with a Pop Quiz containing a bunch of questions like these:
- How long would it take to watch 1 ExaByte (EB) of Standard Definition content encoded at 1Mps?
- How long would it take to watch 1 EB of Blu-ray content (54Mbps)?
- How big would your data center network have to be to deliver 1 EB in a year?
- How long does it take YouTube to stream 1 EB of video?
- How long does it take BitTorrent to deliver 1 EB of data?
Cue the Jeopardy theme song.
After letting these sink in for a bit, I’d begin revealing the answers one by one.
To watch an ExaByte of video at 1Mbps (without bathroom breaks) would take over 292,000 years. If that’s too long, you could sit in front of the Blu-ray player for 5,400 years instead. At the macro level, you’d need 3.5Tbps of sustained network capacity to deliver 1 EB in a year. No small feat considering at the time, all of YouTube required 20 months (1.7 years) to deliver that much video.
But in those days, BitTorrent routinely delivered an EB of data—every single month—over networks at the edge that were unable to handle even simple Netflix. And we did it with software, a team of 20 people and nary a single server, let alone a data center or the cloud.
Powerful stuff, and the only viable technology capable of scaling at the edge.
Yesterday’s realties still apply today
Obviously these talks described an Internet barely capable of ExaByte scale in its entirety and only if you had the right solution.
And if instead you preferred to use the Cloud, then you needed Google’s budget to pull it off.
Roughly 10 years later, an ExaByte is still quite a bit of data to deliver over networks and scaling challenges at the Edge that are no less daunting.
And yet, some of our largest customers are already operating at or near ExaByte scale using Resilio Connect and growing fast.
For each of them, the ExaByte may be different: from daily builds of the latest video game under development, to mountains of raw data needed for training AI algorithms, or the most recent operating system updates to secure your commercial vehicle fleet to large data delivered to systems in tens of thousands of retail restaurants.
None of these customers sees this data getting smaller over time, and yet they know Resilio will scale seamlessly no matter what they throw at us.
We’ve been doing it for years.
Because whatever the data, reaching ExaByte scale is easier with the right technology in place.
