Rather than search for this in Quora or the web and end up with outdated or poorly researched numbers. I’ll keep track of it here. (last updated 5/29/14)
Earlier this year, I noticed a post claiming that Amazon S3 was storing more than 262 billion objects. That blew me away, as just a year before, Amazon had claimed 102 billion objects. 150% year over year growth!
S3 is now on an exponential growth curve, passing 566 billion objects in October, and it is often cited as a cloud success story. But, what’s driving this growth you may ask? Primarily an application that lets consumers access their files across their tablets, smartphones and PCs: Dropbox.
S3 is great technology, but without an application, it is strictly the province of developers who need a place to put bits. A consumer application like Dropbox leverages the Amazon cloud without users ever knowing it. As you can see from the chart below, if Dropbox’s own claim of reaching 100 billion files is to be believed, they drove nearly 50% of S3’s growth between March 2010 and March 2011.
If you dig a little deeper, you’ll notice the discrepancy between Dropbox claiming 100 billion files, and Amazon claiming hundreds of billions of objects. Amazon protects files in the cloud by creating multiple copies and distributing them in different locations. It typically offers protection against the “concurrent loss of data in two separate storage facilities”, which means it keeps at least 3 copies of each file for its standard service. If one were to interpret this literally, Dropbox would make up nearly all of S3’s growth (see the green line):
This seems a bit unbelievable, as I have to think that the Amazon Cloud Drive, which automatically stores every digital purchase that people make from the Amazon store, is at least making some impression. But the possibility of a single application having so much influence on storage is eye-opening (but it shouldn’t be surprising – storage experts need only look back at Oracle Financials and Microsoft Exchange for enterprise examples of applications driving storage).
The broader storage market should pay attention. “Big Data” is all the rage, but much like Dropbox is driving petabytes of S3 storage for consumers, business mobility is primed to become the dominant driver of enterprise file storage systems.
– Leo (@lleung)