There’s a recent survey by IDC in which the vast majority of enterprise respondents name IBM as the vendor able to most effectively provide Infrastructure as a Service (IaaS). Surprisingly (sort of), the megascale public cloud providers, Google, Microsoft and Amazon Web Services come in 5th, 6th and 7th respectively. As a former AWS employee and cloud analyst who firmly believes that a public cloud with essentially unlimited scale, relentless consistency and automated metered service is the “real” cloud, I generally agree with the sentiment that the old line IT companies like IBM and HP have fallen way behind. But from a business strategy and perception viewpoint, it might be a different story.
I was having a conversation the other day about Dropbox, and the topic turned to how much storage individuals had purchased over the years. I decided to take a look.
The best source I could find was Western Digital’s quarterly fact sheet, quite a wealth of storage numbers. Using their numbers to extrapolate the rest of the market, I calculated that over 325 million external drives have been shipped with a total of nearly 200 exabytes of capacity over the last five years.
That’s 66,667 times the amount of digital data stored by the Library of Congress.
That’s 100 times what the new NSA data center can store.
Bold statement? I was a bit surprised at the conclusion myself. But, let’s take a deeper look.
I’ve obviously made certain choices regarding storage, and have done my own comparisons at certain points in time, but was prompted to update my thoughts after a recent tweet by Chris Mims of the WSJ:
Strong words from Chris, but they ring true. It’s easy to forget that your device can be stolen, and your typical protections (e.g. a lot of folks don’t even bother to set a device login) can be circumvented way more easily than a well-designed, well-funded cloud storage service. Historically, according to the Open Security Foundation, reported exposures of personally identifiable information (PII) resulting from lost or stolen devices and media made up more than 22% of all incidents, while hacking made up 30%. The numbers are comparable. While the percentage of incidents related to hacking has increased dramatically in the last year, the um, Target has been easier prey like retailers.
Looking more broadly than security, I wondered if a cloud storage service could actually be better than storage on a device.
Most people I know have at least three devices. A laptop, a tablet, and a smartphone. Some have two of each, one for home, and one for work. If you’ve gone all Apple or all Android, the systems aren’t totally fragmented and disjointed, only partially so. Any deviation from either ecosystem, either at the device, app, or cloud service, and you start running into trouble.
I think it’s because the vendors don’t really know what we want.
They are learning the new style of computing at the same time we are. The cloud app vendors like Evernote and Dropbox are way ahead of the device and OS vendors, and it shows.
Data storage is a massive market ($22.5 billion for the 2013 high end storage market alone), and has reached a critical point in its evolution, highlighted by dramatic changes in base technologies, interfaces, and service models. EMC’s recent acquisition of DSSD was another thought-provoking data point.
- Hardware commoditization and component improvements have been a big part of the story. Magnetic storage has continued its relentless increase in density (Kryder’s Law), albeit with an extended pause in price declines due to a supply constraint from the Thai floods. Solid state storage is making significant inroads into enterprise and cloud data centers, pushing massive valuations for both enterprise flash vendors ($3 billion for Pure Storage), and flash-based cloud vendors ($153 million for Digital Ocean). Acceptance of x86 architectures for high scale storage systems, paired with open source hardware designs and scale-out software, has reached a pinnacle. Quanta Computer’s emergence as a direct system supplier and Red Hat’s acquisition of Ceph professional services firm Inktank are relevant data points there.
EMC kicked off EMC World 2014 with the announcement of their acquisition of DSSD, a shadowy company founded by eccentric geniuses from Sun Microsystems: Andy Bechtolsheim, Jeff Bonwick and Bill Moore. DSSD has been working for three years to accelerate data access and organization for huge datasets by building object storage capabilities directly onto a custom chipset. This is a fascinating story, and can be seen as another step in the struggle of vendors to own information intelligence and speed access to exabytes of data being created.
Higher resolution images from my LinkedIn post: The Game of Clouds: Why everyone wants your data
Silicon Valley received another jolt this morning as Facebook again paid billions to acquire a
nasal nascent technology. This time, it was the Digital Scent Technology company Scentsitive, which recently raised $15,041 on Kickstarter. Scentsitive has a prototype “scent printer,” which currently produces a range of fermented scents ranging from balsamic vinegar to chou doufu. During the announcement, Facebook CEO Mark Zuckerberg, who was accompanied by the freshly “printed” smell of aged gouda cheese (a personal fav), proclaimed:
Scent is absolutely the next great social platform. It will change (a times violently) who we work, play, and communicate with.
Scentsitive, a 2.5 person shop, will be run as a separate company and be left to “innovate independently.” Scentsitive CEO Brent Whiffley said, “It became clear to our team, and our pet dog, that Facebook was the right place for us to grow and realize our, er, vision.”
A spokesperson from Givaudan, creator of such masterpieces as Calvin Klein’s Obsession and J’Adore, praised the Facebook move, calling it “inspired. The emotional pleasure of fragrance, if targeted, would be like Eros firing advertisements straight into the prefrontal cortex.”
No word on when hearing, taste, touch, or the rumored sixth sense will be appropriated.
Happy April Fool’s Day!
Like most folks, we are now buried under the weight of multiple generations of digital music, photo and video. In our case, 13 years of it.
We’ve had 2 generations of digital cameras, a digital camcorder (remember those?), 2 generations of smartphones, Apple AAC format, MP3s, DVDs, Blu-rays, and digital movies purchased from multiple different stores. We’ve had five generations of Mac OS X and countless versions of iTunes and iPhoto. And we’ve never had the time and patience to figure out an end-to-end strategy to manage all the content we’ve generated.
Businesses usually spend the time to address this for their data. They try to centralize the data to make it easier for different users to access it. They manage the centralized data through its lifecycle so that it is faster to access when it is most needed, protected so it is quickly retrievable when it is most valuable, and replicated in a secondary site in case of disasters.
Having worked in the storage market for about ten years, I knew what needed to be done, but the data kept growing, and my “free time” kept shrinking. Fortunately, the tools have gotten significantly better (with some caveats). A relatively low key end of year vacation was just the thing I needed.
First, here’s how my environment looked: