|By Tom Major||
|February 22, 2013 09:00 AM EST||
I have been working for technology companies for 33 years now, so I don't know why I'm always surprised at the technology myths that proliferate. For example, there is a popular notion running around that Solid State Drives (SSDs) will replace Hard Disk Drives (HDDs) as the dominant storage media. So, which will win? Let me give you my answer upfront before I take you through my arguments: both win, and the market size for both continues to grow.
As I offer up this data and this perspective, I'll repeat some advice I received early in my career. A mentor once told me, when searching for the truth in business and technology "look to the economics" (maybe my version of "go to the mattresses" for you "Godfather" fans).
Here is what I read and hear on a regular basis regarding the battle of HDDs and SSDs:
- SSDs run circles around HDDs for performance
- SSDs will soon replace HDDs
- The improving density of SSDs will collapse the current cost premium to HDDs
Sound familiar? Well, as with most claims of this ilk, there are some elements of truth to them but each of these statements goes too far when viewed in light of some data.
Yes, it's true that SSDs are fundamentally faster than HDDs. This is particularly true of random I/O data reading (as with databases). With large block sequential data (i.e., rich media like video), however, this difference tends to be very small.
Here's a claim where the data just doesn't add up. While the attractiveness of the SSD technology would lead one to believe this, the economics really don't hold up. I should probably point more specifically to NAND flash here (the "solid state" in SSDs) because another solid state technology might actually achieve this, but that appears to be years and tens of billions of dollars away.
Let's talk for a minute about the world's demand for storage. Last year approximately 2500 exabytes of data was created and/or replicated. And...this is doubling about every two years. This needs to be serviced largely by HDDs and SSDs. Last year, the NAND flash industry produced somewhere between 30 and 40 exabytes of storage; with somewhere around 3.5 exabytes finding its way into SSDs (the balance in phones, tablets, cameras, etc.). What is that, 1.4% of our total storage need? But production is ramping up... at a capital cost of about $1.5 billion per Exabyte (semiconductor fabs are expensive)! So, from an available supply perspective, SSDs replacing HDDs seems implausible.
Improving Technology Makes SSDs Cheaper
Agree. The NAND flash technology point is at 21nm line widths with plans to move to 19, then 16 nm. Storage density is further improving with the use of multi-level cell (MLC) capabilities vs single-level cell (SLC). This is bringing down the cost of solid state storage in much the same way the areal density increases seen in HDDs brought down the cost of hard drive storage.
Notice how I said "brought" down (i.e., past tense)? Because a real density growth in the hard drive industry has slowed to a crawl, the rapid erosion of cost/Gb has also slowed to a crawl. Now, the HDD industry needs to move to its next technology (HAMR?) to continue to take cost out and HAMR is a number of years away. HAMR will require significant capital investment by the HDD companies. Significant capital investment will be required anyway to keep up with storage demand (even more so with slowing areal density growth). This all spells out a flattening of HDD costs for the foreseeable future. Some would speculate that a renewed interest in improving the utilization of HDD capacity is an artifact of these economics.
Here's what's preventing the complete collapse of the price difference between the two technologies. The NAND flash suppliers have a similar problem as the HDD manufacturers... the implications of shrinking technology geometries. As the line widths shrink, the ability of flash to sustain multiple write-erase cycles declines. To make up for this deficiency, sophisticated error-correction algorithms and "brute force" overprovisioning (to allow some cells to wear out) are being used. This and the above mentioned semiconductor fab costs tend to mitigate the progress SSDs are making in closing the cost gap to HDDs.
Having made my arguments that SSDs won't take over the world, I have to say that IT solutions need this technology. I think you can see that SSDs aren't the answer to all our storage needs, but they allow us to address a crying storage requirement. At a high level, storage is called on to produce two key deliverables:
- Make data available to an application or user in an appropriate timeframe (i.e., performance) and in today's environment this need is growing.
- Store data reliably (i.e., capacity) and in today's environment this need is growing
By and large, to date, systems with hard drives have been architected to deliver both of these capabilities. It's hard to argue that hard drives haven't done a good job of delivering affordable capacity. But to deliver against the performance requirements of IT solutions, hard drives:
- Have been developed with higher speeds, but arguably little progress has been made in the last ten years
- Have been "short stroked" (a technique limiting the stroke of the actuator to improve performance)
- Have been grouped together to allow striping of data across a large number of drives to aggregate their performance.
With each of these approaches the user suffers from higher power requirements, and with the last two approaches their system has been overprovisioned (leaving stranded capacity) to deliver performance. This all adds up to significant system cost that can be avoided with new storage architectures.
Here's where SSDs come in. SSDs are proving to be a technology answer to the new generation of storage needs, both the growing performance and the growing capacity requirements. Here's the simple, logical way to think about SSDs and HDDs and their role in storage solutions. Use the right tool for the job. That is, take advantage of SSDs for performance (particularly small block, random I/O) and HDDs for capacity.
You might challenge me now and ask, "Does adding SSDs (Flash) to IT solutions make economic sense?" SSDs are expensive, but, used appropriately, SSDs can also minimize the number of HDDs required in a given solution. The secret is in optimizing the use of both, that is, avoid overprovisioning of both. Thin provisioning of capacity has become popular as a cost saver. Similarly, thin provisioning of performance is a similar cost saver. (This is a benefit of virtualized caching or tiering capabilities of newer storage solutions.)
I'll close with a few proof points. Look to the latest economically sensible storage solutions that are answering today's performance and capacity calls. The vast majority of affordable "Ultrabooks" incorporate both flash and hard drive technology, as does Apple's latest "Fusion" drive.
Growth hacking is common for startups to make unheard-of progress in building their business. Career Hacks can help Geek Girls and those who support them (yes, that's you too, Dad!) to excel in this typically male-dominated world. Get ready to learn the facts: Is there a bias against women in the tech / developer communities? Why are women 50% of the workforce, but hold only 24% of the STEM or IT positions? Some beginnings of what to do about it! In her Day 2 Keynote at 17th Cloud Expo, San...
Nov. 27, 2015 01:00 AM EST Reads: 537
Apps and devices shouldn't stop working when there's limited or no network connectivity. Learn how to bring data stored in a cloud database to the edge of the network (and back again) whenever an Internet connection is available. In his session at 17th Cloud Expo, Ben Perlmutter, a Sales Engineer with IBM Cloudant, demonstrated techniques for replicating cloud databases with devices in order to build offline-first mobile or Internet of Things (IoT) apps that can provide a better, faster user e...
Nov. 27, 2015 12:45 AM EST Reads: 391
Cloud computing delivers on-demand resources that provide businesses with flexibility and cost-savings. The challenge in moving workloads to the cloud has been the cost and complexity of ensuring the initial and ongoing security and regulatory (PCI, HIPAA, FFIEC) compliance across private and public clouds. Manual security compliance is slow, prone to human error, and represents over 50% of the cost of managing cloud applications. Determining how to automate cloud security compliance is critical...
Nov. 26, 2015 04:00 PM EST Reads: 382
Internet of @ThingsExpo, taking place June 7-9, 2016 at Javits Center, New York City and Nov 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with the 18th International @CloudExpo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world and ThingsExpo New York Call for Papers is now open.
Nov. 26, 2015 03:30 PM EST Reads: 525
With major technology companies and startups seriously embracing IoT strategies, now is the perfect time to attend @ThingsExpo 2016 in New York and Silicon Valley. Learn what is going on, contribute to the discussions, and ensure that your enterprise is as "IoT-Ready" as it can be! Internet of @ThingsExpo, taking place Nov 3-5, 2015, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 17th Cloud Expo and will feature technical sessions from a rock star conference faculty ...
Nov. 26, 2015 03:15 PM EST Reads: 521
We are rapidly moving to a brave new world of interconnected smart homes, cars, offices and factories known as the Internet of Things (IoT). Sensors and monitoring devices will touch every part of our lives. Let's take a closer look at the Internet of Things. The Internet of Things is a worldwide network of objects and devices connected to the Internet. They are electronics, sensors, software and more. These objects connect to the Internet and can be controlled remotely via apps and programs. ...
Nov. 26, 2015 02:15 PM EST Reads: 499
Today air travel is a minefield of delays, hassles and customer disappointment. Airlines struggle to revitalize the experience. GE and M2Mi will demonstrate practical examples of how IoT solutions are helping airlines bring back personalization, reduce trip time and improve reliability. In their session at @ThingsExpo, Shyam Varan Nath, Principal Architect with GE, and Dr. Sarah Cooper, M2Mi’s VP Business Development and Engineering, explored the IoT cloud-based platform technologies driving t...
Nov. 26, 2015 01:00 PM EST Reads: 392
As organizations shift towards IT-as-a-service models, the need for managing & protecting data residing across physical, virtual, and now cloud environments grows with it. CommVault can ensure protection & E-Discovery of your data - whether in a private cloud, a Service Provider delivered public cloud, or a hybrid cloud environment – across the heterogeneous enterprise.
Nov. 26, 2015 11:30 AM EST Reads: 121
In recent years, at least 40% of companies using cloud applications have experienced data loss. One of the best prevention against cloud data loss is backing up your cloud data. In his General Session at 17th Cloud Expo, Sam McIntyre, Partner Enablement Specialist at eFolder, presented how organizations can use eFolder Cloudfinder to automate backups of cloud application data. He also demonstrated how easy it is to search and restore cloud application data using Cloudfinder.
Nov. 26, 2015 11:00 AM EST Reads: 125
The Internet of Things (IoT) is growing rapidly by extending current technologies, products and networks. By 2020, Cisco estimates there will be 50 billion connected devices. Gartner has forecast revenues of over $300 billion, just to IoT suppliers. Now is the time to figure out how you’ll make money – not just create innovative products. With hundreds of new products and companies jumping into the IoT fray every month, there’s no shortage of innovation. Despite this, McKinsey/VisionMobile data...
Nov. 26, 2015 11:00 AM EST Reads: 445
As organizations realize the scope of the Internet of Things, gaining key insights from Big Data, through the use of advanced analytics, becomes crucial. However, IoT also creates the need for petabyte scale storage of data from millions of devices. A new type of Storage is required which seamlessly integrates robust data analytics with massive scale. These storage systems will act as “smart systems” provide in-place analytics that speed discovery and enable businesses to quickly derive meaningf...
Nov. 26, 2015 09:30 AM EST Reads: 379
In his keynote at @ThingsExpo, Chris Matthieu, Director of IoT Engineering at Citrix and co-founder and CTO of Octoblu, focused on building an IoT platform and company. He provided a behind-the-scenes look at Octoblu’s platform, business, and pivots along the way (including the Citrix acquisition of Octoblu).
Nov. 26, 2015 09:00 AM EST Reads: 479
In his General Session at 17th Cloud Expo, Bruce Swann, Senior Product Marketing Manager for Adobe Campaign, explored the key ingredients of cross-channel marketing in a digital world. Learn how the Adobe Marketing Cloud can help marketers embrace opportunities for personalized, relevant and real-time customer engagement across offline (direct mail, point of sale, call center) and digital (email, website, SMS, mobile apps, social networks, connected objects).
Nov. 26, 2015 08:45 AM EST Reads: 270
The Internet of Everything is re-shaping technology trends–moving away from “request/response” architecture to an “always-on” Streaming Web where data is in constant motion and secure, reliable communication is an absolute necessity. As more and more THINGS go online, the challenges that developers will need to address will only increase exponentially. In his session at @ThingsExpo, Todd Greene, Founder & CEO of PubNub, exploreed the current state of IoT connectivity and review key trends and t...
Nov. 26, 2015 06:45 AM EST Reads: 406
Too often with compelling new technologies market participants become overly enamored with that attractiveness of the technology and neglect underlying business drivers. This tendency, what some call the “newest shiny object syndrome” is understandable given that virtually all of us are heavily engaged in technology. But it is also mistaken. Without concrete business cases driving its deployment, IoT, like many other technologies before it, will fade into obscurity.
Nov. 26, 2015 03:00 AM EST Reads: 313
With all the incredible momentum behind the Internet of Things (IoT) industry, it is easy to forget that not a single CEO wakes up and wonders if “my IoT is broken.” What they wonder is if they are making the right decisions to do all they can to increase revenue, decrease costs, and improve customer experience – effectively the same challenges they have always had in growing their business. The exciting thing about the IoT industry is now these decisions can be better, faster, and smarter. Now ...
Nov. 26, 2015 02:00 AM EST Reads: 201
The Internet of Things is clearly many things: data collection and analytics, wearables, Smart Grids and Smart Cities, the Industrial Internet, and more. Cool platforms like Arduino, Raspberry Pi, Intel's Galileo and Edison, and a diverse world of sensors are making the IoT a great toy box for developers in all these areas. In this Power Panel at @ThingsExpo, moderated by Conference Chair Roger Strukhoff, panelists discussed what things are the most important, which will have the most profound...
Nov. 26, 2015 01:30 AM EST Reads: 442
In today's enterprise, digital transformation represents organizational change even more so than technology change, as customer preferences and behavior drive end-to-end transformation across lines of business as well as IT. To capitalize on the ubiquitous disruption driving this transformation, companies must be able to innovate at an increasingly rapid pace. Traditional approaches for driving innovation are now woefully inadequate for keeping up with the breadth of disruption and change facin...
Nov. 26, 2015 01:00 AM EST Reads: 445
Discussions of cloud computing have evolved in recent years from a focus on specific types of cloud, to a world of hybrid cloud, and to a world dominated by the APIs that make today's multi-cloud environments and hybrid clouds possible. In this Power Panel at 17th Cloud Expo, moderated by Conference Chair Roger Strukhoff, panelists addressed the importance of customers being able to use the specific technologies they need, through environments and ecosystems that expose their APIs to make true ...
Nov. 26, 2015 12:00 AM EST Reads: 501
There are over 120 breakout sessions in all, with Keynotes, General Sessions, and Power Panels adding to three days of incredibly rich presentations and content. Join @ThingsExpo conference chair Roger Strukhoff (@IoT2040), June 7-9, 2016 in New York City, for three days of intense 'Internet of Things' discussion and focus, including Big Data's indespensable role in IoT, Smart Grids and Industrial Internet of Things, Wearables and Consumer IoT, as well as (new) IoT's use in Vertical Markets.
Nov. 25, 2015 12:00 PM EST Reads: 534