Click here to close now.

Welcome!

BigDataExpo® Blog Authors: Elizabeth White, Ian Khan, Pat Romanski, Liz McMillan, Carmen Gonzalez

Related Topics: BigDataExpo® Blog, @MicroservicesE Blog, Open Source Cloud, @ContainersExpo Blog, CloudExpo® Blog, Apache

BigDataExpo® Blog: Article

Intel’s Going into the Hadoop Biz

Its reason for joining the Hadoop push is to sell more high-end Xeon server chips into scalable Hadoop clusters

Intel has gone into the open source Apache Hadoop business with its own distribution, which it calls the Intel Distribution for Apache Hadoop or simply the Intel Distribution.

Its reason for joining the Hadoop push is to sell more high-end Xeon server chips into scalable Hadoop clusters by goosing Hadoop's development. It also wants to move solid-state memory and its own networking.

Obviously it thinks the stuff is pretty fundamental.

As part of its effort Intel has added features to the Hadoop widgetry that nobody else has like silicon-based security. That makes Intel's the only Hadoop distribution to include complete encryption.

Since the widgetry supports the AES instructions in the Xeon chip, it's supposed to maximize Big Data performance and, together with the other improvements Intel has added, promises a 40% performance boost.

Intel's got 20 partners supporting the initial launch of its Hadoopery, as the Register calls it, a term we'll happily steal. These partners are supposed to integrate Intel's software into next-generation platforms and solutions and enable deployment in public and private cloud environments.

The partners include Cisco, Dell, Pentaho, Red Hat, SAP, SAS, Savvis, SuperMicro, Tableau, Teradata, Wipro and Zettaset.

Intel says it will open source all but it's most prized Hadoop platform enhancements as well as invest in more R&D to build analytic solutions for Hadoop. Whether this largesse gives rivals like Cloudera, MapR and Hortonworks a leg up remains to be seen. Intel's subscription pricing is supposed to be competitive.

Intel's off-limits proprietary software so far includes Intel Manager for Apache Hadoop software, which is supposed to simplify the deployment, configuration and monitoring of clusters for system administrators as they set up new applications.

Then there's Intel Active Tuner for Apache Hadoop software, which is supposed to take the guesswork out of performance tuning. Intel says until now this required a special understanding of each application's use of system resources along with Hadoop configuration and performance benchmarks. Now it doesn't.

Meanwhile, Intel's started Project Rhino, an open source effort to improve the data protection capabilities of the Hadoop ecosystem. The point is to improve encryption and authentication and make security more granular.

Intel imagines - and no one can gainsay this - that there will be literally tons of information coming from billions of sensors and intelligent systems; it estimates that the world generates a petabyte of data every 11 seconds, the equivalent of 13 years of HD video.

It thinks it can exploit the fact that "only a small fraction of the world is able to extract meaning from all of this information because the technologies, techniques and skills available today are either too rigid for the data types or too expensive to deploy."

And it figures the information derived from Hadoop can enrich our lives, not just make us easy marks for advertisers, by, say, accurately pinpointing customized treatments for terminal diseases.

Intel's distribution is supposed to analyze a terabyte of data, which normally takes more than four hours to fully process, in seven minutes because of its data-crunching chips and software.

Intel has been fooling around with its own Hadoop distribution for a few years ever since it formed a relationship with Yahoo and HP and is now on its third iteration based on the Apache Hadoop stack.

It has done work on the Hadoop Distributed File System, the YARN MapReduce 2.0 distributed processing framework, the Hive SQL query tool and the HBase key-value store.

Intel was reportedly pushed to putting something on the market next quarter by China Unicom and China Mobile, a couple of Chinese telecoms, to help with some performance issues in the Hadoop stack when running on Xeon chips.

Intel will distribute its version of Hadoop, developed in China, through vendors and service providers and sell its own technical support services. It says it won't fork the system.

More Stories By Maureen O'Gara

Maureen O'Gara the most read technology reporter for the past 20 years, is the Cloud Computing and Virtualization News Desk editor of SYS-CON Media. She is the publisher of famous "Billygrams" and the editor-in-chief of "Client/Server News" for more than a decade. One of the most respected technology reporters in the business, Maureen can be reached by email at maureen(at)sys-con.com or paperboy(at)g2news.com, and by phone at 516 759-7025. Twitter: @MaureenOGara

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


@BigDataExpo Stories
The 4th International Internet of @ThingsExpo, co-located with the 17th International Cloud Expo - to be held November 3-5, 2015, at the Santa Clara Convention Center in Santa Clara, CA - announces that its Call for Papers is open. The Internet of Things (IoT) is the biggest idea since the creation of the Worldwide Web more than 20 years ago.
The enterprise market will drive IoT device adoption over the next five years. In his session at @ThingsExpo, John Greenough, an analyst at BI Intelligence, division of Business Insider, will analyze how companies will adopt IoT products and the associated cost of adopting those products. John Greenough is the lead analyst covering the Internet of Things for BI Intelligence- Business Insider’s paid research service. Numerous IoT companies have cited his analysis of the IoT. Prior to joining B...
As the Internet of Things unfolds, mobile and wearable devices are blurring the line between physical and digital, integrating ever more closely with our interests, our routines, our daily lives. Contextual computing and smart, sensor-equipped spaces bring the potential to walk through a world that recognizes us and responds accordingly. We become continuous transmitters and receivers of data. In his session at @ThingsExpo, Andrew Bolwell, Director of Innovation for HP's Printing and Personal S...
SYS-CON Events announced today that BMC will exhibit at SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. BMC delivers software solutions that help IT transform digital enterprises for the ultimate competitive business advantage. BMC has worked with thousands of leading companies to create and deliver powerful IT management services. From mainframe to cloud to mobile, BMC pairs high-speed digital innovation with robust...
The world is at a tipping point where the technology, the device and global adoption are converging to such a point that we will see an explosion of a world where smartphone devices not only allow us to talk to each other, but allow for communication between everything – serving as a central hub from which we control our world – MediaTek is at the heart of both driving this and allowing the markets to drive this reality forward themselves. The next wave of consumer gadgets is here – smart, con...
All major researchers estimate there will be tens of billions devices - computers, smartphones, tablets, and sensors - connected to the Internet by 2020. This number will continue to grow at a rapid pace for the next several decades. With major technology companies and startups seriously embracing IoT strategies, now is the perfect time to attend @ThingsExpo, June 9-11, 2015, at the Javits Center in New York City. Learn what is going on, contribute to the discussions, and ensure that your enter...
SYS-CON Events announced today that MetraTech, now part of Ericsson, has been named “Silver Sponsor” of SYS-CON's 16th International Cloud Expo®, which will take place on June 9–11, 2015, at the Javits Center in New York, NY. Ericsson is the driving force behind the Networked Society- a world leader in communications infrastructure, software and services. Some 40% of the world’s mobile traffic runs through networks Ericsson has supplied, serving more than 2.5 billion subscribers.
The Internet of Things is not only adding billions of sensors and billions of terabytes to the Internet. It is also forcing a fundamental change in the way we envision Information Technology. For the first time, more data is being created by devices at the edge of the Internet rather than from centralized systems. What does this mean for today's IT professional? In this Power Panel at @ThingsExpo, moderated by Conference Chair Roger Strukhoff, panelists will addresses this very serious issue o...
The multi-trillion economic opportunity around the "Internet of Things" (IoT) is emerging as the hottest topic for investors in 2015. As we connect the physical world with information technology, data from actions, processes and the environment can increase sales, improve efficiencies, automate daily activities and minimize risk. In his session at @ThingsExpo, Ed Maguire, Senior Analyst at CLSA Americas, will describe what is new and different about IoT, explore financial, technological and re...
"People are a lot more knowledgeable about APIs now. There are two types of people who work with APIs - IT people who want to use APIs for something internal and the product managers who want to do something outside APIs for people to connect to them," explained Roberto Medrano, Executive Vice President at SOA Software, in this SYS-CON.tv interview at Cloud Expo, held Nov 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA.
SYS-CON Events announced today that O'Reilly Media has been named “Media Sponsor” of SYS-CON's 16th International Cloud Expo®, which will take place on June 9–11, 2015, at the Javits Center in New York City, NY. O'Reilly Media spreads the knowledge of innovators through its books, online services, magazines, and conferences. Since 1978, O'Reilly Media has been a chronicler and catalyst of cutting-edge development, homing in on the technology trends that really matter and spurring their adoption...
There will be 150 billion connected devices by 2020. New digital businesses have already disrupted value chains across every industry. APIs are at the center of the digital business. You need to understand what assets you have that can be exposed digitally, what their digital value chain is, and how to create an effective business model around that value chain to compete in this economy. No enterprise can be complacent and not engage in the digital economy. Learn how to be the disruptor and not ...
An effective way of thinking in Big Data is composed of a methodical framework for dealing with the predicted shortage of 50-60% of the qualified Big Data resources in the U.S. This holistic model comprises the scientific and engineering steps that are involved in accelerating Big Data solutions: problem, diagnosis, facts, analysis, hypothesis, solution, prototype and implementation. In his session at Big Data Expo®, Tony Shan focused on the concept, importance, and considerations for each of t...
The web app is Agile. The REST API is Agile. The testing and planning are Agile. But alas, Data infrastructures certainly are not. Once an application matures, changing the shape or indexing scheme of data often forces at best a top down planning exercise and at worst includes schema changes which force downtime. The time has come for a new approach that fundamentally advances the agility of distributed data infrastructures. Come learn about a new solution to the problems faced by software orga...
With major technology companies and startups seriously embracing IoT strategies, now is the perfect time to attend @ThingsExpo in Silicon Valley. Learn what is going on, contribute to the discussions, and ensure that your enterprise is as "IoT-Ready" as it can be! Internet of @ThingsExpo, taking place Nov 3-5, 2015, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 17th Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading in...
Can the spatial component of your Big Data be harnessed and visualized, adding another dimension of power and analytics to your data? In his session at Big Data Expo®, John Meza, Product Engineer and Performance Engineering Team Lead at Esri, discussed the spatial queries that can be used within the Hadoop ecosystem and their integration with GeoSpatial applications. The GIS Tools for Hadoop project was also discussed and its implementation to discover location-based patterns and relationships...
Cloud and Big Data present unique dilemmas: embracing the benefits of these new technologies while maintaining the security of your organization's assets. When an outside party owns, controls and manages your infrastructure and computational resources, how can you be assured that sensitive data remains private and secure? How do you best protect data in mixed use cloud and big data infrastructure sets? Can you still satisfy the full range of reporting, compliance and regulatory requirements? In...
Are your Big Data initiatives resulting in Big Impact or Big Mess? In her session at Big Data Expo, Penelope Everall Gordon, Emerging Technology Strategist at 1Plug Corporation, shared her successes in improving Big Decision outcomes by building stories compelling to the target audience – and her failures when she lost sight of the plotline, distracted by the glitter of technology and the lure of buried insights. The cast of characters includes the agency head [city official? elected official?...
Storage administrators find themselves walking a line between meeting employees’ demands to use public cloud storage services, and their organizations’ need to store information on-premises for security, performance, cost and compliance reasons. However, as file sharing protocols like CIFS and NFS continue to lose their relevance, simply relying only on a NAS-based environment creates inefficiencies that hurt productivity and the bottom line. IT wants to implement cloud storage it can purchase a...
The emergence of cloud computing and Big Data warrants a greater role for the PMO to successfully manage enterprise transformation driven by these powerful trends. As the adoption of cloud-based services continues to grow, a governance model is needed to orchestrate enterprise cloud implementations and harness the power of Big Data analytics. In his session at Cloud Expo, Mahesh Singh, President of BigData, Inc., discussed how the Enterprise PMO takes center stage not only in developing the app...