Welcome!

@BigDataExpo Authors: Elizabeth White, Liz McMillan, ManageEngine IT Matters, Pat Romanski, Rajeev Kozhikkattuthodi

Related Topics: @BigDataExpo, Java IoT, Linux Containers, Open Source Cloud, SDN Journal

@BigDataExpo: Article

Planet @Hadoop By @ABridgwater | @BigDataExpo [#BigData]

Hadoop has been called a foundational technology, rather than ‘just’ a database by some commentators

What Kinds of People Live on Planet Hadoop?

Labor market analytics firm Wanted Analytics recently assessed the market for technology professionals and found that demand for people with proficient levels of Hadoop expertise had skyrocketed by around 33% since last year - it is true, Hadoop is hard technology to master and the labor market is not exactly flooded with an over-abundance of skilled practitioners.

Hadoop has been called a foundational technology, rather than ‘just' a database by some commentators - this almost pushes it towards being an ‘environment' rather than it being a single software product... and this all goes towards making Hadoop even harder to master, many will agree.

Interestingly, we can essentially define Hadoop as a free Java-based ‘programming framework' that supports the processing of large data sets in a distributed computing environment with robust data processing and storage power.

Hadoop confusion
Is Hadoop a foundational technology, a data environment or programming framework? It is all of these things of course, but this may be where some of the challenge (and, specifically, the skills challenge) lies.

What kinds of people live on Planet Hadoop? Well, quite apart from being able to breath a noxious mixture of data, storage and Java-flavored oxygen, Hadoop native beings (we'll call them Hadoopers) are deeply innovative.

Hadoopers are the kind of people that are driven by an inherent desire to push the ‘analytics envelope' so far forward that it becomes an operational business process that in itself drives innovation - and therefore greater profits.

These are the sort of people who are prepared to help ‘industrialize the analytical process' and make it part of the way a firm operates from first principles. As Adrian Jones of SAS has put it, the analytics factory provides an approach that offers flexibility for analysts and a structured framework for governance.

"Your data preparation process should make it easy to quickly move between the two states. This step requires the creativity of the business combined with the process and operational efficiency of IT. Taking a factory approach to data preparation will remove this gray area from many organizations, reducing the conflict that comes through duplications and inefficiencies," writes Jones.

Hard-wired into Hadoop
As operational processes throughout the business start to get hard-wired into Hadoop, we see analytics deployed in a production state where it becomes part of the core workflow of all operations across the business - the truly analytical company starts to flourish from this point onward.

This Hadoop planet (or company, or environment, or biosphere) is populated by people with ability and aptitude of course; Hadoop is no place for dummies unfortunately... but what other traits to Hadoopers exhibit?

"Resourcefully creative with an ability to think on their feet," - would be a good way of describing these folk, i.e., they are capable of surveying the analytics landscape and knowing instinctively which datasets to combine, splice, analyze and focus on - but more than anything, they can perceive the kind of outcome that ‘might' be useful without actually knowing how it should initially be quantified and qualified.

A steely stoicism
Being able to convert the analytic process into an operational process requires an aptitude for practicality and a steely stoicism with the patience to wait and look for the right outcomes.

Really successful Hadoopers are corporate beasts with an appreciation for their company's regulatory processes and systems, i.e., there's not much point in producing great Hadoop analytics if it remains untamed analytics. Good corporate Hadoop citizens are capable of making Hadoop insights ‘universally accessible' so that every stakeholder in the organization gets the appropriate level of access.

As Fiona McNeill, Product Marketing, SAS, puts it, "The difference with analytically mature, innovative organizations, is that insights are universally accessible - whether you work in HR, finance, sales, logistics, marketing or services. The data is recognized as a corporate asset and analytical methods become intellectual property."

Nimble feet, fingers and foreheads
This discussion could go on and on. Good Hadoop pros exhibit many, many attributes, but one that is appropriate to finish with is nimbleness. Although this term ‘nimble' has become hackneyed, overused and overtired across the IT industry, we will bid to use it this one last time.

Hadoop is all about trial and error in terms of finding out what works best in relation to any particular analytics job. Nimble Hadoopers are unyielding in their ability to keep trying out analytics scenarios until they get what they need - or, perhaps more important, until they get what they want for the business.

These are some of the many elements currently populating the prosperous parts of Hadoop, whether we survive in this brave new world or not is dependent on our ability to be this kind of Hadooper - welcome to the new world.

This post is brought to you by SAS.

SAS is a leader in business analytics software and services and the largest independent vendor in the business intelligence market.

More Stories By Adrian Bridgwater

Adrian Bridgwater is a freelance journalist and corporate content creation specialist focusing on cross platform software application development as well as all related aspects software engineering, project management and technology as a whole.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


@BigDataExpo Stories
You know you need the cloud, but you’re hesitant to simply dump everything at Amazon since you know that not all workloads are suitable for cloud. You know that you want the kind of ease of use and scalability that you get with public cloud, but your applications are architected in a way that makes the public cloud a non-starter. You’re looking at private cloud solutions based on hyperconverged infrastructure, but you’re concerned with the limits inherent in those technologies.
"We focus on composable infrastructure. Composable infrastructure has been named by companies like Gartner as the evolution of the IT infrastructure where everything is now driven by software," explained Bruno Andrade, CEO and Founder of HTBase, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
Artificial intelligence, machine learning, neural networks. We’re in the midst of a wave of excitement around AI such as hasn’t been seen for a few decades. But those previous periods of inflated expectations led to troughs of disappointment. Will this time be different? Most likely. Applications of AI such as predictive analytics are already decreasing costs and improving reliability of industrial machinery. Furthermore, the funding and research going into AI now comes from a wide range of com...
In this presentation, Striim CTO and founder Steve Wilkes will discuss practical strategies for counteracting fraud and cyberattacks by leveraging real-time streaming analytics. In his session at @ThingsExpo, Steve Wilkes, Founder and Chief Technology Officer at Striim, will provide a detailed look into leveraging streaming data management to correlate events in real time, and identify potential breaches across IoT and non-IoT systems throughout the enterprise. Strategies for processing massive ...
SYS-CON Events announced today that GrapeUp, the leading provider of rapid product development at the speed of business, will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Grape Up is a software company, specialized in cloud native application development and professional services related to Cloud Foundry PaaS. With five expert teams that operate in various sectors of the market acr...
SYS-CON Events announced today that Ayehu will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on October 31 - November 2, 2017 at the Santa Clara Convention Center in Santa Clara California. Ayehu provides IT Process Automation & Orchestration solutions for IT and Security professionals to identify and resolve critical incidents and enable rapid containment, eradication, and recovery from cyber security breaches. Ayehu provides customers greater control over IT infras...
Internet of @ThingsExpo, taking place October 31 - November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 21st Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The Internet of Things (IoT) is the most profound change in personal and enterprise IT since the creation of the Worldwide Web more than 20 years ago. All major researchers estimate there will be tens of billions devic...
SYS-CON Events announced today that MobiDev, a client-oriented software development company, will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. MobiDev is a software company that develops and delivers turn-key mobile apps, websites, web services, and complex software systems for startups and enterprises. Since 2009 it has grown from a small group of passionate engineers and business...
With major technology companies and startups seriously embracing Cloud strategies, now is the perfect time to attend 21st Cloud Expo October 31 - November 2, 2017, at the Santa Clara Convention Center, CA, and June 12-14, 2018, at the Javits Center in New York City, NY, and learn what is going on, contribute to the discussions, and ensure that your enterprise is on the right path to Digital Transformation.
Join us at Cloud Expo June 6-8 to find out how to securely connect your cloud app to any cloud or on-premises data source – without complex firewall changes. More users are demanding access to on-premises data from their cloud applications. It’s no longer a “nice-to-have” but an important differentiator that drives competitive advantages. It’s the new “must have” in the hybrid era. Users want capabilities that give them a unified view of the data to get closer to customers and grow business. The...
21st International Cloud Expo, taking place October 31 - November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA, will feature technical sessions from a rock star conference faculty and the leading industry players in the world. Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud strategy. Me...
SYS-CON Events announced today that Cloud Academy named "Bronze Sponsor" of 21st International Cloud Expo which will take place October 31 - November 2, 2017 at the Santa Clara Convention Center in Santa Clara, CA. Cloud Academy is the industry’s most innovative, vendor-neutral cloud technology training platform. Cloud Academy provides continuous learning solutions for individuals and enterprise teams for Amazon Web Services, Microsoft Azure, Google Cloud Platform, and the most popular cloud com...
Automation is enabling enterprises to design, deploy, and manage more complex, hybrid cloud environments. Yet the people who manage these environments must be trained in and understanding these environments better than ever before. A new era of analytics and cognitive computing is adding intelligence, but also more complexity, to these cloud environments. How smart is your cloud? How smart should it be? In this power panel at 20th Cloud Expo, moderated by Conference Chair Roger Strukhoff, paneli...
DevOps at Cloud Expo, taking place October 31 - November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 21st Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is no time to w...
The current age of digital transformation means that IT organizations must adapt their toolset to cover all digital experiences, beyond just the end users’. Today’s businesses can no longer focus solely on the digital interactions they manage with employees or customers; they must now contend with non-traditional factors. Whether it's the power of brand to make or break a company, the need to monitor across all locations 24/7, or the ability to proactively resolve issues, companies must adapt to...
SYS-CON Events announced today that CA Technologies has been named "Platinum Sponsor" of SYS-CON's 21st International Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. CA Technologies helps customers succeed in a future where every business - from apparel to energy - is being rewritten by software. From planning to development to management to security, CA creates software that fuels transformation for companies in the applic...
SYS-CON Events announced today that IBM has been named “Diamond Sponsor” of SYS-CON's 21st Cloud Expo, which will take place on October 31 through November 2nd 2017 at the Santa Clara Convention Center in Santa Clara, California.
We build IoT infrastructure products - when you have to integrate different devices, different systems and cloud you have to build an application to do that but we eliminate the need to build an application. Our products can integrate any device, any system, any cloud regardless of protocol," explained Peter Jung, Chief Product Officer at Pulzze Systems, in this SYS-CON.tv interview at @ThingsExpo, held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA
Amazon started as an online bookseller 20 years ago. Since then, it has evolved into a technology juggernaut that has disrupted multiple markets and industries and touches many aspects of our lives. It is a relentless technology and business model innovator driving disruption throughout numerous ecosystems. Amazon’s AWS revenues alone are approaching $16B a year making it one of the largest IT companies in the world. With dominant offerings in Cloud, IoT, eCommerce, Big Data, AI, Digital Assista...
SYS-CON Events announced today that Enzu will exhibit at SYS-CON's 21st Int\ernational Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Enzu’s mission is to be the leading provider of enterprise cloud solutions worldwide. Enzu enables online businesses to use its IT infrastructure to their competitive advantage. By offering a suite of proven hosting and management services, Enzu wants companies to focus on the core of their ...