Welcome!

@BigDataExpo Authors: Yeshim Deniz, Carmen Gonzalez, Jnan Dash, Liz McMillan, Pat Romanski

Related Topics: @BigDataExpo, @CloudExpo, @ThingsExpo

@BigDataExpo: Blog Feed Post

Better Predictors | @CloudExpo @Schmarzo #BigData #IoT #AI #ML #Analytics

I love the simplicity of the data science concepts as taught by the book 'Moneyball'

Data Science: Identifying Variables That Might Be Better Predictors

I love the simplicity of the data science concepts as taught by the book “Moneyball.” Everyone wants to jump right into the real meaty, highly technical data science books. But I recommend to my students to start with the book “Moneyball.” The book does a great job of making the power of data science come to life (and the movie doesn’t count, as my wife saw it and “Brad Pitt is so cute!” was her only takeaway…ugh). One of my favorite lessons out of the book is the definition of data science:

Data Science is about identifying those variables and metrics that might be better predictors of performance

This straightforward definition sets the stage for defining the roles and responsibilities of the business stakeholders and the data science team:

  • Business stakeholders are responsible for identifying (brainstorming) those variables and metrics that might be better predictors of performance, and
  • The Data Science team is responsible for quantifying which variables and metrics actually are better predictors of performance

This approach takes advantage of what the business stakeholders know best – which is the business. And this approach takes advantage of what the data science team knows best – which is data transformation, data enrichment, data exploration and analytic modeling. The perfect data science team!

Note: the word “might” is probably the most important word in the data science definition. Business stakeholders must feel comfortable brainstorming different variables and metrics that might be better predictors of performance without feeling like their ideas will be judged. Some of our best ideas come from people whose voices typically don’t get heard. Our Big Data Vision Workshop process considers all ideas to be worthy of consideration. If you do not embrace that concept, then you risk constraining the creative thinking of the business stakeholders, or worse, miss out on surfacing potentially valuable data insight.

This blog serves to expand on the approach that the data science team uses to identify (and quantify) which variables and metrics are better predictors of performance. Let me walk through an example.

We recently had an engagement with a financial services organization where we were asked to predict customer attritors; that is, to identify which customers were at-risk of ending their relationship with the organization. As we typically do in a Big Data Vision Workshop, we held facilitated brainstorming sessions with the business stakeholders to identify those variables and metrics that might be better predictors of performance (see Figure 1).

Figure 1: Brainstorming the Variables and Metrics that Might Be Better Predictors

Note: I had to blur the exact metrics that we identified for client reasons of competitive advantage. Yea, I like that!

From this list of variables and metrics, the data science team sought to create an “Attrition Score” that can be used to identify (or score) at-risk customers. The data science team embraced the iterative, “fail fast / learn faster” process in testing different combinations of variables and metrics.   The data science team tested different data enrichment and transformation techniques and different analytic algorithms with different combinations of the variables and metrics to see which combinations of variables yielded the best results (see Figure 2).

Figure 2: Exploring Different Combinations of Variables and Metrics

The challenge for the data science team is to not settle on the first model that “works.” The data science teams needs to constantly push the envelope and as a result, fail enough in their testing of different combinations of variables to feel personally confident in the results of the final model.

After much testing and failing – and testing and failing – and testing and failing, the data science team came up with an “Attrition Score” model that had failed enough times for them to feel confident about its results (see Figure 3).

Figure 3: Identifying Variables and Metrics that ARE Better Predictors

We needed an approach that got the best out of everyone on the project – the business stakeholders brainstorming variables and metrics, and the data science team creatively testing different combinations. The final results in this engagement were quite impressive (see Figure 4):

Figure 4: Final Attrition Model Results

The creative data science process of combining different variables and metrics is highly dependent upon the success of the business stakeholder brainstorming exercises. If the business stakeholders are not brought into this process early and allowed to think creatively about what variables and metrics might be better predictors of performance, then the collection of variables and metrics that the data science team will seek to test will be limited. Put another way, the success of the data science process and the creation of the actionable score is highly dependent upon the creative involvement of the business stakeholders at the beginning of the process.

And that’s the power of our Big Data Vision Workshop process.

——————–

If you are interested in learning more about the Dell EMC Big Data Vision Workshop, check out the blogs below:

The post Data Science: Identifying Variables That Might Be Better Predictors appeared first on InFocus Blog | Dell EMC Services.

More Stories By William Schmarzo

Bill Schmarzo, author of “Big Data: Understanding How Data Powers Big Business”, is responsible for setting the strategy and defining the Big Data service line offerings and capabilities for the EMC Global Services organization. As part of Bill’s CTO charter, he is responsible for working with organizations to help them identify where and how to start their big data journeys. He’s written several white papers, avid blogger and is a frequent speaker on the use of Big Data and advanced analytics to power organization’s key business initiatives. He also teaches the “Big Data MBA” at the University of San Francisco School of Management.

Bill has nearly three decades of experience in data warehousing, BI and analytics. Bill authored EMC’s Vision Workshop methodology that links an organization’s strategic business initiatives with their supporting data and analytic requirements, and co-authored with Ralph Kimball a series of articles on analytic applications. Bill has served on The Data Warehouse Institute’s faculty as the head of the analytic applications curriculum.

Previously, Bill was the Vice President of Advertiser Analytics at Yahoo and the Vice President of Analytic Applications at Business Objects.

@BigDataExpo Stories
SYS-CON Events announced today that CA Technologies has been named "Platinum Sponsor" of SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, New York, and 21st International Cloud Expo, which will take place in November in Silicon Valley, California.
The 20th International Cloud Expo has announced that its Call for Papers is open. Cloud Expo, to be held June 6-8, 2017, at the Javits Center in New York City, brings together Cloud Computing, Big Data, Internet of Things, DevOps, Containers, Microservices and WebRTC to one location. With cloud computing driving a higher percentage of enterprise IT budgets every year, it becomes increasingly important to plant your flag in this fast-expanding business opportunity. Submit your speaking proposal ...
SYS-CON Events announced today that Juniper Networks (NYSE: JNPR), an industry leader in automated, scalable and secure networks, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Juniper Networks challenges the status quo with products, solutions and services that transform the economics of networking. The company co-innovates with customers and partners to deliver automated, scalable and secure network...
New competitors, disruptive technologies, and growing expectations are pushing every business to both adopt and deliver new digital services. This ‘Digital Transformation’ demands rapid delivery and continuous iteration of new competitive services via multiple channels, which in turn demands new service delivery techniques – including DevOps. In this power panel at @DevOpsSummit 20th Cloud Expo, moderated by DevOps Conference Co-Chair Andi Mann, panelists will examine how DevOps helps to meet th...
SYS-CON Events announced today that Juniper Networks (NYSE: JNPR), an industry leader in automated, scalable and secure networks, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Juniper Networks challenges the status quo with products, solutions and services that transform the economics of networking. The company co-innovates with customers and partners to deliver automated, scalable and secure network...
DevOps is being widely accepted (if not fully adopted) as essential in enterprise IT. But as Enterprise DevOps gains maturity, expands scope, and increases velocity, the need for data-driven decisions across teams becomes more acute. DevOps teams in any modern business must wrangle the ‘digital exhaust’ from the delivery toolchain, "pervasive" and "cognitive" computing, APIs and services, mobile devices and applications, the Internet of Things, and now even blockchain.
SYS-CON Events announced today that Hitachi, the leading provider the Internet of Things and Digital Transformation, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Hitachi Data Systems, a wholly owned subsidiary of Hitachi, Ltd., offers an integrated portfolio of services and solutions that enable digital transformation through enhanced data management, governance, mobility and analytics. We help globa...
SYS-CON Events announced today that T-Mobile will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. As America's Un-carrier, T-Mobile US, Inc., is redefining the way consumers and businesses buy wireless services through leading product and service innovation. The Company's advanced nationwide 4G LTE network delivers outstanding wireless experiences to 67.4 million customers who are unwilling to compromise on ...
20th Cloud Expo, taking place June 6-8, 2017, at the Javits Center in New York City, NY, will feature technical sessions from a rock star conference faculty and the leading industry players in the world. Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud strategy.
With major technology companies and startups seriously embracing IoT strategies, now is the perfect time to attend @ThingsExpo 2016 in New York. Learn what is going on, contribute to the discussions, and ensure that your enterprise is as "IoT-Ready" as it can be! Internet of @ThingsExpo, taking place June 6-8, 2017, at the Javits Center in New York City, New York, is co-located with 20th Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry p...
With major technology companies and startups seriously embracing Cloud strategies, now is the perfect time to attend @CloudExpo | @ThingsExpo, June 6-8, 2017, at the Javits Center in New York City, NY and October 31 - November 2, 2017, Santa Clara Convention Center, CA. Learn what is going on, contribute to the discussions, and ensure that your enterprise is on the right path to Digital Transformation.
SYS-CON Events announced today that SoftLayer, an IBM Company, has been named “Gold Sponsor” of SYS-CON's 18th Cloud Expo, which will take place on June 7-9, 2016, at the Javits Center in New York, New York. SoftLayer, an IBM Company, provides cloud infrastructure as a service from a growing number of data centers and network points of presence around the world. SoftLayer’s customers range from Web startups to global enterprises.
SYS-CON Events announced today that Hitachi, the leading provider the Internet of Things and Digital Transformation, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Hitachi Data Systems, a wholly owned subsidiary of Hitachi, Ltd., offers an integrated portfolio of services and solutions that enable digital transformation through enhanced data management, governance, mobility and analytics. We help globa...
Blockchain is a shared, secure record of exchange that establishes trust, accountability and transparency across supply chain networks. Supported by the Linux Foundation's open source, open-standards based Hyperledger Project, Blockchain has the potential to improve regulatory compliance, reduce cost and time for product recall as well as advance trade. Are you curious about Blockchain and how it can provide you with new opportunities for innovation and growth? In her session at 20th Cloud Exp...
In his keynote at 19th Cloud Expo, Sheng Liang, co-founder and CEO of Rancher Labs, discussed the technological advances and new business opportunities created by the rapid adoption of containers. With the success of Amazon Web Services (AWS) and various open source technologies used to build private clouds, cloud computing has become an essential component of IT strategy. However, users continue to face challenges in implementing clouds, as older technologies evolve and newer ones like Docker c...
NHK, Japan Broadcasting, will feature the upcoming @ThingsExpo Silicon Valley in a special 'Internet of Things' and smart technology documentary that will be filmed on the expo floor between November 3 to 5, 2015, in Santa Clara. NHK is the sole public TV network in Japan equivalent to the BBC in the UK and the largest in Asia with many award-winning science and technology programs. Japanese TV is producing a documentary about IoT and Smart technology and will be covering @ThingsExpo Silicon Val...
The age of Digital Disruption is evolving into the next era – Digital Cohesion, an age in which applications securely self-assemble and deliver predictive services that continuously adapt to user behavior. Information from devices, sensors and applications around us will drive services seamlessly across mobile and fixed devices/infrastructure. This evolution is happening now in software defined services and secure networking. Four key drivers – Performance, Economics, Interoperability and Trust ...
DevOps is often described as a combination of technology and culture. Without both, DevOps isn't complete. However, applying the culture to outdated technology is a recipe for disaster; as response times grow and connections between teams are delayed by technology, the culture will die. A Nutanix Enterprise Cloud has many benefits that provide the needed base for a true DevOps paradigm.
SYS-CON Events announced today that CollabNet, a global leader in enterprise software development, release automation and DevOps solutions, will be a Bronze Sponsor of SYS-CON's 20th International Cloud Expo®, taking place from June 6-8, 2017, at the Javits Center in New York City, NY. CollabNet offers a broad range of solutions with the mission of helping modern organizations deliver quality software at speed. The company’s latest innovation, the DevOps Lifecycle Manager (DLM), supports Value S...
"Peak 10 is a national cloud data center solutions managed services provider, and part of that is disaster recovery. We see a growing trend in the industry where companies are coming to us looking for assistance in their DR strategy," stated Andrew Cole, Director of Solutions Engineering at Peak 10, in this SYS-CON.tv interview at 18th Cloud Expo, held June 7-9, 2016, at the Javits Center in New York City, NY.