Welcome!

@BigDataExpo Authors: Pat Romanski, Elizabeth White, Liz McMillan, Rene Buest, Kevin Benedict

Related Topics: @BigDataExpo, @CloudExpo, @ThingsExpo

@BigDataExpo: Blog Feed Post

Connecting Big Data Project Management with Enterprise Data Strategy By @DDMcD | @BigDataExpo #BigData

Making the data analysis process effective and efficient is where good project planning and management come in

The Tip of the Spear II: Connecting Big Data Project Management with Enterprise Data Strategy
By Dennis D. McDonald

“If data analysis is Big Data’s "tip of the spear" when it comes to delivering data-dependent value to customers or clients, we also must address how that spear is shaped, sharpened, aimed, and thrown – and, of course, whether or not it hits its intended target.”

Introduction
In Meeting the Mission of Transportation Safety, Richard McKinney, U.S. Department of Transportation's CIO, describes four components for what I call an “enterprise data strategy”:

  1. Data governance
  2. Data sharing
  3. Data standards
  4. Data analysis

He also mentions additional factors relevant to DOT’s data strategy:

  1. The volume of data is increasing and we need to be ready for it.
  2. Managing data is not the same as analyzing it.
  3. We need to be thinking now about what type of analysis we need to be doing and what resources will be needed to do the analysis.

bdpm

Based on the 20+ personal, telephone, and email interviews I’ve conducted so far[2] as part of my big data project management research I would add a fourth item to McKinney's list:

  1. We need to spend at least as much time to planning and managing the people and business processes that make data analysis possible as we do the analysis process itself and the technologies that support it.

Tip of the Spear
If data analysis is Big Data’s “tip of the spear” when it comes to delivering data-dependent value to customers or clients, we also must address how that spear is shaped, sharpened, aimed, and thrown – and, of course, whether or not it hits its intended target.

We also want the processes associated with throwing that spear to be both effective and efficient.

Making the data analysis process – the tip of the Big Data spear -- effective and efficient is where good project planning and management come in.  Challenges to doing this in connection with data intensive projects are identifiable and include:

  1. Siloes. Data are often generated and managed in system- or mission-specific siloes. As a result, creating and implementing an effective enterprise-level data strategy that rises above and encompasses multiple programs, systems, and/or missions requires not just data analysis skills but a mix of technical, organizational, and political skills – not just good “project management.”
  2. Sharing. Making data accessible and useful often means that data need to be shared with systems and processes outside the control of those who "own" the data to be analyzed. Key steps in sharing data are that (a) data need to be identified and inventoried, and (b) technical and business ownership of the inventories data must be determined. In many organizations this inventorying is easier said than done and may require both manual and automated approaches to creating the necessary inventories.
  3. Standards. Efficient and sustainable analysis of data and metadata may require development or implementation of data standards. Existence and use of such standards differs by industry, data type, and system. The costs for developing and adopting standards to facilitate data sharing and analysis will also vary and may have cost and schedule implications at the project, program, enterprise, and industry or community levels.
  4. Delivering value. Modern data analysis tools and techniques provide mechanisms to identify patterns and trends from the increasing volumes of data generated by a steadily widening variety of data capture mechanisms. Challenges in predicting what will be found when data are analyzed places a premium on making sure we are asking the right questions. This in turn impacts our ability to justify project expenditures in advance.

Portfolio Management
Responding to the above challenges requires not only project management skills but also a project planning process that takes into consideration alignment with an organization’s goals and objectives.

As one of my interviewees suggested, the challenge faced in complex “big data” projects has just as much – if not more -- to do with overall strategy and “portfolio management” as with how individual projects are planned and managed. Effectively designing and governing a portfolio of projects and processes requires not only an understanding of how the portfolio supports (relates to, is aligned with, interacts with) the organization’s objectives; it should also incorporate a rational process for defining project requirements and then governing how the organization’s resources are managed and applied.

Given how pervasive and fundamental data are to an organization’s operation, skill in data science and analytics is a necessary element but such skill will not be, in many cases, a guarantor of success. Technical and analytical skills must be accompanied by effective planning, oversight, and management in order to ensure that the data analysis “spear” is being thrown in the right direction.

Delivering Value Quickly
Ideally a portfolio of projects will support an organization’s strategic plan and the goals or missions the organization is charged with pursuing. We may also need to “get tactical” by delivering value to the customer or client as quickly as possible, perhaps by focusing on better-controlled and better-understood product-centric data early on via a “data lake” approach.

Doing so will be good for the customer and will help create a relationship of trust moving forward. Such a relationship will be needed when complications or uncertainties arise and need to be dealt with.

In organizations that are not historically “data centric” or in organizations where management and staff have a low level of data literacy, an early demonstration of value from data analysis is especially important. An agile approach to project management, accompanied by openness, transparency, and collaboration, will help to accomplish this.

Unfortunately, challenges such as those identified above in many cases cannot be addressed effectively in tactically focused short-term projects given the usual pressures of time and budget. Such challenges can be complex or rooted in how the organization has been traditionally structured and managed.

Still, it’s not unusual for a tactically-focused “sprint” project, even while delivering an effective model or other deliverable, to uncover the need for a more global (or strategic) approach to managing data, metadata, data security, privacy, or data quality.

Balancing Tactics and Strategy
When focusing on delivery of useful data-related deliverables it always pays to keep two questions in mind:

  1. What needs to be done immediately to make data useful?
  2. What does this tell us about what needs to be done more globally in order to maintain and increase data usefulness?

Attention to enterprise-level data strategy while delivering useful results in the short term has implications beyond what is being attempted in an individual project’s scope. Treating data as an enterprise resource may even require changes to how the enterprise itself is managed. As we all know, it’s not unusual for change to be resisted.

An effective enterprise level data strategy will be one that balances the management of a portfolio of individual data intensive “agile” projects with parallel development of an upgraded enterprise data strategy. Doing one without the other could have negative consequences, for example:

  1. Focusing only on a narrowly defined data intensive analytics project by itself may generate immediate value through frequent useful deliverables but may not address underlying technical process issues that impact long-term efficiency and sustainability.
  2. Focusing only on an enterprise data strategy without delivering tactical benefits reduces the possibility that that less data-savvy managers understand the “big picture” down the road.

As experienced project managers know, concentrating on “quick and dirty” or “low hanging fruit” when under the gun to deliver value to a client in the short term can generate short term benefits. This same approach, however, may actually increase costs over time if strategic data management issues related to data standards or quality are repeatedly kicked “down the road.” Also, delivering a “strategy” without also engaging users in development of real-world analytical deliverables might mean that strategically important recommendations ends up gathering dust on the shelf somewhere.

Communication Strategy
As experienced project managers understand all too well one of the most important elements in effective project management is communication:

  • Communication among project staff
  • Communication with the client
  • Communication with stakeholders

In the case of the big data or data intensive project, even when focused on delivering incremental value to the customer by focusing initially on specific or narrowly targeted goals, we want communications about project activities, especially among key stakeholders, to focus both on tactical as well as strategic objectives.

This may require accommodating a variety of communication styles as well as different levels of data and analytical literacy especially when both business-focused and technology- or analytics-focused staff are involved. But if we do follow this balanced approach we will:

  1. Deliver a useful project.
  2. Develop a trusted relationship with the client.
  3. Build the foundation for a realistic sustainable enterprise data strategy going forward.

Summary
In summary, how a data-intensive project is planned must take into account both short- and long-term goals. This planning process must be a collaborative one and, even if led by the organization’s IT department – not an unusual situation – it must involve business or operating units right from the start in order to ensure success.

I’ll be turning my attention to this planning process in future posts. If you’re interested in learning more about this process please let me know.

Related reading:

[1] Copyright (c) 2015 by Dennis D. McDonald, Ph.D. Dennis is an independent Washington DC area management consultant. His services include preproposal research and analysis, proposal development and costing, marketing and sales support, project and program management, project plan development, requirements analysis, and strategic planning. Reach him by phone at 703-402-7382 or by email at ddmcd@outlook.com. An earlier version of this post was published at http://www.ddmcd.com/spear.html and distributed at the Dec. 8, 2015 ATARC Federal Big Data Summit in Washington, DC.

[2] Thanks are due the following for sharing their thoughts with me: Aldo Bello, Kirk Borne, Clive Boulton, Doug Brockway, Ana Ferreras, Keith Gates, Douglas Glenn, Jennifer Goodwin, Jason Hare, Christina Ho, Randy Howard, Catherine Ives, Ian Kalin, Michael Kaplan, Jim Lola, David McClure, Jim McLennan, Trevor Monroe, Brian Pagels, John Parkinson, Dan Ruggles, Nelson Searles, Sankar Subramanian, and Tom Suder.

Read the original blog entry...

More Stories By Bob Gourley

Bob Gourley writes on enterprise IT. He is a founder and partner at Cognitio Corp and publsher of CTOvision.com

@BigDataExpo Stories
SYS-CON Events announced today that App2Cloud will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct. 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. App2Cloud is an online Platform, specializing in migrating legacy applications to any Cloud Providers (AWS, Azure, Google Cloud).
Cloud resources, although available in abundance, are inherently volatile. For transactional computing, like ERP and most enterprise software, this is a challenge as transactional integrity and data fidelity is paramount – making it a challenge to create cloud native applications while relying on RDBMS. In his session at 21st Cloud Expo, Claus Jepsen, Chief Architect and Head of Innovation Labs at Unit4, will explore that in order to create distributed and scalable solutions ensuring high availa...
Internet-of-Things discussions can end up either going down the consumer gadget rabbit hole or focused on the sort of data logging that industrial manufacturers have been doing forever. However, in fact, companies today are already using IoT data both to optimize their operational technology and to improve the experience of customer interactions in novel ways. In his session at @ThingsExpo, Gordon Haff, Red Hat Technology Evangelist, shared examples from a wide range of industries – including en...
Detecting internal user threats in the Big Data eco-system is challenging and cumbersome. Many organizations monitor internal usage of the Big Data eco-system using a set of alerts. This is not a scalable process given the increase in the number of alerts with the accelerating growth in data volume and user base. Organizations are increasingly leveraging machine learning to monitor only those data elements that are sensitive and critical, autonomously establish monitoring policies, and to detect...
To get the most out of their data, successful companies are not focusing on queries and data lakes, they are actively integrating analytics into their operations with a data-first application development approach. Real-time adjustments to improve revenues, reduce costs, or mitigate risk rely on applications that minimize latency on a variety of data sources. Jack Norris reviews best practices to show how companies develop, deploy, and dynamically update these applications and how this data-first...
Intelligent Automation is now one of the key business imperatives for CIOs and CISOs impacting all areas of business today. In his session at 21st Cloud Expo, Brian Boeggeman, VP Alliances & Partnerships at Ayehu, will talk about how business value is created and delivered through intelligent automation to today’s enterprises. The open ecosystem platform approach toward Intelligent Automation that Ayehu delivers to the market is core to enabling the creation of the self-driving enterprise.
SYS-CON Events announced today that Grape Up will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct. 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Grape Up is a software company specializing in cloud native application development and professional services related to Cloud Foundry PaaS. With five expert teams that operate in various sectors of the market across the U.S. and Europe, Grape Up works with a variety of customers from emergi...
You know you need the cloud, but you’re hesitant to simply dump everything at Amazon since you know that not all workloads are suitable for cloud. You know that you want the kind of ease of use and scalability that you get with public cloud, but your applications are architected in a way that makes the public cloud a non-starter. You’re looking at private cloud solutions based on hyperconverged infrastructure, but you’re concerned with the limits inherent in those technologies.
SYS-CON Events announced today that Massive Networks will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Massive Networks mission is simple. To help your business operate seamlessly with fast, reliable, and secure internet and network solutions. Improve your customer's experience with outstanding connections to your cloud.
Because IoT devices are deployed in mission-critical environments more than ever before, it’s increasingly imperative they be truly smart. IoT sensors simply stockpiling data isn’t useful. IoT must be artificially and naturally intelligent in order to provide more value In his session at @ThingsExpo, John Crupi, Vice President and Engineering System Architect at Greenwave Systems, will discuss how IoT artificial intelligence (AI) can be carried out via edge analytics and machine learning techn...
Everything run by electricity will eventually be connected to the Internet. Get ahead of the Internet of Things revolution and join Akvelon expert and IoT industry leader, Sergey Grebnov, in his session at @ThingsExpo, for an educational dive into the world of managing your home, workplace and all the devices they contain with the power of machine-based AI and intelligent Bot services for a completely streamlined experience.
Existing Big Data solutions are mainly focused on the discovery and analysis of data. The solutions are scalable and highly available but tedious when swapping in and swapping out occurs in disarray and thrashing takes place. The resolution for thrashing through machine learning algorithms and support nomenclature is through simple techniques. Organizations that have been collecting large customer data are increasingly seeing the need to use the data for swapping in and out and thrashing occurs ...
SYS-CON Events announced today that Datera, that offers a radically new data management architecture, has been named "Exhibitor" of SYS-CON's 21st International Cloud Expo ®, which will take place on Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Datera is transforming the traditional datacenter model through modern cloud simplicity. The technology industry is at another major inflection point. The rise of mobile, the Internet of Things, data storage and Big...
SYS-CON Events announced today that GrapeUp, the leading provider of rapid product development at the speed of business, will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Grape Up is a software company, specialized in cloud native application development and professional services related to Cloud Foundry PaaS. With five expert teams that operate in various sectors of the market acr...
In the enterprise today, connected IoT devices are everywhere – both inside and outside corporate environments. The need to identify, manage, control and secure a quickly growing web of connections and outside devices is making the already challenging task of security even more important, and onerous. In his session at @ThingsExpo, Rich Boyer, CISO and Chief Architect for Security at NTT i3, discussed new ways of thinking and the approaches needed to address the emerging challenges of security i...
Blockchain is a shared, secure record of exchange that establishes trust, accountability and transparency across business networks. Supported by the Linux Foundation's open source, open-standards based Hyperledger Project, Blockchain has the potential to improve regulatory compliance, reduce cost as well as advance trade. Are you curious about how Blockchain is built for business? In her session at 21st Cloud Expo, René Bostic, Technical VP of the IBM Cloud Unit in North America, will discuss th...
SYS-CON Events announced today that CA Technologies has been named "Platinum Sponsor" of SYS-CON's 21st International Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. CA Technologies helps customers succeed in a future where every business - from apparel to energy - is being rewritten by software. From planning to development to management to security, CA creates software that fuels transformation for companies in the applic...
In his opening keynote at 20th Cloud Expo, Michael Maximilien, Research Scientist, Architect, and Engineer at IBM, discussed the full potential of the cloud and social data requires artificial intelligence. By mixing Cloud Foundry and the rich set of Watson services, IBM's Bluemix is the best cloud operating system for enterprises today, providing rapid development and deployment of applications that can take advantage of the rich catalog of Watson services to help drive insights from the vast t...
Cloud adoption is often driven by a desire to increase efficiency, boost agility and save money. All too often, however, the reality involves unpredictable cost spikes and lack of oversight due to resource limitations. In his session at 20th Cloud Expo, Joe Kinsella, CTO and Founder of CloudHealth Technologies, tackled the question: “How do you build a fully optimized cloud?” He will examine: Why TCO is critical to achieving cloud success – and why attendees should be thinking holistically ab...
Internet of @ThingsExpo, taking place October 31 - November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 21st Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The Internet of Things (IoT) is the most profound change in personal and enterprise IT since the creation of the Worldwide Web more than 20 years ago. All major researchers estimate there will be tens of billions devic...