Welcome!

@BigDataExpo Authors: Elizabeth White, Shelly Palmer, Pat Romanski, Karthick Viswanathan, Yeshim Deniz

Related Topics: @DevOpsSummit, Java IoT, Containers Expo Blog, @CloudExpo, @BigDataExpo, SDN Journal

@DevOpsSummit: Article

The DevOps Database | Part 2

Applying Systems Thinking to Database Change Management

In my first post in this series, I discussed the underpinning principles of all DevOps patterns as eloquently stated by Gene Kim, author of "The Phoenix Project."  In this post I'd like to dig a little deeper into The First Way.  As a refresher:

The First Way: Systems Thinking - This Way stresses the performance of the entire system of value delivery.  Instead of becoming laser focused on the part of the process for which an individual or team is responsible, the individual or team works to understand the entire process from requirements generation to customer delivery.  The goal is to eliminate the delivery impediments that arise when a project transitions from one isolated silo to another.  Understanding the entire system allows business, development, and operations to work towards a common goal in a consistent manner.

When we started Datical, our first step was to perform extensive market validation.  We quickly learned that database schema management was going to be tough nut to crack. In the scores of conversations we had with people throughout the ALM spectrum, we learned that the process for managing and updating the database schema that supports an application was at best murky and at worst a black box.  So how do we elevate a process owned and understood by a few people in an organization to the level of visibility required to understand that process as part of a larger system?

What follows is a list of concepts pertaining to System Thinking that we've rallied around in providing our database change management solution Datical DB. Instead of the black box, database change management can become a transparent and flexible part of your value delivery system.

Start With Reality
When beginning a new project based on previous development, don't rely on a stack of SQL scripts on a file server or even in a source code control system. As you know, databases evolve over time and sometimes out of process changes happen.  When a database schema is modified to resolve an error condition or performance degradation, these alterations are usually handled in a support ticketing system.  You can never be certain that they were added to the stack of scripts used to build out a fresh environment.  In light of this, any database change management solution should start by generating a baseline from the working system: your production schema.  This ensures that design and development activity are taking into account not only those schema objects generated in Dev, but also those which originated in other stops in the system.

Don't Script! Model!
To keep this blog post short(er) I'll point you now to a blog post I wrote a few months ago on modelling vs. scripting.

Version Twice, Deploy Once
The first level of versioning any database change management tool should provide is versioning of the gold standard. Versioning the scripts or model that you use to build a fresh database instance for your application provides a ton of benefits.  You can track how your schema has evolved over time and across releases, you can tightly couple a schema definition to the version of the application it supports using the same branch/tag/merge workflow that you use for your application code, and you can quickly stand up a new database instance that you know is correct for any released version or experimental branch you are working in.  The second level of versioning takes place in each database instance.  Tracking the version of the schema deployed on a specific instance makes deployment and troubleshooting much easier.  Will this application build work with the schema on this instance?  What changes need to be applied to this database to catch it up to the latest version?  Were the right changes applied to this test database to validate a closed defect?  If you are tracking the individual changes applied in a database and the impetus for those changes, these questions can be answered quickly and easily.

Unify Your Modes of Delivery
We've found that a lot of uncertainty was generated when database updates were handled by several individuals using their own tools and methods to affect the required changes.  To remove this uncertainty, all database change must be executed using the same tools and processes across departments and individuals.   The tricky part: a continuous integration system has different requirements than a developer working iteratively or a DBA processing a batch of changes in a headless environment during a maintenance window.  In order to unify your database change process, the solution you use should be accessible to all of these individuals while maintaining the consistency of deployment activities.  That's why Datical DB provides a rich GUI experience for developers that helps them craft & deploy changes in dev environments; tight integrations with popular build and release automation frameworks that preserve the frameworks' workflows while providing Datical DB functionality; and a command line interface that allows users in headless environments to deploy changes in the exact same manner that the GUI or a 3rd-party integration would.  By unifying the modes of delivery you are constantly testing your release practices. By the time the production push rolls around, your system works.

More Stories By Pete Pickerill

Pete Pickerill is Vice President of Products and Co-founder of Datical. Pete is a software industry veteran who has built his career in Austin’s technology sector. Prior to co-founding Datical, he was employee number one at Phurnace Software and helped lead the company to a high profile acquisition by BMC Software, Inc. Pete has spent the majority of his career in successful startups and the companies that acquired them including Loop One (acquired by NeoPost Solutions), WholeSecurity (acquired by Symantec, Inc.) and Phurnace Software.

@BigDataExpo Stories
SYS-CON Events announced today that N3N will exhibit at SYS-CON's @ThingsExpo, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. N3N’s solutions increase the effectiveness of operations and control centers, increase the value of IoT investments, and facilitate real-time operational decision making. N3N enables operations teams with a four dimensional digital “big board” that consolidates real-time live video feeds alongside IoT sensor data a...
Internet of @ThingsExpo, taking place October 31 - November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 21st Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The Internet of Things (IoT) is the most profound change in personal and enterprise IT since the creation of the Worldwide Web more than 20 years ago. All major researchers estimate there will be tens of billions devic...
SYS-CON Events announced today that SourceForge has been named “Media Sponsor” of SYS-CON's 21st International Cloud Expo, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. SourceForge is the largest, most trusted destination for Open Source Software development, collaboration, discovery and download on the web serving over 32 million viewers, 150 million downloads and over 460,000 active development projects each and every month.
Today traditional IT approaches leverage well-architected compute/networking domains to control what applications can access what data, and how. DevOps includes rapid application development/deployment leveraging concepts like containerization, third-party sourced applications and databases. Such applications need access to production data for its test and iteration cycles. Data Security? That sounds like a roadblock to DevOps vs. protecting the crown jewels to those in IT.
SYS-CON Events announced today that Interface Corporation will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Interface Corporation is a company developing, manufacturing and marketing high quality and wide variety of industrial computers and interface modules such as PCIs and PCI express. For more information, visit http://www.i...
SYS-CON Events announced today that mruby Forum will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. mruby is the lightweight implementation of the Ruby language. We introduce mruby and the mruby IoT framework that enhances development productivity. For more information, visit http://forum.mruby.org/.
What is the best strategy for selecting the right offshore company for your business? In his session at 21st Cloud Expo, Alan Winters, U.S. Head of Business Development at MobiDev, will discuss the things to look for - positive and negative - in evaluating your options. He will also discuss how to maximize productivity with your offshore developers. Before you start your search, clearly understand your business needs and how that impacts software choices.
SYS-CON Events announced today that Cedexis will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Cedexis is the leader in data-driven enterprise global traffic management. Whether optimizing traffic through datacenters, clouds, CDNs, or any combination, Cedexis solutions drive quality and cost-effectiveness.
SYS-CON Events announced today that Mobile Create USA will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Mobile Create USA Inc. is an MVNO-based business model that uses portable communication devices and cellular-based infrastructure in the development, sales, operation and mobile communications systems incorporating GPS capabi...
SYS-CON Events announced today that MIRAI Inc. will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. MIRAI Inc. are IT consultants from the public sector whose mission is to solve social issues by technology and innovation and to create a meaningful future for people.
SYS-CON Events announced today that Keisoku Research Consultant Co. will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Keisoku Research Consultant, Co. offers research and consulting in a wide range of civil engineering-related fields from information construction to preservation of cultural properties. For more information, vi...
SYS-CON Events announced today that Massive Networks, that helps your business operate seamlessly with fast, reliable, and secure internet and network solutions, has been named "Exhibitor" of SYS-CON's 21st International Cloud Expo ®, which will take place on Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. As a premier telecommunications provider, Massive Networks is headquartered out of Louisville, Colorado. With years of experience under their belt, their team of...
SYS-CON Events announced today that Enroute Lab will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Enroute Lab is an industrial design, research and development company of unmanned robotic vehicle system. For more information, please visit http://elab.co.jp/.
SYS-CON Events announced today that Ryobi Systems will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Ryobi Systems Co., Ltd., as an information service company, specialized in business support for local governments and medical industry. We are challenging to achive the precision farming with AI. For more information, visit http:...
21st International Cloud Expo, taking place October 31 - November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA, will feature technical sessions from a rock star conference faculty and the leading industry players in the world. Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud strategy. Me...
Real IoT production deployments running at scale are collecting sensor data from hundreds / thousands / millions of devices. The goal is to take business-critical actions on the real-time data and find insights from stored datasets. In his session at @ThingsExpo, John Walicki, Watson IoT Developer Advocate at IBM Cloud, will provide a fast-paced developer journey that follows the IoT sensor data from generation, to edge gateway, to edge analytics, to encryption, to the IBM Bluemix cloud, to Wa...
SYS-CON Events announced today that Fusic will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Fusic Co. provides mocks as virtual IoT devices. You can customize mocks, and get any amount of data at any time in your test. For more information, visit https://fusic.co.jp/english/.
The “Digital Era” is forcing us to engage with new methods to build, operate and maintain applications. This transformation also implies an evolution to more and more intelligent applications to better engage with the customers, while creating significant market differentiators. In both cases, the cloud has become a key enabler to embrace this digital revolution. So, moving to the cloud is no longer the question; the new questions are HOW and WHEN. To make this equation even more complex, most ...
SYS-CON Events announced today that B2Cloud will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. B2Cloud specializes in IoT devices for preventive and predictive maintenance in any kind of equipment retrieving data like Energy consumption, working time, temperature, humidity, pressure, etc.
SYS-CON Events announced today that NetApp has been named “Bronze Sponsor” of SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. NetApp is the data authority for hybrid cloud. NetApp provides a full range of hybrid cloud data services that simplify management of applications and data across cloud and on-premises environments to accelerate digital transformation. Together with their partners, NetApp em...