Welcome!

@BigDataExpo Authors: Liz McMillan, Ed Featherston, Yeshim Deniz, Pat Romanski, Elizabeth White

Related Topics: @DevOpsSummit, Java IoT, Linux Containers, @CloudExpo, @BigDataExpo, SDN Journal

@DevOpsSummit: Article

Chicken or Egg: Abstraction and Automation

But will abstraction necessarily lead to automation?

For many SDN and DevOps enthusiasts, the natural outcome of this wave of technological change is a highly-automated network that is well-orchestrated with surrounding systems and applications. One of the prevailing thoughts is that this level of automation is a well-formed abstraction layer. With the abstractions in place, the army of network engineers will be unencumbered by device configuration, and automation will ensure.

Or will it?

First off, let me say that abstraction is absolutely necessary. There is no doubt that networking will only advance if we can both remove unnecessary elements and simplify those that remain. We have to accomplish this in a way that is vendor (and ideally technology) agnostic. Abstraction is clearly the path forward.

But will abstraction necessarily lead to automation?
For the vast majority of network engineers who are designing and actively managing networks today, automation means writing shell or Perl scripts. The scripts themselves qualify as automation insofar as they remove keystrokes, but they basically execute the same serial logic that has dominated networking devices for decades.

When you want to make a switch or a router do something, you specify some configuration. Then you specify some other configuration. And again and again until you get through the litany of parameters that collectively make the device work. This workflow is so ingrained in our collective psyche that we inherently serialize the tasks required to make the network work.

There have actually been companies that have done a decent job of breaking the habit of serialized configuration. Juniper’s flagship operating system Junos moved to a more code-like representation of configuration, making no assumptions about the ordering of specific tasks. But our training runs deep, and even the Juniper guys will tell you that the biggest barrier to entry is familiarity with the UI.

We are addicted to our serialized behavior.
One of the side effects of highly-serialized configuration is that we tend to think extremely linearly and transactionally. There are a lot of network engineers for whom any kind of object-oriented approach is almost too foreign to really embrace. So when they try to automate tasks, they fall back into a sequencing of steps, repeated as many times as necessary. Automation without reuse is painfully difficult to propagate beyond only the most repetitive tasks.

And so we end up in a scenario where automation is basically synonymous with scripting, and where the value is largely applied only to the most frequently-executed tasks.

Where could automation take us?
If we think through where automation could take us, we ideally aim a little higher. Automation could mean the automated exchange of data between collaborative systems in support of some task. For instance, you might want your servers to communicate to your network so that when a new application is spun up (or a VM moves), you get corresponding policy changes, firewall or load balancer changes, and potentially network capacity allocation.

For most network engineers, the idea that infrastructure communicates and dynamically provides a service is science fiction. Our serialized mode of operation simply doesn’t support this kind of multilateral communication. Even if the abstractions remove some of the configuration complexity, the mental block is around sequencing.

If the current networking model has taught us anything, it should be that our network engineers are quite capable of managing tons of inputs and outputs. Now, whether that ought to be a requirement for the job is another question entirely. But as a group, network engineers are certainly capable of handling a lot of variables. That abstraction reduces these variables to the most meaningful is very interesting, but it wouldn’t seem that input management is the biggest bottleneck to automation.

The barrier to automation

Rather, the biggest barrier to automation is that workflow is so structured. First, it was the devices themselves that forced the structure. Then it was the processes (ITIL anyone?) that forced it. The end result is that we have built a discipline so dependent on structure that it actually impedes our own progress.

If we want to get to automation, we need to find a way to work around—or perhaps work within—this structure.

What we are really talking about is changing how we think about provisioning and managing a network. Why do you think there is so much angst when people talk about network engineers needing to learn to code? It’s because moving from a serialized set of steps to an object-oriented way of thinking about the problem is extremely difficult.

People aren’t pushing back because learning a new language is hard. Or at least they shouldn’t be. Look at any networking device configuration and tell me that you aren’t already a master coder. The biggest difference is that you you are using an interpreted language called Cisco CLI (or Junos CLI or whatever CLI).

What next?
What we need to do is bite off small (dare I say tiny) workflow elements, automate those, and then string them together into larger workflows. This implies a couple of things. First, we need to think less about discrete capabilities and more about how they exist within some broader workflow context. Second, we need to understand how these building blocks fit together. It’s the connecting of individual workflows that forms the basis for automation, and those connections highlight the pieces of information that flow across workflow boundaries.

More bluntly, the data that stitches together workflows ends up being the stuff that needs to be in an abstraction. It very well could be that getting the automation parts right will help us get to better abstractions.

Obviously we have to work the process from both ends – abstraction down, and workflow up. I don’t think it’s as simple as one or the other, which is why abstraction and automation might be a networking incarnation of the age-old chicken-and-egg question.

[Today’s fun fact: A parrot’s vocabulary is generally no more than twenty words. Who knew parrot’s and politicians had so much in common?]

The post Chicken or egg: Abstraction and automation appeared first on Plexxi.

More Stories By Michael Bushong

The best marketing efforts leverage deep technology understanding with a highly-approachable means of communicating. Plexxi's Vice President of Marketing Michael Bushong has acquired these skills having spent 12 years at Juniper Networks where he led product management, product strategy and product marketing organizations for Juniper's flagship operating system, Junos. Michael spent the last several years at Juniper leading their SDN efforts across both service provider and enterprise markets. Prior to Juniper, Michael spent time at database supplier Sybase, and ASIC design tool companies Synopsis and Magma Design Automation. Michael's undergraduate work at the University of California Berkeley in advanced fluid mechanics and heat transfer lend new meaning to the marketing phrase "This isn't rocket science."

@BigDataExpo Stories
24Notion is full-service global creative digital marketing, technology and lifestyle agency that combines strategic ideas with customized tactical execution. With a broad understand of the art of traditional marketing, new media, communications and social influence, 24Notion uniquely understands how to connect your brand strategy with the right consumer. 24Notion ranked #12 on Corporate Social Responsibility - Book of List.
Businesses are struggling to manage the information flow and interactions between all of these new devices and things jumping on their network, and the apps and IT systems they control. The data businesses gather is only helpful if they can do something with it. In his session at @ThingsExpo, Chris Witeck, Principal Technology Strategist at Citrix, will discuss how different the impact of IoT will be for large businesses, expanding how IoT will allow large organizations to make their legacy ap...
DevOps at Cloud Expo – being held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA – announces that its Call for Papers is open. Born out of proven success in agile development, cloud computing, and process automation, DevOps is a macro trend you cannot afford to miss. From showcase success stories from early adopters and web-scale businesses, DevOps is expanding to organizations of all sizes, including the world's largest enterprises – and delivering real results. Am...
What does it look like when you have access to cloud infrastructure and platform under the same roof? Let’s talk about the different layers of Technology as a Service: who cares, what runs where, and how does it all fit together. In his session at 18th Cloud Expo, Phil Jackson, Lead Technology Evangelist at SoftLayer, an IBM company, spoke about the picture being painted by IBM Cloud and how the tools being crafted can help fill the gaps in your IT infrastructure.
SYS-CON Events announced today that Niagara Networks will exhibit at the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Niagara Networks offers the highest port-density systems, and the most complete Next-Generation Network Visibility systems including Network Packet Brokers, Bypass Switches, and Network TAPs.
What happens when the different parts of a vehicle become smarter than the vehicle itself? As we move toward the era of smart everything, hundreds of entities in a vehicle that communicate with each other, the vehicle and external systems create a need for identity orchestration so that all entities work as a conglomerate. Much like an orchestra without a conductor, without the ability to secure, control, and connect the link between a vehicle’s head unit, devices, and systems and to manage the ...
In this strange new world where more and more power is drawn from business technology, companies are effectively straddling two paths on the road to innovation and transformation into digital enterprises. The first path is the heritage trail – with “legacy” technology forming the background. Here, extant technologies are transformed by core IT teams to provide more API-driven approaches. Legacy systems can restrict companies that are transitioning into digital enterprises. To truly become a lea...
SYS-CON Events announced today that Sheng Liang to Keynote at SYS-CON's 19th Cloud Expo, which will take place on November 1-3, 2016 at the Santa Clara Convention Center in Santa Clara, California.
In his session at @ThingsExpo, Kausik Sridharabalan, founder and CTO of Pulzze Systems, Inc., will focus on key challenges in building an Internet of Things solution infrastructure. He will shed light on efficient ways of defining interactions within IoT solutions, leading to cost and time reduction. He will also introduce ways to handle data and how one can develop IoT solutions that are lean, flexible and configurable, thus making IoT infrastructure agile and scalable.
So, you bought into the current machine learning craze and went on to collect millions/billions of records from this promising new data source. Now, what do you do with them? Too often, the abundance of data quickly turns into an abundance of problems. How do you extract that "magic essence" from your data without falling into the common pitfalls? In her session at @ThingsExpo, Natalia Ponomareva, Software Engineer at Google, provided tips on how to be successful in large scale machine learning...
Cognitive Computing is becoming the foundation for a new generation of solutions that have the potential to transform business. Unlike traditional approaches to building solutions, a cognitive computing approach allows the data to help determine the way applications are designed. This contrasts with conventional software development that begins with defining logic based on the current way a business operates. In her session at 18th Cloud Expo, Judith S. Hurwitz, President and CEO of Hurwitz & ...
The Transparent Cloud-computing Consortium (abbreviation: T-Cloud Consortium) will conduct research activities into changes in the computing model as a result of collaboration between "device" and "cloud" and the creation of new value and markets through organic data processing High speed and high quality networks, and dramatic improvements in computer processing capabilities, have greatly changed the nature of applications and made the storing and processing of data on the network commonplace.
An IoT product’s log files speak volumes about what’s happening with your products in the field, pinpointing current and potential issues, and enabling you to predict failures and save millions of dollars in inventory. But until recently, no one knew how to listen. In his session at @ThingsExpo, Dan Gettens, Chief Research Officer at OnProcess, will discuss recent research by Massachusetts Institute of Technology and OnProcess Technology, where MIT created a new, breakthrough analytics model f...
Without a clear strategy for cost control and an architecture designed with cloud services in mind, costs and operational performance can quickly get out of control. To avoid multiple architectural redesigns requires extensive thought and planning. Boundary (now part of BMC) launched a new public-facing multi-tenant high resolution monitoring service on Amazon AWS two years ago, facing challenges and learning best practices in the early days of the new service. In his session at 19th Cloud Exp...
The Internet of Things can drive efficiency for airlines and airports. In their session at @ThingsExpo, Shyam Varan Nath, Principal Architect with GE, and Sudip Majumder, senior director of development at Oracle, will discuss the technical details of the connected airline baggage and related social media solutions. These IoT applications will enhance travelers' journey experience and drive efficiency for the airlines and the airports. The session will include a working demo and a technical d...
Almost two-thirds of companies either have or soon will have IoT as the backbone of their business in 2016. However, IoT is far more complex than most firms expected. How can you not get trapped in the pitfalls? In his session at @ThingsExpo, Tony Shan, a renowned visionary and thought leader, will introduce a holistic method of IoTification, which is the process of IoTifying the existing technology and business models to adopt and leverage IoT. He will drill down to the components in this fra...
Digital transformation is too big and important for our future success to not understand the rules that apply to it. The first three rules for winning in this age of hyper-digital transformation are: Advantages in speed, analytics and operational tempos must be captured by implementing an optimized information logistics system (OILS) Real-time operational tempos (IT, people and business processes) must be achieved Businesses that can "analyze data and act and with speed" will dominate those t...
I'm a lonely sensor. I spend all day telling the world how I'm feeling, but none of the other sensors seem to care. I want to be connected. I want to build relationships with other sensors to be more useful for my human. I want my human to understand that when my friends next door are too hot for a while, I'll soon be flaming. And when all my friends go outside without me, I may be left behind. Don't just log my data; use the relationship graph. In his session at @ThingsExpo, Ryan Boyd, Engi...
Internet of @ThingsExpo, taking place November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with the 19th International Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world and ThingsExpo Silicon Valley Call for Papers is now open.
Adobe is changing the world though digital experiences. Adobe helps customers develop and deliver high-impact experiences that differentiate brands, build loyalty, and drive revenue across every screen, including smartphones, computers, tablets and TVs. Adobe content solutions are used daily by millions of companies worldwide-from publishers and broadcasters, to enterprises, marketing agencies and household-name brands. Building on its established design leadership, Adobe enables customers not o...