Welcome!

Big Data Journal Authors: Jeremy Geelan, Pat Romanski, Elizabeth White, Wiqar Chaudry, Alex Henthorn-Iwane

Related Topics: Virtualization, SOA & WOA, Web 2.0, Cloud Expo, Big Data Journal, SDN Journal

Virtualization: Blog Feed Post

Design Docs, Code Comments, and . . .

We need to treat our projects more like research projects, and keep a “notebook” somewhere

When first you build a bookshelf from raw wood, on your own, you learn a lot. While the first may be the only one you ever want to build, by the time you’re done, you know you could have done X, Y, or Z better. This is true of so many crafts. You get better by doing, in large part because you learn new tricks, figure out how to make useful tools, get more efficient because you purchased more useful tools… I know that while I would never consider myself an artist, painting models and miniatures has made me progress to where some have called me one. (Picture is a battery of M7 “Priest” artillery I painted up – models by Brittania miniatures)

The thing is, when it’s a hobby, or you’re a one man shop, or the job is not particularly taxing, this knowledge is easy to keep around. You have that jig you built to make cool shelf angles hanging on the wall, or the box of perfect fasteners is still on the shelf. The glue or stain you settled on is what you kept when the last shelf was built, etc.

But when the job is complex, and the problems increasingly so, it is not so simple. When the tools used to achieve a solid solution – the things used to learn and used to improve your productivity – are on a disk somewhere in the org, things get a little more complex.

But not irretrievably. You just need to pay a little more attention.

When you’re developing an app or configuring a network, there is the design, showing what is to be done, the code comments describing what was done, and the finished product for use by the enterprise. You have accumulated knowledge, which is good for you and the organization, but you also did some other things that normally fall through the cracks. That tool you cobbled together to check the validity of file formats, or the browser plug-in that helped you resolve network issues, or the DNS tool you downloaded that helped you get redirection correct… These things are important. Just as important as the tool I discovered to make the mud up the side of that vehicle. More important, because in a hobby I have time to mess around and reinvent the wheel if I must. in the enterprise, I’ve never met the IT staffer that went “you know, I did this seven months ago, I think I’ll just go figure out how to do it all over again, I’ve got the free time”. Never.

So what you need to do is keep track of these things. Put them in version control, store them on a special share for just such tools under a heading/directory that associates the tool with the project it helped with. Even silly little things. And put them in a word document if they’re things like links to a web page. Sure the page may be gone when next the word doc is opened, but that’s true of bookmarks in your browser too, and your word doc is likely to have greater longevity. This is for you – because in many orgs if you created it, it’s your problem for eternity – but it is also for the organization, because eventually it will not be your problem, and it will save the next poor person from figuring out how to do the same things. I’m not talking about tools like lint, if you don’t have a corporate standard to use that type of thing though, you could include it. I’m talking about that library you found that does what you wanted, but was a nightmare to configure. So document it. While you’re celebrating that you got it going. Save someone else the headache. Or yourself, should something horrendous happen and you have to configure it from scratch again.

Devops has helped in this regard with networking, by stressing repeatable processes, but developers are asked to do something different almost every app. Patterns are great, but it’s rarely patterns that trip us up, it’s some weird exception case that only occurs in this language/environment/requirements document. Much more difficult to turn into repeatable, when if it was simply repeatable you’d have found source or used a library to achieve it quickly.

What we do is more science than art. We need to treat our projects more like research projects, and keep a “notebook”. somewhere. Those tools, random observations, difficult to discern but very profound solutions… All of that could be part of the intellectual property of the app/tool/network in question, making everyone more productive. But document it in English (or the tongue spoken by the bulk of your coworkers, for my international friends), not in your “from here I understand a ton, I’ll write about the bit I just learned”.

Sometime soon – hopefully next week – I will be writing up my solution (and recommended more generic solution) to the several issues found in Android SSL. By writing about it out here, I am documenting what I went through, and have to write about it in plain English. You need to do the same (but normally internally only, so you don’t have to explain the entire environment/problem-set), so that five years from now, the local Intern can figure out how to work on your app/network. Or you can remember what the heck you did to make it work. ;-)

You’ll thank yourself. Most good developers I know have forgotten more coding projects than they remember, and this might provide the clues to get back up to speed when it circles around and you have to work on the solution again. I assume network admins have the same problem. Your employer will thank you, because you (and potentially anyone else asked to work on it) will be more productive, just like using that jig over again is more productive. It’s not necessarily part of the solution, but it makes building the solution far easier.

Meanwhile, keep building the next generation of apps and networks, we’re expecting lots, and even though complaints are louder than compliments, no one can argue that developers and network admins as a whole haven’t responded in astounding ways over the last 30 years. Unless you prefer disconnected mainframes I suppose...

Read the original blog entry...

More Stories By Don MacVittie

Don MacVittie is a Technical Marketing Manager at F5 Networks. In this role, he supports outbound marketing, education, and evangelism efforts around development, storage, and IT management topics related to F5 solutions. His role includes authoring technical materials, participating in social and community-based forums, and providing guidance for the development of marketing resources. As an industry veteran, MacVittie has extensive programming experience along with project management, IT management, and systems/network administration expertise.

Prior to joining F5, MacVittie was a Senior Technology Editor at Network Computing, where he conducted product research and evaluated storage and server systems, as well as development and outsourcing solutions. He has authored numerous articles on a variety of topics aimed at IT professionals. MacVittie holds a B.S. in Computer Science from Northern Michigan University, and an M.S. in Computer Science from Nova Southeastern University.

Cloud Expo Breaking News
Enterprises can't close their doors just because integration tools won't cope with the volume of information that their systems produce. As each day goes by, their information will become larger and more complicated, and enterprises must constantly struggle to manage the integration of dozens (or hundreds) of systems. Apache Hadoop has quickly become the technology of choice for enterprises that need to perform complex analysis of petabytes of data, but few are aware of its potential to hand...
Our more interconnected planet is accelerating the adoption and convergence of next-generation architectures, in the form of cloud, mobile and instrumented physical assets. Organizations that can effectively balance optimization and innovation, will be in a position to leverage new systems of engagement, out maneuver their peers and achieve desired outcomes. In the Opening Keynote at 12th Cloud Expo | Cloud Expo New York, IBM GM & Next Generation Platform CTO Dr Danny Sabbah will detail the crit...
The cloud-enabled data center sits at the center of IT transformation. It facilitates the interconnection and communities that come together, propelling growth for both buyers and sellers. In his session at the 12th International Cloud Expo, Gerry Fassig, CoreSite’s Vice President of Sales, will discuss how CoreSite is bringing together best-of-breed partners through the Open Cloud Exchange resulting in public, private, and hybrid cloud interconnection and management as well as connectivity to...
Companies around the world are collecting massive amounts of data everyday that’s sitting around and not being utilized. Take for example the fact that companies collect demographic and location-based data via mobile devices all the time, but have to figure out how to monetize that data. In this session, Joyent CTO and founder Jason Hoffman will examine the state of Big Data, taking a look at what we're doing now to discussing what's on the horizon, as companies prepare and realign their busines...
The massive computing and storage resources that are needed to support big data applications make cloud environments an ideal fit. In Nati Shalom's upcoming session at 12th Cloud Expo | Cloud Expo New York [June 10-13, 2013], you'll learn how to build your big data "database on-demand" using MongoDB, Cassandra, Solr, MySQL, or any other big data solution, as well as manage your big data application using a new open source framework called “Cloudify.” All this, on top of the OpenStack cloud.
Planning scalable environments isn't terribly difficult, but it does require a change of perspective. During this session we'll broaden our views to think on an Internet Scale by dissecting a video publishing application built with The SoftLayer Platform, Message Queuing, Object Storage, and Drupal. By examining a scalable modular application build that can handle unpredictable traffic, you'll be able to grow your development arsenal and pick up a few strategies to apply to your own projects.
If zettabytes of data exist, why is less than 1% of the world’s data being analyzed today? Seasoned entrepreneur and startup CEO Radhika Subramanian believes that the inability to analyze and gain value from Big Data is that organizations are taking a services-centered approach. As the title of the session implies, Subramanian believes that the data needs to do the talking, not armies of analysts searching and querying databases. Her company has developed high-speed, advanced algorithms to autom...
Cloud enables SMBs to access new, scalable resources – previously only available to enterprises – in flexible and cost-effective ways. McKinsey’s SMB Cloud Report projects the public cloud market to reach $40-$50 billion by 2015, with SMBs comprising 65% of public cloud spending in 2015. But selling cloud to SMBs raises the questions of who, what and how. In this session Manjula Talreja, VP of Cisco’s Global Cloud Business Development Team, will discuss the importance of knowing who SMB...
SYS-CON Events announced today that Zyrion Inc., the leading provider of Cloud and IT Monitoring software solutions, has been named “Entrance Carpet Sponsor” of SYS-CON's 12th International Cloud Expo, which will take place on June 10–13, 2013, at the Javits Center in New York City, New York. Zyrion is the leading provider of integrated Cloud and Network monitoring software for distributed and complex datacenter environments, and offers the most scalable monitoring platform in the industry. Zyr...
The world’s first vendor neutral marketplace for IaaS (Infrastructure as a Service) cloud computing is being built. This marketplace fills the current gap in the value chain by offering standardized products and by addressing the needs of providers and consumers of cloud computing resources. Zimory is the technical partner for the settlement process of this project. In his session at 12th Cloud Expo | Cloud Expo New York [June 10-13, 2013], Zimory CEO Rüdiger Baumann session will introduce th...