Welcome!

Big Data Journal Authors: Elad Yoran, Elizabeth White, David Honan, Liz McMillan, Mark van Rijmenam

Related Topics: Cloud Expo, Java, SOA & WOA, Virtualization, GovIT, Big Data Journal, SDN Journal

Cloud Expo: Article

Lightning in the Clouds, Big Data

Venture into the clouds, but prepare for lightning

Cloud computing has been marketed as one of the key advances in technology and every day we hear about new areas where cloud services are being utilized. Cloud is the bright shining star that is being leveraged for it's elastic, on-demand, resource pooling capabilities. However there have been Cloud outages recently that have adversely impacted businesses. These outages highlight the risks of the Cloud and bring into focus that such risks need to be effectively managed. Cloud outages are like lightning in the Clouds, lightning can cause problems where it strikes and preparation is important to avoid damage.

This year Amazon, Salesforce, Google, Gmail, Google App Engine, Microsoft Office 365, Azure had outages that impacted businesses. During this holiday season, some Netflix subscribers were hit with an outage on Christmas Eve that was caused by Amazon cloud servers.  Amazon tracked the issue to Elastic Load Balancing, that enables spreading traffic across many servers.The wasn't good timing for the outage as subscribers were looking forward to watching movies during this period. Microsoft Azure storage had an outage during the holidays that impacted the management portal. Big Data services and  providers have also reported access issues.

There are many ways of minimizing risks associated with such outages.  The key is to distribute the load that comes in and to have redundancy, so that failure in one area can be picked up by another area. There are many approaches to achieving such redundancy. One way to do this is to distribute systems across locations so that if one location is down, the other one can pick up the load. Multiple location failover can be more expensive, hence the costs have to be weighed against the benefits and availability requirements. Another approach is to have back up clouds for applications with requirements for high availability, to which traffic can be diverted to reduce the risks of downtime in a specific cloud. Again, this can be more costly compared to having the regular clouds. A similar approach is to have the applications spread out across many clouds, so that if there are downtime issues, not all applications go down at once.   If applications are located in many clouds, this can enhance availability if one of the clouds goes down.

It is important to have adequate monitoring for applications and cloud services to be informed when the services are down and to take necessary actions. Both availability and performance monitoring should be conducted to identify any problems. Availability monitoring tracks if the applications and services are up and running, performance monitoring looks into  performance metrics such as response time for applications and services. Availability and performance should be specified for services and applications based on the requirements.  Key events have to be identified for monitoring and specific alerts have to be set up and as soon as these alerts send notifications, specific actions should be taken.  All these aspects should be defined in a plan that lays out all the details of events and related actions so that any outages can be handled effectively. Preparation is key to ensure the proper actions are taken to address and recover from any such issues. The risks related to Cloud outages can be managed by having the proper approaches for availability and performance, regularly monitoring and taking appropriate actions in a timely manner.

(This has been extracted from and is reference to Ajay Budhraja's blog)

More Stories By Ajay Budhraja

Ajay Budhraja has over 23 years in Information Technology with experience in areas such as Executive leadership, management, strategic planning, enterprise architecture, system architecture, software engineering, training, methodologies, networks, and databases. He has provided Senior Executive leadership for nationwide and global programs and has implemented integrated Enterprise Information Technology solutions.

Ajay has a Masters in Engineering (Computer Science), and a Masters in Management and Bachelors in Engineering. He is a Project Management Professional certified by the PMI and is also CICM, CSM, ECM (AIIM) Master, SOA, RUP, SEI-CMMI, ITIL-F, Security + certified.

Ajay has led large-scale projects for big organizations and has extensive IT experience related to telecom, business, manufacturing, airlines, finance and government. He has delivered internet based technology solutions and strategies for e-business platforms, portals, mobile e-business, collaboration and content management. He has worked extensively in the areas of application development, infrastructure development, networks, security and has contributed significantly in the areas of Enterprise and Business Transformation, Strategic Planning, Change Management, Technology innovation, Performance management, Agile management and development, Service Oriented Architecture, Cloud.

Ajay has been leading organizations as Senior Executive, he is the Co-Chair for the Federal SOA COP and has served as President DOL-APAC, AEA-DC, Co-Chair Executive Forum Federal Executive Institute SES Program. As Adjunct Faculty, he has taught courses for several universities. He has received many awards, authored articles and presented papers at worldwide conferences.

Cloud Expo Breaking News
In an ideal developer/systems administrator’s world, most applications would deploy seamlessly to multiple platforms and scale elastically with minimal effort bringing the unprecedented agility of the cloud within immediate reach of developer teams and IT organizations. OpenStack, a RackSpace and NASA initiative, is now managed by an independent foundation and is supported by multiple vendors. It defines APIs for compute, storage, networking, services, monitoring, and additional infrastructure...
Companies around the world are moving into on-premise private cloud environments. Many connect their private cloud to their public cloud service providers. In his session at 12th Cloud Expo | Cloud Expo New York [June 10-13], Brian Patrick Donaghy will talk about examples of what worked, what failed and why we should think about this evolution.
A recent study by analyst firm IDC reports that in 2012, 1.7 million cloud computing-related roles across the globe could not be filled due to the lack of training, certification and experience in the applicant pool. As the global demand for cloud and big data expertise increases, employers are finding it difficult to recruit talent, which is slowing down the ability for organizations to adopt, implement, and realize benefits from innovative platforms like OpenStack. In this session join Clo...
Enterprise cloud adoption revolves around pushing the BYOD movement and focusing on data security. In his session at the 12th International Cloud Expo, Ross Brouse, COO and President of Solar VPS, will cover how cloud adoption is driven by consumerism, humanity’s need to socialize, our addiction to new gadgets and the ability of data to stay secure in a growing collaborative world. The cloud is a drug and we’re just getting hooked. Ross Brouse is the COO and President of Solar VPS. He is a tr...
Organizations across the world are increasingly starting to see the benefits of moving more and more services to the cloud. The focus on the cost-saving potential of cloud is rapidly shifting to completely transforming the business with cloud. As organizations are investing enormous sums on technology they are starting to realize that in order to maximize the return on investment and accelerate the business transformation process the first area of focus should be people. By ensuring the organiza...
Enterprises can't close their doors just because integration tools won't cope with the volume of information that their systems produce. As each day goes by, their information will become larger and more complicated, and enterprises must constantly struggle to manage the integration of dozens (or hundreds) of systems. Apache Hadoop has quickly become the technology of choice for enterprises that need to perform complex analysis of petabytes of data, but few are aware of its potential to hand...
Our more interconnected planet is accelerating the adoption and convergence of next-generation architectures, in the form of cloud, mobile and instrumented physical assets. Organizations that can effectively balance optimization and innovation, will be in a position to leverage new systems of engagement, out maneuver their peers and achieve desired outcomes. In the Opening Keynote at 12th Cloud Expo | Cloud Expo New York, IBM GM & Next Generation Platform CTO Dr Danny Sabbah will detail the crit...
The cloud-enabled data center sits at the center of IT transformation. It facilitates the interconnection and communities that come together, propelling growth for both buyers and sellers. In his session at the 12th International Cloud Expo, Gerry Fassig, CoreSite’s Vice President of Sales, will discuss how CoreSite is bringing together best-of-breed partners through the Open Cloud Exchange resulting in public, private, and hybrid cloud interconnection and management as well as connectivity to...
Companies around the world are collecting massive amounts of data everyday that’s sitting around and not being utilized. Take for example the fact that companies collect demographic and location-based data via mobile devices all the time, but have to figure out how to monetize that data. In this session, Joyent CTO and founder Jason Hoffman will examine the state of Big Data, taking a look at what we're doing now to discussing what's on the horizon, as companies prepare and realign their busines...
Cloud enables SMBs to access new, scalable resources – previously only available to enterprises – in flexible and cost-effective ways. McKinsey’s SMB Cloud Report projects the public cloud market to reach $40-$50 billion by 2015, with SMBs comprising 65% of public cloud spending in 2015. But selling cloud to SMBs raises the questions of who, what and how. In this session Manjula Talreja, VP of Cisco’s Global Cloud Business Development Team, will discuss the importance of knowing who SMB...