Welcome!

Big Data Journal Authors: Esmeralda Swartz, Pat Romanski, Elizabeth White, Liz McMillan, Kira Makagon

Related Topics: Cloud Expo, SOA & WOA, Web 2.0, Big Data Journal, SDN Journal, DevOps Journal

Cloud Expo: Article

In-Memory Data Grids and Cloud Computing

The promise of the cloud is a reduction in total cost of ownership

The use of in-memory data grids (IMDGs) for scaling application performance has rapidly increased in recent years as firms have seen their application workloads explode. This trend runs across nearly every vertical market, touching online applications for financial services, ecommerce, travel, manufacturing, social media, mobile, and more. At the same time, many firms are also looking to leverage the use of cloud computing to meet the challenge of ever increasing workloads. One of the fundamental promises of the cloud is elastic, transparent, on-demand scalability -- a key capability that has become practical with the use of in-memory data grid technology. As such IMDGs are becoming a vital factor in the cloud, just as they have been for on-premise applications.

What makes IMDGs such a good fit with cloud computing? The promise of the cloud is a reduction in total cost of ownership. Part of that reduction comes from the ability to quickly provision and use new server capacity (without having to own the hardware). The essential synergy between IMDGs and the cloud derives from their common elasticity. IMDGs can scale out their memory-based storage and performance linearly as servers are added to the grid, and they can gracefully scale back when fewer servers are needed. IMDGs take full advantage of the cloud's ability to easily spin-up or remove servers. IMDGs enable cloud-hosted applications to be quickly and easily deployed on an elastic pool of cloud servers to deliver scalable performance, maintaining fast data access even as workloads increase. This is an ideal solution for fast-growing companies and for applications whose workloads create widely varying demands (like online flowers for Mother's Day, concert tickets, etc.). These companies no longer need to create space, power, and cooling for new hardware to meet these fluctuating workloads. Instead, with a few button clicks, they can start up an IMDG-enabled cloud architecture, which transparently meets their performance demands at a cost is solely based on usage.

Expanding on the promise of the cloud, some in-memory data grids can span both on-premise and cloud environments to provide seamless "cloud bursting" for handling high workloads. Let's say your e-commerce application stores shopping carts in an IMDG to give customers fast response times. To spur sales, your marketing group plans to run a special online sales event. Because projected traffic is expected to double during this event, additional web servers will be needed to handle the workload. Of course, maintaining fast response times as the workload increases is essential to success. By deploying your web app in the cloud and connecting it to your on-premise server farm with an IMDG, you can seamlessly double your traffic-handling capacity without interrupting current shopping activity on your site. You don't even need to make changes to your application. The combined deployments transparently work together to serve web traffic, and data freely flows between them within the IMDGs at both sites.

These synergies form a solid basis for making 2014 a watershed year for IMDGs in the cloud. But, there's another big trend that will further drive adoption. As the discussion around "Big Data" analysis heats up, the emerging combination of Big Data and cloud computing - cloud-based analytics - promises to fundamentally change the technology of data mining, machine learning and many other analytics use cases. In 2014, we expect to see the trend toward in-memory, predictive analytics sharply increase, and cloud computing will be a fundamental enabler of that trend.

IMDGs integrate memory-based data storage and computing to make real-time data analysis easily accessible to users and help extend a company's competitive edge. IMDGs automatically take full advantage of the cloud's elasticity to run analytics in parallel across cloud servers with lightning fast performance. Now it's possible to host a real-time analytics engine in the cloud and provide on-demand analytics to a wide range of users, from SaaS services for mobile devices to business simulations for corporate users. Or, maybe you want to spin-up servers with, say, a terabyte of memory, load the grid, run analytics across that data, and then release the resources. In an extreme example, chemistry researchers recently used Amazon Web Services to achieve a "petaflop" of computing power</a> running an analysis of 205,000 molecules for just one week. The elasticity of the cloud again makes the difference by providing the equivalent of a parallel processing supercomputer at your fingertips without the huge capital investment (it costs $33,000 total).

To sum-up, in 2014 we expect firms to adopt cloud computing and cloud-hosted IMDGs at a rapid rate, and the trends of in-memory computing and data analytics will converge to enable fast adoption of in-memory data grid technology in public, private, and hybrid cloud environments. Enterprises that take advantage of this convergence are expected to enjoy a quantum leap in the value of their data without the need to break their IT budgets.

More Stories By William Bain

Dr. William L. Bain is founder and CEO of ScaleOut Software, Inc. Bill has a Ph.D. in electrical engineering/parallel computing from Rice University, and he has worked at Bell Labs research, Intel, and Microsoft. Bill founded and ran three start-up companies prior to joining Microsoft. In the most recent company (Valence Research), he developed a distributed Web load-balancing software solution that was acquired by Microsoft and is now called Network Load Balanc¬ing within the Windows Server operating system. Dr. Bain holds several patents in computer architecture and distributed computing. As a member of the Seattle-based Alliance of Angels, Dr. Bain is actively involved in entrepreneurship and the angel community.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


@BigDataExpo Stories
"SAP had made a big transition into the cloud as we believe it has significant value for our customers, drives innovation and is easy to consume. When you look at the SAP portfolio, SAP HANA is the underlying platform and it powers all of our platforms and all of our analytics," explained Thorsten Leiduck, VP ISVs & Digital Commerce at SAP, in this SYS-CON.tv interview at 15th Cloud Expo, held Nov 4-6, 2014, at the Santa Clara Convention Center in Santa Clara, CA.
"Desktop as a Service is emerging as a very big trend. One of the big influencers of this – for Esri – is that we have a large user base that uses virtualization and they are looking at Desktop as a Service right now," explained John Meza, Product Engineer at Esri, in this SYS-CON.tv interview at Cloud Expo, held Nov 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA.
SAP is delivering break-through innovation combined with fantastic user experience powered by the market-leading in-memory technology, SAP HANA. In his General Session at 15th Cloud Expo, Thorsten Leiduck, VP ISVs & Digital Commerce, SAP, discussed how SAP and partners provide cloud and hybrid cloud solutions as well as real-time Big Data offerings that help companies of all sizes and industries run better. SAP launched an application challenge to award the most innovative SAP HANA and SAP HANA...
The Industrial Internet revolution is now underway, enabled by connected machines and billions of devices that communicate and collaborate. The massive amounts of Big Data requiring real-time analysis is flooding legacy IT systems and giving way to cloud environments that can handle the unpredictable workloads. Yet many barriers remain until we can fully realize the opportunities and benefits from the convergence of machines and devices with Big Data and the cloud, including interoperability, ...
The Internet of Things is tied together with a thin strand that is known as time. Coincidentally, at the core of nearly all data analytics is a timestamp. When working with time series data there are a few core principles that everyone should consider, especially across datasets where time is the common boundary. In his session at Internet of @ThingsExpo, Jim Scott, Director of Enterprise Strategy & Architecture at MapR Technologies, discussed single-value, geo-spatial, and log time series dat...
Cultural, regulatory, environmental, political and economic (CREPE) conditions over the past decade are creating cross-industry solution spaces that require processes and technologies from both the Internet of Things (IoT), and Data Management and Analytics (DMA). These solution spaces are evolving into Sensor Analytics Ecosystems (SAE) that represent significant new opportunities for organizations of all types. Public Utilities throughout the world, providing electricity, natural gas and water,...
There is no doubt that Big Data is here and getting bigger every day. Building a Big Data infrastructure today is no easy task. There are an enormous number of choices for database engines and technologies. To make things even more challenging, requirements are getting more sophisticated, and the standard paradigm of supporting historical analytics queries is often just one facet of what is needed. As Big Data growth continues, organizations are demanding real-time access to data, allowing immed...
The Internet of Things will greatly expand the opportunities for data collection and new business models driven off of that data. In her session at @ThingsExpo, Esmeralda Swartz, CMO of MetraTech, discussed how for this to be effective you not only need to have infrastructure and operational models capable of utilizing this new phenomenon, but increasingly service providers will need to convince a skeptical public to participate. Get ready to show them the money!
The 3rd International Internet of @ThingsExpo, co-located with the 16th International Cloud Expo - to be held June 9-11, 2015, at the Javits Center in New York City, NY - announces that its Call for Papers is now open. The Internet of Things (IoT) is the biggest idea since the creation of the Worldwide Web more than 20 years ago.
The 4th International DevOps Summit, co-located with16th International Cloud Expo – being held June 9-11, 2015, at the Javits Center in New York City, NY – announces that its Call for Papers is now open. Born out of proven success in agile development, cloud computing, and process automation, DevOps is a macro trend you cannot afford to miss. From showcase success stories from early adopters and web-scale businesses, DevOps is expanding to organizations of all sizes, including the world's large...
Cloud Expo 2014 TV commercials will feature @ThingsExpo, which was launched in June, 2014 at New York City's Javits Center as the largest 'Internet of Things' event in the world.
“We help people build clusters, in the classical sense of the cluster. We help people put a full stack on top of every single one of those machines. We do the full bare metal install," explained Greg Bruno, Vice President of Engineering and co-founder of StackIQ, in this SYS-CON.tv interview at 15th Cloud Expo, held Nov 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA.
In this demo at 15th Cloud Expo, John Meza, Product Engineer at Esri, showed how Esri products hook into Hadoop cluster to allow you to do spatial analysis on the spatial data within your cluster, and he demonstrated rendering from a data center with ArcGIS Pro, a new product that has a brand new rendering engine.
"People are a lot more knowledgeable about APIs now. There are two types of people who work with APIs - IT people who want to use APIs for something internal and the product managers who want to do something outside APIs for people to connect to them," explained Roberto Medrano, Executive Vice President at SOA Software, in this SYS-CON.tv interview at Cloud Expo, held Nov 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA.
The Software Defined Data Center (SDDC), which enables organizations to seamlessly run in a hybrid cloud model (public + private cloud), is here to stay. IDC estimates that the software-defined networking market will be valued at $3.7 billion by 2016. Security is a key component and benefit of the SDDC, and offers an opportunity to build security 'from the ground up' and weave it into the environment from day one. In his session at 16th Cloud Expo, Reuven Harrison, CTO and Co-Founder of Tufin,...
SYS-CON Media announced that Splunk, a provider of the leading software platform for real-time Operational Intelligence, has launched an ad campaign on Big Data Journal. Splunk software and cloud services enable organizations to search, monitor, analyze and visualize machine-generated big data coming from websites, applications, servers, networks, sensors and mobile devices. The ads focus on delivering ROI - how improved uptime delivered $6M in annual ROI, improving customer operations by minin...
Can the spatial component of your Big Data be harnessed and visualized, adding another dimension of power and analytics to your data? In his session at Big Data Expo®, John Meza, Product Engineer and Performance Engineering Team Lead at Esri, discussed the spatial queries that can be used within the Hadoop ecosystem and their integration with GeoSpatial applications. The GIS Tools for Hadoop project was also discussed and its implementation to discover location-based patterns and relationships...
DevOps Summit 2015 New York, co-located with the 16th International Cloud Expo - to be held June 9-11, 2015, at the Javits Center in New York City, NY - announces that it is now accepting Keynote Proposals. The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is no time to wait for long development cycles that produce software that is obsolete...
Software AG and Wipro Ltd. have announced a joint solution platform for streaming analytics that provides real-time actionable intelligence for the Internet of Things (IoT) market. “The key to successfully addressing the IoT market is the ability to rapidly build and evolve apps that tap into, analyze and make smart decisions on fast, big data”, said John Bates, Global Head of Industry Solutions and CMO, Software AG. To address the huge market potential created by streaming analytics in conj...
We’re no longer looking to the future for the IoT wave. It’s no longer a distant dream but a reality that has arrived. It’s now time to make sure the industry is in alignment to meet the IoT growing pains – cooperate and collaborate as well as innovate. In his session at @ThingsExpo, Jim Hunter, Chief Scientist & Technology Evangelist at Greenwave Systems, will examine the key ingredients to IoT success and identify solutions to challenges the industry is facing. The deep industry expertise be...