Welcome!

@BigDataExpo Authors: Pat Romanski, Elizabeth White, Kevin Benedict, Nitin Donde, Astadia CloudGPS

Related Topics: @BigDataExpo, Microsoft Cloud, @DevOpsSummit

@BigDataExpo: Blog Post

Detecting Bad Deployments [#APM with #DevOps] | @DevOpsSummit

Resource usage monitoring is as important as response time and throughput.

Detecting Bad Deployments on Resource Impact & Not Response Time

This story came in from Joseph - one of our fellow dynaTrace users and a performance engineer at a large fleet management service company. Their fleet management software runs on .NET, is developed in-house, is load tested with JMeter and monitored in Production with dynaTrace. A usage and configuration change of their dependency injection library turned out to dramatically impact CPU and memory usage while not yet impacting end user experience. Lessons learned: resource usage monitoring is as important as response time and throughput.

On Wednesday, July 3, Joseph's ops team deployed the latest version into their production environment. Load (=throughput) and response time are two key application health measures the application owner team has on their production dashboards. Especially after a new deployment, an unchanged behavior is what you are looking/ hoping for:

APM Lesson #1: Don't judge a deployment based on response time and throughput alone

Joseph is an experienced performance engineer, so he knew to check more than just these two metrics. Looking at host and process monitoring data he identified a 6x increase in CPU and 40% increase in process memory usage, which alarmed the folks responsible for the infrastructure that hosted the ASP.NET Services:

APM Lesson #2: Expand your alerting to resource usage and infrastructure health

Despite the fact that the app still shows the same response time and throughput this is very alarming. Why? Because it means that the application is much more expensive to run as compared to the previous build.

Whether it runs on physical, virtualized or in a cloud environment: more resource consumption means higher operational costs. And who knows what happens during peak load: will CPU and memory be enough or will it eventually impact response time and throughput? That's the reason why analyzing changes in resource consumption is as critical to performance engineering as optimizing response time and throughput.

If you are a performance engineer, application owner or developer I suggest you read the rest of the blog to learn about Joseph's best practices on analyzing and dealing with situations like this and what they changed in their development and deployment process:

Step 1: Analyzing the Impact of Memory Usage on CPU
Runtime engines such as the JVM (more on Java Enterprise Performance) or the .NET CLR (more on.NET Performance Analysis) provide detailed performance counters to start analyzing the resource usage hotspots on CPU and memory. A good start is to look at memory usage in the different heap generations as well as the activity of the garbage collector. The following screenshot shows the .NET garbage collection generations (# of Collections and Heap Size) as well as the time spent in garbage collection metrics of their production environment:

APM Lesson #3: Look at memory metrics to figure out how memory usage and GC behavior has changed

What can you read from these two graphs?

  1. Most of the objects live in generation 0 and never get promoted to an older generation. That means we are talking about short-lived objects that are probably created and cleared on a per request basis. If we saw a shift to older generations it is more likely that these objects are part of a cache.
  2. We don't seem to have a memory leak as the garbage collector seems to clear objects from Gen 0 and not promote them to higher generations, which could eventually lead to an out-of-memory crash.
  3. The increase in # of collections as well as time spent in garbage collection indicates that the application either uses more short-lived objects or these objects grew significantly in size.

Step 3: More Objects or Larger Objects or Both?
The next step is to find out what has changed in terms of objects on the heap. Joseph therefore takes regular dynaTrace memory snapshots (YouTube Tutorial) and keeps them to later compare heap usage patterns between releases. The next screenshot shows the difference between the previous version and the current version and highlights that there are not only about twice as many ASP.NET MVC controller objects on the heap but these objects are also about 6x larger than before:

APM Lesson #4: Continuously take memory snapshots to identify regressions between builds

It turned out it that this pattern was not exclusive to the Planning Controller shown in the screenshot but happened across all ASP.NET MVC Controller objects. Following the Deep Memory Leak Snapshots best practices, Joseph found the actual referenced objects that caused these controller objects to grow in size. It turned out to be related to objects that were instantiated and referenced using dependency injection.

Finding and identifying the problem was the first step which raised two more questions:

  1. Why wasn't this found earlier?
  2. Which code change caused this problem?

For answers, click here for the full article.

DevOps: Disruptive but Essential in a Cloud Computing Universe

The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is no time to wait for long development cycles that produce software that is obsolete at launch. DevOps may be disruptive, but it is essential.

The DevOps Summit at Cloud Expo--to be held November 4-6 at the Santa Clara Convention Center in the heart of Silicon Valley--will expand the DevOps community, enable a wide sharing of knowledge, and educate delegates and technology providers alike. Recent research has shown that DevOps dramatically reduces development time, the amount of enterprise IT professionals put out fires, and support time generally. Time spent on infrastructure development is significantly increased, and DevOps practitioners report more software releases and higher quality.

DevOps Summit content doubled from a single track in New York to two simultaneous tracks: "Dev" Developer Focus and "Ops" Operations Focus.

Join DevOps Summit Chair Andi Mann and Host Alan Shimel, November 4-6 for three days of intense DevOps discussion and focus.

We'll see you in Santa Clara! See you in November!

DevOps Summit Conference Schedule Announced! ▸ Here

Register For DevOps Summit "FREE" (limited time) ▸ Here

Website: DevOps Summit

Twitter: @DevOpsSummit

Bookmark "DevOps Journal" ▸ here

@DevOpsSummit Silicon Valley (November 4-6, 2014, Santa Clara, CA)
@DevOPsSummit New York (June 9-11, 2015, New York, NY)

DevOps Summit New York 2015 Call for Papers Now Open

Call for Papers will remain open through December 31, 2014. DevOps Summit New York 2015 will take place June 9-11, at the Javits Center, in New York City. Tracks and topics for DevOps Summit at Cloud Expo are listed below. Submit your speaking proposals here DevOps Summit Call for Papers

  • Application delivery
  • ROI and business value
  • Site reliability
  • Continuous delivery
  • Kanban, Scrum, and Agile
  • Continuous integration
  • Release scalability
  • Continuous release
  • Service virtualization
  • Operational monitoring
  • Capacity management
  • Load testing
  • Quality assurance
  • Manager and exec support
  • Release automation
  • Cultural change
  • Breaking down IT silos
  • Cloud development
  • Cloud platforms
  • Test automation
  • Teaming
  • Application security
  • API management
  • Identity and access
  • Audit and control
  • DevOps and ITIL, ISO, Six Sigma, COBIT
  • Mobile DevOps
  • DevOps for legacy systems
  • Software-defined servers
  • Network automation
  • Server automation
  • Configuration automation
  • Continuous support
  • DevOps anti-patterns
  • Enterprise DevOps
  • DevOps system architecture
  • IT orchestration
  • Containerization
  • DevOps skills and training
  • WebOps, CloudOps, ChatOps, NoOps
  • Change management

Video Highlights From DevOps Summit at Cloud Expo New York

While there are many ways to define DevOps, the goal of the concept is to be able to deliver IT solutions faster, leveraging several technology tools to add value for business. Cloud companies have demonstrated how they can manage massive pools of IT infrastructure, giving the developer a very flexible resource pool that they can leverage and IT is eager to do the same.


Is DevOps Really Changing How IT Is Working?

In this Power Panel at 2nd DevOps Summit moderated by Cloud Expo Tech Chair Larry Carvalho, Bernard Golden, VP, Strategy at ActiveState; Andi Mann, VP of Strategic Solutions at CA Technologies; John Willis, VP of Customer Enablement at Stateless Networks; Randy Bias, Co-Founder and CTO of Cloudscaling; and Vasu Sankhavaram, Chief Strategist of DevOps and Cross Portfolio Architecture Alignment within HP Software, discuss this movement that is delivering frictionless IT to the enterprise.


DevOps and Security

From showcase success stories from early adopters and web-scale businesses, DevOps is expanding to organizations of all sizes, including the world's largest enterprises - and delivering real results. In this DevOps Summit Power Panel (http://DevOpsSummit.SYS-CON.com/) on June 9 at our New York City studio, moderated by Andi Mann, DevOps Summit conference chair and VP of Strategic Solutions at CA Technologies, John Willis, VP of Customer Enablement for Stateless Networks; JP Morgenthal, Director, Cloud Computing Practice at Perficient, Inc.; Vanessa Alvarez, Cloud Expo Conference Chair and Marketing Team at Amazon; and Gordon Haff, Cloud Evangelist at Red Hat; discussed how security needs to be embraced and brought into the DevOps movement.


Is DevOps Changing Enterprise IT?

Born out of proven success in agile development, cloud computing, and process automation, DevOps is a macro trend you cannot afford to miss. In this DevOps Summit Power Panel on June 9 at our New York City studio, moderated by Andi Mann, DevOps Summit conference chair and VP of Strategic Solutions at CA Technologies, JP Morgenthal, Director of Cloud Computing Practice at Perficient, Inc., Gordon Haff, Cloud Evangelist at Red Hat; Larry Carvalho, Cloud Expo Tech Chair and Research Manager & Lead Analyst, Platform as a Service at IDC; and John Willis, VP of Customer Enablement for Stateless Networks; discussed how DevOps is changing the way IT works, how businesses interact with customers, and how organizations are buying, building, and delivering software.

Schedule for 15th @CloudExpo, @BigDataExpo and @ThingsExpo

Are you ready to put your data in the cloud?

What is the future of security in the cloud?

Does Docker quickly advance the development of an IoT application?

What are the implications of Moore's Law on Hadoop deployments?

Get all these questions and hundreds more like them answered at the 15th Cloud Expo, November 4-6, 2014, at the Santa Clara Convention Center, in Santa Clara, CA. The Cloud ExpoBig Data Expo@ThingsExpoDevOps Summitprograms are now available for you to inspect and investigate in advance.

Our upcoming November 4-6 event in Santa Clara, California will present a total of 10 simultaneous tracks by an all-star faculty, over three days, plus a two-day "Cloud Computing Bootcamp" presented by Janakiram MSV, an Analyst with the Gigaom Research analyst network, where he covers the Cloud Services landscape.

Cloud and Big Data topics and tracks include: Enterprise Cloud AdoptionAPM & Cloud Computing | Hot TopicsCloud APIs & BusinessCloud Security | MobilityBig Data | Analytics.

@ThingsExpo content tripled from a single track in New York to three simultaneous tracks: Consumer IoTEnterprise IoTIoT Developer | WebRTC Convergence.

Schedule for Cloud Expo / Big Data Expo / @ThingsExpo ▸ Here

Now that we have published the full conference schedule, please check back for daily updates as we finalize new session abstracts by working with our distinguished faculty members. For your questions please contact us at events (at) sys-con.com. Last but not least we will announce our keynotes on the hottest subjects to be delivered by world-class speakers!

The largest 'Internet of Things' event in the world has announced "sponsorship opportunities" and "call for papers."

The 1st International Internet of @ThingsExpo was launched this June at the Javits Center in New York City with over 6,000 delegates in attendance. The 2nd International Internet of @ThingsExpo will take place November 4-6, 2014, at the Santa Clara Convention Center in Santa Clara, California, with an estimated 7,000 plus delegates attending over three days.

@ThingsExpo is co-located with 15th International Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading IoT industry players in the world. In 2014, more than 200 companies will be present on the @ThingsExpo show floor, including global players and the hottest new technology pioneers.

Sponsorship and Exhibit Opportunities for @DevOpsSummit Silicon Valley and New York Are Now Available
Sponsors and exhibitors of Internet of @DevOpsSummit will benefit from unmatched branding, profile building and lead generation opportunities through:

  • Featured on-site presentation and ongoing on-demand webcast exposure to a captive audience of industry decision-makers.
  • Showcase exhibition during our new extended dedicated expo hours
  • Breakout Session Priority scheduling for sponsors that have been guaranteed a 35-minute technical session
  • Online advertising in SYS-CON's i-Technology publications
  • Capitalize on our comprehensive marketing efforts leading up to the show with print mailings, e-newsletters and extensive online media coverage.
  • Unprecedented PR Coverage: Editorial coverage on IoT.sys-con.com, Tweets to our 75,000 plus followers, press releases sent on major wire services to over 500 combined analysts and press members who attended Internet of @ThingsExpo - making it the best-covered "Internet of Things" conference in the world

For more information on sponsorship, exhibit, and keynote opportunities contact Carmen Gonzalez by email at events (at) sys-con.com, or by phone 201 802-3021. Book both events for additional savings!

@DevOpsSummit Silicon Valley (November 4-6, 2014, Santa Clara, CA)
@DevOPsSummit New York (June 9-11, 2015, New York, NY)

Secure Your VIP Pass to Attend @DevOpsSummit Silicon Valley
DevOps Summit at Cloud Expo announced today a limited time free "Expo Plus" registration option. The onsite registration price of $600 will be set to 'free' for delegates who register during special offer period.

To take advantage of this opportunity, attendees can use the coupon code and secure their registration to attend all keynotes and general sessions, as well as a limited number of technical sessions each day of the show, in addition to full access to the expo floor.

The registration page is located here.


@ThingsExpo New York 2015 'Call for Papers' Now Open
The 3rd International Internet of @ThingsExpo, to be held June 9-11, 2015, at the Javits Center in New York City, New York announces that its 'Call for Papers' is now open. The event will feature a world class, all-star faculty with the hottest IoT topics covered in three distinct tracks.

Track 1 - Consumer IoT and Wearables: Smart Appliances, Wearables, Smart Cars, Smartphones 2.0, Smart Travel, Personal Fitness, Health Care, Personalized Marketing, Customized Shopping, Personal Finance, The Digital Divide, Mobile Cash & Markets, Games & the IoT, The Future of Education, Virtual Reality

Track 2 - Enterprise IoT: The Business Case for IoT, Smart Grids, Smart Cities, Smart Transportation, The Smart Home, M2M, Authentication/Security, Wiring the IoT, The Internet of Everything, Digital Transformation of Enterprise IT, Agriculture, Transportation, Manufacturing, Local & State Government, Federal Government

Track 3 -  Developer IoT: WebRTC, Eclipse Foundation, Cloud Foundry, Docker & Linux Containers, Node-Red, Open Source Hardware, Leveraging SOA, Multi-Cloud IoT, Evolving Standards, WebSockets, Security & Privacy Protocols, GPS & Proximity Services, Bluetooth/RFID/etc., XMPP, Nest Labs


@ThingsExpo billboard is viewed by more than 1.3 million motorists per week on Highway 101, in the heart of Silicon Valley

Help plant your flag in the fast-expanding business opportunity that is the Internet of Things: Submit your speaking proposal today here!

Download @ThingsExpo Newsletter Today ▸ Here

Chris Matthieu Named @ThingsExpo Tech Chair

Internet of @ThingsExpo named Chris Matthieu tech chair of Internet of @ThingsExpo 2014 Silicon Valley.

Chris Matthieu has two decades of telecom and web experience. He launched his Teleku cloud communications-as-a-service platform at eComm in 2010, which was acquired by Voxeo. Next he built an open source Node.JS PaaS called Nodester, which was acquired by AppFog. His latest startups include Twelephone. Leveraging HTML5 and WebRTC, Twelephone's BHAG (Big Hairy Audacious Goal) is to become the next generation telecom company running in the Web browser. Chris is currently co-founder and CTO of Octoblu.

Website: http://www.ThingsExpo.com

Twitter: http://www.Twitter.com/ThingsExpo

About SYS-CON Media & Events
SYS-CON Media (www.sys-con.com) has since 1994 been connecting technology companies and customers through a comprehensive content stream - featuring over forty focused subject areas, from Cloud Computing to Web Security - interwoven with market-leading full-scale conferences produced by SYS-CON Events. The company's internationally recognized brands include among othersCloud Expo® (CloudComputingExpo.com@CloudExpo), Big Data Expo (BigDataExpo.net@BigDataExpo), DevOps Summit (DevOpsSummit.sys-con.com@DevOpsSummit),  Internet of @ThingsExpo (ThingsExpo.com@ThingsExpo) and Cloud Computing Bootcamp (CloudComputingBootcamp.com).

Cloud Expo®, Big Data Expo® and @ThingsExpo® are registered trademarks of Cloud Expo, Inc., a SYS-CON Events company.

More Stories By Andreas Grabner

Andreas Grabner has been helping companies improve their application performance for 15+ years. He is a regular contributor within Web Performance and DevOps communities and a prolific speaker at user groups and conferences around the world. Reach him at @grabnerandi

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


@BigDataExpo Stories
"Peak 10 is a hybrid infrastructure provider across the nation. We are in the thick of things when it comes to hybrid IT," explained , Chief Technology Officer at Peak 10, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
"We are still a relatively small software house and we are focusing on certain industries like FinTech, med tech, energy and utilities. We help our customers with their digital transformation," noted Piotr Stawinski, Founder and CEO of EARP Integration, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
"We've been engaging with a lot of customers including Panasonic, we've been involved with Cisco and now we're working with the U.S. government - the Department of Homeland Security," explained Peter Jung, Chief Product Officer at Pulzze Systems, in this SYS-CON.tv interview at @ThingsExpo, held June 6-8, 2017, at the Javits Center in New York City, NY.
In the enterprise today, connected IoT devices are everywhere – both inside and outside corporate environments. The need to identify, manage, control and secure a quickly growing web of connections and outside devices is making the already challenging task of security even more important, and onerous. In his session at @ThingsExpo, Rich Boyer, CISO and Chief Architect for Security at NTT i3, discussed new ways of thinking and the approaches needed to address the emerging challenges of security i...
The financial services market is one of the most data-driven industries in the world, yet it’s bogged down by legacy CPU technologies that simply can’t keep up with the task of querying and visualizing billions of records. In his session at 20th Cloud Expo, Karthik Lalithraj, a Principal Solutions Architect at Kinetica, discussed how the advent of advanced in-database analytics on the GPU makes it possible to run sophisticated data science workloads on the same database that is housing the rich...
SYS-CON Events announced today that Massive Networks will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Massive Networks mission is simple. To help your business operate seamlessly with fast, reliable, and secure internet and network solutions. Improve your customer's experience with outstanding connections to your cloud.
"We want to show that our solution is far less expensive with a much better total cost of ownership so we announced several key features. One is called geo-distributed erasure coding, another is support for KVM and we introduced a new capability called Multi-Part," explained Tim Desai, Senior Product Marketing Manager at Hitachi Data Systems, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
DX World EXPO, LLC., a Lighthouse Point, Florida-based startup trade show producer and the creator of "DXWorldEXPO® - Digital Transformation Conference & Expo" has announced its executive management team. The team is headed by Levent Selamoglu, who has been named CEO. "Now is the time for a truly global DX event, to bring together the leading minds from the technology world in a conversation about Digital Transformation," he said in making the announcement.
Internet of @ThingsExpo, taking place October 31 - November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 21st Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The Internet of Things (IoT) is the most profound change in personal and enterprise IT since the creation of the Worldwide Web more than 20 years ago. All major researchers estimate there will be tens of billions devic...
"The Striim platform is a full end-to-end streaming integration and analytics platform that is middleware that covers a lot of different use cases," explained Steve Wilkes, Founder and CTO at Striim, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
With tough new regulations coming to Europe on data privacy in May 2018, Calligo will explain why in reality the effect is global and transforms how you consider critical data. EU GDPR fundamentally rewrites the rules for cloud, Big Data and IoT. In his session at 21st Cloud Expo, Adam Ryan, Vice President and General Manager EMEA at Calligo, will examine the regulations and provide insight on how it affects technology, challenges the established rules and will usher in new levels of diligence...
SYS-CON Events announced today that Calligo, an innovative cloud service provider offering mid-sized companies the highest levels of data privacy and security, has been named "Bronze Sponsor" of SYS-CON's 21st International Cloud Expo ®, which will take place on Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Calligo offers unparalleled application performance guarantees, commercial flexibility and a personalised support service from its globally located cloud plat...
SYS-CON Events announced today that SkyScale will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. SkyScale is a world-class provider of cloud-based, ultra-fast multi-GPU hardware platforms for lease to customers desiring the fastest performance available as a service anywhere in the world. SkyScale builds, configures, and manages dedicated systems strategically located in maximum-securit...
21st International Cloud Expo, taking place October 31 - November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA, will feature technical sessions from a rock star conference faculty and the leading industry players in the world. Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud strategy. Me...
SYS-CON Events announced today that DXWorldExpo has been named “Global Sponsor” of SYS-CON's 21st International Cloud Expo, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Digital Transformation is the key issue driving the global enterprise IT business. Digital Transformation is most prominent among Global 2000 enterprises and government institutions.
SYS-CON Events announced today that Datera, that offers a radically new data management architecture, has been named "Exhibitor" of SYS-CON's 21st International Cloud Expo ®, which will take place on Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Datera is transforming the traditional datacenter model through modern cloud simplicity. The technology industry is at another major inflection point. The rise of mobile, the Internet of Things, data storage and Big...
"With Digital Experience Monitoring what used to be a simple visit to a web page has exploded into app on phones, data from social media feeds, competitive benchmarking - these are all components that are only available because of some type of digital asset," explained Leo Vasiliou, Director of Web Performance Engineering at Catchpoint Systems, in this SYS-CON.tv interview at DevOps Summit at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
"Outscale was founded in 2010, is based in France, is a strategic partner to Dassault Systémes and has done quite a bit of work with divisions of Dassault," explained Jackie Funk, Digital Marketing exec at Outscale, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
"We were founded in 2003 and the way we were founded was about good backup and good disaster recovery for our clients, and for the last 20 years we've been pretty consistent with that," noted Marc Malafronte, Territory Manager at StorageCraft, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
While the focus and objectives of IoT initiatives are many and diverse, they all share a few common attributes, and one of those is the network. Commonly, that network includes the Internet, over which there isn't any real control for performance and availability. Or is there? The current state of the art for Big Data analytics, as applied to network telemetry, offers new opportunities for improving and assuring operational integrity. In his session at @ThingsExpo, Jim Frey, Vice President of S...