Welcome!

@BigDataExpo Authors: Elizabeth White, Yeshim Deniz, Liz McMillan, Carmen Gonzalez, William Schmarzo

Related Topics: @BigDataExpo, Microsoft Cloud, @DevOpsSummit

@BigDataExpo: Blog Post

Detecting Bad Deployments [#APM with #DevOps] | @DevOpsSummit

Resource usage monitoring is as important as response time and throughput.

Detecting Bad Deployments on Resource Impact & Not Response Time

This story came in from Joseph - one of our fellow dynaTrace users and a performance engineer at a large fleet management service company. Their fleet management software runs on .NET, is developed in-house, is load tested with JMeter and monitored in Production with dynaTrace. A usage and configuration change of their dependency injection library turned out to dramatically impact CPU and memory usage while not yet impacting end user experience. Lessons learned: resource usage monitoring is as important as response time and throughput.

On Wednesday, July 3, Joseph's ops team deployed the latest version into their production environment. Load (=throughput) and response time are two key application health measures the application owner team has on their production dashboards. Especially after a new deployment, an unchanged behavior is what you are looking/ hoping for:

APM Lesson #1: Don't judge a deployment based on response time and throughput alone

Joseph is an experienced performance engineer, so he knew to check more than just these two metrics. Looking at host and process monitoring data he identified a 6x increase in CPU and 40% increase in process memory usage, which alarmed the folks responsible for the infrastructure that hosted the ASP.NET Services:

APM Lesson #2: Expand your alerting to resource usage and infrastructure health

Despite the fact that the app still shows the same response time and throughput this is very alarming. Why? Because it means that the application is much more expensive to run as compared to the previous build.

Whether it runs on physical, virtualized or in a cloud environment: more resource consumption means higher operational costs. And who knows what happens during peak load: will CPU and memory be enough or will it eventually impact response time and throughput? That's the reason why analyzing changes in resource consumption is as critical to performance engineering as optimizing response time and throughput.

If you are a performance engineer, application owner or developer I suggest you read the rest of the blog to learn about Joseph's best practices on analyzing and dealing with situations like this and what they changed in their development and deployment process:

Step 1: Analyzing the Impact of Memory Usage on CPU
Runtime engines such as the JVM (more on Java Enterprise Performance) or the .NET CLR (more on.NET Performance Analysis) provide detailed performance counters to start analyzing the resource usage hotspots on CPU and memory. A good start is to look at memory usage in the different heap generations as well as the activity of the garbage collector. The following screenshot shows the .NET garbage collection generations (# of Collections and Heap Size) as well as the time spent in garbage collection metrics of their production environment:

APM Lesson #3: Look at memory metrics to figure out how memory usage and GC behavior has changed

What can you read from these two graphs?

  1. Most of the objects live in generation 0 and never get promoted to an older generation. That means we are talking about short-lived objects that are probably created and cleared on a per request basis. If we saw a shift to older generations it is more likely that these objects are part of a cache.
  2. We don't seem to have a memory leak as the garbage collector seems to clear objects from Gen 0 and not promote them to higher generations, which could eventually lead to an out-of-memory crash.
  3. The increase in # of collections as well as time spent in garbage collection indicates that the application either uses more short-lived objects or these objects grew significantly in size.

Step 3: More Objects or Larger Objects or Both?
The next step is to find out what has changed in terms of objects on the heap. Joseph therefore takes regular dynaTrace memory snapshots (YouTube Tutorial) and keeps them to later compare heap usage patterns between releases. The next screenshot shows the difference between the previous version and the current version and highlights that there are not only about twice as many ASP.NET MVC controller objects on the heap but these objects are also about 6x larger than before:

APM Lesson #4: Continuously take memory snapshots to identify regressions between builds

It turned out it that this pattern was not exclusive to the Planning Controller shown in the screenshot but happened across all ASP.NET MVC Controller objects. Following the Deep Memory Leak Snapshots best practices, Joseph found the actual referenced objects that caused these controller objects to grow in size. It turned out to be related to objects that were instantiated and referenced using dependency injection.

Finding and identifying the problem was the first step which raised two more questions:

  1. Why wasn't this found earlier?
  2. Which code change caused this problem?

For answers, click here for the full article.

DevOps: Disruptive but Essential in a Cloud Computing Universe

The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is no time to wait for long development cycles that produce software that is obsolete at launch. DevOps may be disruptive, but it is essential.

The DevOps Summit at Cloud Expo--to be held November 4-6 at the Santa Clara Convention Center in the heart of Silicon Valley--will expand the DevOps community, enable a wide sharing of knowledge, and educate delegates and technology providers alike. Recent research has shown that DevOps dramatically reduces development time, the amount of enterprise IT professionals put out fires, and support time generally. Time spent on infrastructure development is significantly increased, and DevOps practitioners report more software releases and higher quality.

DevOps Summit content doubled from a single track in New York to two simultaneous tracks: "Dev" Developer Focus and "Ops" Operations Focus.

Join DevOps Summit Chair Andi Mann and Host Alan Shimel, November 4-6 for three days of intense DevOps discussion and focus.

We'll see you in Santa Clara! See you in November!

DevOps Summit Conference Schedule Announced! ▸ Here

Register For DevOps Summit "FREE" (limited time) ▸ Here

Website: DevOps Summit

Twitter: @DevOpsSummit

Bookmark "DevOps Journal" ▸ here

@DevOpsSummit Silicon Valley (November 4-6, 2014, Santa Clara, CA)
@DevOPsSummit New York (June 9-11, 2015, New York, NY)

DevOps Summit New York 2015 Call for Papers Now Open

Call for Papers will remain open through December 31, 2014. DevOps Summit New York 2015 will take place June 9-11, at the Javits Center, in New York City. Tracks and topics for DevOps Summit at Cloud Expo are listed below. Submit your speaking proposals here DevOps Summit Call for Papers

  • Application delivery
  • ROI and business value
  • Site reliability
  • Continuous delivery
  • Kanban, Scrum, and Agile
  • Continuous integration
  • Release scalability
  • Continuous release
  • Service virtualization
  • Operational monitoring
  • Capacity management
  • Load testing
  • Quality assurance
  • Manager and exec support
  • Release automation
  • Cultural change
  • Breaking down IT silos
  • Cloud development
  • Cloud platforms
  • Test automation
  • Teaming
  • Application security
  • API management
  • Identity and access
  • Audit and control
  • DevOps and ITIL, ISO, Six Sigma, COBIT
  • Mobile DevOps
  • DevOps for legacy systems
  • Software-defined servers
  • Network automation
  • Server automation
  • Configuration automation
  • Continuous support
  • DevOps anti-patterns
  • Enterprise DevOps
  • DevOps system architecture
  • IT orchestration
  • Containerization
  • DevOps skills and training
  • WebOps, CloudOps, ChatOps, NoOps
  • Change management

Video Highlights From DevOps Summit at Cloud Expo New York

While there are many ways to define DevOps, the goal of the concept is to be able to deliver IT solutions faster, leveraging several technology tools to add value for business. Cloud companies have demonstrated how they can manage massive pools of IT infrastructure, giving the developer a very flexible resource pool that they can leverage and IT is eager to do the same.


Is DevOps Really Changing How IT Is Working?

In this Power Panel at 2nd DevOps Summit moderated by Cloud Expo Tech Chair Larry Carvalho, Bernard Golden, VP, Strategy at ActiveState; Andi Mann, VP of Strategic Solutions at CA Technologies; John Willis, VP of Customer Enablement at Stateless Networks; Randy Bias, Co-Founder and CTO of Cloudscaling; and Vasu Sankhavaram, Chief Strategist of DevOps and Cross Portfolio Architecture Alignment within HP Software, discuss this movement that is delivering frictionless IT to the enterprise.


DevOps and Security

From showcase success stories from early adopters and web-scale businesses, DevOps is expanding to organizations of all sizes, including the world's largest enterprises - and delivering real results. In this DevOps Summit Power Panel (http://DevOpsSummit.SYS-CON.com/) on June 9 at our New York City studio, moderated by Andi Mann, DevOps Summit conference chair and VP of Strategic Solutions at CA Technologies, John Willis, VP of Customer Enablement for Stateless Networks; JP Morgenthal, Director, Cloud Computing Practice at Perficient, Inc.; Vanessa Alvarez, Cloud Expo Conference Chair and Marketing Team at Amazon; and Gordon Haff, Cloud Evangelist at Red Hat; discussed how security needs to be embraced and brought into the DevOps movement.


Is DevOps Changing Enterprise IT?

Born out of proven success in agile development, cloud computing, and process automation, DevOps is a macro trend you cannot afford to miss. In this DevOps Summit Power Panel on June 9 at our New York City studio, moderated by Andi Mann, DevOps Summit conference chair and VP of Strategic Solutions at CA Technologies, JP Morgenthal, Director of Cloud Computing Practice at Perficient, Inc., Gordon Haff, Cloud Evangelist at Red Hat; Larry Carvalho, Cloud Expo Tech Chair and Research Manager & Lead Analyst, Platform as a Service at IDC; and John Willis, VP of Customer Enablement for Stateless Networks; discussed how DevOps is changing the way IT works, how businesses interact with customers, and how organizations are buying, building, and delivering software.

Schedule for 15th @CloudExpo, @BigDataExpo and @ThingsExpo

Are you ready to put your data in the cloud?

What is the future of security in the cloud?

Does Docker quickly advance the development of an IoT application?

What are the implications of Moore's Law on Hadoop deployments?

Get all these questions and hundreds more like them answered at the 15th Cloud Expo, November 4-6, 2014, at the Santa Clara Convention Center, in Santa Clara, CA. The Cloud ExpoBig Data Expo@ThingsExpoDevOps Summitprograms are now available for you to inspect and investigate in advance.

Our upcoming November 4-6 event in Santa Clara, California will present a total of 10 simultaneous tracks by an all-star faculty, over three days, plus a two-day "Cloud Computing Bootcamp" presented by Janakiram MSV, an Analyst with the Gigaom Research analyst network, where he covers the Cloud Services landscape.

Cloud and Big Data topics and tracks include: Enterprise Cloud AdoptionAPM & Cloud Computing | Hot TopicsCloud APIs & BusinessCloud Security | MobilityBig Data | Analytics.

@ThingsExpo content tripled from a single track in New York to three simultaneous tracks: Consumer IoTEnterprise IoTIoT Developer | WebRTC Convergence.

Schedule for Cloud Expo / Big Data Expo / @ThingsExpo ▸ Here

Now that we have published the full conference schedule, please check back for daily updates as we finalize new session abstracts by working with our distinguished faculty members. For your questions please contact us at events (at) sys-con.com. Last but not least we will announce our keynotes on the hottest subjects to be delivered by world-class speakers!

The largest 'Internet of Things' event in the world has announced "sponsorship opportunities" and "call for papers."

The 1st International Internet of @ThingsExpo was launched this June at the Javits Center in New York City with over 6,000 delegates in attendance. The 2nd International Internet of @ThingsExpo will take place November 4-6, 2014, at the Santa Clara Convention Center in Santa Clara, California, with an estimated 7,000 plus delegates attending over three days.

@ThingsExpo is co-located with 15th International Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading IoT industry players in the world. In 2014, more than 200 companies will be present on the @ThingsExpo show floor, including global players and the hottest new technology pioneers.

Sponsorship and Exhibit Opportunities for @DevOpsSummit Silicon Valley and New York Are Now Available
Sponsors and exhibitors of Internet of @DevOpsSummit will benefit from unmatched branding, profile building and lead generation opportunities through:

  • Featured on-site presentation and ongoing on-demand webcast exposure to a captive audience of industry decision-makers.
  • Showcase exhibition during our new extended dedicated expo hours
  • Breakout Session Priority scheduling for sponsors that have been guaranteed a 35-minute technical session
  • Online advertising in SYS-CON's i-Technology publications
  • Capitalize on our comprehensive marketing efforts leading up to the show with print mailings, e-newsletters and extensive online media coverage.
  • Unprecedented PR Coverage: Editorial coverage on IoT.sys-con.com, Tweets to our 75,000 plus followers, press releases sent on major wire services to over 500 combined analysts and press members who attended Internet of @ThingsExpo - making it the best-covered "Internet of Things" conference in the world

For more information on sponsorship, exhibit, and keynote opportunities contact Carmen Gonzalez by email at events (at) sys-con.com, or by phone 201 802-3021. Book both events for additional savings!

@DevOpsSummit Silicon Valley (November 4-6, 2014, Santa Clara, CA)
@DevOPsSummit New York (June 9-11, 2015, New York, NY)

Secure Your VIP Pass to Attend @DevOpsSummit Silicon Valley
DevOps Summit at Cloud Expo announced today a limited time free "Expo Plus" registration option. The onsite registration price of $600 will be set to 'free' for delegates who register during special offer period.

To take advantage of this opportunity, attendees can use the coupon code and secure their registration to attend all keynotes and general sessions, as well as a limited number of technical sessions each day of the show, in addition to full access to the expo floor.

The registration page is located here.


@ThingsExpo New York 2015 'Call for Papers' Now Open
The 3rd International Internet of @ThingsExpo, to be held June 9-11, 2015, at the Javits Center in New York City, New York announces that its 'Call for Papers' is now open. The event will feature a world class, all-star faculty with the hottest IoT topics covered in three distinct tracks.

Track 1 - Consumer IoT and Wearables: Smart Appliances, Wearables, Smart Cars, Smartphones 2.0, Smart Travel, Personal Fitness, Health Care, Personalized Marketing, Customized Shopping, Personal Finance, The Digital Divide, Mobile Cash & Markets, Games & the IoT, The Future of Education, Virtual Reality

Track 2 - Enterprise IoT: The Business Case for IoT, Smart Grids, Smart Cities, Smart Transportation, The Smart Home, M2M, Authentication/Security, Wiring the IoT, The Internet of Everything, Digital Transformation of Enterprise IT, Agriculture, Transportation, Manufacturing, Local & State Government, Federal Government

Track 3 -  Developer IoT: WebRTC, Eclipse Foundation, Cloud Foundry, Docker & Linux Containers, Node-Red, Open Source Hardware, Leveraging SOA, Multi-Cloud IoT, Evolving Standards, WebSockets, Security & Privacy Protocols, GPS & Proximity Services, Bluetooth/RFID/etc., XMPP, Nest Labs


@ThingsExpo billboard is viewed by more than 1.3 million motorists per week on Highway 101, in the heart of Silicon Valley

Help plant your flag in the fast-expanding business opportunity that is the Internet of Things: Submit your speaking proposal today here!

Download @ThingsExpo Newsletter Today ▸ Here

Chris Matthieu Named @ThingsExpo Tech Chair

Internet of @ThingsExpo named Chris Matthieu tech chair of Internet of @ThingsExpo 2014 Silicon Valley.

Chris Matthieu has two decades of telecom and web experience. He launched his Teleku cloud communications-as-a-service platform at eComm in 2010, which was acquired by Voxeo. Next he built an open source Node.JS PaaS called Nodester, which was acquired by AppFog. His latest startups include Twelephone. Leveraging HTML5 and WebRTC, Twelephone's BHAG (Big Hairy Audacious Goal) is to become the next generation telecom company running in the Web browser. Chris is currently co-founder and CTO of Octoblu.

Website: http://www.ThingsExpo.com

Twitter: http://www.Twitter.com/ThingsExpo

About SYS-CON Media & Events
SYS-CON Media (www.sys-con.com) has since 1994 been connecting technology companies and customers through a comprehensive content stream - featuring over forty focused subject areas, from Cloud Computing to Web Security - interwoven with market-leading full-scale conferences produced by SYS-CON Events. The company's internationally recognized brands include among othersCloud Expo® (CloudComputingExpo.com@CloudExpo), Big Data Expo (BigDataExpo.net@BigDataExpo), DevOps Summit (DevOpsSummit.sys-con.com@DevOpsSummit),  Internet of @ThingsExpo (ThingsExpo.com@ThingsExpo) and Cloud Computing Bootcamp (CloudComputingBootcamp.com).

Cloud Expo®, Big Data Expo® and @ThingsExpo® are registered trademarks of Cloud Expo, Inc., a SYS-CON Events company.

More Stories By Andreas Grabner

Andreas Grabner has been helping companies improve their application performance for 15+ years. He is a regular contributor within Web Performance and DevOps communities and a prolific speaker at user groups and conferences around the world. Reach him at @grabnerandi

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


@BigDataExpo Stories
SYS-CON Events announced today that delaPlex will exhibit at SYS-CON's @CloudExpo, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. delaPlex pioneered Software Development as a Service (SDaaS), which provides scalable resources to build, test, and deploy software. It’s a fast and more reliable way to develop a new product or expand your in-house team.
Most DevOps journeys involve several phases of maturity. Research shows that the inflection point where organizations begin to see maximum value is when they implement tight integration deploying their code to their infrastructure. Success at this level is the last barrier to at-will deployment. Storage, for instance, is more capable than where we read and write data. In his session at @DevOpsSummit at 20th Cloud Expo, Josh Atwell, a Developer Advocate for NetApp, will discuss the role and value...
In order to meet the rapidly changing demands of today’s customers, companies are continually forced to redefine their business strategies in order to meet these needs, stay relevant and continue to see profitable growth. IoT deployment and development is integral in this transformation, and today businesses are increasingly seeing the value of investing their resources into IoT deployments. These technologies are able increase ROI through projects such as connecting supply chains or enabling sm...
SYS-CON Events announced today that Cloudistics, an on-premises cloud computing company, has been named “Bronze Sponsor” of SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Cloudistics delivers a complete public cloud experience with composable on-premises infrastructures to medium and large enterprises. Its software-defined technology natively converges network, storage, compute, virtualization, and management into a ...
New competitors, disruptive technologies, and growing expectations are pushing every business to both adopt and deliver new digital services. This ‘Digital Transformation’ demands rapid delivery and continuous iteration of new competitive services via multiple channels, which in turn demands new service delivery techniques – including DevOps. In this power panel at @DevOpsSummit 20th Cloud Expo, moderated by DevOps Conference Co-Chair Andi Mann, panelists will examine how DevOps helps to meet th...
A strange thing is happening along the way to the Internet of Things, namely far too many devices to work with and manage. It has become clear that we'll need much higher efficiency user experiences that can allow us to more easily and scalably work with the thousands of devices that will soon be in each of our lives. Enter the conversational interface revolution, combining bots we can literally talk with, gesture to, and even direct with our thoughts, with embedded artificial intelligence, whic...
Every successful software product evolves from an idea to an enterprise system. Notably, the same way is passed by the product owner's company. In his session at 20th Cloud Expo, Oleg Lola, CEO of MobiDev, will provide a generalized overview of the evolution of a software product, the product owner, the needs that arise at various stages of this process, and the value brought by a software development partner to the product owner as a response to these needs.
SYS-CON Events announced today that A&I Solutions has been named “Bronze Sponsor” of SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Founded in 1999, A&I Solutions is a leading information technology (IT) software and services provider focusing on best-in-class enterprise solutions. By partnering with industry leaders in technology, A&I assures customers high performance levels across all IT environments including: mai...
Most technology leaders, contemporary and from the hardware era, are reshaping their businesses to do software in the hope of capturing value in IoT. Although IoT is relatively new in the market, it has already gone through many promotional terms such as IoE, IoX, SDX, Edge/Fog, Mist Compute, etc. Ultimately, irrespective of the name, it is about deriving value from independent software assets participating in an ecosystem as one comprehensive solution.
SYS-CON Events announced today that EARP will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. "We are a software house, so we perfectly understand challenges that other software houses face in their projects. We can augment a team, that will work with the same standards and processes as our partners' internal teams. Our teams will deliver the same quality within the required time and budget just as our partn...
SYS-CON Events announced today that delaPlex will exhibit at SYS-CON's @ThingsExpo, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. delaPlex pioneered Software Development as a Service (SDaaS), which provides scalable resources to build, test, and deploy software. It’s a fast and more reliable way to develop a new product or expand your in-house team.
SYS-CON Events announced today that Tappest will exhibit MooseFS at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. MooseFS is a breakthrough concept in the storage industry. It allows you to secure stored data with either duplication or erasure coding using any server. The newest – 4.0 version of the software enables users to maintain the redundancy level with even 50% less hard drive space required. The software func...
IBM helps FinTechs and financial services companies build and monetize cognitive-enabled financial services apps quickly and at scale. Hosted on IBM Bluemix, IBM’s platform builds in customer insights, regulatory compliance analytics and security to help reduce development time and testing. In his session at 20th Cloud Expo, Tom Eck, Industry Platforms CTO at IBM Cloud, will discuss how these tools simplify the time-consuming tasks of selection, mapping and data integration, allowing developers ...
SYS-CON Events announced today that Outscale, a global pure play Infrastructure as a Service provider and strategic partner of Dassault Systèmes, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Founded in 2010, Outscale simplifies infrastructure complexities and boosts the business agility of its customers. Outscale delivers a secure, reliable and industrial strength solution for its customers, which in...
SYS-CON Events announced today that Outscale will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Outscale's technology makes an automated and adaptable Cloud available to businesses, supporting them in the most complex IT projects while controlling their operational aspects. You boost your IT infrastructure's reactivity, with request responses that only take a few seconds.
SYS-CON Events announced today that Systena America will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Systena Group has been in business for various software development and verification in Japan, US, ASEAN, and China by utilizing the knowledge we gained from all types of device development for various industries including smartphones (Android/iOS), wireless communication, security technology and IoT serv...
DevOps at Cloud Expo – being held October 31 - November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA – announces that its Call for Papers is open. Born out of proven success in agile development, cloud computing, and process automation, DevOps is a macro trend you cannot afford to miss. From showcase success stories from early adopters and web-scale businesses, DevOps is expanding to organizations of all sizes, including the world's largest enterprises – and delivering real r...
Cloud applications are seeing a deluge of requests to support the exploding advanced analytics market. “Open analytics” is the emerging strategy to deliver that data through an open data access layer, in the cloud, to be directly consumed by external analytics tools and popular programming languages. An increasing number of data engineers and data scientists use a variety of platforms and advanced analytics languages such as SAS, R, Python and Java, as well as frameworks such as Hadoop and Spark...
SYS-CON Events announced today that CollabNet, a global leader in enterprise software development, release automation and DevOps solutions, will be a Bronze Sponsor of SYS-CON's 20th International Cloud Expo®, taking place from June 6-8, 2017, at the Javits Center in New York City, NY. CollabNet offers a broad range of solutions with the mission of helping modern organizations deliver quality software at speed. The company’s latest innovation, the DevOps Lifecycle Manager (DLM), supports Value S...
SYS-CON Events announced today that Peak 10, Inc., a national IT infrastructure and cloud services provider, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Peak 10 provides reliable, tailored data center and network services, cloud and managed services. Its solutions are designed to scale and adapt to customers’ changing business needs, enabling them to lower costs, improve performance and focus intern...