Welcome!

Big Data Journal Authors: Liz McMillan, Frank Huerta, Elizabeth White, Pat Romanski, Esmeralda Swartz

Related Topics: SOA & WOA, Java, Search, Web 2.0, Cloud Expo, Big Data Journal

SOA & WOA: Article

Five Uses for Log Data That You Never Thought of

Cars, law enforcement, lighting, marketing metrics, and beer and more...

When you think of logs, what do you think of? It's most likely troubleshooting software applications and the infrastructure that underlies them, keeping an eye on your production apps...perhaps even database logs and some other things like that. Traditional log management stuff...I'm guessing it's not sports cars, law enforcement, lighting, marketing metrics, and beer. Well guess what? It can be!

1. Debunk an Inaccurate Journalist
Back in February of 2013 The New York Times published a negative review of the Tesla Model S. To summarize, the reviewer wrote of a very negative experience that nearly left him stranded in the middle of the highway. Despite using range-optimization techniques, he elaborated, the driving distance that a Model S can go on one charge was much lower than stated by the company. The story got a lot of press as Tesla was ready to roll out a large number of the cars, with quite a bit of fanfare, in short order.

Lucky for Tesla, they had data on their side...log data that is, which was published to the Tesla blog. When NY Times author John Broder published his review, Tesla founder Elon Musk found his experience strange as it didn't fit with other reporters' stories or test data. Musk's post went on to pick apart every one of Broder's experiences and show that the reporter was being less than accurate with many facts. Check out the blog post to see how log data was used to debunk the journalist's stories, complete with some great graphs, annotated maps and more...all courtesy of the log data collected by the car's computer.

2. Catch an Alleged Hacker
In a series of events that began in September 2010, Reddit co-founder Aaron Swartz allegedly began mass downloading documents from MIT's JSTOR, an online journal archive. As this article from MIT's The Tech newspaper states, it was "'network flow data' and DCBP log information" provided to the Secret Service by MIT that led to his arrest and charges being filed. (Sadly, after The Tech article was written, Aaron Swartz committed suicide under the pressure of the criminal case against him.)

3. Turn a Light On & Off with Log Data
As first detailed on the Logentries blog by user Jason Ruane, technical director at Moposa, log data can be used to switch lights on and off. While Jason uses this functionality to convey high-priority alerts to his team, it's easy to see other ways that this methodology could be used. Once you can use logs to turn electronics on and off, that opens the door to gathering data and using it to control the environment. Which brings us to our final example...

4. Make Better Beer
As a craft beer lover, this last one hits home for me and shows how log data can be used pretty much anywhere. A company called BrewBit is releasing a set of sensors to help in brewing beer. The first will be a temperature controller that you can use to log your temperature data as you ferment beer (to ensure a good fermentation) or put it into a kegerator (to ensure ideal serving temperature). With setups like we discussed above, temperate can be controlled to maintain ideal conditions. As they role out more sensors to monitor other data points, this log data can be used to see how different factors affect each other over time.

5. Track Marketing Metrics
In this blog post from back in October, we detailed how we here at Logentries eat our own dogfood across every department. In it, we showed how Marketing at Logentries uses log data, along with tags, alerts, search and graphing, to understand how well our marketing efforts our working and how people are using the product. Since then, we've even began using the logs to understand the result of A|B tests we've run on the site. Once you've got this data, there's so much information you can pull out of it, and Logentries helps us do that.

So, as you can see, the use for log data is myriad...and these examples only just begin to scratch the surface of uses. To be honest, though, once I found out you could use it to make better beer, I was sold ;)

Win a Logentries T-Shirt
Send us your own interesting logs as data story and we'll send you one of our brand new Logentries t-shirts! Just email [email protected].

Of course, there are many more traditional use cases currently for log data that are equally valuable, such as monitoring your application, making sure everything is running smooth and giving you a way to quickly debug problems or repair your production environment. We're happy to talk about any of them, traditional or not. And if you're interested in checking out our log analysis tool, then please be our guest:

Logentries_Try_It_Free_Promo_W

More Stories By Trevor Parsons

Trevor Parsons is Chief Scientist and Co-founder of Logentries. Trevor has over 10 years experience in enterprise software and, in particular, has specialized in developing enterprise monitoring and performance tools for distributed systems. He is also a research fellow at the Performance Engineering Lab Research Group and was formerly a Scientist at the IBM Center for Advanced Studies. Trevor holds a PhD from University College Dublin, Ireland.

Cloud Expo Breaking News
There is no doubt that Big Data is here and getting bigger every day. Building a Big Data infrastructure today is no easy task. There are an enormous number of choices for database engines and technologies. To make things even more challenging, requirements are getting more sophisticated, and the standard paradigm of supporting historical analytics queries is often just one facet of what is needed. As Big Data growth continues, organizations are demanding real-time access to data, allowing immediate and actionable interpretation of events as they happen. Another aspect concerns how to deliver data in a meaningful way, one that really delivers on what end-users need to maintain a competitive position in fast-changing markets.
SYS-CON Events announced today that Gigaom Research has been named "Media Sponsor" of SYS-CON's 15th International Cloud Expo®, which will take place on November 4-6, 2014, at the Santa Clara Convention Center in Santa Clara, CA. Ashar Baig, Research Director, Cloud, at Gigaom Research, will also lead a Power Panel on the topic "Choosing the Right Cloud Option." Gigaom Research provides timely, in-depth analysis of emerging technologies for individual and corporate subscribers. Gigaom Research's network of 200+ independent analysts provides new content daily that bridges the gap between breaking news and long-range research.
"We are automated capacity control software, which basically looks at all the supply and demand and running a virtual cloud environment and does a deep analysis of that and says where should things go," explained Andrew Hillier, Co-founder & CTO of CiRBA, in this SYS-CON.tv interview at the 14th International Cloud Expo®, held June 10-12, 2014, at the Javits Center in New York City. Cloud Expo® 2014 Silicon Valley, November 4–6, at the Santa Clara Convention Center in Santa Clara, CA, will feature technical sessions from a rock star conference faculty and the leading Cloud industry players in the world.
Almost everyone sees the potential of Internet of Things but how can businesses truly unlock that potential. The key will be in the ability to discover business insight in the midst of an ocean of Big Data generated from billions of embedded devices via Systems of Discover. Businesses will also need to ensure that they can sustain that insight by leveraging the cloud for global reach, scale and elasticity. In his session at Internet of @ThingsExpo, Mac Devine, Distinguished Engineer at IBM, will discuss bringing these three elements together via Systems of Discover.
"In my session I spoke about enterprise cloud analytics and how we can leverage analytics as a service," explained Ajay Budhraja, CTO at the Department of Justice, in this SYS-CON.tv interview at the 14th International Cloud Expo®, held June 10-12, 2014, at the Javits Center in New York City. Cloud Expo® 2014 Silicon Valley, November 4–6, at the Santa Clara Convention Center in Santa Clara, CA, will feature technical sessions from a rock star conference faculty and the leading Cloud industry players in the world.
“We are starting to see people move beyond the commodity cloud and enterprises need to start focusing on additional value added services in order to really drive their adoption," explained Jason Mondanaro, Director of Product Management at MetraTech, in this SYS-CON.tv interview at the 14th International Cloud Expo®, held June 10-12, 2014, at the Javits Center in New York City. Cloud Expo® 2014 Silicon Valley, November 4–6, at the Santa Clara Convention Center in Santa Clara, CA, will feature technical sessions from a rock star conference faculty and the leading Cloud industry players in the world.
The Internet of Things promises to transform businesses (and lives), but navigating the business and technical path to success can be difficult to understand. In his session at 15th Internet of @ThingsExpo, Chad Jones, Vice President, Product Strategy of LogMeIn's Xively IoT Platform, will show you how to approach creating broadly successful connected customer solutions using real world business transformation studies including New England BioLabs and more.
All too many discussions about DevOps conclude that the solution is an all-purpose player: developer and operations guru, complete with pager for round-the-clock duty. For most organizations that is not the way forward. In his session at DevOps Summit, Bernard Golden, Vice President of Strategy at ActiveState, will discuss how to achieve the agility and speed of end-to-end automation without requiring an organization stocked with Supermen and Superwomen.
“The Internet of Things is a wave that has arrived and it’s growing really fast. The concern at Aria Systems is making sure that people understand the ramifications of their attempts to monetize whatever it is they build on the Internet of Things," explained C Brendan O’Brien, Co-founder and Chief Architect at Aria Systems, in this SYS-CON.tv interview at the Internet of @ThingsExpo, held June 10-12, 2014, at the Javits Center in New York City. Internet of @ThingsExpo 2014 Silicon Valley, November 4–6, at the Santa Clara Convention Center in Santa Clara, CA, will feature technical sessions from a rock star conference faculty and the leading IoT industry players in the world.
“About two years ago Brother launched a new group called Brother Online. We thought it was a good idea for a hardware company to get into the cloud services market and our first step into that market was web conferencing,” explained Courtney Behrens, Senior Marketing Manager at Brother International, in this SYS-CON.tv interview at the 14th International Cloud Expo®, held June 10-12, 2014, at the Javits Center in New York City. Cloud Expo® 2014 Silicon Valley, November 4–6, at the Santa Clara Convention Center in Santa Clara, CA, will feature technical sessions from a rock star conference faculty and the leading Cloud industry players in the world.
"So many teams have been stuck in their own practices for the last ten years and it's really hard to sell all the latest and greatest tools, but the reality is all the tools around operations have evolved so much you are starting to see a big shift toward upgrading that tool set, focusing more on automation, more on cloud," explained Dustin Whittle, Developer Evangelist at AppDynamics, in this SYS-CON.tv interview at the DevOps Summit, held June 10-12, 2014, at the Javits Center in New York City.
“We provide disaster recovery services as well as solutions. We also provide back-up solutions that work across your internal on-premise assets as well as in the public and private cloud," stated Joel Ferman, Vice President of Marketing at InMage Systems, in this SYS-CON.tv interview at the 14th International Cloud Expo® (http://www.CloudComputingExpo.com/), held June 10-12, 2014, at the Javits Center in New York City. Cloud Expo® 2014 Silicon Valley, November 4–6, at the Santa Clara Convention Center in Santa Clara, CA, will feature technical sessions from a rock star conference faculty and the leading Cloud industry players in the world.
SYS-CON Events announced today that Harbinger Systems will exhibit at SYS-CON's 15th International Cloud Expo®, which will take place on November 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA. Harbinger Systems is a global company providing software technology services. Since 1990, Harbinger has developed a strong customer base worldwide. Its customers include software product companies ranging from hi-tech start-ups in Silicon Valley to leading product companies in the US and large in-house IT organizations.
“Distrix fits into the overall cloud and IoT model around software-defined networking. There’s a broad category around software-defined networking that’s focused on data center, and we focus on the WAN,” explained Jay Friedman, President of Distrix, in this SYS-CON.tv interview at the Internet of @ThingsExpo, held June 10-12, 2014, at the Javits Center in New York City. Internet of @ThingsExpo 2014 Silicon Valley, November 4–6, at the Santa Clara Convention Center in Santa Clara, CA, will feature technical sessions from a rock star conference faculty and the leading IoT industry players in the world.
“Dell Cloud Manager is a cloud management environment for the consumption of cloud resources and we provide a consistent interface, both in terms of API and in terms of user interface for doing a wide variety of activities core to deploying and operating software in cloud," explained James Urquhart, Technologist & Director of Cloud Management Solutions at Dell, in this SYS-CON.tv interview at the 14th International Cloud Expo®, held June 10-12, 2014, at the Javits Center in New York City. Cloud Expo® 2014 Silicon Valley, November 4–6, at the Santa Clara Convention Center in Santa Clara, CA, will feature technical sessions from a rock star conference faculty and the leading Cloud industry players in the world.