Welcome!

Big Data Journal Authors: Paige Leidig, Trevor Parsons, Lori MacVittie, Cynthia Dunlop, Elizabeth White

Related Topics: Big Data Journal, Security, SDN Journal

Big Data Journal: Article

The Role of Big Data in Threat Detection

The promise of Big Data lies in the ability to derive insight, reduce risk, and improve future security performance

Arriving at Actionable Insight: The Role of Big Data in Threat Detection

According to Gartner, Big Data refers to "high volume, high velocity, and/or high variety information assets" - and, this is the key - "that require new forms of processing to enable enhanced decision making, insight discovery and process optimization." While Big Data may seem like an invaluable tool that all security teams should try to leverage, it is not practical for everyone to attempt to harness it on their own. Finding insight from data is rarely as simple as it seems. We are still in the early stages of the Big Data revolution, with people only now beginning to understand what is possible, and what it takes to get there. Simply investing in tools and development is not enough. The fact is security teams are still struggling to identify and respond to incidents in an effective way. The Verizon Data Breach Investigations Report of 2013 noted that outside parties, whether it be a telecom provider, credit card issuer, third-party vendor or the FBI, were responsible for 70% of data breach notification, demonstrating that security teams are still missing the signs of detrimental threats that face organizations each and every day.

There is clearly promise in Big Data, but how do organizations get there? First, there is a need for human talent and expertise, as tools alone are not enough. Beyond creating a security operations center (SOC) to coordinate a cyber security strategy, it is critical for organizations to employ a data scientist or someone who is capable of consuming and analyzing the information to create effective models for identifying threats. Unfortunately, there is a vast talent gap in the field of data science, particularly at the intersection of data and security. There are also technical hurdles preventing development. Integrating high volumes and varieties of data sources and formats, both internal and external, into a security framework requires both technical expertise and resources. In many ways, these programs are for select organizations. Enterprises must facilitate financial resources, technical know-how, and data science expertise to execute a holistic and effective Big Data program.

The promise of Big Data lies not in the collection of millions of records, but in the ability to derive insight, reduce risk, and improve future security performance. So, if the barriers to an internal Big Data program are high, but the potential benefits are great, how do we arrive at the insight needed to reduce risk and improve future security performance? Fortunately, there are options available for organizations to gain these advantages without having to make the commitment to a full-scale Big Data program.

One emerging option is the possibility of Big Data as a Service (BDaaS). Through the perimeterless nature of the Internet, vendors can access, analyze, and provide actionable insight into potential - or even future - threats. For example, card issuers often turn to outside vendors for Common Point of Purchase (CPP) analysis to detect potential fraud associated with theft or breach. By outsourcing the collection and analysis, businesses can streamline their path to insight.

Organizations of all sizes face challenges of data collection and analysis on a daily basis. In order to gain insight from data, companies must invest in the tools, strategies, and staff needed to make sense of accessible information. Once the appropriate protocol is in place, insight from Big Data may function as a way of reducing risk, protecting enterprises from our hostile threat landscape.

More Stories By Stephen Boyer

Stephen Boyer cofounded BitSight in 2011 and serves as Chief Technology Officer. Prior to founding BitSight, he was President and Cofounder of Saperix, a company spun out of the MIT Lincoln Laboratory focused on vulnerability and network topology risk analysis. Saperix was acquired by FireMon in 2011.

While at the MIT Lincoln Laboratory, Stephen was a member of the Cyber Systems and Technology Group where he led R&D programs solving large-scale national cybersecurity problems. His work at the MIT Lincoln Laboratory included research, development, and evaluation of next generation intrusion detection correlation architectures, attack graph vulnerability analysis, large-scale cyber situational awareness, security risk measurement, and cyber simulation and testing.

Prior to joining the MIT Lincoln Laboratory, Stephen designed, developed, and tested products at one of the earliest Linux startup companies, Caldera Systems.

Stephen holds a Bachelors degree in Computer Science from Brigham Young University and Master of Science in Engineering and Management from the Massachusetts Institute of Technology.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


@BigDataExpo Stories
Dyn solutions are at the core of Internet Performance. Through traffic management, message management and performance assurance, Dyn is connecting people through the Internet and ensuring information gets where it needs to go, faster and more reliably than ever before. Founded in 2001 at WPI, Dyn’s global presence services more than four million enterprise, small business and personal customers.
SimpleECM is the only platform to offer a powerful combination of enterprise content management (ECM) services, capture solutions, and third-party business services providing simplified integrations and workflow development for solution providers. SimpleECM is opening the market to businesses of all sizes by reinventing the delivery of ECM services. Our APIs make the development of ECM services simple with the use of familiar technologies for a frictionless integration directly into web applicat...
Samsung VP Jacopo Lenzi, who headed the company's recent SmartThings acquisition under the auspices of Samsung's Open Innovaction Center (OIC), answered a few questions we had about the deal. This interview was in conjunction with our interview with SmartThings CEO Alex Hawkinson. IoT Journal: SmartThings was developed in an open, standards-agnostic platform, and will now be part of Samsung's Open Innovation Center. Can you elaborate on your commitment to keep the platform open? Jacopo Lenzi: S...
The Internet of Things will greatly expand the opportunities for data collection and new business models driven off of that data. In her session at Internet of @ThingsExpo, Esmeralda Swartz, CMO of MetraTech, will discuss how for this to be effective you not only need to have infrastructure and operational models capable of utilizing this new phenomenon, but increasingly service providers will need to convince a skeptical public to participate. Get ready to show them the money! Speaker Bio: ...
Things are being built upon cloud foundations to transform organizations. This CEO Power Panel at 15th Cloud Expo, moderated by Roger Strukhoff, Cloud Expo and @ThingsExpo conference chair, will address the big issues involving these technologies and, more important, the results they will achieve. How important are public, private, and hybrid cloud to the enterprise? How does one define Big Data? And how is the IoT tying all this together?
The only place to be June 9-11 is Cloud Expo & @ThingsExpo 2015 East at the Javits Center in New York City. Join us there as delegates from all over the world come to listen to and engage with speakers & sponsors from the leading Cloud Computing, IoT & Big Data companies. Cloud Expo & @ThingsExpo are the leading events covering the booming market of Cloud Computing, IoT & Big Data for the enterprise. Speakers from all over the world will be hand-picked for their ability to explore the economic...
SYS-CON Events announced today that Cloudian, Inc., the leading provider of hybrid cloud storage solutions, has been named “Bronze Sponsor” of SYS-CON's 15th International Cloud Expo®, which will take place on November 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA. Cloudian is a Foster City, Calif.-based software company specializing in cloud storage. Cloudian HyperStore® is an S3-compatible cloud object storage platform that enables service providers and enterprises to bui...
SAP is delivering break-through innovation combined with fantastic user experience powered by the market-leading in-memory technology, SAP HANA. In his General Session at 15th Cloud Expo, Thorsten Leiduck, VP ISVs & Digital Commerce, SAP, will discuss how SAP and partners provide cloud and hybrid cloud solutions as well as real-time Big Data offerings that help companies of all sizes and industries run better. SAP launched an application challenge to award the most innovative SAP HANA and SAP ...
SYS-CON Events announced today that O'Reilly Media has been named “Media Sponsor” of SYS-CON's 15th International Cloud Expo®, which will take place on November 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA. O'Reilly Media spreads the knowledge of innovators through its books, online services, magazines, and conferences. Since 1978, O'Reilly Media has been a chronicler and catalyst of cutting-edge development, homing in on the technology trends that really matter and spurri...
The Internet of Things (IoT) is going to require a new way of thinking and of developing software for speed, security and innovation. This requires IT leaders to balance business as usual while anticipating for the next market and technology trends. Cloud provides the right IT asset portfolio to help today’s IT leaders manage the old and prepare for the new. Today the cloud conversation is evolving from private and public to hybrid. This session will provide use cases and insights to reinforce t...
Samsung promises to be one of the 800-pound gorillas of the IoT, if its success in recent years with Android devices and other consumer electronics is any guide. Showing its willingness to be a big IoT player, the company recently acquired SmartThings, a recent startup that's developed an open smarthome appliation that currently supports 1,000 devices and 8,000 apps. SmartThings will now work under the auspices of Samsung's Open Innovation Center (OIC). SmartThings Founder and CEO Alex Hawkinson...
What process has your provider undertaken to ensure that the cloud tenant will receive predictable performance and service? What was involved in the planning? Who owns and operates the data center? What technology is being used? How is it being supported? In his session at 14th Cloud Expo, Dave Weisbrot, Cloud Business Manager for QTS, will provide the attendees a look into what it takes to stand up and stand behind a highly available certified cloud IaaS.
SYS-CON Events announced today that Gigaom Research has been named "Media Sponsor" of SYS-CON's 15th International Cloud Expo®, which will take place on November 4-6, 2014, at the Santa Clara Convention Center in Santa Clara, CA. Ashar Baig, Research Director, Cloud, at Gigaom Research, will also lead a Power Panel on the topic "Choosing the Right Cloud Option." Gigaom Research provides timely, in-depth analysis of emerging technologies for individual and corporate subscribers. Gigaom Research'...
I'll be hosting an SAP HANA Cloud webinar at 11am eastern time, Wednesday, October 29. You can sign up now. Featured speakers will be Allan Adler, Managing Partner, Channel Cloud Consulting, and Thorsten Leiduck, VP ISVs & Digital Commerce, SAP. Attendees will learn about • Cloud economics, hybrid cloud strategy, market size and opportunity • Introduction to SAP HANA Cloud Platform and how to: - Build new next-generation applications - Extend on-premise solutions non-disruptively throu...
Join both SAP and Channel Cloud Consulting for our webcast and uncover how you can extend your reach to capture a piece of the US$17 billion cloud application services market with SAP. Learn about SAPs market-leading SAP HANA Cloud Platform and an exciting opportunity to join SAPs growing ecosystem of Application Development partners. When: October 29, 11:00am EST Speakers: Allan Adler, Managing Partner, Channel Cloud Consulting Thorsten Leiduck, Vice President ISVs & Digital Commerce, SAP
Application Performance Management (APM) has been bred with all the right elements to give us the insights we need to see the health of our applications. Similar to your most trusted watch dog, it alerts us to anomalies when events occur, providing awareness to the environment that only they can observe. As enterprises embrace the DevOps philosophy, and the coalescence of the Development and Operations continues, I foresee the conditions ripening to foster innovative methods of making applicati...
SYS-CON Events announced today that IBM is holding a Bluemix Developer Playground on November 5, 10:30 am to 5:30 pm at 15th Cloud Expo. 15th Cloud Expo, co-located with @ThingsExpo, Big Data Expo, and DevOps Summit is taking place Nov. 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA. The labs, for developers of all levels, will highlight the ease of use of Bluemix, its services and functionality and provide short-term introductory projects that developers can complete betw...
The Industrial Internet revolution is now underway, enabled by connected machines and billions of devices that communicate and collaborate. The massive amounts of Big Data requiring real-time analysis is flooding legacy IT systems and giving way to cloud environments that can handle the unpredictable workloads. Yet many barriers remain until we can fully realize the opportunities and benefits from the convergence of machines and devices with Big Data and the cloud, including interoperability, da...
Software AG helps organizations transform into Digital Enterprises, so they can differentiate from competitors and better engage customers, partners and employees. Using the Software AG Suite, companies can close the gap between business and IT to create digital systems of differentiation that drive front-line agility. We offer four on-ramps to the Digital Enterprise: alignment through collaborative process analysis; transformation through portfolio management; agility through process automation...
How do you know when a technology has become mainstream? A good clue may be when politicians start talking about it on the campaign trail and with mainstream media. David Cameron, the UK prime minister, was the latest, indicating that the world was now on “fast-forward” with the Internet of Things (IoT) ushering in the new industrial revolution. No mention of IoT targeted at the masses would be complete without the clichéd example of the communicating fridge. While it is easy to get caught up in...