Welcome!

@DXWorldExpo Authors: Kevin Benedict, Zakia Bouachraoui, Elizabeth White, Maria C. Horton, Liz McMillan

Related Topics: @DXWorldExpo, Java IoT, Linux Containers, Agile Computing, Wearables, @CloudExpo, @ThingsExpo

@DXWorldExpo: Article

Challenges with Big Data for the IoT | @ThingsExpo #BigData #IoT #IIoT #M2M

The design and implementation of smarter infrastructure

The biggest challenge to IOT is thus big data that requires an infrastructure that is capable of processing billions or more data points each second.

If you thought e-commerce revolutionized the retail industry over the past two decades, something much bigger is coming over the next ten years and that is the Internet of Things (IoT). By deploying sensors and network connectivity to everyday physical objects like cars, refrigerators, television and buildings, it is possible to remotely connect, manage and engage with these entities in ways that are not possible today.

Proponents of IoT describe a future where your car could interact with your air conditioner at home so that your house could be appropriately cooled by the time you reach home, or such devices could be automatically turned off each time you leave the house. While such examples are not far from truth, the true test for IoT shall come from policy makers.

The prospects for IoT in administration is so good that in countries like India and the UK, IoT is a matter of state policy today. However, it here that IoT is likely to meet its largest set of detractors. This is because IoT in state administration comes with huge stakes. Take the example of smart traffic management where Internet-enabled devices can monitor traffic movements across the city and can smartly administer lanes to ease congestion. Insufficient infrastructure or a badly executed IoT system could crumble under heavy load and could paralyze traffic movements across the city during peak times.

The biggest challenge to IoT is Big Data. The Internet of Things in administration could require an infrastructure that is capable of processing billions or more data points each second. Essentially, when IoT becomes main stream, we will be dealing with information that is much bigger than what the Big Data tools of today process on average. For mission-critical services like traffic management, this necessitates sophisticated data infrastructure that can handle the humongous volume of data thrown in by these large scale IoT systems.

Some experts argue that IoT and Big Data are essentially two sides of the same coin - like Big Data, the solution to a successful IoT implementation lies in storing, processing and extracting value from the data received. Protocols like Mosquitto and platforms like Hadoop and Hive do a good job in queuing and storing data while custom analytics tools will help process and derive meaningful information from this data. In both cases, however, the underlying infrastructure can make or break the system. A recent study published by CA Technologies found that among enterprise Big Data players, nearly 32% believe that their existing infrastructure is a major obstacle and not adequate to implement Big Data projects. It is not surprising then that close to half of the respondents believed their organizations needed major investments in infrastructure to implement their projects.

According to technology writer Arthur Cole, the solution to the infrastructural challenges facing IoT and the associated big data is not bigger infrastructure. Rather, it is in the design and implementation of smarter infrastructure that can process data in transit as well as optimize resources through prescriptive platform analytics and active copy analytics. Such an infrastructure could use machine learning algorithms to dynamically allocate the resources required to back up job processes, monitor services and produce output for the user.

This argument makes sense since focusing merely on larger infrastructure could create a bull-whip effect that could lead to exponential rise in infrastructural investment that could make IoT unviable. By focusing on smarter infrastructure that can possibly make use of current investments to process much larger volumes of data, we could be looking at larger adoption that could in effect lead to quicker mainstreaming of IoT. And that's a good thing for everyone in the ecosystem.

More Stories By Harry Trott

Harry Trott is an IT consultant from Perth, WA. He is currently working on a long term project in Bangalore, India. Harry has over 7 years of work experience on cloud and networking based projects. He is also working on a SaaS based startup which is currently in stealth mode.

DXWorldEXPO Digital Transformation Stories
@CloudEXPO and @ExpoDX, two of the most influential technology events in the world, have hosted hundreds of sponsors and exhibitors since our launch 10 years ago. @CloudEXPO and @ExpoDX New York and Silicon Valley provide a full year of face-to-face marketing opportunities for your company. Each sponsorship and exhibit package comes with pre and post-show marketing programs. By sponsoring and exhibiting in New York and Silicon Valley, you reach a full complement of decision makers and buyers in ...
There are many examples of disruption in consumer space – Uber disrupting the cab industry, Airbnb disrupting the hospitality industry and so on; but have you wondered who is disrupting support and operations? AISERA helps make businesses and customers successful by offering consumer-like user experience for support and operations. We have built the world’s first AI-driven IT / HR / Cloud / Customer Support and Operations solution.
LogRocket helps product teams develop better experiences for users by recording videos of user sessions with logs and network data. It identifies UX problems and reveals the root cause of every bug. LogRocket presents impactful errors on a website, and how to reproduce it. With LogRocket, users can replay problems.
Data Theorem is a leading provider of modern application security. Its core mission is to analyze and secure any modern application anytime, anywhere. The Data Theorem Analyzer Engine continuously scans APIs and mobile applications in search of security flaws and data privacy gaps. Data Theorem products help organizations build safer applications that maximize data security and brand protection. The company has detected more than 300 million application eavesdropping incidents and currently secu...
Rafay enables developers to automate the distribution, operations, cross-region scaling and lifecycle management of containerized microservices across public and private clouds, and service provider networks. Rafay's platform is built around foundational elements that together deliver an optimal abstraction layer across disparate infrastructure, making it easy for developers to scale and operate applications across any number of locations or regions. Consumed as a service, Rafay's platform elimi...
In his general session at 21st Cloud Expo, Greg Dumas, Calligo’s Vice President and G.M. of US operations, discussed the new Global Data Protection Regulation and how Calligo can help business stay compliant in digitally globalized world. Greg Dumas is Calligo's Vice President and G.M. of US operations. Calligo is an established service provider that provides an innovative platform for trusted cloud solutions. Calligo’s customers are typically most concerned about GDPR compliance, application p...
Growth hacking is common for startups to make unheard-of progress in building their business. Career Hacks can help Geek Girls and those who support them (yes, that's you too, Dad!) to excel in this typically male-dominated world. Get ready to learn the facts: Is there a bias against women in the tech / developer communities? Why are women 50% of the workforce, but hold only 24% of the STEM or IT positions? Some beginnings of what to do about it! In her Day 2 Keynote at 17th Cloud Expo, Sandy Ca...
New competitors, disruptive technologies, and growing expectations are pushing every business to both adopt and deliver new digital services. This ‘Digital Transformation’ demands rapid delivery and continuous iteration of new competitive services via multiple channels, which in turn demands new service delivery techniques – including DevOps. In this power panel at @DevOpsSummit 20th Cloud Expo, moderated by DevOps Conference Co-Chair Andi Mann, panelists examined how DevOps helps to meet the de...
According to Forrester Research, every business will become either a digital predator or digital prey by 2020. To avoid demise, organizations must rapidly create new sources of value in their end-to-end customer experiences. True digital predators also must break down information and process silos and extend digital transformation initiatives to empower employees with the digital resources needed to win, serve, and retain customers.
In his keynote at 18th Cloud Expo, Andrew Keys, Co-Founder of ConsenSys Enterprise, will provide an overview of the evolution of the Internet and the Database and the future of their combination – the Blockchain. Andrew Keys is Co-Founder of ConsenSys Enterprise. He comes to ConsenSys Enterprise with capital markets, technology and entrepreneurial experience. Previously, he worked for UBS investment bank in equities analysis. Later, he was responsible for the creation and distribution of life ...