Distributed Analytics in IoT – Why Positioning is Key

analytics-word-cloud

The current global focus on the “Internet of Things (IoT)” have highlighted extreme importance of sensor-based intelligent and ubiquitous systems contributing to improving and introducing increased efficiency into our lives. There is a natural challenge in this, as the load on our networks and cloud infrastructures from a data perspective continues to increase. Velocity, variety and volume are attributes to consider when designed your IoT solution, and then it is necessary to design where and where the execution of analytical algorithms on the data sets should be placed.

Apart from classical data centers, there is a huge potential in looking at the various compute sources across the IoT landscape. We live in a world where compute is at every juncture, from us to our mobile phones, our sensor devices and gateways to our cars. Leveraging this normally idle compute is important in meeting the data analytics requirements in IoT. Future research will attempt to consider these challenges. There are three main classical architecture principles that can be applied to analytics. 1: Centralized 2: Decentralized and 3: Distributed.

The first, centralized is the most known and understood today. Pretty simple concept. Centralized compute across clusters of physical nodes is the landing zone (ingestion) for data coming from multiple locations. Data is thus in one place for analytics. By contrast, a decentralized architecture utilizes multiple big distributed clusters are hierarchically located in a tree like architecture. Consider the analogy where the leaves are close to the sources, can compute the data earlier or distribute the data more efficiently to perform the analysis. This can have some form of grouping applied to it, for example – per geographical location or some form of hierarchy setup to distribute the jobs.

Lastly, in a distributed architecture, which is the most suitable for devices in IoT, the compute is everywhere. Generally speaking, the further from centralized, the size of the compute decreases, right down to the silicon on the devices themselves. Therefore, it should be possible to push analytics tasks closer to the device. In that way, these analytics jobs can act as a sort of data filter and decision maker, to determine whether quick insight can be got from smaller data-sets at the edge or beyond, and whether or not to push the data to the cloud or discard. Naturally with this type of architecture, there are more constraints and requirements for effective network management, security and monitoring of not only the devices, but the traffic itself. It makes more sense to bring the computation power to the data, rather than the data to a centralized processing location. 

There is a direct relationship between the smartness of the devices and the selection and effectiveness of these three outlined architectures. As our silicon gets smarter and more powerful and efficient, this will mean that more and more compute will become available, which should result in the less strain on the cloud. As we distribute the compute, it should mean more resilience in our solutions, as there is no single point of failure.

In summary, the “Intelligent Infrastructures” now form the crux of the IoT paradigm. This means that there will be more choice for IoT practitioners to determine where they place their analytics jobs to ensure they are best utilizing the compute that is available, and ensuring they control the latency for faster response, to meet the real time requirements for the business metamorphosis that is ongoing.

Published by

deniscanty

DENIS CANTY IS EXCITED TO BEGIN IN JULY 2017 WITH MCKESSON, A FORTUNE 5 COMPANY – AS THEIR SENIOR DIRECTOR OF CYBER SOFTWARE ENGINEERING IN CORK. HIS LAST ROLE (TO JUNE 2017) WAS AS THE LEAD TECHNOLOGIST FOR IOT WITH JOHNSON CONTROLS INNOVATION GROUP BASED IN CORK, IRELAND. THAT ROLE MEANT COLLABORATING EXTENSIVELY BETWEEN HIS TECHNICAL AND SALES TEAMS TO DRIVE FURTHER COMMERCIALISATION OPPORTUNITY THROUGH TECHNOLOGY (BOTH OUR OWN AND PARTNERS/STARTUPS) INTO OUR SALES CHANNELS, SPECIFICALLY LOOKING AT THE EMERGING SMART BUILDING MARKET. THE PROJECTS INCLUDE OUR EXISTING TECHNOLOGIES – BUILDING SECURITY, RETAIL, HVAC AND BUILDING ENERGY – AND EMERGING TECHNOLOGIES SUCH AS IOT, AR AND MACHINE LEARNING. A KEY COMPONENT WAS TAKING KEY INPUT FROM NUMEROUS STAKEHOLDERS AND PROCESSES TO DELIVER ROI FOR CUSTOMERS AND PARTNERS. HE THEN LED THE TEAM TO BUILD AND DEPLOY THE SOLUTIONS IN AN LEAN AGILE MANNER. DENIS SPOKE ON THE NATIONAL AND INTERNATIONAL CIRCUIT FOR JOHNSON CONTROLS AT NUMEROUS TECHNOLOGY CONFERENCES. HIS LEADERSHIP STYLE IS LEADERSHIP THROUGH TRUST AND DELIVERY, AND I TAKE RESPONSIBILITY FOR MY TEAM, COMPASSION AND HUMILITY ARE ALSO IMPORTANT AS A LEADER IN MY OPINION. I LIKE TO BUILD A BALANCED CULTURE, WITH THE PEOPLES PERSONALITIES IMPORTANT INPUTS INTO THAT. DENIS HAS A DEGREE IN ELECTRONIC ENGINEERING (2H) FROM CORK INSTITUTE OF TECHNOLOGY, A MASTERS IN MICROELECTRONIC CHIP DESIGN (1H) FROM UNIVERSITY COLLEGE CORK AND A MASTERS IN COMPUTER SCIENCE (1H) FROM DUBLIN CITY UNIVERSITY. PRIOR TO JOHNSON CONTROLS, DENIS HELD A POSITION OF PRINCIPAL DATA ARCHITECT AND DEVELOPMENT MANAGER WITH EMC FROM 2010 TO 2015, SPENDING 2011 IN SILICON VALLEY. HE LED A TEAM FOCUSED AT REDUCING AND CONSUMING NINE TEST AUTOMATION PLATFORMS FROM EXTERNAL MANUFACTURERS TO ONE EMC CLOUD HOSTED PLATFORM. HE ALSO WORKED ON A NUMBER OF WORKFLOW AUTOMATION SOFTWARE REPLACING TEDIOUS MANUAL EXTRACT, SEARCH AND REPORT COMPILATION THAT RESULTED IN EFFICIENCY GAIN (WRITTEN IN PYTHON). I ALSO BUILT PREDICTIVE ANALYTICS APPLICATION IN MANUFACTURING AND DATA SCIENCE MODELS FOR THE CUSTOMER VERTICAL WITH THE CTO OFFICE. DENIS BROUGHT MICROSERVICES BASED DESIGN ALONG WITH DISTRIBUTED STORAGE AND PROCESSING TO THE GROUP, CHANGING THE DEVELOPMENT CULTURE IN THE PROCESS. DENIS WAS ALSO A MEMBER OF EMC’S GLOBAL INNOVATION COUNCIL AND AS AN AMBASSADOR WITH THEIR OFFICE OF THE CTO, LEADING THEIR CUSTOMER INSIGHT SOFTWARE DEVELOPMENT. DENIS WON TWO GLOBAL INNOVATION AWARDS IN HIS TIME WITH EMC, IN THE AREAS OF SUSTAINABILITY AND E-SERVICES, AND HAS A PATENT IN INTELLIGENT POWER MANAGEMENT ON STORAGE ARCHITECTURE. HE ALSO WORKED PREVIOUSLY FOR ALPS AUTOMOTIVE DIVISION FROM 2005-2010, IN A VARIETY OF ROLES, INCLUDING AS THE LEAD COMPUTER VISION ENGINEER, AND THE LEAD TECHNOLOGIST ON EUROPEAN RESEARCH PROJECTS IN THE AREAS OF IN-VEHICLE DISTRACTION MONITORING AND SMART HOME DEVICES. DENIS ALSO SPENT TIME CONSULTING IN THE START-UP WORLD, SUCH AS A HEALTHCARE INFORMATICS CONSULTANT WITH ACE HEALTH, LEADING THE DEVELOPMENT FOR AN APPLICATION WHICH HELPS HEALTHCARE SERVICE PROVIDERS ACHIEVE BETTER PATIENT OUTCOMES AND CUT COSTS THROUGH A REGULATOR-APPROVED PREDICTIVE ANALYTICS PLATFORM IN THE DUTCH AND US MARKETS. HE ALSO HAD HELPED NUMEROUS STARTUPS ON BUILDING THEIR TECHNOLOGY ROADMAP TO ALIGN WITH DEFINED TARGET MARKETS AND CUSTOMER BASES.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s