Creating an effective and scalable IoT infrastructure by introducing Edge Workload Consolidation (eWLC)

Executive Summary

After the first wave of Industrial IoT initiatives, companies are forced to manage a huge crowd of new initiatives, plenty of new data, new hardware and software providers, challenging projects with news technologies, facing among others, the following challenges:

• A set of equipment and devices extremely heterogeneous in terms of capabilities, functionalities, robustness, and degree of security.
• The diversity of software and OS combinations are everchanging and this only increases with time.
• The need to combine an increasingly broad set of management tools.
• Disorganized analytics that does not establish clear criteria on what data is relevant for which process (having all the data is simply the least of the worst solutions). Also, this data analysis is frequently limited by a monetization model focused on the provision of cloud services by a third party that knows the asset or business, not on the knowledge gathered by the final customer.
• The dissemination of valuable operational information in multiple dedicated clouds whose access must be secured and protected using different mechanisms in each provider.

Intel and Minsait-IT are working together to establish the Edge Unified Service Framework facing those problems. This IT/OT high-level framework is composed of the ten most essential disciplines (or aspects) any customer should resolve to deploy a scalable and manageable IoT Solution that provides real business  value. Minsait-IT is a pioneer in the industry implementing and solidifying this vision, allowing customers to speed up the digital transformation journey.

This paper is focused on understanding the roots of the previously mentioned challenges, how Edge Computing, in general, can help and more specifically, how an Edge Workload Consolidation platform can create an effective and scalable IoT/ Edge Hybrid infrastructure for delivering value to the end customer.

Authors

• Mariano Ortega de Mues, Edge CTO Sr.  Practice Manager, Minsait by Indra
• Daniel Seseña, Head of Industry 4.0, Minsait by Indra
• Cesar Martinez Spessot, IoT  Engineering Director, Sr. IoT  Solutions Architect, Intel Corporation
• Marcos Carranza, Sr. IoT Solutions Architect, Intel Corporation
• Jorge Lang, Account Executive, Intel Corporation

Cloud first and the IoT initial solutions

Industrial Internet of Things (IIoT) is coming to transform the way Operational Technologies (OT) work, by improving scalability, open connectivity, and reducing costs (ultimately contributing to  an increase in the efficiency of our systems and a different way of thinking about processes). It is also a matter of fact that the initial Industrial IoT (Figure 1) architectural approach has been the cloud-centric or cloud-first schema. However, today the traditional IIoT paradigm, based on cloud-centric computing is suffering four major problems:

1. is beginning to generate a complexity of deployment that in some extent recalls the scenario of silos systems of traditional automation technology,
2. is not enough to solve increasingly demanding use-cases on real-time/low latency decisions,
3. does not cover the security effort on raw data
4. and finally, it releases in a high-cost component environment.

Most of the current customers who opted for rapid adoption of the first generation IIoT solutions are facing today those problems after starting a digital transformation program.
The truth is that the decisions about what technology to use or with which suite of products to deploy in that program has led, in the majority of cases, to the adoption of a mix of solutions that we can place between a continuum of the two
traditional options:

(i) opt for already packaged solutions

     or

(ii) develop your own in-house.

- Vertical IoT Solutions: The first IoT solutions (vertical ones, not generalist solutions providing mainly technological components) are end-2-end systems that cover everything from field equipment to services delivered in the cloud. With these systems, the client rarely integrates their IT systems directly with the brownfield and focuses on cloud-2-cloud communication, delegating the deployment of analytical capacity to the cloud infrastructure.

The monetization of the data in this scheme (initial digital platforms) always goes through the mediation of the third party that finally transforms the machine itself, into the data set already treated that it generates (digital twin). With this approach, the panorama ends up becoming a set of value-added services on assets that should be consumed continuously without the customer being able to openly develop his analytical capacity and improve their knowledge of the operation.

The added value of these solutions is still far from being inserted in the rapid day to day operation, although it begins to manifest in important insights for aspects such as maintenance, management of the workforce or asset monitoring that previously required the human in-situ supervision.

- Horizontal IoT Platforms: At the other extreme, customers who, instead of opting for very specialized vertical solutions, preferred generalist platforms that offer the components  and the stack necessary to cover the ingestion, process
and analysis of the information; building the solutions by themselves. In this case,

1. A rapid evolution of data management technology that has abandoned the approach of "Big Data for everything" around large Data Lakes, expensive to maintain and evolve by the use of ML and AI focus on small concrete problems,
2. or lack of clear standards for the implementation of each of the components despite the claim of open source have led most of the customers to an increasingly widespread use of proprietary PaaS services of each generalist platform, where once again the actual monetization of  the  data requires the mediation of a third party, in this case, cloud providers themselves.

Figure 1. IoT Cloud oriented architecture

Both situations are not only produced but also encouraged by the cloud-centric model where the underlying technological control point is the computation in cloud as such. Both situations are mutually reinforced with each other since most of the vertical solutions (except some born from the beginning in specific niches) employ the main generalist cloud providers in partnership comprehensive ecosystems. The mix of both situations produces for IT/OT teams, a complicated management and leadership environment, and  a slowdown in programs for implementing new technologies and getting large deployments that facilitate the production of value.

Edge Workload Consolidation concept

The evolution of the Edge Computing is coming to resolve these issues, including real-time analytics and a high level of decentralization. Edge Computing could be defined as a “distributed topology where information processing is placed closer to the things or people that produce and/or consume information for fast service delivery”. Edge Computing extends the "divide and conquer" mantra applied to the algorithmic resolution of complex problems, with its application to the responsibility for process and data (division of concern).

Edge Computing, not only divides the algorithms into simpler elements, is a philosophy for stratifying responsibility in the calculations to be performed in each layer of the processes that also requires establishing a set of different levels of
data abstraction that range from raw data as is produced by machines up to the business indicator in the business results dashboard. Thanks to Edge Computing, cheap direct access to raw data is possible and the application of analytics on this information is in the hands of the end customer who exploits the asset in his facilities.

The potential impact of edge computing is huge, but to make the most of it, it should be understood not only as a technical matter but as a key lever to accelerate new software solutions for the industrial operation, providing lower latency and advanced distributed data analytics. In this way Edge Workload Consolidation responds to the need to manage and obtain value from the huge among of data machine generated and to support new operational use cases with the need for advanced analytics and low latency. For this aim, Edge Workload Consolidation offers a unified way to distribute the computing, storage and service capacity between the different layers of the infrastructure. In this sense, Edge Workload Consolidation complements the traditional IoT using three main characteristics to face the new challenges:

• Distributed Processing: High-speed and flexible computational intelligence to distribute workloads in a fast way allowing and efficient management of different software and device tools.
• Distributed Storage: Scalable storage infrastructure to manage different sources of data from Legacy, Business tools and operational and non-operational data.
• Fast Delivery: Fast data distribution for both, machines and operators, which allows the deployment of advanced use cases adding value to raw data.

Figure 2. Cloud oriented vs Edge oriented architecture

Figure 3. Edge Computing and Workload Consolidation framework

Open Edge Workload Consolidation Architecture

Intel and Minsait have defined an architecture platform that can help with the orchestration of workloads by including Virtual Machine and Container technologies. Both Virtual Machines and Containers have strengths and weaknesses. Containers are the preferred solution when applications and processes  need to be up and running as fast as possible in dynamic execution scenarios. 

With this, the infrastructure costs can be reduced, make systems functionally safe and secure, and management is simplified. Workload consolidation allows having a high efficiency on workload deployment considering dynamic scenarios requiring workload transfer from a tier in the architecture to another tier, or between nodes in the same tier. 

The support for multiple types of “workload packaging” (VMs, Container, Native Runtimes) provides flexibility to customers. The ability to run legacy solutions on top of the platform translates into an optimal use of the current infrastructures without forcing users to chose only one type of workload packaging.

On the multi-tier system schema shown in Figure 4, as part of the edge-computing infrastructure, there are four main components:

• Edge Devices: These are the devices commonly referred to IoT Gateways or Edge Compute Devices, having enough processing power to host several IoT solutions (Intel® Core i3, i5 and i7 processors). They are deployed close to sensors, receiving data directly from them, and interacting with actuators.
• Edge Server: On-premise server clusters (usually Intel® Xeon processors) required for consolidating data coming from many Edge Nodes. These local servers provide independence from cloud providers, allowing higher availability, low latency, and reduction of data transmission.
• Edge Core: Server clusters (Intel® Xeon processors) used to deploy the fundamental management tools to orchestrate the Edge Devices and Edge Server devices.

This tier provides the necessary agility and speed to face real-time use cases and manage a wide amount of data efficiently.

• Edge Cloud: Group of elements reserved to deploy cloud tools allowing cloud operation in-plant like ingest, digital twin or cluster oversizing.  This architecture makes possible the deployment of solutions without compromisingthe data security within the facility.

Figure 4. Edge Compute and Workload Consolidation architecture

One of the main objectives directing the specification of the proposed technical architecture is the orderly and consistent deployment of machine intelligence (or AI) in the different use cases that can be covered. This deployment highlights the collaboration between more traditional computing technologies mentioned above (including GPUs (IRIS®) with new elements that are generalized from other fields of application such as FPGAs (Intel® Arria® 10 FPGA, Stratix*) or
that are of recent creation like the VPUs such asIntel® Movidius™ Myriad™ 2 Vision Processing Unit (VPU) or the neural architectures such as Intel® Nervana™ Neural Network Processor (Intel® Nervana™ NNP)”.

Onesait Platform Things Implementation

The following images (Figures 5 and 6) represent the two different approaches to implement both Edge Workload Consolidation architectures. The first one, represented by Figure 5, is defined as “Edge#1 Isolated Edge Nodes architecture”. This architecture allows an agile signal integration and management between IT- OT layers. The second architecture, represented by Figure 6, is identified as “Edge#2 Tiered k8s clusters architecture” This is designed to deploy advanced use cases requiring edge analytics and optimizing edge-cloud computing workload.Both implementations are available into two different modalities either as Open Source software or as the complete eWLC solution with all the analytical power capabilities of Intel and Minsait. Furthermore, eWLC provides support for 3rd party solutions deployed at the platform.
Figure 5. Isolated Edge Nodes

Figure 6. Tiered k8s clusters

Making eWLC a Business Goal

Today, enterprise corporations are caught in a compute environment which is in constant flux. To stay ahead of these changes, companies need to react in an agile manner. Intel and Minsait propose a five state iterative and agile framework (Figure 7):

A. Business Context: the main objective of this state is to identify the business context to understand the technological capabilities and necessities of the company to be competitive in the sector.

B. Use Cases: once the necessities are identified, it is possible to define the relevant use cases that will empower the business capabilities of the company.

C. Unify service Framework: during this state, the necessities defined for each use case are studied using the Unified Edge Framework (UEF) of Intel and Minsait. Therefore, it is possible to classify and define the evolution of the company on each feather defined on the UEF.

D. White Book Design: during this step, the evolution plan defined in state C is detailed to define how is going to be deployed and configured the eWLC global solution.

E. Roadmap & Program definition: This final state specifies in time the implementation of the rest of use cases prioritized by the business impact that will generate on the company and the challenge considering the necessary technical development for each company.

Companies need to face a new challenging environment  that requires the maximum and highly efficient computing capabilities, so that a Unified Edge Framework has been defined to state a proper approach to face this new situation. This framework is composed of the 10 disciplines (aspects) that a customer should tackle to deploy a scalable and manageable IoT Solution that provides business value. These 10 disciplines are defined as follow:

Figure 7. Digital transformation iterative process

Figure 8. Unified Edge Framework (UEF)

A sector sample: Challenges of Industrial IoT

The architectures and methodologies presented here are globally applicable to many sectors:

1. Digital transformation of energy networks with the integration of Distributed Energy Resources (DERs) and the virtualization of Primary and Secondary Substations.
2. Deployment of advanced analytics in Digital Oil Field.
3. Optimization of pumping and monitoring of Water and Gas Pipelines, etc.

Because of the importance eWLC has for the productive fabric, we will briefly explore the impact on the manufacturing sector. The massive deployment of Industry 4.0 presents the opportunity for market makers for reaching new levels of efficiency (such as  building on the standardization of scalable eWLC systems), productivity and the development of differential competitive capacities.
However, these generational [improvements/changes] also pose  new challenges in terms of the management of the operational data to the extent that customers and the competitive environment require, assuring:

• An efficient management of computing, storage and analysis of big data to extract real value from generate impact in the business at short term.
• Proper management of the balance between centralized decision-making vs Edge.
• The deployment of secure and scalable data architectures
• An efficient, safe and scalable management of a wide variety of sensors, equipment, software solutions, communication protocols, hardware providers, integrators, etc.
• The assurance of regulatory compliance in the management of the company’s data.

The evolution of edge computing is coming to resolve these challenges, including real-time analytics and some level of decentralization. Particularly, an important area related to this is Edge Workload Consolidation, which helps to consolidate new and old legacy systems on an edge computing infrastructure that can serve multiple purposes.

As a new IoT Infrastructure, industrial companies have yet to learn what Edge Computing is and what it can provide. Most industrial players only focus on physical devices or communication endpoints. However, Edge Computing and Workload Consolidation are accelerators to capture the opportunity of massive digitization and Industry 4.0.

Specifically, efficient agile deployment of Workload Consolidation will allow relevant impact both for the industrial sectors as a whole and for key players within companies.

Main impacts for the industrial sectors

Workload Consolidation will allow the development of new advanced use cases that require advanced analytics and very low response time. Proper deployment of Workload Consolidation will provide the leading companies with high tech solutions in some of the following areas:

• Agile and automatized production.
  • AGVs and robots: dynamic optimization, safer human-machine operation.
  • Augmented and Virtual Reality: deployment of enriched immersive environments, remote operation.
  • Digital worker: advanced use cases without connection, safer human-machine operation.
• Digital quality
  • Intelligent monitoring and predictive quality: enriched monitoring, analytical capacity in the edge, analysis and action, learning and sharing.
  • Images recognition: high computational capacity in the edge, high-quality images and videos processing.
• Smart Assets Management
  • Intelligent monitoring and predictive quality maintenance, analytical capacity in the edge, analysis and action, learning and sharing
• Energy efficiency
  • Intelligent monitoring, dynamic optimization of resources and production flows, computational capacity in the edge, energy consume- generation orchestration
• Advanced Planning & Scheduling
  • Ability to execute complex algorithms in near- real-time; machine learning, scheduling scenarios simulation
• Traceability and optimization
  • Dynamic routing optimization, safer human-machine operation, industrial autonomous vehicles operational integration
• Cybersecurity
  • Vulnerability and risk analysis in real-time, scenarios simulation and semi- automatized actions

In conclusion, Workload Consolidation will allow industrial companies to consolidate the technical operation of the first wave of Industry 4.0 solutions and address new advanced use cases.

Main impacts for the key players within the companies

The benefits of Workload Consolidation exceed the scope of the "technical operation" to enable the generation of real value for the business and functional managers, in addition to the benefits given to the technical managers. Some of the benefits for the key players will be the following:

• Chief Information Officers and Chief Technical/Technology Officers (CIO & CTOs)
  • Simplification of the operation of digital systems/solutions
  • Reduction of integration complexity and maintenance
  • Reduction of infrastructure and operational costs
  • Standardization of architectures, devices, equipment, languages 
  • Scalability and agnostic HW-SW approach
  • Increase in cybersecurity
• Chief Digital Officers (CDO)
  • Data architecture simplification: accessibility and availability
  • Efficient management of computing, storage and data service
  • Regulatory compliance
  • Securing cybersecurity
• Chief Operations Officers (COO)
  • Global operation: centralized and distributed monitoring, analysis, decision making and management
  • Best practices learning and benchmarking
  • Rapid deployment of innovative and digital projects
• Plant Directors and Operational Managers
  • Development of advanced use cases requires requiring advanced analytics and very
  • low response time: agile and automatized production, digital quality, smart assets management, energy efficiency.
  • Benefits derived from new computational capabilities (machine learning, deep learning, neuronal networks) deployed on the edge
  • Reduction of latency analysis- response
  • Ability to deploy digital solutions working offline and remotely

Impacts and value generated

As previously mentioned, the benefits of Workload Consolidation and Edge Computing refer to the realm of technology and industrial operations and could be summarized as follows:

• Substantial reduction of cost driven by the volume of data
• Improvement of cybersecurity and regulatory compliance
• Enablement of higher data analytics
• Acceleration of contextual machine learning models
• Provision of lower latency and higher process optimization
• Empowerment of the OT layer with new tools for understanding machine behavior and patterns
• Extract domain knowledge from OT

What´s next?

Edge Computing and Workload Consolidation represent a huge opportunity for technology and industrial companies to move digitalization to a new global scale and accelerate the generation of impact on both their business and operations.

However, there are still multiple technological and operational challenges, such as the generation, and management of a broad ecosystem of partners, the generalization and dissemination of the concept and its transfer to use cases and projects with a quantifiable impact on the company's profit and loss.

Therefore, the following steps should revolve around the implementation of technological-functional initiatives
in controlled operational areas and supported by technologically advanced industrial companies, which allow to demonstrate the value in the short term and generate new opportunities at the global level.

Intel and Minsait share the goal to boost and extend usage of Edge Workload Consolidation architecture presented through this document and offer third party companies with high-value products the require support for the creation of a partnership community that could contribute with innovative solutions on the mentioned sectors. For this purpose, do not hesitate into contact either Intel or Minsait team referenced  at the initial section of this paper.

White Paper