New technologies that comes with recent trends like Internet of Things, Cloud and 5G are promoting new platforms and communication solutions. These trends have also started to impact traditional industrial automation systems, since end customers are starting to expect new services, such as, business intelligence and diagnostic information anywhere at any time. Moreover, the cloud providers lease their infrastructure to be used for deployment of applications using virtualization. Therefore, this paper aims at exploring the possibilities of different virtualization platforms offered by various cloud providers and benchmark the technologies with platforms frequently used within industry. The platform performance was evaluated by conducting experiments with an industrial application, focusing on typical industrial aspect such as latency, jitter and availability. In addition to more industrial focused metrics, the findings in comparison with other related experiments indicate that specific application requirements have an effect on performance. Hence, application specific evaluations may be necessary before taking any decision on where an industrial application may be deployed.

Industry 4.0 is the German vision for the future of manufacturing, where smart factories use information and communication technologies to digitise their processes to achieve improved quality, lower costs, and increased efficiency. It is likely to bring a massive change to the way control systems function today. Future distributed control systems are expected to have an increased connectivity to the Internet, in order to capitalize on new offers and research findings related to digitalization, such as cloud, big data, and machine learning. A key technology in the realization of distributed control systems is middleware, which is usually described as a reusable software layer between operating system and distributed applications. Various middleware technologies have been proposed to facilitate communication in industrial control systems and hide the heterogeneity amongst the subsystems, such as OPC UA, DDS, and RTCORBA. These technologies can significantly simplify the system design and integration of devices despite their heterogeneity. However, each of these technologies has its own characteristics that may work better for particular applications. Selection of the best middleware for a specific application is a critical issue for system designers. In this paper, we conduct a survey on available standard middleware technologies, including OPC UA, DDS, and RT-CORBA, and show new trends for different industrial domains.

In the industrial domain customers expect a product longevity of 10-20 years, with high reliability and availability. Since industrial distributed control systems often are safety critical, aspects such as determinism, low latency and jitter are crucial. More and more industrial systems are becoming connected to the Internet, since end customers are requiring e.g. business intelligence and diagnostic information, anywhere at any time. Industrial systems that traditionally have been isolated are now facing entirely new challenges that will require new competences and ways of working. Introducing a new type of network in the industrial domain is a big investment, with high risks, often lacking known best practices. Time to market with sufficient quality is of high importance. A lot of time is spent on isolated activates, such as, simulations, updating tools, collecting requirements, design, coding, debugging, documentation, creating testbeds, validation and reviews. Therefore, there is a need to improve the efficiency when moving between the research and development phases for several reasons, e.g., integrate innovative research findings into industrial systems, shorten time to market, and improve product quality. This thesis focuses on improving efficiency during research and development of communication software. First, network evaluation methods are studied, and key industrial challenges are identified. For example, despite a huge research effort on network simulators and virtualization, there are still challenges that need to be addressed, in order for increased industrial benefits. Secondly, this thesis propose a flexible communication stack design that supports different run-time behaviors, from real-time operating system to bare-metal systems without an operating system, and different types of communication protocols, from real-time to non-real-time. Finally, this thesis propose a set of key features from network simulators, that are implemented and used as a case study in a research project. These contributions lead to simplification and increased automation, hence reducing the amount of manual work during research and development.

Trends like Internet of Things (IoT), 5G and Cloud are pushing for device connectivity to the Internet, which affects industrial embedded systems with e.g., an increase in code base and functionality. Due to different application requirements, there are relative little reuse between embedded systems with different run-time context (from super loop to multi-threaded), and different types of communication (best effort and real time). In order to improve code reuse and changeability, we propose a flexible communication stack design, that can be configured for time sensitive communication with a real-time operating system (RTOS), or configured for best effort communication with either a super loop or an operating system (OS). Experiments demonstrate the flexibility and simplicity of the design with different configurations, e.g., super loop, single threaded, multi-threaded. Measurements show that the variations in performance related to run-time context scales as expected.

Trends like the Cloud, Internet of Things and 5G are pushing for an increase in connectivity, but, introducing a new type of network in an industrial distributed control system is a big investment with high risks. Time to market with sufficient quality is crucial. However, when getting through the Research and Development (R&D) phases, a lot of time is spent on isolated activates, e.g., simulations, collecting requirements, design, coding, debugging, creating testbeds, and performing various tests. Therefore, there is a need to improve efficiency when moving between the R&D phases. For verification and validation of communication software, the most common network evaluation method in industry are real testbeds, mostly since a testbed can be very similar to the deployed system. Testbeds are, however, hard to debug and costly to maintain. Other network evaluation methods like simulators, have some strengths that testbeds are lacking, like repeatability, control over the network, and lower cost. However, code from simulators can seldom be reused, especially in industrial time-sensitive target systems, due to different abstraction levels, run-time behavior and system timing. This paper presents findings from a case study that targets improved efficiency, getting from research theories, to deployed devices in a homogeneous Industrial Wireless Sensor and Actuator Network (IWSAN). We propose a small subset of network simulators features which eases changeability, reuse, and debugging of communication software. The selected simulator features are evaluated with a proof of concept implementation that is customized to a research platform. The findings indicate improved efficiency when moving back and forth between activities in different R&D phases.

A trend in the industrial domain is that the networks are growing and becoming more complex, this is further accelerated by the digitalization trend. In order to address this, there is a need to improve the efficiency when moving between the R&D phases. For example, integrate innovative research findings into industrial systems, shorten time to market, improve product quality and reduce the number of issues. Despite a huge research effort on network simulators and emulators there are still some issues that needs to be addressed. This paper presents challenges that needs to be resolved, in order for the industry to adopt and benefit from using network simulators and emulators. The major challenges streamlining the workflow in and between the different R&D phases while preserving the real-time aspects in the entire industrial distributed control system.

The run-time context in industrial embedded systems varies from bare-metal microcontrollers, to multicore-processors running real-time operating systems. Due to the longevity of industrial systems, reusability and evolvability are often considered crucial quality attributes. This paper presents a new flexible task design that enables tasks to be agnostic to run-time context. Evaluations of the design were made by conducting experiments using a proof of concept implementation of the proposed design. The experiments were based on typical industrial constructs, such as periodic tasks, and event signaling from interrupts. Findings from the experiments show that tasks can be more agnostic to run-time context and still deliver functionality normally used within industry. The results indicate that it is feasible to improve reusability and evolvability between different run-time contexts, and in addition, support hybrid configurations that can reduce resource usage, since e.g. a thread can be easily shared among several tasks.

Most evaluations of industrial real-time software are conducted on real embedded systems. The use of simulators that provides easily reproducible evaluations is often limited, due to different levels of abstraction, e.g., programming languages and run-time contexts. This paper extends previous work on a flexible task design, enabling tasks to be agnostic to run-time context, with evaluations conducted on bare-metal and real-time operating systems. Based on the same design and experiments we extend the proof-of-concept implementation in a discrete event simulation context, executing on a Windows based simulation host. Our experiments show that the flexible task design can be driven in a simulation run-time context, and still support typical industrial constructs. The result indicates that improved code reuse between discrete event simulators and industrial embedded systems is feasible.

In order to facilitate deterministic behavior; industrial real-time communication stacks need another design than non-real-time communication stacks typically found in e.g., Internet of Things and Cloud solutions. We propose a flexible stack design that enable code reuse between testbeds and simulators, as well as how stack layers are driven and prioritized. The design can be generalized and used for non-real-time bare-metal solutions like battery powered Internet of Things. Our approach aims at extending the use of simulation during development of industrial systems in order to find logical errors and wrong assumptions earlier in the development. Conducted and evaluated experiments show that the proposed solutions are able to extend the use of simulation during development of real-time communication software. This is achieved by reusing the same code on an industrial testbed and in a discrete event simulator. In addition, the experiments show that the stack design is generalizable and enable reuse with other non-real-time embedded systems. The contribution consists of a set of building blocks for real-time systems that enable control over the system timing when executing on a simulation host while reusing the source code from an industrial testbed. Overall, this will improve the engineering situation, with respect to code reuse, flexibility and debugging. © 2023 The Author(s)

Industrial real-time software is commonly evaluated on real embedded systems, while simulators are less used, since the abstraction level and purpose vary with for example, different programming languages and run-time contexts. This paper extends, applies, and evaluates previous work on a flexible task design that improve code reuse between discrete event simulators and embedded real-time systems. The paper focuses on two parts: (i) The performance cost of the proposed design in practice, by comparing the flexible task design with a traditional threaded approach. (ii) The potential of an alternative way to support legacy code in combination with discrete event simulation. The experiments indicate an almost negligible performance cost with respect to the real-time behavior, i.e., latency and jitter, while enabling improved code reuse between discrete event simulation and industrial embedded real-time systems.

Embedded systems are today often self-sufficient with limited and predefined communication. However, this traditional view of embedded systems is changing through advancements in technologies such as, communication, cloud technologies, and advanced analytics including machine learning. These advancements have increased the benefits of building Systems of Systems (SoS) that can provide a functionality with unique capabilities that none of the included subsystems can accomplish separately. By this gain of functionality the embedded system is evolving towards a “dual use” purpose¹¹In this paper we define dual usage as a control system having two purposes. In other contexts such as politics, diplomacy and export control, the term “dual-use” refers to technology that can be used for both peaceful and military aims, e.g., nuclear power technology., The use is dual in the sense that the system still needs to handle its original task, e.g., control and protect of an asset, and it must provide information for creating the SoS. Larger installations, e.g., industry plants, power systems and generation, have in most cases a long expected life-cycle, some up to 30–40 years without significant updates, compared to analytical functions that evolve and change much faster, i.e., requiring new types of data sets from the subsystems, not know at its first deployment. This difference in development cycles calls for new solutions supporting updates related to new requirements inherent in analytical functions. In this paper, within the context of “dual usage” of systems and subsystems, we analyze the impact on an embedded system, new or legacy, when it is required to provide analytic data with high quality. We compare a reference system, implementing all functions in one CPU core, to three other alternative solutions: a) a multi-core system where we are using a separate core for analytics, b) using a separate analytics CPU and c) analytics functionality located in a separate subsystem. Our conclusion is that the choice of analytics information collection method should to be based on intended usage, along with resulting complexity and cost of updates compared to hardware cost.

With the evolution of 5G it is envisioned that industrial applications with different requirements on latency and availability can be offloaded to a distributed cloud infrastructure. For example, some applications with stringent timing requirements can be hosted at the edge of the mobile network, closer to the control hardware, whereas some applications with relaxed timing requirements can be hosted in a cloud located geographically further away.This paper presents a feasibility study of hosting control applications based on OPC UA communication in a distributed cloud with LTE connectivity to the control hardware. The study includes measurements of communication round-trip time and availability of the network comparing cases where the application is hosted at a local or regional cloud. The results indicate that it is feasible to deploy industrial applications in a distributed cloud with timing requirements in the order of 100ms.

Internet of Things and cloud services are one of main enablers in fourth industrial revolution. Real-time industrial systems have high availability requirements of 99.9% to 99.999% whereas architectures built on regional cloud services and IoT do not provide similar guarantees or Service Level Agreement. These differences of QoS and SLA availability between Operational Technology and Information Technology has become a main challenge in adoption of Industrial Internet of Things (IIoT) for real-time applications.This work presents an approach to find end-to-end QoS and availability for an IIoT architecture. Device-to-cloud, cloud-to-cloud and inside-cloud experiments have been performed over eight weeks where each experiment have more then four million QoS measurements. Our availability analysis shows that a remote IoT connected to a less busy cloud region gives higher availability as compared to an IoT device inside a busy cloud region. IIoT and regional cloud services provide good QoS with 99% to 99.9% availability for 1sec soft real-time requirements. In 100ms applications, more efforts are required to achieve higher then 95% availability and design industrial SLA. IIoT applications with 10sec latency like machine learning models can get 99.9% availability with cloud. Availability loss due to communication is almost 1% for 100ms applications. These results also provide requirements and future work of industrial edge computing for IIoT on real-time cloud.