As oil and gas upstream and midstream production executives navigate the challenges of integrating their backend legacy IT and operational systems, like SCADA, with edge devices via the Industrial Internet of Things (IIoT) cloud infrastructure, many factors must be considered. These include evaluating approaches to acquire and manage real-time metrics on the performance of thousands of edge devices, ensuring best practices of IIoT security that will considerably exceed that provided by SCADA systems, while minimizing stress on their network. Also, employing solutions to lower wireless connectivity costs to edge devices and eliminating interruption of existing SCADA functionality while integrating with IIoT are considered.

The IIoT is creating new methodologies to collect, process and push data from the network edge to backend servers using cloud-based systems. For oil and gas producers, making the right decisions on integrating their edge-to-HMI connectivity can have considerable benefits, such as: 

  • Real-time alarming without the need for high rate polling;
  • Better use of bandwidth;
  • Multi-application solutions;
  • Edge analytics; 
  • Higher process efficiency;
  • Lower cost of operation.

Straddling IT and OT systems 

These new IIoT methodologies are challenging automation architectural paradigms in oil and gas operations, and the organizational structures that maintain them. The IT communications groups have historically focused on providing private wireless networks for data services over serial and Ethernet connections. While the SCADA groups use their networks to gather edge data for use in the SCADA HMI and backend systems. The communications groups are usually unaware of the details of the protocols and applications running on the operational technology (OT) networks, making them resistant to taking on new roles associated with application specifics. Conversely, the SCADA groups fear losing control of the backend regarding what data is collected, and when.

Legacy system (Source: MACHFU)
Legacy system (Source: MACHFU)

These new methodologies use IIoT gateways at the edge to gather and process data, moving functionality that traditionally is handled by backend systems to the edge. This change in architecture has a significant impact on the OT organizations–as the new architecture straddles the traditional siloed responsibilities of the communications groups and the SCADA groups. 

These IIoT technologies continue to advance into the realm of oil and gas production. The new IIoT architectures leverage Internet technologies enabling scale, robustness and security. SCADA HMI platforms are building IoT data connectors on open standards, simplifying connectivity, and moving things to plug-and-play. Edge computing and edge analytics are enriching data and making it more actionable by users across the enterprise. And, standards-based ecosystems are allowing vendors to compete and collaborate, speeding innovation.

MQTT Protocol—Connecting Edge Devices 

As an example, MQTT is a cloud connector deployed as a solution for connecting edge devices to backend servers and cloud infrastructure. MQTT is a standards-based publish-subscribe messaging protocol that works on top of TCP/IP. It is designed for connections with remote locations with limited network bandwidth. The systems consist of clients communicating with a server, called a broker. The broker organizes data in hierarchical structures called topics. A client may either publish or subscribe to topics to share or gather data. The client who publishes data to a topic does not need to know the number of subscribers or their locations. Subscribers, in turn, do not require configuration with the publishers. 

IoT system (Source: MACHFU)
IoT system (Source: MACHFU)

Most leading OPC-UA HMI vendors have implemented MQTT to an OPC bridge mechanism. This bridging minimizes the impact on the OT groups and eases the effort required to move to modern IIoT connectivity with existing HMI platforms. The use of MQTT enables edge gateways to process and apply business logic to determine what to push to the broker and what to store locally. The logic may be simple, such as an update on a state change or percentage change, or it may run an advanced set of analytics making control decisions locally and publishing the new state to a topic for others in the system to subscribe and act on.

IIoT gateways have device-specific drivers that gather data by poll, the legacy assets use their native protocols and create hierarchical data models, or topics, that can be published using MQTT. Using this approach reduces the required bandwidth and dramatically reduces latency in reporting alarms and changes to the monitored parameters.

IIoT application gateways

The emergence of MQTT protocol, and IIoT platforms such as the OPC Unified Architecture (OPC UA)—which has been widely adopted by oil and gas producers—has opened the door to a new paradigm of industrial IIoT application gateway technology. These new gateways provide a rich set of edge functionality, cost-effectively providing wireless and wired connectivity that enable the unification of modern web-based applications and traditional SCADA infrastructure. 

An excellent example of these new IIOT gateways is the MACHGateway, developed by MACHFU. Its methodologies enable data collection from edge assets and push those data to the cloud, without interrupting existing SCADA functionality or needing to replace existing hardware, and with the simplicity of plug-and-play.

Noninterruption of SCADA

Traditional SCADA uses centralized back-office systems to gather information from remote devices. These systems use polled architectures on serial connections developed 30 years ago, before the advent of the Internet and high-performance processing. New IIoT architectures are removing the limitations of these older polled systems and bridging existing devices to leverage IIoT systems that move intelligence to the network edge. Supporting this are the new IIoT gateways. 

The MACHGateway collects data locally by polling the device and maintaining device connectivity with existing SCADA systems. Such gateways regularly poll devices at the edge, assessing the system state every second, compared to a traditional 15 minute, or longer, SCADA polling interval. The data gathered are used to create a model that can be easily understood by other applications and translated to other protocols, allowing data sharing in real-time with the ecosystem of emerging IIoT applications.


As opposed to traditional approaches for industrial edge connectivity, application gateways, including MACHGateway, leverage the new high-performance hardware platforms using a hardware-independent Android/Linux-based software platform. Application gateways simultaneously leverage chip/module-based connectivity solutions that are lowering wireless connectivity costs.

These gateways provide many features that simplify the development of edge applications by reducing the time to create and integrate them, reducing development time from months or years to weeks or even days. This enables innovative edge applications to be created and rapidly integrated into existing infrastructure and IIoT applications. New applications and business models can be continuously created over the life of the system.


Application gateways have defense-in-depth security built across the layers of the communications stack, all the way up to the applications layer, including role-based access control. Consequently, they implement the best practices of IT security, considerably exceeding that provided by SCADA systems.

The MACHGateway platform has a secure sandbox for different applications, each with its own cryptotechnology for data and sharing of data between applications. Importantly, however, the security built into this gateway does not put a heavy burden on the system.

Data collection

In traditional applications, specific data from a single operation is monitored and collected by SCADA, which then issues commands to control that single operation. With IIoT platforms utilizing new application gateways, that same well data can be used in multiple applications, including overall equipment effectiveness, loss, waste, production efficiency, and even identifying if differences of efficiency exist between wells on the pad. 

Within this framework, data can be collected to a Big Data cloud database, enabling a higher volume of data can be gathered and made available in a common format, to allow for broader usage and analytics. 

IIoT application: monitoring of rod pump controls

An example of using MQTT protocol and MACHgateway methodology for oilfield automation is monitoring the state of rod pump controllers (RPC). The RPC controls the operation of an oil well based on data from a downhole card, surface card, and motor torque. The rod pump controller calculates a downhole card on every stroke of the pumping unit, utilizing complex mathematical computations. There are often additional I/O algorithms that accurately control and analyze the rod pumping system. 

Data flow system (Source: MACHFU) 
Data flow system (Source: MACHFU) 

There are thousands of RPCs in the field, each capturing data from every stroke of the well occurring on 10-second intervals. Given that there is limited memory to store the data and insufficient bandwidth to collect it via a legacy polled system, 99.99% of the data captured is discarded. The RPC is only able to store some basic trends and worst-case conditions. Due to these limitations, SCADA systems typically collect one set of surface-card data and downhole-card data per day, and a basic status several times per hour.

Conversely, a local gateway can gather and process the RPC data, monitoring of every stroke, and provide real-time notification of issues. Real-time data can be time-stamped and stored, enabling backend systems to collect high-resolution data around flagged issues. 

Analytics can also be performed on-site to flag issues and take corrective action. If high-speed communication is used, like LTE or Broadband PMP, the gateway simplifies pushing bulk data to the cloud. 

High-resolution time-stamped data stored at the edge can be correlated and conditionally sent to backend systems, minimizing the data that needs to be sent. All of the data can be pushed as MQTT topic to a broker, which simplifies integration with other tools and dashboards, enabling productivity improvements and real-time alarming throughout the enterprise.

Simplifying Edge-to-HMI Connectivity

As oil and gas executives evaluate possible IIoT protocol and application gateway solutions to achieve an optimum edge-to-HMI connectivity, one factor may be considered as playing a pivotal role—that being simplicity of system design.

Simplicity in the development of open, standardized web-based solutions that enable sharing of data across the enterprise, without abandoning existing hardware and middleware that is already in place. Simplicity in the development of edge applications where they can be created and rapidly integrated, plug-and-play, to existing infrastructure and IIoT applications. And, simplicity of network security, which does not tax the system.

For upstream and midstream oil and gas producers, these operational simplicities will open the door to improved asset utilization, higher process efficiency and productivity, and lower cost of operation.