As oil and gas upstream and midstream production executives navigate the challenges of integrating their backend legacy IT and operational systems, like SCADA, with edge devices via industrial IoT (IIoT) cloud infrastructure, many factors must be considered. 

Factors include: a) evaluating approaches to acquire and manage real-time metrics on the performance of thousands of edge devices like sensors, alarms, controllers, equipment, valves and actuators; b) ensuring best practices of IIoT security that will considerably exceed that provided by SCADA systems, while minimizing stress on their system; c) employing solutions to lower wireless connectivity costs to edge devices; d) eliminating interruption of existing SCADA functionality while integrating with IIoT; and e) minimizing the need to acquire new software, instead utilizing existing backend systems.

The IIoT is creating new methodologies to collect, process and push data from the network edge to backend servers using cloud-based systems. For oil and gas producers, making the right decisions on integrating their edge-to-HMI connectivity can have considerable benefits, such as:

  • Real-time alarming without the need for high rate polling;
  • Better use of bandwidth;
  • Multi-application solutions;
  • Edge analytics; 
  • Higher process efficiency; and
  • Lower cost of operation.

New IIoT Methodologies Straddle IT and OT Systems

These new IIoT methodologies are challenging automation architectural paradigms in oil and gas operations, and the organizational structures that maintain them. The IT communications groups have historically focused on providing private wireless networks for data services over serial and Ethernet connections. While the SCADA groups use their networks to gather edge data for use in the SCADA HMI and backend systems. The communications groups are usually unaware of the details of the protocols and applications running on the operational technology (OT) networks, making them resistant to taking on new roles associated with application specifics. Conversely, the SCADA groups fear losing control of the backend regarding what data is collected, and when.

Legacy System (Source: Machfu Inc.)
Legacy System (Source: Machfu Inc.)

These new methodologies use IIoT gateways at the edge to gather and process data, moving functionality that traditionally is handled by backend systems to the edge. This change in architecture has a significant impact on the OT organizations—as the new architecture straddles the traditional siloed responsibilities of the communications groups and the SCADA groups.

These IIoT technologies continue to advance into the realm of oil and gas production. The new IIoT architectures leverage Internet technologies enabling scale, robustness and security. SCADA HMI platforms are building IoT data connectors on open standards, simplifying connectivity and moving things to plug-and-play. Edge computing and edge analytics are enriching data and making it more actionable by users across the enterprise. And, standards-based ecosystems are allowing vendors to compete and collaborate, speeding innovation.

MQTT Protocol—Connecting Edge Devices

As an example, MQTT is a cloud connector being deployed as a solution for connecting edge devices to backend servers and cloud infrastructure. MQTT is a standards-based publish-subscribe messaging protocol that works on top of TCP/IP. It is designed for connections with remote locations with limited network bandwidth. The systems consist of clients communicating with a server, called a broker. The broker organizes data in hierarchical structures called topics. A client may either publish or subscribe to topics to share or gather data. The client who publishes data to a topic does not need to have any knowledge of the number or locations of subscribers, and subscribers in turn do not have to be configured with any information about the publishers. 

IoT system (Source: Machfu Inc.)
IoT system (Source: Machfu Inc.)

Most leading OPC-UA HMI vendors have implemented MQTT to an OPC bridge mechanism. This bridging minimizes the impact on the OT groups and eases the effort required to move to modern IIoT connectivity with existing HMI platforms. The use of MQTT enables edge gateways to process and apply business logic to determine what to push to the broker and what to store locally. The logic may be simple, such as update on a state change or percentage change or it may run an advanced set of analytics making control decisions locally and publishing the new state to a topic for others in the system to subscribe and act on.

To make these new systems plug-and-play, the IIoT gateways have device-specific drivers that gather the data by poll, the legacy assets use their native protocols and create hierarchical data models, or topics, that can be published using MQTT. Using this approach reduces the required bandwidth and dramatically reduces latency in reporting alarms and changes to the monitored parameters.

IIoT Application Gateways

The emergence of MQTT protocol, and IIoT platforms such as the OPC Unified Architecture (OPC UA)—which has been widely adopted by oil and gas producers—has opened the door to a new paradigm of industrial IIoT application gateway technology.  These new gateways provide a rich set of edge functionality, cost-effectively providing wireless and wired connectivity that enable the unification of modern web-based applications and traditional SCADA infrastructure.

An excellent example of these new IIOT gateways is the MACHGateway®, developed by MACHFU. Its methodologies enable collection of data from edge assets and push that data to the cloud, without interrupting existing SCADA functionality, without the need to replace existing hardware, and with the simplicity of plug-and-play.

Non-Interruption of SCADA

Traditional SCADA uses centralized back office systems to gather information from remote devices. These systems use polled architectures on serial connections developed 30 years ago, before the advent of the Internet and high-performance processing. New IIoT architectures are removing the limitations of these older polled systems, and bridging existing devices to leverage IIoT systems that move intelligence to the network edge. Supporting this are the new IIoT gateways.

The MACHgateway collects data locally by polling the device, and maintaining device connectivity with existing SCADA systems. Such gateways constantly poll devices at the edge, assessing the system state every second, compared to a traditional 15 minute, or longer, SCADA polling interval. The data gathered is used to create a data model that can be easily understood by other applications and translated to other protocols, allowing the data to be shared in real time with the ecosystem of emerging IIoT applications.


As opposed to traditional approaches for industrial edge connectivity, which are based on developing custom embedded solutions that require reengineering for each device, application gateways, including MACHGateway, leverage the new high-performance hardware platforms using a hardware-independent Android/Linux-based software platform. Application gateways simultaneously leverage chip/module-based connectivity solutions that are lowering wireless connectivity costs.

These gateways provide many features that simplify the development of edge applications by reducing the time to create and integrate them, reducing development time from months or years, to weeks or even days.  This enables innovative edge applications to be created and rapidly integrated to existing infrastructure and IIoT applications. In addition, new applications and business models can be continuously created over the life of the system.


Application gateways have defense-in-depth security built across the layers of the communications stack, all the way up to the applications layer, including role-based access control. Consequently, they implement the best practices of IT security, considerably exceeding that provided by SCADA systems.

The MACHGateway, for example, was built from the ground up with security in mind. The platform has a secure sandbox for different applications, each with its own crypto-technology for data and sharing of data between applications.  Importantly, however, the security built into this gateway does not put a heavy burden on the system.

Edge Data and Big Data Collection

In traditional oil and gas applications, specific data from a single operation is monitored and collected by SCADA, which then issues commands to control that single operation. With IIoT platforms utilizing new application gateways, that same well data can be used in multiple applications, including overall equipment effectiveness, loss, waste, production efficiency, and even identifying if differences of efficiency exist between wells on the pad.

Within this framework, data can be collected to a big-data cloud database which is designed to efficiently provide information to different applications that are requesting data. This means that a much higher volume of data can be gathered from the oil pad and made available in a common format, to allow for broader usage and analytics.

IIoT Application of MQTT and MACHGateway Monitoring of Rod Pump Controls

An example of using MQTT protocol and MACHgateway methodology for oil field automation can be exemplified in monitoring the state of rod pump controllers (RPC). The RPC controls the operation of an oil well based on data from downhole card, surface card and motor torque. The rod pump controller calculates a downhole card on every stroke of the pumping unit, utilizing complex mathematical computations. In addition, there are often additional I/O algorithms that accurately control and analyze the rod pumping system.

Data flow system (Source: Machfu Inc.)
Data flow system (Source: Machfu Inc.)

There are hundreds-of-thousands of RPCs in the field, each capturing data from every stroke of the well, occurring approximately on 10-second intervals. Given that there is limited memory to store the data and insufficient bandwidth to collect it via a legacy polled system, 99.99% of the data captured is discarded. The RPC is only able to store some basic trends and worst-case conditions. Due to these limitations, SCADA systems typically collect one set of surface-card data and downhole-card data per day, and a basic status several times per hour.

Conversely, a local gateway can gather and process the RPC data, monitoring of every stroke, and provide real-time notification of issues. Real-time data can be time-stamped and stored, enabling backend systems to collect high-resolution data around flagged issues. Analytics can also be performed on site to flag issues and take corrective action. If high-speed communication is used, like LTE or Broadband PMP, the gateway simplifies pushing bulk data to the cloud.

High-resolution time stamped data stored at the edge can be correlated and conditionally sent to backend systems, minimizing the data that needs to be sent. All of the data can be pushed as MQTT topic to a broker, which simplifies integration with other tools and dashboards, enabling productivity improvements and real-time alarming throughout the enterprise.

Simplifying Edge-to-HMI Connectivity

As oil and gas executives evaluate possible IIoT protocol and application gateway solutions to achieve an optimum edge-to-HMI connectivity, one factor may be considered as playing a pivotal role—that being simplicity of system design.

Simplicity in the development of open, standardized web-based solutions that enable sharing of data across the enterprise, without abandoning existing hardware and middleware that is already in place. Simplicity in the development of edge applications where they can be created and rapidly integrated, plug-and-play, to existing infrastructure and IIoT applications. And, simplicity of network security, which does not tax the system.

For upstream and midstream oil and gas producers, these operational simplicities will open the door to improved asset utilization, higher process efficiency and productivity, and lower cost of operation.

About Machfu:

Machfu simplifies the complex landscape of Industrial IoT by easily connecting data at the edge to the cloud and legacy enterprise systems, for business analytics. Its products, the Machfu Platform and Gateway, bring edge-to-enterprise connectivity to existing industrial infrastructure that can be deployed and scaled quickly.

Machfu’s gateway platform has the hardware interfaces, device protocol libraries, cloud protocol libraries and legacy SCADA system connections to enable connecting any industrial device to any back-end system easily.

For more information, contact John Geiger, vice president of business development at Machfu, at or 301-540-5372.