Recently E&P sat down with digital experts to discuss the Big Data challenges facing companies on the digital transformation journey. Among the group were Philippe Herve, vice president of oil and gas solutions for Spark Cognition, and Jonathon Shea, president of TAB. The conversation touched on many topics, including artificial intelligence (AI), predictive analytics, personnel issues and more.

E&P: What are the three biggest data challenges facing the upstream oil and gas industry?

Herve: The first is one of scaling. Predictive maintenance has the potential to be hugely beneficial to oil and gas operations.

But when performed manually, predictive analytics requires data scientists and technicians to be constantly collecting and analyzing new sensor data from all assets to monitor their health.

The operations of drilling rigs are too vast and contain too many assets for a team of humans to keep up with.

The second is a lack of data science talent.

Even if it were financially feasible to hire the sheer number of data scientists that would be required to make this approach work, there simply aren’t enough data scientists available to hire.

The gap between the supply and demand for trained data scientists has grown too wide.

The third is model creation and maintenance. Predictive maintenance requires the creation of a model based on prior data, which can be used to predict asset health and operating states of a given asset. In traditional approaches, these models rely on pre-programmed rules and physics-based models, which take a great deal of time for data scientists to design and implement and perform poorly in predicting edge cases that occur under extreme or unusual operating parameters. Furthermore, if a single variable of the asset is changed, the model is instantly rendered useless.

Shea: Technology is rapidly changing how energy companies do business.

Overall, companies need to embrace the benefits that technology and digitalization can bring to their business or risk falling behind their competitors.

Even though the price of oil is improving, the industry is still feeling the impact of cutting back on resources as a result of the downturn.

There is still the challenge to do more with less.

Companies need to work more efficiently, mitigate risk and be profitable.

Information is a key corporate asset, and timely access to accurate information provides a strong competitive edge.

There are many challenges associated with Big Data, including how it is captured, stored, shared and updated.

It’s time for the energy industry to embrace the power of digitalization and technology as it relates to managing business-critical information.

The benefits of a comprehensive Big Data strategy are significant: greater access to important information that is crucial in decision-making, the ability to extract valuable insights from those data, cost savings, streamlined processes, risk mitigation and increased profitability.

The brisk pace of mergers, acquisitions and divestitures pose many challenges. According to Deloitte, more than $227.7 billion in deals are projected to take place in North America in 2018. With any deal comes data, which are key to maximizing a new asset as quickly as possible. Companies organize and manage their data differently— some files may be physical, such as well files, while others are digital. In acquisitions, a company needs to be able to organize and integrate this information quickly, so it is easily accessible, accurate and up to date.

From a practical perspective, data management needs to be considered from the pre-deal stage forward. It’s part of the asset being bought or sold, and how this transition and integration process happens should be part of the conversation throughout the negotiations and deal process. Time is money. If there is a clear, efficient process for integrating new information, companies will realize greater profitability from a new asset sooner, while also minimizing risk to their organization.

Establishing consistent industry best practices for how information is managed throughout the deal process is also key to success. Every deal is different, and how and when information is shared varies. Imagine if an energy company has an aggressive acquisition strategy and it deals with multiple collections that are a combination of physical and digital files that are all organized differently. It becomes a huge challenge to get the data organized and accessible to those who need it.

The transition to a digital environment is the biggest challenge for energy companies. Moving away from physical files to a digital format is essential for companies to remain competitive, but it requires a shift in company culture (people love paper) and business processes. It’s not as simple as scanning documents. A company needs systems in place that will help it organize its information in a way that is accessible to employees and that ensures all the required documentation to meet regulatory requirements. This is a major undertaking that permeates all aspects of a company as it needs employees to be willing to change their behaviors, and it needs to be able to manage all documentation on a day-forward basis so that it is compliant with industry rules and regulations ongoing.

Companies who invest in a proper digitalization strategy will see the benefits including a reduced physical footprint, greater efficiency, increased productivity and profitability, and reduced risk with information compliance. Energy companies who have successfully gone through the transformation process incorporate best processes, such as automated document recognition, for organization and classification. They are willing to rethink their business processes and capture and share Big Data in different ways, such as cloud-based solutions.

E&P: It’s been said that the energy industry uses just 5% of the data it collects. What are the best strategies to extract and act on the value of the collected data?

Herve: The fact is no strategy allows human operators to sift through and analyze these massive quantities of data. AI is a necessity to manage data and extract its value efficiently. An increasing number of companies are now employing AI to uncover new potential in their operations by analyzing new and historical data. Not all data are equally valuable; the quality of an AI model’s insights are dependent on the quality of the data it is fed. High-quality data mean data with a consistent level of time granularity, sensor resolution and reading accuracy. It also needs to be data in which one can reasonably expect to find a pattern.

Shea: Companies need to take advantage of the analytics that comes from Big Data. Energy companies deal with huge volumes of information, but unlocking the power of that information can lead to important insights that will direct business decisions. AI and automated document recognition can be utilized to capture additional metadata from previous digitalization or data capture projects.

E&P: How can the oil and gas industry, known for being secretive, become more open with its data while also maintaining its competitiveness?

Herve: Again, this is where artificial intelligence comes in. Oil and gas companies have historically faced the challenging decision of whether to allow outsiders to see their sensitive data or keep it hidden and lose out on the valuable insights it contains. By integrating artificial intelligence solutions into operations, oil and gas operators can ensure a much higher level of data confidentiality, since it is not humans that will be combing through their proprietary data, but machines.

Shea: An industry commitment to data management standards and guidelines will facilitate more transparency. There are standards in the oil and gas industry that have been successful, such as WITSML [wellsite information transfer standard markup language]. In relation to exchanging drilling data, we believe that a standard should exist for data management.

E&P: For companies just beginning to embrace digital transformation for its systems/services, what advice do you have in selecting the right collaborative partner?

Herve: Selecting the right partner to work with on an AI implementation can be daunting, with so many companies claiming to sell “AI” products and services. Selecting a vendor means first vetting their AI credentials—does the company specialize in AI solutions, or is AI merely a tacked-on feature to preexisting products? The latter will never give the results or value of the former. It’s also important to work with a vendor that understands the importance of collaboration with all personnel within an oil and gas operation to ensure smooth implementation at all levels of the client’s organization. An external machine learning company is not going to replace in-house data scientists but must understand how to augment tasks and seamlessly work with them.

Shea: Look for a partner with extensive experience and an in-depth understanding of the energy industry and its information requirements. Companies need a partner with a wide range of solutions to optimize business processes and to organize, access and manage paper and electronic documents. Given the unique challenges of the industry and the strict compliance issues, a partner that offers everything from filing and storage products to software, consulting and services is needed. Start by doing your research. The right partner will work closely with an energy company to develop a customized plan that will guide the organization through all the necessary steps and processes to successfully migrate to a digital environment.

E&P: When it comes to people, how can a company best harness the knowledge of its older employees to train/ guide its younger employees?

Herve: The problem many companies are facing is that they are rapidly losing experienced employees to the “Great Crew Change.” AI systems are crucial to help companies preserve the tribal knowledge of experienced SMEs [subject matter experts] as they retire and pass that knowledge on to the younger generation. Those experienced employees are also still a critical resource for companies as they work to implement AI, and not just in passing their experience along into AI systems. Machine learning companies need people inside the client oil and gas company who understand the analytics at hand. A successful machine learning implementation depends on the collaboration between vendors who have experience with machine learning and internal data scientists who have intimate knowledge of their own company’s operations.

Shea: Experienced employees have extensive knowledge about documents and data, and how it is used within a company. One of the best things that experienced employees can do is to champion and support the transition to a digital environment. Given the complexity of this transition, a company needs employees who will embrace the move away from paper to digital information. Digitalization must become part of a company’s culture as well as its processes to be truly successful. Experienced employees can help guide the transition from physical to digital data and can teach younger employees how data are used in various aspects of the business.

E&P: How far along do you see the industry in five years on its digital transformation journey?

Herve: We are at a critical turning point for the oil and gas industry. After the recent string of downturns, oil and gas companies increasingly realize that they cannot simply rely on traditional methods and hope that it will be enough. New strategies are necessary for operators that wish to remain competitive, and more and more companies realize this. That being the case, five years from now I believe the oil and gas industry will already look substantially different from today, as organizations integrate artificial intelligence, automated model building and predictive analytics into the core of their operations.

Shea: We are optimistic that more and more energy companies of all sizes are seeing the value of digitizing their information. The oil and gas industry is highly competitive and must adapt quickly to the fluctuating price of oil and changing market demands. Companies often mention that the state of their records (data) is a reflection of their organization and want to be seen as best in class, particularly when records are exposed during A&D [acquisitions and divestitures]. Having quick access to accurate information is essential to making important business decisions. More and more companies are recognizing this, and we think there will be a strong push across all energy companies to embrace digitalization in the next few years.