[Editor’s note: Opinions expressed by the author are his own.]

It’s an old truism of commodity markets that the best cure for high prices is, eventually, high prices.

Demand gets curbed, new supply gets stimulated and in time prices retreat as the market moves back into balance. But in European energy markets right now, that old truism is being tested.

Natural gas prices have soared to the highest on record, both in continental Europe and in the U.K. (at least for the time of year) and with it everything from the price of home heating to electricity is rising sharply.

In countries such as Spain and Germany, the soaring cost of electricity has moved to near the top of the political agenda. In the U.K. the energy regulator has said its so-called price cap on energy bills will need to rise by £139 (US$192.26) later this year, or more than 10%. The Bank of England warned this week that inflation could reach 4% by the end of this year, driven in large part by rising energy costs. 

The issue, beyond the short-term pain for consumers’ wallets, is there are reasons to believe high natural gas prices might not stimulate the usual response.

The reason for this is that natural gas is a fuel that occupies an awkward space at the nexus of the energy and climate debates.

It is far cleaner than coal, producing roughly half as much CO2 when used to generate electricity, but is still disdained by environmentalists as a fossil fuel that contributes to emissions and climate change.

But in the short term, a stepped-up fight against climate change is boosting demand for natural gas. Carbon prices—one of the EU and the U.K.’s main tools for combating climate change—have soared in response to increasingly ambitious political targets to cut emissions.

The result is coal becomes less economic to burn for electricity, which is a good thing for the climate. But it also means it gets replaced with natural gas in many instances where renewables cannot yet take up the slack.

Energy companies are in a bind. The old solution would have been to invest more in raising gas production. But with most developed countries adopting plans to be “net zero” on carbon emissions by 2050 or earlier, the appetite for throwing billions at long-term gas projects is diminished.

At the same time the gas market has become more global. LNG, which can be shipped on tankers around the world, has started to connect up once disparate gas markets from South America to Asia. 

As pressure is put on developing economies such as China to start to slow their growth in coal consumption, guess what fuel they are turning to instead? LNG imports are soaring in Asia, but this has an obvious impact on the amount available to sail to Europe. 

The result is something approaching a global gas crunch.

The only country that arguably has spare capacity available in the short term is Russia. So far Gazprom, its monopoly pipeline gas exporter, has declined to send any additional supplies to Europe and is seen as unlikely to do so before the controversial Nord Stream 2 pipeline is approved.

Prices could ease off later this year if there’s a mild winter, but hope is rarely the best strategy. More LNG will hit the market later this decade, but BP’s CFO Murray Auchincloss said this week that he did not expect prices to ease significantly until 2022 at the earliest, and even then there is a great deal of uncertainty.

None of this is to argue that countries should not be looking to raise carbon prices or curb demand for fossil fuels. But it illustrates the kind of ruptures we should probably expect to see more of during the energy transition. 

In the oil industry there are already those who think a period of +$100-a-barrel oil is on the horizon, as companies scale back investments in future supplies, while demand is expected to keep rising for most of this decade at a minimum.

The International Energy Agency argues that if the world is truly to achieve net zero before 2050, then no more investments are needed in new oil or gas fields. But their own forecasts show that on the current trajectory demand is still rising fast enough to suggest new fields will be needed.

The key challenge for governments is therefore to accelerate efforts to tackle demand. Volatility in energy prices during the transition is probably a given.

But they are also a good opportunity to speed up measures to tackle the root cause of the problem, which isn’t simply that too much gas is being produced but that too much is being consumed. If they can manage that the old truism should still ring true.

Energy Source is a twice-weekly energy newsletter from the Financial Times. It is written and edited by Derek Brower, Myles McCormick, Justin Jacobs and Emily Goldberg.