Understanding Crude Oil Prices
by James D. Hamilton in Energy Politics Archives Winter 2008
As world oil prices face renewed downward pressure in 2025, revisiting sound historical analyses is essential to our understanding of the broader structural and speculative dynamics at play. James D. Hamilton’s 2008 article, Understanding Crude Oil Prices, provides one of the most comprehensive investigations into the interplay between economic theory, statistical patterns, and real-world supply-demand behaviors in oil markets. Through an integrated review of price elasticity, futures markets, the role of speculators, and Hotelling’s principle of resource depletion, Hamilton argues that the apparent unpredictability of oil prices is not a failure of theory, but a confirmation of it—particularly under conditions of low elasticities and sparse excess capacity.
The insights from this article are perhaps even more relevant today. As global prices slump despite persistent geopolitical tensions and the exhaustion of low-cost production in mature basins, analysts can apply Hamilton's framework to understand whether current price shifts reflect speculative retrenchment, waning demand from major consumers like China, or storage dynamics typical of price reversion phases. The article’s findings—that oil prices follow a near-random walk, and that speculative forces may cause large, temporary misalignments—are especially instructive when examining the recent divergence between futures curves and spot benchmarks. For policymakers and market participants navigating today’s volatile energy landscape, this retrospective provides both a cautionary tale and an analytical toolkit.
This paper examines the factors responsible for changes in crude oil prices. The paper reviews the statistical behaviour of oil prices, relates these to the predictions of theory, and looks in detail at key features of petroleum demand and supply. Topics discussed include the role of commodity speculation, OPEC, and resource depletion. The paper concludes that although scarcity rent made a negligible contribution to the price of oil in 1997, it may be an important feature of the most recent data.
How would one go about explaining what oil prices have been doing and predicting where they might be headed next? This paper explores three broad ways one might approach this. The first is a statistical investigation of the basic correlations in the historical data. The second is to look at the predictions of economic theory as to how oil prices should behave over time. The third is to examine in detail the fundamental determinants and prospects for demand and supply.
Reconciling the conclusions drawn from these different perspectives is an interesting intellectual challenge, and necessary if we are to claim to understand what is going on.
In terms of statistical regularities, the paper notes that changes in the real price of oil have historically tended to be (1) permanent, (2) difficult to predict, and (3) governed by very different regimes at different points in time.
From the perspective of economic theory, we review three separate restrictions on the time path of crude oil prices that should all hold in equilibrium. The first of these arises from storage arbitrage, the second from financial futures contracts, and the third from the fact that oil is a depletable resource. We also discuss whether commodity futures speculation by investors with no direct role in the supply or demand for oil itself could be regarded as a separate force influencing oil prices.
In terms of the determinants of demand, we note that the price elasticity of demand is challenging to measure but appears to be quite low and to have decreased in the most recent data. Income elasticity is easier to estimate, and is near unity for countries in an early stage of development but substantially less than one in recent U.S. data. On the supply side, we note problems with interpreting OPEC as a traditional cartel and with cataloguing intermediate-term supply prospects despite the very long development lead times in the industry. We also relate the challenge of depletion to the past and possible future geographic distribution of production.
Our overall conclusion is that the low price-elasticity of short-run demand and supply, the vulnerability of supplies to disruptions, and the peak in U.S. oil production account for the broad behavior of oil prices over 1970-1997. Although the traditional economic theory of exhaustible resources does not fit in an obvious way into this historical account, the profound change in demand coming from the newly industrialized countries and recognition of the finiteness of this resource offers a plausible explanation for more recent developments. In other words, the scarcity rent may have been negligible for previous generations but is now becoming significant….
Conclusions
In this paper we have reviewed a number of theories as to what has produced the current high price of oil, including commodity price speculation, strong world demand, time delays or geological limitations on increasing production, OPEC monopoly pricing, and an increasingly important contribution of the scarcity rent. Rather than think of these as competing hypotheses, one possibility is that there is an element of truth to all of them.
Unquestionably the two key features in any account are a decrease in the price elasticity of demand and the strong growth in demand from China, the Middle East, and other newly industrialized economies. These twin facts explain the initial strong pressure on prices that may have triggered commodity speculation in the first place. Speculation could have edged producers like Saudi Arabia into the discovery that small production declines could increase current revenues and may be in their long run interests as well. And the strong demand may have moved us into a regime in which scarcity rents, while negligible in 1997, are now an important permanent factor in the price of petroleum.
Notwithstanding, different emphases among these explanations would produce profoundly different predictions as to what will happen next. If speculation and short-run price inelasticity are the key driving factors, we would expect shortly to see potentially dramatic moves downward in price. The scarcity rent, by contrast, is expected to increase, not decrease, over time.
The evidence reviewed in Section 2 highlights the hazards of offering a prediction about what happens next. But the algebra of compound growth suggests that if demand continues to grow in China and other countries at its current rate, the date at which the scarcity rent will start to make an important contribution to the price, if not here already, cannot be far away.
*I thank Menzie Chinn for helpful comments on an earlier draft. James D. Hamilton
Read the full paper here.
ABOUT THE AUTHOR:
James Hamilton received his Ph.D. in Economics from the University of California at Berkeley in 1983. He has been a professor at the University of California, San Diego since 1990 and served as Chair of the Economics Department from 1999 to 2002. He is the author of Time Series Analysis, the leading text on forecasting and statistical analysis of dynamic economic relationships. He has done extensive research on business cycles, monetary policy, and oil shocks, and has been a research adviser and visiting scholar with the Federal Reserve System for 20 years.
Reprinted by the permission of UC Energy Institute 2547 Channing Way Berkeley, California 94720-5180 www.ucei.org .
This paper is part of the University of California Energy Institute's (UCEI) Energy Policy and Economics Working Paper Series. UCEI is a multi-campus research unit of the University of California located on the Berkeley campus.
This report is issued in order to disseminate results of and information about energy research at the University of California campuses. Any conclusions or opinions expressed are those of the authors and not necessarily those of the Regents of the University of California, the University of California Energy Institute or the sponsors of the research. Readers with further interest in or questions about the subject matter of the report are encouraged to contact the authors directly.