Pete Hodgson, Convenor of the Ministerial Group on Climate Change, today released a review of two significant economic
models used to analyse the possible effects of ratification of the Kyoto Protocol.
The review has been released in response to an Official Information Act request. It has also been supplied to
Parliament’s Foreign Affairs, Defence and Trade Committee, which is examining the government’s National Interest
Analysis of ratification of the Protocol, in response to requests from committee members.
Commissioned by the government’s Climate Change Project, the review by Arthur Grimes of GT Research and Consulting
investigates economic modelling by the Australian Bureau of Agricultural and Resource Economics (ABARE) for the
government, and by the New Zealand Institute of Economic Research (NZIER) for the Greenhouse Policy Coalition and the
Petroleum Exploration Association of New Zealand.
Ends
AN INVESTIGATION OF THE ABARE & NZIER MODELS USED
TO ANALYSE EFFECTS OF THE KYOTO PROTOCOLS
ON NEW ZEALAND
ARTHUR GRIMES
GT Research & Consulting
10 February 2002
1. PURPOSE
The purpose of this report is to investigate the nature of the models used by ABARE (Australian Bureau of Agricultural
and Resource Economics) and NZIER (New Zealand Institute of Economic Research) to analyse the effects on the New Zealand
economy of New Zealand ratifying the Kyoto Protocols. The report assesses the suitability of the models for that task.
It then suggests ways in which one of the models could be extended to take additional factors into account that may be
relevant to, and so further improve, the analysis.
This comparison of models is important in order to give comfort that the model results present a reasonable assessment
of the effects on the New Zealand economy of ratifying Kyoto. The comparison is even more important given that the two
models appear to give very different assessments of the effects of Kyoto on New Zealand economic outcomes. While the
range of scenarios investigated in the two reports vary, the overall conclusion of the ABARE simulations can be taken to
imply that Kyoto will have only relatively minor economic effects in aggregate (albeit with some substantial sectoral
effects); in contrast, the NZIER assessment can be taken to imply that Kyoto will have strongly negative effects on New
Zealand economic outcomes, particularly on GDP over time.
Differences in the scenarios’ assumptions (e.g. on international emission permit prices, industry coverage, inclusion of
different countries within the protocol, etc) can explain some of the differences in results. However, the generally
more negative conclusions of the NZIER results relative to the ABARE results is likely to be due, at least in part, to
different modelling approaches or to differences in the properties of the two models. For this reason it is important to
assess which tool (if either) is more suitable for the task of assessing Kyoto’s effects on New Zealand.
The approach adopted here has been to use information presented in the reports of the two agencies. This material has
been supplemented by personal interviews with personnel of the two organisations plus additional material supplied by
the agencies to gather further detail.
A brief description of the two models is presented below. Most emphasis in the report is placed on particular issues
which indicate suitability or lack of suitability for the task at hand. In the ABARE case, just one model has to be
assessed; in the NZIER, the two models used in their report are assessed.
The report does not comment specifically on the scenario assumptions within each of the reports. It is an investigation
of the tools rather than of the inputs. Thus no comment is made about the reasonableness of the final results (which
depend both on the tools and the scenario inputs). The report can, however, be used to determine which tool (or variant
of a tool) is best placed to analyse scenarios based on given policy assumptions.
2. ABARE REPORT
The ABARE report uses the GTEM model. GTEM is an international, dynamic 33-sector general equilibrium (GE) model with
considerable disaggregation in industries (including agriculture) that will potentially be affected in a major way by
the Kyoto protocol. It is a standard GE-type model with considerable documentation available (including on the website:
www.abareconomics.com).
In keeping with other GE models, GTEM assumes that each of capital, labour, land and natural resources are used in
production of goods and services in each sector of the economy. Prices adjust to ensure full employment of resources.
Hence it is an equilibrium model which does not attempt to model transitional effects of policy and so cannot model
“temporary” unemployment or other disequilibrium phenomena arising from policy changes.
The standard GTEM model has been extended to include natural resources in order to model Kyoto issues. To extend the
model, ABARE uses the GTAP database (version 4.0e). As noted in the NZIER discussion below, the data which ABARE uses
for New Zealand (e.g. input-output weights) is not fully up-to-date and may miss some points that are country-specific
to New Zealand. This may have some effects on its results.
Emission response functions (ERFs) within sectors are modelled. The ERFs enable reductions in carbon dioxide emissions
through energy efficiency improvements and fuel switching and from reductions in noncombustion sources through adoption
of new technologies and management practices. These responses occur as a reaction to price changes. This is an important
feature to model.
Overall, subject to the points discussed below, the model is appropriate for a long-run analysis of the issues at hand.
Specific Points Related to the ABARE Model
- Each of the scenarios is modelled relative to a reference case where the Kyoto protocol is not ratified. Overall, the
reference case looks reasonable; export and import growth rates appear a little low (at approximately the same rate as
GDP growth, whereas historically the rate of trade growth is higher) but this should not lead to any obvious bias in the
scenario results one way or another.
- It is important to keep in mind that the ABARE results are long run. While the tables refer to results for 2010, there
is nothing in the model which suggests that the long-run transitions depicted will be completed by 2010. Indeed, given
many countries’ experiences with major relative price changes after deregulation (including in New Zealand) it is likely
that the new long-run equilibrium industry configuration could take 10-20 years to complete, with many downside
adjustments occurring faster than many upside adjustments. Thus the present value of the costs of adjustment could well
exceed those documented in the ABARE report.
- Land price effects are incorporated in the ABARE model, which is entirely appropriate and necessary, especially for
agricultural industry reallocation. However no detail is given on the size of the land price adjustments. This
information is important in assessing whether there would be major bankruptcies occurring (especially in agriculture)
and hence whether there would be considerable pressure on the banking system. It is now recognised that the
macroeconomic effects of banking problems (caused by factors such as major land price declines) can be severe and
long-lasting (e.g. the economic effects of the Asian financial crisis, New Zealand and Australia after 1987, and the
Savings and Loans crisis in the USA in the 1980s). If agricultural land prices fell significantly as a result of Kyoto
(this depends on how the protocols are implemented in New Zealand) the negative output effects on the whole economy
could be considerably worse than modelled by ABARE. (Some government intervention as occurred successfully after the
major land-price declines precipitated by the removal of agricultural support in 1984/85 might then be required to
prevent such macroeconomic problems.) This issue is unlikely to be germane if agriculture is exempted from Kyoto
measures.
- ABARE does not model the effects of new forestry plantings in generating new carbon sink entitlements. However,
depending on the nature of the actual policies implemented by government, owners of new plantings may be “rewarded” with
emission credits which would act to increase plantings relative to the effects modelled here. This incomplete modelling
of the positive effects for forestry of the effects of carbon sinks linked to new plantings is likely to lead to an
understatement of the long-run benefits of the Kyoto protocol on forestry and the New Zealand economy generally.
- The assumption that extra government revenues (from the imposition of emission abatement policies) are distributed as
a lump-sum tax back to households also leads to an understatement of the benefits of the protocol for the New Zealand
economy. In practice, increased revenues would be used to lower existing distortionary taxes (e.g. on labour income,
company profits or GST). This lowering in other distortionary taxes would help to bolster economic activity and reduce
existing distortions within the economy. If government revenues from emission credits were significant, this factor
could assume major importance (and is an element in the “double dividend” environmental literature).
Overall, the ABARE results indicate fairly minor economy-wide effects of the Kyoto protocols on macro-economic
variables, but some substantial industry-specific results. This is similar to what has been observed from large tariff
changes in the past and so appears reasonable. Again, however, the historical experience is that positive adjustments of
sectors not originally “hit’ by the policy change can take considerably more time than the negative impacts on directly
affected sectors. Also the effects on the banking system caused by major land price declines (only in those scenarios
where these are predicted to occur - i.e. where methane charges are included) could exacerbate the negative short-run
adjustments. By contrast, the model appears to under-estimate some long-run positive economic effects (arising from
positive effects on forest plantings due to sink credits and from the beneficial economic effects arising from cutting
distortionary taxes). Overall, there is no clear direction of bias caused by this range of issues.
3. NZIER REPORT
NZIER uses two modelling approaches: a static general equilibrium model (similar, in principle, to the ABARE model,
albeit based on a static rather than dynamic approach) and an econometrically estimated equation (or system of
equations) which aims to model a production function with output explained by inputs of labour, capital and energy. The
two modelling approaches are quite separate and these approaches need to be examined separately.
In discussing the results, NZIER reports results from each separate model and reports combined results using effects of
both models. One model (the GE model) is a reasonable attempt to address the issues and the results of that model
deserve attention. There are issues attached to this model which are documented below, but in principle these issues
could be overcome and in some respects they provide an important contrast to alternative assumptions within the ABARE
model.
By contrast, as discussed further below, the econometric approach has no credibility whatsoever. There is virtually no
documentation of these equations available and no staff member is able to discuss, even verbally, what lies behind them.
The statistical estimates and data are not available to enable an independent reviewer to ascertain what has been done
and how reasonable the results are. Quite simply, in the circumstances they should not have been published and/or used
in the study. The results of that approach, and of combined results incorporating that approach, should be entirely
disregarded, at least until such time as NZIER fully documents this work and has had it externally peer-reviewed.
Specific comments on the models follow.
Specific Points Related to the NZIER GE Model
- The NZIER GE model is a newly compiled model for which this is its first assignment. As such, it does not have the
advantage of comprehensive documentation available for the ABARE model. Some documentation is available in Appendix A of
the NZIER report and, while terse, this is a reasonable attempt to explain the content of the model. However, no
justification is given for many of the assumptions underlying the model, e.g. that the consumer trades off a composite
energy bundle of goods with a non-energy composite bundle; that the income elasticity of all demand for all non-energy
goods is unity; that the elasticity of substitution between the composite energy good and the value added bundle for
most sectors is 0.5; that the elasticity of substitution between primary factors is unity (whereas other NZ work
suggests this should be lower). It would be useful in a new model such as this to document the bases for these
assumptions more fully. This is not to say that the assumptions are unreasonable; but it is difficult to assess the
reasonableness of all the assumptions without further documentation.
- NZIER uses more up-to-date input-output weights than does ABARE, and it also takes account of the fact that natural
gas is not piped to the South Island, unlike ABARE. Given the increased share of agriculture in output over the past
decade and given the increasing importance of the South Island dairy industry (a major coal user), inclusion of each of
these factors is likely to make the NZIER results an improvement - in this respect - on the ABARE results. It is
difficult to judge the magnitude of this effect without further modelling using sensitivity tests.
- The NZIER and ABARE models incorporate different elasticities of substitution between domestic and imported goods.
This affects the results. ABARE uses international, historically-based assumptions which may be on the low side for an
open economy such as New Zealand. NZIER uses considerably higher assumed elasticities (based on judgement) which, if
anything, may be on the high side. Thus the two studies seem to set the bounds of reasonableness for these assumptions;
but neither can be considered unreasonable. The empirical work has not been done in NZ at the required level of
disaggregation to ascertain better estimates of these elasticities. Sensitivity tests using either or both models would
be helpful to uncover the magnitude of difference to the results that occurs as a result.
- The modelling of the cement and iron & steel industries incorporates specific threshold effects which allow for the industry closing down once output falls
below a certain level. This contrasts with the ABARE approach which assumes smooth adjustment. The NZIER approach
appears quite reasonable for these industries (and this may be a worthwhile addition to the ABARE model), although the
reasonableness of the exact specification of the threshold effects is difficult to judge. (For instance, the graph on
page 60 indicates that the price of Portland cement is currently approximately 40% below the import price, suggesting
that there is quite a margin in costs that needs to occur before the shut-down scenario occurs. By contrast, from the
results in Table 19, only a very small margin is allowed for before shut-down occurs.)
- As with ABARE, any revenues gained by government from carbon sink entitlements are redistributed back to households
through lump-sum taxes. This ignores the beneficial effects that can be gained by reducing other distortionary taxes and
so leads to an overstatement of the costs of Kyoto under some scenarios.
- Also, as with ABARE, NZIER does not model the effects of new forestry plantings in generating new carbon sink
entitlements. This is also likely to lead to an understatement of the long-run benefits of the Kyoto protocol on
forestry and the New Zealand economy generally.
- Unlike the ABARE report, there are no explicit emission response functions included (although some substitution is
allowed between land, labour, capital and energy). For instance, it appears that no response for agricultural emissions
of methane and nitrous oxide to a methane charge is included (see page 16), thus implicitly assuming that there are no
technologies available currently or in future to reduce these emissions. This may be reasonable for a short to medium
term horizon, but not longer term. Thus the results presented are more “medium term” than long run results.
- For the other sectors, since emission response functions are not included, this will lead to an overstatement of the
negative effects of Kyoto since ability to adjust within a sector to a less emission-intensive production mechanism is
diminished, thus increasing the need to make adjustments between sectors. As an example, discussion on page 55 notes
that the wood products sector should be able to save significant amounts of energy through better utilisation of wood
waste and waste heat, but this is not allowed for in the modelling. As another example, discussion on page 52 notes that
drying milk is energy intensive. It is likely therefore that the dairy industry would shift (at least at the margin)
towards more cheese and less milk powder output if energy rises in price, but such a switch is not allowed for in the
modelling. As another example, the model does not allow for switches in the transport industry to vehicles with
different emissions intensity, but this is a switch that can be expected to occur as energy prices rise.
- It is interesting to note that in the NZIER GE model, energy is solely an intermediate good in the production function
(primary factors being land, labour, capital). Thus energy does not contribute to value added - i.e. to GDP (see page
82). This seems reasonable for most sectors.(To the extent that the documentation allows, it appears that this in direct
contradiction to the NZIER’s econometric approach.)
- While not specific to the underlying models, I note that much of the NZIER discussion is difficult to follow, and some
is potentially misleading. In reporting the GE results, NZIER reports gross output changes rather than value added
(contribution to GDP) changes. It is the latter, not the former that is more relevant for New Zealand’s welfare. As
noted in the previous bullet point, since NZIER assumes that energy does not contribute to value added in most
industries, a change in energy prices can be expected to lead to a drop in gross output (as energy input declines) but
to little change in value added. Thus NZIER would be better to discuss value added (GDP) changes rather than gross
output changes. It is unclear why they do not do so. Because gross output figures are misleading indicators of welfare
changes and because GDP changes are not reported, the best reported figures within the NZIER document to concentrate on
in interpreting the outcomes for the NZ economy are those presented for “household consumption”. These figures
essentially represent long-run income changes for the country.
Specific Points Related to NZIER’s Econometric Work
- The econometric results (section 2.3 and Appendix C) are extraordinary for their lack of detail. There is no detail
provided as to the nature of the production function that has been estimated (e.g. whether it is modelling gross or net
output, whether it nests value added within gross output, etc), there are no details on the data sources used, and there
are no details on the econometric results. Without access to this detail, none of these results can be given any
credence.
- No discussion of how the estimated function(s) relate to other New Zealand empirical work in this area (including
recently published Treasury work). This is normal practice to allow the reader to judge the reasonableness of the
approach adopted.
- No detail is provided as to how the estimated sector and aggregate production functions match up with the production
functions used in the GE model. In the GE model, energy does not contribute to value added, but it is impossible to tell
whether this is the same in the econometric model.
- The estimates are all based on just 20 observations (1980-1999). This is a small sample and (depending on the
econometric results) is likely to make for quite large confidence intervals around the estimates. Nowhere is this
discussed.
- It is stated that an aggregate function is estimated and sectoral production functions are estimated. Nowhere is it
stated whether the aggregate function is consistent with the sum of the sectoral functions.
- It is likely that the estimates treat all vintages of capital as the same in terms of energy intensity. However,
consistent with the discussion on page 22 of the NZIER document, a “vintage-capital” model should be used to allow new
vintages of capital to have different energy intensities and in-built technology than existing vintages. This would have
to be taken into account in econometric work for the purpose of modelling the effects of ratifying Kyoto.
Overall, the NZIER’s GE model is a reasonable starting point to model the issues at hand, although it is previously
“untried” and the write-up is not fully complete. Some aspects also are not entirely appropriate for modelling Kyoto
issues, particularly the lack of emission response functions (unlike ABARE). In other respects, particularly the more
up-to-date input-output coefficients and the specific treatment of South Island energy choices, the NZIER model has some
advantages over the ABARE model.
In contrast, no attention should be paid to the NZIER’s econometric work or to any discussion based fully or in part on
it. It is simply not credible.
Some of the discussion in the NZIER report could be interpreted as being more demanding on the “pro-Kyoto” stance than
on the “anti-Kyoto” stance. An example is the straw man on page 35: “The double dividend could theoretically occur where
¡K revenues from [the environmental] tax are sufficient to finance the entire public budget, and hence eliminate all
distorting taxes”. For the double dividend to occur, it is not necessary that the environmental tax revenues finance the
entire public budget; all that is needed is that they replace some of the existing distortionary taxes. As another
example, the discussion about whether leakage could be lessened is one-sided and is odd in assuming that if New Zealand
were to impose a “carbon tariff’ this may wreak havoc with the world trading system. This is clearly a gross
exaggeration which ignores the usual small country assumption. A third example concerns the sensitivity tests which
varies the size of sinks. The study uses 20 Mt as the base-case and then models only the effects of 5 Mt and 10 Mt
rather than adopting sensitivity tests which are symmetric about the base-case. This unorthodox approach to sensitivity
testing is not justified in the text.
4. COMPARISON OF ABARE & NZIER GE MODELS
The ABARE & NZIER GE models each has strengths and weaknesses. The ABARE model has a well-established pedigree, although it is not
specific to New Zealand. In contrast, the NZIER model, while previously untried, attempts to incorporate assumptions
specific to New Zealand, including use of more up to date input-output data, and takes account of the lack of natural
gas in the South Island. It suffers, relative to ABARE, however, in not incorporating explicit emission response
functions. Also it is unclear whether it is able to report value added changes given that these figures do not appear in
its report.
The two models differ substantially in some of their elasticity assumptions, especially relating to the Armington
elasticities determining consumers’ abilities to substitute between domestic and foreign goods in the same sector.
Neither model appears to have unreasonable estimates and sensitivity analysis is required to test the effects on results
of the differing estimates.
Neither model incorporates the effects of: new forestry plantings on additional sink credits, land prices on the
financial sector and thence on to economic activity, transitional effects, or government reducing distortionary taxes as
a result of sink credit revenues. These effects work in opposing directions and, for each model, it is therefore
impossible to conclude on the direction of any bias in the results arising from omission of these factors.
5. FURTHER MODELLING APPROACHES
Given that the ABARE model is based on a well established GE model and incorporates emission response functions, it
would be useful to use this model to conduct further analysis of the effects of Kyoto ratification.
However, it would be extremely useful to conduct sensitivity tests of its results using, as inputs, some of the
approaches used in the NZIER model. In particular, for any given scenario (preferably one close to the government’s most
likely policy option) it would be useful to compute the results of ratificiation based on the following model variants:
1. The base case model as used in the previous ABARE work.
2. Base case using up-to-date (NZIER’s/Statistics NZ’s) input-output coefficients.
3. Base case using NZIER’s approach for energy substitution for the South Island.
4. Base case using threshold effects in cement and iron & steel.
5. Base case using higher Armington elasticities to proxy NZIER approach.
6. Base case with ability to generate additional sink credits through new forestry plantings.
7. Base case with government cutting distortionary taxes to recycle extra revenues.
(Note that it is unlikely that the model could handle transition effects or effects of land prices on the banking
system.)
After further consideration of the reasonableness of these various approaches, some combinations of variants could be
considered. For instance, a combination of cases 2, 3, 4, 6 & 7 could possibly be considered a “best practice” approach and could be well worth applying to investigate the effects
of likely policy options.