27 February 2014
Data error prompts process improvements
The Treasury and Statistics New Zealand are improving their quality assurance processes after data miscalculations
affected estimates of disposable income, Treasury Chief Economist Girol Karacaoglu and Statistics New Zealand Deputy
Government Statistician Vince Galvin said today.
Estimates of household disposable income (income from all sources after deducting income tax and adding tax credits)
were miscalculated for 2007/08 and for 2009/10 to 2011/12 in a way that tended to overestimate the disposable incomes of
households on low to middle incomes.
The problem is limited to this particular area of work and has no bearing on other Treasury or Statistics New Zealand
work streams or data. Policy advice to the government is unchanged, and there has been no impact on benefit payments or
taxation for individuals or households.
“The Treasury and Statistics New Zealand hold themselves to very high standards. We are deeply disappointed to have failed to meet those standards with these data calculation
errors,” said Dr Karacaoglu.
“The miscalculations were human error. But the main problem was that communication protocols between the Treasury and
Statistics New Zealand were inadequate for this data and neither agency had an overview of the complete process to
ensure the anomalies, once identified, were properly explained. Both agencies are determined to fix that,” said Mr
Galvin.
“Statistics New Zealand and the Treasury have changed their quality assurance and communications processes to ensure this problem doesn’t happen again. These new processes will be
in place for the next round of estimates for 2013, which begin in March.”
The impact on two major pieces of work was assessed in detail:
• In the Ministry of Social Development’s Household Incomes Report, the trends and most of the findings remain
broadly valid. The main difference is that the revised figures show that the global financial crisis had a greater
one-off impact on household incomes in 2009 than the original data showed. This was especially the case for lower-income
households with children in private rental accommodation.
• Estimates used by the Tax Working Group in 2009 and by the Treasury in Budget 2010 are unchanged within
statistical margins of error. The main impact is that corrected estimates show the increase in the number of children in
households with less than 50% of median income after the Budget 2010 tax changes is less than previously advised.
“We recognise that issues of estimates of household disposable income are important and we considered carefully what impact the corrected data may have had on the Treasury’s policy advice,”
said Dr Karacaoglu. “We concluded that that advice would not have changed, as the effects of the miscalculations on the
data we used are too small to have altered our judgements. We consulted with the chair of the Tax Working Group,
Professor Bob Buckle, who agreed that the impacts were not material and would have been most unlikely to have changed
the Tax Working Group’s findings.
“It is also important to note that there are no ‘real world’ impacts on New Zealanders from the miscalculations. There
is zero effect on the individual or household benefit payments and tax credits people get, or the tax they pay,” said Dr
Karacaoglu.
The table below summarises the key miscalculation impacts.
MiscalculationScaleKey statistics affectedAccommodation Supplement was double-counted in estimates of disposable income for 2009/10, 2010/11 and 2011/12.$1.2bnThe largest differences are for families with children. For example:
the revised 2011/12 rate is 14% rather than 12% of children (150,000 rather than 125,000) in households with less than
50% of median income
using the 60% of median after housing costs measure, the revised 2011/12 rate is 27% rather than 25% (285,000 rather
than 265,000).
The proportion of all households with a high ‘outgoings-to-income ratio’ (OTI) increases from 26% to 27%, with lower
income households even more likely to have high OTIs, increasing from 38% to 41%.Working for Families payments were overestimated in Treasury estimates of disposable income in 2007/08.
*************
Accommodation Supplement payments were underestimated in Treasury estimates of disposable income in 2007/08.$900m
*************
$650mIn combination these miscalculations had no statistically significant impact on estimates of the number of households on
estimated incomes below 50% and 60% of median income.
Approximately 11,000 fewer children were in households below 50% of median income after the Budget 2010 tax changes than
previously estimated. No statistically significant change to the 60% measure.
The estimates of household disposable income are jointly produced by Statistics New Zealand and the Treasury, and
involve modelling of Statistics New Zealand’s Household Economic Survey (HES) figures by Treasury using its Taxwell
micro-simulation model. The Taxwell-generated estimates are then added by Statistics New Zealand to the HES customised
dataset before being sent on to the Ministry of Social Development and others. Through the Ministry of Social
Development’s Household Incomes Report, analysis of this data has become the basis of almost all detailed analysis of income distribution in New Zealand.
The Treasury and Statistics New Zealand have conducted two thorough reviews into the miscalculations, covering:
• how the errors occurred, including the internal and co-production processes used by Statistics New Zealand and
Treasury
• the Taxwell dataset and associated documentation
Both of these reviews have been externally peer-reviewed by Deloitte. All of the reviews are available at http://www.treasury.govt.nz/publications/informationreleases/disposableincome.
The Ministry of Social Development has published on its website corrected figures for the most commonly used statistics,
together with a short commentary on the impact of the revised data.
ENDS