Email from MfE to DoC, cced to Stats NZ, dated 5
October 2017 - with 3 page attachment
Withheld under Cc:
section 9(2)(a) Subject:
Environmental Reporting - International Visitor Survey data
Thursday, 5 October 2017 9:25:02 a.m.
IVS - QA summary.docx
Kia ora Elaine
We’re following up the International Visitor Survey data not being included as an
indicator/measure in our upcoming land domain report.
We’ve put together a high-level document briefly explaining:
· The QA process done by Stats NZ when they consider a new measure for inclusion in our
Environmental Reports, specifically the data quality criteria
· The national environmental reporting requirements
· The outcome of the QA for IVS, specifically how the IVS did not meet our quality
thresholds for relevance to our legislative topics (which is one of the data quality
We’ve attached the document for you to read and we’re happy to follow up and discuss this with
you and/or your colleagues over the phone. While it has not met our purposes for an
indicator/measure this time round, we are still exploring options on how the IVS information
could be incorporated into the report.
and the land domain team
, Operations / Environmental Reporting
Ministry for the Environment – Manatu Mo Te TaiaoWebsite: www.mfe.govt.nz
3 The Terrace, PO Box 10362, Wellington 6143
Please consider the environment before printing this email
Background – Environmental Reporting Quality assurance (QA) process
When considering a new measure for inclusion in our Environmental Reports, we assess its
applicability and quality at two stages:
• Stage 1 – a conceptual fitness for purpose review of the data and methods, prior to data
delivery, to determine its relevancy and application to our regulatory reporting topics. For
environmental reporting. If the data seem applicable, it is requested for potential inclusion
as a new measure.
• Stage 2 – Quality assurance (QA) checking of the data to ensure it was compiled and
prepared to the standard expected,(as per the data request) and includes checking for
missing values, outliers, or unusual movements or levels once final data is in.
In both stage 1 and 2, we assess the data against six data quality criteria (Principles and Protocols for
Producers of Tier 1 Statistics)
. Table 1 lists the six data quality criteria and shows some of the
additional considerations in relation to environmental data.
Table 1: Data quality criteria and their applicability to environmental statistics
Applicability to environment
How much the statistical product meets Issues include: the extent of geographic
customer needs in coverage, content,
coverage, fit to topic, and coherence with
the pressure-state-impact framework.
How much the information correctly
Lack of consistent use of definitions. Many
describes the phenomena it was
sources of uncertainty, including that of
designed to measure.
Whether data produced are up-to-date, Many collections are ad hoc or on a one-off
published frequently, and delivered to
basis; data may be collected at different
How easily customers are able to
Extensive use of modelling, for which the
access and understand the statistical
input data are often not readily available or
data and its supporting information.
understood. The underlying lowest-
frequency or untransformed data is often not
Whether statistical information can be Measurement approaches may change
successfully brought together with other over time. International statistical standards
statistical information – within a broad
still being finalized.
analytical framework and over time.
Interpretability The availability of supplementary
Ability to make national inferences from
information and metadata that is
non-random samples. As administrative
necessary to interpret and use the
data, there may not be complete metadata.
The relevance and accurac
Released y of the data are the main data quality criteria for determining robustness
and fitness for purpose. We also consider the remaining data quality criteria (timeliness,
accessibility, coherence/consistency, interpretability) in determining the measures overall data
quality, which is broken up into three categories (a national indicator, a case study, or as supporting
National Environmental Reporting (ER) requirements
Our aim for the Land 2018 report was to develop a pressure
indicator for use of the conservation
estate under the Resource use and management and other human activities
regulatory topic. We do
not currently report on visitor pressures on the land environnment and this is a information gap
under the regulated topic.
We assessed the International Visitor Survey (IVS),
along with other potential data sources, including
the Domestic Travel Survey and the International Travel and Migration dataset, to
see if we could
produce a measure that met our purposes for ER.
Our target concept was the pressure of visitors on the conservation estate in terms of volume of
visitors, the areas most visited and least visited (e.g. National parks, glaciers, caves), the sorts of
activities participated in, and the extent to which these activities are participated in. The premise
being increasing visitor numbers can put pressure on our land environment, affecting the state of
the land and its flora and fauna.
We accessed the IVS at the first stage of QA (the conceptual fitness for purpose check).
For the purposes of environmental reporting the IVS did not meet our quality thresholds for
• The IVS asks about participation in activities, not how much they participate, so the quantity
of demand for different activities and where it occurs does not meet our requirements
• The statistics and surveys collected make it difficult to accurately attribute recreational use
to the attraction of particular natural land areas or ecosystems
• The IVS is focused on marketing characteristics of international visitors – e.g. the propensity
of different nationalities to go walking and hiking – not the amount or the natural areas they
Given these issues, the IVS was not considered suitable for showing the physical effect of visitors on
the state of the land domain. We would have to repurpose the content of the IVS for a distinct
purpose that it was not designed for and have to infer a lot about the pressures from tourist
patterns on the land environment.
The link between Tier 1 statistics and ER
Our Environmental reports contain a vast range of measures, some of which are Tier 1 statistics or
might become Tier 1 statistics over time. Some existing Tier 1 indicators won’t fit our regulatory
framework, and would therefore fail our relevance to the topic criteria. Tier 1 stats, and indicators
for ER, are two distinct things with different purposes.
Many Tier 1 statistics features are common to those used in environmental reporting. However, Tier
1 statistics do not automatically ‘qualify’ for inclusion in environmental reporting – due either to
relevance (eg the ability to relate the measure to our regulatory topics and within our pressure-
state-impact framework) or to accuracy (eg using a Tier 1 statistic at a lower level than it can be
reliably reported on).
Good practice guide for environmental reporting