Email from MfE to DoC, cced to Stats NZ, dated 5
October 2017 - with 3 page attachment
From:
To:
[email address]
Withheld under
Cc:
;
section 9(2)(a)
Subject:
Environmental Reporting - International Visitor Survey data
Date:
Thursday, 5 October 2017 9:25:02 a.m.
Attachments:
image001.jpg
IVS - QA summary.docx
Kia ora Elaine
We’re following up the International Visitor Survey data not being included as an
indicator/measure in our upcoming land domain report.
We’ve put together a high-level document briefly explaining:
· The QA process done by Stats NZ when they consider a new measure for inclusion in our
1982
Environmental Reports, specifically the data quality criteria
· The national environmental reporting requirements
Act
· The outcome of the QA for IVS, specifically how the IVS did not meet our quality
thresholds for relevance to our legislative topics (which is one of the data quality
criteria)
We’ve attached the document for you to read and we’re happy to follow up and discuss this with
you and/or your colleagues over the phone. While it has not met our purposes for an
indicator/measure this time round, we are still exploring options on how the IVS information
Information
could be incorporated into the report.
Nga mihi
Withheld under
and the land domain team
section 9(2)(a)
Official
Withheld under
section 9(2)(a)
, Operations / Environmental Reporting
the
Ministry for the Environment – Manatu Mo Te Taiao
Website: www.mfe.govt.nz 3 The Terrace, PO Box 10362, Wellington 6143
under
cid:image003.jpg@01D26A65.99010DF0
Released
P
Please consider the environment before printing this email
Background – Environmental Reporting Quality assurance (QA) process
When considering a new measure for inclusion in our Environmental Reports, we assess its
applicability and quality at two stages:
• Stage 1 – a conceptual fitness for purpose review of the data and methods, prior to data
delivery, to determine its relevancy and application to our regulatory reporting topics. For
environmental reporting. If the data seem applicable, it is requested for potential inclusion
as a new measure.
• Stage 2 – Quality assurance (QA) checking of the data to ensure it was compiled and
prepared to the standard expected,(as per the data request) and includes checking for
missing values, outliers, or unusual movements or levels once final data is in.
In both stage 1 and 2, we assess the data against six data quality criteria
(Principles and Protocols for
1982
Producers of Tier 1 Statistics). Table 1 lists the six data quality criteria and shows some of the
additional considerations in relation to environmental data.
Act
Table 1: Data quality criteria and their applicability to environmental statistics
Criterion
Standard descriptor
Applicability to environment
Relevance
How much the statistical product meets Issues include: the extent of geographic
customer needs in coverage, content,
coverage, fit to topic, and coherence with
and detail.
the pressure-state-impact framework.
Accuracy
How much the information correctly
Lack of consistent use of definitions. Many
Information
describes the phenomena it was
sources of uncertainty, including that of
designed to measure.
model selection.
Timeliness
Whether data produced are up-to-date, Many collections are ad hoc or on a one-off
published frequently, and delivered to
basis; data may be collected at different
schedule.
frequencies.
Accessibility
How easily customers are able to
Extensive use of modelling, for which the
Official
access and understand the statistical
input data are often not readily available or
data and its supporting information.
understood. The underlying lowest-
frequency or untransformed data is often not
the
available.
Coherence/
Whether statistical information can be Measurement approaches may change
consistency
successfully brought together with other over time. International statistical standards
statistical information – within a broad
still being finalized.
under
analytical framework and over time.
Interpretability The availability of supplementary
Ability to make national inferences from
information and metadata that is
non-random samples. As administrative
necessary to interpret and use the
data, there may not be complete metadata.
statistics effectively.
The relevance and accurac
Released y of the data are the main data quality criteria for determining robustness
and fitness for purpose. We also consider the remaining data quality criteria (timeliness,
accessibility, coherence/consistency, interpretability) in determining the measures overall data
quality, which is broken up into three categories (a national indicator, a case study, or as supporting
information).
1
National Environmental Reporting (ER) requirements
Our aim for the Land 2018 report was to develop a
pressure indicator for use of the conservation
estate under the
Resource use and management and other human activities regulatory topic. We do
not currently report on visitor pressures on the land environnment and this is a information gap
under the regulated topic.
We assessed the
International Visitor Survey (IVS), along with other potential data sources, including
the Domestic Travel Survey and the International Travel and Migration dataset, to
see if we could
produce a measure that met our purposes for ER.
Our target concept was the pressure of visitors on the conservation estate in terms of volume of
visitors, the areas most visited and least visited (e.g. National parks, glaciers, caves), the sorts of
activities participated in, and the extent to which these activities are participated in. The premise
being increasing visitor numbers can put pressure on our land environment, affecting the state of
1982
the land and its flora and fauna.
Act
Outcome
We accessed the IVS at the first stage of QA (the conceptual fitness for purpose check).
For the purposes of environmental reporting the IVS did not meet our quality thresholds for
relevance:
• The IVS asks about participation in activities, not how much they participate, so the quantity
Information
of demand for different activities and where it occurs does not meet our requirements
• The statistics and surveys collected make it difficult to accurately attribute recreational use
to the attraction of particular natural land areas or ecosystems
• The IVS is focused on marketing characteristics of international visitors – e.g. the propensity
Official
of different nationalities to go walking and hiking – not the amount or the natural areas they
visit.
the
Given these issues, the IVS was not considered suitable for showing the physical effect of visitors on
the state of the land domain. We would have to repurpose the content of the IVS for a distinct
purpose that it was not designed for and have to infer a lot about the pressures from tourist
patterns on the land environment.
under
The link between Tier 1 statistics and ER
Our Environmental reports contain a vast range of measures, some of which are Tier 1 statistics or
might become Tier 1 statistics over time. Some existing Tier 1 indicators won’t fit our regulatory
Released
framework, and would therefore fail our relevance to the topic criteria. Tier 1 stats, and indicators
for ER, are two distinct things with different purposes.
Many Tier 1 statistics features are common to those used in environmental reporting. However, Tier
1 statistics do not automatically ‘qualify’ for inclusion in environmental reporting – due either to
relevance (eg the ability to relate the measure to our regulatory topics and within our pressure-
state-impact framework) or to accuracy (eg using a Tier 1 statistic at a lower level than it can be
reliably reported on).
2
References
Good practice guide for environmental reporting
http://www.stats.govt.nz/browse_for_stats/environment/environmental-reporting-series/good-
practice-guide/dissemination.aspx#
1982
Act
Information
Official
the
under
Released
3
Document Outline
