Content recommendations on topics for the 2016 Volume Test
Purpose
To best prepare decision makers to make informed decisions on which topics are included
and excluded from the 2016 Volume Test.
In scope of this document
This document highlights potential content change (for trial in the Volume Test or for paral el
testing)
QMD and RI perspectives have been considered as part of content considerations and these
perspectives are included for reference
Out of scope of this document
This document does not detail topics included but with no change and these are detailed in
linked documentation.
Variables Specifications are living background documents. A comprehensive Volume Test
version of these wil be made available for reference during this decision making process and
for development of systems and tools.
This document does not address final wording of forms which wil be signed off in a separate
process driven by RI in June 2016.
Primary Content Test Objective
Test the proposed 2018 Census new and changed questionnaire content:
a. Is assessed as feasible for a self-completed questionnaire where concepts are clearly
defined and individual questions are clearly understood by respondents
b. meets requirements and customer’s needs for data quality - expected quality
Opportunity
Cognitive and mass completion testing to date has been paper based. The Volume Test is an
important opportunity to test both internet and paper forms. Internet form design may provide
solutions to some issues raised through testing to date.
Towards Census Test 2017
The Volume Test is a key opportunity to gather evidence in a test driven census environment. It is not
the only opportunity. Paral el testing wil be progressed for key topics to feed into content
considerations for Census Test 2017. What goes into the Volume Test may be different from what
goes into Census Test 2017.
Census content Volume Test recommendations
Content to be included - Individual Form
Existing content to remain New content
Changes to existing content
the same
Name
Second residence
Sex
Age
Usual residence one year ago
Post-school qualification
Usual residence
Step-families
Highest qualification
Years at usual residence
Main means of travel to
Disability/activities limitation
Usual residence five years
education/address
Main means of travel to work
ago
Sexual orientation
Tenure holder (yet to be
Census night address
confirmed – proposed new
Birthplace
licence to occupy category
Years since arrival in New
removed, but other changes
Zealand
possible)
Ethnicity
Languages spoken
Maori descent
Iwi affiliation
Religious affiliation
Partnership status in current
relationship
Living arrangements (and the
existing range of derived
family and household
variables)
Legal y registered
relationship status
Cigarette smoking behaviour
Number of children born
alive
Highest secondary school
qualification
Study participation
Total personal income
Sources of personal income
Work and labour force status
Status in employment
Occupation
Sector of ownership
Industry
Workplace address
Hours worked per week
Unpaid activities
Content to be included – Dwel ing Form
Existing content to remain New content
Changes to existing content
the same
Dwel ing address
Housing quality – access to
Occupied dwel ing type
Dwel ing counts and types of basic amenities, dampness,
Sector of landlord
unoccupied dwel ings (note – mould
Access to telecommunication
this isn’t just from the
systems (fax category dropped)
dwel ing form, it also requires
Type of heating (previously fuel
information from the field)
types)
Number of occupants on
Number of motor vehicles
census night
(output change)
Relationship to reference
person (and the existing
range of derived family and
household variables)
Number of bedrooms
Number of rooms
Tenure of household (but
minor changes possible)
Weekly rent paid by
household
Census volume test - content recommendations
Dwel ing form
Included
New content
Housing quality – damp, mould and amenities
Content recommendation
We recommend including mould, damp and access to
basic amenities in the VMO.
Content summary
It is important to keep testing and gather evidence on
the fol owing variables together:
Mould
Dampness
access to basic amenities.
There is a high level of customer interest in this data
and content have ranked this topic highly in the
determination process.
The content team acknowledges current issues in
questionnaire development but the importance of
these topics as a cluster makes gathering further
quantitative evidence is a high priority. Inclusion in
the Volume Test wil al ow quantitative analysis and
assessment of overal data quality.
In addition there wil be minimal impact on the flow of
or the rest of the the questionnaire.
Note col ecting information on insulation has already
been considered and discounted, as indicated in the
variable spec.
Mould
Both QMD and RI have questioned how A4 mould
size relates to effects on health and the usefulness of
this as an indicator. This information is currently used
to assess risk of negative health effects at the
population level. (A4 size equates to category 2 on
the mould severity scale they used in fit for purpose
rental housing research. This amount of mould was
highly predictive of wheezing). Also, it is a practical
way of quantifying the extent of mould and more
reliable than a subjective question that asks
respondents whether mould is a minor problem,
major problem, or no problem.
Dampness
Further external engagement about dampness
confirmed that this information is an important
measure of the indoor environment, has an
independent ef ect on health, and would be very
important information, especial y if it was combined
with the mould question.
Amenities
Those lacking amenities are expected to be a smal
population so it’s important to include this in a large
scale test. An amenity has to be useable (functional)
to be counted.
Some changes to the amenities question may be
needed – see the recently updated variable
spec.Changing to the opposite approach can be
considered but would make the question style
dif erent from others such as telecoms and might lead
to respondent error.
Another idea is separate yes/no questions, but this
would take up more space.
One intended use of the amenities data is to help
measure severe housing deprivation
QMD
Mould
Recommended for inclusion in Volume Test, with
some reservations
Dampness
Recommended for inclusion in Volume Test, with
some reservations
Amenities
Not recommended for inclusion in Volume Test in
current format.
Recommend further testing in communities where we
expect some deprivation of these
basic amenities to be prevalent.
RI
Mould
Disagree with inclusion
Dampness
Disagree with inclusion
Amenities
Disagree with inclusion
Next Steps
Further customer consultation on housing
quality
Paralel testing and development as required
with target smal population groups
Changes to existing content
Number of rooms
Content recommendation
We recommend inclusion but with some minor
changes.
Content summary
Agree that use of caravans is probably relatively
uncommon, and testing so far may not have included
many in this situation, however this is part of what’s
defined as a bedroom in the standard and it’s
important to include them, as we’ve done previously.
Not including them could affect data comparability
over time and artificial y inflate household crowding
rates.
As discussed with QMD, it might be awkward to add
another bul et point about caravans.
The 2013 approach had bedrooms as a separate
question so it worked then.
Other possible options discussed with QMD to
address the caravan issue were:
Move the caravans category to the
bot om of the list (would this help)
Have a ‘bubble’ for it. But there doesn’t
seem to be space for that.
Have a guidenote. But people may not
realise they need to think about caravans
they use as a bedroom and so may not
refer to the guidenotes.
Not discussed with QMD - Maybe add
the necessary wording to the bedroom
response option. But this would make it
long.
QMD
Recommend this question format for inclusion in
Volume Test, with the removal of ‘caravans …’ as a
standalone category.
RI
Agree with inclusion
Next Steps
Minor - refinements to question wording
Heating appliances
Content recommendation
We recommend this be included, possibly with some
further refinement to the question.
Content summary
Central heating is probably quite rare in NZ and
doesn’t warrant its own response option, but open
fires may need its own response category, possibly
one for wood and one for coal. Some research on
where these can stil be used is needed.
We would like to stick with asking main method for
now. We need to know whether respondents can tel
us this information and see the level of multiple
response. A guide note has already been writ en to
help respondents answer this question.
The ‘most often’ wording is a way of operationalising
‘main’. The content team have suggested using the
word “main” in the question but this may be too
vague.
QMD
Recommend this question format for inclusion in
Volume Test with some tweaks
RI
Agree with inclusion
Next Steps
Minor refinements to question
Excluded
We recommended the fol owing be excluded from the VMO.
New content
Licence to occupy
Content recommendation
It is recommended that this be excluded from the
volume test due to the issues that have arisen with
people not understanding this term.
Content Summary
The 2013 approach was to only mention LTO in the
guidenotes, however we may want to consider if the
previous guidenote should be changed or kept the
same.
See the Individual Form, licence to occupy (tenure
holder) for further comments on this.
QMD
Not recommended for inclusion in the Volume Test
RI
Agree with exclusion; strongly recommend looking
into admin data instead, given that al respondents
tested (including those that had a license to occupy)
were confused by the question/response option
Next Steps
Continue to develop and test outside VMO
Includes testing 2013 Census
approach
Go back to external experts to discuss significant
issues and to explore administrative data sources
Housing quality - cold
Content recommendation
This is recommended for exclusion due to the
variation in how it may be answered by dif erent
people living in the same household.
Content Summary
In the consultation, this came out as lower priority for
col ection in the census as it is more dif icult to come
up with a good question that gets good quality data
on this due to its subjective nature.
Note – we have already considered whether
information on insulation could be col ected and
discounted this, as indicated in the variable
specification. Many people would not know the
answer to this question so it would not be possible to
col ect good quality data.
QMD
Recommended for inclusion in Volume Test, with
some reservations
RI
Agree with exclusion
Next Steps
Inform experts of decisions and monitor responses
Census Volume Test - content recommendations
Individual form
Inclusion
New content
Sexual orientation
Content recommendation
Sexual orientation is recommended for inclusion.
Content Summary
Sexual orientation data has been identified as an
ongoing unmet information need, due to the distinct
health needs of the lesbian and gay subpopulations,
need for an indicator to inform research as wel as
the need for national and subnational counts to
inform policy decisions.
The question format proposed has been
implemented by the ONS and in New Zealand
(MOH) in interviewer administered surveys. Thus far
it has performed adequately in QMD testing and has
recommended for inclusion in the Volume Test to
assess how this question wil perform in a ‘real world’
environment.
Unlike the gender identity question tested, this
variable wil not conflict with the existing sex and
number of children born alive questions.
QMD
Recommend inclusion of Sprint 5 question version for
Volume Test with help information available.
RI
Agree with inclusion; though more work needs to be
done around how the data wil be output and the
suitability of asking this question without any age
limitations
Next Steps
Continue paral el testing
Continue to work with CS &D on planning for
a new classification
Monitor international trends
Establish inclusive experts working group for
ongoing advice (internal and external)
Second address
Content recommendation
Recommended for inclusion
Request that this question be implemented towards
end of sprint timelines
Content Summary
Second residence was strongly rated as an
information need, particulaly for highlighting shared
care arrangements of children, and better
understanding the service populations of many
councils.
However, there are a number of issues highlighted
during testing. The reference of 30 days per year is
dif icult for some respondents. They also had
dif iculties in interpreting the response categories with
‘other’ being most popular. Parents with shared
arrangements found it dif icult to determine which
was their childs usual and second residence. There is
a concern the question may impact how people
answer usual residence.
While it is acknowledged there a number of dif iculties
to overcome, we believe it is important that this
variable be included in the Volume Test. It is a
valuable opportunity to get large scale quantitative
data, and to see if extra address questions place too
great a burden on respondents.
QMD
Not recommended for inclusion in the Volume Test.
Recommend further testing for possible later
inclusion (Census Test 2017).
RI
Disagree with inclusion
Next Step
Changes to the latest question are
recommended before the Volume Test. For
instance, an open text field for “other” in Q8
would al ow us to see these responses are.
Any other cognitive testing done before the
Volume Test can also feed into question
development. However, large scale changes
are not expected.
Further testing can be done in paral el with
the Volume Test.
Agree best option to tease out ef ects of one
address question on another
Investigate relationships between alternative
current, 1 and 5 yr address questions on 2nd
residence
Usual residence one year ago and usual residence five years ago
Content recommendation
Both Usual residence one year ago and usual
residence five years ago are recommended for
inclusion in the Volume Test.
Content Summary
In our preliminary view we suggested that both usual
residence one year ago and usual residnce five years
ago be included. However, if a trade-of needed to be
made, usual residence five years ago was considered
by some customers to be a lower priority. Address
five years ago ties in with the date of the previous
census and so enables al the inter-censal
components of population change to be estimated.
Due to its importance in helping with national and
sub-national population estimates there was strong
advocacy for usual residence one year ago to be
included in the 2018 Census. Population projections
are very important for infrastructure planning and
social service provision.
The content team has recently gone back to internal
stakeholders chal enging the need for both one and
five years ago.
Feedback was stil strongly in favour of having both
usual residence one and five years ago. The
combined data from the two questions give the best
possible insights into migration flows including return
and repeat migration. Some consideration has been
given to the IDI of ering alternative linking
opportunities for five years ago. However, feedback
from IDI is that there are problems relying on the IDI
for this information. In addition, the continued
inclusion of five years ago is considered useful in any
further linking work.
The addition of a question for usual residence one
year ago is relatively straightforward, although the
main issue may be respondent burden, especial y
with the introduction of second residence and
education address.
However, it is our view that the best way to find out
about the impact of this burden is to include al
addressing questions in the Volume Test.
QMD
Recommend inclusion of “address 1 year ago’ only in
the Volume Test.
RI
Wil need further investigation with Population
Statistics and other SMAs about prioritisation
Next Steps
Investigate relationship of this address question to
other address questions
Step families
Content recommendation
This is recommended for inclusion because it would
be useful to have a large scale test of this to see how
wel it is answered, including by Maori, and assess
data quality.
Content Summary
We need to investigate moving this question away
from the living arrangements question and trying a
new, simpler question.
We are checking whether a dif erent approach can
be used that would stil meet information
requirements
Inclusion in the Volume Test would al ow quantitative
analysis of responses such as cross-tabulation with
ethnicity which wil help us assess how dif erent
ethnic groups are answering this question. It is
expected to be a smal population group so inclusion
in a large scale test is important to al ow us to assess
overal data quality.
QMD
Not recommended for inclusion in the Volume Test.
Recommend further testing for possible later
inclusion (Census Test 2017).
RI
Disagree with inclusion – concerns around Māori
view of stepfamilies
Next Steps
Investigate alternative approaches to
col ecting this information
Progress concept work within a Māori world
view with internal Māori advisors
Dicuss testing progress and quality of data
issues with internal and external experts on
step families seek advice on further testing of
concept
Investigate potential impacts of inclusion of
this question on household and family data
Travel to education/address
Content recommendation
Recommend inclusion of travel to education and
usual travel means of travel to education.
Content Summary
Consultation demonstrated strong support for travel
to education.
Consistency with ‘travel to work’ means that ‘usual’ is
preferred to census day travel to education.
As indicated in the variable specification col ecting
information on usual travel wil provide bet er
information. This wil provide better information for
transport planners and this would also be consistent
with the proposed approach for travel to work. For
example - travel wil not be subject to the weather on
census day, data wil be more comparable over time,
more inclusive. In addition this approach helps to
resolve the issue of many people using a broader
window of time to complete forms than census day
itself.
The information for the address can be cut down to
name, campus (suburb), and city/town. That would be
suf icient to meet the information need. This has
implications for ICS development and for processing.
QMD
Recommend inclusion in Volume Test, with fol owing
changes:
Revert to travel on census day (which wil
require response option of “did not travel
to study on census day’).
Ask only for ‘name of educational institution’
and ‘city, town or campus’ rather than
exact street address of educational
institution.
RI
Agree with inclusion
Next Steps
Investigate sources of addressing data
Activities limitation (disability)
Content recommendation
Activities limitation is recommended for inclusion in
the Volume Test.
Content Summary
Output data on disability or functional limitations at a
subnational level has been an ongoing identified
need by customers.
The Washington Group questions are the
recommended approach to fulfil this need by SMAs
and have been run in the 2016 GSS question set.
QMD has identified a number of issues around
interpretation of the questions and categories for
respondents. Given the ‘fixed’ state of this question
set, modifications to the set are undesirable outside
of what has already been implemented (matrix style
on paper form).
Quantitative evidence wil help to assess whether the
issues identified by QMD impact the data of quality to
be output.
This wil provide stronger evidence to justify
inclusion/exclusion to customers.
QMD
Not recommended for inclusion in the Volume Test.
Conduct further, targeted, cognitive testing for
possible future inclusion (Census Test 2017).
RI
Disagree with inclusion
Next Steps
Go back to Washington Group experts and
seek advice on appropriateness of using
these questions in a self-complete matrix
format
Check with SNZ experts on comparability of
Volume Test data with GSS outputs
Go back to Disability reference group to
discuss testing progress and concerns raise
Assess quality (fit for purpose) of output data
from Volume Test with internal and external
experts
Changes to existing content
Sex
Content recommendation
Testing of a third category for the sex question is
recommended for the Volume Test.
Content Summary
The 2013 Census sex question presents a binary
concept of sex. It is compulsory for people to
complete census forms. Only those who consider
themselves to be ‘male’ or ‘female’ could honestly
answer this question. Alternative responses were
coded as ‘male’ or ‘female’ by Statistics NZ.
The existing sex standard has never had a ful
review. It al ows for a third category and wil
accommodate col ection of another category which
counts al those not included by the terms ‘male’ and
‘female’.
A human rights complaint has recently been lodged
against Statistics NZ.
Consultation demonstrated a clear need for baseline
information on the sup-population of people who are
niether ‘male’ or ‘female’ from health sectors and the
af ected groups. This potential new census
information was not considered high priority by other
key customers. If the information was available there
is a possibility that it would be more widely used.
Testing in the Volume Test wil provide larger scale
evidence about the feasibility of a third category and
its potential impacts. As sex is a Priority 1 variable
there is an identified risk to key counts. The best way
to understand the realities around this risk is to test a
third category. Classifications have pointed out that
this risk wil only kick in for census data itself rather
than Volume Test.
Classifications are interested in using an
‘other…please specify’ response in the Volume Test
to inform their understanding of how a future review
of the sex classification might be considered. This
evidence wil also inform thinking around statistical
standards across sectors.
It wil also help to assess public response to this
question format in a real world context. A third
category has tested satisfactorily in QMD testing thus
far and any evidence gathered from the Volume Test
wil help to inform any future decision for this variable.
Further work wil be required to understand
implementation impacts of a third category on 2018
Census edits, derivations and imputation.
QMD
Recommend inclusion of Sprint 5 question version for
Volume Test with help information available.
RI
Agree with inclusion
Next Steps
Clarity on the intent of the third category in
the classification would help further
development of this question.
Assess the interest in subpopulations of
interest within a third category and work
further on definitions and revise topic
specification
Learn from Australian census and other
international initiatives in this area
Travel to work
Content recommendation
Recommend continued inclusion but with change to
usual means of travel to work.
Content Summary
There was strong support for changing it to usual
means, as indicated in the variable spec.
Note, for people with multiple jobs, they should
indicate the means of travel used to get to their main
job, so that the information fits together with the
workplace address information, which wil be for their
main job.
There is customer interest in understanding more
about multi-mode travel behaviour in NZ. This has
been discounted for further testing at this stage.
QMD
Recommend inclusion in Volume Test, with smal
wording changes
RI
Agree with inclusion
Next Steps
Continue to test gather evidence on respective
viability of ‘usual’ and ‘main’ means of travel to work
Highest qualification change
Content recommendation
We recommend continued inclusion of highest
qualification. There wil be two minor questionnaire
changes to this variable.
Content Summary
The level of post-school qualification wil be col ected
in a tick box as opposed to a free text field. There wil
stil be a free text field to col ect subject of post-
school qualification. We also propose a tick box
question added to indicate whether the qualification
was gained overseas.
Changing the response options to tick boxes was
recognised as being an improvement to data quality
and helping to reduce the time and cost of
processing.
There has been a need identified to col ect
information on post-school qualifications gained
overseas. The reasons for this was to improve the
accuracy of estimates of the costs and benefits of
obtaining a qualification in NZ and to understand the
profile of those in the workforce with overseas
qualifications.
Respondents have coped wel with the question
changes in testing. The main issue is respondents
cannot easily recal the level of their qualification (but
is not due to questionnaire changes). No issues have
been observed with the overseas qualification
question.
QMD
Recommend inclusion in Volume Test, with one
potential tweak
RI
Agree with inclusion
Next Steps
Look at classification and consider potential to
col apse some response options
Voluntary work
Content recommendation
TBC
Inclusion of the 2013 Unpaid Acitivities format is
recommended for the Volume Test.
Content Summary
Voluntary work (with hours) was tested in order to
assess whether the census could better meet users
needs in this area. QMD testing indicated that
respondents were having dif iculty with the time
component of the question.
Given that a time component is not viewed as
feasible for a census style questionnaire, the 2013
format is recommended in order to retain data
comparability and the positive respondent
interactions with the previous format.
QMD
Recommended for inclusion in Volume Test, with
RI
Agree with inclusion of the 2013 Census version of
this question for Volume Test
Next Steps
Inform external experts of direction for the Volume
Test and monitor responses
Confirm direction for Volume Test with internal
experts re removal of time component
Excluded
We recommended the fol owing be excluded from the Volume Test. Please note that
exclusion in the Volume Test does not neccesarialy mean we recommending exclusion in
2018.
New content
Gender identity
Content recommendation
Gender Identity is not recommended for the Volume
Test.
Content Summary
A question on gender identity question was tested in
order to assess the ability of a question to meet the
needs of health providers for identifying transgender
and other gender diverse populations, as wel as
providing a baseline national count and and indicator
to inform researchers.
QMD testing has indicated that while a simple gender
identity question ‘works’ in terms of gathering
responses, any potential format wil not necessarily
identify the subpopulations this variable needs to
identify in order to meet the information need
identified. A sex and gender question on the same
form was also causing issues and confusion for
respondents. Therefore it is not recommended for
inclusion.
QMD
Recommend inclusion of Sprint 5 question version for
Volume Test with help information available.
RI
Agree with exclusion
Next Steps
Further testing outside the Volume Test to gather
evidence on alternative wording and approaches to
the topic cluster gender identity fits within.
License to occupy
Content recommendation
We are recommending for exclusion of the licence to
occupy category.
We recommend retaining a tenure holder question in
the Volume Test in both the DF and IF.
Content Summary
The re-design of this question is partly to address
issues with family trusts, not just for identifying
licence to occupy and it would be useful to see how
wel this change is working in a large scale test.
Dropping the licence to occupy category means the
quetion wording may need minor tweaking and we
may also need to think about guide notes for those
who know they have a licence to occupy. This would
help respondents know how to answer but wouldn’t
meet the information need for separate identification
of those with a licence to occupy.
The information col ected on the IF (Tenure holder) is
dif erent from that col ected on the dwel ing form,
which is tenure at the household level. It is
considered useful to have both measures. A dwel ing
may be owned by someone with a household, but not
everone within that household may own it. There
could be adutl children, flatmates, relatives etc who
do not themselves own the dwel ing. Asking it at the
individual level also al ows analysis with personal
characteristics such as age and ethnicity, which is
very useful.
QMD
Licence to occupy category not recommended for
inclusion in the Volume Test.
Recommend question revert to 2013 approach.
RI
Agree with exclusion; strongly recommend looking
into admin data instead, given that al respondents
tested (including those that had a license to occupy)
were confused by the question/response option
Next Steps
Given the issues that have arisen with licence to
occupy, more work is required on this. It is an
important information need, so we do not recommend
dropping it completely. It may be possible to try a
dif erent way of col ecting this information eg by
exploring whether we could derive information on
whether a dwel ing is in a retirement vil age and
estimate LTO fom that, or by asking directly whether
a dwel ing is in a retirement vil age. This could
possibly identify the vast majority of licence to occupy
situations.
