Communication with SLT on 6/4/2016
Context: The VMO is a key opportunity to gather evidence in a test driven census environment. It is
not the only opportunity. Paral el testing wil be progressed on a smal number of identified topics
to feed into content considerations for Census Test 2. What goes into the VMO wil look different to
what goes into Census Test 2.
Note: Dwel ing F = DF, Individual Form = IF
Content considerations on variables for volume test fol owing QMD testing feedback
High level overview of content recommendations:
Removal of Gender Identity questions on IF
Removal of tenure question on DF and tick box in Tenure Holder question on IF
Removal of Cold question on DF
Inclusion of al other tested content with some refinement for some questions
Sex (third response option) – INCLUDE in volume test with ‘indeterminate’ to test public response
1. Testing feedback is that third category ‘intersex’ works fine but ‘indeterminate’ was also working
for self-completed form
2. QMD recommends intersex as third option for volume test with help information available
3. Working group (primarily classifications) flagged that using ‘intersex’ would necessitate a major
review of the standard, current testing indicates that either term would work (and meet human
rights issue) even if intersex respondents preferred ‘intersex’
VVL: Agree with inclusion
Though there’s stil a lot of work to do in terms of how to adapt the imputation process and
derivations, including how to assess the quality of the derivations
Gender Identity – OUT of VMO, further work outside of census TBD
1. Question was working for respondents to complete but non-lgbtqi respondents found it
confusing at times
2. QMD recommends inclusion in VMO with help information
3. Evidence from testing suggests it is not possible to distinctly identify trans-gender subpopulation
in a self-completed form given the complexity of their identity (often respond with male/female)
VVL: Agree with exclusion
This question is not designed to identify transgender respondents so would not be able to
meet that need
This has caused quite a bit of noise with other variables such as Sex and Number Born
(Fertility)
Sexual Orientation – INCLUDE in volume test
1. QMD recommends inclusion in volume test with help information available
2. Volume test inclusion wil enable assessment of quantifiable data
VVL: Agree with inclusion; though more work needs to be done around how the data wil be output
and the suitability of asking this question without any age limitations
Second Address – INCLUDE with further refinement for VMO
1. QMD recommends not including with further testing for possible inclusion in dress rehearsal
2. Need to include guide-notes and test later in form further from usual address
3. Working group has recommended inclusion of other tickbox for VMO to al ow assessment of
other types of second dwel ings to better understand other occurences
VVL: DISAGREE with inclusion
This has caused noise/confusion with usual residence (which is critical to the census) and for
family/household data (just a reminder that we have half the time for data evaluation this
time around; in 2013 we needed al the time al otted for data evaluation due to the
complexities around family/household data).
This has included previous residences (if the respondent has lived less than a year at their
usual residence) and instances where an adult lives with a parent part of the time (which
does not meet users’ needs for shared care/custody), and quite a few responses under
‘other’
The noise/confusion/examples above means that second residence is unlikely to help with
NRFU – we are capturing addresses that do not provide occupancy status, may not be
relevant for the timeframe of the census, and may need further investigation beyond what
we have planned for Field/Geography.
Address 1 yr / 5 yrs – INCLUDE ONE YEAR AND FIVE YEARS
1. QMD recommends inclusion of 1 year only
2. Need to ful y assess burden in real world environment (along with online form design in
addressing space)
VVL: Wil need further discussions with PopStats/other SMAs about prioritization
Step Families – INCLUDE in VMO with further refinement of question
1. large scale testing wil enable analysis of Māori interpretation and prevalence
2. QMD recommends further testing outside of VMO
3. content meeting – suggested simplification of question (QMD raised issue of identifying nature
of step relationships complex)
() have no step family
() step family I live with at usual address
() Step family I do not live with at usual address
- issues with living arrangements question
- issues with Māori interpretation raised
VVL: DISAGREE with inclusion
Large scale testing wil not provide insight into Māori interpretation and understanding,
especial y since we have no way to fol ow-up with respondents as to how they understood
the question/response options; this is more appropriate for targeted discussions/cogtesting
with Māori respondents
This has created noise/confusion with the living arrangements question, which is critical for
family coding
The proposed simplified response options above has not been cogtested; based on previous
tests with respondents, it may stil confuse people because of the use of ‘usual address’
The recommended reworded response options also do not address what respondents had
raised, i.e., who counts as ‘step-family’ – which may indicate that the question may not meet
users’ needs
Travel to education/address – INCLUDE in VMO with refinement (with usual means) – simplify
educational institution address
1. QMD recommends inclusion with travel on census day, asking only for ‘name of educational
institution’ and ‘city, town or campus’ rather than exact street address of educational institution
2. Change in concept is important for users
VVL: Agree with inclusion; but we wil likely need more details around the institution (e.g., add
suburb), particularly for educational institutions that have multiple campuses in one city.
Travel to work – INCLUDE (with usual means)
1. QMD recommends inclusion but with census day travel as conceptual y easier for respondents
VVL: Agree with inclusion
Tenure-holder: License to occupy – OUT of VMO, more work required prior to Census Test 2
content decision considerations
1. QMD recommends out of VMO, revert to 2013 approach
2. Include tenure holder question on IF
VVL: Agree with exclusion; strongly recommend looking into admin data instead, given that al
respondents tested (including those that had a license to occupy) were confused by the
question/response option
Highest qualification change – INCLUDE with smal change
1. QMD recommends inclusion with col apsing categories if possible
VVL: Agree with inclusion
Smoking – INCLUDE
1. QMD suggests removal as cyclic topic to make space
2. Strong user need in this area for this data at this point in time
VVL: Agree with inclusion, with reservations (as this is a cyclical topic)
Babies – INCLUDE
1. QMD suggests removal as cyclic topic to make space/remove complications with new gender/sex
questions
2. Issues with gender question wil be minimised by removal of gender identity
VVL: Agree with inclusion
Voluntary work/Unpaid Activities – INCLUDE with refinement (likely revert to old unpaid activities
format)
1. If data cannot be col ected on hours, additional value of new format wil be minimal. Old format
wil be better for respondents and retain data comparability
2. QMD recommends inclusion but mentions concept is less inclusive of old unpaid activities
response options and therefore less respondents indicating in
3. Respondents have difficulty with hours over past 4 weeks
4. Some difficulty with volunteering for other households category conceptual y
VVL: Agree with inclusion of the 2013 version of the question/response options
Activities Limitation / Disability – INCLUDE
1. Need for quantifiable data to analyse for this variable to properly assess quality/comparability
with interview administered versions of the Washington Group questions
2. QMD advises more testing/refinement before possible later inclusion in Dress Rehearsal
3. Matrix format recommended for paper form -> question by question format online
4. Uncertainty around comparability of either question set with international measures (self
complete vs interviewer)
5. QMD advises there are difficulties with interpretation re: assistive devices and some/a lot of
difficulty (which would impact on main indicator of limitations output), also difficulty assessing
answers within context of a health problem
VVL: DISAGREE with inclusion
Clear respondent burden, which, given the priority level, would need justification
May not meet users’ needs – as QMD has indicated, responses may not be due to
health/physical limitation. So while respondents may be able to provide responses, I would
caution interpreting the data as indicative of health/physical limitations when they may not
be related to that at al
Number of rooms – INCLUDE with refinement
QMD recommends inclusion with minor tweaks
VVL: Agree with inclusion
Heating appliances – INCLUDE with refinement
QMD recommends inclusion with minor tweaks
VVL: Agree with inclusion
Housing quality (below):
Content is getting a feedback statement from key users on importance of this data and usability of
data points (ie. a4 paper size mould)
Amenities – INCLUDE with possible refinement (need more data responses to assess how this is
working)
1. QMD recommends further testing outside of VMO
2. targeted and large scale testing wil assist in this area
3. possible inversion of question format to reduce burden
VVL: DISAGREE with inclusion
Respondent burden as most people tested have al amenities
QMD has also indicated that respondents who may not have some of the amenities may be
hesitant to report it
Targeted testing would potential y provide more information – this cannot be done in the
volume test
Cold – OUT of VMO (too subjective relative to other data points on housing quality)
QMD recommends inclusion with reservations around subjectivity
VVL: Agree with exclusion
Damp – INCLUDE
QMD recommends inclusion with reservations
VVL: DISAGREE with inclusion; this is just as subjective as the cold question
Mould – INCLUDE
QMD recommends inclusion with reservations
VVL: DISAGREE with inclusion
While we have relied on cogtesting to test if people understand and can reply, we have no
way of testing if how they responded is accurate – that is, does their dwel ing actual y have
that amount of mould? This is why al the previous research has relied not just on
respondent’s answers but used a trained assessor as wel .
There is no health implication for mould that is larger than an A4 sheet of paper. We are
creating an expectation among respondents that this has health implications, and we could
potential y be causing them stress/concern. QMD’s research has found that it’s the amount
and type of mould that has health implications, which we cannot assess in a self-response
survey like the census.
