1982
Act
Information
Official
the
under
Released
Delib
Hamish Woodside
* Used GOES software tool.
1982
Act
Information
Official
the
under
Released
xx
ŽĐƵŵĞŶƚ dǁŽ
Interviews for GOES review – May 2017
National Library Positioning for the future
Jayne Beggs
1982
Tell me the background to your consultation
• May 2016 - let's do it mood
•
Act
2030 decided as the date
• Project - not pure policy - initially go to cabinet - then decided not to
• Barbara had been talking to Susan. Unaware GOES in pilot.
• Pressure to get consultation doc sorted to then into GOES.
How do you go about the process of consultation? [when to
engage/digital/offline]
• Initially full consultation/ lots of promotion/involvement. Library staff paranoid about
Information
what library world thought of them.
• Risk adverse.
• Had taken awhile to get going
• In the end - some promotion (fb/twitter).
Official
• Senior leaders had to be sold on it.
• Submission had to be copy and pasted.
•
the
Worry about the questions - what do we want to find out.
• Less control of submissions (publishing).
• If doing quantitative - use GOES. If qualitative - need backend tool - GOES doesn’t
give you that.
•
under
Needs more guidance about what it does suit.
• (Half via GOES - half via email).
• Only feedback about GOES was negative (maybe older, less internet savvy).
What’s your experience of running a consultation?
• Janyne had been involved in gambling related consultations.
Released
What is the most important factor for you when choosing how to
consult/what tool to use? Eg functionality (what the product does), support,
price?
• No structure training in-house for doing consultations.
Page 1 of 14
• Need to be aware of traditional methods (eg cabinet sign-off). Big issues need
sign off.
• Would normally pay nothing (put it up on website) - so would need to be pretty
cheap.
At what stage did you decide to use GOES?
• Already decided by Babara and Susan before Jayne got on board. Get to be pilot
for free. Not well communicated on the leadership team (GOES plain - linear).
1982
• Not much time to get sorted. Deadline close. Too much for 1 person (Susan). Felt
there was pushback on functionality requests.
Act
Where do you think the value lies in having consultations on Govt.nz?
• Still had something on National Library websites - then pointing to it.
• (Problems with search - couldn’t find it in a meeting - Bill showing to senior
people)
Do you see benefits in the consultation listing? Who for?
• Yes - want people informed about consultations.
Information
• . How did GOES compare to tools that you’ve used in the past? [easy to use,
faster etc]
• Haven’t used a tool. Document on website.
• Analysis in excel spreadsheet for NL consult. Hindsight - would have used more
yes/no questions.
Official
• Tags - useless. Too high level for value (analysis).
the
Was there a difference in the quality of submissions?
• Main benefit was structuring questions better.
What was its data collection and analysis like? Did it affect the decision
under
making?
• Gave rigour/data viz around the structured questions.
• (extreme influence from an individual - fait accompli for outcome)
How transparent was the decision making?
• Leadership team could read submissions publicly.
Released
• Use transparent. Decision making not transparent.
Are there areas for improvement?
• Clear upfront what it can do for you (not qualitative). Quite rigid - wanted more
flexibility in answering questions (multi-select).
Page 2 of 14
• Focus on language - try to plain English - but internal people wedded to certain
phrases. Problem because no focus on customer.
• Would like to have more control over how it was written. Want transparency
through the whole process. Need to intervene early.
• Te reo translation - useful. No one in submitted in Te reo. Not much feedback on
it (except for senior management).
• Keep languages together (needs to be much shorter).
• Inaccessible submission docs on Govt.nz - NL wouldn’t let them be on the site.
1982
• Multi-lingual good way to go - but need to see where the impact is.
• Archiving needs to be done.
Act
Where does the value lie for you?
• Quantitative
• Having it in place (Govt.nz good)
What are the differences in response and feedback from the public when
using GOES compared to what you have used in the past [online comparison
only]? [quality of submission]
Information
• Only feedback not great. Restrictive, plus word limit found frustrating.
• In hindsight - would have been useful getting feedback on the process (especially
because it was so qualitative).
Official
From your past experience, what problems do people face when giving their
feedback to government?
•
the
Knowledge that it’s even going on. Some legislation puts ads in paper. Email lists
for notification - but may be other people. Not everyone’s online - how to
incorporate those feedback.
Have you got any feedback or ideas on how government can consult better?
under
[part consultation plays in better decision making/policy etc, co-design,
micro-consultations etc]
• Having regular dialogue with people better than 1 off ‘big bang’ approach. Library
got slammed
• Don’t appreciate how much influence govt can have. Get people involved, but
don’t get back to them, keep them in the loop.
Released
SSC and the Policy Project
Leonie Parminter, Lis Cowley, Jo Hartigan, Sally Washington, Amy White.
• Sally - how we do consultations website
Page 3 of 14
• Jo - guidance - overcooked.
• How does GOES facilitate something better. Digital version of paper process. Asking
people a list of questions not the best way. Designing for the past.
• Need different ways to play.
• Experience
• Ideas
• Decision making Insights bank - use your system - put back. “If GOES could think like a 1982
social researcher”
• Too focussed on tool - less focussed on what we should be doing.
• DIA - take care of the the tools - ‘toolkit’ idea.
Act
• Impact - multiple logins with different tools . Will become problematic - tool
proliferation.
• Build suite (eg Nesta digital democracy infographic)
• GOES needs to be push. Send to mob (MindHive)
• Leonie - OGP - used Delib & GOES. Being able to see peoples comments & responses
to peoples comments - richness
• DIA - think futures - what principles/functionality. Information
• Innovation by procurement.
• Make it easier for agencies to engage.
• About engagement (wider than online). No stories without data.
• Social science very important.
• Acts like a social scientist. How can we overlay information from other sources.
Official
• Leonie - upskilling of agencies.Can’t upskill in the heat of consultation.
• Intelligent customers. the
• What’s quality. Start seeing good examples.
• We are the front door to what the market has, ‘broker’ of tools Gold standard
• Leonie - NAP up online before Delib. Support really valuable - moderation document -
value of govt context. GOES - value - takes away the risk for govt. IMpact/trust -
under
speaking to the risks that are there for govt departs.
• Legitimacy in authorising environment.
• Reach - did you get to audiences that you wouldn’t have otherwise?
• What does public engagement for public servants look like?
• Jo - working on methods for policy.
• Market is already providing what people need.
Released
• Panel of providers (cloud questions, body of knowledge)
DIA – BDM Fees
Jo Arnold, Anna Chapman
Page 4 of 14
• Normally a policy piece. 1st time for ops to do.
• Hard when don’t have policy background - go through policy, finance, sign off
• BDM access review - done by policy the old way. Doc on website, PDF to fill in. (didn’t
use it for access review because GOES didn’t have what they needed).
• Main stakeholders advised.
• Good to have it on Govt.nz
• New to the processes - good to have Govt.nz guide us through.
1982
• Consultation listing - good to have it all in in place
• Compare with access review. Lots of work to compile/analyse.
•
Act
Data analysis. Good because can get excel spreadsheet and do pivot graphs. (easy
with 37 responses). Liked the tagging - analysing as it comes in.
• Would have been nice to have more people respond. Supported preferred option.
• Decision making very transparent - published everything.
• Areas for improvement. No cheat sheet - struggled with how to use it.
• Look at savings for government.
Inland Revenue – Information sharing between IR and MSD
Information
Fernanda Borba-Nunes
• Proposed legislative changes - go for consultation. Preparing discussion document,
releasing to the public.
• Database of ppl who subscribe to tax policy - they receive alerts.
Official
• Ministers do a press release.
• Have to go wider with wider interest topics - eg student allowances.
the
• Target people specifically, but get feedback from ‘usual suspects’ eg Privacy
commissioner.
• Price very important - didn’t want spend much.
• Less than 10 submissions (only 3-4 individuals).
under
• Decided to use GOES from the beginning.
• Govt.nz - good sitting it on it as gives a view of all of govt.
• Using GOES gives it a bigger audience - more exposure than having it on tax policy
website. Able to reply anonymously - if they went through usual channel have to say
who they are.
• Faster to get individual people's responses (big orgs just use email).
•
Released
No difference in quality of submission. Maybe if it was a forum - would be a different
result.
• Numbers too low to use data analysis tagging.
• Analytics useful - number large - but submitters low.
Page 5 of 14
• Improvement - more autonomy in terms of functionality. Had to ask Susan to do
things, wasn’t allowed to do things themselves. Need a guide/help. Not having it
meant that had to ask a lot of questions.
• Easy to do if you’re not an expert. Got a lot of support from Susan.
• Making tax easier - being reused.
• Wouldn’t pay much for small consultations - up to $5k.
• Future state - provide easier access. Like idea of micro consultations, ongoing
consultations.
1982
Archives 2057 Discussion Document
Act
Kate Hodgetts
• 2016 - wanted overarching long term strategy for Archives. Recognised need for
stakeholder engagement. Found out because of National Library using GOES.
• At first thought just targeted stakeholder consultations, but then decided to go
broader - go public.
• Discussion document had very open ended questions. Invited qualitative information.
• Also did a roadshow - but pointed people towards online.
Information
• If didn’t use GOES, wouldn’t have looked for tools. Would have put a doc on the
website.
• Govt.nz - valuable from a citizen perspective (vs being on Archives). Might get a
broader reach. Think this was the case.
• Consultations listing - high value - ppl can see what’s
Official
• Have used survey monkey in the past. Value of GOES - have the document on the
website. Meets web standards. Couldn’t use survey monkey for that (+ open ended)
the
• Total - 120 online submission - 25 emailed submissions. No difference in quality.
• Had someone do the analysis. Found the back-end difficult to use. Difficult to pre-
empt what tags would make sense. Hard to change and update tags. Easier to export
to excel.
•
under
Technical limitation. Creating a PDF of the submissions laborious. Publishing big
submissions.
• Barbara Reed - NSW - independent reviewed the document based on submissions.
• Hard to search for submissions.
• Te reo translation time pressured. Not many submissions from Maori/iwi - not sure
online best way to engage
• Analytics - showed that people read Te reo.
Released
• Moderation straight forward.
• Cost recovery? Up to $2k
• Susan had to do a bit of support. Converting word document to HTML - make sure
okay. Carin Wilton did some editing. Web editing not a skill Archives had.
• Value - ease for submitters, transparency around submissions.
Page 6 of 14
• 1 person struggled to use it - hidden save button.
• Getting people over the line to commit. Incentives to get people to do things
• Huge interest in reinstating quarterly meetings - would like to think of online way.
Police – Information sharing about gangs
9(2)(a)
•
1982
Gang Intelligence Centre - separate unit.
• Govt plan reduce harm gangs cause. Enforcement and social programme.
• Consultation - only share a certain amount of info.
Act
• Whole point is to be collaborative - makes it slower.
• Online perceived as easier. DIA/IR behind the mechanics of how to run it.
• Get concepts of running it openly.
• Didn't have written criteria of how to choose it.
• Choose it because Susan etc did most of the work.
• Appreciate support/brokerage. 'a few thousand'. Easier because multiple agencies
(unusual for Police to do).
Information
• Value of having it on Govt.nz - value of the multi-agency - not branded as a specific
agency - (plus established site).
• Consultations listing - a bit unhelpful having local and central.
• Did have email as well. Makes it harder for analysis.
• 29 submissions (24 through GOES). Written ones organisations (law commission,
Official
sensible sentencing trust). Online - more structured.
• Decision making - themes coming through about what could be done better. Made
the
amendments as a result of the consultation.
• Areas for Improvement. Data analysis - didn't like output.
• Combining online and offline.
• Tagging - only worked because of low number of submissions. Also problem of
under
needing to know what themes people would submit on before the submissions came
in.
• Problems of reach - getting people interested in the niche topic.
• Challenge - taking a dry document and turn into something potentially more
interesting! Who's going to read that!
• Level of investment by Product Owner a selling point - marketing, advice on the
document. Wouldn't have been able to do it if just said 'go use this tool'
Released
DIA – Dangerous dogs
Fergus Broom
Page 7 of 14
• Focussed heavily on F2F engagement with interested parties - council, dog control,
plastic surgeons.
• Had consultation on National Strategy on the dog reduction strategy. Public survey
lead by Minister.
• Collating info from stakeholders.
• Survey of public.
• Criteria - don’t know what other options were considered.
•
1982
Greater reach - could do better at MoJ.
• Govt.nz - better because Ministerial led. Better website than DIA. Think that made a
difference in reach.
Act
• Consultations listing - niche. People know about it or don’t.
• Social media valuable in getting people involved.
• GOES - easy to use. Some problems with questions set up.
• Structured and online makes it easier and improves reach.
Data analysis
• Probs - duplicate entries. Maybe hit send multiple times?
Information
• 1 in 10 were duplicates. (300)
• Entire system broke because some put an = sign in. Delayed analysis.
• Tags didn’t show up in excel - so time wasted.
• NVivo - should be used for analysis.
• Doesn’t favour manual analysis
Official
• Be prepared for larger number.
the
• Decision making - transparent - Minister referred to it a lot - into decision making.
Improvement?
• Development of questions - way you ask questions determining outcome.
under
• Had a system of coding free text responses.
• Small surveys - use survey monkey.
• Depend on the scale of surveys. Pay per use?
• If department paid for it - would need more help/guidance - end to end. Engagement
on how to structure questions, use the tool.
• Users - too technical - barrier to engagement. Impact of engagement - keeping
Released
people in the loop. Visibility - what’s going in govt.
Future state
• Engagement not being a specialised thing.
• Use social media to engage better.
Page 8 of 14
• Getter a greater reach with people.
Agencies interviewed who didn’t use
GOES
1982
Transparency International
Suzanne Snively, Janine McGruddy
Act
• How do we make sure that we’re not missing something. 2 way communications (not
just 1 way broadcast).
• OGP commitments - Al Morrison wanted a commitment around working with iwi
leaders so core public sector officials could manage relationships more effectively.
Addressing the core cultural structure of Maori - hapu. Controlling factors.
Whanau>Hapu>Iwi. 2200 hapu. 84 territorial authorities, community services people.
• Proposal to SSC. Start by breaking populations down.
Information
• Formative consultation.
• Value from any consultation -
transparency + impact.
• Govt putting gagging clauses/competitiveness.
• Effort involved feeding into consultations.
• Taken people’s voice away.
Official
Future of consultations / engagement
the
• People feel empowered by consultation process that govt will hear their needs.
Decision making.
• Public trust.
• Cultural. under
Ministry of Justice
Suzanne Jones (previously Electricity Authority)
• EA - does 30-40 consultations a year. Specialist audience generally, but are done
publically.
Released
• EA contributes to listing service, but no analytics evidence that people ever came
from it to the consultations on the EA site.
• Where does the consultation sits?
Page 9 of 14
• Split across websites (Govt.nz & agency website - need context on agency site and
pointing to Govt.nz) Where is the audience going?
• Value - speed to publish consultation. Up against the wire.
• Price - doesn’t come into it as just put a document on the website. Nothing from
GOES that’s worth paying for yet.
• Would pay for:
• Content analytics (data smarts)
•
1982
Deliberative techniques
• Data visualisation
• Also - all of govt procurement for tools
Act
• Future - collaborative perspective for doing an engagement (x-agency) - eg with
Maori.
• Forward schedule - what consultations different agencies are proposing to do.
Delib
Hamish Woodside
Information
• Devolve consultations to policy teams
• Anyone ‘information as a point of response’
• What questions to ask?
Official
Bottlenecks:-
• Education around consultation best practices.
the
• Procurement - security assessments
• MoE - $25k for 1 consultation.
• MoJ - 22 consultations a year - getting a big data set - what should we do with it? API
• 46 separate survey monkey accounts in 1 agency
under
• API - consultations aggregator
• We can teach people standards - but can’t teach people to write - should have team
of content people.
Ministry for Women
Gill Palmer
Released
• Different type of consultations.
• Any paper got to SoC have a gender view on it.
• Policy/operations work. Leadership - working it as part of the policy process.
Page 10 of 14
• Where relationships important - Maori/Pacific trades training - building up those
communities (ongoing conversation)
• Outward facing - govt/NGOs etc
• Women in leadership - collaboration and sourcing data -
• Decision making / reach / inclusiveness - better outcomes.
• Family whanau violence bill. Not user testing.
• Ministerials that cover a few sectors - talk to other organisations.
1982
• Gill ran project 2013 - women’s employment dived after Canterbury earthquake -
emphasis on construction. Women underutilised. Workshopped with people in
Canterbury. Report online 'getting it done'.
Act
• Collaboration - prompt action and support it.
• Operating model working well. Develop - then share and influence. Sometimes longer
term, sometimes short term. "Influencing others" developing, presenting and
engaging on others with your evidence. Digital harm project - focus groups
(outsource research, rapid lit review, contract out).
• How we work as opposed to what we do.
• Visibility - making sure work shared - getting images of real women.
Information
Ministry of Health
Anna Pethig
•
Official
Consultations - mostly policy teams. Policy document - plus
• Policy cycle - tricky part. Expectation with what happens with submissions. Whether
they're going to be release, what will be redacted.
the
• No help with analysis at the moment.
• Help sort and analyse -
• Govt doesn't have a role in developing a tool.
• Ideal consultations scenarios eg there as an expectation that all consultations on
under
behalf of the govt will have their submissions publically released. Creating
expectations of what transparent consultations should do.
• Guidance and recommendations about tools. What branding should be applied -
ideally consistent experience.
• Cognito forms - can stop and start - can save, forward onto someone else for
approval. Good for collaborative creation of submissions (eg stakeholders)
•
Released
Someone in the agency responsible for consultations. Accountability. See across the
whole process.
• Useful to understand advice about formats - exclude people who didn't use digital.
What are you going to miss out on?
• Simplified version of policy documents. Top 3 things - make it easier for people.
(People resort to 'postcards')
Page 11 of 14
• Whether we provide a summary of submissions - not done consistently. Whether
appropriate to pull names of individual submitters by default.
• Whether they have the chance to speak to the document.
• Developed 'workshop' for people to run their own workshops - eg healthy ageing
policy.
• Discussion forum - health strategy. discuss.health.govt.nz Catalyst platform (IRD use
it) - $3k to stand it up - more reusable. NOt that successful because need to promote
it.
1982
• Need to think about who you want to answer, make it easy. Barrier of policy speak.
• Not impressed by MoJ's implementation of Delib. Loomio not good because want to
Act
moderate comments.
• Standard - what should constitute a submission eg conversations on social media.
• Feedback from people – they find Word documents difficult - pre-set up Word form.
• Presentation of consultations - linking between document/analysis of
submissions/submissions themselves - guidance + outcomes.
Te Puni Kokiri
Nancy Tuaine
Information
• Who engages with iwi -(agencies) where's the record of that?
• Tension between co-design & govt process.
• TPK undertaken strong process Maori Land Court - 3 round of engagement included,
1st brainstorm etc.
Official
• Government engage with iwi on particular workstreams eg fresh water. iwi group
meet with Ministers, whanau ora, justice, family violence. Key subjects that govt has
the
direct partnership with iwi on.
• Govt has minimised approach to key centres - have to travel to Wellington.
• Submission process - filling out someone elses form - Have the right to submit to govt
on the manner that suits you. Health Disability Commissioner.
•
under
Big subject matter that makes a difference - get a templated answer - won't get gold,
but will get impact.
• Technology impacted by access. ON the west coast can't get data signals. Restrictions
by money to go satellite - or the internet. Access to broadband and the price.
• Terrible system like RealMe - validation process - 2 step.
Ministry of Primary Industries
Released
Sid Pickering, 9(2)(a)
, Stacey Moir
• No one way doing consultation at MPI
• 'Tricky' stakeholders - do pre-engagment/consultations. Reach out through industry
bodies.
Page 12 of 14
• Becomes trickier with recreational fisher people - takes more work. They're both
politically engaged.
• Doing something next month in the Hawkes Bay. See if they can get 3rd party
endorsers - trying to be representative and also be seen to be representative.
• Sustainability rounds - run through media, meetings, commercial fishers - info on
website.
• Things could be made easier for submitting.
• Use digital channels - direct advertising with Facebook.
1982
• Make info easy to understand important.
• Could be a lot of different ways that could be
Act
• Concerned people can't understand what they're consulting on.
• Sometime put a summary document - a lot of work - do it when
• Consistency needed internally - strategic thought - what are trigger points
• Website - most common point of interactions.
• Overlap in consultations / with other departments.
• Standards need to serve a purpose.
• Policy documents difficult.
Information
• Have a voice - opt for voice recording system.
• Digital postcard - no going to give it more weight - thousands of responses.
• Flexible enough -
• Suite of tools to use + skills needed. Different levels of engagement with business.
Official
• Don't have the resources to re-write documents.
• Need to be practically useful.
• Different tools - show different ways of doing things.
the
• Exposing business rules
• Asking the right question up front. How do we get this done in time?
• Having tools might shift that mindset.
under
• Other agencies doing it.
• Power of the outcome. Currently a faith based argument - would be good to have
case studies to prove it.
• Examples highlight - material impact on the policy.
• 'Over consultation' a problem
• Kaikoura - lots of govt agencies involved. Lots of people talking to people. Could have
talked to people about their priorities.
Released
Wellington City Council
Matt Lane.
• Last 5 years - consultation platform 'Consult 24' SAS. Bespoke for NZ local govt.
Page 13 of 14
• For formal consultations. Good job of managing consultations (not wider
engagement)
• Administered by democratic service team.
• 9(2)(a)
- not adequate for 21st century. Replaces front end not back.
• 10 year plan - used survey tool. Typeform.
• Want to focus on the business process.
• Did increase participation, provide.
1982
• Not repeatable, scaleable, economic.
• Training/advice/guidance.
•
Act
WCC - template - people need to complete before consult - not sure if they do.
• Need to make it simple - not big document. Social media - bringing it all in.
• Mature engagement - ongoing conversation.
• Wellington Town Belt act - every time reading in parliament.
• Other tools end up as spreadsheet - cf CRM
• Much easier way to have a privacy leak
• DCC - built own system from scratch - specifically designed to manage annual plan
process.
Information
Official
the
under
Released
Page 14 of 14
