A1004040
Tertiary Education Commission
Audit Report
under the
National Council of YMCAs of New Zealand
Incorporated
Youth Guarantee, ntensive Literacy and Numeracy and Workplace
Literacy Funding
Released
Official Information Act 1982
Edumis Number: 9388
Audit Dates: 18 – 22 July 2016
Draft Report Release Date: 10 August 2016
Final Report Release Date: 19 August 2016
1 Purpose
The purpose of the audit was to provide assurance that the National Council of
YMCAs of New Zealand Incorporated (YMCA) is meeting the Investment Plan
Funding Conditions as referred to in the Tertiary Education Commission (TEC)
letters of approval dated 17 November and 21 December 2015. The funding
approved includes Youth Guarantee, Intensive Literacy and Numeracy and
Workplace Literacy funding.
2 Scope
The scope of the audit was aligned to the performance commitments in the TEC
Investment Plan and the associated funding obligations between the TEC and
YMCA. The scope was outlined in the audit arrangements letter.
The focus areas included:
The reporting through the Single Data Return, including recording enrolments
and withdrawals.
The financial support for the equipment infrastructure required to deliver the
approved qualifications.
Whether inducements or benefits have been provided to students.
Responsibility for subcontracting arrangements.
under the
Compliance with Youth Guarantee funding conditions.
Compliance with Intensive Literacy and Numeracy funding conditions.
Compliance with Workplace Literacy funding conditions.
Compliance with the requirements in Part 18 of the Education Act 1989.
Monitoring the action plan for the TEC 2014 audit.
Any other TEC funding matters
An audit is a snapshot of an organisation’s performance at a particular point in
time and may not, as such, provide a view of ongoing compliance. An audit is
based on sampling and issues may remain undetected. The outcome of this audit
wil contribute to decisions made by the TEC relating to current and future
funding.
Released
3 Background
This audit is part of the TEC’s ongoing monitoring of Tertiary Education
Organisations.
YMCA is a Private Training Establishment with its head office in Lower Hutt and
centres throughout New Zealand. The YMCA Centres visited during the audit
Official Information Act 1982
were South and Mid Canterbury (Timaru), Invercargil , Nelson, New Plymouth
and Porirua.
The organisation receives Youth Guarantee funding for the following:
NCEAL1 National Certificate in Educational Achievement (Level 1)
NCEAP2 Manufacturing and Technology
NCEAP3 Primary Industries
2
4 Key Findings
The key findings were:
Youth Guarantee funding
Youth Guarantee students studying in Christchurch had been reported against
the Main Campus in Lower Hutt in the December 2015 and April 2016 Single
Data Returns. This site is the head office and no students study at this location.
Approved Course Sites
A review of approved course sites registered with the TEC through the Services
for Tertiary Education Organisations (STEO) website when compared with sites
reported in the December 2015 and April 2016 Single Data Returns found
several discrepancies that impact on the accuracy of data reported. This is
summarised in Appendix 1 – Approved Course Sites in STEO with eight sites
reporting SDR data.
Quality Management System
YMCA has a Quality Management System; however in the student and learner
records sampled version controls on documents were not in place and therefore
no system for monitoring the consistency and quality of enrolment decisions
made between YMCA Centres. Two examples of this inconsistency are:
a. YMCA South and Mid Canterbury provide detailed offer of placement
letters to prospective students and retain a copy for audit trails
under the
whereas YMCA Invercargil has a short template that is overwritten
and no copy is retained for audit trai s.
b. Students enrolled at YMCA Sou h and Mid Canterbury sign the
enrolment form but it is not counter signed by the PTE whereas at
YMCA Invercargil both pa ties sign the form.
Intensive Literacy and Numeracy funding
YMCA has commenced an internal audit of Hawke’s Bay Intensive Literacy and
Numeracy records reported for 2015 following recordkeeping issues identified
in the ecords pre-selec ed for this audit.
Audit trails did not always detail how learners met the entry criteria in Intensive
Released
Literacy and Numeracy programmes specified in funding condition ILN004 –
TEO to ensure lea ners meet criteria.
For YMCA Central Intensive Literacy and Numeracy learner, NSN 9(2)(a)
,
and Gisborne District learner, NSN 9(2)(a)
, there was no evidence
confirming domestic status.
Workplace Literacy funding
Audit trails did not always detail how learners met the entry criteria in
Official Information Act 1982
Workplace Literacy programmes specified in funding condition WLN004 – TEO
to ensure learners meet criteria.
YMCA Invercargil check employer records to ensure learners were in paid
employment in Workplace Literacy programmes but this evidence was not
included in the decision-making process recorded in the enrolment checklist.
4
YMCA Invercargil Workplace Literacy learner, NSN 9(2)(a)
, indicated on
their enrolment form that they held a diploma of 9(2)(a) but there was no audit
trail detailing how the learner was eligible.
5 Recommendations
The main recommendations are:
Youth Guarantee funding
Report Youth Guarantee students in the Single Data Return accurately against
the relevant approved course delivery site.
Approved Course Sites Review the Course Delivery Site field1 and update STEO approved course sites
(Appendix 1).
Source site approval letters from NZQA for all approved sites.
Quality Management System
Implement version controls on student enrolment and assessment documents
used to ensure consistency across all YMCA Centres.
Develop an administrative section in the Quality Management System covering
TEC funding conditions and Single Data Return reporting requirements.
under the
Ensure all enrolment forms are counter-signed by both the student and the
PTE.
Intensive Literacy and Numeracy funding
YMCA to provide a report to the TEC by 1 October 2016 on the results of the
internal audit of Hawke’s Bay Intensive Literacy and Numeracy records.
Include the In ensive Literacy and Numeracy funding condition (Appendix 2) in
the enrolment process to ensure decision-making is structured and covers all
components of the funding condition.
Workplace Literacy funding
Released
Include the workplace literacy funding condition (Appendix 2) in the enrolment
process to ensure decision-making is structured and covers all components of
the funding condition.
Official Information Act 1982
1 Pages 77-78 in the Ministry of Education – Single Data Return 2016 Manual.
5
course enrolment patterns to ensure the limits were not exceeded. A
subsequent review of the programme structure has sought to
eliminate unit standards that were common to more than one
qualification and this has been successful.
Data entry
Al data is loaded into Educate Plus at each of the YMCA Centres.
Access controls at each Centre are either specific to the Administrator
(with full access) or Read only. There is an inherent risk to the
integrity of data reported when full access to the student management
system is decentralised; however, this can be managed by regular
monitoring and sampling of records.
Single Data Return attestation
Prior to the submission of each Single Data Return trial reports are
run to identify and resolve any reporting erro s.
The attestation is signed off the General Manager – YMCA PTE
Attendance
Student attendance is monitored hrough attendance registers for
each programme.
Withdrawal
Student withdrawal policies and procedures are in place to ensure
decisions are documented and withdrawal is accurately reported.
Variation of the Enrolment process used between YMCA Centres
under the
In the sample of student records rev ewed across all funded
programmes (Youth Guarantee Intensive Literacy and Numeracy and
Workpla e Literacy) there were variations between YMCA Centres in
the d cumentation that was used.
Part of a quality management system seeks to standardise generic
d cumentation with controls in place to ensure current versions of
documents are being used.
Each YMCA Centre that enrols students follows a standard enrolment
process but there are variations between Centres in the range of
documents that are used and information that is provided to students.
This has resulted in a Centre-specific enrolment process modelled on
a generic enrolment process rather than an overarching and
Released
controlled YMCA enrolment process.
This carries a risk that individual Centres may develop forms and
processes that sit outside the quality management system (or
determine to not adhere to processes), such as occurred in 2014
when only Central consistently used the Tutor Management System
(TMS). These are issues for management when monitoring the
quality and consistency of enrolment decision-making.
Course completions reported through the Single Data Return
Official Information Act 1982
Across the organisation, the pastoral care needs of students are
assessed and time is taken to ensure students are ready to engage in
study. Once students start achieving unit standards these are
marked and moderated and course completions reported through the
student management system. Evidence was sighted that courses are
being completed and accurately reported in the Single Data Return.
7
2. The financial
Youth Guarantee students are not required to individually purchase
support for the
any item of hardware or equipment necessary to gain the approved
equipment
qualification.
infrastructure
required to
deliver the
approved
qualifications.
3. Whether
From the sample of records reviewed there was no evidence of any
inducements or
Youth Guarantee student being provided with an inducement or
benefits have
benefit to study.
been provided
to students.
4. Responsibility
There is no subcontracting of any component of programmes being
for
offered by the YMCA.
subcontracting
arrangements.
5. Compliance
Youth Guarantee students eligible to enrol
with Youth
Guarantee
Al Youth Guarantee student records in the audit sample were valid
funding
enrolments for funding purposes domestic student status was
conditions.
confirmed and minimum attendance requireme ts met.
6. Compliance
Hawke’s Bay records under review
with Intensive
under the
Literacy and
In the Hawke’s Bay sample of Intensive Literacy and Numeracy
Numeracy
records selected pre-audit concerns identified by YMCA related to
funding
recordkeeping for 2015 deliv ry YMCA is undertaking an internal
conditions.
audit of the learners reported and wil provide a separate report to the
TEC
The TEC wil rev ew and evaluate the report and an outcome may be
a full review of these records.
Intensive Literacy and Numeracy decision-making
Audit trails did not always detail how learners met the entry criteria in
Intensive Literacy and Numeracy programmes specified in funding
Released
condition:
ILN004 – TEO to ensure learners meet criteria.
YMCA should embed the Intensive Literacy and Numeracy funding
condition (Appendix 2) in the enrolment process to ensure decision-
making is both structured and covers all components of the funding
condition. Valid exceptions to policy, such as where a learning
disorder has been established, should also be documented to
safeguard funding.
Ineligible learners for funding purposes
Official Information Act 1982
For YMCA Central Intensive Literacy and Numeracy learner, NSN
9(2)(a)
, and Gisborne District learner, NSN 9(2)(a)
, there
was no evidence retained confirming domestic status.
The TEC could seek recovery of funding for these two learners.
8
7. Compliance
Workplace Literacy decision-making
with Workplace
Literacy
Audit trails did not always detail how learners met the entry criteria as
funding
specified in Workplace Literacy programmes funding condition:
conditions.
WLN004 – TEO to ensure learners meet criteria.
YMCA Invercargil check employer records to ensure learners were in
paid employment in Workplace Literacy programmes but this
evidence was not included as an entry requirement and recorded on
the enrolment checklist.
YMCA should include the Workplace Literacy funding condition
(Appendix 2) in the enrolment process to ensure entry requirements
cover al components of the funding condition.
Ineligible learner for Workplace Literacy
9(2)(a)
YMCA Wanganui District Workplace Literacy learner, NSN 9(2)(a)
, 2
held two qualifications, the Certificate in
Level 4 and 5, but audit trails could not confirm eligibility to access
funding by demonstrating a low skil level in literacy and numeracy.
No recovery of 17 hours of Workplace Literacy funding is sought for
this learner because of over-delivery.
Audit trails incomplete to establish eligibility to enrol
YMCA Invercargil Workplace Literacy learner NSN 9(2)(a)
,
indicated on their enrolment form that they eld a diploma of 9(2)(a)
but there was no audit trail detailing how the learner was eligible.
under th
8. Monitoring the
The main recommendations from the 2014 audit were:
action plan for
the TEC 2014
a Review and document a protocol for staff access privileges
audit.
for data entry into the student management system, Puk-e-
DATA.
i. This is stil an issue and not covered in the 2016
Quality Management System.
b. Rev ew, document and stardardise the enrolment practice
across all sites. The amended enrolment policy and
procedures then need to be documented in a section of the
Quality Management System Manual.
i. This has been partial y addressed and is covered
Released earlier in this report under the heading, Enrolment
process variations between Centres.
c. The issue of duplicate unit standards within elective
modules requires the YMCA to develop revised modules
and submit to NZQA for approval.
i. This issue has been successfully resolved.
Official Information Act 1
9. Compliance
The overal standard of recordkeeping was reviewed as satisfactory,
with the
notwithstanding two caveats, the first relating to Intensive Literacy
requirements
and Numeracy records for Hawke’s Bay are currently under review by
in Part 18 of
YMCA (covered in section 6 above) and the second relating to
Education Act
enrolment process variations between YMCA Centres (covered in
1989.
section 1 above).
Part 18 of the Education Act 1989 includes, but is not limited to
student programme information and records.
9
In the Quality Management System (section 12, pages 45-46) there is
an absence of detail around archiving of enrolment and student
records, specifically the timeframe for keeping records. There is also
nothing specific requiring YMCA to retain programme approval letters
(including tracking al and any Type 1 and 2 changes) and course site
approval letters from NZQA indefinitely.
YMCA has programme approval from NZQA for the programmes that
are funded by TEC. Copies of programme approval letters were
available for review.
A sample of timetables of programmes being provided across
Centres confirmed teaching hours were in accordance with NZQA
course approval documentation.
The timetabling of programmes needs to be standardised to ensure
YMCA managers can be confident NZQA approved face-to-face and
self-directed hours are being complied with.
Correct programme hours are a requirement of the TEC funding
conditions.
10. Any other TEC
Approved Course Sites and reporting of Youth Guarantee students
funding
matters.
Youth Guarantee students from Christchurch had been incorrectly
reported against the Main Campus in Lower Hutt in the December
2015 and April 2016 Single Data Returns. The reason for this was
that Christchurch was not set-up as an approved site.
A review of approved course sites identified several discrepancies.
under the
This is summarised in Appendix 1 – Approved Course Sites in STEO
with eight sites reporting SDR data.
It is impor ant that students are correctly reported to approved course
sites as the information reported is used by the Ministry of Education
to calculate student participation by geographic regions and can
assist the TEC when making investment decisions.
Literacy and Numeracy Adults Assessment Tool (Assessment Tool)
Al students and learners enrolled in TEC-funded programmes are
assessed using the Assessment Tool with an initial and progress
ass ssment. The Youth Assessment option has been available from
mid-2014.
Released
Official Information Act 1982
10
APPENDIX 2
Eligibility criteria for Intensive Literacy and Numeracy and Workplace Literacy
and Numeracy extracted from the 2016 funding conditions letter dated 21
December 2015
Pages 23-24
“
ILN004: TEO to ensure learners meet criteria
The TEO must ensure that each learner that the TEO enrols in an eligible programme of
study or training:
a) is, and continues to be, a New Zealand Citizen or resident; and
b) has low skil levels in literacy, numeracy, or literacy and numerac .
Definition of low skill level in literacy, numeracy, or literacy and numeracy
For the purposes of this condition, a learner has a low skil le el in literacy, numeracy, or
literacy and numeracy:
a) if the learner has fewer than 80 credits on the New Zealand Qualifications Framework
(NZQF); or
b) if the learner has more than 80 credits on the NZQF, the learne has fewer than 12
credits in literacy and/or numeracy in unit standards and/or achievement standards;
or
c) if the learner is assessed using the Literacy and Numeracy for Adults Assessment
under the
Tool and presents at Step 1 or 2 on the Learning Progressions for reading and/or
numeracy.”
Page 31:
“
WLN004: TEO to ensure earners meet riteria
“The TEO must ensure that each learner that the TEO enrols in an eligible programme of
workplace train ng:
a) is, and continues to be, a New Zealand Citizen or resident; and
b) has low skil levels in literacy, numeracy, or literacy and numeracy; and
c) is, and continues to be, in the paid workforce; and
Released
d) is not:
(i)
enrol ed a a full-time student at a TEO; or
(i )
accessing TEC-funded literacy, numeracy, or literacy and numeracy,
prov sion at another TEO or workplace.
Definition of low skill level in literacy, numeracy, or literacy and numeracy
Fo the purposes of this condition, a learner has a low skil level in literacy, numeracy, or
literacy and numeracy:
Official Information Act 1982
a) if the learner has fewer than 80 credits on the New Zealand Qualifications Framework
(NZQF); or
b) if the learner has more than 80 credits on the NZQF, the learner has fewer than 12
credits in literacy and/or numeracy in unit standards and/or achievement standards;
or
c) if the learner is assessed using the Literacy and Numeracy for Adults Assessment
Tool and presents at Step 3 or below on the Learning Progressions for reading and/or
Step 4 or below on the Learning Progressions for numeracy.”
12
