2.10 Institute and Department Performance Audit (IDPA)

1. The IDPA provides an opportunity for the Department, Institute and University to evaluate the effectiveness of an Institute/Department’s management, learning, teaching and enhancement and quality assurance procedures and monitoring of performance against relevant KPIs and for the University to assure itself and be able to demonstrate to external bodies (QAA) that management of quality and standards which are the responsibility of the University as a whole are being delivered successfully.

2. An Audit Team, reporting to the Quality Assurance Committee will, on behalf of the University, audit departments on a 5-6 year rotating basis, to examine that all relevant processes and mechanisms for quality and standards are in place, operating effectively and efficiently, and that recommended enhancements to teaching and learning are being implemented.  Any collaborative provision will be included as part of the quality audit.

3. Although the audit will focus on activities at a department level, links to and the activities of relevant institute level committees and management structures will also be explored.

4. If deemed appropriate an Institute Audit may also be conducted in place of, or in addition to, individual academic department audits. An Institute Audit will be conducted on the same basis as a departmental audit, though the Audit Team may focus on the operation of institute processes, systems, compliance and training and management structure and the overall portfolio as opposed to more detailed investigation of undergraduate and postgraduate taught modules and schemes.

5. Student representatives from the department should be encouraged to participate fully in the department’s preparations for the audit and in completing the self-evaluation document (SED). Alternatively, student representatives may submit their own self-evaluation document, or alternatively attach an appendix to the department submission.

Details of the IDPA process

6. The focus of a visit will be clearly specified to a Department in advance but will relate to the delivery of all of the Department’s provision. When an Audit Team visits the following areas will be considered in the review:

(i) Undergraduate and postgraduate schemes and modules; postgraduate research provision; collaborative provision;
(ii) Feedback Systems;
(iii) Compliance;
(iv) Staff Development and Training.

7. An audit visit will typically last one or two days and involve the following aspects:

(i) Inspection of a self-evaluation document, relevant committee minutes and all other documentation concerned with teaching and learning in a Department;
(ii) Meeting(s) with Head of Department and staff with responsibility for different aspects of teaching and learning (e.g. examinations, student support, admissions);
(iii) Meeting(s) with student representatives, undergraduate, taught postgraduate and research postgraduates;
(iv) Meeting(s) with other staff involved in quality assurance and/or teaching and learning (e.g. new staff members / administrators);
(v) Meetings with any relevant Institute level staff e.g. Institute Directors of Undergraduate Studies, Postgraduate Studies or Learning and Teaching, Research, Director of Graduate School; these meetings may occur at subsequent Institute audit if not at the departmental audit.

Audit Team Membership and Indicative Schedule

8. Membership of the Audit Team will be discussed with the Chair of QAC and/or Director of AQRO. Normally an Audit Team will include the following members:

(i) Chair (senior member of AU academic staff e.g. PVC, ID, HoD or other academic member of staff with a senior institute role);
(ii) A member of QAC, a previous Audit Team member or another ID;
(iii) Head of Department from a department scheduled for audit in the near future;
(iv) External Academic Assessor (Academic Subject expert and if appropriate an industry expert);
(v) Director of AQRO or their representative;
(vi) Student representative;
(vi) A member of AQRO staff will serve as Secretary to the Audit Team, overseeing the exercise and drafting the report.

An observer (a member of AQRO staff or an internal member of academic staff may also sit as an observer in order to gain experience before becoming a full Audit Team member);

9. No more than two members of the University Executive should normally be members of a Departmental Audit Team.  Audits will normally be chaired by the Pro Vice-Chancellor, Chair of QAC or by another Institute Director. The internal auditors for each visit will be identified by AQRO. All members of staff appointed to serve as members of Audit Teams will be given appropriate training and guidance, and the External Assessor will be issued with formal notes of guidance.

10. Departments will be invited to nominate three professional colleagues from the United Kingdom who could act as an External Assessor for which a fee and expenses are paid.  An External Assessor should not be a current External Examiner from within the Department and should preferably have knowledge of quality assurance processes and procedures.  The choice of external assessor will be made by the Chair and the Secretary of the Audit Team in consultation with the Department.

Indicative Schedule

11. The following is an indicative schedule for a one-day audit (Departmental or Institute). The Chair and Secretary may meet with students ahead of the main event, and submit a written summary of the discussion for consideration by the full audit team:

No

Time

Event

1

  9.15 am

Private meeting of Audit Team members

2

10.00 am

Meeting with the Head of Department/Institute Director

3

11.00 am

Private meeting of Audit Team members

4

11.30 am

Meeting with senior academic staff with key roles in the Department/Institute (It may be appropriate to see two separate groups)

5

12.30 pm

LUNCH

6

1.30 pm

Private Meeting of Audit Team

7

2.00 pm

Meeting with students*

8

2.45 pm

Private meeting of Audit Team members

9

3.15pm

Meetingwith other staff including administrative and part time staff involved with quality assurance and/or teaching and learning

10

4.15 pm

Private meeting of Audit Team members

11

4.45 pm

Feedback to Head of Department

12

5.00 pm

Finish

*Where appropriate the Chair and Audit Secretary may meet with students ahead of the Panel.

Details of the process

12. The focus of a visit will relate to all schemes (from Foundation and undergraduate degrees to taught postgraduate Master’s schemes, and Postgraduate Research). The following items will be included in the review:

 Undergraduate and Postgraduate Schemes and Modules

(i) the standards of teaching and learning in modules.
(ii) Methods of assessment in their relationship to:

  • learning outcomes
  • course content
  • learning strategies
  • transferable skills

(iii) Student Progression and Achievement
(iv) Student Support and Guidelines
(v) Student representation
(vi) Teaching and Learning Resources
(vii) Progression between different levels of undergraduate courses (Level 0 through to level M in the case of Integrated Masters schemes), and the relationship between undergraduate and postgraduate modules.
(viii) Holistic nature of the course (including Joint Honours).
(ix) Teaching and learning innovations e.g. VLE
(x) Student Handbooks and other information provided to students
(xi) Any Franchising, collaborative or Distance Learning schemes
(xii) Welsh Medium teaching (if applicable)
(xiii) Placements or work experience (if applicable)
(xiv) Exchange and Erasmus
(xv) Support, training, supervision and monitoring of research students.

Feedback Systems:

(i) External Examiners’ Reports
(ii) Feedback from Professional, Statutory and Regulatory Bodies (if applicable)
(iii) Student Module and Scheme Evaluations, reports on MEQs, Tell us Now
(iv) Annual Monitoring of Taught Schemes
(v) KPIs including National Student Survey Results

Compliance: The Audit Team will also ensure that Departments are complying with the University’s Regulations and the Academic Quality Handbook, and engaging with and implementing University policy, including, for example, the University’s Learning and Teaching Strategy.

Staff Development and Training: The Audit Team will discuss the following issues with Departments:

(i) University and Departmental induction and training, and the development of staff.
(ii) Peer Observation process.

Documentation

13. Departments will be expected to evaluate themselves against a set of statements in relation to quality and standards of teaching and learning, using the SED template which will include KPIs. The statements cover the following areas: academic schemes, teaching and learning, assessment and feedback, research students, student participation and experience, academic staff development, collaborative provision and external examiners.

14. The comments section should be used to reference supporting evidence and identify action points for quality enhancement and indicate the steps the department has identified that it needs to take to enhance quality and/or improve performance.

15. In addition, the Department is asked to write a commentary (no more than two sides of A4) reflecting on the major strengths, weaknesses, opportunities and threats it has identified (SWOT analysis). The scores and comments will be used as a basis for discussion during the audit.

16. The Department should provide the following documentation which should be uploaded onto the AQRO SharePoint site no later than three weeks before the audit visit:

(i) Departmental Structure Roles and Responsibilities: Details of the composition and responsibilities of department and, where relevant, Institute level Committees involved in teaching and research and an explanation of the reporting lines of these groups within the department and Institute identifying key staff roles and responsibilities.
(ii) Committee Minutes for the previous twelve months of the following:
Department or Institute executive committee
Department (and/or Institute) Learning and Teaching Committees
Staff-Student Consultative Committees
Committees dealing with collaborative provision or other distributed learning.
(iii) Strategy Documents (Institute level documents) for example:
Teaching & Learning Strategy
NSS Action Plan
Employability Strategy
TELG Action Plan
(iv) Reports on Periodic Quality Assurance Processes:
Period Scheme Review report(s)– report of most recent review
Annual Monitoring of Taught Schemes (AMTS1 forms) documentation for the previous year.
The most recent report of a professional body or external organisation (where relevant) on schemes offered by the department.
Evidence of student feedback on modules and any actions taken from the previous and current (if applicable) academic year.
(v) Departmental Publications:
Department or Institute Welcome Week or induction programme.
Department student handbooks for the current academic year
Staff Handbook(s)/guidance
(vi)Any other documentation which may be relevant.

AQRO and the Planning Department will also provide the following on the SharePoint site:

(i) External Examiners’ Reports for the previous three academic sessions.
(ii) Previous IDPA Report and Progress Report.
(iii) Annual Departmental Review Report prepared by the Planning Department including statistical data relating to applicants, students and staff.
(iv) Key Information Sets.

The audit team may request additional evidence after the meeting if specific issues need to be explored further.

Outcome of the Departmental Performance Audit visit

17. The Chair and Secretary may invite the External Assessor to submit a written report for consideration by the Audit Team during the IDPA event. AQRO will also liaise with the External Assessor and provide information on University level QA processes as appropriate.

18. Following the audit, a report will be produced which will highlight areas of good practice and identify recommendations for the Department, for the Institute and for the University. This report will be considered by QAC. The Head of Department will be given an opportunity to comment on factual accuracy prior to the draft report being circulated to the Institute Director and the Institute Audit Team members. The Department will not be expected to respond in full with an Action Plan until a later date.

19. The Department will be asked to submit an Action Plan within three months of the publication of the report, indicating how it intends to implement the recommendations of the Audit Team. A template for the Action Plan will be provided with the report.

20. The Department and, if appropriate, the Institute will be required to provide a written follow up to the report within 12 months of the audit, indicating how it is implementing the recommendations identified in the Action Plan. Where a specific issue gives cause for concern the Department and/or Institute may be required to respond to this within a shorter period. The written response will be considered by QAC.

21. If, as a consequence of the written response, QAC is not satisfied with the progress or implementation of the action points, it can request the following:

(i) A visit to the Department to discuss the action points in more detail by the Chair and Secretary of the Audit Team.
(ii) That a follow up internal audit be conducted within the Department between 12 and 36 months.

Institute Audits

22. This section provides guidance on operating a two stage audit, with the Preliminary Departmental Audit being followed by an Institute Audit. Stage 1:Preliminary Departmental Audits will be conducted as separate one day events prior to a panel being convened to review the operation of the Institute. The Audit Team may or may not meet with Institute staff at the departmental audit, some or all may be present in their capacity as members of the academic department being reviewed. Stage 2:The Institute Audit should ideally be held within a reasonable timescale following the last of the departmental audits.

23. The Institute Director will be invited to provide a written commentary on the departmental audit reports, SEDs, KPIs and Planning Departmental Performance Review document.

24. A Pro Vice-Chancellor or Chair of QAC will chair the Institute Audit Panel and its membership will include the chairs of the department panels, and an external assessor with senior university level management experience (who will not be expected to be a subject expert), student representation and representation from AQRO. The indicative schedule for a one day Institute Audit will follow the format outlined in paragraph 11.

25. The Panel will meet with the Institute Director, senior institute staff, and any other staff with institute roles, e.g. administrative staff, admissions tutors etc and the institute student representatives. The indicative schedule for a one day Institute Audit will follow the format outlined in paragraph 11.

26. A separate report will be prepared on the Institute Audit Panel meetings, drawing out recommendations for the institute and for the university; this report together with the individual departmental reports will constitute the full report for the institute and be reported to QAC and beyond.