View Current

Academic Calibration Procedure

This is the current version of this document. To view historic versions, click the link in the document's navigation bar.

Section 1 - Preamble

(1) Academic Calibration is an external peer review process that is undertaken in collaboration with participating Universities to support meeting the Higher Education Standards Framework (Thresholds Standards) 2021. Specifically, that: 

  1. review and improvement activities include regular external referencing of the success of student cohorts against comparable courses of study; and
  2. the assessment methods and grading demonstrates the standards of learning outcomes. 
Top of Page

Section 2 - Purpose

(2) The procedure provides instructions to ensure the academic calibration process at Charles Darwin University (CDU) is applied comprehensively and consistently across faculties.

(3) The purpose of this procedure is to improve the quality of course work curriculum by providing mechanisms for a comparable review and constructive feedback for selected higher education units on:

  1. grades awarded;
  2. the relationship between assessment and learning outcomes;
  3. the relationship and appropriateness of a unit, within its designated course structures;
  4. the clarity and appropriateness of assessment design, learning outcomes, and supporting material for a unit; and
  5. the comparison of the assessment and supporting items to that of other institutions.
Top of Page

Section 3 - Scope

(4) These procedures apply to all higher education coursework awards offered by the University and all higher education teaching academics.

Top of Page

Section 4 - Procedure 

Methods of academic calibration

(5) There are two (2) methods for conducting academic calibration: 

  1. Calibration of CDU units by another provider (outgoing calibrations); and
  2. Review of units from another provider by CDU academic staff (incoming calibrations).

(6) The calibration of units must include a review of assessment tasks and learning outcomes.

(7) At times, a calibration may also include a review of the entire unit, at the request of the Unit Coordinator.

Calibration of CDU units 

Number of units selected

(8) A minimum of one (1) unit from every discipline will be calibrated each year. 

(9) The number of units to be calibrated will be reviewed annually by the University’s Learning and Teaching Committee (LTC).

(10) The Associate Dean Learning Futures (ADLF) is responsible for ensuring that each course has had at least one (1) unit calibrated during the course accreditation cycle of seven (7) years (or earlier if required for professional accreditation). 

Unit eligibility and selection

(11) The Chair of each Faculty Learning and Teaching Committee (FLTC) is responsible for selecting units for the following year in consultation with discipline staff. 

(12) The Quality Specialist Academic Calibration and Benchmarking will collate selected units for calibration into the yearly Academic Calibration Program (ACP) Schedule and provide to the LTC for endorsement in February each year. 

(13) LTC reviews and make any determinations on any particular focus or distribution method for the upcoming academic calibration schedule.

(14) Units will be selected based on the following considerations: 

  1. Course structure: 
    1. Capstone units, which will take preference in the selection process;
    2. Core units;
    3. Specialist electives; and
    4. Work Integrated Learning units, which will be assessed for eligibility on a per unit basis to identify complications from commercial in confidence or client-based information. The Unit Coordinator will inform any third party about their potential involvement in the calibration process. Both the relevant third party’s written permission and the Faculty Pro Vice-Chancellor's permission are mandatory for calibration to proceed.
  2. As an outcome of review and monitoring processes, such as the Higher Education Assessment Review Panel or Annual Course Monitoring.
  3. Informed by reviews of professional bodies, including requirements relating to professional accreditation. 

(15) A unit calibration will be applicable for all courses that include the unit in the course structure. 

(16) The ADLF will inform Unit Coordinators whose units have been selected for academic calibration within two (2) weeks of LTAC endorsement of the schedule.

Assessment selection

(17) The Unit Coordinator will select a single assessment task, or combination of smaller tasks, to review for each nominated unit.

  1. Priority is given to assessment tasks that cover a high number of unit learning outcomes and that have an assessment weighting of at least 30% of overall marks. 

(18) Assessment tasks must be de-identified without obscuring the ability to evaluate.

(19) An academic calibration is expected to be able to be completed within eight (8) hours. 

(20) If the entire unit is considered for review, criteria should be determined in advance of the calibration. 

(21) Criteria for unit review can include, but are not limited to:

  1. overall summary judgement of the unit; 
  2. unit design to support students to meet learning outcomes;
  3. alignment and coherence of unit learning outcomes, unit content and assessment; 
  4. appropriateness of unit content and learning outcomes for unit and AQF level;
  5. unit content compared against relevant discipline best practice, currency, and relevance;
  6. comparison of learning outcomes and assessment with those of similar units at other institutions; 
  7. design of Learning Management System (LMS) for ease of use and organisation of information for student access; 
  8. evidence of student support; 
  9. evidence of feedback on assessment; 
  10. review of assessment task appropriateness;
  11. clarity of assessment requirements and marking criteria; and
  12. appropriateness of assessment grades awarded.  

(22) Access to all unit materials on the LMS is arranged by the Quality Specialist Academic Calibration and Benchmarking if the entire unit is reviewed. 

Identifying external calibrators

(23) The Quality Specialist Academic Calibration and Benchmarking will coordinate the calibrations.

(24) The Quality Specialist Academic Calibration and Benchmarking will curate information provided by the relevant Unit Coordinator and send to participating Universities to seek suitable calibrators. 

(25) Information sent to participating universities by the Quality Specialist Academic Calibration and Benchmarking for each calibration will include the following:

  1. unit code;
  2. unit name;
  3. name of the faculty;
  4. unit catalogue web link;
  5. assessment task selected;
  6. assessment task information, as a link to the unit catalogue or summary statement: and
  7. the date that marked student samples will be available for review. 

(26) Calibrators may be nominated by multiple institutions, and it is the final decision of the ADLF to make the selection. 

(27) The Quality Specialist Academic Calibration and Benchmarking will send all relevant nominations to the Unit Coordinator, whose unit is being calibrated and the ADLF. 

(28) The ADLF will select the calibrator after consulting the Unit Coordinator.

(29) Calibrators are not expected to be teaching a similar unit or to have taught a similar unit. It is sufficient that the nominated calibrator is experienced in the discipline and possesses a well-developed sense of academic standards.

(30) Where there are no suitable academics within the participating Universities, the Quality Specialist Academic Calibration and Benchmarking will look for suitable academics from other universities, who have expertise in the same discipline.

Calibration documentation 

(31) Unit Coordinators in consultation with the Quality Specialist Academic Calibration and Benchmarking will collate the calibration documentation of student assessment samples and supporting materials relating to the selected assessment task and unit, which will include: 

  1. information about the course structure/s of which the unit is a part of;
  2. course level learning outcomes, and their relationship to the unit level learning outcomes;
  3. the unit outline provided to students;
  4. unit learning outcomes;
  5. grade distribution of the selected assessment task;
  6. grade distribution of the unit for the calibrating semester (optional);
  7. selected assessment task detail;
  8. the marking rubric for the selected assessment task;
  9. a Context Statement that describes important features of the unit, or the context of the specific teaching period in which the unit was offered that may not be immediately obvious through standard documentation; and
  10. the marked student samples with feedback in the following grade ranges, or as close as possible:
    1. Pass – low, medium, high
    2. Credit – low, medium, high
    3. Distinction – low, medium, high
    4. High distinction – low, medium, high

(32) The Quality Specialist Academic Calibration and Benchmarking is responsible for the overall curation of the calibration documentation. 

(33) The Quality Specialist Academic Calibration and Benchmarking will ensure student samples are de-identified prior to submission to the calibrator. 

  1. If de-identification is not possible, the ADLF will decide whether the calibration will proceed.

(34) The inclusion of additional information to support the calibration of a unit can be raised by the Unit Coordinator with the Quality Specialist Academic Calibration and Benchmarking, who will negotiate with the corresponding university.

Calibration timeframes

(35) Once the calibrator receives the calibration documentation, they must evaluate all items provided and complete the academic calibration forms and report within six (6) weeks. 

(36) If the calibration package is not made available to the calibrator within the agreed time frame, the agreed date for the return of the calibration report will be re-negotiated by the the Quality Specialist Academic Calibration and Benchmarking, in consultation with the Unit Coordinator.

Return and review of calibration reports (CDU Units) 

(37) External calibrators will return completed reports to the Quality Specialist Academic Calibration and Benchmarking.

(38) The Quality Specialist Academic Calibration and Benchmarking will:

  1. review the report and clarify any information with the calibrating institution; and 
  2. forward completed reports to the relevant Unit Coordinator and the ADLF for review.

(39) All parties must review the report and provide a response seeking feedback or clarification of the report to the Quality Specialist Academic Calibration and Benchmarking within ten (10) working days. 

(40) If no response is received within two (2) weeks, it is assumed the report is accepted.

Report recommendations

(41) A calibration report may identify risks to the future quality of the unit and will be used as evidence to inform further evaluation and review of a unit.

(42) The Unit Coordinator and ADLF is responsible for proposing any actions based on recommendations of the unit calibration report.

(43) Actions that involve unit changes must be incorporated within existing unit change and accreditation processes.

(44) Any unit change proposals must comply with Higher Education Course and Unit Accreditation Procedure.

Report implementation: course accreditation and professional accreditation

(45) Calibration reports must be submitted as part of the regular review and accreditation of all coursework awards and associated units.

(46) The ADLF is responsible for determining whether calibration reports are submitted as part of the external professional accreditation requirements for a specific course.

Reporting

(47) The Quality Specialist Academic Calibration and Benchmarking will provide an annual report to LTC to provide a comparative review of the quality of assessments and learning outcomes. 

(48) The report will include any actions taken to change unit details following a calibration.

Calibration of external units

Recruitment and approval of calibrators

(49) The Quality Specialist Academic Calibration and Benchmarking will maintain a central register of academics who have expressed interest in becoming reviewers. All academic staff of the University may apply to register. 

(50) The Quality Specialist Academic Calibration and Benchmarking will promote registration opportunities with Faculty Learning and Teaching Committee, being recognised as relevant to their professional development plans and scholarship. 

(51) Staff who wish to register as a calibrator will contact the Quality Specialist Academic Calibration and Benchmarking before registering.

  1. No staff member may register without the approval of the ADLF. 

(52) The Quality Specialist Academic Calibration and Benchmarking will contact the ADLF or appropriate academic staff in the discipline to consider nominating to calibrate a unit posted for calibration by another institution that reflects their area of expertise. 

(53) The Quality Specialist Academic Calibration and Benchmarking will supply staff with a link to the web-based registration form or will collate materials on a staff member’s behalf and assist with lodgement of the application.

(54) Academic staff members who nominate as calibrators will be required to provide an up-to-date online academic staff profile on the University’s website and a curriculum vitae.

Eligibility and selection of calibrators

(55) Academic staff members must have experience in the selected discipline of the unit seeking calibrators, as well as demonstrated teaching experience, including the grading of student assessment work.

(56) Academic staff members will be at least an Academic Level C or equivalent, however, an Academic Level B with extensive experience in a specialised area will also be considered.

(57) The calibrator will be selected from all nominated academic staff by the University requesting the calibration. 

Requirements and expectations of calibrators

(58) Calibrators must sign an Academic Calibration Declaration Form, which acknowledges academic experience requirements and the acceptance that once reports are returned to the external institution, the calibrator retains no rights, including copyright and moral rights, in connection with the materials produced for the review.

(59) It is required that calibrators will:

  1. be fair, and timely;
  2. provide feedback that is comprehensive and constructive;
  3. act in confidence and not disclose the content or outcome of the calibration;
  4. be informed about, and comply with, the criteria being applied;
  5. declare all conflicts of interest, including professional views or values, which may influence the calibration process;
  6. consider teaching and assessment that challenges or changes accepted ways of thinking; and
  7. not participate in calibrations where the unit is outside one’s area or level of expertise, or where this occurs, without declaring the limitations.

Calibration payment

(60) Calibration is considered a core quality function of academic roles and does not normally include an honorarium. 

(61) The selected method for payment of calibrators will be determined between the corresponding universities based on the nature of the review and on a case by case basis.

Records

(62) Reports will be retained and stored in the relevant records management system in accordance with the University’s Records and Information Management Policy and Procedure.

Top of Page

Section 5 - Non-Compliance

(63) Non-compliance with Governance Documents is considered a breach of the Code of Conduct – Staff or the Code of Conduct – Students, as applicable, and is treated seriously by the University. Reports of concerns about non-compliance will be managed in accordance with the applicable disciplinary procedures outlined in the Charles Darwin University and Union Enterprise Agreement 2022 and the Code of Conduct – Students.

(64) Complaints may be raised in accordance with the Complaints and Grievance Policy and Procedure - Employees and Complaints Policy - Students.

(65) All staff members have an individual responsibility to raise any suspicion, allegation or report of fraud or corruption in accordance with the Fraud and Corruption Control Policy and Whistleblower Reporting (Improper Conduct) Procedure.