About the CIP
Since the launch of the Certification System in 2018, NAATI has rolled out certification testing in more than 50 languages and will be continuing to offer more over the coming years. We have accumulated a significant amount of data in that time, with several thousand tests now complete, which provides the opportunity to conduct the first refinement of the system.
As part of its commitment to proactive continuous improvement, NAATI launched a Continuous Improvement Program (CIP) in March 2021.
The primary aim of CIP is to undertake an in-depth analysis of the Certification System to ensure it remains fit for purpose, and to ensure that NAATI’s certification tests remain valid, reliable and practical to run. The program focuses on non-specialist certification tests: Certified Translator (CT), Certified Provisional Interpreter (CPI) and Certified Interpreter (CI).
The program targets six specific focus areas:
- Examiner Panels’ Training & Performance
- Test Reliability
- Test Delivery & Technology
- Prerequisites
- Task Design
- Assessment Rubrics.
First phase complete
The initial consultation process and data-collection phase has been completed. This involved gathering data through comprehensive surveys of NAATI Examiners and test candidates, an interrater reliability study, more than 200 examiner training workshops and 30+ hours of consultations with a dedicated CIP Working Group of experts.
The first phase of this program identified 20 recommendations, divided into areas of focus, below. Some of these recommendations are test-specific, others are broader and may involve stakeholders other than NAATI.
NAATI will start implementation progressively from July 2022 onwards, with the aim to implement most of the recommendations by the end of the 2023-2024 financial year.
-
1. Evaluating and monitoring examiner performance
Current processes around Examiner certification test setting and marking performances will be consolidated by introducing a Performance Evaluation Framework that will enable NAATI to give systematic and data-driven feedback on individual examiners and panels’ performances. Training modules to support Examiners will also be provided as part of this framework.
02. Recognition of the NAATI Examiner status
NAATI Examiners may disclose if they wish, their examiner status publicly. To enable this change NAATI will provide Examiners with an Examiner badge they can use to advertise their status.
03. Rewarding examiner expertise and contribution
Best performing examiners will be rewarded for their expertise and contributions through a variety of mechanisms including professional opportunities (e.g., mentoring, consultancy opportunities)and a NAATI awards ceremony.
04. Creating an online Examiner space
An online Examiner space will be createdto centralise important information relevant to examiners, to provide easy access to shared resources and to enable discussion forums moderated by NAATI staff.
05. Enabling examiners through live interaction opportunities
Professional development events will be proposed to examiners in the form of webinars and live Q&A sessions to create more interactions with NAATI and consolidate aspects of the training or processes relevant to examiners.
06. Streamlining the non-specialist interpreting stream
The non-specialist interpreting stream will be refined by:
- Removing the dialogue tasks from the Certified Interpreter (CI) test
- Splitting the CI test into two sessions:
– a Certified Provisional Interpreter(CPI) test with three CPI dialogue tasks and;
– a Certified Interpreter (CI) test composed of six interpreting tasks (monologues and sight translation tasks). - Providing dialogue task briefs three days prior to testing
07. Improving testing conditions for the Sight Translation task
A mark-up and text highlighting functionality will be integrated to the online testing platform for the sight translation tasks and candidates will
be provided with internet access during preparation time.
08. Allowing access to online resources and spell-check functionalities for the Certified Translator test
Candidates will be given access to a whitelist of relevant LOTE online resources in addition to the already permitted downloadable and printed resources, and the spell-check functionality available in NAATI’s testing platform will be enabled.
09. English source texts to be culturally and locality neutral to support international testing
English source texts for both the Translation and Revision tasks will be culturally and locality neutral to cater for a broader international audience.
10. Advising candidates of the range of errors in the Revision task
Candidates will be provided with a range of errors to identify in the text to be revised to mitigate a tendency to overcorrect and support the assessment process.
11. Sponsor a research project to investigate current industry practice for revision assignments and develop a set of industry standards in this area
NAATI will co-fund and support a collaborative research project involving industry experts to investigate revision industry practice, current market expectations and technological evolutions. This project will help define future improvements to the Certified Translator test revision task while defining industry standards in this area.
12. Adapt the dialogue task introduction to allow for familiarisation and introduction of the interpreter’s role
The current dialogue introductory prompts for spoken languages be replaced with a new format allowing for a more natural and meaningful introduction between role-players and candidates. This format will support accent and dialect familiarisation as well as introduction of the interpreter’s role.
13. Review interactional management triggers
Alternative interaction management triggers, in line with the disruptions interpreters face in either online or onsite assignments, be introduced in the dialogue tasks.
14. Improving and consolidating role-player training and performance
Role-player training will be reviewed in line with Recommendation 12 and 13, and more systematic interactions with or feedback from examiners will be encouraged to create a continuum between dialogue task creation, delivery, and assessment.
15. Collaborating with training institutions to develop resources on Interaction Management
The development of a training module on Interaction Management will be sponsored via NAATI’s Research Development Fund, developed collaboratively by a cohort of training institutions, and will be made available to prospective candidates, trainers, and NAATI Examiners.
16. Improving the wording of assessment rubrics band descriptors for clarity and transparency
A full review of the Certified Translator, the Certified Provisional Interpreter and the Certified Interpreter assessment rubrics descriptors will be undertaken with the view to create more clarity in the wording, limit errors of judgement and provide more guidance on aspects of the performance.
17. Removing Criterion B from the Translation of a Non-Specialised Text Task assessment rubrics and collapsing it under Criterion C – Application of textual norms and conventions
“Skill B – Follow the Translation Brief” will be removed from the Certified Translator – Translation of a Non-Specialised Text Task assessment rubrics and assessment rubrics will only contain three criteria: A: Meaning Transfer skill, C: Application of Textual Norms and Conventions and D: Language Proficiency Enabling Meaning Transfer (Target language – LOTE or English).
18. Making system improvements to support examiner marking
Technical enhancements will be investigated and scoped to improve the current Examiner portal with the provision of a dropped-down menu for feedback comments to create efficiencies in the assessment process and ensure consistency of feedback comments.
19. Consolidating the continuum between NAATI, training institutions and candidates through support and resource sharing
Collaboration and cross-information between NAATI and endorsed institutions will be extended more broadly around key areas of the Certification System, with the provision of information sessions to training institutions and through the sharing of available resources.
20. Phasing out the Skill Set training package
The Skill Set course will be phased out from the training prerequisites once the reviewed Diploma and Advanced Diploma packages become universally available.
A pdf version of the recommendations is available to download below:
Contact Us
If you have questions about the program, please contact us at certification_cip@naati.com.au
