This fact sheet provides guidance to Registered Training Organisations in determining that their assessment system is meeting the requirements of the Standards for Registered Training Organisations (RTOs) 2015 for Clause 1.8, and for Clauses 1.9, 1.10 and 1.11.
This fact sheet does not fully address the requirements of Clause 1.25 and Schedule 2 of the Standards for RTOs which relate to the independent validation of training and assessment qualifications. Further information on these requirements can be found in TAC Fact Sheet Delivery of Training and Assessment Qualifications and further information on TACs Assessment workshop series is available via the TAC Recordings and Resources webpage.
Assessment Validation Explained:
Clause 1.8a requires that the RTO's assessment systems comply with the assessment requirements of the relevant training packages or VET accredited courses, and Clause 1.8b requires RTOs to ensure that evidence gathered is valid (one of the Rules of Evidence) and that assessment processes and outcomes are valid (one of the Principles of Assessment). These requirements must be met and demonstrated by all the assessment policies, procedures, materials and tools of the RTO. Clause 1.8 relates primarily to the development (or purchase) of the RTO's assessment resources.
Assessment validation is emphasised in the Standards for RTOs, and RTOs are required to:
- develop and implement a comprehensive plan for the ongoing systematic validation of assessment that includes all training products on the RTO's scope of delivery (Clause 1.9);
- validate the assessment practices and judgements for each training product at least once every five years with at least 50% of products to be validated within the first three years of each five year cycle (Clause 1.10); and
- ensure that validation is conducted by one or more suitably qualified persons who are not directly involved in delivery and/or assessment of the training product being validated. (Clause 1.11).
These Clauses relate primarily to the actual delivery and outcomes of the RTO's assessment systems, including the performance of the RTO's assessors.
is assessment validation?
Assessment validation should answer the question:
'Do the assessment practices meet the requirements of the training package and the Standards for RTOs and are assessment judgements being consistently applied so that certification issued is accurate and credible?'
Assessment validation needs to be undertaken across all stages of the assessment process of design, the application of evidence-gathering tools, and the assessment judgement.
The validity of the assessment system design process is addressed under Clause 1.8 and applies to all assessment resources. Validation of the application of the assessment system is addressed through Clauses 1.9 - 1.11 and is conducted with a sample of assessments.
The overall outcome of assessment validation is to ensure:
- assessments are valid and meet the training package or accredited course requirements;
- the principles of fairness, flexibility, validity and reliability are met;
- the evidence used to make a decision about competence is valid, sufficient, authentic and current;
- assessors are familiar with the outcomes being assessed;
- assessors reach accurate and consistent decisions that inform assessment outcomes; and
- potential improvements to assessment practices are identified and acted upon.
can be involved in validation of assessment?
The validity of the assessment system (Clause 1.8) needs to be built into the design and development process through the RTO's policies and procedures and the planning and preparation by the assessor and needs to be demonstrated by the RTO assessor responsible for the process.
In contrast, any assessor directly involved in the application of the assessment system (Clause 1.9) cannot be responsible for the validation as they are in part the object of the validation (Clause 1.11).
This validation process can be undertaken by a team or individual approach.
If a team approach is used, the assessors who are directly involved in the
assessments being validated may participate in the process, however they cannot
contribute to meeting the team requirements of Clause 1.11.
If an individual approach is used, validation must be undertaken by a
person who meets all the requirements of Clause 1.11.
Clause 1.11 requires participation in the validation process by a person or persons other than the assessors directly involved in the particular instance of delivery and assessment being validated that hold or collectively have:
- vocational competencies and current industry skills relevant to the assessment being validated;
- current knowledge and skills in vocational teaching and learning; and
- the training and assessment credential specified in Item 2 or Item 5 of Schedule 1.
Industry experts may be involved in validation to ensure there is the combination of the above expertise.
The validity of the design
or purchase of assessment resources
The design or purchase of assessment tools is considered part of the RTO's assessment system under Clause 1.8.
All assessment resources, whether purchased or designed internally, must be validated prior to use in accordance with the Principles of Assessment and the Rules of Evidence. A mapping document where assessment tools are mapped to the specific requirements of the unit of competency is typical evidence that an RTO has checked the validity of the assessment tools and would generally be considered partial evidence of the validity of assessment if it was accurate.
The purpose of assessment validation at the design or purchase stage is to verify if the assessment resources:
- meet the principles of fairness, flexibility, validity and reliability;
- meet all the rules of evidence of validity, sufficiency, currency and authenticity;
- provide clear instructions for evidence gatherers and candidates; and
- provide transparent and defensible information to RTOs in relation to their competency decisions and awarding of nationally recognised qualifications and/or statements of attainment.
Where to start?
It is important RTOs have a clear understanding of the terms being used before they commence developing assessment validation plans and schedules under
Clauses 1.9 -1.11.
For example, in the context of the Standards for RTOs:
- What is a training product?
- What is the intent of assessment validation?
- What is a statistically valid sample?
The answers to all of these questions are detailed in the Glossary of the Standards for RTOs.
Clause 1.10 of the Standards for RTOs states:
|||"…each training product is validated at least once every five years, with at least 50% of products validated within the first three years of each five year cycle….."|
The intent of this requirement is that over five years the assessment practices and judgements of all training products on an RTO's scope of delivery must be systemically validated including those where there have been no enrolments.
As noted earlier, the intent of assessment validation is to establish:
|||Is the assessment process actually doing what it is supposed to be doing?''|
This question needs to be answered across both stages of the application of the assessment process including:
- the gathering of evidence using the assessment tools; and
- the assessment judgement based on the evidence.
These stages will most likely be a starting point for informing the RTO's assessment validation plan.
This stage of the assessment validation process is a quality process that evaluates if the:
- assessors reached accurate and consistent decisions that informed nationally recognised training outcomes;
- Rules of Evidence have been met through the collection of valid, sufficient, authentic and current evidence;
- competency decisions are valid, fair and reliable in accordance with the Principles of Assessment; and
- assessment validation outcomes and actions are used to inform continuous improvement of assessment.
When validating a whole qualification or a skill set the assessment practices and judgements of a minimum of at least two units of competency must be validated. The two units selected need to be typical of the qualification or skill set.
It is anticipated that RTOs will use a risk analysis approach considering the complexity of the qualification and the risks associated with each unit of competency to determine the actual number of units of competency that will be validated. If the risk analysis highlights the presence of a number of risk factors that may compromise the quality of the assessment, or your validation indicates that assessment processes are not valid then then more units should be validated.
When validating stand-alone units of competency, samples of assessment judgements for the full unit must be validated.
Statistically valid sample
In the Glossary of the Standards for RTOs a statistically valid sample:
|||'… means for the purposes of these Standards, a random sample of appropriate size is selected to enable confidence that the result is sufficiently accurate to be accepted as representative of the total population of assessments being validated.'|
The method an RTO uses to determine their assessment validation sample is an individual RTO decision. An RTO must validate a random sample of assessments. The selection of a completely random sample has many strengths, but the critical aspect of any sample is that it must be sufficient to ensure consistency of assessment decisions across each training product.
It is suggested that a 'stratified random sample' be used. This involves identifying distinct groups of candidates (such as campus based, workplace based, RPL, etc) and randomly selecting an appropriate number of students from each group for close analysis. Over time, all assessors, assessment methods and tools should be included in the sample. It is good practice to document all sampling strategies in a written internal verification procedure.
There are several key features that should be considered in selecting an assessment validation sample including (but not limited) to:
- student enrolment numbers in each training product;
- scope of delivery;
- feedback from clients, trainers and students;
- complaints received – formal or informal;
- completion rates;
- risk level of a training product;
- assessment mode (Simulation, workplace, RPL);
- introduction of new assessment activities/ tools;
- introduction of new training products;
- new assessors;
- third party arrangements; and
- previous issues or problems identified from Assessment validation.
RTOs should note that there is no 'one right way' for the conducting of systematic assessment validation of assessment practices, individual RTOs are best placed to establish an appropriate plan and schedule.
the gathering of assessment evidence
An RTO’s assessment validation plan will include monitoring
of the use of assessment tools by evidence-gatherers to ensure they are
implemented appropriately and are being used in accordance with the Rules of Evidence.
The application stage of assessment validation may also
verify those who are gathering evidence are familiar with assessment tools and
the outcomes they are assessing. This may be done by observing the use of
assessment tools to verify that the evidence being gathered is an accurate
reflection of the relevant unit requirements and the candidate’s performance.
The monitoring of the validity of the application of
assessment tools should be ongoing through the assessment process, not just at
the end. Indications of problems may be through student feedback, appeals and
complaints. This allows modifications to the evidence-gathering process before
final assessment judgements are made. Identifying problems at an early stage
will also allow support or training to be provided to evidence-gatherers.
Having validated the processes of gathering evidence, the validation should now focus on how the evidence is used to make the assessment judgement.
This can be done by establishing that there is a direct link between the evidence observed and the judgement of competence.
In particular, the validation could confirm that the Principles of Assessment have been met, including:
- all unit requirements have been observed before competence is awarded;
- different assessors using the same evidence would make consistent reliable judgements;
- all available evidence has been considered before making the judgement; and
- the candidate has had the opportunity to demonstrate all unit requirements.
The assessment validation team or person should have access to the same evidence in the sample that the assessor used to make the original assessment judgment. This is so the validator/s can reliably ensure that the assessment practice and judgement are valid.
assessment validation outcomes
The outcomes of assessment validation may lead to recommendations for improvement to assessment tools, assessment processes or assessment judgements.
Your RTO must have a process to maintain records of assessment validation activities. You should retain evidence of:
- the person/people leading and participating in the assessment validation process including their qualifications, industry skills and VET knowledge;
- the training product and selected units of competency;
- the evidence-gathering tools for those units;
- the assessment validation tools used;
- all assessment samples considered; and
- the assessment validation outcomes.
The assessment validation plan must clearly show how you will document and act on assessment validation outcomes.
The validation plan must demonstrate how the assessment validation decisions will inform the continuous improvement of assessment practices and improvements. These improvements should be introduced across all training products. If changes and improvements are made to an assessment tool this tool should be reviewed again prior to implementation.
An RTO with a well-designed assessment system and accompanying validation process that have been fully implemented will be able to demonstrate that their assessment judgements:
- are valid;
- align with the requirements of the unit of competency or module; and
- comply with the Standards for RTOs.
are the auditors looking for?
During an audit the auditors are looking for evidence that you have established a system that assures the quality of assessment in your RTO. This will include:
- having processes and tools that enable validation at all stages of the lifecycle of a training product in your organisation;
- having a plan for validation that is implemented successfully in your organisation;
- being able to demonstrate that you have managed who participates in the validation process;
- being able to demonstrate that your assessors have made quality assessment decisions; and
- your assessment practice has improved as a result of validation activities.
17/06/2022 2:39 PM