validity and reliability in assessment methods

A warning the following discussion requires a clear understanding of the logical distinction between whether a condition is “necessary” or “sufficient”. The confusion arises from the neglect, when defining units of competence, to distinguish between conditions that are “necessary” components of competence and those that are best thought of as “sufficient”.

Instead, I am referring to the systematic confusion in the core concept of “Competence” as it is applied in the Australian VET sector. It doesn’t, and it can’t resolve the fundamental problem. I am not talking here about “reasonable adjustment” which is a practical technique to resolve the fundamental dilemma when working in the field. The “rigor” of the procedure also supports the reliability across assessors.īut what if the demands of the procedure compromise the validity of the assessment? There is a trade-off but Is there an optimal “balance” between flexibility and reliability? “Good” RTOs design assessment tools to impose a rigorous procedure to ensure only those that demonstrate performance that meets the “standard” are deemed competent. We know that in Competency-Based Assessment (CBA) the “criteria” is the standard of performance.

There is a problem here - and, in my opinion, it is endemic in the Australian VET sector. I was told, in no uncertain terms that 'That's not the way we do things around here!'. Some years ago, as a new assessor at a distinguished, national RTO I was reprimanded by the compliance manager for judging a candidate competent when they had clearly violated the RTO's prescribed format for document submission. But now I think it’s the biggest load of BS that has ever poisoned the Australian VET system. A sage old builder said to me “There are lots of ways to complete the job but there’s only one right way”.

Validity and reliability in assessment methods