How does one design assessments responsibly? Tests are designed with reference to certain principles and requirements. These principles are given shape in the actual instrument which is being developed, and theory allows us to identify at least 14 of them as fundamentals.
BROWN BAG SEMINAR
At the invitation of Kenneth Ong of the Singapore Institute of Technology, I shall do a presentation on Friday this week to consider what responsible design means.
I intend to interpret the term ‘responsible’ in its original meaning. ‘Responsible’, applied to the case I shall be presenting, refers to taking an action of imagining, developing and shaping a language intervention in response to principles of design. Though the intervention I focus on is the design of a language test, the examples referred to will have useful applications in many other fields.
The presentation begins by focusing on those design principles that have been prominent in the literature on language assessment: homogeneity, reliability, validity, differentiation, authenticity, implementability, usefulness, accountability, and fairness. They are so basic and fundamental that responsible assessment design is unthinkable without them.
Can AI be useful?
I specifically examine how authenticity is achieved in devising a test of academic literacy and also identify techniques to measure language ability economically. An assessment technique will be demonstrated that is efficient and economical and can be productively created by an AI tool.
A framework of design principles
The significance of these discussions and demonstrations is that the design principles are clearly useful in other fields than language testing. The framework they offer can be expanded in range to inform the design of assessments in other disciplines.
RELATED ARTICLES:
