This article focuses on two types of assessment validity which relate to transferable skills alongside language and exam skills development: consequential and cognitive validity.
How closely does the language tested align with the skills that will be necessary for the test-taker in their lives beyond a test?
It is fair to say that most test-takers, and most teachers whose students are preparing for examinations in English, see the test event as the final point in their academic process. This is especially true of students and teachers working towards high-stakes barrier tests such as IELTS, TOEFL, or other exams which are used for entry into universities or as proof of language ability for immigration purposes. However, this kind of test is far from the end of an academic process; in fact, it is the beginning of a much more important part of the test-takers’ lives: migration to live and work in a new culture, or an extended period of overseas study. These post-test situations have recently played a part in assessment design, and are referred to as the consequence of the test event.
Academic consequences and consequential outcomes
Just as a test results in outcomes for a student (a passing grade, or a measure of their performance on the test), the test-taker’s result has effects on the following activity which it enables him or her to perform. For example, a score of band 4.5 in an IELTS test may enable a student to enter a foundation year or bridging programme at a UK university, but will not enable them to begin an undergraduate degree. The possibilities that a test-taker is presented with upon receiving their result are known as consequential outcomes. This is a term which has gained importance in recent years due to the increase in the range and type of international study setting which growing numbers of international students can opt for.
Taking consequential outcomes into account is essential for any language assessment activity which is designed to enable students to move forward in their academic or personal life. For this reason, assessment design should ensure that there is a strong connection, or alignment, between what the test-takers are being asked to do on the test and the types of cognitive, social and linguistic activity that they will need to perform in their post-test situation (their chosen professional, academic or personal goals beyond the test itself). The closeness of this alignment is known as consequential validity, and with a carefully-designed test construct, can be used to help students develop key skills for success in their future lives, through the preparation that they undergo prior to the test. This idea will be developed further in the next article in this series, concerning the washback effect in language assessment.
How closely do the interactional and cognitive processes demonstrated by test-takers relate to the types of thinking and interaction that the test-taker will perform in their experience beyond the test?
This is an important question, which is often levelled as a criticism of high-stakes language assessments. A single test construct, used by millions of test-takers every year in countries all over the world, is unlikely to represent the diverse range of linguistic, cultural, social and functional needs that those test-takers have for their various life and study goals. Testing vocabulary and grammar knowledge, and performance in the four skills of reading, writing, speaking and listening is relatively easy through standardised testing, but enrolment and successful performance in an overseas degree programme, for example, requires a lot more than knowledge of language and accurate understanding and use of grammar and vocabulary. If this were the case, then all first-language speakers of English would succeed without difficulty in English-language university settings.
Language ability vs communicative competence
If we take language out of the equation, what other features of a university student lead to achievement in higher education? What makes a ‘good student’ at university? There is a raft of non-linguistic skills which help language-users of any discipline to achieve their communicative goals, and which can be defined and tested alongside language, ensuring deeper alignment between testing and future performance. One framework which aims to define such a set of skills is Bachman and Palmer’s breakdown of communicative competences - features of language use which lead to more effective all-round performance in a language. Bachman and Palmer divide communication into four areas: Linguistic competences, which cover the grammar, vocabulary and skills as mentioned above; sociolinguistic competences, which enable a speaker to engage effectively with people in a range of social settings, using language appropriately to achieve purposes in the situations where they find themselves in life; Discourse competences, which enable language users to organise and frame their message appropriately for their message to be communicated effectively, and to understand the framing and organisation of others’ messages in different communicative situations, and strategic competences, which allow language users to make appropriate linguistic choices to deal with the situations in which they find themselves. These competencies range from the linguistic to the social and cognitive, and represent a sound combination of skills for success in any situation, where language is used for any purpose, not just those laid out on a one-size-fits-all examination. This framework is a useful starting point for assessment design, as it identifies qualities in the language users themselves which can be developed for success, and are therefore testable if the assessment is designed accordingly, i.e. if the assessment construct mirrors the real-life thought processes and resulting language choices which are typically made in the consequential situation where the test-takers will find themselves.
What are communicative competences?
These competences overlap greatly with what have become known as ‘21st century skills’ (the skill-set which is desirable for success in the fast-moving work environment of the 21st century). These include the ability to take initiative with language, to analyse and interpret ideas beyond surface meaning, to change the way information is communicated depending on aspects of genre (in reading and writing) and communicative setting (in speaking and listening), to deal with unknown ideas and language, to manage errors and mistakes, to reflect on learning and development processes, to combine and communicate ideas from different sources effectively… the list goes on. As you can see, these skills, as performed in a second language, depend on, but are not defined by, a test-taker’s linguistic ability. This means that they can be tested alongside language through test items which are designed to elicit these behaviours.
This alignment between the internal processes undertaken by a test-taker when they are engaging with test items, and those required for success in their post-test situation is known as cognitive validity, an aspect of contemporary assessment design which is relatively new, and has only been applied in a handful of test constructs which are used today. Cognitive validity can be designed in to a test to allow for assessment of the thinking processes which are necessary for effective performance at work, university, or just in living in a country where a given language is spoken. This in turn can provide a much more realistic gauge of to what extent the test-taker is prepared for their future life in a more holistic way. An assessment with high cognitive validity can prevent the all-too common phenomenon of students arriving at a British or American university, for example, bearing acceptable entry test scores as defined by that institution, but being unable to function linguistically or socially in that new environment.
In summary, contemporary language assessment has gone beyond the testing of simple language knowledge, and has moved towards a more holistic view of language learners, taking into account their future goals, applications of English and the ways in which they will need to think, behave and use language beyond the test situation itself. These aspects of language testing are an integral step towards a higher-functioning, happier and more prepared international student body.
In part 5 of this article series, we will look at the ways in which language assessment can be harnessed as a tool for language development, and the ways in which test design can influence the quality of learning leading up to the test event: the washback effect.