AI-powered assessment: focusing higher education on what really matters

AI has triggered a paradigm shift in university assessment. While the effectiveness of essays and multiple choice questions at assessing learning has long been in question, students can now easily use AI to generate effective responses.

The initial reaction was to attempt to ban students from using AI in assessment. But there is a growing realisation that this stifles a much more important debate about what higher education is actually for, and how it needs to adapt to remain relevant (Aoun, 2018).

How should faculty and universities adapt assessment?

Traditionally, the purpose of summative assessment has been to enable universities to determine whether or not a student has acquired sufficient knowledge and skills to be awarded a credential. This has made its primary purpose ‘assessment security’ rather than ‘assessment for learning’ (Kramm & McKenna, 2023).

But as AI continues to reshape the landscape of education and business, universities have a duty to develop students’ AI literacy if they are to prepare them effectively for employment (Brew, Taylor, Havermann & Nerantzi, 2023).

5 ways to incorporate AI in assessment

1. Create scenarios

A common criticism of assessment in higher education is that it fails to prepare students effectively for real-life situations. By using AI to create scenarios, faculty can quickly frame assessments in a more realistic context. For example, an assessment on a business course could position students in a fictional company facing a specific business challenge.

As part of the assessment, the student has to analyse data and current trends to make predictions. Using AI, faculty can quickly build out simulated conversations with different stakeholders across the business. In this way, the student has to think about what is the best approach for this set of stakeholders in this particular context, rather than in business more generally.

The assessment could also ask students to use AI to conduct initial research, then synthesise this with relevant academic literature to confirm or contest the output from the AI. To model a workplace scenario, the student could then present their findings in a simulated business meeting with their line manager to help them practise their presentation skills.

Using AI to build out a scenario makes the assessment more realistic, while also testing high-level skills such as synthesis and communication.

2. Use different formats

AI makes it easy for students to generate text-based assignments. Instead, ask students to submit their work for assessment in formats that also test their ability to present and format information appropriate.

Video and audio presentations focus assessment on students’ ability to communicate effectively. Encourage students to use AI to develop their initial findings, and then ask them to present their views on what they discovered as a video or audio file. In this way, faculty are more able to see and assess what students have learned through their research.

Portfolios are another format that enable more transparent and robust assessment. Again, encourage students to use AI to respond to a prompt or question, and then record their responses in a portfolio. Using AI, students are able to discover information quickly. But by shifting the focus of assessment onto how they compare, contrast and evaluate this information, faculty are able to assess higher level skills of evaluating, debating and interpreting.

3. Assess critical thinking, metacognition and process

One key purpose of higher education is to develop students’ ability to think critically and evaluate. But too often, assessments simply test students’ ability to memorise and reproduce information.

AI enables faculty to shift the focus of assessment towards metacognition, which is a much more valuable attribute. Instead of focusing on the ‘product’ of learning, such as the essay or the test, assessment should enable students to reflect on the process they followed to respond to a question or task. We can then ask them to reflect on the implications of taking a different approach, or asking a different question. We can even ask them to develop different AI prompts, and evaluate the implications of changing specific words.

Designing assessments that encourage students to use AI enables us to assess their decision-making skills, and their ability to evaluate the implications of their choices. This is arguably a much more valuable use of assessment than testing their ability to recall information.

4. Focus on evaluation and verification using academic sources

At the time of writing, AI is unable to access academic databases. This should be brilliant news for faculty, because it enables us to place much greater emphasis on the importance of using relevant academic literature.

For example, we can ask students to use AI to quickly generate a range of responses to a problem. Then, we can ask them to find and use relevant academic sources to verify the accuracy of the AI-generated information. This is hugely important, and a key strategy for developing students’ ability to think critically and evaluate information. If one purpose of universities is to develop students’ AI literacy, then placing more emphasis on the importance of academic referencing plays an important role in achieving this aim.

5. Assess interpersonal skills

Another common criticism of higher education is that graduates are not ready for the workplace. Functioning effectively in a business requires graduates to have strong competencies in collaboration, negotiation, teamwork, conflict resolution and leadership.

Encouraging students to use AI to quickly develop work enables us to use assessment to assess these competencies. Creating team-based assignments that encourage students to use AI to respond to a problem or question means we can assess each student’s approach to collaboration. We can use peer assessment to help them learn how to critique each other’s work and provide constructive feedback. And we can assess the extent to which they are able to manage conflict, support their team members and take the lead where necessary.

Using assessment in this way is more likely to develop graduates who possess the interpersonal skills required to be effective as they move beyond university.

Advice for designing AI-proof assessment

While there are many ways to redesign assessments in response to AI, three effective strategies are:

1. Set clear learning outcomes

Begin by clearly defining the learning outcomes for each assessment. Each outcome should clearly state what students should be able to do at the end of the course, programme or project. Begin each outcome with a suitable verb from Bloom’s Taxonomy such as ‘evaluate’, ‘apply’, ‘negotiate’, or ‘debate’ to ensure assessment focuses on process and not on product.

2. Diversify assessment formats

Ask students to submit their work in ways that make their learning visible. Presentations, portfolios, critical reflections, debates, panel discussions and podcasts all make it much easier for faculty to see what a student has actually learned.

3. Provide guidance on AI use

Provide Guidance on AI Use: Encourage students to use AI in their assessed work, and provide explicit instructions on how you expect them to use AI tools (see Ethan Mollick’s AI policy). Additionally, offer guidance on how to interpret and apply AI-generated feedback to their study strategies.