QS graduate employability rankings: What they measure and what they don’t

This article, by jason Brown and I, was originally published as a feature story in Campus Morning Mail.

We read with interest Angel Calderon’s CMM column on the new QS employability ranking. However we are not persuaded that those rankings are a valid measure of employability. In this response, we explain why.

Mr Calderon ends his column by encouraging readers to “look beyond the vagaries of the metrics of this ranking” and accept them as “valuable tools to drive social and labour market policy reforms”. But what are “vagaries of the metrics” if not gaps in their validity? By our analysis, the QS employability rankings are a thoroughly inadequate representation of what they claim to measure.

He correctly notes that employability is a slippery term, and that employment and career success is inherently tied to many kinds of social inequity. Despite this, he does not confront the fact that the QS employability rankings are a better measure of university prestige than they are of any defensible conceptualisation of employability. A previous close look at what the component metrics measure and what they don’t demonstrated this, as well as how the rankings actively disadvantage regional, online, and/or otherwise modest universities.

Our specific concerns include:

• “Employer reputation”: Only 1.92% of respondents are Australian firms and there does not seem to be any certainty that employers actually employ graduates from the universities they nominate

• “Alumni outcomes”: Considers only the highest of high-fliers, such as senior leadership of the top 500 firms in the world, global lists of influence by the likes of Forbes, Fortune, and Time, and prestigious medal and award winners. These are outcomes that will be removed by 20 years, at least, from those people’s initial university degree. This measure disregards the more humble aspirations and achievements of most students, and nurses and teachers may as well not exist

• “Partnership with employers”: Excludes government agencies, such as education and healthcare, and privileges fast-track job applications and work experience arrangements, which are not a feature of Australian graduate recruitment

• “Employer-student connections”: privileges on-campus and discounts online activities, in effect discriminating against regional universities and those that serve part-time students with work and caring responsibilities.

• “Graduate employment rates”: perhaps the most valid and transparent measure, but still one that conflates employability with employment

Most importantly, the imprecise use of term employability outcomes continues to confuse rather than inform. We argue that employment should not be considered a direct outcome of employability. An individual’s employability (i.e., their professional knowledge, skills, and attributes) can influence the quality of job that is obtained but does not necessarily predict the attainment of employment itself.

There are several factors that complicate the relationship between employability and employment. Firstly, in the absence of obtaining a desired graduate job, most people will accept other forms of employment to pay their bills. The 2020 Graduate Outcome Survey reports that 28 per cent of graduates perceive themselves to be overqualified for the job they hold. Secondly, an employment outcome requires the availability of suitable job vacancies in the labour market. 70 per cent of those graduates who say they are overqualified noted that they weren’t in more suitable work simply because there was none available.

Finally, there is recent research that questions the assumption that so-called employability skills and graduate qualities translate into employment outcomes. Using 110,000 responses to the Graduate Outcome Survey and Course Experience Questionnaire, Brown et al. (2021) demonstrated near zero correlation between graduates’ perceptions of their skills and qualities and employment outcomes. Rather, quality employment outcomes are achieved through complex self-regulatory job search behaviours over a sustained period, starting well before the end of a student’s degree.

These factors are fundamentally outside the influence of universities. So, why do we measure universities’ performance on the employment outcomes of their graduates? And how do we justify comparing the employment outcomes of students in regional Queensland, for example, with those studying the same field of study in Sydney and Melbourne, let alone in Beijing or Los Angeles?

Universities should certainly be responsible for supporting the careers and employability learning of their graduates. But we need to do with much more meaningful and precise concepts and metrics for understanding employability outcomes than those offered by the QS rankings.