Merit versus diversity in NHMRC grants

The NHMRC discussion paper reads like polemic. The data analysis is one-sided and incomplete. The only options considered are four different versions of affirmative action The possibility of just waiting for more females to wash through the STEMM population was not countenanced at all. There was no consideration of the possible counterarguments that I have listed above. It is hard to avoid the conclusion that this was all part of a Twitter and change.org motivated activist movement.

The role of the council is primarily to allocate tax-payer funded grants to those projects that will deliver the greatest benefits. Their primary responsibility is to ensure robust allocation of funds through robust evaluation of projects and researchers. A broader interpretation of their role might include medium term incentives to attract the best applications.

So can they assure us that their assessments are indeed robust and unbiased? No, they really can't.

They should have audited fully all their assessment processes. Are they biased towards any identity group? Are they biased towards consensus positions, whether these be traditional or more modern? How large are these biases?

It is still not too late. They should start an audit right now!

But I am not optimistic, because I have found that those who claim systemic bias in institutional systems are often loathe to investigate where it is and how strong it is. Rather, they tend to dismiss the possibility of merit-based assessment at all and fall back on crude statistical arguments that any deviations from demographic benchmarks indicate bias. So grants should be 50-50 by sex.

The NHMRC should present evidence of any bias in (a) project quality assessment, (c) researcher track record and (c) grant success rates. They have the data on the quality assessment of every project, on a 1-7 scale. To what extent does this assessment depend on the sex and age of the assessor as well as how close the project is to their own research area? Do final success rates depend on the gender or age or subject matter of the applicant, over and above their track record and project quality assessment?

NHMRC are experts in this kind of data analytic assessment.

This failure to audit and improve the assessment processes is the opportunity cost of focusing on gender counts. But fine grain policy analysis doesn't generate outrage or change.org petitions. As things stand, can we be confident that the NHMRC are doing their best to fund and encourage the best research?