Tag Archives: REF

Semantic Scholar: the AI which, knows your field better than you (and a Short Rant to Science Policy-makers)

“What if a cure for an intractable cancer is hidden within the tedious reports on thousands of clinical studies? In 20 years’ time, AI will be able to read — and more importantly, understand — scientific text. These AI readers will be able to connect the dots between disparate studies to identify novel hypotheses and to suggest experiments which would otherwise be missed.

AI-based discovery engines will help find the answers to science’s thorniest problems.”

— Oren Etzioni

“Semantic Scholar” is a better idea than the REF. But wait, what do they have in common? Here’s what.  Semantic scholar looks through existing scholarship and lets you use it and build up on it. It may uncover an article which no one has read many years after it was published, and let a new researcher learn something. It helps cut through the bullshit – perhaps not in the best possible way, but AIs are work in progress and will surely evolve into cleverer versions. The REF, conversely, fosters the production of bullshit. Its existence scares scientists into producing more crap (sorry, dear colleagues) because that’s the way in which we, and our university departments, are assessed. We all know it’s a game, and some refuse to play it, but people in their early careers have more incentive to play along than to protest by producing fewer, better pieces of work – even though it actually is in our long-term interests, we are fooled by fear. And so we write, and publish, instead of thinking and publishing less, better stuff.

If I were a science policy-maker, I’d put my money on tools that facilitate tedious, or downright impossible, tasks such as sifting, navigating and organising existing knowledge and debates.  And I would leave scientists with a bit more freedom to actually think, be curious, produce ideas, hypotheses and sometimes even knowledge, and – very importantly – also to make mistakes in the process. Oh, and having access to jobs that last longer than a year or two before having to move continent with or without your significant other, or instead of deciding to have children, would help. But that’s another rant, perhaps for a future post entitled “If I were a science policy-maker”.

For more about Semantic Scholar, see: https://www.newscientist.com/article/dn28434-ai-tool-scours-all-the-science-on-the-web-to-find-new-knowledge/

Tagged , , , ,

Is “embracing ordinariliness” really the only way to cope with the impossible demands of contemporary universities?

Another depressing article: astute analysis of the problems in contemporary academia (which affect both women and men, but the average man tends to have better invisible support in coping with them)…sadly followed by a call to “embrace ordinariliness”.


“Given that women still bear the brunt of responsibility for running households and organising the school schedules of children and so on, the question I was asking myself was how can women academics ever hope to achieve success in their working lives when this kind of pattern is seen as not just normal but entirely unremarkable, especially in a sector deemed by and large to be well-disposed towards working parents? Deciding not to have children, and having a partner who is also an academic or at least very familiar with these kinds of schedules would seem like the obvious answer.

the ideal career track in the academy especially one which carried all the laurels of prizes, awards, fellowships and a high volume of grants seemed to have been tailored around the image of the brilliant young man untrammelled by any of the fine details of domestic life. And if the young woman was to follow this pathway and plan the right time to have a child, then when would this right time be? The first few years of full time work (34-38) are marked by all kinds of expectations, and so it may be that just before getting to 40 having children could be embarked upon.”

Tagged , , , , , , , , , , ,

Killing me softly with your REF

If you work in an UK university you probably know about the REF (understatement). If you don’t, then you might be forgiven for thinking that it is an American black comedy from the 1990s (known in some countries as Hostile hostages). Although the film’s poster is strangely appropriate, that’s not it. Here is the official definition:  The Research Excellence Framework (REF) is the new system for assessing the quality of research in UK higher education institutions.” (http://www.ref.ac.uk, 18 Dec 2014).

I just discovered an article which apparently made quite a splash when it was first published in 2012. I agree with many of the criticisms in the first half of the text. The best point of the article is about the ambiguous nature of the REF. It chimes with something I’ve been trying to argue in conversations with British academics: namely, that the REF does at least have a good reason behind it, and in other countries where there exists tenure, but little or no accountability, professors can sometimes slack off and lie on their laurels instead of doing research, and that is bad for the careers of younger aspiring academics:

“This ambiguity, of processes which are managerialist, anti-collective, even oppressive, and at the same time potentially progressive, pervades both the REF, and academics’ engagements with it.  The whole evaluation of submissions is by panels of disciplinary peers,  resulting from the collective, if tacit, understanding that it is better done like this than any other way”

It turns out that things weren’t so different in the UK, only happened earlier. I hadn’t realised that the turn to Human Resource Management was initially brought about by well-meaning people who wanted to change old inefficient practices:

. […] Along with customer satisfaction surveys for lectures, an individualized, a per capita REF enables (it seems) the atomization of universities’ intellectual endeavour, and, the performance of each faculty member to be exposed, measured and judged, on its own terms, and against that of others. This is helped by the techniques of human resource management (HRM) – probationary periods, target setting, annual performance appraisals,  and performance related pay. HRM is also associated with disciplining, by both its proponents, and its Foucauldian critics. But against those who see mutual cause and effect in the simultaneous rise of HRM and Thatcherism, I counterpose my anecdotal experience of its practices being developed and realized in public sector by the political left.  When I worked for Leeds City Council in the 1980s, advocates of what now are seen as everyday HRM tools –  job descriptions, person specifications, annual appraisal – presented them as objective methods to counter racism and sexism. It was the stroppy, brave people from councils’ and health authorities’ equal opportunities units who drove the HRM movement, as much as it was privatising, union-busting right-to-managers.  When I left Leeds for my first proper HRM job at British Telecom in London, I found most of my colleagues developing its new entrepreneurial culture had a similar public sector background.”

The article also goes off on a very long rant about the forced disclosure of exceptional circumstances in the second half, which I don’t like as much. But it is very worth reading:

P.S. if you want a serious sociological critique of performance-based appraisal systems, check out Lisa Lucas’ 2006 book “The research game in academic life”. (It is based on her PhD thesis which, I was surprised to learn, was written at the department of sociology at Warwick!)

Or read these slides (I wish I knew who their author is!) which give a nice summary of Lucas’ book, as well as a useful bibliography.

Tagged , , , ,

I wonder…

Peter Higgs, Emeritus Professor at Edinburgh, who gave the name to the Higgs boson, has never sent an email. In a recent interview for the Guardian he called himself “an embarrassment to the department when they did research assessment exercises” and said that it’s “difficult to imagine how I would ever have enough peace and quiet in the present sort of climate to do what I did in 1964.”

I was just watching some lectures by Richard Feynman with my office lunch. I don’t understand most of the physics… yet… but I can now see that he was not only a great writer but also a great lecturer. That made me wonder what would Feynman say about the REF (UK research excellence framework). Actually, having read a couple of his books, I don’t really wonder at all. I think he would have called it “bullshit” and gone on to produce research with far more impact than most of us ever dream of producing, and be very bad at “creating demonstrable impact”. I realise that universities can’t be run by people like Higgs and Feynman. But I do wish that impact assessment wasn’t actually impeding great research and stifling the flight of thought of scientists and researchers.

Tagged , , , , ,
%d bloggers like this: