Slow interview #1 – An interview with Paolo Cherubini

Author: Sofia Pagliarin

Impact Factor: how a useful tool turned into fever

Paolo Cherubini is a senior scientist at the Swiss Federal Research Institute WSL studying forest ecological processes using tree rings, i.e., dendroecology. Dendroecology provides information that can help us understand and predict how our forests and environment have changed over centuries, from carbon dioxide concentrations to warm or cold, dry or wet weather conditions.

I met Paolo Cherubini when I was working as a post-doc at the Swiss Federal Research Institute WSL. During an informal event, I discovered that in 2008, Paolo published a letter titled “Impact Factor Fever”  in Science (Vol. 322), where he strongly criticised the abuse and misuse of bibliometric in evaluating academic life and performance. He also told me different stories about the Impact Factor and his opinion about it.

As a contributor to the Slow Science network, I hence got the idea to organise in written form those early conversations. Actually, we scheduled a Skype interview where we talked about his thoughts about the Impact Factor, its increasing importance in academia and his experience as an author, reviewer and editor with it.

The following text is a synthetic re-elaboration of these interviews, structured in two posts and reviewed together with Paolo before its online publication on the Slow Science blog.

paolo1
Paolo Cherubini

Sofia: Paolo, first of all, what is the Impact Factor?

Paolo: The Impact Factor is a measure that calculates how frequently articles published in a journal are cited. The Impact Factor of a journal for a certain year (e.g., 2015) is calculated by dividing the number of citations accumulated by all citeable items, e.g., articles, reviews, personal commentaries, of this journal with the average of all published citeable items in the two previous years (e.g., 2013-2014). So, if a journal had 100 citations over 50 published items, its Impact Factor will be 2. The Impact Factor is a great tool that was developed by Eugene K. Garfield, the owner and founder of the Institute of Scientific Information (ISI), when staff was still typewriting and comparing the number of times the publications in a certain journal were cited by other journals, and now run by Thompson Reuters.

Sofia: In your letter to Science published 2008, you wrote that “The exacerbated pressure to publish we all suffer from is induced by an exaggerated reverence for the impact factor”. How did this all happen?

Paolo: Well, the story is long but basically the Impact Factor comes out of a true necessity to navigate through the different journals and rank them according to their reputation and utility. Although it is a measure that changes over time, it can give a good idea of how important a certain journal is for scientific publications in a certain domain, based on the ratio of citations of the journal articles.

However, this practical need for comparing international scholarly journals has “backfired” and evolved into a way to evaluate scientific performance, institutes and personal careers and a “citation rush”. So today not only journals are fighting to increase their Impact Factor, but also authors crave for the quantification of their research publication records. Personally, I am not against the Impact Factor; as an editor-in-chief of a small journal, I care about it for my journal, but I also know that it’s a measure that does not tell everything about a journal.

Sofia: But why is there such a “reverence”?

Paolo: Because the quantification of scientific results through the Impact Factor is extremely effective for decision-making: it is easy to use. From a measure useful to rank journals, the impact factor now is used to decide which departments have to die and which ones can survive, live or are “of excellence” and be funded. It is a measure that is functional if one has to demonstrate that “objective” decisions have been taken on who should be hired, who be funded and who be prized because of high scientific productivity.

The Impact Factor is functional to maintain a certain structure and functioning of academia, which is beneficial to many parties, that’s why I call it reverence. The journals gain from it as true “brands” in a scientific sector, and the scientists and universities gain, because they all compete for funding and the Impact Factor serves as objective quantitative measure to figure out who has been a good or a bad scientist. In a way, “it’s a simple game for simple minds”.

Sofia: Can you provide an example?

Paolo: Yes, sure. The Impact Factor serves specific academic structures and arrangements. For instance, the supervisor always has to be included in the publications of her/his post-docs and doctoral students. This is not only to increase her/his productivity record, but also because without the supervisor having such a productivity measure, s/he will not be able to get funding, and nurture this hierarchical structure of post-docs and PhDs who otherwise will not be working. Don’t get me wrong: this is not a problem per se, but of course it affects how academic research is done, the number of students in a certain department or research group, and their mentoring and supervising.

Another example is the impact this had on the social sciences, that have been forced to adapt to this parameter, created in and for the natural sciences; the social sciences have started to compete with the natural sciences for funding using the same measures and weapons, in spite of the fact that probably in the social sciences publishing a book makes more sense than publishing articles in ISI journals, making so the use of the Impact Factor perhaps less appropriate.

Sofia: So do you think we should get rid of it?

Paolo: Let’s say that the Impact Factor was invented for a certain purpose, that is ranking international scientific journals. Over time it altered into something that is almost a dictatorship in scientific research and production. It has been misused, and we should get rid of it being used in such a wrong way.

However, it is also true that the Impact Factor was a good thing in those university systems where competitiveness and productivity were not awarded. For instance, in Italy, where I originally come from, the traditional university system was based on professors making a career publishing one book every 10 or 20 years, or only a bunch of articles written in the local language in non-ISI journals. This corrupts the system and makes it extremely inefficient and not innovative at all, which is especially frustrating for students and for the careers of new researchers. It was a system that worked autoreferentially for decades, and recently, thanks to the Impact Factor culture, the university system has been changing. [See here for more background information on the effect of adopting metrices on self-citational practices in an Italian context]

Once I was giving a talk in a formerly Eastern European country, and when I said that I was critical about the Impact Factor, because it is not necessarily a good measure of good research, some colleagues got angry because they were trying to change the academic system there in order to put more pressure on the “dinosaurs” that did not publish and reward young researchers and professors who really did make an effort to compete on the international research scene.

So, it’s good to have a measure such as the Impact Factor that can tell us which journals are best and who is publishing the most. But the Janus-face of this is that, for instance in China, professors get paid thousands of euros more if they get published in one of those top-end journals in the Impact Factor list. This isn’t good, it can become a problem if academic research is solely oriented to get a publication done and cited: the quality of academic research will suffer.

Sofia: Thank you for your time, Paolo. Anything to add?

Paolo: Yes. As a dendroecologist and forest scientist I don’t think these are so far from the social sciences. Actually, ecological systems are very much connected to social sciences … So I believe we are all on the same boat, and the Impact Fair is currently misused in both the natural and the social sciences. Thank you for this opportunity to share with you these thoughts, that I have been discussing with so many other colleagues over the past two decades.

Sofia: Thanks to you Paolo!

 


 

“Slow Interviews” is a column published on the blog of the Slow Science network. The “Slow Interviews” articles/posts are conceived, written and reviewed by Sofia Pagliarin, as one of the collaborators of the Slow Science network. The publication and the content of each interview, in one or multiple posts, are discussed, reviewed and reciprocally agreed through a cooperative dialogue and effort taking place between Sofia Pagliarin, the interviewee(s) and some other members of the network.

These interviews have the aim to enrich the topics and debates that are central to the network, and are conceived to be in direct dialogue with the course on the critical analysis of academia and academic production organised annually by the Slow Science network: https://slowscience.be/our-doctoral-school/. They add to our understanding through adding different points of view, and are not intended to replace the topics and debates dealt with during the course. Interviewees are academics or informants that have experience and/or knowledge on a particular topic, and who are not necessarily related to either the Slow Science Network or the course.

Disclaimer: The views and opinions expressed in the “Slow Interviews” articles and blog posts are those of the authors and respondents and do not necessarily reflect an official policy or position of the Slow Science network.

 

Debate role of universities – Monday April 23rd at deBuren, Brussels

Dutch follows English

Dear all,

On Monday April 23rd , deBuren in collaboration with KU Leuven, UA, UGent and VUB are organizing a debate on the democratic legacy of 1968 and the role of universities then and now. The debate is organized as part of the doctoral school course “What does it mean to be a researcher in 21st century academia?”, but it is open to a broader audience. For everyone interested in slow science and/or the societal role of universities, this event is not to be missed!

You can find all relevant information via the website of deBuren: https://www.deburen.eu/programma/4603/de-democratische-erfenis-van-68-waar-staat-de-universiteit-vandaag

 

Practical information:

Monday April 23rd, 19h30-21h
deBuren
Leopoldstraat 6, Brussels

Tickets (5€/3€) are available through the website. The event is free for those who follow the workshop.

 

We hope to meet you there!

 

——–

Beste allen,

Aanstaande maandag organiseert deBuren in samenwerking met KU Leuven, UA, Ugent en VUB een debat over de democratische erfenis van 1968 en de rol van de universiteit, zowel toen als nu.

Het debat kadert in de doctoral school workshop “What does it mean to be a researcher in 21st century academia?” maar is open voor publiek. Voor iedereen die geïnteresseerd is in slow science en/of de maatschappelijke rol van universiteiten is dit een niet te missen evenement!

Meer informatie is te vinden op de website van deBuren: https://www.deburen.eu/programma/4603/de-democratische-erfenis-van-68-waar-staat-de-universiteit-vandaag

 

Praktische informatie:

Maandag 23 april, 19u30-21u
deBuren
Leopoldstraat 6, Brussel

Tickets (5€/3€) zijn te verkrijgen via de website. Het debat is gratis voor wie de workshop volgt.

 

We hopen jullie daar te mogen verwelkomen!

UCU strike in the UK and how to help

At the moment, the University and College Union (UCU) organizes a strike to fight against the new pension plans. A majority of UK academic institutions are taking part in this strike. For more information and latest updates about the strike:

https://www.ucu.org.uk/strikeforuss

This issue is also relevant for the broader slow science movement, as it is part of the marketization of universities. To give only one but very pertinent example: final pensions would depend on how the stock market performs, not on contributions.

As one of the strikers puts it:

“The real problem behind pensions is this: Universities have borrowed billions in bonds to spend on fancy new buildings – they are becoming property developers. These bonds are then traded on the financial markets so that the more they are ‘de-risked’ the more they are worth. De-Risking the pension to the extent that its modelled on all universities going bust simultaneously isn’t a realistic expectation, but it is a theoretical risk that when transferred to individual academics increases the bond’s value. We are losing our pension security to make more money for bankers in other words.”

 

HOW CAN YOU HELP?

  • You can tweet your messages of support to @UCU (union nationally) or to the local initiatives of different universities.

 

  • Donate to the fighting fund of UCU: there are lots of precarious, early career academics, and single parents on strike who will need financial support for the wages they are losing. https://www.ucu.org.uk/fightingfund

 

  • The Southampton University, in the midst of the strike, has received an email to make sure to ‘prioritize the partner organisations of Southampton University as part of the Internationalisation Strategy’, of which KU Leuven is apparently the number 2 partner. Therefore we want to use this internationalization strategy to show our solidarity with our colleagues. We would like to collect pictures of you showing a card/paper with (something like) “I support my colleagues in the UCU-strike @ Southampton University [name, university]”. This will only take 5 minutes of your time, but it would be really great if we can collect a big number of pictures. They will be shown on the website of those who strike in Southampton. Of course, scholars from KU Leuven are especially encouraged, but a broad support from all colleagues is highly appreciated. You can send your picture to Valerie, and they will be sent collectively to the UCU strikers @ Southampton.

 

  • Boycott the institutions from this list, as they engage in punitive behaviour during and between strike days

 

  • All universities could benefit from your support, so please reach out to any of your colleagues on strike at the moment and ask how you can help them. We can post the actions on our website or help circulate through other means.

 

 

Universiteit Gent Women’s Strike 2018

WOMEN’S STRIKE Ghent University 2018
#WOMENSTRIKEUGent2018

The official website of the event is now available:
https://womenstrikeugent.wixsite.com/2018

Please read the demands and sign the petition:
https://womenstrikeugent.wixsite.com/2018/blog

We willen onze eisen kracht bijzetten door een feministische actie te organiseren op 8 maart 2018: Overeenkomstig de loonkloof, zullen we een “walk-out” organiseren om 15u27. Studenten en docenten worden gevraagd om de lessen te staken en samen met andere personeelsleden en sympathisanten te verzamelen voor het rectoraat, alwaar we graag een open “mic” willen houden, met koffie en een vieruurtje.
Wij hopen dat de Women’s Strike massaal op uw steun kan rekenen!

We want to assert our demands with a feminist action on March 8 2018: a walk-out at 15h27, symbolising the general gender pay gap (on average a woman is paid from 8h00 until 15h27 compared to a fully paid full work day of a man). Students and lecturers will be asked to leave their classes and join other staff and sympathizers at the rectorate, where we will hold an open “mic”, with coffee and a 4 o’clock snack.
Let’s stand together and strike for another university and a more equal and diverse community!

Our demands:

TO THE RECTOR AND VICE-RECTOR OF UGENT:

We strongly support your commitment to a more participatory and inclusive university, as expressed in your election campaign. We want to join you in working actively towards a more diverse and stress free workplace, with equal opportunities for all. We would like to remind you of our demands and propose the following actions:

1. DEDICATION TO EQUALITY

All deans and 75% of professors are men, and a majority of cleaning staff are women and people of colour. Let’s end sexual and racial segregation in all jobs and eradicate bias and privilege in recruitment:
elevate the minimum wages at our university!
stop the outsourcing of service jobs to put an end to precarious working conditions
install quota to increase the number of female professors to at 50%
develop programs to eradicate the barriers for underprivileged and minority groups in entering and continuing in the university

2. NO MORE SILENCE ABOUT VIOLENCE

Racist, sexist, ableist, trans- and homophobic discrimination have no place at the university. Let’s not be part of the silent majority that enables this kind of violence and speak up for one another! An open debate and transparent policy that takes unequal power relations at the university into account is the urgent and necessary thing to do.
set up an anonymous contact point and consultation centre to report not only sexual harassment but all discrimination and/or violence in the workplace!
mandatory gender and diversity training for all tenured staff!
unfund any organisation that promotes any form of discrimination or violence!

3. SAY YES TO LESS STRESS

Good-quality work requires healthy workers with a good work-life balance. We want “slow science” instead of “publish or perish.” Let’s reduce burnouts by reducing working hours, (administrative) demands, unpaid tasks and pointless competition for scarce funds.
promote a culture of collaboration and care instead of competition and control!
reform the current career model and system of personal goals!
Work towards reform the financial allocation model for higher education

4. LET’S PRACTICE WHAT WE PREACH

Ghent University proclaims to be a socially committed, pluralistic university, with critical thinking as its baseline. Let’s practice what we preach and actively work towards a more caring and diverse university:

we want more effective support for maternity, parent, caring and sickness leave, for all levels, all contracts!
reinstate the recently abolished coverage of anti-conception and abortion by the UGent health insurance plan!
decolonize the university by funding relevant programs and courses and promote feminist and postcolonial mentality that embraces differences!

universiteit gent women's strike

Waarom het debat over de universiteiten iedereen aanbelangt

Capture
This piece was published on Knack

Het Slow Science Network bestudeert het academische milieu waarin we ons als jonge onderzoekers bevinden. Onze voornaamste bezorgdheid daarbij is de balans tussen de verschillende taken van de universiteit: onderwijs, onderzoek, en maatschappelijke dienstverlening. Met dit stuk willen we onze stem toevoegen aan de roep om het huidige financieringsmodel te herdenken. Tegelijkertijd willen we ook enkele kanttekeningen plaatsen bij de manier waarop het debat tot nu toe gevoerd is.

Reeds tien jaar verschijnen er regelmatig verhalen in de media over problematische situaties aan de universiteit. Hierbij ging het bijvoorbeeld om problemen met het welbevinden op de werkvloer, de afschaffing van studierichtingen, wetenschapsfraude en andere bedenkelijke praktijken, en een publicatiedruk die het nemen van dergelijke risico’s zou aanmoedigen. Ook de afgelopen weken kwamen dergelijke thema’s weer aan bod.

De rode draad is steeds de rol van het financieringsmodel van onze universiteiten. Hierbij is niet alleen de hoeveelheid geld, maar ook (en vooral) de manier waarop dat geld verdeeld wordt de oorzaak van de huidige malaise. Deze verdeling verloopt via verschillende kanalen, die als gemeenschappelijk kenmerk hebben dat de middelen verdeeld worden op basis van hoe de universiteiten scoren op een aantal kwantitatieve indicatoren zoals het aantal publicaties of doctorandi. Dat heeft natuurlijk als voordeel dat de verdeling van het geld tussen de verschillende universiteiten op een snelle en objectieve manier berekend kan worden. Het nadeel is echter dat dit in de praktijk leidt tot een blind nastreven van die indicatoren.

Wat de zaak nog problematischer maakt, is dat die middelen verdeeld worden vanuit een vaste pot. Dat wil zeggen: het te verdelen geld ligt al vast nog voor men naar de indicatoren kijkt. In het geval dat alle universiteiten er op vooruit gegaan zijn, zal het globaal toegekende geld niet mee toenemen. Dit heeft tot gevolg dat een universiteit die erin slaagt beter te scoren op bepaalde indicatoren, toch achteruit gaat wat betreft haar financiering wanneer anderen het nog beter doen. Een doorgedreven concurrentieklimaat is het gevolg. Universiteiten willen het niet alleen zelf beter doen, maar hopen ook dat andere universiteiten het minder goed doen.

Die analyse is niet nieuw. De problemen zijn welbekend en niemand is gelukkig met de huidige situatie. Verandering blijft echter uit. Niemand wil immers bij de verliezers zijn als het gewicht van de ene of de andere indicator bij de verdeelsleutels aangepast wordt. Op die manier zijn we in een situatie beland waarin iedereen het erover eens is dat er iets grondig veranderd moet worden, maar niemand daar concrete stappen toe zet. Deze patstelling kan opgelost worden door wanneer een instantie die de belangen van de individuele partijen overstijgt, de zaak in handen neemt. In dit geval is dat de overheid. We zien echter dat zij de zaak terug doorspeelt naar de universiteiten. Die leggen op hun beurt de verantwoordelijkheid bij het niveau van het bestuur en het onvermogen daar om beslissingen te nemen. De zwartepiet wordt doorgeschoven, met tragische gevolgen voor zowel de kwaliteit van ons onderwijs, de relevantie van ons onderzoek en de levensvatbaarheid van wetenschappelijk dienstverlening die niet direct geld in het laatje brengt.

Die situatie is echter niet louter een probleem van academici. Uiteraard zijn het zij die als eerste en op dagelijkse basis geconfronteerd worden met de beperkingen van het huidige systeem en daar soms een zware tol voor betalen. Tegelijkertijd mag men niet uit het oog verliezen dat het huidige systeem ook negatieve effecten heeft op het vermogen van de universiteit om haar drie kerntaken te vervullen: onderzoek, onderwijs en maatschappelijke dienstverlening. Op die manier is het huidige debat er een dat iedereen aanbelangt, niet alleen een handvol academici.

Neem nu de focus op wetenschappelijke publicaties, waarschijnlijk de meest beruchte indicator in de verdeelsleutel. De eenzijdige nadruk op publiceren in internationale tijdschriften leidt, in combinatie met de louter kwantitatieve benadering en de vermelde concurrentie, in de praktijk vaak tot een vervlakking van het onderzoek. Niet wat er gepubliceerd wordt, maar dat er gepubliceerd wordt, telt (letterlijk).

Onderzoeksonderwerpen worden vaak bepaald door de trends in de vaktijdschriften. Die internationale trends komen echter niet noodzakelijk overeen met de noden en vragen van de maatschappij die voor het onderzoek betaalt. Bovendien staat de nadruk op publicaties in internationale, Engelstalige tijdschriften de maatschappelijke verspreiding van aan de universiteiten geproduceerde kennis in de weg. Zo zijn er voorbeelden van Nederlandstalige vaktijdschriften die de boeken hebben moeten neerleggen wegens een gebrek aan publicaties. Onderzoek heeft aangewezen dat die vaktijdschriften, die bijvoorbeeld ook gelezen werden door leerkrachten, een belangrijke rol speelden in de verspreiding van onderzoeksresultaten naar een niet-academisch publiek. Meer algemeen heeft de nadruk op het tellen van publicaties tot gevolg dat er minder aandacht gaat naar alles wat niet meegeteld wordt. Communicatie aan een breder publiek of deelname aan het publieke debat komt in het huidige systeem bijna neer op gevaarlijke tijdsverspilling. Op die manier wordt er weer minder ingezet op het terugvloeien van kennis uit de universiteit naar de bredere maatschappij.

Dat universiteiten en haar activiteiten geëvalueerd worden, is normaal. Aangezien zij draaien op belastinggeld, moeten zij ook rekenschap kunnen afleggen voor wat zij met dat geld doen. De ironie van de huidige situatie echter is dat het huidige evaluatiesysteem er net voor zorgt dat de maatschappij minder waar krijgt voor haar geld. Het is dus hoog tijd om terug naar de tekentafel te gaan en met een beter model op de proppen te komen. Het is tijd voor onze rectoren om de kaarten samen te leggen en voor de ministers van Onderwijs en Wetenschap om het nodige leiderschap te tonen. Niet alleen omdat het huidige systeem nadelig is voor de academici, maar ook omdat het de universiteit ervan weerhoudt haar maatschappelijke rol te vervullen.

[Dit opiniestuk werd gepubliceerd op Knack.be.]