Academic evaluation and hiring in Italy: The curious incident of Giovanni Sambin in the ASN 2016

Disclaimer: This post might contain imprecisions about the ASN, since I have never worked at an Italian university myself and I have never applied for the Italian ASN. I welcome corrections from whoever reads this post and has experience with this Italian evaluation exercise. Let me state at the outset that what I write pertains to fields such as computer science and mathematics. I do not know what is done in the humanities. 

An Italian law dated 30 December 2010 specifies a procedure for academic hirings in Italy at the level of associate and full professor. According to that law, recruiting for those positions should be "based on scientific qualification criteria. A national commission evaluates and assesses the candidates scientific qualification." See this outdated web site, which should be compared with the one in Italian.  Only candidates that have obtained the so-called Abilitazione Scientifica Nazionale (ASN, National Scientific Qualification) can then apply for a professor position at an Italian university, if and when such positions are advertised.

One can wonder why Italy uses this two-step system for academic hirings, whose need is not felt in any of the countries where I have worked so far. I guess that the first step is meant to filter out potential candidates who do not meet minimum requirements for being a professor at any of the 63 public universities in Italy. 

According to the regulations a "national commission evaluates and assesses the candidates scientific qualification." In fact, there is one national commission for each of the many scientific areas considered in Italy. Many of these commissions have to examine hundreds of applications, and their members play the role of gatekeepers and paladins of quality in the Italian university system. I can only imagine how much work is needed to do a thoughtful job in one of those committees and how easy it is to make enemies regardless of how considerate one is in justifying one's opinions. The evaluation is partly based on bibliometric criteria, which are known beforehand, should simplify the work of the commissions and should give a look of objectivity to their decisions. However, as far as I know, the commissions can also base their decisions on a qualitative analysis of the applicants.

Given the crucial role played by the members of the evaluation committees, one would expect that their members are chosen by taking the candidates' scientific profile and experience carefully into account. As it turns out, however, the qualifications of candidates for the committee are evaluated using only the following three bibliometric criteria:

1. Number of publications in the period 2006-2016 (threshold 9);
2. Total number of citations  in the period 2001-2015 (threshold 80);
3. H-index in the last 15 years (threshold 5). 

In order to be eligible, one has to meet the thresholds in at least two of the above criteria. This might even seem reasonable. Note, however, that only publications indexed in Web of Science or Scopus count. In particular, journal papers published in outlets that are not indexed by Web of Science/Scopus are not taken into consideration (regardless of their content and impact) and conference papers don't count at all. Books and monographs don't count either, regardless of how influential they might be. Web of Science/Scopus are also used for calculating citations and the h-index. Again, this provides a smaller coverage than the one offered by Google Scholar, say.

By way of example, recently Giovanni Sambin, one of the most famous, currently active Italian logicians and an expert academic  one would trust to lead a national evaluation committee for Mathematical Logic, was considered to be ineligible as an evaluator because he met only one of the above-mentioned criteria. His Google Scholar profile is here.

This kind of decisions makes me wonder whether there is an overemphasis on bibliometric evaluations in Italian academia. If experience over a long and distinguished academic career plays second fiddle to fairly arbitrary thresholds calculated using only Web of Science and Scopus, I wonder how reliable the decisions of the evaluation committees will be considered by Italian academics. Most importantly, having so many people spend a lot of time seeking the holy grail of the national qualification and small committees devote endless hours examining their qualifications looks like a huge waste of energy and resources. I cannot help but think that that energy and time would be best used for research, teaching and all the other tasks that make up our work.