Benchmarking Metrics: What and Why?

In his post entitled "Top 10 theory schools?", Jonathan Katz gives a list of what he considers to be the top 10 theory departments in the US. (Read the post and the comments for an update on the discussion.) One of the comments reads as follows:

I would agree about:


MIT (Silvio, Shafi, Ron, Michel, …)
Cornell (Rafael, Eva, Jon, Bobby,…)
Berkeley (Luca, Christos, Umesh,…)
CMU (Venkat, Manuel, Avrim, Ryan,…)
Princeton (Boaz, Sanjeev, Moses, Bernard,..)

The others are a bit flakier:

GA Tech (Santosh, Chris, Vijay, Sasha, …)
UT Austin (Adam, David, Brent)
UCSD (Mihir, 1/4 Russell, Daniele, maybe Hovav)
U Washington (Anna, Paul, Anup)

I would say at least he following schools are VERY comparable to above:

Stamford (Dan, Serge, Tim, Amin,…)
NYU (Subhash, Assaf, Yevgeniy, Richard, …)
Harvard (Salil, Michael(s), Leslie)
Columbia (Mihalis, Rocco, Tal)

I would say there are top 5, and then top 10 following them.

So this commentator measures the quality of a theory group using the perceived research quality of the very best researchers at a given institution. Another possible metric would be the size of the group of active researchers with high international visibility. Yet others could be the number of peer-reviewed publications in top-class journals and conferences, or the average such number per staff member.

What criteria are most useful and why (bearing in mind that, at the end of the day, we are always making subjective judgements)? I am interested in this topic since the School of Computer Science I am working at is presently undertaking a benchmarking exercise. The aim of the exercise is to find three to four departments in the Nordic countries with which we aim at comparing ourselves in the very short term, within five years and within 10-15 years. One of the interesting aspects of this exercise is that our school is substantially smaller than most of the departments elsewhere. So, what do you think would be good metrics for the benchmarking exercise? Standing of the top scientists within the school? Average number of peer-reviewed publications and citations? Or what?

EATCS Award 2010: Call for Nominations

The Call for Nominations for the EATCS Award 2010 has been published (see this pdf). Nominations and supporting data should be sent to the chairman of the EATCS Awards Committee, Emo Welzl. The next award is to be presented during ICALP'2010 in Bordeaux. The deadline for nominations is: December 15, 2009. So get your act together quickly and nominate one of the many "obvious suspects" for the award! 

Note that nominations are now kept alive for three years. Since I sent in an unsuccessful nomination last year, I don't need to do anything this time around and will just be a very interested observer :-)


I wish the award committee the best of luck with their work.

Five Years of Logical Methods in Computer Science

I have received this letter via email. I post it here since I strongly support journals like LMCS and I encourage my readers to submit good papers to it. Happy birthday LMCS! May your reputation grow with the years.



Dear Colleague:

We would like to bring the community up to date on the journal

   Logical Methods in Computer Science
   www.lmcs-online.org

We started this fully refereed, open access, free electronic journal in January 2005, intending to create a high-level platform for publications in all theoretical and practical areas in computer science involving logical methods, taken in a broad sense. We are now on Issue 3 of Volume 5 (there are four issues a year). So far, we have received more than 350 submissions of which we have published 162. In addition to individual submissions, our journal publishes special issues, e.g., of selected papers of high-level international conferences such as LICS, IJCAR, CAV, CSL, and RTA.

We are continuing actively to develop the journal. For example, we accept survey articles, and are developing `live' surveys, which can be continually updated as knowledge progresses. In another direction, we are considering allowing authors to provide additional material of an expository nature, such as slides and videos, to enable them to interest a wider spectrum of readers in their contribution.

The journal is an overlay of CoRR, the computer science repository of arXiv. There are no fees for authors nor for readers. Every paper is refereed by two or more referees, and high standards are applied. The editorial board consists of about sixty top specialists in all areas of logic in computer science.

The journal is covered by Mathematical Reviews, the ISI Web of Knowledge, and the DBLP Database.

We welcome your comments and suggestions, and we seek your contributions! For more information please consult our web pages:

     www.lmcs-online.org

Yours,

Editor-in-Chief:   Dana S. Scott <dana.scott@cs.cmu.edu>
Managing Editors:  Benjamin C. Pierce <bcpierce@cis.upenn.edu>
                  Gordon D. Plotkin <gdp@inf.ed.ac.uk>
                  Moshe Y. Vardi <vardi@cs.rice.edu>
Executive Editors: Jiri Adamek <adamek@iti.cs.tu-bs.de>
                  Stefan Milius  <s.milius@tu-bs.de>

IFIP 1.8 workshop on FORMAL METHODS FOR EMBEDDED SYSTEMS

I have received the following announcement from Bas Luttik. I am happy to post it here since the event will be of interest to concurrency theorists and people working on formal methods at large, many of whom will be in Eindhoven for FM week.  (Not to mention the fact that I am the outgoing chair of IFIP WG 1.8.)

==============================

========================================
                       CALL FOR PARTICIPATION
======================================================================
      IFIP 1.8 workshop on FORMAL METHODS FOR EMBEDDED SYSTEMS

               http://www.cse.unsw.edu.au/~rvg/FMES/

           Eindhoven, The Netherlands, November 5, 2009
======================================================================

This IFIP 1.8 workshop is organised as part of the Formal Methods Week,
which takes place in Eindhoven from November 2 until November 6, 2009.
The goal of the workshop is to summarise research from different areas
of formal methods targeted to embedded systems, and to promote the use
of formal methods in different applications and in the engineering
discipline for embedded systems.

PROGRAMME:
----------
 8:30  Registration and coffee
 8:50  Opening

 9:00  Bert van Beek -
         The Compositional Interchange Format: concepts, formal basis,
         and applications
 9:45  Holger Hermanns -
         Synchronous vs. Asynchronous Performance Models of Industrial
         Networks on Chip Designs

 10:30 Coffee break

 11:00 Catuscia Palamidessi -
         Synchronization in the pi-calculus
 11:45 Joost-Pieter Katoen -
         Analysis and Semantics of Extended AADL Models

 12:30 Lunch
 14:00 The End

For abstracts of the talks and further details about the workshop we
refer to http://www.cse.unsw.edu.au/~rvg/FMES/.

REGISTRATION
------------
The registration fee for the workshop is 45 euros and covers coffee/tea
and lunch. You also need to register for FMweek, which costs an
additional 35 euros (administration costs). Please register via the
FMweek website: http://www.win.tue.nl/fmweek

WORKSHOP ORGANISERS:
---------------------
Rob van Glabbeek (National ICT Australia)
Ursula Goltz (Technical University Braunschweig, Germany)
Bas Luttik (Technische Universiteit Eindhoven, The Netherlands)
Uwe Nestmann (Technical University Berlin, Germany)

Presburger Award

In an old post I announced that the EATCS was about to initiate a young researcher award. I am happy to see that the call for nominations for the first Presburger Award has now been advertised. See here for the details. The award is meant for scientists in TCS who are 35 or younger.

The award is named after Mojzesz Presburger who accomplished his ground-breaking work on decidability of the theory of addition (which today is called Presburger arithmetic) as a student in 1929.

The award includes an amount of 1000 € and an invitation to ICALP 2010 for a lecture.

I hope you will take the time to nominate young scientists for the award. Who would be your favourite candidates?