ESRC Peer Review Process

As I said in my the previous post, I took part in a PGCHE mock panel today. I preparation I did some background reading on the ESRC process, and thought it would be interesting to set down the way that they assess applications.

•   First Stage – ESRC receive the application

o   Roughly 10% of applications get rejected at this stage on technicalities, such as not having the right attachments, sections not being filled, format not being adhered to, etc

•  Second Stage – External Reviewers

o   Each application gets sent to at least 3 academic reviewers, and a user reviewer (if relevant). These are identified using key words.

o   Reviewers use a scale of 1 (low) -6 (high) to rate the application.

o   Applications are assessed on:

§  Originality and potential contribution to knowledge;

§  Research design and methods;

§  Value for money;

§  Outputs, dissemination and impact;

§  Scheme specific criteria (not relevant to responsive mode).

o   Any application with an average score of less than 4/6 it will be rejected at this stage. This applies to about 30% of applications

•  Third Stage – Introducers

o   The remaining applications are allocated to Grant Assessment Panel (GAP) members who will act as 'introducers' at the panel meeting. There are usually two introducers per application.

o   The ESRC tries to match the applications to the GAP member with the most relevant experience. However, there are only three GAPs, which cover a wide range of disciplines, so applications may well be introduced by someone with limited knowledge or understanding of the discipline.

§  The disciplines covered by the three GAPs are:

GAP A Education  Psychology  Linguistics

GAP B Sociology  Social Work  Social Policy  Social Legal  Area Studies  Anthropology  Statistics and Methods  Politics and International Studies  Science and Technology Studies

GAP C Economics  Management  Demography  Environmental Planning  Geography  History

§  If it is felt that there is no one with relevant experience on the GAP, they can either cross refer to another GAP, or even to another Research Council.

o   Each introducer gets around 7-10 proposals to assess each meeting, and 4-5 weeks to write their assessments. Their assessments will highlight key strengths and identify any weaknesses that need to be addressed.

o   Based on this assessment, they rate the application using a scale of 1 (low) – 10 (high).

o   The ESRC will analyse these scores and work out approximately what score the applications need to have got in order to go forward to the GAP meeting. Roughly 30% of applications get rejected at this stage, and realistically only those scoring 6 or above are likely to be funded.

•  Fourth Stage – The Panel Meeting

o   Each application is introduced by the two GAP members. It is not always clear beforehand who will be the leading introducer and who will be the seconder.

o   The panel works through the applications in order of introducers’ scores: the ones with the highest score get discussed first, those with a lowest score get discussed last.

o   The Panel works with PDFs which save on paper but do make it difficult to refer back, check, and follow the discussion of applications easily.

o   ESRC officers are present, and do have input:

§  By saying roughly how many applications they can fund in that round;

§  By highlighting any problems with applications that have not been picked up before (eg they have been submitted to the Council before)

o   Whilst the panel will take a steer from the introducers, the panel discussions allow for proposals to be pulled up or down the rankings. Most of the discussion is around marginal or controversial proposals.

o   The Chair is key. S/he summarises the discussion and, if there is no consensus, has the final say.