12 January 2010 – Performance Measurement and Analysis Division Publication of DWP Research Report 616: International Review of Performance Management Systems in Public Employment Services

A report is published today by the Department for Work and Pensions on the findings to explore how other Public Employment Services (PES) across Europe use performance measurement in support of their organisational objectives in order that Jobcentre Plus can learn from this when considering future improvements to its performance measurement regime. The main aim of this review was to understand existing labour market targets and whether these would be appropriate for Jobcentre Plus.

The review was undertaken between December 2008 and February 2009 and included a review of the academic and policy related literature; an in-depth focus on a large number of European and other Public Employment Services; an online consultation with PES officials in Europe and noted experts and finally a small number of supplementary interviews with respondents in relation to five ‘case study’ PES performance measurement systems.

The research involved a literature review and a rapid online consultation survey was undertaken with all Directors of Public Employment Services in the European PES Directors network and with SYSDEM network¹ country experts.

[1] The SYSDEM network is a network of independent labour market experts which operates within the European Employment Observatory.  Its aim is to report on employment and labour market policy developments and statistical trends and to provide research and evaluation services to the European Commission. More information is available at: http://www.eu-employment-observatory.net/en/about/abt03_01.htm.

The key findings of the literature review were:

Performance measurement in the public sector has been implemented generally in the context of increasing privatisation of public services and as a mechanism to ensure that contractors or public sector arms length delivery organisations act in the ways that the purchaser of services would want.  Other prominent objectives underpinning performance measurement in the public sector relate to the potential to use performance data to aid public and political accountability.

Performance measurement systems take different types of performance indicators and attempt to link them together, particularly to show the relationship between inputs (the resources devoted to a particular issue), outputs (the immediate results of activity, such as the number of interventions completed) and outcomes (the ultimate effects of interventions).  Example performance measurement systems include ‘Balanced Score Cards’ and ‘Performance Prisms’.

Commentary on an international review of performance measurement

The review of performance measurement in comparative Public Employment Services in Europe, North America and Australia showed that a wide range of approaches to performance measurement are used. The core components of these are set out below in relation to input, output, intermediate outcome and final outcome measures.

Input measures relate to the presentation of resource information (e.g. budgets, staffing) alongside achieved performance, so as to enable a comparison of the relationship between resources and performance. While this is not widespread, some countries do adopt these measures and they are incorporated in the European PES Benchmarking Project and the European Employment Strategy approach.

Output measures count the volume and quality of activity and interventions undertaken by the PES. Common indicators in PES performance measurement systems include:

• Vacancy registrations – this commonly refers to the vacancies registered by employers with the PES and acts as a measure of the role of the PES in providing a passive information-based function in the labour market.  Some variants go further to identify measures of the quality or type of vacancies that they register.
• Interventions – Interventions related to the more active measures used by PESs on the supply-side of the labour market, usually (though not exclusively) with unemployed jobseekers or inactive benefit claimants.  These include measures of:
  • Activation interviews.
  • Referrals to particular employment or support programmes, including training and development programmes.
  • Work trials/intermediate employment programmes.
  • Individual plan completions setting out mutual expectations, labour market aspirations and the support needed to achieve these.
  • Sanctions – reductions in benefit payments as a result of not actively seeking employment.
• Penetration measures – These measure interventions in relation to a particular group of clients (e.g. the proportion of lone parents who have been provided with child care advice).
• Process quality measures – these tend to be less numerical and include qualitative checks on the quality of interviews undertaken, the plans completed for individual jobseekers or the use of feedback from customers on the quality of service delivery.

Outcome measures can be divided into two categories: Intermediate and Final outcomes.  Intermediate outcomes measure those effects that result directly from the activities of the PES.  Final outcome measures incorporate assessments of the overall and often long-term desired outcomes of PES activity.  Widely used intermediate outcomes include:

• General off-flows from benefits – these measures are common and tend just to record where individual benefit claimants terminate their claims.  A focus only on general off-flows can be misleading as the reasons for a claim termination include transitions into the informal economy, to another benefit, migration, to prison or even death, in addition to employment.
• Specific off-flows from benefits – these measures include a consideration of the destination of people leaving the unemployment register or benefit claimant count. Destinations commonly include some indication that this includes formal employment, but might also go further to measure separately different types or quality of employment achieved with a view to assessing the quality of outcome achieved by PES activity. They also commonly ensure a link back to a particular intervention to enable comparison of what sorts of interventions are most successful, and in what way, for particular groups of jobseekers.
• Penetration – Penetration measures again focus on the outcomes for specific groups of jobseekers or inactive claimants.
• Benefit duration – Benefit duration measures focus on how quickly the PES is able to help people return to work from unemployment, and are sometimes referred to as rapid re-employment targets. In one PES, there is also an indicator of how quickly individuals return back to the unemployment system.

Final outcome indicators typically focus on overall objectives, such as increasing the employment rate or reducing the rate of claiming particular benefits. They are often though quite difficult to link back to PES interventions because of the wide range of other influences that act on them and therefore are not often associated with a particular target

The conclusions drawn from the evidence review, consultation and case studies suggest that some general principles are important in designing performance measurement systems.

• Performance measurement should be based on robust data.
• Decision-making requires data which is both comprehensive and timely.
• The collection of performance data should be assessed against its relative cost and benefit.
• Performance measurement needs to avoid generating perverse incentives
• The incentive effects of performance measures may vary at different organisational levels.

Notes to Editors

1. The DWP research report International Review of Performance Management Systems in Public Employment Services will be published on 12 January 2010.
2. Policy Research Institute at Leeds Metropolitan University conducted the research.  The report authors are Alex Nunn, Tim Bickerstaffe and Ben Mitchell.

Media Enquiries: 0203 267 5141
Out of hours: 07659 108 883
Website: www.dwp.gov.uk