Updated 20 December 2012
Public Opinion of DWP Service levels technical description
| Short title | Public Opinion of DWP Service Levels |
|---|---|
| Technical definition | The indicator is an overall satisfaction score (either fairly or very satisfied) of the percentage of people who have had meaningful contact with the Department over a 6 month period. This involves an interaction with either of the former agencies Jobcentre Plus (JCP) or the Pensions Disability and Carers Service (PDCS). This indicator is measured using data from the DWP Claimant Service and Experience survey to generate a pan-departmental score of overall claimant satisfaction with the Department’s services. The survey uses data from approximately 7,000 telephone interviews. |
| Rationale | The indicator demonstrates DWP’s performance against SRP6 -Improve our service to the public. It provides strategic insight into performance management as it explores the relationship between satisfaction and service delivery. The indicator has previously been generated from the JCP and PDCS surveys which fed into agency Performance Management Frameworks and provided robust, national assessment of claimant service in each agency, allowing the agencies to review performance and improve services accordingly. However, the two surveys have been aligned for the 2012 survey and will feed into the office of the Chief Operating Officer (COO). The survey has Ministerial approval. |
| Formula | The indicator is derived from a combination of the satisfaction scores from the sample base of PDCS and JCP claimants who had meaningful contact or interaction with those former agencies over a six month period. Satisfaction is assessed through a single item relating to general perception of satisfaction. Worked Example: PDCS satisfaction = 90% (90% of people report being fairly or very satisfied). JCP satisfaction = 70%. If the ratio of contacting claimants between JCP and PDCS was a perfect match 1:1 then the combined measure would be 80% satisfaction. If there are twice as many JCP contacting claimants as PDCS, a ratio of 2:1, then the combined measure would be 77% satisfaction. If there were half as many JCP contacting claimants as PDCS, a ratio of 1:2, then the combined measure would be 83% satisfaction. |
| Start date | Data on service satisfaction is already published and is available from 2003. In 2009/10 PDCS moved to a combined survey to include both pensions and disability carers (http://research.dwp.gov.uk/asd/asd5/rrs-index.asp). However comparisons longitudinally are limited due to changes in sample design, questionnaire and business structures. From 2010 -11 data was combined from the PDCS and JCP surveys to generate a pan-departmental score which was available from Autumn 2011. The data for the 2012 indicator will be published in December 2012. |
| Good performance | This is a baseline measure for the Department that allows good performance to be agreed and reflected in maintaining or improving the level of satisfaction (Depending upon the level of change in the Department). Satisfaction ratings are reported at the 95% confidence level and subtle changes which are significant can be detected (e.g. 1% change). |
| Behavioural impact | There are no perverse incentives generated by data collection to determine the indicator. Data is only reported at the regional and business level, and includes a random sample of customers, so it is not possible for staff to change behaviour in order to manipulate results. |
| Comparability | The survey offers limited time series comparability from 2010/11 because this was when the item and sampling strategy was standardised. The satisfaction measure could potentially be compared to other organisations if overall satisfaction of those organisations is measured with the same question item and response scale. |
| Collection frequency | Annual |
| Time lag | Headline reporting of the indicator is four months from the end of the survey period |
| Data source (which data collection it comes from) | Survey data collection commissioned from a third party research organisation contracted through the DWP research framework. |
| Type of data (Whether it is an official statistic, national statistic, survey, MI ) | Survey data |
| Robustness and data limitations | The research contractor carrying out the survey is part of the DWP research framework. As such, they have gone through extensive checks to ensure they comply with the Departments quality standards and those of the Government Social Research profession. There is an assurance plan in place as part of the wider project plan, which is agreed prior to contract signing. A series of meetings are conducted between DWP and the contractor to finalise and sign off the details of the research requirements, sampling strategy, data collection, analytic plan and strategy. Potential risks and mitigations are also jointly identified at this stage. The DWP project manager works closely with the contractor throughout the design, delivery and dissemination of the project. The agreed milestones are central to the delivery of the project and are signed off by the DWP project manager as part of their oversight of the contractor. Steps taken by the DWP project manager for oversight of the survey include: signing off the final question set, agreeing opt out letters, attending interviewer briefing sessions, observing telephone interviews, regular progress updates and meetings, and monitoring and sense checking the data and analysis. This ensures that quality standards are monitored and the contractor’s processes and controls are adhered to. The survey items were developed through cognitive testing and a large scale pilot accurately capturing claimant opinions. Significant differences are reported at the 95% confidence interval. The current survey measures perceptions of benefit claimants and does not include those who have received a service from other agencies of the Department (e.g. the Health and Safety Executive). |
| Collecting organisation | External contractor TNS-BMRB Annually |
| Return format | Dataset returned in specified format as well as full research report |
| Geographical coverage | Nationally and JCP Group level (where applicable). |
| How indicator can be broken down | Satisfaction can be broken down by key demographics. The following breakdowns are available:, gender, age, ethnicity, work status, disability, sexual orientation, marital status, religion, highest qualification and whether claimants have children or not, English as a second language. Satisfaction can also be broken down by benefit. |
| Further guidance | A full research report will be published in February 2013 for the 2012 indicator. |
(1) While information is gathered that relates to individual demographics it may not be possible to robustly compare the satisfaction levels of some groups because of the limited number of participants in such groups.
