20 December 2005 - Publication of DWP research report 305: The Job Retention and Rehabilitation Pilot: Reflections on Running a Randomised Controlled Trial
A research report published today by the Department for Work and Pensions provides a detailed account of the running of one of the first Randomised Controlled Trials (RCT) within a large-scale voluntary labour market programme in the UK.
Based on the running of the Job Retention and Rehabilitation Pilot (JRRP), this report provides information about the practical and methodological aspects of running an RCT, including recruitment, the screening process and randomisation, service delivery and management, and issues around winding down the trial. The findings will be useful for policy-makers and evaluators considering the use of RCT methodologies to pilot future labour market programmes.
Key Findings
- Recruitment: providers used a range of methods to reach volunteers but recruitment was found to be difficult and predictions were optimistic. Both the marketing and the voluntary nature of the trial influenced the composition of the sample, as marketing was often focused where the highest returns were expected and those with low motivation will not have volunteered.
- Screening process: overall, the screening process worked well, and the advantages of centralising the process appeared to outweigh the disadvantages. Only a small percentage of would-be participants were screened out, and ‘deadweight’ – the number of trial entrants who would have returned to work without an intervention - was contained below 50 per cent.
- Randomisation: the randomisation process allocated entrants to the four groups as expected. The level of attrition differed by randomisation group and area, but was no greater than expected.
- Service Delivery: JRRP demonstrates the benefits of putting in place procedures for delivering specific aspects of the service in advance. For example, the consent procedures worked well. Another lesson is the desirability of maintaining a measure of standardisation and central control.
- Winding down the trial: a key lesson is the importance of forward planning, and developing a specific wind-down strategy paid dividends.
- Management and communication: An important theme is the need to establish and maintain good working relationships. These are fostered by a clear, accountable management structure combined with effective communication.
Notes to Editors
- The Job Retention and Rehabilitation Pilot: Reflections on Running a Randomised Controlled Trial’ is published today in the DWP Research Report Series: report number 305. A summary and copy of the report are available on the DWP website at http://www.dwp.gov.uk/asd/asd5.
- The report is based on data collected by three ‘Research Advisers’ who conducted site visits to Providers and Contact Centres to monitor the trial throughout its operation, as well as data from two surveys of participants and the screening instrument.
- The Job Retention and Rehabilitation Pilot is a four-way Randomised Controlled Trial designed to test the relative and net impacts of a boosted health, workplace-based and combined intervention for people who are off work sick for between six and 26 weeks and are screened as at risk of losing their job. The pilot was run by four providers in six areas. A total of 2845 people were screened onto the programme between April 2003 and December 2005.
- The screening process was designed to minimise ‘deadweight’ by ensuring that those participating in the pilot were those at most risk of losing their job. The screening tool was administered from a central Contact Centre, which ensured a consistent message and minimise ‘contamination’ of the control group. The tool consisted of a number of questions, with the five strongest predictors of a return to work used to produce a risk score which screened potential participants onto, or out of, the pilot. The screening tool was designed to contain ‘deadweight’ at 50 per cent.
Press enquiries: 0207238 0866
Out of hours press enquiries: 07659 108 883
Public enquiries: 020 7712 2171
Website: www.dwp.gov.uk/asd/asd5
Prepared by: Department for Work and PensionsWork, Welfare and Poverty Directorate
Prepared by: DWP Cross Cutting Strategic Analysis Division