Voice Risk Analysis (VRA)

The Department has conducted research into the suitability of Voice Risk Analysis (VRA) to help validate benefit claims and target fraud and error work.

VRA combines the measurement of voice patterns with behavioural analysis and conversation management techniques. The process is used in the private sector, for example, in assessing the risk associated with insurance claims.

Research objectives

The specific objectives of the project were to:

• improve customer service by speeding up claims processing and reducing the level of intrusion for the majority of claimants
• provide a service that is more efficient and therefore less costly
• improve fraud and error prevention and detection

Method

In 2008, six local authorities (LAs) and Jobcentre Plus conducted an initial trial on Housing Benefit (HB) and Council Tax Benefit (CTB) claims and new claims for Income Support and Jobseeker’s Allowance.

A second trial involving a total of 24 LAs was then conducted. This was rolled out from summer 2008, with each trial lasting up to 12 months. Information from a total of 45,000 VRA telephone calls were included in the evaluation, of which a subset of 5,000 calls were validated. LAs tested VRA when:

• customers contacted them to make a new claim
• customers contacted them to report a change of circumstance
• reviewing HB and CTB claims that were already in payment

Customers who were part of the trial provided information over the telephone. The call handler classified the call as high or low risk by using:

• agreed scripts
• VRA technology
• behavioural analysis techniques

A high risk call was defined as one where the call handler, using the process, determined that the customer had:

• not disclosed all the relevant information, or
• provided misleading information

A selection of customers from both high and low risk groups were then followed up with a more detailed review of their circumstances. This was to establish whether the call handler had correctly classified the call as high or low risk. This further investigation revealed whether the amount of HB and CTB in payment was correct.

Evaluation

The evaluation focussed on whether VRA successfully predicted whether the customer would be overpaid benefit or not.

The criteria for success was that VRA correctly identified overpaid claims as high risk and included only a small proportion of claims with no overpayments in the high risk group.

Details of the evaluation are in The application of Voice Risk Analysis within the Benefits System – evaluation report (191KB)

Results

The evaluation is based on the result from the 23 pilots which provided auditable information in time for evaluation. The main conclusions are:

• of the nine local authorities testing VRA at new claims stage, three were judged successful
• of the two local authorities testing VRA on changes in circumstance, one was judged successful
• of the twelve local authorities testing VRA on in-claim reviews, one was judged successful

Conclusion

DWP conducted the research to investigate whether VRA worked when applied to the benefit system. From our findings we cannot conclude that VRA works effectively and consistently in the benefits environment. The evidence is not compelling enough to recommend the use of VRA within DWP.

At no stage did the evaluation carried out by the Department explicitly consider the effectiveness of the technological aspects of VRA. However, social research evidence suggests that the scripting and training elements of the trial were successful.