Intelligence, judged by the outcome.
Connected knowledge informs prediction, judgement and digital-first action. We assess the full chain: the chosen treatment, timing and channel; the digital conversation and expert support; and the customer and commercial outcome. A completed interaction starts the evaluation.
Judge prevention and recovery together.
We assess whether prevention and recovery together improve the overall portfolio outcome. That means sustainable payment and worthwhile recovery after the full service cost, alongside appropriate customer treatment. The evidence supports a practical decision about where to continue, change or extend the service.
Keep prevention and recovery populations distinct.
For prevention, identify the eligible accounts that are not in arrears at the start of the test. Define the payment event, observation period and conditions that constitute a new arrears entry. Keep the starting population visible, including accounts that receive no contact.
For recovery, define the accounts already in arrears, their balance and debt age, and any relevant disputes or existing arrangements. Differences in those characteristics can change the apparent result before an intervention begins.
Track movements between the populations. Successful prevention may leave a smaller, harder recovery portfolio. A lower recovery percentage alone would then be an incomplete assessment of the service.
Give each measure a meaning the business can inspect.
Keep the prediction and selected action beside the actual delivery record. That includes digital voice where selected, expert queries made during the conversation, contextual handovers and guidance to the specialist. Prediction quality and treatment effect answer different questions.
| Measure | Definition to agree before the test |
|---|---|
| New arrears entry rate | Eligible accounts entering the defined arrears state during the observation period, divided by the starting eligible prevention population. |
| Sustained payment | Accounts meeting the agreed payment conditions across a defined follow-up period. State how partial payments and changes to arrangements are treated. |
| Net cash recovered | Cash received from the defined recovery population, adjusted for reversals and refunds. Attribute improvement through the comparison method. |
| Repeat arrears | Accounts returning to the agreed arrears state after a cure or sustained arrangement, within the defined follow-up window. |
| Full service cost | Implementation, integration, digital delivery, expert consultation, specialist handling, continued assistance and agreed fees. Include relevant client effort and separate one-off from continuing costs. |
| Customer treatment | Appropriate contact, customer choice, handling of disputes and support needs, suitable arrangements, access to a person and adherence to decision limits. |
Establish what changed because of the intervention.
Choose a credible comparison.
Where appropriate, compare eligible groups assigned to the new treatment and the usual service. Essential support and required action must remain available. If assignment is not feasible, use a defensible comparable population or baseline and state the limitations. Match the observation period and definitions, and account for material differences between the groups.
Separate selection from treatment.
Accounts selected for treatment may already differ in risk and likely response. Comparing contacted customers with all uncontacted customers can therefore misstate treatment effect. Assess the population assigned to the approach, including failed contacts, alongside records of the digital conversation, expert consultation and specialist involvement.
Keep other changes visible.
Record seasonality, tariff or policy changes, account mix, payment-calendar effects and concurrent initiatives. A rolling baseline helps manage an evolving operation, while the original baseline preserves the starting point. Neither removes the need to understand changes outside the intervention.
Review the outcome when it has had time to occur.
Set review windows that fit payment dates, arrangement duration and the expected effect of the intervention. Early contact or task-completion measures can explain delivery. They cannot substitute for later evidence of sustained payment, reduced arrears or customer outcomes.
What fewer collections entries can establish.
Suppose two comparable groups each contain 1,000 eligible accounts. During the agreed period, 80 accounts enter arrears under the usual service and 60 under the new treatment. The observed rates are 8% and 6%, a difference of two percentage points, or 20 accounts.
These invented figures demonstrate the calculation only. The design, uncertainty and other differences determine how confidently the result can be attributed. A relative reduction of 25% is mathematically correct, but it is not an ellio performance claim.
Count the value once and include the work it creates.
Prevention, recovery and fewer service contacts can affect the same account. Build a benefit record that explains which effect is counted where. Cash collected, accounting loss avoided and operational capacity released are different measures; they cannot simply be added together.
Include digital conversations, expert queries, specialist handling and continued assistance when assessing client net value, alongside implementation, integration and fees. Explain whether released capacity changes staffing cost or is redeployed. Review treatment quality and customer outcomes alongside the financial result.
Finish with an explicit service decision.
| Decision | Evidence to consider |
|---|---|
| Scale | The effect is credible, economically worthwhile and consistent with the agreed customer safeguards; delivery capacity and integration support wider scope. |
| Adjust and test again | The opportunity remains plausible, but a treatment, population, handover or measurement weakness needs a defined change. |
| Stop or narrow the intervention | The evidence shows insufficient value, unacceptable customer effects, unreliable delivery or uncertainty that the available test cannot resolve. |
Evidence your business can act on.
Connect what the service knew, the decision it made and the action it took with the outcome that followed. The evidence informs the next improvement.