Comparing Operational Risk across different Managers
Institutional investors rarely assess managers that resemble one another closely.
A manager universe may contain a large global asset manager, a specialist credit boutique, a private-equity GP and a relatively lean real-assets firm. Their governance, technology, transaction volumes, outsourcing arrangements and organisational depth may be fundamentally different.
Yet the investor still needs a coherent basis for comparing the judgements it makes about their separate operational setups.
The obvious solution is standardisation. Apply the same questionnaire, examine the same topics and assess managers against the same organisational expectations.
Taken too far, that creates false comparability.
The more useful objective is to make the reasoning consistent while allowing the managers themselves to remain different. Similar evidence should be interpreted through a stable judgement framework, without assuming that good organisations all need to look the same.
¨The relevant question is therefore not whether a manager conforms to a preferred organisational design. It is whether the arrangement makes sense for the business it supports...¨
Understand the operating model before comparing it
Visible organisational features are poor benchmarks on their own.
A manager with a larger team is not automatically more resilient. Extensive committee governance does not necessarily mean that authority is well distributed. Outsourcing does not inherently weaken an operating model, and automation is not always superior to a well-controlled manual process.
ODD practitioners encounter both sides of these comparisons. A sophisticated governance structure can coexist with important decisions remaining concentrated around a small number of people. A much leaner organisation may operate extremely well because responsibilities, deputies and escalation are unusually clear.
The relevant question is therefore not whether a manager conforms to a preferred organisational design. It is whether the arrangement makes sense for the business it supports and whether the risks created by that design are understood and controlled.
Comparison should begin only after that operating reality is clearly understood.
Compare the mechanism, not the label
Consider two managers that both rely on a relatively manual operational process.
At one, transaction volumes are modest, responsibilities are clearly segregated, material changes require independent verification and the reviewer can see evidence that those controls have operated consistently. At another, volumes have increased substantially, several spreadsheets feed the process, responsibilities overlap and recurring exceptions still require senior intervention.
Both processes are manual.
The operational conclusion should not be the same.
Labels such as manual process, outsourcing dependency or key-person risk are useful shorthand, but they can become misleading when used as the unit of comparison. The real work is understanding the mechanism through which an observation could affect the organisation and the mitigants surrounding it.
Sometimes that also means looking beyond the individual finding. A governance issue, an outsourcing concern and recurring control exceptions may arise in different parts of a review. They can still point towards the same underlying problem: concentrated authority, weak retained capability or ineffective escalation.
This is the role of the structural layer within Lestrade's approach: topical ODD identifies risks where they arise. structural analysis considers whether those observations reveal wider organisational mechanisms or dependencies.
That gives the investor a more useful basis for comparison than counting findings by category.
“Separating residual risk from evidential confidence therefore makes comparison more informative.”
Compare how strongly the conclusion is established
Residual risk alone is also an incomplete comparison.
Suppose two managers are both assessed as presenting relatively low operational risk in a particular area.
At the first, formal documentation, interviews and operating evidence all support the conclusion. At the second, the assessment depends largely on management's description because relatively little evidence of operation is available.
The risk conclusion may currently be similar. The confidence that should be placed in it is not.
Different evidence sources establish different things. A policy can demonstrate that a control is formally designed without demonstrating that it operates. An interview can explain how responsibility is understood while operating evidence shows what actually happened. Where different sources contradict one another, the contradiction may itself matter.
Separating residual risk from evidential confidence therefore makes comparison more informative.
Across a manager universe, this separation lets an investor distinguish between reassuring conclusions that are strongly established and reassuring conclusions that remain more provisional. The headline risk assessment may be similar, but the monitoring or further work justified by it may not be.
Calibrate the reasoning without removing judgement
Professional judgement will never be perfectly uniform, nor would that be desirable.
Experienced ODD professionals notice different things. Someone who has spent years assessing private-market firms will bring a different reference set from someone whose experience is concentrated in large liquid-market organisations.
The risk arises when those differences in experience begin to produce unexplained differences in outcome.
A useful calibration test is not whether two reviewers would write the same report. It is whether, faced with broadly comparable evidence, they would recognise the same important questions, apply a reasonably similar threshold for escalation and be able to explain why their eventual conclusions differ.
The underlying Lestrade framework is designed for precisely this problem. It does not aim to guarantee identical conclusions; it aims to make judgement more structured and evidence-led so that outcomes depend less on reviewer preference while retaining professional judgement.
This becomes increasingly important as an ODD program grows. A handful of experienced people can calibrate informally. Across more managers, more reviewers and more years of work, unexplained variation becomes harder to defend.
Consistency should therefore sit in the reasoning process, not in identical wording or ratings.
¨The purpose of comparability is therefore not to rank managers by how closely they resemble a preferred operating model. It is to allow an investor to explain why one difference is benign, another is material, and why that judgement would remain defensible even if a different reviewer had performed the work.¨
Preserve the differences that actually matter
The test of a good comparison framework is whether it can create consistency without flattening context.
A private-equity GP may deserve greater attention to valuation governance, organisational dependencies and significant but infrequent cash processes. A large liquid-markets manager may require more emphasis on trading infrastructure, mandate compliance or technology resilience.
Those differences should survive the analysis. Indeed, good ODD becomes weaker when unfamiliar organisational forms are treated as inherently riskier simply because they do not resemble the investor's usual managers.
What should remain stable is the discipline underneath the review. Understand the operating model, identify the mechanism through which risks could arise, test the relevant propositions against evidence, assess what residual exposure remains and be explicit about how securely the conclusion is supported.
The purpose of comparability is therefore not to rank managers by how closely they resemble a preferred operating model. It is to allow an investor to explain why one difference is benign, another is material, and why that judgement would remain defensible even if a different reviewer had performed the work.
