Testing performance on independent data, new sites and later time periods, not just the development sample.
AUROC, precision-recall, calibration curves and decision-curve analysis, reported to TRIPOD standards.
Performance across patient groups, sites and settings, with clear reporting of where a model falls short.
Why performance changes between health systems and populations, and what can be done about it.
Post-deployment monitoring plans and recalibration or updating strategies as data and practice change.
Evidence packages that support go / no-go decisions, governance review and reporting to oversight bodies.
Evaluating vendor or in-house models before and after go-live.
Independent evidence of performance for customers, regulators and publications.
Objective assessment of AI tools proposed or used in funded programs.
Methods for testing whether a prediction model holds up in a new site, population or period.
Developing methods to estimate how outcome risk may shift when a model is introduced into clinical use.
Research on how the timing of clinical observations can bias early-warning signals, with a manuscript currently under peer review.
A short conversation to understand your question, data and deadline.
A written analysis or evaluation plan with methods, deliverables and timeline.
Analysis by senior specialists, with reproducible code and clear documentation.
Help interpreting results, responding to reviewers and preparing outputs.
Developers have an interest in favorable results. An independent evaluation gives decision-makers, regulators and reviewers evidence they can trust.
Not always. Many evaluations need only model outputs (risk scores) and outcomes on an evaluation dataset.
We follow established reporting guidance for prediction models, such as TRIPOD and TRIPOD+AI, and tailor reporting to the audience.
Work with SmartDaaS
SmartDaaS is a principal-led firm. Every engagement is directed by the principal and staffed with the senior specialists it needs from our network of biostatisticians, epidemiologists, health economists, data scientists and clinicians.
Start a conversation