# Best Practice Research

#### Jenna Reps, Peter R. Rijnbeek

#### 2025-10-15

## Best practice publications using the OHDSI PatientLevelPrediction framework

Topic | Research Summary | Link
--- | --- | ---
Problem Specification | When is prediction suitable in observational data? | Guidelines needed
Data Creation | Comparison of cohort vs case-control design | Journal of Big Data
Data Creation | Addressing loss to follow-up (right censoring) | BMC medical informatics and decision makingk
Data Creation | Investigating how to address left censoring in features construction | BMC Medical Research Methodology
Data Creation | Impact of over/under-sampling | Journal of big data
Data Creation | Impact of phenotypes | Study Done - Paper submitted
Model development | How much data do we need for prediction - Learning curves at scale | International Journal of Medical Informatics
Model development | What impact does test/train/validation design have on model performance | BMJ Open
Model development | What is the impact of the classifier | JAMIA
Model development | Can we find hyper-parameter combinations per classifier that consistently lead to good performing models when using claims/EHR data? | Study needs to be done
Model development | Can we use ensembles to combine different algorithm models within a database to improve models transportability? | Caring is Sharing–Exploiting the Value in Data for Health and Innovation
Model development | Can we use ensembles to combine models developed using different databases to improve models transportability? | BMC Medical Informatics and Decision Making
Model development | Impact of regularization method | JAMIA
Evaluation | Why prediction is not suitable for risk factor identification | Machine Learning for Healthcare Conference
Evaluation | Iterative pairwise external validation to put validation into context | Drug Safety
Evaluation | A novel method to estimate external validation using aggregate statistics | Study under review
Evaluation | How should we present model performance? (e.g., new visualizations) | JAMIA Open
Evaluation | How to interpret external validation performance (can we figure out why the performance drops or stays consistent)? | Study needs to be done
Evaluation | Recalibration methods | Study needs to be done
Evaluation | Is there a way to automatically simplify models? | Study protocol under development