RISK & ML GOVERNANCE SHOWCASE

Model Evaluation Gates & Target Reframing.

Senior ML Governance: Model v1 was withheld after rigorous temporal holdout evaluation. Model v2 reframes the target to Expected Resolution Time with real signal.

Model v2: Time-to-Close Regression (hist-gbr-2025-v2)

ACTIVE_PRODUCTION

Target: predicted_ttc_hours · Challenger: HistGradientBoostingRegressor (does not beat grouped-median baseline) · Served: grouped median of observed closure hours per (complaint type, borough), computed live from requests

Passed Temporal Holdout Gate

Holdout Test MAE

5.81 hrs

Mean Absolute Error

Holdout MedAE

3.25 hrs

Median Absolute Error

Holdout RMSE

8.39 hrs

Root Mean Squared Error

Grouped Baseline MAE

5.07 hrs

Problem × Borough Median

Learned Feature Importances

complaint_type99.9%

Permutation importance (validation MAE increase) for complaint_type

spatial_density0.1%

Permutation importance (validation MAE increase) for spatial_density

agency0%

Permutation importance (validation MAE increase) for agency

borough0%

Permutation importance (validation MAE increase) for borough

hour_of_day0%

Permutation importance (validation MAE increase) for hour_of_day

day_of_week0%

Permutation importance (validation MAE increase) for day_of_week

WITHHELD_AT_GATEHoldout base rate: 0.017% (3 of 17,289)

MODEL v1 EVALUATION GATE

Ranking withheld after test

logreg-2025-jan-v1 · held-out 2025-01-27 to 2025-02-01 (exclusive)

Only 3 of 17,289 test requests lacked a valid closure within seven days. The logistic model did not beat the grouped historical-rate baseline on PR-AUC, and neither method found a positive case in its top 100.

Logistic PR-AUC0.000513

Grouped baseline PR-AUC0.000554

Top 100 precision0%

High-risk band precision0%

The validation-derived high-risk threshold selected 1,226 test requests and found no positive cases. These results do not support individual probabilities or a review queue for this slice.

Read the model criteria ↗

This is a portfolio analysis product. It cannot assign or prioritize work inside a city agency.