# Owner-framing evidence classifier evaluation - 2026-08-10

## Data added

- New synthetic reviewed rows: 66
- Combined reviewed rows after merge/dedupe: 547
- Focus: meeting-minutes action ownership and wording where the transcript says one person will share a document so another can review it.
- Specific weak spots covered:
  - QMS handoff should become Orla reviewing the QMS Manual, not Jacqui sharing it.
  - Label review offer should become Orla sharing the label with Jenny for review, not Jenny owning a standalone review action.
  - HPRA invoice/bill discussion should become Jacqui reviewing the invoice fee, not Orla sharing invoice/email as the published action.
  - DoC risk-rationale discussion should become John-Paul reviewing DoCs for MDR/PPE compliance.

## Training result

- overall_status macro F1: 0.568
- action_state macro F1: 0.596
- action_state micro F1: 0.644
- signals macro F1: 0.738
- signals micro F1: 0.812

The overall evidence score is broadly steady versus the prior DITA handoff bundle. The important change is not a headline F1 jump; it is a more targeted action-framing behaviour on DITA-style reviewer/share-recipient examples.

## Local probe verdict

Direct classifier smoke on the DITA client transcript returned six actions:

1. Orla Skally - Review the QMS Manual
2. Orla Skally - Follow up with Cody for the MedEnvoy project plan or task list
3. Orla Skally - Share a copy of the new label with Jenny for review
4. Orla Skally - Share the country and language list for the Declarations of Conformity
5. John-Paul Hughes - Review the DoCs for sunglasses for MDR and PPE compliance
6. Jacqui Fox - Review the HPRA invoice fee

## VPS live verdict

After deployment and one Node merge/dedupe fix, the hosted staged-actions endpoint returned the same six action rows above.

Model bundle:

```text
/root/meeting-minutes-evidence-model/runs/owner-framing-20260810T224329Z/model_bundle
```

Training CSV:

```text
/root/meeting-minutes-evidence-model/source_uploads/meeting_minutes_evidence_training_owner_framing_20260810.csv
```
