Halyard Labs
Evaluation and interpretability tooling for models already in production.
- Sector
- Evaluation & interpretability
- Headquarters
- Cambridge, MA
- Ownership
- Majority
- Team
- 28
Overview
Halyard builds the instrumentation that tells an operator whether a deployed model is still doing its job — and, when it is not, which part of the system moved.
Why this exists
Pre-deployment benchmarks answer a question nobody has after launch. The real question is narrower and harder: did behavior on our traffic change this week, and was it the model, the retrieval layer, the prompt, or the users?
Halyard runs continuous evaluation against live production distributions, holds a versioned history of every scored interaction, and attributes regressions to a specific component rather than reporting a single number that fell. For regulated customers the same record doubles as the audit trail they are already required to keep.
Why we hold it
Evaluation history is the rare asset that compounds without being copied. A customer three years into a Halyard deployment has a longitudinal record of their own system that no competitor can reconstruct or migrate.
Current focus
- Attribution across retrieval, prompt, and weight changes
- Per-customer eval sets built from production traffic without retaining PII
- Reporting that satisfies EU AI Act post-market monitoring obligations