Solutions · AI models and their users

Test your model against outcomes it has never seen.

Healthcare AI is trained and benchmarked on claims and records assembled for billing, then judged against more of the same. Circles Datasets are different: whole episodes of care recorded prospectively, across sites, with outcomes months and years later, each label traceable to the clinician or patient who entered it.

The executive case

Return, liability, compliance — answered up front.

Return

A model is only as good as the evidence it is tested on. Of 1,357 cleared AI devices reviewed in 2026, 34 were linked to prospective trials and 3 were evaluated on patient outcomes. A Circles Dataset gives a developer, or the health system buying the model, outcome-based evidence that it works, and keeps working.

Liability

When a model’s output is challenged, the question is what it was tested against. A test set assembled from brokered data cannot show who recorded each label or when. In a Circles Dataset every label carries its author and time, so performance claims can be defended record by record. Among 950 AI devices studied, 6.3% were recalled, 43% of those within a year of clearance.

Compliance

FDA guidance asks that test data be independent of training data, sequestered from development and drawn from different sites, with a plan to monitor performance after deployment. The EU AI Act requires training, validation and testing data that are relevant, sufficiently representative, and, as far as possible, free of errors and complete. Circles Datasets are built to those requirements.

5 solutions for AI developers and users

Why AI developers, device makers and health systems partner with RegenMed.

Each runs on the same four pillars: the Observational Protocol, the Circle, the closed system and the technical stack. The detail sits in the linked white papers.

1

Independent reference datasets

Prospective, multi-site Cases with outcomes the model has never seen, cut to the population and decision the model serves.

2

Post-deployment performance monitoring

Compare predictions with outcomes as they are recorded.

FDA asked in 2025 how AI performance should be measured in the real world, noting reliance on retrospective testing and static benchmarks. Benchmarks are fixed; clinical practice, populations and models change. A Circle keeps recording, so a deployed model can be re-measured against new outcomes each quarter, and drift is visible before it reaches patients. Verification of results without moving records is in pilot.

3

Sequestered evaluation sets

Reserve Cases from any training license, so an evaluation set stays independent of every model it tests, as FDA guidance requires.

4

Training data with provenance and consent

Where training data is licensed, every record shows its source and the patient’s consent terms, unlike data scraped or brokered.

5

Domain expert clinical panels

Circle Members who design and record the protocol can review model outputs and adjudicate disputed cases, with their judgments recorded and attributed.

By the numbers
1,600+
AI-enabled medical devices authorized by FDA
As of September 2026
3 of 1,357
Cleared AI devices evaluated on patient outcomes
34 linked to prospective trials
6.3%
Of 950 AI devices associated with a recall
43% of recalls within 12 months of clearance
Aug 2028
EU AI Act high-risk obligations for AI in medical devices
Data governance for training, validation and testing data

Sources: FDA (2026); Abulibdeh et al., PLOS Digital Health (2026); Lee et al., JAMA Health Forum (2025); Regulation (EU) 2024/1689, as amended by Regulation (EU) 2026/1744.

Frequently asked questions
Why not test against the EHR or claims data we already have?

Those records were created to document visits and pay claims. They lack a baseline, rarely record outcomes after discharge, and code suspicions as diagnoses. Many models were trained on the same kinds of records, so testing against them measures agreement with familiar data, not accuracy for patients.

Can a Circles Dataset be used for both training and evaluation?

Yes, but never the same Cases for both. Cases licensed for training are excluded from evaluation sets, so an evaluation stays independent of the model it tests.

Can we evaluate a model without receiving patient records?

Licensed datasets are de-identified under HIPAA. Verification of model results where the data resides, so that no record moves, is in pilot.

Do you build AI models?

RegenMed uses AI to research clinical questions when drafting protocols; clinical experts finalize every protocol, and no model enters data into a Case. RegenMed supplies the ground truth that models are tested against.

Bring us the decision your model supports, and we will show you the outcome data to test it against.

Talk to our team