To solve this, Paradigm deployed and optimized traditional best practice, healthcare domain-specific ML and NLP models to extract and interpret medical record data. These models were trained and evaluated on golden data sets curated by expert clinicians. However, this approach was slow and burdensome.

Because traditional models only worked so well, clinicians are required to manually review model output to verify that quality standards have been met.

Paradigm believed that LLMs, with their ability to summarize unstructured text, could be a great fit for their use case and replace the approach of building one-off ML models. They explored two potential paths: integrating with a custom LLM trained on medical use cases, or integrating with GPT‑4 through OpenAI’s API.

Paradigm believed they would need a specialized medical model to get good results. They were “shocked” to find that GPT‑4 outperformed a team of highly trained human experts on complex data evaluation tasks.