Mendel Unveils Groundbreaking Neuro-Symbolic AI System Outperforming GPT-4 for Automatic Cohort Retrieval in New Study
July 02 2024 - 11:00AM
Business Wire
Mendel, a leader in Clinical AI, today announced the results of
its latest research on Neuro-Symbolic AI where Mendel’s Clinical AI
system can automate the identification of patient cohorts from
unstructured and structured EMRs, outperforming GPT-4 in several
benchmarks. Mendel’s unique clinical AI approach couples large
language models (LLMs) with its proprietary hypergraph reasoning
engine. The research unveiled by Mendel showed how it is able to
power significant advancements in Automatic Cohort Retrieval (ACR),
a fundamental task for clinical research and patient care. This
research findings can be read in full here.
Transforming cohort retrieval
Identifying patient cohorts is essential for clinical trials,
retrospective studies, and other healthcare applications.
Traditional methods relying on automated queries of structured data
combined with manual curation are time-consuming and often yield
low-quality results. Mendel’s AI offerings utilize a unique
approach that couples a world-class clinical LLM trained to
understand structured and unstructured text with a proprietary
reasoning engine infused with medical knowledge reviewed by medical
professionals to apply a clinician’s mind to complex and varied
medical situations. This ability to apply clinical reasoning to ACR
has been demonstrated to offer significant improvements over
existing Retrieval-Augmented Generation (RAG) and LLM
techniques.
“Our latest research at Mendel marks a significant milestone in
the field of AI in general, and healthcare in particular,” said
Wael Salloum, Cofounder and Chief Science Officer at Mendel. “We
are the leader in clinical reasoning by coupling LLMs with our
hypergraph reasoning, enhancing both the effectiveness and
efficiency of patient cohort retrieval. This work is critical in
paving the way for more robust and scalable clinical reasoning.
This breakthrough underscores our commitment to advance the AI
field to transform clinical research and improve patient
outcomes.”
Key findings of the study include:
This research introduces two types of reasoning to the AI
field:
- Longitudinal Reasoning: Mendel’s neuro-symbolic
architecture outperformed pure LLM approaches by efficiently
handling the longitudinal nature of unstructured Electronic Medical
Records (EMRs). As a patient’s record unfolds over time, the system
reasons over the emerging facts, contrasting, rejecting, and
consolidating them into a symbolic patient journey. Unlike LLM-only
approaches, this approach processes a patient’s EMR just once,
offline, to construct a journey that can be queried repeatedly at
minimal cost.
- Large-Scale Reasoning: Mendel's integration of real-time
hypergraph reasoning and a clinical LLM achieved higher Precision
and Recall in cohort retrieval tasks. Unlike LLM-only solutions,
which process the entire patient database for each query—making
them infeasible for healthcare applications—Mendel’s approach
maintains a fixed cost per query, regardless of the database
size.
Benchmark and evaluation
Mendel’s research introduces a new benchmark task for ACR,
featuring a comprehensive query dataset and an evaluation
framework. The study compared the performance of Retrieval
Augmented Generation (RAG) and LLM-based solutions and Mendel’s
neuro-symbolic systems, providing a detailed analysis of their
effectiveness and efficiency.
In the evaluation, Mendel had a 1.4K patient data set, and
Mendel evaluated several embeddings and found Ada outperformed
others. The evaluation report compares Ada with GPT-4 (RAG) to
Mendel’s Neuro-symbolic System, Hypercube. F1 score is the key
metric used to evaluate the accuracy of models, balancing both
precision (how many of the results are relevant) and recall (how
many relevant results were identified). This score provides a
comprehensive measure of the model’s performance.
Below are the sample results of F1 scores:
EXISTING LLMS
MENDEL
ADA+GPT-4
HYPERCUBE
IMPROVEMENT VS BEST
COMPETITOR
Query complexity by clinical experts 1
20.8
62.9
42.1
Query complexity by gold cohort size 2
52.7
79.4
26.7
Longitudinal complexity by document count
3
37.3
65.7
28.4
- Medium complexity with most queries (52)
- Broad cohort size
- Document count is greater than 74
Future implications
The findings underscore the transformative potential of Mendel’s
Neuro-Symbolic AI system by combining LLMs with domain-specific
knowledge embedded in hypergraphs. This approach enhances the
accuracy and efficiency of cohort retrieval, facilitating more
precise patient stratification and targeted interventions for new
therapies. It also paves the way for broader applications in
clinical research and patient care.
Mendel is offering a free “TryMe” demo to showcase the company's
Neuro-Symbolic AI system.
About Mendel
Mendel AI supercharges clinical data workflows by coupling large
language models with a proprietary clinical hypergraph, delivering
scalable clinical reasoning without hallucinations and ensuring
100% explainability. Headquartered in San Jose, California, Mendel
is backed by blue-chip investors, including Oak HC/FT and DCM.
For more information, visit www.mendel.ai or contact
marketing@mendel.ai
View source
version on businesswire.com: https://www.businesswire.com/news/home/20240702942909/en/
Media Contact Sylvia Aranda (on behalf of Mendel)
saranda@realchemistry.com