This accomplishment signifies John Snow Labs’ dedication to delivering the most accurate medical LLMs available. Here are three key milestones:
- Overall Accuracy: Their LLM achieved an impressive 87.35 score on the Open Medical LLM leaderboard’s standardized test harness. This surpasses well-known models like Med-PaLM2, GPT-4, and OpenBioLLMLlama.
- Efficiency Powerhouse: A 7-billion parameter model outperformed all previous models of similar size, including GPT-4, on the PubMedQA benchmark (78.4 vs. 75.2). This dataset focuses on reasoning through biomedical research texts, with a single human achieving a near-identical 78% accuracy.
