Use the vitals package with ellmer to evaluate and compare the accuracy of LLMs, including writing evals to test local models.
Framework Desktop Review: Small and Mighty, but Shy of Upgrade Greatness ...