Tag

Performance Metrics

All articles tagged with #performance metrics

artificial-intelligence1 year ago

"Rapid Acquisition of Unforeseen Abilities by Large Language Models"

Researchers have observed that large language models (LLMs) exhibit sudden leaps in ability on certain tasks, which has been described as "breakthrough" behavior. However, a new paper from Stanford University suggests that these abilities may not be as unpredictable as previously thought, attributing the sudden appearance to the way researchers measure LLM performance. The study argues that the emergence of abilities is more gradual and predictable than previously believed, and that the choice of metric used to evaluate LLMs plays a significant role in determining their perceived capabilities.

healthcare2 years ago

VA sets tougher terms for EHR vendor contracts after past failures.

The Department of Veterans Affairs has renewed its multibillion-dollar contract for a new Electronic Health Record (EHR) with Oracle-Cerner, with new terms meant to hold the vendor accountable for persistent outages. The renegotiated contract includes larger fines Oracle-Cerner will need to pay VA if the EHR doesn’t meet performance targets. The new contract includes 28 performance metrics for Oracle-Cerner to meet or exceed and includes a higher standard for reliability and uptime. The VA has obligated about $9.4 billion on the EHR modernization program since fiscal 2018.