Researchers from Hospital for Special Surgery (HSS), a world leader in musculoskeletal research and care, are calling for greater consensus in the orthopedic community around standardizing methods used to calculate patient-reported outcome measures (PROMs) for hip and knee replacements. Their research shows that current performance measures varied significantly depending on which method was applied when assessing outcomes from these procedures. These results were presented today at the 2018 American Academy of Orthopaedic Surgeons (AAOS) Annual Meeting.

"While significant progress has been made collecting PROMs data for joint replacements, no guidelines exist on the most accurate way to analyze these data to objectively measure and improve performance," said lead author Stephen Lyman, PhD, associate scientist at HSS. "Before policies are enacted, clinicians and payers must reach a consensus about which measures reflects the meaningful improvement in joint-related health."

PROMs are considered the gold standard for evaluating the effectiveness of hip or knee replacement procedures. Two of the most commonly utilized PROMs for the hip and knee are the Hip Disability and Osteoarthritis Outcomes Survey (HOOS), the Knee Injury and Osteoarthritis Outcomes Survey (KOOS), respectively, and their short form versions HOOS, JR. and KOOS, JR., respectively. The Centers for Medicare and Medicaid Services (CMS) have adopted these surveys in their Comprehensive Care for Joint Replacement (CJR) Model, which is a bundled payment plan for all primary elective total joint replacements performed in eligible Medicare patients.

This study, performed at HSS, included 2,323 people who underwent a total hip replacement procedure, and 2,630 people who underwent a total knee replacement procedure. Meaningful change of PROMs can be assessed using three measures: Minimal Detectable Change (MDC), Minimal Clinically Important Change, and Substantial Clinical Benefit. Answers from the HOOS and KOOS (original or JR.) are used to score these measures and currently several different methods can be applied to calculate the total score.

The results showed that there is wide variability in benchmarks when these methods are used to calculate them:

• Distribution-based approach, which relies on distribution of the data and assumes a normal distribution of responses and does not include patient input.
• Anchor-based approach, a patient-centric approach where an anchor question on the HOOS/KOOS regarding patient perception of a change in health is used to assess the change in PROMs scores.
• Anchor-based ROC approach, which uses the area under the ROC curve to determine the optimal delta score that best predicts those who experienced a minimal improvement and those who did not and does not include patient input.

"We believe meaningful use of PROMs to assess care delivery and quality requires an understanding of how much change over time should be expected and how much change matters to patients," Alexander McLawhorn, MD, MBA, an orthopedic surgeon at HSS. "The anchor-based assessment is the only method that accounts for the patients’ perspective."

"We hope this analysis sparks important discussions about how quality and appropriateness are measured," said senior author Catherine H. MacLean, MD, PhD, chief value medical officer at HSS. "The orthopedic community must be careful before we start using PROMs scores for performance evaluations, and as a next step HSS researchers plan to explore in depth the care experience and broad outcomes of patients in relation to changes in their patient reported outcome scores."