Why we need more and better ratings systems for medical care

Savvy consumers have learned to rely on ratings to reserve a restaurant table or book a hotel room. Doctors and hospitals: not so much. Sure, medical care is a lot more complicated than whether a hamburger is cooked the way you asked. But patients deserve to know: Why can’t medicine, which accounts for one-sixth of our economy, get with the 21st century and help us with difficult choices?  It’s a question we really want to ask especially when something bad happens and you wake up in an unfamiliar facility wondering,  How good is this place and its care?

Complex medical care used to be offered only in big hospitals. But then came stand-alone surgery centers, imaging, urgent care, and now: rehabilitation hospitals.

Rehab hospitals have become a relief valve for “regular” hospitals, taking on patients who aren’t ready to go home because they still need care, but not at the peak levels offered at big hospitals.

Hospital space is costly to build and maintain. The bills for their intensive care mount fast. Patient advocates and regulators have pressed hospitals to discharge those needing care from expensive beds and pricey buildings, pronto. They also have crafted financial and regulatory incentives to keep the sick and injured from boomerang readmissions.

But patients pushed into rehab facilities can find that their options are limited. They may sense that not only have they lost control over where their care occurs but also they may be at greater risk of harm in the new places, as the New York Times reported:

“Rehab hospitals that help people recover from major surgeries and injuries have become a highly lucrative slice of the health care business. But federal data and inspection reports show that some run by the dominant company, Encompass Health Corporation, and other for-profit corporations have had rare but serious incidents of patient harm and perform below average on two key safety measures tracked by Medicare. Yet even when inspections reveal grave cases of injury, federal health officials do not inform consumers or impose fines the way they do for nursing homes. And Medicare doesn’t provide easy to understand five-star ratings as it does for general hospitals. In the most serious problems documented by regulators, rehab hospital errors involved patient deaths.”

The newspaper found that “rehab hospital errors” led to fatal incidents, including lethal medication mistakes, flawed oversight of facility construction leading to a patient’s fatal carbon monoxide poisoning, and the discovery of a patient’s lifeless body in a pool of blood on the floor — this after a nurse switched off a bed alarm.

The newspaper cited federal statistics that show how fast for-profit rehab hospitals, like the 168 operated by Encompass, have grown to dominate and become profitable:

“Across the nation, there are now nearly 400 stand-alone rehab hospitals, the bulk of which are for-profit. These hospitals collectively generate profits of 10%, more than general hospitals, which earn about 6%, and far more than skilled nursing homes, which make less than half a percent.”

On their positive side, the newspaper reported:

“Encompass tends to be better at helping most patients return home and remain there. In a two-year period ending in September 2023, Medicare rated 233 rehab facilities as performing better than the national rate for this major metric, called discharge to community. Most rehabs with better community discharge rates are for-profit, and Encompass owns 79 of them.”

Reasons for concern

Still, patients and their advocates have reason for concern, the New York Times reported:

“[D]ata from Medicare also reveals Encompass owns many of the rehabs with worse rates of potentially preventable, unplanned readmissions to general hospitals. Medicare evaluates how often patients are re-hospitalized for conditions that might have been averted with proper care, including infections, bedsores, dehydration, and kidney failures. Encompass accounts for about one in seven rehab facilities nationally, but owned 34 of the 41 inpatient rehab facilities that Medicare rated as having statistically significantly worse rates of potentially preventable readmissions for discharged patients … And it owned 28 of the 87 rehab facilities — 65 of which were for-profit — that had worse rates of potentially preventable readmissions to general hospitals during patient stays …”

Encompass’s lawyer defended the company’s performance, saying its facilities offer “superior” care. He criticized comparisons involving Medicare data, which he assailed as “crude” and argued that the newspaper had over-emphasized what he termed “rare” and extreme problems in facilities’ care.

The newspaper, though, reported that patient advocates are asking if it is past time for greater transparency about rehab hospitals and their care. Why doesn’t the Centers for Medicare and Medicaid Services (CMS)— the federal oversight agency — help patients more by using its array of data and offering an easier, more convenient rating, as occurs with other hospitals? The previous administration had pledged to develop star ratings for rehab hospitals. But no decision on them has occurred with current CMS officials.

By the way, patients and their loved ones, as well as the primary care doctors or specialists treating them, have a say before hospitals transfer the sick or injured to rehab facilities. But nonprofit hospitals, as the New York Times reported, often have closed their own such ventures and partnered with chain-operators of rehab institutions. Patient transfer decisions typically are made with urgency and those affected don’t have a lot of time to research options. That makes consumer-friendly ratings even more important for patients and families.

Resistance by the medical industry to simple ratings systems began to crumble with the 1999 release of a landmark report by the respected Institute of Medicine that exposed big issues in  the quality and safety of U.S. medicine. As reported in a review in a nursing journal a few years later:

“On Nov. 29, 1999, the Institute of Medicine (IOM) released a report called To Err is Human: Building a Safer Health System. The IOM released the report before the intended date because it had been leaked, and one of the major news networks was planning to run a story on the evening news. Media throughout the country recognized this opportunity for a headline story describing a very large number of hospital deaths from medical errors —possibly as great as 98,000 per year. The problem in other care settings was unknown, but suspected to be great.

“The search was on to find out who was to blame and how to fix the problem. Congressional hearings were subsequently held. Governmental agencies, professional groups, accrediting organizations, insurers, and others quickly responded with plans to define events and develop reporting systems. Health care organizations were put on the defensive. Recognizing that individual accountability is necessary for the small proportion of health professionals whose behavior is unacceptable, reckless, or criminal, the public held organizational leadership, boards, and staff accountable for unsafe conditions. Yet imposing reporting requirements and holding people or organizations accountable do not, by themselves, make systems safer.”

Transformational research

The medical journal Health Affairs reported in 2018 how the IOM study proved so important:

“To Err Is Human was transformational for patient safety. It brought the problem of medical errors into the public eye and highlighted why every health care organization in the U.S. must consider safety as a priority. Before the report’s release, many—including leaders in major health care organizations—simply did not. The report made several major points: Errors are common, they are costly, systems-related problems cause errors, errors can be prevented, and safety can be improved. Important changes resulted, including a significant increase in patient safety research sponsored mainly by the Agency for Healthcare Research and Quality (AHRQ) and hospital programs focused on measurement, accreditation, and regulation. The number of studies to address safety gaps increased by more than 250% over several years, and many occurred in areas that had not received previous attention.”

Dating to 2002, though, the efforts to collect, analyze, and report crucial information on hospital care proved complex, controversial, and slow. Researchers, notably those working now as part of a federal intervention, found themselves dealing with big data and wrestling with determining exactly what their information showed.

Doctors and hospitals participated in the work, but they raised objection after objection, including arguments about disclosing serious problems that they asserted could lead to lawsuits and were reported to state and federal authorities privately. Hospitals also argued that they had created their own watchdog organization known as the Joint Commission. It cites its accreditation process and the information it provides about participating institutions as important quality and safety metrics for the public. But critics note the commission is supported by hospitals and has industry-friendly staff.

Others step in 

Outside groups went where the feds were slow to go. U.S. News and World Report, for example, “started publishing rankings in higher education in 1983 and then expanded to U.S .hospitals in 1990 and pediatric hospitals specifically in 2007,” Health Affairs reported. A study in the Journal of Health Economics noted that the magazine’s early efforts were based on a survey of doctors. “Beginning in 1993, USNWR contracted with the National Opinion Research Center at the University of Chicago to publish an ‘objective’ ranking system that used underlying hospital data to calculate which hospitals they considered to be ‘America’s Best Hospitals.’” Over the years, the magazine has expanded and deepened its publicly disclosed rankings methods, consulting with top experts to refute earlier criticism that its ratings were mostly based on hospitals’ reputations among doctors and patients.

In the meantime, a group of business leaders — with growing alarm about the safety, cost, and quality of U.S. medical care — coalesced as the Leapfrog Group. These folks worked in the late 1990s with the “Business Roundtable, leading researchers and academics, and top quality and patient safety experts” to improve medical care. Their efforts, as reported on their website, resulted in the Leapfrog Hospital Safety Grades. These were launched in 2012 after the group, starting in 2001, surveyed hospitals. Their queries came in different categories intended to show patients key measures about their care. The group pushed hard for its system, the “letter grade assigned to all general hospitals rating how safe they are for patients. Issued twice per year, the Leapfrog Hospital Safety Grade empowers regular people to find a safer hospital in their community by using the easy-to-understand A, B, C, D, F grading system.”

With a magazine offering numeric rankings and Leapfrog producing letter-grade summaries, U.S. officials lagged in efforts to offer a similar public system. CMS did launch a flawed, five-star system for nursing homes in 2008. The 1-5 U.S. star ratings didn’t apply to hospitals until 2016. Then the controversial hospital ratings got a big push from the Affordable Care Act and groups like the seniors’ organization AARP. As a news article in the Washington Post by the independent, nonpartisan Kaiser Health News service reported back then:

“Over the past decade, the federal government has devised 115 different ways to measure medical quality in hospitals, from assessing wait times in emergency departments and noise levels outside hospital rooms to tracking blood clots in surgical patients. But the latest effort, to combine dozens of metrics into one patient-friendly quality indicator, has proven the most contentious … Hospital leaders … say the [ratings] formula is skewed against institutions that treat the poorest or toughest patients, meaning those with complex illnesses. The number of stars would be based on 64 different measures, which CMS has posted on Medicare’s Hospital Compare website. The metrics on mortality, readmission, patient experience, and patient safety are the most influential, each representing 22% of a facility’s rating.”

The hospital star ratings proved shocking to institutions, doctors, and patients, Kaiser Health News service reported in an news article published in the San Jose Mercury News:

“The federal government released its first overall hospital quality rating … slapping average or below-average scores on many of the nation’s best-known hospitals while awarding top scores to dozens of unheralded ones. The Centers for Medicare & Medicaid Services rated 3,617 hospitals on a one- to five-star scale, angering the hospital industry, which has been pressing the Obama Administration and Congress to block the ratings. Hospitals argue the ratings will make places that treat the toughest cases look bad, but Medicare has held firm, saying that consumers need a simple way to objectively gauge quality. Medicare does factor in the health of patients when comparing hospitals, though not as much as some hospitals would like. Just 102 hospitals received the top rating of five stars, and few are those considered as the nation’s best by private ratings sources such as U.S. News & World Report or viewed as the most elite within the medical profession.”

Still, CMS and patient advocates have pressed on, with hospitals, doctors, researchers, and policy makers continuing to improve the ratings to better serve patients.

While rating systems have proven to be far from perfect, their utility and importance to users have skyrocketed. Ensuring their accuracy and reliability is crucial. So is ensuring  consumer access to important information about vital products and services like medical care.

For hospitals and other providers, reducing the complex care to a simple few stars is, inherently, problematic. They long have criticized, with some reason, the unfairness of patients putting so much weight into one, highly visible metric. It fails to show challenges that medical providers confront, such as big city hospitals treating a disproportionate share of sick, badly injured, homeless, and chronically ill patients. Teaching hospitals complain they are penalized because they offer complex, long-term, and innovative care. Big hospitals complain that smaller competitors look better statistically by treating fewer patients — and, thus, don’t have outlier cases that skew data.

Hospitals also criticize CMS experts for constantly shifting the weight given to elements in the total score that culminates in their star rating. A hospital, for example, might see its score drop because first responders suddenly make it the local, go-to treatment facility for the homeless, drug abusers, or indigent patients. A hospital’s surrounding community may be especially gray, meaning it has disproportionate numbers of patients cycling in and out of its doors as they fail in old age.

Fueling ratings’ criticism

Critics note that the federal oversight agency rated nursing homes ahead of hospital because officials thought the public had greater need for consumer-friendly information on facilities for the aged. The nursing home ratings, however, suffered the first of at least two powerful critiques of their flaws shortly after CMS issued them, with the New York Times reporting:

“[A]n examination of the rating system by The New York Times has found that …[many] top-ranked nursing homes have been given a seal of approval that is based on incomplete information and that can seriously mislead consumers, investors and others about conditions at the homes. The Medicare ratings, which have become the gold standard across the industry, are based in large part on self-reported data by the nursing homes that the government does not verify. Only one of the three criteria used to determine the star ratings — the results of annual health inspections — relies on assessments from independent reviewers. The other measures — staff levels and quality statistics — are reported by the nursing homes and accepted by Medicare, with limited exceptions, at face value. The ratings also do not take into account entire sets of potentially negative information, including fines and other enforcement actions by state, rather than federal, authorities, as well as complaints filed by consumers with state agencies.”

As the coronavirus pandemic slammed nursing homes and resulted in disproportionate disease deaths, critics scrutinized the star ratings anew. Defensive officials claimed that they had reviewed and purportedly made major ratings improvements, but the journalists found: “Despite years of warnings, the [CMS facility rating system] provided a badly distorted picture of the quality of care at the nation’s nursing homes. Many relied on sleight-of-hand maneuvers to improve their ratings and hide shortcomings that contributed to the damage when the pandemic struck.”

The New York Times’ deep dig into nursing home data, reviewing huge amounts of public data and other information less available, found:

“Much of the information submitted to CMS is wrong. Almost always, that incorrect information makes the homes seem cleaner and safer than they are.    Some nursing homes inflate their staffing levels by, for example, including employees who are on vacation. The number of patients on dangerous antipsychotic medications is frequently understated. Residents’ accidents and health problems often go unreported. In one sign of the problems with the self-reported data, nursing homes that earn five stars for their quality of care are nearly as likely to flunk in-person inspections as to ace them.

“But the government rarely audits the nursing homes’ data. Data suggest that at least some nursing homes know in advance about what are supposed to be surprise inspections. Health inspectors still routinely found problems with abuse and neglect at five-star facilities, yet they rarely deemed the infractions serious enough to merit lower ratings.”

With hospital ratings, a health journalist group’s online posting offered a snapshot of how the savvy would be wary about the star ratings. The journalists cautioned that not all hospitals are rated. They noted that hospital ratings follow a bell curve, so most institutions end up clustered in the middle. This makes it look like many institutions are middling and alike. The ratings rely on data that can also be dated, the journalists noted. They also said this is a byproduct of analyses requiring large volumes of information in certain areas to ensure accurate results. Biases can creep into key information sources, the journalists said. They said,  for example, data can fail to accurately portray patient mortality (stats skewed by differences in reporting on patients who die in hospice) or readmissions (affected by overnight stays of those billed as outpatients).

Patients may not have as much choice as they would like when it comes to picking the providers for their medical care.

In an emergency, for example, first responders simply may take badly injured or seriously ill patients to the nearest emergency room or appropriate hospital. With elective procedures, research shows patients rely on primary care physicians and specialists to help them choose hospitals. This is a good thing.

This practical approach, patient advocates hope, will be enhanced by providing patients ever-increasing, already available information, like the CMS star ratings and rankings by others. Patients can see data on the safety and quality of their prospective care, sometimes even  information on outcomes of their planned procedures.

The ratings by Leapfrog, U.S. News, and CMS often differ, of course. They conflict with local reputations and professionals’ views that institutions build in their communities, That standing that can be affected by hospitals’ aggressive communications, marketing, and advertising.

How patients can use ratings

The Mayo Clinic has posted an online guide about how patients can scrutinize the ratings — and the methods by which they are derived. The experts urge patients to look at multiple assessments — from CMS, Leapfrog, and U.S. News. They should take time to dig into elements that make up the rankings, especially for insights about their planned procedures. As the Mayo post reported:

“Your priorities may change based on the type of care you need. When considering primary care, you may be more concerned with how well a hospital or clinic manages chronic diseases or its patient satisfaction scores. If you need a complex heart surgery, you may be more concerned with hospital readmissions and 30-day mortality. Focus your research and attention on the rankings that highlight the elements most important to you now.”

The site also advises this:

“It’s important to use critical thinking skills when analyzing data, especially if it influences where you seek health care. [Mayo’s experts recommend] that patients review the methodology and metrics that the ranking or rating organization uses while studying if they truly lead to a positive health care experience … maintaining a questioning approach to research.”

Forcing improvements

While hospitals may criticize rankings, patients should understand the major benefits that occur from external scrutiny of medical providers’ work and facilities, researchers say. The Health Affairs journal cited, for example, improvements that occurred when data become public on nasty infections resulting from lines inserted into patient veins:

“[H]ospital-acquired infections [once] were considered a cost of doing business. Central line–associated bloodstream infections (a type of hospital-acquired infection) represent a notable example. Peter Pronovost and his team from Johns Hopkins University showed that by following a bundle of safety procedures, they could reduce the incidence of these infections to nearly zero. The bundle included steps to follow in central venous catheter insertion, the handling and maintenance of lines, and the prompt removal of unnecessary lines. Many felt that these initial results might be too good to be true, but Pronovost and colleagues were later able to replicate the results across the state of Michigan. This resulted in a change in how people thought about harm, because even in situations in which no obvious error had been made, it was possible to dramatically reduce the risk of harm.”

As rankings took into account the prevalence of medication errors, doctors, nurses, hospitals, and researchers made improvements to reduce these mistakes, the journal also reported. Drug containers now, for example, carry bar codes. Computers and machines rigorously dispense the number of pills or other dosages. Associated software also can check patient prescriptions to warn of allergies or risky interactions.

There’s more to the ratings, the University of Chicago’s Booth Review reported:

“[T]he ratings have a lot of influence, starting with patient decisions. Looking at a decade’s worth of nonemergency Medicare-patient hospital visits, Chicago Booth’s Devin G. Pope finds that the average hospital experiences a 5% change in patient volume due to fluctuations in its U.S. News & World Report hospital rankings. He estimates that from 1993 to 2004, this accounted for 15,000 Medicare patients switching from lower-ranked to higher-ranked hospitals for nonemergency care, resulting in more than $750 million changing hands. An improvement in rank for a hospital’s specialty also corresponded with a rise in both the number of nonemergency patients treated and the revenue generated in that specialty …

“The CMS ratings have a particularly strong influence in the industry, in part because they affect a hospital’s contract negotiations with insurance companies. A better rating can give a hospital more leverage, while a drop can hurt it. A contract includes agreements on the amount insurance companies will pay for certain tests and procedures, which is enormously important to a hospital …The CMS ratings can also affect Medicare and Medicaid reimbursements, which, according to a 2019 analysis from data provider Definitive Healthcare, can comprise a whopping 30% of hospitals’ revenues.”