Evaluating Managed Care Organization Quality

One of the great potential advantages of purchasing health care through a managed care organization is the cost-effective ongoing quality assurance and continuous quality improvement that these plans can provide. The question for the employer/purchaser of an MCO is how to evaluate the quality of its supplier's QA/CQI programs. One approach to this question is to look for accreditation by an independent organization that has evaluated the quality of the MCO. Today, there are two major accrediting organizations for MCOs: the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) and the National Committee for Quality Assurance (NCQA).

The NCQA is the most experienced of the MCO accrediting organizations. It has reviewed a majority of MCOs in the United States. The NCQA accreditation process involves a review of MCO quality-related systems, including quality improvement, processes for reviewing and authorizing medical care, quality of provider network, and members' rights and responsibilities. Documentation of these processes provided by the MCO are analyzed, and a site survey is conducted involving both physician and administrative reviewers.

In addition to its process-oriented assessments, the NCQA has developed the Health Plan Employer Data and Information Set (HEDIS®) to help standardize the measurement and reporting of health plan performance. HEDIS measures have become the basis of performance measures produced by many health plans and purchasing coalitions.HEDIS has measures applicable to commercial, Medicaid, and Medicare plans. In the area of effectiveness of care, the measures include such items as breast cancer screening, controlling high blood pressure, and follow-up after hospitalization for mental illness. In the area of access/availability of care, measures include getting needed care and getting care quickly. In the area of satisfaction, HEDIS incorporates the Consumer Assessment of Health Plans (CAHPS) instrument—a reliable and valid survey and reporting kit developed by a consortium of Harvard Medical School, the RAND Corporation, and the Research Triangle Institute under the sponsorship of the Agency for Health Care Policy and Research.

Based upon these reviews, and MCO performance on HEDIS and CAHPS, the MCO is granted one of the following levels of accreditation status:

• Excellent: Demonstrated performance that meets or exceeds NCQA requirements for consumer protection and quality improvement (QI), and HEDIS results among the highest scoring plans nationally or regionally.
• Commendable: Demonstrated performance that meets or exceeds NCQA requirements for consumer protection and QI.
• Accredited: Performance meeting most of NCQA's requirements for consumer protection and QI.
• Provisional: Compliance with some, but not all, of NCQA's consumer protection and QI requirements.
• Denied: Failure to meet NCQA requirements for consumer protection and QI requirements
• Suspended: NCQA accreditation for a plan has been withdrawn to investigate and implement corrective action
• Under Review: Accreditation is under review at the request of the plan
• Discretionary Review: NCQA is assessing the appropriateness of a plan's current accreditation status.

NCQA has a similar process for accreditation of PPO plans based upon assessments of access, service, and qualified providers. PPO accreditation levels are Full, One-Year, Provisional, and Denied.

The performance of health plans can be compared using accreditation level and HEDIS measures. Individual plan performance and performance benchmarks can be accessed through NCQA's Quality Compass (see www.ncqa.org). NCQA has also created an economic model for projecting the comparative performance of health plans. The Quality Dividend Calculator^TM projects cost savings that an individual employer can expect from choosing a high quality MCO. Projections are based upon how health care quality as measured by HEDIS reduces absenteeism and increases productivity among employees. The calculator is also available through the NCQA web site. It should be noted that not all health plans collect and publish HEDIS data, and although NCQA audits HEDIS data, the data are collected and analyzed by the health plans themselves, with the potential for bias that is inherent in this approach.

The approach of the Joint Commission on Accreditation of Healthcare Organizations to accrediting MCOs is comparable to that of the NCQA and results in assignment of an MCO to one of the following categories of accreditation: provisional accreditation, accreditation with commendation, accreditation with or without recommendations, conditional accreditation, or nonaccreditation.

In assessing health plan quality of care, it would be worthwhile to ask the following questions:

• Has your MCO applied for accreditation from either NCQA or the Joint Commission on Accreditation of Healthcare Organizations?
• If so, when was your most recent review, and what category of accreditation did your MCO receive?
• Will the MCO provide a summary of the findings of the accreditation process?

A list of health plans reviewed by NCQA for accreditation is available on-line at http://www.ncqa.org. While reviewing the results of these accreditation processes can be informative, the accreditation organizations explicitly warn that they do not warranty any third parties (e.g., employers) regarding the quality of care of an MCO. In addition, many MCOs have not yet undergone accreditation. Therefore, whenever an employer or employee is purchasing MCO services, it would be advisable to do some additional evaluation, including contacting your state departments of insurance and/or public health, reviewing some minimal documentation related to MCO quality, and making a site visit.

State governments generally have some regulatory authority over MCOs operating within their borders. This regulatory authority may reside with the department of public health, the department of insurance, or some combination of these. A call to one or both of these agencies in your state, asking for information about the status of a particular MCO, can be informative. If the MCO of interest is an HMO, you may want to ask for a copy of the HMO's annual report, which must be filed with the state department of insurance.

Requesting and reviewing the following information from the MCO also can be helpful:

• Credentialing criteria/processes for network physicians, hospitals, and ancillary providers (e.g., laboratory, X-ray, home health agencies): Do these criteria and processes include those mentioned above under physician and hospital quality? Are provider credentials verified by the MCO, or do they accept a provider's self-report? How frequently are providers recredentialed? Does the recredentialing process include routine, systematic consideration of member complaints, member satisfaction, and other quality indicators?
• A copy of the most recent quality assurance, quality management, or CQI plan and annual report (individual provider and patient identifiers can be removed to protect confidentiality): Does the plan include reliable and valid measures and standards of appropriateness of care, excellence in care, and satisfaction with care as described above? Are providers educated about these measures and standards? Are performance measures documented and routinely fed back to providers? Is meaningful reinforcement and support provided for performance improvement? Are there credible, specific documented examples of performance improvement over the preceding year?
• Routine provider quality profiles (i.e., sample reports on provider performance routinely analyzed by the MCO): How reliable, valid, and useful to quality improvement are the data contained in the reports? To what extent has the quality performance monitoring described in the QA plan been incorporated into MCO reporting systems?
• Reimbursement formula for physicians in the MCO: Are there substantial financial incentives for physicians to withhold necessary care? Conversely, are there substantial financial incentives for physicians to provide quality care? (It has been this author's observation that MCOs providing such financial incentives are more likely to have reliable and valid measures of physician quality and systems for monitoring and feedback of these measures.)
• Preventive care programs offered and participation rates: What preventive care programs does the MCO offer, at what location, and with what frequency? What member cost sharing, if any, is required? What are the participation and success rates for these programs?
• Plan-wide measures of quality: Will the MCO provide the most recent report of performance using HEDIS measures? Did it use survey instruments recommended in HEDIS for assessing member satisfaction and health status? If not, how did it ensure the reliability and validity of the instruments? What were the response rates to these surveys?

An additional step that can be immensely helpful in assessing the quality of an MCO is to conduct a brief site visit to "kick the tires." In this author's experience, it is not uncommon to come away from such a visit with an entirely different assessment of MCO quality than is conveyed in written material from the organization. Consultants with some knowledge of managed care can be helpful but are not necessary. For a site visit to be most helpful, the following guidelines are recommended:

• Allow four to eight hours for the visit.
• Try to limit the time devoted to marketing and formal presentations.
• Arrange to meet key staff, including the medical director and the heads of member services, quality assurance, utilization management, and finance: What is their relevant training and experience? Are they credible and involved? What is their level of commitment?
• Devote the most time on site to direct observation and questioning of MCO operations staff, and listening to staff on the telephone in member services, claims administration, and utilization management: What is their relevant training and experience? What is their level of commitment? What is the quality of their customer service? Do they document members' complaints, concerns, and questions, and follow up? Do you see signs of a pervasive CQI program with posted performance standards and measures?
• Discuss quality-related information provided prior to the site visit (see above). What are the processes for collection and quality control of data? What were the most successful improvement initiatives in the preceding year? Review minutes of the most recent quality assurance committee meetings.
• Assess the philosophy of the MCO: Is it a good fit with your own and that of your organization? Is the MCO interested in you as a customer, your quality concerns, and your business needs?

Patient, member, and/or physician confidentiality should not be a barrier to conducting a site visit as long as reviewers are willing to sign confidentiality agreements.

For larger employers, all of the above elements can be incorporated into a formal competitive bid process involving multiple MCOs. A nationwide effort, spearheaded by the National Business Coalition on Health and Watson Wyatt Worldwide, has been joined to gather MCO responses to a standardized Request For Information (RFI). Information on this initiative, named eValue8, is available at www.evalue8.org.

Other resources to consider when evaluating MCO quality include the following:

• National Committee for Quality Assurance (NCQA), Washington, DC. www.ncqa.org
• America's Health Insurance Plans (AHIP), Washington, DC. www.ahip.org
• National Coalition on Health Care, Washington, DC. www.nchc.org
• Institute for Health Care Improvement (IHI), Roxbury, MA. www.ihi.org
• Centers for Medicare & Medicaid Services, Baltimore, MD. www.cms.hhs.gov
• Agency for Healthcare Research and Quality (AHRQ), Rockville, MD. www.ahrq.gov
• Foundation for Accountability, Portland, OR. www.facct.org

Evaluating MCO quality, like physician and hospital quality assessment, can be a time-consuming process. Yet, this may be a relatively small investment of time when weighed against the resources spent by employer and employee on health care and the risks posed by the purchase of poor-quality health care.