IOM Standards and Clinical Practice Guidelines

Clinical practice guidelines have played an increasingly important role in dictating the practice of medicine in the United States over the past 20 years. During that time period, the number of organizations creating those guidelines has increased, as have the guidelines themselves. The Agency of Healthcare Research and Quality archives approximately 2700 clinical practice guidelines in their National Guideline Clearinghouse (NGC); more than 6800 reside in the Guidelines International Network.

Guidelines are intended to help reduce inappropriate practice variation, accelerate the process of research findings into clinical practice, and improve the quality and safety of healthcare. However, according to researchers, there are concerns about the validity and reliability of some of the guidelines.

The concerns have focused on the quality of evidence on which the guidelines are based, the tendency of the guidelines to promote more care rather than more effective care, and the challenge of customizing population-based recommendations to individual cases. Lack of transparency in the process of clinical practice guidelines creation is another area of concern.

In March 2011, the Institute of Medicine (IOM) issued a new set of standards for clinical practice guidelines. The standards were developed to improve the transparency and objectivity of guidelines being created and to standardize the procedure by which guidelines are developed.

Researchers recently conducted a study to examine adherence to the IOM standards by guidelines archived in the NGC website. They compared their findings to those of an earlier analysis published in 1999 by Shaneyfelt et al. Results of the current study were reported in Archives of Internal Medicine [2012;172(21):1628-1633].

In June 2011, the researchers randomly selected 5 clinical practice guidelines from each of the 26 Medical Subject Headings (MeSH) under the general MeSH topic of Diseases. Of the 130 guidelines selected, 16 were listed under 2 separate MeSHs; those were considered only once in the analysis, resulting in 114 individual guidelines analyzed. Of those, 15 were randomly chosen for comparison with the original versions of the guidelines published on the sponsoring organization’s website; 4 were not accessible, thus, 11 were included in the comparison.

The researchers selected 18 standards for the development and reporting of clinical practice guidelines from the IOM recommendations in Clinical Practice Guidelines We Can Trust. Each clinical practice guideline summary was independently evaluated for compliance by 2 of the study’s authors. Discrepancies in the evaluations were resolved through open discussion.

Overall, the median number of IOM standards satisfied was 8 of 18 (44.4%); interquartile range, 7 to 10 (38.9% to 55.6%). Fewer than half of the guidelines surveyed met >50% of the IOM standards. Subspecialty societies were the worst in this regard; barely a third of the guidelines satisfied >50% of the IOM standards included in the analysis. Only a quarter of guidelines associated with obstetrics/gynecology met the target. The researchers noted that in neither case were the poor performances significantly different from the performances of the groups with which they were compared.

The IOM standards include recommendations for guideline development and format. Adherence to those standards was poor, with information on conflicts of interest (COIs) provided in <50% of the guidelines in the analysis. Guidelines produced by medical societies and non-US developers were less likely to include information on COIs. Committee membership was also not up to IOM standards, rarely including either an information scientist or a patient or patient representative.

With regard to IOM standards for evidence identification and summary, adherence was substantially better, other than for the use of non-English literature, unpublished data, and/or abstracts. Adherence to standards for formulation and recommendations was also good, except that differences of opinion were rarely described. Free public access to guidelines was nearly universal.

Guidelines published from 2006 through 2011 had little variation in terms of the average number of IOM standards satisfied. When the researchers compared the results of the current analysis to those of Shaneyfelt et al, the distributions of the mean number of standards satisfied were similar.

“Analysis of a random sample of clinical practice guidelines archived on the NGC website as of June 2011 demonstrated poor compliance with IOM standards, with little, if any, improvement over the past 2 decades," the researchers concluded.