Flawed Logic of Regional Accreditation Criteria

Regional Accreditation Evaluative Criteria and Quality Assurance

William Troutt, Assistant Director of the Tennessee Higher Education Commission

The Journal of Higher Education, Vol. 50:2. (Mar. - Apr., 1979), pp. 199-210.

Regional accrediting associations serve many purposes, but the public identifies them primarily with their quality assurance function. Various higher education consumers rely heavily on the judgments of regional accrediting associations to satisfy concerns about institutional quality. Students, faculty, employers, federal and state governments, and the public in general see regional accrediting association approval as evidence that an institution meets qualitative criteria.

For all the public confidence they possess, regional accrediting association standards receive considerable criticism. Some critics charge that accreditation's evaluative criteria fail to relate in any way to the educational accomplishments of students [7]. Others go so far as to say that regional accreditation has stopped, for the most part, making quality distinctions [18].

Despite their importance, accreditation criteria receive very little attention from researchers or scholars in higher education. Even criticisms of accreditation criteria fail to offer any detailed information either about specific accreditation criteria or their overall character. This review focuses on (1) identifying regional accreditation criteria that propose to assure institutional quality, (2) recognizing major assumptions underlying these criteria, and (3) reviewing accreditation criteria in light of available research on correlates of educational quality. Unlike most commentaries on regional accreditation, this paper represents a perspective from outside the accrediting community. This analysis emerges, not out of experiences in accreditation, but from a review of publications of the six regional accrediting associations. It does not ad-dress specific steps in the accrediting process such as the self-study. As a [[200]] critique of accreditation practices, this study is useful to the extent that published standards or guidelines from each regional accrediting association (available as of January 1978) reflect actual practice.

Regional Accreditation Evaluative Criteria

Regional accrediting association evaluative criteria vary in format, emphasis, and terminology, but they share common areas of concern and similar assumptions about appraising educational quality.¹ A review of published evaluative criteria of the commissions on higher education of the six regional accrediting associations suggests eight areas of concern for the most part common to all associations. These areas include: (1) institutional purposes and objectives, (2) organization and administration, (3) financial resources, (4) physical resources, (5) library/learning center, (6) student services, (7) faculty, and (8) educational program. Three regional accrediting associations also employ criteria for research, graduate study, and special activities such as off-campus programs.

Statements in the published criteria of regional accrediting associations identify these criteria with the maintenance of educational quality. An examination of accrediting association criteria reveals, however, that they serve a variety of purposes. It also suggests that not all criteria claim either a direct or indirect relationship to quality.

A textual analysis of published regional accrediting association criteria reveals that five of the eight criteria listed above claim some association with quality assurance. Most regional associations suggest a relationship exists between institutional quality² and criteria for: (1) institutional purposes and objectives, (2) educational program, (3) financial resources, (4) faculty, and (5) library/learning center. Assuming some consensus exists that these five areas represent checks of institutional quality, the question becomes whether or not any relationship exists between these criteria and institutional quality. Identification of the assumptions underlying these criteria, however, needs to precede that examination. [[201]]

Regional accrediting association³ criteria for assuring quality appear to rest on three basic assumptions. First, accreditation criteria assume that judgments about institutional quality should rest on inferences from certain conditions rather than direct assessment of student performance. Second, they assume that no common benchmarks exist for assessing institutional quality. Third, current accreditation criteria equate higher education with a production process.

Inferential Character of Quality Assurance

Regional accreditation criteria assume that certain conditions and resources assure institutional quality. Accrediting associations defend this indirect approach to quality assurance on the grounds that a direct assessment of quality in terms of student performance would infringe on institutional autonomy. The Northwest Association indicates, "The standards for the accreditation of postsecondary institutions describe conditions and principles which characterize educational effectiveness rather than prescribe the means of achieving effectiveness" [17, p. 15]. It does not necessarily follow, though, that a direct assessment of ends necessitates a prescription of means. .On the contrary, requiring institutions to fulfill certain conditions and have certain resources would constrain an institution far more than a check of end results.

Defenders of accreditation practices might argue that these indirect means for assuring quality represent the best available measures of institutional quality. Accreditation criteria of most regional associations suggest checking the curriculum is more reliable than checking the student. Such an argument equates evaluating institutional quality through direct assessment of student performance with standardized tests. It ignores the long history in higher education of the direct assessment of student achievement by faculty.

Relative Definition of Quality

Accreditation criteria generally assume that no common benchmarks exist for assessing institutional quality. Accrediting associations suggest that imposing any common measure of institutional quality would destroy [[202]] institutional diversity. All regional associations agree that institutional evaluation should be solely in terms of stated purposes and objectives. Accrediting associations refer to this relative approach to institutional assessment as qualitative. The Northwest Association states, "Because of the diversity of institutional purposes and the processes for achieving those purposes, the standards are qualitative rather than being stated in quantitative terms" [17, p. 15]. This universal dissatisfaction among the regional accrediting associations with quantitative data may grow out of the disapproval of quantitative benchmarks that bear no relation to institutional quality. Current accreditation standards generally assume, though, that no quantitative standards or common benchmarks are acceptable.

Accrediting each institution solely in terms of the achievement of its stated purposes assumes that each institution has worthy purposes. According to this logic, any institution could be accredited without assessing the character of its goals. Accreditation basically attests only to the fulfillment of conditions and provision of resources necessary to make achievement on institutional goals likely.

Fundamentally, regional accreditation's relative definition of quality emerges because of a lack of a commonly accepted definition of a college degree. Since the abandonment of the classical curriculum and the rise of the elective system, the meaning of a college degree has been unclear. Accrediting agencies remain either reluctant or unable to deal directly with this problem.

Production Model for Higher Education

Any assessment of institutional quality hinges on assumptions about the nature of the educational process. Assuming regional accrediting associations associate institutional quality with student achievement, regional accrediting association criteria would appear to equate higher education with a production process requiring certain treatments and conditions. Reliance on the presence of certain conditions and resources suggests that students represent raw material to be refined by the education process. Most individuals associated with accreditation would decry any association of accreditation with an industrial model of higher education. They would contend that higher education does not lend itself to such a model and point out that their qualitative approach emphasizes the nonindustrial nature of education.

Accrediting associations nonetheless appear to partially adopt an industrial model for higher education. Instead of checking on the quality of production outcomes, i.e., student achievement, criteria generally check [[203]] on the quality of the assembly line, i.e., curricula, faculty, resources, etc. Criteria assume a direct relationship exists between the quality of the assembly line and the quality of the product.

Concerns about the outcomes of the educational process generally appear only as appendages to accreditation criteria. North Central Association guidelines state, "A direct assessment of educational and learning experiences in terms of desired outcomes should be undertaken [16, p. 65]. New England Association standards ask, "Does the institution have information about the post-graduate performance of its students? If so, what does this assessment suggest about the institution's programs and objectives?" [14, p. 1]. Western Association guidelines indicate that "an institution is expected to provide evidence of educational outcomes in harmony with its objectives and appropriate processes" [24, p. viii]. Middle States Association standards offer a plan for the measurement of outcomes [13]. Southern Association standards call for evidence of evaluation procedures that certify the effective learning outcomes of students when member institutions submit programs at variance with standards [20]. These statements illustrate a basic anomaly of most accreditation criteria, i.e., they encourage the direct assessment of educational outcomes but rely on more indirect measures for assuring institutional quality.

Research on Correlates of Educational Quality

Given accrediting association assumptions about the relation of some criteria to educational quality, it seems appropriate to examine these criteria in light of available research on correlates of educational quality, i.e., investigations into whether or not certain institutional characteristics influence student intellectual achievement. Accrediting association publications, like most higher education literature, fail to specifically define educational quality. This review assumes educational quality must relate in some way to the intellectual attainment of students.

Any research useful in determining correlates of educational quality must satisfy the following criteria: (1) employ measures of intellectual achievement or cognitive outcomes, (2) collect data from contrasting types of institutions, and (3) provide information on student change between admission and a subsequent point in time. The first criterion eliminates perhaps 95 percent of the literature on college impact, and the other two criteria rule out most of the rest. In a synthesis of this literature, Feldman and Newcomb [5] review over 1,000 studies of the effects of college on students, but only a handful deal with intellectual outcomes. [[204]]

Many of these do not provide multi-institutional comparisons. Only the following four studies survive this screening process.

In one of the earliest college impact research efforts, Learned and Wood [11] administer comprehensive achievement tests to nearly 45,000 Pennsylvania students. This research involves successive achievement examinations of the 1932 college class in 1928 (upon leaving high school), in 1930 (at the close of the sophomore year), and in 1932 (at graduation). The study concludes that no relationship exists between the amount of student exposure to an educational program and tested student achievement. To illustrate their findings, Learned and Wood note that basing graduation on intellectual attainment rather than accumulated credits would produce far different results. One college's graduating class would consist of 28 percent seniors, 21 percent juniors, 19 percent sophomores, and 15 percent freshmen. Each contingent represents roughly one-fourth of the new class. This study's major weakness lies in its failure to separate the effects potentially attributable to maturation or student ability at college entrance.

Nichols [15] examines student academic growth in terms of Graduate Record Examination (GRE) Aptitude Test performance. In this study, scores of the National Merit Scholarship Qualifying Test (NMSQT) serve as a control for differences in academic ability prior to college. Nichols' study includes a sample of 356 students at ninety-one colleges. Nichols predicts GRE Aptitude Test scores based on college entrance examination scores and subtracts these predicted scores from actual GRE Aptitude Test scores to obtain scores independent of initial student characteristics such as faculty-student ratio, library books per student, ability level of the student body, and college affluence unrelated to changes in academic performance.

Nichols' study suffers from some important limitations. In addition to its small size, the sample includes only students taking the NMSQT. The use of a general aptitude test rather than an achievement test further limits the usefulness of the study.

Astin [2], in a later study of student academic growth, also uses the NMSQT as a measure of academic ability prior to college. However, he more appropriately uses as output measures the GRE Area Tests (general college-level achievement tests in the areas of social science, humanities, and natural science). Like Nichols, Astin uses an "input-output" model to arrive at scores independent of any input characteristics. The study involves undergraduates attending thirty-eight institutions, mostly liberal arts colleges, representing both rich and poor as well as selective and unselective institutions. Astin concludes that traditional indices of institu- [[205]] tional quality, i.e., a large library, high expenditures per student, a highly trained faculty, a low faculty-student ratio, and selective admissions policy, do not appear to contribute to student achievement. Astin may present a methodologically sound study, but a relatively small sample limited to students taking the NMSQT prior to college restricts his study.

Rock, Centra, and Linn [19] report different results in a study of the relationship between college characteristics and student achievement. Their study involves a sample of 6,855 students from ninety-five colleges. This research also uses GRE Area Tests as the measure of academic achievement but employs the Scholastic Aptitude Test (SAT) to control for differences in student academic ability at college entrance. Researchers use an "input-output" model to arrive at scores independent of input characteristics. Unlike previous research, this study finds a small but significant relationship between student achievement and two college characteristics, college income per student and the proportion of faculty with the doctorate.

This study's shortcoming lies in the sampling of participating institutions. Of the ninety-five institutions sampled, only ten enroll over 2,000 students and only four represent state college or universities. The sample hardly represents all of American higher education.

Several studies failing to meet the earlier listed criteria relate in some way to correlates of educational quality. Knapp and Goodrich [10] identify certain types of institutions as highly productive by using graduate attainment of advanced degrees and other awards as criteria for institutional quality. Later studies by Holland [8], Thistlehwaite [22], and Astin [1] find that differences in institutional productivity disappear with adjustments for the abilities, talents, and goals of students. Some studies of college effects such as those by Jacob [9], Lehman [12], and Chickering [4] find little or no relationship between student change and college characteristics. Several early college impact studies report changes in scores on various measures of intellectual performance, but they fail to provide multi-institutional data for institutional characteristic comparisons.

In his study of the relative quality of graduate departments, Cartter [3] finds that library resources and faculty salaries correlate with quality ratings, i.e., prestige relative to other institutions. A recent study of the quality of doctoral programs by the Educational Testing Service [21] concludes, though, that high ratings on indicators of quality such as reputation, physical and financial resources, and faculty publications will probably mean lower ratings on indicators of quality such as student satisfaction and ratings of teaching and the learning environment. [[206]]

The Relation of Research on Correlates of Educational Quality to Accreditation Criteria

Five quality assurance criteria for regional accrediting associations were identified earlier. These criteria cover institutional purposes and objectives, the educational program, financial resources, faculty, and the library/learning center. Research on correlates of educational quality ad-dresses each criterion in the following ways.

Institutional Purposes and Objectives

The clarity of institutional purposes and objectives and the degree to which various constituencies understand them represent one index of institutional quality according to regional accreditation criteria. Unfortunately no research exists to confirm or deny this claim. Only two studies even remotely relate to institutional quality and institutional purposes and objectives. In a study of college impact on affective student outcomes at thirteen liberal arts colleges, Chickering and his associates [4] find no support for the notion that_, small liberal arts colleges with distinctive purposes and orientation produce more change in students than less distinctive institutions. In a major study of colleges and university goals, Gross and Grambsch [6] note a strong relation between prestige (a popular image of quality) and certain types of institutional goals. Their study does not address institutional quality in terms of student attainment, however, or the clarity of goals or purposes.

Educational Program

Educational program adequacy represents another accrediting association quality check. Educational programs must be congruent with institutional purposes and contain a certain amount of general education. Limited research fails to confirm a relationship, however, between educational program differences and student achievement.

Learned and Wood's massive study of Pennsylvania college students finds no correlation between educational programs and student achievement. In view of these results, Learned and Wood [11] call for a new approach to educational programming based on student attainment rather than exposure to the curriculum. College impact research on affective outcomes also fails to establish a relationship between student change and different types of educational programs. For instance, Jacob [9] finds little significant change in student values attributable to curriculum. Like-wise, Lehman [12] reports that informal, nonacademic experiences seem to produce the most change in student affective outcomes. [[207]]

Financial Resources

Accrediting associations also view financial resources as a determinant of institutional quality. Criteria ask for evidence of financial stability, but define financial stability solely in terms of resources necessary to carry out institutional purposes. Limited research on the relation of financial resources to educational quality produces inconclusive results.

In his examination of student academic growth in terms of GRE Aptitude Test performance, Nichols [15] finds no relation between academic growth and college affluence. In a similar study using GRE Area Tests as performance measures, Astin [2] concludes that high expenditure per student does not appear to contribute to student achievement. In another study using GRE Area Tests as measures of achievement, Rock, Centra, and Linn [19] report a small relationship between student achievement and college income per student but no relationship between achievement and college per student expenditures.

Faculty

Accrediting associations indicate that faculty qualifications represent another significant check of institutional quality. Accreditation standards uniformly state that faculty preparation and experience must be such as to further institutional purposes. Available research provides differing results on the relationship of faculty qualifications to educational quality.

Nichols' 1964 study [15] fails to test the relationship between faculty credentials and academic growth, but he does note the lack of a relationship between student-faculty ratio and student academic performance. Astin [2], in his 1968 study, finds measures of student achievement unrelated to either a highly trained faculty or a low student-faculty ratio. A 1970 study by Rock, Centra, and Linn [19], however, does report a small but significant correlation between student achievement and the proportion of faculty with the doctorate.

Library/Learning Center

All regional accrediting associations note the importance of the library as a benchmark of institutional quality. Accreditation criteria do not specify minimal library size, but they do indicate library resources must adequately support institutional programs. Available research finds no relationship between differences in library resources and student achievement.

Nichols' research [15] uncovers no relation between library books per student and changes in student academic performance. Astin [2] also fails [[208]] to find a relation between library size and academic outcomes. Rock, Centra, and Linn [19] likewise discover no relationship between student achievement test performance and either the total number of library books or the number of library books per student. Cartter's study [3] of departmental rankings represents the only literature showing a correlation between any measure of institutional quality and differences in library resources. Cartter bases his research, however, on perceived prestige rather than measures of academic performance.

Conclusions

The preceding review suggests available research cannot substantiate the claim that certain accrediting association criteria assure institutional quality. It also illustrates, however, the difficulty in authoritatively stating that research demonstrates the lack of any relationship between ac-crediting association criteria and institutional quality. This test of accreditation standards by available research suffers from three major weaknesses.

First, the small number and inherent limitations of the studies involved makes this review inconclusive. Four studies hardly represent a definitive examination of American higher education, especially four studies limited in their sampling either by type of institution, type of student, number of students, or geography. Second, the examinations used to measure academic performance further limit the interpretation of research results. A test of intellectual aptitude, such as the one employed by Nichols, measures relative ability rather than achievement. Even achievement tests, however, represent limited measures of educational quality. The two achievement tests used in these studies measure only the broad general outcomes of education and do not reflect the distinctive curricular emphases of individual institutions. Third, the "input-output" model for assessing college impact employed in three of the four studies reviewed may fail to accurately reflect actual increases in student achievement. Some critics of this approach suggest that corrections for entering student aptitude and achievement may under certain conditions mask the effects of college environments [5, p. 360]. Fourth, substantiation of accreditation criteria does not represent the initial purpose of these four studies. These studies view institutional characteristics, e.g., number of library books, in terms of highs versus lows rather than acceptable versus unacceptable. Ideally, a study designed to substantiate accreditation criteria might more appropriately focus on institutional characteristics of accredited institutions and the achievements of their students versus the institu [[209]] tional characteristics of nonaccredited institutions and the achievements of their students. Unfortunately, the limited number of unaccredited institutions makes such a study unlikely.

Conclusions drawn from this review hinge on where the burden of proof lies. Do accrediting associations have an obligation to substantiate the claim that their criteria assure institutional quality? Or, should accrediting associations consider their quality assurance claims valid until research concludes otherwise?

Accreditation criteria do not emerge out of empirical research. They grow out of experienced educators' judgments as to what characteristics constitute a reputable institution. In this respect, accreditation criteria resemble moral precepts or criteria for club membership.

Accreditation criteria, however, function in a broader context. They make factual assumptions about institutional quality that can affect unaccredited institutions very adversely. Accreditation in many instances may be synonymous with institutional survival. Accreditation decisions should continue to be based on the opinions of professional educators, but their judgments should rest on solid empirical and philosophical grounds.

In identifying underlying assumptions of accreditation criteria, this review questions the philosophical basis of accreditation criteria. This review also points out the frail empirical basis of many accrediting association criteria. It further illustrates the difficulties in substantiating any indirect approach to quality assurance. Other approaches to quality assurance through a system of regional accreditation merit investigation.⁴

Notes

¹For example, the North Central Association and the Western Association refer to their evaluative criteria as "guidelines" while the other four regional associations call these criteria "standards." Whether their approach to evaluation emphasizes a very general guide or specific standards, each regional accrediting association points to several areas considered important in assessing the quality of postsecondary institutions.

²This study assumes that any definition of institutional quality relates in some way to the academic performance of an institution's students or the intellectual achievement of its graduates.

³Regional accrediting association or regional accreditation refers to the dominant perspective of the six regional accrediting associations. The published evaluation criteria of the six regional accrediting associations share many common features, more than enough to merit dealing with them collectively, but one or two associations may vary from other associations on a given point.

⁴For a discussion of other approaches to quality assurance see [23].

References

1. Astin, A. W. "A Re-examination of College Productivity." Journal of Educational Psychology, 52 (1961), 173-78.

2. Astin, A. W. "Undergraduate Achievement and Institutional Excellence." Science, 161 (1968), 661-68.

3. Cartter, A. M. An Assessment of Quality in Graduate Education. Washington D.C.: American Council on Education, 1966.

4. Chickering, A. W., J. McDowell, and D. Campagna. "Institutional Differences and Student Development." Journal of Educational Psychology, 60 (1969), 315-26.

5. Feldman, K. A., and T. M. Newcomb. The Impact of College on Students, Vol. I. San Francisco: Jossey-Bass, 1969.

6. Gross, E., and P. V. Grambsch. University Goals and Academic Power. Washington D.C.: American Council on Education, 1968. [[210]]

7. Hefferlin, J. B. L. "Avoiding Cut-Rate Credits and Discount Degrees." In Planning Non-Traditional Programs, edited by K. P. Cross and J. R. Valley. San Francisco: Jossey-Bass, 1974.

8. Holland, J. L. "Undergraduate Origins of American Scientists." Science, 126 (1957), 433-37.

9. Jacob, P. E. Changing Values in College: An Exploratory Study of the Impact of College Teaching. New York: Harper and Row, 1957.

10. Knapp, R. H., and H. B. Goodrich. Origins of American Scientists. Chicago: University of Chicago Press, 1952.

11. Learned, W. S., and B. D. Wood. The Student and His Knowledge. New York: The Carnegie Foundation for the Advancement of Teaching, 1938.

12. Lehman, I. J. "Changes in Critical Thinking, Attitudes, and Values From Freshman to Senior Years." Journal of Educational Psychology, 54 (1963), 305-15.

13. Middle States Association of Colleges and Schools. Characteristics of Excellence in Higher Education. Philadelphia: MSACS, 1978.

14. New England Association of Schools and Colleges. Standards for Membership-Institutions of Higher Education. Burlington, Mass.: NEASC, 1976.

15. Nichols, R. C. "Effects of Various College Characteristics on Student Aptitude Test Scores." Journal of Educational Psychology, 55 (1964), 45-54.

16. North Central Association of Colleges and Schools. Handbook on Accreditation. Boulder, Colo.: NCACS, 1977.

17. Northwest Association of Schools and Colleges. Manual of Standards and Guide for Self-Study. Seattle: NASC, 1975.

18. Orlans, H. Private Accreditation and Public Eligibility. Lexington, Mass.: Lexington Books, 1975.

19. Rock, D. A., J. A. Centra, and R. L. Linn. "Relationships between College Characteristics and Student Achievement." American Educational Research Journal, 7 (1970), 109-21.

20. Southern Association of Colleges and Schools. Standards of the College Delegate Assembly. Atlanta: SACS, 1976.

21. The Assessment of Quality in Graduate Education: Summary of a Multidimensional Approach. Princeton, N.J.: Educational Testing Service, 1976.

22. Thistlehwaite, D. L. "College Environments on the Development of Talent," Science, 130 (1959), 71-76.

23. Troutt, W. E. "The Quality Assurance Function of Regional Accreditation." Unpublished doctoral dissertation, George Peabody College for Teachers, 1978.

24. Western Association of Schools and Colleges. Handbook of Accreditation. Oakland, Calif.: WASC, 1975.