Statements and Letters

Our Students’ Best Work: A Framework for Accountability Worthy of Our Mission Also available for purchase or download (pdf)

Preface

On behalf of the Association of American Colleges and Universities (AAC&U), we are pleased to present Our Students’ Best Work: A Framework for Accountability Worthy of Our Mission. This statement, framed and approved by the AAC&U Board of Directors, is designed to help campuses respond to calls for greater accountability in ways that strengthen the quality of student learning in college.

The national dialogue about these issues is becoming intense, with discussions taking place in statehouses across the country and in the U.S. Congress. We believe that it is essential that they focus on the knowledge, skills, and capacities that are most important for today’s students. They also should take into account the best campus practices already developed to advance and assess these outcomes.

Thoughtful and forceful leadership from within the academy itself on both assessment and accountability is more essential today than ever. Educational leaders are already doing good work to define what academic excellence really means in today’s world. It is vitally important that we build on this foundation to find improved ways to demonstrate achievement of academic excellence by students and institutions.

This statement is designed to help colleges and universities as they continue to improve upon strategies of assessment and accountability appropriate to their own missions. It also is intended to assist institutions as they respond to questions about accountability from both policy makers and members of the general public.

We hope Our Students’ Best Work will prove useful in your own campus efforts and we encourage you to share with AAC&U the promising assessment strategies that you are developing. We will do our best to disseminate your work throughout the AAC&U community and among external stakeholders throughout the country.

Elisabeth A. Zinser, Chair of the Board of Directors

Carol Geary Schneider, President

Introduction

For the past two decades, the Association of American Colleges and Universities (AAC&U) has repeatedly called for the academy to take responsibility for assessing the quality of student learning in college. The vision developed in its 1984 report, Integrity in the College Curriculum, and, most recently, in its 2002 report, Greater Expectations: A New Vision for Learning as a Nation Goes to College, contains three elements:

• A clearly articulated, collective conception of the qualities of a college-educated person
• Intentionality and coherence in educational programs to cultivate those qualities
• Assessment to determine the extent to which the desired learning has been achieved

Yet, despite the development over the past two decades of a veritable “assessment movement,” too many institutions and programs still are unable to answer legitimate questions about what their students are learning in college.

The lack of evidence on student learning outcomes has proved damaging. In the absence of consistent and broad-based leadership on assessment and accountability from the academy, a politically popular accountability ideology has swept statehouses across the country and is capturing the allegiance of many lawmakers of both major parties. This alternative ideology, in brief, threatens to shortchange accountability by holding the academy to standards for students’ higher learning that are much too low.

While specific accountability proposals from this new source vary, they have one feature in common. They regard a particular kind of standardized testing—multiplechoice, “one-best-answer” tests—as the right way to assess student knowledge and hold the academy “accountable.” Leaders of the testing industry encourage enthusiasm for this kind of thinking by extolling the virtues of their tests.

Interest in mass testing has been fueled nationally by the No Child Left Behind law that mandates school testing in multiple grades. Schools that do not measure up on the chosen tests face serious consequences. While it is certainly a major step forward to hold the schools accountable for all students’ academic achievement, knowledgeable researchers have pointed to multiple problems with the state tests being used at various levels, including in high schools. For example, many state high school tests still focus only on easy-to-measure factual knowledge and reactive answers (Achieve 2002), rather than higher-order abilities such as critical thinking, evidence-based reasoning, integrative thinking, and problem solving.Most of the state tests evaluate only a ninth- or tenth-grade level of achievement (Achieve 2004).

Ignoring these problems, many policy makers now want to use the same logic and make a similar form of mass testing the focus of accountability in higher education. This would be an enormous misuse of time and scarce resources. It is the wrong approach to the challenge of holding higher education accountable.

Students study at the college level in hundreds of different academic departments and programs. These programs reflect very different communities of inquiry and practice. The kind of tests being used for school assessment cannot begin to probe the distinctive forms of excellence expected across this multitude of different fields.

Assessing what students have learned in colleges and universities requires a sophisticated understanding both of context and of how knowledge and skills are to be used. Students typically do their best and most advanced work in their major fields of specialization, and they should be held accountable for knowledge and skills that are deemed essential at an advanced level, whether the field is physics, psychology, or pharmacy.

What is regarded as excellent writing in chemistry, for example, because of its direct, descriptive, and succinct language, is very different from the well-told analytical narrative in history or the evidence-based scan of policy alternatives appropriate to public administration. A common test of communication skills cannot probe students’ highest skill level, because advanced skill takes different forms in different fields. Professional fields such as law and medicine do not test educational accomplishments with the same generic test, and undergraduate institutions should not do so either.

This does not mean educators and educational institutions should be exempt from accountability. Rather, accountability for the highest standards of learning calls for new forms of critical inquiry and reflective practice—forms that are both appropriate to higher education’s mission and feasible in the contemporary academy.

There is considerable promise in tests now being developed that give students a small library of new materials related to a problem in a particular domain (e.g., the social sciences) and ask them both to assess the quality of the evidence and to write complex answers to questions based on the evidence (taking into account its limitations). Measures such as these come significantly closer to life’s real challenges and therefore are more appropriate as assessments of college education.

Such tests are not yet widely available. Eventually, they may complement discipline-centered assessments by providing evidence of students’ abilities to apply both knowledge and analytical skills in domains of learning such as the natural sciences, social sciences, humanities, and arts. But even if better tests emerge, standardized tests alone are an inadequate and inappropriate strategy to foster advanced learning and accountability in higher education.

What Is to Be Done?

AAC&U affirms that accountability is essential, but that the form it takes must be worthy of our mission. This means we must hold ourselves accountable for assessing our students’ best work, not generic skills and not introductory levels of learning. Any accountability framework must first, of course, respect the diversity of institutional missions and students’ educational goals in the contemporary academy. The framework suggested below is designed to accommodate differences in institutional mission while still holding higher education institutions accountable for a set of key learning outcomes that all college graduates should achieve regardless of their field of study or choice of institution.

The first step is to establish clarity about the kinds of learning that make a difference for all college graduates over time: as thoughtful people, as participants in the economy, and as citizens.

AAC&U’s more than 1100 college and university members represent the spectrum of postsecondary institutions: two-year and four-year; public and private; large and small. All are committed to ensuring that every student experiences the benefits—intellectual, economic, civic, social, and intercultural—of a well-designed and intellectually challenging liberal education.

Liberal education, as a respected educational tradition, has guided U.S. colleges and universities to unrivaled, world-class standing. Any tradition with deep historical roots necessarily adapts to reflect the many social, economic, cultural, and technological changes that occur over the years. Consider two examples. In the nineteenth century, liberal education primarily served young men who were preparing for leadership positions, often in the clergy, medicine, and law. Now, liberal education aims to be inclusive and to provide an empowering education to widely diverse students. In the twentieth century, many came to contrast liberal education with professional educa-tion and to regard it as, by definition, not “practical.” But in today’s knowledge-based economy, a good liberal education embraces science and new technologies, hands-on research, global knowledge, teamwork, cross-cultural learning, active engagement with the world beyond the academy, and a commitment to lifelong learning, as well as the acquisition of knowledge and skills.

These forms of learning provide a strong foundation for success in a dynamic economy. They are also essential as a foundation for civic participation and for a meaningful life.

The opposite of liberal education is narrow, situation-specific training. While situation-specific training has many good uses, by itself it is insufficient preparation for a world characterized by complexity, conflicting judgments, and accelerating change. Even students in technical fields, therefore, need and deserve the complementary benefits of a liberal education to help them make sense of the social and environmental contexts in which they will use their skills, and to prepare them for lifelong work rather than just an initial job.

In short, a contemporary liberal education rests on a vital historic tradition and reflects current realities. New frameworks for educational accountability should focus on students’ high level of achievement in the college outcomes that characterize a liberal education.

Focusing on Key Educational Outcomes

The public has questions about the quality of education that colleges and universities are providing, and it deserves to know how well students are doing. It is time for leaders of education to embrace a small number of highly valued and widely affirmed educational goals, establish high standards for each, and assess their achievement across the curriculum.

AAC&U has summarized several aims of undergraduate liberal education in its 2002 report, Greater Expectations: ANew Vision for Learning as a Nation Goes to College. We propose selecting from them five key outcomes as a concentrated focus for assessment.

In brief, the outcomes we propose are:

1. strong analytical, communication, quantitative, and information skills— achieved and demonstrated through learning in a range of fields, settings, and media, and through advanced studies in one or more areas of concentration;
2. deep understanding of and hands-on experience with the inquiry practices of disciplines that explore the natural, social, and cultural realms—achieved and demonstrated through studies that build conceptual knowledge by engaging learners in concepts and modes of inquiry that are basic to the natural sciences, social sciences, humanities, and arts;
3. intercultural knowledge and collaborative problem-solving skills—achieved and demonstrated in a variety of collaborative contexts (classroom, communitybased, international, and online) that prepare students both for democratic citizenship and for work;
4. a proactive sense of responsibility for individual, civic, and social choices— achieved and demonstrated through forms of learning that connect knowledge, skills, values, and public action, and through reflection on students’ own roles and responsibilities in social and civic contexts;
5. habits of mind that foster integrative thinking and the ability to transfer skills and knowledge from one setting to another—achieved and demonstrated through advanced research and/or creative projects in which students take the primary responsibility for framing questions, carrying out an analysis, and producing work of substantial complexity and quality.

These outcomes for student learning are not arbitrarily chosen. Rather, there is an emerging consensus across many professions, the business community, civic leadership, and the academy that these liberal education capabilities are valuable for work, citizenship, and a satisfying life.

In a recent comparison of their standards for accreditation, for example, leaders from several professions, the regional accreditation organizations, industry, and educational associations discovered that they all viewed the outcomes listed above as integral aspects of a good education and, in the case of the professions, of preparation for business, education, engineering, and nursing (AAC&U 2004).

These outcomes are valuable, it is now widely agreed, because they prepare students to bring knowledge, experience, and reflective judgment to the complexity of the contemporary world. They give graduates a strong foundation to deal with issues that are challenging, unscripted, and often vigorously contested. They teach students to find and evaluate evidence and to take into account competing perspectives as they form judgments about significant questions. They help develop both a respect for the value of human diversity and a set of internal values that serve as a compass in an era of accelerating change.

Cultivating and Assessing Liberal Education Outcomes

In proposing this focus for accountability, we offer the following guiding principles:

• These outcomes do not emerge from taking only one or two relevant courses. Rather, these are complex capabilities which are appropriately cultivated from school through the final year of college, at increasingly higher levels of challenge and expected accomplishment.
• There are many ways of fostering these outcomes for today’s diverse students and academic institutions. AAC&U does not endorse a “one-size-fits-all” approach to college learning or its assessment.
• College education should help all students achieve the array of liberal education outcomes described above, whatever their particular areas of study or major field(s).
• These liberal education outcomes will reach their highest level of cultivation in the context of the student’s area of specialization or major field(s), where advanced achievement appropriately takes different forms.

In other words, even though the outcomes characteristic of liberal education can be described generally, they must be cultivated and assessed in context. Analytical skill, for example, has one kind of applied meaning for an English major, and a quite different kind of applied meaning for an engineer. Similarly, the civic, social, or intercultural questions faced by a student preparing for teaching are likely to be very different from those encountered by a student studying economics or biology.

These insights point toward a curricular strategy for educational accountability, rather than a reliance on standardized and generic testing. The previously listed outcomes of liberal learning should be addressed and cultivated throughout the entire educational experience. Whatever the field of study, therefore, a student’s progress in achieving liberal education outcomes ought to be assessed periodically from the initial to the final year, in both general education and the chosen major field(s).

Within the college or university context, a comprehensive accountability and assessment framework should include the following elements:

1. Orientation should be provided for the student during the first year about the institution’s expectations for important learning outcomes, and a diagnostic assessment of each student’s demonstrated accomplishment and expected further progress in relation to these outcomes should be made.
2. A plan of study, constructed with the student’s advisor, should transparently connect the expected outcomes to the student’s choice of courses and major field(s).
3. Milestone assessments as students progress in their studies in both general education and the major should be tied to key outcomes with timely feedback to the student and his or her advisor. These assessments should be aligned between two-year and four-year campuses so that successful transfers are possible, and they can be compiled in a portfolio that demonstrates each student’s progress.
4. Capstone or culminating experiences in the major field(s) allow students to actively demonstrate their cumulative accomplishments in liberal education. The capstone is a critical element of this framework because it provides a designated place in the regular curriculum where students do their best work. It should be conceived as both a culminating integrative experience and as the centerpiece of the effort to assess sophisticated learning.

An important foundation for this approach to accountability has already been laid at many colleges and universities. The National Survey of Student Engagement (NSSE) reports that 58 percent of college seniors currently are expected to complete a capstone or culminating experience of some kind (NSSE 2003). Typically, capstones are completed in the student’s major field, although some institutions require capstone experiences in general education as well.

Many other institutions and programs already require students to compile portfolios of their work as a requirement for graduation. Experiments are underway across the country to put such portfolios online.

Capstone projects and portfolios provide promising anchors for a meaningful approach to educational accountability. They provide contexts in which student work can be assessed for the crosscutting outcomes of liberal education described above as well as for conceptual knowledge and skills appropriate to the students’ selected major(s).

In some cases, assignments for portfolios and capstones may need redesign to encompass the array of important liberal education outcomes. In other cases, the primary change needed will be a fuller reading of the available evidence on student’s cumulative achievement of the key liberal education outcomes.

For every institution, the first accountability questions that should be asked are these:

1. Are all students expected to produce advanced, culminating work?
2. Is this culminating work assessed for broad liberal education outcomes as well as knowledge relevant to the specific field?
3. Have standards been established and made public for what is expected at this advanced level in each program?
4. Are examples of this advanced work and the related standards regularly peer reviewed in the context of accreditation?
5. Have milestone assessments been established that prepare students to meet advanced standards and, where relevant, to plan for successful transfer from one institution to another?
6. Does the curriculum effectively prepare students to meet the standards that will be expected in milestone and culminating assessments?

Summarizing Results and Reporting to the Public

In the current climate it is not enough for an institution to assess its students in ways that are grounded in the curriculum; colleges and universities also must provide useful knowledge to the public about goals, standards, accountability practices, and the quality of student learning. Common rubrics will be needed to summarize levels of student achievement across different academic fields and institutions and for particular groups of students.

But here again, much progress has been made. The National Assessment of Educational Progress grades student achievement in four levels: advanced, proficient, basic, and below basic. On each of the five outcomes that we propose as the heart of college-level learning, these four levels can be described in concrete terms and in enough detail that they can be reliably scored. Faculty members responsible for milestone and capstone assessments can be trained to judge the level of each student’s achievement on each of the five expected liberal education outcomes and on their accomplishment in their chosen fields.

A summary report to an accreditation body, a state official, or the general public can be prepared that aggregates the data across the institution. Because it may include results from many students majoring in different disciplines, a summary report can include examples to illustrate the larger meaning of its results.

Like standardized testing, this method will allow for summarizing the outcomes of student learning with a few scores. But unlike tests based on quick responses to multiple-choice questions, these will be summaries of higher-order skills such as communication, analytic ability, and integration of knowledge, and will reflect meaningful educational projects judged by professionals.

Also, when the data are available, each campus can take steps to engage faculty and students in interpreting the meaning and implications of assessment outcomes. Faculty members should use the findings as a basis for discussion and a catalyst for needed changes in the academic programs.

Ten Recommendations for a New Accountability

1. Make Liberal Education the New Standard of Excellence for All Students Liberal education should become the new standard of excellence for all students, whatever their major or anticipated career. Five key liberal education outcomes were noted above, and a more comprehensive set of learning outcomes may be found in Greater Expectations: ANew Vision for Learning as a Nation Goes to College. These outcomes are important indicators of what students need to accomplish as citizens, as workers in a particular profession or field, and as thoughtful, creative, responsible human beings.
2. Articulate Locally Owned Goals for Student Learning Outcomes Clarity about essential learning outcomes is the foundation of both a robust educational program and an accountability framework. For higher education to be accountable for liberal education outcomes, individual institutions (and systems) must translate these outcomes into goals and language that are meaningful in local contexts. Goals for student accomplishment should be developed and articulated in dialogues that include both faculty members and members of the wider community. To meet the highest standards of excellence, campus (and system) goals for student learning should be challenging, public, and evaluated.
3. Set Standards in Each Goal Area for Basic, Proficient, and Advanced Performance Levels of performance should be specified in concrete and detailed ways so that student work can bereviewed and judged similarly by different individuals. One of the important benefits of having clearly described goals and levels of achievement is that students themselves will begin to understand the standards for quality in different fields and become more capable of assessing their own learning. Another benefit is that complementary expectations for assessments in the second and final years of college will help students in two-year colleges meet local standards, while anticipating more advanced standards should they elect to transfer.
4. Develop Clear and Complementary Responsibilities between General Education and Departmental Programs for Liberal Education Outcomes It does little good to agree on valued goals for students if responsibility for cultivating them is not fixed. Similarly, the usual assignment of responsibility for general education goals to one group of faculty, typically in the liberal arts and sciences, and for specialized program goals to another group of faculty members in departmental programs, virtually guarantees a fragmented education. It is far better for students to experience their general education and major as integrated and coherent. Although specific responsibility may be assigned, it is best if there is discussion and understanding among faculty about what is expected in both parts of the curriculum, if students are encouraged to make connections between courses, and if advanced courses intentionally build on prior work.
5. Charge Departments with Responsibility for the Level and Quality of Students’ Most Advanced Work Once goals for student learning have been articulated at the campus level (and, in public higher education, at a system level), they should be translated into program-specific goals for student accomplishment. Goals should be set for general education in ways that respect an individual campus program’s particular aims, design, and character. In addition, student learning goals should be articulated within the context of academic majors. For example, while the campus as a whole may hold all students accountable for analysis, communication, and intercultural knowledge, these expectations will and should have different implications for specific departments and programs. Each department should translate campus-wide goals for liberal education into goals appropriate to the field. Departments also should articulate field-specific goals for their majors. Each department should communicate how and why these standards contribute to effective accomplishment in that field.
6. Create Milestone Assessments across the Curriculum
   Assessments of student progress in achieving goals should be built into the ongoing curriculum and embedded in designated courses or assignments in both general education and departmental majors. Assessment of student progress over time requires that campuses distinguish among advanced, proficient, basic, and below basic levels in relation to specific goals. Students should be taught to gauge their progress against high expectations for their most advanced work. No student should learn for the first time about shortfalls in meeting proficiency standards at the point of graduation.
7. Set Clear Expectations for Culminating Work Performed at a High Level of Accomplishment
   Each department should identify expected proficiency standards and culminating work—encompassing liberal education and specialized outcomes— that will both cultivate advanced knowledge and skill and demonstrate students’ cumulative learning. Culminating work may include research projects, supervised internships, capstone courses, public performances, licensure, or other validated tests in a field, and/or cumulative portfolios providing examples of student achievement in relation to specific goals.
8. Provide Periodic External Review and Validation of Assessment Practices and Standards
   There should be periodic evaluation by external reviewers of the goals, the proficiency standards, and work samples submitted by students to meet standards. Such external reviews provide validation of both the goals and standards. A representative sample of student performances in different fields will provide sufficient evidence for external feedback.
9. Make Assessment Findings Part of a Campus-wide Commitment to Faculty Inquiry and Educational Improvement
   Accountability efforts should be part of a continuous engagement with the quality of students’ actual achievement in relation to important educational goals. Each campus and department should review the quality and level of
   students’ best work, and seek ways to ensure that the curriculum provides
   repeated opportunities for students to practice and reach expected levels
   of learning. Campus reward systems should incorporate the importance of
   faculty members’ intellectual and professional leadership in both assessment and educational improvement.
10. Provide Public Accountability and Transparency
    Each college and university should make public on its Web site: a. General and departmental goals for student learning b. Proficiency expectations for rating levels of student achievement in relation to these goals c. A description of the kinds and range of performances that are used in assessing student progress (with links to different programs and departments) d. A report on student achievement levels (e.g., advanced, proficient, basic, and below basic) in relation to each goal ensure that the curriculum provides repeated opportunities for students to practice and reach expected levels of learning. Campus reward systems should incorporate the importance of faculty members’ intellectual and professional leadership in both assessment and educational improvement.

For purposes of publicity and comparison, a campus may translate department-specific assessments back into general categories (e.g., 75 percent of the students in the college of arts and sciences met a proficient standard for analytical skill and collaborative problem-solving). The institution should also make public its procedures for reviewing and validating assessment practices, standards, and findings.

These recommendations are offered as guidelines for institutions of all sorts working to demonstrate the level of student achievement of key outcomes of liberal education and to respond with integrity to calls for greater accountability.

Additional Resources

• American Council on Education. 2004. Public accountability for student learning in higher education: Issues and options.Washington, DC: ACE.
  This position paper from the Business-Higher Education Forum provides a comprehensive explanatory framework for issues of accountability for student learning in higher education. It provides a discussion of the reasons for the heightened demand for evidence about performance in higher education and a description of various approaches to learning assessments, institutional performance review, and quality review in higher education. It also includes a set of recommendations for design principles for a public accountability structure appropriate to the diverse system of American higher education.
• Association of American Colleges and Universities. 2002. Peer Review, 4 (Winter/Spring).
  This special issue of Peer Review, “Value Added Assessment of Liberal Education,” highlights an initiative of the RAND Corporation’s Council for Aid to Education’s Value Added Assessment Initiative (now called the College Learning Assessment Initiative). This initiative is a long-term project developing sophisticated assessment tools to measure the quality of undergraduate learning in America at the institutional level. This issue features articles summarizing the aims of the initiative and the importance of developing value added assessment measurement tools to raise the level of student accomplishment. It features both a review of the literature and the advantages and disadvantages of current approaches to assessing higher education quality.
• Middle States Commission on Higher Education
  On its Web site, the Middle States regional accrediting association provides many useful resources and publications on assessment and accreditation. See www.msache.org.
• Palomba, Catherine A., and Trudy W. Banta. 1999. Assessment essentials: Planning, implementing, and improving assessment in higher education. San Francisco: Jossey-Bass.
  This volume presents an overview of widely used assessment practices and provides a useful introduction to how assessment is currently being accomplished on college campuses today.
• State Higher Education Executive Officers, National Commission on Accountability in Higher Education. The National Commission on Accountability in Higher Education was organized by the national association of State Higher Education Executive Officers (SHEEO) to reflect on the state of American higher education, the states, and the nation in articulating priorities and assessing and improving performance in higher education; to articulate significant lessons and observations gleaned from the collective experience of institutions across the country; and to recommend principles and practices to help institutions, states, and the nation make continuous progress toward our shared goals for higher education. A report is planned for release in 2005.

Works Cited

Achieve, Inc. 2004. Do graduation tests measure up? Acloser look at state high school exit exams.Washington, DC: Achieve, Inc.

Achieve, Inc. 2002. Staying the course: Standards-based reform in America’s schools: Progress and prospects.Washington, DC: Achieve, Inc.

Association of American Colleges. 1985. Integrity in the college curriculum.Washington, DC: AAC.

Association of American Colleges and Universities. 2002. Greater expectations: A new vision for learning as a nation goes to college.Washington, DC: AAC&U.

Association of American Colleges and Universities. 2004. Taking responsibility for the quality of the baccalaureate degree.Washington, DC: AAC&U.

National Survey of Student Engagement. 2003. Converting data into action: Expanding the boundaries of institutional improvement. Bloomington, IN: NSSE.