Liberal Education

The VALUE of Learning: Meaningful Assessment on the Rise

How well are we doing? This question is central to the enterprise of higher education—students want to know what grade was received on the paper or test, faculty want to know what it will take to reach tenure or be reappointed, admissions staff want to know how many students need to be enrolled in order to provide the tuition revenue required to pay the bills, and so forth. In essence, individuals in higher education spend a good portion of their time measuring and assessing how well they are doing and whether they are meeting expectations or goals. It is ironic, then, that higher education institutions are so often described as resistant to assessment and standards of performance.

Just over ten years ago, the report of the Secretary of Education’s Commission on the Future of Higher Education took postsecondary education to task for not being accountable for student success and challenged institutions to demonstrate serious attention to performance measures for students.1 Among the commission’s favored ways to address the perceived lack of accountability for student learning was through the utilization of standardized testing. The recommendations in the report, which came to be known as the Spellings report, followed closely on the heels of the federal No Child Left Behind Act, which also targeted the measurement of achievement, but in primary and secondary education, through increased standardized testing of all students in key areas of learning—for example, language arts and mathematics. Following the release of the Spelling’s report, significant pushback came from higher education leaders and organizations.

In response to widespread objections to the use of standardized testing in higher education, the US Department of Education issued a special call for proposals, inviting higher education partnerships to design alternative approaches to measure student learning. The Association of American Colleges and Universities (AAC&U), along with the American Association of State Colleges and Universities and the National Association of State Land Grant Universities (now the Association of Public Land-grant Universities), received a grant from the department for a project called Rising to the Challenge: Meaningful Assessment of Student Learning. Through this project, AAC&U oversaw the development of sixteen rubrics, each keyed to a specific learning outcome that faculty and employers alike regard as essential to success in life and employment after college.

AAC&U’s work on assessment

For well over two decades—beginning long before the release of the Spellings report—AAC&U has worked with faculty and other educational professionals, students, regional and professional accreditors, and employers to develop responsive curricula and to identify learning outcomes that are essential for success in life, democratic society, and careers in a global environment. AAC&U’s work in this area has been guided by four underlying principles: (1) the measurement of student success should be multifaceted, (2) expected learning outcomes should reflect broad consensus among educators and employers, (3) education providers have valuable expertise and are central to improving student achievement, and (4) examining the actual work students produce in relation to their education yields the best evidence of how well educators and students are doing. In brief, AAC&U has for decades been engaged simultaneously with assessment for learning improvement and assessment for accountability, viewing these two strands as intertwined by necessity and practice.

Through a series of projects, surveys, and focus groups over several years, AAC&U has been exploring these principles. As the measurement of student learning has become a critical component of the accreditation process, for example, and as accreditation has become more important for student access to financial aid, AAC&U has engaged with regional and professional accrediting organizations to encourage their recognition of the need for multiple measures to satisfy internal academic and external policymaking audiences. Indeed, while the Spellings Commission was preparing its 2006 report, AAC&U was already summarizing a decade’s worth of findings from surveys and roundtables of employers as well as results from work with faculty across all types of campuses and organizations.

In a 2007 report, the National Leadership Council for AAC&U’s Liberal Education and America’s Promise (LEAP) initiative described and documented a broad consensus on a particular set of learning outcomes that are closely linked with academic success, employability, and civic engagement. The report articulated these consensus expectations for college learning as the LEAP Essential Learning Outcomes, which have been formally adopted and adapted by higher education organizations across the country and abroad (see sidebar).

The LEAP Essential Learning Outcomes
Beginning in school, and continuing at successively higher levels across their college studies, students should prepare for twenty- first-century challenges by gaining:

Knowledge of Human Cultures and
the Physical and Natural World

•    Through study in the sciences and mathematics, social sciences, humanities, histories, languages, and the arts
Focused by engagement with big questions, both contemporary and enduring

Intellectual and Practical Skills, Including
•    Inquiry and analysis
•    Critical and creative thinking
•    Written and oral communication
•    Quantitative literacy
•    Information literacy
•    Teamwork and problem solving
Practiced extensively, across the curriculum, in the context of progressively more challenging problems, projects, and standards for performance

Personal and Social Responsibility, Including
•    Civic knowledge and engagement—local and global
•    Intercultural knowledge and competence
•    Ethical reasoning and action
•    Foundations and skills for lifelong learning
Anchored through active involvement with diverse communities and real-world challenges

Integrative and Applied Learning, Including
•    Synthesis and advanced accomplishment across general and specialized studies
Demonstrated through the application of knowledge, skills, and responsibilities to new settings and complex problems

Contrary to the approach recommended in the Spellings report, AAC&U’s work has been centered on the recognition that, in order to achieve the learning that educators, employers, and the broader public say they expect of college graduates, faculty and other educators must be at the center of the improvement process. Of course, a key premise of the centrality of educators is that higher education does indeed promote higher-order learning, which involves mentored and mediated assistance as learners move into new or unchartered spaces and subjects. A contingent premise is that the assignments that accompany the instruction require students to demonstrate the expected learning outcomes at a desired level of proficiency or achievement. Therefore, the work that students produce in response to assignments or prompts from educators would logically be the best representation of the students’ learning.

A role for technology

One of the twenty-first-century drivers of life, both in and out of the academy, is the explosion in technology, which creates new possibilities for information processing and communication. Technological change is a given for today’s students, and it needs to become an integral part of the decision-making process within higher education organizations and institutions. The biggest cautionary point may be the inclination to fixate, at least temporarily, on the next shiny technological thing to appear on the scene. Whether it be MOOCs (massive open online courses) or data analytics (mining data points and throwing them into correlational relationships to increase retention and graduation success), there always is enough benefit to engender investment and exploration. When the results fail to live up to expectations—even when there are observable benefits—we simply move on to the next shiny thing that emerges to capture primacy.

In Open and Integrative: Designing Liberal Education for the New Digital Ecosystem, Randy Bass and Bret Eynon offer a singularly insightful exploration of the nexus between higher education and technology.2 They argue for an approach to technology that is in service of student learning, rather than technology utilization per se. They call not for breaking the educational process into discrete and disconnected pieces that promise greater efficiency or cost savings, but for a focus on how to integrate and make meaning of learning in ways that also yield efficiencies and cost savings through higher rates of student retention, graduation, and engagement. In other words, Bass and Eynon argue for re-bundling higher education, rather than unbundling it.

A prominent technological medium already utilized across higher education institutions is the eportfolio. When done well, the use of eportfolios can be a high-impact practice. That is, eportfolio use has been identified as one of a set of educational practices that share several traits: they demand focused time and effort, facilitate learning outside and inside the classroom, require meaningful interactions with faculty and students, encourage collaboration with diverse others, and provide frequent and substantive feedback to students and faculty.

As George Kuh recently observed, “the eportfolio is much more than a just-in-time twenty-first-century electronic record keeping system. It is an intentionally designed instructional approach that among other advantages prompts students to periodically reflect on and deepen what they are learning and helps them connect and make sense of their various experiences inside and outside the classroom that—taken together—add up to more than the sum of their parts.” As Kuh explains, the eportfolio has the potential to “serve as a portable, expandable, updatable vehicle for accumulating and presenting evidence of authentic student accomplishment including the curation of specific proficiencies and dispositions at given points in time”; to “document, integrate, and enhance the positive effects of other [high-impact practices]”; and to “make the extended educational transcript (something like a cocurricular transcript on steroids) initiative now being tested even more attractive to employers, institutions, and students themselves.”3

Perhaps most significantly, eportfolios can be used effectively at higher education institutions of all kinds and to benefit all students. Further, the eportfolio accommodates the use of smart phones, social media platforms, networking, information search and processing, and other types of technology associated with today’s students as well as learning gained in online environments and from prior or parallel experiences.

The role of accreditation

Assessment of learning in higher education has been spurred and sustained primarily through regional and specialized accreditation. As a result, much of the culture among higher education providers has centered on compliance with accreditation standards. This, in turn, has

been reflected in policy circles by demands for fairly simple metrics to measure student success. It is unfortunate that the assessment movement has been directed into this unsatisfying sidetrack.

As a critical component of quality higher education, accreditation has historically offered a way to ensure that a purveyor of higher education is financially capable of sustaining teaching and learning throughout the time needed to complete a course of study (hopefully culminating in a recognized credential) and that it provides the teaching and learning through well qualified and prepared individuals, curricula, and associated support services. The assumption has been that the ensuing learning would be of sufficient quality to warrant the investment.

Accreditation has offered accreditation-seekers the opportunity to make their own case and to reflect upon their enterprise in terms of demonstrating the myriad standards of capability and implementation, at least at an accepted minimal or average level of performance. By and large, higher education institutions have responded well to the standards, sufficiently meeting the expectations for reaccreditation with few, if any, recommendations for improvement in processes and information. However, as the economic situation has changed and new demands from employers, students and their families, legislators, and others have focused attention on the outcomes for graduates, the nature of the expectations has also changed. Institutions are now required to produce more robust and varied evidence of what students are learning in preparation for life after college.

The good news, despite many claims to the contrary, is that accreditors now increasingly require more and better evidence of expected student learning. Multiple surveys and focus groups have shown that, although they sometimes use different language, educators and employers agree on a core set of essential and robust learning outcomes that all college graduates should be able to demonstrate at stated levels of competence or proficiency.

The challenge has been to determine how adequately to measure the desired levels of learning across the agreed upon outcomes. In the absence of widely accepted measures of learning to judge student success, proxies for learning have been used as the primary default metrics. These proxy measures—job placements, for example, or salaries for first jobs—
do not necessarily reflect either the core learning that educators believe graduates need and deserve for a lifetime of flourishing or what employers indicate they seek in hiring college graduates.4 It was precisely this lack of widely vetted measures of essential student learning outcomes that prompted the AAC&U-led Rising to the Challenge proposal to the US Department of Education in the wake of the Spellings report a decade ago and that led to the creation of AAC&U’s Valid Assessment of Learning in Undergraduate Education (VALUE) rubrics.

The VALUE rubrics

Developed by teams of faculty and other educators from public and private higher education institutions across the country, the VALUE rubrics are based on previously existing rubrics and on research related to the key components and dimensions of each of sixteen learning outcomes.5 Before they were released publicly in the fall of 2009, the rubrics underwent two to three rounds of testing and revision by faculty on over a hundred campuses. The VALUE rubrics were designed to reflect expected levels of learning as demonstrated by the work students produce in response to curricular and cocurricular assignments across two- and four-year degree programs. That is, the same rubric can be used to assess student achievement across different types of institutions, across different disciplinary areas, and by faculty from different fields of study.

Rather than assess what students cannot do, the VALUE rubrics were designed to assess what students can do and the level of learning demonstrated by their actual work. The rubrics reflect the level of learning, not the year in school; diverse students have diverse patterns of learning strengths and weaknesses that are not necessarily parallel to their year in school. The rubrics capture and reflect variation in learning by providing assessment across the multiple, key dimensions of the learning that underlay each of the learning outcomes.

Thousands of educational organizations have explored and used the VALUE rubrics since 2009, modifying and adapting them as needed to conform to their own missions or priorities. The consistent feedback from educators has indicated that the rubrics effectively capture the key dimensions of learning for each outcome, that the information gained through the use of the rubrics leads to improved pedagogy and assignment construction, and that the availability of the assessment results enables discussion of student learning and engenders faculty collaboration within and across departments and colleges.

The VALUE/Multi-State Collaborative

Beyond the anecdotal evidence that has emerged since 2009 from educational institutions and organizations using the VALUE rubrics for assessment, a national initiative—called the VALUE/Multi-State Collaborative to Advance Quality Student Learning—is now underway to determine whether rubric-based assessment of student learning outcomes can be taken effectively to scale. A partnership between AAC&U and the State Higher Education Executive Officers Association, this national initiative involves nearly a hundred institutions—two year and four year, public and private—and thirteen state higher education offices. The participants have committed to collect samples of student work from their respective institutions, identify faculty to score the work on three or four learning outcomes, and use the VALUE rubrics as the shared metric to assess the student work samples. In addition, the initiative is conducting extensive reliability and validity studies and analyses of disaggregated results to enable examination of student demographic and institutional variables in relation to the assessment findings.

After engaging with faculty in the application of VALUE rubrics to assess student learning, Dan Berrett reported on the VALUE/Multi-State Collaborative for the Chronicle of Higher Education. “It’s the kind of acronym-heavy, jargon-laced endeavor that’s easily overlooked,” he observed. “But by measuring students’ intellectual skills, it might turn out to provide telling insight into one of higher education’s central functions.” Berrett noted that “what makes the effort notable is its subject of analysis: the authentic stuff of college—the homework, problem sets, and papers that students regularly produce. From those, evaluators . . . can produce generalizable and comparable findings across disciplines, institutions, and states about students’ critical-thinking, writing, and quantitative-reasoning skills.” He concluded that, citing the view of George Kuh, “the rubrics’ fundamental connection to the daily work of education . . . means this attempt may succeed where others have foundered.”6

Assessment for learning is happening

Simplistic dichotomizing conceptions of assessment—accountability versus improvement, faculty led versus externally imposed, compliance versus learner centered—are being replaced by a recognition of the importance of demonstrating student learning and a reconsideration of what constitutes the best evidence of learning. Over the past ten years, higher education faculty and institutions have begun to meet the challenge of accountability by placing faculty and educator expertise and judgment at the center of assessment efforts that have student learning improvement as their primary purpose.

Building on its earlier work, AAC&U has helped facilitate the transformation of assessment into a high-impact practice. Assessment can be used effectively to improve student learning—not only content knowledge, but also the skills and abilities needed to apply knowledge to complex, unscripted problems and for career success in an ever-changing, technology-driven global environment. Indeed, we have now reached a point where learning improvement can be demonstrated in meaningful ways to students, ourselves, and those outside the academy.

Notes

1. The Secretary of Education’s Commission on the Future of Higher Education, A Test of Leadership: Charting the Future of US Higher Education (Washington, DC: US Department of Education, 2006).

2. Randy Bass and Bret Eynon, Open and Integrative: Designing Liberal Education for the New Digital Ecosystem (Washington, DC: Association of American Colleges and Universities, 2016).

3. George D. Kuh, “And Now There Are Eleven,” foreword to High-Impact ePortfolio Practice: A Catalyst for Student, Faculty, and Institutional Learning, by Bret Eynon and Laura Gambino (Sterling, VA: Stylus Publishing, 2017).

4. See, for example, Hart Research Associates, It Takes More Than a Major: Employer Priorities for College Learning and Student Success (Washington, DC: Association of American Colleges and Universities, 2013).

5. The VALUE rubrics are available for download from http://www.aacu.org/value/rubrics.

6. Dan Berrett, “The Next Great Hope for Measuring Learning,” Chronicle of Higher Education, October 16, 2016, http://www.chronicle.com/article/The-Next-Great-Hope-for/238075.

To respond to this article, e-mail liberaled@aacu.org, with the author’s name on the subject line.


Terrel Rhodes is vice president for quality, curriculum, and assessment at the Association of American Colleges and Universities, where he also serves as executive director of the Valid Assessment of Learning in Undergraduate Education project.

Previous Issues