Select any filter and click on Apply to see results
Table of Contents
Charting the Future of US Higher Education: A Look at the Spellings Report Ten Years Later
September marked the tenth anniversary of the release of A Test of Leadership: Charting the Future of US Higher Education, the report of the Secretary of Education’s Commission on the Future of Higher Education, also known as the Spellings Commission. Blue ribbon panels rarely say much that is new or stimulate much sustained attention, but the Spellings report seemed to strike a chord and did a bit of both. In 2009, Stanford educational psychologist Richard Shavelson observed that the commission had spurred debate and prompted a variety of new initiatives to assess undergraduates’ learning and to “hold higher education accountable.”1 The report’s central theme, on target then and perhaps even more significant now, proclaims that “to meet the challenges of the 21st century, higher education must change from a system primarily based on reputation to one based on performance.”2
The frustratingly difficult challenge, fully acknowledged by the commission at the time, is how to define and assess performance. What is it that four years of college-level study is designed to accomplish?
The two predominant, if awkwardly discrete, answers to this question reflect the views of two sets of interested parties. The answer provided by students (and perhaps even more so by their parents) is “to get a good job”—an unambiguous bottom line and an outcome that is actually subject to calculations of return on investment. Members of the professoriate, who are more than a little ambivalent about a role as occupational placement specialists, have a different answer. The purpose of college in their view is to develop the student’s capacity for critical thinking, abstract analysis, and complex problem solving. This is why it does not particularly matter whether you have selected English literature or biology; if you do well, you can still get that good job at Google because you are a critical thinker.
Accordingly, there have been two important lines of activity and research in the wake of the Spellings Commission—one studying the articulation between college and successful, well-compensated careers, and the other taking a serious look at what critical thinking actually is and what in the college experience might facilitate it.
College and successful careers
One of the commission’s most specific recommendations was the creation of “a consumer-friendly information database on higher education with useful, reliable information on institutions, coupled with a search engine to enable students, parents, policymakers and others to weigh and rank comparative institutional performance.”3 Although some members of the commission were hopeful that a sophisticated assessment of the “value added” of a college education could be developed, many in the higher education community were highly and vocally skeptical. They argued, first, that one-size-fits-all testing could not possibly capture the diversity of institutions and curricula that are found in American higher education and, second, that the federal government is not the appropriate entity to direct the assessment of learning outcomes. As a result of this resistance, the following language was inserted in the relevant section of the 2008 Higher Education Opportunity Act: “Nothing in this section shall be construed to permit the Secretary to establish any criteria that specifies, defines, or prescribes the standards that accrediting agencies or associations shall use to assess any institution’s success with respect to student achievement.”4
Secretary of Education Margaret Spellings and commission cochair Charles Miller felt the key to getting better data on students, many of whom attended multiple institutions over time, was the development of a presumptively anonymous unit record system that could be used to track education and employment. That turned out to be a nonstarter, however, because of privacy concerns. David Warren of the National Association of Independent Colleges and Universities, for example, protested that “although the commission report calls for ‘non-identifiable’ data, this is inconsistent with the commission’s desire to collect data on transfer students, and to track labor-force outcomes. Finding effective ways to track the progress of individuals, without having their identities known in some originating database, seems to us to be impossible. We also fear that the existence of such a massive registry will prove irresistible to future demands for ancillary uses of the data, and for additions to the data for non-educational purposes.”5
In an attempt to preempt a federal system for evaluating educational outcomes, the Association of Public and Land-grant Universities and the American Association of State Colleges and Universities created the Voluntary System of Accountability, whose College Portrait website enables comparisons among participating public universities based on a variety of metrics, including class size, campus safety, tuition, student satisfaction, and learning outcomes.6 Many public institutions opted not to participate, however, and private higher education was not included in the process.
The Obama administration picked up the ball in 2013, creating the interactive College Scorecard, which “provides students and families the critical information they need to make smart decisions about where to enroll for higher education.”7 The Department of Education described this as part of President Obama’s broader effort to hold colleges accountable for cost, value, and quality. Ironically, because of the statutory prohibitions on assessing learning outcomes—and, of course, the immense difficulties of definition and measurement—the scorecard is built around just three fully quantitative measures: tuition, graduation rates, and the average income of graduates. Parents and prospective students can use the scorecard to calculate a crude return on investment by comparing tuition and postgraduation salaries.
The scorecard raises the fundamental question: Is college really worth it? Given the prominence of billionaire college dropouts Bill Gates and Mark Zuckerberg and fellow billionaire Peter Thiel’s fellowship initiative, which annually awards $100,000 to twenty people under twenty years old in order to spur them to drop out of college and create their own ventures, the question is a serious one. Economists have been crunching the numbers on this question for years. Most commonly reported is an annualized return on investment of 10 percent, which makes completion of a four-year college degree pretty attractive. There is a lot of individual variation, of course, and there are numerous anecdotes about Princeton history majors flipping burgers. A recent report in Science concludes that the earnings gap between college and high school graduates has more than doubled in the United States over the past three decades and is now reaching a lifetime differential of about half a million dollars, including adjustments for tuition, inflation, and foregone income while attending school.8 Although student debt remains a significant problem, the average debt of about $29,000 doesn’t look quite so intimidating.
While it may be understandable that prospective students and their parents are interested in return on investment, the average salary of college graduates is a very crude and partial index of what an undergraduate education is designed to accomplish.
College and learning outcomes
There is clearly something of a student assessment movement afoot. Experienced observers note wryly that the impulse to try to assess college learning outcomes waxes and wanes with the rhythm of the hand-wringing of blue ribbon commissions, the public perception of tuition and student debt crises, and various announcements of critical shortages of highly skilled workers. Before the release of the Spellings report, the outcomes-assessment frenzy was driven, in part, by the Organization for Economic Cooperation and Development’s comparisons of national higher education systems—comparisons in which the United States appeared somewhere in the undistinguished middle of the pack on most measures. Moreover, as European college students were actively moving around the European Union and transferring credits, the so-called Bologna Process led to a variety of creative assessment initiatives and credit-hour evaluations. This process was imported to the United States by Lumina Foundation, which encourages disciplinary “tuning” and created a “degree qualifications profile” that identifies high-level intellectual skills.9
Because of the broad resistance to the Spellings Commission’s call for direct federal engagement in the assessment of higher education outcomes, the Bush administration and, later, the Obama administration turned to a Plan B: working through the existing six primary regional higher education accreditation entities to tie the renewal of accreditation every five years to serious efforts on the part of colleges and universities under review to create and utilize meaningful learning outcomes measures. The idea is that the faculty and administration of these institutions know the local conditions best and can most appropriately tailor the process—self-assessment to preempt federal assessment. This is certainly politically viable, but it is not clear that the incentives are in place for much more than various forms of symbolic compliance.
An actual failure of an institution to be reaccredited is extraordinarily rare, so the positive effects of the process tend to be indirect. The players in this process are hardworking and earnest, and from time to time study groups of faculty and administrators do get excited about the potential of the assessment process to produce real reform. But it is relatively rare that the meetings, PowerPoint presentations, and five-year plans full of student learning outcomes actually make a difference in the typical classroom. Historians note that institutions of higher education over the last one thousand years have been almost unique in their capacity to resist change—a phenomenon well captured by the quip that it is easier to move a graveyard than to change a college curriculum.
Nonetheless, there are signs that meaningful structural reform may be underway. This is due not to a single historical impetus, but rather to a confluence of forces. During the last decade, because of extreme economic pressures on state-level support for higher education, state schools lost 30–45 percent of their state funding. Further, as a result of the fact that 70 percent of college costs are staff related, the usual increased efficiencies from technological advances are not in evidence. The result is that, controlling for inflation, tuition has risen over the last decade by 25 percent for private, nonprofit institutions and by 34 percent for public institutions.10 It is unrealistic to imagine that these growth rates can continue.
Is it actually possible to meaningfully assess intellectual skills, such as critical thinking, creativity, and problem solving? Although many have expressed skepticism over the years, the Collegiate Learning Assessment (CLA) initiative, which was already well underway when the Spellings Commission was convened and was approvingly acknowledged in the final report, has turned out to be a key development. The CLA indirectly acknowledges that the facts and figures remembered from a college classroom—the sorts of things captured by multiple-choice tests—may help you on Jeopardy but are unlikely to be of much direct use in a professional career.
The advantage gained from college is, presumably, the ability to critically evaluate and make sense of new information and new situations. The CLA exercises address this directly by assessing written communication skills in response to performance tasks derived from a domain of real-world, on-the-job decision making. Test-takers are confronted with a critical situation, often involving conflicting indicators, and are required to evaluate pros and cons, make a recommendation, and successfully articulate their reasoning. The test is conducted entirely online, with responses scored by machine or by a trained cadre of readers who grade the answers online.
Although the CLA has its critics, the fact that it exists and is widely used represents an important development in the culture of higher education and signals a new seriousness about evaluation and outcomes. That’s the good news. The not-so-good news is that a major study of CLA results found that only minimal learning gains are derived from the college experience. Richard Arum and Josipa Roksa tracked 2,300 traditional-age college students who were enrolled at a range of four-year colleges and universities. They found that 45 percent of the students “did not demonstrate any statistically significant improvement in CLA performance during the first two years of college,” and that 36 percent “did not demonstrate any significant improvement in learning” over four years of college.11 And when in evidence, the improvements were statistically modest.
Many observers were not terribly surprised by these findings and agreed with the authors’ conclusions that too few college courses assign much reading or require much writing, and that too many undergraduates drift through college without a clear sense of purpose or an appreciation of how their classwork may ultimately relate to professional achievement. If this is largely true, then perhaps we should not blame the students but rather the professoriate, which views the world through the lens of disciplinary specialization. The faculty member who takes student careers or even a holistic view of critical thinking seriously is a rarity. If careers and critical thinking are the keys to collegiate performance, then we have a challenge in moving from a system primarily based on reputation to one based on performance.
Ten years after the release of the Spellings report, American higher education has made demonstrable progress in taking its performance more seriously. One would be hard pressed to argue that this was a direct result of the commission’s report; it seems to result from a shift in the zeitgeist, albeit a very gradual one. In the world of accreditation and, increasingly, in higher education administration, the message has been received and understood. In the world of the teaching faculty? Not so much.
A more dramatic change in higher education may require an external shock like the Sputnik crisis of the 1950s. The advent of the MOOC and the growth of for-profit higher education have not had much of an effect on educational practice in traditional public and private higher education. The perception of a crisis in student debt has potential. But by their nature, financial dynamics move slowly. As a result, in the absence of some sort of sudden student debt meltdown, a crisis mentality is unlikely to take hold. So we are likely to witness a much more measured rate of change.
The next ten years
If the spirit of the Spellings Commission’s call for a focus on performance rather than reputation in higher education has resonated at all in the last decade, it has been at the institutional level. Concerns about university budgets and tuition increases, about accreditation and the clarification of educational objectives and assessment of learning outcomes, seldom make themselves felt in classrooms and faculty offices.
Faculty skepticism makes sense. Assessment may well be seen as simply asking for trouble, and there are no obvious incentives for faculty members to pursue such experimentation with much energy. For many faculty, “assessment” probably resonates with the curious phenomenon of student evaluation questionnaires. There is little evidence that a student’s “liking” a course is correlated with demonstrable progress on learning outcomes. Indeed, there may well be an inverse correlation between the difficulty and time demands of a course and student satisfaction. And it is well known that “easy A’s” often correlate with customer satisfaction.
There is a further complication. To the extent that evidence of successful teaching is valued and rewarded, the process takes place largely within each academic department. As a result, evaluation is conducted using the specialized lens of expertise esteemed in particular disciplines, rather than elusive and broadly defined questions of critical thinking or intellectual integration.
So if the next ten years are to distinguish themselves from the last ten, especially concerning faculty engagement, we are going to need a deus ex machina or some sort of environmental intervention to shift the direction of this narrative arc. Any candidates?
Well it turns out that there is a candidate. It is the machina after all (this time minus the deus), a dramatic growth in attention to instructional technology. Some of the changing technical environment is primarily mechanical—the video projector replaces the blackboard, the MacBook replaces the notebook, the physical library is supplemented by variety of resources accessible to students online. But it gets especially interesting when we examine the wake of the MOOC phenomenon.
MOOC is the infelicitous acronym for “massive open online course.” Free or relatively inexpensive online education based on video lectures, online readings, and interactive exercises and exams from a variety of sources has become an important enrichment of the environment of higher education. MOOCs, however, have not put traditional higher education out of business, as some were speculating when MOOCs came into popular focus in 2012. But it would be a mistake to dismiss the online hubbub as just a passing fashion. Perhaps more important than the MOOC itself is what might be termed “the wake of the MOOC.”
How can the techniques of the MOOC be used to enrich and extend the reach of traditional education and, importantly, to reduce the seemingly intransigent and challenging costs of providing quality higher education? Enter the flipped course, blended learning, and hybrid instruction through which a traditional faculty member moves the lectures, readings, and some exercises online in order to leave face-to-face classroom time for interactive problem solving, group work, and in-depth analysis and synthesis.
When faculty members begin to explore the use of technology to enhance their classes, something important and largely unexpected occurs. Some viewing at a distance may assume going digital involves a simple process of lecture capture and the digitization of course packs. But it turns out that lecture capture almost never works well. That in itself tells us something. A video of the traditional blackboard-based lecture with the instructor’s back to the camera or a PowerPoint-based lecture with an instructor reading through the slides without the prospect of actual instructor-student interaction is stunningly dull and enervating. YouTube and some ill-conceived MOOCs provide ample evidence of this. It quickly becomes clear that flipping and blending require the instructor to rethink both the learning objectives and the appropriate means to those objectives.
There is a scenario that plays out again and again as a faculty member sits down for the first time with an instructional technology specialist and asks, in effect, “How do I do this?” The response is not a list of available technical tools, but rather another question: “What are your educational objectives?” How best to design online readings, video lessons, online student discussions, and interactive modules depends critically on the nature of the intellectual material to be conveyed.
It is something of a magic moment, or at least it can be under the right circumstances. What at first might seem like a routine technical question, such as whether to use a laser pointer, turns out to be nothing of the sort. It is more like translating prose or poetry from one language to another, which requires fluency and artistry in both languages. Flipping and blending require rethinking pedagogical fundamentals. The fact that thoughtless lecture capture is a non-starter quickly becomes clear to faculty member and student alike.
What is largely missing from the scenario, so far at least, is a recognition that moving online offers dramatic new possibilities for assessment. As each student moves through the instructional materials, they leave digital footprints. This allows for adaptive strategies to cycle those students who are struggling with the material to get additional help. It allows students to choose the instructional format of particular materials that best meets their learning styles.
Some face-to-face lecturers hand out three-by-five cards for an every-student-response exercise. Others experiment with wireless hand-held clickers for student responses. Online, the opportunity for an every-student response is not only continuous and ongoing, it is essential for successful pedagogical design.
Experimentation with digitally enhanced teaching in higher education will likely move slowly and unevenly among different institutions. But the potential for engaging faculty in the assessment of student learning outcomes should not be overlooked. It will require that presidents, deans, department chairs, and faculty themselves engage in, well, some critical thinking.
1. Richard J. Shavelson, Measuring College Learning Responsibly: Accountability in a New Era (Stanford, CA: Stanford University Press, 2009).
2. The Secretary of Education’s Commission on the Future of Higher Education, A Test of Leadership: Charting the Future of U.S. Higher Education (Washington, DC: US Department of Education, 2006), 21.
4. Higher Education Opportunity Act, 20 U.S.C. § 1099b (2008).
5. David Warren, quoted in Robert Atwell, “Reactions to the Spellings Commission Report,” National Crosstalk (Fall 2006).
6. See http://www.collegeportraits.org.
7. See http://collegescorecard.ed.gov.
8. David Autor, “Skills, Education, and the Rise of Earnings Inequality among the ‘Other 99 Percent,’” Science 344 (2014): 843.
9. See http://degreeprofile.org.
10. See Michael Leachman, Nick Albares, Kathleen Masterson, and Marlana Wallace, “Most States Have Cut School Funding, and Some Continue Cutting,” Center on Budget and Policy Priorities, January 25, 2016, http://www.cbpp.org/research/state-budget-and-tax/most-states-have-cut-school-funding-and-some-continue-cutting; William G. Bowen, Higher Education in the Digital Age (Princeton, NJ: Princeton University Press, 2013); National Center for Educational Statistics, “Tuition Costs of Colleges and Universities,” Digest of Education Statistics, 2014 (Washington, DC: US Department of Education, 2016).
11. Richard Arum and Josipa Roksa, Academically Adrift: Limited Learning on College Campuses (Chicago: University of Chicago Press, 2010), 121.
To respond to this article, e-mail email@example.com, with the author’s name on the subject line.
W. Russell Neuman is professor of media technology in the Steinhardt School of Culture, Education, and Human Development at New York University.