Peer Review

Making Excellence Inclusive in Oregon

Motivated by institutional strategic planning and impetus from two external drivers—the release of the Spellings Report and revisions to accreditation standards—the Oregon University System (OUS) began discussions in fall 2007 on the complex task of framing student learning expectations and advancing the use of meaningful assessment not only at each of the system’s seven institutions but also across the system. In 2008, following a visit from the Association of American Colleges and Universities (AAC&U), the OUS Provosts’ Council convened the Learning Outcomes and Assessment (LO&A) Task Group, composed of one representative from each of the seven OUS campuses and staff from the Chancellor’s Office.

The LO&A initiative began with a directive to develop a framework for a system-wide approach to the assessment of student learning and accountability for student achievement. The initiative sought to help each institution and the OUS as a whole develop, interpret, and wisely use direct evidence of cumulative student learning, while considering best practices in the use of evidence to meet rising expectations for transparency and accountability.

LO&A began by mapping each university’s general education (GE) learning outcomes and comparing them with AAC&U’s LEAP outcomes. It was clear that although the methods of implementation differed at each institution, the institutions’ outcomes were closely aligned not only with each other but also with the LEAP outcomes. The LO&A concluded that GE in Oregon would be enhanced by joining AAC&U’s national work and requested that OUS be included in the LEAP-States-sponsored project called Give Students a Compass: A Tri-State Partnership for College Learning, General Education, and Underserved Student Success
(known as the Compass project).

The partnership fueled LO&A’s institutional reform work, placing the task force at the forefront of national debates. Collaboration with like-minded colleagues from across the nation gave LO&A the confidence to decline participation in the Voluntary System of Accountability—a program sponsored by the American Association of State Colleges and Universities (AASCU) and the Association of Public and Land-grant Universities (APLU) for undergraduate institutions to communicate data about learning outcomes, demographics, and students’ experiences and perceptions of their undergraduate experience. Instead we lobbied for the administration of the National Survey of Student Engagement (NSSE). Several OUS campuses have since decided to continue to use NSSE to develop robust longitudinal data on the impact of their students’ engagement in high-impact practices (HIPs), with particular attention to their effects on underrepresented students.

LO&A’s efforts to create a plan meaningful to faculty and defensible to a general public were facilitated by creation of an effective bond among group members that fostered trust and allowed frank conversations on difficult issues. This atmosphere allowed us to address the charged issues of assessment, accreditation, and accountability with common goals, despite our seven different institutions and cultures. Effective leadership redirected the conversation from reactive to proactive, from reaction to demands for accountability to external audiences to proactive attention to students and support for teaching, learning, and meaningful assessment.

Raising the Significance of Lliberal Education

Fall 2010 marked the end of the first phase of LO&A’s system-wide work, which culminated in a set of recommendations now moving through review in the Provosts’ Council, OUS, and the board. While formal institutional commitments to the LO&A process also concluded, the group has every intention of continuing in its efforts, focusing on specific issues that need more work and fostering a much deeper involvement of faculty.

The leadership, resources, and synergy provided by AAC&U enhanced LO&A’s efforts to raise the significance of liberal education across the system. Three of the seven universities in OUS (Southern Oregon University, Eastern Oregon University, and Portland State University) worked closely as Compass project beta sites with AAC&U colleagues in pilot studies while the other four institutions (Western Oregon University, University of Oregon, Oregon Institute of Technology, and Oregon State University) served as “critical friends,” thus ensuring that the work was system-wide, rather than simply site specific.

By design, work associated with the Compass project was deliberately integrated and interwoven into more general academic improvement strategies on each campus, allowing activities to support and enhance one another in a multifaceted approach to student achievement. Through the LO&A initiative, assessment coordinators on each campus enjoy the benefits of coordinated, collaborative effort. They have learned from each other and have developed lasting relationships. LEAP and the Compass project have helped to extend that collaboration to faculty, fostering greater faculty involvement and creating grass-roots pockets of talent, experience, and conviction. The value of LO&A and the Compass project flows from collaboration that creates coherence across campuses and around the system.

Compass at SOU

At Southern Oregon University (SOU), the Compass project had two primary goals: (1) to conduct a mixed-method study of GE to identify underserved students’ experience and discover factors contributing to their success; and (2) to connect essential learning outcomes to documented proficiency levels, using students’ work.

Initial program data showed that the first-year seminar was helping students meet three GE outcomes: improving their writing, thinking, and information literacy skills. What was not clear, however, was whether students were continuing that academic growth throughout their remaining years at SOU. The Compass team decided to focus part of its study on the required graduation capstones.

The team invited capstone faculty to participate in a yearlong study. The process itself turned out to be transformational and renewing for faculty. One result of the study was the creation of four campus-wide high-impact capstone rubrics for original experimental research, research-based persuasive writing, internships/community-based learning, and oral presentations.

In addition, the Compass project team completed a mixed-method study of first-generation students who persisted beyond the first year. The study required statistical analysis of enrollment data, narrative analysis of institutional policy, and mining of NSSE data. These data were triangulated with student interviews and a survey aligned to NSSE findings. The study revealed unique characteristics of first-generation, low-income students, and identified institutional practices that directly contributed to this population’s success. The SOU Compass project team is now integrating the two studies to create a model for other institutions.

Compass at EOU

The Compass project at Eastern Oregon University (EOU) focused on the establishment of a first-year experience (FYE) high-impact seminar for all students—on campus and online—and articulation of university-wide learning goals with which academic and cocurricular programs could align. This plan enabled systemic communication and data collection, and produced common templates that resulted in remapping of degree program and cocurricular goals and assessment processes.

To address poor retention of first-year students, academic affairs and student affairs partnered on a one-credit extended orientation seminar (EasTrek) for all first-year students entering with fewer than thirty credit hours—both on campus and online. Focus groups of faculty and support services staff designed a curriculum that included three essential ingredients for student success: social engagement, academic engagement, and first-year advising. The seminars are facilitated principally by student affairs educators teamed with faculty mentors. Students have an opportunity to choose from among exploratory, professional, and interest-based seminars. An FYE coordinator provides a point of contact and oversight for a common syllabus and professional development for facilitators. Student course evaluations indicate that students benefit from being informed about the learning environment that supports their success. Retention rates for the 2008 cohort increased by 18 percentage points.

EOU’s GE curriculum revision began in June 2007. The mission, core learning outcomes, and rubrics developed for the GE Core formed the nucleus for curricular transformation. Through the Compass project, the campus had a significant opportunity to develop a phased approach to curricular alignment with degree programs and the cocurriculum. The initiative supported several communication venues that resulted in articulation of University Learning Outcomes—mapped to AAC&U’s LEAP essential learning outcomes—that were assessable and to which all curricula could align. The recent purchase of planning and assessment software will further enhance the university’s capacity to track and document student success. To bring the Compass project to successful completion will require some necessary next steps on leadership, data collection and assessment, and professional development.

Compass at PSU

Portland State University has a history of over fifteen years with a GE program consisting of year-long courses with smaller, twice-weekly mentoring sessions attached. PSU used the Compass project to evaluate a partnership between the residence halls and the first-year GE courses. They now have over one hundred students sharing on-campus housing and a first-year course. From this program, they have available to share both materials and data on retention.

PSU has also used the Compass project to expand e-portfolio use in the first-year GE course and to analyze the effectiveness of this expansion. A question on e-portfolio work was added to PSU’s annual survey of freshmen in the GE program. Student responses were connected to the students’ scores on the annual portfolio assessment. Positive results persuaded faculty to increase the use of e-portfolios. PSU also started using Google sites as the e-portfolio platform, which has made e-portfolio creation easier and quicker for students. PSU has also created an e-portfolio gallery (http://tinyurl.com/33jrbgk), a step-by-step guide for students (http://tinyurl.com/394kut4), and several templates for PSU e-portfolios (http://tinyurl.com/2whp6sl).

Finally, Portland State has taken a unique approach to one of its most successful programs—the GE senior capstone, required of almost every graduating student. These award-winning senior capstones were instrumental in initiating the service-learning movement in higher education. Program assessment of senior capstones has been a challenge due to the diversity of the courses and the focus on useful products for the community partners. As a part of their Compass work, PSU decided to use a course portfolio to assess the senior capstone. Participating instructors have shared a portfolio site, a best-practice website, and a rubric for their course portfolios.

These examples illustrate the specific actions generated on the three Compass project beta campuses. What is more difficult to document is the way these inquiries fed into the work of the LO&A and provided additional support for educational improvement on the other four campuses. The evidence, however, that Compass influenced the whole Oregon University System was best illustrated at the first inter-institutional conference on teaching and learning specifically designed for university faculty. Held in Portland in October 2010, Teaching TALKS: Today’s Academic Linking Knowledge and Skills set the tone for dialogue to come. According to a member of the LO&A team:

In my work with my LO&A and AAC&U colleagues I have been challenged to analyze my teaching practice in deep and meaningful ways as we addressed the tensions between accountability and assessment…between compliance and the improvement of the teaching and learning processes. But the greatest value of this work has been in finding like-minded colleagues who share a commitment to deciphering the complexities of teaching and learning.

Conference evaluations issued a strong call for regular meetings of this nature. Specific topics for which there was high demand included the nature and evidence-based effectiveness of HIPs, and methods to institutionalize work on student learning outcomes and assessment. It is important to note that GE reform is a result of faculty action. Providing faculty with continued opportunities to collaborate with like-minded colleagues is critical to sustaining an environment of renewal, both on individual campuses and system-wide.

The OUS Final Report

The final report of the LO&A task force was delivered to the OUS Provosts’ Council in January 2011. The report makes two recommendations and suggests directions for continuing this work. The first recommendation is the adoption by all OUS campuses of a framework of expectations for the assessment of student learning, and the second is the establishment of an OUS Student Learning Assessment web page to serve as the public face for transparency, accountability, and accreditation. The Task Group report closes with suggestions, based on the experiences of LO&A in Oregon and in collaboration with Compass project systems in California and Wisconsin, for continuing this work.

Results thus far in Oregon suggest that partnerships in this area may be fostered through

  • shared goals for student learning and success for all students emanating from a sustained high-level emphasis infused throughout the individual campus communities;
  • flexibility in the means of achieving the goals, respectful of individual institutional cultures, administrative structures, student populations, priorities, and autonomy;\freedom to attempt innovative practices and fail without fear of reprisal;
  • sufficient focus to prevent the dilution of both message and action framed around too many priorities.

As we stated in our October 2010 Compass report, transformation can be messy, but it need not be. First, an institution must begin transformation with a plan that is strategically sound, coherent, sustainable, and has a five- to ten-year horizon. That is, leadership needs to develop a framework that is simple, elegant, and mutable enough to accommodate its mission within a system, its shifting strategic goals over the years, and curriculum designs that meet the evolving needs of its current and potential students.

Acknowledgments

The authors wish to thank Rowanna Carpenter, Shawn Smallman, and Sarah Witte for contributing to this essay and Bob Turner, Elisabeth Zinser, and Ruth Keele for leadership and support.


Kay M. Sagmiller is the faculty director of academic assessment at Southern Oregon University; Kenneth M. Doxsee is the associate vice provost for academic affairs, Office of Academic Affairs, and professor, department of chemistry at the University of Oregon.

Previous Issues