Peer Review

The Freshman Academies Assessment Protocol: A Reflective Tool for Student Success

Queensborough Community College, located in Bayside, Queens, is one of six community colleges in the twenty-three-institution City University of New York system. Queensborough has pioneered a freshman academies initiative that provides enhanced academic and student support for all first-time, full-time students. This flagship initiative led to Queensborough being named one of the twelve leadership institutions in AAC&U’s Developing a Community College Roadmap project. The Freshman Academies initiative was designed with an assessment protocol, which, since the program’s launch in fall 2009, has in turn provided a roadmap for the college community to better understand how the elements of the initiative are working. Through this assessment, Queensborough reports improved retention and graduation rates, and it can also identify the most impactful interventions. The protocol has also become a tool of reflection for the college.

All first-time, full-time students enter Queensborough into one of six academies based on field of major interest. They receive enhanced orientation and advisement from Freshman Coordinators (“relationship guides” who provide support throughout the first two semesters), and they take an Introduction to College Life class. Also, within the first thirty credits, all first-time, full-time students are enrolled into at least two courses identified as providing high-impact practices (HIPs). These experiences include service learning, e-portfolio use, writing-intensive components, learning communities, and first-year experiences. Queensborough aims to infuse the common threads of reflection, interdisciplinary collaboration, and active learning throughout these practices, and it also provides social learning in its extensive and innovative use of cohorts through face-to-face and online collaborations. E-portfolios serve as vehicles to deliver these practices at entrance to the college and throughout a student’s academic career. Faculty coordinators assess HIPs in multi-disciplinary faculty cohorts using rubrics which align with the college’s ten general education objectives. Faculty Coordinators also collaborate with the Freshman Coordinators to design cocurricular experiences which align with course and program objectives.

To connect the strands of the freshman academies, Queensborough hired an outside consultant, Derek Price of DVP Praxis LTD, to create the Freshman Academies Assessment Protocol. The protocol serves multiple functions. It sets expectations of how various aspects of the Freshman Academies initiative—primarily the Freshman Coordinators and the use of HIPs—should inform change. It specifies the kinds of beneficial changes that should occur, such as increased faculty and student engagement, increased course success rates, and increased retention and graduation rates. The protocol also defines the ways that research on the academies is to be carried out by specifying data collection methods, such as surveys and focus groups. An important characteristic of the protocol is that it states outcome expectations in very general terms, giving the college the flexibility to specify the details. For example, the Protocol expects that Freshman Academy students will perform better in gateway courses, and the College determines which courses are considered gateway. The employment of an outside consultant provided important advantages, providing an external perspective, freedom within the design process, dedicated time to the task, and expertise in the area of research design. Queensborough chose the fall 2006 cohort as a baseline group with which to compare the cohort of first-time, full-time students who entered the academies in fall 2009, the first year of the initiative. Measures of success, such as course pass rates and retention, have been compared between the baseline fall 2006 cohort and the fall 2009 academy cohort.

The college applied for and received Institutional Research Board approval for the protocol-specified methods of data collection, which was beneficial in multiple ways. It helped to focus those involved in the research process to carefully consider ways to ensure that human subjects would not be harmed. More broadly, it permitted the college to designate the findings within the definition of “research,” and thus it allowed for publication of its results. It also brought into the process a skilled, external group to critique aspects of the research, enabling further refinement of the research plan.

The Freshman Academies Assessment Protocol has been used since fall 2009 to guide the collection and analysis of research data. In addition to providing evidence that the Freshman Academies initiative is associated with benefits to our students, the results have pinpointed which HIPs have been most effective. In certain courses it has revealed where particular practices work best. This feedback from the research findings has led to a continuous improvement process. Queensborough has therefore been able to focus its resources and expand those practices which encourage higher pass rates in gateway courses.

When looking at the effectiveness of HIPs, all students enrolled for a particular course are observed. A comparison is made of pass rates and retention rates between students who are in course sections with and without HIPs. There is a separate analysis for remedial and nonremedial students. According to the Freshman Academies Assessment Protocol, it is expected that the effects of the Freshman Coordinators and HIPs, all in concert, would result in higher retention rates across the academy cohorts.

Retention rates for the fall 2006 baseline group of first-time, full-time freshmen, enrolled before the academies, were used to make comparisons to the cohorts of freshmen who were enrolled within the academies. Half-year and one-year retention rates were found to be higher in the initial cohort of freshmen in the academies than the baseline comparison group. The baseline group of freshmen from fall 2006 had a 65.8 percent one-year retention rate while the fall 2009 Academy cohort had a one-year retention rate of 71.5 percent. The second academy cohort likewise showed high retention rates, with a 72.1 percent one-year rate. Two very important outcomes arose from the findings of the protocol. The most important expected long-term outcome for Queensborough students was increased retention and graduation rates. After three years, Queensborough has data on its three-year graduation rate, which suggests that this initiative has made a difference (see fig. 1.).

Figure 1. Data on Queensborough’s Three-Year Graduation Rate

Cohort
Total Number of Students
Number Graduating
Three-Year
Grad. Rate
Fall 2006 Comparison Cohort
2,051
262
12.8%
Fall 2009 Academy Cohort
3,226
523
16.2%

The other important finding is that the protocol itself needs to be a constantly evolving and revisited document. Undergoing the very process of continuous assessment through findings of the protocol has led the college to form the academies review taskforce. The Freshman Academies Protocol, the data it produces, and the processes it enacts inform our understanding of the Queensborough student experience, leading to improved, evidence-based decision making.


Arthur Corradetti is the associate dean for accreditation, assessment, and institutional effectiveness; Michele Cuomo is the associate dean for academic affairs; Victor Fichera is an institutional research manager; Susan Madera is an academic program specialist—all of Queensborough Community College

Previous Issues