Peer Review

Transparency and Problem Solving: The UHD Experience

The University of Houston-Downtown (UHD) is a comprehensive four-year university located in the heart of the city of Houston, Texas. The university reflects the diversity of the Greater Houston Area as a Hispanic-Serving Institution, and through its academic programs engages with the community to address the needs and advance the development of the region. UHD is an inclusive community dedicated to integrating teaching, service, and scholarly research to develop students’ talents and prepare them for success in a dynamic global society. Our vision is to engage every student in high-impact educational experiences, ensuring that students graduate with twenty-first-century skills.

Through a competitive process, UHD was selected as one of seven institutions in the nation to participate in AAC&U’s Advancing Underserved Student Success through Faculty Intentionality in Problem-Centered Learning project. One of the primary reasons UHD was excited to join the AAC&U research project was because the research aligned with the university’s newly developed Quality Enhancement Plan (QEP), which aims to promote integrative learning while engaging underserved student in high-impact educational practices. Our institution successfully completed a general education (GE) reform in fall 2014, implementing the Texas Common Core. However, UHD’s GE program extends beyond the common core into the disciplines. With that in mind, our team is eager to share the results of our research work in implementing transparency and problem-centered learning into our courses over the spring 2015 semester. Our goal for this article is to educate faculty and administration on the purpose and structure of transparency practices and how they can enhance overall course experiences.

Diverse Transparency Practices

The faculty members engaged in this project are representative of the diversity of our university and the students who participated in our research. The team members served in different faculty ranks and represented three colleges on our campus: the College of Humanities and Social Sciences, the College of Public Service, and the College of Sciences and Technology. The diversity of the team allowed each faculty member to bring a unique set of skills and life experiences in order to create and embed problem-centered learning experiences and assignments into their courses. Additionally, each instructor utilized a variety of transparency techniques, with a focus on transparent assignment design. The four courses selected for this project ranged in format from face-to-face to fully online, and they were already utilizing a variety of high-impact practices before being selected for participation in this project. Table 1 illustrates the variability between all four courses and gives an at-a-glance view of the methods used for implementing transparency.

After initial project orientation, our team followed the AAC&U Problem Solving VALUE Rubric to help guide the design of assignments, utilized lessons learned about transparency to modify content delivery, and collected overall course data including results of initial and final transparency surveys from intervention and control courses. The results of these surveys have been published and highlight how implementation of transparency can have a positive effect on students’ perception of gains in skills valued by employers. We have chosen to highlight some of the analysis of our course data and share lessons learned from reflections, team discussions, and discussions with project leaders and teams from other institutions.

Table 1. UHD Control (C) and Intervention (I) Courses

Course

Face to Face

Online

Class Size

Level

High-Impact Practice

Transparency Techniques

General Genetics

C, I

 

C-42
I-41

Junior

Problem solving in real life; team-based learning

C 2,3,6
I 1,2,3*,6*

General Biology

C, I

 

C-45
I-45

Freshman

Team-based learning

C 2,6
I 2*,3,6*

Introduction to Special Populations

 

C, I

C-39
I-40

Junior

Problem solving in real life; collaborative learning

C 1,2,3,4,6
I 1*,2*,3*,4*,5,6*

Small Group

 I

C

C-30

Junior

Collaborative learning and service learning

C 1,5,6

Each instructor taught a control and intervention course in spring 2015. Transparency techniques 1–6 are as follows: 1. Create flexible formats that appeal equitably to various learners’ strengths. 2. Build students’ critical-thinking skills in a logical sequence (problem solving). 3. Set criteria for student success (provide to students in advance.). 4. Critique student work (provide examples to students in advance). 5. Perform self, peer, and group evaluations (provide to students in advance). 6. Explicate assignments’ purpose, task, and criteria in advance.
*For the same techniques applied to both sessions, application was more intentional in the intervention sessions

Results

As an integral part of the project, our team utilized the rubric in order to design or modify an existing assignment or sets of assignments for our courses. The goal was to be intentional in the implementation of transparency, both in assignment design and course delivery, in the intervention course. Each instructor selected an assignment to represent an initial sample and then an assignment to represent a final sample. Ten student samples from each intervention and control section were collected. Evaluation of assignments utilizing the four-point scale of the six-dimension rubric was performed blindly by another member of the team. Before scoring student samples, the team participated in two different calibrating exercises. The scoring process highlighted the importance of assignment alignment to the rubric and proved challenging. Assessing all dimensions of the rubric in a single assignment, especially when the course was designed to build problem-solving skills progressively, and conducting assessment in introductory level courses emerged as common challenges. Below is a short description of how each course utilized the rubric for assignment design and a brief overview of results, followed by faculty reflection.

Small Group Communication Course

Implementation and Results. Implementation and design of both transparency and problem-centered learning presented a challenge for this course since the control course was delivered fully online whereas the intervention course was delivered face-to-face. Additionally, the intervention course also had a service-learning component. While the assignment itself was identical for both courses, students in the control course had freedom to create their own company for analysis whereas students enrolled in the intervention course had to select an existing company. These and other variations may account for the high variability seen in the initial assignment and precluded any meaningful data analysis.

Lessons Learned. Initial participation in this project was under the assumption that the area of communication studies and this course were already very transparent. Based on participation in this project, future classes taught in both the face-to-face and the online formats in the communication studies area will move their focus toward the need to explicate all assignments, course objectives, and project design based on the types of learners present in our classes.

General Genetics Course

Implementation and Results. For this course the problem-solving rubric was used to generate an assignment titled “Why Is There a Divided Opinion on GMO Food,” which was content-related but not specific to the course curriculum. Students had the opportunity to focus on an open-ended problem that is relevant and highly publicized. The assignments were built progressively and the initial sample aimed to only assess the first three initial rubric dimensions. The final assignment aimed to assess all or most of all of the rubric dimensions. The results of this course are summarized in figure 1. Both the experimental section and the control course section had a similar baseline for the initial assignment. The experimental section showed a marked increase as compared to the control when the average scores across dimensions were compared. Additionally, 100 percent of students in the experimental section improved their score in at least one dimension as compared to 89 percent of control section students.

Lessons Learned. The encouraging results can be attributed to the careful planning of the assignment based on the rubric and the experimental design. The relevance of the assignment to students’ daily lives aroused their interest and enhanced learning. Their final assignment was to generate a survey on GMOs and administer it to their family and friends. There was strong evidence of learning according to the analysis of the survey results and student reflection. The significant improvement in this final assignment in the intervention course further supports the importance of explicating each assignment’s purpose and establishing criteria for success in advance.

 

General Biology Course

Implementation and Results. This course utilized a set of six assignments to make connections across biology while focusing on insulin and diabetes. The assignments were created to progressively strengthen students’ critical-thinking and problem-solving skills. The second of these assignments was chosen as the initial sample and the last assignment was chosen as the final sample. For both courses there was a small improvement when the average scores across rubric dimensions were compared. Additionally, 70 percent of students in the experimental section improved their score in at least one dimension as compared to 50 percent of control section students.

Lessons Learned. Since this was a freshman-level course, the instructor struggled with designing a content-specific assignment that would meet all rubric criteria without exceeding the expectations of the freshman-level students. Therefore, the assignments addressed only the first four dimensions of the rubric. A more open-ended and less content-driven assignment will likely allow for a better assessment of rubric dimensions. There was a marked change in how students in the intervention course interacted with the instructor and reflected on their performance in these assignment sets. There was a shift from the traditional “Why did I lose points?” attitude to a “What would make it better?” attitude. This change was apparent in course evaluation comments, where students in the intervention courses focused more on what and how they learned from the course than on likes or dislikes about the course or instructor, as was seen previously and in the control course.

PR_WISP16_-Kang_Fig1_Web.jpg

(Click on figure to enlarge)

 

Introduction to Special Populations Course

Implementation and Results. For problem solving, there were six assignments that were content specific and required students to differentiate instruction for six different disability groups. Each assignment asked students to develop one accommodation in five different domains (content, instruction, setting, behavior, and affect) to foster a positive learning environment for students with that disability. Each activity met the criteria of open-ended and real-world applications. The first assignment, learning disabilities, was used for the initial sample and the fifth assignment, Autism, was used for the final sample. Comparing the rubric scores, 90 percent of students in the experimental section improved their score in at least one dimension as compared to 55 percent of control section students.

Lessons Learned. The findings indicate that students could be more engaged in the problem-solving process based on all dimensions of the rubric. This has led to adopting a more backward design that will provide a more mindful approach to the activities specified in the problem-solving process.

Both the General Biology and Introduction to Special Populations courses exhibited an increase in transparency as measured by the transparency survey. To gain a better understanding of how students’ perceived transparency related to their self-assessment and course assessment in final course evaluations, we mined data from the IDEA Students Ratings of Instruction (IDEA SRI), the current course evaluation system in place at UHD, and analyzed components related to three separate areas: transparency, motivation and metacognition, and perceived progress by the student. Table 2 summarizes these results. Overall, students from intervention sections reported feeling that they received more meaningful feedback, that instructors had a personal interest in their learning and were more available outside of class, and that instructors encouraged students to reflect on their own learning and progress. Most notably, and relevant to the problem-centered aspect of the project, students noticed a marked increase in “learning to analyze and critically evaluate ideas, arguments and points of view.” These data suggest that our existing course evaluation system (IDEA SRI) can be used similarly to the transparency survey utilized in this project to shed light on students’ perception of transparency. High transparency as perceived by students appears to correlate with their self-assessment of critical-thinking and problem-solving skills.

Table 2. Improvement in Course Evaluation Areas for Courses in which Transparency was Implemented

Area % Improvement in
Intervention Course
General Biology Intro. to Spec. Pop.
Transparency
Instructor provided meaningful feedback on student’s academic performance 6 6
Instructor encouraged student-faculty interaction outside of class 17 11
Instructor explained the reasons for
criticisms of student’s academic performance
5 1

Instructor displayed a personal interest in students and their learning

5 7
Motivation and Metacognition
Instructor encouraged students to reflect on what they have learned 13 4
Instructors stimulated students to intellectual effort beyond that required by most courses 7 4

Instructor found ways to help students answer their own questions

5 12
Instructor demonstrated the importance and significance of the subject matter 5 5
As a result of this course, I have more positive feelings for this field of study 8 10
Perceived Progress by Student
Learning to analyze and critically evaluate ideas, arguments, and point of view 23 11
Learning to apply course material (to improve thinking, problem solving, and decisions) 9 6
Acquiring skills in working with others as a member of a team 7 35
IDEA SRI course evaluation data was analyzed for components related to three separate areas: transparency, motivation and metacognition, and perceived progress by student. These components were then compared between intervention and control courses. There was a marked or slight improvement in every component analyzed for both courses. General Genetics and Small Group Communication did not have enough students to take either the transparency survey or the IDEA evaluation and thus were precluded from this data analysis.

 

Summary and Moving Forward

This experiential project provided hands-on learning experience for faculty and students. Through participation in this project, faculty became more intentional at every step of designing, implementing, and assessing a problem-based assignment. Our students demonstrated higher learning or reported better engagement based on the data collected from the transparency report, IDEA evaluation, and problem-solving rubrics. As a team, we identified four points that are key for future studies: (1) backward design of assignment to ensure proper alignment to rubric is essential; (2) remaining unbiased in application of transparency between control and intervention sessions proved problematic; (3) course characteristics such as delivery mode, student classification, and size should be taken into consideration for data interpretation; and (4) detailed faculty training and continued support is vital to successful implementation.

Moving forward, our group will incorporate more transparency techniques in our teaching in order to improve student learning and retention. We also have the unique vantage point to share our knowledge with other UHD faculty members. Our team has devised a dissemination plan to contribute to the general education reform by providing faculty training on transparency and intentionality. To do this, we will provide training modules and workshops through collaboration with the UHD Center for Teaching and Learning Excellence. We have also identified stakeholders throughout the university who are committed to ensuring university-, college-, and department-level dissemination of transparency implementation techniques. Through their partnership we will recruit new faculty members to apply transparency to their courses and engage in an active dialogue on best practices. We hope that by sharing our lessons learned from participation in this AAC&U project and providing high-quality training, we can help build a culture of transparency and intentionality among UHD faculty.

The authors would like to acknowledge Dr. Rachna Sadana for her participation in this project and her design of General Biology assignments as well as UHD’s Center for Teaching and Learning Excellence (CTLE) and University College for their support of this project. 

 

References

Association of American Colleges and Universities. 2015. “Transparency and Problem-Centered Learning.” www.aacu.org/problemcenteredlearning.

Association of American Colleges and Universities. “VALUE Rubric Development Project.” www.aacu.org/value/rubrics.

Winkelmes, Mary-Ann, David E. Copeland, Ed Jorgensen, Alison Sloat, Anna Smedley, Peter Pizor, Katharine Johnson, and Sharon Jalene. 2015. “Benefits (Some Unexpected) of Transparent Assignment Design.” National Teaching and Learning Forum 24 (4): 4–6.


YuanYuan Kang, assistant professor, biology; John Kelly, assistant professor, special education; Creshema Murray, assistant professor, corporate communication; Adriana Visbal, lecturer, biology—University of Houston-Downtown

Previous Issues