Peer Review

The STIRS Framework and Integrative Liberal Education

AAC&U’s Scientific Thinking and Integrative Reasoning Skills (STIRS) framework is designed to guide the development of curriculum in evidence-based thinking. This framework has been developed, used, and revised as part of the LEAP Challenge, leading to capstone and signature work. The STIRS project was developed as an exemplar of integrative liberal education.

What Is the STIRS Framework?

The STIRS framework consists of four components designed to fit together as part of an integrative bachelor’s degree. The four components are

  1. Evidence—What It Is and How It Is Used: Defining and using evidence across the disciplines
  2. Research Methods: Obtaining and ensuring the quality of evidence
  3. Evidence-Based Problem Solving: Using evidence to define and solve problems
  4. Evidence-Based Decision Making: Using evidence to define options and make decisions.

The goal of the LEAP Challenge and STIRS is to graduate engaged and productive citizens who are prepared to address the critical challenges of the twenty-first century. Therefore, graduates in all fields of study need to be able to

  • ƒ Describe the range of definitions and uses of evidence in the sciences, social sciences, health, humanities, and fine arts including identifying common ground and interdisciplinary approaches.
  • ƒ Describe qualitative and quantitative study designs and inferential reasoning principles, and other relevant frameworks, to obtain and evaluate evidence in a variety of disciplines.
  • ƒ Discuss how evidence can be used to advance knowledge and/or to inform subsequent research.
  • ƒ Apply an evidence-based problem-solving approach that moves from problem identification to identification of causal factors; evidence-based recommended solutions; and implementation and evaluation of outcomes, including the role of reflection.
  • ƒ Apply an evidence-based decision-making approach, identifying elements that frame and drive decision making for problems in the sciences, social sciences, health, humanities, and the fine arts, including preprofessional education.
  • ƒ Analyze the operation of complex systems using evidence and analysis of systems.
  • ƒ Analyze ethics issues which are inherent in research and the use of evidence.
  • ƒ Synthesize evidence to formulate responses to complex problems and/or make recommendations for new approaches to disciplinary and interdisciplinary issues.

The STIRS project has undergone several phases since its 2012 inception. The first phase developed the initial version of the STIRS framework. The second “case studies” phase, begun in 2013–14, identified thirteen STIRS Scholars and produced sixteen peer-reviewed case studies that were linked to the initial STIRS framework (Stanford 2016). The “institutional phase,” begun in 2015, identified four institutions that are developing, implementing, and evaluating integrative evidence-based learning models based on the STIRS framework as exemplars of integrative liberal education (Anthony 2016).

In 2015–16, the STIRS framework was revised to more formally include the humanities and fine arts and to provide a structure that more readily suggests ways to incorporate STIRS into integrative liberal education. This article reports on the STIRS framework that has emerged from this process.

How Do the STIRS Components Connect with Integrative Liberal Education?

The STIRS framework is designed to outline the content and process areas that should be covered in an evidence-based curriculum using an integrative liberal education framework applicable to disciplines ranging across the sciences, social sciences, health, humanities, and fine arts. It should be applicable to preprofessional programs as well as degrees preparing for entry into the workforce.

The STIRS framework provides a guide for teaching evidence-based thinking throughout the bachelor’s degree. As such, the framework must be hardwired to an integrative liberal education approach to bachelor’s degree education that interdigitates the general education or essential studies with majors or specialized knowledge.

Figure 1 (see p. 8) represents an AAC&U-endorsed model of integrative liberal education which intentionally links general education and essential studies with specialized knowledge or majors. To accomplish this goal, the model focuses on three levels of interaction labeled cornerstone, connector, and capstone.

PR_FA16_Riegelman_Fig1_Web.jpg

To connect the STIRS framework to integrative liberal education, it was essential that the framework be structured so that the relationship of the components to the cornerstone, connector, and capstone elements of the liberal education model were obvious and provided ample opportunity for creative ways to implement the framework.

What Are the Components of the STIRS Framework and How Do They Fit Together?

The four components of the STIRS framework fit together in ways that allow them to be hardwired to the integrative liberal education model. The components and their links to the cornerstone, connector, and capstone approach are illustrated in figure 2.

Thus the STIRS framework suggests the need for a scaffolded approach aiming to achieve a coherent whole by using multiple components strategically distributed throughout the bachelor’s degree.

PR_FA16_Riegelman_Fig2_Web.jpg

What Are the STIRS Framework Outcomes?

From the beginning, the STIRS framework aimed to make critical and analytic thinking more precise, more expansive, and more applied. The STIRS framework aims to facilitate communication, curriculum planning, and eventually curriculum evaluation in the area of evidence-based thinking.

In order for the framework to serve all of these purposes, the terms used in the framework need to be defined. This is especially important since the scope of application of the STIRS framework stretches from the sciences to the fine arts, fields that rarely have talked to each other using the same language or even the same meaning for the same words. Therefore, the STIRS framework includes over eighty definitions of terms designed to promote a common language for communicating key concepts.

The STIRS framework is not a set of specific learning outcomes. Rather, it aims to define the “enduring understandings” that all students should retain from their bachelor’s degree education. The four components of the STIRS framework each include multiple subcomponents. Box 1 provides an outline of the four STIRS framework components and their subcomponents. The full STIRS framework, including definitions of terms, is available on the AAC&U website (www.aacu.org/STIRS).

To better understand what the STIRS components are seeking to achieve, this paper examines in more detail a few of the subcomponents of the four STIRS framework components.

Box 1. STIRS Framework Components and Subcomponents

Evidence—What It Is and How It Is Used

  1. Reductionist approaches to the use of evidence
  2. Integrative approaches to the use of evidence
  3. Theories and paradigm shifts
  4. Uses and display of evidence
  5. Roles of statistical reasoning
  6. Roles of analytical, intuitive, and logical reasoning  

Research Methods

  1. Qualitative and quantitative evidence
  2. Data collection and analysis
  3. Assignment to groups and assessment of outcomes
  4. Interpretation criteria for cause and effect
  5. Extrapolation
  6. Ethical principles for research

Evidence-Based Problem Solving

  1. Approaches to evidence-based problem solving
  2. Problem framing and description
  3. Etiology/efficacy and evidence-based recommendations
  4. Implementation and evaluation
  5. Methods of evidence-based problem solving, such as data synthesis and translational research

Evidence-Based Decision Making

  1. Heuristics and decision rules
  2. Comparing benefits and harms
  3. Principles of testing
  4. Prediction and prediction rules
  5. Methods of evidence-based problem solving, such as data synthesis and translational research
  6. Ethical principles for decision making

Component 1: Evidence—What It Is and How It Is Used
Subcomponent 2: Integrative approaches to the use of evidence
Integrative approaches often build upon reductionist approaches. Integrative approaches draw from multiple disciplines incorporating multiple influences or determinants of outcomes; look for interactions between factors; and use evidence-based approaches to understand and propose strategies for addressing complex problems. Integrative approaches aim to model the multiple influences or determinants of a single outcome rather than test a hypothesis. In doing this they view outcomes as the results of complex interacting systems. Reflecting on complex systems and questions before problems are posed and solutions considered is a key skill for integrative approaches.

This description of integrative approaches to evidence-based thinking highlights the complementary roles of integrative and reductionist thinking. It outlines roles that integrative thinking can play, which can be applied across a range of disciplines as diverse as meteorology, geography, anthropology, sociology, linguistics, history, and public health. This subcomponent also emphasizes the role of reflection, which is fundamental to the humanities and fine arts.

Component 2: Research Methods
Subcomponent 6: Ethical principles for research
Research needs to be conducted based on ethical research principles including the principles of respect for persons, beneficence, and justice. These require prior review of research proposals by an objective external body, high-quality research designs, informed consent for human interventional research, and an expanding set of safeguards to ensure ethical implementation. Ethical standards for animal and laboratory research also need to be established and maintained.

This subcomponent emphasizes the connection between ethics and other humanities with the types of research methods that are increasingly applied in the sciences, social sciences, and health. It puts forth the now traditional principles of ethical research and emphasizes that high quality study design is itself an ethical imperative. Ethical standards in the conduct of research as well as its design are emphasized.

Component 3: Evidence-Based Problem Solving
Subcomponent 2: Problem framing and description
Problem framing requires the use of evidence. Discipline-specific evidence is often needed, including unique methods applicable to the humanities and fine arts. Specific details, such as the timeline, burden, and financial costs, can be central to describing the problem. Describing the problem may provide a framework for integrating available qualitative and quantitative evidence to define what is known. It may also assist in developing a strategy for producing additional needed evidence. Evidence that describes the problem may be used to generate hypotheses.

This subcomponent outlines the use of a widely applied step-by-step process of evidence-based problem solving. It then focuses on the essential issue of framing the problem, which often sets the agenda for how problem solving is approached. The important role that the humanities and fine arts play in framing problems is central to this subcomponent.

Component 4: Evidence-Based Decision Making
Subcomponent 1: Heuristics and decision rules
Heuristics or “rules of thumb” often govern human decision making due to humans’ limited ability to process large quantities of data. Heuristics are an essential part of everyday decision making but are prone to a range of analytical and logical limitations. Decision rules, such as maximizing expected utility and satisficing, provide an objective basis for combining harms and benefits and selecting between options as part of evidence-based decision making. Unique methods are used in the humanities and fine arts to describe and challenge conventional approaches to decision making, including understanding a text or historical event differently, which adds to the understanding.

Once again, this subcomponent seeks to link the contributions of the humanities and fine arts with the traditional analytical approaches to decision making. This subcomponent assumes an emerging understanding of human decision making including its strengths and weaknesses. This approach is taking on increasing importance for twenty-first-century students confronting technological decision aids and massive amounts of data.

What Does the STIRS Framework Imply About Implementation?

The STIRS framework is meant to link to the integrative liberal education model, while leaving ample room for creative implementation approaches. A variety of approaches can be used to integrate and link the components throughout the course of a bachelor’s degree.

A common initial curriculum is often desirable. It should define and illustrate the use of evidence in a wide range of disciplines selected from natural sciences, social sciences, health, humanities, and fine arts. It should introduce students to the thinking processes in multiple disciplines and help them appreciate the fit between the disciplines and their own interests and talents.

Integration of evidence-based thinking into a variety of majors and/or concentrations can facilitate accomplishing the remaining components. A synthesis or capstone activity, ideally including signature work, can ensure a coherent approach to integrating the four components of the STIRS framework.

How Was the STIRS Framework Developed and Revised?

The initial STIRS framework was drafted by AAC&U’s Office of Integrative Liberal Learning and the Global Commons. The effort was led by former AAC&U senior director of global learning and curricular change, Kevin Hovland, with wide input, including that of the author of this article. The aim was to put forth a first approximation of the knowledge and skills that are central to applying evidence-based thinking to a wide spectrum of students in a broad range of disciplines.

From the beginning, definitions of terms were included in the framework in order to encourage communication using a common well-defined lexicon of evidence-based thinking terms. The initial draft framework was reviewed by the AAC&U STEM community through Project Kaleidoscope and the AAC&U general education community through the General Education and Assessment network.

The initial framework, developed in 2012–13, remained on AAC&U’s STIRS website for three years and guided the development of both the STIRS case studies and institutional phases. The use of the initial STIRS framework made it clear that it has two major limitations: (1) it did not explicitly integrate the use of evidence into the humanities or the fine arts and (2) it was not organized to hardwire to the cornerstone, connector, and capstone components of integrative liberal education.

Therefore, an intensive effort was made in 2015–16 to restructure and revise the STIRS framework to ensure that it was in sync with the integrative liberal education model increasingly being used by AAC&U. The revised STIRS framework also sought to explicitly and implicitly incorporate the humanities and the fine arts.

STIRS Scholars, STIRS Fellows, and AAC&U staff all provided feedback on the process. Additional feedback was solicited at the 2016 AAC&U annual meeting. Once drafted, the revised STIRS framework was announced in AAC&U News with an invitation to the entire AAC&U community to provide input. The revised STIRS framework, which is featured on the AAC&U website (www.aacu.org/stirs), incorporates a large number of recommendations suggested by this diverse community.

How Can the STIRS Framework Be Used?

The STIRS framework has already been used to help organize the STIRS case studies. Each of these case studies was expected to illustrate one or more of the original STIRS framework components. The STIRS framework revision took place concurrently with the institutional phase of STIRS and benefited from the development of the cornerstone, connector, and capstone model of integrative liberal education upon which the institutional phase is built.

The STIRS Fellows also linked the STIRS framework with the AAC&U VALUE rubrics. Initially the STIRS project leadership refrained from developing an “evidence-based thinking” VALUE rubric, recognizing that the key components of evidence-based thinking were already dispersed through many of the VALUE rubrics. After some thought, however, the AAC&U VALUE rubric project team encouraged the development of an Evidence-Based Thinking VALUE rubric, which was created by adapting appropriate benchmarks from the other AAC&U VALUE rubrics.

Each STIRS Fellow is now developing and utilizing his or her own evidence-based thinking VALUE rubric. The next step is to compare them and then integrate these rubrics into each institution’s evaluation efforts. It is anticipated that the results of this process will be reported in the near future.

A key application of the STIRS framework is to utilize the components of the STIRS framework to scaffold the curriculum. For instance:

  • ƒ Evidence—What It Is and How it Is Used (Component 1). This component can serve as the basis for a cornerstone curriculum. This curriculum might introduce students to basic principles of evidence and might be followed by discussions with faculty from a range of disciplines on their approaches to using evidence. This could allow faculty from disciplines as diverse as physics, economics, psychology, philosophy, architecture, and design to discuss what evidence means and how it is used in their discipline. A natural outgrowth of this approach is for the faculty to encourage students to try out their disciplines and for students to better understand whether their talents and interests are a good fit with a discipline.
  • ƒ Research Methods (Component 2). Integrative liberal education requires explicit efforts to connect general education and the major. Research methods provide an excellent opportunity for this to occur. General education curricula are capable of engaging students in exploring basic principles of research design and analysis. The use of research methods in the major can then build on this foundation and apply discipline-specific approaches and tools for research.
  • ƒ Evidence-Based Problem Solving and Evidence-Based Decision Making (Components 3 and 4). These components of the STIRS framework are the keys to applying the STIRS frameworks. They are also key to developing the types of problem-solving and decision-making skills that AAC&U has consistently found are sought by employers. These components can be illustrated in the major, but full implementation requires a capstone synthesizing general education and the major, which is ideally framed as a “signature work.”

As suggested in these examples, it is possible to hardwire the four components of the STIRS framework to the cornerstone, connector, and capstone components of the integrative liberal education model. One option for implementation is to develop the STIRS framework as a “Guided Learning Pathway” connecting evidence-based curriculum taken in two-year colleges with curriculum leading to signature work taken as part of a bachelor’s degree.

The STIRS framework as developed and widely reviewed might be said to have “face validity.” Face validity is a solid starting point but not the end of the process. Formal implementation and evaluation is needed before it can be said with confidence that the STIRS framework can be used as the structure for successfully teaching evidence-based thinking.

Therefore, another phase of STIRS is needed that could be called the “STIRS: Show Us the Evidence” phase. This potential next phase of STIRS will require larger scale planning; implementation; evaluation; and, if successful, dissemination. If successful, it will allow the next generation of college students to master the art and science of evidence-based thinking regardless of their institution or their major. 

 

References

Anthony, Seth, Wesley Barker, Tami Carmichael, Catherine Pride, and Ryan Zerr. 2016. “Creating Intentional Pathways: The STIRS Institutional Phase.” Peer Review 18 (4): 19–22.

Association of American Colleges and Universities. Guided Learning Pathways. Washington, DC: Association of American Colleges and Universities. Available at http://www.aacu.org/node/16204 .

Stanford, Jennifer S., Loren Byrne, and Katherine Hunting. 2016. “Promoting Evidence-Based Thinking through the STIRS Case Studies.” Peer Review 18 (4): 14−18.


Richard K. Riegelman, professor and founding dean, The Milken Institute School of Public Health, The George Washington University

 

Previous Issues