Register for What Matters Most: EL Education 2020 Virtual Summit
Header image

Measuring Joy: A Social Justice Issue

  • Date

  • Author

    Madora Soutter

My daughter attends a preK-8 school where students splash in puddles, take time to look at clouds, and come home with paint in their hair. Her teachers nurture friendships, hold the children’s hands when they are sad, encourage risk taking, and give even the youngest students opportunities to discuss and to celebrate acts of kindness and peace. The entire school seems to be infused with exploration, engagement, curiosity, and joy. Read the full story here, or below:

But the happiness that emanates from this lovely place fills me not only with deep gratitude for my daughter’s teachers, but also with a sense of guilt that as an education professor and researcher, I try to measure and calculate what my daughter’s teachers so beautifully and holistically bring to everything they do. Much of the joy that I observe in my daughter and her classmates seems to arise from teachers’ explicit efforts to cultivate students’ social and emotional learning (SEL) alongside their academic growth, and as a scholar, my job is to ask questions and to understand how educators can foster this kind of atmosphere and to report on SEL outcomes.

However, when I analyze survey responses and apply statistical methods to track student growth in qualities like empathy and open-mindedness, I sometimes feel as though I am wrenching the soul out of what SEL is meant to be. Is measuring something as crucial — and as beautiful — as empathy really a good idea? Or is it so reductionist that it threatens the authenticity and joy of being in a classroom where these qualities are nourished?

Why measure feelings and dispositions?

In an era of test-based accountability, the mere decision to measure things like engagement, curiosity, and joy sends a strong signal to teachers and parents that these things matter — it may even help convince skeptics that it is worthwhile to invest in SEL programs and instruction (see, for example, Immordino-Yang, Darling-Hammond, & Krone, 2018; Mahoney, Durlak, & Weissberg, 2018).

Ideally, efforts to measure students’ social and emotional growth, motivation, happiness, and other traditionally nonacademic outcomes will also demonstrate that specific approaches to SEL instruction are, in fact, effective. At present, educators are confronted with an overwhelming number of SEL models from which to choose (Berg et al., 2017), with little guidance as to which ones are most promising or whether all students will respond to them in the same way. Further, even if educators choose the most promising models, it remains to be seen whether they will implement them effectively or equitably (Jagers, Rivas-Drake & Borowski, 2018; Jagers, Rivas-Drake, & Williams, 2019; Jones, McGarrah, & Kahn, 2019; Simmons, 2017).

Indeed, measurement is a social justice issue. Tracking, measuring, and studying SEL and other nonacademic instruction and outcomes is an important way to determine whether all children are afforded opportunities to develop a strong sense of agency, for example, or to become more empathetic, attuned to the well-being of the broader community, and empowered to participate in the civic arena (Levinson, 2012). And if we hold educators accountable to such student outcomes, then perhaps they will be less likely to rely on the kind of authoritarian, back-to-basics teaching model that Martin Haberman (1991) famously referred to as the “pedagogy of poverty.”

Thanks to research by Haberman and many other scholars, we know that disproportionately large numbers of students of color are given disciplinary referrals by teachers for minor behavioral issues (Fergus, 2019; Gregory, Skiba, & Noguera, 2014; Wallace et al., 2008); feel that their teachers don’t support them or care about their success (Love, 2019; Noguera, 2002); and feel no real sense of belonging at school (Brown & Tam, 2019; McMahon & Wernsman, 2009; Moreno, 2008).

Researchers have also found, though, that schools can work to overcome these troubling dynamics if teachers make it a priority to build warmer relationships with students of color (Gregory, Skiba, & Noguera, 2014; Weinstein, Curran, & Tomlinson-Clarke, 2003); create a more caring environment, in which all students feel safe and supported (Gregory, Skiba, & Noguera, 2014); and make it the norm to respect and celebrate all students’ cultures and identities (Jagers, Rivas-Drake, & Borowski, 2018; Metropolitan Center for Urban Education, 2008; Paris & Alim, 2017; Weinstein, Curran, & Tomlinson-Clarke, 2003).

It is no simple matter to measure such outcomes. Feelings and dispositions are inherently nuanced, complex, and elusive. For example, it is notoriously difficult to distinguish between a student who genuinely feels safe and supported and one who only appears so on the surface. If we want to highlight these kinds of inequities and ensure that schools are addressing them, then we have no choice but to measure them. But we must be extremely careful in deciding which methods and tools to use, and how we are interpreting and using the data.

For example, if schools make high-stakes decisions based on narrow interpretations of nuanced survey data, they risk making flawed and even unethical decisions that overlook possible misinterpretation by the participant; potential reference bias (when students’ responses are skewed based on their own frame of reference); and the impact of school context (Duckworth & Yeager, 2015; McKown, 2019). For instance, because traditional survey tools, such as climate surveys administered to students, elicit individual responses, they can easily reinforce the assumption that attitudes, behaviors, and feelings —perseverance, optimism, joy, belonging, and so on — are entirely personal, having to do solely with the individual’s temperament, and not the surrounding environment or broader systemic influences (e.g., Jones, McGarrah & Kahn, 2019; Mehta, 2015; Simmons, 2017).

Some researchers, however, have begun to design and use tools that measure the ways in which a given school community (including the attitudes and behaviors of teachers, administrators, students, and others) affects students’ social and emotional development (Clark, Soutter, & Lee, 2020; Duckworth & Yeager, 2015; Jones, McGarrah, & Kahn, 2019; McKown, 2019). If we want to measure joy, it may be best to take a holistic look at the school’s climate, curriculum, and classroom practices, rather than zeroing in only on the feelings and attitudes of individual students.

A measurement model

EL Education (formerly Expeditionary Learning) is a national network of more than 150 schools that uses an expanded definition of student achievement that prioritizes character as a crucial, interlinking dimension. The organization recently rolled out a series of measurement tools designed not only to track social-emotional growth in their students, but also to ensure equitable learning environments for all students. I collaborated with their research team over the past year to document these efforts (see Clark, Soutter, & Lee, 2020) and found that EL Education shares my simultaneous commitment to and skepticism of trying to assess character strengths. Their measurement efforts provide a promising example of how schools might reconcile their desire for concrete, formative data about individual students’ social and emotional development with an emphasis on the larger school environment and its contribution to making learning joyful. So how are they putting these ideas in action?

EL Education champions a vision of character where students work to become effective learners and ethical people who contribute to a better, more equitable world; their first step was to compile, pilot, and validate character survey measures and align them with this definition. Their final toolbox includes a series of measures that fall into each of these dimensions of character. For example, two of the student survey tools that align with “becoming effective learners” are a growth mindset scale and a self-management scale; two surveys that encompass “becoming ethical people” include an empathy and perspective taking scale and a compassion scale; two examples of surveys that capture elements of students’ commitment to “contributing to a better world” include a civic responsibility scale and a civic empowerment scale.

Second, EL Education developed a data platform tool that enables school leadership teams to look at students’ character survey data in multiple ways. For example, they can disaggregate character measurement data by subgroups so that leaders can assess whether their teachers are serving all students equally well, and, if not, how to make the necessary adjustments. It is important to note that the purpose of this disaggregation is for teachers and leaders to recognize where they themselves can improve, and how they can improve the school community, and not, of course, to reinforce biased perceptions or to further entrench stereotypes.

Third, in addition to using self-report tools (that measure changes in specific qualities such as compassion and integrity over time), EL Education developed a walk-through tool that encourages school leaders to observe classrooms and nonacademic spaces throughout their schools and gauge the extent to which students’ experiences in the school support social and emotional well-being. School leaders conduct most walk-throughs, but teachers can participate as well. As more educators participate, there are greater opportunities for comparison and discussion. Instead of evaluating only individual character qualities, these walk-through tools attend to the ecosystem of the classroom, which includes how well students actively demonstrate social-emotional skills and support one another’s development and whether the teacher is providing opportunities for students to engage in these interactions and effectively facilitating them as they occur. This holistic focus helps prevent teachers and school leaders from simply giving a low character rating to a student who is having a particularly challenging class period and instead directs that observer to see how the rest of the class perhaps rises (or doesn’t) to support that student.

By examining survey data and walk-through data in relation to one another, EL Education schools are able to identify patterns and red flags, and the resulting information can guide next steps. For example, if disaggregated survey data suggest that members of a particular student population tend not to feel a strong sense of belonging at a school, the school might use the walk-through tool to better understand the classroom experience of students with that particular identity. Or if students are reporting high self-management skills, but the walk-through data contradicts this, teachers and students may need to recalibrate and have a conversation about how everyone in the building defines and practices these skills.

Finally, unlike traditional survey tools, these kinds of observations can also serve as a catalyst for authentic conversations around what social and emotional learning should look like and sound like in the classroom and beyond. What explicit strategies might teachers employ to ensure that all students feel a sense of belonging? What is the evidence that students are being kind and empathetic? How can the teachers within the school learn from each other’s practices? Is there evidence that social-emotional skills are transferring beyond the classroom (e.g., to lunch, recess, or pickup time)? Many EL Education schools share survey and walk-through data with students to allow them to ask questions, explain what they’re experiencing, and generate ideas for improvement. In this way, the data collection, observation, and reflection process responds to students and the school environment, empowering teachers and students to take ownership of improving the school climate. Because their measurements are driven by educators and what they are observing in their schools, EL Education is able to capture the nuances of what SEL instruction and outcomes look like in actual school settings in a way that traditional psychometrics have not.

Importantly, despite this commitment to measuring social and emotional learning in their schools, EL Education maintains a level of caution when doing so. Because they are aware of the ethical ramifications and potential pitfalls of measuring individual character strengths, they do not use the measures as high-stakes outcomes, a way to rank students, or a method of comparing schools externally.

Tracking what’s important

I’ve been skeptical about measuring social and emotional learning since I was first introduced to the idea as a 2nd-grade teacher. My school prioritized SEL for both students and families, the educators I worked were fiercely dedicated to treating their students as whole people, and our students cared for and supported each other. However, on a spring day in the copy room, a school leader for whom I have tremendous respect told me that she was looking into ways to measure character. I was not just horrified, but truly fearful. Fearful that reducing community and caring and kindness to nothing but numbers would extract the essence from respecting students as whole people. Fearful that these efforts would reduce our wonderful students to statistics. Fearful that it would take away all the joy that our focus on character and community were bringing to the classroom. One of the primary reasons I loved working at that school and believed in its mission was its all-encompassing commitment to prioritizing our students’ healthy development. I was afraid — and still am — that measuring character, social-emotional learning, and similar outcomes might completely detract from the actual purpose of promoting them.

But, if such measures are designed with great care, then I stand behind their use. I believe in the value of studying and tracking results. If we don’t examine carefully the plethora of SEL programming in schools in a critical, thoughtful, and yes, measured way, we can tread into dangerous waters where SEL certifications are hastily awarded to schools that sign up for a program, complete a checklist, or purchase a curriculum, without regard to whether and how well they are actually achieving the aims of promoting an emotionally safe culture. And if we ignore the widely documented disparities related to how student experiences of belonging, connectedness,  teacher support, discipline, and civic empowerment divide along racial and socioeconomic lines, we risk entrenching these divides further (Brown & Tam, 2019; Campbell, 2012; Fergus, 2019; Gregory, Skiba, & Noguera, 2014; Levinson, 2012; Love, 2019; McMahon & Wernsman, 2009; Moreno, 2008; Noguera, 2002; Spring, Dietz, & Grimm, 2007; Wallace et al., 2008; ).

Measurement must be done, but done thoughtfully, with equity in mind, and in such a way that embraces the complexity of the task and does not snuff the joy out of whole child education. Social and emotional learning in schools must not be reduced to a growth chart, a report card, or something to be rewarded with a sticker. As educators, we have measurement tools, and we need to hold ourselves accountable for authentic, meaningful social-emotional learning. But sometimes, we also need to put those tools down, walk outside with our students, marvel at the clouds they are pointing to, and be joyful — and not try to measure it.

References

Berg, J., Osher, D., Same, M.R., Nolan, E., Benson, D., & Jacobs, N. (2017). Identifying, defining, and measuring social and emotional competencies. Washington, DC: American Institutes for Research.

Brown, C.S. & Tam, M. (2019). Ethnic discrimination predicting academic attitudes for Latinx students in middle childhood. Journal of Applied Developmental Psychology, 65, 1-11.

Campbell, D.E. (2012). Civic education in traditional public, charter, and private schools: Moving from comparison to explanation. In D.E. Campbell, M. Levinson, & F.M. Hess (Eds.), Making civics count: Citizenship education for a new generation (pp. 229–246). Cambridge, MA: Harvard Education Press.

Clark, S., Soutter, M., & Lee, A. (2020). Measuring character: EL Education’s journey in developing tools for improvement and impact in social, emotional, and academic learning: Theoretical, psychometric, and educator considerations. New York, NY: EL Education.

Duckworth, A.L. & Yeager, D.S. (2015). Measurement matters: Assessing personal qualities other than cognitive ability for educational purposes. Educational Researcher, 44 (4), 237-251.

Fergus, E. (2019). Confronting our beliefs about poverty and discipline. Phi Delta Kappan, 100 (5), 31-34.

Gregory, A., Skiba, R.J., & Noguera, P.A. (2014). The achievement gap and the discipline gap: Two sides of the same coin? Educational Researcher, 39 (1), 59-68.

Haberman, M. (1991). The pedagogy of poverty versus good teaching. Phi Delta Kappan, 73 (4), 290-294.

Immordino-Yang, M., Darling-Hammond, L., & Krone, C. (2018). The brain basis for integrated social, emotional, and academic development: How emotions and social relationships drive learning. Washington, DC: The Aspen Institute: National Commission on Social, Emotional, & Academic Development.

Jagers, R.J., Rivas-Drake, D., & Borowski, T. (2018, November). Equity and social emotional learning: A cultural analysis. Chicago, IL: CASEL.

Jagers, R.J., Rivas-Drake, D., & Williams, B. (2019). Transformative social and emotional learning (SEL): Toward SEL in service of educational equity and excellence. Educational Excellence, 54 (3), 162-184.

Jones, S.M., McGarrah, M.W., & Kahn, J. (2019). Social and emotional learning: A principled science of human development in context. Educational Psychologist, 54 (3), 129-143.

Levinson, M. (2012). No citizen left behind. Cambridge, MA: Harvard University Press.

Love, B.L. (2019). We want to do more than survive: Abolitionist teaching and the pursuit of educational freedom. Boston, MA: Beacon Press.

Mahoney, J.J., Durlak, J.A., & Weissberg, R.P. (2018). An update on social and emotional learning outcome research. Phi Delta Kappan, 100 (4), 18-23.

McKown, C. (2019). Challenges and opportunities in the applied assessment of student social and emotional learning. Educational Psychologist, 54 (3), 1-17.

McMahon, S.D. & Wernsman, J. (2009). The relation of classroom environment and school belonging to academic self-efficacy among urban fourth- and fifth-grade students. The Elementary School Journal, 109 (3), 267-281.

Mehta, J. (2015, April 27). The problem with grit. Education Week.

Metropolitan Center for Urban Education. (2008, October). Culturally responsive classroom management strategies. New York, NY: NYU Steinhardt School of Culture, Education, and Human Development.

Moreno, M. (2008). Lessons of belonging and citizenship among hijas/os de inmigrantes Mexicanos. Social Justice, 35 (1), 50-75.

Noguera, P.A. (2002, December). Joaquin’s Dilemma: Understanding the link between racial identity and school-related behaviors. In Motion Magazine.

Paris, D. & Alim, S.H. (2017). Culturally sustaining pedagogies: Teaching and learning for justice in a changing world. New York, NY: Teachers College Press.

Simmons, D. (2017, June 7). Is social-emotional learning really going to work for students of color? Education Week.

Spring, K., Dietz, N., & Grimm, R., Jr. (2007). Leveling the path to participation: Volunteering and civic engagement among youth from disadvantaged circumstances. Washington, DC: Corporation for National and Community Service.

Wallace, J.M., Jr., Goodkind, S., Wallace, C.M., & Bachman, J.G. (2008). Racial, ethnic, and gender differences in school discipline among U.S. high school students: 1991-2005. Negro Educational Review, 59, 47–62.

Weinstein, C., Curran, M., & Tomlinson-Clarke, S. (2003). Culturally responsive classroom management: Awareness into action. Theory Into Practice, 42 (4), 269-276.