sel and measurement

By Sam Piha

​There is a growing consensus that social emotional learning (SEL) and character building are important parts of a young person’s education and preparation for life. This has resulted in a call for measurement and related tools. There are a number of questions we have:

• Who or what is the subject of SEL measurement? Are they individual youth, the adult youth workers, or the program climate?

• What is the purpose of SEL measurement data? Is it program improvement or improved skills of individual youth over time?

In a recent commentary in EdWeek entitled, We Should Measure Students’ Noncognitive Skills, the author suggested that we should be measuring the skills of individual youth in order to identify those youth that have low SEL skills. We had some concerns about this approach, so we recently asked a number of experts to respond to our questions above. Below are some of their responses.
​
Q: Who or what is the subject of SEL measurement?

---

Corey Newhouse (Public Profit): As you know, I’m frequently concerned by the rush to youth-level outcomes in measurement, without consideration for the work that the adults need to do first. I therefore appreciate that you include prompts around setting and adult practice.

​While we are ultimately interested in how youth participants benefit from a particular SEL-focused program or intervention, it is critical to first focus on the adult practice. Without assuring that adults are creating the kinds of experiences that we know help kids to build these skills, we can’t then look for evidence of changes in young people’s skills or mindsets.
​
Once we’re assured that adult practices have shifted, then we should look for evidence that kids are changing, too. Unfortunately, we’ve gotten in a bad habit of blaming kids for things that adult staff didn’t do. This is a good opportunity to change that.

---

Roger Weissberg (CASEL): For me, the answer is all three. There should be a theory of change that explains how the setting (climate) and staff practices enhance the development of which youth participants and in what ways.

---

Christina Russell (Policy Studies Associates): It’s important to first understand the practice and capacity of adult leaders:  how do they implement programming that supports SEL development and character building for youth?

​Observations of program implementation and collection of data about the capacity of adult leaders provides context to the program’s readiness to foster these skills in youth.  Adult practice is closely connected to the climate and setting of the program; are youth in a strength-based environment that infuses SEL throughout the program and offers experiences in which youth collaborate, reflect, and think critically?  With these structures in place, youth data can provide information to refine the skills targeted by the program, based on youth needs.

---

Dale Blyth (University of Minnesota): The subject depends on the purpose of the assessment but in my view we need data of all three types. We need to know where youth are on these SEL skills and attitudes so we can better plan and support effective strategies for growing these competencies. Such data used for improvement, not judgment, can also open discussions with the youth, parents and others. A key to remember with data on youth is not only confidentiality but making sure it is not used to blame youth for their lack of skills. SEL competencies need to be approached as strengths, not deficits, and need to be grown and built, not treated.

​We need data about the dispositions and SEL competencies and teaching strategies of teachers. They are the key to improvement of students’ skills. Since SEL skills are more caught than taught and teachers are one of the people they catch them from, data that can support better understanding and improvement of teacher’s skills and attitudes are key. Building such data into teacher evaluation systems is too hot a political football to do well and not something I would recommend.
Finally, data on school climate is also critical for the purpose of understanding the broader environment in which students spend their time – whether school or out of school programs.
So for me, the issue is not about which is the subject of the data but how to use data from each in an integrated improvement-centered approach.

---

Q: What is the purpose of SEL measurement data? 

Corey Newhouse: Ideally, we get a two-fer from measurement data. It can help staff members make data-informed changes to their practices, and demonstrate that young people are building the skills and mindsets we seek.

Measuring change is very, very tricky, however, especially since so many of the tools in this area are in development, and we don’t yet have a clear connection between specific program practices and specific skills and mindsets. For now, we tend to think of youth-focused measurements as a way to track the proportion of youth who are demonstrating the skills and mindsets we seek.

Roger Weissberg: I think it is important to improve program practices that will enhance students’ skills and behavior from pre to post to follow up. It is also helpful to know to measure kids’ competence levels at the start of the program so we can better understand which youth benefit – the kids with low skill levels, high skill levels, or all kids!

Christina Russell: A primary purpose of SEL and character-building data should be to inform program practice. Too often there is a disconnect between program implementation and desired outcomes:  if a program goal is to help youth strengthen SEL skills, activities should explicitly address, reinforce, and provide opportunities to practice those skills. Data can be used to more clearly define priority skills for youth to develop, to identify professional development opportunities that build staff capacity to support those skills, and to design activities that are aligned to and that target the desired skills. Without a clear pathway from implementation to outcomes, it’s not appropriate to use data to hold program staff—or youth—accountable for improvements in SEL or character building competencies.
​
Dale Blyth: ​Measurement of key concepts and the effective use of data require clear purposes. It is where people who are thinking about gathering data or doing analysis should start and the answer may determine both whether and how to proceed. It is particularly important, in my view, to separate out universal efforts to improve these skills and attitudes from efforts to screen youth for services. The purpose behind the first is to build these strengths for all youth while the second is to identify youth who need extra services or interventions because of a lack of skills or the problems that result. While both purposes are honorable and can be appropriate, we too often slip into the second when we should be more focused on the former.