I have had the privilege of presenting at the ASCD annual conference over several years. I've been an ASCD member nearly my whole career. It's an organization that, as the rebranded conference name suggests, empowers educators in all roles to support students.
This year, I am presenting on the qualitative side of data. My session description is "If 'not everything that counts can be counted,' as Albert Einstein suggested, then how do we measure and represent student growth beyond test scores and grades? In this interactive session, you will learn strategies that capture student learning in multiple ways, as well as how to communicate feedback about the whole child using data visualization. Join the conversation about how to apply digital and analog tools to tell your students' stories and report the full spectrum of student learning."
The challenge of doing a presentation like this is that I have to submit the description more than six months before the conference. Whatever it is that I had in mind to talk about in August was long gone before I received notification that the proposal was accepted...let alone when I sat down to build the content. I am influenced, too, by all the things I have learned in the interim.
The basic story arc did finally emerge. I'll start first by talking a bit about why data visualization can be a powerful tool. This is my usual lead-in, and I think it helps to provide a few easy to grasp examples before launching into new territory. The next hook is to talk about achievement data. Now, this particular piece does not explicitly fit the session description, but my goal is to move from the larger scope of the purpose of data viz to what we typically see in education, and finally into non-traditional ways to represent education data...and perhaps even a little further than that.
I heard a presenter this morning say that "schools embrace business ideas as they are fading." In other words, what was hot in the private sector 5 or 10 years ago becomes the things that schools are talking about now. I have seen this happen a lot over the course of my career. And what worries me most now is that decisions about data privacy and access are being made now that will affect schools in ways they haven't even anticipated yet. I am not going to claim that I can change the world with my presentation and suddenly schools will make these conversations a priority...but it's a start.
My call to action for them is around being in control of creating their own narratives using data and to think about what they want to represent, not necessarily what they are told to represent. All too often, the public view of school data is just annual test scores. But children are so much more than the sum of their test scores. They deserve a more robust approach to sharing their stories (and to be involved in that process, as well).
I have an ancient (by web standards) wiki where I have placed materials for this session. Someday, I'll move everything over to GitHub...but for now, it's a reminder of the journey I've taken to this point and perhaps a place to shape the ideas ahead.
Tuesday, March 28, 2017
Sunday, March 26, 2017
ASCD 2017: Data Tools
This weekend, I am at ASCD's annual conference, now referred to as Empower. This is not the fourth year that I've attended (and presented...but more on that in another post), but it is the fourth time I've wandered through the exhibit hall with an eye toward what various companies are promoting to schools. The versions from 2013, 2014, and 2015 are available, if you'd like a trip down memory lane. In fact, that might not be a bad idea, because (Spoiler alert!) there was nothing particularly new or outstanding.
Data management and reports
There are no stand-alone systems here this year. Lots of vendors who focus on assessment and grading, however, do have displays and reports for managing student information. I asked the same question that I have asked for the last five years: Who builds your displays and reports? And yes, of course the developers make the magic happen behind the scenes, but there is still the same disturbing number of companies out there where that is the only answer. Or, there may be something along the lines of "we got feedback from teachers." This is all well and good---I support user involvement. I also know that developers and teachers are not data designers. People are spending a lot of time on these products, but they don't care enough to make the effort to ensure that effective communication with the data actually happens.
In one particular exchange, the project creator told me that all her charts were a result of her research. I have no doubt that the project is spawned from many years of hard work with teachers...but I know that she does no research with data design or communication. In fact, she was a little upset that anyone would ask about how she came to make her choices. I won't link to it here, but it's a new partnership with ASCD that you can look for, if you're interested.
Student assessment
There were some different displays this year for various flavours of student assessment. Scantron is making a big show this year. I chatted with them a bit and they have a few sessions this weekend. Perhaps I just hadn't noticed before that they are more than the "bubble sheet" company, but it looks like they're diversifying and growing into student assessment. I can't speak to the quality of this new content, but in an age of apps, google tools, and other options, it seems wise to be more than a company involved with scoring assessments.
Pacific Metrics is a content aggregators for various assessment banks, like the ACT or district-developed items. It produces no reports---it just integrates with existing school information systems. I think this is a desirable option for a lot of districts, especially smaller ones who may not have the resources to develop their own content. If someone else has valid and reliable items for you...and those can be automagically scored and then imported into your gradebook...it could be helpful. It's not a replacement for professional judgment---teachers would still need to ensure that the assessment matches the content.
The product I liked best came from Exemplars K - 12. This company provides not only rich tasks for the classroom, but also scoring tools and anchor papers. This last piece is incredibly valuable. These exemplars are representative of actual student work and can show teachers how to implement the rubrics. While I will always advocate for teachers to come together to develop tasks, score student work, and engage in conversations about student learning as a result, I can't deny that these banks would also support good teaching. I think this is especially important for rural or small districts, or schools with a high rate of turnover so that new teachers have a consistent framework in place.
Have you seen something this year for student data or assessment that you like?
Data management and reports
There are no stand-alone systems here this year. Lots of vendors who focus on assessment and grading, however, do have displays and reports for managing student information. I asked the same question that I have asked for the last five years: Who builds your displays and reports? And yes, of course the developers make the magic happen behind the scenes, but there is still the same disturbing number of companies out there where that is the only answer. Or, there may be something along the lines of "we got feedback from teachers." This is all well and good---I support user involvement. I also know that developers and teachers are not data designers. People are spending a lot of time on these products, but they don't care enough to make the effort to ensure that effective communication with the data actually happens.
In one particular exchange, the project creator told me that all her charts were a result of her research. I have no doubt that the project is spawned from many years of hard work with teachers...but I know that she does no research with data design or communication. In fact, she was a little upset that anyone would ask about how she came to make her choices. I won't link to it here, but it's a new partnership with ASCD that you can look for, if you're interested.
Student assessment
There were some different displays this year for various flavours of student assessment. Scantron is making a big show this year. I chatted with them a bit and they have a few sessions this weekend. Perhaps I just hadn't noticed before that they are more than the "bubble sheet" company, but it looks like they're diversifying and growing into student assessment. I can't speak to the quality of this new content, but in an age of apps, google tools, and other options, it seems wise to be more than a company involved with scoring assessments.
Pacific Metrics is a content aggregators for various assessment banks, like the ACT or district-developed items. It produces no reports---it just integrates with existing school information systems. I think this is a desirable option for a lot of districts, especially smaller ones who may not have the resources to develop their own content. If someone else has valid and reliable items for you...and those can be automagically scored and then imported into your gradebook...it could be helpful. It's not a replacement for professional judgment---teachers would still need to ensure that the assessment matches the content.
The product I liked best came from Exemplars K - 12. This company provides not only rich tasks for the classroom, but also scoring tools and anchor papers. This last piece is incredibly valuable. These exemplars are representative of actual student work and can show teachers how to implement the rubrics. While I will always advocate for teachers to come together to develop tasks, score student work, and engage in conversations about student learning as a result, I can't deny that these banks would also support good teaching. I think this is especially important for rural or small districts, or schools with a high rate of turnover so that new teachers have a consistent framework in place.
Have you seen something this year for student data or assessment that you like?
Monday, March 13, 2017
Three: Student Information
In theory, I was going to publish one data story a month this year. In reality, it's March (the school year started in September) and I'm only on my third one. I am way behind on my goal. But I am learning to make my peace with that sad state of affairs. This project is going to run into the next school year...and I'm okay with that.
So let's talk about number three. It's a magic number, is it not?
This month(ish), we're looking at our various student information systems. Each collection of squares represents one system. On the left is Skyward, our district system...and data flows various directions from there to other systems, including TIDE on the right that we use for state assessments.
Each group has layers that are colour-coded by the type of accounts/users it houses. Green is for students, yellow is for teachers, pink is for school administrators, and little red pins are for district administrators. Only one system has blue, representing parents. The sizes of the squares tell you something about the number of people represented by the data set. Each square inch is 50 people. The green squares are largest and district administrators the smallest. All of the systems, excepts for one, include students.
Two of the systems that I chose to represent (SWIS and Google Apps) are connected to our system with a broken line, because there is not a direct data connection. Instead, a system of imports and exports is used.
I also built some charts to show a bit about how families are accessing Skyward. Generally speaking, they log in about twice a month, during the work week, and in the morning.
I don't have any specific data on how many users are represented by our state data warehouse (CEDARS) or GoogleApps. I can only tell you how many data points we transfer in a given week (~250K to CEDARS) or documents we share online (over 600K in Google).
Bottom line: There's a lot of data flowing around.
Questions to Ponder
I selected the topic of information systems because they really are invisible...yet their impact is very real. Me? I'm represented by those nearly invisible red pins in the center of almost every square. I can see all these data, but there are a lot of people who can't due to their permissions or system access. This data story project this year is about sharing data beyond the usual suspects like attendance or achievement. Information systems are a good place to shine some light.
In the end, this is really a story about power and privacy. You'd think that the biggest group in these systems (students) would have the greatest power to use these data, but the fact is simply that they have none at all. Some systems look large, like TIDE, and yet a student or teacher might log in only once or twice a year. Others, like Homeroom, look insignificant and yet they are our most powerful tools for reviewing student information. Looks are deceiving.
Bonus Round
While the offline bulletin board is intended to be a conversational piece, as well as a way to reach audiences that might not have an Internet connection, I always put together an online component, too. This time around, I share a video about the historical origins of personal privacy and provide a way for you to look at how the clicks you and others contribute to our web site add up.
Peekaboo...I see you.
So let's talk about number three. It's a magic number, is it not?
This month(ish), we're looking at our various student information systems. Each collection of squares represents one system. On the left is Skyward, our district system...and data flows various directions from there to other systems, including TIDE on the right that we use for state assessments.
Two of the systems that I chose to represent (SWIS and Google Apps) are connected to our system with a broken line, because there is not a direct data connection. Instead, a system of imports and exports is used.
I also built some charts to show a bit about how families are accessing Skyward. Generally speaking, they log in about twice a month, during the work week, and in the morning.I don't have any specific data on how many users are represented by our state data warehouse (CEDARS) or GoogleApps. I can only tell you how many data points we transfer in a given week (~250K to CEDARS) or documents we share online (over 600K in Google).
Bottom line: There's a lot of data flowing around.
Questions to Ponder
I selected the topic of information systems because they really are invisible...yet their impact is very real. Me? I'm represented by those nearly invisible red pins in the center of almost every square. I can see all these data, but there are a lot of people who can't due to their permissions or system access. This data story project this year is about sharing data beyond the usual suspects like attendance or achievement. Information systems are a good place to shine some light.
In the end, this is really a story about power and privacy. You'd think that the biggest group in these systems (students) would have the greatest power to use these data, but the fact is simply that they have none at all. Some systems look large, like TIDE, and yet a student or teacher might log in only once or twice a year. Others, like Homeroom, look insignificant and yet they are our most powerful tools for reviewing student information. Looks are deceiving.
Bonus Round
While the offline bulletin board is intended to be a conversational piece, as well as a way to reach audiences that might not have an Internet connection, I always put together an online component, too. This time around, I share a video about the historical origins of personal privacy and provide a way for you to look at how the clicks you and others contribute to our web site add up.
Peekaboo...I see you.
Subscribe to:
Posts (Atom)