Sunday, November 26, 2017

Walls that invite others in: Transform your conversations about data

Last spring, I was invited to put together a 3.5 hour workshop for our state assessment conference that happens every December. I was told that the topic could be anything I wished. And, of course I agreed to do just that and hastily wrote a description in early June...and by the time my summer holiday started in July, I was already panicking. What the hell had I written? How was I going to deliver on this description?

Public narratives about schools often focus on measures of student performance. However, students are more than the sum of their test scores, demographics, and program participation. Join our conversation about how to repurpose and leverage data walls to create, inspire, and communicate with audiences about what matters most. As part of this interactive session, we will share examples of innovative data stories from both school and district levels, as well as tools and strategies for constructing new narratives about student and school outcomes.

Basically, I said I would facilitate some learning around how to build the sort of data (story) walls we've been doing recently. This sounds great on the surface. I truly believe that we have change the types of conversations we are having with and about data in education. In practicality, however, it turns out to be really challenging to develop guidance on how to listen critically for opportunities to transform these interactions...much less how to tap into and be confident about being creative with displays. As Isaac Asimov said, "The world in general disapproves of creativity...and to be creative in public is particularly bad."

I made a lot of notes this summer based around my own questions about using data, and this fall, I've been chipping away at organizing all my notes from the last 18 months into some sort of outline. Combined with the data academy work I'm guiding this fall, I have felt overwhelmed by the amount of adult learning I need to generate...all while trying to do my "regular" work.
But, at long last, I think I'm ready for this workshop...which is a good thing, because it will happen on Wednesday. I don't have a lot of time left to fuss with it. It's time to fish or get off the pot, so to speak.

The slide deck is 150 slides (and 90 MB). I have around 10,000 words in my speaker notes. I've incorporated references to King Kong, Frankenstein, Pink Floyd, ETA Hoffman, and The Inferno. And, I've built a companion site on GitHub for all of the other links, references, handouts, and structure.

We'll see how this goes. Three and a half hours is a long time to spend with 40'ish people on one topic. The whole premise---building these oversized, experiential data installations---requires a different sort of investment and risk-taking on their part. I am not foolish enough to believe that they will all go back to their schools and districts and put up these displays. But let's say three of them do. What a wonderful start...and how I would love to know of others who are doing this work. It all has to start somewhere. I will plant seeds now and see what sprouts.

Monday, November 6, 2017

Five: On the Bubble

Student achievement data is among the most public data sets that exist for education. Scores from annual summative tests wind up in all sorts of places, from the front page of the newspaper to Zillow. Because of that, I've been shying away from using these data as the source of a data story. They already get enough attention, right?

But then I started hearing some conversations in my district that piqued my interest. (You can read about that on my other blog.) So I pulled scores for our current sixth grade students. I looked at their scale scores and performance levels from third, fourth, and fifth grade in the area of English language arts. I was interested in a few different things:
  • What is the average growth students made between grades 3 and 5?
  • What are the common characteristics of students who are persistently low-performing (or high-performing)? What about the ones in the middle (a/k/a "the bubble")?
  • What do we mean when we refer to groups of students as red kids or bubble kids?

The Data
Calculating the growth was easy: just subtract the 2015 score from the 2017 score. But understanding the significance of that change was a bit more challenging. Is it better to gain 75 points vs the 70 necessary to show two grade levels of growth? Probably, but how much better? I wasn't ready to just declare an arbitrary cut. I was going to need some stats to guide me.

Now, all you stats lovers out there can probably think of several ways to represent this (and maybe a few will leave comments to tell me how I can do it mo betta)...but in the end, I decided to do something very simple. I transformed the values representing the amount of change in scores to z-scores. I normalized them. Why? Because this gave me a quick idea of how many students had performance that was relatively typical to their peers (within one standard deviation) and which ones had even larger amounts of change (two or three deviations from the mean).

The Build
Once I had the various groups (-3, -2, -1, 1, 2, 3 SD), I thought more about how to represent this. But to back up a little bit, I have to say that my favourite starting idea was to cut different diameters of dowel rods into lengths that represented the amount of change for each students performance. Since we often talk about bubble kids, I wanted the data to look like bubbles. However, I don't have a saw that would make this simple work...and thinking about all the measurement involved and sorting things out made me a little queasy. I wanted a round shape, though. Lucky for me, Amazon sells bags of wooden disks in 1", 2", and 3" sizes. A-ha! Now I had something that corresponded to the calculated SDs and could easily paint them to match the level of student performance.

I still wanted to work in the concept of change. In other words, do students tend to stay at one particular level of academic performance? To represent this, I decided to stack the disks. If a student had spent only one out of the last three years at the most recent performance level, they got one disk. Two years at a level was shown by two, and three years with three disks.

Another challenge was to represent negative change. The raw difference between 2015 scores and 2017 scores ranged between -101 and +226. Although I could still place the disk on the display according to the 2017 score, I decided to paint the disks black to represent the "hole" in performance over time. Then, a roofing nail was attached to the back and the raw score written on the edge for placement reference.

The other materials for the build included five shades (orange, yellow, green, blue...and black) of shiny enamel paint in sample sized cans, wood glue and super glue, .75" roofing nails (leftover from the last story), fabric with small circles to reflect the bubble theme, and glittery grey paper for the lettering. My assistant and I punched 1" discs in the paper and used an enlarged dot matrix font to guide the creation of the letters. We slid the template for the words behind the fabric on the bulletin board, which was enough for us to see where to peel and stick each dot before sliding the template back out.

The "bubbles" were pushed into the board like oversized push pins, using the y-axis of the board as a guide for placing each disk in position (axis ranged from 2200 - 2800, with 2502 = met standard). There was no particular value assigned to the x-axis. We placed disks in that direction purely based on aesthetics of the overall display.

Lessons Learned
When I get an idea for one of these, it comes together very quickly. This one took a week from inception of idea to final product. And yes, that includes a lot of weekend time to pull, clean, organize, and transform the data. But I am always grateful to The Muse, when she deigns to visit, and willing to give in to the compulsion to create these works. I am also grateful to have an enthusiastic and supportive working environment for these. I try to keep them cost effective (the last two have included ~$100 in materials) and am always sure to keep up with my regular work while cranking out this other stuff.

These builds include a lot of trial and error. I am embarrassed to share how long it took us to figure out that we could just slide a patterned piece of paper behind the fabric on the board to use a template for the letters. We had googled for ideas for paper-to-fabric transfer. We plotted different ideas using a measuring stick and T-square. We drew things in PowerPoint. And then, duh, we figured it out. We did try different types of glue to see which was best for getting the non-pointy end of a nail to stick to wood. And so on. The lesson here is that even when you have to kill your darlings during the ideation phase, you can't give up during the build. Keep going.

To learn more about this data story, please visit the companion page for the analog build.