Jon Krohn: 00:00:00 My guest today took a public school that was about to be shut down and turned it into the number one school in Boston, and AI is her latest secret weapon. Welcome to episode number 983 of the SuperDataScience Podcast. I’m your host, Joh Krohn. Today’s guest is Traci Walker-Griffith, principal of the Elliot School in Boston, a kindergarten through eighth grade school that thrives on AI innovation. In a long overdue episode on AI for supporting children’s education, hear directly from Principal Traci , how her teachers have been experimenting with AI in classrooms, what works, what doesn’t work, and what’s next for kids as LLMs continue to improve. Enjoy this one. This episode of SuperDataScience is made possible by Anthropic, Acceldata, and Cisco. Traci, welcome to the SuperDataScience Podcast. I am so excited for today’s episode. Where are you calling in from today?
Traci W.: 00:00:53 So we’re calling in from the north end in Boston. Super excited to be here, Jon. Thank you so much for having us.
Jon Krohn: 00:00:59 Yes, we have been doing this show for almost 10 years now. I’ve been hosting for over five. And we haven’t had an episode dedicated to how in all of this AI change that’s engulfing the world, how do we deal with the children? How are we going to educate kids in this scenario where AI is doing so much where it has so much opportunity to be helpful, but also there’s lots of risks. Parents have so many questions. So many of my guests in recent episodes have brought up topics that make us wonder how we’re going to deal with AI with our kids. And I’m so excited to finally be able to dig into it with you.
Traci W.: 00:01:43 Me too.This is our passion and we’re excited to share our story and what’s next.
Jon Krohn: 00:01:50 Nice. And so I’m going to ask you in the very next question what your job is, and you can explain that to the audience. But first I just want to explain how we know each other because we do have a personal connection. So you and I haven’t met in person. We’re recording remotely, but I’ve spent a lot of time with someone who I believe is your cousin, whom you call Mike Walker Jr., And I just call him Mike Walker. And so Mike Walker works at Bloomberg in New York, and him and I have collaborated on lots of ads for television. So for clients like Nvidia, Dell, AT&T, we have made lots of nationwide ads together. He’s sensational to work with. So thoughtful, so thorough, an amazing journalist in his own right. And yeah, he’s your cousin, right?
Traci W.: 00:02:36 And he’s my cousin. We’ve grown up going to the beach together, going on vacations, spending many hours as kids on the beach at the Cape. And as adults, we ended up on the beach this past summer talking about AI. And we have some strong opinions, both of us, me being in education and Michael Walker Jr. Being in journalism. And so when we left the beach, he said, “I’ve got to email my friend Jon Krohn because you would love him.” And here we are, almost eight months, nine months later.
Jon Krohn: 00:03:13 Yeah, exactly. It took a little while to get together because there’s lots of potential podcast episodes that we can be covering, but we’ve got a sensational episode planned in terms of content. So let’s not make our audience wait any longer. Let’s get right into it. So explain, we know that you’re in Boston and you’re a principal in Boston at what sounds like a pretty special school, the Elliot School.
Traci W.: 00:03:38 So I am the proud principal of the Elliot School and it’s located in the historic north end. I’ve been at the school for 19 years and I’ve worked in the Boston public schools for 34 years. The running joke is I started when I was 12, although that is not actually true. And I am so excited that as a school that in April Fool’s Day, which is almost 19 years ago to the day, in 2007, I walked into the Elliot School and it was on the closure list and I was told it needed transformation or improvement at the very least. And 19 years later, I stand in a school community that has three buildings from 150 students to, in 2007 to 800 students across three buildings in the north end. So it’s a very exciting journey and it’s the coolest job in America for sure.
Jon Krohn: 00:04:38 Nice. And those three buildings, they represent three different levels of elementary school, right?
Traci W.: 00:04:43 That’s correct. So we have a lower school for our early childhood ages three to six years old. We have the intermediate school, so that grades second, third, and fourth. And then our upper school is grades five through eight. So our age band scans ages three to 14 across three buildings.
Jon Krohn: 00:05:06 I love the idea. I imagine if I was a kid in that school, when you’re in one of the younger two schools, you’d look at the upper school and be like, “Those guys are so old. I can’t wait to be there.” And then when you get there, you must feel like a boss. Well,
Traci W.: 00:05:20 It’s so interesting you should say that because the Littles, we call them the Littles, actually travel to the upper school. We have a robust STEM programming. So in the Littles, they have STEM and computer science, so they’re coding and design thinking. And then in fourth grade, they start robotics. And so we do a collaborative opportunity for the upper school students to host the Littles, and then the Littles get to actually host the upper school students when they’re showcasing some of their STEM projects or their coding with Scratch Junior or their other work with Kibo, some really cool opportunities to continue to collaborate.
Jon Krohn: 00:06:02 And this wasn’t on my planned path for the conversation, but this is too fascinating to not dig into a little bit more. Tell us about these STEM projects and robotics in more detail. What at these different age levels, what’s the ideal programming interface? How do you get kids? I mean, if we have listeners with kids across this age range that you cover from I guess four to 14, basically that kind of age ranger.
Traci W.: 00:06:27 Yes, that’s accurate. I mean, this is the most … I mean, we love everything that happens at the Elliot. And I think part of my journey as I was a technology teacher in the early ’90s, thinking about dial up and Netscape Navigator and thinking about the internet and the way it would change the future for my students in 1990. So when I took over the Elliot, part of my commitment was, yes, we need reading, writing, and math, and we need what does technology do to support our students’ journey? So our four year olds are interacting with Kibo and BBOTs and thinking about programming and steps and that input is directly related to output. So good input gets you great output. Mediocre input, not so great output, so that from the age of four, students are understanding their role as the lead learner in all parts of their day.
00:07:32 And so it’s really amazing kids are understanding the design thinking process and it’s not about product as much as it is about process.
Jon Krohn: 00:07:42 Wow. You talking about the quality of inputs reminds me of a very popular saying in data science that you may be aware of already, which is garbage in, garbage out.
Traci W.: 00:07:50 That’s exactly right. And I mean, we even have math problems where you have the in and the out. So this is not just in one space, it’s permeating, this is the collaborative community that we cultivate at the Elliot.
Jon Krohn: 00:08:07 Really cool. And so now let’s talk about AI. So we’ve talked about technology a little bit in general. You could have had students doing programming and working with robots for many years now, but it’s only probably very recently that they can be interacting with LLMs. And tell us about that journey. When you first proposed some of the topics that we could be covering in this episode, at that time, you were using Claude as the key LLM internally, but now there’s been a switch to Gemini. For our listeners to understand, as the AI landscape continues to evolve, how does one make a decision for what kind of LLM or what kind of system or what kind of framework they should be deploying for kids to work with?
Traci W.: 00:08:54 So that’s a great question. I think we think about this all the time. When we launched our work, it was the spring of 24, and we were thinking about that we could not get feedback to students at the rate that we knew would impact the outcomes, right? We wanted accelerated outcomes. And so we had a problem to be solved and with thinking about AI, and we were just at that little, that early adoption, that Roger’s adoption skill, we live by it. When we take on initiatives that as an innovation school, we’re an innovation school, an incubator for innovative ideas, we decided that we were going to use Claude as teacher facing and think about ways that we could build a system where we could train our Claude to give feedback to student writing. Obviously, under the hood, the teachers had to do a lot of work behind it.
00:09:57 And then fast forward to the fall-
Jon Krohn: 00:10:01 If you don’t mind me just interrupting and clarifying quickly there. So when you talk about that you had to kind of change your Claude, what you mean there is prompt engineering, I assume.
Traci W.: 00:10:10 Yes, we call it the good prompt whisper. So we created some really great prompt whisperers and prompt engineering was so hard. I mean, Jon, when I tell you, in the spring, we were just doing it as a group of teachers that were failing forward, right? And that’s, we embraced that fail forward. And in the fall, we were, well, late summer, we were given an opportunity to join a cohort of schools from across the country, mostly high schools. When someone opens the door, we push right through to join a collaborative, it was called the AI Collaborative with leading educators at that time as Learning Accelerator and Play Lab. And we took our problem to be solved to this convening that was in Denver in October and we built this kind of system or the way in which we thought about it with Claude as our LLM, because at that time, we felt very confident with everything that we had explored that Claude was that LLM that would be maybe less hallucinogenic at the time and would be able to give us a place to, as a repository for all of our work.
00:11:32 And so we took on a year long journey with Claude and our ELA humanities team in grades five through eight, and we started early on giving feedback to students using Claude. And as I said, the prompt engineering was very frustrating and it was great at the same time because when things don’t work, it allows people, the humans or the learners, the adult learners at this point to interact with one another and with Claude and they were getting better together. And this is that collective genius that we talk so much about. I’m a big fan of Linda Hill and the work that’s coming out of, it’s not just one genius, it’s a collective genius. And this creative abrasion that was happening, I mean, it was magical. I mean, this is, I mean, I think of the way in which the Elliot has transformed from an underperforming, under-enrolled school on the closure list to the highest performing school in Boston, the number 13 performing school in Massachusetts because-
Jon Krohn: 00:12:44 Wow, congrats.
Traci W.: 00:12:45 Yeah, we just keep chipping away at everything that we’re in a constant learning edge. We’re never satisfied. And we say with the kids and the adults, when you think you’re done, you’ve only just begun. So with prompt engineering, that was the area that although we lost many hours in this way in which we were trying to figure out how to prompt Claude to give the most impactful feedback for students, people were meeting together, they were excited, they were laughing and crying and cheering when they got it right and they were like, “Oh, this is amazing.” That is just for that one student though, Jon. It wasn’t, you can give feedback to all the kids like this. It was very clear to us that the under the hood had to continue to be iterated with rubrics and an update on, so if a student was that far behind in their reading level, giving writing feedback that was not digestible was definitely not going to accelerate for our students.
00:13:56 And then we asked the consumers, right? We want to create critical consumers. We asked the kids, “How do you like the feedback?” And then we desegregated that feedback and said, so that students were already on level or above level, they just wanted more feedback. The students that were in the middle, like just on the cusp, they needed a little bit more, and that’s where Claude gave us more opportunities to create small group instruction. And then students that were the furthest behind the standard, we needed more help with creating targeted small group instruction and Claude was able to even help move that work. And then at the end of the day, it goes back to the data, what did the pre to the post assessment tell us with three iterations of feedback, and we were seeing a lot of gain, a lot of gains with our multilingual learners and our students with disabilities, which for us, that is where we wanted to move the most, that students, some of our students need to make over a year and a half growth just to continue to meet or exceed that growth that we’re setting. So it was a really exciting time in the fall of 24 all the way into the spring of 25, and that was our early adopter innovative group, and then our science team was starting to take on some of that work, and it was truly a pebble in the pond, right? Who’s leading this work, and then you move through it, and at the same time, introducing it to teachers in the lower grades, and I use this infinity symbol where everything is connected, right? So we’re three buildings, one school, how are we thinking about AI and AI enabled practices to accelerate student learning?
Jon Krohn: 00:15:48 Exciting. And so as Claude spread across the school, it started to make an impact in different ways, to different levels. And we’ll talk in a moment about how you’ve set things up differently for different age groups, but tell us a little bit more about the journey more recently. It sounds like you’ve now moved to Gemini.
Traci W.: 00:16:08 So we, in the spring of 25, when we’re a Google district, in the Google suite, Gemini was released and our district, we worked very closely with them because the recommended levels were grade seven and up for student facing. So phase two was, how are we thinking about student facing AI? And so by being able to use Gemini for grades five and six, we knew that students understood they were getting feedback from Claude and AI generated feedback, and they wanted to be more participatory in how AI was impacting their learning. And so in the younger grades, we were thinking about data and thinking about student writing and how to use Gemini and create Gemini Gems to think about small group instruction. So students in kindergarten through grade four are not seeing the AI student facing. The teachers are doing that work and building their skills and then grades five through eight, this fall we started thinking about more strategically, how does student facing AI work in our classrooms across … We have a block at the end of the day called Epic.
00:17:30 It’s Elliot Play Innovate Create where teachers pitch ideas and students pitch ideas to have 18 days at the end of the day to do a project. And there’s a lot of AI pitches going on right now, so it’s really cool.
Jon Krohn: 00:17:46 Really cool.
00:17:47 Yeah. There’s amazing things now just in the past couple months for building fully functional applications that are pretty slick. I can imagine a lot of students thinking of projects in that way. And of course, AI would be helpful for just brainstorming on ideas. Tell us a bit more about, with the younger kids, with the littles who aren’t interacting with the AI system directly themselves, how are the teachers using AI in that scenario?
Traci W.: 00:18:14 So that’s been really interesting. And I think it goes back to that adoption curve where the upper school and especially the ELA team at the ELA humanities team at the upper school was so deep in the work that we wanted to find ways for our teachers of the younger students to see the impact that AI could have. So what we do as a school community is every month, we’re pulling our 70 teachers together for professional development and our upper school teachers or our intermediate school teachers that have been doing some of that AI enabled practice, were sharing that work with teachers. So there was a deep commitment to the adult learning. So for brass tacks, right? So we think about the younger teachers have 25 students, they do a pre-assessment, the writing comes in, they can scan the writing. Obviously for the privacy laws, there’s no … Even though Gemini is in our Google suite, best practice, or we say next practice is we’re not using student identifiers, we’re putting that data into the Gemini gem that we’ve created based and we put the rubric in and we taught the Gemini Gem, how do we give feedback?
00:19:36 You’re the type of writer that does this. I see that there’s very clear ways that there’s try one, try two, try three that we’ve built into our K to eight writing feedback system. And then the teachers-
Jon Krohn: 00:19:53 Sorry, just to clarify a little bit to make sure I understand. So first of all, for people who don’t use Gemini regularly, a Gem is analogous to a project in ChatGPT or in Claude, where you can pre-configure specific context and examples of outputs that you’d like to have. And so this gives you a powerful way to have repeatable and standardized outcomes. So it sounds like this … So when you talk about pre-assessment, that could be … I guess it could be in any subject, but let’s say writing, you could have a pre-assessment of a writing sample of a child, and that goes into the gem without any PII, personally identifiable information. And then based on all of the context and the way that you structured your prompts in that Gemini gem, it comes out with then three separate layers of feedback to try to … That’s kind of my understanding of what you’ve been saying to … And those three layers give kind of three chances for the AI system to be providing feedback that will move the needle on a post assessment.
Traci W.: 00:21:02 So that’s … It’s somewhat right. I think part of what we’re … Well, I mean,
Jon Krohn: 00:21:08 Obviously- That’s why I’m asking.
Traci W.: 00:21:09 Yeah. No, it’s really great because I think part of why we’re so excited to talk to you is that we’re moving at such a break next beat. It’s really important for us to slow down and kind of talk through how does this actually happen, right? And so, as we think about what goes in and what comes out, it goes back to we have to train the gem or we have to train Claude. It’s a different genre. So it could be that they’re doing narrative fiction writing, it could be a nonfiction, they’re doing a literary essay that we’re saying to the Claude or the Gem, we upload the standards, here’s how we want feedback to be given. And clearly there’s the … Obviously if kids are not writing in complete sentences, and we talk to kids about this, right? How do you know what feedback is working for you?
00:22:13 But in the lower grades, the teachers are using the gems to also see patterns and trends so that they can then create groups from that data to accelerate around certain standards that are showing up in their writing. And when you’re meeting with a collaborative team, that data is going to showcase that … So in Principal Traci ‘s class, the kids really got that conclusion paragraph and they hit it. And in the three other first grade classrooms, that wasn’t there. So let’s talk to Traci about what she did, and that’s done in like five to 10 minutes rather than … I mean, I just remember sitting in a … We’re passing papers, we’re looking at the rubric, we’re thinking and we’re talking through it. AI gives it the first pass and then the human interaction and the human touch really goes deeper, which leads to more impactful conversations for our teachers.
Jon Krohn: 00:23:19 I see. Yeah. So this is … So for the younger grades at least, the AI tools are there to help teachers be better teachers, to think about what they can be doing to move the needle on from the pre-assessment to the post-assessment.
Traci W.: 00:23:32 Yep, totally.
Jon Krohn: 00:23:33 Cool. All right. So now let’s talk about the older kids who interact with the AI systems directly. So I think it’s grade, it’s five through eight. And it’s funny how I come from Canada, so we say grade five to eight, but you say fifth grade to eighth grade. It should
Traci W.: 00:23:46 Be pretty- I can say it always, Jon. Whatever we can do. So the fifth through eighth grade this year is amazing. I mean, I will tell you, I was in a classroom yesterday, we had a district visit, so the chief academic officer, the director of digital learning for Boston Public Schools and the deputy chief of academic learning, they visited and the director of digital learning hadn’t been here for a year. So it was amazing because last year, as we talked about, we were just launching the AI co-lab, we had some ideas, we were flushing out, we were really excited. And this year they got to see the student facing AI. And so the students, and I’ll just give you a real example because I love a good story. So I walk with these amazing people who are so excited to see the best school in Boston, and we walk into a classroom and the kids have a … There’s like a question that they’re exploring, and the question is, how can AI make us a better test taker?
00:25:03 So to give you context, it’s March, we’re getting ready for our Super Bowl, which is the state test, and we’re in a repertoire unit. We’re not in a test taking unit, we’re in a repertoire unit because in our repertoire, we have all of these skills that we can show what we know in this test. And so because we’ve been using AI all year, we thought what better way is for students to have an opportunity to think about AI. And so we have student versus AI and we’ve used … So Magic School is another opportunity for us to continue to use other systems and Magic School because it’s part of our suite in the Boston Public Schools. And so teachers in the ELA humanities team work together to create this … I call it complicated because I think it was … I was not understanding it until my fifth grade friend was, Principal Traci , here’s how it works.
00:26:01 He’s got like four tabs open on his computer. They’ve uploaded these practice short pieces of text. There’s the extended response, which is the writing responses that we’ve really … Year over year from like 2007, it has been a problem of practice for our students to be able to explain their thinking clearly in writing. So this critical thinking and analytical writing. And so there’s open response, there’s multiple choice, and then there’s the notebook is next to the computer, and the kids are doing what they call chunking. So they’re writing down what they’re reading and chunking it. Then they’re answering a question in magic school. And I said to the student, I said,” How is AI even helping you? “And he said,” Well, let me tell you, I’m writing my response, then I’m letting AI write a response, and then I’m comparing the two to see who’s better.
00:27:02 “And then I have to explain if I think mine is better or if I think the AI is better. So the idea of reflection right here is, I mean, that’s mission critical to thinking about thinking and accelerating outcomes. And then in addition to that, I said,” Well, what about the multiple choice? “So he’s like, ” Oh, I got this one wrong. “And I said,” Well, why do you think he got it wrong? “And he said,” Why do you think he got it wrong? “He goes,” Well, this was the almost right answer, but this was the right answer. “And I said,” Okay, so what do you do with that information? “He said,” Well, actually, it goes right into this tracker so Ms. Duggan, my teacher, can see if there’s a pattern that might emerge. “And I’m like, ” A pattern that might emerge. What does that mean?
00:27:49 “I mean, just the conversation. So I’m with the CEO, she’s looking at me, I’m looking at her, and I’m saying,” This is fifth grade. “So in four years, this child will be in high school, and this opportunity for this child to take this learning … There’s a hundred kids in fifth grade all having the same experience with this interface with Magic School, which was teacher created and collaborated on. And now the student is having an opportunity that this is not the first opportunity that the student is working with AI. This is like a culmination, and that’s why we’re talking about a repertoire. This student is driving his learning. Another student is getting feedback based on what they need, and it’s personalized, but it’s related to we’re not lowering the bar, we’re raising the bar. AI raised the floor. Teachers are raising the ceiling. What’s better than that?
Jon Krohn: 00:28:50 That is sensational. I love how the older kids are able to figure out their own way based on the way that they learn, based on the places that they feel like they need work or the AI system says they need work. They figure out their own way to be working between these different applications, Gemini, Magic School, a notebook. And so what is Magic School exactly? Actually, it came up in our research as well, but I didn’t dig into it very much.
Traci W.: 00:29:17 So it’s another platform, and it’s built for education to be school safe. And as I said, we try to stay platform informed, not platform loyal. We want to make sure that PlayLab, which is another amazing … And it’s powered by Claude in the backend. We’re using PlayLab personally, I’ve done it with some principals. I created a budget tool for Play-Lab that Magic School allows teachers to prompt and control that interface. So we didn’t just start with Magic School. I want to be clear. The Because I think right out of the box doesn’t help the learning that teachers need to do and the AI literacy for both the adult learners and the student learners. That’s at the foundation. We need to know what we don’t know and give students opportunities to think about AI and AI ethicality and create … Our students are critical consumers of everything in their world.
00:30:28 AI is like number one. We need to make sure from the youngest age we’re thinking about AI literacy.
Jon Krohn: 00:30:35 I love this. This is really exciting. For people who are trying to figure out what they should be doing with their kids at home, or maybe hopefully we have other educators listening to today’s episode and figuring out what they can be doing in their schools. Is it the consensus of the research out there, as well as your maybe personal experience, that this bifurcation of having younger kids not be working directly with the AI because maybe they anthropomorphize too much, whereas the older kids do get that direct access, that bifurcation, that’s basically like that would be a recommendation that you have for everyone at this time, right?
Traci W.: 00:31:14 Well, so that’s a really interesting question. It’s similar to the question, should three year olds be in school all day? We believe yes. So if this is where we are, thinking about it in our timeline that I see two year olds with iPhones in their hand. So they are actually interfacing with AI, whether we want to think about that. I think in schools we need to be thinking about the ways we talk about AI. And I mean, 19 years ago, I mean, I didn’t even have an iPhone. That was about 19 years ago. I had my little Blackberry. So that we know that the world is shifting at such a breakneck speed. We need to create critical consumers for our families. And what’s the family education look like at Elliot is we’re still building that out. So even that old, it’s not old, but it’s like the social dilemma and it’s now showing up on a streaming service.
00:32:19 And one of our eighth grade teachers was like, “Should we show that to our seventh and eighth grade families?” Just to retool this idea of how much is too much exposure. So just like calculators, when you think about people are like, “You can’t use a calculator in math.” Well, would we say that now?
00:32:42 So I think we should be always thinking about, for families especially, what is your child interacting with at home? Kind of do a, I don’t know, a survey at your house. And we always say reading is thinking. So please make sure there are like hardcover books and paper books. And if you are letting your student or child use the computer, what are they using? It’s about decisions too, because the decisions you make as a family at home impact how they interact at school too. And let’s go back to the infinity, right? Everything in and out, it keeps connecting back.
Jon Krohn: 00:33:27 The physical books thing is interesting because I find it so much easier to stay focused and just to comprehend what I’m reading if I’m leaving through a physical book, as opposed to having something on say a tablet. Is that something that … Yeah, I guess this is like a research backed thing that we should be having kids working with physical books. And what’s interesting about that is obviously we’re not talking about university education or high school education really very much in this episode, but a lot of the big educational publishers have moved to digital only for the textbooks that they provide. I went recently to, I did my undergrad at a small university in Ontario called Wilford Laurier and I had, when I was home, because my family still lives in that area, when I was home over the Christmas break, this most recent year, the alumni office took me on a tour of the campus and it ended at the bookstore.
00:34:31 And the bookstore when I was there was full of books and now there are no books. There was one kind of book. They still have anatomy textbooks because I guess somehow that like second year anatomy, there’s like hundreds of things on a page and it’s probably just too unwieldy to get into a digital format and have that work. But other than that, they had a couple like touristy kind of books about the university or about the region, but that bookstore that was once full of books was now full of sweaters and key chains and that kind of stuff. And yeah, I don’t know. It seems weird to me. I wonder if I could have succeeded the way I did as an undergraduate student, if all the learning that I had to do was on a computer instead of out of books.
Traci W.: 00:35:25 You’re speaking my love language and I think it’s an interesting thought. Obviously as a school, we do believe so deeply in … If you walked into my office, there’s books everywhere. We love reading. And for our kids, we send book bags home with kids. We have bags and books and we also have kids reading online and kids can bring in a Kindle if that’s the way that they feel inspired to read, because at the end of the day, we have all that research that says kids need to read and they have to read at least 30 minutes at home and they have to read at least 30 minutes during school, at least to accelerate. And so we go to the library here. I love bookstores. They’re like few and far between, but we do have a little one in the north end, which makes my heart sing that, both my kids went to the Elliot, my youngest, she’s like, “I’ll meet you at the bookstore after school, mom.” And she’s like 24 years old.
00:36:37 So this love of reading also inspires a love of learning because we know reading is thinking. And then simultaneously in the upper school where the kids want to … Everybody has a Chromebook at the Elliot and in Boston. There was a clear commitment to ensuring technology, especially during COVID. But I would say BC, before COVID, there was always this idea that we need to make sure that kids have access to high quality literature that reflects not only them and their identities, but gives them a window to the world. And that’s what books do. And if we just say you have to go, you can only use the computer, we need to have multiple modes because every learner is different. So I just, that makes me sad that bookstores are on the out, but there are hundreds of thousands of people that believe so deeply in the actual pen to paper, which we also do.
00:37:45 We have Google Classrooms and we have the old black composition notebooks where kids are capturing and keeping a repository of their thinking in notebooks as well.
Jon Krohn: 00:37:57 Right. That makes a lot of sense to me that we have this blend of approaches and I could see that working. There are some advantages, of course, to digital learning. Resources can be updated in real time. You can search for terms more rapidly. You can have just way more content at a much lower price in a digital format. So there are some advantages, but yeah, there’s a … I don’t know if I’ve said this on air before, but something that I’ve definitely been saying to people over the years, my fantasy day is to be able to be in … So at that same undergrad university that I went to, I just had like a simple flip phone at that time, could really only make phone calls on it. You could text, but you had to get each character, you had to press the number a certain number of times.
00:38:47 So it was pretty tedious. You wrote really short texts, see you with just the letters C in you. And I could go into a study room and I have the specific study room in mind that was called the Fishbowl because it had windows on all sides. And so you could go into the Fishbowl, you get a desk to yourself, nobody had a laptop in that space. We had computer labs that you would go into when you were typing up a report or whatever. But in these study areas, people didn’t have laptops, they had textbooks out and notebooks. And I specifically, one of the most enjoyable textbooks for me personally was calculus for some reason. As you work through those calculus pages and your mind opens to some completely new concept that just a few pages ago would have been impossible and how that journey through those hundreds of pages through the calculus book, if you were to start at the back, it’s impossible and it looks like nonsense.
00:39:51 But if you work through it from page one through page 400 or whatever, by the time you get to the end there, all of that, what would be nonsense makes sense. And that is, I don’t know, it’s such a cool experience that to be able to sit there for two hours uninterrupted by emails or phone calls and to just be able to work on calculus problems, that’s like my deepest, darkest fantasy.
Traci W.: 00:40:13 I mean, I was right with you. I’m thinking about the fishbowl and the ways … I mean, I’m in an office right now. I’ve never sat this long in a space without interacting with others. And at the same time, it’s just really important for us to create those spaces for our students. So I really, I appreciate you sharing that with us.
Jon Krohn: 00:40:35 Yeah. So anyway, and I ended up talking a bit more about myself than I need to on a podcast that’s really about you and the kids. But prior to recording this episode, you sent me an email where you talked about there being three distinct failure modes as you were developing these AI systems that required complete prompt redesigns. And so it seems to me like for any of our listeners, whether they’re parents or teachers or whatever, that are trying to figure out ways that they can be leveraging AI tools to help themselves or help kids learn, what were those failure modes that you encountered and how did you fix them?
Traci W.: 00:41:11 So that’s great. We’re always in failure mode, and I think that’s the mantra of failing forward. The first one definitely was this idea of generic feedback. If we didn’t actually give Claude or Gem the correct way in which we were thinking about giving feedback, it was just so generic, like, “Great job.” And that was the second failure, which is too much encouragement, like, “Great job, great use of this. ” Well, that’s not feedback. That’s just something else that has two letters that I can’t use on a podcast. And we knew that even the teachers, you can’t trust it, right? So this idea of what we want students to be critical consumers, we wanted our teachers to be critical consumers to not just throw something in and send it to a child. So this idea of the generic feedback, then the next thing was too much, good job, good job, pat on the back.
00:42:18 And then the last thing is over trust. And so how we worked through it is we had to use this as an actionable way to train the AI. And as we continue to train the AI, I think it was a missed opportunity for us to capture all the ways in which we were prompting. So that fail forward was great, except for teachers were doing it in different spaces, although we had like a coordinated Claude and we had projects in the Claude, we had a shared Claude account with Claude with multiple users. We weren’t always like connecting all of the work together. So one of our lead teachers and one of our other fifth grade teachers were like talking at night and saying, “Oh, I tried this, I tried that. ” And then it wasn’t in a repository that we could actually then learn from because maybe we didn’t meet that week or we were meeting … We were meeting weekly, but also trying to document all of that work.
00:43:21 So it’s been, for us, the journey and the challenges of the journey have made the journey that much more impactful for both the students and the teachers.
Jon Krohn: 00:43:36 Thank you for sharing those and yeah, provides us with some useful context on the ways that we can be developing AI systems ourselves. A particular group that you mentioned earlier in the episode, well, you talked about two groups earlier in the episode that I want to kind of highlight here. You talked about multilingual learners and students with disabilities as being particular categories of students that can leverage this AI world positively. And so tell us about how AI can be a differentiator for those groups.
Traci W.: 00:44:11 So I think for us, we talked about AI as being part of a solution. It is not the solution, it is one of the tools. We always start with a problem of practice, right? So we know from our data, there were groups of students in different grade levels, whether it was a student that was a multilingual learner or a multilingual learner and a student with disability, or just a student with a disability, that clearly there was an opportunity for us to think about the data that was coming out, that we could adjust the feedback so that the teacher could prompt and say, “This is the writing from a student who is on this Lexile level, which is a reading level for the non-educators in your audience, that if they’re reading at a … So say the student is in fifth grade, their reading level is in third grade.” The standard is the same, but you want to be able to provide actionable feedback for this particular student.
00:45:16 You can prompt Claude or Prompt Gemini, whatever the LLM you’re using, so that that student is receiving that feedback in a way that’s actionable at their reading level. And then that’s where we were seeing that prompt engineering and creating the under the hood, I guess, the architect part of the creating the response was it was clear that we needed to make sure that everything was in there so students could move based on that feedback. And it’s all connected to data, Jon, because if you’re doing something and you’re not collecting data and you can’t show the growth, you can’t abandon something because you keep thinking to yourself, and there’s like the, we’re over prompting ourselves saying, “Good job,” even though it’s not a good job because you’re not actually accelerating the learning and AI was a differentiator in that way and then teachers are coming together when they see this data and this one teacher was seeing a lot of gains.
00:46:19 So then the teachers were like, “Okay, how did you prompt? What was the way you did that that got us to there?” And it’s all connected.
Jon Krohn: 00:46:30 You’re talking there about data and the importance of tracking data effectively from some written conversation that you and I had also before recording this episode, something that came up are the kinds of metrics that you’re tracking, which I think are interesting. So in the data science world, the kinds of metrics when we’re thinking about evaluating a model or process, we talk about these things like accuracy, precision, recall. The metrics that you’re measuring include things like writing growth, teacher time saved, student metacognition, and ethical understanding of AI. Can you walk us through some of those measurements?
Traci W.: 00:47:06 We sure can, and we can also be super honest that this is an area that we continue to grow with and iterate on. I think for us, capturing a data system, and as I said when we started in Denver, we had an idea and created a Google sheet and started to think about, we’re doing something, we have to collect data and so that the actual writing growth data was really important, setting goals for students. And then the piece around, there’s an assessment called the SAO, Student Assessment of Youth Outcomes. And so teachers are being surveyed. Surveys are hard and they’re important at the same time. So thinking about triangulating all of those data sources is something that we continue to try to figure out how to ensure that we’re not just spending all of our time assessing and also not losing the opportunity to find ways to document and in real time say, “The data is showing this, this, and this.
00:48:22 ” I’m trying to think, what was the other?
Jon Krohn: 00:48:25 Student metacognition, ethical understanding of AI. Is that what you’re …
Traci W.: 00:48:31 Yes. So thinking … We always say thinking about thinking. So how do we track thinking about thinking? And we have heads of school and we’re thinking about the students that when we go … When I was telling you about going in, I go and visit classrooms all day. Kids will be wondering where I am today for an hour and a half, because usually I visit probably at least 20 classrooms a day across the three buildings. I get a lot of steps and this idea of validating and valuing students being able to share their thinking and be able to explain their work and their reflections so that we have a repository, we have a Google classroom. So when we’re thinking about the reflections, that it’s built into the units of study, that there’s portfolios, and our next level of work is ensuring that year over year, there’s a way to capture that learning, because we know what we’re doing is absolutely amazing, and it has to be replicable and scalable.
00:49:39 Those two pieces, like we can continue to do this work and move the needle, and we need to bring everybody else along because this is not just here, this is we’re talking about access and equity and closing opportunity and wealth gaps, not opening them and widening them.
Jon Krohn: 00:49:57 Some of those metrics like writing growth, it’s kind of easy to understand, you’d have a pre and post evaluation on writing quality, teacher time saved, requires some logging of teacher time, but you can imagine that’s quantifiable. For something like student metacognition, is that something that is converted into some kind of quantity that can be tracked over time?
Traci W.: 00:50:22 We think about this all the time. We’ve always had this problem. So probably in your audience, this could be another startup around capturing data around student metacognition. And I imagine maybe there is something out there. We’re also thinking about those 21st century skills, the soft skills that we know are mission critical in the workforce, and we always keep talking about the future of learning connected to the future of work. And when we have a portrait of a graduate and we have a … Right before COVID, we were launching into deeper learning and thinking about how all of our units should be considering problems to be solved and the ways in which students have access to think about their lives beyond the fourth grade classroom they’re sitting in and giving them access to people from different walks of life and in their professional lives. And it’s just so hard to measure.
00:51:26 And then there’s the ethical considerations around how are we tracking this? How are we thinking about getting this information and sharing it? So a lot to think about and we’ll keep thinking about our thinking.
Jon Krohn: 00:51:41 Yeah. It’s a fast moving space in terms of AI capability. And so this kind of this idea of fail forward and experimenting with different things, tracking what data you can, it seems like the best that you could possibly be doing in this scenario. You are retiring in June. I don’t know if this is … Are we announcing that on air? Is this breaking news?
Traci W.: 00:52:05 I mean, you just announced it, Jon, so I announced it in the end of September. I started socializing this idea and creating space and opportunity for the next generation of leaders to continue this amazing work at the Elliot and beyond.
Jon Krohn: 00:52:22 So how do you build an AI program like this where we’ve been talking about how flexible it needs to be and adaptable. It seems like a big part of this working. I mean, obviously all of the teachers there need to be on board with making something like this work, but you’ve been a big piece of it. How do you build an AI program so that it can continue to grow once you’ve retired?
Traci W.: 00:52:44 So I think it’s more than just an AI program. I think it’s about what’s the foundation in our public schools that encourage and invite innovation to the forefront of accelerating student outcomes. So for the past 19 years, we went from a small single building school to two buildings to three buildings. And in that time, the foundation of the work that we do and the culture of collaboration was critical to our success. So for anyone thinking about AI work, they also need to look under their hood, right? They need to look at how their school is organized for learning and leading and failing forward and doing some type of kind of survey of their own learning organization and then thinking about AI is not going to replace the humans. It should amplify the work that you are doing and the work of improving student outcomes is why we are in education and in service of children.
00:54:01 And there’s so much that goes into the structure and systems that need to be in schools, that learning, both the adult learning culture and the student learning culture is equally important. I don’t know if I said this to you, but if you don’t feed the teachers, they eat the kids, right? You’ve
Jon Krohn: 00:54:22 Definitely not
Traci W.: 00:54:24 Said
Jon Krohn: 00:54:24 That to me. It’s a
Traci W.: 00:54:24 Good club. It’s intuitive, right? Counterintuitive in the sense. If teachers don’t get the professional learning diet they need, how can they accelerate the learning of their students? And so, the big picture is how do we, as a country, think about schooling and public schools, because the transformation that can happen in a school is … I mean, I can speak to it because I’ve been here 19 years, I’ve been in one district for 34 years. At the Elliot, this is the foundation of innovation. Whether you’re an innovation school and we’re an autonomous school, so we have some more flexibility around certain autonomies around curriculum and instruction. At the end of the day, when we started at the Elliot, we were a traditional public school and we still had the same mindset, which was we’re all learning this. Whatever we’re learning, we’re learning it together.
Jon Krohn: 00:55:29 A report from Brookings from January of this year concluded that the risks of gen AI in children’s education currently outweigh the benefits. You’ve been doing this now for a while and are seeing great benefits. What the heck happened in that report? What are they getting wrong?
Traci W.: 00:55:47 So that’s a really important report because everybody’s reading it. And when you think about the world and where the world is around using AI, I mean, you’ve probably seen that graphic came out a couple, I don’t know, a couple weeks ago or something. People send me so much because they know I’m all on about AI at this moment, but I’m not just all on about AI. I’m all on about being prepared for the future. And so we’re not taking children and just flopping them in front of a computer with an AI driven … There is the human interaction that is happening all the time. And so maybe what I took out of Brookings was don’t deploy, don’t deploy. And I’m saying deploy carefully, deploy intentionally and be thoughtful and intentional about how you’re thinking about the work. And in most of the work, I often talk about this with other leaders as well.
00:56:49 I have this one, three, five, right? One day, three days, five days, one month, three months, five months, one year, three years, five years. So in five years from now, when I’m looking back and listening to this podcast, and I now have my own podcast show, and Jon, you’ve helped me start it, and we’re doing some work together. I’m living in that utopian world at the moment. I’m looking back and saying, “Do you remember five years ago when we were talking about that Brookings report?” The schools that deployed carefully and thoughtfully, look at where they are, because that’s another metric. We need to be looking at schools that are doing this work really well and have data to show that. And then we’re sharing that broadly, not just in one district or in the commonwealth of Massachusetts or in the West Coast or the East Coast.
00:57:40 We’re finding a way to share practice that is going to raise the level so that the United States in PISA is number one, right? We don’t want to be in the middle of the pack. We want to be at the top. And in order to do that, we have to find a coherent, consistent way to share this work in public spaces, in public education. I presented at the AI show in April, and even since then, how much has changed.
00:58:11 The social emotional learning that has to happen in order for AI to be even used, we believe that the social emotional learning is at the foundation of anything you do in a school, and we have to share this and not think that, “Oh, it’s just this school that can do it, ” or because they have this crazy principal who thinks like this. It’s not about that. It’s about committing to public education and changing the world. That’s it. It’s very simple, I think.
Jon Krohn: 00:58:41 It does seem to me like you talk about going up the piece of rankings, so PISA, I believe it’s spelled their rankings of education across the world. You can compare country by country, and it does seem to me like something like AI adoption could make a big impact and get the US up those rankings because the US does lead the OECD in things like bandwidth, connectivity, early adoption of technologies, access to AI, and obviously all of the world’s leading frontier labs are in the US. So it’s an interesting possibility there. For our listeners, wherever they are in the world who are listening to this episode, and they want to be improving their own education or particularly the education of their kids, what can they practically be doing? Do you have guidance? So in some previous episodes, like episode number 975, which we had recently, the guest in that show, Zack Kass was talking about private schools that are in the US and starting to proliferate around AI education.
00:59:52 So things like Alpha School was one that he particularly highlighted, but not everyone is going to live … Near one of those schools, not everyone is going to be able to be in the position that they can put their kid in a private school. What do you recommend to, I guess, public school systems, to parents, to teachers? What can they be doing today to be taking advantage of the benefits of AI?
Traci W.: 01:00:19 That’s a really great question. I think I’m currently serving on a task force through the Department of Elementary and Secondary Ed in Massachusetts. And so there are a lot of different organizations that are thinking about this. And I think anyone that’s … There’s so many resources out there that we need people to really look at the resources. And I mean, I overuse this, but critical consumers of any resource that don’t expect AI to solve a problem. You actually have to understand what the problem to be solved as a district or a school community. Currently in Boston Public Schools, we’re really excited about kind of thinking about the guidelines around AI and an AI policy. And either one is never going to be stagnant. So it’s no longer that you have one acceptable use policy that we had when I started in 1992 with kids going onto the internet.
01:01:23 But there is a clear commitment from many, or if not all, public school districts, looking closely at policies and guidelines and procedures. It’s really important that you slow down and move fast at the same time. That’s that leadership on the line. That’s one of my favorite books too. That this is important that as an organization that has leaders, superintendents, or even principals, that there is an ongoing conversation about the ways in which we can put AI into our practices. And we’re thinking about AI enabled practices, not just an AI practice. It has to have a clear connection to a problem to be solved.
Jon Krohn: 01:02:13 Nice. I like that. Clear guidance. Obviously, there’s no one size fits all answer, and that’s kind of your main point, is that it’s an ongoing conversation, but being able to track things and know that things are improving, sounds like a key part of your approach throughout. With your retirement coming up, are you planning on spreading the good word of AI?
Traci W.: 01:02:36 Yes, indeed I am. I’m really excited to continue to support schools, support districts. Specifically, I’m a lifelong Bostonian, so I’ll stay continuing to work with the Elliot School and Boston Public Schools. And I literally was texting with the superintendent saying I was going to be on Jon Crone’s podcast and she was super excited.This is lifelong work and I am a lifelong educator and Bostonian. And I want to see this go beyond. There clearly is a need across cities, suburban towns and rural areas in our country, from the Northeast to the Northwest, to the Southeast to the Southwest, that schools need opportunities to think about this work, not just about AI, but public schooling, because this is our workforce. And I’m all in, Jon. So if your listeners want to get in touch or they want to come and see the work at the Elliot School and want to talk to our friends, the fifth and sixth graders who in 10 years from now will be running our world, definitely reach out to us.
01:03:52 I know that you’ll have all my information. My LinkedIn will be live. And I’m so excited and grateful for this opportunity to share where we are right now and knowing that we’re always going to be getting better and learning.
Jon Krohn: 01:04:11 Yeah. Exciting times ahead. Exciting times ahead indeed. And I’m a big techno optimist. It seems to me like the having abundant, affordable intelligence all over the place accessible to us. There’s so much possibility for positive impact. And yeah, education is one of those big places. So thank you so much, principal, Traci Walker-Griffith for taking the time in today’s show. The very last thing before I let you go, you’ve already kind of given us how we should be continuing to stay in touch with you after the show. But my final question for you, which is perfect, given the love of books that we both been expressing in today’s episode, is what book recommendation do you have for our listeners today?
Traci W.: 01:05:00 So that’s so great. I mean, I love books and I don’t know if I, in earlier in the episode, there’s out of HBS, Linda Hill, she has a new book that it’s actually being shipped to me as we speak. It has not arrived, but it’s called Genius at Scale and it continues to embrace this idea of collective genius and cultivating genius that … I mean, I’m not the genius. The students are the genius, the teachers are the genius here. And so I’m looking forward to thinking about leadership. And this is a book that if you’re a leader in any organization, it’s thinking about genius at scale, how great leaders drive innovation. And I mean, we can read it together and maybe you can check it out and offline, we can think about this. This is a big responsibility that we are embracing as a school and as a country and in our world that we need to have people thinking it’s not just me, it’s us, right?
01:06:06 There’s no I in team, right? And I do believe that. And this is one of the books that I’m hoping that’s going to continue to grow me and my thinking as a leader. So I hope you like the book. I’m looking forward to reading it.
Jon Krohn: 01:06:21 Sensational, Traci . Thank you so much for taking the time, making the time to hear that you’ve never sat in all of your years of teaching from starting in 1992 to retiring this coming summer, that this is the first time that you’ve sat still for 90 minutes in an office. Thank you for doing that for us. We really appreciate it. What a special episode. And yeah, looking forward to checking in in the future again and seeing how the AI education journey is unfolding.
Traci W.: 01:06:51 Thanks, Jon. And I hope maybe Michael Walker Jr. And I and you can meet at a beach someday soon.
Jon Krohn: 01:06:56 That sounds great. Looking forward to it.
Traci W.: 01:06:58 Thanks, Jon.
Jon Krohn: 01:07:02 Such a valuable and long overdue episode from Traci Walker-Griffith today. In it, she covered how she transformed the Elliot School from an underperforming school on the closure list into the highest performing school in Boston. She talked about how kids as young as four at the Elliot work with robots and coding tools like Kibo and Scratch Junior, learning that the quality of their input determines the quality of their output, garbage in, garbage out. She talked about how for younger students in kindergarten through fourth grade, teachers use AI behind the scenes, scanning student writing, feeding it into custom Gemini gems with rubrics, and using the AI generated feedback to identify patterns and form targeted small groups. She talked about then how students in grades five through eight interact with AI directly, comparing their own writing responses to AI generated ones and reflecting on which is better and why, building metacognition and critical thinking.
01:07:54 And Traci ‘s provided her guidance for schools considering AI. Don’t expect AI to solve a problem you haven’t clearly defined. Invest deeply in teacher professional learning, deploy carefully and intentionally, and always connect what you’re doing back to data. All right, that’s it. As always, you can get all the show notes, including the transcript for this episode, the video recording, any materials mentioned on the show, the URLs for Traci ‘s social media profiles, as well as my own, at superdatascience.com/983. Thanks to everyone on the SuperDataScience Podcast team, our podcast manager, Sonja Brajovic, media editor, Mario Pombo, our partnerships team Natalie Ziajski, our researcher, Serg Masís writer, Dr. Zara Karschay, and our founder Kirill Eremenko, for producing another excellent, another educational episode for us today. For enabling that super team to create this free podcast for you, we are deeply grateful to our sponsors. You can support the show by checking out our sponsor’s links, which are in the show notes.
01:08:48 And if you yourself are ever interested in sponsoring an episode, you can get the details on how by making your way to JonKrohn.com/podcast. Otherwise, share this episode with someone who would also benefit from Principal Traci ‘s perspective on children’s education with AI. Review this episode on your favorite podcasting app or on YouTube. Subscribe if you’re not already a subscriber, but most importantly, I hope you’ll just keep on tuning in. I’m so grateful to have you listening, and I hope I can continue to make episodes you’d love for years and years to come. Till next time, keep on rocking it out there, and I’m looking forward to enjoying another round of the Super Data Science Podcast with you very soon.
