Game-Based Learning and Assessment, with Rebecca Kantar

 In this episode, we are joined by Rebecca Kantar, Vice President of Education at Roblox. Rebecca shares her experiences and insights into the world of game-based and simulation-based learning and assessment. She digs into the benefits and limitations of these approaches as well as the exciting opportunities she sees for the future. Visit AVID Open Access to learn more.

Transcript

#266 — Game-Based Learning and Assessment with Rebecca Kantar

AVID Open Access
46 min

Keywords

roblox, game, assessment, students, folks, test, roblox studio, learning, experiences, simulation, create, work, build, process, skills, assessments, educators, thinking, company, environment

Speakers

Speaker 1 (43%), Rebecca (26%), Paul (14%), Winston (9%), Rena (6%), Transition (1%)

Rebecca Kantar  0:00  

Students do have the opportunity to demonstrate authentic problem-solving, akin to what they'd face on the job or in school. No matter the context you choose to situate your game or your simulation within, I want to watch you solve problems. I want to watch you understand systems. I want to watch you create and innovate based on what you see happening in the environment and the data you get back.

Winston Benjamin  0:26  

The topic for today's podcast is Game-Based Learning and Assessment, with Rebecca Kantar. Unpacking Education is brought to by avid.org. AVID believes in seeing the potential of every student. To learn more about AVID, visit their website at avid.org.

Rena Clark  0:48  

Welcome to Unpacking Education, the podcast where we explore current issues and best practices in education. I'm Rena Clark.

Paul Beckermann  0:59  

I'm Paul Beckermann.

Winston Benjamin  1:00  

And I'm Winston Benjamin. We are educators.

Paul Beckermann  1:04  

And we're here to share insights and actionable strategies.

Transition Music  1:09  

Education is our passport to the future. 

Winston Benjamin  1:12  

Our quote for today is from Dr. Robert Sternberg, former president of American Psychological Association, APA, and current professor at Cornell University. He says, "So our big mistake is thinking that people who are good at solving multiple-choice problems that are fairly trivial, that have a unique correct answer, and that measure things that are never going to matter again in your whole life, that that is going to be predictive of who is going to be able to handle really tough, unstructured, ambiguous, real-world problems."

Paul Beckermann  1:59  

That might be the new record for the longest quote.

Winston Benjamin  2:04  

It was fun, though. What do y'all think? Rena? Paul? What are you thinking about that? 

Rena Clark  2:09  

First of all, this makes me feel good, because my husband is so good at trivia. And I always feel really not the smartest around him. And I'm just confirming that that's not true. But really thinking about how these traditional types of multi-choice assessments, they don't take into account just as I was saying about diverse styles of learning, of our students, or humans as a whole. And as pointed out, these types of assessments also don't prepare students for life beyond the classroom, where real world problems are messy. There's lots of multiple pathways to solve these problems. And you have to be creative in your thinking. There's no one right answer. So how do we shift our focus towards those types of assessments away from these multi-choice, one singular answer type of assessment? 

Paul Beckermann  3:08  

Well, you know, my big mantra I've said it before is, what's the most authentic thing somebody can do with this information, this content? Let's assess them that way. Because multiple choice questions are kind of to make the assessment process more efficient, more practical, and more mass producing all these people through the system. You can drill it down to a number and make it really efficient. So I guess the question is, how do we make it authentic and efficient? And you know, hopefully, some new technologies will be on the horizon here, and maybe somewhere in development now that we can maybe get to some of those solutions in the future.

Winston Benjamin  3:47  

I appreciate you said efficiency, because efficiency, for me, talks about the person grading, not the actual learner. So today, I'm so glad that we're going to jump into a topic about how do you assess and how to use different forms of assessment with our guest, Rebecca Kantar. You're the Vice President of Education at Roblox. I wanted to say that my niece is excited about Roblox. She always wants me to play with her. But I didn't want to say that until we got an introduction. Hello, welcome to our podcast. How are you?

Rebecca Kantar  4:21  

Yeah, thank you so much. It's great to be here. And I'm very well. I'm looking forward to this conversation. I spent many, many years thinking about everything in that opening quotation. So thanks for bringing that to start the conversation today.

Winston Benjamin  4:33  

I appreciate that. Thank you. Just to help get our guests to understand you, understand your expertise, do you mind giving us a little bit of a background about how you got to this point in your career? 

Rebecca Kantar  4:47  

Yeah, happy to. So my background started as an entrepreneur really during college. I wasn't loving college, to be honest. I was finding the work repetitive, monotonous. A lot of what I felt high school, when I was drowning in AP courses, was like, actually. And I found myself, very fortunately, during high school in my early college time, being asked to work in a variety of nonprofit and kind of social-impact projects, some centered around child sex trafficking in the United States, so very heavy subject material. And I loved the complexity and the authenticity, to borrow your words, Paul, of the challenges that I was confronting in that work. And I eventually felt like that nature of work, where I was constantly thrown into different types of problem-solving situations, where I was having some traceable impact, where I was constantly pushed to demonstrate and try new sets of skills, that's what I was drawn to. And my classwork was no longer providing that. So I took some time off from school. I began my first company that was connecting some of the outstanding young people I met throughout my social impact work with Fortune 100 companies, who I saw as having maybe a deficit of some of these long-term, social impact-oriented folks around the table earlier rather than later in their careers. And I saw an opportunity to connect those folks. And so I created what's called an expert network that was eventually acquired by a firm called Gerson Lehrman Group. And once I got to GLG, I was very fortunate to have a group of executives and colleagues there, who were very interested in education. And I had started reflecting a lot on why it was that I didn't want to go back to school, and what it was about my extracurricular activities that was so compelling, and what it would have been like had high school and college embraced, instead of resisted, allowing my participation and that kind of project-based, real-world-centric type of learning that I created for myself with my peers, but wasn't able to enjoy as the primary mode of exploring all different subject areas. And, therefore, I kind of became deeply interested in the default settings of most systems. I was so fortunate. I grew up in a very privileged public district outside of Boston. And I recognized that I had, you know, really superb education. I might not have liked every minute of it, but no doubt, I learned how to think critically. I learned how to read. I learned how to write. I never struggled with testing. I never really thought all that much about our state standardized tests, for example. It was kind of a given that I would perform fine on our state standardized tests. And I had a moment to myself once I was at GLG, to just reflect on that reality and how lucky I was, and to think about what would it take for our system, on the whole, to raise the collective floor for all students, but also to make sure that students felt very connected to the nature of work and opportunity they were going to face after school, whether they went on to college or not, after high school, such that they'd feel really equipped for adulthood. 

And I wrestled with what some of the default settings in our public school system were like that kept kids perhaps not feeling that way, at least a lot of the young people I'd gotten to know over the many years in kind of more entrepreneurial circles. And one of the recurring themes that you realize has a lot of downward-shaping effect on what and how we teach, and upward-shaping effect on where you can go and what opportunities open to you was college admissions testing. And so I became obsessed with understanding the origins of the SAT, the ACT, how these tests work, why they use multiple choice, why they're so long, why they're so expensive. Everyone hates them. And I got so many emails when I was starting this exploration of "I hate the SAT more than you'll ever know" from all sorts of folks. But ultimately, fast forward nearly 10 years now, I immersed myself in understanding what would it take to build the kinds of assessments that really tried to provide an authentic opportunity for demonstrating one's skills, capacities, and interests, as opposed to assessments that were very one-size-fits all, and focused on just testing a set of, I'd say some skills, but a lot of knowledge that is imminently teachable, and somewhat gameable with enough resources. And I do think we have to give credit where credit's due. There are some advantages of a test like the SAT that has been very robust to cheating and coaching, such that the Varsity Blues scandal, for example, is a demonstration of what it takes to compromise a test like that. So just looking at the security issues around testing, but privilege and access issues around testing. That's what really got me interested in building Imbellus, which was a game-based assessments company using games and simulations as a medium for creating those authentic performance environments, but being able to do so at scale, and in a way that respected the conditions and constraints that beget any standardized test that's being used in a large scale, high stakes, i.e., college admissions or state standardized testing scenario. And we were fortunate in 2020, we'd had many years of working with corporations on workforce hiring assessment, we'd done some work exploring, building state assessments for state summative programs. And then truthfully, COVID threw a wrench on a lot of those state testing plans, because kids weren't going to be having summative assessment for a year, if not two. So we were very fortunate that Roblox had seen a lot of our testing work in the corporate market, and acquired the company in 2020. So that's how I ended up at Roblox. And I like to say, I went from about the least popular profession you could talk about with kids working on standardized tests, to the most popular company you could talk about with kids at Roblox, and I feel really fortunate to have made that transition and not be hated. So that's been a blast. And I'm happy to get into more of our Roblox work today. But that's a very long story of how I became obsessed with standardized testing, and brought gaming and simulations to bear, and I'm sure we'll get into all the details of what those mediums can do and why I'm still a fan.

Rena Clark  11:29  

Thank you so much for sharing your story, your windy story. You talked a little bit about the limitations of assessments and dug into those, but what are some of the benefits from assessments, specifically the type of assessments that you've been developing and using? 

Speaker 1  11:50  

Yeah, so let's first start with traditional tests for a moment. And some of the advantages they confer. I mentioned the idea of security and robustness against cheating and coaching, which is a must if we really want a fair assessment system. The other advantage, as you all pointed out in some of the introduction to this podcast is the efficiency both of test creation and of scoring at scale. And when you do have millions of students going through an assessment program each year, if you just think about stepping back, one zoom out, someone has to pay to develop whatever the test items are, and someone has to pay for the scoring and administration of that subject matter. And so the challenge becomes if, in the course of test development, let's say 50% of the questions or the items that we come up with, might eventually fail for one statistical reason or another. Maybe they showed a gender difference where, you know, young women did better than young men did on the test, or folks who are non-binary struggled with a particular question where other folks didn't, or whatever the gender differences we might see that could rule a question invalidated. And it might not be eligible then to be included on tests. And we want that to happen because if an item is performing in a way that's biased or unfair, it should not be used in high stakes assessment contexts. So, by the time you've looked at the content validity, the nature of the statistical properties of these questions, you might end up creating tens of thousands of initial questions from your learning, science, and psychometrics teams, only to find that only a few hundred of them, or a few thousand of them make it through onto a final test form. And then if you think about a test like the SAT, for example, that is administered many times a year, in different locations, in different geographies, you have to be worried about the physical security of those test items because if one region has a test in March, and another region has a test administered three months later, you need to make sure that none of those items are compromised and shared such that, again, that later group has an advantage. And when you do have a test, like the SAT or the ACT that confer such tremendous opportunity on the other side, they're so high stakes, you unfortunately, have a lot of folks who are interested in sabotaging a particular test session with all sorts of organized crime attempts to bring those items out and then sell them and give an advantage. So, you deal with a security apparatus and a testing behind the scenes creation apparatus that's expensive. And the reality I came to terms with throughout years of thinking about how to build a better assessment is the market for these tests. 

Rebecca Kantar  14:32  

For college admissions tests, it's a little bit bigger, but for state summative tests, remember, these are states who are putting out different RFPs every few years, and they have really strict bidding requirements. And of course, these tests are somewhat paid for with taxpayer dollars. So you want them to be lean and efficient. And you want them to be responsible stewards in the bidding process of searching for the best assessment company at the best value or price tag. So you run into all of these constraints when you think about what could a better test be like. And so while I take a lot of credence in much of the criticism and I have participated in over the years, of the multiple choice tests that we're familiar with lacking the richness of opportunity to demonstrate one's thinking, one's critical reasoning skills, one's spiky talents, or performance-based capacities that could never show up with pen and paper, I'm also sympathetic to how difficult it is to build something that's fair, valid, reliable, highly scaled, efficient to create and administer, and achieve those things. The good news is flipping to the kinds of tests I build. I do think game-based and simulation-based assessments for certain types of skills and competencies, begins to approach the capacity to do that. Why do I use all that hedging language? Well, I learned through having nearly 60 folks working on these kinds of task-based games or simulations, that these kinds of experiences, where I drop a student into a natural forest environment, for example, and I want to see them problem solve in this forest environment. I'm giving them everything they need. They don't need to study. There's no bits of knowledge they need to necessarily learn in advance. I'm just assuming that you're a person who's somewhat familiar with trees, with the sky, with animals, with these basic concepts of life. And that when you come in here, I'm gonna give you things that you have to reason with. I want to watch you solve problems. I want to watch you understand systems. I want to watch you create and innovate based on what you see happening in the environment and the data you get back. But in order to set that up, I need a team that's like a well-oiled machine, akin to the kind of folks you'd have making a movie or making a commercial video game. I have a whole gaming team. And I have a whole learning science, psychometrics data science team. Those folks need to work together at every single stage of creating one of these immersive game-based or simulation-based tasks. And what we came to understand over years of doing this together is that's hard. It's expensive. And unlike a multiple choice question, where if it's not working, you kind of throw it out, and you go on to the next, dissecting exactly where you went wrong once you've created a 30-minute or an hour-long video game or simulation-based task for someone to perform in, is expensive and difficult. Was it the art that threw someone off? Was it a UI button? Was it the narrative? Was it the core design? So you have to get really good at testing each layer of one of those tasks as you build them up. So, I think there's a lot of promise for game-based and simulation-based assessment because when you get them right, students do have the opportunity to demonstrate authentic problem-solving, akin to what they'd face on the job or in school, no matter the context you choose to situate your game or your simulation within. And when you got it right, students also have a lot more dynamism in the learning environment slash assessment environment or asking them to contend with, so you get a sense of the adaptability, the resilience, the persistence, a lot of those 21st century skills that I think we often hope to capture with assessment, but can often be squirrely, again, when you're limited to pen and paper. So there's so much opportunity to use these kinds of environments, but they are very expensive and difficult to build. And they're very difficult to establish standardized and fair and reliable scoring for. So I hope we'll see with generative AI, actually, a lot more folks attempting in this arena to come. But at the time we were starting out, certainly we were among the only folks trying to do this at scale, and certainly the only for high stakes. So apologies for the long answer. But hopefully that gives you a sense of where my hesitancies lie and also where my excitement still remains.

Paul Beckermann  19:08  

That's so cool. I love you describing what you were doing there at Imbellus because I want to take that test. And that's silly to say I want to take a test. But it sounds fun. Sounds like something I'd want to do. I'm just kind of curious as you're going through that process, and you're you're really redefining assessments into these simulation-based experiences, what kinds of things, you already talked about a couple, but what did you learn along the way that kind of informs you moving forward?

Speaker 1  19:41  

Well, I think eventually, we came to understand that these kinds of tasks or simulations had to evaluate your process ofhow you think, and that was perhaps the most special advantage these types of simulation environments confer that a traditional test might not. Now a math test, where you have to show every single step, would be approaching or perhaps an open response essay where you have to annotate why you're writing the sections you're writing, where you're writing them, but that metacognitive ability to leave us these little breadcrumbs without knowing you're doing so but matter for us, at least, to see exactly where you were stuck, and how you became unstuck, what you looked at in what sequence, that type of process data, about how you reasoned your way to completion of a task, was an excellent opportunity for richer scoring and evaluation of someone's true skill level. At the same time, those types of scores, how we process these long strings of clicks, or timestamps, or mouse movements, were really finicky to create. So it took us a long time to get practiced and skilled at separating out that kind of process thinking from an outcome in one of the scenarios we'd build. And we called those product scores. What did you get? How did your collection turnout? Or how did the system you develop turn out? Or how did the problem you solve resolve? So, you have an opportunity in game-based and simulation-based assessment to absolutely look at the deeper and the more critical thinking, but only if you can engineer your assessment to capture that in a fair, reliable, and valid way. My same three words, always. So, that was a big learning for us over the years. And then I think the other learning is these tests can thrive in a high stakes environment, but I think they're, perhaps, even better in a formative environment because the ability to take that process data, look at it for each student, and then tailor an adaptive simulation or game that has different levels that kind of go deeper on or meet students at different points of skill, or different conceptual gaps, is really compelling. And again, I think with generative AI, the cost of producing that kind of branching logic content in any one of those games or simulations is coming down. So I think that will be a compelling next chapter for these kinds of assessments. Certainly, in a high stakes environment, one of the hardest additional constraints we were always solving for was how to keep those tests constantly varied, such that the nature of the environment, and the nature of the solution space, both are constantly changing. So we didn't have cheating or learning effects if folks took the test multiple times. So thinking about that was a big learning curve for us, as well, and took us many, many years to get right because, of course, I need to make sure that Test Taker A and Test Taker B, even if they're six weeks apart, have an equivalent assessment in terms of difficulty. So that presents many challenges because you're dealing, perhaps, with different widgets and different right answers. But I need to look at the nature of the possibility space of what both of you faced and then statistically demonstrate that you had kind of an equal opportunity, particularly again, if we're looking at a high stakes assessment, say for employment or for college admission, so probably more technical than you ever wanted to know about assessment. But those were many of my learnings from the past decade.

Winston Benjamin  23:15  

Absolutely not, you're talking to a bunch of teachers. I like digging into this. Yeah, like, don't stop. But here's my question. From what I'm understanding, you're deeply in the science and understanding of assessment, then you're on the highway of life, and then all of a sudden, you pull a whoop, a real quick left turn, and you ended up at Roblox. Can you explain what brought you there? What were they working on and why you felt compelled to go there? And also for a little bit of our audience, just to give a little background of what the company is, just so that we have an understanding of like the shift, if that makes sense in your approach?

Speaker 1  24:06  

Well, for your younger audience members, I'm sure Roblox needs absolutely no introduction, but for parents and grandparents or anyone else who might not yet be on the platform, Roblox is a massive online community of millions of creators and millions of community members of the Robox ecosystem who are able to engage in digital co-experiences. So, that might be playing games, it might be exploring branded experiences, going to concerts or fashion shows or, in our case, learning with Roblox. So Roblox is a platform that brings people together through play, through co-experience, through co-creation, and it seems like a far-flung 180 from what Imbellus was working on. But, in fact, Roblox had always been very thoughtful about its hiring process and its assessments within its hiring process, and when my colleague, Jack Buckley, who's also an assessment veteran, and I were leading our M&A process during 2020 again and the peak COVID time, Roblox had seen some of the work we'd done with McKinsey and Company in the past. So previously, MLS built a problem-solving assessment that's still in use at McKinsey and Company today. And, Roblox folks were thinking a lot about how to renovate their hiring process and perhaps include games on Roblox as evaluations in that hiring process. And we met at just the right place, right time. So fast forward. Dr. Jack Buckley leads our assessment team now at Roblox. I still am very close with all of those folks. They've been with me for many years. But we have a fully on Roblox game-based assessment at Roblox for all new hires. And you guys can include in your show notes, if you want, a link to more information about that. We encourage products folks and engineers to check that out and think about the company, certainly if you're in the Bay Area. And we were able to, very quickly, build the kinds of assessments that we've been building for other corporate customers at Roblox. So those are exciting, they're new, they work really well, and they've been a joy to implement here. But when I arrived at Roblox, I recognized that the company's first chapter was really started in learning with games because Dave Baszucki, our founder, historically had a previous company called Knowledge Revolution that was, in fact, education software. Really, his idea for Roblox as this kind of build-your-own-game environment anything was predicated on his experimentation with his previous company in classrooms where kids just like to kind of build things and knock them down, and really push the physics engine and see how things work, see how they came back together. And so when I met Dave and the team at Roblox, I recognized an opportunity to carry forward a lot of the organic work that had already happened, specifically on Roblox Studio, the authoring side of the platform, where folks go and create experiences that then can be played on the Roblox application that most of our users are enjoying every day. And I saw this chance for that organic work around Roblox Studio that usually happened in kind of computer science camps or coding camps or tech boot camps to be just the beginning of Roblox's entrance into the more formal education space and, of course, supporting informal learners wherever they are, as well. But I thought about us extending far beyond teaching computer science and coding with Roblox Studio which, don't get me wrong, is a fantastic way to learn computer science and basic coding skills because you instantly get to see the fruits of your labor and all of your friends get to experience them right away with use. It's a very rewarding process and feedback mechanism. But I saw the opportunity for us to work with outstanding developers and outstanding educational content providers and educators to build these experiences that really were emblematic of what I feel like folks in game-based learning have always talked about, of this deep marriage between core game mechanics and a set of learning objectives that are standards aligned, that are rigorous, that are higher up on Bloom's Taxonomy, and folks subscribe to it. But these deeper thinking skills, more challenging require, you know, dynamic feedback loops, the fast forwarding and rewinding of time, the immersion, the 3D graphics--where all of these components that Roblox has out of the box, multiplayer chief among them, as well, could be incorporated in a useful way to educators that doesn't pull them out of what they already need to be teaching throughout the course of the year, but fits nicely in. And I felt like with the right network of existing educational organizations and partners, we could offer that value proposition in a very respectful way that minded the pacing, the sequencing, they've already gamed out in their respective curricular materials, and just invited them to think about upgrading some of those sections or concepts that tend to hold kids up or maybe weren't as rich or deep as they hoped they could be, to bring those on to Roblox. So a few years later, since I've joined the company, we have our Roblox Community Funds makes grants between these outstanding educational organizations and Roblox developers. And you can see a whole bunch of learning experiences now on the platform: Mission Mars from the Museum of Science; Robot Champions from First Robotics and tournament games. I know AVID has one coming up. So a lot of outstanding content now that educators and students are free to use and enjoy and that these educational partners are now carrying into the school day, as well. 

Speaker 1  24:40  

So, let's dig into that a little bit more because there's a website for the educational side of Roblox, and I believe it has sections: Build with Us, Teach with Us, and Learn with Us. So can you just tell our listeners a little bit more about that and how they could access some of that content that you're alluding to?

Speaker 1  30:10  

Yes, so just check out Education at roblox.com, if you're interested. If you're a developer, please do come build with us. That's the thrust of the first page there. So learn about what it takes to build in Roblox Studio and the many tools and platform advantages available to you. And, of course, if that fits what you're interested in, reach out, and we're happy to help introduce you to some outstanding educational organizations, which brings us to the next section, the Teach with Us. So, if you're an educator, if you're an educational provider, and you have a concept for an experience you'd like to bring to life that's aligned to standards, be those national standards, international standards, or state standards, all of the above are interesting to us, particularly right now looking at the social sciences, looking at English language arts, more math, we're always looking for more outstanding math experiences that folks dream up, social and emotional learning, as well. Then we invite you to check that page out and let us know if you have an idea for a grant from our Roblox Community Fund. You do have to have pre-existing, established relationships and rapport in schools or in large scale, after-school model so that we know that these experiences kind of make it into classrooms in a way that are  supervised and facilitated and rigorous. And then finally, if you're a parent or caretaker or an individual educator, and you're interested in just, you know, working with people in your life who could benefit from this kind of immersive, 3D interactive and social learning environment, we encourage you to check out that Learn with Us page, where you can see all of the free curricular materials available and get a sense of a number of those educational experiences are already on the platform.

Paul Beckermann  30:10  

Awesome. Gonna have to go check that out, for sure. And we will link to those things on our description of this podcast on our webpage, too. So people have quick and easy access to those; go check those out. Before we move on into our Toolkit and One Thing and those those pieces, I want to talk a little bit about game-based learning in a more general sense. You've talked about some of the benefits of them. But how can educators really leverage this game-based learning in the classroom? And where do you see that kind of going? So like, now, what can they do? But where do you also see that going down in the future?

Speaker 1  32:31  

Right now, I think the main advantage for using game-based learning, in places where in the curriculum, it's appropriate, relevant, and interesting is to get deeper and think about challenging students. Again, back to our theme of the day, it seems to this more authentic type of demonstration of learning and coming up to speed on a new environment, a novel situation, a novel problem, and working together to solve that problem or understand and act in whatever environment or situation. And I stress this want to prioritize game-based or simulation-based learning for those times when you need to go deeper, or where you need to allow students to really unpack and wrestle with a difficult concept or a difficult stacking, perhaps, of many concepts or many skills that you've been practicing in isolation. Because I see games affording that kind of back-and-forth and feedback mechanism in a way that's not overwhelming can be just right for your skill level, because you're driving it and that offer outstanding engagement throughout that entire learn and loop cycle, such that you're likely to do it over and over again, and not necessarily throw your hands up in frustration and just accept that maybe you don't understand stocks and flows, but instead to persist, because that's what well-designed games make us do, in fact, sometimes for too long, where we're saying to ourselves, "How has it been three hours, but it was fun and mesmerizing and challenging." And folks often ask me, you know, how do I make my game based educational experience really fun? And I say, look, sometimes it should be fun. But sometimes it's okay, if it's just really challenging, right? A lot of us enjoy Sudoku. It isn't necessarily fun in the moment. I mean, you're not getting fun and rewards and points and stars. But it's a challenging exercise of logic and reasoning, and it's rewarding when we finish something challenging. So I also asked folks in this community to think about, sometimes it should be, especially with some of our younger learners, perhaps more rewards-based and quick, fast loops that are exciting and silly and zany. But sometimes, it's also okay for not every single educational game to be extremely fun, so long as it's really meaty and interesting and draws us in and it keeps us there with its engagement. So that's what I see excellent game-based learning experiences doing now. In the future, as I mentioned, I'm pretty excited about the extremely low cost of producing the kind of content that I think you'll be able to create on Roblox, for example, with newer generative AI features baked in to Roblox Studio. And I'm pretty excited about the dynamism and adaptability and personalization that that kind of content creation can afford. So suddenly, instead of having all students in the class kind of progressing, and maybe waiting for each other at different levels of an experience, maybe we really can direct you towards the best possible next step or path for you. And if you need a little time, or you want to review a concept, maybe I can do that in a way that's not just complete repetition, but that brings you through a nuanced, different type of learning loop that gets at precisely what it is you were kind of struggling with in the previous level. So, I think that kind of dynamism and adaptability is going to be an exciting future, and it's hard to do when you have to kind of handcraft all of those design loops and all of those levels and all of the assets for them because, again, they're expensive. Anything digital, immersive with so many folks working behind the scenes is expensive to produce. So I see great advantage afforded when more folks can be creators, and Roblox encourages that everyone has the capacity to jump into Roblox Studio, check out some of our free online materials, check out some of the tutorial videos. Many folks teach with it, so that you can learn and have the opportunity to start creating yourself.

Paul Beckermann  33:35  

That's cool. I think that's the perfect place to jump into our toolkit. 

Transition Music  36:29  

Check it out, check it out. Check it out. Check it out. What's in the tookit? What is in the tookit? So, what's in the toolkit? Check it out. 

Paul Beckermann  36:54  

Winston, we're gonna let you start.

Winston Benjamin  36:57  

So from the point from where Rebecca started, she clearly hit words that I love, which are standards. And for me, I think the most important part is deeply understanding the standards and the key aspects of the skills that you're really trying to examine so that you could find ways for the students that demonstrate those knowledge skills, right? It's not just about the oh, let's make it fun. But like, what is the purpose of this engagement that you're doing so that students are proving their knowledge. So, that to me stands out as a toolkit. Know your standards. 

Paul Beckermann  37:35  

Rena?

Rena Clark  37:35  

Winston kind of took mine, but that's okay. I'll say it's aligning your learning objectives and the game elements. And as Rebecca pointed out, I love the idea of it being appropriate, relevant, interesting, so you're actually enhancing rather than distracting because I've seen it go both ways. So how are you using this to enhance? And then the other thing is, we've been talking about some pretty complex things. But there's also a way to repurpose tools and things that you already know how to use to create opportunities for some game-based learning. I work with elementary students. So, it's a pretty simplistic example, but they were doing social studies and the kids, in different ways, created games for other kids to play through showing their learning. And they just did "choose your own adventures," but use the slide template. And I'm telling you, those kids had so much more in-depth knowledge because they also had to have knowledge of like, what actually happened. But what could happen, if this happened. It was really like coding an if-then  block. So they kind of had some of that immersed, and their understanding was much deeper. And then they played each other's games and reflected on it. It's a little bit different than what we're talking about. But trying to also think how we can implement in elementary, and especially using tools that many of the educators I work with are already familiar with.

Paul Beckermann  38:58  

Yeah, and I, I love all that. And I love the game- based learning piece and bringing in the gamification with gaming elements. And you caught my attention, Rebecca, when you started talking about your studio, Roblox Studio. So I like quick, pull it up for a second. I want to see more about this because, I mean, I was a little bit familiar, but so tell me if I'm wrong, but educators can go on there and it's like a free creation tool right within the Roblox site. Is that correct?

Transition Music  39:29  

That is correct. 

Paul Beckermann  39:32  

And I think that's a perfect thing for the toolkit. Go to the Roblox site, the Education at Roblox.com, and go into that. I think it's under the Partners tab. It's cool. It looks really neat. And I would explore that. Check it out.

Rebecca Kantar  39:47  

Definitely. We can get you all the links for your show notes, as well. And I'd say also, the Roblox developer community is robust and ready to help and it's a huge can-do spirit and attitude. So, if you do decide to get started and you get stuck anywhere, don't worry, because there's always someone there to help you, including many folks at Roblox itself who spend a lot of time in that community. So yes, definite plus one, though, to analog game-based design loops and I think anything that kind of asks students to really reason deeply and to anticipate orders of consequence, is a great precursor for the type of deeper thinking that a game-based or simulation-based environment will require of them. And certainly, my hope is always in a way that approximates the nature of thinking you'll encounter as an adult, as well. Because having that situational awareness and that broader understanding of systems and dynamics, I think that's what prepares our students for dealing with very ambiguous environments, and they're certainly going to encounter, especially with generative AI, more and more challenging environments as they enter adulthood and start their careers.

Speaker 1  40:56  

For sure. So do you have like an additional tool you want to drop in the toolkit, Rebecca?

Rebecca Kantar  41:03  

You know, I'm a big fan of the next gen science standards. I know, we've mentioned standards at large. But one of the things I appreciate about the next gen science standards, I think more than maybe any others I've seen, is just that cross-cutting skills nature of this expectation that you're not just building on each leg of the stool independently, but you're, in fact, constantly integrating them. And you're demonstrating those skills and novel stacking orders or sequences. So, I love seeing experiences, whether they're analog games, board games, or digital experiences, that take a few of the more rigorous next gen science standards, and use them as a real capstone moment for students to demonstrate that they can kind of bring it all together. And I think deploying that kind of experience in combination with really good facilitation in class, and maybe a project around it as well, is a really promising way to cement a deep understanding of the themes and recurring science ideas in there, in addition to just the specific knowledge of any one unit.

Rena Clark  42:16  

Okay, we're ready for our one thing. So what's our one thing that we still maybe have left or that we took away? 

Transition Music  42:25  

It's time for that one thing. One thing. One thing. Time for that one thing. It's that one thing.

Rena Clark  42:37  

Let's go ahead, Winston.

Winston Benjamin  42:40  

Test knowledge, not skills, right? Sometimes you could do the skill over and over again, but do you really learn. Test the knowledge, not the skill.

Paul Beckermann  42:51  

I was struck by the complexity of designing that gaming simulation. So like, just to make sure that everything is fair and equitable all the way through, and it's not a test question that gets thrown out. But it's not a part of the problem-solving process that gets thrown out. Just how complex that is. I have a whole new appreciation for that when I go through them. And then the fact that it's assessing the process that the learner is going through to solve those problems. Very complex, but really cool, and I think has lots of value added down the road.

Rena Clark  43:33  

You said it at the very beginning, but this idea of embracing instead of resisting. And then, as you just pointed out, Paul, how do we then evaluate the process of how we think, so embracing that component of how we think instead of just one way, that's the only way. Do you have anything to add, Rebecca?

Rebecca Kantar  43:53  

My final thought is generative AI comes up a lot for me in thinking about efficiency of creating outstanding educational content materials. But I think there's an equally important, if not more important, question of how to teach our students with generative AI as a tool that they have available to them and to embrace it, rather than shut it out as an opportunity for us to expect even more of what they're capable of doing now that they have this very handy and dynamic assistant. Can we push for that deeper thinking, those more rigorous examples of skills, and that willingness to tackle any ambiguous environment because of this new tool by their side, so I'm very excited to see that evolve and look forward to hearing from all of you on that in the future.

Winston Benjamin  44:43  

Thank you so much, Rebecca, for giving us your time. I'm gonna say that when I first heard that we were going to talk about this topic, I was really excited because I play video games all the time. And if anybody out there from games, they'll understand the Demon's Souls, Dark Souls, the hardness of these games. It really forces you to actually learn everything about the boss that you're doing. And, individually, your own personal playing style, you utilize that knowledge that you gained after every loss. The reason why I say this is that even though it was hard and I hated every moment and wanted to throw my remote at the wall, I had resiliency, and I stuck through the hardship of that learning process. Again, the goal is helping students develop the willingness to go through and find ways of solving the ambiguous real-world problems that they live through. So how do we support their ability to take what they've learned and not just do trivial, meaningless tasks, but engage deeply in something that they find valuable, that they are able to apply. Let's think about different ways of assessing our students' skills. Let's push forward. Thank you so much for listening. Have a good night.

Rena Clark  46:13  

Thanks for listening to Unpacking Education.

Winston Benjamin  46:16  

We invite you to visit us at AvidOpenAccess.org, where you can discover resources to support student agency, equity, and academic tenacity to create a classroom for future-ready learners. 

Paul Beckermann  46:31  

We'll be back here next Wednesday for a fresh episode of Unpacking Education. 

Rena Clark  46:35  

And remember, go forth and be awesome.

Winston Benjamin  46:39  

Thank you for all you do. 

Paul Beckermann  46:40  

You make a difference.

Transcribed by https://otter.ai