(00:00:08) Ken Ono: Welcome to Hoos in STEM. I'm Ken Ono, your host, and the STEM Advisor to the Provost and the Marvin Rosenbl Professor of Mathematics at UVA. Our goal is to evoke flights of imagination and wonder by showcasing the cornucopia of all that is STEM at UVA: the marvelous world of UVA science, technology, engineering, and mathematics. Now, in our last episode, we had a great chat with Mark Esser about UVA's exciting plans to push the boundaries of medical technology, especially with the help of artificial intelligence in drug discovery. He's leading UVA's new Manning Institute for Biotechnology, but AI is actually present everywhere these days. In my own work as a mathematician, I'm part of an exciting team at Epoch AI that is working to assess advances in mathematical reasoning, can machines learn to think like a mathematician, with the idea that maybe we can reduce the number of hallucinations that pop up when you use ChatGPT. On this show, on Hoos in STEM, we've had the pleasure of chatting with Scott Acton. He's the chair of the ECE department, Electrical and Computer Engineering, and a lot of his work is grounded in a commitment to AI research. We've also been inspired by Darden professors Yael Grushka-Cockayne and Raj Venkatesan. You might remember that they have founded an institute where the goal is to come up with policy by which ethical uses of AI become standard business practice. And if you love the stars, who can forget our conversation with Paul Torrey, who introduced us to his amazing AI co-pilot. It's a tool that's been revolutionizing his research and the research here at UVA in astronomy.

Now, today, we're continuing the AI theme. And it's very important to point out that most of the research we described a moment ago in terms of AI is really research. And let's not forget one of the most important roles of a university is to educate and disseminate. So, it's with that in mind that I'm thrilled to welcome my friend Kemi Jona to the show. Kemi is UVA's Vice Provost for Online Education and Digital Innovation. Kemi, it's great to have you. Welcome to Hoos in STEM.

(00:02:33) Kemi Jona: It's great to be here, Ken. Thanks for having me.

(00:02:37) Ono: You might be surprised to learn that Kemi here sends me more texts than, in fact, maybe everyone other than the immediate members of my family. You see, it's really important to Kemi that I, as a representative for STEM in the university, be well aware of all things that are happening related to AI, innovation, and education. So, on a daily basis, he's texting me fascinating, sometimes alarming, articles about how AI is transforming well, almost everything, from education to the workplace, and often, including my own research as a scientist. So Kemi, for our listeners who don't receive your texts, what have you been reading lately?

(00:03:21) Jona: You know, the things that are really, I think, front of mind for me, Ken, and I think all of us, have started to see it because it's these stories are showing up in mainstream outlets like the Wall Street Journal and the New York Times, not just sort of scientific publications. There have been a lot of stories about you know, like what does it mean for students to be using ChatGPT and other AI tools in school, whether it's you know K-12, high school, certainly universities, and there's a lot of hand wringing going on about whether is this a good thing, is it a bad thing, how do we make sense of it. You know, what do young people think, because as we know, right, they're always out in front, and our early adopters on almost every kind of technology. And to me, you know, what that says is that we have a lot of really hard questions to ask ourselves about how we've structured education and whether that has to change meaningfully as we come to grips with these powerful new AI tools.

(00:04:29) Ono: So, in particular, there's there many layers to that. I remember when ChatGPT first came out. It came out at a time where I was joining the College’s faculty as part of their freshman engagements program. And I think, like most of the other faculty that teach in this program, it was eye opening just to see how much ChatGPT could offer students. And it was, it was surprising, now that I reflect on it. I think all of us in the room, our first reaction was so how do we make sure that students aren't using ChatGPT to write their essays? So, Kemi, tell me, what are the questions that you want us to be thinking about with regards to introducing AI into education?

(00:05:20) Jona: You know, one of the things that we all saw right away is exactly what you said, which is, oh my gosh, we've got this tool that can literally, you know, do your math homework, do your physics homework, write the essays that, you know, get assigned. You know, we've seen, you know, that some student at Columbia like, you know, bragging that he used it to cheat his way through all the way through school, or even cheat his way through a job interview, right? And so, what it brings to the surface is a real question about how we've structured what school looks like, right? And unfortunately, you know, when you talk to a lot of young people, especially high school students, a lot of them sadly feel that school, whether it's high school, college, whatever, has just turned into a bunch of meaningless hoops to jump through, right? And they... we've lost the narrative, right? It's just like, you know, do this assignment, get your grade, try to get a high GPA. That's going to unlock your college success, and then when you're in college, you know, try to get a good GPA so that you can get you can be competitive for your internship or your job, right? It's just a sort of long chain of jumping through these hoops. And what I worry, and have worried about for a long time, but I think it's really coming to focus for all of us now, is well, what does that even mean anymore if those hoops that you can jump through can essentially be done by ChatGPT? 

(00:06:47) Ono: It's funny, just this morning I got an email from a friend. He's a professor at Oxford, and the educational system in England is very different from what it is here in the United States. You will take year-long courses. There's no homework. There are no exams until the very end of the year. And the students have to sit down with a professor or a collection of professors for an oral examination. And I thought for quite some time that this would be super stressful, and it is. But in the era of AI, I'd come to the conclusion that what a wonderful way to test students because you have a one-on-one engagement with the students, and you get to discuss the material. So, this email I got from a colleague at Oxford this morning was particularly shocking. He had just finished examining 58 master's students in my field, in mathematics, and although these were oral examinations, he also gave the same oral examination to ChatGPT o4-mini-high with mathematical reasoning, and he graded the transcript. Out of the 58 students, only one scored better than ChatGPT. And I would have thought that there would be no chance for ChatGPT to perform at that level. And so, it puts us in a position where, adding to your point, we really need to rethink what our goals in education are, and you're doing that. So, question, we're here at Thomas Jefferson's university. We love to talk about how Jefferson inspired the faculty to pursue the illimitable search for knowledge, something along those lines. So, what does it mean to you in your role to aim to successfully educate the next generation?

(00:08:47) Jona: Well, that that is the eternal question. So, I don't know that I have a particularly better insight into it than you know, any of the other faculty, or really any parent who thinks about how to bring up their kids, you know, to succeed in the world. That's what we all want to do. But you know, I think the example that I think might make a lot of sense for you, Ken, is to think about your experience working with these cutting edge, you know, AI models, right, and you put some of the most advanced models through the paces of a very advanced, you know, mathematics problem and you know, your job was to kind of figure out, well, like, can it do, you know, can it do the work, right, just like the story you told about your Oxford colleague. And the thing for me is, you needed expertise in order to assess whether the output of that AI was correct, and accurate, and rigorous, just like your colleague at Oxford had to have the expertise to both examine all of the 58 students, and assess the output of AI. And so, the question is how do you get that expertise, right? How do you acquire expertise to be able to even work with an AI and decide whether that output is garbage, as you said, whether it's full of hallucinations, right, or no, whether it's actually done something really meaningful and that's acceptable, right. In your case, you put them through exercises that only a very few number of mathematicians would actually be able to do as a test, but it's the same principle for everybody. And so, you know, the thing I worry about a lot, Ken, for education, whether it's you know bringing up your kids or whatever, in the area of AI is how are we going to sort of instill that expertise if everybody is leaning on these AIs to do most of their thinking, right? And the same is true for employers, like how are you going to get expert employees if you've just eliminated all the entry-level roles so you have no junior employees? Well, how do you get senior employees without junior employees, right? Like we all, both educators, and employers, and parents, right, all have to figure this out and we have a lot of work to do.

(00:11:11) Ono: The story that you're describing for listeners was, kind of appeared in an almost semi-viral Scientific American article from a few weeks ago. In May, at Epoch AI at Berkeley, I was leading a team of about three dozen mathematicians who were charged with the task of assembling 50 benchmark math problems. In the room were three Fields medalists. If you don't know what that means, think Nobel Prize of Mathematics, and you would be pretty close to right. And you would think that with this level of expertise in the room, that it would be super simple to assemble 50 math problems that are humanly solvable but are not solvable by ChatGPT or a large language model. Now, as the article goes on to explain, we struggled to do that. Now, make no mistake that the problem of assembling benchmark problems doesn't exactly align with what mathematicians do. Just like with all sorts of metrics, they're just proxies. So, please take what I'm about to say with a grain of salt. It was hard for us. We had a very difficult time assembling these 50 problems. And after a day, I thought, well, how could I come up with problems that the computer would have no chance at solving? And I thought of one. And I thought of one where I was probably the only person in my field, and please don't take this the wrong way, but in the world, who could have answered this question because the question relied on a very subtle part of a paper I had written a few years ago, that even I misunderstood the implications of when I wrote the paper, and in fact, it took me four years of deep thought to figure out that this paper had these applications. So, I asked ChatGPT this open problem and it says, "What is the fifth power moment for elliptic curves, their Tamagawa numbers." If it sounds complicated to you, yeah, there probably 20 people in the world would understand what I was asking and I sat... and it was extraordinary. The computer, over five minutes, navigated the literature, made some poor choices, made a bunch of mistakes, but then returned to literature searches. Then it decided to just, on its own, compute a number of examples. A little pop-up window appears on the screen and says, "I computed 50 examples. I still don't know how to do it." But then after 5 minutes, locates my paper from 2021, which doesn't even have the words in the question I asked in it, and it was able to put like a thousand two-and-twos together to recognize that this one paper out of the millions in science would be relevant. And then it found the observation that I'd only realized after four years, the night before, and it answered the problem.

(00:14:14) Jona: Incredible.

(00:14:15) Ono: And in terms of translation, the formula that it produced is not a formula that is easily calculable. So, I thought, can it actually translate this theory and compute some numbers? And it thought for 5 minutes which, by the way, if a computer is thinking for five minutes, is pretty hard so, in a perverted way I was quite pleased that it took the computer five minutes to compute it. All I did was ask two questions, and the computer did the rest of the work. What do you think about that?

(00:14:43) Jona: I think it's amazing. I mean, you're describing a completely new way of doing your professional work that literally did not exist 6 or 12 months ago. And you've changed how you literally do your work every single day, right? And so I think, you know, the question that we all have to wrestle with is, you know, well, how would that look for all the other jobs that are out there, and roles, right? Not just to be a very distinguished mathematician like you are, but for you know all the jobs out there. And then, you know, the next step for us as educators here at UVA is well, what does that mean to us in terms of how we prepare students for this kind of literally brave new world? And how can we guess how the jobs and roles will change in you know, the next year, two years, five years? At least that we can envision, right, and so you know, I think to me on the one hand, it's super scary. I think it is somewhat scary even, you know, for you, right, to really wrap your head around how different and how powerful this tool is. So, I think it really is worrisome. On the other hand, for me as someone who has spent, you know, most of my career thinking about what does effective learning look like and how do we create environments that support humans in doing that learning across the lifespan, you know, one thing that I think is worth celebrating is it's really shined a light on some of the weaknesses in the model of how school works.

(00:16:21) Ono: Yeah. So, I know there's a favorite skit you like to share, so I guess the question is, is there a funny way of describing what education should not be?

(00:16:33) Jona: Yeah, I have a favorite clip that I often use in my talks that I think really illustrates this idea of the fallacy of, sort of, memorization as the core of education. It's by a comedian Don Novello who, you know, plays a character on some of the original Saturday Night Live shows and did a really hilarious skit in a sort of a special with Gilda Radner.

(00:16:58) Ono: Let's play it.

(00:16:59) Recording of Don Novello: I find that education, I think it don't matter where you go to school. Italy, America, Brazil, it's all the same. It's all just memorization. And it don't matter how long you can remember anything, just so you can parrot it back for the test. And I got this idea for a school I would like to start, something called the Five Minute University. And the idea is that in five minutes, you learn what the average college graduate remembers five years after he or she's out of school. Would cost like $20.

(00:17:42) Ono: So, Kemi, what do we learn from that?

(00:17:43) Jona: A couple things. One is, the reason it is funny is that we all recognize it to be true, right? Like it resonates for us. We don't listen to that clip and go like, what is he talking about, right? Like, we all we all get it and the truth of the matter is that you know if you just do rote memorization, which a lot of school involves, you will forget it, right, and that's the deep insight from cognitive science is that if you just learn something from  memorization and you don't apply it and you don't practice it, you will forget it, right? You just, you won't remember that stuff. And the things that we learn, the most enduring memories from school, often are you know in teachers or faculty who really motivated us, inspired us, to learn because of their passion and then, that's something that, you know, we really dug into ourselves, right? And so it really is use it or lose it, and so as we think about well, what is it going to mean for AI, in terms of transforming how we think about teaching and learning, we have to obviously get away from this idea of you know, education as memorization, which was the funny part of the of that clip, and think about how do we master the tools, like you described, Ken, of using AI but then learning how to apply it, learning how to discern whether it's right or wrong, whether the, you know, the solution makes sense in a particular context. So, what does that look like? Well, it means doing more sort of project based learning, right? Doubling down on the kind of human skills that AI is never going to replace, right? Teamwork, collaboration, leadership, ethical reasoning, you know, giving and receiving feedback, negotiating out a difficult disagreement on a particular, whether it's a presentation you're working on or, you know, personal or interpersonal issue, those are all human skills, right? And so, we really need to shift away from this idea of like memorizing or, you know, kind of mastering formulas and facts and move towards, how do we really use this stuff in real world context? And each context is different, right, so there's no such thing as just sort of learning X, and then you know it across the board. You have to learn it, and then you have to apply it subtly in a particular context, so that's really what it's going to be about

(00:20:14) Ono: I think there's some terminology out there that has been quite bantered about in higher education circles, experiential learning, right? So, can you describe what that means and maybe offer us an example that we can all learn from?

(00:20:30) Jona: Absolutely. So, when we talk about experiential learning, you know, it goes by a number of names. Applied learning, work-based learning, career-connected learning, right? And I think it's especially important for a bunch of reasons. First what we just talked about, which is context matters, right? And so, whatever you learn in school, you're going to have to apply it in a particular context. And so, this idea that you can sort of like isolate your learning from these real-world contexts of application is kind of the big lie of education, right? And so, we have to shift back towards allowing students the chance to apply their learning in sort of real world projects, whether it's, you know, community engaged learning, service learning, volunteerism, or you know, internships and that sort of thing. And so, that's one. And the other reason, I think is more important today than ever, is when you engage in an employer project, whether it's a traditional internship or a virtual project, which we'll talk about in a second, it puts you, the student, in contact with what the real world looks like today. And that is more important than ever because it's changing so fast, right? The top 20 skills that employers look for in a particular job are changing between half and three quarters every four years. That means for the same job, like a software engineer or a financial analyst, the same job title is going to expect different skills by half or even three quarters between the time when a first year shows up and when they walk the lawn, right? And so, in that kind of a world, the only hope you have of staying current is to really spend a lot of time doing work-based learning projects, career-connected learning so that you're staying in sync with what does it mean to apply AI in financial services or health care or environmental science.

(00:22:29) Ono: Right, and it's not just AI, it's everything. And the traditional view of college had been well, there might be a College of Arts and Sciences, a Department of Biology, there's a Department of Chemistry, there's a Department of Statistics, History, so on and so forth, and the topics that you might study would be completely siloed from department to department. And I can give an example in the math department, where we have been not great at this, but we're getting much better at it through HHMI, which might have an episode on later, where we teach the rigorous math courses in a way where the students that take them have a difficult time of recognizing when the theory that is learned is actually relevant. And historically, it's been well, what do I need from these courses as prerequisites for the next course, without really addressing how any of this mathematics might relate to the world.

(00:23:24) Jona: There's a term for that, actually, it's called inert knowledge

(00:23:27) Ono: Oh, inert knowledge, yeah, so, we have to combat that. So, one point I want to pick up on here is that I didn't hear you even mention the need to grade our students. So, what are your thoughts about the role of grades, or maybe your view on how we evaluate students that we train in an experiential learning environment?

(00:23:47) Jona: Well, it's a great question. It does come back to this idea of like, how are we going to navigate this new era of AI, and instead of sort of trying to police it and focus on the plagiarism. And so, look, I think one of the silver linings here with, as AI kind of permeates through, you know, the world of the university and our society is, we all are going to have to rethink how we do tests, and assessments, and quizzes, and grades so that we can ensure that the students are really learning. You know, one of the fears and the themes of the articles that I keep texting you about, that you mentioned, is this idea of cognitive offloading, right? What does that mean? It means that if you kind of shift your thinking over to an AI, then you're not learning yourself, right? And I think the easiest way to think about that, Ken, is you know the analogy of sort of like buying a really expensive gym membership and then sending your friend to work out for you, right? And like, nobody in their right mind would do that, right? Because we all understand that, like if you're going to get the benefits of working out, whether you're an athlete or just, you know, for health, you can't send somebody else to do it. You've got to do it yourself. And in fact, not only do you have to do it, but you have to push yourself, because if you're not pushing yourself, you're not going to get stronger, healthier. But that's what school, and especially university, is really like. There's going to have to be a big mind shift around that, with both students and parents, which is you're going to pay a lot of money to go to school, and if you give all of that to ChatGPT, you're like sending your friend or sending a robot to work out for you. And so, when we come back around to it, we have to think about well, how are we going to assess learning and do grading in a way that really ensures that the students are doing that mental workout, that cognitive workout, and that they come to see it as a cognitive workout and not just as hoops to jump through, right? So, that's going to take a pretty big cultural shift. And to your point about your friend at Oxford, it's probably going to mean more work for the faculty to really engage deeply, whether one-on-one or small groups. And I think that you know the north star for that is shifting from the idea of product, which is like turning in your paper or turning in a test, to process, which is what is the process that you use to figure this out? 

You've talked a lot about that in your sort of mathematical proofs, where you're looking at what are the steps you went through, where  did you go wrong, how did you recover from that, and the same is true with, whether you're writing a paper, you're researching, you're solving a problem. I think the faculty, the instructor, is going to have to really deeply engage in what is this process, tell me the ideas that you thought about, tell me the ideas that you rejected, why did you reject them, what evidence did you bring to bear? All of those process steps take a lot more time and effort, but ultimately are going to reveal, and help you kind of do that mental exercise that it's going to take to really become you know, an expert, or even a proficient learner.

(00:27:07) Ono: That's all part of also getting to know oneself better. If you don't graduate from the university having a better sense of self than when you started, then I think we have all sort of missed out on something. Right, you may get grades, you may get your diploma, but those are just proxies for what you have become. Speaking of grades, so the traditional grades, A, B, C, D, GPA of 4.0, what do you think about that? I get the impression that you'll say it's antiquated, so on and so forth, ut I want to hear what do you think about our need, our society's needs for evaluation?

(00:27:48) Jona: It's a great question, and it's also I think a fraught one because we're just we're so used to the whole system being organized around this idea of grades. But, if I take my sort of official Vice Provost hat off and just speak sort of as a learning scientist for a minute, you know, the idea of GPA, I think, is really antithetical to learning because it doesn't encourage exactly the kind of thing that makes for powerful learning, which is making mistakes and learning from mistakes, right? The thing we know about human learning is that if you're not making a mistake, you already know everything, right? It's only when you come up at, as you said, at the limits of what you're able to do, or able to understand, or you try something and you make a mistake, or you get feedback from your professor that like, you know, this doesn't make sense or you forgot X, Y, or Z. That's when you come up against the limits of your knowledge and that's when you learn, right? 
And so, you know, grades can be used for good but often are creating an environment that creates a risk aversion. People don't want to take chances and make mistakes because they're afraid of hurting their grade and their GPA. And that's exactly the opposite of the kind of environment that makes for the most powerful learning. Taking risks, putting yourself out there, putting an idea out that might not be totally baked and then revising it.

(00:29:18) Ono: I'm really glad that you articulated those thoughts, because I strongly believe in them. The way I would describe it is that our need, our societal need for assigning grades can introduce horrible negative feedback loops.

(00:29:35) Jona: Absolutely.

(00:29:36) Ono: If you get a B in a class, does a student then think I'm no good at it?

(00:29:41) Jona: Or I'm a B person.

(00:29:42) Ono: Or I'm a B person. And it may surprise listeners to know that when I graduated from the University of Chicago, I graduated with a 2.7 GPA on a 4-point scale. I didn't get my first A in a math class till my senior year at the University of Chicago. And I by no means think that I couldn't teach all of those classes now, honestly, probably better than the courses that I took, because through the benefit of time and thinking deeply about the subject, not getting good grades straight away helped me become an expert. So yeah, I'm not a fan of grades. I recognize that we have to give them, and I strongly believe that giving an A could be a misleading. Giving a B could result in negative feedback loops. And I don't think anybody in their right mind really believes that grades are perfect proxies for evaluating student performance, but it just happens to be what we in higher education have settled on going back many, many decades. So, we've been talking quite theoretically about innovation and education, let's start putting our fingers on things. Let's start hanging our hats on stuff. So, Kemi, tell us, what have you been doing now that you've been here about two years. What can you tell us about the work you've done at UVA?

(00:31:08) Jona: We've been doing a few different things, Ken. The office that I lead, the Office of Online Education and Digital Innovation, has two parts, as the name might imply. So, on online education, it's what you would sort of expect, which is we're here to support the strategic plan, the 2030 plan's goal of making UVA accessible to those who can't come to Grounds, right? Grounds is an incredibly special place for those who are students here or alumni, they know it. But let's be honest, getting to Charlottesville is not that easy for a lot of people, especially if you are not 18 years old anymore and can't spend full-time here. And so, our office works with the Schools to help find and put programs online, particularly grad programs, so master's programs, so that you know, anybody anywhere in the world could take advantage of the spectacular UVA faculty and UVA education. So, that's a big part of our work. For example, we've just helped Batten, the policy school here, put two graduate certificates online for their Master's of Public Policy. That program is sort of a must-have for anyone who's going into public policy and wants to move up but until now, you've had to come to Charlottesville to take any of those classes. And guess what? There's a lot of people in DC, including a lot of our own alumni who are working there, who just can't do that, right? And so, that's a big part of the online education part of our work.

(00:32:49) Ono: So, for students that are interested in these courses, how do they find out about it? Where can they go? What website can they visit to learn more about these certificates?

(00:33:02) Jona: The, you know, all of the programs are going to be on the school websites, right? So, you go to the Batten website and you know, you discover them there. We do have kind of an outdated central site, online.virginia.edu, that needs a little bit of cleaning up, just because it's got a ton of stuff in there and it's hard to sort through. So, we're in the process of sort of revising that site to make it easy for people who want to find online courses, but really, they usually come in because they want a particular degree or topic, right? They know about Batten and then they want to find it there. So, that's where it'll be. It'll be available, but we have a lot of them already. So, as you probably know, education has, you know, well over half of their grad students, who are mostly teachers, are taking online programs. The School of Engineering has, you know, a handful of engineering master’s degrees. Data Science has a very well-known, both in person and online, program. So, that's the online education part of the work that we do.

The Digital Innovation does actually sort of inform you know, the students who are here on Grounds more, who you know, tend to be sort of in-person and residential. So, some of the things that we've been doing have been in what we call the co-curricular space. So, not necessarily trying to turn you know our undergrad courses online because they are delivered, you know, extremely well here in person. So, one example is we've been launching a set of career academies, right, as we talked about the experiential learning and helping students bridge kind of those in-demand skills that they need to be competitive for an internship, or a job, or even being competitive to apply to say, McIntire, or you know, any of the other Schools that they want to get into. And so, what does that look like? So, we started in J-Term with a career academy where you could take some of the Google professional certificate courses and then do an applied experiential project with a real employer. We had 13 employers. and so during J-Term we had about 55 students from a few student clubs, Women in Computer science, the data science club, data science and analytics. This summer, we're running it again with almost 90 students. So, we almost doubled the number of students, and interestingly, we've got about a third of them are student athletes. So, as you might suspect, there's a bunch of athletes who have to stay here on Grounds in the summer, and practice and work out. We have a number of athletes, actually some of the swimmers that you work with, Ken, who are not on Grounds. They go back home and they work out at their home pools, but they're doing two-a-day workouts and so, all of these, we have, I think, three or four international students, some of whom are athletes who are back home at their home countries. But the thing they all have in common is that they can't access a traditional in-person internship. Their schedules are too busy, they had to go home, you know, aside from the athletes who are working out, practicing, but others who just go home and, you know, maybe they have to do family, you know, child care or they have to work at, you know, the family business. And so, we've made those virtual internships accessible and the online learning accessible to them to fit around their schedule, right? So, we've gotten, I think, a lot of excitement and we'll be running it again in the fall. And we're looking forward to doing that sort of thing. We're also about to announce some partnerships with some of the big AI companies to bring some of those tools to Grounds so that students and researchers can have access to some of the latest and greatest.

(00:36:47) Ono: In terms of these internships and in terms of these partnerships, could you give some examples of companies that have offered these internships and maybe name some of these AI labs that will be coming to Grounds to uplift this work?

(00:37:16) Jona: Well, interestingly, these virtual internships that we're doing, they tend to be with small and midsize businesses, right? You know, they're not the Fortune 500 ones because they tend to do the traditional in-person internships, right?

(00:37:33) Ono: And they run their own operations in house.

(00:37:35) Jona: Exactly. Right. And so, you know, there's one example of one of these small businesses, is called Toys Electronics, right? And so, we get them on the platform through our partner, Ripen, that helps us recruit them. We've been doubling down on trying to recruit Virginia-based businesses so that we can actually provide a lift to our community with smart UVA students who are helping them with real projects. We also do projects that are sponsored by UVA, right? So that you're doing analytics work or other things, but for the university. So, you're building experience. A lot of students put it on their LinkedIn profiles, which is great because that's the whole idea. And you know, what's exciting is that when you're working with some of these small mid-size businesses, you get to work right with the senior people, that you wouldn't get in a Fortune 500. So, you're working with the Chief Marketing Officer or the COO. So, it might seem sort of less glamorous on the surface than like a big, you know, like working for Google or Apple, but you get to do really meaningful work, and your voice is really valued.

(00:38:41) Ono: You see why it matters, right?

(00:38:42) Jona: Absolutely. And you're wrestling, as we talked about earlier, Ken, with real issues, right? So, I'll give you one story. You know, the students who worked at J-Term, you know, doing some analytics projects, well, they would get data back that was completely dirty, and messy, and incomplete. And so, they had to negotiate out with the business leaders, what questions you could actually answer with the data. It wasn't this like clean packaged assignment that you would get in class, where you have the whole data file and it's perfect, right? So those are the real world issues that come to life when you're doing these experiential learning projects.

(00:39:19) Ono: That's great. If you are running a small business, and you'd like to partner with talented UVA students, Kemi Jona, send him an email, because it's very much a part of our growing educational and vocational offerings.

(00:39:34) Jona: And an effort to lift up, you know, as a public university, our mission is to serve the Commonwealth.

(00:39:41) Ono: That's right. We are the University of the State of Virginia. So, as many of you may know, it was recently announced that our President, Jim Ryan, has resigned, largely due to pressures that are related to some reporting with the Department of Justice. I don't really want to go into that now, because that's really not the point of what we're celebrating here today, but I do want to emphasize that this idea that the University of Virginia should be uplifting the entire state for good has really, and has been, Jim Ryan's rallying call. You know, I am sad that he is leaving the University of Virginia. So, I'll hang my hat on that. But regardless of where we end up moving forward, the work goes on. The work that Kemi is doing is very important, and I hope we continue to use Jim's mantra, that his vision, and I think it should be our collective vision for UVA, is that we are both great and good in all that we do. So, we have a couple minutes left. I like to talk about how all of my guests have ended up here at the University of Virginia. So, Kemi, how did you end up here at UVA as our Vice Provost for Online Education and Digital Innovation?

(00:41:09) Jona: You know, I've always been really interested in technology, and especially with AI and learning. So, in high school, I, you know way back, got really into programming with some of the first computers. This was an Apple II, for those who go that far back.  I was also inspired by the movie 2001: A Space Odyssey, if you remember, which had you know, the computer character Hal, that was sort of running the spaceship was like an artificial intelligence. Of course, it turned out to be a, spoiler alert, sort of an evil artificial intelligence, but I was always really fascinated by like, how could we make computers to be that smart? And that was really the thing that drove me. So, you know, I went to undergrad at Wisconsin, where you were a faculty member, although maybe a little earlier than that. You know, and there was no major of AI, right, this was well before that, so I had to kind of make my own way, and I sort of double majored in computer science and cognitive psychology, to sort of come at it from both sides. And then I went on to graduate school in AI, to work with one of the godfathers of the field, Roger Schank, who was at Yale at the time, and then later at Northwestern University. And one of the shifts that happened there was this idea of not just focusing on making a smart computer, but could we have computers help people get smarter, right? And so, it was really using the same insights about human learning, memory, cognition that we were trying to apply to make smart computers, smart programs, to how do we take a computer and help it make a person smarter by helping them learn in ways that are aligned to how memory really works, not how we heard Don Novello in in that funny clip talk about, right? So, the opposite of that. And so, that's really been animating for me. And then, you know, I started my career at Northwestern as a research faculty and then, you know, focused on building learning technologies that would expand access to high quality learning for K-12 students. And then, from there, went to a number of other schools, ended up at Northeastern University in Boston, which is a big experiential learning school.

(00:43:37) Ono: Yeah, I certainly know about that. They seem to be everywhere these days.

(00:43:40) Jona: They do. And you know, had a role very similar to the one I have here, around innovation but focused on building close employer partnerships to have students you know, benefit from that experience and then, had the good fortune to apply for and get selected for this role. And you know, what really excited me particularly about this role, Ken, is you know, I started my career as an undergrad at Wisconsin, which is a great public institution and I've had the opportunity, the privilege, to work at many really highly regarded universities but they've all been private, Northwestern, Northeastern, Carnegie Mellon and others. And you know it drew me back the public mission that you talked about, being great and good and making education a ticket to economic mobility for all, has really been the animating mission here, and the work that we do at my office now about making UVA accessible to those who can't come to Grounds is a key part of that.

(00:44:44) Ono: Yeah. Well, we're glad to have you here. So, when you were first hired, you were hired into an office that was that was only a concept. So, tell us, how's it going?

(00:44:55) Jona: Well, we are, yeah, we're small but mighty. You know, we've got a great team. I think we're up to five now, in addition to myself. So, we've got Jennifer, and Sarah, Rebecca, Taylor, and Jaden, who are all sort of, you know, expanding the abilities that we have to serve and partner with our, you know, with the Schools across Grounds.

(00:45:21) Ono: So, we have to wrap up now. For students that want to get involved, if there's a role for students in your office, how can they get involved?

(00:45:29) Jona: Well, we love to have student voices because ultimately that's what we're here for, is just to make the experience that students have on Grounds or even online, more engaging and more accessible, like we talked about with the athletes and the other students this summer. So, you know, we'd love to have student interns, if they want to come and and actually join the team and be part of the work. And for those who, you know, want to join the programs like the ones I described, you can email us at oedi@virginia.edu. Right, online education, digital innovation. And we'd love to, you know, sign you up for the next round or get your suggestions for the kinds of tools, technologies, programs that students think about wanting to do. The other exciting opportunity that's coming up is one of our colleagues, Mona Sloane, who is in the College, is leading...

(00:46:26) Ono: And the School of Data Science. She is kind of a dynamo.

(00:46:30) Jona: She is. Yeah, she is a dynamo. She's launching a really exciting new project called the Student Technology Council, which seeks to sort of extend the student governance tradition here at UVA, and convene a group of student representatives who want their voices heard about how the university adopts technology, what our policies are, which tools we should or shouldn't be adopting. So that's a really exciting opportunity, and I encourage students to reach out to Mona if they're interested in helping organize that, or just participate in it.

(00:47:06) Ono: So, it is time to wrap up, but I would be remiss if I didn't ask you to tell me a funny story. Can you share a fun fact, a funny story?

(00:47:16) Jona: Well, I knew this question was coming, Ken, so I did my research, and knowing that that you and I both spent many winters in Madison, Wisconsin. When I was an undergrad there in computer science, we didn't have laptops, and I know this was an era when you were also learning too. We had to go to the computer science building, into the basement, to access the terminals that, you know, that we would use to program or do our assignments. And trust me, when you have to walk across a huge campus in the middle of winter in Madison at midnight because that's the only time that the terminals free up, then you really appreciate the technology. And think about the fact that the phones that we have in our pocket are 10 times more powerful, or even more, than that computer that we had to walk to in the basement. So, you know, imagine and that was you know, 30 plus years ago. Imagine, in 30 more years, you know, how powerful the computers and the software are compared to where we were then. So, that's sort of my, you know, semi-funny Madison story about being a young computer science major trudging through the snow to try to get on a terminal and finish my programming homework.

(00:48:41) Ono: Yeah, I remember the lake would freeze, and it wouldn't thaw until May. So, Kemi, it's been great having you on the show. Thank you for your continued commitment to building new things so that the university can be future-oriented. And it's hard for us because we are very traditional. We're well known as your epitome of traditional elite, high quality, public university. And so, it's not so easy for us to incorporate these new tools, but I think we all recognize that's really important. So, the work that you're doing is both great and good. I'm Ken Ono, STEM Advisor to the Provost and the Marvin Rosenblum Professor of Mathematics and you've been listening to Hoos in STEM.

• Kemi Jona
• Online Education and Digital Innovation