How can we ensure that the AI we build is trustworthy? Responsible? An agent of the great and good, even as their rapidly increasing capabilities continue to astonish us? Thanks to the generous support of UVA Darden alum David LaCross, the LaCross Institute for Ethical AI in Business at Darden has been created to address these very concerns. We invited the Institute’s co-directors, Professors of Business Administration Yael Grushka-Cockayne and Rajkumar Venkatesan, to discuss the future of this inspiring new initiative, and how they came from unconventional backgrounds to bear the torch of ethical considerations amidst rapid technological innovation.
- Related Links
- Transcript
- 0:07
welcome to Who's in STEM i'm Ken Ono your host and the STEM adviser to the provost and the Marvin Rosenlum 0:13Professor of Mathematics at UVA our goal is to evoke flights of imagination and 0:19wonder by showcasing the cornucopia of all that is STEM at UVA the marvelous 0:24world of UVA science technology engineering and mathematics now this is our second 0:32episode of our third season and in our last episode which we kicked off the 0:37third season with we had a great conversation with Professor Paul Tory about the incredible opportunity for 0:44scientific advancement that AI represents for a growing team of astronomers astrophysicists 0:51statisticians data scientists and more that was our conversation about cosmic 0:57AI now today uh we're going to stick with the AI theme uh we're going to talk about the other side i mean how do we 1:04navigate a rapidly changing world where computers and AI seem to possess 1:10superhuman intelligence and well to set the stage I 1:15think many listeners will recognize the next clip from this iconic film 2001 A 1:21Space Odyssey open the pod bay doors Hal 1:28i'm sorry Dave i'm afraid I can't do that i think that clip haunts us and it 1:36might be the most famous haunting clip but there are many others let me remind you of the film Terminator 3: Rise of 1:42the Machines in that 2003 film the Malevolent AI called Skynet sent 1:49machines to Earth to attack people hoping to destroy human civilization 1:55although that's the stuff of film with the rapid evolution and introduction to AI which is occurring at an 2:02extraordinary rate my head's spinning i think all of our heads are spinning both with excitement but also with fear and 2:10worry it's kind of a twisted version of Oscar Wild's famous quote "Life imitates 2:16art far more than art imitates life." Fortunately faculty across UVA have 2:24anticipated the need to grapple with some of our pressing questions how can we ensure that the AI we build is 2:31trustworthy responsible and an agent for good benevolence in other words how can 2:37we establish and embed a system of ethics in our AI tools as their 2:42increasing abilities continue to astonish us so thanks to the generous 2:48support of UVA Darden alum David Lacrosse the Lacrosse Institute for 2:53Ethical AI and Business at Darden has been created a very generous gift and 3:00the charge the purpose of this institute is to address these very concerns so I'm 3:05delighted to have with us today inaugural co-directors here in conversation professors Yael Grushka 3:12Cocaine and Raj Venatessan to discuss how this exciting new initiative is 3:17taking shape yael Raj welcome to who's in STEM thank you for having us thank 3:23you for having us it's a pleasure to be here congratulations institutes don't just pop up it's really a very special 3:30day at the university when institutes are founded and it's it's a great honor 3:36for you to be leading this effort i guess the best way to get started is 3:42tell us about the institute's mission let's just dive right in well one way to think about it and to start and one of 3:47the inspirations for uh Dave and Kathy Lacrosse was Darden's mission itself uh 3:53Darden as a business school has always had at its uh front and center to not 3:58only educate business leaders but to educate responsible business leaders and 4:04to ground them in making responsible decisions uh for their businesses and for the world around them and so the 4:11institute um is directly aligned with the school's mission not only is ethics 4:16in every single decision that every business manager uh makes um but definitely related to AI um and uh we 4:24also take what we call an enterprise perspective or an interdisciplinary perspective darden has always cared 4:30about that and the lacrosse institute similarly believes that these ethical AI 4:36related decisions are not just one person's responsibility the techn you know the technical lead of an 4:43organization but it is all business leaders that have to have front and center some of those ethical 4:49considerations so the same mission of the school gets reflected in the mission of the AI 4:55institute as well and it is to ask hard questions about responsible deployment 5:01of AI in business which could include bias misinformation deep fakes or even income 5:07inequality so I do some work with uh women's empowerment digital empowerment 5:12in India and you can see that uh there's a lot of bias and misinformation online 5:17on WhatsApp for example there are forwards on WhatsApp that ask women to 5:23give their jewelry and they will be doubled in 2 weeks or uh send money to a 5:29certain number to get a free tractor so this is something like another version of the Nigerian prince who keeps 5:35emailing us yeah don't uh reply to that prince uh so you can see that without AI 5:43there is still misinformation and AI just puts this on supercharge right it 5:50just exacerbates this uh whole issue of bias and misinformation and add to this there are 5:57other challenges about uh AI in business that are good as well as uh bad and 6:05risks with deploying AI in business the mission of the institute is to 6:11understand how you need to balance the good with the bad and also how we can 6:17use this once in a generation opportunity to reassess the role of 6:22technology in business and society and how we might align the financial and moral incentives to really create a 6:30brighter future yeah I guess that kind of question is why we're here today 6:36right it's President Ryan's charge to all of us in the university community to 6:43not only be great but also to be good and really good at being good and uh the 6:50mission of the of your institute is like a perfect example of that you want to be 6:56great in business you want to support economic growth but you have to be doing 7:01it in a benevolent way the moral and how do you balance those goals that could be 7:09pointing in different directions so I guess that's the biggest question of all 7:14i think it was the fall of 2022 when open AI released chat GPT when it was 7:23clear to the world that we've entered the world of AI it's one thing to have Siri that you can talk to on your phone 7:30uh but chatpt entirely changed and made this issue 7:35front and center that's exactly the point so if you remember and you go back to that time there are many questions 7:41that arise with that kind of power you alluded to this in your introduction but with great power comes great 7:48responsibility if I can talk and quote Spider-Man but it's the understanding the um opportunity and also some of that 7:56fear of the future that encourages us to think more deeply about the set of responsibilities that business leaders 8:02have we have responsibilities all along when we're developing the technology 8:08when we're deciding to provide the technology as available to the to the masses we as business leaders or 8:15educating business leaders we believe that this should be a conversation at the table how do the tech how does the 8:21tech sector work together handinand with society to ensure that we benefit from 8:28all the great accomplishments that we can uh uh strive for these days but also do so in a responsible manner it was it 8:36was a havoc or we all had to kind of reorganize ourselves to consider how 8:41we're going to deal with chat GPT when it was launched there was a little bit of a moment of panic um uh but 8:48potentially we could have seen that coming and potentially worked with the business leaders to do that in a more seamless manner but now that we have 8:55that in mind organizations are trying to to take the steps and we're hoping to develop methods and frameworks for them 9:01to do it in a more responsible way we can also think about this in the terms of short-term and long-term you 9:08can see with the launch of chat GPT is it made sense for the company but there 9:13are also longerterm issues that are uncertain that they're trying to uh balance in terms of the growth of this 9:20and what effect this has on society right and that's what we want our students to understand as well and to 9:26have this essential practice of having value based conversations that are 9:32necessary for society and for the workplace to thrive right so you can think of social media firms trying to 9:39maximize short-term user engagement because that's going to drive their advertising revenue but we all know the 9:46long-term consequences on society and brand reputation that comes with just 9:53maximizing user engagement so the essential skill for our students and for 9:59managers these days in the role of AI is acknowledging our flaws as humans and 10:07the challenge we have when making decisions with uncertain data and 10:12uncertain futures and the way we approach this is also to consider what has happened in 10:18the past and what can we learn from it we have certainly seen the wave of the internet 10:25and the slowness in understanding its all its consequences and the slowness in 10:32regulation has had some challenges and if you go even further back when the printing press was first 10:39launched we first got the Salem witch hunt before we got the scientific 10:45revolution so it's important to have this multid-disciplinary view where you're looking at stakeholders different 10:53stakeholders industry leaders you're bringing in philosophers and students together uh to think about this in a 11:01more multid-disciplinary enterprise perspective to follow up on your comment 11:06about the Salem witch hunt those were very difficult times although that's certainly not our era we can totally 11:13understand how that time would have been very frightening but I just think about 11:19the events of the last few years and certainly the events over the last few months where uh there's so much turmoil 11:26whether it's politics or the fear for the future and the environment it seems 11:31to be that like there's catastrophes happening in you know on the planet in universities everywhere you turn the 11:38news seems to be bad so Yael and Raj I want to know with your institute and 11:45your expertise you know what do you have to offer us that's uplifting ah great question and I uh I really 11:52cherish the opportunity to talk to this i think for me it's grounded in curiosity part of why I was drawn to 11:58academia to begin with and part of why I enjoy being part of this institution and watching AI evolve is that it fuels our 12:06curiosity and curiosity can take us in many different ways and that's you know as we react to fear if we're curious we 12:14can try to think about frightening situations in many different ways and with the evolution in AI and 12:20demonstrating things that are possible that we may have never imagined are possible it's an invitation to start to 12:28think about what could be and is a scary future in perhaps a different way it's 12:33an invitation to learn to learn about something new new technology to imagine ways in which the technology gets 12:40infused into our day-to-day in new ways that we haven't conceptualized and we're pushed maybe from a place of fear a 12:47little bit of fear but hopefully that fear is uh combined with some uh uh not 12:53only willingness to learn and willingness to discuss and converse but also a curiosity to try and overcome and 13:00to say we can only rely on ourselves or we must rely on ourselves to push ourselves to find solutions to a better 13:07future take any one of the challenges that you mentioned and there are questions related to AI that we can 13:13foster related to the environment related to policy definitely related to misinformation that you all were talking 13:18about before related to lifestyle choices and related to to our health and 13:23related to everything that we do there could be an AI uh complement or discovery that can push us to come up 13:30with better solutions and that's what we hope that our institution will foster and also adding to that is the ability 13:38to get this positive future and the bright future also relies on having 13:44strong structures and processes and regulations in place and that's where 13:49with our institute and with um the UVA community broadly we're looking at how 13:57to better foster the benefits and the positives from AI and what kind of uh 14:05systems and processes and structures are needed in business and in society to 14:10realize this better future so thank you Raj Yel you know it's it's wonderful 14:15that uh the lacrosse institute is set it set up at a time where it seems like the 14:22university's leaders actually anticipated this need right you don't right it takes time to establish an 14:28institute so that the timing has worked out very well for us I genuinely and 14:34deeply hope that the work allows us to um uh 14:40restore benevolence greatness and goodness just just like President Ryan 14:46expects of us so I bet there's many activities that you're launching right 14:51so on the one hand you have faculty conducting research and interfacing with maybe the business community but there's 14:58a flip side how can others get involved yeah great question and um and uh really 15:03to your point the the university leaders President Ryan our dean Scott Beardsley together with Dave um and Kathy Lacrosse 15:11that have seen some of this bubbling up in practice dave is a runs you know ran 15:16businesses in his past um they really had the insight that this institute should bring together people this is a 15:23convening uh institute we organize conferences we have events and seminars we encourage people to come to talk and 15:29interact with one another um across the various disciplines across the various sectors um our recent conference that um 15:37I'll let Raj tell you more about since he was the the the chair of the conference uh was a great example in 15:43December where we brought together uh 200 people but it was really from 15:48students to faculty to industry leaders to share and to uh together paint a 15:53picture for what is needed in the future and the conference uh that we organized uh in December was uh the value chain of 16:01ethical AI and it was uh it had more than 200 people from all around UVA and 16:07Charlottesville who were interested in AI and who were some of the speakers uh so some of the speakers you can we had 16:14Madur Bell who was uh from engineering who at the Cavlar racing he talked Yeah 16:20we had Madur on the show he was fantastic we had uh Kristen Martin who is a ethicist who was a PhD student from 16:27UVA who's now a chat professor at Notre Dame who came uh to the conference we 16:33had Tom Davenport who's a thought leader in AI in business who spoke at the event 16:39as well and and Raj was a speaker as well oh yeah too humble to mention himself as an expert in AI and marketing 16:45thank you and uh we had a lot of our faculty who were conducting research uh 16:52in AI across the university from the Baton School from 16:57McIntyre uh from engineering and from the sciences and Darden and it was a great 17:05event and I think it was great to see so many people who are interested in AI 17:10around the university come in and discuss this topic and we also 17:16termed it value chain of AI because if we we had these listening sessions across the university to see what was 17:23happening in AI what were people interested in AI and it turned out that 17:30just like any university we are doing an eclectic collection of topics and what 17:36really brings it together is to think of AI as a supply chain from the point of 17:41raw materials which is the data to bigger societal issues like ethics and 17:46regulations so this value chain kind of connects all the different things that 17:52you can see happening around the university and we've continued that with 17:57uh we just had our uh fair symposium where we uh award grants to two uh 18:05research teams and we had more than 19 submissions and it was a really difficult decision to select and these 18:13were really multiddisciplinary uh teams so what are the two projects going to be about uh so the two projects one is 18:21around uh better algorithms for uh privacy protection in AI and another is 18:27better algorithms for uh reducing bias in AI and ensuring AI 18:35is effective for every subgroup in the population and the collaborators were 18:42from Darden and from the school of data science and from computer engineering 18:48ing so really a multi-disiplinary set of uh scholars who've come together and 18:54that's something we're really looking forward to building this community of uh connections across the university and we 19:02will continue in the spring with seminars that are open to anybody who wants to get involved to come and attend 19:08to talk about current topics such as uh deepseek and everything else that we see in the news if they're curious if they 19:14want to hear a multid-disciplinary perspective they should join our mailing list and kind of uh join the events 19:20definitely make sure I'm on that mailing list uh AI is actually everywhere probably will won't come as a surprise 19:27that mathematicians and data scientists we're almost at the birth of AI so I'm 19:32part of a group called Frontier Math uh we have been charged with um testing the 19:38limits of the mathematical capabilities of the various large language models and 19:44the advances just since October is is mind-boggling last summer we were 19:52focusing on the question as to whether uh Chachi PT could earn um a medal at 19:59the International Math Olympiad which is an elite high school math test and by October we were already in a 20:07position where large language models could solve open PhD level problems that 20:13took uh new PhDs 7 years to complete at Ivy League 20:19institutions and the the rate at which the AI landscape is changing everywhere 20:27is amazing so in fact even the hypothesis that uh large language model 20:34trainings will be a huge percentage of the expected power resources even that 20:40has now come into question with with deepseek um and not to mention the political ramifications what 85% of the 20:47chips made in the world come out of Taiwan and we're supposed to be very frightened about that but maybe by July 20:53we'll discover that uh there'll be new discoveries and smarter training uh that 20:59will completely change our world view there's no such thing as um a status quo 21:06right now and I and I think it's for this reason it's really important for us to have an institute where people are 21:12literally on top keeping their finger on the pulse of an incredibly rapidly 21:18changing environment and it's not only the technology as you mentioned the the rapid evolution of the models themselves 21:24and their capabilities i think what uh we're all watching and studying is the 21:30rapid uh manner with which um companies are able to start utilizing some of 21:36these technologies they seem that they become available pretty readily and uh 21:41you you open a browser and every day there is something new that is embedding another aspect of AI and it's that rapid 21:49change we're not used to that and that is where especially one of the roles where our institute can play a role 21:56because it is around how businesses embrace new technology how we teach our leaders to be agile how do we think 22:02about change how do we think about changing with our populations which is uh really timely Now I would be remiss 22:09if I didn't spend a little bit of time talking about the two of you right we have a large faculty but the two of you 22:15you were selected to be these leaders and so I'm wondering what led you both 22:21and and Raj maybe I'll start with you Raj what led you to not only end up in 22:28this role but end up in a position where you think about AI I'd be surprised if 22:3310 years ago when you were uh early in your career that you thinking about ethics and so on and so forth there must 22:39be a path here well I have been thinking about AI for a long time now okay then I 22:45take that back i stand corrected when I was a computer engineer at the 22:50University of Madras in the late '9s I'm aging myself here that's when you know 22:56it was the stage of the AI winter we were thinking about neural networks and genetic algorithms at that time and this 23:03was the time of Y2K and I came to the United States as a graduate student at 23:09the University of Houston uh and started working in the business school for an 23:15MBA program there and realized at the time uh 23:20that the things that I learned in computer engineering are getting used in business and we had uh there there were 23:27some nice little applications that could be done using computer science in 23:33business and It also taught me that there is a purpose for all these algorithms that I was learning and I 23:41really liked about humans and psychology and marketing seemed to be the right spot for all of that and the first uh 23:49I'm probably one of the weird ones where the first economics class where I learned about supply and demand the 23:55world started making sense to me right and I said I need to be in this and learn more and so I've always pursued 24:02this side and I've worked on how technology affects marketing and from 24:07the time of CRM systems to uh mobile marketing to now AI the evolution has 24:15which has moved with different technologies but always applying it in the context of marketing well I uh so I 24:22graduated with a around the same time uh that Raj described kind of the mid to 24:27late 90s i graduated with a degree in industrial engineering from Bengurion University in Israel and I um following 24:34my degree I specialized in information systems i studied you know different coding languages i studied uh different 24:40artificial intelligence at the time systems like Lisp um and the like um and I really enjoyed that and I went to 24:47California and lived in San Francisco for a while selling um as you mentioned Raj enterprise resource planning systems 24:53and working in sales engineering uh with organizations to think about their processes and how they leverage 24:59technology and their working processes it led me to want to go back to school i was uh always very driven to be back in 25:06academia and um really my uh thesis that I identified from working uh in the in 25:12the private sector was around how decisions get made at the at the higher level of an organization when it comes 25:19to do with planning their resources forecasting their needs and making these decisions um which is really where I 25:26stand today a lot of my work is in the intersection of forecasting and and decision-m and that's where the AI 25:32starts to creep in forecasting when we talk about whether experts are doing the forecasting or we talk about systems 25:38doing the forecasting um that's what AI is doing in many instances is predictions t different prediction tasks 25:45even large language models at the heart of them have prediction in them how do we predict the the next word the next 25:50sentence the next uh uh paragraph and the like um and so uh that led me to 25:56where I am today where I study a lot um to do with decision-m and forecasting thank you when I hear you described 26:02large language models and AI as wonderful and rapid predictors right ai 26:08on the one hand uh offers essentially almost immediate instantaneous access to 26:15the accumulation of human knowledge one that's already scary but then also combining that with the ability to make 26:22strong predictions is a wonderful thing but one of the reasons why we have to worry about the implementation of these 26:28tools is that predictions aren't always correct we don't want to get led into 26:35the abyss because a large language model makes a prediction that we that is either inaccurate or flawed based on 26:43inappropriate assumptions and it's not only assumption ai models uh rely on data they they were trained on data 26:50forecasting very often relies on data um and so how does that data inform the our 26:56predictions about the future and then the the actions that we take um it's true that we see this uh pretty clearly 27:02with AI um however working with a lot of expert forecasters individual human 27:09expert forecasters human judges uh when we predict the future we often rely on 27:15our own intuition our own data the data that is in us that we've trained our forecasting capabilities on and so we 27:22see it and we've been talking about it in the context of human forecasters how do you kind of think about your biases 27:28are aware of them and how do you make predictions for the future knowing what you know about your your past right i I 27:34I want to dig in a little bit here Yael right so let's not be a professor for a moment let's be a marketer let's give us 27:40an example of this work at play real world example that we all get uh yeah 27:46from of my research real world example so I have an a project that will maybe bring some to life something to life or 27:52related to uh expert judges uh forecasting the future i've worked with 27:57um uh an NFL team that collects scout assessments of their players um which 28:02you can think about as uh forecasts actually um it's a form of forecasting i'm forecasting as a scout the success 28:09of a future player and um I've worked myself and and my co-authors have worked 28:15with this uh team to think about how you take those assessments and how you actually quantify them to understand 28:22more about the uncertainty that is expressed through this assessment how do you combine multiple if I have five 28:28scouts assessing one uh specific player how do I combine their opinions when their opinions are not always uh 28:34quantitatively provided up front so the forecasts that we are gi are given are 28:40somewhat abstract on a scale on more of a categorical scale and you can think about this problem in many different 28:45contexts it's the fact that we studied it in in an NFL and scouting context is fun but it um and of course it's a 28:53domain with a lot of money that gets spent on choosing these players uh adequately but any hiring decision any 29:00tenure decision in academia any um assessment of uh scientific proposals or 29:06grants or innovations they all go through a process where multiple individuals express an opinion on 29:12somewhat of a of a qualitative scale and the decision maker is left to try and 29:17make sense out of it um and our our work helps provide a framework for doing that 29:23raj same question so have you heard of toys artist it was once a great toy 29:28store once a great toy store and have you gone to toys.com 29:34never because uh well that's the reason why they don't exist anymore uh so my 29:41research which I worked on uh showed that firms should appreciate customers 29:48who shop across all their channels the store website the mobile app and firms should 29:56make all channels available to consumers not just try to push consumers to just 30:02stay in the store but to keep them engaged for as long as possible and so 30:08we showed how multi-channel shoppers as we term them are more profitable and 30:14stay longer with the firm than shoppers who come to only one channel but the 30:19idea with stores is also that you are going online and shopping in the same 30:25store if you are using the store's app or if you are going to the store in 30:32different occasions you're choosing that store over other options that were possible so you're showing more 30:39loyalty and you're using that store for different things the app could be for 30:45repeat purchases the store could be just for exploration and the website could be for 30:52comparison comparing products and their features so this so people who are using 30:58the product across different channels are really the ones who are bought into the products and are uh valuable of 31:06course this is an average effect it can change by industry could be that airlines you're normally shopping it in 31:13the in the app but you can also see I fly Delta a lot but nowadays I just book 31:19on the app i'm not going to the website anymore or I look in Google flights and 31:26then find where Delta is Delta good option for the flight and then I open my 31:31app and book straight in the app it's much more convenient so that's kind of what we did at the time is to embrace 31:38different options and make all options available to consumers the same way with AI now a lot of companies are 31:46implementing AI and they're starting to see benefits in the back office in terms 31:52of improving the processes all the examples that are coming in now are which makes sense is efficiency 31:59improvements but you'll start seeing the ones that embrace this and have AI that 32:05is customerf facing really allowing the AI to be the chatbot of your company 32:12firms that are embracing new technology and enabling uh these technologies for 32:17their consumers are going to cultivate better customer relationships and going 32:22to be stronger brands in the future so thank you Yael and Raj this has been great conversation time is running short 32:29here but I do have one last question before we begin to wrap up and uh the 32:36fact that we are talking about AI won't be surprising to anyone turn on the news 32:42everywhere you turn someone's talking about AI but I think it's important to point out that here at the University of 32:47Virginia it's not just talk because everyone's supposed to talk about it can 32:52you comment on that yes uh together with uh Scott Actton I'm uh really honored to serve as one of uh the provost's 32:59adviserss on AI and we work closely with um his office and uh the individuals 33:04there um there are investments made across grounds that are really exciting 33:10professorships that are being offered uh to coordinate the efforts across the different schools themes that are being 33:16identified for instance AI in in the medical space and in the healthc care uh uh domain uh the school of data science 33:24engineering uh uh the college so there are serious coordinated efforts from the 33:30uh from from the provost office uh to encourage different ways to access AI 33:36and to get involved and we're really um ahead of the curve in many ways by coordinating these efforts and putting 33:42together an AI at UVA um uh umbrella uh 33:48organization that will allow folks to join and flourish so our time is up but 33:54I think it's important to talk a little bit about both of you as non-academics one fun fact Raj tell us something about 34:01yourself well I'm uh I'm very active in tennis after uh the co especially so I 34:08play tennis a lot in fact I have a game today i'm not so sure i think today is 34:13going to be a tough match and are you a fan of the UVA tennis team i think they're pretty We had a couple people in 34:18the Olympics oh yes i think UVA tennis is the best i mean it is the best 34:25experience to see such great players and you saw the women's tennis i mean Emma Navaro did very well with the recent 34:33Australian Open and Dan and Collins has been great and uh I think the men's too 34:38are doing very well the team is poised to be doing well they beat Texas just 34:44over this weekend uh which was a great uh game so love of uh UVA tennis go yael 34:51fun fact i do love uh running so I'm a jogger um I like um uh watching 34:58televisions and movies i'm kneedeep into everything that is uh current and I like to watch all the Oscar movies before the 35:04uh before the Oscar ceremony a tip do you have a tip for us i mean there are so this year is extraordinary really the 35:11quality of the movies is really really high and the girl power in the movies is impressive so lots of female strong 35:17female roles which is really exciting to see um and I have a little fetish for shoes that I'm known for so um like oh 35:24my gosh I do see Oh whoa what do you call those shoes platform pink and white 35:29plaid shoes I would say yeah well it's been a pleasure to have both of you thank you for being on Who's in STEM all 35:35of us wish the Lacrosse Institute godspeed in its important and vital mission please do your best to help keep 35:43AI benevolent i think there's no doubt it will be a tool to streamline and um 35:50make what seems impossible humanly impossible possible and well it should go without 35:58saying but I I have to say it anyway you are fulfilling President Ryan's mission to be both great and good so thank you 36:05for your service and I'm Ken Ono STEM adviser to the Provost and the Marvin Rosenlum Professor of Mathematics and 36:12you've been listening to who's in STEM who's in STEM is a production of WTJU 91.1 FM and the office of the provost at 36:19the University of Virginia who's in STEM is produced by Katherine Casabum Claire Keren Benjamin Larson Mary Gardner McGee 36:26Katie Nichols and Ria Verma our music is composed and performed by Robert Schneider and John Ferguson of Apples 36:32and Stereo follow us on Facebook Instagram and Twitter listen and subscribe to Who's in STEM on Apple 36:38Podcast Spotify or wherever you get your podcasts we'll be back soon with another 36:43conversation about scientific and technological innovation at the university