AI, Well-Being and the Human Connection

View Transcript

Artificial intelligence is changing healthcare — but what does that mean for the people who deliver and receive care? Cornelius James, M.D., FACP, FNAP, a clinical assistant professor and primary care physician at for U-M Departments of Internal Medicine and Pediatrics, joins Chief Well-Being Officer Elizabeth Harry, M.D., to explore AI literacy, clinical judgment, governance, education and well-being.

They discuss how healthcare can embrace new technologies while protecting human connection, empathy, wisdom and the meaningful moments that define care.

Episode guest:

Cornelius James, M.D., FACP, FNAP

Learn more about how the Office of Well-Being aids it team members and brings a culture of belonging, resilience and support. 

Transcript

Elizabeth Harry:

Welcome. I'm Dr. Liz Harry, and this is the Well-Being at Michigan Medicine Podcast. Welcome to our discussion on AI in healthcare. I'm thrilled today to introduce Dr. Cornelius James, assistant professor in internal medicine, pediatrics and learning health sciences here at the University of Michigan. Dr. James is not only a practicing primary care, but also a leader in AI and machine learning education and implementation in clinical practice. He heads the DATA-MD team, which is preparing clinicians for the AI-powered future and developing web-based curricula to bring interprofessional perspectives, including patients, into the conversation.

As AI tools become deeply embedded in healthcare, recent reports like the JAMA Summit Report on AI, which came out in October of 2025, highlight our urgent need for stronger systems in evaluation, monitoring, governance, infrastructure, and incentives. At the same time, there are real world risks, like a London School of Economics analysis showing a Gemma generated summary in social care may downplay women's health needs and research in PNAS warning that large language models often misestimate wellbeing in underrepresented contexts.

Today, we'll dive deep into these challenges, what it takes to become an AI literate clinician, how we should rethink medical training, the incentive shaping health system choices and the practical tools and guardrails needed to ensure AI delivers real benefit without unintended harms. Welcome, Dr. James. Can I call you Cornelius?

Cornelius James:

Of course, only if I can call you Liz.

Elizabeth Harry:

Yes, yes, absolutely.

Cornelius James:

Okay, great. I really appreciate the introduction, Liz, and I'm very excited to be here and honored to be here.

Elizabeth Harry:

Oh, I'm so excited. This is just a great topic. It feels like it's top of mind. You can't get on anything these days without seeing a post about AI or a conversation about it. And so as we think about, one of the big parts of your role that I think is so cool is really thinking about how we build AI literate clinicians, how we train the current generation, the next generation of clinicians, so that they know how to interact with this technology in a really effective way. So first off, what does that mean and what should they be able to do next week?

Cornelius James:

Absolutely. Really appreciate the question. It's challenging because, as medical educators, as clinicians, and I say more specifically as medical educators, it's challenging because these tools are already out there. They're already in the wild. So in some respects, students, learners, et cetera, are using these tools, unfortunately, without very much training. And that's okay. And I'm sure that we have a lot to learn from those that are currently using the tools, but there's also some risk associated with taking that approach.

So we usually see people using generative AI or using large language models, OpenEvidence, et cetera, to inform clinical decisions and so on. And that's okay as long as there's a healthy level of skepticism or a person being inclined to always ensure that the outputs that they're receiving from these models are accurate. So that's one part of things.

But there's other AI out there. And unfortunately, I think generative AI, really cool, really exciting, it's important, but it's not the only type of AI that's out there. Generative AI comes from a long history of work that's been done on other types of AI models, and those are tools that we're going to have to interact with as well as clinicians. And we have to start preparing folks to interact with certainly generative AI. That's going to require learning how to collaborate or interact with the tool, so to speak.

But when it comes to those other models, there's other things that have to be considered. And some of it'll overlap with what we do for generative AI. And one thing I'll say is I usually say when it comes to all of these models, you're not going to have to be a developer, you're not going to have to be a world-class computer scientist or engineer or anything like that, but I often compare it to a randomized controlled trial.

So if you're going to implement or use whatever is being studied in a randomized controlled trial, as a clinician, you should be able to read a randomized controlled trial, determine whether or not there's a high risk of bias, internal validity, external validity, generalizability, et cetera. Being able to do that, in my opinion, is going to be very important, meaning from a randomized control trial from a just clinical intervention perspective. But then when it comes to AI, yeah, again, similarly, you're not going to have to develop a model, but to understand where a model's data comes from, who developed it, why was it developed, et cetera, to ask those types of questions, that's what clinicians are going to have to be able to do so that they can effectively integrate the outputs of these tools into their clinical decision making.

Elizabeth Harry:

And so you mentioned large language models and generative AI as sort of one category. Give us an example of something else that you see in a different category that you think might be on the horizon for clinicians to start thinking about how they would engage with it.

Cornelius James:

Interesting. So I can give you one that's not even on the horizon, but that is actually out there in clinical practice sort of in the wild. So there's the Epic Sepsis Model as an example or deterioration models, those are widely deployed, available. So the Epic Sepsis Model, you have a patient that's hospitalized or one of your patients is hospitalized, you get a risk score that pops up saying that this person is at risk of sepsis. Well, what does that mean for my particular patient? How do I interpret that? How do I be the human in the loop to say, well, that doesn't quite make sense for my patient, or I just saw Mr. Jones, I just reviewed his history, that doesn't quite fit with this picture? Or you know what? I didn't think about that particular point, so maybe I do need to think a little bit more about that or monitor this value a little bit more.

Because we can go in either direction. We can say, okay, I'm going to either automation bias, okay, person's at risk for sepsis, I'm going to start antibiotics, send this person to the ICU. Or I'm going to be a bit more nuanced, and say, I know my patient well, I feel confident in my clinical experience, my team's experience, the input that I'm giving, let's monitor this a little bit and then we can make decisions from there. So that's one example in an inpatient setting.

In an outpatient setting, diabetic retinopathy. So we know that was the first autonomous FDA approved model to screen for diabetic retinopathy that is again out there. And if a person has a high risk or the model suggests that there's a high risk of diabetic retinopathy, do you just refer? Is there, again, that automation bias, or is there that under reliance where you're not trusting it for a given reason? And that's where background, experience, reading, understanding of how these tools work is really important. And those two models, the retinopathy model, that's a deep learning model. It's not generative AI. The other models that I mentioned are predictive models that are not generative AI.

Elizabeth Harry:

What I love about your analogy that you're tying it to being able to understand a randomized control trial is that, as a clinician, everyone doesn't need to have the skills to run a randomized control trial, but it reminds me of that study, parachutes don't decrease the risk of mortality when jumping outside of a plane. But then if you look at the methods, they had the people jumping off the wing on a grounded plane. And the importance is that you have to know how to read a study because the click bait can be really misleading if you don't have that skill.

And what I'm really hearing you say is AI is the same. That if you don't understand some of the background, some of how the sausage is made, if you will, then you might get in this very sort of automated habit that could put your patients at risk because you're over testing or overprescribing or alternatively not having that nuanced skill. And so some terms that I hear thrown around when we talk about medical education and our learners, particularly around AI, is this sort of skilling, de-skilling, never skilling. In the AI era, what do these terms mean to you? And when you think of them, what should we as educators be thinking about? What should our learners be thinking about? And do they apply to more than just our learners too?

Cornelius James:

Appreciate that question. It's something that I've been thinking about for a while, and my understanding or thoughts about this have sort of evolved. I will say broadly, there will be a necessary de-skilling that's going to be important. And I believe that that's something that's very challenging to think about, to come to grips with.

Elizabeth Harry:

And what does that mean, a necessary de-skilling?

Cornelius James:

That there will be some things that are going to... We don't need to know the Krebs cycle anymore as an example. So hopefully medical students out there are clapping and very happy because I'm saying things like that will probably no longer be necessary.

But at the same time, we are sort of trying to skate to where the puck is going in that we know what these tools in many respects are capable of, but we don't know what it's going to look like in real world clinical practice across the board. So the necessary de-skilling will involve those things that we absolutely should stop teaching our learners now, right now. And in my opinion, there are things that are out there like that. I mentioned the Krebs cycle kind of jokingly, but there are other things out there that I think they're sort of those fatted calves as some would call them where we don't really want to sacrifice those things because of various reasons, whether it is because of professional identity or other things, it's understandable and I want to be sensitive to that, understanding that, but it's going to be important if we ultimately want to take good care of our patients.

And then there's going to be harmful de-skilling where we should not. There were some things, some blind spots, so to speak, that we didn't sort of account for that's going to... And I would say that, it's as important, but that's what we definitely want to make sure that we avoid. And the same is true for never skilling. Never skilling, it could be either harmful or good, again, in that there are some things that we should never teach students again, but then we also want to make sure that we are teaching things that definitely need to be taught and that we avoid that harmful never skilling.

So it's not going to be easy. Christy Boscardin, Brian Gin, and Raja-Elie Abdulnour wrote a great paper in NEJM. It was an editorial describing what this could potentially look like. And they were very thoughtful in thinking, yes, there are some things where we have to think about sort of where learners are in their trajectory where we do say, make sure that they have been skilled here so that we can now start to prepare them or allow them to interact with these tools just so that we can make sure that they have that foundation that's going to be necessary if there's a failure of technologies and so on and so on.

But the failure of technologies, I get it, but I also caution against just saying we have to always teach these things because you never know if technology's going to break down. I often give the example of, well, I don't live right around the corner from my clinical office because I'm concerned my car is going to break down or I didn't learn to ride a horse so that I can make sure that I can continue to get to work. I'm going to make decisions that are ultimately, yeah, there's some risk involved with it, but ultimately it's going to, in the grand scheme of things, serve me, my family, et cetera, well.

Elizabeth Harry:

And so there's this tension of this idea of do we have to sort of have learners hold off on using these tools until there's kind of a baseline? I think I've even seen this graph and I can't think of, it was in a talk I saw, so I can't cite the paper, but maybe it's the one you're describing, where if you're at this certain knowledge level, then it is an accelerant.

And I've felt that way in my clinical care. I feel very confident that using some of these resources, my patients are getting much better care with the questions I'm able to ask and the things I'm able to check. And this idea that if you have not hit this sort of level of competency, then you could fall into this never skilling risk. What are your thoughts about that concept?

Cornelius James:

Yeah, absolutely. And we are certainly talking about the same paper. And I do believe that a lot of that is important. And there has to be a trust between students and/or learners or residents and house officers, fellows, there has to be a trust so that they'll believe that we're doing what's best for them because they're smart. They're super smart human beings. And eventually the resident or the medical student is going to say, "I could spend so much more time with my patients. I could look them in the eye more. I could be more well or have better wellbeing if I didn't have to be so concerned about writing this note."

So that, in my opinion, it's one of those things where I'm not suggesting that we move in one direction versus another, but those are things that we sort of have to weigh. Yes, I understand that there's literature suggesting that writing or putting pen to paper or typing, it's helpful for developing clinical reasoning and formatting plans and truly understanding the patient. But what's the balance? What's the trade off when it comes to learner wellness and their engagement, and also preparing them for what the real world is going to actually be like?

Elizabeth Harry:

Yeah. I love this. An analogous thing that I've spent some time thinking about is we teach people to present patients. I haven't presented a patient since I left residency. When was the last time you had to present a patient? And yet we spend so much in internal medicine, like half our day, training people to do a skill that they will never ever do again in the real world.

And I find it so fascinating some of the historic, and we could talk, it's a whole other discussion about why do we do presentations, and where did that come from and why do we still do them in the way that we do them? But I think this idea, and I love your analogy about the horse and the car and where you live, this idea of are we sort of insisting, is it like cursive, are we insisting that they learn something that they're not ever going to use, or is there some mandatory minimum threshold?

Cornelius James:

Yeah, I agree. And unfortunately, I don't believe we have the answers to all of those questions. But I've also stated in the past, we're trying to sort of skate to where the puck is going, but moving in that direction, because this is a big, huge enterprise, certainly here at Michigan Medicine, but healthcare in general, medical education is huge. And to get it to shift or pivot, that's like moving the Earth or turning the Earth. So getting that to happen is really challenging. But I say that because it's going to take bold initiatives, in my opinion, to get us to where we need to be when it comes to engaging with these technologies.

This is our Flexnerian moment, so to speak. So Flexner wrote that report back in 1910 or so, commissioned by, I believe the AMA and the Carnegie Foundation, but wrote that report because medical education was in a horrible state at that point. You barely had to even have a college degree to be a doctor. So I don't agree with everything that happened in the Flexner report because there was some racist things that happened there too. But it also laid the foundation for things like the biomedical model, which we still use, and also ensuring that learners are able to engage in the clinical environment because that's what they're going to be doing. That was a monumental shift for medical education and for healthcare at that time. I believe we are in a similar moment right now with AI where we can sort of ignore it, but it's happening for sure.

Elizabeth Harry:

Yeah. And it's interesting because there's people kind of all over the spectrum on that. There's people that are like, "Oh, maybe it's a bubble." It feels like there's no way this could be a bubble. And then there is dialogue of the world will never be the same. And really our ability to deliver effective care, sustain our profession. When I think about wellbeing, I think about people being able to come and do the job that they meant to do that gives them meaning and purpose, care for others, in a way that doesn't detract from their own ability to also care for themselves. And see this as an opportunity to facilitate that.

Cornelius James:

100%. 100%. The challenge though, in my opinion, is going to be the care and what that looks like. I suspect that what it means to be a clinician, to be a healthcare provider, is going to be a bit different. It's no longer going to be encyclopedic knowledge. It's going to be important for us to have these facts to move science forward, et cetera, to know, to have great medical knowledge. But to care, to team, to be empathic. Those are going to be changes where we're going to say that is a great clinician because they're able to communicate well, they're able to team well, they're able to collaborate, they're able to critically think, critically appraise.

Those are going to be things that are going to be really important relative to what's been emphasized in the past around memorization, rote knowledge. You know what I mean? And it's going to sort of change. I don't want to be hyperbolic here, but professional identity and who chooses to become a doctor, a nurse, a pharmacist, et cetera, this is going to have a major impact on that even, in my opinion.

Elizabeth Harry:

What I love about what you're saying is that, in some ways the technology puts the human connection back at the forefront.

Cornelius James:

100%, 100%. And that is what a lot of my work is, I try to gear it toward. I try to start a lot of the conversations that I have with the old Peabody quote, the care of the patient, the science of medicine and the art of medicine, not antagonistic, but complimentary. And enforcing and encouraging that clinician-patient relationship, making that stronger.

It pains me at times that I can't go to more patient funerals to be with their family. That there's not enough time for me to, when appropriate and when well received, to give a patient a hug, to tell them I'm praying for you, or let's pray together if that's what they want. Because I'm so busy with so many other things that are not really patient care related, like finishing a note or checking a in-basket message and so on and so on. I'm rambling now. I'm sorry.

Elizabeth Harry:

No, but it ties beautifully. I mean, we actually had Dr. Sanjay Saint on many episodes ago talking about the Sacred Moments work. We can put a link in the show notes to that particular episode, but that's what you're naming is the ability to create space for these sacred moments in healthcare. And then we had Dr. Vic Strecher on talking about purpose, and create more space for the things that bring purpose.

And I think that's better for everybody involved, ourselves as clinicians, our patients, ourselves as clinician patients, because we also have recently talked about that. And it sounds like, so there was this report in JAMA, which seemed to be kind of a big moment in trying to summarize where we're at. And my read of it is that it basically says what we're saying here. It's going to transform healthcare. But one of my takeaways from it is that maybe our evaluation and our oversight systems and our governance aren't ready. And I'm curious your take on that and where are you seeing that gap most critically, and what do we need to do to try to address it?

Cornelius James:

So again, appreciate that question. Governance oversight at both the national and even international levels, extremely important. Oversight at the local level, very important. Nicholson Price wrote a great paper in Nature describing this collaborative governance that's going to be necessary. I would suggest taking a look at that if you can. It was written in 2023 or so. But it's basically suggesting that the federal government should have a responsibility to say, these are the policies, these are the guardrails, these are the things that we are going to regulate, are not going to regulate.

And then you have local committees because they know their environment, they know their patients, their populations, their culture, et cetera. They're able to sort of say, let's take these practices, make sure that we're meeting them, but we have to also make sure we're tailoring what we're doing to our particular population. So the governance there is important.

The work that I do is a bit more meso macro in that I'm thinking a bit more about the oversight of the practicing frontline clinician. Because ultimately they're going to be the ones responsible for making decisions around these tools, and making sure that they are properly equipped to make those decisions, whether it's through initial training and education in medical school or residency or whatever professional school that a person's going through, but then also amongst teams. How does this change team dynamics now? How are these conversations taking place now that we've got a tool, perhaps we see a shared output from a model? Well, how do we all communicate or talk about that? What does the oversight of that technology or that particular output look like and how do we come together as a healthcare team to apply this to our patient to make sure that they're receiving the best care?

So I believe generally speaking, that middle level that I just mentioned, is probably where we're doing a bit better. And I know Michigan specifically is doing very well in that middle level with the local governance, the clinical intelligence committee, which I'm honored to sit on. Great. I'm there when we're vetting these tools and ensuring that they are what they should be, what they need to be, they're going to be helpful.

But it's that federal level that's challenging that we really can't do a lot about in some respects. I know there are things that we can do, but that's challenging. But then there's that micro level that is, I believe we need to pay a bit more... Those are probably going to be the harder levels to address that federal level and that micro level. In my opinion, those are going to be the biggest lifts, so to speak, when it comes to governance.

Elizabeth Harry:

And it seems like part of the skilling and also part of that micro level is really shifting our competencies or our capabilities from generating ourself, generating a note, or generating things, to sort of evaluating and reviewing, and the ability to be very discerning reviewers rather than sort of blank slate generators. Is that fair?

Cornelius James:

With proper balance. So in that summit, that JAMA Summit Report, they described this algorithmovigilance, which is great. Julia Adler-Milstein out at UCSF wrote a great paper about that as well. In that paper, she mentioned, yeah, clinicians are going to have to be vigilant, but we can get to a point where now it becomes overwhelming that they have to check everything. So we also don't want these technologies to be deployed or implemented in a way where clinicians are like, "I'm spending so much time reviewing the results and making sure that they add up. I'm trying to be vigilant, but now that's sort of taking the place of whatever I was doing before." So there's going to have to be that vigilance for sure, but how do we balance it is going to really be key.

Elizabeth Harry:

Yeah, It's interesting because even when we've looked at the ambient documentation data, people's self-reported burnout has declined and their perception of cognitive load appears to have declined, but this sort of work after work seems to vary. Some self-reported work after work or sort of perceived work home conflict seems to go down. But to me, it seems that there's a little bit of a mixed signal when you actually look at the data in Epic of how long are they spending that we're slightly shifting where the work is maybe because of this phenomenon you're talking about.

Cornelius James:

That's interesting. I guess as you were speaking, so I do some qualitative work, and it would actually be interesting because, and I'll speak for myself, if I'm doing work at home that I believe is meaningful, that I really believe is moving the needle and helping people, I guess I don't know that I'd mind as much. Now you'd have to ask my wife and kids how much they mind, but I don't know that I would mind that. So it would be interesting to know the quality of that work that's happening at home. Do you feel that improvement when it comes to burnout and wellness and wellbeing, but do you still have that same amount of time that you're working at home, but are you doing something that's meaningful? That's something that would be interesting to me.

Elizabeth Harry:

And do you have a sense of autonomy? So I always joke with my team because they do look at signal data and they do look at how are clinics doing in terms of this. And I always joke with them, don't look at my pajama time because I do almost all of my kind of clinical care, if I'm not actually in clinic after hours, just because of the way that my life is structured and my day job, if you will, takes up this part. And so then I do that, big change, it takes a lot of time. But I make that choice. And so it doesn't contribute to burnout for me because I can come home and have dinner with the kids and get them to bed and then look at the things. And that's okay with me because I feel a sense of autonomy around it. And so I think that question of autonomy and control comes into play there as well.

I think you've been teaching us a lot already. So we've talked about a little bit between what an LLM is and what generative AI is versus some of these deep learning models or kind of predictive models. Another area that you and I have messaged about and talked about a little bit is this idea of an open model versus a closed model. Could you teach us a little bit about what is the difference between those two things, both sort of technically and operationally?

Cornelius James:

Sure. So I'll use some brand names here if that's okay. So your open models are going to be things like your Geminis, your ChatGPTs, your Claudes. So those are huge models that are foundation models that have been pre-trained on large amounts of data. And then they've been fine-tuned for various purposes, image generation, language, maybe multiple things. So they are trained, they're pre-trained, fine-tuned, deployed, that's it. No one has access to what data they used, architecture, weights, et cetera. It is closed. If you want to access it or use it at your institution in a special, you can use an API, but you're not going to go in there and manipulate data, manipulate weights, et cetera, et cetera. So that's what closed models are.

Your open models are going to be, I'll use again some brand names here, your Gemmas or Llama, right? Or for a medical example, there's something called Meditron that has been trained or pre-trained on large amounts of medical data. So those models are actually pre-trained, and it's going to sort of vary or there's a range of openness, so to speak. So maybe one shares their data or their architecture or their weight, but maybe not all of that.

So those models are going to be a bit more flexible, a bit more transparent. Flexible in that, yes, we can take this model, and we can fine tune it in a way that we want to use it in our setting. And we know what's going on in many respects with this particular model because we've done the work to fine tune it in a way that's going to be beneficial for us. So it's flexible in that way, but it's again, also transparent that we know sort of what the... You could say that there's more perhaps, yeah, I'll just use the word transparency here when it comes to what's actually put into developing those types of models.

Elizabeth Harry:

And it seems to me that, as an individual, our frontline clinician either has access to what their organization puts forward, which an organization may engage with one of these more open models, or it seems like they have access to these sort of closed, large models. Is that fair?

Cornelius James:

Okay. So the proprietary model or the closed models, they're very smart because they want everyone to use them. So yes, you can have access to what your institution sort of endorses. Yes, we have Copilot in our Microsoft suite. So there's that. But generally, and this is one of the things that's amazing about AI, is that everybody has access to it, patients, et cetera. So yeah, you can have these closed models or access those, but on the other hand, everybody can access the open models and manipulate them if they have the compute power and so on, if they have the hardware and so on.

But they can take those and they can fine tune them in a way that's most appropriate for what they want to do. So you do have the institutional things, but people also have access to, there's a free version of just about all of them, so you can access them. Because again, people really want you to use them at this point. And in many cases that's using them so that if you're not careful, your data, your prompts, et cetera, that your input, your output, all of those things are being used to train or develop models. Whereas at Michigan, we know here that when it comes to many of the models that we're using, that is not true there. We're not using data and inputs and outputs, et cetera, to train the models that they've developed here at Michigan.

Elizabeth Harry:

There's so much richness in what you said. So one is sort of this great equalizer, that the accessibility, and acknowledging that the more open models, you might have to have some computing power or some hardware, and maybe some prior knowledge and how to do that weighting. But these closed models for sure are really the great normalizer. There are all these stories of people putting in their symptoms and GPT comes up with something. There are also scary stories. But this great equalizer of access to information, which seems to be really powerful, how do you think that plays into then how we think about our profession moving forward and we think about how we're training our learners?

Cornelius James:

So I often think about these different levels of input and output. So data basically just being something that has not been analyzed, it's just raw. You have data, and then you have information. And I think information is more so you do a Google search and it just brings up whatever is searched for or whatever comes up the most, so to speak, or whatever is most relevant in many respects. So there's information. And you have knowledge where there's someone that is sort of looking at the information and they're applying it to something effectively.

And I think that's where we're going to have to see, there's information that's going to be out there and large language models, in many respects, they do generate information, but it can also generate some knowledge. But I think a lot of it is going to depend upon the end user. So someone can interpret or look at information that's been generated, but whether or not they have the knowledge, and then next the wisdom to actually apply that information in an effective way, that's going to be really important.

Now when it comes to the different health professions and medicine, nursing, all of the other health professions that are out there, I think we're going to have to sort of think about that or have that in mind that are we dealing with data, information, knowledge? Where's the wisdom that's required? How do we train people? Because in my opinion, the wisdom in many respects is going to really be what sets us apart from these technologies because we have that life experience. We have that, I've felt pain, I've felt anguish, I've felt those things. I've seen other people or other folks. Now how do I take this information and take my knowledge and my wisdom and apply that in an effective way? That's what I think is going to be sort of next level when it comes to clinicians and how they engage with this. I feel like I did not answer your question.

Elizabeth Harry:

No, you totally did. And I think that it loops back to the previous point of that we have to train different competencies, and that may mean that we're self-selecting for people that are focused in different things. Because this gets a lot more into the humanism of medicine. And I love that idea of focusing on the wisdom.

Let's jump back a little bit to, you talked about guardrails, you talked about protection, and you really focus at the frontline clinician or clinical team. Tell us how you think about what the companies are doing to protect us, to protect our patients, and what we need to be doing either as organizations, as people are listening to this, or as individuals, as they're sort of engaging with these tools, what are important guardrails to keep ourselves and our patients safe?

Cornelius James:

That is a great and big question, and I'm probably more skeptical than most people. I believe that there are well-meaning developers and companies out there and people working for those companies. But I believe that, as a profession, medicine, or in healthcare generally, it is impossible for those companies to truly know what our patients need. So it's going to be imperative for us to partner with these developers and make sure that the right people are partnering with them and communicating with them so that the solutions or the technologies that are being developed for our patients are actually what our patients need. And not only our patients, but our clinicians as well, that they are actually going to allow them to do what they do better.

Because I believe that, minus that, unfortunately, and I don't want to suggest that there are some people's values that are better than others. I am not saying that, but it's just that sometimes they're not as well aligned as we'd like them to be. So that's why I think those conversations are going to have to happen because I don't suspect that developers are going to know all of the guardrails to put up. But it's going to be up to us as clinicians to have the vocabulary, to know the lingo. And not all of us, maybe we're just training folks to be able to do that a bit more effectively, although I do believe we all hold some responsibility there, but it's going to be very important that we are prepared to speak that language, to have a feedback loop to make sure that, and not only clinicians, patients as well, to make sure that the solutions that are being developed are appropriate.

So again, not sure if I answered your question, but I think the biggest or the best guardrail that we can sort of put up right now is going to be having clinicians involved. And I guess in my mind, I sort of heard the word, if I'm not mistaken, companies, et cetera. And when I think company, I guess I just think about the bottom line generally. And that's what folks tend to, not in all cases, not in all cases, but that tends to be a pretty big focus. Whereas in healthcare, and maybe I'm being too idealistic here, but I think our focus is on, and not that this isn't the case for companies, but it's ultimately on keeping people safe and saving lives and allowing people to be the best that they can possibly be in every way.

And the perspective that I believe we have as healthcare providers, as a profession, you have to live it. And you know better than I do, you have to live it in order to really understand what that means, what that looks like, the type of guardrails that are necessary, et cetera, et cetera.

Elizabeth Harry:

So if a clinician was listening to this, and was like, this is just too much and I'm worried that I don't understand it. It's a black box. I am nervous and I don't want it to hurt my patients, so I think I'll just not engage. What would you say to them?

Cornelius James:

First thing, there's certainly, I know that that is the case we have, that there are some that may be feeling that way. I think the EHR is a great example where folks are like, "This is just too much. I don't want to do this anymore." I think we learned some good lessons from the EHR, in that we recognized that clinicians had to be involved with the development or with the implementation of that.

But I believe it's going to be important for us to truly see the benefits with these technologies, to appreciate how it can actually make us and our patients better. If that is ultimately the focus, then I would hope that we would be excited about engaging with something that's new. And I acknowledge that it can be scary, but I also want to go back to that point that I made. You're not going to have to be a statistician. You're not going to have to be a trialist. You're not going to have to be a world-class researcher. You're going to have to just be able to be a bit adaptive, have those principles, those humanistic principles and those skills that I mentioned earlier, and be willing to change, and at least at this point be a little bit comfortable with being uncomfortable.

But I would say ultimately, I believe we're headed in the right direction. I know of lots of wonderful people locally and nationally that are doing this work and that are advocating for us to use these technologies in the right way. And if we don't, going back to the conversation or what we discussed related to the developers and companies, again, I believe that there are well-meaning companies out there, but if we don't engage, then that's when we can expect that... So Abraham Flexner was not a physician or a healthcare provider. But imagine this person coming in and telling people, I'm not saying that they can't think through this and say what it could and should look like, but he wasn't a physician, a nurse, a pharmacist, a physical therapist, a social worker, an MA, et cetera. He was an educator, but he wasn't a physician. But he was able to come in and say, "This is what it should look like."

I don't think it should happen that way with AI. If we don't have enough clinicians that are learning, and you don't have to be on Capitol Hill, but if we don't have enough people that are saying, "No, this is what it should be like, this is what's happening in real world clinical practice," then the companies, I believe, will dictate how we practice medicine. If it's not the companies, it'll be politicians or other people that'll dictate how we practice medicine. I hope that does not sound dire or overly scary.

Elizabeth Harry:

We've seen it before, right? I mean, if we don't advocate enough in reimbursements, et cetera, other things, those things are dictated for us. And it brings me to this sort of worry that I've had where I look at, so I have a paid subscription to one of these closed models that I use a lot. I don't put anything confidential in there or anything like that. I'm very thoughtful about what I put in there. But boy, has it transformed my ability to run my home and organize my life and made a lot of workflows really easy.

When I compare that to products that are available for some of our workplace use, it's clear to me that there are different tiers, and it depends on what we've paid for and it depends on what guardrails and restrictions we put around them. And it seems like there might be a risk of a two-tier wellbeing future, people that have access to good AI and that understand how to use it to make a digital twin, if you will, as people have spoken about, and then people that are using it like Google, sort of a glorified Google format. What are your thoughts about that and what needs to be put into place to prevent that?

Cornelius James:

Yeah. So it sounds like you're describing a digital divide, but just in a different way. As these technologies become ubiquitous, they're integrated more, I suspect, especially because these are, and this is challenge because we do have to be a bit specific about the type of AI, and I appreciate you being specific with large language models or the generative AI models.

Those are going to become general purpose technologies. And like I said, lots of people already have access to them. But I do think about people in rural communities with internet access or the signal may not be as robust. So those are things that we certainly have to address. But I do suspect that as these technologies become more sort of ingrained in our culture and society, I suspect that they will become less expensive. I suspect that they'll become better.

But my concern is not so much around that divide and that there are some with better, quote, "technologies." I really don't suspect that that's going to be the issue. I get far more concerned about these technologies becoming really good. And now there are some who are able to access a doctor, which is a premium, meaning a human doctor, but others can only access or are only accessing AI. That's what I get a bit more concerned about because these tools are going to become better, they're going to become smarter.

There was a bill, and even in that summit report, they did mention digital doctors, but then there was a bill introduced in Congress, I believe in 2025 or so, where a Congressman US proposed or put a bill forth that would allow states to determine if AI can prescribe medications. I know there were some responses to that from national organizations. And I get it. In my opinion, that's coming, but I do believe that the technologies have to be vetted and so on and so on, and there's going to have to be lots of policies around that. I believe it's coming.

But again, my concern is, because that's coming, does it now become a premium to be able to see a human that can provide that communication, that can provide that touch, that empathy, that can give you what you need human to human, that can understand what you're dealing with, what you're going through? I get a bit more concerned about that divide.

Elizabeth Harry:

Wow. I mean, that's amazing to even think about. And I mean, just envisioning this future where it is sort of optional that your healthcare would involve a human is really profound. And so if you're-

Cornelius James:

Liz, can I just really quickly say?

Elizabeth Harry:

Yeah.

Cornelius James:

I get a little bit more concerned about that being the case for marginalized minoritized populations, not just because there's the premium there when it comes to being able to see a clinician. But we did a study relatively recently that it was in JAMA Network Open. It was just like a secondary analysis, and it was looking at older adults' use of digital health technologies. And interestingly, racially minoritized folks, Black folks, Hispanic folks, they used digital health technologies, and this was a surprising finding, they used digital health technologies more often than people that were in a majority population. So that was interesting.

And one of the questions that someone posed to me was, "Well, should we just start pushing this and suggesting that these populations use this more?" I said, "No, we should address the reason why they're not seeing doctors." And I get a little bit concerned about, because of mistrust and distrust of the healthcare system, that people do tend to turn toward these technologies as opposed to humans because of the distrust and the mistrust and so on and so on. So that's a bit tangential, but I think it's sort of related in that I see that potential there too.

Elizabeth Harry:

Well, and it's so interesting because, on the one hand, it's like this idea of the great equalizer that people that have no care right now, is it a step up from where they are? That there's a sort of foreshadowed physician shortage that is quite startling when we look at some of those numbers that have been predicted. And so thinking about, okay, well, are we comparing it to having no care, or are we comparing it to having a human? And then part of the question I start asking myself is, might we get to a point, I think we'll never not want that human connection.

Cornelius James:

I hope not. I hope so.

Elizabeth Harry:

I hope not too. But might we get to the point where the AI is actually maybe better in terms of the cognitive piece of it?

Cornelius James:

Oh, for sure.

Elizabeth Harry:

Right. And so then it's like, are there people that are going to say, "I don't care about that human piece. I want that 100% you've searched everything in the universe that exists related to my symptoms guarantee which a human can't give me in the same way."

Cornelius James:

I think if we go to our medical ethics principles with the autonomy piece, I think that's going to be fine. There are some people that prefer that. There are some patients where I say, "Please take this statin. It's going to be very important," or, "Please take aspirin," and they will not do it. And I think having that agency, having the autonomy to make that decision, if someone would prefer to see AI. But is it available? Meaning is that human, is it accessible? If I needed it, if I really felt. Because at the same time, while my patients will say, "I don't want to take the statin," they know you portal me, I'm going to send it for you, and I'm going to congratulate you for making that decision. But if we're to just not be available, that's another issue.

Elizabeth Harry:

So with all of this, do you think we're at any risk of overmedicalizing AI, treating it like a device problem when it's really sort of maybe a system or operations or kind of human problem?

Cornelius James:

So I think you can probably appreciate that I'm expecting that these technologies are going to get better. They're going to be better at diagnosis, particularly the thinking part of diagnosis. There's already been literature, it's not quite real world clinical practice, but I think many are familiar with that paper published in JAMA Network Open suggesting that AI alone performed better than either humans alone or humans with resources like UpToDate and so on. But it's going to get better. It's going to show, quote, "superhuman performance" that's going to happen.

Bob Wachter wrote a paper in JAMA, I think it was published in 2024 or so, it was around this productivity paradox. And what he suggested was these innovations are going to get better or these technological innovations particularly are going to get better. But there needs to be a parallel innovation in multiple areas for us to successfully integrate these tools into healthcare. There has to be innovation in medical education. There has to be innovation in the culture of healthcare systems, innovation when it comes to teamwork. There has to be so much innovation.

So to your point, yes, I do suspect that it's going to be more of a human problem because the technology's going to get better because people are going to continue to innovate. We see these technologies changing by the month, by every few months or so. But we need to have this parallel race toward innovation when it comes to culture, teaming, education, et cetera, et cetera, that are very, very, and implementation science or implementation. There needs to be the same degree of funding and interest and so on in those areas as well if we're going to see these technologies truly benefit patients and clinicians.

Elizabeth Harry:

It's sort of naming that sociotechnical system. Don't just focus on the technology, but the social piece in which it lives. I love that. So if a clinician's listening, what's one thing they could do Monday morning, next week, to integrate some of these learnings or to try to move a little bit forward in their use of AI?

Cornelius James:

So I suspect that we have folks, I met with someone yesterday, I'm not going to give the number because then they'll know who it was, but they said that they had only used AI a handful of times, and this is someone at Michigan with lots of AI around to use for free. So my opinion is to just try it, not necessarily for a clinical purpose at this point, try it to just see what it does. If you're planning that trip to Chicago, ask AI, what are some hotel, or ask the gen AI or a model, use U-M GPT or something like that, how can I get there? And once I get there, where should I stay? And I'm traveling with my wife and two kids, or my husband and two kids, or my partner and two kids. Ask it, just use it for simple things like that. I want to make spaghetti. Give me a good recipe for making spaghetti. And just sort of see what it does and appreciate some of the benefits, but also the limitations or the challenges associated with it.

And then if you move forward, try something like OpenEvidence, which is freely available and just sort of see where the benefits lie. You'll be able to see, yeah, JAMA, NEJM, the American Diabetes Association are now making their content openly available to this technology. Maybe there's something behind this.

And then maybe you get over to where you say, okay, I'm going to try out this Ambient Scribe thing, see if it really moves the needle or has an impact. I think those are small things that people can do, recognizing that there are folks that are sort of across the spectrum. And I would say if you are a heavy user, try to certainly be respectful of those who are sort of lagging behind, but there's literature suggesting that clinicians or trusted clinicians have a major impact or a significant impact on what their peers do. If we allow people or tell or talk it up or talk about how this technology has actually been helpful in your clinical life, but also maybe even your personal life.

Elizabeth Harry:

I love that. And if you're a healthcare system leader listening to this, and maybe you're sort of bought into the ambient documentation because most health systems seem to be, but you're not sure where you should be thinking next as a leader of a health system, and this is not your space, what would you advise them to be thinking about next week?

Cornelius James:

Oh, I have my bias that's like, I'm going to say education, education, education. I think that's going to be really important to educate the end users. But I would suggest surrounding oneself with, first and foremost, people who their mission, vision, values are aligned with the university, but that also have the technical expertise, the clinical expertise, all of the expertise that's needed to ensure that ultimately we're making... And this is what I love. At Michigan, yes, have we deployed more models than everybody? Have we put out the coolest models? But it's been thoughtful because it's been thoughtful implementation integrate... Would I love to see it go a little faster sometimes? Absolutely. But I believe that it's been thoughtful because the right people are in the room and having these discussions. And when I say the right people, I'm talking about diversity in many ways, the right people are in the room, and therefore that's causing it to actually work well as opposed to not.

Elizabeth Harry:

And we're all patients at some point, but if you're a patient listening to this and thinking about how to incorporate AI into your patient or health or personal wellness journey, what would you say?

Cornelius James:

So the digital literacy question that you asked earlier was really important. You're likely familiar with there's a model that Epic recently released. It's either Emmie or Emma, one of those two, where it's basically a patient facing AI that's going to eventually allow them to ask questions and it's going to explain results and so on and so on.

So those things are coming, and I would suggest very similar to what I mentioned about clinicians, just start to think about using these technologies, maybe in personal life for personal things. And then eventually, once you get to a point where you feel comfortable, it's okay. It is okay to ask the AI questions. But, but, big but, it's going to be very important that you also consult with your healthcare provider to ensure that some of the recommendations, the suggestions from the AI, are actually evidence-based, safe, and appropriate for you. So I think engaging with the technology's great idea, but it's important, just like the clinicians, to do so in a safe and responsible manner.

Elizabeth Harry:

Yeah, to get that wisdom.

Cornelius James:

Yeah, for sure.

Elizabeth Harry:

Well, I mean, this has been incredible and I have learned so much. So thank you so much, Dr. James, for sharing your insights on the evolving-

Cornelius James:

You said Cornelius though.

Elizabeth Harry:

Yeah, but we're closing it.

Cornelius James:

Okay. Okay. I'm sorry. I'm kidding.

Elizabeth Harry:

But no, no, it's perfect. So thank you for sharing your insights on the evolving landscape of AI and healthcare. And we've covered lots of things from building true AI literacy in clinicians to leaders to patients, which we all are, of course, to pitfalls in how we think about the different types of models and model evaluation and governance, and why motivations behind these things matter, and having the right minds at the table to really think through this, thinking about how language-based AI can affect wellbeing, and where we need to be really thoughtful to make sure that if we're doing this to give access and to help create access where we're having access issues, that's wonderful. If it creates access issues, because we sort of start filtering people through AI instead of having that human experience, that we really need to watch for that.

And as the JAMA Summit Report suggests, this transformation is well underway and our challenge is to close the gap between the technological innovation and meaningful oversight as well as the social piece that you talked about and that wisdom. I really love that. And it's clear that interdisciplinary collaboration and patient-centered thinking has to be at the center, the humanness of this. I really appreciated that.

So to our audience, thank you for joining. Keep asking critical questions, stay curious, consider how you can foster more informed use of AI in your day-to-day. We have links and resources in the show notes to Michigan AI resources available to improve your daily workflows. And come join us again. Thank you, and thank you so much for joining.

Cornelius James:

Thank you.