I'm Dr. Karen Lenick. I'm from 3M Corporate Occupational Medicine. I'm going to take this off because I'm very jingly, and I like to move around when I talk. Can you guys hear me OK? OK. Perfect. Thank you. All right. So I'm Dr. Lenick from 3M. I'm going to do the intro slides for ACOM, and then I will give my talk about 3M. Then Dr. McKinney will talk about the work being done at Health Partners Institute. And then Arthur Sanchez will present for UCSD. So just to give an overview here of the Specialty Society's Advancing Adult Immunizations Grant. So since 2021, ACOM has partnered with CMSS in a CDC cooperative agreement. ACOM and health systems aim to increase COVID-19, influenza, and routine adult vaccination rates among high-risk adults in occupational health settings, doing this through education, dissemination, and QI initiatives. Partners include 3M, Albertson's Companies, Conservative Care Ock Health, Froedtert Health, Health Partners Institute, and the University of California at San Diego. CDC standards for adult immunization practice include assessing immunization status of all of your patients at every clinical encounter, strongly recommending vaccines that patients need, administering or referring your patients to the vaccination provider, and documenting vaccines received by your patients. ACOM has developed a website with vaccine-specific information. It's acomvaxinfo.org. There are podcasts, guidance statements, vaccine messaging, and strategies to address vaccine hesitancy. You can get more information at the membership booth. And with that, I will go into 3M and our Quality Improvement Project. I have no disclosures, and our participation in ACOM's vaccine initiative is supported by the CDC. I apologize. Thank you. Sorry about that. So, this is attributable to the CDC and the Department of Health and Human Services as part of a financial assistance award to CMSS. Okay. So, background. 3M is a global workforce of approximately 89,000 employees with about 30,000 located in the U.S. 3M Maplewood Center is in Minnesota and houses a full-service clinic and the Corporate Occupational Medicine Department. We have three physicians, several certified occupational health nurses. Most manufacturing sites and research and development centers have an on-site OHN. 3M has provided on-site flu vaccinations for several decades and provided on-site COVID vaccines during the pandemic. Our on-site COVID vaccines were manually entered into our EMR and then into state immunization information systems by regulatory requirement. But there was no electronic communication. 3M Corporate Occ Med received this vaccination grant in October of 2022. We were able to onboard our QI manager, project manager, and really get going in early 2023. Our objectives for last year were to reduce vaccine hesitancy and increase vaccination rates for influenza and COVID-19 among the 3M workforce at two pilot sites, also to explore the feasibility of developing a bidirectional electronic feed between our EMR, which is COERDI, and many of you might have COERDI, and state immunization registries. The framework that I like to look at, I like to consider, and that I talk to the nurses about, I really love this diagram from Dr. LaSalle that talks about the continuum of vaccine acceptance. So at the plunger of the needle, you're going to have people who refuse all vaccines. And then in the middle there, you have that vaccine-hesitant population where they might refuse, but they're unsure, they delay, or they accept but are unsure. And at the tip of the needle are those that accept all. So you tell them what they're due for if they go out and get the vaccine. Those are the very rewarding ones, right? And take up the least amount of counseling time. So our goal was really to target those in the middle, where they would gain some additional knowledge and perhaps some additional trust in the CDC and their healthcare providers, and move that extra step towards getting vaccinated. So our timeline was that in May, we had our study undergo IRB approval through 3M Institutional Review Board. We started recruitment at our Minnesota manufacturing site and our Texas office and R&D site. Informed consent was done electronically, which was a unique component for us at 3M. And then the baseline survey was done online, and it included demographics, health characteristics, vaccination status for tetanus, flu, and COVID. We added in tetanus, because it's a vaccine that all adults are recommended to receive, and to try to de-stigmatize COVID. And then we also included the Adult Vaccine Hesitancy Scale by ACLE. It was published in 2021. It's 10 different questions, and then we added in some additional questions as well to evaluate vaccine beliefs pre- and post-survey, or pre- and post-intervention. The vaccine education modules were done in September through December of 2023, so in the fall. They included an online module through our 3M Learn system, the Lunch and Learns that were offered at Texas, and an OHN one-on-one consult, where we could actually pull up the vaccine records then, because it was the intended purpose of the visit, and then counsel people on what vaccines they were due for, and often refer out, because we have a limited number of vaccines we offer at our sites. And then the post-survey was given out in January and February. So there could still have been a few people, right, who might have gone and got the influenza vaccine, but really kind of targeting those who we'd really like to see get the vaccine, you know, in the fall or early winter of the season. And then we looked at change in health and vaccination status, and again, repeated the Adult Vaccine Hesitancy Scale. The Learn, the 3M Learn module and Lunch and Learn talked about reasons for adults to get vaccinated. The Lunch and Learn had a little more time, so they talked about what to expect after vaccinations and went into a little bit more about risk. Both covered which vaccines are recommended for all adults, tetanus, flu, and COVID, and then additional vaccines that might be recommended based on your age or health conditions. They talked about ways to find your vaccine records, which is very key. And then the Lunch and Learn had a tutorial of the Texas IMTRAC2 system, so they could actually see how to get their records. And then how to get vaccinated. If they had the one-on-one consult, and this is kind of small, but if you had the one-on-one consult, you went through in very great detail your specific vaccination history, what the recommendations were, and then general information. This was based on a handout the CDC had developed a few years ago, and then we updated it to include some additional information and then tweaked it for our next round and changed the color format to the new logo. So, the old logo was green and blue. So, study participation, we invited about 1,000 employees to participate, and 126 study participants completed informed consent and the baseline survey. A hundred and twenty went through the vaccine education choices, with the most commonly selected being the online module. About 11 attended the Lunch and Learn, and seven had the one-on-one nurse consult. About 113 participants completed the post-survey, so the analysis will be based on those 113. So, looking at our demographics, it was about split between those who were under 50 or 50 and older, and just as you get older you have, you know, potentially more concern in being vaccinated and protecting yourself from certain conditions. Sixty-four percent were male, which I think was great, because from an education standpoint, women tend to sign up more, I feel like, for education activities. Primarily the race and ethnicity was non-Hispanic white. The primary marital status was married. About 54 percent had a bachelor's degree or higher, but, again, 46 percent had technical certificates, high school, or associate's degrees, so still a pretty good representation of our workforce. Sixty-one percent were in production or manufacturing environment, and 68 percent worked on-site. We do, the rest were either remote, completely remote, or a hybrid approach. So, for health characteristics, the most reported health characteristic was having diabetes, followed by respiratory problems. There were about a handful that were immunocompromised or had a close contact with someone who was severely immunocompromised, and 11 percent were first responders at 3M or elsewhere, which leads into our potential look at hepatitis B in the future. So, vaccination status at baseline, again, we really kind of brought tetanus in just to get people thinking about vaccination, and we had about 80 percent reporting being up-to-date on tetanus, which was considered having had at least one dose of Tdap as an adult and a dose of TD or Tdap in the last 10 years. But still about 20, about 18 percent were unsure of their status, so we were hoping through this process they would learn more about where they stand. The most important focus was on those who had received the prior year's influenza vaccine, which was about 66 percent. That's actually fairly good overall, but then you can see the drop-off in terms of who would receive that bivalent booster, the one that was available through September 10, 2023, where we only had 30 percent of people who had received the booster, so we had a lot of room for improvement there. And then CDC and ACOM asked us to track the number of doses received. So, surprisingly, we did get six people who had never received a single COVID vaccine dose, more than I had expected from voluntary participation with the only reward being a very, very nice-looking but small 3M-branded cooler, so I thought that was still impressive. And the majority, you know, had received two or three doses, so had, you know, were at least completed a primary series, with a few very great overachievers that had received five or more, which was great. When we looked at the change in beliefs, I'm kind of categorizing these in groups. They're not specifically subscales, so you know. But when I look at the importance of getting vaccinated, so the two questions that were from the hesitancy scale, were vaccines are important to my health, and being vaccinated is important for the health of others in my community. We added on two additional questions to look at whether people thought more highly of protecting their family members or their coworkers, kind of using that socioecological framework. And on both, you can see, we started fairly high. So this is on a Likert scale, so a four would be agree, somewhat agree, and that's kind of where we came in at the start for our pre, on average, our pre-mean. And then our post-mean, you do see movement. It's small, but it does move even closer into the strongly agree category, which was really exciting. And those were all statistically significant findings. In terms of effectiveness of vaccines, again, we started fairly high at about an average of four, so at that agree, somewhat agree category. And again, we saw that small shift upward towards strongly agreeing that vaccines are effective. In terms of trust or knowledge, I'm sure our grant sponsor will be very happy to see that we started around a 3.5, so somewhere between neutral and undecided, and agreeing, and moved upward a little bit, but still statistically significant, moved upward towards a little bit more trust in the CDC and the reliability of the information that they are receiving. We started very high for trusting your healthcare provider and for your site nurse, again, had some forward movement for the healthcare provider. Unfortunately for our nurse, they love our site nurses, but our manufacturing site lost one of our nurses to the spinoff company, so they didn't have a nurse for two months. So otherwise, I think we would have seen even more movement there. And then I added this question in about knowing which vaccines are recommended for my age group and health conditions, so really more of a knowledge piece. So the people who know are more likely then to go and obtain the vaccines, and that also had a forward shift. Now these are a little more tricky. This is from the scale, and these three questions are worded in a way that I had to reverse code them. So the actual question was, I do not need vaccines for diseases that are not common anymore. So if you look at that in the reverse way, after we reverse coded, we're really looking at, I do need vaccines for diseases that are not common anymore. And that did move forward as well, which was great, and it was statistically significant. And I would look at that as the risk of the disease, and then the next two to me are looking at more of the risk of the vaccine itself. So then when you look at those and you reverse code, and we're looking at, I am not concerned about serious adverse effects of vaccines. That started in the neutral or undecided category, and unfortunately it stayed in the neutral undecided category. So we have room for improvement. The same thing, when we reverse code and we look at it as, new vaccines do not carry more risk than older vaccines, again, low, almost in that partially disagree slash neutral category, and again, no movement on those statements. When we look at the total count from the Likert responses, when we started off at baseline and we added up all the different responses, the most common was that four out of five that agree or somewhat agree. By the end, the most commonly responded to out of those 14 questions was strongly agree. So if we're thinking about that model of trying to move people closer to the tip of the needle, I feel like we're seeing some movement and progress in that direction. How that played out then in the change in seasonal vaccination, overall we had a decrease of four people between who reported receiving the earlier seasonal vaccine versus the most recent seasonal vaccine. Again, the questionnaire was in January, so maybe there were a couple more people that went ahead and got the vaccine. Also, overall, we've been seeing a trend in the U.S. of lower flu uptake, flu vaccine uptake, since COVID. So it might mean that we're pretty much on par or a little better maybe than the decrease in the country. We don't have those rates yet for 2023-2024 for the population. But what was very exciting was that 11 people, there was an increase of 11 people who received the most recent of the 2023-2024 formulation of the COVID vaccine compared to the prior booster. So I feel like that was really exciting progress. I know it's a small sample size. And one of the people who had never received a single dose of COVID vaccine got vaccinated. So that was also very exciting. I know. Got to move the needle somehow. So overall, I would say the belief in vaccine importance or effectiveness and the trust in CDC and healthcare providers was relatively high in our population on average at baseline. But we did have a small but statistically significant increase, suggesting even greater vaccine acceptance. There is a concern about adverse effects of vaccines or risk from newer vaccines. It was about neutral on average. And we didn't see a change in either of those two topics. Overall, an increase in self-reported COVID-19 vaccination. So with that, we feel like there's enough there to continue the pilot. We will optimize survey questions and education modules, try to see if maybe wording the questions in a positive manner all the way through makes any difference in terms of having those kind of negatively or reverse worded questions. We also are tweaking our education modules, adding in some more specific information about adverse effects and risk to try to get more at the heart of those concerns. Our pilot will continue at one of the two sites. And we will add on two more sites, another large manufacturing facility, and it's in South Carolina, and then a subset of a group at 3M Center. So hopefully with that increased sample size and assessing the effective, we can assess the effectiveness of the education options. We're really hoping to get some more OHN consults completed. And with that, then we are able to look at is there a difference. And our long-term plan is to have this worked in with our health benefits. So how many points will they receive for doing the online module? How many points would they receive if they did the one-on-one nurse consult? And so we can give that information and move that ahead. So long-term, there will not be coolers, or Stanley cups is our newest idea. For incentives, it would be saving money on your health insurance premiums, which is something we all want to do. And then finally, we need to find a solution to connect our EMR to the immunization information systems, and that is in the works for 2024. And so we have the possible, it's possible to do the connectivity, and now we have someone assigned in a resource in order to move that forward. And if we can get that in place, we can really grow and expand the project. So that's it for now. We have two more people, so I'm going to let them present, and then we can take questions at the end, if that's OK, unless anybody has one right now. Okay, and Zeke's slides are shared this time. We're good. Hi everybody, thank you so much for coming. Hopefully you're not in that postprandial food coma, but you know, if you are, whatever. Though that's a tough act to follow. Dr. Lenick is amazing, those slides are amazing, 3M is amazing. So anyway, my name's Zeke McKay, I'm the Program Director for the Health Partners Occupational Environmental Medicine Residency Program in Minnesota. The only such residency in Minnesota, one of only two in the Midwest. So we are more of a clinical site, whereas 3M is a corporate site, so it'll be a bit different in how I present this. Also, I present super fast, because I don't want you to read these slides. You can look at this later at home. In fact, I'll take this pointer. You know, but just focus on what I'm talking about. So we're gonna breeze right through, don't even worry about it. So this is just really our team here, but a few people worth calling out, of course, would be Natalie Belzer, who's one of our senior residents, and she was helping us with some of the data collection elements. Elise Haven is our Project Manager. Linda Fletcher, really important informaticist, who's been really helping us with understanding how to map what we're trying to measure to the actual outcomes that CDC wants to know about. By the way, this sounds really loud to me. Is this too loud? Oh, okay. It feels like really reverberant and whatever. Okay, great. So a few things about health partners. We're like one of the largest consumer-owned nonprofit healthcare organizations in the country. Minnesota is like the home of the HMOs, if you guys didn't know that. So we cover a lot of Minnesota and a bit of western Wisconsin. So really what I tell people is health partners is just like Kaiser. Hospitals and clinics, insurance plan, grant-funded research, there you go. And like I said, only residency in Minnesota, 1.2 million patients. All right, that's the end of all that. So more importantly, like I said, nonprofit and the Health Partners Institute has 400 investigators, including myself. And so we got this grant from ACOM and their partners. Importantly, we also are a site for vaccine safety data link as well as other vaccine-related research, which is important. And I think the cool thing about our institute is we have a lot of really great support there for programming, informatics, statisticians, grant development, the whole nine yards, full-stack research, you know, if you want to call it that way. And so, like I said, clinically, we are consisted of four occupational environmental medicine clinics in the Twin Cities metro area and a little bit beyond that. I work primarily at the West End Clinic, that first one on the list. This one's just south of downtown St. Paul. Stillwater's right on the border of Wisconsin. And then we have one in Minneapolis. And then actually since this time, we've opened up a fifth location actually in Western Wisconsin at this point. We actually had more sites in this before COVID, but of course, you know, a lot of OCMED contracted right when COVID happened. So just in case anybody doesn't know, I actually talk a lot about vaccine hesitancy. This is just a picture from when I was presenting for the School Nurses of Minnesota. They invited me back in November to talk about vaccine hesitancy. Because I like go off about this stuff. How do you talk to patients about it? So anyway, you know, I like to get into that. All right, now digging into this project for real. So Karen already talked about this. Dr. Lenick, I apologize. I talked about the CDC standards for adult immunization practice, where it really comes down to these four steps of assessment, recommendation, administration, and referral, and then ultimately documenting and following up that that administration happened. We don't need to get into the detail of that either. Okay, now where really the rubber hits the road was. So for us, we wanted to see that every patient coming into our OCMED Clinic was potentially eligible for this intervention, so be it. We excluded any nurse-only visits, because again, we didn't think it would be appropriate only to put that onto our nursing staff, and the clinicians would have to talk to them too. And if somebody had already gone through this process in the past six months, we weren't gonna do it again. And so what we wanted to do, and this is just like the most simple way I can just describe it. First thing we did was give the person a copy of their immunization record from our state's immunization information surveillance system and Minnesota actually has a famously good one. And so I personally thought that was one of the most important things we could do here, just truly just handing somebody their immunization record and said, this is what the state has on file for you. Take it or leave it. I don't know if it's right, but this is what we got. And so then our nursing staff, our CMAs would talk to them about it real quick and say, hey, do you want a referral for this? And then they would come and see the clinician, an MD, a PA or whatever. And we talked to them about exactly what we're doing. And then we'd say, again, do you want this referral if they hadn't already taken it? And then we actually would place this referral in. So then it would, as a representation, I mean, cause we're not actually referring anyone per se and we could, but it's really more of a data representation that in fact, yes, they were accepting this vaccine. One big problem with that process was that, because we were kind of doing this on paper, which I'll get into, then somebody had to go back later and putting these referral orders, which actually is a big process limitation. So originally this was kind of our pilot project where we started with just COVID-19 and influenza and just two of our four clinics. The intent was then to expand to our four clinics, those same vaccines, and then later expand the number of vaccines in all the clinics. And so we're still kind of in this pilot phase, moving into the second phase soon. So this is just an example, you don't need to read it, but it basically just showed that this person was given these vaccines. This is like from the state's data. And then down here, it started onto the next page saying, hey, here's what you're due for and when. And so again, that's great. The person's getting the best immunization information from the state. And at the bottom there, we had links to, you know, the vaccine information statements, the viz, on those which are available in many languages on vaccine.org. So that was great. Now, the problem was we didn't have a lot of good information there originally that by the way, this is a research project and like, we think we could have reorganized this data better and we did that later. And our original form actually had 10 questions for the clinician or the nursing staff to answer, probably too many, I get it. But we did it on paper, because you guys know how it is in clinic. If you want somebody to do some extra process where you got to go into your electronic health record and answer all these questions, no, no, no, no, no, no. So I just said here, just write it on the paper. And so basically just to ask, you know, is the person, did you give the person the report, yes or no? Did, does it say that they're up to date on everything? Because if they're not, then let's keep going and try to give them some stuff if they want it. Do they agree with it? That's a question I felt very passionate about this entire time. Because if somebody looks at their vaccine records and says, no, no, no, no, no, I've already gotten all these vaccines, or inversely, no, I've never had these vaccines, then I don't want to disrupt that. And I say, okay, look, now you got to go to the state or go to your primary care and talk to them about fixing this record. That's a different layer of a problem that isn't even part of this. But if they do agree, then are they even willing to talk about it? Will they take a referral to get a vaccine? And then the same questions are repeated after the CMA for the clinician. And so that's twice the opportunity for the people to get a chance to be asked about it. But it turned out, okay, well, maybe the language wasn't totally clear to our staff originally. Maybe it wasn't clear for everybody to follow, okay, what happens if they say yes or no versus all this? The paper form obviously requires manual data entry, which is hard. And then of course, this independently having to put in referral orders after the fact was a big limitation. So this was our original thing. We've done better since then. But from that, the question was, okay, how do we actually measure who got what into the framework of what CDC wants? And so basically for anybody who theoretically needs more than one vaccine, if they're up to date and they agree to talk about it, well, then great, let's keep going. But if not, then we'll exclude them because we'll assume they're up to date or at least the data may not be valid. If they're willing to talk about the vaccination, cool, and they keep going. But if they told both the CMA and the clinician they don't wanna talk about it, all right, move along. Hey, we've asked them and they don't wanna go there. And if they're willing to get a referral, great. But if they told both the CMA and the clinician, hey, I don't want a referral, then kick it out again. And so then once we actually have that referral in place, again, which we got rid of later, but it's this time, then all we were gonna do to verify this for the documentation step was look at the IIS data and say, hey, did the person actually get that vaccine within three months, proving that they went through with what we recommended. And so really just following those steps of what the CDC wanted, I made this correlation here. You can look at that later. And so we put this into place in August of 2023, so not even a year ago. And so immediately when the patient checked in, we would print out these forms that came along with whatever their normal clinic face sheet and kind of go through that process. And again, originally at the original two clinics. And so we originally surveyed our clinicians using a usability scale. This is where I'm gonna get into some informatics jargon, don't go crazy, in that first couple of weeks after that. And even just initially there was 900 encounters of which we had about 219 or nearly 25% where we had any type of response on there. The question, at least at that time in analyzing it was, well, how much delay was there between when we actually saw the person in real life and the forms actually made their way into our process of getting checked on it. So just from a usability perspective, we use a system usability scale, which has been around for about 40 years. And so it's a validated 10 question survey with Likert scores from one to five. And then we just let them write on there, hey, tell us whatever else you think, who cares. And it gives a score from zero to 100, where 100 would be like, this is the most usable thing ever. And zero is like, I don't know if anybody could use this. And it's trying to measure effectiveness, efficiency and satisfaction, which are the three kind of measures on which people design systems and usability. And effectiveness being like, can people do it? Efficiency is like, how easy is it to do it? And satisfaction is like, does it annoy you to do it? So the system usability scale looks like this in terms of questions. I think I would like to use this, and you could say process, but we replaced it with this vaccine intervention or whatever. Oh, by the way, I forgot to say the most fun thing. The name of this project, Worker Health Assessment of Timely System-Based Vaccine Promotion, what's up, a la Bulgari, if you will. Anyway, I forgot to even make that good joke at the beginning. So part of the trick of being a researcher is coming up with good names for your grants. Just remember that. Okay, anyway. All right, so what we found for the 17 staff of ours, including the clinicians and the nursing staff, that they had an average, gave an average SUS score of 54.43 where again, so I mean, you can see it's not great. It was kind of in the middle on everything or maybe a little lower. And so to contextualize that for you in terms of what does that mean? So again, an SUS score of like 90 or 100 is like, this is the best system I can imagine. Whereas, and again, down here, you can see it again, like 50 or 60, it's kind of like, ah, this is okay. Now for some real life examples, people gave TurboTax like an 82. Not our staff, but in studies that have been done historically. People gave Microsoft Office like a 75. People gave Blackboard like a 65. What's all the way down here? AutoCAD is like a 60. So my point is, I'm sure you guys can think of things that are even lower than that, that have even annoyed you to death to use. I bet Netflix would probably be pretty high. That's easy to navigate, for example. So what happened? Well, number one, we were able to actually implement this pilot study, great. And like I said, to me, one of the most effective and important things that we did was actually just give every single patient, here's a copy of your immunization record. And so many patients were like, wow, is this really what's written there for me? I've never even seen this before, which we all probably imagined. And so we were able to get this automatic printing done. Clinic staff were able to do it. And like I said, patients liked it. So that was cool. But what we found out was, it wasn't perfect. One of the problem is that, you know, in terms of forecasting who needs what vaccine, it's really great data that we can do in our electronic health record, namely Epic, but what the state has is a little bit different than that. And so the question was, how do we resolve those two? And ultimately, we settled on using the state's data because not every patient we're gonna see are naturally patients in our health system. You know, the question about whether or not the vaccines are due is really up to the human being looking at the form to say, yes, this person actually needs that based on the data that's here. And again, how do we actually give this information to telehealth patients when, again, this is printing at the time that people are coming to clinic? And that's actually a problem we haven't totally solved yet either. We thought about maybe using like a patient portal, such as what would be called MyChart and Epic. And so anyway, there's still some things to resolve that aren't perfect. What we really heard was, you know, if patients had questions, sometimes the doctors who aren't me, the principal investigator of the project, didn't really know how to answer it in the context of like, well, this is a research study. And again, if there's any issues in the data entry, like if it's lagged, somebody putting that paper form into our electronic system, then the patient might get the intervention again less than six months later, that's bad. Sometimes the immunization report, even though we can actually see it in our IS, for whatever technical reason, didn't actually make it to the paper, which is a problem. And that's a very rare case, but it happens. And then the referral that we were giving to primary care wasn't really a great handoff and actually was causing some problems in terms of our process. And overall, what do you guys expect? You know, the doctors are saying, hey, I don't have time to do this, man. That's bad, and it's true. And so there were a lot of ways in which this was really complex, especially considering that we had both the nursing staff and the doctors asking people the same questions twice, which I thought was good to give them an opportunity, but it turned out to be maybe too onerous for our staff. So one thing was, get rid of some questions. Okay, great idea. Why don't we, you know, for now, just keep it limited to the small number of vaccines and maybe even on the vaccine report. We didn't end up doing that, but we could have. And then as clinicians got more familiar with it, it did get easier. And so then one thing about, you know, the question about do patients agree with it, this has been a great argument internally. I'm always arguing, yes, I really want to know if patients agree with this. And everybody's like, yeah, but that's unnecessary. Do we really need to know that? Well, I hope so. And then ultimately, you know, not just ACOM, I shouldn't have called you guys out here, but with even CDC, there has been a lot of questions amongst all participating sites about exactly how do we interpret the data we're getting in terms of the metrics of who's been vaccinated or not. So our new form broken down to four questions from those original 10, pretty good. So question one says, did the person get the report and did you review with them? Yes or no, or they don't want to look at it. Cool. Number two, does the report indicate that they are overdue for any of the following? In this case, COVID and flu. Cool. Do they agree with it? Cool. And if they do agree with it, now do we want to proceed on and just ask them, hey, do you intend to get it? Not a formal referral, but just like now that I'm telling you you're due for it, are you going to get it? And so that really helped, but nonetheless still being on paper, there's a chance for a lot of logical errors. We also, much like Dr. Lennox showed at this second phase of this gave an informational handout, which we didn't originally, and I'll give credit to Dr. Lennox and our folks from UCSD and all the other sites that we met at ACOM back in November of 22 when we reviewed this, because I realized that was something we weren't giving, like information about what are these vaccines doing? And so we have a similar one page handout to what everybody else was showing, just giving basic high level information about adult vaccinations. So translating the CDC guidelines to what we're currently doing, again, the assessment is we gave them the report. The recommendation is, are they up to date on it? And do they agree? It did in terms of administering, well, we're asking you. Now I'm telling you to go get it. Are you going to? And then four, based on those results, can we compare that with what did they actually get in real life? Now, what I found out now is some clinicians may not even believe adults necessarily need to be vaccinated such as for COVID, which I thought was weird. And I haven't had that conversation with people yet, but okay, whatever. And it's actually more upsetting because I'm hearing that maybe those same clinicians are convincing their nursing staff that this is not an important thing to do too. So when you're doing this in an actual clinical setting, you may want to get all your colleagues on board with each other about, you know, do we really believe this is something we need to be doing? And so again, I haven't had a chance to get through the usability data for our second round, but I'm hoping it'll be significantly better and we'll be able to compare it to what happened before. So without taking up any more time, you know, many thanks to ACOM, CDC, CMSS for supporting this work, to all my colleagues, and as well to my folks locally who are doing all this. Thank you very much, and I'll pass it on to my friends at UCSD. Thank you. to call after what's up for Dr. King. I was like, dang, I have to be right after you, see? Anyhow, but you know what? We're coming with strength from San Diego. Alright? Alright, so actually I'm just going to talk, you know, a little bit. I'm Marcia Sakari from UC San Diego. We have three lines of business. We see campus employees, about 30,000, plus health system employees, about 12,000. And we're going to focus on the 12,000 employees. And we have a commercial line, too. And Arthur, our champion, will do most of the talk. Alrighty. Nothing to disclose. So what I wanted to talk, actually, is how was our strategy? It was a little, we think we're a little bit different, is that we had like a very good opportunity that we integrated our EHR. And the nice thing is that not only we integrated fully the occupational medicine department into this large EHR, we also created firewalls, and we were able to interface with PeopleSoft. So everybody who gets employed at UCSD will get logged in, in our large EHR. And there's firewalls and, you know, there are different methods. So we're able to have peer-to-peer vaccination and stuff like that. So our approach was through two prongs. One is system and data, and we talked about that. And I'm going to show you the dashboard. When we started, we had a theory. You know, we came from San Diego, and, you know, diversity and inclusiveness is, in a way, it's super important. We have a large Latino population. Myself from Brazil, Arthur is Cuban-American, so we're pretty passionate about, you know, bridging any gaps of social inequality for the Latinos. So when we started this, I was like, I'm going to prove, you know, with the large data, with every single of these 12,000 employees, that the Latinos were not getting vaccinated as much. Thus, we need more intervention. And you're going to see the results a little bit later, okay? And then people, you know, in the Latino culture, consejeras, you know, it's very effective. You know, family, community, it's super important. And so our approach is, you know, listen, you know, what exactly moves people, develop partnership, and bridge gaps. And first we initiated thinking maybe we're going to use the Latino consejeras because we thought that we're going to see discrepancy on that. But we quickly adjusted. So this is our dashboard. You know, as you can see, the difference that we see, actually, you know, from the grand total of 80% is on the SX union and on the K6. It's too small to not see that. So you can see definitely there's a difference in that in the percentage, right? And going back a little bit of what we can measure, we can actually, for the whole 12,000 people, we can break it down by union, by department, job code. The race and ethnicity got a little bit tricky on that, but we were able to break down into Latino and non-Latino. And that's what we found. Actually, there was not a difference between Hispanic and non-Hispanic. But as you saw on the other slide, that the difference was actually based by union. With that, then, okay, let's change our strategy, right? So with that, the difference is on union. Well, the employer has control over that. And then why we don't have the employer approach, meaning that on-site visit. And then the question is, is it convenience or is it only hesitancy? So then we started thinking, you know, of course hesitancy has a lot to do, but convenience can be a very effective countermeasure. And why not hopefully, you know, put some ideas and possibly having government incentives, like for COVID, it was so effective. So I thought that this was, you know, super great that it actually does prove, you know, even more in our specialty, our role and how impactful we can be, you know, as an employee-based strategy. And I'll pass to Arthur how we did. So like Dr. Izakari talked, initially we thought we're going to mimic the population, right? Hispanics, Latinos are going to have lower vaccination rates. That was not the case at all. And it actually turned out that discrepancy was really by union. So we were trying to figure out, well, how do we, how do we like attack this problem, right? Like how do we reach out to these different unions? And of course, part of our strategy is developing partnerships. So we reached out to our employee health and safety folks. And it turns out these guys have a really great on-the-ground presence. They meet with the different staffs, particularly for the SX union, which is service custodians, food service custodians. And I think food service custodians, what am I forgetting? Maintenance is K6, the journeyman also. So we reached out to our EH&S folks and we had this conversation like, hey, can you set us up with the different department leaders? Like how can you get us in, right? And we also found out they did these monthly safety topic presentations. And of course, they had really good partnerships with the department leadership. So we decided to leverage that. We wanted to put a face to COEM, say, hey, we're here. We take care of your employees. It's, you know, whenever you get injured, we're here. But we're also here to help you prevent injury and kind of just be able to say, hey, these are, this is what we do. And these are our values and we're here to help. So part of what we identified through these clinics that we had was, number one, our times for vaccination when we were having our mass flu campaigns was really between 9 and 4, pretty much every day. The problem is, is that these particular unions, they have multiple shift changes. And they're not always working in clinical sites where they have access to peer-to-peer vaccination. Also, one of the things that we found is our custodians food service and our facilities engineering folks are about half a mile away from where these vaccine clinics are. And imagine you're walking this in the summer in San Diego. It's not terrible, but it's also not fun to spend your break time or whatever walking half a mile to get a shot and then half a mile back. So we had these vaccination clinics. We had eight of them. They were each one to two hours long. We targeted service workers and our facilities folks. We tried to schedule them around shift changes and their staff meetings where we could have the most amount of people. Basically, one medical assistant, myself, a cooler, a bunch of vaccines, and a 2013 Prius. And we went out. We were like, hey, we're here. And we decided to give the vaccines. So this is a couple pictures that we took during our vaccination clinics. It was really cool because we actually got to sit down and stand and talk to our folks and say, hey, this is who we are. We're COEM. We're here to take care of you. We're here to offer you the vaccine. Do you have any questions? And take a moment and just answer their questions, too. Right. And we were pretty successful. And we also learned quite a few things. One of those we found was really helpful to improving vaccination rates was peer pressure. So basically, between the employees themselves, they're like, no, you're not going to get it. Oh, you're a wuss. You should get the vaccine. I'm going to get it. And we're like, okay, cool. Everybody get it. Having us on site, I think, was really helpful, too. A lot of the employees had questions like, is this safe? Am I going to get sick? I always get sick. And having us there to kind of explain, hey, guys, you're not going to get sick. Feeling a little bit kind of junky after getting the vaccine is pretty normal. Your body is building a robust immune response. This is a good thing. Value of convenience, right? Being there, we found the employees really loved it. They're like, this is great, not having to go out, not having my manager harass me. This is wonderful, right? One of the other funny things, while we were doing the food service, one of the custodians came up. And he's like, oh, I want to get my vaccine. And then he's like, can you do this for us, too? So he grabbed me and took me downstairs to his leadership. And we ended up doing the custodians, too. That was at La Jolla. Leaders being vaccinated. So having their leaders there and seeing them getting vaccinated we felt was really huge. Lead by example. And gratitude, right? Thankful that we were there, but also us thankful that they were letting us be there and letting us vaccinate them. And honestly, swag. These pens, I don't know why, but people go bananas for these pens. They literally have a screwdriver, a flashlight, and a stylus. I don't know what it is about these things, but man, they were fighting for them. So, needless to say, for this year, we're going to order a whole heck of a lot of them. So, our intranet website, we went through and we did some updating. One of the things that we did, this is Andy, she's my medical assistant, and she was the MA that went to help me vaccinate, and I don't know why it's doing that. But she lost her sister during COVID. Her sister passed away during COVID. So, we felt that her story was really impactful. So, we reached out to our folks that do the in-house production. We recorded a video and we posted it up on our intranet site. And I apologize, I should have played. I don't know if it'll, yeah, I just don't know if it'll have sound. Yeah, it's not. One of the nice things about the partnership is that we partnered with UC Riverside. They had an educational champion. So, they came and did the video and it was really fantastic. And we're finding lots of friends and making lots of friends in the process. Again, we updated our intranet. And I think, Zeke, you brought this up at our last partner meeting, not using the word myths. So, we went back and we changed it. And we also, February of this year, we had a grand rounds in our department where we talked to our provider groups and our back office clinicians. Like, I know you were talking about, some of your docs were saying, hey, do you really need this? So, we felt it was important to educate our people and talk about hesitancy, strategies, communications. Also, Zeke did a really great podcast that's on that ACOM Vaxinfo website, which I may or may not have watched before I did this. But, super helpful and highly recommend. And then, let's talk a little bit about results. I think this was really interesting because, again, these were eight clinics. They were each about an hour long. And I think we were most impactful in our service folks. I think there was 100 people vaccinated, 100 people more vaccinated. Unfortunately, when all of this was going on, UCSD was acquiring a hospital. So, our numbers are a little skewed. We're kind of like, we're hoping that this year will be more normal. So, I don't think that the numbers are 100% accurate, but you do see an increase in the service folks. And our K6, which are our journeymen, very moderate increase. But, we only targeted K6 in one of our facilities. We're two, well, technically three now. So, I'm hoping for this year we'll be able to hit our other facilities, our Hillcrest folks. And now that we know with them that their shift change happens at 10 p.m. and they're gone by six, we know we need to be a little earlier so we can get those night shift folks, which I think is actually what brings that number down. And again, we talked about challenges. We're in the process of acquiring two, well, one now. We finished one, we're getting another one. Yeah, first it was almost 800 employees that were on board in two months, and now 1,500. 1,500. So, we're fighting for resources, ITIS, our own leadership resources to make things happen. And with that... Yes, sir. Is this on? Was this directed at anyone in particular? Okay, I'll give it to you. So, I think for us, for this year, one of the things is we're going to look at this data again, but I think we want to tailor those vaccine clinics even more. So, we really want to hit the people that don't have easy access to the vaccines. So, we know that our nurses, these folks that work on the floors, they have pretty high rates of flu vaccine compliance, but these folks that are the service workers, the K6, they don't have that access to peer-to-peer because they would have to go into the clinical areas to get the vaccine. So, we want to make sure that we're really hitting them and really putting a lot of resources. Last year was kind of like we're figuring out what we're doing, and now that we have a better idea, I think we're going to push in that direction. And the other approach is, of course, to address social discrepancies and inequalities, but also we found out that although we've been there for 25 years, they don't know much about us, occupational medicine, even where to go when you get hurt and stuff like that. So, one of the second agenda we have is to build trust, and with that, build a couple more preventive services and so forth on that. So, we just think this is just the beginning. Yeah, I mean, I think much like I described, we're still in the midst of some changes, but I think the big thing where I still have a gap to address is educating the clinicians, not only on making sure that people are in alignment about why vaccination is important and maybe doing some surveying, much like Dr. Lennick was describing, but also on how to maybe have some of these conversations in a pretty quick time period. I've been surprised that even on occasions when I would anticipate a patient might be like, ah, I don't want that, they're like, oh, actually, yeah, maybe I will go ahead and go get that, and I'm like, oh, all right, great. So, just like with smoking cessation, us bringing it up can make a huge difference. For us, I feel like on the online module, they access the adult vaccine quiz from the CDC, so they have that interaction, which is really great, that they would also get one-on-one with the nurse, but we haven't been getting that handout to them, so I want to try to find a way to get the handout to everyone who's participating. The other piece is looking at more ways to engage with the workforce. So, we've been giving on-site flu vaccines for decades, and they're inexpensive, so we can purchase them, and then during the pandemic, when COVID vaccines were free, we did all kinds of vaccine clinics for them, but now that there's a cost associated and we don't process personal health insurance, it's a barrier. However, to give a lot of credit to our nurse in Texas, she was able to get an external party to come on-site, and they processed the health insurance. So, she was able to still do another COVID vaccine clinic in an affordable manner. So, we're trying to look at how can we get that model in place, since now vaccines keep going up and up in price, how do we get the vaccine into the arm as conveniently as possible? Our numbers are too low to go into any further analysis, so that's where, overall, when we were able to look at it, we ended up picking up those 11 persons, but we need a larger sample size to evaluate more. So, we did a little bit of work, but I can't say anything with confidence because we don't have enough power. I will make a comment and say that there is definitely a correlation between people's intent to get it and them getting it, but to what my colleagues at UCSD were saying, convenience is a huge thing. So, if you can find out somebody's intention and have the vaccine for them right at the same exact moment, you've reduced a lot of barriers right there. Actually, to that point, while we were doing one of our clinics for our facilities folks, we met with some of the plumbers and were like, hey, did you get vaccinated for Hep A? And they were like, no, was I supposed to do that? And we're like, maybe you should. So, thankfully, their site is not very far from our clinic, so I actually went back and picked up a pack of Hep A vaccines and we gave them the Hep A vaccines on the spot. So, I think that that gives further, I think it gives credence to that whole hesitancy versus convenience. So, if I'm there and I have it and you need it, why not, right? And we care. We really try to instill that we do care and our institution cares. And then with that cascading to support not only for our senior leadership to recognize that they have to demonstrate that they care, to have a happy, productive, and that occupational medicine can be the bridge for that and, again, be one of the keys for their strategy for a productive population. All right, the gentleman in the back had his hand up for a while. Yes, sir. Thank you. It's a virus that's part of the disease. And it's a virus that's part of the population. And on that note, we have to consider that it's part of the native population. So we have to consider that it's part of the native population. And it's a disease that's developing. And we have to consider that it's part of the native population. So we have to consider that it's part of the native population. And we have to consider that it's part of the native population. Oh, you've asked the right panel about that. Do you want to go first? Actually, there is a recent paper about that. But it was more on pediatrics and the risk factors and stuff like that. So when we build the firewall, actually, when we are in this environment that we are vaccinated, we're hiding comorbids from ourselves. We could potentially, theoretically, run. But definitely, we would like to then have a... Well, part of it is already doing that, right? But we're thinking about using artificial intelligence actually more for workers' calm. So to the point of AI, we're doing some work with generating some educational videos with AI. Admittedly, it's a little bit painful, the software that we're using, Eli.io. But we do have one video that Dr. Zakari's son may or may not have put together for us. But yes, we're trying. So I think that's a great question. And I think there is a lot of application that you could put in there. So if you think about the WHO's 5Cs model for vaccine hesitancy and actually setting up an AI chat bot oriented exactly in those lanes so that when people talk with it and say, oh, I'm concerned about blah, it says, well, did you know? I think we can get there. We haven't seen a lot of that implemented out in the world yet. But I think we're... Not us, but I mean the world or clinical practice is probably only a few steps away from that. But that's a great idea and maybe a good session for next year. Did you want to... Oh, okay. I didn't want to add to it. Thomas, John, I think you were next, sir. So I understand you're going to focus on the environment in terms of vaccine confidence, of course, but also... Uh, you want to take it? We would love to do that. Our limitation has always been that we've only administered at our flu vaccine clinics. We do have family members come up. We've only ever vaccinated adults. But I think all of our messaging during Covid and through this have always been about protecting yourself and your loved ones. So I don't know if you guys have more of an opportunity for that, but we're kind of within the confines of what, what our system has always done. Um, and what vaccines we could provide because you also have to have different doses too. Yeah, I would agree with Dr Lennox. I mean, obviously as an occupational health clinic, we're not prepared to give vaccines to kids. Framing it around protecting your family is a big one. Uh, as I mentioned, of course, in clinical practice, more than even where these two institutions are at, like time is really of the essence and going to, oh, let me ask you about why you don't want to get your kids back. Like we, I don't think I could even push that yet, but maybe one day you guys may have more of a little opportunity for that. So actually, I just, just from my practice at UCSD, you know, we do have like some vaccine hesitant adults and just even simple things like happy. I don't know if I want that. And, um, part of my conversation with them sometimes when trying to, you know, explain to them why I'd tell them, Hey, these are like childhood vaccines. Like, chances are you already got this and you're fine. Um, so I don't, I don't know if that necessarily answers your question, but, um, but we're trying to like, you know, tie it back to like, you've may have gotten some of these already and you were just fine. But that's a very important point. Yeah. Yeah. So for education, it would work for us because we're focusing on on site doing work and we kind of crushing and their staff meetings. Um, it will be difficult and also to have this stuff be able to, uh, give the vaccine for a child. We're just not again, not prepared for that. Um, yeah, yeah. I'm a lot. Yeah. Uh huh. Yeah. And yeah. Well, um, yeah. Uh huh. Yeah. Uh huh. Yes. Okay. So, um, yeah. Yeah. Oh, yeah. Yeah. Yeah. I'll sorry for a second. Yeah. So that was a consideration we had in our clinical practice in terms of making these recommendations. Uh, I didn't dig into this in my slides, but you know, we started originally with asking people, you know, okay, well, can you just go to your primary care clinician or whatever clinic of your choosing? But for those who are uninsured, of course, figuring out what county they live in and referring them their local county public health and having those phone numbers available. And it may be for your local area, you know, 1, 2, 3, 4 counties may be enough to get people where they're going. But that would be my, I guess, first recommendation for sure. I used to be at health partners, and I remember still at our ST paul clinic. We always had a list of sliding fee scale clinics and things in the area that they could go to. So if they had high blood pressure, they just needed to go get that treated to pass their D. O. T. They'd be able to get an affordable visit, get started with their new employer, and then they have health insurance. So I think it's the same for vaccines in our learn module or three and learn module we created. We also have a link at the bottom on how to specifically get vaccines. If you're uninsured, we contemplated whether to have it or not because it's employees who have been offered a health care plan. Um, but then we thought, well, they might want it for a family member. Um, and we do have message board slides that go up at the plant. So you might not be a three M employee, and you might see this message board and be like, Oh, I can go get that vaccine somewhere else. So so I can get that website for you and send it to you if you want to connect with me. Yeah. Well, it's a little bit of a twist of an answer here, but I think the question I have is, I want to be wonderful if we could promote, uh, government incentives for employers to provide vaccine. Because if you had problems like that on a site that has like 10 people, you know, we will drive right with our 2013, uh, Prius and what the value that brings to the company is not like it's worth like three and worth investment that 30 bucks on that to not have people sick and stuff like that. But right now, what we face is that the covid cost like 100, $150. And, uh, we actually weren't conversations on partner with with Moderna on education promotion, but then end up with the commerce, our conversation. And because it's like, can you give us vaccine? It's like, huh, right? All right. I think we might be a little over, but I'm gonna call out. Is that? Is that Dr Grove in the back there? You had your hands up. Anyone here has a in the shop? Yeah. Um, what? Great. Great. I'll just say, I mean, obviously, from the project perspective, I don't know that any of us have had enough time to answer that question. I've read a lot of vaccine hesitancy literature, and I've not seen it time gated like that as much as, uh, what is the source of information? You know, friends, family, new sources, different particular sources, maybe more focal points for individual people. When we were preparing for the mandate before, you know, it didn't go through. We had one site with the highest vaccination rates reported out of the whole company, and it didn't have a site nurse. It just had this H. R. Person who had had a similar experience to what you were saying about Andy, um, where they had lost. I think was their husband or a very close member of their family to Cove it, and they were so passionate about getting people vaccinated that they just drove up the vaccine rates. So I think it's very I think it's personality based, honestly, like you have a history of trust, but you bring in some people just have that charisma, right? And you have, you know, so you need that charismatic personality whose passion and cares, and that goes a really long way. I think for us, actually, was almost instant. So Arthur says, Dr. Sarkar, come with me. So here I am, you know, driving, you know, to to the med center. Actually, I know my car. Yeah, but they were so grateful, and that was like we were so ourselves so energized and empowered because we're like, gosh, that's what we really like our work. So just to be there and show that we cared that, you know, we sat down and for five minutes answers the Q and A talk to their leader. And then someone say, Hey, can I go somewhere else? Yeah, by all means, we'll go. And we did say, you know, we really do care. And you so I, you know, for that experience itself, I think one of the things that it makes us so energized to do this and even multiply the site visits is to replicate that experience, right? We have time for one more. Yeah. Oh, he was just so Dr. Baker was saying was while it's optimal to improve adult vector adult vaccination rates in all age groups. This particular project was a collaboration between CDC and a com to specifically improve adult vaccination rates. But that doesn't mean your point. Thomas John was not well taken. All right. Thank you. I think I think we're here. We're way over. Yeah. Oh, yeah. We're having a talk. Thank you all for your engagement.

Dr. Karen Lenick

3M

adult immunization rates

vaccine information

quality improvement

workplace vaccinations

Health Partners Institute

Zeke McKay

vaccine promotion program

immunization records