[MUSIC] Let's go to a specific case. So, you were saying about a week ago you were asked get the, tell me that story. >> I was, a physician at Hopkins asked me how many blood clots occur. How many VT events are there? And a lot of times when I get questions, particularly from clinicians or researchers who might not have a lot of experience with databases, I get a question in the vernacular. That makes perfect sense to them, but might not make sense when you try to pull data out of databases. >> Because the computer is stupid. >> Because the computer stores information in a different way than we ask it. >> [LAUGH] You're very kind to the computer, keep going. >> But the very simple question of, how many blood clots are there, has a lot of different components that you need to consider when trying to answer that question. Because we need to understand where blood clots occur, where are blood clot diagnoses documented. How are they documented? How are they diagnosed? And it oftentimes takes working with the person who asks the question to really understand, what information are they trying to get? What question are they trying to answer? What are they trying to inform or influence as a result of their question? To make sure that you get the right data to adequately address what they're asking. And in this particular case, going through a series of questions with that physician, we arrived at he was looking for data for one particular year at the Johns Hopkins Hospital. As you know, we have multiple hospitals within our health system. So within the Johns Hopkins Hospital, we were looking for blood clots that occur in the hospital, not after discharge. We were looking for blood clots that occur during hospitalization after 24 hours. Sometimes we have patients who come to the hospital with a blood clot that's considered present on admission. We wanted to identify blood clots that we had an opportunity to prevent within the hospital. And then the question of where do we actually get the data? Because a lot of times, clinicians, or people asking things, think that if you ask what is a blood clot, what is the heart rate, it's just stored in one convenient area. And we can pull that information out. >> Piece of cake. >> Piece of cake. Unfortunately, it requires a little more refinement. >> So if we could do all that, you left, I think, the reason, I think you kind of hinted at it, but the reason they wanted this number was that- >> So there's a lot of effort on blood clot prevention across the country. And as I mentioned there are a number of organizations that measure blood clot prevention practices and outcomes. And unfortunately very few have similar criteria in determining prevention practice. As a matter of fact we recently published a paper comparing all the blood clot prevention measures, quality measures that are used around the world. And we found that they all seem to have different criteria. >> Are you going to make a new one? >> Well I think that it's important when you're looking at this from a quality and patient safety perspective. That when a blood clot happens, a patient has experienced harm. And depending on which quality measure you use, the patient could have been considered to receive perfect care. That there was nothing more that we could have done. Because the quality measures are defined to be measurable and easily reportable. However, when you look at the full continuum of care that a patient receives in the hospital, there might actually be opportunities to improve practice. So what we look at in our blood clot prevention would encompass the measures that are reported. But actually goes a few steps farther to consider, was the patient risk assessed? Was the patient prescribed the right prevention regimen? Did the patient receive every aspect of that prevention regimen? >> A couple of things come to mind. So first of all, the goal of this query, this request, it's not a query. The goal of this request implicitly was, I want to know how we're doing so we can improve things. So that's my understanding. >> That was the ultimate goal of the request. Where the request started was, how many events are happening. >> Right, I understand that. Like I ask my kids, they ask me a question and my first response is, okay, what's the real question that you're asking? But the other thing is you start out the whole conversation by saying that VT prophylaxis and treatment is a system event. So what number do you come up with that helps you that's sensitive to changes of the behavior of the system? >> Multiple numbers, unfortunately. >> Right. So that's why when you hear what is the number VT, at the same time, you know it's a system. See something's a little bit of a mismatch. >> It is to some degree. However, as I mentioned earlier we look at different things. We look at process measures, we look at outcome measures and we look at process linked outcomes. >> Right, but this person was asking for the number. >> This person was asking purely for an outcome measure. >> For the outcome number, okay. Now is this somebody who knows about VTEs or somebody that you had to educate about VTEs? >> This is somebody who knows about VTE events. And the question there was purely to gauge the absolute number. >> But I just want to point out you're playing an interesting role. You're not just the informatician, you're actually a domain expert at this point as well. Because you know VT really well, so you're not just the intermediary between a database and a client. >> That's very kind of you to say. But I also think that it's the responsibility of the informatician to know certainly a basic level of the clinical context of the question. Because there's no way that you can deliver effective data or effective information without understanding why it is that something is being asked. >> So we have this issue currently in informatics, can we put up a self-serve kiosk of our databases? So you just have the clinician go to the machine, they put in their value sets, they put in their time horizons, whatever it is. They press a button, they get a number, they walk away, and they're happy. And it sounds like you're saying, well. >> Well, I think that it's an achievable goal. But I don't think that the information will be 100% correct in the first month of implementing that type of solution. Well is it correct on the machine side, but much harder to correct on the clinician side? >> Well, I think that it needs to be correct on both sides. On the machine side, the machine will give you the information that you're asking for. It's just whether or not that's the right information to answer the question that you're asking. >> It's like Monty Python. What's your favorite color? Blue, no red. So you can get a right answer or an inaccurate answer or an answer that doesn't mean anything. >> I mean, so in the example that I was giving, for example, if we did not exclude blood clots or VT events that were diagnosed as present on admission. Our number of events reported would actually be multiplicatively higher than if we hadn't excluded that. So it's- >> And it would not be answering the real question, is, what can I prevent? >> What can we actually prevent? >> Right. >> So it's very important to consider all of these aspects that influence the question before you start randomly pulling data out of machines. >> How many times do you have to go back between the data and the client? >> So the longest meeting is oftentimes the first meeting. And the way that I like to approach this when talking to clinicians, and this example and in others, is to really walk through the clinical workflow and the clinical experience. If you have an outcome that they're trying to get at, what are all of the processes that lead up to that data being documented somewhere in an electronic health record system? What are all of the potential influences that can tell us what the quality of the information coming out is what will actually address the information? And after walking through that clinical workflow, seeing it through the clinician's eyes and translating it into something that we can actually query a database. Then I start breaking apart that question and asking in specific details. I want to define what the population is, who is it that we're pooling information from, what exactly is our outcome? So for example If you're asking me about what is the average weight of the patient. Is it the weight when the patient was admitted to the hospital? Is it the average weight throughout? Is it the weight at discharge? You certainly have patients in the ICU who are getting many litres of fluid and their weight changes over the course of hospitalization. These questions aren't just one-time answers. So really understanding what that outcome of interest is. >> I think you used, in order to frame your query, you used the PICO or PICOT approach, patient intervention comparator outcome and time horizon. Which in this course we actually discussed in the evidence-based medicine section, in the intervention decisions board course. And then another way of looking at this is the who, what, when, where, why, and maybe how. So to kind of make sure you get your bases covered. Can you say anything about the query that you ended up doing for this particular story? >> So in the end, we defined one year that we were going to look at. And specifically at the Johns Hopkins Hospital, we were interested in patients who developed VT events that were not present on admission, but that occurred during hospitalization. So not after discharge. >> Now, hold on a second. Let's go back to the time. The year you chose was, how many years ago was that year that you chose? >> So the year was one year ago, one entire academic year. >> And the reason you chose that, why was that? >> The reason that we chose one academic year is that residents begin on July 1st and end their residency on June 30th. So new interns who are coming in and who are largely responsible for risk assessing patients may start out with lower performance than we might like. We implement interventions to try to improve performance which can- >> During the year. >> During the year. So we wanted to have that wash in period so to speak. And that we wanted to have a nice sustained period of time that we could look at practice over the course of time. >> But you didn't go two years ago because? >> We didn't go two years ago because our quality measures are reported on an annual basis. And so we wanted to measure what our outcomes were against what some other organization measured us against. >> And you didn't do it this year because the year's over. >> Correct. >> What I'm ultimately going to is some notion of trying to also project forward. So you want a full year because you want to project forward what's going to happen this year or the following year.
