So, I'm here with Felix. And today we're going to talk about logic models. So Felix, obviously when we want to evaluate our intervention, we need to understand how it works and we can model that in different ways. So can you tell us a bit more about these logic models? >> Absolutely. So think about digital health evaluation. It's an interesting business and actually if I'm honest I think there's a lot of digital health stuff out there in the real world that isn't very well evaluated at all. Not to rigorous scientific and academic standards. And sometimes that's because it's people from within digital industries or people that aren't used to sort of the medical academic way of doing things and that's understandable. You see a lot of things evaluated by I don't know, how many hits, how many downloads, time spent on screen and that's okay, but often in health, we're trying to move health outcomes. We're trying to change people's behaviors. We're trying to make people healthier, lead longer healthier happier lives. So we really need to understand what we're trying to move, so that's why we need a logic model. So in this process, I'd like people to think of a structured way to understand their intervention and what it's trying to do, how it's trying to change the world. There's lots of different language people use in different settings here. And so I'm going to use the language of a logic model and that's something from the traditions of healthcare and evidence-based medicine, but you might hear it referred to in other bits of businesses, benefits realization or plain old evaluation. But it's sometimes in the theory. It's a theory of change but it all means kind of the same thing. So what we're trying to do is take the intervention and understand what it does and the process it goes through to achieve the change in health that it's trying to do. And what I'm going to talk to you today is a structured process to allow you to do that. >> So is this something that we should do before we think about evaluation or something that we should do early on in the process? >> Absolutely killer question. You've got to think about it at the start. If you try and evaluate something post-hoc, it never quite works out, you find you're chasing your tail. So as you're designing your intervention, you should also be designing your evaluation. If you do it this way around I promise you it's going to save you time for the long run. >> So thinking about these logic models, they're obviously very important for our evaluation but in practice, how do we go about developing one of these models? >> Great question. So a simple way to do it is to follow the template that we provided. I mean there are other more sophisticated ways, but you'll find in the notes and the readings, sort of a guide to do it step by step. So that might be a sensible way to do. So the first thing you need to do is think about your overall question, your hypothesis. What is it that you're trying to move? What is it that you're trying to measure? I also want you to think about the context that you're working in. Describe the environment, the organization, the culture. The wider political environment and think about some of those issues. I want you to think about the barriers that might exist. What are the things that might stop or prevent or harm or slow down the adoption of your intervention? So that's describing the wider environment and then I want you to think about the classic components of a logic model. Structure, process, and outcome. Structure, it's well those things that you've got available to you. Your intervention, the money you've got, the people you've got, the resources. Then it's process. What are the things in your health system or with the people, if you're trying to change people's behavior, that you're trying to change? The simple activities as part of their everyday day-to-day life. And then the outcomes are those health outcomes you're trying to measure and they might be both be short-term and long-term. Long-term are normally those oval things like improving health, making people live longer healthier happier lives. If you try and measure them, you often get stuck because it takes ages to change them. So one of the more proximal outcomes that you want to measure. So for example, if you've got a physical activity intervention, it might be counting the number of steps they've taken or the number of active minutes they're walking but it's trying to find the proportionate outcome measure that gets you towards your eventual health goals. >> So it sounds like it's a real team effort to develop one of these logic models. >> Well, absolutely. So you probably need to get all the people who are working together with to develop your intervention into a room. You probably need the digital people, the clinical people, maybe even some extra academic input and get them all sitting around the table. And actually the act of trying to plot what you're trying to do and come to a shared consensus about the vision and your intentions. It's actually a really useful task and many people find it provides additional clarity, if you do this process at the start and you agree on the goals that you're trying to achieve and change. >> So we've talked a lot about evaluation of digital health interventions and why it's so important and but why do you think logic models in the process of developing these models is so pertinent for digital health? >> Well as I mentioned digital health tends to be under-evaluated. People tend to measure and use the wrong outcomes in their evaluations. And by going through this structured process, you really think downstream to those behaviors or health outcomes that you're trying to change and then once you've worked out which of those indicators you're trying to move, you then need to think carefully and practically and pragmatically about what type of experimental design you want to do. But almost shouldn't be thinking about an experimental design until you've worked out which of the indicators you want to move. So there's almost a sequence to it. Build and design your evaluation. While you're doing that, build and design your theory of change in your logic model. And once you've come to that conclusion about what you're trying to move think about what the best method to do it, whether it's experimental, observational. And there's a process to go through but with everything digital it's iterative. Think about what you're doing, stop it, change it, be agile in your thinking. >> So it sounds like logic models can really help us be a stepping stone around making sure we're on the right track with our evaluation and making sure we've considered all aspects of our digital health intervention very early on? >> Absolutely and one of the common things actually about digital health is sometimes it has unintended consequences. There's some really brilliant things that digital health does. It often increases access, reach, and provides things at lower costs. But sometimes it produces things that we're not so keen on. Perhaps the access will but only be in certain groups. It might even widen inequalities and in public health, we're always concerned to make sure that we're ideally reducing and definitely not widening inequalities. So there's something useful sometimes in the logic model about saying are there any outcomes you want to be measuring to make sure that they aren't going in the right direction. This has been referred to in the literature as a dark logic model. So always be thinking about those unintended consequences as well. >> That's great. Thank you so much for your time Felix. >> Very good. Thanks. [MUSIC]
