Lessons Learned: Out of the Crisis #11: Jeremy Howard on the power of masks, health policy, and data science in medicine

Monday, June 29, 2020

Out of the Crisis #11: Jeremy Howard on the power of masks, health policy, and data science in medicine

Jeremy Howard has had tremendous impact on the world of AI entrepreneurship. fast.ai, the company he co-founded with Rachel Thomas and Sylvain Gugger is working to "radically democratize" deep learning by making it accessible to people beyond the tech world through courses, software and other methods. Enlitic builds machine learning-based software for radiologists that produces more accurate and faster assessments.

Data science is the foundation of all of these enterprises. It's also what led Jeremy to sound the alarm about the value and effectiveness of wearing masks when the pandemic arrived in the U.S. The "drip drip drip" of data he was seeing from countries that were hit first told him what was coming, and he made it his mission to start a campaign for masks -- Masks4All. His goals are two-fold: to get the American public to understand how crucial mask-wearing is not just for individual protection but for the protection of everyone around us; and to lobby for policy change that would make wearing them the law.

He was joined in his efforts immediately, as he told me, by dedicated and talented "self-selected people" who connected him with media outlets, politicians and more. An article he wrote for the Washington Post, published on March 28th, when America was still reeling from the exponential growth and the CDC was telling the public not to wear masks, was the first real call to arms for masks. It was followed a month later by a piece he collaborated on with Zeynep Tufekci and Trisha Greenhalgh, explaining that the real reason to wear a mask is to protect others, rather than ourselves. It detailed modeling that shows that "if 80 percent of people wear masks that are 60 percent effective, easily achievable with cloth...that’s enough to halt the spread of the disease."

Jeremy and I sat down to talk about the creation and growth of Masks4All, what it was like to sit at the center of health policy debates, statistics, and many other things. In the time since our conversation, data and science have shown again and again that wearing masks is an incredibly effective, low-cost and simple way to prevent viral spread. The difference in caseloads between counties and states that have done it and those that have not is stark. Models show that using them can make a huge impact on what lies ahead, too.

You can listen to our conversation on Apple, Google, or wherever you like to download podcasts.

A full transcript follows the show resources below.

Highlights from the show:

Jeremy on his background in AI, data science, and his education (2:17)
Jeremy on starting at McKinsey at age 18 (4:30)
On how his ability to look at the data early on affected his quarantine setup (6:57)
On the collaborative efforts among strangers to fight COVID (7:37)
What Jeremy was working on when the pandemic hit (10:44)
How masks came into his life and how he founded #Masks4All (14:23)
On the moment when we were all told not to wear masks and how that changed (18:05)
On the reasoning for not wearing masks and the science going on behind the scenes to prove they were effective (21:33)
On how health policy is made and communicated (25:58)
The class, youtube video and article that became the first U.S. call to action on masks (28:10)
Petr Ludwig and nearly 100% mask usage in the Czechia (32:30)
Wearing masks as an act of resistance in Czechia (35:35)
Jeremy on people are taking matters and health into their own hands to make change (37:35)
The sudden availability of experts as volunteers (42:43)
Jeremy's TV appearances on the mask issue and what surprised him as policy changes occurred (44:31)
Credentials: why they matter and why they shouldn't (52:22)
Jeremy's public persuasion campaign to get people to contact officials about masks (53:47)
Using Resistbot to get the message across (55:27)
The number of states that had mask laws when Jeremy started, and what happened next (58:13)
States that have mask laws (59:35)
The issue of masks vs. the virus intruding on rights (1:01:49)
Conversations with policy makers (1:02:54)
Measuring the progress of #masks4all (1:04:32)
The effects of doing things in minutes versus doing them in weeks (1:07:46)
Asymmetric upsides and downsides of the pandemic (1:09:19)
The history of masks in the 1918 flu pandemic (1:10:14)
The misguidedness of doing nothing until you can do something perfectly (1:13:40)
Jeremy's advice for what people can do -- the drive train approach (1:15:12)
What Jeremy hopes we take into the future from this experience and where we go from here (1:20:29)

Show-related resources:

Transcript for Out of the Crisis #11: Jeremy Howard

Eric Ries: This is Out of The Crisis. I am Eric Ries. If you go outside, you need to wear a mask. You know that, right? But how do you know that? Think about how quickly we as a society have gone from thinking that masks were something that were pretty strange to where in public to understanding, not yet completely, that they are a necessity. It's not a coincidence. It didn't happen by itself. People had to make a change. Some of them were politicians and leaders, scientists, public health officials, but some were just citizens who realized that something new was needed. I normally start by saying something about where we are in this crisis, however, my guest for this conversation is so concise and clear about the data and what we all need to do that I'm just going to let him summarize it.

Jeremy Howard is not a medical doctor. He's not a lobbyist. He's not a politician. He is a data scientist. This data scientist is the reason that many of our cities and states are telling us all to wear a mask outside right now. Jeremy never thought of himself as an activist, and he certainly didn't expect to be part of the reason that many cities are flattening the curve as we speak. But he is leading the #Masks4All movement. #Masks4All is advocating for everyone to wear a mask to stop the spread of COVID-19. That may sound like common sense today, but it was only a few weeks ago that this was a very radical proposition. This is a citizen-driven movement that has led whole states, countries, and public health agencies to revise their guidance on masks.

Jeremy and I talked about why he is such a big believer in masks and what the data says about why masks are important. I've learned a lot from him about how to use data responsibly, and about how we can be more evidence driven in our decisions as a society, as policymakers, and as individuals. Here's my conversation with Jeremy Howard.

Jeremy Howard: I'm Jeremy Howard, a research scientist at the University of San Francisco. I chair a medical data research lab called WAMRI, and probably best known as the co-founder of fast.ai, although increasingly finding myself known, surprisingly enough to me, as the co-founder of this thing called #Masks4All.

Eric Ries: Jeremy, thanks so much for taking time to talk to me, just give us a little bit of a sense of your background before the crisis hit.

Jeremy Howard: Sure. It's a bit of a weird background. I'm mainly known as an AI guy nowadays, since I'm the co-founder of fast.ai, which is a reasonably well-known research lab that also does teaching possibly the most popular AI deep learning course in the world.

Eric Ries: It's really good.

Jeremy Howard: Thanks. And we have a software library called fast AI, which is the most popular layer that sits on top of PyTorch for deep learning. Before that I founded a company called Enlitic, which was the first company to focus on deep learning and medicine. You can see why I'm known as an AI guy. Before that I was an equal partner in Kaggle, which is a popular data science competition community. But before that I did a range of different things. I spent 10 years creating and running an insurance pricing company, 10 years creating and running a popular email provider called Fastmail, and almost 10 years as a very boring corporate strategy management consultant at McKinsey.

Eric Ries: It's a very eclectic set of work experiences. What was your early life educational background like?

Jeremy Howard: Not much really. I started at McKinsey when I was 18. Although in theory I have a Bachelor of Arts, I didn't actually go to any lectures, I only went to the exams. It was a bit of a stressful time because I didn't go to any lectures so at the end of the term, I'd go to the teachers and say, "Hi, I was actually in your course. Did you have any assignments? Could I do it by tomorrow? Because I know it's late, but you know..." They always said yes, somehow.

Eric Ries: Very kind of them. How did you get into AI originally?

Jeremy Howard: Well, I was always super interested in what people now call data science. That's been the common theme throughout my career. The reason I was able to get into McKinsey at 18 was because I use this data driven approach. McKinsey called me an analytical specialist, which at the time I thought I was the only one, but I found there was two others around the world. We found each other. So I was always doing linear programming and operations research stuff, and regression models and whatever to try to solve corporate strategy problems.

Eric Ries: How did you even have the idea at 18 to take that to McKinsey?

Jeremy Howard: Too much fiddling around with Lotus123, I guess. There was actually a teacher who taught us... I think we had one class about spreadsheets in high school and I was just like, "Wait, this replaces nearly everything we've learned in math and science." So I just got to do nothing but this. So I did spreadsheets and then the guy that lived across the road from me was like, "I'm a management consultant, do you know what that is?" I was like, "I have no idea." And he told me. And I was like, "Okay, I want to do that too." So he said, "Okay, well you can look at all this human resources data for this mine in Ghana. You could probably use your spreadsheet to like crunch it."

And I did and he was thrilled. And then this thing called Microsoft Access came out, which I learned was this thing called a database. And that was like a spreadsheet on steroids. And so I started doing that with this management consulting guy. So it was all like, I don't know, something I should have been doing in high school, but instead I was doing this I suppose.

Eric Ries: Oh my goodness. I can really relate. I have similar stories about computer programming from my early days.

Jeremy Howard: I mean, I was never much of a programmer. It was all like...

Eric Ries: No, I know. That ability as a young person to get adults in the grownup world to take you seriously-

Jeremy Howard: Well it was so much easier then because none of them knew how to do this stuff. So by the time I got to McKinsey, they thought I was some magician, which is good because the rest of them used interviews and expertise and knowledge, which I didn't have any of those things. So data was my fallback.

Eric Ries: I feel like this is like foreshadowing for things that are going to happen later. That's pretty awesome. These are dark times and it's been very stressful for all of us. First of all, how are you? How's your family? What's your quarantine set up like?

Jeremy Howard: Sure. So we looked at the data a few months ago and realized this is going to be bad. We weren't sure how bad. So working on the precautionary principle, we assumed it could be really bad. So we left San Francisco a long time before the lockdown happened, went somewhere remote, but reasonably close to a good hospital. And we've been hanging out there since.

It's been okay, we've got a four year old child and she's just... It's amazing how adaptable she is. She spends more time on zoom each day than I do just hanging out with her friends and they somehow keep themselves amused.

Eric Ries: I'm very familiar.

Jeremy Howard: Yeah. It's been interesting because some of our family are healthcare workers. Some of them have got sick, but then they got better. So there was like scary moments. So plenty of dark bits, but also this weird discovery that you can create incredibly close connections with people that I've never met before and build a global community of people who want nothing more than to help save lives. And that's been remarkable. I've discovered this... I've never had such a collaborative period in my life, but by far, and I say this as somebody who's done a lot of collaborations, so there's been interesting highs and lows like that.

Eric Ries: I had so many similar experiences and I was just, someone was asking me something about one of my collaborators on one of these projects. And they asked me about his background before the crisis. And I realized, I knew absolutely nothing about him.

Jeremy Howard: Yeah.

Eric Ries: I couldn't answer even the first question. I said, listen, we're bonded for life. I would do anything for this person, but I actually don't really know them. We've never met.

Jeremy Howard: Yeah. Well, I mean, same with you, right?

Eric Ries: Yeah.

Jeremy Howard: I vaguely knew who you were because I was at Singularity University overlapping when you did some stuff there, but I don't think we ever bumped into each other and I know some of your work, but then I was introduced to you for reasons, totally unrelated to any of that. And we've been helping each other out or mainly you helping me out, which is an example of this.

Eric Ries: Well, I hope it's been helpful and certainly you've been helping me and so many of us through your work on #Masks4All so I think we all owe you a debt of gratitude for that.

Jeremy Howard: Well, it's boring and horrible as all hell, so I'll take it.

Eric Ries: Yeah, exactly. I don't think this the kind of work that anyone does for fun. Do you have a favorite quarantine tip?

Jeremy Howard: It's possible to live on a life of a diet of nearly entirely carbs that come out of your freezer and not fall apart. It's very different to the diet I used to have of like fresh fruit shopping every day, but it seems to be working okay. I've also discovered that I don't actually need my home Olympic lifting gym as much as I needed. I've discovered much more about the ability to use body weight exercises to keep myself from getting too flabby.

Eric Ries: I think we've all had that learning. That's great. Yeah. Yeah. Appreciate you sharing. All right. So where were you? Do you have a moment that you recall when the severity or the reality of the pandemic first occurred to you or became manifest to you?

Jeremy Howard: No. I mean, for me it was a drip, drip, drip of data. I think as soon as we saw community transmission, strong signs of asymptomatic transmission during, I don't know, January, then into February and happening around the world, it seemed like there was a pretty high chance this is going to be a very serious global pandemic.

Eric Ries: What were you doing at the time? How were you spending your time between fast AI and, and that year university commitments?

Jeremy Howard: Well, they're kind of one and the same. fast.AI and my university is just amazingly great. They're super flexible and helpful. So really everything I do in fast.AI is connected to the university and vice versa. So my focus has been on taking our research software and courses to the next level. I really felt like we had gotten to the point where we had a really good way of doing and teaching deep learning. And so we decided to take 18 months off, pretty much teaching and doing anything else and focus on making our software as good as it could be. So we rewrote it from scratch. We wrote a whole new kind of development environment for writing it from scratch.

And we decided to write a book about it. And we wrote a whole new kind of publishing system for writing the book. So it's kind of like being this really cool eighteen months of working with my colleague and we were coming to the end of that process. So the book was kind of through the middle of the year and recording the course started a few weeks ago. So it was really getting to a very exciting part of, I guess, the last five years or four or five years of my life building up to this moment of the first half of this year.

Eric Ries: And then life throws you a wrench into those plans?

Jeremy Howard: Yeah, a little bit. I mean, we've actually finished the book. And so, hopefully it won't be too late. People have already pre-ordered it, lots of people. And the software already works great. It still needs some documentation cleanup and I'm still teaching the course, which has actually been super weird to spend time every week in the middle of all this stuff that's going on.

Eric Ries: Yeah. But to be clear, I think it's important for people to understand, you're not an epidemiologist and you were not working on masks or quarantine and pandemic related topics, I mean, even as recently as a few months ago.

Jeremy Howard: Oh God, I would never choose to work on masks if I didn't have to. Good Lord, no. I could think of nothing worse. Yeah, no, absolutely. I do a lot of medical work, particularly, because as I mentioned, I was the founder of this company called Enlitic, which was the first company to focus on deep learning and medicine, but my focus is more on medical imaging. We do a lot of work... We work with Stanford and Harvard and UCSF and a bunch of academic medical hospitals to help them bring modern techniques to their projects.

So like one of the really cool things we did was to work with the Salk Institute, which is arguably, the top life biology lab in the world. And we worked with their core microscopy team and built a new algorithm with them that we co-published, that basically allowed them to get orders of magnitude higher resolution from their existing equipment, allowing them to map the connectome of a brain and how it changes over time in ways that have never been done before.

So that was the work I was doing. And it was a lot, lot, lot more interesting than masks.

Eric Ries: So how did masks come into your life and how did you wind up founding this group?

Jeremy Howard: Well, one of the things I care a lot about is how we understand data as evidence. And it drives me crazy to see how most medical research has treated it as this binary decision with randomized controlled trials and P values. And it's like, "Oh, this is statistically significant or it isn't." So one of my long running rants has been about how we should gather all the evidence we have to try and make a best guess as to the distribution of possible outcomes based on possible upsides and possible risks and costs and make an overall assessment of that totality of evidence and impacts. So I was teaching that in this course, because it's not just a deep learning course that covers a lot of machine learning and data science topics like evidence and probabilistic reasoning.

So I wanted a case study and one of the things we talk about a lot in the course is ethics. And we particularly talk about the idea that a data scientist should have a responsibility to not just analyze data, but to do something with it, to stand behind their results, to talk to the people who can use them. So I wanted to get something current and relevant to policy. So I noticed from stories on social media, I guess, that countries that were using masks a lot, seem to be getting better outcomes than countries that didn't at like hundreds of times better, which is exactly the evidence which a lot of medical folks tend to ignore because it's so imperfect.

Eric Ries: What kind of evidence were you looking at?

Jeremy Howard: I mean, just the data like Taiwan, Hong Kong, Mongolia, right next to China, a lot of trade, a lot of social back and forth, Chinese New Year, yet they're in double figures, numbers of deaths. I think it's four or five in Hong Kong. They've kept their economies open. Restaurants are still doing business, compared to nowadays in places like London and New York, you're getting thousands and thousands of deaths every week. And even at that time, it was very, very obvious that the trajectory in these countries was extraordinarily different.

Eric Ries: I think for a lot of people listening, they'll think to themselves, "Wait a minute, I was on social media at the same time as you were. I don't recall seeing this data or understanding it." So just talk a little bit more about what you actually... How that data came into your awareness. What did you actually see? And then what was it about that data that prompted you to want to take action versus viewing it more as an academic curiosity?

Jeremy Howard: Yeah, well when I say social media, for me, that means Twitter and I'm very careful about my use of Twitter. I use the API to find interesting topics and accounts and kind of go out from there to find a range of perspectives. That might be interesting. So my Twitter feed is fairly carefully curated. And I'd say at this point it wasn't, I was determined I was going to make a big deal of this masks thing. It was more, as I said, I was just like, "Oh, I wanted to teach a lesson." And I thought, I'd just explore a little bit for a few hours to see if this would make for a good lesson.

And so really, what happened was I was just shocked by what I found that the quality of the evidence was still a little sketchy, but the apparent size of the impact was astonishing like many orders of magnitude in terms of human life and hundreds of billions of dollars of potential economic impact. And I thought, "Wow, this is really surprising that I just don't hear anybody talking about it." While I was doing this, there was an article that came out in the New York Times from Zeynep Tufekci, basically saying, "Hey, you guys telling us that masks don't work was kind of a dumb thing to do." She's a professor of sociology. So that was-

Eric Ries: We'll put a link to some of her writing in the show notes. It's astonishing how Zeynep is always first to so many of these ideas.

Jeremy Howard: Yeah, yeah. She didn't go as far as saying we should all be wearing them, but she was like, "You should at least not be lying to people as a matter of health policy."

Eric Ries: Explain a little bit about the state of thinking at that time. I remember that there was a strong statement from, I think from the CDC saying don't wear masks. And also explain that we're talking about masks for the general public. We're not talking about N95 masks or the masks for nurses. Just say a little bit about what was the controversy at that time.

Jeremy Howard: People just used the word mask. They didn't just really distinguish the idea that there were multiple types. The surgeon general got out on Twitter as I was doing this research and said, "Masks don't help. So don't use them and let the healthcare workers use them." So he was kind of the one who made the most direct and clearly wrong statement. The CDC and who were just more like highly misleading. The WHO still has a policy of saying, "You should only use a mask if you're sick or you're with somebody who's sick." But of course we have no idea who is sick. So it's kind of--that's weird. Other health bodies like the American and European disease control centers generally had variants of the WHO, which were very different to Taiwan that invested massively... Well, their SMEs basically invested massively to get to a point where they could create 10 million masks.

I think it was 10 million a day for 30 million people in Hong Kong, where about 90% of shops require a mask to go inside. Mongolia, which I think had a law about requiring them. The interesting thing for me, I'm very interested in... I've got an implicit bias in racism and stuff like that. And to me, the idea that these countries that have fairly recent experience and expertise with respiratory pandemics were being ignored when it came to policy. And I kind of thought, "Well, I wonder if that's because they don't look like us." People assume that they're somehow different. And so it's one of these kind of systematic bias issues that I'm very interested in digging into to see whether the data actually tells us something different to what our biased intuition suggests.

Eric Ries: So what was the reasoning for not wearing masks, especially given the data that you were seeing... I don't know if you want to do a rant on statistics.

Jeremy Howard: I mean, I wouldn't exactly call it like that. That's certainly like a lack of probabilistic approach. So there's this thing called evidence based medicine, which drives me crazy. And I do plan to write something about this with folks that are experts on this. Evidence based medicine is this kind of very binary thing where you do a randomized controlled trial and you find out if the P value is less than 0.05 and if it is, you say it's significant and then you recommend it. And this is actually totally out of line with the guidelines of the American Statistical Association. They have seven guidelines around this, which clearly say don't be using P values and stuff as the basis for your policy decisions. Because a P value tells you nothing about whether a relationship doesn't exist. It also tells you nothing about if it does exist, how important it is, like it doesn't actually matter.
So the... And the WHO is very, very, very into this approach. So I've spoken to lots and lots of advisors to the WHO and they're all feeling kind of crazy about this. Because they're all saying, this so much looks like maybe the most important tool we have. It's possibly as important or more important than distancing and it's certainly a hell of a lot cheaper. And the WHO saying well effectively when we're not going to recommend it in the absence of a randomized controlled trial. And here's the thing, it's actually impossible to run one.

So what would a randomized controlled trial for the impact of masks on community transmission look like? Well it would have to be at a community level, otherwise you can't tell the impact on community transmission. So it'd have to pick a hundred cities and then say to fifty of them--and that hundred cities will have to be representative of the population you cared about. And then you have to pick fifty of them and say, "Nobody in these cities is allowed to wear masks." Then the other fifty you'd have to say, "Okay, everybody in these has to wear masks." And that would never get past an Ethics Review Board because you can't... We have such a strong Prior that masks work, that telling large groups of people not to use them would be very likely to lead to a massive amount of deaths.

Eric Ries: So it's interesting. It sounds like there are these theses out there, where the Prior is strong enough that you think it's a good idea that you can't run a trial without it. But the evidence is not strong enough, according to the P values that you can actually endorse it as the policy.

Jeremy Howard: Well, you can't even get a P value.

Eric Ries: You can't. It's unstudiable and therefore not recommendable.

Jeremy Howard: Right. It's studyable using observational techniques. So what I did for this lesson was I tried to show a range of techniques that data scientists can use to gather evidence in the absence of a randomized controlled trial. So we can do things like say, "Okay, how was this transmitted?" And it turns out it's nearly entirely transmitted through droplets. Seems like mainly when we speak, maybe also when we breathe. And it's like, "Okay, let's just look physically at... Do those droplets get stopped by a piece of cloth?" And so I actually came across... Somebody actually reached out to me and said, "Hey, I've been doing a kind of a secret study of this in a laser chamber." And it'd have to be secret because publishing this stuff at that stage was basically a career ending move. So all the scientists I spoke to were not able to go on the record about their work. So I knew about a lot of science that was being done in secret. And they-

Eric Ries: It’s antithetical to the scientific reader

Jeremy Howard: In theory, it should be. Yeah, but the problem is when you do a piece of work that suggests that the leading health policy bodies are saying the wrong thing, it does tend to undermine the credibility of the big health policy bodies. And that's a huge problem, because we need to, people need to take vaccinations seriously and take condoms seriously. So there's a lot of historical reasons why we protect these health policy bodies. And so the fact that they've messed up so badly with masks is not just telling people because they're not wearing masks, but it's also undermining the credibility of a system that we really need people to trust.

Eric Ries: Yeah. I don't know if you're familiar with the book, Good Calories, Bad calories by Gary Taubes, but he's a science reporter who looked into why so much of the nutritional research went astray over the past few decades. And this issue comes up again and again and again, as people discovered contrary findings but suppress them because they didn't want to undermine the public health authorities. Because they lacked that information we're giving people the wrong advice.

Jeremy Howard: Right. Well, I mean, one of the things we can certainly talk about is how much I've learned about how health policy is done. It'd be nice to imagine it was this kind of science- based thing and what the WHO says is based on what scientists say. But unfortunately there's this huge disconnect and the policy in false politics. And so once a health body says something, they have a tendency, particularly in the WHO's case to kind of stick with it, even as new information comes along, for far too long.

And they also tend to communicate in ways that are really unhelpful to the public. So for example, they still have this GIF up on their site, this picture saying "COVID-19, or the particles, is not airborne." Which is strictly speaking, true if you use a really kind of wacky definition of airborne, that doesn't mean borne through the air, but means reaches a certain droplet size, which hangs around for a certain amount of time and potentially go through vents to other floors of a building and so forth. But it's absolutely true for sure that nearly all transmission seems to be borne through the air.

Eric Ries: Right.

Jeremy Howard: So policy bodies communicate in weird ways and they make decisions in weird ways and there's this huge disconnect between kind of science and policy. Which means that for me to get the policy changed here, I actually had to become an advocate and a campaigner. It wasn't enough to just find these scientists that were doing good research and to help kind of publicize them and create our own research and so forth. That's just not enough to move the dial on policy.

Eric Ries: Let's go back to having this realization, that masks were going to be an important part of the public health solution. You're a university professor, I think the caricature of a university professor is maybe you would write a paper about it or teach a class about it. Or maybe if you're feeling really adventurous, tweet about it or something. But you wound up doing something different. Why?

Jeremy Howard: Well, I mean, I was a caricature professor, I taught a class about it. So I said, like, "Hey look, gang..." So I've got about a thousand people in this class today that I teach each week. And I said, "Okay, let's start looking at this evidence based kind of data analysis. So we're doing masks this week. And here's what happens when we look at the population based epidemiological evidence. And here's what we look at when we look at the physical evidence. And here's what we look at when we look at kind of efficacy studies. And overall, we can put all this together and we can estimate the economic value of a mask as being something like $5,000 per person per mask, blah, blah, blah."

And so I don't know, that was like 10, 15 minutes of class. And so my students were like, "You've got to tell other people this. Don't just tell us." Because normally we don't make our material public for a few months. So I was like, "Okay." So I extracted that little bit and put it online.

Eric Ries: Do you remember when this was?

Jeremy Howard: About four weeks ago I guess.

Eric Ries: So this would be mid March basically?

Jeremy Howard: So I can tell you the date, 24th of March.

Eric Ries: So you taught this class on March 24th?

Jeremy Howard: Yes and uploaded the video to YouTube on the 25th. And then on the 26, much to my surprise, an editor from the Washington Post emails me and says, "Hey, Dr. Howard, can you write an article about this?" And I was like, "Well, A, I'm not Doctor." People keep assuming this, is just like, I'm just a data scientist. "And B what do you mean an article?" And he was like, "Well, this actually seems like something that's important." And at this point we had actually, my fast.ai co-founder and wife, Rachel Thomas and I, had actually, couple of weeks earlier written an article on our blog saying like, "COVID-19 is actually kind of going to be a big deal. Please take it seriously." So we'd already kind of done a little deep data science dive into that and much to our surprise, like a million people read it, even though we just put it up on our blog.

Eric Ries: Yeah, I remember that. It was a time when there was very little information that the popular... That was with high quality that members of the public could consume.

Jeremy Howard: Yeah. Yeah. A lot of... We heard from a lot of hospital administrators and people running companies and stuff saying, that was the reason they started taking it seriously and they started canceling their events. And so that was cool.

Eric Ries: We'll include a link to that article in the notes.

Jeremy Howard: Cool.

Eric Ries: So you can... I want people to be able to see how fast, how far we've come in such a short time in terms of our understanding of this pandemic.

Jeremy Howard: Yeah, it's ancient history now.

Eric Ries: Right.

Jeremy Howard: But again, I was tired of this thing of like, "Hey, we're data scientists. This is what the data says. This... That it's going to be hard for us to act rationally because our kind of gut is not used to dealing with exponentials." So I guess when this editor reached out, I was like, "Okay, I guess I know that our writing can make a difference. So, all right, I'll give it a go." So I wrote an op-ed and I'm lucky enough to have some fantastic friends, particularly folks that I've met through the World Economic Forum, who I reached out to, who write op-eds all the time. And they made my op-ed way, way better. And yeah, I sent it in and thanks to them, it turned out great. And apparently over a million people read this thing in the Washington Post, which, the title that I think they picked, I don't... Can't remember who picked, it was, "Simple DIY masks could help flatten the curve. We should all wear them in public." And then the subtitle was, "Got a T-shirt? You could make a mask at home." So that's still the message.

And I think that was the first time in the English speaking world, that there was a clear call to action, saying we should wear masks. But it absolutely was not the first time in the West. In fact, largely I was plagiarizing from a guy called Petr Ludwig who had already done all this in the Czech Republic. And so even when I did the video for my class, a lot of it was basically me saying, "Okay, this guy called Petr Ludwig, looked at this and then he said this. And then he looked at this and then he said this. And then this is what happened because he started this astonishing campaign in the Czech Republic, which was the first country in the West to get to basically about a hundred percent mask usage, weeks and weeks ahead of anybody else. And it was all thanks to his great science communication.

So every time I wondered, like, "Okay, now that I've decided this is important, what should I do next?" It was always like, "Well, can you try to do whatever Petr did?"

Eric Ries: What was Petr's background? And how did you know him?

Jeremy Howard: I didn't know him. As you know, part of the research was like, I kind of thought, there's not much point advocating for masks because nobody's going to wear them in the West. Everybody knows that only Asian people wear masks so why bother? And I actually came across this story that like, again on Twitter, it's like, "Oh actually that's not true. Everybody in the Czech Republic is wearing masks." And I looked into why, and I found this video in Czech from this guy, Petr with subtitles. And it was very, yeah... It was really nicely done. And then one of the people in the Czech Republic had been kind enough to write a Google Doc, basically describing the story that they watched as people saw this video. And they got really into the idea of wearing masks and within three days, the whole country had masks, within three days with no government support. They put up these things they called mask trees on street corners where people would make masks and go and hang them up on the mask tree. And celebrities would have songs and videos about masks and people in fashion would create fashionable masks. And so I...For my Washington Post story, I really wanted to anchor it with a specific example of this. So I reached out to some folks I found on social media who had to close up their bar in Prague and because of social distancing. And two days later they reopened it as a mask making factory. And when I say factory, what I mean is, they went around to their neighbors and asked if anybody had sewing machines and borrowed as many as they could. And within a week they were making 400 masks a day and had a full time driver, nine full time staff and they did the whole thing actually for free. They gave them all away.

So in my Washington Post piece, I was able to have a paragraph describing this story. I was chatting to the people actually from the bar via Twitter DM and they sent me photos and they told me all their names and they described exactly what happened. And so I kind of found myself immersed in the world of the Czech Republic. And they all wanted me to tell this story because they were also proud of what their country had done. And for them, a lot of it was this kind of weird anti-government thing. A lot of them seemed to hate their government and they were annoyed that the government wasn't doing anything. They felt lied to about masks after Petr's video. And so part of it was like, "Well, damn it, this is something we can do ourselves. We don't have to wait--."

Eric Ries: Almost like an act of resistance.

Jeremy Howard: "Yeah. We don't have to wait for the government. If they're not going to do anything, we're going to do it ourselves." So they were super proud that they had kind of, as a grassroots movement had made this happen. And it actually forced their government to change. A week later, the government turned around and started requiring masks in public.

Eric Ries: How many times in this pandemic have we seen this story of civic society rising up and leading the people who are supposed to be the leaders?

Jeremy Howard:I know it's amazing.

Eric Ries: It's been a remarkable thing to see over and over again.

Jeremy Howard: It really has and the deeper I get into kind of policy and I'm now been talking with leaders in Europe and England and Africa and America. The more I find myself talking to random people who... So I'm spending a lot of time working with a musician in England at the moment, who has turned out to be one of the key influencers in their ability to make masks happen in England. Yeah. I mean, people are just seeing that stuff needs to get done and they're seeing it's not getting done. And so they're finding ways to do it. And then that puts the pressure on when the public is saying, "Well, we're doing it anyway." Then I think governments have to respond.

Eric Ries: How does it feel to be in the middle of all that activity? I think people from the outside who haven't seen what you and I have seen up close would expect this to all be chaos and a kind of anti-intellectual mob. If you just say random people will be in charge of things.

Jeremy Howard: Because these are kind of self selected people, they're just super caring and super passionate and doers rather than talkers. Because we don't have companies organizing what we're meant to do or KPIs to meet or whatever. So, one of the great things that happened early on was somebody reached out to me and said like, "Well, I know this Republican Senator who I think might be interested and do you want me to connect you?" And I was like, "Sure." Five minutes later, he's like, "Okay, Senator, you should talk to this guy, Jeremy, he's got this view about masks." And I sent him a couple of paragraphs, 20 minutes later, the Senator emails me back and he's like, "This is great. I'm going to do a video about this." By the next day, he had a video on Twitter of him wearing a mask.

And so this guy who helped me connect and I talked, and we had never met before. And we thought, "Well, this could be really bad if this becomes a partisan issue." So he helped me reach out to a Democratic Senator and so I spoke to his team. I didn't speak directly to the Senator in this case. And I think one or two days later, we had a Democrat Senator doing the same thing in public, showing a picture wearing a mask, telling other people to wear a mask.

And so within another three or four days, I was briefing ten Senate offices on a big conference call on the whole thing. And briefing senators who we're talking to Donald Trump that day and briefing senators who were talking to the lead policymakers in the CDC. They were super interested, they were super helpful. Watching Democrats and Republicans, their offices and the senators, just emailing each other, chatting, working together, figuring out the messaging. Figuring out, like, "Oh, we don't know... How do we get the CDC to change their tune without making them look bad?" Thinking how do we... What are the scientific issues they need to understand? How do we explain them best? That was, it was great.

Eric Ries: So here's a part of the story I don't even know. Because I remember when the Washington Post article came out, I remember there being a lot of excitement among some folks in the tech community, that there was this new idea that could make a big difference and that maybe the official authorities weren't getting it. And the next thing I know there's a masks for all organization, and you're just having impact after impact after impact, as the dominoes start to fall. What took it from an idea that you were just talking to some people about to an actual thing?

Jeremy Howard: I mean it's still not an actual thing, that's the thing. People keep emailing me and saying, "Can I join your organization?" And always like, "Look, it's just a bunch of random people doing random things." I don't like... I'm kind of the face of it in the English speaking world, but obviously Petr is kind of the very much the face of it in Europe.

Eric Ries: Well, you have a domain name in a website. So that's different than just having---

Jeremy Howard: Yeah, we have a domain and a website.

Eric Ries: Where'd that come from?

Jeremy Howard: I went and I went and registered a domain name and then a super helpful guy called Cam Woodsum and I connected, I think, through-

Eric Ries: Yeah, shout-out to Cam.

Jeremy Howard: I think through Meesha Shalem who you probably know. So Meesha is a great kind of connector in the kind of tech policy community and I've known him for a long time. He's a good friend. And I reached out to him and said, like, "I need help." And I think some of the WhatsApp groups that you and I are on Eric, I kind of begged for help. And so, yeah, got connected with Cam who put the whole website together with really minimal supervision. Because I don't really have any supervision to give, I was just like, "I don't know, look at my Twitter feed. That's what I say. And here's my media appearances. Here's the website that I... The domain name I have, tell me where to point the A records.

So he basically made that happen. And so that... And at the same time, that was happening within a day of my Washington Post article coming out. I was getting calls from Good Morning America and Nightline and Jory Reed's team and CNNs team saying, like, "Come and tell this story on TV. So I reached out to a friend who knows everybody and everything, he used to report to Obama when he was in the White House and said, like, " Do you know anybody who can help?" And he hooked me up with some volunteer PR people. Yeah, when I say volunteer, it sounds like they’re some kind of interns or something like that. It's some of the top PR people in the world...

Eric Ries: Well that... And that's a remarkable thing many of us have experienced. That some of the most talented people in the world are all of a sudden available on a part time volunteer basis that would never have been even conceivable a few months ago.

Jeremy Howard: Exactly. No, no. Exactly, absolutely. So I was like, "Could you help with this?" So they helped kind of deal with the logistics of all the inbound. And then when there was new news, I would pass it along and they'd send it off to the bookers. And then I started getting, particularly through the World Economic Forum community, I asked for help and people were like, "Oh, I know this top team at one of the world's top advertising companies. Do you want me to connect you for that?" And yeah, nobody's really got enough work to do so and they all want to do useful things. So people have been super, super helpful. It's always hard to know exactly which bits are our impact and what would have happened anyway, but it's been cool to see, I don't know, for example, The Mayor of London yesterday or the day before wrote something, open letter in The Times saying, "Hey, we want to require everybody-"

Saying, "Hey, we want to require everybody to wear masks, come on, UK government, change your policy," and he only tagged two people. One was me, and the other was a soccer player, Gary Lineker. And I was kind of like, "Okay." Otherwise, I wouldn't have known that my campaigning and neck and stuff made a difference, but when I saw he had done that, I kind of thought, "Oh, that's nice that he is saying that made a difference."

Eric Ries: What are some other things that have surprised you? I feel like I want to pick up the story kind of going back to the moment you wrote the op-ed. You went on TV for the first time, and then you started to see these policy changes happen. Tell us some of those stories. It must have been surreal to be on those calls-

Jeremy Howard: Yeah.

Eric Ries: And to see things translate into action so quickly.

Jeremy Howard: I mean, luckily, I was kind of in the right place to do this because I used to do a lot of TV in Australia. I was kind of, for the main morning news show, I was kind of the go-to IT guy if they wanted to talk about the latest Internet Explorer security problem, or election hacking or whatever. So I was kind of happy, I'm no expert, but I was happy enough to go on TV. So when Good Morning America were the first call and say, "Can you come on tomorrow morning?" I didn't totally embarrass myself, which is good.

Eric Ries: What had you been doing in Australia, by the way, that you had experience with--?

Jeremy Howard: So that was mainly around my companies Fastmail, the email company, and Optimal Decisions, the insurance pricing company, but particularly Fastmail, which was a pretty popular email service, or still is a pretty popular email service. So because it was Australian born, they were happy to have a local Aussie kind of face of IT. They'd always ask me about random stuff that had nothing to do with email, but that's fine.

Eric Ries: So you were accustomed to doing that kind of a kind of TV?

Jeremy Howard: Yeah. Somewhat, yeah. Yeah, so like the way the media works is kind of things pop in and out of the news cycle very quickly. So there was a basically continuous media bookings in the US for a few days. And one of the best ones was I was on, I think it was CNN, as a group, as a part of it. As well as the host, there was also Ashish Jha, who is a Harvard... I think he kind of runs their health policy group or something. And it was really cool because I always feel very like I don't belong in all of these conversations because I'm not a medical doctor or I'm not an epidemiologist or whatever. And there's always a lot of skepticism about whether somebody without the appropriate credentials can have any idea what they're talking about. So being able to be on a big national show like that with an extremely respected person was super helpful.

And actually, that's one of the main things I've tried to do, is to surround myself with people who are far more respectable than me. So one of the things I started finding was we had made a certain amount of progress with policymakers and I had managed to get in touch with kind of people on the inside who were able to tell me like, "Okay, this person's been briefed and that person's been briefed, and this is the conversation that happened, but this person over here isn't convinced yet," that it started becoming a view that the science wasn't clear enough, was causing policy to not move.

So at that point, I decided to try to fill that gap. And I hate writing scientific papers so, so much, but I thought, "This is what has to happen." And I thought, "It has to be respectable, so it needs people's names who are not me." So I reached out to a bunch of people who had been helpful and ended up writing a paper with 19 coauthors, of which the other 18 are extremely respectable people, unlike me, very well known Stanford and UCLA professors and founders of South African public health initiatives and stuff like that.

And that was great, publishing a... It was just a literature review basically, looking at 84 references to say like, "Well, what's the evidence around mask wearing? Do people actually do it? Does it actually block the droplets? What's the efficacy look like? What's the modeled outcomes look like?" Stuff like that.

So that was super helpful. And then that helped me get in touch with this absolutely brilliant Oxford professor called Trisha Greenhalgh, who I think she's the editor of the Oxford evidence-based team for COVID-19. She certainly produces a lot of their content. And she, luckily enough, had just written a letter in the British Medical Journal, not just a letter, I think a paper, describing that people should be wearing masks, even if there isn't enough evidence, because of the precautionary principle, which all countries in the UN I think have signed up to, which is basically, "Hey, if there's something that could save a lot of lives, you should do it, even if you're not a hundred percent sure it's going to work."

And so her and I ended up writing a joint article. It ended up just being on our blog, on Fast AI, but basically describing our kind of joint view. And she's she's super well-respected. And so once that happened, she suddenly started getting lots and lots of calls from the UK press, who have continued to be the slowest, just about, in the world at getting behind every scientific kind of development with COVID-19.

Eric Ries: Yeah, yeah, we'll save the editorializing for a different day, but boy...

Jeremy Howard: Yeah. So one of the things I definitely noticed is people listen a lot more to credentials than to science or data. So one of the main kind of advocacy approaches I've found effective is to find thoughtful open-minded people, convince them, or find people who are already convinced, and then work with them so that then we've got their credentials behind it, which I think maybe for most people even is more convincing than data or science.

Eric Ries: That paper that you wrote strikes me that that's kind of like a full circle moment, taking you all the way back to your very original insight. And in fact, one of the first things you said to me about your personal response to the quarantine principle... Sorry. And one of the first things you said to me about your personal response to the pandemic was that you made personal decisions on the basis of the precautionary principle, that you were looking at this data with your class and looking at it probabilistically, instead of using this more traditional view of data.

Jeremy Howard: Yeah.

Eric Ries: And despite you not having the credential of being the supposed expert in the virology or the epidemiology of this, had this insight that kind of carried you through the whole time. And the paper was like the official world catching up to the insight that you had had privately almost a month earlier.

Jeremy Howard: Yeah. I was kind of pulling it all together. I mean, data scientists like me are not really experts at anything, but what we can do is pull together data from other experts. It could be like raw epidemiological data or data that's shown in tables of studies. So although I was the lead author on that paper, my main job was to kind of try and find all the literature I could. And frankly, a lot of that was a community effort. Folks on Twitter helped find all this literature. And I kind of tried to find interesting-looking tidbits here and there, which I then passed off to the other 18 authors and they were able to say like, "Oh yes, that's correct. That is a helpful thing to know for this modeling perspective, and here's how it integrates with the other pieces." So yeah, I was kind of like a curator and kind of an organizer, I guess.

Eric Ries: I've noticed this phenomenon in a bunch of the efforts that I've seen, that you're describing, that there's people who have the supposed authority to speak to the public who have the credentials. And oftentimes, when they're speaking, if you go behind the scenes, you see there's actually a civilian or a programmer, or some unqualified person who actually did the analysis and got the data and helped them figure out what to say. And I think people assume that people who would have those credentials would have figured those things out on their own. So what does it mean that we've had to kind of do this credential laundering to get the message out?

Jeremy Howard: Right. It depends a bit on the jurisdiction, but like particularly in Britain, I find that the people who were the talking heads are a very, very, very long way away from the analysis that they're talking about. It's somewhat true in the US, at least in the tech sector. There's more people who are actually hands-on who actually do the work.

But yeah, it does mean that I often find myself debating people on TV who I know perfectly well are just parodying what their teams have told them. And so they're not going to change their mind because they don't actually directly have the knowledge or ability to make their own assessment.

Eric Ries: Yeah, they were just given the talking points.

Jeremy Howard: Yeah. So what I've been doing a lot is to try to find the scientific advisors behind the scenes and talk with them so that I know that then that will get passed up and hopefully get turned into new policy.

Eric Ries: Talk about the public persuasion campaign that you ran to get people to contact their officials and push for masks as public policy.

Jeremy Howard: Look, here's the thing about masks, right? Masks are only somewhat okay at protecting the wearer, but they're really, really, really good at blocking stuff coming out from somebody who's infected. And so what that-

Eric Ries: So if you wear a mask, it's not for yourself, it's for the people around you?

Jeremy Howard: Yeah, exactly. But screw the people around me, I want to be safe, so that means I want everybody around me wearing a mask. I don't want to go to the shops and find people around me not wearing masks because I'm unsafe. So how the hell do I get the people around me to wear masks? To do that, we probably need laws or massive kind of celebrity-driven campaigns. So those are the two things I've been working on.

The other reason we need laws, much more importantly, is that if everybody wears masks, then they're keeping their droplets to themselves, which means the reproduction rate, R, decreases, which means that the virus stops transmitting as quickly, which hopefully means, well, A, less people die, and B, we can start to end the lockdown. So there's like basic public policy reasons that everybody needs to wear a mask, just like everybody needs to be vaccinated. It's the same basic public policy decision here. It's not just for you, but for the community you're a part of.

So yeah, so I kind of thought, "Okay, well let's try and do these two things." So I reached out to some folks around kind of advertising to try to build some celebrity-driven messaging campaigns. And then yeah, I was introduced to this thing called Resistbot for trying to help move the needle on policy. For whatever reason, policy is still heavily driven by people who get letters. Letters to senators actually get read, and then the contents of them get told to the senators.

Eric Ries: It's a remarkable thing.

Jeremy Howard: Yeah, it's super weird. And sometimes they have to be faxed, and sometimes they have to be emailed, and sometimes they have to be delivered physically or whatever. So Resistbot is this cool thing where they figured out those details and all you have to do is send a text message and it'll take you through the process of getting that letter or fax or whatever out for you.

And so, the Resist guy I worked with on this, Jason, he was amazing. He basically said, "Well, what's your campaign?"

Eric Ries: And Jason, just for those that don't know, was a long time civic tech entrepreneur. And we'll have a link to the Masks For All campaign on Resistbot in the show notes.

Jeremy Howard: Yeah. So he basically said, "All right, write a letter and then I'll set up the SMS so it gets sent to all these governors every time somebody sends a text message." So that was kind of terrifying. So I was like, "I don't know anything about writing letters to governors." So I reached out around again for help from some of these amazing people who've been so useful, wrote a letter, got some help drafting it, which basically said, "Hey, I want to be safe in my community, and that means you have to make sure everybody wears a mask, and so please enact an executive order to make that happen."

And then it was great because I was on Joy Reid, MSNBC. And one of the things I found very helpful, which I've learned with media, is to talk to the segment producer beforehand and just be totally direct. So I actually said, "I want to speak to the segment producer beforehand." And I said to her, "Look, I want to use Joy's segment to pitch this campaign. And so I want you to have this chyron ready to put on the screen when I say it." And so I was actually able to say, "Okay, text MASKSFORALL to 50409, and this will send it to your governor, and this is going to help make sure that you're safe." And within minutes, Jason said there were thousands of letters.

Eric Ries: Thousands. Oh, the data was unbelievable.

Jeremy Howard: Yeah. It actually turned out I had a mutual acquaintance with Joy, and Joy told our mutual acquaintance, and she was like, "After this segment aired, I was like flooded with emails saying like, 'This is amazing. This is so great. Why aren't you doing more of this?'"

And so that segment was super helpful to kick this thing off. And since that time, it's taken a while, but you're seeing a lot of governors on board now.

Eric Ries: Yeah. At the time that you started the campaign, how many states had mandatory masks?

Jeremy Howard: Zero. Zero states. Zero cities.

Eric Ries: No one had done it yet?

Jeremy Howard: Yeah. But it wasn't just the Resistbot thing. I also tried really hard to find kind of insiders. So there's a lot of kind of highly connected business people who were friends with governors or whatever. And so one of the things I did was I actually made a video, a private YouTube video, where I said, "Governor, we need you to enact an executive order requiring masks for these reasons." So I kind of like personally-

Eric Ries: Yeah.

Jeremy Howard: I didn't say the name because I wanted multiple people to have it, but I tried to make it sound personal as possible. So this was like my lowest view YouTube video ever. It had like 16 views, but basically it's gone out to like--

Eric Ries: But they're all governors?

Jeremy Howard: Yeah, governors. And so then I've been kind of talking to the people who are talking to the governors to say like, "Has he watched the video yet? What did he say?"

Eric Ries: I hope you make the video public at some point because it was powerful to see, I thought. It was very direct and very... I bet governors don't get a lot of messages like that.

Jeremy Howard: I don't know. I have no idea because I have no idea what I'm doing, but it seemed to work at the time. And certainly, some of the governance that I sent it to have enacted these things, and hopefully that was something that they found useful in that decision.

Eric Ries: Where are we in the adoption of masks for all here in the US? Do you have a sense of kind of where we are and where we still need to go?

Jeremy Howard: It's still early. Like the public perception's still not there yet. It's certainly well ahead of where it is in the UK or Australia, for instance. I mean the southern hemisphere is going to be in so much trouble because people in the southern hemisphere think they've done a good job of controlling this thing, when actually, all they've done is lived in a warmer climate. UK is basically not, like I said, we've got the London mayor very much on board, but not much happening elsewhere yet. Hopefully that'll change.

Eric Ries: Yeah. How many states have done it so far?

Jeremy Howard: Obviously the Bay area communities have done it.

Eric Ries: Yeah, we're very fortunate here.

Jeremy Howard: LA was actually the first. And actually, one of the scientists I was working closely with was very much in the ear of the LA mayor's office to make that happen.

Eric Ries: Yeah. And we in San Francisco don't like LA to be first, so we got to be more on the ball for the next thing.

Jeremy Howard: Well, this was one of the things in my pitch to governors, was like, "Hey, who's going to be the first?"

Eric Ries: Yeah.

Jeremy Howard: Like you'll literally be remembered by history as being the first. But yeah, this is the thing that I guess I hadn't realized either, people don't really realize, is these policies that get enacted is only because a whole lot of lobbying that gets done, basically people like me deciding to care a whole lot and figuring out how to convince the right people.

So yeah, so now we've got New York, New Jersey, Maryland, Pennsylvania, Connecticut, Rhode Island, Hawaii, Puerto Rico, LA, Miami, Washington, San Antonio, Dallas County, San Francisco. Massachusetts is rapidly moving as well.

Eric Ries: Yeah. All those localities have something in common. I'll leave it to the listeners to make an exercise in figuring that out.
Jeremy Howard: Right. What is that?

Eric Ries: Well, I don't want to get into the politics of it, but-

Jeremy Howard: But if you're saying politics, I hope you're not suggesting that they're all Democrat, because they're not. So Hogan is Maryland, and he's actually been a bit of a star here because he did this fantastic interview where he talked about rights. And he was like, "Hey, if you're worried this is intruding on your rights, then let me tell you this. If I go out and your virus comes into me because you're not wearing a mask, you have very much intruded on my rights."

Eric Ries: That's exactly right. That's exactly right. And that is a key argument.

Jeremy Howard: Yeah.

Eric Ries: Yeah, we're lucky to have him making it.

Jeremy Howard: So I don't think this is as partisan... It's not that partisan. I mean, it became a bit difficult when Trump said he wasn't personally going to wear a mask, but at the same time he said he's fine with the idea. And he talked about like wearing a scarf or whatever, and to be fair, although it's not a great role model, he actually doesn't have to because everybody around him gets tested first.

Eric Ries: Right.

Jeremy Howard: So there is actually one person in the country who, you could argue, doesn't have to wear a mask.

Eric Ries: I want to hear some more stories about like the actual conversations you have with policymakers, or you must have had some of those moments that felt surreal, where you're talking to someone and you're like, "Why am I the person talking to this person about this topic?" Do you have any stories like that you can tell?

Jeremy Howard: I've never talked to a US politician before. And so when I was first talking to this Republican senator thinking like, well, A, "Why am I talking to you? I'm just some random data scientist." And it's also like, "Why am I talking to you," because my politics are not at all aligned with his. And it was kind of like, yeah, this feeling of how did this happen? Like a few days ago, I really didn't think about masks at all, and suddenly, here I am briefing a senator who's going to be talking to Donald Trump. It was a really weird feeling.

And one of the interesting things was how, yeah, how very unpartisan this kind of whole thing turned out to be. I've been getting so much help from people who have diametrically opposed political views to mine. And we just don't talk about that. We just help each other out because we're all clearly just trying to save lives and save the economy. And so it's been cool to see how people are happy to put their politics aside to respond to a pandemic.

Eric Ries: How do you know that we're making progress towards masks for all?

Jeremy Howard: It's pretty easy to measure progress in terms of policy change. So the CDC now says you should wear a mask, many parts of the country in the US now require a mask. Countries like Indonesia and Israel and the Czech Republic and Slovenia and Mongolia and Singapore require masks. And the vast majority of countries that require masks have only done so since the campaign. Again, no one was talking about it before.

Like Singapore is an interesting example. That's a country that did everything right, testing and tracing, quarantine, but they didn't do masks. They actually had a campaign recommending people only use masks if they're showing symptoms. And because they did all the other stuff right, they didn't have a terrible outbreak, but their R was getting above two. And recently, things have been getting super concerning. And so then a week ago, they changed and they went from officially recommending, "Don't use a mask," to immediately requiring using a mask at all times, including in the workplace.

So looking around the world, we just see the recommendations and the laws changing. And for a lot of those, I've directly been involved in the conversations with the government leaders or their advisors as those things are happening. In every case, there's always scientists and advisors behind the scenes-

Jeremy Howard: Scientists and advises behind the scenes. It's not politicians just deciding to do these things on their own.

Eric Ries: How do you know that you're having an impact? That these things wouldn't have happened anyway, on their own?

Jeremy Howard: These things might well have happened on their own. One would like to think that eventually people must realize that covering your face with a piece of cloth will stop bits of saliva coming out. It's not rocket science. I guess the question with these things is always, how long will it take? I'm concerned that it might have taken all the way up until the next winter when we'll probably have some kind of second outbreak or maybe it would have, I don't know. There was just no sign of it happening. Perhaps it would have gradually grown out of the Czech Republic, very successful campaign anyway. We certainly saw from the Czech Republic that it moved to Austria and Slovenia within weeks. So maybe it would have gradually come from there. But I don't know. I've personally been on so many calls with policymakers and emails with their advisors working directly to make these things happen. So how long would it have taken otherwise? I don't know. But every day counts with this stuff. So it certainly seems to have been helpful.

Eric Ries: That topic has come up a lot in my conversations with people who are either on the sidelines or getting involved in the fight is for a lot of the folks that I know that said it doesn't seem like this is my job, or why am I doing this? The thing that ultimately has been decisive for a number of them is even if all I do is accelerate the right thing happening by a few days in an exponential situation that could mean thousands of lives. So we're all obligated to do what we can. Even if we don't bend the curve of the long run, even if the impact is ... We don't know what it could be, but on the possibility that we could even accelerate the right things happening by a few days, we're called to act.

Jeremy Howard: Yeah, exactly. It's been cool to see how there's certain people in the world who do things in minutes rather than weeks. I think that's been one of the challenges for huge slow moving organizations like the World Health Organization that just don't tend to work at that kind of speed. Or the UK government.

Eric Ries: Listen, you're talking to someone who's been trying to pitch people an organizational transformation for speed and agility as an organizational capability for years.

Jeremy Howard: Yeah.

Eric Ries: To see people talk about it as unnecessary during the good times, and now lament its absence in the bad times, without making the connection that these are choices, there not an inevitability.

Jeremy Howard: Yeah. I mean a pandemic, you look at the difference between places that locked down two weeks earlier versus two weeks later and it's terrifying to look at or look at the difference between the countries that have mask mandates and those that don't.

Eric Ries: We knew that from the 1918 pandemic that even in the US the cities that locked down sooner had dramatically different economic and health outcomes. So we didn't study that history and act on it, it's heartbreaking.

Jeremy Howard: Yeah. It's also this misunderstanding of asymmetric upsides versus downsides.

Eric Ries: Explain what you mean by that.

Jeremy Howard: So people who demand gold standard evidence to do anything, like masks for example. When the thing has almost no cost, the potential upside is huge. So in that situation you should demand gold standard evidence not to do the thing. But this is just an approach, which is very alien to certain professions.

Eric Ries: Have you seen the historical threads about the anti mask leagues and the protest in 1918?

Jeremy Howard: Yeah, that was one of the things I studied in my research.

Eric Ries: Yeah. Talk a little about that history, especially as it relates to masks, and then what you think we should learn from that history as we go forward into this crisis.

Jeremy Howard: Sure. So the 1918 pandemic had a lot of overlaps with this one. It was a respiratory infection like COVID-19, but in that case it was the flu. I think actually we are going to be wanting to study it a whole lot, because it teaches us a lot about how people respond. So in places like San Francisco, people thought as summer came, "Oh, wow we're past it. So we did all this hard work of distancing and doing things outside and wearing masks and that's over, it's behind us. Thank God." So when things started flaring up again the next winter, there was a lot of anger when it was suggested that those hard things should be done again. So for example, the anti mask league appeared as that second winter started coming along with people demanding, "We shouldn't be required to wear masks again." Because they were the law.

They weren't good masks by the way, they were gauze masks, which obviously are not great for source control and things can go through them. I don't know why they used gauze, but there you go. Nowadays we use cotton or paper towels, which works much better. So 2,000 people got together in what we would today call a super spreader event without masks. We don't know exactly what the transmission rate was from that, but it;s pretty likely it was big. There was also war bonds, big ceremony parade in St. Louis I think it was. 250,000 people turned out and it basically turned into this huge statewide massacre. We're seeing similar things here. The fact that Mardi Gras went ahead in Louisiana, despite really as the pandemic was kicking off and now it's been so badly hit.

Yeah. I mean responding quickly and appropriately based on the huge risk versus the limited upside, it's hard to do and people are still not always doing it. But I think places like the Bay Area and LA examples of doing this pretty well. I would have liked to have seen things happen even a little bit earlier. But on the whole they've done a pretty good job and the hospital systems not being overwhelmed. In fact, UCSF healthcare workers are flying off to New York to help that's how well that's been handled.

Eric Ries: I mean, we're very, very, very fortunate and very grateful to the leaders who stepped up and did do the right thing. As you say, maybe not as early as we would have liked.

Jeremy Howard: We're still waiting for Gavin Newsome, I don't know, as a governor, he actually ordered hundreds of millions of masks. I get the impression he's waiting for them to arrive, which is crazy for me because we already have the ability to put a handkerchief over your face. So I don't know why, as a governor, he's taking so long on this important public health measure.

Eric Ries: Yeah. There's some backstory to that, to the masks in China. The whole thing has been just another completely surreal topic maybe for another time.

Jeremy Howard: Well it is one of the interesting stories here is that one of the things we keep facing is jurisdictions that don't act because they don't have the perfect masks yet and they don't have the perfect messaging campaign yet about donning and doffing them and the perfect understanding yet of how to clean them. I find it really weird this idea that we should do nothing at all until we can do everything perfectly. But it's a super common response, which I find really infuriating.

Eric Ries: It's an artifact of 20th century management thinking, not to digress or too much, but the logic of masks production and the economics of that organizational design really reward planning in advance, high efficiency, single action, measure twice, cut once and cut against speed. If you're dealing with a problem where that's a good toolkit, maybe that's okay, but it's especially bad in conditions of extreme uncertainty.

Jeremy Howard: Yeah. Careful you might get me started, I was a management consultant doing that stuff in the 90s. So yeah, I know exactly what you mean. I was not fond of it then and I'm still not fond of it now.

Eric Ries: Yeah. Boy, is change needed here and needed more than ever. So maybe let me switch gears for a moment and ask you if you have advice. There's probably somebody listening to this podcast who's still on the sidelines of this fight or has had an insight, has seen some data like you've seen and has been radicalized by it, or it's just feeling frustrated and they're not sure if they can make a difference. They're not sure what to do. What advice would you give them?

Jeremy Howard: Yeah, the same advice that I give entrepreneurs. As an AI academic who spent years as an entrepreneur, I have a lot of people asking for advice about AI product insight they have. What should they do? In both cases, I think the answer is give it your absolute best shot. Don't piss around, right? So if you think you have an insight, an idea, a product, an app or whatever, that can genuinely help people then work the hell out of it. Write it up as carefully as you can, find everybody you can who could intelligently critique it, get people to pick holes in it, make it as compelling as possible, do things in public.

If you are hearing back from people that ...if people just don't seem interested, don't blame the people. If you're not getting people interested, it's because you're not interesting enough. So, why not? Are you wrong? Are you pitching it the wrong way? I often find, one of the things I talked to my students the most about is tenacity. The biggest difference I find between people who were successful entrepreneurs and successful researchers versus not is whether they stick with things until they finish it. So it's the same here. If you've got something, then stick with it. I can't tell you how many doctors I dealt with, particularly a few weeks ago with this campaign telling me normal people could never learn to use masks properly. "They'll just touch them too much. Oh, it's just going to make them less careful about social distancing, blah, blah, blah." Just have to keep sticking with it anyway.

Eric Ries: So what would it look like to apply what you've called the drive train approach to pandemic interventions?

Jeremy Howard: So the drive train approach was something I came up with when I built this company called Optimal Decisions Group. Which is a particularly boring and not societally great company, which was all about helping insurers to make more money by setting their prices better. So insurers basically set their prices by looking at how, say car insurance, how risky are you? How about how likely are you to crash? How much is that going to cost if you do? Then try to figure out like, "All right, let's make sure we charge more than that so we make a profit." That actually doesn't tell the whole picture because maybe I'm in a hurry. So you could charge me much more. So if you only charge me like 10% more than my risk, then you're leaving a lot of money on the table.

So I developed this thing called the drive train approach, where you build a bunch of models, including in this case, an elasticity model, how likely would I be to accept a particular price and a predictive model of risk and combine them all together into a simulation that says like, "Oh, if you've charged this amount of money, here's what your eventual market share would be. Here's what your risk would be and so forth." Then you can optimize that. So the drive train approach for pandemic response would be, "Okay, what are all of the things we could estimate about how likely are people to socially distance yet? We don't know, let's make our best guess if you are wearing a mask, if you're not, what data do we have, let's try to figure out as best as we can, what it might be."

"What's the probability that you're going to remove it when you're speaking?" So he built all these different models. I think it just chuck it into a spreadsheet or whatever, or do a simple probabilistic programming model that then says, "Okay, so if we have a mask, here's the estimated number of lives lost per month. The estimated value of the economy per month. How long until we might be able to win the lockdown?" Versus if we don't require masks that's the thing I'd love to see people doing, but literally most official evidence-based policies for health care don't actually include the cost of the intervention at all. Which is why we can have something that doesn't have a randomized controlled trial, like a lockdown. And the cost is huge, does happen. That something like masks, which may be just as effective, this be something you can do in conjunction with social distancing rather than instead of. Has almost no cost, but doesn't happen. So from a basic common sense perspective, or from this drive train approach perspective, it makes no sense at all.

Eric Ries: What do you hope people will take away from this experience? What do you hope? If you could pick one thing that we as a society will take into the new normal, after the pandemic passes, what would it be?

Jeremy Howard: Main thing is I hope is that we keep our lives. So I want to see less lives lost. I hope we can keep our economy. Then assuming we find a way to make these things happen, it would be great to see ... Some of the different ways we're seeing to do things where maybe we don't have to travel as much. Maybe we don't need as many certifications, maybe it's okay for doctors to practice across state lines. In a pandemic I guess we start to realize which laws are actually getting in the way more than they're helping. So maybe some of those things can be maintained beyond this as well.

Eric Ries: Where do you think we go from here? How do we get out of the crisis?

Jeremy Howard: Well, there's a lot of uncertainty, so we have to accept that. So any planning has to be aware of the possibility that herd immunity might not be possible, or it might be, or maybe it is possible. A vaccine might not be possible. Maybe it is possible. Maybe those things last for a short amount of time, maybe they last for a long amount of time. These are all things we don't know. It could be that, seems likely to me, that during warmer months things will be a lot easier. Then during winter months, it will be much harder again. It seems very likely that people during summer times will become apathetic. They'll be hubris. That will make the winter times, the first winter way, way worse. So if we're going to get out of this, we need not just one off reactionary policies, but a complete understanding of what the whole dynamic of this looks like.

I mean, we certainly need to be wearing masks in public, especially whenever we're indoors or in very close proximity to others. To not do that is to increase the reproduction rate. So that's just going to be a massacre. At the same time, we need to realize that not everybody can stay locked up forever. We have jobs for a reason. People do need to work. To not do so has a huge cost on lives. So we need to be talking about like, "Well, how do we gradually get back to life, but also know that there will be more breakouts, so how do we be sure that we know when a breakout's happening?" Then, "Do we have the systems in place to track it?" So we need good testing. We need good contact tracing. We just need to also set this expectation that for some period of time, which might be forever, might be years. It might not.

We're all going to have outbreaks from time to time, which means we all need to be testing to know when it's happening, contract tracing to know what's happening. From time to time, regions are going to have to be getting very good at various types of lockdowns to ensure that those breakouts don't turn into new pandemics.

Eric Ries: Jeremy, I wanted to thank you for all the work that you're doing here to keep all of us safe. Thank you for coming against it's the stereotype of an academic and getting into action and seemingly being everywhere at once in defense of these values and the need to save as many lives as possible. Of course, thank you for coming on and sharing in this conversation.

Jeremy Howard: Thank you.

Eric Ries: This has been Out Of The Crisis. Out Of The Crisis is produced by Ben Erlich, edited by Jacob Tender, music composed and performed by Cody Martin hosting is by Breaker. For more information on COVID-19 and ways you can help visit helpwithcovid.com. If you have feedback or you're working on a project related to the pandemic, please reach out to me on Twitter. I'm @E-R-I-C-R-I-E-S. Let's solve this together.

Lessons Learned

Monday, June 29, 2020

Out of the Crisis #11: Jeremy Howard on the power of masks, health policy, and data science in medicine

Highlights from the show:

Show-related resources:

Transcript for Out of the Crisis #11: Jeremy Howard

Highlights from the show:

Show-related resources:

Transcript for Out of the Crisis #11: Jeremy Howard

Lean Startup Book

Get New Essays By Email

Get New Essays by Email

About The Author

About the Author

Blog Archive

Popular Posts

Recent & Upcoming Events

Recommended Reading

Facebook Fans

@ericries on Twitter

@ericries on Twitter

outbrain