Dr. Harrell is the founding chair of the Vanderbilt Biostatistics department. Since 2003 he has been Professor of Biostatistics, Vanderbilt University School of Medicine, and was the department chairman from 2003-2017. He is Expert Statistical Advisor for the Office of Biostatistics for FDA CDER. He is Associate Editor of Statistics in Medicine, a member of the Scientific Advisory Board for Science Translational Medicine, and a member of the Faculty of 1000 Medicine. He is a Fellow of the American Statistical Association and winner of the Association's WJ Dixon Award for Excellence in Statistical Consulting for 2014. His specialties are development of accurate prognostic and diagnostic models, model validation, clinical trials, observational clinical research, cardiovascular research, technology evaluation, pharmaceutical safety, Bayesian methods, quantifying predictive accuracy, missing data imputation, and statistical graphics and reporting.
+ Full Transcript
John Bailer: You may have a philosophy of life, but few of us are likely to have a philosophy of biostatistics. I’m John Bailer from the Department of Statistics at Miami University, and I’m joined by my colleague Richard Campbell from the Department of Media, Journalism and Film. Today we’re going to be chatting about biostatistics philosophy on Stats and Stories with Frank Harrell, Professor of Biostatistics and founding Chair of the Department of Biostatistics at Vanderbilt University. Okay Frank, what led you to formulate a philosophy of biostatistics? And tell us a little bit about what’s contained in it.
Frank Harrell: Glad to, and I’m glad to be here John and Richard. One of the things that made me want to write down a philosophy is that I noticed that most people didn’t do it, and they would get in various dilemmas. And so they might be arguing with a collaborator about some best statistical practice, and they’d never gone on record as saying what a good statistical practice is in a certain part of statistics, and I thought if you went on record you have something you can lean on, maybe in the heat of battle, maybe a contiguous collaboration; and then the other thing is probably more important, but you write down your philosophy of life or biostatistics, then you start critiquing it and if you don’t write it down you don’t really think about it and trying to find fault in it, and so over time you refine your philosophy; if it’s plain enough to read it gives you the ability to check it and get other people to critique it, and find out your blind spots and make it better.
Bailer: So, what’s one or two items in your philosophy?
Harrell: Well, the one that is probably no mystery to anyone is that you want to use statistical methods that actually work, and so they need to be demonstrated to work you can do that either with lot of simulation or statistical theory. Another philosophy relates to my general feeling that skepticism is one of our most important tools, is when you’re working with someone, they may come to you with a certain way of thinking and they may have measured something a certain way, and you often find, when you think about it, that they’ve taken something for granted, because they were not skeptical enough about their own field and they’re using a measurement- it might be a patient outcome in a clinical study, that actually hasn’t been validated, or you can actually shoot holes in it. It may tell you that two patients that have the same response, the response has a different interpretation for those two patients because it doesn’t take into account, in the proper way, where the patient started in their course of the disease. So, questioning measurements and questioning the assumptions that the subject matter expert has made is a very important part of my philosophy. And another one related to that is related to that is that I learned that you do not want to put yourself in the position of giving somebody what that they want; it’s a bad goal. Unless maybe it’s your boss or something, but you should be in a position to give somebody what they really need, not what they say they want; that’s where this questioning and skepticism needs to come in.
Richard Campbell: Does that work for journalists? One of the questions I often ask our statisticians is how well do journalists cover what you do? How well do they understand biostatistics? And what can they do better to tell that story? Because a lot of what you did here for us is translating some of the more complex analysis that goes on. So, talk a little bit about- again, I think that often the best journalists are often the most skeptical ones, but sometimes it’s hard to be skeptical about something you know nothing about so you’re often starting from scratch. So, any suggestions for journalists?
Harrell: Well, I think knowing more about biostatistics is really going to pay off, and one recommendation is, David Spiegelhalter has a book out, it came out this year, it’s called The Art of Statistics. And it’s really covers a lot of stories related to journalism and reporting and claims made in the media, so his Art of Statistics should be read by journalists very widely. I think the other thing that happens with journalists is they may be reporting on a finding by a subject matter expert; it might be some quantification of how much better patients get with a certain treatment, and it might be a statistical statement of the results of a study, and the subject matter expert may not have perfected the exactly correct way to state the statistical interpretation, and a journalist may not know enough about statistical interpretation to really question the subject matter expert. So you get a lot of statements that are overstated and not really put in the right context, and journalists as a whole are under pressure because they want to report on things that people will actually read, so if you start an article by saying here’s 18 caveats you need to know before you read about what’s about to be written, you probably are not going to get many readers.
Bailer: So, you mentioned that by writing down your philosophy you can start critiquing it and possibly changing it, could you mention one item that you’ve changed since you first did this?
Harrell: Well, I think probably the one that’s related to how our results are stated so that they’re useful for decision makers- I was trained like you were at University of North Carolina, John, in more classical, statistical foundations and we had a certain way of stating results and classical degrees of surprise, from the data which is a p-value, but what decision makers need is they need to know is the likelihood that something works, which is not the likelihood of data, but it’s the chance that a treatment works in a clinical trial, for example. And so I didn’t use to think about what are the inputs that the decision makers need and now I’ve got that in my philosophy, that they need to have more actionable inputs that relate to the probability that something has an effect, rather than the probability of getting extreme data if it does not have any effect.
Bailer: Well I’m afraid that’s all the time we have for this episode of Stats and Stories. Frank, thank you so much for being here.
Harrell: Very glad to be here.
Bailer: Stats and Stories is a partnership between Miami University’s Departments of Statistics and Media, Journalism and Film, and the American Statistical Association. You can follow us on Twitter, Apple podcasts, or other places where you can find podcasts. If you’d like to share your thoughts on the program send your email to statsandstories@miamioh.edu, or check us out on statsandstories.net. Be sure to listen for future editions of Stats and Stories where we discuss the statistics behind the stories, and the stories behind the statistics.