The FDA and the Pharmaceutical Industry

0 0

(upbeat music) – We’re delighted to have Dan Carpenter from the Harvard Department of Government. Dan is the Allie FreedProfessor of Government at Harvard University, and he’s developed someof what I personally think is the most interesting workthat we’ve had on bureaucracies and in the disciplineof political science, and this is related to that kind of work. Dan has done a book, whichI probably should have had here to hold up, but Ibought it on a Kindle, so that kind of shamelesspromotion is not possible anymore.- I’ll do that. – All right, excellent, excellent. A book on the FDA a couple of years back, and he’s here to share someinsights from that work and some projects going forward. So Dan, welcome to USC. – Thank you. So thanks for having me. I was invited to talk today about the FDA and the pharmaceutical industryas kind of a general theme. So this talk will have basically two components and two purposes. One is to kind of give you ageneral overview of the stuff that I’ve done in this area and to sort of pass alongsome general lessons, including those in the aforementioned book that Tony just mentioned.And the second is topresent some new results that we’re working on withmy research team at Harvard. And that’s the part, so the first part is kind of in red here, and the second part is after including. The first part is largely published, and so you should cite that. It’s really the second partwhere I’m asking you to be perhaps a little carefulwith the citation patterns of what I’m conveying today.So why is pharmaceuticalregulation of interest to students of public policy, tostudents of political science, economics, things like that? Well, by comparison with awide range of other industries, there’s actually much heaviergovernmental involvement in this industry, both in theUnited States and worldwide. So states and by whichI mean nation states, although sometimes insome cases as you’ve seen in California and Texaswith some of the bond issues and referendum passed measures on funding basic thingslike stem cell research, states fund some of the basic research that goes into this industry.Most of the applied researchand most of the money that is spent onpharmaceutical and biotech R&D is in fact private money,but still it’s fair to say that there is a kind of a complementarity and a kind of a mutualdependence between that work, which is often built offof some of the basic things that are funded in part bypharmaceutical companies and private foundations, butalso in part by government agencies like the NationalScience Foundation or the National Institutes of Health.The state regulates muchof applied research, what I’m going to refer to as the conceptual power ofthe regulatory state. Basically the ability of the government to define the vocabularies, methods and concepts that people use in research. And so much of the history ofthe pharmaceutical industry in the 20th century isactually the history not so much of science developing exogenously from government regulation, but science often developing endogenously within government regulation. I’m gonna show you oneexample of that, okay. And so just, for instance, if you want in the UnitedStates to test a drug and to basically transport that drug across state boundaries, you have to get an exemption from the FDA, which is called anInvestigational New Drug exemption or IND, all right.And that is basically aprerequisite in order to engage in medical research with clinical subjects, which is to say humansubjects in the United States. By virtue of the diffusionof those rules worldwide, that is now basically a globalorder of regulation, right. The state is also a veto player over R&D, so if you’re a car company andyou design a new automobile, for the most part thegovernment is not sitting there at the end of the linewith the ability to veto whether that project entersthe marketplace or not.Right, I mean, you cantalk about different ways in which a wide variety ofmarkets the state enters: licensing, land use permitting,environmental permitting, things like that, but againhere it’s quite strong and it’s product specific,it’s not licensing firms. So Pfizer in some way or Merckare not licensed by the FDA, although they’re kind of certified with the way they produce drugs, but each and every drugthat they would wish to introduce to the market and on which they wouldseek to generate profits has to be approved by the FDA, all right.And finally less so in the United States, but increasingly worldwide, once those products are on the market, the state also regulatestheir post-market life, the prices they can charge, all right, which we see sort of inEurope, say for instance, in the United Kingdom throughthe National Health Service or for that matter in Canada, but also the way they are distributed. So there’s been a coupleof recent developments at the FDA with the distributionof opioid related drugs and the FDA basically trying to induce pharmaceutical companies toless tamper resistant drugs. So that hydrocodone andoxycodone based medications can’t be kind of mixed into a soup that is more addictive, right. Now the general theorythat I’ve been working on, which kind of functionsas a background to this is what I call the theoryof approval regulation.And the idea is, is that the state is, again, in this capacity ofbeing a veto player over R&D, and that there is kind of asimultaneity between firms, which seek to kind of developprofitable investments, but those investments are kind of, the value of those investments is known only with a great degree of uncertainty, and that the firm is alsoregulating those investments, but again knows perhaps even less than the firm does about them. And so it’s a world wherebasically companies are trying to bring these products to market, they are not sure thattheir product is profitable, all right, they need toin part test the product through the R&D processto be sure of its quality or at least to gain moreinformation about it, but again, there’s thisveto player, right.Now, so the regulator is adecision maker under uncertainty, which I described as kindof stochastic, all right. Part of what we do is thatwe then model an estimate, statistically a set ofpolitical constraints on the regulator, and whenI refer to the regulator, just think FDA in itsapproval behavior, all right. So some of the work that we’ve done, and this is work thatother people have done too is to describe how even in the absence of political capture or political protection, large firms are often gonnado better under this process. Why? In part because they’re morefamiliar to the FDA, all right. People who enter a certain market niches earlier often do better, not again because there’ssome capture dynamic, but because the regulatorcan approve drugs for say a new cancer therapy as a way of kind of throwing a bone to patient advocate groupsand things like that.So this is summarized insome work that I’ve done with the title protection without capture where one gets protectionfor larger producers, older firms and older or firstentrance to a marketplace without there being any degree of kind of political purchasing or bribingof the regulatory process. And then finally, in some other modeling that I’ve done with Mike Ting, who I referenced on the first slide, we’ve looked at theendogeneity of R&D decisions and regulatory approval.And there’s been somesubsequent development where we’ve looked at what happens in this worldto consumer confidence. So basically peoplecoming into a marketplace, in which there’s a certaindegree of screening. So in theory bad productsmight be screened out, the products that you do havethat enter the marketplace, there’s a lot of data produced about them, such as randomized clinical trials, summaries of those data,make it into the label. And so the question is,what happens to consumption? And then there is a more general model that seems to be applied to more to antitrust by two economists, Ottaviani and Wickelgren, all right, but one problem. Do I have this here? I don’t have that. One problem here is that we actually lack, we have a lot of data intheory about this problem, which is basically howthe regulators decide. We have a lot of data andtheory about how firms develop their products in R&D. We have very little data, although now again justsome emerging theory on basically how R&D andregulatory approval strategies respond to one another. Question, yeah. – So like my impression wasthat the FDA approval was they have more objectiveprocess than your clinical trial shows your drug wasstatistically significant, and that there is not muchsubjectiveness or discretion, but these stories suggests that, are you suggesting thattrials for social diseases that more advocacy or something,even if they don’t show they have a differentspecial for approval? – Yeah, so if it werereally that objective, I think you’d find lessdisagreement within the FDA and less disagreement withinthe advisory committees that offer their counselto the FDA than you see.So I guess I would kind ofdisagree, it’s not that I, basically, yes, theprocess is very scientific. Yes, there’s a lot ofdata that informs it, but science, number one, doesn’teliminate the uncertainty. And sometimes the sciencegenerates more controversy than in fact reduces. So I think the process isshot through with science and in fact, rigor.I mean, what we know about theseproducts coming into market probably is greater than just about that for any other sort of formof industrial organization. That said, sometimes that information can generate controversy and subjectivity. For instance, we’ll knowa lot about these products because they’ve been testedin randomized clinical trials with thousands of patients, right, but from those trials wemight get a safety signal that suggests that, well, wait a minute, sometimes after 18 months, there’s some hepatotoxicity, right, that’s developing in the liver, right. How do you interpret that? Do you interpret that as something, which is so important that weshould thereby reject the drug or something that we should thereby or thereafter attach awarning to the label, right. That’s a controversy whichactually is generated by the scientific process, and which is actually notso much reduced by it.Does that kind of…? Yeah, okay. So the book that I’ve done, and so if Tony had had the hard copy here, he would have been able to give you this. It was published in 2010,which tries to unify both historically, theoreticallyin a conceptual manner and empirically a largenumber of these observations, all right. And so let me just give you two before I get to the sort of newer work.Let me just give you two kindof lessons from this book that I can sort of talk about. The first is, is thatit’s commonly thought that basically the waythat the FDA evolved was in sort of three kindof crucial enactments. Number one, the 1906Pure Food and Drugs Act, which gave to the FDA,actually was then a bureau in the US Department of Agriculture, power in interstate commerceto govern food and drugs. In 1938, it got thispre-market approval power, but only for the question of safety, not whether drugs actuallyworked, all right. And then along comes thethalidomide tragedy in 1962, which essentially didn’toccur in the United States because this woman, FrancesKelsey, held up this drug which was Contergan, thalidomide, which made its way into Germany.There were thousands of birthdefects, things like that, but the usual story is,is that only in 1962 after that tragedy in Europe that the FDA begin to regulate efficacy. And in fact, people haveused that sort of before and after comparison ina wide variety of studies and economics and political science to try to essentiallyestimate what the effect of efficacy regulation isversus safety regulation. Well, basically onehistorical lesson of this book with a lot of time spentin the FDA archives as well as pharmaceutical company archives is in fact that the FDAwas regulating efficacy more continuously in kind of an upslope from the late 1940s allthe way up until 1962. So there’s no sort of tightboundary pre and post, right? So here’s just an example. Erwin Nelson, who is thehead of the drugs division in the FDA in 1949 gives a speech to pharmaceutical company representatives, in which he says, look,we want proof of safety. That’s what the law says, but we also want proof of efficacy. This is one of those cases where simply by communicatingthings in a speech, a federal agency manageror bureaucrat or regulator can often go beyond what the law says, not in a way that’s illegal, right, but in a way that’s kindof non-statutory, right.And so nowhere did the FDA’s rules say that you have toprove efficacy to get it, your drug approved, but increasingly inspeeches they’re trying to giving this message. And if you look at sort of the trade journalsduring this period, industry trade journalswhere they’re talking about, what’s the best new investmentsin the world of chemicals, pharmaceuticals drugs, foods? They’re basically complainingthat the FDA is making a lot of these kinds of statements. Like, all right, now we don’tknow what the criteria are because we thought it wassafety 10 years ago in 1938, but increasingly it seems to be efficacy. And if you want lots andlots and lots of those quotes with lots and lots of lots of sites, consult chapter three of my book, which is about 100 pages long, all right.Too long, but it’s got allthat data available for you. As evidence of also of this,basically the FDA began to use, refuse to file or RTF judgments, which is to say we’renot even going to review your drug application unless it meets certain minimum criteria, and I’ve sort of listed those here. And this was a draft federalregister document in ’54, the new drug applicationform was finalized in 1956.That’s five to six yearsbefore thalidomide hits, and again, the drug efficacyamendments were passed. And it says, an application, I’m just gonna read this for you, may be incomplete or may be refused unless it includes fullreports of adequate tests by all methods reasonably applicable to show whether or notthe drug is safe for use. That was a way that theyenabled efficacy regulation by saying not just safetyin terms of toxicity. Like do you explodewhen you take the pill, but safe as used, right? And that was a way of getting into how was the drug gonna beused and for what purposes, and with what effects? The reports ordinarilyshould include detailed data derived from appropriate animal or other biological experiments and reports of all clinicaltesting by experts. Those experts must be qualified by scientific training and experience. That was code for you better have a PhD in clinical pharmacology on your team, otherwise we’re not even gonnalook at your application, all right. And it should include detailed information pertaining to each individual treated, including all these variables, results of clinical andlaboratory examinations made.So if you took a bloodsample from this person at the beginning and/or in themiddle of a clinical trial, the results of that hadto be available on paper, not just in sort of a summary statistic, and a full statementof any adverse effects in therapeutic resultsobserved, all right. So you have to tell usthe therapeutic results in order for us to even look at your drug application, right. This again was six yearsbefore thalidomide occurred. One of the things that the FDAwas doing about three decades before it occurred inEurope was literally getting the raw data from all thesenew drug applications.What happened in Europewas companies would send statistical summaries fromtheir clinical trials, often highly observational,not randomized. In the US and this predates thalidomide, you’d not only have the raw data in the sense of the numerical dataset, you would have all the paper data from which the numbers were coded, and they would literallygo back and recode and examine thesensitivity of assumptions. They were literally decades ahead at least in terms ofstatistical methodology, replicability of whereEurope was at that time. If you actually look at theapproval time distribution, how long did it take from the time that a drug was sent in forthose drugs which is approved, we’re only looking for drugswhich were approved here.How long did it take them to get approved? Okay. You see that in the early 1950s, and these are quantiles of theapproval time distribution. So this is the time bywhich down here 25%, the first 25% of the drugs are approved, the first 50% of the drugs are approved, the first 75% of the drugs are approved, and here 90% of the drugsare approved, right. So this tells you somethingabout, if you will, the tail or the outer tail ofthat distribution, right.If you look in the early1950s, it’s very quick. And in fact the statutory standard is, they’re supposed to beapproved within six months, all right, or reviewed within six months. So if approved, thenapproved within six months, but you can see here a sharp uptick, not only in the median,but also the tails, right, whereby by 1960 before anybodyknows what thalidomide is, all right, before there’s any idea about officially adding efficacy, the FDA is already at themedian, all right, going through, it’s the congressional standardswhich were not binding, all right, but at leastwere recommendations.Now is this proof of efficacy regulation? No, right, but is itconsistent with the story that the FDA was getting more stringent during this time period. So worth keeping in mind thatif you just estimate this in a sort of regression model,turning out like basically how long does it take theFDA to approve these drugs, and you control for the amount of staff that the FDA had at this time, right, the effects get stronger, not weaker. Why? Because actually the FDA staff was tripling during this period, right. – [Man] (indistinct) the volumeof application (indistinct) – Yeah, yeah, so if youcontrol for all those things, this is, it’s pretty clearthat this is something other than backlogand/or resources, right.And, again, I’m just showingyou some statistics here, this is not just from anestimation here, right. And again, this is not efficacy per se, it could be a whole bunch of things, but basically it’sconsistent with the story, the procedural story that youcan tell elsewhere, all right. Second lesson, so that is, if you will, kind of gatekeeping power, all right.When we talk about the vetopower, the gatekeeping power that the FDA has over the marketplace, this is one form of it, and they get to definewhat standards are used separating the wheatfrom the chaff, right. And in so doing they’reactually able to define other kinds of standards. So if you read the financial pages, say of The Wall StreetJournal or The New York Times, and you refer to a biotech stock, right, something you’re kind of interested in, you’ll often hear this,okay, Verolta Pharmaceuticals had a candidate promising fornon-small cell lung cancer that failed in Phase 2 trials. You might ask, so where does this Phase 1, Phase 2, Phase 3 stuff come from, right? Well, again, this is ageneral lesson of the book, consult chapters four andfive, if you want more, but basically this is acreation of the regulatory state imposed upon medical researchand scientific research, not the other way around, right, and there’s a long history that goes into literally when these phasesbegan to get drawn up, all right.If you look at, for instance, and the key rules werewritten in 1963, right. There were a few phase trialsbefore that were actually sanctioned by theNational Cancer Institute. In fact, most of them run bythe National Cancer Institute. So the story of the developmentof phased experimentation, the idea that one not onlyruns a test for a drug, but you run one set of tests, successful passage through whichbecomes a sufficient hurdle to go to the next set of tests, sufficient passage through which becomes the sufficient hurdle for thethird set of tests, right.This idea of sequentialexperiments, right. That is a regulatory imposition, not only on the pharmaceutical industry, but in fact, on theentire medical industrial university complex in the United States, and in fact worldwide. Every human clinical trialnow that involves a drug, all right, of any sort isessentially going to be classified into one, two or three. Now there’s four, andthere’s technically a zero, but those are just further glosses on this basic structure, all right. – [Man] And the originof that was the FDA? – FDA, yeah, and you can find original documents in sites there. Again, some ideas aboutthis were thrown around by the National Cancer Instituteas well in the late 1950s, but the original idea for this idea of sequential experimentationactually comes out of a pharmacologist, animalpharmacologist in the 1940s looking at how to test for the safety and nutritional value ofdifferent feeds for livestock.And part of what they’re interested in is what’s the acute effect andwhat’s the chronic effect? And if you think aboutPhase 1 and Phase 2, it’s kind of a development from that. You’re looking in Phase1 at kind of, all right, do people explode whenthey take this pill? Do they basically fall over? Phase 2 and Phase 3 are whatare those longer term effects? You’re moving from acuteto chronic, all right. Well, again, this is not only or not purely endogenous to science. In fact, if anything,it’s imposed upon science. And if you follow the pharmaceuticalindustry, you’ll know, for instance, that if acompany is not publicly traded and it’s getting its moneyfrom venture capital, the people in that company are often paid by benchmarks, right.Have you met a certain benchmark? Then the money comes in. Well, the benchmark ina lot of these cases, which is by the way the money that people make in the biotech industry is often the successfulcompletion of a phase. So literally the way thatpharmaceutical payment contracts are structured in the biotech sphere, for those companies thatare not publicly traded is in fact shaped by theseregulatory categories. So it’s not simplyconceptual power in science, it’s conceptual powerin science that shapes the structure of industryand payment contracts. So too if you want to look atwhere the big movements occur in asset prices forpharmaceutical companies, it’s often on the announcementof Phase 1, Phase 2 or Phase 3 results,often are also approval, advisory committees and things like that.So the major pivots for stock prices, for those companiesthat are publicly traded also observe at some levelthis conceptual structure. It’s been a very powerful,it’s a simple idea, right. Let’s just set up a setof experiments in sequence in seriatim, but it’saffected not only science, everything that goes on, not everything, but most of the things that go on at the Health Sciences Institute here, but it also affects thestructure of business, right. Again, read the book, ifyou like more on that. So now I wanna shift gears a little bit to talk about a claim that’s commonly made about pharmaceuticalregulation and innovation, and sort of here is the morespeculative part of the talk and also the part thatmight be more relevant for pharmaceutical, publicpolicy, excuse me, communities.So there have been numerous claims made about the effects of this kindof regulation on innovation. What do we mean by innovation? The number of new drugs, particularly new molecular entities, molecules never beforemarketed, never before used in widespread treatmentin any other capacity. And the claim has often been made that this regulation hasreduced those, that innovation. Not necessarily by the way in a way that’s net cost beneficialnegative because you could say, well, look, we’re getting ridof all these safety problems, we’re getting rid of the crap, could be that we’re better off. But the argument has been nonetheless an observational argument,an empirical argument that in fact, after theimposition of this regulation things went down, I’llget to that in a minute. So claims have been madecomparing things before and after major laws, including some of the work that I’ve done.Claims have been made internationally, so there was this old literaturecalled “About the Drug Lag” in the 1970s about how these, many of these drugs werereaching England in particular and some other countries in Europe before they were reachingthe United States, particularly with thingslike beta blockers, cardiovascular treatments, right. The claims again are usuallyabout reduced innovation, although there are argumentsthat go the other way and say, actually innovation or thelarger sort of properties of the health system are improved, that sort of go off thelemons argument in Akerlof.The argument sort of loosely stated as, well, once you start gettingrid of quack cancer treatments or once you yanktranquilizers off the market as the FDA did in the 1970s, you start to improve themarket for cancer treatments because the bad stuff doesn’tcrowd out the good stuff, all right, but again, theseare just a set of claims. The problem with a lot ofthese claims is twofold, and I’m gonna separatewhat we usually refer to as endogeneity into two senses here. Strict endogeneity inthe sense that basically regulation often responds topatterns of economic activity, which themselves respondto regulation, right. That’s the endogeneity ofthe kind that we can model and that I have modeledwith Mike Ting, all right.So in approval regulation, all these things coming to market, right, only the FDA can’t regulate, or at least can’t sort ofmake a decision on something that hasn’t been submitted to it, right, but firms develop and submit according to their expectationsof regulatory behavior. And those expectationsare probably correlated for what it’s worth witha lot of other things that change around the time of regulation. So if you’re looking at thelate 1930s, early 1960s, a wide range of scientificchanges going on in terms of pharmacology,applied chemistry and things like that, all right. The other problem isnon-random assignment, which is the usual thing we care about in these kinds of questions, right. I’m separating that from endogeneity because, again, endogeneity is something at least partially we can model. Non-random assignment,I don’t know everything that might be correlated withthe application of regulation in the new deal in theearly 1960s, in early 1990s, but suffice it to sayif our research design is premised up on a beforeand after comparison, well, lots of things might becorrelated with that, right.So here’s an example from oneof the most famous studies of Sam Peltzman on the 1962 amendments. And so what he did is he looked at 1962, which was when theseefficacy amendments passed. And he said, well, look,the actual number of NCEs, which is this seriesright here, went down. Now if you, by his productionfunction the way he sets it, it shouldn’t have gone down that much, it should have stayed higher. And so he has a counterfactual, which is the higher one here, and the split between thesetwo functions occurs in 1962.And he wants to arguethat difference after 1962 can be attributed to regulation and he finds or claims in other work that the cost of this is not made up by better therapeutics, right. Now, this is a pretty influential article, and to give him his due, thiswas published in the 1970s, but one might worry aboutessentially basing policy on a 14 point time series followed by a 10 point time series, right, and estimating two differentproduction and functions there. But the second problem is, is that this isn’t reallykind of a treatment or an intervention in any way that we can plausibly callexperimental, all right.And again, this is where I think an historical perspectiveactually helps to matter. For one, as you notice thesort of new chemical entities are falling from a peak inthe late 50s, early 1960s, before 1962 happens. And perhaps my chapter and some of my work on the application of efficacy regulation in the 1950s might explain that, but at the very least we don’thave a clean before control after treatment kind of world here, right. If in fact the numbers Iwas showing you earlier that basically the FDA is beginning to regulate efficacy here, and we really can’t trust alot of the kind of judgements that we’re making bycomparing things before and after a given date.Again, to be fair, he was writing somethingthree decades ago. – Does this in some sense coincide well with your previous figure, which showed that therewas a structural break few years before probablysometime in 1960, ’59. And this kind of shows that, yes, there is also astructural break in the- – Yeah, right, so I think minecould explain that, right, in part there’s two other problems here. One is he doesn’t nor do I control for industry concentration, and there’s some emergingevidence from the literature that actually suggests that one reason we’ve seen a little bit lessinnovation in recent years is precisely because of mergerand acquisitions activity.I can reference that separately, and that was occurring heavilyin this period as well. Now you could say, well,that’s endogenous to regulation because people arefacing a tough regulator, they wanna develop regulatoryaffairs departments, get big to basically beable to handle all this. That’s quite possible, it’s tough to kind ofdisentangle and sort that out. I agree actually that if we’relooking for the reason of why we come from this roughmean down to this rough mean? Probably that smoother regulatory function is probably a plausible candidate, right, but the point still remains that than a before and after comparisonusing 1962 is not valid.- Yeah.- Right, yeah, so, okay. So what to do? Well, here’s where we have an idea, and this is a storythat’s actually taken from in part the first chapter orthe introduction of my book, “Reputation and Power”but I’m repeating it here and actually talking about some features that I don’t talk about in the book. So you may know of Genentech, it’s kind of a darling of theCalifornia biotech industry. It’s now a quite big and profitable firm, goes up and down, but it usedto be a tiny little firm, and it had a very small drug called tissue plasminogen activator or Activase, and it submitted it to the FDA and was quite confident in fact that it was going tobe approved, all right, but a Food and DrugAdministration panel in June 1987 basically said no, voted against approval of the drug, right. And basically it wasn’t, and it’s important to keep in mind when the FDA says no to a drug, it never says we will neveraccept this molecule ever. All right, they wouldn’teven do that for cyanide.I mean, legally they can’t. What they say is, and it’s kind of like if you’re an academic and yousubmit papers to journals, it’s like getting an endless R&R, again and again and again and again without the certainty of evergetting an approval, right? So sometimes when the journaleditor comes back to you and says, look, next time,I’m gonna give you an up or down decision, the FDA never says that. And that’s actually ahuge source of complaint among pharmaceutical companies, like give us an if then statement, so that if we provide you this evidence, we’re gonna do that. Now with some work I’mdoing with a game theorist and another work I’mdoing with an historian, we’re actually trying totease out why the FDA follows this kind of strategy of ambiguity.And the difficulty is,is it’s very reluctant to kind of commit to a certainmodel of saying, all right, if you do this, then we’ll do this because then they feel that the firms or other firms can, number one, gain that and just basically come upwith a weak satisfaction of the if part of the hypothesis; and second, that they’re setting, and this is I think the real reason, they’re setting implicit and sometimes explicitprecedents for other firms. And that’s the other reason they do it. I’m not saying by theway that’s good policy, I’m just saying, that’s the rationale. I think that we think was going on, but this was bad newsfor Genentech, all right. This happened on a Friday, and if you follow government agencies, particularly in Washington theyoften announce these things after the market closes,this was one such example, but when the market reopenedfor trading on Monday, right, Genentech stock dropped by about a quarter and about a billion dollarsvanished, just like that, right.And so, this is kind ofinteresting for two reasons. One, there were kind ofsurprises to this, right. A lot of people did not see this coming, including a lot of peoplewho had bet a lot of money on Genentech, not justpeople at the company itself, but Genentech was publicly traded, right. So, and you can insert if youwant your snarky reference to the Romney victoryparty in Boston here, but they actually had planned a company executive victory bash, right, which wilted, and I justwouldn’t be able to write this as well myself into a combinationwake and strategy session.Try that sometime after your next professional difficulty, okay. And then the other thingis, is there’s kind of, if you will a peer or alter effect, a lot of other firms arelooking at this and saying, oh, crap, Genentech just got shot down, now what are we gonna do? Right. And so here’s one of thesepeople quoted anonymously.It’s like, well, wait a minute, now the FDA has kind ofchanged the ball game here. Something that we thought was a sure thing they’ve kind of raised the bar or we’re not sure where the bar is. So you see what we’re getting into. So here’s the idea, all right, it doesn’t solve every problemthat I just talked about, but it gets at how to assessthe effects of regulation or regulatory decisions on innovation. We’re going to use events like this, they come with a certaindegree of surprise. We can measure that surprise in a general equilibriumfinancial market, all right. We’re then gonna use thosesurprises as weights. So every time the FDA makesone of these decisions, it’s going to be weightedonly to the degree that it moves the market. We’re going to filter that price to try to get rid of othercontaminants, all right.And then we’re gonna usethat essentially to affect what other firms do,not what Genentech does after it gets its drug shot down, but what other firms do with that? Okay, that’s the strategy. And by the way, I thinkthis is at some level consistent with the larger storythat the book tries to tell because gatekeeping power, andfor those of you who are in political science who study vetoes, right, the power of the veto is notsimply the power to say no to something that comes your way, it’s to induce everybodyelse who would send something your way to begin thinking twice about whether they want tosend it in the first place. All right, so gatekeeping power is not simply the power of decision, it’s the power of induced anticipation. Question? – Doesn’t this kind ofregulation or regulatory change? It’s not just like theFDA changed its stance instead Genentech might have disappeared, and that influences thebehavior of competitors.So competitors are respondingboth to FDA getting stricter about antibody agents, butthey’re also responding to the fact that– Right. – Genentech might no longerbe in the market, right. – So there’s a set ofcomplicated effects here, and so for purposes of statistics, what I’m presenting to you is an average across all of those. It’s what a statistician would call an average treatment effect of this.That is gonna combine both theresponse to the FDA, right. It could be the higher bar,it could be FDA uncertainty, and it’s going to combine the fact that other people might see opportunities, which means that if anything,I’m probably underestimating these effects upon innovation, right, because what I’m gonnashow you is an average, it’s a composite of all those things, but one of those composites is probably, I can’t say for sure becausewe’d have to net this out, and we’re in the process of doing that, but one of the buildingblocks of that composite is probably positive, which is to say other firms might see an opportunity here and might actually continuewith their development projects, not pull them back.I do tend to think actually that the way that most firms respond to these things is that the regulatory effect washes out any like market opening. You see that quite commonlybecause the bottom line is all these other companies, right, who would wish to getinto the market, who say, ah, Genentech might no longer be there, but if they’re gonnabe where Genentech was, take up that niche, they’re gonna have to passthrough the regulator too, right. So, again, so what you’re saying is very interesting and useful, and basically it’s gonna depend on defining the set ofcompetitors quite exactly. What’s the therapeuticmarketplace or niche? What’s the mechanism of action? And we’re doing that in afurther extension to this, but right now what I’mgiving you is essentially an average across all those. – (indistinct) when itsdecision comes through, other firms in the industry,if they’re in Phase 1 or 2 or 3, they’re not pulling their drug at that point, are they? – Oh yeah.- Yeah. – Voluntarily?- Oh yeah. – They’re not going through that phase and seeing how the results.- No.So I’m in Phase 2, I’m plucking my- – People drop midstream all the time. – They simply not on theresults of that current phase. – The external factor by the way doesn’t have to be regulatory. It could be we had a bad budget shock, we had a new sort of ChiefFinancial Officer come in, looked at our portfolio ofactive projects and said, we don’t like this. And if you’re going tomake that decision to kill, why wait until something is done.If you think you haveenough evidence already and you’re just gonna, you’regonna make a business decision to say, all right, stopthis clinical trial. Now there are issues abouthuman subjects protection and things like that thatmight extend the clinical trial a little bit furtherin today’s environment, but again, this does happen midstream. – [Man] But there’s plenty ofevidence of drug doing well in Phase 1 or Phase 2and still being pulled. – Oh yeah, absolutely, absolutely, yep. Now that’s anecdotal, Imean, it’s kind of hard to sort of quantify drug doingwell in Phase 1 and Phase 2, we’ve got some ideas about how to do that, but plenty of exampleswhere that’s occurred. – So when you look at these events, are you looking at events where the FDA decision was a surprise? Because in this Genentechcase, it seems like they actually showed that their drug reduced this particular enzymeor whatever thing it was.And they just (indistinct)that means improved survival. And FDA didn’t buy the data (indistinct) versus a clinical trialwhere it just failed because there was- – We’re not looking at those because those would havehappened anyway, right. So we’re looking at cases where it’s the regulatorassociated with an event and we’re using the stock market shift as an indicator of the surprise, right. And the idea here is ifwe’re trying to sort of be kosher with our statistical estimation, we want something that’sboth non-anticipable, which is another wayof defining randomness, and two, conditionally not correlated with all the other thingsthat we’re worried about that might be correlated with that, right? So I don’t have abackground model here today, but basically here’s the kind of approach that we’re talking about.So imagine that a firm ischoosing dynamically every moment, okay, in time, DT, if you will, between a certain drugthat it’s developing, and this is by the way not Genentech, this is Genentech’s competitor, right. Between a drug and a safer investment, which gives you a known return, which we’re just gonna calla put option, all right.And it values this, the value of its investment is stochastic, and it basically is afunction of an initial state followed by an exponentiated X, all right. So this is basically always positive, think this as kind of analogousto a stock price, right. And this X is gonna be awhat I call a Levy process, what we call a Levy process, all right. And that means it can havethese more continuous things like a Brownian motion or Wiener process. It can also have jumps,which are these kind of very discontinuous up anddown movements, all right.Now if I give you thefollowing, and there’s a, I’m just gonna wave my handsat the French mathematician, Paul Levy, if I assumethe following things, independence of theincrements from one another. So given any given history, thenext movement is independent of what came in the past, all right. Stationarity, all right,so basically the idea that the expectation ofthese movements at any time is itself moving in a stationary way. And the continuity, when I mean continuity inprobability of the increments, obviously there’s discontinuityin the jumps itself, but the probability functiondescribing them as continuous. There’s a something calledthe Levy decomposition theorem and a set of other results that basically anytime youmake just these three results, you always get a Levy process. The Levy process in turn isessentially described by, and I’m just gonna wave my hands, I’m being kosher to give thekind of full equation here, but it’s a linear trend, all right, which could be zero, right. Brownian motion, which isthis kind of little thing, butterfly popping around.And then, again, I’m justdoing this to be kosher because there’s a knot at one that is, and again in the kosher theory,you can’t integrate over it, jumps, so all this stuff hereis just discontinuous jumps, all right. So every Levy process isa sum of Brownian motion, a trend in jumps, and each component, the trend, the jumps and the Brownian motion areindependent of one another, all right. So the idea here is this, again, what we wanna do is focus on these jumps, again, just I’m gonna wavemy hands at all this kind of, lovely math and say, that’s jumps.What’s left over is somethingthat at least in a reasonably functioning generalequilibrium financial market is already priced in, right. And then noise, right, which means actually there’s, every time we observe one of these jumps, a little bit of it is due to this, right. So we actually have a littlebit of measurement error, but we can plausibly claimthat measurement error is itself random or not anticipable, okay. So that’s what’s happeningfor a given firm, but maybe the firm, and this is, again, one of Genentech’s competitors, okay. So let’s call it genome therapeutics or something, all right. Maybe its decisionsdepend on its observations of another firm like Genentech, right. So that the value, alpha is a function, both of its own product, but also some function ofanother product, not its own, whose success or failure, and that includes success or failure in the regulatory domain tells that firm something useful aboutits own product, right.Now we don’t see that otherproduct as analysts, right, as somebody crunching the numbers, I don’t see what’s goingon with that other product, but I do see a stock price that’s based in part upon that product, right. And what I’m just gonna focushere is on the negative jumps. And I’m gonna do the same Levydecomposition I did earlier. Right. If again, it has these properties, I can reduce it to lineartrend, noise and jumps. I’m sorry, yeah, noiseand jumps, all right. So those jumps in theory, and we can actually testsome of these things, should be not anticipable, you can’t tell they’recoming ahead of time. One sufficient but notnecessary way of getting there is just to assume a perfect market. If you could know you’dmake a lot of money, therefore you’d make a lot of money, and all that information isalready priced in, all right.But again, it’s also, if notanticipable, uncorrelated, given the information up tothat point in expectation with other bases of firminformation, all right. So I’m gonna make the claimthis is plausibly random, it’s not an experiment, butas you know, plausibly random. So here’s the idea, the research design is, we’regonna use Wall Street Journal stories on FDA rejection,request for more data, for drugs under NDA submission,but not yet approved. Right, so we’re gonna take these stories, we’re gonna compute either theday those stories come out, the day the FDA makes the announcement or sometimes the companydoes or the day after, if that’s the trading date that’s relevant like the Genentech case,just the one day shift in the asset price for thatsponsor, the stock price, right. You could say, we should do more, and we’ve done a little bit of that and we’re looking at other filters, but the idea is we want tocapture only what that event had and not some other event that might happen like somebody got firedor somebody came in, there was a some newsales figure that came in, we wanna capture only that event, right.We apply that as a predictor to whether all other firm’sdevelopment projects, which is to say all the thousands of drugs they’re developing happened to get dumped in the months followingor continued, okay. So we observe from theearly 70s to December 2003, and this is actually forthe most part 1987 to 2003 or 1985, most of our analysisis focused in those 18 years, about 187 of these, right. And if we analyze basically what’s the correlationof those shocks, right, that the shock in the stock movement with a set of things that we can measure, we tend to find not much correlation. So do the shocks get bigger over time? No, they don’t get bigger or smaller. Are they correlated withthe beginning price? Because one of the wayswe’re measuring these things as the percentage change, so you might be concernedabout a denominator effect. Again, 0.05 correlation, notstatistically significant. Are they partially correlatedwith the size of firms that are developingdrugs at the same time? Again, they’re not. Are they correlated withthe general movement in the stock market that day? Well, not surprisingly, yes, because on the same dayit could have happened.The Labor Department couldhave come out with a report that said unemploymentis going up or down, it could have been somemajor market shift. It is correlated, although not a ton, and one might, but one thingwe can do in which we do do, and I can describe thisas we essentially purge our estimates of this general movement. So what we’re looking at isessentially the specific firm’s movement purged of the generalmarket movement, right. And we’re working on tests, whether these satisfied Levy properties. So some threats to inference might occur. Let me just sort ofgive you a little bit of the soft underbelly of theresearch design here, okay.What finance specialists will call volatility clustering is a possibility. And that’s the idea that,well, you can’t predict whether the stock is goingup or down on a given day, but if the stock is movingaround a lot, one week, it’s been shown that it’s more likely to move around a lot the next week. So there’s first moment, independence, but there’s not secondmoment, independence and stationarity in many cases, right. And we are, again, stillworking on a purge. Again, what that woulddo is not so much change. If this were a problemwould not so much change the sort of the validity, it would change the interpretation of our estimates from one of sort of the FDA is changing its bar, raising its bar or lowering its bar to the FDA is becoming more uncertain, but that’s a significantenough change in interpretation that we want to track that down.The second course is,the FDA does not report on all of its negative decisions, so you actually have to goto new services, all right, including The WallStreet Journal or others to track when the FDA handsout a negative decision. And the reason is, it’sa complicated exception to the Freedom of Information Act. If you ask the FDA, is a drug from Pfizer currently under review at your agency? The FDA cannot answer yes or no. That is consideredproprietary trade information.You cannot request informationabout that application under the Freedom of Information Act. Again, because it’sproprietary trade information, whatever whether that’sa good policy or bad, it sticks, right. So we actually have to look in the news for reports of this sort, and it could be that onlysurprises of a certain magnitude are likely to get reported. That does not change thefact that the day before they’re reported, they’renot anticipable, right, but it might change something about the distributionof what we’re observing. And then finally there issomeone who actually knows that these decisions are coming, right, and that’s the regulator or the regulators themselves, right. So you might know of Martha Stewart and the time she spentas a guest of the state. I hope she doesn’t watch the YouTube here. She was actually brought upon charges of insider trading, but actually got convictedon charges of perjury in that investigation. Sam Waksal was also, I believe indicted, I don’tknow whether he went to, I don’t know the exact story, but he was also part of that case.Here’s a case where aninsider, a chemist at the FDA, all right, knew that drugswere going to be turned down or delayed, all right, often focused on small biotechs, right, and bet on shares fallingafter negative decisions and sold shares to avoid losses. So exactly the kind ofthing that were occurring. If this occurred a lot, all right, like this was an everyday occurrence, and people like this didn’t get caught, that would be a big problemfor the research design I’m presenting you becauseessentially it would mean that a certain part of thatsurprise is essentially priced out or priced intothe market before it occurs because of all this kindof trading, all right.Reason I don’t think that that’s, but I’m presenting itbecause it is a concern, but the reason I don’t think it violates the sort of validity of thisresearch design is twofold. First off, these people do get caught. Mr. Liang is now serving fiveyears in a federal prison, all right. Second, the extent to whichthey can make money off of this, right, is limited by the degreethat if they traded so much as to cause me as an analyst problems, they would be all themore likely to get caught.So they can make a lot ofmoney for an individual, right, they can’t make so much money that they begin to reallychange the stock price. If they do, they’re far morelikely to get caught, right. If this couple of days before this and you see like a 2, 3,4% swing in a stock price due to one individual’s trading,even the SEC, I’m sorry, but the SEC has been gettinga lot of criticism lately, a seven-year-old with aspreadsheet would probably be able to pick up that kind of activity and detect the insider trading, okay. So here’s what these assetprice shifts look like. This was a fraction change, so if you’re looking for percentages, just multiply by 100, all right. So the mean is about a 10,20% drop in stock price after one of these things occurred. Sometimes there’s justnot much of an event, so these are the kinds that get essentially weighted to zero,it’s as if they don’t occur, those rejections don’t occur. Some of them are companieslosing 75% of its value.Now one of the things youmight be concerned about, again, is that some companiesmight be more likely conditioned on this happening to lose more their value than others.So one of the things we do, in addition to using the raw value purged of the general movement is also to binarize the treatment, which is to say, let’s have a cutoff sayright here, all right. Did the stock price fallmore than this amount as opposed to that amount? For what it’s worth actually,that does reduce the error and the models that we estimatequite a bit, all right. So that might suggestthat there’s a lot of extreme bouncing around thisdistribution, all right, but we do both, all right. And the other thing we do is essentially we observe a list ofthousands of drug projects that are undergoing developmentat a given point in time. And essentially if you’veused Cox models before, we essentially use aCox model of duration, how long does it last beforeit’s abandoned, all right, but it’s a little different, in that the analysis isconducted not only across drugs, but within drugs.And the idea if you’re sortof into kind of epidemiology is this is kind of withinsubject treatment, all right. So we’re controlling for all the features of the drugs themselvesthat are under development. The non-Genentechs, ifyou will, all right, but we’re looking sort of whathappens within those drugs. As a supplement, one ofthe things I’m gonna do is use a linear probabilitymodel, all right, which is basically zerowhen the drug is continuing, one when it gets abandoned, all right. Just gonna run a simple generalized Least Squares Regression on that and include a fixed effectfor each and every drug, which is namely 15,000 of them, so it’s kind of highly saturated model.And, again, that’s gonna turn this into a differences indifferences estimation. And that’s also gonna be awithin the subject treatment, all right. So here’s what it looks like,I’m sorry, here’s the data. So if you will, the dependent variable is, we wanna find out whethercompanies are moving on with their projects toward further testing or submission to the FDA orwhether they’re ditching them saying enough of this, right. We have about 14,000projects under development between the mid late70s and December 2003, and these are followed monthly.So we’ve got about a half a million observations in our database. The coverage is better after 1987 because this is a proprietary database produced by pharma projects, right. This is a private company that’s been following themarket for a long time that aggregates a lotof these market reports. The coverages, again, gets better, and so one of the things wewanna do is say, all right, let’s only look at the dataafter a certain amount of time and then change that, just to see whether our results still hold up. One limitation and I’m sort of trying to get a grant for this, this is all before the Vioxx tragedy, which by some estimates contributed to 20, 30,40,000 excess deaths, things like that. There’s an argument that the FDA got more procedurally conservativeafter the Vioxx tragedy that I think needs to be tested, but we’re not gonna seethat in these data, right.We have two different measuresof abandonment, right. One is when the company justsays, we’re done with this, and they come out withan announcement, right. Often companies don’twanna say those things, in part because they wannasort of keep their options open and things like that, sowe have an implicit one, which is where this database reports no development reported, all right. Once that happens for two years, we go back and code it fromthe time it originally started being coded as such andsay the drug was abandoned. We use each of these alternativelyand then we combine them. All right, so that we’re notdependent on given one measure. We allow the effects then ofthese shocks to be generic which is to say applying to every firm or applying to a firm,which is a rough competitor or an entrant into the therapeutic niche, say cancer drugs, centralnervous system drugs, cardiovascular drugs,in which the bad events or the negative news forone company happened, right. We’re defining this class very broadly, this gets to your questionabout the competitive effects.So one of the ways we’regonna do that here, and we could do it muchmore narrowly with kind of a refined data on the mechanism of action. Right now I’m just gonna use the division structure of CDER. Now in part CDER by the way is the Center for DrugEvaluation and Research. It is the FDA bureau that makes these decisions on the drugs. And so, one reason we mightwanna do that is because if, the extent that thesefolks are making inferences about the FDA and saying, oh my goodness, the FDA is getting muchtighter, they’re not just making a judgment about the FDA generally, but about the particular rule, the particular decision makersin the oncology division or in the cardiovascular drugs division who may have changedtheir standards and said, oh no, no, no, P less than 0.10 is no longer statistically significant, we’re gonna say that’s P less than 0.05.Or we’re gonna demandanother different kind of clinical trial with anotherdifferent kind of treatment arm or control arm before we sendsomething onto the next stage or approve it, right. They might be making in otherwords decisions or inferences, not about the bureau orthe regulator writ large, but about sub-regulatorswithin that bureau, right, which is one way ofactually thinking about possibly a way of kind ofquantifying agency reputations and sort of de-compartmentalizingor compartmentaling, decomposing the agency writ large. Go ahead. – I’m still not sure that this is, if you could interpret thissolely as changes at the FDA, this could just be scientific surprises. So we’re doing a clinicaltrial for a certain drug and you were hoping it willwork, but it didn’t work, and that changed science andthe stock price plummeted for this company becauseeveryone thought it would work, but it didn’t work, andit’s got nothing to do with how FDA validated it or insome sense it’s a mixture of, (indistinct) exactly, I don’t know whether Iwould interpret this solely as changes within the FDA.- Well, so it’s always true, I mean, so here’s the problem, right, is that every regulatorydecision is a decision about the merits of a given drug, right. Now if it’s a decision aboutthe merits of a given drug, right, then we should clearlysee a within firm effect, which is to say Genentech gotthis bad news about its drug, they should drop it there. It’s not clear that that logicextends to everybody else including outside of the therapeutic area. – [Man] Like samemechanism of (indistinct) – Right, so that’s, that’sexactly why we’re doing this. If you’re right, we should observe a lotof class specific effects.- Or it could also belike a financial shock. I think you’re a VC andGenentech stock plunges, you’re like, I’m out of all biotech, I’m investing in cars instead. – [Dan] Yeah, you’re outof all biotech precisely because the FDA ruledagainst your (indistinct) – No, but not becausethe FDA rules against you because the science was bad, then there’s a lot of hopethat biotech is gonna produce great medicine and Genentech trial fails, I changed my expectationsabout biotech more generally. So this is bad science, I need to invest in nanotechnology or something else. – Yeah, first off I don’t think in sort of a general equilibriummarket, that’s gonna happen. I mean, basically especiallywith a publicly traded company, right, there’s enough other people to say, look, there’s a possibility here, and it’s possible there’sgonna be an overreaction and things like that. To the extent that it’s about purely, it’s picking up purely likea scientific development, first off, that’s notinconsistent with my story, right.Basically this is, thescience is being produced, but the science is being produced and judged by the regulator, by the regulator’s advisory committee. So you can view this asa scientific revelation in many cases, right, but again, this revelationwould not happen in the absence of approval regulation because we’ve already had the announcement of Phase 1, Phase 2 and Phase 3 trials. This is all after all of that, right. So it can’t just be, it could be a further scientific signal, but it’s a scientific signalfrom the regulator, right, and I think that’s the key. The other thing, again,is, is to the extent that it really is aboutmechanism of action, I’m not worried about like the whole world abandoning biotech.I’d be much more concerned about saying, look, in this market likethe FDA is being too tough or we’ve had this failure, we should see basically highdegree of class specific action and not non-specific action. It turns out we’re gonna see both. – And I think since you’re basing this on The Wall Street Journal stories, maybe if you have someoneread through those stories and try to say how many ofthese stories were about, people complaining that theFDA made the wrong decision or made a very strict decision. – [Dan] We do that actually. – Okay, I think that– Sure. So some evidence for (indistinct) these are very large estimates for when the FDAhas an advisory committee and the advisory committeevotes it down surprisingly, right. And that is consistent with the idea that it’s not simply the FDA, but also the scientific advisors giving a negative judgmenton the drug, right, but again, that’s not the only place we observe a lot of these.So if the FDA says, no,look, we want another test or, no, we want a set of other things. And, again, remember, keep in mind, all three phases of clinicaltrials have been completed for almost all of these,at least two have, right. So it can’t be just that aclinical trial previously when… You’re right that theremay be some revelation of scientific information still left, but again, that’s only coming because we have this regulatory process. So here is the effects ofone of these shocks, right, and I’m just gonna generalize this to say, all right, let’s justimagine one of these shocks is 10% drop in the(indistinct) stock price. What happens to the hazard rate of abandonment for all other firms? That is to say month by month by month, what’s the increased rate at which companies abandon theirdrugs given that 10% shock? Now one thing I do here is, is T plus zero is the month of the shock.So one of the things we do is actually we include some leads here, and that’s a test of two hypothesis. One, it’s kind of what youmight call a placebo test. The idea that these shocks should not be predicting something thatthey really can’t predict, which is abandonment ahead of time. And it’s comforting inthis respect to know that these by the way, these reds are the parameter estimates, these are 95% confidence intervals, both individually andjointly these are zero, okay. The second is, is this isa test of anticipability. If in fact, these things could in fact be hedged ahead of time, you should see other companies adjusting their developmentstrategies in the months before.And again, this isstatistically zero, all right. Where one sees the effects is essentially beginning in the second month, and continuing roughly if youwant to sort of judge that as on the margin ofstatistical significant until about the sixth month. It takes time, in other words,for these to filter their way through firms and their decision processes to make judgements about that. This is by the way generic, this is both therapeutic specific effects and non-therapeutic specificeffects combined, all right. Once you get out here,there’s just enough noise that there’s reallyjust not much going on. If I run that linear probability model, I talked about earlier,okay, so this is not, this is a little less interpretable.Basically, if you will, this is, what’s the change in theprobability of abandonment? Again, we have to adjust the things. It’s for lack of a better term, essentially the same resultsalthough a little bit less statistical significance we get these two, T plus two and T plus four. If you actually compare these two, they have the essentially the same shape, even though basicallynothing going on early, right around T plus two toT plus four arise, oops, and then down to wherethere’s just a lot of noise, all right, which iscomforting in the sense that basically the linearprobability model relies heavily upon these fixed effects to generate a within subject treatment. So it can’t be any feature forthe linear probability model, excuse me, it can’t beany feature of the drug that’s currently under development, right. It has to be only the shock that’s generating thisresponse, all right. And, again, notice that thelead values are all zero. So there’s not anticipability here. If I, again, just get rid of all the leads and everything past six months, things bounce around afair bit more, all right, but the average of this is quite positive.If you will, each 10% shock, if I integrate over these distributions, each 10% shock leads to aboutfour to six drugs abandoned in the six months following, okay. We can’t say that those drugs would have eventually become approved, we can’t say that they wouldbecome useful treatments, so that they would havebeen marketed well, all we can say is they havean increased probability of the firms themselves pullingthe plugs in response to that.Okay. So now if we look withintherapeutic category, we look at this division chart, these are the therapeuticcategories we’re going to use. Essentially there’s 14 and not 15 because this one is OTC,over-the-counter drug products, we’re not looking at those. So it could be skin and dental, it could be antiviral,it could be anesthetics, it could be pulmonary, things like that. Some of these names maybe recognizable. Robert Temple is one ofthe most influential people in the history of 20thcentury pharmaceuticals.Again, he’s got a, now a kind of a top level deputy commissioner post, but at this point he was the head of one of these drug reviewing divisions. This guy is often very controversial, is often taken to task in TheWall Street Journal editorial pages as being sort of adrag on cancer treatments. And so some of these namesare kind of very well-known. If we look at the effect of the 10% shock in therapy targeted, we get stuff that’s verysimilar to what we had. It bounces around a bit, but very much similar towhat we observed before. The second thing we can do is say, well, what happens whenwe kind of break these events down by what was happening? So let’s just examine five categories. And for those of you who do work in statistical text analysis or coding or content analysis, this would be a great applicationof those kinds of methods. Basically look at whatkind of decision this was and try to classify it, but it could be a case wherea company abandoned the drug on its own incited FDA regulationas a reason for doing so, so we code that separately; it could be an FDA request for more data; it could be an advisorycommittee voting and saying no; it could be the FDA saying, we’re not ready to makea decision on this yet.Okay. Each of these outside of the FDA saying, we’re just rejecting this, all right, doesn’t seem to have an effect. Now remember, one reasonmight not have an effect is because this is probably theeasiest one to anticipate where the FDA on the deadlines says, we’ve made a decision up or down, and that the firm iskind of communicating, oh, we’re not gettinggreat signals from the FDA, so it’s not surprisingessentially that that’s zero. Technically it might be statistically significantly negative, but I don’t put much in it, all right. The biggest effects are from when an advisory committee suggests no. There’s a bunch of reasons for that, I would bet or hedge. Number one, that’s the firstread on the FDA’s thinking and outside committee, whichis going to advise the FDA after these phase trials, all right. Sometimes there’s a public today and often there’s a public report released by the FDA review or theFDA review team in advance, but the period we’re dealing with, that report was often releasedat this meeting, right.So there’s a whole bunch ofthings that are folded in here. Second, this is a sort of a judgment, not simply about what the FDA thinks, but what a panel of sort ofindependent cardiologists who advise the FDA thinks. So this gets in partto your question about to what extent is thisa signal from science? Well, again, it’s both, but here again it’s where we’reletting the sort of advisors speak a little bit independentlyof the FDA as well, right. It turns out that a fair degree happens just from the cases where the FDA says, we’re not ready to makea decision on this yet. And it’s tough to figureout the reasons for that, it could be that we’d like more data, so we don’t think that, we think that it looks good, but we’d like more proof,a bigger sample size, a smaller confidence interval, or we’re just, we’re notready to make a decision yet.So it could be, themail room isn’t working, we need a plumbing repair on floor three, something like that,but that also generates a higher degree of company abandonment and other companies abandoning and citing the FDA as a reason or citing regulatory factors as a reason also leads to about a 4%increase in the hazard rate. These by the way aresummed across six months, I’m sorry, seven, the month of plus thefollowing six months. And here’s what we do if webinarize the treatment, right. So this is where we’vetaken that stock price shock and we purge it, all right. And then we say, all right,we’re gonna assign a one, if it drops by more than 3%, and zero if it doesn’t drop, I mean, it doesn’t drop by more than 3%, and then we’re gonna sum across 12 lags. And essentially most of this effect is occurring withintherapeutic categories, right. And that’s a very large hazard ratio because that is beingmultiplied by month across firms many, many times over.So now if we take theseas kind of our evidence, we’re talking about 30, 40,50 drugs getting dropped after one of these eventsand not just a few, but keep in mind that someof this is also occurring generically, which is to sayoutside of therapeutic class. So you can’t ignore the fact that some people are making inferences, not just about what the FDAoncology division is thinking, but about the FDA writ large, right.This is specifically codedas to say, all right, an oncology drug goes down, What is the reactionof people in cardiology developing cardiology drugsor infectious diseases drugs? AlL right. And this is a case wherewe actually control for a few other things, right. So what do plausibly abandoneddrug projects look like? Well, essentially weexpect those with a shock and then what happenstwo periods afterwards? All right, which wasone of the significant, statistically significantparameter estimates that we have. So we can’t know whetherthese in fact were caused, we just say it’s consistentwith the causal story. These would be predictedto have a higher level of regulator induced abandonment, okay. So it turns out that over95% of those abandoned are in Phase 3, which from an efficiencystandpoint is bad news. If you wanted these to be abandoned, you’d like them to beabandoned early before all that capital is sunk in, right. Now I can’t say whether over 95% of drugs that are abandoned are in Phase 3 because we don’t have greatdata on where these things are.And the further they go in the process, the more likely they areto be reported at all. So all I can say is for those drugs for which we have phase data,Phase 1, Phase 2, Phase 3, 95% of these are in Phase 2, but that’s highly, highly selected because if you get toPhase 3 in this database, it’s much more likely thatthe people who put this database together are able toreport that you’re in Phase 3. What you can say I think though is that a fair numberof these are in Phase 3 and are dumped, right. We can’t say that 4,000 drugs were dumped because of regulatory factors, right. We can just say that among those that occur in these events right after two, four months afterthese, a high number of those for which we know thephase seem to be Phase 3, all right, and we have to sniff some more to kind of dig where that is. Most of these are, again, implicit abandonments and non-explicit, but if you look at the, andI can send you the paper or you can even lookat the previous slide, you get very similar results as opposed to whether you focus on explicit abandonments or implicit abandonments, all right.So choosing one oranother of those measures actually doesn’t seem to affect much the results that you getfrom these estimations, which is somewhat comforting. So to conclude on this part, well, I think this is still speculative. I mean, one thing I’dlike to be able to say is give you a harder estimate of, well, when one of these things happens, the following numberof drugs are abandoned, and they’re abandoned in thisphase and things like that. There are some limits on the data, which I think will preventus from ever being able to do that in a fully satisfactorymanner, but one can do that. It’s also important to saythat this is not an evaluation of what happens in responseto regulation generally like the issuance of a new rule, but the issuance of aregulatory decision, all right.And that points I think tothe difficulty of measuring the overall effects of a policy because regulations usuallycome in bundles, right, and regulatory decisionsusually come in bundles. So you say, well, let’s evaluate the effect of this regulation on Y. Well, what part of theregulation are you picking up? Because the regulation isprobably a statute, right, or a rule with seven different components. And is it component two or component five? So there’s a lot of debate rightnow about what’s the effect of the Dodd-Frank Acton the financial realm? Well, the Dodd-Frank Actis prudential regulation, which is to say large banks, it’s regulation of credit rating agencies, it’s regulation of thehome mortgage market, it’s the new Consumer FinancialProtection Bureau, right, it’s 20 different things.In fact, really it’s morelike 20,000 different things going on in that bill, right. And so, assessing the effect of a piece, of regulation writ large ora piece of regulatory statute is very hard because thesethings come in bundles, and it’s very difficult to disentangle one part of that component from the other. And so the more you focus up, the more you basically giveup in terms of granularity. The more you go in terms of granularity, the less you’re able to focuson regulation writ large. I don’t think this problemis fully escapable, right. I don’t think it’s possible to just say, well, there’s a strategy out there that will allow us to speakabout regulation writ large and also to have thiskind of granular approach. This is what I think at somelevel political scientists can teach to those whowish to evaluate policy, policies come in bundles,and it’s hard to disentangle one part of the bundlefrom another, right.It’s difficult also todraw policy conclusions, again, all I can say isthat firms are more likely to pull the plug on these projects. I cannot say, right, that theseprojects were of high value. We might be able to follow later on in some of thesetherapeutic areas and say, were there cost beneficialnew products introduced? What happened to morbidity, mortality, some public health measures in these areas where there were more surprise rejections? We might be able to follow that, but I haven’t done it today. And, again, the more youstart to sort of take into to account some of thesetherapeutic area specific measures, the more you’re beginning tosort of introduce other areas, which can contaminate, right. There’s no way of knowing essentially what the health effects would have been, in other words had thesethings gone to market or what the economicprofitability would have been had these things gone to market. That said this method does open the black box a little bit, all right. We know that it’s not simplythe FDA rejecting a drug that might lead to less innovation, which is to say the FDA making a decision, no on something that’s sent to it, but the effect that that ishaving on firm’s own decisions not to continue their ownproduct development processes and not to seek approval forthose projects later, right.It’s also potentially generalizable. In theory if you can find regulatory enforcement decisions in otherdomains, focus them on firms, compute what happens to those firms as to whether they’regoing up and down, right. And then I think this is the key, can you get a large databasewith high granularity on what other firms inthat domain are developing? Energy development projects, right, consumer financial orsystemic financial innovation.It’s really that dependentvariable kind of data that one needs to be able to evaluate, the stock market data, and then some cases theregulatory enforcement or decision data is always there. What you really want is ahigh granularity database at the level of firm decision-making to be able to evaluatewhat happens with R&D. So I’ll conclude there andopen it up to questions as, other questions as you like. – Have you done similar research with devices, medical devices? – I have a graduatestudent who is doing that, and I may end up joiningher on that project or not, but that’s exactly one of thethings where that’s occurring.Yeah. – How much control do thedirector of the difference (indistinct) have over like what the thresholds they’re using? Just I was wondering if youhave data on who is in charge and whether they have areputation of being (indistinct) cars or something (indistinct) – Yeah, that’s great question. So actually the womanwho just asked a question has a copy of my book there. Thank you. And one of in the historical period, in historical work that I do, I describe this process of sub-delegation. So in theory this power ofveto is given to the secretary namely Kathleen Sebelius,but in the 1960s and 1970s, it kept on gettingsub-delegated to the fact that you’ve got careerbureaucrats making these decisions now in a way that’s almost never overturned by higher levels.The only case recentlyand we talked about this at dinner last night wherethere’s been an overturning was the Plan B decisionwhen Obama and Sibelius basically turned downthe approval of Plan B for over-the-counter status, but that’s the exceptionthat in some ways, although I worry about theprecedent that it might set for kind of overturning doing that. It is possible and Idid it a long time ago and I kind of gave up on it to, if you can get approvaltime data to net out the effect of differentreviewers basically by like computing a fixed effectfor each reviewer, and then just to examinethe fixed effects.And I just never went very far with it, but I’ve got all my datafrom this book online, not all of it, but a lot of it. And if you went to it, I could probably give you some others. We basically coded the entireCDER employee directory from the eighties through the early 2000s, so we have like 5,000employees in this database. And you can see in many cases which one of them did the review, what the review team composition was. And you can net out theeffect of a division director and things like that. That assumes, of course,that you’re controlling for everything else thatmight be correlated with that. So in theory that’s possible, but I never went so far withthat as to do it in part because there’s a lot of missing data on who made the decision in thiscase, who made the decision.It’d be easier to do in more recent years because the FDA is actuallypretty good on the whole, given the limits of theFreedom of Information Act about putting a lot of this data online. – So, Dan, (indistinct) storyabout the these approval that’s being kind ofendless in our process, but we don’t get the signal that, (indistinct) point specifically. Is there anything otherthan insider trading that could signal that(indistinct) to the market, right? Because now you’re talkingabout the financial markets, so if there’s any bleedingthrough congressional committees or anything like that? I’m just kind of (indistinct) – So actually, I mean, two things.I mean, there is at some levelkind of a continuous kind of information at least aboutthe way these things happen. I’m not worried about thatin terms of internal validity because again, that gets priced in. So I’m looking at what happens the day of, what happens the day after. That’s another reason forfocusing just on that, one day shock, but I think there’s amore interesting process by which some of this gets to… So one of the other thingsthat you could actually do by the way is look at what happens to other firm’s stock values right after. So I’ve looked at what other firms do with their development decisions, you could look at otherfirm’s stock values. The problem is that couldbe responding to a lot.And in fact, not least theregulatory decision itself, like I’m a competitor in this market. Maybe it goes up becausenow there’s space, but more likely it probably goes down because they have to passthrough the same gauntlet. Now hearings, I’m not so concerned about, but there’s a constantcommunication between the firm. And so what gets releasedto the marketplace, things like that, I mean,so what we do know is that in theory the review teamsdeliberations are lockbox. It’s only at or just beforea today an advisory committee that the review team’smemos are put online. There’s often a lot ofmovement right there. If we had more recent data, we might be able to kind of exploit that.The clinical trials are lockboxfor a number of reasons. One is blinding, right, so you can’t inspectthe data halfway through and say, does this looklike it’s going well or look like it’s not? Although if this idea for more Bayesian clinical trials takes off, you might see more of that,which could actually create some interesting problemswith insider trading that I really hadn’t thought about that. Yeah, that’s interesting, but at least again, themore traditional model now that’s lockbox, andthat’s in part FDA regs, but it’s also humansubjects and blinding regs.There’s a lot ofcommunication that goes on between these review teams. And, again, the problem is,is if you’re a company person and you’re holding stock andyou’re privy to some of these. The one advantage that the SED has is, it knows who is privy tothat information, right. So it knows who has accessto the database at the FDA, and it knows all thepeople at the company. And you will see people at these companies getting hauled into courtand sometimes put in jail for having heard bad information and then going selling the stock or having heard good information ahead and going and buying the stock, hedging one way or the other. But a little bit of it does, there is, I mean, it’s a little bit more continuous that I’m stating here. There are some huge discontinuities, but it is a little more continuous. – I wonder what is the mostreasonable cause for mechanisms behind these, say one analogyI could think of is that among academics, so is (indistinct) the paper rejected by a journal, and it reduces my (laughs) my urge to submit it to the same journal because it lower my expectation, but in your case maybe the problem is there’s only one journal, better you do it or you don’t, right.So you have no (indistinct)journals with something too. So that is more a psychological event, or would that be more like whatthe gentleman referred to is kind of revealing some kindof the underlying scientific promise of a certain mindset of thinking. – [Dan] Right. – So what would be your take on that, what would be the actualcausal mechanisms behind it? – Well, I actually think that not all this is purelyrational expectations, right, but in order for mystory to work, I don’t, this doesn’t have tohave full rationality.To the extent that people are kind of scared off by the FDAperhaps irrationally, so that they should have continued on. My story doesn’t changebecause it’s a story about the effects of policy. And I do think actually, this mainly comes not fromthe quantitative research, but years of looking atthese industry trade journals like the pink sheets andother things like that, there’s a lot of fear in this industry because they recognizethat they’re sort of in front of the all powerful regulator. And even though we like to tell stories about the pharmaceuticalindustry dominating the FDA, that’s number one, a morerecent development where the pharmaceutical industryhas had that kind of power. And number two, firm by firm, these companies are stillvery afraid of the FDA and these drug reviewersand things like that. So I think actually a lot of this is basically being scared off. Some of that fear may beirrational or inflated and some of it may be rational, which is to say we thinkthings have happened here. It’s hard to really nail the mechanism.I think part of it isexactly what he is saying, this is a revelation of science. I actually don’t think, again, that’s inconsistent with the story because that revelationwouldn’t be happening but for the regulatory process, right. In other words, if you could… Just required everybody togo through three phases, announce those phasesand then go to market, you wouldn’t be seeing these effects because the three phase trials are already being priced in oncewe’ve seen this, right, but I think and a lot of this because precisely becauseit’s happening both within therapeutic class andoutside of therapeutic class is judgements about the FDA.What I can’t say here, althoughI think I probably could with a little more confidencewith some more data is whether this is the FDA raising the bar or more uncertaintyabout where the bar is. And I think that’s animportant policy question. My sense, again, just eyeballing the datais that sum of both, and I’ll probably need to kind of do some auxiliary test to kind of do that, but both of those are important questions. You can make an argument thatfrom a policy standpoint, you might wanna havehigher bars or lower bars in certain points, butit’s always better off to maybe know where the baris to have less uncertainty for the industry and forscience and things like that. Although there’s an argument that ambiguity can also serve purposes because it doesn’t allow the firms to gain the system as much.It keeps them kind ofon their toes as well, but I do think it’s being scared off whether it’s by uncertainty or by the bar changing, that is probably the mechanism here. – So this is (indistinct)question that this is in context of (indistinct) but is there evidence (indistinct) are they’rerelated to other markets (indistinct) structured for? – So here is, again, theproblem this gets to Yan’s nice point about there beingone journal editor, right. One reason that the FDA is so powerful is because it controls access to the most profitable pharmaceuticalmarket in the world. So, yes, if you want, youcan go introduce your market, your drug to the European market, but it’s gonna be price controlled.It’s gonna be in a countrythat’s not as rich, and it’s actually not agingas fast as ours, right? So we have highpharmaceutical consumption, basically zero price controlsat the margins with a couple and maybe with Medicare PartD, in the future we will, but what makes the FDAso powerful in this world is precisely the fact thatit’s a stringent regulator in a world where priceregulation is not stringent. So gatekeeping power, in other words is directly proportional, theamount of gatekeeping power to the price that you’rekeeping aspirants from, right. And the FDA doesn’t control the fact that there aren’t pricingregulations in the US and it doesn’t controlthe political economy of the United States, but its gatekeeping powerbenefits at some level from these other factors. Last question (indistinct) – If there’s one more question we might have time for, if not…- Yeah, go ahead. – So I think coming back tothis point about mechanisms, one thing I was struggling with was, what these like, you’ve convinced that these changes were exogenous that if they were like a randomshop and were a big change. – [Dan] Mm-hmm. – But it’s (indistinct) to know whether they were just transient or more problem, problem in the sense, was this having asked for areviewer on the review committee and rejecting a trial (laughs) or was this a change in the FDA’s stance about their threshold? – So I can say that,yep, but we could, right, because I could say, all right, was this followed byleader decisions, right. – And I don’t know how to, those leader decisionsare more complicated because as you said because of endogeneity that now you know there’sa higher threshold, you don’t take drugs with the FDA, which thin are gonna– Right.- [Man] (indistinct) think all that- – But once you begin to analyzethe process in this way, can you at least openthe door to answering some of these questions.- Yeah. – But I agree I haven’t done it yet. I mean, essentially what we wanna do is trace not only the decisionsas weighted by shocks, but a series of decisionsthemselves and say, is there a pattern here? And at some level that’s kindof descriptive in their bones, but I think you kind of need to do it, step away from the internalvalidity church for a minute, and then kind of focus more on kind of descriptive features in order to get.And that, again, allowsus to go a little more from regulatory decisionsto regulation writ large. – [Anthony] That’s great. Right, well, let’s thank Dan. – Thank you.(audience applauds) Thank you..

As found on YouTube

Free Coupon for the Pharmacy

About Post Author

Happy

0 0 %

Sad

0 0 %

Excited

0 0 %

Sleepy

0 0 %

Angry

0 0 %

Surprise

0 0 %