From: To: Cc: Subject: Date: Attachments: Unger, Ellis jrnlprod.ANA@cenveo.com Unger, Ellis RE: Your article 24842 (24842) from Annals of Neurology is available for download Wednesday, January 11, 2017 2:45:30 PM Editor Ref # 16-1449.pdf Dear Sirs: I have attached a PDF scan of the manuscript with queries answered.  Please let me know if you have any questions. Thanks! Ellis Unger, M.D. -----Original Message----From: jrnlprod.ANA@cenveo.com [mailto:jrnlprod.ANA@cenveo.com] Sent: Wednesday, January 11, 2017 7:54 AM To: Unger, Ellis Cc: jrnlprod.ANA@cenveo.com; jrnlprod.ANA@cenveo.com Subject: Your article 24842 (24842) from Annals of Neurology is available for download Importance: High Annals of Neurology © Published by Wiley-Blackwell, USA Dear Author, Your article page proof for Annals of Neurology is ready for your final content correction within our rapid production workflow. The PDF file found at the URL given below is generated to provide you with a proof of the content of your manuscript. Once you have submitted your corrections, the production office will proceed with the publication of your article. John Wiley & Sons has made this article available to you online for faster, more efficient editing. Please follow the instructions below and you will be able to access a PDF version of your article as well as relevant accompanying paperwork. First, make sure you have a copy of Adobe Acrobat Reader software to read these files. This is free software and is available for user downloading at http://www.adobe.com/products/acrobat/readstep.html. Open your web browser, and enter the following web address: http://115.111.50.156/jw/retrieval.aspx?pwd= You will be prompted to login, and asked for a password. Your login name will be your email address, and your password will be Example: Login:  your e-mail address (b) (4) Password: The site contains one file, containing: • Author Instructions Checklist • Annotated PDF Instructions • Reprint Order Information • Color Charge Form FDACDER00041 • A copy of your page proofs for your article In order to speed the proofing process, we strongly encourage authors to correct proofs by annotating PDF files. Any corrections should be returned to jrnlprod.ANA@cenveo.com 1 to 2 business days after receipt of this email. Please see the Instructions on the Annotation of PDF files included with your page proofs. Please take care to answer all queries on the last page of the PDF proof; proofread any tables and equations carefully; and check that any Greek characters (especially "mu") have converted correctly. Please check your figure legends carefully. - answer all queries on the last page of the PDF proof - proofread any tables and equations carefully - check your figure(s) and legends for accuracy Within 1 to 2 business days, please return page proofs with corrections and any relevant forms to: Production Editor, ANA E-mail: jrnlprod.ANA@cenveo.com Technical problems? If you experience problems downloading your file or any other problem with the website listed above, please contact Balaji/Sam/Ganesh (e-mail: Wiley.CS@cenveo.com, phone: +91 (44) 4205-8810 (ext.308)). Be sure to include your article number. Questions regarding your article? Please don’t hesitate to contact me with any questions about the article itself, or if you have trouble interpreting any of the questions listed at the end of your file. REMEMBER TO INCLUDE YOUR ARTICLE  NO. (i.e., 24842) WITH ALL CORRESPONDENCE. This will help both of us address your query most efficiently. As this e-proofing system was designed to make the publishing process easier for everyone, we welcome any and all feedback. Thanks for participating in our new e-proofing system! Sincerely Production Editor, ANA E-mail: jrnlprod.ANA@cenveo.com **** FDACDER00042 A04 ANA Customer ANA24342 Cadmus Art: ANA24342 Ed. Ref. No; 16-1449 Date: 11-January-17 Stage: Page: 1 Regarding ?Eteplirsen for the Treatment of Duchenne Muscular Ellis Unger, MD and Robert M. (:aliff. MD This letter is in reference to the article "Eteplirsen for the treat- ment of Duchenne muscular published in the Annals of Neurology. The principal conclusion of the study was: ?Eteplirsen restored in the 30 and 50 cohorts, and in subsequently treated, placebo-controlled subjects." We believe that the reported findings for the 48?week study are based on unreliable data and that the conclusions, based on these erroneous findings, are misleading. We therefore urge that the article be correcred or retracted from the Annals afNenmlogy In the course of the US Food and Drug Administration's (FDA) review ofa New Drug Application for eteplirsen for the treatment of Duchenne muscular in patients with a confirmed mutation of the DMD gene that is amenable to exon 51 skipping, the FDA inspected the Columbus, Ohio, facility where the study had been conduCted. As nored in docu- ments pOSted on the Internet subsequent to our September 19, 2016, approval,? a number of significant methodological prob? lems were found that importantly alfecr the study results. The Patients and Methods section of the publication states: "To evaluate fiber levels, immunohis? tochemical Staining was done on 10pm frozen secrions of 3 separate blocks of biopsy tissue separated by at leasr 200 itrn and evaluated by blinded expert muscle patholo? gisrs. For localization, seetions were Stained with MANDY5106 NCL, a mouse monoclonal antibody to amino acids 16] using standard immu? nofluorescence methodology.[9, l4] fibers and tOtal fibers were counted. and the percentage of fibers was calculated across all samples." Note that the firsr sentence states that the tissues were evaluated by ?blinded expert muscle pathologists." implying more titan one such evaluator. The third sentence explains that and total muscle fibers were counted, and becausr: this sentence closely follows the first sentence and is in the same paragraph, there is the implication that the fiber counting was also performed by "blinded expert muscle patho? logists.? In facr, the counting was performed by a single histotechnician. ?Downloaded September 29, 2016 from http:anvw.accessdataldagovl ID: pachlyappanm Time: 17:54 LETTER The FDA inspectional ?ndings pertinent to the fiber counting are reprinted from pp. 12 to 15 of the Office Director Decisional Memo.? ?Although the technician had been blinded to treatment group, access to the treatment code was not protected with the kinds of safeguards and firewalls that one would ordinarily put in place for an adequate and well controlled trial. The immunohiStocherniStry images were only faintly stained, and had been read by a single technician using an older liquid display (LCD) computer monitor in a windowed room where lighting was not controlled. (The technician had to suspend reading around mid-day, when brighter light began to ?ll the room and reading became impossible.) These issues are well described in a summary of inspectional findings in Dr. Breder's clinical review (page 27). There was also concern that the reader, although masked to treatment assignment, was nor masked to sequenceitime (see below). lmportantly, in a trial where all patients eventually received the active drug, knowledge of sequence could lead to the false appearance of a treatment effect, the appearance of increasing expression with time, simply by having a lower threshold for calling ?bers ?positive? at later time points in the Study. Having uncovered numerous technical and operational shortcomings in Columbus. our team worked collabora? tively with the applicant to develop improved methods for a reassessment of the Stored images. We suggested a re- read of all images by 3 independent masked readers, such that blinding could be assured and inter- and intra- observer variability could be characrerized. We also sug- gested the use of better equipment, specifically, high- quality light-emitting diode (LED) computer monitors, in darkened rooms. The applicant undertook a blinded re-analysis of the images on the server as FDA suggeSted. Unfortunately, the re-analyses failed to show a significant increase in fiber counts in eteplirsen treated patients (Figure 3). Note also that for patients who switched from placebo to eteplitsen at \?i?eek 24 (dashed red and black lines), there was no response between Weeks 24 and 48." The results of the original analysis (left) and the blinded reanalysis by three readers (right) are shown graphi- cally below. These show aCtual percent positive fiber counts, without baseline subtraction. Note the marked differences between readings in the left and right plats, particularly at later time points. 2016 American 1 Teurological Association 1 ANA Customer ANA24842 Cadmus An: ANA24842 Ed. Ref. No.: 16-1449 Date: 11-January-17 Stage: Page: 2 ANNALS of Neurology Original reading at Blinded rc-read by 3 palholugisls Nationwide Children?s Hospital ng?kg (m4) 1' BO 5 50 Placebo lo 30 {mil} 2 50 .. so I Placebo to 30 mgikg a; Placebo to 50 ?a Placebo time (weeks) time (weeks) The original results were shown in Table 2 of the article Based on data received by the FDA. the revised and markedly different results from the blinded reread of three as percent positive ?bers; absolute change From baseline, reprinted below: pathologists are shown using the same format: Week 12 Mean Week 24 Mean Week 48 Mean ft fr ,2 Cohov?r gee; LS (in?ame-E All Eteplirsen (8) NA NA 47.3, 3.89 ($0.001) Eteplirsen 30mg/kg (4) ND 22.9, 2.90 (30.002) 51.7. 3.54 (30.001) Eteplirsen 50 mg/kg (4) 0.8, 3.55 (NS) ND 42.9, 672(50008) Placebo/Delayed Eteplirsen 2.92 -4.0, 2.92 37.7, 6.30 (30.009)? Eteplirsen 30 mg/kg (2) ND ?7.48, 1.00 33.6. 5.23 Ereplirsen 50 mg/kg (2) -O.6, 5.16 ND 41.8, 13.30 (M000. 46X) Week 12 I Week 24 Mean Week 48 Mean COMOV Change from Change from Change from BL (SE) BL (SE) BL (SE) All Etep1irsen (8) NA NA 9.8, 2.38 Eteplirsen 30 mgikg (4) 13.7. 2.04 9.6, 4.54 Eteplirsen 50 mg/kg (4) 1.8, 3.57 ND 10, 2.41 Placebo delayed Eteplirsen (4) 1.69 -0.8, 1.96 Ereplirsen 30 mg/kg (2) ND -0.4, 3.4 2.02 Eteplirsen 50 (2) -2.1. 2.05 ND -0.6, 4.34 Volume 00, No. 00 . 44 ID: pachiyappanm Time: 17:54 1 Path: ANA Customer ANA24842 Cadmus Art: ANA24842 Ed. Ref. No.: 16-1449 Date: 11-January-17 Stage: Note that the absolute change from baseline originally reported was approximately 50% for all eteplirsen patients, whereas the percent change reported on independent reanalysis is approximately 10%. Many time points and treatment groups were examined here; given that there was no attempt to control the type I error rate, we did not compute nominal values for these results. Importantly, the independent reanalysis did not con?rm the results reported in the article and is much less sup- portive of the stated conclusions. Moreover, based on our comparison of the immunohisto- chemistry images in Figure 3 of the article and the raw images of all photomicrographs collected from each patient biopsy received in the New Drug Application, the images in the publi- cation appear markedly enhanced and thereby deceptive; cer- tainly, the images in the publication are in no way ?representative,? as claimed by the authors. Finally, we found technical problems with the supportive western blot analyses reported in the article. We have yet to post the details of our review on the Internet; therefore, we can? not provide any speci?c comments with respect to the western blot analyses at this time. We believe that correcting these ?ndings is important because they have been cited in a manner that overestimates the effect of eteplirsen on Furthermore, as human translational science advances and the time lag between discov- ery and human clinical trials shortens, We are seeing more criti? cal evidence for new therapies emanating from academic Month 2017 Page: 3 laboratories. Accordingly, the numerous methodological short- comings should be noted to assist others who may be involved in producing evidence of a quality needed for regulatory submissions. In summary, the muscle ?ber count? ing, as originally performed by the authors, was incorrect. In a blinded reanalysis by three readers, the original counts were shown to be markedly inflated, and these incorrect counts pro- vided the basis for the conclusions of the study. In light of the above, we do not believe that the article in its current form meets the high standards of Annals of Neurology Potential Conflicts of Interest Norhing to report. US Food and Drug Adminis- tration, Silver Spring, MD Reference 1. Mendell JR, Rodino-Klapac LR, Sahenk 2, et al. Eteplirsen for the treatment of Duchenne muscular Ann Neurol 2013;74: 637-647. DOI: 10.1002/ana.24842 AQ2 AQ3 ID: pachiyappartm Time: 17:54 Path: l_ ANA Customer ANA24842 Cadmus Art: ANA24842 Ed. Ref. No.: 16-1449 Datez11-January-17 Stage: Page: 4 Please con?rm or correct title for letter to the editor. AQ2: A disclosure statement reporting no con?icts of interest has been inserted below ?Potential Con?icts of Interest? in the Acknowledgment section; please con?rm as accurate. i AQ3: Please con?rm or correCt author af?liation. AQ4: Please con?rm that given names (red) and surnames/family names (green) have been identi?ed correctly. 01C ID: pachiyappanm Time: 17:54 I Path: From: To: Subject: Date: Attachments: Unger, Ellis "Annals of Neurology" RE: Retraction of correction of paper in Ann Neurol Sunday, December 04, 2016 10:46:00 PM Saper final.doc Dear Dr. Roe,   I’m happy to provide you with a Word version as requested.   Best,   Ellis F. Unger, M.D. Director Office of Drug Evaluation-I Office of New Drugs Center for Drug Evaluation and Research US FDA   From: Annals of Neurology [mailto:aon@bidmc.harvard.edu] Sent: Sunday, December 04, 2016 4:14 PM To: Unger, Ellis Subject: Re: Retraction of correction of paper in Ann Neurol Dear Dr. Unger,   I am submitting your letter regarding Dr. Mendell et al.'s work to Annals of Neurology, but our system needs your letter to be in a .doc file. Can you send along a .doc file version of this letter?   Thank you,   Daniel Roe, PhD Managing Editor Annals of Neurology From: Unger, Ellis Sent: Friday, November 4, 2016 11:35 AM To: Annals of Neurology Cc: Unger, Ellis; Califf, Robert; Borio, Luciana; Woodcock, Janet; Jenkins, John K; Bastings, Eric Subject: Retraction of correction of paper in Ann Neurol   Clifford B. Saper, M.D., Ph.D. Editor-in-Chief Annals of Neurology 330 Brookline Avenue FDACDER00047 Boston, MA 02215   Dear Dr. Saper:   This letter is in reference to the paper “Eteplirsen for the treatment of Duchenne muscular dystrophy,” by Jerry R. Mendell, et. al., Ann Neurol 2013;74:637–647.   The principal conclusion of the study was: “Eteplirsen restored dystrophin in the 30 and 50 mg/kg/wk cohorts, and in subsequently treated, placebo-controlled subjects.”  We believe that the reported findings for the study are based on unreliable data and that the conclusions are, therefore, misleading.  We urge that the paper be corrected or retracted from the Annals of Neurology.  We provide the full text of our concerns in the attached letter.   Thank you in advance for your consideration of this matter.   Sincerely,     Ellis F. Unger, M.D. Director Office of Drug Evaluation-I Office of New Drugs Center for Drug Evaluation and Research US Food and Drug Administration       This message is intended for the use of the person(s) to whom it may be addressed. It may contain information that is privileged, confidential, or otherwise protected from disclosure under applicable law. If you are not the intended recipient, any dissemination, distribution, copying, or use of this information is prohibited. If you have received this message in error, please permanently delete it and immediately notify the sender. Thank you. FDACDER00048 DEPARTMENT OF HEALTH AND HUMAN SERVICES Food and Drug Administration Silver Spring MD 20993 Clifford B. Saper, M.D., Ph.D. Editor-in-Chief Annals of Neurology c/o Managing Editor 330 Brookline Avenue Boston, MA 02215 email: aon@bidmc.harvard.edu Dear Dr. Saper: This letter is in reference to the paper “Eteplirsen for the treatment of Duchenne muscular dystrophy,” by Jerry R. Mendell, Louise R. Rodino-Klapac, Zarife Sahenk, Kandice Roush, Loren Bird, Linda P. Lowes, Lindsay Alfano, Ann Maria Gomez, Sarah Lewis, Janaiah Kota, Vinod Malik, Kim Shontz, Christopher M. Walker, Kevin M. Flanigan, Marco Corridore MD, John R. Kean, Hugh D. Allen, Chris Shilling, Kathleen R. Melia, Peter Sazani, Jay B. Saoud, and Edward M. Kaye, published in the Annals of Neurology (Ann Neurol 2013;74:637–647). The principal conclusion of the study was: “Eteplirsen restored dystrophin in the 30 and 50 mg/kg/wk cohorts, and in subsequently treated, placebo-controlled subjects.” We believe that the reported findings for the 48-week study are based on unreliable data and that the conclusions, based on these erroneous findings, are misleading. We therefore urge that the paper be corrected or retracted from the Annals of Neurology. In the course of FDA’s review of a New Drug Application for eteplirsen for the treatment of Duchenne muscular dystrophy in patients with a confirmed mutation of the DMD gene that is amenable to exon 51 skipping, FDA inspected the Columbus, Ohio facility where the study had been conducted. As noted in documents posted on the internet subsequent to our September 19, 2016, approval, 1 a number of significant methodological problems were found that importantly affect the study results. The Patients and Methods section of the publication states: “To evaluate dystrophin-positive fiber levels, immunohistochemical staining was done on 10 μm frozen sections of 3 separate blocks of biopsy tissue separated by at least 200 μm and evaluated by blinded expert muscle pathologists. For dystrophin localization, sections were stained with MANDYS106 NCL, a mouse monoclonal antibody to amino acids 1749–2248,[15, 16] using standard immunofluorescence methodology.[9, 14] Dystrophin-positive fibers and total fibers were counted, and the percentage of dystrophin-positive fibers was calculated across all samples.” 1 Downloaded 9/29/16 from http://www.accessdata.fda.gov/drugsatfda docs/nda/2016/206488 summary%20review Redacted.pdf FDACDER00049 Note that the first sentence states that the tissues were evaluated by “blinded expert muscle pathologists,” implying more than one such evaluator. The third sentence explains that dystrophin-positive and total muscle fibers were counted, and because this sentence closely follows the first sentence and is in the same paragraph, there is the implication that the fiber counting was also performed by “blinded expert muscle pathologists.” In fact, the counting was performed by a single histotechnician. The FDA inspectional findings pertinent to the fiber counting are reprinted from pp. 12 to 15 of the Office Director Decisional Memo.1 "Although the technician had been blinded to treatment group, access to the treatment code was not protected with the kinds of safeguards and firewalls that one would ordinarily put in place for an adequate and well controlled trial. The immunohistochemistry images were only faintly stained, and had been read by a single technician using an older liquid crystal display (LCD) computer monitor in a windowed room where lighting was not controlled. (The technician had to suspend reading around mid-day, when brighter light began to fill the room and reading became impossible.) These issues are well described in a summary of inspectional findings in Dr. Breder’s clinical review (page 27). There was also concern that the reader, although masked to treatment assignment, was not masked to sequence/time (see below). Importantly, in a trial where all patients eventually received the active drug, knowledge of sequence could lead to the false appearance of a treatment effect, i.e., the appearance of increasing dystrophin expression with time, simply by having a lower threshold for calling fibers “positive” at later time points in the study. Having uncovered numerous technical and operational shortcomings in Columbus, our team worked collaboratively with the applicant to develop improved methods for a reassessment of the stored images. We suggested a re-read of all images by 3 independent masked readers, such that blinding could be assured and inter- and intraobserver variability could be characterized. We also suggested the use of better equipment, specifically, high-quality light-emitting diode (LED) computer monitors, in darkened rooms. The applicant undertook a blinded re-analysis of the images on the server as FDA suggested. Unfortunately, the re-analyses failed to show a significant increase in dystrophin-positive fiber counts in eteplirsen treated patients (Figure 3). Note also that for patients who switched from placebo to eteplirsen at Week 24 (dashed red and black lines), there was no response between Weeks 24 and 48.” The results of the original analysis (left) and the blinded re-analysis by 3 readers (right) are shown graphically below. These show actual percent positive fiber counts, without baseline subtraction. Note the marked differences between readings in the left and right plots, particularly at later time points. FDACDER00050 Original reading at Nationwide Children’s Hospital Blinded re-read by 3 pathologists The original results were shown in Table 2 of the paper as percent positive fibers; absolute change from baseline, reprinted below: Based on data received by FDA, the revised and markedly different results from the blinded reread of 3 pathologists are shown using the same format: All Eteplirsen (8) Week 12 Mean Change from BL (SE) Week 24 Mean Change from BL (SE) Week 48 Mean Change from BL (SE) NA NA 9.8, 2.38 13.7, 2.04 9.6, 4.54 ND 10, 2.41 -1.2, 1.69 -0.8, 1.96 Eteplirsen 30 mg/kg (4) Eteplirsen 50 mg/kg (4) 1.8, 3.57 Placebo - delayed Eteplirsen (4) Eteplirsen 30 mg/kg (2) ND -0.4, 3.4 -1, 2.02 Eteplirsen 50 mg/kg (2) -2.1, 2.05 ND -0.6, 4.34 FDACDER00051 Note that the absolute change from baseline originally reported was approximately 50% for all eteplirsen patients, whereas the percent change reported on independent re-analysis is approximately 10%. Many time points and treatment groups were examined here; given that there was no attempt to control the type-I error rate, we did not compute nominal p-values for these results. Importantly, the independent re-analysis did not confirm the results reported in the paper and is much less supportive of the stated conclusions. Moreover, based on our comparison of the immunohistochemistry images in Figure 3 of the paper and the raw images of all photomicrographs collected from each patient biopsy received in the New Drug Application, the images in the publication appear markedly enhanced and thereby deceptive; certainly, the images in the publication are in no way “representative,” as claimed by the authors. Finally, we found technical problems with the supportive Western blot analyses reported in the paper. We have yet to post the details of our review on the internet; therefore, we cannot provide any specific comments with respect to the Western blot analyses at this time. We believe that correcting these findings is important because they have been cited in a manner that overestimates the effect of eteplirsen on dystrophin. Furthermore, as human translational science advances and the time lag between discovery and human clinical trials shortens, we are seeing more critical evidence for new therapies emanating from academic laboratories. Accordingly, the numerous methodological shortcomings should be noted to assist others who may be involved in producing evidence of a quality needed for regulatory submissions. In summary, the dystrophin-positive muscle fiber counting, as originally performed by the authors, was incorrect. In a blinded re-analysis by 3 readers, the original counts were shown to be markedly inflated, and these incorrect counts provided the basis for the conclusions of the study. In light of the above, we do not believe that the paper in its current form meets the high standards of Annals of Neurology. Sincerely, Ellis F. Unger, M.D. Director Office of Drug Evaluation I Office of New Drugs Center for Drug Evaluation and Research US Food and Drug Administration Robert M. Califf, M.D. Commissioner US Food and Drug Administration FDACDER00052 From: To: Cc: Subject: Date: Unger, Ellis Saper,Clifford Califf, Robert RE: Retraction of correction of paper in Ann Neurol Tuesday, November 29, 2016 2:36:00 PM Dear Dr. Saper,   For all the reasons stated in my email of November 14, 2016 (12:05 AM), I think the best course of action would be for your journal to publish our letter and let the authors respond.   Thanks again for your consideration.   Ellis       From: Saper,Clifford [mailto:csaper@bidmc.harvard.edu] Sent: Monday, November 14, 2016 11:36 AM To: Unger, Ellis Subject: Re: Retraction of correction of paper in Ann Neurol Okay.  I was afraid that might be the case. Well, our policy is that we will publish your letter, but also their response.  In most cases, that will be the end of it.  In other words, this will not amend the original paper (which cannot be changed in any case; it will stay online unless it is totally retracted, and a notation of an erratum would be added to its PubMed listing if one is published).  The only way that we could institute an editorial retraction would be if their reply to you were either false or did not address your points, or it indicated that their main conclusions were wrong and the paper therefore required retraction.  My prediction is that Dr. Mendell will instead provide a vigorous defense, perhaps ceding that your methods were more stringent, but that the effect went in the same direction, and that more work is needed.  That would not provoke a retraction.  If your goal is to place a mark on this paper that indicates that some of the methods were not sufficiently stringent, and to replace those results, then I urge you to accept the Erratum.  If we go the route of a Letter there will be a Reply (and by Annals policy, that will be the end of it: no further letters or replies on those points).  And we will end up with a muddy discussion in which Mendell gets in the last work, and from which readers can draw their own conclusions. Clif Saper   Clifford B. Saper, MD, PhD James Jackson Putnam Professor of Neurology and Neuroscience, Harvard Medical School Chairman, Dept of Neurology, Beth Israel Deaconess Medical Center 330 Brookline Drive, Boston MA 02215 FDACDER00053 Phone: 617-667-2622, FAX: 617-975-5161, email: csaper@bidmc.harvard.edu   Information in this email may be privileged or confidential.  If you should receive this email by mistake, please inform me, and delete it. From: Unger, Ellis Sent: Monday, November 14, 2016 12:05 AM To: Saper,Clifford Cc: Califf, Robert Subject: RE: Retraction of correction of paper in Ann Neurol   Dear Dr Saper,    The response by Dr. Mendell and colleagues is not satisfactory in our opinion.     In essence, their view is that they only need to replace the table with the new data from the reread, without changing the text of the paper or its conclusion.    Their concluding comment states only that FDA “felt” that the data were less supportive of  the conclusion; they do not retract or alter their own overstated conclusion.   “The FDA reviewers felt this reported data was much less supportive of the statement in the paper that ‘eteplirsen is a safe and effective disease-modifying therapy for DMD’.”   In our view, they need to replace not only the table (figure 2a), but also the graph (figure 2b).  They also need to revise the paper’s abstract, results, and conclusions.  We also noted that the photomicrographs are exaggeratory and in no way “representative” as claimed by the authors.  The authors didn’t address that at all.  These photomicrographs should be removed or replaced.   We also note that their proposed text (below) gives the false impression that there was nothing wrong with their original immunohistochemistry reading – the text suggests only that FDA wanted a re-read of the images using new criteria:   “Three blinded pathologists recounted the number of positive fibers using a similar format but criteria were agreed upon before the count and excluded any fibers with membrane staining at the margins of the image and fibers with borderline threshold positivity.”   Given all the attention that is being paid to this, we would prefer that you publish our letter.   We would like to stress that we’re not accusing the authors of deceit, but of making mistakes that simply need to be corrected for the record so as not to mislead others.   Thank you very much for your attention to this matter.   Sincerely,   FDACDER00054 Ellis Unger, MD     From: Saper,Clifford [mailto:csaper@bidmc.harvard.edu] Sent: Wednesday, November 09, 2016 5:57 PM To: Unger, Ellis Cc: Califf, Robert Subject: RE: Retraction of correction of paper in Ann Neurol Dear Drs. Unger and Califf.  Dr. Mendell and colleagues offer the attached draft as an Erratum that they are willing to submit for their paper.  Would this satisfy your concerns?  I would prefer to get this on record in a way that will be attached in PubMed to the original paper, and which is not pejorative in either direction.  If you insist, I will publish your full letter to me instead, but I would have to let Dr. Mendell respond, and that may not produce as satisfactory a response. Let me know your thoughts on this. Clif   Clifford B. Saper, MD, PhD James Jackson Putnam Professor of Neurology and Neuroscience, Harvard Medical School Chairman, Department of Neurology Beth Israel Deaconess Medical Center 330 Brookline Avenue, Boston, MA 02215 USA Phone: 617-667-2622; Fax: 617-975-5161 Email: csaper@bidmc.harvard.edu   Note: This message is confidential and may be legally privileged. It is intended solely for the addressee. If you receive it in error, please delete this message and notify the sender immediately. Any use by an unintended recipient is prohibited.   From: Unger, Ellis [mailto:Ellis.Unger@fda.hhs.gov] Sent: Friday, November 04, 2016 3:01 PM To: Saper,Clifford; Annals of Neurology Cc: Califf, Robert Subject: RE: Retraction of correction of paper in Ann Neurol Dear Dr. Saper,   I think this represents sloppy science – not scientific misconduct.   Thanks,   Ellis Unger, MD   From: Saper,Clifford [mailto:csaper@bidmc.harvard.edu] Sent: Friday, November 04, 2016 2:08 PM To: Annals of Neurology; Unger, Ellis Subject: RE: Retraction of correction of paper in Ann Neurol Dear Dr. Unger, FDACDER00055 Thank you for your email letter concerning the paper by Mendell et al., Annals of Neurology 2013; 74:637. Before I act on this request, I need to know whether you are accusing the authors of scientific misconduct in this paper, or just sloppy science.  The difference is that if you believe that this is sloppy science (and the Editors agree), we will just forward this to the authors, and ask for a response.  They may choose to retract the paper; or perhaps to write an erratum to explain the differences and give the revised data, and how this effects their conclusions;  or they may disagree with you, and feel that no correction is needed (and the Editors may or may not agree with their decision, and if necessary may take the action of retracting the paper ourselves, if we feel that is warranted, based on your evidence and their response).  If you (or the Editors after reviewing your observations in detail) believe that this rises to scientific misconduct (deliberate intent to deceive) then we would handle it differently, and pass this on to the authors’ institution for a full investigation.  In the case of the latter, we would place a note of editorial concern on the paper until it is either withdrawn, or the institution decides that is not required.  If you are agnostic on this issue, the Editors of the journal will make our own decision on which course to take (and if you decide one way or the other, we will still review things internally and see if we agree with you), but knowing your intention would be important in determining the course we take. Clif Saper   Clifford B. Saper, MD, PhD James Jackson Putnam Professor of Neurology and Neuroscience, Harvard Medical School Chairman, Department of Neurology, Beth Israel Deaconess Medical Center 330 Brookline Avenue, Boston, MA 02215 Phone: 617-667-2622; Fax: 617-975-5161; Email: csaper@bidmc.harvard.edu   Note: This message is confidential and may be legally privileged. It is intended solely for the addressee. If you receive it in error, please delete this message and notify the sender immediately. Any use by an unintended recipient is prohibited.   From: Unger, Ellis Sent: Friday, November 4, 2016 11:35 AM To: Annals of Neurology Cc: Unger, Ellis; Califf, Robert; Borio, Luciana; Woodcock, Janet; Jenkins, John K; Bastings, Eric Subject: Retraction of correction of paper in Ann Neurol   Clifford B. Saper, M.D., Ph.D. Editor-in-Chief Annals of Neurology 330 Brookline Avenue Boston, MA 02215   Dear Dr. Saper:   This letter is in reference to the paper “Eteplirsen for the treatment of Duchenne muscular FDACDER00056 dystrophy,” by Jerry R. Mendell, et. al., Ann Neurol 2013;74:637–647.   The principal conclusion of the study was: “Eteplirsen restored dystrophin in the 30 and 50 mg/kg/wk cohorts, and in subsequently treated, placebo-controlled subjects.”  We believe that the reported findings for the study are based on unreliable data and that the conclusions are, therefore, misleading.  We urge that the paper be corrected or retracted from the Annals of Neurology.  We provide the full text of our concerns in the attached letter.   Thank you in advance for your consideration of this matter.   Sincerely,     Ellis F. Unger, M.D. Director Office of Drug Evaluation-I Office of New Drugs Center for Drug Evaluation and Research US Food and Drug Administration       This message is intended for the use of the person(s) to whom it may be addressed. It may contain information that is privileged, confidential, or otherwise protected from disclosure under applicable law. If you are not the intended recipient, any dissemination, distribution, copying, or use of this information is prohibited. If you have received this message in error, please permanently delete it and immediately notify the sender. Thank you. FDACDER00057 From: To: Cc: Subject: Date: Attachments: Unger, Ellis aon@bidmc.harvard.edu Unger, Ellis; Califf, Robert; Borio, Luciana; Woodcock, Janet; Jenkins, John K; Bastings, Eric Retraction of correction of paper in Ann Neurol Friday, November 04, 2016 11:35:46 AM Mendell letter to editor.pdf Clifford B. Saper, M.D., Ph.D. Editor-in-Chief Annals of Neurology 330 Brookline Avenue Boston, MA 02215   Dear Dr. Saper:   This letter is in reference to the paper “Eteplirsen for the treatment of Duchenne muscular dystrophy,” by Jerry R. Mendell, et. al., Ann Neurol 2013;74:637–647.   The principal conclusion of the study was: “Eteplirsen restored dystrophin in the 30 and 50 mg/kg/wk cohorts, and in subsequently treated, placebo-controlled subjects.”  We believe that the reported findings for the study are based on unreliable data and that the conclusions are, therefore, misleading.  We urge that the paper be corrected or retracted from the Annals of Neurology.  We provide the full text of our concerns in the attached letter.   Thank you in advance for your consideration of this matter.   Sincerely,     Ellis F. Unger, M.D. Director Office of Drug Evaluation-I Office of New Drugs Center for Drug Evaluation and Research US Food and Drug Administration     FDACDER00058 DEPARTMENT OF HEALTH AND HUMAN SERVICES Food and Drug Administration Silver Spring MD 20993 Clifford B. Saper, M.D., Ph.D. Editor-in-Chief Annals of Neurology c/o Managing Editor 330 Brookline Avenue Boston, MA 02215 email: aon@bidmc.harvard.edu Dear Dr. Saper: This letter is in reference to the paper “Eteplirsen for the treatment of Duchenne muscular dystrophy,” by Jerry R. Mendell, Louise R. Rodino-Klapac, Zarife Sahenk, Kandice Roush, Loren Bird, Linda P. Lowes, Lindsay Alfano, Ann Maria Gomez, Sarah Lewis, Janaiah Kota, Vinod Malik, Kim Shontz, Christopher M. Walker, Kevin M. Flanigan, Marco Corridore MD, John R. Kean, Hugh D. Allen, Chris Shilling, Kathleen R. Melia, Peter Sazani, Jay B. Saoud, and Edward M. Kaye, published in the Annals of Neurology (Ann Neurol 2013;74:637–647). The principal conclusion of the study was: “Eteplirsen restored dystrophin in the 30 and 50 mg/kg/wk cohorts, and in subsequently treated, placebo-controlled subjects.” We believe that the reported findings for the 48-week study are based on unreliable data and that the conclusions, based on these erroneous findings, are misleading. We therefore urge that the paper be corrected or retracted from the Annals of Neurology. In the course of FDA’s review of a New Drug Application for eteplirsen for the treatment of Duchenne muscular dystrophy in patients with a confirmed mutation of the DMD gene that is amenable to exon 51 skipping, FDA inspected the Columbus, Ohio facility where the study had been conducted. As noted in documents posted on the internet subsequent to our September 19, 2016, approval, 1 a number of significant methodological problems were found that importantly affect the study results. The Patients and Methods section of the publication states: “To evaluate dystrophin-positive fiber levels, immunohistochemical staining was done on 10 μm frozen sections of 3 separate blocks of biopsy tissue separated by at least 200 μm and evaluated by blinded expert muscle pathologists. For dystrophin localization, sections were stained with MANDYS106 NCL, a mouse monoclonal antibody to amino acids 1749–2248,[15, 16] using standard immunofluorescence methodology.[9, 14] Dystrophin-positive fibers and total fibers were counted, and the percentage of dystrophin-positive fibers was calculated across all samples.” 1 Downloaded 9/29/16 from http://www.accessdata.fda.gov/drugsatfda docs/nda/2016/206488 summary%20review Redacted.pdf FDACDER00059 Note that the first sentence states that the tissues were evaluated by “blinded expert muscle pathologists,” implying more than one such evaluator. The third sentence explains that dystrophin-positive and total muscle fibers were counted, and because this sentence closely follows the first sentence and is in the same paragraph, there is the implication that the fiber counting was also performed by “blinded expert muscle pathologists.” In fact, the counting was performed by a single histotechnician. The FDA inspectional findings pertinent to the fiber counting are reprinted from pp. 12 to 15 of the Office Director Decisional Memo.1 "Although the technician had been blinded to treatment group, access to the treatment code was not protected with the kinds of safeguards and firewalls that one would ordinarily put in place for an adequate and well controlled trial. The immunohistochemistry images were only faintly stained, and had been read by a single technician using an older liquid crystal display (LCD) computer monitor in a windowed room where lighting was not controlled. (The technician had to suspend reading around mid-day, when brighter light began to fill the room and reading became impossible.) These issues are well described in a summary of inspectional findings in Dr. Breder’s clinical review (page 27). There was also concern that the reader, although masked to treatment assignment, was not masked to sequence/time (see below). Importantly, in a trial where all patients eventually received the active drug, knowledge of sequence could lead to the false appearance of a treatment effect, i.e., the appearance of increasing dystrophin expression with time, simply by having a lower threshold for calling fibers “positive” at later time points in the study. Having uncovered numerous technical and operational shortcomings in Columbus, our team worked collaboratively with the applicant to develop improved methods for a reassessment of the stored images. We suggested a re-read of all images by 3 independent masked readers, such that blinding could be assured and inter- and intraobserver variability could be characterized. We also suggested the use of better equipment, specifically, high-quality light-emitting diode (LED) computer monitors, in darkened rooms. The applicant undertook a blinded re-analysis of the images on the server as FDA suggested. Unfortunately, the re-analyses failed to show a significant increase in dystrophin-positive fiber counts in eteplirsen treated patients (Figure 3). Note also that for patients who switched from placebo to eteplirsen at Week 24 (dashed red and black lines), there was no response between Weeks 24 and 48.” The results of the original analysis (left) and the blinded re-analysis by 3 readers (right) are shown graphically below. These show actual percent positive fiber counts, without baseline subtraction. Note the marked differences between readings in the left and right plots, particularly at later time points. FDACDER00060 Original reading at Nationwide Children’s Hospital Blinded re-read by 3 pathologists The original results were shown in Table 2 of the paper as percent positive fibers; absolute change from baseline, reprinted below: Based on data received by FDA, the revised and markedly different results from the blinded reread of 3 pathologists are shown using the same format: FDACDER00061 Note that the absolute change from baseline originally reported was approximately 50% for all eteplirsen patients, whereas the percent change reported on independent re-analysis is approximately 10%. Many time points and treatment groups were examined here; given that there was no attempt to control the type-I error rate, we did not compute nominal p-values for these results. Importantly, the independent re-analysis did not confirm the results reported in the paper and is much less supportive of the stated conclusions. Moreover, based on our comparison of the immunohistochemistry images in Figure 3 of the paper and the raw images of all photomicrographs collected from each patient biopsy received in the New Drug Application, the images in the publication appear markedly enhanced and thereby deceptive; certainly, the images in the publication are in no way “representative,” as claimed by the authors. Finally, we found technical problems with the supportive Western blot analyses reported in the paper. We have yet to post the details of our review on the internet; therefore, we cannot provide any specific comments with respect to the Western blot analyses at this time. We believe that correcting these findings is important because they have been cited in a manner that overestimates the effect of eteplirsen on dystrophin. Furthermore, as human translational science advances and the time lag between discovery and human clinical trials shortens, we are seeing more critical evidence for new therapies emanating from academic laboratories. Accordingly, the numerous methodological shortcomings should be noted to assist others who may be involved in producing evidence of a quality needed for regulatory submissions. In summary, the dystrophin-positive muscle fiber counting, as originally performed by the authors, was incorrect. In a blinded re-analysis by 3 readers, the original counts were shown to be markedly inflated, and these incorrect counts provided the basis for the conclusions of the study. In light of the above, we do not believe that the paper in its current form meets the high standards of Annals of Neurology. Sincerely, Ellis F. Unger, M.D. Director Office of Drug Evaluation I Office of New Drugs Center for Drug Evaluation and Research US Food and Drug Administration Robert M. Califf, M.D. Commissioner US Food and Drug Administration FDACDER00062