6 Everett Street, Suite 4119
Cambridge, MA 02138
617.496.2058 (tel.)
617.384.7633 (fax)

August 7, 2018
By Electronic Submission to www.regulations.gov
Acting Administrator Andrew Wheeler
U.S. Environmental Protection Agency
1200 Pennsylvania Avenue, NW
Washington, D.C. 20460
Docket ID No. EPA-HQ-OA-2018-0259
Re:

COMMENTS ON PROPOSED RULE, STRENGTHENING TRANSPARENCY IN
REGULATORY SCIENCE, 83 FED. REG. 18,768 (Apr. 30, 2018)

Dear Acting Administrator Wheeler:
The Emmett Environmental Law & Policy Clinic at Harvard Law School submits this letter on
behalf of a distinguished group of experts committed to the advancement of research to improve
the health and safety of Americans and people around the world. The signatories include the
President of Harvard University, the Presidents and a number of Department Chairs and Chiefs
of four of the world’s foremost research and teaching hospitals (Beth Israel Deaconess Medical
Center, Brigham and Women’s Hospital, Massachusetts Eye and Ear, and Massachusetts General
Hospital), the Deans of Harvard’s T.H. Chan School of Public Health and Harvard Medical
School, preeminent faculty at the Harvard T.H. Chan School of Public Health, the Harvard
Medical School, and the Harvard School of Engineering and Applied Sciences, and numerous
esteemed research and clinical doctors affiliated with Harvard and its research hospitals. Work
done by the signatories and/or their institutions addresses a broad spectrum of health impacts on
infants, children, and adults from exposures to chemicals and activities that are regulated by the
U.S. Environmental Protection Agency (“EPA”) under various statutes including the Safe
Drinking Water Act, the Toxic Substances Control Act, the Comprehensive Environmental
Response, Compensation, and Liability Act, the Resource Conservation and Recovery Act, the
Clean Water Act, and the Clean Air Act, collectively referred to herein as “the Statutes.”
Specifically, signatories of this letter have conducted research to determine whether and how
exposures to chemical substances such as lead and mercury in food, water, soil, and air affects
the development of fetuses, infant mortality, children’s development, and children’s educational
performance. They have also studied the health effects of indoor and outdoor chemical
exposures on adult health and safety, including worker productivity and well-being.

1

 Comments re: Docket ID No. EPA-HQ-OA-2018-0259
August 7, 2018
Page 2 of 26

Some of the signatories’ research is used to develop vaccines and cures for cancer, improve the
medical care of infants, children and adults, improve public and private building design, and plan
responses to emergencies. The results are also used to demonstrate the benefits of proposed
regulatory actions in accordance with statutory and regulatory requirements. 1
Their research is routinely relied upon by international, federal, and state agencies—including
EPA—when they set standards and establish rules and best practices for the protection of human
health, safety, and the environment. As explained below, the proposed rule would—for no
rational reason—prevent EPA from relying on much of the research that the signatories, their
institutions, and other public health and environmental exposure researchers have conducted and
continue to conduct. The rule will cripple EPA’s ability to implement the aforementioned
Statutes and will jeopardize the health and safety of infants, children, and adults in the United
States and beyond. 2
Without the ability to protect and respect patient/human subject privacy and confidentiality,
signatories and other researchers would not be able to conduct the studies that are pivotal to their
work and to EPA’s ability to fulfill its statutory duty to protect public health. The proposed rule
ignores a host of existing methods and best practices already established—and adhered to—by
the research community to ensure the transparency, reproducibility, replicability, objectivity, and
validity of studies, analyses, models, and reports. 3 The proposed rule thus does not serve its
stated purpose to ensure that regulatory decisions are based on “valid” science. 4
1

Signatories’ research—which analyzes the human health and environmental impacts of the presence of chemicals
in air, soil, drinking water, food, and consumer products—is relevant to EPA’s required determinations under the
Statutes that its regulations provide societal benefits by reducing harm to human health and the environment. Such
research is also critically important to identifying the benefits of EPA regulations when the agency is required by the
Statutes or Executive Order to conduct a formal cost-benefit analysis. See Exec. Order No. 13783, 82 Fed. Reg.
16,093, §1(e) (Mar. 31, 2017) (“It is also the policy of the United States that necessary and appropriate
environmental regulations comply with the law, are of greater benefit than cost, when permissible, achieve
environmental improvements for the American people, and are developed through transparent processes that employ
the best available peer-reviewed science and economics.”).
2

David Cutler & Francesca Dominici, A Breath of Bad Air: Cost of the Trump Environmental Agenda May Lead to
80 000 Extra Deaths per Decade, JAMA NETWORK (June 12, 2018),
https://jamanetwork.com/journals/jama/fullarticle/2684596 (copy attached for inclusion in the administrative record,
Attachment 1).

3

See Section IV, below, for a discussion of best practices. EPA already has detailed policy and procedural guidance
for ensuring and maximizing the quality of information the agency disseminates. See EPA, Guidelines for Ensuring
and Maximizing the Quality, Objectivity, Utility and Integrity of Information Disseminated by the Environmental
Protection Agency (Oct. 2002), https://www.epa.gov/sites/production/files/2017-03/documents/epa-info-qualityguidelines.pdf. Note further that the proposed rule incorrectly uses the terms “reproducibility” and “replicability” as
though they are interchangeable. In fact, they have different meanings. Typically, in the scientific community,
“reproducibility refers to the ability of a researcher to duplicate the results of a prior study using the same materials
as were used by the original investigator.” Steven N. Goodman, et al., What does research reproducibility mean?, 8
SCIENCE TRANSLATIONAL MEDICINE 341ps12 (2016). By contrast, “replicability” refers to the ability of a researcher
to duplicate the results of a prior study following the same procedures but collecting new data. Id.
4

See 83 Fed. Reg. 18,768, 18,773 (Apr. 30, 2018) (stated purpose “to ensure that the regulatory science underlying
its actions is publicly available in a manner sufficient for independent validation”); see also id. at 18,770 (“It is the
charge of regulators to ensure that key findings [of science that informs regulatory actions] are valid and credible.”).

2

 Comments re: Docket ID No. EPA-HQ-OA-2018-0259
August 7, 2018
Page 3 of 26

Signatories teach graduate and undergraduate students and doctors-in-training about best
practices in the conduct of public health, medical, and scientific research. They publish their
research results in the most reliable, highest-quality, peer-reviewed medical and scientific
journals, including Lancet, Nature, Science, New England Journal of Medicine, Journal of the
American Medical Association, Cell, and Environmental Health Perspectives. They conduct
peer reviews of the work of other researchers. The approach advocated in the proposed rule is
inconsistent with professional best practices in their respective disciplines for conducting,
reviewing, and confirming the results/findings of studies, especially those based on confidential
personal health data of study participants. As will be shown below, the proposed rule will wreak
havoc on public health, medical, and scientific research and undermine the protection of public
health and safety.
Accordingly, the signatories strenuously object to the proposed rule and urge EPA to withdraw
it.
I.

THE PROPOSED RULE WOULD PREVENT EPA FROM RELYING ON THE
BEST AVAILABLE INFORMATION AND SCIENCE

In the proposed rule, EPA acknowledges that it must use the “best available science” in all of its
regulatory actions. 5 The signatories agree that is the correct starting point for EPA. They
disagree, however, with EPA’s new position in this proposed rule that science is not the “best”
unless the associated raw data are released to the public. 6 As an initial matter, releasing raw data
will not improve the quality of the resulting report/study/analysis, and therefore will do nothing
to render any individual study “better.” EPA itself affirmed this point as recently as 2016. 7
Moreover, while it might be helpful in some situations to make raw data publicly available, it is
neither practical nor desirable to impose this requirement as a one-size-fits-all approach.
Instead, there are a variety of other best practices that already exist to test and ensure the rigor,
quality, and validity of research. These include the peer review process, which evaluates
whether the work is based on the best available scientific understanding, and scientists’ detailed
description of their research methods, code and non-confidential data in their published articles.
That detail allows other researchers not only to challenge the study results, but also to reproduce
or validate them using the original data, and/or replicate them via other studies using different
data sets. The scientific community considers results valid if they are or can be replicated by
other researchers conducting studies using new data, but the same method. 8
5

83 Fed. Reg. at 18,769 (citing Exec. Order No. 13563, 76 Fed. Reg. 8,321 (Jan. 21, 2011)).

6

See 83 Fed. Reg. at 18,772 (rule would require that “dose response data and models underlying pivotal regulatory
science are publicly available in a manner sufficient for independent validation.”).

7

EPA, Plan to Increase Access to Results of EPA-Funded Scientific Research, at 4-5 (2016),
https://www.epa.gov/sites/production/files/2016-12/documents/epascientificresearchtransperancyplan.pdf (“Whether
research data are fully available to the public or available to researchers through other means does not affect the
validity of the scientific conclusions from peer-reviewed research publications.”).
8

See, e.g., Memorandum from Alison Cullen, Chair, SAB Work Group on EPA Planned Actions for SAB
Consideration of the Underlying Science to Members of the Chartered SAB and SAB Liaisons, Preparations for
Chartered Science Advisory Board (SAB) Discussions of Proposed Rule: Strengthening Transparency in Regulatory

3

 Comments re: Docket ID No. EPA-HQ-OA-2018-0259
August 7, 2018
Page 4 of 26

Contrary to EPA’s stated goal of improving the basis for its regulatory decisions, requiring the
public availability of all raw data will instead undermine EPA’s ability to make reasonable
decisions. This requirement will effectively prohibit EPA from considering studies that by
design are based on data that cannot be made publicly available due to laws and contracts
designed to protect patient and human subject privacy and ensure willingness of people to
participate in research by sharing their private information with researchers. The proposed rule
precludes consideration of studies based on confidential data, even when those results have been
confirmed by other studies. 9 Hence, the proposal would in many instances prohibit EPA from
relying on the best available science relevant to many of the regulatory issues that the agency
faces.
Moreover, this proposed requirement contravenes five decades of EPA practice. EPA has
repeatedly affirmed that its mission requires it to rely on the best available scientific evidence,
without ever asserting that it should exclude from consideration studies for which the underlying
data were not publicly available. For example, in its 1997 strategic plan, EPA declared one of its
seven overall purposes was to ensure that “efforts to reduce environmental risk are based on the
best available scientific information.” 10 In 2002, EPA issued Information Quality Guidelines in
which it took the position that the standard set forth in the Safe Drinking Water Act — “the best
available, peer-reviewed science” 11 — should apply to all of the agency’s risk assessments. 12

Science RIN (2080-AA14) 4 (May 12, 2018), https://perma.cc/MM3J-CHEA [hereinafter “SAB Memo”]; Bernard
Goldstein, Op-Ed., This is Why EPA’s “Secret Science” Proposal Alarms Public Health Experts, THE
CONVERSATION (May 18, 2018), https://theconversation.com/why-the-epas-secret-science-proposal-alarms-publichealth-experts-96000.
9

One example is the Six Cities Study, Douglas W. Dockery, et al., An Association between Air Pollution and
Mortality in Six U.S. Cities, 329 NEW ENGLAND J. MED. 1753 (1993), whose results were subsequently confirmed
by independent reanalysis, Health Effects Institute, Reanalysis of the Harvard Six Cities Study and the American
Cancer Society Study of Particulate Air Pollution and Mortality (2000),
https://www.healtheffects.org/system/files/HEI-Reanalysis-2000.pdf. Indeed, both the Six Cities Study and the
American Cancer Study of Particulate Air Pollution and Mortality have each been reproduced and replicated. The
findings are consistent with the original studies. See, e.g., Qian Di, Francesca Dominici, Joel D. Schwartz, et al., Air
Pollution and Mortality in the Medicare Population, 376 NEW ENGLAND J. MED. 2513-2522 (2017) (copy attached
for inclusion in the administrative record, Attachment 2).
10

EPA, EPA/190-R-97-002, EPA Strategic Plan, at 16 (1997), https://nepis.epa.gov/Exe/ZyPDF.cgi?Dockey=40000
9JX.PDF. Earlier, in a March 1992 report titled Safeguarding the Future: Credible Science, Credible Decisions, an
independent committee convened by EPA declared that “science is one of the soundest investments the nation can
make for the future. Strong science provides the foundation for credible environmental decision making. With a
better understanding of environmental risks to people and ecosystems, EPA can target the hazards that pose the
greatest risks, anticipate environmental problems before they reach a critical level, and develop strategies that use
the nation’s, and the world’s, environmental protection dollars wisely.” EPA, Safeguarding the Future: Credible
Science, Credible Decisions, at 15 (Mar. 1992),
https://nepis.epa.gov/Exe/ZyPDF.cgi/30001ZWJ.PDF?Dockey=30001ZWJ.PDF.

11

42 U.S.C. § 300g-1(b)(3)(A)(i).

12

EPA, Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information
Disseminated by the Environmental Protection Agency, at 21-23 (2005),
https://www.epa.gov/sites/production/files/2017-03/documents/epa-info-quality-guidelines.pdf.

4

 Comments re: Docket ID No. EPA-HQ-OA-2018-0259
August 7, 2018
Page 5 of 26

EPA’s historic position is consistent with the Statutes. For example, one of EPA’s core duties
under the Clean Air Act is to set and periodically review the National Ambient Air Quality
Standards (“NAAQS”) for six common air pollutants. In carrying out this responsibility,
Congress commanded EPA to use “the latest scientific knowledge useful in indicating the kind
and extent of all identifiable effects [of air pollution] on public health or welfare.” 13 Similarly,
the Safe Drinking Water Act commands EPA in general to use “the best available, peer-reviewed
science” and when deciding whether to regulate a particular contaminant to consider the “best
available public health information.” 14 The Toxic Substances Control Act requires that
regulation of chemical substances be “consistent with the best available science” and that EPA
make decisions “based on the weight of the scientific evidence.” 15 The water quality criteria that
EPA develops under the Clean Water Act must “accurately reflect[] the latest scientific
knowledge” on a variety of factors. 16
Furthermore, because EPA is required under the Statutes to assess the public health benefits of
its regulations, it must take into account all relevant science and cannot arbitrarily exclude
certain studies demonstrating those benefits. Under the Clean Air Act, EPA must set the
NAAQS at a level “requisite to protect the public health.” 17 Under the Safe Drinking Water Act,
EPA must determine whether a contaminant “may have an adverse effect on the health of
persons” before deciding to regulate it. 18
Many of the fundamental public health studies on which EPA has based key rules and standards
under the Statutes are studies for which the raw data were not or could not have been released.
Attachment 3 to this letter contains a partial list of studies that likely contain confidential data;
these are all studies on which EPA has relied and cited as the basis for its actions under some of
the Statutes. Until now, release of the underlying raw data was not an EPA criterion for
determining the “best available” reports, studies, analyses, or models. Indeed, none of the
Statutes invoked by EPA as support for the proposed rule limits EPA in this fashion; none of the
Statutes requires EPA to make raw data publicly available. 19

13

42 U.S.C. § 7408(a)(2).

14

42 U.S.C. §§ 300g-1(b)(3)(A)(i), 300g-1(b)(1)(B)(ii)(II).

15

15 U.S.C. § 2625(h), (i).

16

33 U.S.C. § 1314(a)(1).

17

42 U.S.C. § 7409(b).

18

42 U.S.C. § 300g-1(b)(1)(A)(i).

19

When litigants in the past argued that EPA could not rely on studies for which the raw data had not been publicly
available, the D.C. Circuit soundly rejected their argument. As the court explained in one case:
Claiming neither that they were unable to obtain the studies, nor that the studies were improperly
published or peer reviewed, Petitioners instead urge us to impose a general requirement that EPA
obtain and publicize the data underlying published studies on which the Agency relies. The Clean
Air Act imposes no such obligation. . . . More generally, we agree with EPA that requiring
agencies to obtain and publicize the data underlying all studies on which they rely “would be
impractical and unnecessary.”
[…]

5

 Comments re: Docket ID No. EPA-HQ-OA-2018-0259
August 7, 2018
Page 6 of 26

EPA’s proposed new approach, which conflicts with the agency’s obligations and curtails its
authority, is irrational at best and detrimental to public health and safety at worst.
II.

THE PROPOSED RULE WOULD EXCLUDE CRITICAL STUDIES FROM
CONSIDERATION IN FUTURE EPA RULEMAKING

There are at least two categories of critically-important, health-based studies for which it will be
impractical or illegal to make the underlying data publicly available. Within each category are
studies that have already formed the basis for decades of EPA regulatory actions producing
enormous public health and safety benefits. The proposal would require that EPA stop relying
on these studies and prohibit automatic consideration of, or reliance on, others like them in the
future for no other reason than that the raw data cannot be released to the public. 20 This result
would be extremely harmful to human health, safety, and the environment.
A.

THE PROPOSAL WOULD PREVENT EPA FROM RELYING ON
STUDIES BASED ON CONFIDENTIAL HUMAN HEALTH DATA

For many studies, disclosure of the raw data would violate researchers’ statutory or contractual
duties to protect patient or human research participant confidentiality. Many types of crucial
health impact studies cannot be conducted without human participants. For any research carried
out by healthcare providers that involves the handling of individually identifiable health
information, the Health Insurance Portability and Accountability Act of 1996 (“HIPAA”)

As EPA persuasively stated in denying Petitioners’ original request for information:
If EPA and other governmental agencies could not rely on published studies without
conducting an independent analysis of the enormous volume of raw data underlying
them, then much plainly relevant scientific information would become unavailable to
EPA for use in setting standards to protect public health and the environment. . . . Such
data are often the property of scientific investigators and are often not readily available
because of ... proprietary interests ... or because of [confidentiality] arrangements [with
study participants].
Am. Trucking Associations, Inc. v. E.P.A., 283 F.3d 355, 372 (D.C. Cir. 2002) (quoting Particulate Matter NAAQS,
62 Fed. Reg. 38,652, 38,689 (July 18, 1997)). The court reiterated this holding six years later in a challenge to the
2008 lead NAAQS. Coal. of Battery Recyclers Ass’n v. E.P.A., 604 F.3d 613, 622 (D.C. Cir. 2010). In that case,
the litigants had sought access to the raw data underlying Bruce P. Lanphear, et al., Low-Level Environmental Lead
Exposure and Children’s Intellectual Function: An International Pooled Analysis, 113 ENVTL. HEALTH PERSP. 894
(2005).
20
The proposal allows EPA to decide to consider such studies on a case-by-case basis. See 83 Fed. Reg. at 18,772.
The factors EPA identifies for providing individual exemptions—that such disclosure cannot be done “in a fashion
that is consistent with law, protects privacy, confidentiality, confidential business information, and is sensitive to
national and homeland security”—merely reiterates the main reasons that data are not currently made publicly
available. Id. at 18,773. If EPA always allows data to be withheld for those reasons, the rule is meaningless and has
no effect. On the other hand, if EPA instead picks and chooses when to allow data to be withheld for those reasons,
it will be doing so based on no meaningful standards. Cf. Pearson v. Shalala, 164 F.3d 650, 660 (D.C. Cir. 1999)
(“It simply will not do for a government agency to declare—without explanation—that a proposed course of private
action is not approved. To refuse to define the criteria it is applying is equivalent to simply saying no without
explanation.”).

6

 Comments re: Docket ID No. EPA-HQ-OA-2018-0259
August 7, 2018
Page 7 of 26

Privacy Rule imposes strict confidentiality requirements. 21 Federally-funded research involving
human subjects is governed by the Federal Policy for the Protection of Human Subjects, also
known as the Common Rule. 22 The Common Rule requires that researchers obtain Institutional
Review Board (“IRB”) approval and informed consent of research subjects, during which
process the researcher will typically need to make promises regarding confidentiality. 23 Most
institutions have committed to comply with the Common Rule for all of their research, 24 even
when it is not federally-funded. 25
EPA’s suggestion in the proposed rule that “simple data masking, coding, and de-identification,”
83 Fed. Reg. at 18,771, will be able to overcome these confidentiality concerns is incorrect. As
explained by the EPA’s own Science Advisory Board (“SAB”), “[i]n some cases, the data cannot
be released simply by redacting portions of it. For example, data may have been collected with
an assurance to the participating individuals that their data would be kept confidential.” 26
Researchers cannot violate those promises after the fact, particularly if they want to be able to
continue to find participants for their studies. In addition, “[i]n the case of clinical trials, there
are studies in which removal of all identifying data negates its scientific value.” 27
The understanding of what counts as identifying data is continually expanding: true deidentification of the data may not be possible for some studies, such as those in which the
participants come from a small geographical area and/or a specific profession. One study found
that the researchers could re-identify approximately one-quarter of the records in a subset of a
21

45 C.F.R. Part 160 and Subparts A and E of Part 164.

22

45 C.F.R. 46 subpart A is the U.S. Department of Health and Human Services (“HHS”) citation for the Common
Rule. A total of 18 federal agencies have adopted it; each agency has its own separate entry in the Code of Federal
Regulations. This federal rule governs ethical constraints that federally funded studies must follow, including
academic research, responding to earlier concerns of ethical lapses in medical research. See, e.g., Jerry Menikoff,
Could Tuskegee Happen Today?, 1 ST. LOUIS U. J. HEALTH L. & POL’Y 311, 312-16 (2008) (describing the
Congressional response to public outcry when the details of the Tuskegee experiment were brought to light). The
thrust of the Common Rule is to address such matters of research ethics as informed consent, informational risk, and
institutional oversight when research involves human subjects. See 82 Fed. Reg. 7,149-7,274.
23

For example, under its “Basic elements of informed consent” provisions, the Common Rule provides that “in
seeking informed consent the following information shall be provided to each subject: . . . A statement describing
the extent, if any, to which confidentiality of records identifying the subject will be maintained.” 45 C.F.R.
§ 46.116(b)(5). The Common Rule also requires that the IRB ensure that the researchers make “adequate provisions
to protect the privacy of subjects and to maintain the confidentiality of data.” 45 C.F.R. § 46.111(a)(7).
24

See Federalwide Assurance (FWA) for the Protection of Human Subjects, HHS,
https://www.hhs.gov/ohrp/register-irbs-and-obtain-fwas/fwas/fwa-protection-of-human-subjecct/index.html
(describing Common Rule policy for institutions performing government-funded human subject research) (last
visited August 3, 2018).
25

Harvard University, for example, has established policies for all university research that go beyond the
requirements of the Common Rule. Statement of Policies and Procedures Governing the Use of Human Subjects in
Research at Harvard University, HARVARD UNIVERSITY, https://provost.harvard.edu/use-human-subjects-research
(last visited August 3, 2018).
26

SAB Memo, supra note 8, at 4.

27

Lynn R. Goldman & Ellen K. Silbergeld, Assuring Access to Data for Chemical Evaluations, 121 ENVTL. HEALTH
PERSPECTIVES 149, 150 (2013).

7

 Comments re: Docket ID No. EPA-HQ-OA-2018-0259
August 7, 2018
Page 8 of 26

HIPAA-compliant environmental health data set. 28 For some studies, it may not be possible to
de-identify the data set while still protecting patient or research subject confidentiality.
The proposed rule would prohibit the continued and future use of these studies by EPA thereby
obstructing EPA’s statutory duty to consider the “best,” “reasonably” available information in its
decision-making processes. The resulting information vacuum would occur for no other reason
than that the underlying human subject data is private and cannot be publicly disseminated. 29
The proposed rule would also impede EPA’s ability to address new and emerging public health
risks in future rulemakings. For example, former Administrator Pruitt announced on May 22,
2018, that EPA will begin to develop maximum contaminant levels under the Safe Drinking
Water Act for two fluorochemicals, perfluorooctanoic acid (“PFOA”) and perfluorooctane
sulfonate (“PFOS”). 30 EPA also plans to designate PFOA and PFOS as hazardous chemicals,
potentially under the Comprehensive Environmental Response, Compensation, and Liability
Act. 31 If finalized, however, the proposed rule would prevent these EPA actions. 32
When EPA issued health advisories for these two chemicals in 2016, the Health Effects Support
Documents relied extensively on epidemiological studies generated by the C8 Health Project. 33
A key component of the evidence for the harmfulness of these chemicals consists of
epidemiological studies based on data that are not publicly available. Researchers published
more than three dozen papers based on these data, identifying probable links between PFOA
28

Latanya Sweeney, et al., Re-identification Risks in HIPAA Safe Harbor Data: A Study of Data from One
Environmental Health Study, TECH. SCI., 2017082801 (Aug. 28, 2017), https://techscience.org/a/2017082801.
29

Note that some of the Statutes require EPA to use the “best” available information and others have a lower
standard. For example, the Toxic Substances Control Act compels EPA to take “reasonably” available information
into account. 15 U.S.C. § 2625(k).
30
Amena H. Saiyid, Pruitt Plans to Declare Two Fluorochemicals Hazardous, BLOOMBERG BNA (May 22, 2018),
https://news.bloombergenvironment.com/environment-and-energy/pruitt-plans-to-declare-two-fluorochemicalshazardous.
31
Press Release, EPA, Administrator Pruitt Kicks Off National Leadership Summit on PFAS (May 22, 2018),
https://www.epa.gov/newsreleases/administrator-pruitt-kicks-national-leadership-summit-pfas.
32

Epidemiological studies, which were essential to discovering the immunotoxicity of perfluorinated alkylate
substances, including PFOA and PFOS, were based on confidential human health data. See Philippe Grandjean,
Delayed discovery, dissemination, and decisions on intervention in environmental health: a case study on
immunotoxicity of perfluorinated alkylate substances, 17:62 ENVTL. HEALTH 1 (2018) (copy attached for inclusion
in the administrative record, Attachment 4).
33

EPA, EPA 822-R-16-003, Health Effects Support Document for Perfluorooctanoic Acid (PFOA), at 3-1 to 3-60
(May 2016), https://www.epa.gov/sites/production/files/2016-05/documents/pfoa_hesd_final-plain.pdf; EPA, EPA
822-R-16-002, Health Effects Support Document for Perfluorooctane Sulfonate (PFOS), at 3-1 to 3-49 (May 2016),
https://www.epa.gov/sites/production/files/2016-05/documents/pfos_hesd_final_508.pdf. The C8 Health Project
was funded through the settlement agreement in a lawsuit brought over drinking water contaminated by PFOA from
the DuPont Washington Works facility near Parkersburg, West Virginia. The study involved close to 70,000
participants, for each of whom “demographic data, medical diagnoses (both self-report and medical records review),
clinical laboratory testing, and determination of serum concentrations of 10 perfluorocarbons (PFCs)” were
collected. Stephanie J. Frisbee et al., The C8 Health Project: Design, Methods, and Participants, 117 ENVTL.
HEALTH PERSP. 1873, 1876 (2009) (“To protect participant privacy, the presiding judge subsequently sealed the data
set.”).

8

 Comments re: Docket ID No. EPA-HQ-OA-2018-0259
August 7, 2018
Page 9 of 26

(also known as C8) exposure and “diagnosed high cholesterol, ulcerative colitis, thyroid disease,
testicular cancer, kidney cancer, and pregnancy-induced hypertension.” 34
This situation underlines the arbitrariness and irrationality of the proposed rule. On the one
hand, EPA is proposing to take regulatory action to protect the American people from emerging
health threats. On the other—through the proposed rule—it is simultaneously undermining its
own ability to follow through on those proposals.
B.

THE PROPOSAL WOULD PREVENT EPA FROM RELYING ON
STUDIES CONDUCTED MANY YEARS AGO FOR WHICH DATA ARE
NO LONGER AVAILABLE

Many key EPA regulatory decisions in effect today were based on studies conducted decades
ago. Due to the passage of time, the raw data from these studies may no longer be available.
Records may have been lost; researchers may have retired or passed away. Or, the data may
have been stored in electronic media such as tapes that are no longer compatible with existing
systems or otherwise difficult to access. 35 As noted by John Ioannidis, who is a strong advocate
of data transparency, 36 “we should recognize that most of the raw data from past studies are not
publicly available. . . . If the proposed rule is approved, science will be practically eliminated
from all decision-making processes. Regulation would then depend uniquely on opinion and
whim.” 37
C.

STUDIES THAT EPA WILL BE PROHIBITED FROM CONSIDERING
UNDER THE PROPOSAL HAVE SERVED AS THE BASIS FOR
MULTIPLE RULEMAKINGS BY EPA AND OTHER AGENCIES

Studies that would be excluded from EPA consideration under the proposal form the basis for
multiple regulatory actions that EPA and other agencies have taken over the course of many
years. Consider, for example, early studies on the neurological effects of low-dose exposure to
lead such as Herbert Needleman’s 1979 paper finding a negative relationship between the level
of lead in children’s teeth and IQ scores. 38 EPA relied on this study in its 1986 Air Quality

34

The Science Panel Website, C8 SCIENCE PANEL, http://www.c8sciencepanel.org/index.html (last updated Jan. 4,
2017). Even the scientists selected to lead the research were provided with access only to de-identified data from the
participants, except in the case of some participants who consented to provide additional data for follow-up studies.
35

Goldman & Silbergeld, supra note 27, at 150.

36

Ioannidis was one of the authors of Marcus R. Munafò et al., A Manifesto for Reproducible
Science, 1 NATURE HUMAN BEHAVIOUR 1 (2017), DOI: 10.1038/s41562-0160021, http://www.nature.com/articles/s41562-016-0021.pdf.
37

John P.A. Ioannidis, All Science Should Inform Policy and Regulation, 15(5) PLOS MEDICINE 1, 1-2 (May 3,
2018), https://doi.org/10.1371/journal.pmed.1002576.
38

Herbert L. Needleman, et al., Deficits in Psychologic and Classroom Performance of Children with Elevated
Dentine Lead Levels, 300 NEW ENGLAND J. MEDICINE 689 (1979).

9

 Comments re: Docket ID No. EPA-HQ-OA-2018-0259
August 7, 2018
Page 10 of 26

Criteria document for lead. 39 EPA’s Lead and Copper Rule, which established the federal
regulations for lead under the Safe Drinking Water Act, in turn relied on that Air Quality Criteria
document to identify blood lead levels of concern. 40 EPA relied on both the 1986 Air Quality
Criteria and on Needleman’s research directly in establishing standards for lead-based paint
hazards under the Toxic Substances Control Act. 41 Needleman’s work, and subsequent studies
building upon it, also supported EPA’s decision to revise the NAAQS for lead in 2008. 42 The
D.C. Circuit specifically ruled that the underlying data from one of the studies on which EPA
relied in this rulemaking did not need to be publicly available for EPA to rely on the study. 43
After 40 years, and with the principal investigator no longer alive, it is not clear that the raw data
from the Needleman study is available. Even if the data were, they could not be made publicly
available without invading the privacy of the study participants. Importantly, it would not be
possible to conduct that same study at this time, because children no longer have blood or dental
lead levels as high as they did in the 1970s as a result of EPA’s implementation of the Statutes.
EPA’s drinking water standard for arsenic under the Safe Drinking Water Act is similarly
dependent on studies that the agency would now be compelled to ignore under the proposed rule.
EPA established a drinking water standard of 10 ppb for arsenic in 2001. 44 The Food and Drug
Administration (“FDA”) then relied on EPA’s determination. 45 In setting this standard, EPA
relied on a National Research Council review of the scientific evidence, which “concluded that
[certain epidemiological] studies from Taiwan provided the current best available data for the
risk assessment of inorganic arsenic-induced cancer.” 46 The Taiwanese papers looked at rates of
skin cancer and blackfoot disease in villagers from southwestern Taiwan who were exposed to

39

EPA, AIR QUALITY CRITERIA FOR LEAD, VOL. IV, 12-86 to 12-88, 12-95 (1986),
https://nepis.epa.gov/Exe/ZyPDF.cgi/9101HLA1.PDF?Dockey=9101HLA1.PDF.
40

Maximum Contaminant Level Goals and National Primary Drinking Water Regulations for Lead and Copper, 56
Fed. Reg. 26,460, 26,468–26,469 (June 7, 1991).

41

Lead; Identification of Dangerous Levels of Lead, 63 Fed. Reg. 30,302, 30,316–30,317 (proposed June 3, 1998).
The final rule was published at 66 Fed. Reg. 1,206 (Jan. 5, 2001).
42

National Ambient Air Quality Standards for Lead, 73 Fed. Reg. 66,964 (Nov. 12, 2008).

43
Coal. of Battery Recyclers Ass’n v. E.P.A., 604 F.3d 613, 622-624 (D.C. Cir. 2010) (rejecting need to make raw
data publicly available from Bruce P. Lanphear, et al., Low-Level Environmental Lead Exposure and Children’s
Intellectual Function: An International Pooled Analysis, 113 ENVTL. HEALTH PERSPECTIVES 894 (2005)).
44
National Primary Drinking Water Regulations; Arsenic and Clarifications to Compliance and New Source
Contaminants Monitoring, 66 Fed. Reg. 6,976, 7,036 (Jan. 22, 2001).
45

The FDA subsequently relied on EPA’s drinking water standard, as well as the research underlying it, when it
proposed an action level for arsenic for apple juice in 2013. See Draft Guidance for Industry on Arsenic in Apple
Juice: Action Level; Supporting Document for Action Level for Arsenic in Apple Juice; A Quantitative Assessment
of Inorganic Arsenic in Apple Juice; Availability, 78 Fed. Reg. 42,086 (July 15, 2013); see also Clark D. Carrington
et al., FDA, A Quantitative Assessment of Inorganic Arsenic in Apple Juice (2013),
https://www.fda.gov/downloads/Food/FoodScienceResearch/RiskSafetyAssessment/UCM360016.pdf.
46
National Primary Drinking Water Regulations; Arsenic and Clarifications to Compliance and New Source
Contaminants Monitoring, 65 Fed. Reg. 38,888, 38,902 (proposed June 22, 2000).

10

 Comments re: Docket ID No. EPA-HQ-OA-2018-0259
August 7, 2018
Page 11 of 26

high levels of arsenic in their drinking water. 47 These studies were based on data from clinical
examinations of the research subjects and therefore included confidential patient data that likely
cannot be released to the public. In addition, given that the first data were collected more than
50 years ago, the studies are based on data that may no longer be available.
Even though the proposed rule “is intended to apply prospectively,” it will also have a retroactive
impact. Some of the Statutes require EPA to periodically review its prior regulatory decisions.
For example, EPA must reconsider the lead NAAQS every five years. 48 EPA is also in the
process of reconsidering the Lead and Copper Rule under the Safe Drinking Water Act. 49 The
proposed rule would prohibit EPA from continuing to rely on Needleman’s critically-important
study in future reconsiderations of the lead NAAQS and revisions to the Lead and Copper Rule.
Other future rulemakings would also be undermined by the proposed rule. In 2011, EPA decided
to regulate perchlorate as a contaminant under the Safe Drinking Water Act. 50 “Perchlorate is
commonly used as an oxidizer in rocket propellants, munitions, fireworks, airbag initiators for
vehicles, matches, and signal flares” and is also present in some fertilizers. 51 It is known to
disrupt thyroid function by competitively inhibiting the uptake of iodide by the thyroid, and EPA
in 2011 concluded “that there is a substantial likelihood that perchlorate will occur in public
water systems with a frequency and at levels of public health concern.” 52 Late in 2017, EPA
issued a draft report identifying potential approaches to deriving a maximum contaminant level
goal for perchlorate. 53 To develop these approaches, EPA focused on five epidemiological
studies. 54 All five studies relied on confidential patient data. In addition, all five studies were
47

The original papers were W.P. Tseng et al., Prevalence of Skin Cancer in an Endemic Area of Chronic Arsenicism
in Taiwan, 40 J. NAT’L CANCER INST. 453 (1968) and Wen-Ping Tseng, Effects and Dose Response Relationships of
Skin Cancer and Blackfoot Disease with Arsenic, 19 ENVTL. HEALTH PERSP. 109 (1978). Subsequent articles
discussed longer-term health effects among the study cohort.
48

42 U.S.C. § 7409(d)(1).

49

See Lead and Copper Rule Long-Term Revisions, EPA, https://perma.cc/U5GV-B93M.

50

Drinking Water: Regulatory Determination on Perchlorate, 76 Fed. Reg. 7,762 (Feb. 11, 2011).

51

Perchlorate in Drinking Water, EPA, https://www.epa.gov/dwstandardsregulations/perchlorate-drinking-water
(last visited August 3, 2018).
52

76 Fed. Reg. at 7,763.

53

EPA, Draft Report: Proposed Approaches to Inform the Derivation of a Maximum Contaminant Level Goal for
Perchlorate in Drinking Water (2017), https://www.regulations.gov/document?D=EPA-HQ-OW-2016-0438-0019.

54

Id. at 6-1 to 6-19 (citing Tim I. M. Korevaar et al., Association of Maternal Thyroid Function during Early
Pregnancy with Offspring IQ and Brain Morphology in Childhood: A Population-based Prospective Cohort Study, 4
THE LANCET DIABETES & ENDOCRINOLOGY 35 (2016); Martijn J. J. Finken et al. Maternal Hypothyroxinemia in
Early Pregnancy Predicts Reduced Performance in Reaction Time Tests in 5- to 6-Year-Old Offspring, 98 J.
CLINICAL ENDOCRINOLOGY & METABOLISM 1417 (2013); F. Vermiglio et al., Attention Deficit and Hyperactivity
Disorders in the Offspring of Mothers Exposed to Mild-Moderate Iodine Deficiency: A Possible Novel Iodine
Deficiency Disorder in Developed Countries, 89 J. CLINICAL ENDOCRINOLOGY & METABOLISM 6054 (2004); Victor
J. Pop et al., Maternal Hypothyroxinemia during Early Pregnancy and Subsequent Child Development: A 3-year
Follow-up Study, 59 CLINICAL ENDOCRINOLOGY 282 (2003); Victor J. Pop et al., Low Maternal Free Thyroxine
Concentrations during Early Pregnancy Are Associated with Impaired Psychomotor Development in Infancy, 50
CLINICAL ENDOCRINOLOGY 149 (1999)).

11

 Comments re: Docket ID No. EPA-HQ-OA-2018-0259
August 7, 2018
Page 12 of 26

carried out in Europe, where scientists may be subject to different data confidentiality
requirements than in the United States. As a result, the proposed rule risks undermining the
scientific basis for this EPA action as well.
Many other EPA rulemakings and decisions have relied on studies that cannot be replicated and
whose data likely could not be made publicly available. For example:
•

PCBs: EPA’s regulations establishing water quality standards for polychlorinated
biphenyls (“PCBs”) under the Clean Water Act were based in part on long-term
epidemiological studies of cancer rates in workers exposed to PCBs. 55

•

Radionuclides: EPA’s Safe Drinking Water Act regulation for radionuclides relied on
epidemiological studies of survivors from the Hiroshima and Nagasaki atomic bomb
attacks. 56

•

Particulate matter: EPA’s 1997, 2006, and 2012 NAAQS for fine particulate matter all
relied on studies using confidential data, such as the Six Cities Study. 57

•

Methylmercury: EPA’s reference dose for methylmercury in fish that will be consumed
by humans relied on data from human exposures in the Faroe Islands. 58

Precluding reliance on these and other studies for the sole reason that the underlying raw data has
not been or cannot be released to the public is arbitrary, capricious, contrary to professional best
practices, and antithetical to protection of public health and safety as required by the Statutes.
The proposed rule will prevent EPA from relying on the “best available science.”
III.

“TRANSPARENCY” IN SCIENCE DOES NOT REQUIRE RELEASE OF
PRIVATE INFORMATION; IT REQUIRES A CLEAR STATEMENT AND
DETAILED DESCRIPTION OF THE METHODOLOGY USED BY THE
RESEARCHER

Transparency is valuable and important. As used in the draft rule, however, transparency is a
guise for excluding large bodies of valid—and best available—science. The concept of

55

Thomas Sinks et al., Mortality among Workers Exposed to Polychlorinated Biphenyls, 136 AM. J. EPIDEMIOLOGY
389 (1992); Pier Alberto Bertazzi et al., Cancer Mortality of Capacitor Manufacturing Workers, 11 AM. J. INDUS.
MED. 165 (1987).
56

See Environmental Data and Governance Initiative (“EDGI”), Public Protections Under Threat at the EPA:
Examining Safeguards and Programs That Would Have Been Blocked by H.R. 1430 9-10 (2017),
https://perma.cc/3NUU-MDHM.
57

Douglas W. Dockery, et al., An Association between Air Pollution and Mortality in Six U.S. Cities, 329 NEW
ENGLAND J. MED. 1753 (1993).
58
P. Grandjean, et al., Cognitive Deficit in 7-Year-Old Children with Prenatal Exposure to Methylmercury, 19(6)
NEUROTOXICOL TERATOL 417 (1997).

12

 Comments re: Docket ID No. EPA-HQ-OA-2018-0259
August 7, 2018
Page 13 of 26

transparency promoted by the draft rule is harmful to good decision-making, to implementation
of the Statutes, and, most of all, to protection of public health and safety.
In the professional scientific and medical research community, “transparency” means clear and
detailed disclosure of all methods, data, assumptions, and uncertainties. Studies are considered
“transparent” when the study design and methodology are clear enough to allow other scientists
to challenge assumptions, test hypotheses, and either reproduce or replicate the study to
determine whether the results obtained are consistent with the original study. Having the raw
data associated with the original study is not usually necessary to validate a study. 59
Transparency does not mean violating the confidentiality of study participants or making all raw
data publicly available. The proposed rule does not comport with the fundamental approach to
conducting scientific and medical research that is the standard practice for experienced,
advanced scholars and researchers.
Nor is it necessary to reproduce 60 a study to validate it. The proposal provides that
“[i]nformation is considered ‘publicly available in a manner sufficient for independent
validation’ when it includes the information necessary for the public to understand, assess, and
replicate [sic] findings.” 61 Neither reproducing nor replicating studies is always possible.
Indeed in some circumstances it would be inhumane, immoral, or physically impossible to do so.
Some studies involve natural disasters, other one-time events, or exposures and conditions that
no longer exist and cannot be reproduced or replicated. Those studies are valid but would be
excluded by the proposed rule. Examples include:
•

Studies of Hiroshima and Nagasaki survivors that underlie Safe Drinking Water Act
radionuclides regulation;

•

Studies of the effects of lead from 1970s, when blood lead levels were higher than they
are now;

•

Studies of worker exposure to polychlorinated biphenyls before PCBs were banned; these
studies formed the basis of water quality standards for PCBs under the Clean Water Act;

•

Long-term cohort studies of benzene exposure in workers which formed the basis of
EPA’s 2007 Clean Air Act regulation for emissions of hazardous air pollutants from
mobile sources; and

59

See supra notes 3, 7, 8. In the rare instance when the raw data is needed to validate a study, EPA already has the
ability to request it. This should be the exception, not the default as it has become in the proposed rule. If,
ultimately, EPA is unable to obtain the raw data to verify the study results, it is within the agency’s discretion to
categorize such data as “qualitative,” and taking into consideration inherent uncertainties, weigh the study relative to
other evidence. See EPA, Guidance for Considering and Using Open Literature Toxicity Studies to Support Human
Health Risk Assessments 9 (Aug. 28, 2012), https://www.epa.gov/sites/production/files/2015-07/documents/litstudies.pdf.

60

In the proposed rule, EPA incorrectly uses the term “replicate.” See note 3, above.

61

83 Fed. Reg. at 18,773–18,774.

13

 Comments re: Docket ID No. EPA-HQ-OA-2018-0259
August 7, 2018
Page 14 of 26

•
IV.

Studies based on the massive oil leak at Deepwater Horizon.
THE PROPOSED RULE IGNORES MECHANISMS THAT ALREADY EXIST
TO DEAL WITH CONCERNS ABOUT ACCESS TO RAW DATA

The proposed rule fails to acknowledge numerous federal laws, regulations, and guidance that
regulate the quality of and access to raw data. These include: the Information Quality Act,
Office of Management and Budget (“OMB”) Uniform Administrative Requirements, Cost
Principles, and Audit Requirements for Federal Awards (“OMB Uniform Guidance”), 62 and
EPA’s own Information Quality Guidelines. These already address the data access concerns that
EPA raises in the proposed rule. Moreover, the proposed rule is inconsistent with some aspects
of these other requirements. For example, OMB Uniform Guidance exempts from its definition
of “research data” subject to disclosure any “medical information and similar information the
disclosure of which would constitute a clearly unwarranted invasion of personal privacy, such as
information that could be used to identify a particular person in a research study.” 63 In contrast,
the proposed rule would generally prohibit EPA from relying on studies based on data not
disclosed to the public, even when disclosure would be a clearly unwarranted invasion of
personal privacy. Any decision to consider the study while allowing the data to remain
confidential is left to the whim of the EPA Administrator. This standardless, case-by-case
approach is inconsistent with OMB’s uniform privacy protections.
In the proposed rule, EPA ignores a variety of commonly-used mechanisms for assessing and
ensuring the validity of studies without requiring public disclosure of the raw data. These
mechanisms include peer reviews, pre-registration of study methodology, corroboration of
results by subsequent studies, and in some instances special agreements that enable an
independent third party, such as the Health Effects Institute (“HEI”), to re-analyze the raw data.
As explained by the Science Advisory Board, the HEI’s reanalysis of the Six Cities Study,
through “an unusually rigorous form of peer review and independent reanalysis, coupled with
many follow-up studies, has accomplished a measure of confidence in findings without public
access to data and analytic methods.” 64
For these reasons, the public health, medical, and scientific research community does not regard
the public disclosure of all raw data as necessary. For example, the Committee on Publication
Ethics (“COPE”), which has over 12,100 member journals and editors covering all areas of
scholarly inquiry, has established 10 core practices. COPE’s core practice #5 on data and
reproducibility provides that “[j]ournals should include policies on data availability and
encourage the use of reporting guidelines and registration of clinical trials and other study

62

See OMB, Uniform Administrative Requirements, Cost Principles, and Audit Requirements for Federal Awards,
78 Fed. Reg. 78,590, at 78,631, 2 C.F.R. § 200.315(e)(3) (Dec. 26, 2013) (guidance incorporated from OMB, OMB
Circular A–110, Uniform Administrative Requirements for Grants and Agreements With Institutions of Higher
Education, Hospitals, and Other Non-Profit Organizations § 36(d) (as amended Sept. 30,1999)) [hereinafter “OMB,
Uniform Guidance”].
63

OMB, Uniform Guidance, 2 C.F.R. § 200.315(e)(3)(ii).

64

SAB Memo, supra note 8, at 4.

14

 Comments re: Docket ID No. EPA-HQ-OA-2018-0259
August 7, 2018
Page 15 of 26

designs according to standard practice in their discipline.” 65 The simplicity and generality of this
core practice statement signals that the question of standards for data transparency, data access,
data sharing, data peer review, and replication and reproducibility practices are far from settled.
There is no one-size-fits-all approach to the critical questions of data transparency, data sharing,
and reproducibility.
The proposed rule was announced by EPA without any meaningful consultation with the broad
research community despite the fact that it addresses a complex and contentious issue that is not
yet ripe for regulatory action. There are ample and adequate safeguards in place at the leading
journals to ensure “transparency” – the ability of other researchers to question, challenge, and
validate the results of published studies. This would include the journals’ policies on treatment
of data from research published years and even decades ago. It is contrary to good scientific
study and practice and the advancement of knowledge for EPA to arrogate to itself the
determination of what constitutes useable research and data, and to grant sweeping discretion to
the Administrator—who may not even be a scientist—to make those determinations.
In a rare joint statement, the editors of the journals Science, Nature, PLOS One,
Proceedings of the National Academy of Sciences, and Cell explained:
It does not strengthen policies based on scientific evidence to limit the scientific
evidence that can inform them; rather, it is paramount that the full suite of relevant
science vetted through peer review, which includes ever more rigorous features,
inform the landscape of decision making. Excluding relevant studies simply
because they do not meet rigid transparency standards will adversely affect
decision-making processes. 66
As has long been recognized by the professional public health, medical, and scientific research
community—and by EPA itself until now 67 —whether or not the raw data underlying a study is
released does not determine the quality of the study. Rather, it is the scientific method that is
determinative. The proposed rule fails to take into account the fact that studies are reliable and
constitute the best available science when they comply with professionally-established best
practices for describing the methodology, sampling size, sampling procedure and assumptions
utilized and the results are consistent with those of other studies.

65

Core Practices, COPE, https://publicationethics.org/core-practices (last visited August 3, 2018) (copy attached for
inclusion in the administrative record, Attachment 5).
66

Jeremy Berg et al., Joint Statement on EPA Proposed Rule and Public Availability of Data,
SCIENCE (Apr. 30, 2018), http://science.sciencemag.org/content/early/2018/04/30/science.aau0116 (copy attached
for inclusion in the administrative record, Attachment 6).

67

See supra note 7.

15

 Comments re: Docket ID No. EPA-HQ-OA-2018-0259
August 7, 2018
Page 16 of 26

V.

THE PROPOSAL WOULD IMPOSE AN IMMENSE AND UNNECESSARY
COST AND PAPERWORK BURDEN ON EPA, OTHER FEDERAL AGENCIES,
AND THE RESEARCH COMMUNITY

EPA has not established a legitimate need for the proposed rule. EPA has made thousands of
regulatory decisions over the last 50 years. The Congressional Budget Office estimates that EPA
“relies on about 50,000 scientific studies annually to perform its mission.” 68 The proposed rule
fails to identify a single regulatory action based on faulty science. 69 The rule is not needed or
warranted. It will do far more harm than good.
Although OMB did not have a meaningful opportunity to review the proposed rule before former
Administrator Pruitt signed and released it (OMB had a mere five days) and presumably did not
intend to allow EPA’s new definitions to modify OMB’s Uniform Guidance, one might argue
that that is an effect of the proposed rule. If so, its radical and erroneous “transparency”
requirements would extend to all federal agencies, wreaking chaos.
The CBO estimates that it will cost between $10,000 and $30,000 per study to make the raw data
available. 70 If EPA continues to rely on roughly the same number of studies, it could cost
hundreds of millions of dollars a year to implement the proposal. Imposing these costs on all
federal agencies would be a staggering burden. Given the cost and the impracticality of releasing
all raw data to the public, EPA will have effectively but wrongly undermined public health and
safety. 71
Even if EPA or the researchers do spend this money and considerable time to de-identify data to
comply with the proposed rule, that effort will not necessarily protect patient or research subject
confidentiality. As mentioned above, it is frequently possible to re-identify individuals from
supposedly de-identified datasets. For example, one study found that the researchers could re-

68

Congressional Budget Office (“CBO”), Cost Estimate: H.R. 1030, Secret Science Reform Act of 2015 2 (Mar. 11,
2015), https://www.cbo.gov/sites/default/files/114th-congress-2015-2016/costestimate/hr1030.pdf.
69

Importantly, this proposed rule shifts the presumption of validity away from non-biased, peer-reviewed studies
conducted by professional and academic researchers to non-peer reviewed studies conducted by the interested,
regulated enterprises. In fact, if there is a problem anywhere in the science on which EPA relies, it is in the industry
studies submitted for licensing and permitting—yet these actions are excluded from the coverage of the rule by the
definition of “regulatory decisions.” See Thomas O. McGarity, Beyond Buckman: Wrongful Manipulation of the
Regulatory Process in the Law of Torts, 41 WASHBURN L.J. 549, 559-63 (2002) (detailing incidents in which data
required to be submitted by manufacturers or their contractors under the Federal Fungicide, Insecticide, and
Rodenticide Act (“FIFRA”) and the Food, Drug, and Cosmetic Act (“FDCA”) were either withheld or were
misleading or fraudulent); cf. SHELDON KRIMSKY, SCIENCE IN THE PRIVATE INTEREST: HAS THE LURE OF PROFITS
CORRUPTED THE VIRTURE OF BIOMEDICAL RESEARCH? (2003) (discussing this problem throughout the book and
providing considerable support).
70

CBO, supra note 68, at 2.

71

In the proposal, EPA cites a paper prepared by Randall Lutter and David Zorn for the Mercatus Center, which
arrives at a lower cost estimate than the CBO, to support its conclusion that “the benefits of this proposed rule justify
the costs.” 83 Fed. Reg. at 18,772 & n.24. EPA cannot abdicate its responsibility to conduct its own analysis of the
costs and benefits of this regulation by relying on this paper.

16

 Comments re: Docket ID No. EPA-HQ-OA-2018-0259
August 7, 2018
Page 17 of 26

identify approximately one-quarter of the records in a subset of a HIPAA-compliant
environmental health dataset. 72
Relatedly, for some studies (e.g. prospective cohort studies that include extensive personal data;
environmental health effects studies), it is impossible to de-identify the data without negating its
scientific value. To protect against re-identification, it would be necessary to remove so much
demographic information from the dataset that other scientists would not be able to perform
meaningful re-analyses of the data.
VI.

THE PROPOSED RULE WOULD CREATE CONFUSION AND CHAOS
DETRIMENTAL TO THE PROTECTION OF PUBLIC HEALTH

The proposal, as drafted, contains significant ambiguities. As a result, it is entirely unclear what
the effect of the proposed rule will be on studies that have already formed the basis of existing
rules but as to which the underlying raw data has not been and cannot be made available for
various reasons. These studies are considered by professionals to be the “best” available science.
The following crucial questions are not addressed by the proposed rule:
1. Will EPA continue to rely on those studies or will they now arbitrarily be excluded from
consideration?
2. Will EPA implement the new rule by ensuring that raw data are made available (very
costly) or simply by ignoring existing, valid studies as to which the data cannot be made
available or would be extremely expensive to de-identify?
3. How will EPA implement its exemption authority? What are the governing standards for
when the Administrator will exercise this authority?
4. Will the proposed rule apply to old studies or only new ones and to past regulatory
decisions or only new ones? The latter point is especially a concern under statutes that
require EPA to revise standards periodically. Will previously-established standards be
abandoned because the data from the studies underlying those decisions (in many cases
decades old) is no longer available?
5. How will the proposal affect the actions of other agencies that rely on EPA’s findings or
decisions or that provide information to EPA? For example, what will the effect be on
Agency for Toxic Substances and Disease Registry (“ATSDR”) analyses that EPA is
required to consider pursuant to the Comprehensive Environmental Response,
Compensation, and Liability Act?
6. How will EPA’s re-interpretation of OMB’s Uniform Guidance and other rules that apply
uniformly to the entire federal government be administered? For example, how will the
Food and Drug Administration’s review of applications for new drugs be affected?
72

Sweeney, et al., supra note 28.

17

 Comments re: Docket ID No. EPA-HQ-OA-2018-0259
August 7, 2018
Page 18 of 26

In addition, EPA has not included any analysis of the impact of the proposed rule on its existing
or future regulations.
Many of the signatories conduct studies, reports, analyses, and models that are used to support
the work of numerous state and federal agencies. The proposed rule will interfere with the
ability of these agencies to work together as required by some statutes to develop joint
approaches to protection of public health and safety due to the restrictions in the proposed rule.
Specifically, the rule will impede EPA’s ability to work effectively with the Food and Drug
Administration, ATSDR, the Department of Agriculture, and other agencies whose mission is to
protect public health.
VII.

THE PROPOSED NEW APPROACH TO DOSE-RESPONSE MODELING IS
ANTITHETICAL TO PROPER SCIENTIFIC METHODOLOGY AND
CONTRAVENES THE ADVICE OF EXPERTS IN THE FIELD, INCLUDING
THE NATIONAL ACADEMIES OF SCIENCES, ENGINEERING, AND
MEDICINE

EPA proposes to use “default assumptions, including assumptions of a linear, no-threshold dose
response, on a case-by-case basis....When available, EPA shall give explicit consideration to high
quality studies that explore: a broad class of parametric dose-response models; a robust set of
potential confounding variables; nonparametric models that incorporate fewer assumptions;
various threshold models across the dose or exposure range; and models that investigate factors
that might account for spatial heterogeneity.” 73 This proposed new approach allows for
assuming a safe threshold below which humans can be exposed to chemicals in circumstances
where data may be sparse. This approach runs counter to EPA’s own historic practice and to the
best practice employed by the scientific community when conducting risk assessments.
Specifically, the National Research Council has recommended that linear and conceptual models
be used “unless data is sufficient to reject low-dose linearity.” 74 The scientific research and risk
assessment community have also reached a consensus that cancer and non-cancer risk
assessment should be unified so that all compounds, not just carcinogens, should be subjected to
benchmark dose modeling. 75 This means that researchers should not assume a safe threshold of
exposure even for non-carcinogens such as lead and mercury. 76

73

83 Fed. Reg. at 18,774.

74
This has also been the position of the federal government since 1983. Eileen Abt, et al. Science and Decisions:
Advancing Risk Assessment, 30 RISK ANALYSIS 1028 (2010); Committee on the Institutional Means for Assessment
of Risks to Public Health, Commission on Life Sciences and National Research Center, Risk Assessment in the
Federal Government: Managing the Process (1983), http://www.nap.edu/catalog/366/risk-assessment-in-thefederal-government-managing-the-process.
75

EPA, Risk Assessment Forum, Benchmark Dose Technical Guidance (June 2012),
https://www.epa.gov/sites/production/files/2015-01/documents/benchmark_dose_guidance.pdf; Eileen Abt, et al.
Science and Decisions: Advancing Risk Assessment, 30 RISK ANALYSIS 1028 (2010).
76

EPA, supra note 75; Eileen Abt, et al., supra note 75.

18

 Comments re: Docket ID No. EPA-HQ-OA-2018-0259
August 7, 2018
Page 19 of 26

The approach EPA proposes also conflicts with the advice of EPA’s own Science Advisory
Board as well as the advice of the National Academies of Sciences, Engineering, and
Medicine. 77 And, EPA’s proposed new approach directly conflicts with the statutory mandates
that it must protect develop rules that protect human health “with an adequate margin of
safety.” 78
EPA’s assertion in the proposed rule that there is “growing empirical evidence of non-linearity in
the concentration-response function for specific pollutants and health effects” is dangerous and
unsupported by scientific evidence. 79 In recent years, several toxicants such as lead and
particulate matter air pollution have been shown to have either superlinear responses at low dose
or no threshold. 80 The consensus of the academic scientific community has been for over a
decade that threshold effects should not be presumed in the absence of robust concentrationresponse data. 81 Accordingly, this comment letter endorses and incorporates by reference the
comments on this point that have been submitted by: The National Academies of Sciences,
Engineering, and Medicine dated July 16, 2018, and the Center for Science in the Public Interest
dated July 17, 2018.
VIII. THE RULE SHOULD BE WITHDRAWN
The proposed rule will undermine EPA’s ability to fulfill its mission to protect human health,
safety, and the environment by using the best available information and science. First, the
proposed rule would exclude from EPA’s consideration any reports, studies, analyses, and
models that rely on confidential, inaccessible, or unavailable data but that historically have been
considered the best available science and therefore used to support regulations and standards
designed to protect public health and safety. Second, in so doing, the rule also eliminates EPA’s
access to fundamental information necessary for identifying and calculating the “health benefits”
of rules and standards needed to protect public health. Finally, it threatens to impose significant
costs on both the federal government and independent scientists. Worst of all, the proposed rule
creates these multiple problems without providing any significant countervailing benefits.

77

EPA, supra note 75; Eileen Abt, et al., supra note 75.

78

For example, the Clean Air Act, 42 U.S.C. § 7409(b)(1) (setting NAAQS); the Safe Drinking Water Act, 42
U.S.C. § 300g-1(b)(4)(A) (setting Maximum Contaminant Level Goals (“MCLG’s”)).

79

83 Fed. Reg. at 18,770.

80

Bruce P. Lanphear, et al., Low-Level Environmental Lead Exposure and Children’s Intellectual Function: An
International Pooled Analysis, 113 ENVTL. HEALTH PERSP. 894 (July 2005),
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1257652/; Joel Schwartz, Assessing Confounding, Effect
Modification, and Thresholds in the Association between Ambient Particles and Daily Deaths, 108 ENVTL. HEALTH
PERSP. 563 (June 2000), https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1638159/pdf/envhper00307-0129.pdf;
Qian Di, et al., Association of Short-term Exposure to Air Pollution With Mortality in Older Adults, JAMA
NETWORK (Dec. 26, 2017), https://jamanetwork.com/journals/jama/fullarticle/2667069.
81

Eileen Abt, et al., supra note 75.

19

 Comments re: Docket ID No. EPA-HQ-OA-2018-0259
August 7, 2018
Page 20 of 26

For these and all of the reasons explicated above, the proposed rule should be withdrawn.

By: ________________________
Wendy B. Jacobs, Esq.
Emmett Clinical Professor of Environmental Law and Clinic Director
Shaun A. Goho, Esq.
Deputy Director and Senior Staff Attorney
Emmett Environmental Law & Policy Clinic
Harvard Law School
6 Everett Street, Suite 4119
Cambridge, MA 02138
wjacobs@law.harvard.edu

On behalf of the following signatories:
HARVARD UNIVERSITY
President of Harvard University, Lawrence S. Bacow JD PhD

HARVARD T.H. CHAN SCHOOL OF PUBLIC HEALTH
Dean of Harvard T.H. Chan School of Public Health, Michelle A. Williams ScD
Senior Associate Dean and K.T. Li Professor Global Health; Director, Harvard Global
Health Institute, Ashish Jha MD MPH
Frederick Lee Hisaw Professor of Reproductive Physiology and Chair, Department of
Environmental Health, Russ Hauser MD ScD MPH
Irene Heinz Given Professor and Chair, Department of Immunology and Infectious
Diseases; Associate Physician, Brigham and Women's Hospital, Eric J. Rubin MD PhD
Professor of Epidemiology, Department of Epidemiology and Chair, Master of Public
Health Program, Murray Mittleman MD DrPH
Professor of Environmental Epidemiology and Associate Chair, Department of
Environmental Health and Director of the Exposure, Epidemiology, and Risk Program;
Member, EPA Chartered Scientific Advisory Board 2012-2017, Francine Laden MS ScD

20

 Comments re: Docket ID No. EPA-HQ-OA-2018-0259
August 7, 2018
Page 21 of 26

Elkan Blout Professor of Environmental Genetics, Departments of Environmental Health
and Epidemiology; Professor of Medicine, Pulmonary and Critical Care Division,
Department of Medicine, David Christiani MD MPH
John L. Loeb and Frances Lehman Loeb Research Professor of Environmental
Epidemiology, Departments of Environmental Health and Epidemiology; Director,
Harvard-NIEHS Center for Environmental Health, Douglas Dockery MS ScD
John LaPorte Given Professor of Immunology and Infectious Diseases; TB Program
Director, Ragon Institute of MGH, MIT and Harvard, Sarah Fortune MD
Akira Yamaguchi Professor of Environmental Health and Human Habitation; Program
Director, Nature, Health and the Built Environment, John Spengler MS PhD
Cecil K. and Philip Drinker Professor of Environmental Epidemiology and Physiology,
Departments of Environmental Health and Epidemiology, Marc Weisskopf PhD ScD
Professor of Biostatistics, Department of Biostatistics; Co-Director of the Data Science
Initiative, Francesca Dominici PhD
Professor of Epidemiology, Departments of Epidemiology and Immunology and Infectious
Diseases; Director, Center for Communicable Disease Dynamics, Marc Lipsitch, PhD
Professor of Environmental Epidemiology, Departments of Environmental Health and
Epidemiology, Joel Schwartz PhD
Professor of Epidemiology and Nutrition, Departments of Nutrition and Epidemiology,
Walter Willett MD PhD
Associate Professor of Nutrition and Epidemiology, Departments of Nutrition and
Epidemiology, Jorge Chavarro MD ScD
Assistant Professor of Exposure Assessment Science, Department of Environmental
Health; Co-Director, Center for Climate, Health and the Global Environment (CCHANGE), Joseph Allen MPH D.Sc.
Co-Director, Center for Climate, Health and the Global Environment (C-CHANGE);
Hospitalist, Division of General Pediatrics, Boston Children’s Hospital, Aaron Bernstein
MD MPH
Adjunct Professor of Environmental Health, Department of Environmental Health,
Philippe Grandjean MD

21

 Comments re: Docket ID No. EPA-HQ-OA-2018-0259
August 7, 2018
Page 22 of 26

Adjunct Professor, Department of Environmental Health; Professor and Interim Chair,
Department of Environmental Health, Boston University School of Public Health,
Jonathan Levy ScD
Associate Professor, Department of Immunology and Infectious Disease, Shahin Lockman
MD MSc
Visiting Professor, Department of Environmental Health; Senior Professor, Heidelberg
University and former Head of the Institute of Public Health at Heidelberg University
Hospital 1997-2016; Member of the Intergovernmental Panel on Climate Change (IPCC)
2010-2014, Rainer Sauerborn MD PhD MPH Msc

HARVARD MEDICAL SCHOOL AND AFFILIATED TEACHING HOSPITALS
Dean of Harvard Medical School, George Q. Daley MD PhD
Higgins Professor of Microbiology and Molecular Genetics, Department of Microbiology
and Immunobiology, David M. Knipe PhD

Beth Israel Deaconess Medical Center
President of Beth Israel Deaconess Medical Center, Peter Healy
CEO of Beth Israel Deaconess Medical Center, Kevin Tabb MD
Chief, Department of Surgery, Elliot L. Chaikof, MD
Chief, Department of Orthopaedic Surgery, Mark C. Gebhardt, MD
Chief, Department of Psychiatry, William E. Greenberg, MD
Chief, Department of Radiology, Jonathan B. Kruskal, MD, PhD
Chief, Department of Dermatology, Suzanne Olbricht, MD
Chief, Department of Neonatology, DeWayne M. Pursely, MD, MPH
Interim Chief Academic Officer; Chief, Department of Pathology, Jeffrey E. Saffitz, MD,
PhD
Chief, Department of Neurology, Clifford B. Saper, MD, PhD
Chief, Department of Radiation Oncology, MaryAnn Stevenson, MD, PhD
22

 Comments re: Docket ID No. EPA-HQ-OA-2018-0259
August 7, 2018
Page 23 of 26

Chief, Department of Anesthesia, Pain Management and Critical Care, Daniel S. Talmor,
MD, MPH
Chief, Emergency Medicine, Richard E. Wolfe, MD
Chief of Department of Medicine; Herrman Ludwig Blumgart Professor of Medicine,
Mark L. Zeidel MD
Ellen and Melvin Gordon Professor of Medical Education, Richard M. Schwartzstein
MD
Associate Professor of Medicine, Robert J. Thomas MD
Assistant Professor of Medicine, Katherine Berg MD
Assistant Professor of Medicine, Robert Hallowell MD
Assistant Professor of Medicine, Margaret M. Hayes MD
Assistant Professor of Medicine, Mary Rice MD MPH
Assistant Professor of Medicine, Jeremy Richards MD MA
Assistant Professor of Medicine, Elisabeth Riviello MD
Assistant Professor of Medicine, Amy Sullivan, EdD
Instructor in Medicine, Anjali Ahn MD
Instructor in Medicine, Douglas Beach MD
Instructor in Medicine, Elias Baedorf Kassis MD
Instructor in Medicine, Sean Levy MD
Instructor in Medicine, Debby Ngo MD
Instructor in Medicine, Mihir S. Parikh MD
Instructor in Medicine, Melanie S. Pogach MD
Instructor in Medicine, Laura Rock MD

23

 Comments re: Docket ID No. EPA-HQ-OA-2018-0259
August 7, 2018
Page 24 of 26

Brigham and Women’s Hospital
President of Brigham and Women’s Hospital, Elizabeth G. Nabel MD
Hersey Professor of the Theory and Practice of Medicine and Chair, Department of
Medicine, Joseph Loscalzo MD PhD
Professor of Medicine and Chief, Channing Division of Network Medicine, Edwin
Silverman MD PhD
Edward H. Kass Distinguished Professor of Medicine, Frank E. Speizer MD
Associate Professor of Medicine and Deputy Editor, New England Journal of Medicine,
Caren Solomon MD MPH
Assistant Professor of Medicine, Channing Division of Network Medicine, Jaime Hart
MS ScD

Massachusetts Eye and Ear
President of Massachusetts Eye and Ear, John Fernandez
Chief of Ophthalmology and Chair, Department of Ophthalmology and David Glendenning
Cogan Professor of Ophthalmology, Joan Miller MD
Charles L. Schepens Professor of Ophthalmology and Professor of Pathology, Schepens
Eye Research Institute, Patricia D’Amore PhD MBA
Charles Edward Whitten Professor of Ophthalmology and Director of the Retina Service,
Evangelos Gragoudas MD
David Glendenning Cogan Professor of Ophthalmology and Director of NeuroOphthalmology Service, Joseph Rizzo MD

Massachusetts General Hospital
President of Massachusetts General Hospital, Peter Slavin MD
Chief, Obstetrics and Gynecology and Joe Vincent Meigs Professor of Obstetrics and
Gynecology, Jeffery Ecker MD
Chief, Department of Dermatology and Professor of Dermatology, David E. Fisher MD
PhD

24

 Comments re: Docket ID No. EPA-HQ-OA-2018-0259
August 7, 2018
Page 25 of 26

Chief, Department of Emergency Medicine and MGH Trustees Professor of Emergency
Medicine, David F.M. Brown MD
Chief, Department of Pediatrics and Physician-in-Chief at MassGeneral Hospital for
Children; Charles Wilder Professor of Pediatrics, Ronald Kleinman MD
Professor of Emergency Medicine, Carlos Camargo Jr. MD DrPH MPH
Professor of Emergency Medicine, Joshua Goldstein MD PhD
Associate Professor of Emergency Medicine, N. Stuart Harris MD
Associate Professor of Emergency Medicine and Director of Center for Vascular
Emergencies, Christopher Kabrhel MD MPH
Associate Professor of Emergency Medicine and Director of the Center for Ultrasound
Research and Education, Andrew Liteplo MD
Associate Professor of Emergency Medicine and Director of Emergency Medicine
Research Program, John T Nagurney MD
Associate Professor of Emergency Medicine, Hamid Shokoohi MD
Assistant Professor of Medicine, Mark Eisenberg MD
Assistant Professor of Medicine, Regina LaRocque MD MPH
Assistant Professor of Emergency Medicine, Emily Miller MD
Assistant Professor of Emergency Medicine, Brian Yun MD MBA MPH
Instructor of Medicine, George Alba MD
Instructor of Medicine, Director of Inpatient Medicine, Marjory Bravard MD
Instructor of Pediatrics, Kathryn Brigham MD
Instructor of Emergency Medicine, Sayon Dutta MD
Instructor of Emergency Medicine, Kamal Medlej MD
Instructor of Emergency Medicine, Renee N. Salas MD MPH MS
Instructor of Emergency Medicine, Jonathan Slutzman MD
Associate Director, Recovery Research Institute, Brandon Bergman PhD
25

 Comments re: Docket ID No. EPA-HQ-OA-2018-0259
August 7, 2018
Page 26 of 26

Other
Assistant Professor of Psychiatry, Cambridge Health Alliance, Jim Recht MD
Instructor in Emergency Medicine, Mount Auburn Hospital, Justin Pitman MD

HARVARD JOHN A. PAULSON SCHOOL OF ENGINEERING AND APPLIED SCIENCES
Philip S. Weld Professor of Atmospheric Chemistry, James Anderson PhD
Gordon McKay Professor of Environmental Chemistry, Elsie M. Sunderland PhD

26

 Emmett Environmental Law & Policy Clinic - Attachment 1
Comments re: Docket ID No. EPA-HQ-OA-2018-0259, August 7, 2018
News & Analysis

The JAMA Forum

A Breath of Bad Air: Cost of the Trump Environmental Agenda
May Lead to 80 000 Extra Deaths per Decade
David Cutler, PhD; Francesca Dominici, PhD

P

resident Donald Trump and Environmental Protection Agency
(EPA) Administrator Scott Pruitt
have pledged to reexamine landmark environmental policies and to repeal regulations. In their view, excessive regulations
are harming US industry, and thus reducing regulation will be good for business. As
Donald Trump has said, seemingly without
irony, “We are going to get rid of the regulations that are just destroying us. You
can’t breathe—you cannot breathe.”

As has become apparent, however, it is
the changes Trump is proposing that are
likely to make breathing more difficult. A
central feature of his agenda is environmental damage: making the air dirtier and exposing people to more toxic chemicals. The beneficiaries, in contrast, will be a relatively few
well-connected companies.

Stockbyte/Thinkstock

The Trump Agenda
In pursuit of its wide-ranging environmental agenda, the administration has already
reversed or proposed to reverse more
than 60 environmental rules. The full
extent of the effects on health has not
been tabulated and is hard to quantify, but
guesses can be made for some of the
larger ones (see the Table).
The largest health consequences
are likely to come through changes in air
quality. The Trump administration has
announced its intention to repeal the Clean
Power Plan rule, President Barack Obama’s
signature policy on climate change. The
rule provides for the EPA to assign each
state a goal for limiting emissions from
jama.com

existing power plants and gives the states
latitude in meeting those goals, such as
switching from coal to natural gas or building new wind or solar farms. Based on the
regulatory impact analysis done by the EPA
when the rule was implemented (as well as
other analyses), repealing the rule would
lead to an estimated 36 000 deaths each
decade and nearly 630 000 cases of respiratory infection in children alone.
The administration is also targeting
the control of air pollution from motor
vehicles, indicating a desire to weaken
greenhouse gas and fuel economy targets for
automobiles. Nothing formal has been proposed, but Trump has spoken about rolling
back new rules put in place by the Obama
administration. Based on the regulatory
impact analysis performed when those
rules were proposed, it was estimated that
they would lead to a reduction of 5500
deaths and 140 000 cases of respiratory
ailments in children over a decade—
benefits that would be lost if the rules are
rolled back. Repealing these rules will also
have negative effects on certain types of jobs,
the environment (global warming pollution),
and consumer savings. The administration
is also planning to repeal the emission
requirements for glider vehicles—rebuilt
trucks that do not meet current environmental standards—a loophole that could
lead to as many as 41 000 premature
deaths per decade and 900 000 cases of
respiratory tract symptoms.
Other elements of the administration’s
environmental agenda will also affect health,
though it is hard to know by how much.
Withdrawing from the Paris agreement on
global warming, imposing tariffs on solar
panels, and rolling back the “once in, always in” rule for industrial plants will all lead
to increases in fine particulate matter and additional exposure to pollutants such as sulfur dioxide, nitrogen oxides, mercury, and
others that adversely affect respiratory and
cardiovascular health.
Water quality is also being targeted.
The Trump EPA has proposed to rescind

the Waters of the United States rule published in 2015, which brought more US
streams and wetlands areas under the
Clean Water Act. Rivers and streams are
sources of drinking water for more than
130 million people and if polluted, might
pose major health risks. The rule itself
does not mandate any specific changes in
water cleanliness, so we do not estimate a
specific health consequence of repealing
this rule.
Finally, the administration is proposing to withdraw or not implement regulatory actions affecting particular chemicals
shown to be harmful to health, including
lead, agricultural pesticides, and coal ash
waste. Exposure to these hazardous substances will affect fewer people than the
number of individuals affected by air pollution, but each will affect a concentrated
number. As Christine Todd Whitman, head
of the EPA under President George W.
Bush, said: “You stop enforcing those
regulations and [deaths] will go way up.”
Overall, an extremely conservative
estimate is that the Trump environmental
agenda is likely to cost the lives of over
80 000 US residents per decade and lead
to respiratory problems for many more
than 1 million people. This sobering statistic captures only a small fraction of the
cumulative public health damages associated with the full range of rollbacks and
systemic actions proposed by the Trump
administration.

An Attack on Science
One might imagine that the science that
supported enactment of these rules would
make repealing them difficult. But that is
not the case. Even as it is targeting environmental rules, the Trump administration is
taking aim at the use of science that supports public policy.
Scott Pruitt recently signed a controversial rule stipulating that policy can
be based only on research for which the
underlying data have been made accessible to the general public. The idea is to

(Reprinted) JAMA June 12, 2018 Volume 319, Number 22

© 2018 American Medical Association. All rights reserved.

Downloaded From: by a Harvard University User on 07/23/2018

2261

 News & Analysis

Proposed Changes in Environmental Protections and Possible Effects
Area

Actions

Projected Effects

Air Quality

Repeal of Clean Power Plan

• Increases exposure to small atmospheric
particulate matter
• An estimated 36 000 deaths over a decade
• An estimated 630 000 cases of respiratory ailments
in children over a decade
• Increases exposure to small atmospheric particulate
matter and ozone
• An estimated 5500 deaths over a decade
• An estimated 140 000 cases of respiratory ailments
in children over a decade
• Allows noncompliant diesel trucks on the roads
• An estimated 41 000 premature deaths over a decade
• An estimated 900 000 cases of respiratory ailments
over a decade
• Potential for industrial plants to increase emissions
by 4 times
• Endangering those living near power plants
(areas of high poverty)
• Exposes water sources for approximately 117
million US residents
• At least 1 million people in each of 21 different states
depend on small streams for their drinking water
• Leaves an estimated 4 million households with
children at risk of exposure to high levels of lead
• Approximately 500 000 children currently have
elevated blood lead levels
• Exposes toddlers and older children to 11 to 15 times
the recommended levels of chlorpyrifos
(because of denial of ban on use in agriculture)
• Exposes public to 3 carcinogens (methylene chloride,
trichloroethylene, and N-Methylpyrrolidone) used in
furniture stripping, grease removal, and dry cleaning
(action delayed)
• More than 100 million tons of coal ash are produced
annually, resulting in more than 100 documented
cases of coal ash poison contamination in the
drinking water, wetlands, creeks, and rivers between
1948 and 2008

Rollback of CAFEa
standards for automobiles

Repeal of emission
requirements for glider
vehicles
Loosening of other air
pollution rules (eg, power
plants, solar power tariffs)
Water Quality

Repeal of Waters of the
United States rule

Chemicals

Scale back of lead-risk
reduction program

Delay or reduction of
chemical bans

Weakening of rules on
coal ash waste

a There is substantial uncertainty with respect to the extent of the rollback of the Corporate Average Fuel Economy (CAFE)

rules. Projected health effects are calculated based on the assumption of full achievement of CAFE standards vs rollback
of those standards.

remove most observational studies of
health effects of air pollution exposure
from being considered in regulatory settings, unless the individual health records
are made publicly available. This is a nearly
impossible task because the health data
are collected under the agreement to
maintain patient confidentiality. With no
evidence of harms (because of constraints
on presenting the available evidence),
regulations cannot be sustained. On April
23, 985 scientists sent him a letter urging
him to abandon the proposal.

2262

Fortunately for those interested in public health, the regulatory process will take
many years. Whoever is sworn in as President in January 2021 will have a large effect
on whether the Trump administration’s full
environmental agenda goes into effect.

Implications for Physicians and Policy
For physicians, the manifestation of these
changes is likely to be an increase in disease
and number of deaths. Respiratory and
cardiovascular problems are most likely, but
a wide variety of conditions are likely to

be seen. Poor, black, or elderly populations
are likely to be affected the most. People
working with chemicals in industrial settings will also be affected, as will people
who live in areas with high concentrations of
power plants such as the Ohio River Valley
from Indiana to Pennsylvania, and in the
southeast from Alabama and Georgia
to Maryland.
One could debate the merits of these
tradeoffs if there were a large number of
people who would benefit economically
from these changes. In practice, however,
any economic benefits are not likely to accrue to those most in need. Employment is
down in many fossil fuel industries because
technology has made workers less necessary
for production, not because of environmental regulations. And even if a large number
of coal jobs were restored, it would come at
the expense of employment in new industries such as wind and solar, which are
already being hurt by the Trump administration policies. Not having to comply with environmental rules will increase corporate
profits, but not worker bank accounts.
Overall, the ultimate effects of the
Trump administration’s policies seem clear,
even through the haze they will create.
Author Affiliations: Otto Eckstein Professor of
Applied Economics in the Department of
Economics and Kennedy School of Government at
Harvard University and a member of the Institute of
Medicine (Cutler); professor of biostatistics at the
Harvard T.H. Chan School of Public Health and
codirector of the Harvard Data Science Initiative
(Dominici).
Corresponding Author: David Cutler, PhD
(dcutler@harvard.edu).
Published Online: May 10, 2018, at https:
//newsatjama.jama.com/category/the-jama-forum/.
Disclaimer: Each entry in The JAMA Forum
expresses the opinions of the author but does not
necessarily reflect the views or opinions of JAMA,
the editorial staff, or the American Medical
Association.
Additional Information: Information about
The JAMA Forum, including disclosures of
potential conflicts of interest, is available at
https://newsatjama.jama.com/about/.
Note: Source references are available through
embedded hyperlinks in the article text online.

JAMA June 12, 2018 Volume 319, Number 22 (Reprinted)

© 2018 American Medical Association. All rights reserved.

Downloaded From: by a Harvard University User on 07/23/2018

jama.com

 Emmett Environmental Law & Policy Clinic - Attachment 2
Comments re: Docket ID No. EPA-HQ-OA-2018-0259, August 7, 2018

new england
journal of medicine
The

established in 1812 

June 29, 2017

vol. 376  no. 26

Air Pollution and Mortality in the Medicare Population
Qian Di, M.S., Yan Wang, M.S., Antonella Zanobetti, Ph.D., Yun Wang, Ph.D., Petros Koutrakis, Ph.D.,
Christine Choirat, Ph.D., Francesca Dominici, Ph.D., and Joel D. Schwartz, Ph.D.​​

a bs t r ac t
BACKGROUND

Studies have shown that long-term exposure to air pollution increases mortality.
However, evidence is limited for air-pollution levels below the most recent National Ambient Air Quality Standards. Previous studies involved predominantly
urban populations and did not have the statistical power to estimate the health
effects in underrepresented groups.
METHODS

We constructed an open cohort of all Medicare beneficiaries (60,925,443 persons)
in the continental United States from the years 2000 through 2012, with
460,310,521 person-years of follow-up. Annual averages of fine particulate matter
(particles with a mass median aerodynamic diameter of less than 2.5 μm [PM2.5])
and ozone were estimated according to the ZIP Code of residence for each enrollee with the use of previously validated prediction models. We estimated the risk
of death associated with exposure to increases of 10 μg per cubic meter for PM2.5
and 10 parts per billion (ppb) for ozone using a two-pollutant Cox proportionalhazards model that controlled for demographic characteristics, Medicaid eligibility, and area-level covariates.

From the Departments of Environmental
Health (Q.D., Yan Wang, A.Z., P.K., J.D.S.)
and Biostatistics (Yun Wang, C.C., F.D.),
Harvard T.H. Chan School of Public Health,
Boston. Address reprint requests to Dr.
Dominici at Harvard T.H. Chan School of
Public Health, Biostatistics Department,
Bldg. 2, 4th Flr., 655 Huntington Ave.,
Boston, MA 02115, or at f­ dominic@​­hsph​
.­harvard​.­edu.
N Engl J Med 2017;376:2513-22.
DOI: 10.1056/NEJMoa1702747
Copyright © 2017 Massachusetts Medical Society.

RESULTS

Increases of 10 μg per cubic meter in PM2.5 and of 10 ppb in ozone were associated with increases in all-cause mortality of 7.3% (95% confidence interval [CI],
7.1 to 7.5) and 1.1% (95% CI, 1.0 to 1.2), respectively. When the analysis was restricted to person-years with exposure to PM2.5 of less than 12 μg per cubic meter
and ozone of less than 50 ppb, the same increases in PM2.5 and ozone were associated with increases in the risk of death of 13.6% (95% CI, 13.1 to 14.1) and
1.0% (95% CI, 0.9 to 1.1), respectively. For PM2.5, the risk of death among men,
blacks, and people with Medicaid eligibility was higher than that in the rest of the
population.
CONCLUSIONS

In the entire Medicare population, there was significant evidence of adverse effects
related to exposure to PM2.5 and ozone at concentrations below current national
standards. This effect was most pronounced among self-identified racial minorities and people with low income. (Supported by the Health Effects Institute and
others.)

n engl j med 376;26 nejm.org  June 29, 2017

The New England Journal of Medicine
Downloaded from nejm.org at Harvard Library on July 23, 2018. For personal use only. No other uses without permission.
Copyright © 2017 Massachusetts Medical Society. All rights reserved.

2513

 The

n e w e ng l a n d j o u r na l

T
A Quick Take
is available at
NEJM.org

he adverse health effects associated with long-term exposure to air pollution are well documented.1,2 Studies suggest that fine particles (particles with a mass
median aerodynamic diameter of less than 2.5 μm
[PM2.5]) are a public health concern,3 with exposure linked to decreased life expectancy.4-6 Longterm exposure to ozone has also been associated
with reduced survival in several recent studies,
although evidence is sparse.4,7-9
Studies with large cohorts have investigated
the relationship between long-term exposures to
PM2.5 and ozone and mortality4,9-13; others have
estimated the health effects of fine particles at
low concentrations (e.g., below 12 μg per cubic
meter for PM2.5).14-18 However, most of these
studies have included populations whose socioeconomic status is higher than the national average and who reside in well-monitored urban areas.
Consequently, these studies provide limited information on the health effects of long-term exposure to low levels of air pollution in smaller
cities and rural areas or among minorities or
persons with low socioeconomic status.
To address these gaps in knowledge, we conducted a nationwide cohort study involving all
Medicare beneficiaries from 2000 through 2012,
a population of 61 million, with 460 million
person-years of follow-up. We used a survival
analysis to estimate the risk of death from any
cause associated with long-term exposure (yearly
average) to PM2.5 concentrations lower than the
current annual National Ambient Air Quality
Standard (NAAQS) of 12 μg per cubic meter and
to ozone concentrations below 50 parts per billion
(ppb). Subgroup analyses were conducted to identify populations with a higher or lower level of
pollution-associated risk of death from any cause.

Me thods
Mortality Data

We obtained the Medicare beneficiary denominator file from the Centers for Medicare and
Medicaid Services, which contains information
on all persons in the United States covered by
Medicare and more than 96% of the population
65 years of age or older. We constructed an open
cohort consisting of all beneficiaries in this age
group in the continental United States from
2000 through 2012, with all-cause mortality as
the outcome. For each beneficiary, we extracted
2514

of

m e dic i n e

the date of death (up to December 31, 2012), age
at year of Medicare entry, year of entry, sex, race,
ZIP Code of residence, and Medicaid eligibility
(a proxy for low socioeconomic status). Persons
who were alive on January 1 of the year following their enrollment in Medicare were entered
into the open cohort for the survival analysis.
Follow-up periods were defined according to
calendar years.
Assessment of Exposure to Air Pollution

Ambient levels of ozone and PM2.5 were estimated
and validated on the basis of previously published prediction models.19,20 Briefly, we used an
artificial neural network that incorporated satellite-based measurements, simulation outputs from
a chemical transport model, land-use terms,
meteorologic data, and other data to predict
daily concentrations of PM2.5 and ozone at unmonitored locations. We fit the neural network
with monitoring data from the Environmental
Protection Agency (EPA) Air Quality System
(AQS) (in which there are 1928 monitoring stations for PM2.5 and 1877 monitoring stations for
ozone). We then predicted daily PM2.5 and ozone
concentrations for nationwide grids that were
1 km by 1 km. Cross-validation indicated that
predictions were good across the entire study
area. The coefficients of determination (R2) for
PM2.5 and ozone were 0.83 and 0.80, respectively;
the mean square errors between the target and
forecasting values for PM2.5 and ozone were 1.29 μg
per cubic meter and 2.91 ppb, respectively. Data
on daily air temperature and relative humidity
were retrieved from North American Regional
Reanalysis with grids that were approximately
32 km by 32 km; data were averaged annually.21
For each calendar year during which a person
was at risk of death, we assigned to that person
a value for the annual average PM2.5 concentration,
a value for average ozone level during the warm
season (April 1 through September 30), and values
for annual average temperature and humidity according to the ZIP Code of the person’s residence.
The warm-season ozone concentration was used
to compare our results with those of previous
studies.10 In this study, “ozone concentration”
refers to the average concentration during the
warm season, unless specified otherwise.
As part of a sensitivity analysis, we also obtained data on PM2.5 and ozone concentrations
from the EPA AQS and matched that data with

n engl j med 376;26 nejm.org  June 29, 2017

The New England Journal of Medicine
Downloaded from nejm.org at Harvard Library on July 23, 2018. For personal use only. No other uses without permission.
Copyright © 2017 Massachusetts Medical Society. All rights reserved.

 Air Pollution and Mortality in the Medicare Population

each person in our study on the basis of the nearest monitoring site within a distance of 50 km.
(Details are provided in Section 1 in the Supplementary Appendix, available with the full text of
this article at NEJM.org.)
Statistical Analysis

We fit a two-pollutant Cox proportional-hazards
model with a generalized estimating equation to
account for the correlation between ZIP Codes.22
In this way, the risk of death from any cause
associated with long-term exposure to PM2.5 was
always adjusted for long-term exposure to ozone,
and the risk of death from any cause associated
with long-term exposure to ozone was always
adjusted for long-term exposure to PM2.5, unless
noted otherwise. We also conducted singlepollutant analyses for comparability. We allowed
baseline mortality rates to differ according to
sex, race, Medicaid eligibility, and 5-year categories of age at study entry. To adjust for potential
confounding, we also obtained 15 ZIP-Code or
county-level variables from various sources and
a regional dummy variable to account for compositional differences in PM2.5 across the United
States (Table 1, and Section 1 in the Supplementary Appendix). We conducted this same statistical analysis but restricted it to person-years with
PM2.5 exposures lower than 12 μg per cubic
meter and ozone exposures lower than 50 ppb
(low-exposure analysis) (Table 1, and Section 1 in
the Supplementary Appendix).
To identify populations at a higher or lower
pollution-associated risk of death from any cause,
we refit the same two-pollutant Cox model for
some subgroups (e.g., male vs. female, white vs.
black, and Medicaid eligible vs. Medicaid ineligible). To estimate the concentration-response
function of air pollution and mortality, we fit a
log-linear model with a thin-plate spline of both
PM2.5 and ozone and controlled for all the individual and ecologic variables used in our main
analysis model (Section 7 in the Supplementary
Appendix). To examine the robustness of our
results, we conducted sensitivity analyses and
compared the extent to which estimates of risk
changed with respect to differences in confounding adjustment and estimation approaches
(Sections S2 through S4 in the Supplementary
Appendix).
Data on some important individual-level covariates were not available for the Medicare co-

hort, including data on smoking status, bodymass index (BMI), and income. We obtained data
from the Medicare Current Beneficiary Survey
(MCBS), a representative subsample of Medicare
enrollees (133,964 records and 57,154 enrollees
for the period 2000 through 2012), with individuallevel data on smoking, BMI, income, and many
other variables collected by means of telephone
survey. Using MCBS data, we investigated how
the lack of adjustment for these risk factors
could have affected our calculated risk estimates
in the Medicare cohort (Section 5 in the Supplementary Appendix). The computations in this
article were run on the Odyssey cluster, which is
supported by the FAS Division of Science, Research Computing Group, and on the Research
Computing Environment, which is supported by
the Institute for Quantitative Social Science in the
Faculty of Arts and Sciences, both at Harvard
University. We used R software, version 3.3.2
(R Project for Statistical Computing), and SAS
software, version 9.4 (SAS Institute).

R e sult s
Cohort Analyses

The full cohort included 60,925,443 persons living
in 39,716 different ZIP Codes with 460,310,521
person-years of follow-up. The median follow-up
was 7 years. The total number of deaths was
22,567,924. There were 11,908,888 deaths and
247,682,367 person-years of follow-up when the
PM2.5 concentration was below 12 μg per cubic
meter and 17,470,128 deaths and 353,831,836
person-years of follow-up when the ozone concentration was below 50 ppb. These data provided
excellent power to estimate the risk of death at
air-pollution levels below the current annual
NAAQS for PM2.5 and at low concentrations for
ozone (Table 1).
Annual average PM2.5 concentrations across the
continental United States during the study period
ranged from 6.21 to 15.64 μg per cubic meter
(5th and 95th percentiles, respectively), and the
warm-season average ozone concentrations ranged
from 36.27 to 55.86 ppb (5th and 95th percentiles, respectively). The highest PM2.5 concentrations were in California and the eastern and
southeastern United States. The Mountain region
and California had the highest ozone concentrations; the eastern states had lower ozone concentrations (Fig. 1).

n engl j med 376;26 nejm.org  June 29, 2017

The New England Journal of Medicine
Downloaded from nejm.org at Harvard Library on July 23, 2018. For personal use only. No other uses without permission.
Copyright © 2017 Massachusetts Medical Society. All rights reserved.

2515

 n e w e ng l a n d j o u r na l

The

of

m e dic i n e

Table 1. Cohort Characteristics and Ecologic and Meteorologic Variables.
Characteristic or Variable

Entire Cohort

Ozone Concentration
≥50 ppb*

<50 ppb

PM2.5 Concentration
≥12 μg/m3

<12 μg/m3
32,779,950

Population
Persons (no.)

60,925,443

14,405,094

46,520,349

28,145,493

Deaths (no.)

22,567,924

5,097,796

17,470,128

10,659,036

11,908,888

Total person-yr†

460,310,521

106,478,685

353,831,836

212,628,154

247,682,367

7

7

7

7

7

Ozone (ppb)

46.3

52.8

44.4

48.0

45.3

PM2.5 (μg/m3)

11.0

10.9

11.0

13.3

9.6

44.0

44.3

43.8

43.1

44.7

Median yr of follow-up
Average air-pollutant concentrations‡

Individual covariates‡
Male sex (%)
Race or ethnic group (%)§
White

85.4

86.6

85.1

82.0

88.4

Black

8.7

7.2

9.2

12.0

5.9

Asian

1.8

1.8

1.8

2.1

1.6

Hispanic

1.9

2.0

1.9

1.9

1.9

Native American

0.3

0.6

0.3

0.1

0.6

Eligible for Medicaid (%)

16.5

15.3

16.8

17.8

15.3

Average age at study entry (yr)

70.1

69.7

70.2

70.1

70.0

BMI

28.2

27.9

28.4

28.0

28.4

Ever smoked (%)

46.0

44.9

46.2

45.8

46.0

9.5

13.4

8.4

8.4

10.0

Ecologic variables‡

Population including all people 65 yr of age
or older (%)
Hispanic
Black
Median household income (1000s of $)
Median value of housing (1000s of $)

8.8

7.2

9.3

13.3

6.3

47.4

51.0

46.4

47.3

47.4
159.8

160.5

175.8

156.3

161.7

Below poverty level (%)

12.2

11.4

12.4

12.5

12.0

Did not complete high school (%)

32.3

30.7

32.7

35.3

30.6

Owner-occupied housing (%)

71.5

71.3

71.6

68.6

73.2

3.2

0.7

3.8

4.8

2.2

Low-density lipoprotein level measured (%)

92.2

92.0

92.2

92.2

92.2

Glycated hemoglobin level measured (%)

94.8

94.6

94.8

94.8

94.8

≥1 Ambulatory visits (%)¶

91.7

92.2

91.6

91.7

91.7

Population density (persons/km2)

Meteorologic variables‡
Average temperature (°C)

14.0

14.9

13.8

14.5

13.7

Relative humidity (%)

71.1

60.8

73.9

73.7

69.6

*   Summary statistics were calculated separately for persons residing in ZIP Codes where average ozone levels were below or above 50 ppb
and where PM2.5 levels were below or above 12 μg per cubic meter. The value 12 μg per cubic meter was chosen as the current annual
National Ambient Air Quality Standard (NAAQS) (e.g., the “safe” level) for PM2.5. BMI denotes body-mass index (the weight in kilograms
divided by the square of the height in meters) and ppb parts per billion.
†  The number for total person-years of follow-up indicates the sum of individual units of time that the persons in the study population were at
risk of death from 2000 through 2012.
‡  The average values for air pollution levels and for ecologic and meteorologic variables were computed by averaging values over all ZIP
Codes from 2000 through 2012.
§   Data on race and ethnic group were obtained from Medicare beneficiary files.
¶  The variable for ambulatory visits refers to the average annual percentage of Medicare enrollees who had at least one ambulatory visit to a
primary care physician.

2516

n engl j med 376;26 nejm.org  June 29, 2017

The New England Journal of Medicine
Downloaded from nejm.org at Harvard Library on July 23, 2018. For personal use only. No other uses without permission.
Copyright © 2017 Massachusetts Medical Society. All rights reserved.

 Air Pollution and Mortality in the Medicare Population

A Average Concentrations of PM2.5
14

12

µg/m3

10

8

6

4

B Average Warm-Season Concentrations of Ozone
60

55

ppb

50

45

40

35

Figure 1. Average PM2.5 and Ozone Concentrations in the Continental United States, 2000 through 2012.
Panel A shows the average concentrations of fine particulate matter (particles with a mass median aerodynamic
diameter of less than 2.5 μm [PM2.5]) in micrograms per cubic meter, as estimated on the basis of all daily predictions during the study period. Panel B shows the concentration of ozone levels in parts per billion as averaged from
April 1 through September 30 throughout the study period.

In a two-pollutant analysis, each increase of
10 μg per cubic meter in annual exposure to
PM2.5 (estimated independently of ozone) and
each increase of 10 ppb in warm-season exposure to ozone (estimated independently of PM2.5)
was associated with an increase in all-cause
mortality of 7.3% (95% confidence interval [CI],
7.1 to 7.5) and 1.1% (95% CI, 1.0 to 1.2), respecn engl j med 376;26

tively. Estimates of risk based on predictive, ZIPCode–specific assessments of exposure were
slightly higher than those provided by the nearest data-monitoring site (Table 2). When we restricted the PM2.5 and ozone analyses to locationyears with low concentrations, we continued to
see significant associations between exposure
and mortality (Table 2). Analysis of the MCBS

nejm.org

June 29, 2017

The New England Journal of Medicine
Downloaded from nejm.org at Harvard Library on July 23, 2018. For personal use only. No other uses without permission.
Copyright © 2017 Massachusetts Medical Society. All rights reserved.

2517

 The

n e w e ng l a n d j o u r na l

of

m e dic i n e

Table 2. Risk of Death Associated with an Increase of 10 μg per Cubic Meter in PM2.5 or an Increase of 10 ppb in Ozone
Concentration.*
Model

PM2.5

Ozone
hazard ratio (95% CI)

Two-pollutant analysis
Main analysis

1.073 (1.071–1.075)

1.011 (1.010–1.012)

Low-exposure analysis

1.136 (1.131–1.141)

1.010 (1.009–1.011)

Analysis based on data from nearest
monitoring site (nearest-monitor analysis)†

1.061 (1.059–1.063)

1.001 (1.000–1.002)

1.084 (1.081–1.086)

1.023 (1.022–1.024)

Single-pollutant analysis‡

*   Hazard ratios and 95% confidence intervals were calculated on the basis of an increase of 10 μg per cubic meter in exposure to PM2.5 and an increase of 10 ppb in exposure to ozone.
†  Daily average monitoring data on PM2.5 and ozone were obtained from the Environmental Protection Agency Air Quality
System. Daily ozone concentrations were averaged from April 1 through September 30 for the computation of warmseason averages. Data on PM2.5 and ozone levels were obtained from the nearest monitoring site within 50 km. If there
was more than one monitoring site within 50 km, the nearest site was chosen. Persons who lived more than 50 km
from a monitoring site were excluded.
‡  For the single-pollutant analysis, model specifications were the same as those used in the main analysis, except that
ozone was not included in the model when the main effect of PM2.5 was estimated and PM2.5 was not included in the
model when the main effect of ozone was estimated.

subsample provided strong evidence that smok- cubic meter and 30 ppb, respectively (Fig. 3, and
ing and income are not likely to be confounders Fig. S8 in the Supplementary Appendix).
because they do not have a significant association
with PM2.5 or ozone (Section 5 in the SupplemenDiscussion
tary Appendix).
This study involving an open cohort of all perSubgroup Analyses
sons receiving Medicare, including those from
Subgroup analyses revealed that men; black, small cities and rural areas, showed that longAsian, and Hispanic persons; and persons who term exposures to PM2.5 and ozone were associwere eligible for Medicaid (i.e., those who had ated with an increased risk of death, even at levels
low socioeconomic status) had a higher estimated below the current annual NAAQS for PM2.5. Furrisk of death from any cause in association with thermore, the study showed that black men and
PM2.5 exposure than the general population. The persons eligible to receive Medicaid had a much
risk of death associated with ozone exposure higher risk of death associated with exposure to
was higher among white, Medicaid-eligible per- air pollution than other subgroups. These findsons and was significantly below 1 in some ra- ings suggest that lowering the annual NAAQS
cial subgroups (Fig. 2). Among black persons, may produce important public health benefits
the effect estimate for PM2.5 was three times as overall, especially among self-identified racial
high as that for the overall population (Table S3 minorities and people with low income.
in the Supplementary Appendix). Overall, the risk
The strengths of this study include the asof death associated with ozone exposure was sessment of exposure with high spatial and
smaller and somewhat less robust than that as- temporal resolution, the use of a cohort of alsociated with PM2.5 exposure. We also detected a most 61 million Medicare beneficiaries across
small but significant interaction between ozone the entire continental United States followed for
exposure and PM2.5 exposure (Table S8 in the up to 13 consecutive years, and the ability to perSupplementary Appendix). Our thin-plate–spline form subgroup analyses of the health effects of
fit indicated a relationship between PM2.5, ozone, air pollution on groups of disadvantaged persons.
and all-cause mortality that was almost linear, However, Medicare claims do not include extenwith no signal of threshold down to 5 μg per sive individual-level data on behavioral risk fac-

2518

n engl j med 376;26 nejm.org  June 29, 2017

The New England Journal of Medicine
Downloaded from nejm.org at Harvard Library on July 23, 2018. For personal use only. No other uses without permission.
Copyright © 2017 Massachusetts Medical Society. All rights reserved.

 Air Pollution and Mortality in the Medicare Population

Hazard Ratio for Death per 10 µg/m3 Increase in PM2.5

A
1.25

1.20

1.15

1.10
Overall
population

1.05

1.00

Female

Male

Native
White
American

Hispanic

Asian

Black

Not Eligible Eligible for
for Medicaid Medicaid

Hazard Ratio for Death per 10 ppb Increase in Ozone

B
1.04

1.02
Overall
population
1.00

0.98

0.96

0.94

Female

Male

Native
White
American

Hispanic

Asian

Black

Not Eligible Eligible for
for Medicaid Medicaid

Figure 2. Risk of Death Associated with an Increase of 10 μg per Cubic Meter in PM2.5 Concentrations and an Increase
of 10 ppb in Ozone Exposure, According to Study Subgroups.
Hazard ratios and 95% confidence intervals are shown for an increase of 10 μg per cubic meter in PM2.5 and an increase of 10 parts per billion (ppb) in ozone. Subgroup analyses were conducted by first restricting the population
(e.g., considering only male enrollees). The same two-pollutant analysis (the main analysis) was then applied to each
subgroup. Numeric results are presented in Tables S3 and S4 in the Supplementary Appendix. Dashed lines indicate
the estimated hazard ratio for the overall population.

tors, such as smoking and income, which could
be important confounders. Still, our analysis of
the MCBS subsample (Table S6 in the Supplementary Appendix) increased our level of confidence
that the inability to adjust for these individuallevel risk factors in the Medicare cohort did not
lead to biased results (Section 5 in the Supplementary Appendix). In another study, we analyzed a

similar Medicare subsample with detailed individual-level data on smoking, BMI, and many
other potential confounders linked to Medicare
claims.23 In that analysis, we found that for mortality and hospitalization, the risks of exposure
to PM2.5 were not sensitive to the additional
control of individual-level variables that were not
available in the whole Medicare population.

n engl j med 376;26 nejm.org  June 29, 2017

The New England Journal of Medicine
Downloaded from nejm.org at Harvard Library on July 23, 2018. For personal use only. No other uses without permission.
Copyright © 2017 Massachusetts Medical Society. All rights reserved.

2519

 The

n e w e ng l a n d j o u r na l

A Exposure to PM2.5
1.10

Hazard Ratio

1.08
1.06
1.04
1.02
1.00
6

8

10

PM2.5

12

14

16

(µg/m3)

B Exposure to Ozone
1.020

Hazard Ratio

1.015
1.010
1.005
1.000
0.995
30

35

40

45

50

55

60

Ozone (ppb)

Figure 3. Concentration–Response Function of the Joint Effects of Exposure
to PM2.5 and Ozone on All-Cause Mortality.
A log-linear model with a thin-plate spline was fit for both PM2.5 and ozone,
and the shape of the concentration-response surface was estimated (Fig. S8
in the Supplementary Appendix). The concentration–response curve in
Panel A was plotted for an ozone concentration equal to 45 ppb. The concentration–response curve in Panel B was plotted for a PM2.5 concentration equal to 10 μg per cubic meter. These estimated curves were plotted
at the 5th and 95th percentiles of the concentrations of PM2.5 and ozone,
respectively. The complete concentration–response three-dimensional surface is plotted in Fig. S8 in the Supplementary Appendix.

We also found that our results were robust
when we excluded individual and ecologic covariates from the main analysis (Fig. S2 and
Table S2 in the Supplementary Appendix), when
we stratified age at entry into 3-year and 4-year
categories rather than the 5 years used in the
main analysis (Fig. S3 in the Supplementary Appendix), when we varied the estimation procedure (by means of a generalized estimating

2520

of

m e dic i n e

equation as opposed to mixed effects) (Tables S3
and S4 in the Supplementary Appendix), and
when we used different types of statistical software (R, version 3.3.2, vs. SAS, version 9.4). Finally, we found that our results were consistent
with others published in the literature (Section 6
in the Supplementary Appendix).5,17,24-28
There was a significant association between
PM2.5 exposure and mortality when the analysis
was restricted to concentrations below 12 μg per
cubic meter, with a steeper slope below that
level. This association indicated that the healthbenefit-per-unit decrease in the concentration of
PM2.5 is larger for PM2.5 concentrations that are
below the current annual NAAQS than the health
benefit of decreases in PM2.5 concentrations that
are above that level. Similar, steeper concentration-response curves at low concentrations have
been observed in previous studies.29 Moreover,
we found no evidence of a threshold value — the
concentration at which PM2.5 exposure does not
affect mortality — at concentrations as low as
approximately 5 μg per cubic meter (Fig. 3); this
finding is similar to those of other studies.18,30
The current ozone standard for daily exposure is 70 ppb; there is no annual or seasonal
standard. Our results strengthen the argument
for establishing seasonal or annual standards.
Moreover, whereas time-series studies have shown
the short-term effects of ozone exposure, our
results indicate that there are larger effect sizes
for longer-term ozone exposure, including in locations where ozone concentrations never exceed
70 ppb. Unlike the American Cancer Society
Cancer Prevention Study II,9,10 our study reported
a linear connection between ozone concentration
and mortality. This finding is probably the result
of the interaction between PM2.5 and ozone (Section 7 in the Supplementary Appendix). The significant, linear relationship between seasonal
ozone levels and all-cause mortality indicates
that current risk assessments,31-33 which incorporate only the acute effects of ozone exposure on
deaths each day from respiratory mortality, may
be substantially underestimating the contribution
of ozone exposure to the total burden of disease.
The enormous sample size in this study, which
includes the entire Medicare cohort, allowed for
unprecedented accuracy in the estimation of risks
among racial minorities and disadvantaged subgroups. The estimate of effect size for PM2.5 expo-

n engl j med 376;26 nejm.org  June 29, 2017

The New England Journal of Medicine
Downloaded from nejm.org at Harvard Library on July 23, 2018. For personal use only. No other uses without permission.
Copyright © 2017 Massachusetts Medical Society. All rights reserved.

 Air Pollution and Mortality in the Medicare Population

sure was greatest among male, black, and Medicaid-eligible persons. We also estimated risks in
subgroups of persons who were eligible for Medicaid and in whites and blacks alone to ascertain
whether the effect modifications according to
race and Medicaid status were independent. We
found that black persons who were not eligible
for Medicaid (e.g., because of higher income)
continued to have an increased risk of death
from exposure to PM2.5 (Fig. S4 in the Supplementary Appendix). In addition, we found that
there was a difference in the health effects of
PM2.5 exposure between urban and rural populations, a finding that may be due to compositional differences in the particulates (Table S3
Supplementary Appendix).
Although the Medicare cohort includes only
the population of persons 65 years of age or older,
two thirds of all deaths in the United States occur
in people in that age group. Although our exposure models had excellent out-of-sample predictive power on held-out monitors, they do have
limitations. Error in exposure assessment remains
an issue in this type of analysis and could attenuate effect estimates for air pollution.34
The overall association between air pollution
and human health has been well documented
References
1. Ambient (outdoor) air quality and
health. Fact sheet no. 313. Updated March
2014. Geneva:​World Health Organization,
2015.
2. Brook RD, Rajagopalan S, Pope CA
III, et al. Particulate matter air pollution
and cardiovascular disease: an update to
the scientific statement from the American Heart Association. Circulation 2010;
121:​2331-78.
3. Lim SS, Vos T, Flaxman AD, et al. A
comparative risk assessment of burden of
disease and injury attributable to 67 risk
factors and risk factor clusters in 21 regions, 1990-2010: a systematic analysis for
the Global Burden of Disease Study 2010.
Lancet 2012;​380:​2224-60.
4. Crouse DL, Peters PA, Hystad P, et al.
Ambient PM2.5, O3, and NO2₂ exposures and
associations with mortality over 16 years of
follow-up in the Canadian Census Health
and Environment Cohort (CanCHEC). Environ Health Perspect 2015;​123:​1180-6.
5.  Wang Y, Kloog I, Coull BA, Kosheleva
A, Zanobetti A, Schwartz JD. Estimating
causal effects of long-term PM2.5 exposure on mortality in New Jersey. Environ
Health Perspect 2016;​124:​1182-8.
6.  Beelen R, Raaschou-Nielsen O, Stafog-

since the publication of the landmark Harvard
Six Cities Study in 1993.25 With air pollution
declining, it is critical to estimate the health effects of low levels of air pollution — below the
current NAAQS — to determine whether these
levels are adequate to minimize the risk of death.
Since the Clean Air Act requires the EPA to set
air-quality standards that protect sensitive populations, it is also important to focus more effort
on estimating effect sizes in potentially sensitive
populations in order to inform regulatory policy
going forward.
The views expressed in this article are those of the authors
and do not necessarily represent the official views of the funding agencies. Furthermore, these agencies do not endorse the
purchase of any commercial products or services related to this
publication.
Supported by grants from the Health Effects Institute (4953RFA14-3/16-4), the National Institutes of Health (R01 ES02433201A1, ES-000002, ES024012, R01ES026217), the National Cancer Institute (R35CA197449), and the Environmental Protection
Agency (83587201-0 and RD-83479801).
No potential conflict of interest relevant to this article was
reported.
Disclosure forms provided by the authors are available with
the full text of this article at NEJM.org.
We thank Stacey C. Tobin, Ph.D., for editorial assistance on an
earlier version of the manuscript, Sarah L. Duncan and William J.
Horka for their support with the Research Computing Environment, and Ista Zahn at the Institute for Quantitative Social Science, Harvard University, for SAS programming support.

gia M, et al. Effects of long-term exposure
to air pollution on natural-cause mortality: an analysis of 22 European cohorts
within the multicentre ESCAPE project.
Lancet 2014;​383:​785-95.
7. Atkinson RW, Butland BK, Dimitroulopoulou C, et al. Long-term exposure to
ambient ozone and mortality: a quantitative systematic review and meta-analysis of
evidence from cohort studies. BMJ Open
2016;​6(2):​e009493.
8.  Hao Y, Balluz L, Strosnider H, Wen XJ,
Li C, Qualters JR. Ozone, fine particulate
matter, and chronic lower respiratory disease mortality in the United States. Am J
Respir Crit Care Med 2015;​192:​337-41.
9.  Turner MC, Jerrett M, Pope CA III, et al.
Long-term ozone exposure and mortality
in a large prospective study. Am J Respir
Crit Care Med 2016;​193:​1134-42.
10.  Jerrett M, Burnett RT, Pope CA III, et al.
Long-term ozone exposure and mortality.
N Engl J Med 2009;​360:​1085-95.
11.  Krewski D, Jerrett M, Burnett RT, et al.
Extended follow-up and spatial analysis of
the American Cancer Society study linking
particulate air pollution and mortality.
Res Rep Health Eff Inst 2009;​140:​5-114,
discussion 115-136.

12. Carey IM, Atkinson RW, Kent AJ, van
Staa T, Cook DG, Anderson HR. Mortality
associations with long-term exposure to
outdoor air pollution in a national English
cohort. Am J Respir Crit Care Med 2013;
187:​1226-33.
13. Ostro B, Hu J, Goldberg D, et al. Associations of mortality with long-term
exposures to fine and ultrafine particles,
species and sources: results from the California Teachers Study Cohort. Environ
Health Perspect 2015;​123:​549-56.
14.  Crouse DL, Peters PA, van Donkelaar A,
et al. Risk of nonaccidental and cardiovascular mortality in relation to long-term
exposure to low concentrations of fine
particulate matter: a Canadian nationallevel cohort study. Environ Health Perspect 2012;​120:​708-14.
15.  Wang Y, Shi L, Lee M, et al. Long-term
exposure to PM2.5 and mortality among
older adults in the southeastern US. Epidemiology 2017;​28:​207-14.
16.  Thurston GD, Ahn J, Cromar KR, et al.
Ambient particulate matter air pollution
exposure and mortality in the NIH-AARP
Diet and Health cohort. Environ Health
Perspect 2016;​124:​484-90.
17.  Pinault L, Tjepkema M, Crouse DL, et al.

n engl j med 376;26 nejm.org  June 29, 2017

The New England Journal of Medicine
Downloaded from nejm.org at Harvard Library on July 23, 2018. For personal use only. No other uses without permission.
Copyright © 2017 Massachusetts Medical Society. All rights reserved.

2521

 Air Pollution and Mortality in the Medicare Population

Risk estimates of mortality attributed to
low concentrations of ambient fine particulate matter in the Canadian Community Health Survey cohort. Environ Health
2016;​15:​18.
18.  Shi L, Zanobetti A, Kloog I, et al. Lowconcentration PM2.5 and mortality: estimating acute and chronic effects in a
population-based study. Environ Health
Perspect 2016;​124:​46-52.
19. Di Q, Kloog I, Koutrakis P, Lyapustin
A, Wang Y, Schwartz J. Assessing PM2.5
exposures with high spatiotemporal resolution across the continental United States.
Environ Sci Technol 2016;​50:​4712-21.
20. Di Q, Rowland S, Koutrakis P,
Schwartz J. A hybrid model for spatially
and temporally resolved ozone exposures
in the continental United States. J Air
Waste Manag Assoc 2017;​67:​39-52.
21.  Kalnay E, Kanamitsu M, Kistler R, et al.
The NCEP/NCAR 40-Year Reanalysis Project. Bull Am Meteorol Soc 1996;​77:​437-71.
22.  Lee EW, Wei L, Amato DA, Leurgans S.
Cox-type regression analysis for large
numbers of small groups of correlated
failure time observations. In: Klein JP,
Goel PK, eds. Survival analysis: state of
the art. Berlin:​Springer, 1992:​237-47.
23. Makar M, Antonelli J, Di Q, Cutler D,
Schwartz J, Dominici F. Estimating the
causal effect of low levels of fine particulate matter on hospitalization. Epidemiology, May 25, 2016 (http://journals​.lww​.com/

epidem/​Abstract/​publishahead/​Estimating
_the_Causal_Effect_of_Low_Levels_of_
Fine​.98844​.aspx).
24. Kioumourtzoglou MA, Schwartz J,
James P, Dominici F, Zanobetti A. PM2.5
and mortality in 207 US cities: modification by temperature and city characteristics.
Epidemiology 2016;​27:​221-7.
25. Dockery DW, Pope CA III, Xu X, et al.
An association between air pollution and
mortality in six U.S. cities. N Engl J Med
1993;​329:​1753-9.
26. Lepeule J, Laden F, Dockery D,
Schwartz J. Chronic exposure to fine particles and mortality: an extended followup of the Harvard Six Cities study from
1974 to 2009. Environ Health Perspect
2012;​120:​965-70.
27.  Pope CA III, Burnett RT, Thurston GD,
et al. Cardiovascular mortality and longterm exposure to particulate air pollution:
epidemiological evidence of general pathophysiological pathways of disease. Circulation 2004;​109:​71-7.
28. Eftim SE, Samet JM, Janes H, McDermott A, Dominici F. Fine particulate matter and mortality: a comparison of the six
cities and American Cancer Society cohorts
with a Medicare cohort. Epidemiology
2008;​19:​209-16.
29. Pope CA III, Burnett RT, Krewski D,
et al. Cardiovascular mortality and exposure to airborne fine particulate matter
and cigarette smoke: shape of the expo-

sure-response relationship. Circulation
2009;​120:​941-8.
30. Schwartz J, Coull B, Laden F, Ryan L.
The effect of dose and timing of dose on
the association between airborne particles and survival. Environ Health Perspect
2008;​116:​64-9.
31. Smith RL, Xu B, Switzer P. Reassessing the relationship between ozone and
short-term mortality in U.S. urban communities. Inhal Toxicol 2009;​21:​Suppl 2:
37-61.
32.  Zanobetti A, Schwartz J. Mortality displacement in the association of ozone
with mortality: an analysis of 48 cities in
the United States. Am J Respir Crit Care
Med 2008;​177:​184-9.
33. Regulatory impact analysis of the final revisions to the National Ambient Air
Quality Standards for ground-level ozone.
Research Triangle Park, NC:​Environmental Protection Agency, 2015 (https:/​/​w ww
.epa​.gov/​naaqs/​regulatory-impact-analysis
-final-revisions-national-ambient-air
-quality-standards-ground-level).
34.  Spiegelman D. Evaluating public health
interventions. 4. The Nurses’ Health Study
and methods for eliminating bias attributable to measurement error and misclassification. Am J Public Health 2016;​106:
1563-6.
Copyright © 2017 Massachusetts Medical Society.

ARTICLE METRICS NOW AVAILABLE
Visit the article page at NEJM.org and click on the Metrics tab to view
comprehensive and cumulative article metrics compiled from multiple sources,
including Altmetrics. Learn more at www.nejm.org/page/article-metrics-faq.

2522

n engl j med 376;26 nejm.org  June 29, 2017

The New England Journal of Medicine
Downloaded from nejm.org at Harvard Library on July 23, 2018. For personal use only. No other uses without permission.
Copyright © 2017 Massachusetts Medical Society. All rights reserved.

 6 Everett Street, Suite 4119
Cambridge, MA 02138
617.496.2058 (tel.)
617.384.7633 (fax)

Comments re: Docket ID No. EPA-HQ-OA-2018-0259
August 7, 2018
ATTACHMENT 3
Partial List of Studies That May Contain Protected Health Information and
That Have Been Relied on by the Environmental Protection Agency (EPA)
and Cited in EPA Documents
The following studies were cited in supporting or decision making EPA documents.
Safe Drinking Water Act (SDWA)
Six Year Review #1 Health Effects Technical Support Document
• Barrett JH, Parslow RC, McKinney PA, et al. 1998. Nitrate in drinking water and
the incidence of gastric, esophageal, and brain cancer in Yorkshire, England.
Cancer Causes Control. 9:153-159.
• Croen LA, Todoroff K, Shaw GM. 1997. Maternal dietary nitrate exposure and
risk for neural tube defects. Am J Epidemiol. 145:S30.
• Van Loon AJ, Botterweck AA, Goldbohm RA, et al. 1998. Intake of nitrate and
nitrite and the risk of gastric cancer: A prospective cohort study. Br J Cancer.
78:129-135.
• Ward MH, Mark SD, Cantor KP, et al. 1996. Drinking water nitrate and the risk of
non-Hodgkin’s lymphoma. Epidemiology. 7:465-471.
• Weyer PJ, Cerhan JR, Cross BC, et al. 2001. Municipal drinking water nitrate
level and cancer risk in older women: the Iowa Women’s Health Study.
Epidemiology. 12(3):327-338.
Six Year Review #2 Health Effects Technical Support Document
• Moertel, CG et al. 1982. A clinical trial of amygdalin (laetrile) in the treatment of
human cancer. New England J. Med. 306: 201-206.
• Rothman, N; GL Li; M Dosemeci; et al. 1996. Hematotoxicity among Chinese
workers heavily exposed to benzene. Am. J. Ind. Med. 29: 236-246.
• Tajtakova, M; Z Semanova; et al. 2006. Increased thyroid volume and frequency
of thyroid disorders signs in schoolchildren from nitrate polluted area.
Chemosphere. 62(4): 559-564.
• Tseng, WP. 1977. Effects and dose-response relationships of skin cancer and
blackfoot disease with arsenic. Environ. Health Perspect. 19: 109-119.
• Tseng, WP; HM Chu; SW How; et al. 1968. Prevalence of skin cancer in an
endemic area of chronic arsenicism in Taiwan. J. Natl. Cancer Inst. 40: 453-463.
• Wones, RG; BL Stadler; and LA Frohman. 1990. Lack of effect of drinking water
barium on cardiovascular risk factor. Environ. Health. Perspect. 85: 355-9.

 •

Yang, G; S Wang; R Zhou; and S Sun. 1983. Endemic selenium intoxication of
humans in china. American J. Clin. Nutr. 37:351-357.

Six Year Review #3 Health Effects Technical Support Document
• Baccarelli, A; SM Giacomini; C Corbetta; et al. 2008. Neonatal thyroid function in
Seveso 25 years after maternal exposure to dioxin. PLoS Med. 5(7): e161.
• Ciesielski, T; J Weuve; DC Bellinger; J Schwartz; B Lanphear; and RO Wright.
Cadmium exposure and neurodevelopmental outcomes in U.S. children. Environ
Health Perspect. 2012 May;120(5):758-63.
• Mocarelli, P; PM Gerthoux; DG Patterson, Jr; et al. 2008. Dioxin exposure, from
infancy through puberty, produces endocrine disruption and affects human
semen quality. Environ Health Perspect. 116(1): 70-77.
• Nawrot, TS; DS Martens; A Hara; M Plusquin; J Vangronsveld; HA Roels; and JA
Staessen. 2015. Association of total cancer and lung cancer with environmental
exposure to cadmium: the meta-analytical evidence. Cancer Causes Control.
26(9):1281-8.
• Walton, G. 1951. Survey of literature relating to infant methemoglobinemia due to
nitratecontaminated water. Am. J. Public Health. 41(8 Pt 1): 986-996.
• Wones, RG; BL Stadler; and LA Frohman. 1990. Lack of effect of drinking water
barium on cardiovascular risk factor. Environ. Health. Perspect. 85: 355-9.
• Yang GQ; et al. 1983. Endemic selenium intoxication of humans in China. Amer
J Clinic Nutr. 37: 872-881.
• Bassin, E.B., Wypij D., Davis R.B., Mittleman M.A. 2006. “Age-specific Fluoride
Exposure in Drinking Water and Osteosarcoma.” Cancer Causes and Control.
17: 421-8.
• Broadbent, Jonathan M., W. Murray Thomson, Sandhya Ramrakha, Terrie E.
Moffitt, Jiaxu Zeng, Lyndie A. Foster Page, and Richie Poulton. 2015. Community
Water Fluoridation and Intelligence: Prospective Study in New Zealand.
American Journal of Public Health. 105.1 (2015): 72-76.
• Grimes, D.R. 2015. Commentary on are fluoride levels in drinking water
associated with hypothyroidism prevalence in England? A large observational
study of GP practice data and fluoride levels in drinking water. J Epidemiol
Community Health. 69(7): 616.
• Malin, Ashley J., and Christine Till. 2015. Exposure to Fluoridated Water and
Attention Deficit Hyperactivity Disorder Prevalence among Children and
Adolescents in the United States: An Ecological Association. Environmental
Health. 14:17.
• Larsson, SC; N Orsini; and A Wolk. 2015b. Urinary cadmium concentration and
risk of breast cancer: a systematic review and dose-response meta-analysis. Am
J Epidemiol. 182(5):375-80.
Contaminant Candidate List Examples
Boron Health Effects Support Document
• Baker, M.D. and S.C. Bogema. 1986. Ingestion of boric acid by infants. Am. J.
Emerg. Med. 4(4):358-361 (as cited in U.S. EPA, 2004a).
Page 2 of 9

 •

•

•

•

•
•

•

•

Culver, B.D., P.T. Shen, T.H. Taylor, et al. 1994. The relationship of blood- and
urine-boron to boron exposure in borax-workers and the usefulness of urineboron as an exposure marker. Environ. Health Perspect. 102(Suppl. 7):133-137
(as cited in U.S. EPA, 2004a).
Friis-Hansen, B., B. Aggerbeck, and J.A. Jansen. 1982. Unaffected blood boron
levels in newborn infants treated with a boric acid ointment. Food Chem. Toxicol.
20:451-454 (as cited in U.S. EPA, 2004a).
Naghii, M.R. and S. Samman. 1997. The effect of boron supplementation on its
urinary excretion and selected cardiovascular risk factors in healthy male
subjects. Biol. Trace Element Res. 56:273-286 (as cited in U.S. EPA, 2004a).
Nielsen, F.H. 1994. Biochemical and physiologic consequences of boron
deprivation in humans. Environ. Health Perspect. 102(Suppl. 7):59-63 (as cited in
U.S. EPA, 2004a).
Rainey C.J., L.A. Nyquist, R.E. Christensen, et al. 1999. Daily boron intake from
the American diet. J. Am. Diet Assoc. 99(3):335-40.
Usuda, K., K. Kono, K. Nishiuraet al. 1997. Boron diffusion across the dialysis
membrane during hemodialysis. Miner Electrolyte Metab. 23(2):100-104 (as cited
in U.S. EPA, 2004a).
Whorton, D., J. Haas, and L. Trent. 1994a. Reproductive effects of inorganic
borates on male employees: Birth rate assessment. Environ. Health Perspect.
102(Suppl. 7):129-131 (as cited in U.S. EPA, 2004a).
Whorton, D., J. Haas, and L. Trent, et al. 1994b. Reproductive effects of sodium
borates on male employees: birth rate assessment. Occup. Environ. Med.
51:761-767 (as cited in U.S. EPA, 2004a).

Perfluorooctanoic Acid Health Effects Support Document
• Andersen, C.S., C. Fei, M. Gamborg, E.A. Nohr, T.I.A. Sørensen, and J. Olsen.
2010. Prenatal exposures to perfluorinated chemicals and anthropometric
measures in infancy. American Journal of Epidemiology. 172:1230–1237.
• Apelberg, B.J., F.R. Witter, J.B. Herbstman, A.M. Calafat, R.U. Halden, L.L.
Needham, and L.R. Goldman. 2007. Cord serum concentrations of
perfluorooctane sulfonate (PFOS) and perfluorooctanoate (PFOA) in relation to
weight and size at birth. Environmental Health Perspectives. 115:1670–1676.
• Barry, V., A. Winquist, and K. Steenland. 2013. Perfluorooctanoic acid (PFOA)
exposures and incident cancers among adults living near a chemical plant.
Environmental Health Perspectives. 121:1313–1318.
• Barry, V., L.A. Darrow, M. Klein, A. Winquist, and K. Steenland. 2014. Early life
perfluorooctanoic acid (PFOA) exposure and overweight and obesity risk in
adulthood in a community with elevated exposure. Environmental Research.
132:62–69.
• Bloom, M.S., K. Kannan, H.M. Spiethoff, L. Tao, K.M. Aldous, and J.E. Vena.
2010. Exploratory assessment of perfluorinated compounds and human thyroid
function. Physiology & Behavior. 99:240–245.
• Bonefeld-Jørgensen, E.C., M. Long, S.O. Fredslund, R. Bossi, and J. Olsen.
2014. Breast cancer risk after exposure to perfluorinated compounds in Danish

Page 3 of 9

 •

•

•

•

•

•

•

•

•

•

•

•

women: a case-control study nested in the Danish National Birth Cohort. Cancer
Causes & Control. 25(11):1439–1448.
Buck Louis, G.M., Z. Chen, E.F. Schisterman, S. Kim, A.M. Sweeney, R.
Sundaram, C.D. Lynch, R.E. Gore-Langton, and D.B. Barr. 2015.
Perfluorochemicals and human semen quality: The LIFE study. Environmental
Health Perspectives. 123(1):57–63.
Chan, E., I. Burstyn, N. Cherry, F. Bamforth, and J.W. Martin. 2011.
Perfluorinated acids and hypothyroxinemia in pregnant women. Environmental
Research. 111:559–564.
Chang, E.T., H. Adami, P. Boffetta, C. Cole, T.B. Starr, and J.S. Mandel. 2014. A
critical review of perfluorooctanoate and prefluorooctanesulfonate exposure and
cancer risk in humans. Critical Reviews in Toxicology. 44(51):1–81.
Chen, M.-H., E.-H. Ha, T.-W. Wen, Y.-N. Su, G.-W. Lien, C.-Y. Chen, P.-C.
Chen, and W.-S. Hsieh. 2012. Perfluorinated compounds in umbilical cord blood
and adverse birth outcomes. PLoS One. 7(8):e42474.
Darrow, L.A., C.R. Stein, and K. Steenland. 2013. Serum perfluorooctanoic acid
and perfluorooctane sulfonate concentrations in relation to birth outcomes in the
Mid-Ohio Valley, 2005-2010. Environmental Health Perspectives. 121:1207–
1213.
de Cock, M., M.R. de Boer, M. Lamoree, J. Legler, and M. van de Bor. 2014.
Prenatal exposure to endocrine disrupting chemicals in relation to thyroid
hormone levels in infants – a Dutch prospective cohort study. Environmental
Health. 13:106.
Eriksen, K.T., M. Sørensen, J.K. McLaughlin, L. Lipworth, A. Tjønneland, K.
Overvad, and O. Raaschou-Nielsen. 2009. Perfluorooctanoate and
perfluorooctanesulfonate plasma levels and risk of cancer in the general Danish
population. Journal of the National Cancer Institute. 101:605–609.
Eriksen, K.T., O. Raaschou-Nielsen, J.K. McLaughlin, L. Lipworth, A. Tjønneland,
K. Overvad, and M. Sørensen. 2013. Association between plasma PFOA and
PFOS levels and total cholesterol in a middle-aged Danish population. PLoS
ONE. 8:e56969.
Fei, C., J.K. McLaughlin, L. Lipworth, and J. Olsen. 2008b. Prenatal exposure to
perfluorooctanoate (PFOA) and perfluorooctanesulfonate (PFOS) and maternally
reported developmental milestones in infancy. Environmental Health
Perspectives. 116:1391–1395.
Fei, C., J.K. McLaughlin, R.E. Tarone, and J. Olsen. 2008a. Fetal growth
indicators and perfluorinated chemicals: a study in the Danish National Birth
Cohort. American Journal of Epidemiology. 168:66–72.
Fei, C., J.K. McLaughlin, L. Lipworth, and J. Olsen. 2010b. Maternal
concentrations of perfluorooctanesulfate (PFOA) and perfluorooctanoate (PFOA)
and duration of breastfeeding. Scandinavian Journal of Work, Environment &
Health. 36:413–421.
Gallo, V., G. Leonardi, B. Genser, M.-J. Lopez-Espinosa, S.J. Frisbee, L.
Karlsson, A.M. Ducatman, and T. Fletcher. 2012. Serum perfluorooctanoate
(PFOA) and perfluorooctane sulfonate (PFOS) concentrations and liver function

Page 4 of 9

 •

•

•

•

•

biomarkers in a population with elevated PFOA exposure. Environmental Health
Perspectives. 120(5):655–660.
Geiger, S.D., J. Xiao, A. Ducatmen, S. Frisbee, K. Innes, and A. Shankar. 2014a.
The association between PFOA, PFOS and serum lipid levels in adolescents.
Chemosphere. 98:78–83.
Gilliland, F.D., and J.S. Mandel. 1993. Mortality among employees of a
perfluorooctanoic acid production plant. Journal of Occupational Medicine.
35:950–954.
Leonard, R.C., K.H. Kreckmann, C.J. Sakr, and J.M. Symons. 2008.
Retrospective cohort mortality study of workers in a polymer production plant
including a reference population of regional workers. Annals of Epidemiology.
18:15–22.
Liew, Z., B. Ritz, E.C. Bonefeld-Jørgensen, T.B. Henriksen, E.A. Nohr, B.H.
Bech, C. Fei, R. Bossi, O.S. von Ehrenstein, E. Streja, P. Uldall, and J. Olsen.
2014. Prenatal exposure to perfluoroalkyl substances and the risk of congenital
cerebral palsy in children. American Journal of Epidemiology. 180:574–581.
Lopez-Espinosa, M.-J., T. Fletcher, B. Armstron, B. Genser, K. Dhatariya, D.
Mondal, A. Ducatman, and G. Leonardi. 2011. Association of perfluorooctanoic
acid (PFOA) and perfluorooctane sulfonate (PFOS) with age of puberty among
children living near a chemical plant. Environmental Science & Technology.
45(19):8160–816.

Cyanobacterial Toxin Health Effects Support Document
• Carmichael, W. W., Azevedo, S. M. F. O. and An, J.S. 2001. Human fatalities
from cyanobacteria: Chemical and biological evidence for cyanotoxins.
Environmental Health Perspectives. 109(7): 663-668
Naphthalene Health Effects Support Document
• Anziulewicz, J.A., H.J. Dick and E.E. Chiarulli. 1959. Transplacental naphthalene
poisoning. Am. J. Obstet. Gynecol. 78:519-521 (as cited in ATSDR, 1995).
• Athanasiou, M., C. Tsantali, M. Trachana, et al. 1997. Hemolytic anemia in a
female newborn infant whose mother inhaled naphthalene before delivery. J.
Pediatr. 130:680-681.
• Dreisbach, R.H. and W.O. Robertson. 1987. Handbook of poisoning: prevention,
diagnosis and treatment, 12th ed. Norwalk, CT. Appleton and Lange. p. 194 ( as
cited in U.S. EPA, 1990).
• Gerarde, H.W., ed. 1960. Naphthalene. In: Toxicology and biochemistry of
aromatic hydrocarbons. Amsterdam: Elsevier. pp. 225-231 (as cited in U.S. EPA,
1998a).
• Ghetti, G. and L. Mariani. 1956. [Alterazioni oculari da naftalina]. Med. Lavoro.
47(10):533- 538. (original in Italian) (as cited in U.S. EPA, 1998a).
• Gidron, E. and J. Leurer. 1956. Naphthalene poisoning. Lancet. 4:228-230 (as
cited in ATSDR, 1995).
• Gupta, R., P.C. Singhal, M.A. Muthusethupathy, et al. 1979. Cerebral oedema
and renal failure following naphthalene poisoning. J. Assoc. Phys. 27:347-348
(as cited in ATSDR, 1995)
Page 5 of 9

 •

•
•
•
•
•

•

•

Ijiri, I., K. Shimosata, M. Omae, et al. 1987. A case report of death from
naphthalene poisoning. Japan J. Legal Med. 41(1):52-55 (as cited in U.S. EPA
1998a).
Kup, W. 1978. [Work-related origin of cancer in the nose, mouth, and larynx].
Akad. Wiss. 2:20-25 (original in German) (as cited in U.S. EPA, 1998a).
Kurz, J.M. 1987. Naphthalene poisoning: critical care nursing techniques.
Dimens. Crit. Care Nurs. 6:264-270 (as cited in ATSDR, 1995).
Schafer, W.B. 1951. Acute hemolytic anemia related to naphthalene: Report of a
case in a newborn infant. Pediatrics. 7:172-174 (as cited in ATSDR, 1995).
Valaes, T., S.A. Doxiadis and P. Fessas. 1963. Acute hemolysis due to
naphthalene inhalation. J. Pediatr. 63:904-915 (as cited in ATSDR, 1995).
Wolf, O. 1976. [Cancer diseases in chemical workers in a former naphthalene
cleaning plant]. Deutsch. Gesundheitwes. 31:996-999 (original in German) (as
cited in U.S. EPA, 1998a).
Zinkham, W.H. and B. Childs. 1957. Effect of vitamin K and naphthalene
metabolites on glutathione metabolism of erythrocytes from normal newborns
and patients with naphthalene hemolytic anemia. Am. J. Dis. Child. 94:420-423
(as cited in ATSDR, 1995).
Zinkham, W.H. and B. Childs. 1958. A defect of glutathione metabolism of
erythrocytes from patients with naphthalene-induced hemolytic anemia.
Pediatrics. 22:461-471 (as cited in ATSDR, 1995).

Interim Drinking Water Health Advisory for Perchlorates
• Chan, S. and M. D. Kilby. 2000. Thyroid hormone and central nervous system
development. J Endocrinol. 165(1): 1-8.
• Glinoer, D. 2007. Clinical and biological consequences of iodine deficiency
during pregnancy. Endocr Dev. 10: 62-85.
• Delange, F. 2004. Optimal iodine during pregnancy, lactation and the neonatal
period. International Journal of Endocrinology and Metabolism. 3:1-12.
• Rovet, J.F., 2002. Congenital hypothyroidism: an analysis of persisting deficits
and associated factors. Child Neuropsychology. Vol. 8, No. 3. pp. 150-162.
• Zoeller, R.T., and J. Rovet. 2004. Timing of thyroid hormone action in the
developing brain: clinical observations and experimental findings. J
Neuroendocrinology. 16: 809-18.
• Kooistra, L., S. Crawford, A.L. van Baar, E.P. Brouwers, and V.J. Pop. 2006.
Neonatal effects of maternal hypothyroxinemia during early pregnancy.
Pediatrics. 117; 161-167.
• Haddow, J.E., G.E. Palomaki, et al. 1999. Maternal thyroid deficiency during
pregnancy and subsequent neuropsychological development of the child. New
England Journal of Medicine. 341(8): 549-55.
• Kooistra, L., S. Crawford, A.L. van Baar, E.P. Brouwers, and V.J. Pop. 2006.
Neonatal effects of maternal hypothyroxinemia during early pregnancy.
Pediatrics. 117; 161-167.
• Auso E., R. Lavado-Autric, E. Cuevas, F.E. Del Rey, G, Morreale De Escobar,
and P. Berbel. 2004. A moderate and transient deficiency of maternal thyroid
Page 6 of 9

 •

•
•

function at the beginning of fetal neocorticogenesis alters neuronal migration.
Endocrinology. 145: 4037-47.
Blount, B.C., J.L. Pirkle, J.D. Osterloh, L. Valentín-Blasini, and K.L. Caldwell.
2006. Urinary perchlorate and thyroid hormone levels in adolescent and adult
men and women living in the United States. Environmental Health Perspectives.
Vol. 114, No. 12. pp. 1865–1871.
Blount B.C., Valentin-Blasini L., Osterloh J.D., Mauldin J.P., and Pirkle J.L.
Perchlorate exposure of the US population, 2001–2002. J Expo Sci Environ
Epidemiol. 2007: 17: 400–407.
Steinmaus, C., M.D. Miller, R. Howd. 2007. Impact of smoking and thiocyanate
on perchlorate and thyroid hormone associations in the 2001-2002 National
Health and Nutrition Examination Survey. Environ Health Perspect. 115(9):13338.

Additional Documents
Public Health Implications of PCBs (2015)
• Bertazzi PA, Riboldi L, Persatori A, Radice L, Zocchetti C. 1987. Cancer mortality
of capacitor manufacturing workers. Am J Ind Med 11:165-76.
• Chao WY, Hsu CC, Guo GL. 1997. Middle-ear disease in children exposed
prenatally to polychlorinated biphenyls and polychlorinated dibenzofurans. Arch
Environ Health. 52:257-62.
• Chen Y-CJ, Guo Y-L, Hsu C-C, et al. 1992. Cognitive-development of Yu-cheng
(oil disease) children prenatally exposed to heat-degraded PCBs. JAMA.
268:3213-8.
• Cogliano VJ. 1998. Assessing the cancer risk from environmental PCBs. Environ
Health Perspect. 106(6):317-323.
• Fein GG, Jacobson JL, Jacobson SW, et al. 1984. Prenatal exposure to
polychlorinated biphenyls: effects on birth size and gestational age. J Pediatr.
105:315-20.
• Gerstenberger SL, Tarvis OR, Hansen LK, Pratt-Shelley J, Dellinger JA. 1997.
Concentrations of blood and hair mercury and serum PCBs in an Ojibwa
population that consumes Great Lakes region fish. J Toxicol Clin Toxicol. 35:37786.
• Fitzgerald EF, Hwang SA, Bush B, Cook K, Worswick P. 1998. Fish consumption
and breast milk PCB concentrations among Mohawk women at Akwesasne. Am
J Epidemiol. 148:164-72.
• Fitzgerald EF, Brix KA, Deres DA, et al. 1996. Polychlorinated biphenyl (PCB)
and dichlorodiphenyl dichloroethylene (DDE) exposure among Native American
men from contaminated Great Lakes fish and wildlife. Toxicol Ind Health. 12:3618.
• Gustavsson P, Hoisted C, Rapae C. 1986. Short-term mortality and cancer
incidence in capacitor manufacturing workers exposed to polychlorinated
biphenyls (PCBs). Am J Ind Med. 10:341-4.
• Hsu S-T, Ma C-I, Hsu S-K, et al.1985. Discovery and epidemiology of PCB
poisoning in Taiwan: A four year follow-up. Environ Health Perspect. 59:5-10.

Page 7 of 9

 •

•

•
•

•
•
•

Jacobson JL, Jacobson SW, Humphrey HEB. 1990a. Effects of in utero exposure
to polychlorinatedbiphenyls and related contaminants on cognitive-functioning in
young children. J Pediatr. 116:38-45.
Jacobson JL, Jacobson SW, Humphrey HEB. 1990b. Effects of exposure to
PCBs and related compounds on growth and activity in children. Neurotoxicol
Teratol. 12:319-26.
Jacobson JL, Jacobson SW. 1996. Intellectual impairment in children exposed to
polychlorinated biphenyls in utero. N Engl J of Med. 335:783-9.
Koopman-Esseboom C, Morse DC, Weisglas-Kuperus N, et al. 1994. Effects of
dioxins and polychlorinated biphenyls on thyroid hormone status of pregnant
women and their infants. Pediatr Res. 36: 468-73.
Kreiss K, Zack MM, Kimbrough RD, et al. 1981. Association of blood pressure
and polychlorinated biphenyl levels. J Am Med Assoc. 245:2505-9.
Meigs JW, Albom JJ, Kartin BL. 1954. Chloracne from an unusual exposure to
Aroclor. J Am Med Assoc. 154:1417-8.
Rothman N, Cantor KP, Blair A, et al. 1997. A nested case-control study of nonHodgkin lymphoma and serum organochlorine residues. Lancet. 350:240-4.

Health Assessment Document for Trichloroethylene (1985)
Original Document Download Site
• Baerg RD and Kimberg DV. 1970. Centrilobular hepatic necrosis and acute renal
failure in “solvent sniffers.” Ann Intern Med. 1970;73(5):713-720.
• Bernstine JB, Meyer AE, Bernstine RL. Maternal blood and breast mild
estimation following the administration of the chloral hydrate during the
puerperium. BJOG Int Journal of Obstetrics Gynaecology. 1956;63(2):228-231.
• Bernstine JB, Meyer AE, Hayman HB. Maternal and foetal blood estimation
following the administration of chloral hydrate during labour. BJOG Int Journal of
Obstetrics Gynaecology. 1954;61(5):683-685.

Trichloroethylene Health Risk Assessment: Synthesis and Characterization
Document (2001)
Original Document Download Site
• Bovenzi, M; Barbone, F; Betta, A; et al. (1995) Scleroderma and occupational
exposure. Scand J Work Environ Health 21:289–292.
• Bove, FJ; Fulcomer, MC; Klotz, J; et al. (1992) Public drinking water
contamination and birth outcomes. Am J Epidemiol. 141:850–862.
• Brauch, H; Weirich, G; Hornauer, M; et al. (1999) Trichloroethylene exposure and
specific somatic mutations in patients with renal cell carcinoma. J Natl Cancer
Inst. 91:854–861.
• Mellemgaard, A; Engholm, G; McLaughlin, JK; et al. (1994) Occupational risk
factors for renal-cell carcinoma in Denmark. Scand J Work Environ Health.
20:160–165.
• Pastino, GM; Yap, WY; Carroquino, M. (2000) Human variability and
susceptibility to trichloroethylene. Environ Health Perspect. 108(suppl 2):201–
214.
Page 8 of 9

 •
•
•
•
•
•
•
•

Ritz, B. (1999) Cancer mortality among workers exposed to chemicals during
uranium processing. J Occup Environ Med. 41:556–566.
Stacpoole, PW; Moore, GW; Kornhauser, DM. (1979) Toxicity of chronic
dichloroacetate. N Engl J Med. 300:372.
Stacpoole, P; Wright, EC; Baumgartner, TG; et al. (1992) A controlled clinical trial
of dichloroacetate treatment in patients with lactic acidosis. The dichloroacetatelactic acidosis study group. N Engl J Med. 327:1564–1569.
Stewart, RD; Dodd, HC; Gay, HH; et al. (1970) Experimental human exposure to
trichloroethylene. Arch Environ Health. 20:64–71.
Vamvakas, S; Brüning, T; Thomasson, B; et al. (1998) Renal cell cancer
correlated with occupational exposure to trichloroethylene. J Cancer Res Clin
Oncol. 124:374–382.
Vartiainen, T; Pukkala, E; Strandman, T; et al. (1993) Population exposure to triand tetrachloroethylene and cancer risk: two cases of drinking water pollution.
Chemosphere. 27:1171–1181.
Wartenberg, D; Reyner, D; Scott, CS. (2000) Trichloroethylene and cancer:
epidemiologic evidence. Environ Health Perspect. 108(suppl 2)161–176.
Wideroff, L; Gridley, G; Mellemkjaer, L; et al. (1997) Cancer incidence in a
population-based cohort of patients hospitalized with diabetes mellitus in
Denmark. J Natl Cancer Inst. 89:1360–1365.

Mercury Study Report to Congress
• Cragle, D., D. Hollis, J. Qualters, et al. 1984. A mortality study of men exposed to
elemental mercury. J. Occup. Med. 26:817-821.
• Buiatti, E., D. Kriebel, M. Geddes, et al. 1985. A case control study of lung
cancer in Florence, Italy: I. Occupational risk factors. J. Epidemiol. Comm.
Health. 39:244-250.
• Ahlbom, A., S. Norell, Y. Rodvall and M. Nylander. 1986. Dentists, dental nurses,
and brain tumours. Br. Med. J. 292:662.
• Amandus, H. and J. Costello. 1991. Silicosis and lung cancer in U.S. metal
miners. Arch. Environ. Health. 46(2):82-89.
• Ellingsen, D., A. Andersen, H.P. Nordhagen, J. Efskind and H. Kjuus. 1992.
Cancer incidence and mortality among workers exposed to mercury in the
Norwegian chloralkali industry. 8th International Symposium on Epidemiology in
Occupational Health, Paris, France, September 10-12, 1991. Rev. Epidemiol.
Sante Publique. 40(Suppl. 1):S93-S94.
• Barregard, L., B. Hultberg, A. Schutz, et al. 1988. Enzymuria in workers exposed
to inorganic mercury. Int. Arch. Occup. Environ. Health. 61(1-2):65-69.
Barregard, L., G. Sallsten and B. Jarvholm. 1990. Mortality and cancer incidence
in chloralkali workers exposed to inorganic mercury. Br. J. Ind. Med. 47(2):99104. Barregard, L., B. Hogstedt, A. Schutz, et al. 1991. Effects of occupational
exposure to mercury vapor on lymphocyte micronuclei. Scand. J. Work Environ.
Health. 17(4):263-268.

Page 9 of 9

 Emmett Environmental Law & Policy Clinic - Attachment 4
Comments re: Docket ID No. EPA-HQ-OA-2018-0259, August 7, 2018
Grandjean Environmental Health (2018) 17:62
https://doi.org/10.1186/s12940-018-0405-y

EDITORIAL

Open Access

Delayed discovery, dissemination, and
decisions on intervention in environmental
health: a case study on immunotoxicity of
perfluorinated alkylate substances
Philippe Grandjean1,2

Abstract
Identification and characterization of environmental hazards that impact human health must rely on the best possible
science to inform and inspire appropriate public health intervention. The perfluorinated alkylate substances (PFASs) are
persistent emerging pollutants that are now being recognized as important human health hazards. Although the
PFASs have been produced for over 60 years, academic research on environmental health aspects has appeared only
in the most recent 10 years or so. In the meantime, these persistent chemicals accumulated in the global environment.
Some early studies e.g., on population exposures and toxicity, were not released to the public until after year 2000. Still,
the first PFAS risk assessments ignored these reports and relied on scant journal publications. The first guidelines and
legal limits for PFAS exposure, e.g., from drinking water, were proposed 10 years ago. They have decreased
substantially since then, but remain higher than suggested by data on human adverse effects, especially on
the immune system, that occur at background exposure levels. By now, the best-known PFASs are being
phased out, and related PFASs are being introduced as substitutes. Given the substantial delays in discovery of
PFAS toxicity, in dissemination of findings, and in regulatory decisions, PFAS substitutes and other persistent industrial
chemicals should be subjected to prior scrutiny before widespread usage.

Late emergence of early evidence
Industrial chemicals are often regarded inert or safe, unless proven otherwise, i.e., the so-called “untested chemicals assumption,” although this belief is of course not
logical [1, 2]. A high-priority group of environmental chemicals, the perfluorinated alkylate substances (PFASs),
constitute a clear example how narrow reliance on published toxicity studies can be misleading and result in insufficient and delayed protection of public health [3]. New
insight on PFAS immunotoxicity shows that the path from
discovery of toxicity to decisions on intervention can be
stalled for decades (Table 1).
After the beginning of commercial PFAS production
in the 1950s, a brief review article from 1980 [4] for the
first time mentioned industry-sponsored studies, some
Correspondence: pgrand@sdu.dk
1
Department of Environmental Health, Harvard T.H. Chan School of Public
Health, Boston, MA, USA
2
Department of Environmental Medicine, University of Southern Denmark,
Odense, Denmark

of which were carried out in monkeys. Perfluorooctanoic
acid (PFOA) showed specific toxicity to the reticuloendothelial system (i.e. immune system) [5]. In this 90-day
study, compound-related microscopic lesions were seen
in bone marrow, spleen and lymph nodes, thus clearly
suggesting immunotoxicity, although functional tests
were not carried out. A parallel study on perfluorooctanoic sulfonic acid (PFOS), also from 1978, was aborted
due to mortality of the monkeys at all doses (the lowest
being 10 mg/kg/day) [6]. These two internal reports
were eventually shared with the U.S. Environmental Protection Agency (EPA) in 2000 [7] and then became accessible to the public.
A medical thesis from 1992 mentioned the evidence
from the monkey study and noted: “No follow-up studies
of these observations have been reported” [8]. The thesis
analyzed clinical examination data from PFOA production workers and found clear associations between increased PFAS concentrations in the blood and decreased

© The Author(s). 2018 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0
International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and
reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to
the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver
(http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

 Grandjean Environmental Health (2018) 17:62

Page 2 of 6

Table 1 Time course of important developments regarding PFAS exposure and health risks [5, 6, 8, 10, 11, 13, 15, 16, 28, 31, 32, 44,
50]

Unpublished information is shaded

leukocyte counts. The results were not reported in a
scientific journal. However, in connection with a recent law suit, a draft manuscript on this study has
been released (“Peripheral blood lymphocyte count in
men occupationally exposed to perfluorooctanoic
acid” [9]). The draft concluded: “PFOA is associated
with alterations in peripheral blood lymphocyte numbers in PFOA production workers, suggesting that
cell-mediated immunity may be affected by PFOA”.
Other company materials outlined in an expert report
include the comment “We’re working with [the author] regarding some of the wording” [10]. Evidently,
an agreement was not reached, and the findings were
not published.
Human exposure to organofluorine compounds was
discovered as early as 1968 [11] and was later confirmed in a more extensive study [12]. However, the
exact identity and the sources were unknown at the
time. Soon thereafter, PFASs were identified in blood
from production workers, and in 1981 also in umbilical cord blood at a female worker’s childbirth [13].
Although the latter finding signified placental passage
and prenatal PFAS exposure, this observation was not
revealed until 20 years later, after which it was soon
confirmed in a larger study [14]. Of additional public

health significance, an unpublished study on goats
from 1993 showed that PFOS was transferred into
milk [10], and this pathway was verified in humans,
again many years later [15].

New insight into a hidden hazard
By about 2000, the widespread occurrence and persistence of PFASs in the environment became known [7], as
reflected also by the presence of PFASs in serum samples from blood banks [16]. Only after this time, and especially during the most recent 10 years, did the
scientific literature on PFASs expand (Fig. 1) [17]. Immune system deficits in PFOA-exposed mice were at
first observed in studies of peroxisome proliferator activation [18]. Later, experimental studies of PFOS showed
reductions in lymphoid cell numbers and de novo antibody synthesis [19], and a study in mice from 2009 demonstrated that PFOS exposure reduced the survival after
influenza A infection [20]. Then followed in vitro evidence of adverse effects in human white blood cells [21].
Although the 1978 monkey study [5] could have been
obtained from the U.S. EPA, none of these studies referred to these original findings.
Important evidence emerged after the discovery of
PFAS contamination in the Mid-Ohio River Valley and

 Grandjean Environmental Health (2018) 17:62

Page 3 of 6

Fig. 1 Number of publications on PFASs over time, according to the Web of Science database (between 1978 and 2017), using the search terms
“perfluorinated or perfluoro”* and restricting to environmental sciences, toxicology, or public, environmental, and occupational health categories.
This search was further refined using the search terms “immun*” and “child*”

the court-mandated health examinations [22]. In regard
to immunotoxicity, an interim report showed that increased PFOA exposure was associated with changes in
serum concentrations of immunoglobulins [23]. A more
focused study determined antibody responses to flu vaccination [24]. Elevated serum-PFOA concentrations were
associated with a reduced antibody titer rise, particularly
to an A influenza virus strain, with an increased risk of
not attaining the antibody level needed to provide
long-term protection. A later study on 12 adult volunteers with background exposures showed that two of the
subjects failed to respond to a tetanus-diphtheria booster
and that the steepness of the antibody responses was
negatively associated with the serum-PFAS concentrations [25]. Cross-sectional data have also suggested
lower vaccination antibody concentrations at elevated
background PFAS exposures [26].
The first prospective study assessing children’s antibody responses to routine childhood immunizations reported in 2012 that a doubling in exposure to PFOS and
PFOA was associated with an overall decrease by up to
50% in the specific vaccine antibody concentration [27,
28]. When mutually adjusted, the regression coefficients
for PFOA and PFOS changed only little [27]. Booster
vaccine responses in children at age 5 years were lower
at elevated serum-PFAS concentrations [28, 29]. A
smaller Norwegian study of about 50 children aged
3 years also showed tendencies toward lower vaccination
antibody concentrations at higher exposures during

pregnancy [30]. As PFASs are now known to be transferred to the infant via human milk [31], it seems likely
that PFAS exposures in early infancy represent a particular hazard to the adaptive immune system [32]. If true,
the routine modeling of lifetime exposures for risk assessment is inappropriate, as it ignores the presence of
vulnerable time windows.
PFAS exposure can also impact the body’s ability to
fight off common infections, such as colds and gastroenteritis, as seen in the Norwegian study [30]. A larger,
prospective study in Denmark found that increased maternal serum concentrations of PFOA and PFOS were
significantly associated with a higher frequency of fever
and symptoms in the children [33], in agreement with a
subsequent study from Japan that relied on retrospective
assessment of the disease incidence [34]. In contrast, a
substudy from the Danish National Birth Cohort examined the hospitalization rates for a variety of infections,
such as airway infection, middle ear infection, and appendicitis, through to age 11 years and showed no association with PFOS and PFOA in early pregnancy serum
from the mother [35]. However, a recent report from the
project team raised doubt about the validity of the PFAS
analyses [36].

Delayed interventions
Despite the support from both experimental and epidemiological data [37], most regulatory risk assessments
of PFASs have focused on other target organs and have

 Grandjean Environmental Health (2018) 17:62

Page 4 of 6

emphasized toxicity testing in rodents [4]. The first
opinion from the European Food Safety Authority
(EFSA) in 2009 [38] listed a single report on immunotoxicity under “Other endpoints”. That same year, the
EPA issued provisional health advisories and concluded
that “epidemiological studies of exposure to PFOA and
adverse health outcomes in humans are inconclusive at
present” [39]. Neither report referred to the 1978 monkey study that had become available in 2000. Early and
more recent guidelines and recommended limits for
PFOS and PFOA are shown in Table 2.
The EPA prepared more detailed risk assessment reports for PFOA and PFOS in 2014 [40, 41]. These drafts
conclude that the two major PFASs exhibit immunotoxicity in experimental models and that the epidemiological
evidence is additive, although mixed exposures complicate the attribution of effects to specific PFASs. A similar conclusion was reached by an ATSDR ToxProfile on
the perfluoroalkyls in 2015 [42]. The coverage of human
immunotoxicity was very brief, and no mention of this
potential was made in the sections on public health implications. Although the monkey studies were cited, the
risk assessment reports did not refer to the 1992 study
of exposure-associated immune cell abnormalities in
workers.
More recently, the National Toxicology Program
(NTP) in 2016 reviewed the immunotoxicity information
on PFOS and PFOA and concluded that both are “presumed” to constitute immune hazards to humans [37].
The term “presumed” is the strongest below “known” in
Table 2 Guideline values expressed in terms of acceptable
concentrations of PFOS and PFOA in drinking water (ng/L),a as
compared with the estimated limit based on benchmark dose
calculations for immunotoxicity in children [44]
Authority

Year

PFOS

PFOA

Australia
2016

70

560

Canada

2016

600

200

U.S. EPA

2009

200

400

2016

70

70

ATSDR

Minnesota

New Jersey

EFSA

BMDL-based
a

2015

70

100

2018

11

7

2008

300

300

2017

27

35

2007

-

40

2017

13

14

2009

70

700

2018

6.5

3

2013

<1

<1

Estimated from total intake limits, assuming 20% exposure contribution from
water (rounded values)

the NTP vernacular. Both PFASs suppress the antibody
response in animal studies, while the evidence in
humans is “moderate”, as all studies are observational
(not experimental) and refer to mixed PFAS exposures.
The revised ATSDR ToxProfile [43] just released concluded that decreased antibody response to vaccines is a
potential outcome from exposure to all five PFASs commonly found in human blood samples. However, ATSDR
stopped short of using epidemiology evidence for derivation of exposure limits.
Regulatory agencies frequently use benchmark dose
calculations as a basis for generating exposure limits
[38]. This approach relies on fitting a dose-response
function to the data, and the benchmark dose (BMD) is
defined as the dose that leads to a specific loss (or degree of abnormality) known as the benchmark response
(BMR) in the outcome variable. The lower one-sided
95% confidence limit of the BMD is the benchmark dose
level (BMDL), which is used as the point of departure
for calculation of exposure limits. Relying on the vaccine
antibody responses, BMDLs for PFOS and PFOA were
calculated in 2013 to be about 1 μg/L serum [44], i.e.,
levels that are exceeded by a majority of the general
population [45]. However, at first, these results were disregarded because of the absence of an unexposed control group [42], a condition that would be impossible to
meet. Another concern was the high correlation between
exposure components, such as PFOA and PFOS [40, 41,
43]. Still, mutual adjustment is possible and shows clear
negative impacts of both of these major PFASs on immune system responses [27], and other calculations
show virtually unchanged BMDLs for PFOA and PFOS
after such adjustment [46].
In an updated opinion on PFOS and PFOA [47], EFSA
used separate BMD calculations for several outcomes in
humans, including immunotoxicity, relying on summary
data in deciles or quartiles. For the vaccine response
data [28], EFSA assumed that all subjects in the lowest
decile exposure group had the same exposure, and the
BMDs were similar to the average serum concentration
in that group. For this reason, EFSA’s calculated BMDs
are several fold higher than the ones obtained from the
continuous dose-effect relationship [44]. Still, the new
tolerable intake limits are substantially lower than other
published guidelines (Table 2), though quite similar to
the Minimal Risk Levels developed by ATSDR [43].
The “untested chemicals assumption”, as highlighted
by the National Research Council [1] has clearly been inappropriately relied upon in past risk assessments of
PFASs, and these substances must now be added to the
list of environmental hazards [48] where standard risk
assessment has failed. As a major reason, early evidence
on PFAS toxicity was kept secret for 20 years or more,
and even after its release, it was apparently overlooked.

 Grandjean Environmental Health (2018) 17:62

A related reason is the absence of academic PFAS research on the immune system and other sensitive target
organs until about 10 years ago. Further, regulatory
agencies relied on experimental toxicity studies and disregarded emerging epidemiological evidence. As a result,
even some of the current guidelines are orders of magnitude above exposure levels at which associations with
adverse effects have been reported.
The PFASs therefore constitute an unfortunate example that risk assessment may be inappropriate to assess human health risks from chemical exposures when
crucial documentation has not yet been published. Recognizing the weaknesses of conventional risk assessment,
scientists from the U.S. EPA recently recommended to
consider the full range of available data and to include
health endpoints that reflect the range of subtle effects
and morbidities in humans [48]. The present summary
of delayed discovery, dissemination and decision-making
on the PFASs indicates that a more comprehensive assessment of adverse health risks is urgently needed and
that PFAS substitutes, as well as other persistent industrial chemicals, should not be considered innocuous in
the absence of relevant documentation [49].

Conclusions
Early research on environmental PFAS exposures and
their health implications became available at a substantial delay and was not taken into account in initial regulatory decisions on exposure abatement. Only in the last
10 years or so has environmental health research focused on the PFASs and revealed important human
health risks, e.g., to the immune system. Although
guideline values for PFASs in drinking water have decreased over time, they remain too high to protect
against such toxicity. While the most commonly used
PFASs will remain in the environment for many years,
new PFAS substitutes are being introduced, although little information on adverse health risks is available. Given
the serious delays in the discovery of PFAS toxicity, their
persistence in the environment, and their public health
impact, PFAS substitutes and other persistent industrial
chemicals should be subjected to prior research scrutiny
before widespread usage.
Abbreviations
BMD: Benchmark dose; BMDL: Benchmark dose level; BMR: Benchmark
response; EFSA: European Food Safety Authority; EPA: Environmental
Protection Agency; NTP: National Toxicology Program; PFAS: Perfluorinated
alkylate substance; PFOA: Perfluorooctanoic acid; PFOS: Perfluorooctanoic
sulfonic acid
Funding
The author is supported by the NIEHS Superfund Research Program
(P42ES027706).
Authors’ Contributions
The author read and approved the final manuscript.

Page 5 of 6

Competing interests
The author is an editor-in-chief of Environmental Health. The author recently
served as a health expert for the State of Minnesota in a lawsuit against
a PFAS-producing company.

Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published
maps and institutional affiliations.
Received: 20 June 2018 Accepted: 29 June 2018

References
1. National Research Council. Science and decisions: advancing risk
assessment. Washington, D.C.: National Academy Press; 2009.
2. Grandjean P. Science for precautionary decision-making. In: Gee D,
Grandjean P, Hansen SF, van den Hove S, MacGarvin M, Martin J, Nielsen G,
Quist D, Stanners D, editors. Late lessons from early warnings, vol. 2.
Copenhagen: European Environment Agency; 2013. p. 517–35.
3. Grandjean P, Clapp R. Perfluorinated alkyl substances: emerging insights
into health risks. New Solut. 2015;25(2):147–63.
4. Griffith FD, Long JE. Animal toxicity studies with ammonium
perfluorooctanoate. Am Ind Hyg Assoc J. 1980;41(8):576–83.
5. Goldenthal EI, Jessup DC, Geil RG, Mehring JS. Final report, ninety day
subacute rhesus monkey toxicity study, International Research and
Development Corporation, study no. 137–090, November 10, 1978, U.S. EPA
Administrative Record, AR226–0447. 1978.
6. Goldenthal EI, Jessup DC, Geil RG, Mehring JS. Ninety-day subacute rat
toxicity study, with Fluorad® Fluorochemical Surfactant FC-95, International
Research and Development Corporation, project No. 137–085, December
18, 1978, U.S. EPA Administrative Record, AR226–0137. 1978.
7. Lindstrom AB, Strynar MJ, Libelo EL. Polyfluorinated compounds: past,
present, and future. Environ Sci Technol. 2011;45(19):7954–61.
8. Gilliland FD. Fluorocarbons and human health: studies in an occupational
cohort. Minnesota: University of Minnesota; 1992.
9. Gilliland FD, Mandel JS: Peripheral blood lymphocyte count in men
occupationally exposed to perfluorooctanoic acid. 1992. (unpublished
manuscript, available as PTX2498 at https://urldefense.proofpoint.com/v2/
url?u=https-3A__www.ag.state.mn.us_office_contactus.asp&d=DwIGaQ&c=
vh6FgFnduejNhPPD0fl_yRaSfZy8CWbWnIf4XJhSqx8&r=2zTc5apVO6PhpYV
EBS7RA1SznZCjQuanmrAp-aakLhY&m=Cd1tSUYiNfgKihoNG-jZAQGoxUz1jvj_
BW5fLjF5BBs&s=AVjjp–we1Rr1P6Y-7CxFA1IXt7UC7l6tyOk6FTrVxA&e=).
10. Grandjean P. Expert report. Minneapolis: State of Minnesota District Court
for the County of Hennepin Fourth Judicial District; 2017. Civil Action No.
27-cv-10-28862, State of Minnesota, et al. v. 3M company
11. Taves DR. Evidence that there are two forms of fluoride in human serum.
Nature. 1968;217(5133):1050–1.
12. Guy WS, Taves DR, Brey WS. Organic fluorocompounds in human-plasma prevalence and characterization. ACS Symp Ser. 1976;28:117–34.
13. PFCs. Global contaminants: PFOA is a pervasive pollutant in human blood,
as are other PFCs [https://www.ewg.org/research/pfcs-global-contaminants/
pfoa-pervasive-pollutant-human-blood-are-other-pfcs].
14. Inoue K, Okada F, Ito R, Kato S, Sasaki S, Nakajima S, Uno A, Saijo Y, Sata F,
Yoshimura Y, et al. Perfluorooctane sulfonate (PFOS) and related
perfluorinated compounds in human maternal and cord blood samples:
assessment of PFOS exposure in a susceptible population during
pregnancy. Environ Health Perspect. 2004;112(11):1204–7.
15. Kuklenyik Z, Reich JA, Tully JS, Needham LL, Calafat AM. Automated solidphase extraction and measurement of perfluorinated organic acids and
amides in human serum and milk. Environ Sci Technol. 2004;38(13):3698–704.
16. Olsen GW, Church TR, Miller JP, Burris JM, Hansen KJ, Lundberg JK, Armitage
JB, Herron RM, Medhdizadehkashi Z, Nobiletti JB, et al.
Perfluorooctanesulfonate and other fluorochemicals in the serum of
American red Cross adult blood donors. Environ Health Perspect. 2003;
111(16):1892–901.
17. Grandjean P, Eriksen ML, Ellegaard O, Wallin JA. The Matthew effect in
environmental science publication: a bibliometric analysis of chemical
substances in journal articles. Environ Health. 2011;10:96.
18. Yang Q, Xie Y, Alexson SE, Nelson BD, DePierre JW. Involvement of the
peroxisome proliferator-activated receptor alpha in the immunomodulation

 Grandjean Environmental Health (2018) 17:62

19.

20.

21.

22.

23.

24.

25.

26.

27.

28.

29.
30.

31.

32.

33.

34.

35.

36.

37.

38.

caused by peroxisome proliferators in mice. Biochem Pharmacol. 2002;
63(10):1893–900.
DeWitt JC, Peden-Adams MM, Keller JM, Germolec DR. Immunotoxicity
of perfluorinated compounds: recent developments. Toxicol Pathol.
2012;40(2):300–11.
Guruge KS, Hikono H, Shimada N, Murakami K, Hasegawa J, Yeung LW,
Yamanaka N, Yamashita N. Effect of perfluorooctane sulfonate (PFOS) on
influenza a virus-induced mortality in female B6C3F1 mice. J Toxicol Sci.
2009;34(6):687–91.
Corsini E, Sangiovanni E, Avogadro A, Galbiati V, Viviani B, Marinovich M,
Galli CL, Dell'Agli M, Germolec DR. In vitro characterization of the
immunotoxic potential of several perfluorinated compounds (PFCs). Toxicol
Appl Pharmacol. 2012;258(2):248–55.
Steenland K, Fletcher T, Savitz DA. Epidemiologic evidence on the health
effects of perfluorooctanoic acid (PFOA). Environ Health Perspect. 2010;
118(8):1100–8.
C8 Science Panel. In: Fletcher T, Steenland K, Savitz D, editors. Status report:
PFOA and immune biomarkers in adults exposed to PFOA in drinking water
in the mid Ohio valley; 2009.
Looker C, Luster MI, Calafat AM, Johnson VJ, Burleson GR, Burleson FG, Fletcher
T. Influenza vaccine response in adults exposed to perfluorooctanoate and
perfluorooctanesulfonate. Toxicol Sci. 2014;138(1):76–88.
Kielsen K, Shamim Z, Ryder LP, Nielsen F, Grandjean P, Budtz-Jorgensen E,
Heilmann C. Antibody response to booster vaccination with tetanus and
diphtheria in adults exposed to perfluorinated alkylates. J Immunotoxicol.
2016;13(2):270–3.
Stein CR, McGovern KJ, Pajak AM, Maglione PJ, Wolff MS. Perfluoroalkyl and
polyfluoroalkyl substances and indicators of immune function in children
aged 12-19 y: National Health and nutrition examination survey. Pediatr Res.
2016;79(2):348–57.
Mogensen UB, Budtz-Jørgensen E, Heilmann C, Nielsen F, Weihe P,
Grandjean P. Structural equation modeling of immunotoxicity associated
with exposure to perfluorinated compounds. Environ Health. 2015;14:47.
Grandjean P, Andersen EW, Budtz-Jorgensen E, Nielsen F, Molbak K, Weihe
P, Heilmann C. Serum vaccine antibody concentrations in children exposed
to perfluorinated compounds. JAMA. 2012;307(4):391–7.
Grandjean P, Heilmann C. Perfluorinated compounds and immunotoxicity in
children – reply. JAMA. 2012;307:1910–1.
Granum B, Haug LS, Namork E, Stolevik SB, Thomsen C, Aaberge IS, van
Loveren H, Lovik M, Nygaard UC. Pre-natal exposure to perfluoroalkyl
substances may be associated with altered vaccine antibody levels and
immune-related health outcomes in early childhood. J Immunotoxicol.
2013;10(4):373–9.
Mondal D, Weldon RH, Armstrong BG, Gibson LJ, Lopez-Espinosa MJ, Shin
HM, Fletcher T. Breastfeeding: a potential excretion route for mothers and
implications for infant exposure to perfluoroalkyl acids. Environ Health
Perspect. 2014;122(2):187–92.
Grandjean P, Heilmann C, Weihe P, Nielsen F, Mogensen UB, Timmermann
A, Budtz-Jorgensen E. Estimated exposures to perfluorinated compounds in
infancy predict attenuated vaccine antibody concentrations at age 5-years. J
Immunotoxicol. 2017;14(1):188–95.
Dalsager L, Christensen N, Husby S, Kyhl H, Nielsen F, Host A, Grandjean P,
Jensen TK. Association between prenatal exposure to perfluorinated
compounds and symptoms of infections at age 1-4years among 359
children in the Odense child cohort. Environ Int. 2016;96:58–64.
Goudarzi H, Miyashita C, Okada E, Kashino I, Chen CJ, Ito S, Araki A,
Kobayashi S, Matsuura H, Kishi R. Prenatal exposure to perfluoroalkyl acids
and prevalence of infectious diseases up to 4years of age. Environ Int. 2017;
104:132–8.
Fei C, McLaughlin JK, Lipworth L, Olsen J. Prenatal exposure to PFOA and
PFOS and risk of hospitalization for infectious diseases in early childhood.
Environ Res. 2010;110(8):773–7.
Bach CC, Henriksen TB, Bossi R, Bech BH, Fuglsang J, Olsen J, Nohr EA.
Perfluoroalkyl acid concentrations in blood samples subjected to
transportation and processing delay. PLoS One. 2015;10(9):e0137768.
National Toxicology Program. Immunotoxicity associated with exposure to
Perfluorooctanoic acid (PFOA) or Perfluorooctane sulfonate (PFOS). Raleigh:
National Toxicology Program; 2016.
European Food Safety Authority. Guidance of the scientific committee on use
of the benchmark dose approach in risk assessment. EFSA J. 2009;1150:1–72.

Page 6 of 6

39. U.S. Environmental Protection Agency. Provisional health advisories for
perfluorooctanoic acid (PFOA) and perfluorooctane sulfonate (PFOS).
Washington, DC: U.S. Environmental Protection Agency; 2009.
40. U.S. Environmental Protection Agency. Health effects document for
Perfluorooctanoic acid (PFOA). Washington, D.C.: U.S. EPA; 2014.
41. U.S. Environmental Protection Agency. Health effects document for
Perfluorooctane sulfonate (PFOS). Washington, D.C.: U.S. EPA; 2014.
42. Agency for Toxic Substances and Disease Registry. Draft toxicological profile
for perfluoroalkyls. Atlanta: Agency for Toxic Substances and Disease
Registry; 2015.
43. Agency for Toxic Substances and Disease Registry. Draft toxicological profile
for perfluoroalkyls. Atlanta: Agency for Toxic Substances and Disease
Registry; 2018.
44. Grandjean P, Budtz-Jorgensen E. Immunotoxicity of perfluorinated alkylates:
calculation of benchmark doses based on serum concentrations in children.
Environ Health. 2013;12(1):35.
45. CDC: Fourth National Report on human exposure to environmental
chemicals, updated tables. Centers for disease control and prevention; 2015.
46. Budtz-Joergensen E, Grandjean P. Application of benchmark analysis for
mixed contaminant exposures: Mutual adjustment of two perfluoroalkylate
substances associated with immunotoxicity: bioRxiv; 2017. p. 198564.
https://www.biorxiv.org/content/early/2017/10/06/198564.
47. European Food Safety Authority. Risk to human health related to the
presence of perfluorooctane sulfonic acid and perfluorooctanoic acid in
food (draft). EFSA J. 2018;16(5):1–293.
48. Gwinn MR, Axelrad DA, Bahadori T, Bussard D, Cascio WE, Deener K, Dix D,
Thomas RS, Kavlock RJ, Burke TA. Chemical risk assessment: traditional vs
public health perspectives. Am J Public Health. 2017;107(7):1032–9.
49. Birnbaum LS, Grandjean P. Alternatives to PFASs: perspectives on the
science. Environ Health Perspect. 2015;123(5):A104–5.
50. Dewitt JC, Copeland CB, Strynar MJ, Luebke RW. Perfluorooctanoic acidinduced immunomodulation in adult C57BL/6J or C57BL/6N female mice.
Environ Health Perspect. 2008;116(5):644–50.

 Emmett Environmental Law & Policy Clinic - Attachment 5
Comments re: Docket ID No. EPA-HQ-OA-2018-0259, August 7, 2018

Our cOre practices

Core practices are the policies and practices journals and publishers need to reach the highest standards
in publication ethics. We include cases with advice, guidance for day-to-day practice, education modules
and events on topical issues, to support journals and publishers fulfil their policies.

1. Allegations of misconduct

2. Authorship and contributorship

Journals should have a clearly described process for handling
allegations, however they are brought to the journal’s or
publisher’s attention. Journals must take seriously allegations
of misconduct pre-publication and post-publication. Policies
should include how to handle allegations from whistleblowers.

Clear policies (that allow for transparency around who
contributed to the work and in what capacity) should be in
place for requirements for authorship and contributorship
as well as processes for managing potential disputes.

FIND OUT MORE: 

publicationethics.org/misconduct

FIND OUT MORE: 

publicationethics.org/authorship

3. Complaints and appeals

4. Conflicts of interest/Competing interests

Journals should have a clearly described process for
handling complaints against the journal, its staff, editorial
board or publisher.

There must be clear definitions of conflicts of interest
and processes for handling conflicts of interest of authors,
reviewers, editors, journals and publishers, whether identified
before or after publication.

FIND OUT MORE: 

publicationethics.org/appeals

FIND OUT MORE: 

publicationethics.org/competinginterests

5. Data and reproducibility

6. Ethical oversight

Journals should include policies on data availability and
encourage the use of reporting guidelines and registration of
clinical trials and other study designs according to standard
practice in their discipline.

Ethical oversight should include, but is not limited to, policies
on consent to publication, publication on vulnerable populations,
ethical conduct of research using animals, ethical conduct of
research using human subjects, handling confidential data
and of business/marketing practices.

FIND OUT MORE: 

publicationethics.org/data

FIND OUT MORE: 

publicationethics.org/oversight

7. Intellectual property

8. Journal management

All policies on intellectual property, including copyright and
publishing licenses, should be clearly described. In addition,
any costs associated with publishing should be obvious to
authors and readers. Policies should be clear on what counts
as prepublication that will preclude consideration. What
constitutes plagiarism and redundant/overlapping publication
should be specified.

A well-described and implemented infrastructure is essential,
including the business model, policies, processes and software
for efficient running of an editorially independent journal, as
well as the efficient management and training of editorial
boards and editorial and publishing staff.

FIND OUT MORE: 

publicationethics.org/intellectualproperty

FIND OUT MORE: 

publicationethics.org/management

9. Peer review processes

10. Post-publication discussions and corrections

All peer review processes must be transparently described
and well managed. Journals should provide training for editors
and reviewers and have policies on diverse aspects of peer
review, especially with respect to adoption of appropriate
models of review and processes for handling conflicts of
interest, appeals and disputes that may arise in peer review.

Journals must allow debate post publication either on their
site, through letters to the editor, or on an external moderated
site, such as PubMed Commons or PubPeer. They must have
mechanisms for correcting, revising or retracting articles
after publication.

FIND OUT MORE: 

publicationethics.org/peerreview

FIND OUT MORE: 

publicationethics.org/postpublication
publicationethics.org/core-practices

 7/23/2018

Core practices   Committee on Publication Ethics: COPE

Sign in

What are you looking for

Home

About COPE

Core Practices

Resources

Cases

Membership

News

Events

Contact Us

Home

Core practices
COPE's role is to assist editors of scholarly journals and publishers/owners - as well as other parties, such as institutions and funders, albeit less directly - in
their endeavour to preserve and promote the integrity of the scholarly record through policies and practices that reflect the current best principles of
transparency and integrity. COPE’s new recommendations are intended to reflect these aims, in a practical way. COPE have therefore reviewed the Code of
Conduct and Best Practice Guidelines for Editors and Code of Conduct for Journal Publishers and have consolidated them into one, much shorter, document
entitled “Core Practices”. [Available to download as an A4 poster.] Connected to each of these core practices will be hyperlinks to the detailed documents
and resources COPE already publish, which are arrived at through extensive consultation, and which we will be building into a comprehensive, yet
responsive library. The full range of COPE resources can be found here. .The Core Practices are applicable to all involved in publishing the scholarly
literature: editors and their journals, publishers (and institutions).
COPE's Core Practices should be considered alongside specific national and international codes of conduct for research and is not intended to replace
them.
Background to why the Code of Conduct for Journal Editors has been replaced with the Core Practices.

Journals and Publishers should have robust and well-described, publicly documented practices in all the following areas for their journals:

1. Allegations of misconduct
Journals should have a clearly described process for handling

Latest resources

allegations, however they are brought to the journal's or

Association of Research Integrity Officers 2018 (Event)

publisher’s attention. Journals must take seriously allegations

In the News: July Digest (News)

of misconduct pre-publication and post-publication. Policies

In the news: May 2018 Digest (News)

should include how to handle allegations from whistleblowers.

View all Allegations of misconduct resources

2. Authorship and contributorship
Clear policies (that allow for transparency around who

Latest resources

contributed to the work and in what capacity) should be in

Association of Research Integrity Officers 2018 (Event)

place for requirements for authorship and contributorship as

In the News: July Digest (News)

well as processes for managing potential disputes

ISMTE North American Conference (Event)

https://publicationethics.org/core-practices

1/4

 7/23/2018

Core practices   Committee on Publication Ethics: COPE

View all Authorship and contributorship resources

3. Complaints and appeals
Journals should have a clearly described process for handling

Latest resources

complaints against the journal, its staff, editorial board or

In the news: April 2018 Digest (News)

publisher

Complaints and appeals (News)
COPE Education Subcommittee focus: Complaints and Appeals (News)

View all Complaints and appeals resources

4. Conflicts of interest / Competing interests
There must be clear definitions of conflicts of interest and

Latest resources

processes for handling conflicts of interest of authors,

Letter from the COPE co-Chairs: July 2018 (News)

reviewers, editors, journals and publishers, whether identified

In the news: May 2018 Digest (News)

before or after publication

COPE Education Subcommittee focus: Conflicts of Interest (News)

View all Conflicts of interest / Competing interests resources

5. Data and reproducibility
Journals should include policies on data availability and

Latest resources

encourage the use of reporting guidelines and registration of

Letter from the COPE co-Chairs: July 2018 (News)

clinical trials and other study designs according to standard

In the News: July Digest (News)

practice in their discipline

Creating and implementing data research policies: COPE webinar report (News)

View all Data and reproducibility resources

6. Ethical oversight
Ethical oversight should include, but is not limited to, policies

Latest resources

on consent to publication, publication on vulnerable

In the News: July Digest (News)

populations, ethical conduct of research using animals, ethical

In the news: June 2018 Digest (News)

conduct of research using human subjects, handling

COPE Forum discussion: Preprints: continuing the conversation (News)

confidential data and of business/marketing practices

View all Ethical oversight resources

https://publicationethics.org/core-practices

2/4

 7/23/2018

Core practices   Committee on Publication Ethics: COPE

7. Intellectual property
All policies on intellectual property, including copyright and

Latest resources

publishing licenses, should be clearly described. In addition,

Association of Research Integrity Officers 2018 (Event)

any costs associated with publishing should be obvious to

In the News: July Digest (News)

authors and readers. Policies should be clear on what counts

COPE Education Subcommittee focus: Intellectual Property (News)

as prepublication that will preclude consideration. What
constitutes plagiarism and redundant/overlapping publication
should be specified

View all Intellectual property resources

8. Journal management
A well-described and implemented infrastructure is essential,

Latest resources

including the business model, policies, processes and

In the News: July Digest (News)

software for efficient running of an editorially independent

COPE Forum 30 April 2018: Preprints: continuing the conversation (Resource)

journal, as well as the efficient management and training of

COPE Forum discussion: Preprints: continuing the conversation (News)

editorial boards and editorial and publishing staff

View all Journal management resources

9. Peer review processes
All peer review processes must be transparently described

Latest resources

and well managed. Journals should provide training for editors

Letter from the COPE co-Chairs: July 2018 (News)

and reviewers and have policies on diverse aspects of peer

In the News: July Digest (News)

review, especially with respect to adoption of appropriate

ISMTE North American Conference (Event)

models of review and processes for handling conflicts of
interest, appeals and disputes that may arise in peer review

View all Peer review processes resources

10. Post-publication discussions and corrections
Journals must allow debate post publication either on their

Latest resources

site, through letters to the editor, or on an external moderated

In the News: July Digest (News)

site, such as PubPeer. They must have mechanisms for

In the news: April 2018 Digest (News)

correcting, revising or retracting articles after publication

COPE Forum 26 February 2018: Expressions of concern (Resource)

View all Post-publication discussions and corrections resources
https://publicationethics.org/core-practices

3/4

 7/23/2018

Committee on Publication
Ethics
Promoting integrity in research
publication
COPE on Facebook
COPE on Twitter
Privacy Policy
Website Terms and Conditions

Core practices   Committee on Publication Ethics: COPE

Site map

Resources

Home

Cases

About COPE

Core practices

Resources

Flowcharts

Cases

Guidelines

Become a member

eLearning

Membership

Discussion Documents

Events

International standards for editors and
authors

News & Opinion
Contact Us

Sign up to COPE's latest news
Your email address

Sign up

Select Language ▼

COPE Digest
Seminars and webinars
Research
Resources and Further Reading
Chinese resources
COPE Guide
Archive

© Committee on Publication Ethics 2018

https://publicationethics.org/core-practices

4/4

 Emmett Environmental Law & Policy Clinic - Attachment 6
Comments re: Docket ID No. EPA-HQ-OA-2018-0259, August 7, 2018

Cite as: J. Berg., P. Campbell, V. Kiermer,
N. Raikhel, D. Sweet, Science
10.1126/science.aau0116 (2018).

Downloaded from http://science.sciencemag.org/ on July 23, 2018

First release: 30 April 2018

www.sciencemag.org

 Joint statement on EPA proposed rule and public availability of data
Jeremy Berg, Philip Campbell, Veronique Kiermer, Natasha Raikhel and Deborah Sweet

published online April 30, 2018

http://science.sciencemag.org/content/early/2018/04/30/science.aau0116

PERMISSIONS

http://www.sciencemag.org/help/reprints-and-permissions

Use of this article is subject to the Terms of Service
Science (print ISSN 0036-8075; online ISSN 1095-9203) is published by the American Association for the Advancement of
Science, 1200 New York Avenue NW, Washington, DC 20005. 2017 © The Authors, some rights reserved; exclusive licensee
American Association for the Advancement of Science. No claim to original U.S. Government Works. The title Science is a
registered trademark of AAAS.

Downloaded from http://science.sciencemag.org/ on July 23, 2018

ARTICLE TOOLS