j:

"i

Case 2:12-md-02323-AB Document 10672-1 Filed 06/13/19 Page 1 of 12

UNITED STATES DISTRICT COURT
FOR THE EASTERN DISTRICT OF PENNSYLVANIA
No. 2:12-md-02323-AB
IN RE: NATIONAL FOOTBALL
LEAGUE PLAYERS' CONCUSSION
INJURY LITIGATION

MDL No. 2323

- - - - - - - - - - - - - - - - - i Hop. Anita B. Brody

Kevin Turner and Shawn Wooden, on
behalfofthemselves and others similarly
situated,
Plaintiffs,

Civ. Action No.: 14-cv-00029-AB

v.
National Football League and NFL
Properties, LLC, successor-in-interest to
NFL Properties, Inc.
Defendants.

THIS DOCUMENT RELATES TO:
Settlement Class M e m b e r (SPID 100002282)

Hon. Anita B. Brody

THE NFL PARTIES' APPEAL OF THE
SPECIAL MASTER'S DECISION NOT TO CONSULT AN AAPC WHEN
DECIDING THE APPEAL OF CLAIM DETERMINATION F O R -

 Case 2:12-md-02323-AB Document 10672-1 Filed 06/13/19 Page 2 of 12

TABLE OF CONTENTS

TABLE OF CONTENTS ................................................................................................................. i
PRELIMINARY STATEMENT .................................................................................................... 1
BACKGROUND ............................................................................................................................ 2
A.

The Role of AAPCs in Appeals .............................................................................. 2

B.

Performance Validity Testing Under the Settlement Agreement ........................... 3

C.

Mr. • 's Neuropsychological Testing Results and Claim History ..................... 4

ARGUMENT .................................................................................................................................. ?
CONCLUSION ............................................................................................................................. 10

 Case 2:12-md-02323-AB Document 10672-1 Filed 06/13/19 Page 3 of 12

The National Football League and NFL Properties LLC (collectively, the "NFL Parties")
respectfully appeal the Special Master's March 19, 2019 Post-Appeal Notice ofMonetary Award
Claim Determination, which denied the NFL Parties' appeal of

("Claimant's")

Monetary Award Determination (the·· Appeal"). The NFL Parties submit that the Special
Master's failure to consult with an Appeals Advisory Panel Consultant ("AAPC") in deciding the

Ill Appeal, which exclusively concerned the highly technical issue of egregious performance
validity failures on neuropsychological testing, constitutes an abuse of discretion. The NFL Parties
respectfully seek reconsideration of t h e . Appeal with the assistance of an AAPC. ·
PRELIMINARY STATEMENT

Claimant Baseline

Assessment

neuropsychologist Dr.

submitted a claim based on evaluations he received through the
Program

("BAP")

by

neurologist Dr.

and

. Although D r . - (the neurologist) diagnosed Claimant

with Level1.5 Neurocognitive Impairment, Dr. - ( t h e neuropsychologist) determined that
Claimant did not satisfy the criteria for any Qualifying Diagnosis because the neuropsychological
test results "[were] not considered an accurate reflection of [his] optimal cognitive abilities," as
the Settlement Agreement requires, because Claimant failed all seven administered performance
validity tests-including performing substantially below chance on all six measures of forced

choice recognition. In other words, the testing results reflected intentional malingering. As Dr.
-summarized, Claimant "would have performed better if he had randomly guessed."
Because the Qualified BAP Providers disagreed on whether the evaluations merited a
Qualifying Diagnosis of Level 1.5 (per the neurologist) or no diagnosis (per the
neuropsychologist), the BAP Administrator asked a neurologist on the Appeals Advisory Panel
("AAP") to review the medical records. Upon information and belief, the AAP neurologist
reviewed the material without consulting an AAPC neuropsychologist. The AAP then upgraded

 Case 2:12-md-02323-AB Document 10672-1 Filed 06/13/19 Page 4 of 12

the claim to a diagnosis of Level 2 Neurocognitive Impairment despite the fact that neither Dr.
-

nor D r - supported that more severe diagnosis.

The NFL Parties appealed the claim determination based on the extreme performance
validity failures, and specifically requested that the Special Master exercise his discretion to
consult with an AAPC given the clear need for neuropsychological expertise to evaluate Dr.
-conclusion that the testing was unreliable. The Special Master, however, denied the
NFL's appeal without consulting his. neutral AAPC resource. The Special Master's failure to
consult an AAPC-where the entire appeal turned on the highly technical issue of extreme
performance validity failures in neuropsychological testing and the only neuropsychologist to
review that testing had determined that Claimant's severe performance validity failures rendered
the testing unreliable and precluded a Qualifying Diagnosis-was a clear abuse of discretion. The
integrity of the Settlement Program requires consultation with an AAPC in the face of such flagrant
intentional malingering.
For these reasons, the Court should remand the -Appeal to the Special Master for
reconsideration,

in consultation with an AAPC, to determine whether Mr. . ' s

neuropsychological test results are reliable or, instead, reflect intentional malingering.
BACKGROUND
A.

The Role of AAPCs in Appeals

The Settlement Agreement provides that, in deciding appeals of claim determinations, the
Court "may be assisted, in its discretion, by ... an Appeals Advisory Panel Consultant."
(Settlement Agreement, § 9.8.) While the decision to seek such assistance is discretionary, the
Parties to the Settlement Agreement made the AAPCs a central part of the claim appeal process
because they recognized that the Court or its designees (here, the Special Masters) would not be

2

 Case 2:12-md-02323-AB Document 10672-1 Filed 06/13/19 Page 5 of 12

experts in neuropsychology or have the training necessary to evaluate neuropsychological testing
records and the meaning and significance of performance validity failures.
The three

AAPCs

available for

consultation

on

appeals

are

board-certified

neuropsychologists "agreed to and jointly recommended by Co-Lead Class Counsel and Counsel
for the NFL Parties, and appointed by the Court." (!d., § 2.1(g).) The Parties selected these
AAPCs based on their unquestionable expertise and stature in the field of neuropsychology, and
after conducting a stringent vetting process. The AAPCs are incontestably qualified to evaluate
performance validity failures in neuropsychological test results. The AAPCs are true neutrals,
with no ongoing contact with lawyers for claimants, Co-Lead Class Counsel or the NFL Parties.

B.

Performance Validity Testing Under the Settlement Agreement

The definitions of Qualifying Diagnoses were key and heavily negotiated components of
the Settlement Agreement. For a Qualifying Diagnosis of Level 2 Neurocognitive Impairment
made through the BAP, the Settlement Agreement requires "[e]vidence of a severe cognitive
decline from a previous level of performance, as determined by and in accordance with the
standardized neuropsychological testing protocol" provided in the Settlement Agreement. (!d.,
Ex. 1 § 2(a)(ii).) That neuropsychological testing protocol requires that "[f]reestanding, embedded
and regression based performance validity metrics [are] administered to each Retired NFL Football
Player during baseline and, if relevant, subsequent neuropsychological examinations," and that the
neuropsychologist complete the Slick Checklist of validity criteria "in order to determine whether
the Retired NFL Player's test data is a valid reflection of his optimal level of neurocognitive
functioning." (!d., Ex. 2 at 2.) Moreover, "a Retired NFL Player's failure on two or more effort
tests may result in the Retired NFL Football Player's test results being subjected to independent
review, or result in a need for supplemental testing." (!d.)

3

 Case 2:12-md-02323-AB Document 10672-1 Filed 06/13/19 Page 6 of 12

C.

M r - Neuropsychological Testing Results and Claim History

Claimant was evaluated in the BAP by neurologist D r . - and neuropsychologist Dr.
- · Dr. -found that Mr. · · s evaluation was "generally consistent with" the BAP
criteria for a diagnosis ofLevel1.5 Neurocognitive Impairment and rendered that diagnosis, even
though the "generally consistent" standard plainly does not apply to diagnoses made within the
BAP. D r . - · on the other hand, determined that Mr. • did not satisfy the criteria for

any Qualifying Diagnosis, explaining that because Mr. •failed all seven performance validity
tests, Mr. · · s test results "[were] not considered an accurate reflection of [his] optimal
cognitive abilities," as the Settlement Agreement requires. (Ex. 1, - R e p o r t at 2-3; Ex. 2,
-

Addendum at 1.)
A neurologist on the AAP reviewed these conflicting determinations pursuant to the

Settlement Agreement and upgraded Claimant's diagnosis to Level2 Neurocognitive Impairment,
despite the fact that neither Qualified BAP Provider agreed to that Qualifying Diagnosis. (See Ex.
3, Nov. 5, 2018 Letter.) Claimant then submitted a claim based on that upgraded Level2 diagnosis,
and his claim was approved by the Claims Administrator. The NFL Parties appealed that claim
determination based on Claimant's egregious performance validity test failures, as documented by
D r . - · who performed the neuropsychological examination.
Specifically, the NFL Parties argued that Claimant failed to satisfy the requirements for
Level 1.5 or 2 Neurocognitive Impairment because the neuropsychological testing used to support
his alleged diagnosis clearly evidenced malingering and was therefore invalid. As D r . found, Claimant failed all seven performance validity measures, and performed "substantially

below levels of chance on all6 measures offorced-choice recognition" (the TOMM, VSVT, and
4 other ACS measures). (Ex. 1, - R e p o r t at 2-3 (emphasis added).) That is, Claimant
"would have performed better if he had randomly guessed." (/d.) Although an individual with
4

 Case 2:12-md-02323-AB Document 10672-1 Filed 06/13/19 Page 7 of 12

severe dementia may have suboptimal scores on performance validity tests as a result of guessing
or severe limitations, that individual would be expected to perform near chance levels (i.e., the
level expected when randomly guessing). Here, where Claimant performed "substantially below
levels of chance" on all six measures of forced choice recognition, the results indicate intentional
malingering. Put another way, to perform so far below chance on so many measures, Claimant
must have recognized the correct answer (e.g., answer "A"), and then intentionally picked an
incorrect response instead (e.g., answer "B").
Indeed, it is well established that such results demonstrate malingering.

Extensive

neuropsychological research, as well as the TOMM's and VSVT's own test manuals, confirm that
performing below-chance levels, especially substantially and repeatedly as here, goes beyond other
performance validity test failures and is a telltale sign of intentional malingering. 1
Because of this substantial evidence of malingering, D r . - reviewed the Slick
Criteria Checklist, as the Settlement Agreement requires, and determined that "review of the Slick
Criteria suggest that there was a pattern of neuropsychological test performance that is markedly
discrepant from currently accepted models of normal and abnormal central nervous system
function," and that Claimant's "[p]erformance was extremely poor on tests designed to measure
the degree of effort applied to testing." (Ex. 1 , - Report at 3 (emphasis added).) In
addition, as part of the Slick Criteria Checklist, D r - specifically determined that Mr.
Ill's performance on these validity tests was "markedly discrepant" and "below" that seen "in

See, e.g., Ex. 4, Larrabee, G.J., Aggregation Across Multiple Indicators Improves the Detection of Malingering
(2008); Ex. 5, Green, P. & Iverson, G.L., Validation of the Computerized Assessment of Response Bias in
Litigating Patients With Head Injuries (2001); Ex. 6, Clark, et al., Yes/No Versus Forced-Choice Recognition
Memory in Mild Cognitive Impairment and Alzheimer's Disease at 1212-13 (2012); Ex. 7, Dean, et al., Dementia
and Effort Test Performance at 148 (2009); Tombaugh, T. N. (1996). Test of Memory Malingering (TOMM),
New York: Multi-Health Systems, Inc.; Slick, D., et al., Victoria Symptom Validity Test (VSVT) Professional
Manual.

5

 Case 2:12-md-02323-AB Document 10672-1 Filed 06/13/19 Page 8 of 12

patients with [traumatic brain injury] and moderate levels of dementia." (Id.) D r . concluded:
According to guidelines from the Neuropsychologist Handbook
provided by the BAP, M r - test results cannot be considered
valid due to his failure of multiple [performance validity tests}. As
such, I cannot rely on these findings as an accurate representation
of his c~ve abilities. Therefore, it is not possible to conclude
that Mr.- meets criteria for Neurocognitive Impairment at Level
1, 1. 5 or 2 at the present time.
(Id. at 5 (emphases added).) 2

On appeal, the NFL Parties argued that Dr. - · s conclusions were correct and that
Claimant's performance on his neuropsychological examination provided clear and convincing
evidence that he was intentionally malingering and, therefore, his neuropsychological test results
did not represent his optimal neurocognitive performance and could not satisfy the Settlement
Agreement's criteria for Level 2 (or Level 1.5) Neurocognitive Impairment. Because the appeal
dealt exclusively with highly technical issues of failures on neuropsychological performance
validity tests, the NFL Parties explicitly requested that the Special Master consult with an AAPC
in deciding the appeal. (See Ex. 8, NFL Parties' Appeal at 4.)
Claimant opposed the NFL Parties' appeal, arguing that there was no evidence he was
malingering and that deference should be given to the AAP neurologist who reviewed the
Qualified BAP Providers' disagreement and upgraded the diagnosis to Level 2. Claimant also

2

Although Claimant's performance on the seven performance validity tests, standing alone, invalidates his
neuropsychological testing, D r . - found additional indications that Claimant was malingering during the
evaluations. For example, Claimant's validity results on the MMPI-2-RF (a mental health assessment test
included in the BAP) likewise "raise[d] concerns about inconsistent reporting." (Ex. 1, - n Report at 4.)
In particular, there were "concerns about over-reporting [because] he generated a much larger than average
number of infrequent responses . . . and provided a very unusual combination of responses that is strongly
associated with non-credible memory complaints." (Jd) Claimant's "report of cognitive symptoms was much
higher than that typically seen in individuals with known cognitive impairments," indeed it "was nearly 6
standard deviations above the mean." (Jd (emphasis added).) Yet, Claimant also "presented himself in an
extremely positive light by denying minor faults and shortcomings that most people acknowledge." (Id)

6

 Case 2:12-md-02323-AB Document 10672-1 Filed 06/13/19 Page 9 of 12

argued that the NFL Parties' appeal disregarded his functional impairment, and that the request
that the "Special Master ... confer with an AAP member on ... appeal shows why [the] appeal
should be denied." (See Ex. 9, Claimant's Opp'n.) In fact, the NFL Parties requested that the
Special Master consult with an AAPC, rather than an AAP member-a critical difference, as
described herein. (See Ex. 8, NFL Parties' Appeal at 4.)
The Special Master denied the NFL Parties' appeal on March 19, 2019. Despite the NFL
Parties' explicit request that the Special Master consult with an AAPC given the performance
validity issues involved, the Post-Appeal Notice provides no indication that the Special Master
exercised his discretion to consult an AAPC.
ARGUMENT

The Special Master's apparent failure to consult with an AAPC constituted a clear abuse
of discretion given the appeal's singular focus on the highly technical issue of performance validity
failures in Claimant's neuropsychological testing. 3 Only neuropsychologists are properly trained
in the validated science to review such results. Indeed, the only neuropsychologist to review
Claimant's testing, D r . - · unequivocally determined that Claimant's testing could not
support a Qualifying Diagnosis under the terms of the Settlement Agreement. (See Ex. 1 , Report; Settlement Agreement, Exs. 1-2.)
In reviewing for abuse of discretion, the Court must determine, based on the totality of the
circumstances, whether the Special Master's decision not to consult an AAPC was unreasonable. 4

4

See Fed. R. Civ. P. 53(f)(5) ("Unless the appointing order establishes a different standard of review, the court
may set aside a master's ruling on a procedural matter only for an abuse of discretion.").
See Loinaz v. EG & G, Inc., 910 F.2d 1, 7 (1st Cir. 1990) ("There is no neat, standardized test for judging abuse
of discretion; each case must be judged on its own facts and circumstances."); Zervos v. Verizon NY., Inc., 252
F.3d 163, 169 n.6 (2d Cir. 2001) ("Discretion is said to be 'abused' ('exceeded' would be both a more felicitous
and correct term) when the decision reached is not within the range of decision-making authority a reviewing
court determines is acceptable for a given set of facts.") (internal quotation omitted)); U.S. v. Cunningham, 694
F.3d 372, 383 (3d Cir. 2012) (district court's rulings are an abuse of discretion if "clearly unreasonable");

7

 Case 2:12-md-02323-AB Document 10672-1 Filed 06/13/19 Page 10 of 12

The Special Master's decision is entitled to less deference than an appellate court would give to a
trial court's determination. 5 Moreover, this Court expressly retains "continuing and exclusive
jurisdiction to interpret, implement, administer and enforce the Settlement Agreement," 6 and thus
has unfettered authority to opine on the proper implementation of the Settlement Agreement's
provisions governing AAPC use in this context.
The goal in making AAPCs available to the Court (and, by extension, the Special Master)
in connection with claim appeals was to provide a safeguard that the ultimate determinations would
be correct--or at least not clearly wrong-as a matter of clinical neuropsychology. Here, there
can be no pretense that it was reasonable for the Special Master not to consult an AAPC in deciding
this appeal. The NFL Parties' appeal focused solely on extreme performance validity failures,
which only a neuropsychologist has the proper training and expertise to properly and fully
interpret. As noted above, the NFL Parties' appeal detailed Claimant's severe performance
validity failures-including that Claimant failed all seven validity measures and was substantially
below chance on all six forced-choice validity measures-and explained that Dr. - ' the

only neuropsychologist who has reviewed Claimant's testing results, unequivocally concluded that
Claimant's ''test results cannot be considered valid due to his failure of multiple [performance
validity tests]" and cannot be "rel[ied] on ... as an accurate representation of his cognitive
abilities." (Ex. 1, - R e p o r t at 3, 5.) For that reason, the NFL Parties explicitly requested
that the Special Master consult with an AAPC, eminently qualified neuropsychologists handpicked by the Court and Parties for the express purpose of helping the Court and Special Masters

Ferguson v. Eakle, 492 F.2d 26,29 (3d Cir. 1974) (considering "totality of[] circumstances" in determining that
district court abused its discretion).
See Fed. R. Civ. P. 53, Special Committee Notes, 2003 Amendment Subdivision (g) ("The subordinate role of
the master means that the trial court's review for abuse of discretion may be more searching than the review that
an appellate court makes of a trial court.").
6

Am. Final Order and J. § 17, ECF No. 6534; see also Settlement Agreement§ 20.l(n).

8

 Case 2:12-md-02323-AB Document 10672-1 Filed 06/13/19 Page 11 of 12

correctly decide precisely this type of appeal. (See Ex. 8, NFL Parties' Appeal at 4.)
While the AAP member who reviewed Claimant's BAP medical records prior to appeal is,
of course, eminently qualified in his or her field, that field is neurology rather than
neuropsychology, and the NFL Parties' appeal (and Dr. - · s refusal to render a Qualifying

Diagnosis) solely involved issues of neuropsychological testing. The Parties understood that AAP
members, despite their impressive qualifications in the field of neurology, would need assistance
interpreting neuropsychological testing. That is why the Parties ensured that highly-qualified
neuropsychologist AAPCs would be available to the Court to assist with technical
neuropsychological issues, such as the severe performance validity failures that are the subject of
the NFL Parties' appeal.
The NFL Parties' appeal presented a highly technical issue as to which the Special Master's
failure to consult with an AAPC is particularly misguided and unreasonable. The performance
validity metrics in the neuropsychological testing battery are a critical feature to safeguard the
integrity of the Settlement Program and weed out intentional malingering and misrepresentation
of cognitive abilities by retired players. Those performance validity metrics, however, are also
highly technical and require the specific expertise of the AAPC. Unassisted review by the Special
Master, a layperson with no expertise in this highly technical area, simply provides no safeguard
on appeal. The severe performance validity failures at issue on this appeal should presumptively
have the benefit of AAPC expertise. Indeed, the Settlement Agreement specifically provides that
"a Retired NFL Player's failure on two or more effort tests may result in the retired NFL Football
Player's test results being subjected to independent review, or result in a need for supplemental
testing." (Settlement Agreement, Ex. 2 at 2.) Here, Claimant failed not just two effort tests, but
all seven of the performance validity measures, including six below chance level. Simply put, this

9

 Case 2:12-md-02323-AB Document 10672-1 Filed 06/13/19 Page 12 of 12

claim involves textbook intentional malingering, and the payment of this claim without AAPC
review would undermine the integrity of the Settlement Program.
CONCLUSION

For the reasons set forth herein, the NFL Parties respectfully request that the Court remand
the .Appeal and direct the Special Master to consult with an AAPC on the appeal.

Dated: April 8, 2019

Respectfully submitted,

Is/ Brad S. Karp
PAUL, WEISS, RIFKIND, WHARTON
& GARRISON LLP
BradS. Karp
Bruce Birenboim
Claudia Hammerman
Richard C. Tarlowe
Douglas M. Burns
1285 Avenue of the Americas
New York, NY 10019-6064
Tel: (212) 373-3000
bkarp@paulweiss.com
Attorneys for Defendants the National Football
League and NFL Properties LLC

10