PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
Internal Revenue Service (IRS)
Wage & Investment (W&I)
Sponsor: IRS
Dept. No.: P181
Contract No.: TIRNO-99-D-00005
Project No.: 2032H5-18-F-00593
The views, opinions and/or findings
contained in this report are those of The
MITRE Corporation and should not be
construed as an official government position,
policy, or decision, unless designated by
other documentation.
IRS Free File Program
Independent Assessment of the
Free File Program
For Internal MITRE Use. This document was
prepared for authorized distribution only. It
has not been approved for public release
Free File Program Assessment Final Report
This technical data was produced for the
U.S. Government under Contract Number
TIRNO-99-D-00005, and is subject to
Federal Acquisition Regulation Clause
52.227-14, Rights in Data—General, Alt. II,
III and IV (DEC 2007) [Reference
27.409(a)].
October 3, 2019
No other use other than that granted to the
U.S. Government, or to those acting on
behalf of the U.S. Government under that
Clause is authorized without the express
written permission of The MITRE
Corporation.
For further information, please contact The
MITRE Corporation, Contracts Management
Office, 7515 Colshire Drive, McLean, VA
22102-7539, (703) 983-6000.
©2019 The MITRE Corporation.
All rights reserved.
McLean, VA
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
This page intentionally left blank for two-sided printing.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
ii
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
Executive Summary
In January 2019, the Internal Revenue Service (IRS) Free File program (“Free File” or
“the program”) opened for its 17th filing season. Established in 2002, the IRS Free File program
makes free online tax preparation and electronic filing available to 70 percent of the taxpaying
population; for Filing Season (FS) 2019, this equated to taxpayers with an adjusted gross income
(AGI) of $66,000 or less. The program is made possible through a partnership between the IRS
and the tax software industry. The partnership is governed through a mutually negotiated
Memorandum of Understanding (MOU) between the IRS and the Free File Alliance (FFA)1—a
coalition of individual private-sector tax preparation companies2—that has evolved over the
program’s lifespan to address issues as they have arisen.
While cumulatively the program has resulted in more than 53 million free returns e-filed and an
estimated $1.6 billion in savings to taxpayers3, the Free File program has also faced criticism in
recent years as questions have been raised by Congress and the press about low annual
participation rates and assertions of inadequate oversight, deceptive business practices, and
misaligned roles for government and industry. This year’s launch came amid criticisms of the
program and speculation that industry partners were not acting in the best interest of taxpayers.
In response to these assertions as well as recent oversight activities and congressional inquiries,
IRS leadership determined that an independent assessment was necessary to ensure the continued
operations and integrity of the IRS Free File program. Because the MITRE Corporation operates
the Service’s federally funded research and development center (FFRDC), the IRS engaged
MITRE to conduct an independent and objective assessment of the IRS Free File program in its
current state, providing recommendations for improvements to the program, and by extension to
the MOU that governs the program.
Methods
Beginning in June 2019, MITRE established several workstreams for addressing each of the
targeted assessment areas reflected in Figure ES-1. Taken together, these workstreams produced
findings and recommendations related to Free File member compliance with the MOU and
potential enhancements to the program. Highlights of the assessment approach include:
1
In 2014, the Free File Alliance officially changed the name of the organization to Free File, Inc. (FFI) doing business as the
Free File Alliance. While FFI is the listed name on all agreements and MOUs beginning in 2014, the two names are used
interchangeably. The IRS references the Alliance on IRS.gov “Free File: About the Free File Alliance,” and the organization has
not changed the name of its own website: https://freefilealliance.org/. This document will use the term Free File Alliance, the
Alliance, or FFA except when FFI is used in a specific context or quote.
2 Industry participation in the Free File program is voluntary. As stipulated in the MOU, there is a process for applying for
membership to the Free File Alliance and companies may leave the program at their discretion. Also outlined in the MOU, IRS
and FFA leadership can remove members from the program if there are certain MOU violations. Through FS 2019, the Free File
Alliance included 12 member companies; currently there are only 11 participating members of the program. As of July 2019,
Drake [1040.com] chose to withdraw from the program and will not be a part of it going forward. Since they were a part of the
program for FS 2019, they are still included in this assessment. The decision to leave came from the member itself and to
MITRE’s knowledge were not related to any MOU infractions.
3 Internal Revenue Service, "IRS, Free File Alliance announce changes to improve program; Improved taxpayer options available
for 2019 Free File program," Internal Revenue Service, 2 November 2018. [Online]. Available:
https://www.irs.gov/newsroom/irs-free-file-alliance-announce-changes-to-improve-program-improved-taxpayer-optionsavailable-for-2019-free-file-program. [Accessed 2 July 2019].
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
iii
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
• Current Environmental Scan. MITRE conducted an environmental scan to better
understand the Free File program in terms of historical context, taxpayer participation, the
role of IRS and industry in its public-private partnership (PPP), the economics of “free” as
it pertains to Free File, and recent interest in the program.
• Free File Program Analysis. MITRE conducted stakeholder interviews—including IRS
leadership—and performed document reviews to assess the IRS Free File program’s
alignment to objectives, performance, and the IRS management and oversight functions.
• FFA Member Organization Interviews (“site visits”). MITRE conducted interviews
with representatives from each Free File member organization to collect information on
their interactions and agreements with the IRS regarding the Free File program and
applicable business processes, practices, and techniques for how each Free File member
determines if a taxpayer qualifies for Free File. Each of the interviews followed a similar
approach: standard interview protocols, targeted questions (both related to the IRS Free
File program as well as the MOU Article 4, Standards of Practice) and in-person or phone
interviews with participants from each Free File member organizations (selected by the
member company’s leadership).
• Web Forensics Analyses. MITRE performed an analysis to determine if any coding
within the member’s website influences search engine routing. In addition, MITRE
analyzed web search query traffic metrics for internet searches that signaled an intent to
find free tax filing services, based on a representative sample of organic (unpaid) search
keywords for free tax filing. This included a click-through analysis of IRS.gov site usage
metrics to investigate where taxpayers went from the IRS Free File web portal. This
analysis was also used to determine how often taxpayers searched for free tax filing
services (especially during tax filing season) using two primary search engines, Google
and Bing; to what site(s) these users went following their searches; and the differences in
search engine results.
• Taxpayer Journey. MITRE conducted an expert review of the IRS Free File portal as
well as each individual Free File member offer. This review was twofold in its objective:
first, MITRE evaluated the 12 member sites for compliance with MOU requirements
(including an initial landing page analysis); second, MITRE evaluated the 12 member sites
and the Free File-related pages on IRS.gov against recognized software user interface
usability principles (called heuristics) in order to identify potential usability problem
areas. For both parts, MITRE set-up user accounts and ran scenarios to test the sites in a
systematic, exhaustive fashion. MITRE’s usability assessment used both industryrecognized criteria (heuristics) as well as MITRE’s interpretation of how MOU
requirements translate into usability criteria and whether sites are optimized for mobile
devices.
• Taxpayer Usability Testing. MITRE conducted a study to test the usability of the IRS
Free File portal and assess taxpayers’ actual experience in filing tax returns using three
randomly selected Free File software offerings. MITRE conducted in-person usability
testing sessions with 29 taxpayers who filed their taxes electronically using Free File
software in prior year filings. During the testing, taxpayers were observed as they
completed fictitious tax scenarios using the IRS Free File portal and Free File software. In
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
iv
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
addition to data on the usability of the Free File services, MITRE collected and analyzed
data on the frequency of errors committed, taxpayer satisfaction, and experience (via a
brief questionnaire), and taxpayers’ perceptions of the Free File program.
• Legal Analysis. MITRE’s General Counsel, in coordination with outside counsel,
conducted a review of the MOU’s terms and conditions to determine recommended
revisions to the MOU to more clearly define the parties' responsibilities to enhance and
promote the IRS Free File program, address any shortcomings in the MOU language as it
relates to the prohibition against unfair methods of competition, and provide enforcement
capability via liability clauses if the MOU's terms and conditions are breached by a
Free File member organization.
• Recommendations. Based on the data gathered from the workstreams, MITRE developed
recommendations for overall improvements to the Free File program, potential
enhancements to the MOU, and improvements to usability of IRS.gov and some aspects of
member offerings.
Figure ES-1 shows how each of these workstreams collectively contributed to addressing
targeted assessment areas (assessment objectives) including: documenting the current Free File
program environment, assessing Free File program management and oversight, assessing FFA
member compliance with the MOU, assessing taxpayer experience with respect to the program,
and ultimately drafting recommendations for the program and MOU. Outputs from some
workstreams contributed to more than one assessment area.
Figure ES-1. High-Level Approach to Free File Program Assessment (Aligned to Assessment Objectives).
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
v
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
Addressing Public Perceptions
Throughout the assessment, MITRE considered the perspectives of various media, oversight, and
advisory organizations: the recent ProPublica articles on the Free File program, the 2018 Internal
Revenue Service Advisory Council (IRSAC) Annual Report, and the 2018 National Taxpayer
Advocate Annual Report to Congress4. Three primary themes emerged from these external
sources:
1. Member companies use deceptive strategies to steer taxpayers away from using the Free
File program services towards fee-based services.
2. The IRS Free File Program Office does not provide adequate oversight.
3. Low usage rates signal that the Free File program is not working.
However, MITRE’s findings from this assessment determined that while there may be an
element of truth to each of the criticisms, the data indicate the issues are much more nuanced and
don’t tell the full story.
MITRE conducted the assessment of the Free File program in its current state—the tax software
industry providing free tax preparation and filing services under a MOU with the IRS. It was not
in the scope of MITRE’s effort to address the underlying political issue that is the genesis of
much of the controversy surrounding the Free File program—whether the IRS should create its
own free tax preparation and filing program or continue to rely on the tax software industry to
provide these services. While MITRE recognizes that government services provided to its
citizens are continually evolving, and that Congress may determine in the future that the IRS
should offer that service (and provide it with the appropriate funding to do so), for the purposes
of this study, MITRE assumes that industry will continue to be the entity that provides free tax
return preparation and filing offerings to taxpayers. All of MITRE’s recommendations in this
report are based on that assumption.
Search Engine Avoidance
The issue that seems to have created the most publicity and was at the center of much of the
congressional interest is the practice of coding that caused members’ Free File landing pages to
be excluded from an organic search through Google or Bing search engines. During FS 2019,
five of the 12 FFA members did engage in a practice that excludes their company’s Free File
landing page in organic searches. When questioned about it, most members contended this keeps
them in compliance with the MOU (language in the MOU preamble) and in alignment with IRS
marketing that states Free File is “accessible only through IRS.gov.”5 MITRE did not take a
position on the practice—determination of whether this is a deceptive business practice is not
within MITRE’s legal purview. The perception points to the need for the IRS to take a position
on the practice and incorporate new language into the MOU. MITRE also conducted an in-depth
analysis of what the results of organic searches provided.
4
Internal Revenue Service, "Internal Revenue Service Advisory Council Public Report, Publication 5316, Catalog Number
71824A," Internal Revenue Service, Washington, D.C., 2018.; Taxpayer Advocate Service, "National Taxpayer Advocate Annual
Report to Congress," Taxpayer Advocate Service, Washington, D.C., 2018.; Series of ProPublica articles from March 20, 2017 to
June 5, 2019, available at https://www.propublica.org
5 Internal Revenue Service, "Changes strengthen program; Free File can also help navigate new tax law provisions," Internal
Revenue Service, 11 january 2019. [Online]. Available: https://www.irs.gov/newsroom/irs-free-file-opens-today-in-advance-oftax-season. [Accessed 10 September 2019].
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
vi
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
What’s been said …
Member companies use deceptive
strategies to steer taxpayers away from
using the Free File program services
towards paid commercial services
What the data show…
•
Five of the 12 members used a coding device6 to keep
their Free File landing page out of organic searches.
•
Based on a representative sample of organic (unpaid)
search keywords for free tax filing, the highest
number of searches went to IRS.gov (>780,000). The
top member landing page had 698,000 searches. The
next highest was less than 180,000 searches. The
total unpaid traffic to the 12 members was 1.8
million.
•
Paid searches resulted in the highest amount of
traffic (using the same representative sample of
keywords as above), more than 10 million total to the
12 member sites combined, none to IRS.gov (IRS does
not pay for advertising).
•
IRS.gov Free File landing page received 2.8 million
entrances7 (the landing page was the first page a user
saw when arriving on IRS.gov) during FS 2019.
•
Four of the five members who engaged in the
practice had increases in Free Filing in FS 2019, and
one remained relatively flat. Five of the seven who
did not engage in the practice had increases in Free
Filing, and two had decreases.
•
Number of taxpayers who received free filing outside
of Free File through free commercial offerings is not
publicly available. However, according to the FFA, the
current members provided a total of more than 17
million of these free commercial filings in FS 20198.
(ProPublica).
6
“meta
Because the IRS does not employ any tracking codes once a user leaves the site, there is no way to know how many of these 2.8
million entrances became Free File users.
8
The actual number of free offerings may be higher. One company who was a member in 2019 did not provide a number.
7
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
vii
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
Table ES-1 depicts the effects of organic versus paid searches for the top 108 non-branded high
value keyword search terms that drove traffic to Free File member landing pages. This data was
derived from website traffic information downloaded from SEMrush9, a digital market software
as a service company.
Table ES-1. Comparing Paid Search Traffic to Organic Search Traffic9.
The data in Table ES-1 shows that for the organic searches, more traffic went to the IRS.gov
Free File landing page than to any one of the individual member sites’ non-Free File landing
pages. However, the most traffic, as would be expected, went to paid advertising. Seven of the
12 Free File members bought ads for keywords related to free tax filing that directed traffic to
the landing pages for their non-Free File free tax filing software offering, resulting in five times
more traffic than organic searches. The data used for this analysis (not shown in this table)
indicated that no member had paid advertising for their Free File landing pages. This is not
unexpected, as the MOU specifically assigns advertising responsibility to the IRS. However,
during the FFA member interviews, one member stated that their company does employ
advertising practices for the Free File program10.
Program Oversight
The perception that members were undertaking potentially deceptive practices that resulted in
vulnerable taxpayers paying for tax services–compounded by the program’s low participation
rates–led to criticism that the IRS Free File Program Office was not providing adequate oversight
of member behavior. What this critique overlooks is the fact that the Free File program operates
as a public-private partnership and, as such, program oversight is a mutual, collaborative effort
9
SEMrush Inc., "All-in-one Marketing Toolkit," SEMrush Inc., 2019. [Online]. Available: https://www.semrush.com/. [Accessed
week of 27 May 2019].
10 During MITRE interviews with representatives of FFA member organizations, one member indicated that their company pays
for advertising of the Free File program through a number of methods. However, the interview did not include detailed discussion
of advertising practices specific to key word search. For this analysis, MITRE looked specifically at ads purchased for key words
related to free tax filing. The results of this analysis indicated that no purchased ads directed users to the landing pages of Free
File offers.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
viii
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
with the industry, represented by the Free File Alliance. While there is room for improvement,
based on assessment findings, MITRE considers the IRS’s current compliance processes
adequate and effective to support the integrity of the program. In part, because of this longstanding partnership with FFA and continuous assessment and improvement of the program as it
has evolved.
What’s been said …
What the data show…
•
The IRS Free File Program Office does
not provide adequate oversight (IRSAC,
Taxpayer Advocate Service [TAS],
members of Congress).
•
•
•
Because the Free File program is a PPP, and not a
paid contractual relationship, it must work
collaboratively with industry on oversight of the
program.
IRS program office conducts member website
reviews, evaluates member offer coverage and
monitors member acceptance rates, while working
closely with FFA to address, document, and resolve
any identified or potential compliance breaches.
IRS’s existing agency governance of the Free File
program effectively allows the program to fulfill the
IRS’s currently articulated program goals.
While the program office does not have updated,
measurable objectives for internal agency
performance, its shared oversight of member
compliance strictly follows the provisions of the MOU
and meets joint program objectives.
Taxpayer Participation in Free File Program
Finally, what has led many to question the overall effectiveness of the Free File program is the
participation numbers themselves. In 2018, fewer than 3 million filers used the Free File program
to file their tax year 2017 returns out of nearly 104 million taxpayers that meet the eligibility
requirements for the program. However, these numbers don’t tell the entire story and take the
issues of taxpayer choice and taxpayer behavior out of the equation. The Free File program will
generally appeal to those taxpayers that prefer a “do-it-yourself” (DIY) method of tax
preparation and filing. Looking retrospectively at the eligible population from FS 2018 and how
those taxpayers chose to file can help the IRS better understand their target population for Free
File.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
ix
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
What’s been said…
What the data show…
•
Low usage rates signal that the Free
File program is not working (TAS,
ProPublica, members of Congress).
•
•
•
•
•
•
While 103.8 million taxpayers were technically
eligible to use Free File in FS2018, half of those used a
paid preparer, taking them out of the do-it-yourself
(DIY) tax preparation pool in that year.
Another 9.8 million taxpayers in the DIY pool received
refund anticipation checks (RACs) or refund
anticipation loans (RALs), taking them out of the
eligible pool.
Though paper filers are a potential target population
for Free File, the 8.8 million DIY taxpayers who filed
on paper removed themselves from the 2018
candidate pool.
Three million taxpayers went to Volunteer Income
Tax Assistance (VITA) sites, removing them from the
candidate pool.
The actual number of filers using DIY software and
eligible to use Free File in FS 2018 was about 30
million (see Figure ES-2). Of those, about 9 percent
used Free File.
Of those 30 million, a significant percentage (17.7
million for FS 2019 according to FFA) received free
commercial filing from the current members.
The VITA program (the bricks and mortar assisted
free equivalent of Free File) had approximately 3
million taxpayers use their service VITA has
thousands of volunteer preparers, and more than
2,000 partner organizations who conduct outreach
for their 11,000 sites nationwide.
As shown in Figure ES-2, once the number of taxpayers that choose to file via a different
method—e.g., with a paid preparer or through a Volunteer Income Tax Assistance (VITA)—or
elects to receive their tax refund immediately through a refund anticipation loan (RAL) or a
refund anticipation check (RAC)—are removed due to their preferences, the actual pool of
candidates for the Free File program from 2018 is about 30 million taxpayers. A significant
percentage of those receive free filing outside the Free File program. Though the numbers for
free commercial filings are unknown for tax year (TY) 2017, a study by the FFA in 2015 showed
it to be 19 million for that year from the 14 members, and the FFA attested to 17.7 million for
TY 2018 from 11 current members.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
x
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
103,778,000 Eligibile Taxpayers - TY 2017
Potential Add'l Free
Filers, 27,684,000
free commercial
offerings
Paid Preparers
RAC/RAL self-prepared
Paid Preparers,
51,860,000
Free Filers, 2,649,000
Paper filers - self-prepared
VITA/TCE
Free Filers
VITA/TCE, 3,085,000
Potential Add'l Free Filers
Paper filers - selfprepared, 8,750,000
RAC/RAL self-prepared,
9,750,000
Figure ES-2. Break-down of 103,778,000 Taxpayer Eligible for Free File (TY2017).
It is outside the scope of this assessment for MITRE to determine what the proper percentage of
Free File taxpayers should or could be. Without further analysis and research into taxpayer
behavior it is impractical to make that type of determination. Presumably if awareness was at
peak saturation, participation would be greater than it is now. However, there are many
competing free offers in the market that even taxpayers who are aware of Free File may gravitate
to. The IRS’s ability affect the Free File members’ behavior in terms of their commercial
offerings, or taxpayers’ behavior in terms of how they choose to file their return, is limited. What
the IRS does have control over is what and how much they do to ensure program awareness.
Free File Member Assessment Results – MOU Compliance
Recent attention and criticisms notwithstanding, the Free File program has operated for 17 years
under the governance of an MOU—an agreement that all members must abide by or be removed
from the program. The IRS asked MITRE to conduct a fact-finding review of each member to
determine whether members were in compliance with the most recent version of the MOU at the
time of the assessment.
To assess MOU compliance, MITRE drew findings from IRS interviews and document reviews,
interviews with Alliance leadership and an Alliance-contracted reviewer, and fact-finding
interviews with representatives from each of the Free File member organizations. In addition,
MITRE incorporated the results of the hands-on taxpayer journey reviews of member Free File
landing pages and websites, as well as results from the web forensics analyses where applicable.
Overall, based on the various assessment workstreams, all Free File members were compliant
with the MOU, in particular with Article 4, Standards of Practice.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
xi
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
Article 4 outlines level of service, software and website functionality, security, and disclosure of
forms and schedules. In addition, it includes taxpayer service options and other requirements
such as innovation. MITRE found two instances of MOU non-compliance and an inconclusive
finding for nine requirements for some or all members. There were also two requirements that
were not applicable at the time of review for several members.
The two non-compliance instances were:
• One member was non-compliant with a sentence in MOU 4.15.4. The MOU states that
“Members must clearly list their free customer options … on the Member’s Free File
Landing Page (or such page must have a clear and prominent link to such disclosures
directly from this page).” Based on MITRE’s landing page analysis, the link was not
“clear and prominent” on their landing page11.
• One member was non-compliant with MOU 4.32.4. The MOU requires members to
“communicate not less than once annually via email with their taxpayer customers who
used Free File services and completed their returns through Free File in the immediately
preceding tax year.” The representative from the member acknowledged they were
supposed to initiate an email to their prior tax season Free File taxpayers and self-attested
that they did not provide the required email prior to the tax season.
While MITRE deems these to be non-compliant issues, none rise to the level of severity or
concern regarding the integrity of the Free File program.
Taxpayer Experience Analysis
In addition to assessing compliance with MOU requirements, MITRE also conducted research
activities to understand the taxpayer experience using the Free File program. These research
activities were designed to represent the perspective of the end user of the Free File system—the
taxpayer. As such, MITRE identified pain points and areas for improvement to make the
experience easier, clearer, and more accurate for taxpayers. This included an expert review of
IRS.gov and the 12 member sites for potential usability issues, and usability testing interviews
with actual taxpayers.
Taxpayer Journey Analysis
MITRE conducted a taxpayer journey analysis to evaluate potential usability issues with the Free
File member systems and Free File-related web pages. The taxpayer journey analysis also
evaluated member landing pages for compliance with the MOU, as described above. The rest of
the analysis was focused on the taxpayer experience from a usability perspective and should not
be interpreted as evidence for or against member compliance with the MOU.
The taxpayer journey analysis used both established industry criteria and MOU requirements
related to system usability and user expectations for system functionality as anchor points for the
analysis. Specifically, MITRE used the following sets of criteria to conduct the review of the 12
member sites and IRS.gov’s Free File-related web pages:
11
As of the date of this report, the member has updated their Free File landing page.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
xii
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
• Nielsen’s 10 usability heuristics12. These 10 criteria, called “heuristics,” are widely
recognized measures of system usability. Taken together, the 10 heuristics construct a
measure of user experience and include factors such as “match between system and the
real world,” “error prevention,” “flexibility and ease of use,” and “consistency and
standards.”
• MOU requirements, from a usability perspective (user experience [UX] MOU
Adherence). MITRE examined MOU requirements pertaining to the user experience and
constructed usability criteria from these MOU requirements. These criteria assess user
interface elements closely associated with MOU requirements and included factors such as
“understands offerings available” and “transparency around ineligibility.” MITRE’s
assessment against these criteria are not included in the evaluation of MOU compliance
presented in Section 4.0.
• Mobile experience. MITRE evaluated the extent to which the examined sites were
optimized for mobile device use.
For all three of these groups of criteria, the objective of the evaluation was to identify potential
usability issues and prioritize them via a 1-5 (1 = worst, 5 = best) severity score.
MITRE’s analysis found that, while the Free File members adhere to many UX best practices as
measured by Nielsen’s usability heuristics, they could improve the most in “error prevention”
and “match between system and real world” heuristics. With respect to examining the sites in
terms of a usability interpretation of MOU requirements, the area where members have the most
need for improvement was “transparency around ineligibility.”
All of the scores from the review are rolled up into a dashboard that shows a quick view of the
summary scores for user experience, UX MOU adherence, and mobile experience, with a more
detailed breakout of the scores from the heuristic review by each heuristic and grouping of MOU
requirements (see Section 5.1). Figure ES-3 shows the summary dashboard from the taxpayer
journey heuristic analysis, and aggregates Nielsen’s 10 heuristics under “user experience” issues
related to the usability in the MOU under “UX MOU adherence,” and whether or not the site is
mobile-optimized under “mobile experience.” The 12 member sites are shown in the columns by
number and the heuristic dimensions are displayed down the rows.
12
Nielsen’s 10 Usability Heuristics: Nielsen, Jakob. (1994). Heuristic evaluation. In Nielsen, J., and Mack, R.L. (Eds.), Usability
Inspection Methods, John Wiley & Sons, New York, NY
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
xiii
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
Figure ES-3. Heuristic Evaluation Dashboard Summary.
The top two highest-rated members both had exceptional user experience, intuitive mobile
experiences, and followed the MOU’s guidance related to taxpayer experience and usability.
Other members might consider mirroring the transparency Member 10 has around Free File
eligibility, which lets the user know quickly whether or not they are eligible to proceed through
the Free File process.
Two members received the lowest ratings for the usability of their user interfaces. Three
members did not have mobile-optimized sites, which would make it very difficult for users to file
their taxes from a mobile device.
Taxpayer Usability Analysis
MITRE conducted the taxpayer usability study via in-person interviews with 29 taxpayers who
previously used Free File from selected zip codes in Chicago. To select an interview location,
MITRE reviewed the density of TY 2018 Free Filers in multiple clusters of zip codes in U.S.
metropolitan areas; Chicago had a relatively high volume of Free Filers, which allowed MITRE
to send invitation letters to potential participants and elicit a sufficient interview sample. MITRE
randomly selected a subset of Chicago Free Filers to receive invitation letters to participate.
The objectives of these interviews were to understand how taxpayers locate the Free File
program options; understand how taxpayers select a software provider; capture usability issues
with IRS websites related to the Free File program; capture usability issues with specific vendor
platforms; and understand taxpayers’ previous experiences with the Free File program. During
the sessions, participants completed fictitious scenarios to evaluate the experiences of completing
three tasks: 1) locating the Free File program online; 2) selecting a Free File software offer,
including with the software lookup tool; and 3) completing a tax return. MITRE randomly
selected three Free File member software offers to test with taxpayers. Each participant filed a
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
xiv
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
fictitious return with one of the three selected offers. Taxpayers used mocked-up information
(e.g., fictitious Form W-2s and other documentation) to complete the tasks.
MITRE identified and reported all usability issues observed by the research team with each of
the three Free File usability tasks and provided suggested solutions for addressing each of these
issues individually. Drawing on the data collected from the usability tasks, analysis of qualitative
data gathered during debriefing and with semi-structured interview questions, the satisfaction
and user experience questionnaires, and metrics on taxpayer behaviors and selections during the
tasks, MITRE identified the several high-level findings. When it comes to navigating to the Free
File program and selecting a member offer, taxpayers:
• Lacked understanding of what the Free File program is and its relationship to the IRS
• Struggled to find the Free File landing page, including through IRS.gov
• Experienced difficulties and confusion with selecting a software offering, particularly
when evaluating Earned Income Tax Credit (EITC) eligibility
• Expressed that they were overwhelmed with choices and information
• Used strategies beyond eligibility criteria to select software offers
With respect to using member-specific Free File program tax preparation software, common
usability issues included trouble finding where to report adjustments to income, difficulty
choosing between filing statuses, and trouble navigating different systems’ “flow.”
Overall, taxpayer ratings of ease and satisfaction varied by task, vendor, and taxpayer
demographic characteristics.
Findings and Recommendations
Building upon the analysis from the various workstreams in this assessment, MITRE provides
the following findings and recommendations for improvements to the Free File program and to
the Free File MOU that governs the program. The findings and recommendations are grouped by
key themes that emerged through the execution of the MITRE assessment. The recommendations
include actions that the IRS may take to improve the integrity and effectiveness of the Free File
program.
Where appropriate, MITRE provides workstream-specific recommendations the body of this
assessment.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
xv
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
1. Objectives of the Free File program have not been updated since the
program’s inception.
Findings
Recommendations
The IRS and FFA have not revisited the
objectives of the program since its
inception. The objectives (reflected in the
MOU) are outdated and don’t reflect
leadership’s current view of program
success.
MITRE recommends that IRS leadership, in
collaboration with the FFA, define new program
objectives and create metrics to determine the
Program Office’s performance against those
objectives. Recognizing that the IRS and FFA
will need to negotiate any formal changes to
joint program objectives, MITRE recommends
The program objective of providing the
that IRS develop internal objectives and metrics
venue for free tax filing for 70 percent of
that reflect its own priorities. While agreement
the population has been met. The e-filing
between the IRS and FFA would be the ideal,
objective has been met. No other metrics
currently exist by which the program office translating priorities into objectives and metrics
should not be contingent on that agreement.
can measure its performance.
For example, according to IRS leadership, its
focus for free file is less about target number of
participants, and more about maximizing
awareness to ensure people recognize it is a
choice.
Potential metrics could include:
•
•
•
•
•
Number of companies participating,
potentially targeting a specific market
share
Actual number of Free File taxpayer
candidate pool/participants (e.g., xx% of
the DIY community minus RALs/RACs)
Metrics associated with compliance (e.g.,
offering parameters, etc.)
Awareness numbers (see additional
recommendations)
Web analysis metrics (e.g., number of
clicks on the Free File link on the
IRS.gov home page, percentage of select
keyword searches that bring taxpayers to
the IRS.gov landing page, etc.)
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
xvi
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
2. If Free File objectives are revised to focus on taxpayer awareness, more
needs to be known about the target population and taxpayer behavior.
Findings
Recommendations
The IRS has collected demographics data
on users of Free File and migration data of
those users, but they do not have
demographics data of the potential pool of
Free File users. This data would be helpful
in targeting outreach. (see Section 3.3)
MITRE recommends the IRS conduct data
analysis of the demographics of the population
who are prime candidates for Free File but not
using it.
The total population pool of 103 million
taxpayers who are eligible may not
represent an accurate picture of the pool of
taxpayers who are candidates for Free File.
MITRE recommends the IRS conduct a taxpayer
behavior study to better understand the factors
involved in a taxpayer’s choice of filing
methods. Understanding taxpayer motivation
Taxpayers make choices that take them out and choices could help determine what would be
an acceptable Free File number, if a number is
of the candidate pool. About 9 percent of
needed at all. This aligns with the IRS-related
taxpayers who have not taken themselves
out of the candidate pool by (for example) priorities of the internal MITRE research
using a paid preparer or choosing a RAC or program and will be proposed for a future
research project.
RAL, use Free File (not taking into
consideration those who choose free
commercial offerings) There is no
consensus on what would be an acceptable
number.
While each individual member receives
customer feedback in various ways for
their own Free File product, the IRS has
not conducted a customer survey since
2009.
MITRE recommends the IRS conduct a
customer survey of Free File users specifically to
the IRS experience of Free File (not the software
they filed their return through) to use as a
baseline and create an online survey for users to
capture continuous feedback.
The IRS has no access to data that would
indicate whether a DIY filer paid for a
return or received a free commercial
offering. As the issue is whether a lowincome taxpayer has to pay for filing,
knowing how many taxpayers receive free
filings either through Free File or a
commercial product would give the IRS a
better understanding of their target
markets.
MITRE recommends, that in the spirit of
transparency, the IRS propose the industry to use
an indicator on free commercial filings
Or
MITRE recommends the IRS request that FFA
provide them the aggregate number of taxpayers
who file their taxes using free commercial
products each year. When developing metrics,
this could help with understanding a more
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
xvii
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
accurate picture of the population of the
Free File candidate pool.
Leadership acknowledged that the number
of returns submitted through Free File have
not increased in the same way that e-file
numbers have and articulated a goal of
increased awareness. The last awareness
study was conducted in 2011.
MITRE recommends the IRS commission an
awareness study to baseline taxpayer awareness
of Free File, specifically of targeted
demographics determined in the prior
recommendation.
Paid advertising yields results. In MITRE’s
web analysis of those searching for free tax
filing, an estimated 85 percent of the traffic
followed a paid search result to member
sites, while the other 15 percent of visits to
member sites came from an organic search
result. Searchers were more than five times
likely to click on a paid search result.
MITRE recommends the IRS conduct a costbenefit analysis to determine whether to
purchase paid advertising to reach taxpayers
searching for free tax filing. We recognize the
IRS may have to seek statutory authority and
budget-specific funding for such advertising if it
was deemed beneficial.
MITRE further recommends, based on the
results of this study—and other recommended
research—the IRS conduct a targeted awareness
campaign to the desired Free File demographics,
and include other IRS-sponsored venues such as
low-income tax clinics (LITCs) and VITA sites,
including IRS’ own website.
3. IRS Free File program is a long-standing public-private partnership, but
there is a lack of public awareness about what that means to the program.
Findings
Recommendations
Free File program is a partnership between
the IRS and industry that requires
cooperation to achieve success. This
partnership requires a balance that serves
the interests of the government and
taxpayers, but also creates a value to the
for-profit industry that provides the
service.
MITRE recommends that a health assessment of
the PPP become part of the overall metrics for
the Free File program, using a generally
accepted framework provided by external
sources, such as that used by MITRE in this
assessment.
The IRS Free File program could benefit
from providing clarity and scope around
the PPP, both internally and externally.
The purpose and scope of PPPs are not
common, and awareness is key to success.
MITRE further recommends that the IRS
Free File Program Office incorporate talking
points addressing the unique nature of the PPP in
terms of joint governance and the value a PPP
provides to all partners as central themes when
addressing critics or publicly discussing the
program to help address this lack of awareness.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
xviii
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
4. Rules that were put into the MOU to promote fairness among FFA
members and promote choices for taxpayers translate into unclear
presentation of offers and confusion for taxpayers.
Findings
Recommendations
The FFA is obligated to provide Free File
to eligible taxpayers through individual
commercial sites such that, when taken in
aggregate these services are offered to the
lowest 70 percent of the taxpayer
population, calculated using AGI.
Moreover, the MOU requires that each
company must offer services to at least 10
percent of total eligible taxpayers but to no
more than 50 percent of taxpayers (“10/50
rule”)—a requirement set to level the
playing field for smaller providers.
MITRE recommends the IRS conduct an
analysis/study of the 10/50 rule. This could
include a confidential market study of the
members to help understand the economic
incentive to assist in preserving competition
amongst the members and promoting taxpayer
economic welfare; a small pilot to determine the
business impact of removing the rule; a
consideration of whether differing rules for the
two large companies would benefit or create
inequity; and/or an in-depth data analysis of how
the different offerings have impacted members’
free file numbers.
While the 10/50 rule has been (and may
continue to be) beneficial to FFA members
to ensure equity, it’s possible that it has
outlived its usefulness. MITRE did not
have adequate time, nor was it within
scope, to do an analysis of how the 10/50
rule impacts Free Filed returns. However,
from a taxpayer perspective, the different
company offerings create confusion as a
taxpayer navigates the various choices.
Even if the rule is not changed, MITRE
recommends simplifying the display of
members’ offering to reduce taxpayer confusion.
5. Usability activities—both with taxpayers and by usability experts—
provide insight into how taxpayers experience various aspects of the
Free File program.
Findings
Recommendations
MITRE conducted a complete heuristic
assessment of the Free File website and
members websites, and a usability
assessment with Free File users. Both
assessments found that improvements
A complete list of recommendations to improve
usability and the user experience on IRS.gov is
provided in the Taxpayer Experience Results
Report (Appendix G) and taxpayer journey
supplemental materials (Appendix F). While
usability-specific, these recommendations
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
xix
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
could be made to enhance the user
experience.
complement the program-level recommendations
above and as such may be repetitive.
MITRE recommends that IRS:
• Standardize how Free File is referenced on
IRS platforms and communications
• Take steps to increase ease of understanding
eligibility requirements
• Ensure that taxpayers understand that
determining eligibility for a software offer is
ultimately their responsibility
• Recommend best practices to increase
usability of members’ tax preparation
software systems
• Conduct research on the usability taxpayer
access and use of the Free File system
• Conduct research on Free File awareness,
attending to different demographic groups’
awareness
• Eliminate or change the EITC eligibility
question in the lookup tool and on the
software offers page
o If the IRS stated on the Free File site that
all taxpayers eligible for EITC are
eligible for Free File, the question could
be removed from the tool. MITRE
recognizes this would be a topic for
negotiation between the IRS and FFA
because of how that could impact some
companies’ appearance in the tool.
o If the IRS chooses to leave EITC in the
lookup, MITRE recommends linking the
question to the EITC eligibility tool, but
ONLY after conducting usability testing
on the EITC eligibility tool and
addressing the results.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
xx
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
6. The MOU is an evolving document that is added to with each new
negotiation, resulting in a piecemeal agreement that is out of date in some
areas.
Findings
Recommendations
The MOU generally has provisions added
with each revision, but few are taken out.
Much of the same language is used, not
only in the objectives, but also in the
technology portion, that has become
outdated (e.g., text CAPTCHA).
MITRE recommends that the IRS and the
Alliance jointly determine new, measurable
objectives for the Free File program, and draft a
new MOU, pulling and using provisions from
the current MOU that are relevant, discarding
provisions that are not, and creating new
provisions based on the current environment.
All but two of the members’ Free File
offerings are the same as their commercial
offerings. The security/privacy rules of
Free File are the same as e-File, and all
Free File members are also members of the
Security Summit where the IRS works
collaboratively with the industry to
improve security.
MITRE recommends simplifying the MOU in its
security and privacy provisions by simply
requiring members to adhere to the eFile/Security Summit requirements and
recommendations.
MITRE’s review of the individual
members’ processes for applying the
Free File indicator (without looking at the
actual coding) indicated most members
used the same process that should place the
indicator only on Free Filed returns.
However, that is not always the case.
MITRE recommends adding language to the
MOU to specify conditions under which the
indicator is applied to ensure quality control.
In MITRE’s data analysis of returns
marked with the Free File indicator,
several thousand returns from one vendor
indicated the presence of a RAC. The
vendor noted that the IRS informed them
of this irregularity, and they discovered a
coding error had mistakenly placed the
indicator on returns that had not been Free
Filed, which was subsequently corrected.
Another vendor also had several hundred
returns with the Free File indicator on
returns with AGI of more than $66,000.
Though not necessarily dependent on the MOU,
MITRE also recommends the IRS develop a
quality control method that would allow the IRS
to validate returns with the Free File indicator
were actually Free Filed.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
xxi
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
7. Search exclusion techniques are the latest new issue for consideration
when entering into the next round of MOU negotiations.
Findings
Recommendations
Five of the companies used a NOINDEX
code to exclude their company’s Free File
landing page from organic searches. They
reported that their belief was that this kept
them compliant with the MOU. However,
while the MOU states the IRS will provide
links to their site, it does not prohibit
taxpayers from accessing their sites
directly. IRS press releases state that
taxpayers can ONLY access Free File sites
through IRS.gov.
MITRE recommends that if the IRS wants
taxpayers to go to Free File sites only through
IRS.gov, the MOU should reflect that
specifically as a provision, as opposed to the
current reference in the preamble.
MITRE further recommends the IRS determine
their position on the use of NOINDEX and add a
provision to the MOU addressing that use.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
xxii
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
Table of Contents
Executive Summary ........................................................................................................................ iii
Introduction ............................................................................................................................. 1
1.1
Background ...................................................................................................................... 2
1.1.1
How Taxpayers Access and Use Free File ................................................................ 3
1.1.2
How the Free File Program Works: IRS and Industry Partners................................. 5
1.1.3
Economic Drivers of Industry .................................................................................... 9
1.1.4
Recent Interest in the Free File Program ................................................................... 9
1.2
Scope .............................................................................................................................. 11
1.3
Assumptions and Limitations ........................................................................................ 12
1.4
Organization of This Document..................................................................................... 14
Methods Overview ................................................................................................................ 15
2.1
Current Environmental Scan .......................................................................................... 17
2.2
Free File Program Analysis ........................................................................................... 18
2.3
Free File Alliance Member Organization Interviews (Site Visits) ................................ 19
2.4
Web Forensics Analyses ................................................................................................ 20
2.5
Taxpayer Journey ........................................................................................................... 22
2.6
Taxpayer Usability Testing ............................................................................................ 26
2.7
Legal Analysis ............................................................................................................... 28
Eligible Taxpayers and Actual Participation ......................................................................... 29
3.1
Overall Trends ............................................................................................................... 29
3.2
Taxpayer Behavior and Awareness of the Free File Program ....................................... 30
3.3
A Closer Look at Eligibility and Participation .............................................................. 31
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
xxiii
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
3.4
Free File and VITA ........................................................................................................ 33
Compliance with the MOU ................................................................................................... 35
4.1
MOU Provisions Set Criteria for Compliance ............................................................... 35
4.2
Free File Alliance Member Adherence to MOU ........................................................... 39
4.2.1
FFA Member Assessment Results: Overview ......................................................... 39
4.2.2
Emerging Issues: Search Techniques....................................................................... 42
4.3
4.2.2.1
How Taxpayers Find the Free File Program ................................................... 42
4.2.2.2
Search and MOU Compliance ......................................................................... 45
Assuring Adherence to MOU ........................................................................................ 47
4.3.1
Program Objectives .................................................................................................. 47
4.3.2
Program Structure .................................................................................................... 50
4.3.3
Member Oversight and Compliance Activities........................................................ 55
4.4
Assessment of Public-Private Partnership ..................................................................... 58
4.5
MOU Renewal in 2021 .................................................................................................. 61
Taxpayer Experience with Free File ..................................................................................... 63
5.1
Taxpayer Journey ........................................................................................................... 64
5.1.1
Heuristic Review of Member Free File Websites .................................................... 64
5.1.2
Heuristic Review of IRS.gov ................................................................................... 67
5.2
Taxpayer Experience Usability Findings ....................................................................... 71
5.2.1
General Free File Program Findings and Usability Issues....................................... 72
5.2.2
Locating the Free File Program Online ................................................................... 72
5.2.3
Selecting a Free File Software Offer, Including with the Software Lookup Tool ... 73
5.2.4
Completing a Tax Return Using a Specific Free File Offer .................................... 75
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
xxiv
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
5.3
Integrated Look at Taxpayer Experience ....................................................................... 76
Recommendations ................................................................................................................. 79
6.1
Improvements to the Free File Program ........................................................................ 79
6.1.1
Define Updated Program Objectives and Create Corresponding Metrics ............... 80
6.1.2
Take Steps to Better Understand the Free File Target Taxpayer Population and
Behavior ................................................................................................................... 80
6.1.3
Increase Awareness of the Free File as a Long-Standing PPP ................................ 82
6.1.4
Simplify Member Offerings to Reduce Taxpayer Confusion .................................. 82
6.2
Recommendations Related to the MOU ........................................................................ 83
6.2.1
Update MOU to Reflect Current Environment ........................................................ 83
6.2.2
Determine the IRS Position on Search Exclusion and Provide Guidance ............... 84
6.2.3
Legal Analysis of MOU ........................................................................................... 85
6.3
Taxpayer Experience-Specific Recommendations ........................................................ 86
Appendix A
The Economics of IRS Free File ......................................................................... A-1
Appendix B
Crosswalk of External Program Reviews ........................................................... B-1
Appendix C
ProPublica Articles on IRS Free File Program ................................................... C-1
Appendix D
Initial Assessment: Overall Search Engine Results ............................................ D-1
Appendix E
How the Major Web Search Engines Work .........................................................E-1
Appendix F
Taxpayer Journey ................................................................................................. F-1
F.1
Heuristic Review of Free File Web Pages .................................................................... F-1
F.2
Heuristic Review of IRS.gov ........................................................................................ F-1
Appendix G
Taxpayer Experience Testing Report.................................................................. G-1
Appendix H
Acronyms and Initials ......................................................................................... H-1
Appendix I
References ............................................................................................................. I-1
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
xxv
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
Table of Tables
Table 1. Taxpayer Eligibility Criteria by Free File Alliance Member (as of August 2019)........... 8
Table 2. Document Organization. ................................................................................................. 14
Table 3. Stakeholder Interview Participants. ................................................................................ 19
Table 4. Free File Program Agreements and MOUs. ................................................................... 37
Table 5. Top Sources of Traffic to the Free File Member Landing Page. .................................... 42
Table 6. Estimated Traffic Distributions from Organic and Paid Search Result to Landing
Pages/Estimated Cost for Paid Traffic. ......................................................................................... 44
Table 7. Free File Program Office Key Areas of Focus. .............................................................. 52
Table 8. Public-Private Partnership Assessment Criteria for Free File Program. ........................ 59
Table of Figures
Figure 1. Free Options for DIY Tax Filing: Free File Program and Free Commercial Offerings. . 4
Figure 2. High-Level Approach to Free File Program Assessment (Aligned to Assessment
Objectives). ................................................................................................................................... 17
Figure 3. Nielsen's 10 Usability Heuristics. .................................................................................. 24
Figure 4. Heuristics to Evaluate MOU Adherence from a Usability Perspective. ....................... 25
Figure 5. Free File Accepted Returns, FY 2003-2018 .................................................................. 30
Figure 6. Economic Cost by Filing Method.................................................................................. 32
Figure 7. Break-down of 103,778,000 Taxpayer Eligible for Free File (TY 2017). .................... 33
Figure 8. Comparison of Two Free Programs: VITA (Preparer-assisted) and Free File (DIY). .. 34
Figure 9. Top Keywords for Member Free File Landing Pages. .................................................. 43
Figure 10. Seasonality of Free File-Related Searches. ................................................................. 45
Figure 11. The “MOU-specified” Path of Free File Through IRS.gov. ....................................... 46
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
xxvi
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
Figure 12. Key IRS Free File Program Management and Oversight Activities for TFF.............. 57
Figure 13. Public-Private Partnership: Balance of Value to Stakeholders. .................................. 58
Figure 14. The MOU as a Negotiation Tool. ................................................................................ 62
Figure 15. Severity Level and Description. .................................................................................. 65
Figure 16. Heuristic Evaluation Dashboard Summary. ................................................................ 65
Figure 17. Heuristic Evaluation Dashboard. ................................................................................. 66
Figure 18. Samples from Heuristics Review for IRS.gov Landing Page. .................................... 67
Figure 19. Design Alternative #1 to Improve Scanning for FFA Member Offers. ...................... 68
Figure 20. Design Alternative #2 to Improve Scanning for FFA Member Offers. ...................... 69
Figure 21. Observed Taxpayer Paths to Reach Free File.............................................................. 78
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
xxvii
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
Introduction
The IRS Free File program is a 17-year old program that makes free online tax preparation
and electronic filing available to 70 percent of the taxpaying population, specifically those
whose adjusted gross income is $66,000 or less. The program is made possible through a
partnership between the IRS and the tax software industry. The partnership is governed
through a mutually negotiated Memorandum of Understanding that has evolved over the
program’s lifespan to address issues as they have arisen.
While cumulatively the program has resulted in approximately 53 million free returns e-filed
and an estimated $1.6 billion dollars in savings to taxpayers1, the Free File program has also
faced criticism in recent years as questions have been raised by Congress and the press about
low annual participation rates and assertions of inadequate oversight, deceptive business
practices, and misaligned roles for government and industry. Considering this attention, IRS
leadership asked MITRE to conduct an independent and objective assessment into the current
state of the program, and to make recommendations. To conduct this assessment, MITRE
assembled a team to take a comprehensive look into multiple aspects of the program;
including, but not limited to, industry compliance with the current MOU, IRS program
management and oversight activities, emerging issues around web search and avoidance
techniques, what is known about the eligible population and participation rates, and taxpayer
experience with industry websites and relevant IRS.gov pages.
Throughout this document there are some reoccurring key themes based on our findings that
may be helpful to keep in mind, including:
• Compliance with the current MOU is only one part of the story. While member
compliance with the MOU is critical, too narrow a focus on that compliance may
overlook nuances of the program that are worth considering. Taking a broader view of
free options for tax filing may provide a more holistic view of Free File program—with
regards to the IRS, industry partners, and the taxpayers they serve—and potential
enhancements for the future.
• Free File program is a partnership between IRS and industry that depends upon
balance and cooperation to achieve success. Any improvements the IRS wants to
make to the program need to be mutually negotiated and enforced by both sides of the
partnership. This partnership requires a balance that serves the interests of the
government and taxpayers, but also creates a value to the for-profit industry that
provides the service.
• More needs to be known about taxpayer choice and filing behavior when it comes
to participation—or lack of participation—in the Free File program.
In January 2019, the Internal Revenue Service (IRS) Free File program (“Free File” or “the
program”) opened for its 17th filing season (FS), providing access to free online tax preparation
and filing to eligible taxpayers through a long-standing partnership with the tax software
industry. This year’s launch came amid criticisms of the program and speculation that industry
partners were not acting in the best interest of taxpayers. In response to these assertions as well
as recent oversight activities, IRS leadership determined that an independent assessment is
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
1
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
necessary to ensure the continued operations and integrity of the IRS Free File program. As the
Service’s federally funded research and development center (FFRDC), the IRS engaged MITRE
to provide IRS, Wage and Investment (W&I) with an independent and objective assessment of
the IRS Free File program in its current state, providing recommendations for improvements to
the program, and by extension to the Memorandum of Understanding (MOU) that governs the
program.
The intended outcome of this effort is that the IRS will ensure continued equitable application
and integrity of the Free File program through improved situational awareness of adherence to
the Free File MOU, potential amendments to the MOU based on emerging issues and legal
analysis, potential program improvements, and overall taxpayer experience with the program.
As the operator of the Center for Enterprise Modernization FFRDC, sponsored by the
Department of the Treasury and the IRS, MITRE is an independent, objective, conflict-free
organization that has no commercial interest in the Free File Alliance (FFA), its members, or the
Free File program. This message was communicated to all interview and usability testing
participants to encourage candid, truthful responses and to engender trust between the team and
the participants.
1.1 Background
The Free File program was established in 2002—implemented during FS 2003 though
partnership between the IRS and the tax software industry— allowing access to online software
that provides free online tax preparation and filing services to eligible moderate- and low-income
taxpayers. The program came about as a direct result of the Office of Management and Budget’s
(OMB) 2001 Quicksilver Task Force that established 24 e-government initiatives designed to
improve government-to-government, government-to-business, and government-to-citizen
electronic capabilities13. In accordance with this OMB directive, and as part of a separate effort
to increase individual taxpayer filings of electronic returns (e-file), the IRS worked with industry
to envision the program, and subsequently entered into a multi-year partnership agreement with a
collection of private sector tax software companies who came together to form the Free File
Alliance, LLC14 (FFA or “the Alliance”). As part of that agreement, in exchange for the private
industry offering free services, “the federal government has pledged to not enter the tax
preparation software and e-filing services marketplace.”15
In 2014, the Free File Alliance officially changed the name of the organization to Free File, Inc.
(FFI) doing business as (dba) the Free File Alliance. While FFI is the listed name on all
agreements and MOUs beginning in 2014, the two names are used interchangeably. This
13
Internal Revenue Service, "Free File: About the Free File Alliance," Internal Revenue Service, 25 March 2019. [Online].
Available: https://www.irs.gov/e-file-providers/about-the-free-file-alliance. [Accessed 2 June 2019].
14 The IRS references the Alliance on IRS.gov “Free File: About the Free File Alliance”, and the organization has not changed
the name of its own website: https://freefilealliance.org/
15 In each MOU, through the 8th MOU of October 31, 2018. For more information, see Internal Revenue Service, "Free File:
About the Free File Alliance," Internal Revenue Service, 25 March 2019. [Online]. Available: https://www.irs.gov/e-fileproviders/about-the-free-file-alliance. [Accessed 2 June 2019].
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
2
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
document will use the term Free File Alliance, the Alliance, or FFA except when FFI is used in a
specific context or quote.
While the number of taxpayers filing electronically has greatly increased since the program’s
inception—almost 88 percent of all individual tax returns were e-filed in fiscal year (FY) 2017
— the use of the Free File program has remained stagnant for the past several years. About 2.5
million people filed returns using Free File software in FY 2017 compared to over 3 million in
FY 2014, and the peak of about 5 million taxpayers in tax year (TY) 200416. Recent data from the
IRS Compliance Data Warehouse (CDW) database shows that Free File numbers started to pick
up since TY 2016. Compared to TY 2017, approximately 74,000 more returns have been filed to
date for TY 2018. These trends are discussed in more detail in Section 3.1.
1.1.1 How Taxpayers Access and Use Free File
The IRS Free File program is currently available to any taxpayer who has an Adjusted Gross
Income (AGI) of $66,000 or less—or 70 percent of all individual taxpayers. The software is
provided by participating Alliance members. For FS 2019 taxpayers chose from 12 companies’
Free File offerings.17 For each available option, software is the do-it-yourself (DIY) variety, that
is, questionnaire format (rather than form-based) that navigates the taxpayer through the tax
filing process without the intervention of a tax practitioner, like a paid preparer or accountant18.
State tax returns may be free or paid depending on the specific Free File offer.
Ideally, users access Free File through IRS.gov19—either going to IRS.gov and clicking through
to find the program’s landing page (the first page of a website or home page) or by navigating
directly to the program’s IRS.gov landing page through a web search engine. The path taken by
taxpayers looking for a DIY free option to file is shown in Figure 1.
16
Taxpayer Advocate Service, "National Taxpayer Advocate Annual Report to Congress: Volume 1," Taxpayer Advocate
Service, Washington, D.C., 2018.; Data Sources: IRS Data Book (FY 2017) and IRS, 2017 Filing Season Statistics (29 Dec
2017). In TY 2016, just under 2.3 percent of all eligible taxpayers submitted returns using the Free File software, and just 0.2
percent of taxpayers used Free File Fillable Forms to submit their returns. IRS response to TAS information request (7 Sept.
2018).7
17 Currently there are only 11 members participating in the Free File program. As of July 2019, Drake [1040.com] chose to
withdraw from the program. However, since they were a part of the program for FS 2019, they are still included in this
assessment.
18 In 2009, the IRS added FFFF for free filing. Unlike the traditional Free File program assessed here, the FFFF is not a
questionnaire-based form, but resembles as fillable pdf that can be filed electronically free of cost. FFFF is available to all
taxpayers regardless of income. It is an “unbranded” product provided to IRS by industry, but all customer service is the
responsibility of IRS Free File program staff. While there is some mention of FFFF in this document, it is out of scope of the
assessment.
19
Internal Revenue Service, "Free File: Do Your Federal Taxes for Free," Internal Revenue Service, 24 June 2019. [Online].
Available: https://www.irs.gov/filing/free-file-do-your-federal-taxes-for-free. [Accessed 2 July 2019].
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
3
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
Figure 1. Free Options for DIY Tax Filing: Free File Program and Free Commercial Offerings20.
Once on the IRS.gov landing page, taxpayers are presented with a list of all Free File offerings
and a brief description of each. Since each member is permitted to have slightly different criteria
for their offering (see Section 1.1.2), the IRS also provides a software look-up tool (also referred
to as a wizard or “Guide Me To A Company” tool) to help taxpayers make a selection that works
best for their situations.
If taxpayers are eligible for Free File (for FS 2019 if AGI is $66,000 or less) and meet the
specific requirements of the member offer they selected, taxpayers will be able to file their taxes
for free through the Free File program. The selected offer will be subject to all the oversight and
regulation outlined in the program MOU.
There is another path that could also result in a free offering—though, importantly, not through
the Free File program (and as, such, not subject to the program regulation). If taxpayers make
more than the $66,000 ceiling, they are not eligible for the Free File program; however, many
commercial software providers do offer free editions of their products apart from their Free File
program offerings. These products may or may not offer the same features and choosing one of
these products may or may not result in actually being able to file for free. These “free
commercial products” are sometimes confused with Free File products, both anecdotally and by
critics of the program. Only taxpayers that access a Free File offering (online software product)
with a direct link from the IRS.gov landing page, are actually participants in the Free File
program
If taxpayers start at IRS.gov but discover that they have an AGI greater than $66,000, they may
leave Free File to seek other alternatives. Similarly, as discussed in Section 4.1, if taxpayers
begin using a Free File product and discover that they are not eligible for a specific offer
20
Icons created by Vectors Point, Lil Squid and monkik from Noun Project.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
4
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
(because of age or other criteria) there are safeguards in place to keep eligible taxpayers in the
program. That is, taxpayers whose AGI is $66,000 or less but do not qualify for a specific
program for other reasons will be sent back to the IRS.gov landing page to select another offer.
Taxpayers who make more than $66,000 can legitimately be offered the commercial product by
the FFA members, free or otherwise.
1.1.2 How the Free File Program Works: IRS and Industry Partners
The roots of the Free File program lie within two foundational political directives—both of
which are also tied to a broader goal of increasing electronic filing. Taken together, the IRS
response to these directives resulted in the partnership between the IRS and the tax software
industry that exists today to provide free e-file services to taxpayers. These are:
• Restructuring and Reform Act of 1998 (RRA 1998).21 The IRS was asked to cooperate
with the private sector to reach a target to have 80 percent of federal tax and information
returns filed electronically by 2007, with the expectation that increased e-filing would
improve tax compliance, lower operating costs, and reduce human errors in tax data
transcription.
• OMB Quicksilver Task Force (2001).22 Part of President George W. Bush’s Management
Agenda, the Free File initiative from this task force directed the IRS to provide free and
secure online tax return preparation and filing services to taxpayers, in anticipation that
reduced cost of tax preparation would increase e-filing.
In 2002, Treasury Secretary Paul O’Neill asked then IRS
• “[It is not the intent] for the IRS
Commissioner Charles Rossotti to partner with the private
to get into the software
sector for its established expertise in electronic tax preparation
business, but rather to open a
and filing.23 This is a critical piece of the program’s history in
constructive dialogue with those
who already have established
understanding the partnership that ensued. In directing the IRS
expertise in this field. In the end,
to enter into this partnership, Treasury recognized the private
this effort should come up with
sector could accomplish the directive more quickly and at a
a better way to save time and
lower cost than the IRS. The IRS then relinquished any efforts
money for both taxpayers and
to provide e-filing services and that became a primary
the Government”
provision of the agreement between the partners. The
- Treasury Secretary Paul O’Neill
participating industry partners, in turn, assumed the role of
January 30, 2002
providing free tax filing for the agreed-upon population of
taxpayers.
This arrangement led to the growth of a public-private partnership (PPP) between the IRS and
the tax preparation software industry. While there is no consensus of how to define a PPP,24 it is
21
The Internal Revenue Service Restructuring and Reform Act of 1998, Pub.L. 105–206, 112 Stat. 685, enacted July 22, 1998.
Internal Revenue Service, "Free File: About the Free File Alliance," Internal Revenue Service, 25 March 2019. [Online].
Available: https://www.irs.gov/e-file-providers/about-the-free-file-alliance. [Accessed 2 June 2019].
23 U.S. Department of the Treasury, Office of Public Affairs, "Treasury, IRS Announce New Efforts to Expand E-Filing,"
Washington D.C., 2002.
24 M. Marsilio, G. Cappellaro and C. Cuccurullo, "The Intellectual Structure Of Research Into PPPs," Public Management
Review, vol. 13, no. 6, pp. 763-782, 2011
22
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
5
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
generally defined as a cooperative agreement between two or more public and private sectors,
typically of long-term nature.25 MITRE discusses the IRS Free File Program Office in more
detail in Section 4.3 and explores assessing a PPP in Section 4.4.
Memorandum of Understanding: Documenting Roles, Rules, and Program Objectives
The partnership described above is governed by a MOU between the IRS and the Free File
Alliance, composed of individual tax software companies who apply for membership with FFA
and who agree to follow the rules and regulations set forth in the MOU. Individual provisions of
the MOU describe in detail the terms and conditions required for continued participation in the
program—and ultimately for achieving the program’s objectives. An important distinction, the
MOU is not a contract (i.e., between an organization and a set of vendors) but a mutually
negotiated tool for governance agreed upon by the two partners.
Participation in the partnership—by both the IRS and the individual companies—is voluntary.
Software companies have come and gone throughout the years, and at least one company who
participated in the program in FS 2019 will not be back in FS 2020.
While the companies come together to form a single entity in the Alliance; outside the Alliance
they are still individual companies—with varying sizes, resources, proprietary business models
and priorities. These differences were observed during MITRE interactions with member
companies during site visits and interviews.
The influence of the historical factors described above can be seen in the Free File program
objectives, as stated in the current MOU between the IRS and the Free File Alliance:
• Make tax return preparation and filing easier and reduce the burden on individual
taxpayers, particularly the economically disadvantaged and underserved populations
• Support the IRS’s statutory goals of increased electronic filing, pursuant to the IRS
Restructuring and Reform Act of 1998
• Provide greater service and access to the Services to taxpayers
• Implement one of the proposals in the President’s Fiscal Year 2003 budget, specifically to
encourage further growth in electronic filing by providing taxpayers the option to file their
tax return online without charge using cooperation with, and encouraging competition
within, the private sector
While the wording may be slightly different from the original agreement establishing the
program in October 2002, the objectives in the current MOU have not changed in nearly 17
years, even though e-filing now accounts for 88 percent of all individual income tax filings. (See
Section 4.1for further discussion of how the original agreement between the IRS and FFA has
changed over time and Section 4.3.1 for further discussion of Free File program objectives.)
Also included in the MOU—just below the objectives—is the agreement that IRS will not pursue
its own tax preparation and filing software platform: “In recognition of this commitment, the
federal government has pledged not to enter the tax preparation software and e-filing services
25
G. A. Hodge and C. Greve, "On Public–Private Partnership Performance: A Contemporary Review," Public Works
Management & Policy, vol. 22, no. 1, pp. 55-78, 2016
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
6
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
marketplace.”26 This statement has also not changed in the 17-year program history, and program
supporters and critics alike have mentioned this as a foundational element of the current Free
File program.
Free File Offerings: Why Taxpayer Eligibility Differs Member to Member
The primary criterion for taxpayer eligibility in the program is AGI27. That said, the program is
structured in such a way that individual members can set additional eligibility rules, including
limitations on age, state, Earned Income Tax Credit (EITC) eligibility, etc. As discussed in
Section 5.0, Taxpayer Experience with Free File, the difference in eligibility can sometimes be
confusing to taxpayers. The reason for the variety comes from the program structure agreed upon
in the MOU and provisions written into the agreement to encourage competition among
members and provide choice to taxpayers28.
In 2002, the initial agreement between the IRS and the FFA stipulated that Free File be made
available to at least 60 percent of U.S. taxpayers—with no restriction on income.29 30 After three
filing seasons, the IRS and the FFA extended the agreement and amended it to limit that program
to taxpayers with an AGI equal to or less than 70 percent of all taxpayers31, in order to include
those “least able to afford e-filing tax returns, based upon verifiable characteristics in their tax
return.” The IRS and the Alliance have maintained the 70 percent, AGI-based target ever since32.
Importantly, the 70 percent taxpayer coverage requirement is a combined obligation across all
members of the Alliance, and not for each individual member alone. In other words, the FFA is
obligated to provide Free File to eligible taxpayers through individual member Free File sites
such that, when taken in aggregate these services are offered to the lowest 70 percent of the
taxpayer population, calculated using AGI.
In addition, the MOU requires that each company must offer services to at least 10 percent of
total eligible taxpayers but to no more than 50 percent of taxpayers—a requirement set to level
the playing field for smaller providers (referred to in this document as the “10/50 rule”). Finally,
also in aggregate, the MOU stipulates that every eligible taxpayer must be eligible for at least
26
Internal Revenue Service and Free File, Incorporated, "Eigth Memorandum of Understanding on Service Standards and
Disputes Between the Internal Revenue Service and Free File, Incorporated," Internal Revenue Service, 2018.
27 The IRS defines Adjusted Gross Income as “as gross income minus adjustments to income.” Individual gross income is the
combination of all income earned in a year (e.g. wages, rental income, interest income, dividends, retirement distributions, etc.)
AGI accounts for allowable deductions from the gross income. These deductions are subject change each year and can include,
for example, pre-tax dependent care deductions and pre-tax retirement contributions.
28 M. S. Chu and M. M. Kovalick, "An Analysis of the Free File Program," in National Tax Associate Proceedings, 99th Annual
Conference on Taxation, Boston, 2006
29 Internal Revenue Service and Free File Alliance, LLC, “Free On-Line Electronic Filing Agreement”, Internal Revenue Service,
2002.
30 J. G. Russell, "2006 Tax Return Filing Season and the 2007 Budget Proposal for the Internal Revenue Service: Hearing before
the U.S. House Of Representatives Committee on Ways and Means Subcommittee on Oversight (Statement of J. Russell
George)," Department of Treasury, Washington D.C., 2006.
31 Internal Revenue Service and Free File, Incorporated, "Memorandum of Understanding on Service Standards and Disputes,"
Internal Revenue Service, 2005
32 The volume of eligible taxpayers has changed each filing year based on the most current AGI number that equates 70 percent
of all individual income taxpayers.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
7
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
one Free File offer, so while offers may vary across members, there must be full coverage by the
Alliance as a whole.
Table 1 summarizes current eligibility criteria for taxpayers by Alliance members as of August
2019. MITRE discussions with members during site visit interviews helped illustrate—at a highlevel—why companies chose to structure their eligibility criteria. Given that the 10/50 rule limits
the percentage of the eligible population for each member, member companies make choices
about how to tailor their program based on their own business models and any other factors not
discussed. Members can use the “levers” of AGI, age, state, etc. to make adjustments so that they
stay within the 10/50 rule bounds. As discussed in more detail in Section 4.3.3, the IRS provides
members with a tool (spreadsheet) they can use to make adjustments year to year.
Table 1. Taxpayer Eligibility Criteria by Free File Alliance Member (as of August 2019) 33.
Eligibility
based on
EITC
eligibility?
Eligibility
dependent
on state?
Same
eligibility
criteria
applies when
filing with a
foreign
address?
Free for
active
military
for AGI of
$66,000 or
less?
Free for state
return if
eligible for
free federal
return?
Free File Alliance
Member Company
AGI
Age
1040NOW.NET
$66,000
or less
Up to 60
years**
No
Yes
Yes
Yes
No free state
returns
eSmart Free File
Edition
$66,000
or less
Up to 53
years
No
No
No
Yes
No free state
returns
ezTaxReturn.com
$66,000
or less
Any age
No
Yes
No
No
No free state
returns
FileYourTaxes.com
$9,000 $66,000
Up to 65
years
No
No
Yes
Yes
Yes, for some
states
FreeTax Returns.com
$66,000
or less
Up to 70
years
No
Yes
No
Yes
Yes, for some
states
FreeTaxUSA®IRS
Free File Edition
$35,000
or less
Any age
Yes
No
No
Yes
Yes, for some
states
H&R Block's Free File
$66,000
or less
17 years 51 years
Yes
No
Yes
Yes
Yes, for all
states
Online Taxes at
OLT.com
$14,000 $66,000
Any age
No
No
Yes
Yes
Yes, for all
states
TaxAct®Free File
$55,000
or less
Up to 56
years
Yes
No
Yes
Yes
Yes, for all
states
TaxSlayer
$66,000
or less
Up to 50
years
No
No
Yes
No
Yes, for some
states
TurboTax Free File
Program
$34,000
or less
Any age
Yes
No
Yes
Yes
Yes, for all
states
** Applicable only for some states
33
Internal Revenue Service, " Free File Software Offers," Internal Revenue Service, 30 August 2019. [Online]. Available:
https://apps.irs.gov/app/freeFile/. [Accessed 30 August 2019].
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
8
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
These are the offers—and offer parameters—that taxpayers see on the IRS.gov Free File offers
page. Overall, taxpayers utilizing Free File have the primary responsibility of identifying the
right company based on their circumstance. As described in Section 1.1.1, the IRS offers a
lookup tool to help taxpayers in identifying their eligibility across different member offers.
1.1.3 Economic Drivers of Industry
Though the Free File program has its roots in political directives, the design and implementation
of the program is equally driven by economic forces. The MOU between the IRS and the
Free File Alliance, which describes the terms and conditions for achieving the program’s
objectives, is largely shaped by the economic incentives of the programs’ key stakeholders:
taxpayers (consumers), the IRS (administrator), and tax preparation industry members (service
providers).
Consumer demand for Free File is expected to be a function of the “costs”—in time, money, and
effort—of tax preparation and filing through Free File, and the availability of comparable
substitutes. Similarly, the supply of free tax preparation services is a function of the economic
returns that service providers expect to earn from their program investments. Because Alliance
members are required to offer Free File services to eligible taxpayers free of charge, these
companies must use other means to recover their costs and forgone revenue.
The Free File partnership helps generate additional business for Alliance members, including
from customers who end up using paid tax preparation services. Essentially, these commercial
customers who pay for tax software of Alliance members help subsidize the provision of the free
services. The IRS stands in a unique position of having to balance taxpayer expectations from the
program with the industry members’ outlook for economic gains while ensuring that the agency
remains cost-effective in implementing Free File. The MOU serves as a powerful economic
negotiation tool that the IRS and Alliance members use to maximize the expected returns shaped
by stakeholders’ economic incentives.
These are critical factors in understanding industry motivations for participating in the
partnership and providing free returns. Appendix A delves more deeply into the economic theory
and forces driving the Free File program.
1.1.4 Recent Interest in the Free File Program
MITRE collected and reviewed several reports, studies, and articles as part of this
assessment. These references included analysis of the Free File program by media, tax advocacy,
and oversight organizations who proposed numerous recommendations for consideration. Rather
than duplicate the efforts of these earlier studies, MITRE identified common themes and
incorporated them into this assessment where applicable. The following key artifacts were
reviewed by the MITRE team:
• Internal Revenue Service Advisory Council (IRSAC). IRSAC is a federal advisory
committee whose purpose is to serve as an advisor to the Commissioner of the Internal
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
9
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
Revenue. In 2018, the IRS requested that IRSAC evaluate the value of the Free File
program and identify needed improvements34.
The IRSAC assessment concluded that Free File is still a viable program worth IRS
resources and sponsorship. However, the committee assessment also concluded that the
IRS did not provide adequate oversight over the Free File program and that short- and
long-term goals, objectives, and performance measures needed to be clearly defined to
more effectively assess program performance and success moving forward.
• Taxpayer Advocate Service (TAS). The National Taxpayer Advocate heads an
independent organization within the IRS charged with ensuring that taxpayers are treated
fairly and that they understand their rights. In February 2018, TAS issued a report to
Congress that outlines several concerns regarding the IRS Free File program35.
While the TAS report acknowledged that the IRS renewed its agreement with the Alliance
multiple times to bolster taxpayer eligibility, strengthen security and privacy requirements,
and require members to provide an electronic Free File indictor, it outlined several areas
that the program fell short including insufficient oversight and evaluation, declining usage
of the program, and a lack of a robust demographics analysis to determine the cause,
inconsistency in content quality across member companies, and cross-marketing and
advertising of other services on Free File software platforms, which sometimes confuses
taxpayers.
• Treasury Inspector General for Tax Administration (TIGTA). TIGTA conducted a
review of the Free File program in 2007 to determine the IRS’ effectiveness in
administering the program36. At the time of MITRE’s assessment, TIGTA was conducting
an updated review of the Free File program, which is due to be published in Fall 2019. As
such, this report did not consider TIGTA’s most recent findings and recommendations.
• ProPublica Articles. ProPublica published a series of articles over the past several years
on this subject, focusing primarily on the two largest tax software/preparation companies
(the most relevant to this assessment published between March 20, 2017, and June 5,
2019). ProPublica describes itself as a non-profit newsroom whose mission is “To expose
abuses of power and betrayals of the public trust by government, business, and other
institutions, using the moral force of investigative journalism to spur reform through the
sustained spotlighting of wrongdoing.”37 The publication alleged that the companies had a
deliberate strategy to steer customers away from the free product and into paid versions in
deceptive ways. They further contend the companies hid their free offerings from the
Google search engine, and certain tax preparation companies lied to some customers who
sought refunds in the wake of the reporting. See the section below for additional
information on this series38.
34
Internal Revenue Service, "Internal Revenue Service Advisory Council Public Report, Publication 5316, Catalog Number
71824A," Internal Revenue Service, Washington, D.C., 2018.
35 Taxpayer Advocate Service, "National Taxpayer Advocate Annual Report to Congress: Volume 1," Taxpayer Advocate
Service, Washington, D.C., 2018
36 Treasury Inspector General for Tax Administration, "Additional Action Is Needed to Expand the Use and Improve the
Administration of the Free File Program," U.S. Department of Treasury, Washington, D.C., 2007
37
ProPublica, "About Us," ProPublica, 2019. [Online]. Available: https://www.propublica.org/about/. [Accessed 2 July 2019].
38Series of ProPublica articles from March 20, 2017 to June 5, 2019, available at https://www.propublica.org
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
10
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
A more detailed listing of findings and recommendations from external oversight organizations
and stakeholders is in Appendix B, along with a crosswalk to MITRE’s own recommendations.
An annotated list of ProPublica articles is in Appendix C.
Political Interest: Role of Government and Public Sector in Software Preparation and Filing
Underlying the recent focus on the Free File program is a divergence of views over whether the
IRS should provide its own offering for taxpayers to electronically file their tax returns, or
whether that is a function strictly for the private sector to perform as they have for more than 30
years.
A provision in the recently passed Taxpayer First Act that would have legislatively mandated the
Free File program was taken out just before the Act passed. Many believed this provision would
have barred the IRS from ever providing its own offering. In June, a group of nearly 40 advocacy
and community groups, along with 75 academicians and tax experts sent a letter to Congress
urging them to pull the provision and conduct an inquiry into the Free File program. The letter
cited the series of articles by ProPublica described in the previous section.
ProPublica reports that certain tax preparation software companies have lobbied for years to
permanently bar the IRS from offering its own free alternative, arguing that governmental
encroachment at both the federal and state levels may present a continued competitive threat to
the industry for the foreseeable future. They say that the agreement with the companies not only
keeps the federal agency from developing a direct filling option, but also an automated “returnfree” option in which the IRS could calculate most individuals’ taxes.
In 2010, MITRE conducted a feasibility study of the IRS offering its own e-filing program and
concluded it was neither cost-beneficial, nor could the IRS keep pace with the innovation of the
private sector39. MITRE recognizes that the environment of government services provided to its
citizens is continually evolving, and that Congress may determine in the future whether the IRS
should offer that service and provide it with the appropriate funding to do so.
MITRE was not asked to weigh in on this debate with this assessment, but rather to look at the
Free File program itself in terms of the member compliance with the MOU and to make
recommendations for the program based on its assessment. However, it is essential to understand
the underlying political arguments, because simply knowing whether a company is in
compliance with an MOU that was jointly negotiated between government and industry may not
satisfy those who want the program investigated.
In this report, MITRE considered all sides of the program, as evidenced by the broad scope of
the assessment outlined below.
1.2 Scope
As part of this independent assessment, MITRE conducted a comprehensive review of the
current IRS Free File program from three perspectives: Free File Alliance and its member
organizations, IRS Free File Program Office leadership and staff, and taxpayers.
Specifically, the scope of the assessment includes:
39
The MITRE Corporation, "Advancing E-file Study - Phase 2 Report," Internal Revenue Service, Washington, D.C., 2010.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
11
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
• The multi-year agreement/MOU structure that governs the Free File program, and how it
could be improved
• Member compliance with specific provisions of the current MOU
• Compliance and oversight activities conducted by the IRS Free File Program Office
• Stakeholder understanding of historical and current objectives of the program and vision
of success
• Assertions from media (specifically ProPublica) and recommendations from oversight
organizations IRSAC, TAS, and TIGTA
• Taxpayer experience with the Free File program, based both on an assessment by human
factor and user-experience engineers and usability testing with actual taxpayers
• Observations and suggestions from the Alliance, IRS, taxpayers, and other stakeholders
for improvements to the program
• Web forensics analysis with focus on search engine coding and search traffic
• Data analysis of e-filed/Free Filed returns in the ecosystem of return filings
• Analysis of the PPP and the role it plays in the Free File program
• Economic analysis of the Free File program
Areas that were out of scope for the assessment included:
• State tax filing. The assessment focused on the federal Free File program only. While
there are aspects of the program that pertain to states, MITRE did not consider state tax
filing itself, nor did MITRE speak with any state stakeholders.
• Interviews with oversight agencies or media organizations. While MITRE reviewed
relevant documentation from oversight agencies and articles in the press (specifically
ProPublica), MITRE did not conduct interviews or speak directly with either.
• Free File Fillable Forms. Taxpayers with no restriction on AGI can opt to fill out
electronic versions of IRS forms and submit electronically for free from IRS.gov. While
Free File Fillable Forms (FFFF) are provided to the IRS by the FFA, the form is
unbranded. MITRE considered FFFF only in terms of overall usage of the Free File
program and as part of responsibilities of IRS Free File Program Office. Beyond that,
FFFF is not part of this assessment.
1.3 Assumptions and Limitations
MITRE conducted this assessment with the following assumptions:
• As discussed in Section 1.1.4, for the purposes of this study, MITRE assumes that industry
will continue, at least in the near future, to be the entity that provides free return offerings
to taxpayers. All of MITRE’s recommendations in this report follow that assumption.
• Keyword search in our web analysis is based on a representative sample of organic
(unpaid) search keywords for free tax filing, via two primary search engines, Google and
Bing. There are many other search engines that taxpayers could use (Yahoo, Baidu, ASK,
AOL, etc.), though Google and Bing are the largest. While we used more than 100 of the
top search terms, the terms were not exhaustive.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
12
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
• MITRE used TY 2017 data from the IRS CDW in the data analysis portions of the
assessment. Though TY 2018 data is available, because taxpayers with extensions can file
2018 tax returns until October 15, 2019, it is not considered a full data set. Therefore,
MITRE used TY 2017 CDW data for all analysis, unless otherwise stated.
• MITRE conducted an initial assessment that included: a landing page compliance
assessment, a search avoidance analysis, and a legal analysis. The results are included in
this report where appropriate, and results in this report supersede any results shared with
the IRS before this formal deliverable.
In addition, to preserve anonymity, MITRE “masked” member company names throughout this
report—with the exception of the FS 2019 member offerings, which are available publicly on
IRS.gov. MITRE provided the IRS with a member key.
The MITRE team also operated under several limitations and constraints for this study,
including:
• Data limitations.
o All values in Figure ES-2 and Figure 7 were drawn from CDW queries for TY 2017
(FS 2018) with the exception of the fillable form portion of the Free File number.
Because fillable forms were not in the query, MITRE used data provided from the IRS
(W&I Research, Electronic Tax Administration Research and Analysis System
[ETARAS]) of 295,000 fillable form users. The Free File number differs slightly as
the IRS uses calendar year data.
o The number of free commercial filings cited were provided by the FFA by each of the
current members (one member recently dropped out of the program and did not
provide numbers). These numbers are not publicly available, and MITRE cannot
independently verify them.
o Absence of detailed industry, firm, and taxpayer-level data on electronic tax
preparation and filing services it is difficult to develop a full economic
characterization of Free File. Notwithstanding, the economic analysis in Appendix A
uses available proprietary and public data, an extensive literature review, and simple
economic concepts to establish an economic perspective of the program.
• FFA member interviews. Ten of the 12 interviews with members were in person. One
was by phone with written responses provided, and one was just by written response.
• Proprietary information. In general, members were very candid and open. As would be
expected, their legal counsel advised them to not answer questions that were deemed to be
proprietary. However, that was not a limitation in terms of determining compliance with
the MOU and MITRE was able to complete our assessment for each member.
• Self-attestation. MITRE was not able to physically observe compliance with some
provisions of the MOU. For those provisions, the members either verbally or in writing
attested to their compliance.
• Taxpayer journey limitations. For taxpayer journey activities, the MITRE team created
user accounts to access each of the member software packages. In doing so, the team was
able to simulate tax preparation for three scenarios for each package. However, the team
did not actually file taxes using these accounts. Therefore, any screenshots or activities
that would take place after submitting the return were out of scope for the team.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
13
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
• Usability specific limitations. The taxpayer usability study has several limitations that
may affect the external validity of this study, meaning the extent to which the results of
this study can be generalized beyond the context of the study setting. The complete set of
limitations are in the accompanying Taxpayer Experience Results report, Section 1.3.
1.4 Organization of This Document
This document is organized into the following sections, as shown in Table 2.
Table 2. Document Organization.
Section Number and Title
1. Introduction
2. Methods Overview
Contents
Describes the reason for this assessment
and provides relevant background
information to set context. Outlines the
scope of the assessment, any assumption
and limitations, and the organization of this
document.
Provides a high-level overview of the
methodological approach to this study.
Findings
3. Eligible Taxpayers and Actual
Participation
Provides findings from TY 2017 data analysis
of eligible taxpayers
4. Compliance with the MOU
Provides overview of current MOU,
compliance scorecard for all FFA members,
discussion of search and search avoidance,
results of Program Office oversight activity
review, and guidance for assessing publicprivate partnerships
Provides detailed findings from expert
usability review (taxpayer journey) and
taxpayer usability testing
5. Taxpayer Experience with Free File
Recommendations and Appendices
6. Recommendations
7. Appendices
Provides consolidated findings and
recommendations for program office, MOU
updates based on program assessment,
MOU updates based on legal assessment,
taxpayer journey and taxpayer experience
Economics of Free File, crosswalk of MITRE
recommendations and oversight reports,
taxpayer journey results, taxpayer usability
results, search engines overview, etc.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
14
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
Methods Overview
For this comprehensive assessment of the Free File program, MITRE used a crossdisciplinary approach to focus on three key objectives: assessing Free File program
management and oversight, assessing FFA member compliance, and assessing overall
taxpayer experience. Highlights of this approach include:
• Multiple workstreams worked concurrently to consider the program from
different perspectives. This included a data analysis of the eligible population; a
comprehensive web analysis that included web forensics to determine where taxpayers
landed from organic searches and how coding and paid advertising influenced that;
taxpayer usability from actual taxpayers’ experience as well as human factor engineers
conducting heuristic analyses of both IRS.gov and member sites; and a legal analysis of
the MOU to highlight changes that could improve the program.
• Taken together, this allowed MITRE to take a holistic approach to assessing the
Free File program. While these interdisciplinary workstreams worked independently,
they each worked toward the same stated objectives to bring together a holistic view of
MOU compliance and improvements that could be made to the program.
• Each workstream used rigorous methods to produce findings contained in this
report. Each discipline used the appropriate methodology to apply rigor and clearly
document a data-driven, evidence-based approach to determine recommendations.
Beginning in June 2019, MITRE established several workstreams for addressing a set of targeted
assessment areas. Taken together, these workstreams produced findings and recommendations
related to Free File member compliance with the MOU and potential enhancements to the
Free File program. Highlights of the assessment approach include:
• Current environmental scan. MITRE conducted an environmental scan to better
understand the Free File program in terms of historical context, taxpayer participation, the
role of the IRS and industry in its PPP, the economics of “free” as it pertains to Free File,
and recent interest in the program.
• Free File program analysis. MITRE conducted stakeholder interviews—including with
IRS leadership—and performed document reviews to assess the IRS Free File program’s
alignment to objectives, performance, and the IRS management and oversight functions.
• FFA member organization interviews (“site visits”). MITRE conducted interviews with
representatives from each Free File member organization to collect information on their
interactions and agreements with the IRS regarding the Free File program and applicable
business processes, practices, and techniques for how each Free File member determines if
a taxpayer qualifies for Free File. Each of the interviews followed a similar approach:
standard interview protocols, targeted questions (both related to the IRS Free File program
as well as the MOU Article 4, Standards of Practice) and in-person or phone interviews
with participants from each Free File member organizations (selected by the member
company’s leadership).
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
15
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
• Web forensics analyses. MITRE performed an analysis to determine if any coding within
the member’s website influences search engine routing. In addition, MITRE analyzed web
search query traffic metrics for internet searches that signaled an intent to find free tax
filing services, based on a representative sample of organic (unpaid) search keywords for
free tax filing. This included a click-through analysis of IRS.gov site usage metrics to
investigate where taxpayers went from the IRS Free File portal web page. This analysis
was also used to determine how often taxpayers searched for free tax filing services
(especially during tax filing season) using two primary search engines, Google and Bing;
to what site(s) these users went following their searches; and the differences in search
engine results.
• Taxpayer journey. MITRE conducted an expert review of the IRS Free File portal as
well as each individual Free File member offer. This review was twofold in its objective:
first, MITRE evaluated the 12 member sites for compliance with MOU requirements
(including an initial landing page analysis); second, MITRE evaluated the 12 member sites
and the Free File-related pages on IRS.gov against recognized software user interface
usability principles (called heuristics) in order to identify potential usability problem
areas. For both parts, MITRE set-up user accounts and ran scenarios to test the sites in a
systematic, exhaustive fashion. MITRE’s usability assessment used both industryrecognized criteria (heuristics) as well as MITRE’s interpretation of how MOU
requirements translate into usability criteria and whether sites are optimized for mobile
devices.
• Taxpayer usability testing. MITRE conducted a study to test the usability of the IRS
Free File portal and assess taxpayers’ actual experience in filing tax returns using three
randomly selected Free File software offerings. MITRE conducted in-person usability
testing sessions with 29 taxpayers who filed their taxes electronically using Free File
software in prior year filings. During the testing, taxpayers were observed as they
completed fictitious tax scenarios using the IRS Free File portal and Free File software. In
addition to data on the usability of the Free File services, MITRE collected and analyzed
data on the frequency of errors committed, taxpayer satisfaction, and experience (via a
brief questionnaire), and taxpayers’ perceptions of the Free File program.
• Legal analysis. MITRE’s General Counsel, in coordination with outside counsel,
conducted a review of the MOU’s terms and conditions to determine recommended
revisions to the MOU to more clearly define the parties' responsibilities to enhance and
promote the IRS Free File program, address any shortcomings in the MOU language as it
relates to the prohibition against unfair methods of competition, and provide enforcement
capability via liability clauses if the MOU's terms and conditions are breached by a
Free File member organization.
• Recommendations. Based on the data gathered from the workstreams, MITRE developed
recommendations for overall improvements to the Free File program, potential
enhancements to the MOU, and improvements to usability of IRS.gov and some aspects of
member offerings.
Figure 2 shows how each of these workstreams collectively contributed to addressing targeted
assessment areas (assessment objectives) including: documenting the current Free File program
environment, assessing Free File program management and oversight, assessing FFA member
compliance with the MOU, assessing taxpayer experience with respect to the program, and
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
16
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
ultimately drafting recommendations for the program and MOU. Outputs from some
workstreams contributed to more than one assessment area.
Figure 2. High-Level Approach to Free File Program Assessment (Aligned to Assessment Objectives).
2.1 Current Environmental Scan
As part of understanding the overall Free File environment, MITRE conducted a review and
analysis of IRS-provided program documentation, published reports from external stakeholder
groups (i.e., IRSAC, TAS, TIGTA) regarding the Free File program, and recent media
publications.
Public-Private Partnership
Research shows that while collaboration across organizational and sectoral lines is both
necessary and desirable to address complex challenges, collaboration is neither easy nor always
effective. One particular challenge is that government practitioners increasingly face demands
for concrete and tangible ‘results’—i.e., the visible creation of public value.40
40
S. Goldsmith and W. D. Eggers, Governing by Network: The New Shape of the Public Sector, Washington, D.C.: The
Brookings Institution, 2004.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
17
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
Performance and costs, however, are not the only criteria for assessing the value partnerships
create. Research suggests using a broader set of criteria for assessing whether partnerships are
successful. For example, other key attributes of successful PPPs including democratic
responsiveness and process legitimacy, are considered to be equally important values for public
undertakings.41
As part of this assessment, MITRE therefore conducted a review of the Free File PPP through
the perspective of alterative attributes to help evaluate the value of the program. In addition to
interviews with each partner and review of literature and documentation regarding the Free File
program, MITRE applied a comprehensive framework for assessment of cross-sector
collaborations42 to assess the Free File program PPP.
Using indicators of democratic responsiveness, process legitimacy, and substantive outcomes as
the attributes of value, MITRE mapped specific Free File indicators to holistically review the
partnership.
Economics of Free File
MITRE conducted an economic analysis to determine the fundamental links between economic
theory and the Free File program. The analysis started with an environmental scan of the
program and its key stakeholders and looked at taxpayers’ demand for Free File as a function of
the economic price (cost) of the program relative to other tax preparation and filing methods. A
demographic analysis of taxpayers was also performed. The analysis explored the economic
motivations of the IRS to partner with private sector companies in providing Free File.
Subsequently, the analysis addressed the software industry’s market structure and economic
incentives for offering free tax preparation and filing services.
MITRE also looked at the MOU as a two-way tool for economic negotiations between the IRS
and the Alliance that determines the dynamics of demand and supply of Free File services.
Subsequently, the analysis evaluated the economic explanation relative to MITRE’s findings on
IRS’s oversight of the program as well as the Alliance members’ compliance with the existing
MOU. The complete economics report is in Appendix A.
2.2 Free File Program Analysis
MITRE conducted an outcome and process evaluation of the IRS Free File program, guided by
the following research questions:
• Is the IRS Free File program achieving the IRS’s desired objectives?
• Are program resources being used efficiently?
• Are the IRS’s oversight and compliance processes allowing the IRS to effectively manage
the provisions of the MOU?
To obtain an understanding of the current state of the management of the Free File program,
MITRE conducted interviews with IRS program personnel, IRS leadership, and Alliance
leadership. These interviews supplemented MITRE’s review and analysis of IRS-provided
41
R. D. Behn, Rethinking Democratic Accountability, Washington, D.C.: The Brookings Institution, 2001.
S. B. Page, M. M. Stone, J. Bryson and B. C. Crosby, "Public value creation by cross-sector collaborations: A framework and
challenges of assessment," Public Administration, vol. 93, no. 3, pp. 715-732, 2015.
42
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
18
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
program documentation, published reports from external stakeholder groups (i.e., IRSAC, TAS,
TIGTA) regarding the Free File program, and recent media publications.
Stakeholder interviews were focused on gaining a full and accurate understanding of the current
management and operational oversight of the Free File program and MOU compliance by the
IRS as the program sponsor and through its collaboration with the Alliance. A list of stakeholder
interview participants is provided in Table 3.
Table 3. Stakeholder Interview Participants.
Organization
IRS Free File Program
Office
IRS Leadership
IRS W&I Customer
Account Services (CAS)
Electronic Products
Support & Services (EPSS)
Free File Alliance
Leadership and
Contractor
•
•
•
•
•
•
•
•
•
•
•
Title
Free File Program Director (Chief, Industry Engagement
and Strategy)
Free File Program Team Lead
Deputy Director, Submission Processing
Director, e-file Services, Submission Processing
Chief, EPSS
Chief, EPSS Operations Support
e-File Provider Program Manager
Chief of e-Help Operations
FFA Executive Director
FFA Legal Counsel
FFA Independent Compliance Auditor
2.3 Free File Alliance Member Organization Interviews (Site Visits)
MITRE conduced fact-finding interviews with the 12 Free File member companies to assess
alignment with the MOU, perceived value of the program, and any recommendations for
enhancements to the program. The objectives of the interviews were used to inform the
following aspects of the Free File program assessment:
• FFA approach to monitoring member compliance to the MOU
• Member companies’ compliance practices
• Recommendations from FFA member companies for enhancements to the Free File
program
• Suggestions for clarification or improvements to the MOU
MITRE developed an interview questionnaire and conducted phone interviews with FFA
leadership and face-to-face interviews with representatives of 10 of the 12 Free File member
organizations. One interview was conducted by phone with written responses provided, and one
was handled via written correspondence. The member questionnaire contained targeted questions
related to the IRS Free File program as well as the MOU Article 4, Standards of Practice to
determine Free File member compliance with the current Free File MOU and solicit views of the
overall Free File program.
These interviews allowed MITRE to gain an understanding of member interactions and
agreements with the IRS Free File program and applicable business processes, practices, and
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
19
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
techniques for how each Free File member monitors compliance and determines if a taxpayer
qualifies, as well as the procedures or practices to determine when taxpayers do not qualify for
Free File.
2.4 Web Forensics Analyses
MITRE conducted multi-tiered web analyses, including data analysis on web search query traffic
and an analysis of organic and paid web search traffic flow from Google to the Free File member
landing pages and related free tax filing web pages during the FS 2019.
As part of the initial assessment, MITRE captured and reviewed the source code associated with
each Free File program member’s landing page to determine if any coding within the member’s
page violates the terms of the MOU.
MITRE used a variety of web analytics tools and techniques to answer the following research
questions regarding usage metrics during the 2019 tax filing season (to the extent possible). For
each question, the approach that was used to answer the question is shown below.
1. For each partner in the FFA, to which URLs (member landing pages) were taxpayers
directed from the Free File program portal page on the IRS.gov website?
Approach: Followed each of the links on the IRS.gov Free File program pages and
observed any changes in URLs (caused by web page redirects – see also number 3 below).
2. How much web traffic navigated from the Free File portal page on IRS.gov to the 12
member landing pages?
Approach: Used Google Analytics reporting to analyze the web traffic metrics.
3. MITRE observed that some of the member links were redirecting the taxpayer to a URL
that was different than the URL shown on the IRS.gov Free File program page. Did these
pages always redirect? If not, can we see any differences between the original URL and
the new target page?
Approach: MITRE used Wayback Machine®43 to see if a copy of the old URLs existed in
their database. If so, MITRE captured the differences in the two pages.
4. Is there anything about the 12 member landing pages that would modify how these pages
were handled by the major search engines? For example, do they instruct the search
engine to exclude the member Free File landing page from the search index?
Approach: Analyzed the robots.txt file for the presence of "disallow" statements that
would prevent search engines from crawling the member landing pages. Also looked at
43
Internet Archive, "Wayback Machine," Internet Archive, 31 December 2014. [Online]. Available: https://web.archive.org/.
[Accessed 15 July 2019].
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
20
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
the source code for each of the member landing pages for presence of meta
robots=noindex, checked for meta refresh redirects, and also rel=canonical tags.
5. During FS 2019, how much web traffic with search queries such as “free tax filing”
navigated from Google organic and paid search to the various FFA partner sites? Were the
target URLs of this traffic the same or different than the FFA landing pages? Did any
traffic go from Google to the FFA landing pages?
Approach: MITRE used a leading third-party keyword research tool44 to analyze ranking
data of organic and paid search results for each of the members.
6. How much money were FFA partners willing to spend on searchers who were looking for
free tax filing services?
Approach: MITRE used additional third-party data to analyze average cost per click of
purchased search keywords and estimate cost of the projected traffic delivered via these
paid search results.
7. Once a Free File user reached the IRS.gov website, what percentage of people used the
various paths to get to the IRS Free File program web page?
Approach: MITRE used Google Analytics data to track click paths, next page flows, etc.
MITRE included link navigation and site search.
As part of the data analysis on web searches, MITRE captured query traffic metrics for users
who searched with queries that signaled an intent to find free tax filing services. This also
included a click-through analysis of IRS.gov site usage metrics to investigate where taxpayers
went from the IRS Free File web page.
MITRE used this analysis to determine:
• How often taxpayers searched for free tax filing services (especially during tax filing
season)
• To what website(s) did these users go following their searches and (to the extent possible
through available data)
• To which Free File program member site(s) did taxpayers go after arriving at the IRS
Free File page
• Were there any differences between where taxpayers went from the IRS Free File page
versus where they went from using a search engine such as Google
In addition, MITRE downloaded the top 30,000 search terms that drove organic search traffic
and the top 30,000 search terms that drove paid search to each of the 12 Free File member sites
as well as to the IRS.gov website.
44
SEMrush Inc., "All-in-one Marketing Toolkit," SEMrush Inc., 2019. [Online]. Available: https://www.semrush.com/.
[Accessed 15 July 2019].
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
21
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
For each search term that drove traffic to a specified domain, several attributes were captured.
The most important for this effort are:
•
•
•
•
Search keyword (what people were searching for)
The URLs of the Free File member websites that ranked in search for each keyword
Search position of the ranking URL for each keyword
An estimated monthly search volume for each keyword on Google US (the 12-month
average ending in May 2019 was seasonally adjusted for the filing season)
• An estimated average monthly traffic driven to the ranking URL for each keyword
• Cost per click for paid ads purchased for each keyword
MITRE then flagged each URL in the data that matched any of the 12 Free File member site
landing pages.
From the data, MITRE could identify a representative sample of high value keywords that drove
traffic to FFA landing pages (108 keywords) and matched following criteria:
• Caused at least one FFA member page to rank in the top 10 search positions
• The search was not a branded search term (i.e., did not contain a member company’s
commercial product names)
• The search did not contain a year designation within the keyword (e.g., 2016)
• The search did not contain the word “state” (this eliminated the searches targeted for the
free state tax filing products)
MITRE also captured ranked searches for the non-free file landing pages. These were identified
and included in the target dataset for the analysis. For example, a search of “free tax filing”
caused a non-Free File Alliance landing page to rank in the second highest search result position.
2.5 Taxpayer Journey
MITRE performed multiple research activities as part of the taxpayer journey analysis. The
objective of these research activities was twofold: assess member Free File offerings for
compliance with MOU and assess usability of each offering (from a taxpayer’s perspective). For
the first part, MITRE assessed the 12 FFA member companies’ compliance with MOU
requirements by evaluating the member landing pages (as part of the initial assessment),
followed by a more comprehensive review of member sites in which the MITRE team set-up
user accounts and ran scenarios to test for observable MOU compliance. MITRE incorporated
the assessment of observable compliance with these MOU requirements into overall
determinations of member compliance and noncompliance, as reported in Section 4.2.
For both the compliance and usability reviews, MITRE human factors experts used fictitious
taxpayer scenarios to navigate the user experience (UX). The human factors team used three
scenarios to select a member offer, create an account with the member sites, and proceed up to
the point of filing a return (returns were not actually filed for these exercises). The three
scenarios were:
1. Eligible taxpayer “simple” scenario. A single taxpayer with no dependents, who has only
income reported on a Form W-2. The taxpayer is eligible for Free File.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
22
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
2. Eligible taxpayer “complex scenario.” A married filing jointly taxpayer, with a spouse
and two dependent children. The taxpayer reports income from two jobs (two Form W2s) and interest income on Form 1099-INT. The taxpayer is eligible for Free File.
3. Ineligible scenario. A single taxpayer who reports income from a Form W-2. The
taxpayer is ineligible for Free File due to having wage income of $74,000.
For the usability assessment, MITRE conducted an expert usability review of the 12 members
sites in their entirety, as well as IRS.gov web pages related to the Free File program (including
the Free File landing page, software offers page, and software lookup tool, as well as how
taxpayers might access the Free File pages from web queries or other parts of IRS.gov). To
conduct this expert review, MITRE evaluated the sites against criteria known as heuristics.
Heuristics represent widely recognized usability best practices. MITRE used Nielsen’s 10
usability heuristics to assess user experience.
Additionally, MITRE examined the MOU for requirements related to taxpayer experience and
system usability and used these MOU requirements to derive a set of heuristics related to MOU
adherence (UX MOU requirements). This set of MOU-related heuristics should not be
interpreted as evidence of member compliance or noncompliance with the MOU. Finally,
MITRE evaluated the extent to which member sites are optimized for mobile devices.
In summary, MITRE’s heuristic review conducted under the taxpayer journey analysis evaluated
member sites and IRS.gov using the following criteria:
1. Nielsen’s 10 usability heuristics45. These 10 criteria—heuristics—developed in 1994 by
Jakob Nielsen are widely recognized measures of system usability. These general
principles for interaction design, as shown in Figure 3, are well referenced criteria for
describing usable interfaces within the user experience community. Taken together, the
10 heuristics construct a measure of user experience. They include factors such as “match
between system and the real world,” “error prevention,” “flexibility and ease of use,” and
“consistency and standards.”
2. MOU requirements from a usability perspective (UX MOU adherence). MITRE
examined MOU requirements pertaining to the user experience and constructed usability
criteria from these MOU requirements. The usability assessment looked at the users’
experience (or expected experience) with the aspect of the website governed by the
requirement, or how the MOU requirements are demonstrated via the user experience.
These criteria included factors such as “understands offerings available” and
“transparency around ineligibility.” MITRE’s assessment against these criteria are not
included in the evaluation of MOU compliance presented in Section 4.2. For the member
site assessments, MITRE also reviewed specific MOU requirements and assessed
alignment to each from a usability perspective. Note that this is different than the
assessment in Section 4.2, which assessed whether or not the member was in compliance
with the MOU requirement.
45
Nielsen’s 10 Usability Heuristics: Nielsen, Jakob. (1994). Heuristic evaluation. In Nielsen, J., and Mack, R.L. (Eds.), Usability
Inspection Methods, John Wiley & Sons, New York, NY.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
23
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
3. Mobile experience. MITRE evaluated the extent to which the examined sites were
optimized for mobile device use.
Figure 3 displays Nielsen’s 10 heuristics used for the user experience portion of the heuristic
review. Figure 4 displays the heuristics MITRE constructed to evaluate whether the user
experience aligns with MOU requirements.
Figure 3. Nielsen's 10 Usability Heuristics.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
24
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
Figure 4. Heuristics to Evaluate MOU Adherence from a Usability Perspective.
For all three of these groups of criteria, the objective of the evaluation was to identify potential
usability issues and prioritize them via a 1-5 (1 = worst, 5 = best) severity score. When multiple
issues were found against a heuristic, the lowest score was used in calculating the overall score.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
25
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
2.6 Taxpayer Usability Testing
MITRE conducted in-person usability testing sessions with taxpayers who were identified in
CDW as having submitted a tax return with the Free File program in TY 2018. MITRE sent
invitation letters to 2,500 randomly selected prior Free Filers in Chicago, IL, and scheduled
interviews with taxpayers who responded to the invitation. In total, MITRE completed 29
usability testing sessions with Chicago taxpayers.
The objectives of these interviews were to:
• Understand how taxpayers locate the Free File program options; understand how
taxpayers select a software provider
• Capture usability issues with IRS websites related to the Free File program
• Capture usability issues with specific vendor platforms
• Understand taxpayers’ previous experiences with the Free File program
During the 90-minute sessions, participants completed fictitious scenarios to evaluate the
experiences of completing three tasks: 1) locating the Free File program online; 2) selecting a
Free File software offer, including with the software lookup tool; and 3) completing a tax return.
MITRE randomly selected three Free File member software offers to test with taxpayers46; each
participant filed a fictitious return with one of the three selected offers. Taxpayers used mockedup information (e.g., fictitious Form W-2s and other documentation) to complete the tasks.
MITRE researchers observed the session and then asked debriefing questions to establish what
aspects of the Free File system (both the IRS.gov Free File pages and the specific vendor pages)
caused confusion, errors, or frustration. In addition to data on the usability of the Free File
system, MITRE also collected and analyzed data on taxpayer satisfaction and experience (via
brief questionnaires), and taxpayers’ perceptions of the Free File system components and the
Free File program.
To complete this study, MITRE performed the following activities:
1. Identified candidates for participation. The total universe of Free Filers in TY 2018
was 2,343,23547. To select a location to conduct interviews, MITRE reviewed the volume
of TY 2018 Free Filers in four metropolitan areas48 to determine whether the volume was
46
Due to limitations of time, MITRE was only able to test a sample population of 25-30 taxpayers, which limited usability testing
to three Free File member software systems. The three members were selected randomly according to a methodology outlined in
more detail in Appendix G, Taxpayer Experience Testing Report.
47 These data were drawn from CDW (ETARAS_MEF1544 table), with the Free File indicator
(ENGLISH_FREE_FILE_RET_IND=1), accepted return indicator (accepted_rtn_ind=1), and a non-missing software vendor ID
number (software_id_number not null). The data were cleaned to remove duplicate taxpayer identification numbers, records with
international zip codes (zip=0), records with missing zip code, and taxpayers under age 18.
48 MITRE reviewed Free Filer density and characteristics in four clusters of zip codes. The clusters were in Chicago, IL;
Portland, OR; Pittsburgh, PA area; and Dallas, TX area. MITRE previously visited these four sites to conduct a usability study of
the 2018 redesigned 1040 form, and was therefore already familiar with the locations. In that study, MITRE interviewed
taxpayers who filed on paper (a population that does not overlap with Free Filers). The clusters were formed for the F1040 study
because they were geographically small enough to allow MITRE to interview taxpayers from a single interview venue, yet
densely populated enough to yield sufficient taxpayers to send invitation letters. See the Taxpayer Experience Testing Report for
a more detailed review of the sampling procedure and a comparison of the characteristics of the full U.S. Free Filer population
with the sampled Chicago zip codes.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
26
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
enough to send invitation letters and derive an interview sample. MITRE identified
Chicago, IL as the best candidate location for the taxpayer interviews. The Chicago, IL
cluster included 4,106 taxpayers who used Free File in TY 2018 (according to CDW
indicators). From this pool of 4,106, MITRE randomly sampled 2,500 taxpayers to
receive invitation letters. (See the Taxpayer Experience Testing Report appendices for
copies of all recruitment materials.)
2. Recruitment. Of the 2,500 invitation letters mailed, MITRE received responses from 141
potential participants (5.64 percent) either via phone call or via the online form hosted at
www.mitre.org. Additionally, MITRE created a waitlist for participant interviews for
each site in case a minimum necessary number of interviews could not be conducted due
to respondents’ failure to appear. Two participants failed to appear to their scheduled
interview, and one participant cancelled approximately 24 hours in advance of their
scheduled interview. MITRE called two candidates from the waitlist. Twenty-nine total
interviews were deemed “completed,” meaning that the participants were eligible, and
testing took place.
3. Develop Free File program usability testing material. In preparation for the usability
interviews, the MITRE team developed an interview protocol designed to have
participants complete three tasks: 1) locating the Free File program online; 2) selecting a
Free File software offer, including with the software lookup tool; and 3) completing a tax
return using a Free File software offer chosen by MITRE. The MITRE team developed
three fictitious scenarios for taxpayers to use while completing Tasks 2 and 3, in order to
protect taxpayers’ privacy. Two scenarios were used to support tasks where taxpayers
were asked to select a vendor for which a fictitious person was eligible (selection
scenarios 1 and 2). The third scenario was used to support the task where taxpayers were
asked to complete a tax return using a Free File software offer chosen by MITRE (filing
scenario). To design the selection scenarios, MITRE examined eligibility criteria for
different vendors and developed taxpayer characteristics that would return different sets
of possible vendors. To design the filing scenario, the team had two goals: 1) to introduce
enough complexity into the scenario so that taxpayers would have to interact with
multiple components of the tax preparation software, including filing status, dependents,
and adjustments to income, while still keeping the scenario simple enough to complete in
approximately 30 minutes and 2) to ensure that scenarios represented feasible Free Filer
characteristics.
4. Conduct usability testing sessions. During the sessions, participants completed the
fictitious scenarios to evaluate the experiences of completing three tasks: 1) locating the
Free File program online; 2) selecting a Free File software offer, including with the
software lookup tool; and 3) completing a tax return using a Free File software offer
chosen by MITRE. MITRE researchers observed the session and then asked debriefing
questions to establish what aspects of the Free File system (both the IRS.gov Free File
pages and the specific vendor pages) caused confusion, errors, or frustration. In addition,
MITRE also collected and analyzed data on taxpayer satisfaction and experience (via
brief questionnaires), and taxpayers’ perceptions of the Free File system components and
the Free File program. MITRE recorded the sessions with the participants’ permission
(specifically, MITRE recorded audio and an image of the participants’ screens during the
sessions).
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
27
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
5. Data analysis and reporting. MITRE summarized and analyzed the usability testing
session data. MITRE applied aspects of behavioral science to interpret the session data in
terms of assessing how taxpayers interacted with the Free File software programs.
MITRE analyzed the interview data and developed key findings, summarizing the major
themes and patterns of the interview data. The integrated findings from the taxpayer
interviews are included in this report.
2.7 Legal Analysis
MITRE’s General Counsel conducted an analysis of the MOU to determine where improvements
could be made to improve both clarity and specificity, and provided a “red-lined” version of the
MOU to illustrate suggested improvements. In addition:
• An outside counsel retained by MITRE also simultaneously conducted the same analysis
and red-lined the MOU.
• MITRE’s counsel consulted with the IRS counsel to gain insight to the IRS perspective
and discussed potential improvements to the MOU.
• MITRE and the outside counsel collaborated to compare red-lines and come to consensus
on the final red-line.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
28
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
Findings
Eligible Taxpayers and Actual Participation
Because much of the criticism of the Free File program is based on what is perceived to be
low participation, MITRE looked closely at the eligible taxpayer population to understand
whether the criticism was warranted. While that determination could not be made without
additional, unavailable data, what did become clear is that looking only at the number
participating versus the number eligible is not a basis for the criticism. Highlights of this
section include:
• Although 70% of the population is technically eligible for the Free File program,
that number is not representative of likely candidates. Half of those use a paid
preparer, and more than 25 million taxpayers make other choices that take themselves
out of the pool of likely candidates.
• Many factors contribute to choices taxpayers make, and cost is only one. It cannot
be assumed that simply because a program offered by the government is free, people
will use it. Without a deeper analysis into taxpayer awareness and taxpayer behavior,
the IRS should not assume any specific actions they take will increase participation.
• According to the FFA, their members provided free commercial filings last tax
season to nearly 18 million taxpayers. With free venues such as VITA, military
assistance, TCE and free commercial filings, Free File is just one choice taxpayers
have to file their returns for free.
3.1 Overall Trends
The Free File program met with considerable success in its first three years. During FS 2003—
the first year of implementation—2.8 million returns were filed though 17 Alliance member
offerings. While this represented less than 4 percent of eligible filers, the sheer number of Free
Filings was considered a huge success given the novelty of the program. In its second year (FS
2004), 3.5 million returns were filed (an increase of 26 percent), and in the third year (FS 2005),
5.1 million returns (an increase of 46 percent) were filed.49 50FS 2005 marks the peak of
participation.
Data from IRS’ ETARAS shows an overall decline in the number of Free File accepted returns
between 2008 and 2018, as shown in Figure 5. However, new data from the IRS CDW database
49M.
S. Chu and M. M. Kovalick, "An Analysis of the Free File Program," in National Tax Associate Proceedings, 99th Annual
Conference on Taxation, Boston, 2006.
50 Source: IRS Electronic Tax Administration Research and Analysis System
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
29
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
shows that Free File numbers have started to pick up since TY 2016. Compared to TY 2017,
approximately 74,000 more returns were filed to date for TY 2018.
Since its inception, over 53 million returns were filed through the Free File program. The IRS
reported that Free File saved taxpayers about $1.6 billion in tax preparation costs51.
Millions of Returns
6.00
5.00
4.00
3.00
2.00
1.00
2018
2017
2016
2015
2014
2013
2012
2011
2010
2009
2008
2007
2006
2005
2004
2003
0.00
Filing Year
Free File
FFFF
Total
Figure 5. Free File Accepted Returns, FY 2003-2018
3.2 Taxpayer Behavior and Awareness of the Free File Program
A key component of the Free File program is, of course, the taxpayer. In the early days of
Free File, when encouraging electronic filing was one of the primary drivers, the IRS followed
the patterns and behaviors of Free File participants closely through surveys, focus groups, and
studies that looked at awareness and migration.52
As overall e-filing participation increased, and budgets for these types of studies decreased, the
ability for the IRS to understand taxpayer behaviors declined. The last Free File awareness study
was conducted in 2011. Because the environment has radically changed in the past eight years in
terms of technology and filing choices available to taxpayers, the insights that study provided
may no longer have value.
51
Internal Revenue Service, "IRS, Free File Alliance announce changes to improve program; Improved taxpayer options
available for 2019 Free File program," Internal Revenue Service, 2 November 2018. [Online]. Available:
https://www.irs.gov/newsroom/irs-free-file-alliance-announce-changes-to-improve-program-improved-taxpayer-optionsavailable-for-2019-free-file-program. [Accessed 2 July 2019].
52 Internal Revenue Service, "Free File Survey, Publication 4556 (11-2008), Catalog Number 48884," Internal Revenue Service,
Washington, D.C., 2008; Internal Revenue Service, "Findings From The Free File Migration Study, Publication 4684 (3-2008),
Catalog Number 51580P," Internal Revenue Service, Washington, D.C., 2008 Internal Revenue Service, "Report of Findings
From the Free File Marketing Focus Group, Publication 4833 (Rev 09-2010) Catalog Number 54427P," Internal Revenue
Service, Washington, D.C., 2010;Internal Revenue Service, "Free File Demographics and Migration Analysis, Publication 4829
(Rev 11-2011), Catalog Number 54376," Internal Revenue Service, Washington, D.C., 2011; Internal Revenue Service, "Free
File Awareness & Attitudinal Research, Findings From Research Conducted Among Free File Non-Users With Focus On Three
Specific Segments, Publication 4928 (01-2011), Catalog Number 57580X," Internal Revenue Service, Washington, D.C., 2011.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
30
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
Based on the participation rates, it would appear that awareness of the program is low, though
the 2011 study and other e-file studies indicate awareness is not the only factor that determines
usage of the program. While the IRS has continued to collect data on demographics and
migration of Free File users, no recent— if any— comprehensive studies on taxpayer behavior
(i.e., why taxpayers make the choices they do) have been conducted. Understanding the numbers
is instructive and Section 3.3 examines those numbers more closely. However, until the IRS
understands why taxpayers make the choices they do, they can only make programmatic
decisions based on assumptions. A behavioral study helping the IRS understand the choices
taxpayers make in each of the demographics, along with an updated awareness study, could help
the IRS determine how to improve participation, if that is the goal.
3.3 A Closer Look at Eligibility and Participation
As mentioned earlier, we must look at Free File in the context of taxpayer behavior and taxpayer
choices. One cannot assume that everyone in the eligible population would use Free File if they
were aware of it, nor can we assume how many people should use it. But we can narrow down
the population of taxpayers who may be more likely to use it if they know about it.
Taxpayers have several methods to choose from when filing taxes— the IRS allows tax return
submission via paper and electronic forms. Taxpayers’ decision to choose between selfpreparation and a paid tax professional factors in the financial costs as well as the time, effort,
and hassle associated with each filing method. Figure 6 shows a simplified taxpayer decision tree
for choosing different methods of filing and summarizes the associated economic costs.
The economic factors of making those choices are discussed in greater length in Appendix A.
Understanding these factors are important when attempting to determine who should use
Free File, who does use Free File, and how the IRS might increase the awareness of Free File
(and therefore, presumably, the participation).
As stated earlier, the IRS determines who is eligible for the Free File program by looking at prior
year returns and taking the lower 70 percent by AGI. For TY 2017, nearly 104 million taxpayers
were eligible for the program by having an AGI of less than $66,000. Less than 3 million
actually used the program, which has brought the program under the recent scrutiny described in
Section 1.1.4.
But being eligible and being an actual candidate for the Free File program can be very different.
As demonstrated in Figure 6, taxpayers have many choices when they file their returns, and
many of those choices will take them out of the likely candidate pool for Free File.
For example, taxpayers generally have a preference to whether they e-file or file on paper, and
whether they prepare the return themselves or go to a paid preparer. Because Free File is a DIY
tax filing program, a valid comparison can only be made to the overall e-filing DIY population,
i.e., do “self-prepared” taxpayers pay for the software and e-filing, or do they look for a free
version?
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
31
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
Economic Costs by Filing Method
Taxpayer
SelfPrepare
Use Tax
Professional
Online
Paper
Ineligible
for Free
File
Eligible for
Free File
Commercial
ERO
Tax Software
Commercial,
Paid
Fees: Moderate
Time: Low
Commercial,
Free
Fees: None
Time: Low
VITA / Voluntary
Tax Professionals
Fees: None
Time: Moderate
Free File
Free File
Fillable Form
Paper
Tax
Professional
Fees: None
Fees: None
Fees: None
Fees: High
Time: Moderate
Time: High
Time: Low
Effort: Low
Other: Low
Time: Low
Effort: Low
Effort: Low
Effort: Low
Effort: Low
Effort: Moderate
Effort: High
Other: Moderate
Other: Moderate
Other: Moderate
Other: None
Other: Moderate
Other: Moderate
Figure 6. Economic Cost by Filing Method.
There are other free venues that a taxpayer could select, such as using a free commercial product,
getting free assistance through a Volunteer Income Tax Assistance (VITA) or Tax Counseling
for the Elderly (TCE), or through the many free options available for active military at military
bases. Additionally, many taxpayers choose to receive their refund immediately either through a
Refund Anticipation Check (RAC) or a Refund Anticipation Loan (RAL), which has not been
allowed through the Free File program since the 2007 agreement. In TY 2017, of the eligible
population, more than 18 million taxpayers received a RAC or RAL, nearly 10 million of those
were in the self-prepared group. And, of the overall population of eligible taxpayers, nearly 12
million chose to file on paper that year, with 8.6 million of those taxpayers being in the group
who self-prepared. While some paper filers, particularly younger and first-time filers are
traditional target candidates to become e-filers/Free Filers, if we look at TY 2017 retrospectively
(as shown in Figure 7), making a choice to file on paper removed those filers from the candidate
pool for Free File that year (i.e., keeping the comparison to DIY e-filers).
Looking more closely at the filing characteristics of this population demonstrates that while the
IRS clearly has opportunity for increasing the participation in the Free File program, that
opportunity may come from a much smaller population than the actual number of eligible
taxpayers. As such, Figure 7 draws a more accurate picture of that population for TY 2017.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
32
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
103,778,000 Eligibile Taxpayers - TY 2017
Potential Add'l Free
Filers, 27,684,000
free commercial
offerings
Paid Preparers
RAC/RAL self-prepared
Paid Preparers,
51,860,000
Free Filers, 2,649,000
Paper filers - self-prepared
VITA/TCE
VITA/TCE, 3,085,000
Free Filers
Potential Add'l Free Filers
Paper filers - selfprepared, 8,750,000
RAC/RAL self-prepared,
9,750,000
Figure 7. Break-down of 103,778,000 Taxpayer Eligible for Free File (TY 2017).
However, the pool of potential Free Filers shown in Figure 7 is actually smaller than the almost
28,000,000 noted, as most commercial software companies offer a free version of their product
for the simplest returns outside of the Free File program. That information is not publicly
available, though MITRE requested the number of returns filed through a free commercial
offering by the members as an aggregate from the FFA, who attested to 17.7 million for TY 2018
from 11 current members.
An unknown number of taxpayers presumably used free commercial offerings from software
companies outside of the Free File Alliance, which would make the number of users of free
commercial offerings even larger.
While MITRE cannot definitively say whether or how many taxpayers in the groups outside of
the “additional potential free filers” in Figure 7 could or should have been Free Filers, what the
chart reiterates is that taxpayers have many choices when they file their returns. And some of
these choices are free, outside of the Free File program. If the focus is on ensuring taxpayers who
are least able to pay for tax preparation and filing are getting free help, then it raises the question,
should the objective be to increase the participation in Free File program, or more broadly in free
filing.
3.4 Free File and VITA
In breaking the eligible tax filing population of the 103.8 million into those who use a return
preparer and those who self-prepare, a useful comparison may be made between those who pay
for a preparer and those who choose VITA, an assisted free option, to the paid DIY and those
who choose Free File. The VITA program has 2,400 partner organizations spanning government,
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
33
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
corporate, educational, and non-profit, including very large organization such as AARP who
advertise their more than 10,000 tax-help sites nationwide. The IRS has offices and employees
nationwide to administer the VITA program, and thousands of professional volunteers who work
at the sites each year, yet the number of those choosing this free program is roughly equivalent to
those choosing the Free File program (see Figure 8).
Number of taxpayers within
the targeted (103.78 million)
population (Uses paid
preparer or self-prepares)
eligible for assisted help
Access to Program
Volunteer Income Tax Assistance
(Bricks and Mortar Assisted Free
Tax Filing Program)
51.86 million using paid preparers
Less 3.4m paper filers
Less 8.4m RAL/RACs =
40.06 million returns*
Free File Program
(DIY Free Tax Filing Program)
11,000 locations nationwide
Choice of 12 companies through
IRS Free File Landing Page
program office in HQ, Alliance
of 12 software companies with
Executive Director
~ 2.6 million
8.5%
Program Administration
offices and employees nationwide,
>2,000 partner organizations
Number of Users TY2017
Ratio of free to paid
~ 3 million
7.5%
48.82 million Self prepared
Less 8.75m paper filers
Less 9.75m RAL/RACs =
30.32 million returns*
*Because the comparison is being made between e-filed returns, paper returns are backed out of the population, as are RACs and
RALS, both of which are unavailable for VITA and Free File
Figure 8. Comparison of Two Free Programs: VITA (Preparer-assisted) and Free File (DIY).
As noted earlier, without further research to understand taxpayer behavior and what drives
taxpayers to make the choices they do, it’s difficult to know whether the percentages of those
seeking free tax return help or free filing is reasonable or if it’s too low. As there are many
unknowns about taxpayer behavior and choice that will require further research, and these
questions align with MITRE’s internal research program initiatives on improving customer
service, MITRE will consider this for future research.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
34
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
Compliance with the MOU
The purpose of this chapter is to provide a comprehensive assessment of current state
compliance with the Free File agreement; that is, individual Free File member organization
and IRS Free File program compliance with the most current MOU agreement (October 31,
2018). The assessment includes findings from the IRS interviews and document reviews,
interviews with Alliance leadership and Alliance-contracted reviewer, and fact-finding
interviews with representatives from each of the Free File member organizations. In addition,
MITRE conducted hands-on reviews of member Free File websites (taxpayer journey) and
web forensic analyses. See Section 2.0 for methods used for each of these activities.
Key themes from this set of findings include:
•
MOU is a living document that has evolved over time. The original agreement
between the IRS and the software industry has been updated numerous times since the
program’s inception. While key aspects in the MOU have remained intact, as issues
come up, IRS and FFA negotiate resolution and amend the agreement accordingly.
•
Overall, FFA members are generally compliant with the MOU as written.
Assessing against current MOU provisions—and at the time of the assessment—
MITRE found only a few requirements inconclusive, non-compliant, or not applicable.
•
Search marketing is an emerging issue that is not explicitly addressed by the
current MOU. One of the issues at the core of recent criticisms is whether members
employed—or should be able to employ—search routing techniques. For the next
iteration of the MOU, IRS has the opportunity to provide clear guidance in this area.
•
The IRS and FFA work together to ensure MOU compliance. While there is room
for improvement, based on assessment findings, MITRE considers the IRS’s current
compliance processes adequate and effective to support the integrity of the program.
In part, this is the result of its long-standing partnership with FFA and continuous
assessment and improvement of the program as it has evolved.
4.1 MOU Provisions Set Criteria for Compliance
As a result of the most recent MOU negotiation in October 2018, new and revised provisions
were added for implementation in FS 2019. According to the IRS press release announcing the
updates, these changes to the agreement were aimed at making the Free File program “more
taxpayer friendly while strengthening consumer protections in several key areas,”53 presumably
to address public criticism outlined in Section 1.1.4. Based on this updated MOU—which will be
53
Internal Revenue Service, "IRS, Free File Alliance announce changes to improve program; Improved taxpayer options
available for 2019 Free File program," Internal Revenue Service, 2 November 2018. [Online]. Available:
https://www.irs.gov/newsroom/irs-free-file-alliance-announce-changes-to-improve-program-improved-taxpayer-optionsavailable-for-2019-free-file-program. [Accessed 2 July 2019].
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
35
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
in place until the next round of negotiations slated for October 31, 2021—Free File member
organizations must adhere to the following additional requirements (beginning in FS 2019):
• Remove any “value-added” button from Free File member landing pages. Free File
member landing pages cannot include a value-added button—defined as “an icon, link or
any functionality” that direct taxpayers to non-Free File programs (commercial products
or paid services). [MOU Section 4.32.654].
• Provide an option for taxpayers to “easily return” to the IRS Free File page if they
don’t qualify for a particular member offer. If taxpayers do not qualify for members’
Free File offer (i.e., due to the different eligibility characteristics beyond the $66,000 AGI
limit), members must allow taxpayers the option to first return to the IRS landing page on
IRS.gov to see if they qualify for another member’s Free File offer. After that, the member
can offer two additional options, in this order:
o If taxpayers are eligible for Free File (but not the particular member offer), a second
option is offering a free alternative to complete their return with the same member.
o If taxpayers are not eligible for Free File (e.g., has AGI over $66,000), a third option is
to continue to the member’s commercial site to complete their return for a fully
disclosed fee. (MOU Section 4.19.2)
These options are shown graphically Figure 1 in Section 1.1.1.
• Offer returning Free File taxpayers the option to use Free File again when they log
into member’s website. If taxpayers return to a Free File member’s commercial website
after using the Free File program the previous year, the member must first offer taxpayers
the Free File option for the current filing season when taxpayers log in to their account.
The option to continue with the Free File program must be offered before any alternative
commercial products or services. (MOU Section 4.14)
• Send at least one follow-up email to taxpayers who used Free File the previous year
to “welcome them back to Free File service.” For taxpayers who participated in the
Free File program the previous year, members must send at least one email to remind the
taxpayer about the availability of the Free File program and invite them to return to the
member’s Free File landing page. The email communication cannot include offers or links
to non-Free File programs (commercial products or paid services). (MOU Section 4.32.4)
• Participate in-place review process. The latest MOU includes updates to the already
existing annual review process conducted by the IRS and the Free File Alliance. The
MOU explicitly calls out pre-filing season review of members’ landing pages and an
additional review conducted by the IRS that is a random, unannounced review during the
filing season to ensure continued compliance. (MOU Section 4.29.1 and Section 6.1.3)
Evolution of Agreements and MOU
The changes to the MOU implemented in FS 2019 are aligned with what MITRE observed when
reviewing the historical context of the MOU (and overall agreements between the IRS and
54
Section 4.32.5 further clarifies “No Other Sales and Selling Activity” – limiting “marketing, soliciting, sales or selling activity,
or electronic links to such activity” to paid federal return as described in the third option for ineligible taxpayers in the next bullet
[Section 4.19.2] and state tax preparation offers as described elsewhere in the MOU.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
36
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
Free File Alliance). Since the Free File program began in 2002, there have been six formal
agreements between the two parties, corresponding to eight iterations of the MOU, as shown in
Table 4. Each time there is a new agreement or updated MOU there are associated changes to
terms of service—either to address an issue current at the time, or general update to reflect the
current enviroment.
It is important to note that with each iteration, additional information is added to—or in some
cases removed from—the MOU. The numbering and potentially sequence of the provision
changes, but the foundation of the MOU stays the same. In this way, the MOU is iterative; there
is no wholesale “redo” of the MOU.
For example, from the original agreement in 2002 to the first MOU in 2005, limits were placed
on members’ offering RALs and RACs. In this first mention of RALs and RACs, it is agreed that
they are allowed, but with restrictions. However, starting in the second MOU in 2007, RALs and
RACs are prohibited within the Free File program. To address this change, Section 4.2 of the
MOU is shown as “Ommited by Intention.” That language is carried through to the most current
MOU.
Conversely, there are sections of the MOU that have not changed from year to year. For
example, after a slight revision to how the program objectives are characterised from the first
agreement to the first MOU, the MOU continues to contain the same program objectives. See
Section 4.3 for additional discussion of the Free File program objectives.
Similarly, there are at least two foundational statements in the MOU preamble that were in the
first version of the MOU and is still in the most recent version:
• Preamble. “[I]t was agreed that the Alliance will offer the Services and the IRS will
provide taxpayers with links to the Services offered by the Alliance participants offered
through a web page, which is hosted at irs.gov with links from www.usa.gov.”
• Article 2 FFI and IRS objectives. “In recognition of this commitment, the federal
government has pledged not to enter the tax preparation software and e-filing services
marketplace.”
Table 4. Free File Program Agreements and MOUs.
Year
Free File Agreements and MOUs
2002
• First Agreement between the IRS and
Free File Alliance, LLC executed on October
30, 2002, to launch and set parameters for
the IRS Free File program (three-year
agreement)
• Second Agreement between the IRS and FFA
executed on October 31, 2005, to extend and
amend original (four-year renewal
agreement)
• IRS and FFA enter into first MOU as of
December 20, 2005 to govern partnership for
implementation in FS 2006
2005
Examples of Updates
• Guarantee coverage in aggregate to 60
percent taxpayers
• Each member provides minimum of 10
percent coverage
• MOU to administer program
• Guarantee coverage in aggregate to 70
percent taxpayers (up from 60 percent)
• No member can cover more than 50 percent
• Begin using AGI
• Members will provide Free File indicator
• Limits (but does not prohibit) RALs
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
37
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
Year
2007
2008
Early 2009
Late 2009
2014
2015
2018
Free File Agreements and MOUs
Examples of Updates
• IRS and FF enter into second MOU as of
January 12, 2007 for implementation in FS
2007 with revised and additional
requirements for Free File program
• IRS and FFA enter into third MOU as of
January 2, 2008, for implementation in FS
2008 with revised and additional
requirements for Free File program
• IRS and FFA enter into fourth MOU as of
January 13, 2009 for implementation in FS
2009 with revised and additional
requirements for Free File program
• Prohibits RALs/RACs, provisions removed
from MOU
• Identifies permitted sales, limits on ancillary
sales, and selling activity
• Adds implementation of CAPTCHA
• Adds third-party certification of returns as
part of already existing guarantee of
calculations
• Adds unbranded fillable form utility to MOU
(specifications dated December 9, 2008)
• Adds basic package of forms and schedules
required for all members
• Third Agreement between the IRS and FFA
executed on October 30, 2009, to extend and
amend the previous (five-year agreement)
• IRS and FFA enter into fifth MOU as of
October 20, 2009 for implementation in FS
2010 with revised and additional
requirements for Free File program
• Fourth Agreement between the IRS and FFI
effective October 30, 2014, with the intention
to enter into a multi-year agreement (oneyear agreement)
• IRS and FFI enter into sixth MOU signed in
conjunction with 2014-2015 agreement until
next multi-year agreement signed
• Fifth Agreement between the IRS and FFI
effective October 31, 2015, to extend
previous (five-year agreement)
• IRS and FFI enter into seventh MOU signed in
conjunction with 2015 agreement with
revised and additional requirements for
Free File program
• Sixth—and most current—Agreement
between the IRS and FFI effective October 31,
2018, to replace 2015 agreement (five-year
agreement)
• IRS and FFI enter into eighth—and most
current—MOU signed in conjunction with
2018 agreement for implementation in FS
2019
• Both the 2018 agreement and eighth MOU
are set to expire on October 31, 2021.
• Adds core forms and schedules
• Additional guidance under development for
pop-ups, spyware, and marketing tools
• Adds customer satisfaction survey
• Additional security and privacy requirements,
including compliance with e-file standards
• Link on landing page back to irs.gov
• Free File Alliance officially changes name of
organization to Free File Inc.
• Adds annual review and revision of MOU
• Members to offer the same federal forms and
schedules as basic commercial online product
• Promotion of Free File program responsibility
of IRS
• Adds language around innovations, included
pre-population of tax information and forms
• Removal of value-added buttons from
member landing pages
• Define paths for taxpayers who are not
eligible for a member offer (i.e., first back to
IRS.gov, then free alternative, then
commercial)
• Adds requirements for engaging returning
Free File taxpayers (i.e., offer Free File at login, email taxpayers)
• Refines requirements for existing IRS and FFI
review processes
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
38
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
4.2 Free File Alliance Member Adherence to MOU
This section summarizes MITRE’s assessment of each Free File member organization’s
adherence to the current set of MOU requirements, including the latest additions implemented in
FS 2019 (outlined in the previous section, Section 4.1)
Section 4.3 describes the IRS Program Office role in assuring adherence to the MOU, including
implementation of the updated review processes also included in the FS 2019 MOU
requirements (Section 4.1).
4.2.1 FFA Member Assessment Results: Overview
Overall, members were generally compliant with the MOU, in particular with Article 4,
Standards of Practice, which outlines the expected level of service, including software and
website functionality, security, disclosure of forms and schedules as well as taxpayer service
options and other requirements such as innovation. More specifically, MITRE found in assessing
72 MOU requirements for each FFA member only a few were either deemed as inconclusive,
non-compliant, or not applicable.
As mentioned in Section 2.0, MITRE not only collected data from each of the members, either
through in-person or telephone interviews or through written responses to a specific set of
questions, but also through stakeholder interviews and other analytics methods involving web
reviews and taxpayer journey evaluations. FFA member compliance to the MOU was assessed
using several methods. Methods used included:
• Interviews with member companies (site visits). Responses given during the interviews
or provided in writing. These inputs are considered self-attestation.
• Taxpayer journey reviews for each member site. Results from landing page and
comprehensive site reviews. These inputs are considered observable as they can be seen
while interacting with member sites.
• Other stakeholder interviews. Responses given by FFA leadership or key IRS
stakeholders.
MITRE also looked at the results of the IRS website reviews (see Section 4.3 for more
information) as a supplemental input to determine if more information was needed.
Specific Instances of Non-Compliance
There were two instances where members were determined to be non-compliant with certain
provisions of the MOU. They include:
• One member was non-compliant with a sentence in MOU 4.15.4. The MOU states that
“Members must clearly list their free customer options … on the Member’s Free File
Landing Page (or such page must have a clear and prominent link to such disclosures
directly from this page).” Based on MITRE’s landing page analysis, the link was not
“clear and prominent” on their landing page55.
55
As of the date of this report, the member has updated their Free File landing page.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
39
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
• One member was non-compliant with MOU 4.32.4. The MOU requires members to
“communicate not less than once annually via email with their taxpayer customers who
used Free File services and completed their returns through Free File in the immediately
preceding tax year.” The representative from the member acknowledged they were
supposed to initiate an email to their prior tax season Free File taxpayers and self-attested
that they did not provide the required email prior to the tax season.
Instances Deemed Inconclusive or Not Applicable
The following MOU requirements from Article 4 were deemed as inconclusive or not applicable
for specific members, they include:
• 4.15.4 Disclosure of Taxpayer Service Options (inconclusive) – (applicable to 3
members). The MOU states that “Members must provide taxpayers a free electronic
method to obtain a copy and learn the status of their electronically filed return.” Based on
what could be observed on three member sites through taxpayer journey activities—which
were limited to all steps prior to actually submitting the test return—MITRE was unable to
verify that these members offer a method to check the status of the return without first
submitting a return.
• 4.19.2(i -vi) Eligible taxpayers/no promotional codes or rebates/links to paid sites
(inconclusive) – (applicable to 2 members). For this set of provisions, the MOU states
that member programs must “unequivocally inform taxpayers who are ineligible for the
free offer at the earliest feasible point,” and outlines a set of requirements specific to when
a taxpayer is informed of ineligibility (e.g., reason for ineligibility, alternative actions that
must be presented in a specific order, etc.). During the taxpayer journey analysis, MITRE
tested a scenario in which a taxpayer has an AGI over the eligibility limits. For two
members, testers in this ineligible scenario were able to prepare their Free File tax return
without receiving ineligibility notification. In both cases, the testers reached the signature
page where the next step was to file the return and were unable to verify whether or not
the appropriate notifications, reasons, alternative actions, etc. were provided. The two
member programs varied slightly in that:
o One member has a wizard on its landing page that allows a taxpayer to determine
eligibility prior to creating an account. This optional step guides a taxpayer through
eligibility criteria and does provides appropriate language if they are ineligible.
However, this is not using actual taxpayer information and if the taxpayer ignores the
landing page wizard—or does not see it— or guesses information incorrectly, the
software allows the ineligible tester to continue to the point of the signature page, as
described above.
o The other member does not have a wizard but does have generic language on the
landing page describing their offering and eligibility requirements. Presumably, a
taxpayer could determine eligibility from this though it is unclear whether this
constitutes ineligibility notification “at the earliest feasible point.” Again, this
member’s software also permitted the ineligible tester to continue to the signature page
with no notification.
MITRE did receive screenshots from the IRS that were provided by all members on this
issue. Based on the screen shots, it appears taxpayers would receive an ineligibility
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
40
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
message and the members do have the appropriate language, but it is unclear when these
messages are provided, and MITRE could not independently verify this. As explained
above, since the MITRE assessment was limited to steps prior to actually submitting the
test return, MITRE was unable to observe any messages that may have been presented
after submitting the return. MITRE testers were therefore not able to determine whether
reasons and alternative actions were proper. These two members were therefore
inconclusive in all “ineligible” categories.
• 4.32.2 Permitted sales, limits on ancillary sales, and selling activity (inconclusive)–
(applicable to 2 members). The MOU states “when a taxpayer enters a Member’s Free
File Landing Page and begins to complete a return but ultimately cannot qualify for the
Member’s free offer, the member must provide, as a first option, a prominent hyperlink for
the taxpayer to return to the IRS Free File Landing Page (consistent with Paragraph
4.19.2)” The remainder of the paragraph outlines the order of other options and stipulates
that the “charge for such commercial service shall not exceed the usual commercial price
for such products or services.” For the same rationale provided for Paragraph 4.19.2,
compliance for two members was deemed inconclusive, subject to discussion with IRS.
• 4.34 Use of Free File Logo (not applicable) – (applicable to 2 members). This is a
requirement that is only applicable if there is a logo. Since these members did not use a
logo, it is not applicable.
• 4.36.3 Innovations (not applicable) – (applicable to 1 member). Because this is
between the IRS and FFA, it is not possible to rate compliance of individual members.
Nevertheless, all but one member detailed during interviews how they have introduced
multiple and continuing innovations into both products; some noted they were including
capabilities offered to all FFA members by one member. This member self-attested that it
didn’t have the resources to pursue major innovations.
MITRE was unable to verify compliance for the following provisions of the MOU and
determined inconclusive for all members. They were:
• 4.17.3 Blackouts (inconclusive). Based on stakeholder interviews with the IRS and FFA
along with members’ input, there were no unplanned blackouts. As a result, maintaining a
consistent level of service could not be determined nor whether customers seeking access
to the Free File option were directed elsewhere, in particular to the fee-based services of
the member. Hence, MITRE rated the evidence for all members as inconclusive.
• 4.19.2(iv) Eligible Taxpayers/No Promotional Codes or Rebates/Links to Paid Sites
(inconclusive). MITRE did not explicitly verify whether the messaging, language, and
formatting relating to disqualification practice by members adhered to a specific standard
provided by the FFA ED in consultation with the IRS. This was not covered during site
interviews. MITRE did observe that all members—with the exception of the two
previously discussed members—had messaging with regards to disqualification, but
MITRE did not compare against a standard. If no standard language exists, the MOU
should be updated to reflect the guidance that is provided.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
41
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
4.2.2 Emerging Issues: Search Techniques
As discussed in Section 4.1, the MOU is an evolving agreement, changes to which are made
through negotiations between the IRS and the Free File Alliance at specific points in time and/or
as new issues emerge.
The assertions from ProPublica and to some extent recommendations from IRSAC and TAS
opened the door to discussion around the broad topic of pay-per-click search advertising and
search engine optimization techniques (collectively known as search marketing) that enable web
publishers to acquire large amounts of traffic from the major web search engines and exert
control to which pages on their websites this traffic is directed to.
There is no place in the MOU that explicitly addresses the issues related to search marketing or
provides guidance to the members for standards of practice. We have provided the discussion
here—including our data and analysis—for the IRS to consider for the subsequent MOU
negotiation, currently planned to result in an updated MOU by October 2021.
4.2.2.1
How Taxpayers Find the Free File Program
As shown in Table 5, there are three primary ways for users to get to the IRS.gov Free File
landing page: web search engines such as Google and Bing, clicks from referral links on other
third-party websites, and direct traffic (or traffic from unknown sources).
Table 5. Top Sources of Traffic to the Free File Member Landing Page.
Top Sources of Traffic for the
FFI Landing Page
Organic Search
Referrals from third-party websites
Direct or unknown sources
Entrances
1,541,816
761,258
476,060
Percent of Total
55.2%
27.3%
17.1%
Web Search Engine Activity
MITRE conducted website traffic analysis using data downloaded from SEMrush56, a digital
market software as a service company. This analysis examines searcher behaviors and the
numbers of taxpayers that were searching for free tax filing services from a major web search
engine such as Google. It also provides analysis of the seasonal nature of the search activity
relating to free tax filing services.
To help understand the search discussion, it may be helpful to define a few terms. (Also see
Appendix E for an overview of how search engines work).
• Organic search. The primary set of search results on major web search engines such as
Google and Bing that use an algorithmic approach to ranking and displaying search results
to users. This result type is explicitly not influenced by payments from web publishers.
56
SEMrush Inc., "All-in-one Marketing Toolkit," SEMrush Inc., 2019. [Online]. Available: https://www.semrush.com/.
[Accessed week of 27 May 2019].
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
42
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
• Paid search. Unlike organic search results, paid search results are explicitly
advertisements that web publishers purchased on an auction-like bidding platform in
which search ads that have high bid prices and have been shown to be relevant to the
user’s search query will obtain the most prominent positions on the search results page for
a specified search query.
• Keywords. This is shorthand for “search query.” It is the word or phrase a user types into
the search box to express what they are looking for. “Search term” is another synonym of
keyword and search query.
What search terms did taxpayers use to find Free Tax filing services?
MITRE identified a representative sample 108 search queries (keywords) showing the range of
permutations used by taxpayers to search generically for free tax filing services. As discussed in
Section 3.2, a few criteria were used to refine this list of keywords to only those keywords
considered to be highly relevant to this service. There are likely many more less relevant
keywords that also drove traffic to these pages and are not accounted for by this list of 108.
As shown in Figure 9, of the 108 keywords, the top five accounted for 82.5 percent of all organic
search traffic to the FFA member sites and IRS.gov. The top keyword—“free tax filing”—drove
63 percent of all organic search traffic.
Figure 9. Top Keywords for Member Free File Landing Pages57.
Following a free tax filing search, how many taxpayers clicked on paid vs organic search
results?
In total, using the list of 108 free tax filing keywords, MITRE estimated over 12 million visits
went from Google to the FFA member sites and IRS.gov. Of those visits, it is estimated that 85
percent of the traffic followed a paid search result while the other 15 percent of visits came from
an organic search result. Searchers were more than five times likely to click on a paid search
result.
According to the third-party data source used for this analysis, the average cost per click for a
free tax filing search advertisement was $1.69. This results in an estimate of $17.3 million for
total advertising spent during the 2019 tax filing period.
57
SEMrush Inc., "All-in-one Marketing Toolkit," SEMrush Inc., 2019. [Online]. Available: https://www.semrush.com/.
[Accessed week of 27 May 2019].
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
43
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
How much of the search traffic went to one of the FFI landing pages vs other “commercial
type” products that offered “free tax filing” services?
Of the 1.9 million visits that went from Google organic search to an FFA member site, 56,000 (3
percent) went to an FFA landing page operating within the IRS Free File program. This is 0.5
percent of all search visits (including paid) to the FFI websites for the 108 free tax filing
keywords.
Table 6 shows the estimated traffic distribution from organic and paid search results to each of
the 12 FFA member sites. The first two data columns are focused on estimated traffic coming
from organic search results. The third data column is an estimate of visits from paid search
results. Notice that there is no column for estimated paid traffic to FFA landing pages. This is
because the data indicated that none of the FFA members purchased any paid search
advertisements to take taxpayers to their FFA landing page. However, during the FFA member
interviews, one member stated that their company does employ advertising practices for the Free
File program58. The fourth data column estimates the total money spent by the FFA member
using an estimate of $1.69 per click.
Table 6. Estimated Traffic Distributions from Organic and Paid Search Result to Landing Pages/Estimated
Cost for Paid Traffic59.
58
During MITRE interviews with representatives of FFA member organizations, one member indicated that their company pays
for advertising of the Free File program through a number of methods. However, the interview did not include detailed discussion
of advertising practices specific to key word search. For this analysis, MITRE looked specifically at ads purchased for key words
related to free tax filing. The results of this analysis indicated that no purchased ads directed users to the landing pages of Free
File offers.
59
SEMrush Inc., "All-in-one Marketing Toolkit," SEMrush Inc., 2019. [Online]. Available: https://www.semrush.com/.
[Accessed week of 27 May 2019].
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
44
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
What is the seasonal nature of searches for “free tax filing?”
Using the search term “free tax filing” as highly representative of the entire collection of 108
keywords, MITRE submitted this as input to Google Trends to analyze the seasonality of
searches for this service. Figure 10 shows that there is a substantial peak in search activity for
these terms stretching from the end of January to the middle of April. This is completely
consistent with what is referred to as tax filing season. Additionally, MITRE estimates that tax
filing season search activity for people searching for free tax filing services represents 90 percent
of total search traffic for the entire year.
Figure 10. Seasonality of Free File-Related Searches60.
4.2.2.2
Search and MOU Compliance
MITRE’s source code analysis of member Free File landing pages found that five of the 12
companies engaged in a search routing practice that prevented their Free File offering from
appearing in the organic search results of the major web search engines—the use of meta robots
NOINDEX (M)—or the rel=canonical tagFree File (R). Two companies changed the coding after
the ProPublica articles. (See Appendix D for complete source code explanations and findings on
search engine avoidance and Appendix E for how major search engines work.)
60
Google Trends, 2019. [Online]. Available: https://trends.google.com. [Accessed week of 27 May 2019].
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
45
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
When questioned about the practice, most members reported they believed that excluding their
Free File landing pages from search results kept them in accordance with the MOU. The two
companies that removed the coding said they still believe excluding the pages was appropriate,
but after it was raised publicly, didn’t want to create the perception that they were doing
something wrong.
The MOU states in the preamble that … “the Alliance will offer the Services and the IRS will
provide taxpayers with links to the Services offered by the Alliance participants…” While the
MOU doesn’t expressly prohibit taxpayers from accessing Free File directly from the members’
Free File landing page, the IRS stated in a news release announcing the opening of the Free File
program: “The free online software program, accessible only through IRS.gov, ...” This is what
members said led them to exclude their own landing pages in an organic search, so that taxpayers
wouldn’t come directly to their company Free File landing pages.
To illustrate this point, Figure 1 from Section 1.1.1 was annotated to show the MOU-specified
path to Free File offerings. If Free File is only accessible through IRS.gov, taxpayers looking for
the program would have to either go directly from IRS.gov or go to IRS.gov from a web search
like Google or Bing. Taxpayers would not be able to go from other websites (like members’
commercial sites) nor would they be able to use a web search to bypass the IRS.gov landing page
to go directly to a member’s free file landing page.
Figure 11. The “MOU-specified” Path of Free File Through IRS.gov61.
ProPublica asserted the exclusion was used to drive searches for free tax filing to the companies’
commercial offerings, which could result in taxpayers paying for services, though they were
eligible for free tax filing. MITRE’s web analysis of organic searches shows the vast majority of
61
Icons created by Vectors Point, Lil Squid and monkik from Noun Project.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
46
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
those searching for free tax filing landed on commercial sites, and no searches landed on sites of
the five companies who excluded their Free File landing pages, though more searchers landed on
IRS.gov’s Free File landing page than any individual members’ commercial page (for unpaid
traffic only—four members’ web traffic exceeded IRS.gov traffic with paid searches).
It must also be pointed out that excluding the Free File landing pages may not have translated
into fewer Free File returns for those companies. Of the five companies who used the
NOINDEX, four increased their free filed returns in the 2019 filing season, and one remained
relatively flat. Of the seven companies who did not exclude their pages, five had increases in
Free File returns and two had decreases. While it’s impossible to know whether the increases
would have been greater had the landing pages not been excluded, three of the five companies
who excluded had larger increases in Free File over the previous year.
MITRE is not taking a position on the use of NOINDEX. If the IRS wants taxpayers to find
Free File only through IRS.gov, the use of NOINDEX by the FFA members may be appropriate.
However, it may also lead to the perception that companies are engaging in deceptive business
practices when that may not be the case. Adding a provision to the MOU on the use or not of
NOINDEX would provide clear guidance to the FFA members and would aid in clearing up
these perceptions.
4.3 Assuring Adherence to MOU
As the sponsor of the MOU, the IRS has an obligation to ensure FFA and members abide by the
agreed upon provisions outlined in the MOU. The Free File program office fulfills that duty by
closely managing the program and associated relationships. The office makes certain that
member offers are complete, free file coverage is available to all eligible taxpayers, and that both
IRS’s and member Free File sites are ready for filing season—all in accordance with the
specifications of the agreement.
While the IRS has a responsibility to safeguard the provisions of Free File to taxpayers, it shares
the duty to oversee MOU compliance with FFA. The IRS can only be an effective steward by
working in collaboration with FFA. This section provides MITRE’s understanding and
assessment of the overall management of the program, including a review of program objectives,
program structure and member oversight, and compliance activities.
4.3.1 Program Objectives
The mission of the IRS Free File program, as noted in the IRS’s Free File Program Management
Plan, is to “offer free tax preparation and e-file to economically disadvantaged and underserved
populations, at no cost to the government, by partnering with the Free File Inc. (FFI) a
consortium currently made up of 12 tax preparation software companies62.”
In support of that mission, IRS-specific business objectives guide the Free File program office in
their administration of the program. These objectives highlight the program office’s
responsibility to provide Free File through the PPP and the Service’s secure and reliable portal:
62
Draft Program Management Approach for Free File, IRS Free File Program Office, undated [Free File Program Management
Plan.v3.docx.]
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
47
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
•
Provide free electronic tax preparation and filing to taxpayers through a public-private
partnership model with a focus of serving the underserved and disadvantaged
populations.
•
Provide a safe and trusted portal via IRS.gov for taxpayers to link from IRS.gov to FFI
software companies. Deliver web content in clear and easy to understand language.
•
Convey Free File information in a clear and easy to understand language.
As written, these objectives clearly define the what, how, and to whom the IRS is providing the
program.
Article 2 of the Free File MOU outlines the shared objectives for the program partnership:
“Members shall work in concert with the IRS to increase electronic filing of tax returns,
which includes extending the benefits of online federal tax preparation and electronic
filing to economically disadvantaged and underserved populations at no cost to either the
individual user or to the public treasury.”
These objectives were established at the start of the partnership and, while they reflect IRS’s
mission, they include the specific goal to increase electronic filing of tax returns, as discussed in
Section 1.1.2. They are then translated in the MOU to more specific directives for FFA members,
which make direct reference to the two foundational, but dated, legislative mandates for the
program (RRA and the President’s FY2003 Budget).
Members shall also:
•
Make tax return preparation and filing easier and reduce the burden on individual
taxpayers, particularly the economically disadvantaged and underserved populations
•
Support the IRS’s statutory goals of increased electronic filing, pursuant to the IRS
Restructuring and Reform Act of 1998
•
Provide greater service and access to the Services to taxpayers
•
Implement one of the proposals in the President’s Fiscal Year 2003 budget, specifically
to encourage further growth in electronic filing by providing taxpayers the option to file
their tax return online without charge using cooperation with, and encouraging
competition within, the private sector
These directives have changed little since the program was established and they reflect the
nascent program’s focus on increasing volume of the agency’s larger e-file initiative. As noted
previously, given the e-file rate for individual income tax returns was 88 percent in FY 2018, the
IRS has now met and surpassed the original 80 percent goal stated for e-file.63 This issue was the
focus of recent external reviews of the program and both IRSAC and TAS recommended the
development of new goals, objectives, and performance measures for the program; the TAS
specifically recommends “targeted-use percentage” goals.
63
IRS Filing Season Statistics for Week Ending August 31, 2018, available at: https://www.irs.gov/newsroom/filing-seasonstatistics-by-year.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
48
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
However, when asked about their current objectives for the Free File program, IRS Submission
Processing leadership expressed they sought to “ensure there is a free electronic filing option for
lower- and middle-income taxpayers, and that those who are eligible for the program are aware
of it.” It was not their goal, necessarily, to target an increase in the use of the program, but rather
to ensure that all eligible taxpayers know that Free File is an option for the preparation and filing
of their tax returns.
They also discussed the need for and challenge of better defining the true demographic of
Free File users to bring some baseline clarity to the management of the program. Historically,
when reviewers of the program attempted to describe the Free File program’s “success,” they
reference as a measure the annual number of Free File returns as a percentage of the total
taxpayers eligible for the program. As noted in Section 3.3, while the total number of Free Fileeligible taxpayers is 70 percent of all individual filers, there are many reasons taxpayers, even if
they are aware of the option, choose not to use it (e.g., they use a paid preparer, they desire a
faster refund, they use VITA, or they use a free commercial offering).
When members were asked about the objectives of the program, they consistently cited the
principles at the heart of the Article 2 MOU objectives: to provide free online tax preparation and
e-filing to low- and middle-income taxpayers, and the underserved. While all members thought
the original objectives were still valid, several members acknowledged that the program’s early
aspirations were tied to increasing e-filing, which is no longer
the primary aim.
“Americans are a hodge-podge
When asked what success for the program looked like,
members cited two essential characteristics: that eligible
taxpayers are aware of the program as an option—and either
use it or make an informed choice not to—and that taxpayers
have a positive experience with the program. Members want
users to find the program easily accessible, simple, and straight
forward; they strive for taxpayers to be satisfied with the
interaction and consider using the offering in the future.
of people with varying needs
when it comes to preparing and
filing tax returns.
A blanket free option approach
will never cover 100% of the
eligible filers, and it shouldn’t.
[FF is] a great niche program
that will never cover 100% of
eligible taxpayers.”
-FFA member
Based on their stated goals for the IRS program office, the IRS
is meeting their currently articulated objectives for the internal
administration of the program. However, these goals and the shared objectives for FFA/IRS in
the MOU, do not seem to accurately reflect, with sufficient specificity, the true intent and current
strategy of the program today. While the shared objectives are still wholly supported by FFA
members and leadership, they are vestiges of the program’s founding and updating them to more
explicitly convey to both internal and external stakeholders the current program’s desired
outcomes would benefit the program. If awareness and availability are indeed the present goals
for the program, the IRS should use, and negotiate, more precise language that expresses those
outcomes.
Updated goals would enable the IRS to develop and communicate a meaningful measure of
success for the program. Recent external reviews used the program’s current participation rate
against all eligible taxpayers as a proxy to measure program success. Due to the nature of a
PPP—discussed in more detail in Section 4.4—and the vagaries of taxpayer behavior, such a
measure is not apt.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
49
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
IRS and FFA could update and clarify the shared goals for the program by revisiting and
formally redefining objectives for the Free File program and outlining: what outcomes for the
program IRS leadership see as success; what IRS operational measures would indicate progress
and could be used to manage by; and what data is needed to help explain the impact of the
program. If, for example, availability and awareness are the future goals for the program,
ensuring availability would hinge on the continued effective management of the MOU and
relationship with FFA; while increasing awareness would focus on increased outreach and
promotion to reach all eligible Free File taxpayers. Increased participation may be a by-product
of these activities but would not be a goal. Formally evaluating this population, as outlined in
Section 3.2, to determine how candidate taxpayers file and why they file as they do would help
the IRS better understand the true Free File population to target to meet awareness objectives.
MITRE cautions against the addition of prescribed targets for taxpayer participation in the MOU
objectives. Such targets would be both difficult to enforce fairly across members and a most
unwelcome provision to negotiate with voluntary
members of a PPP. As noted above, research into
Stakeholders supported by IRS’s Industry
Free File candidate population behavior and current level
Stakeholder Engagement and Strategy Branch
of awareness would equip the IRS to create meaningful
goals and inform discussions of perceived
• Council for Electronic Revenue
“underutilization.”
4.3.2 Program Structure
IRS’s Free File program is within the e-File Services
branch of W&I, Customer Account Services, Submission
Processing. To evaluate the current structure of the
Free File program, MITRE considered existing program
staffing, funding, and governance levels.
•
•
•
•
•
•
•
•
•
Communication Advancement
National Association of Computerized Tax
Processors
Federation of Tax Administrator’s Tactical
Advisory Group
Largest software companies via the
National Account Management program
IRS Pre-Filing Season Meeting
Reporting Agents Forum
Software Developers Conference
Payroll Industry Calls
Largest Transmitters Conference Calls
Contact Center Forum
Program Staffing
The Free File Program Office is a part of the Industry
Stakeholder Engagement and Strategy Office under eFile Services. It is composed of three full-time
employees—a team lead, senior program analyst, and
business analyst. The senior manager of the Industry Stakeholder Engagement and Strategy
Branch divides management time among the duties of that office, which include:
•
•
•
64
Administering the Free File program
Supporting the needs of internal business organizations to interact with the largest
external partner successfully
Supporting the broad tax ecosystem by enabling external partners to interact successfully
with the IRS64
Internal Revenue Manual, 12-28-2018.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
50
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
The host of internal and external stakeholder groups the Free File Office director supports in the
larger of role of Industry Stakeholder Engagement and Strategy Branch Chief are included in the
box to the right.
The Free File Office manager and team lead both have extensive experience with the Free File
program, numerous years with IRS, and are retirement eligible.
For TFF, the IRS must effectively
manage the relationship with FFA and its
members, while supporting the
collaborative management of oversight
and compliance activities per the MOU.
The Free File program office works with
Online Services (OLS), W&I Operation
Support Strategies and Solutions
Research, and EPSS in its development
and maintenance of program tools and
IRS website requirements; however,
taxpayer customer support is provided by
member companies. If the IRS is advised
of an issue within the TFF program, they
refer the issue directly to the member
companies to remedy.
TFF
The IRS’s FF Program Business Objectives
(highlighted by component)
FFFF
As the full set of business objectives for
the program (highlighted at right)
illustrates, the Free File program consists
of two components: Traditional FF (TFF)
and Free File Fillable Forms (FFFF).
Given the distinct nature of each of these
components, resource requirements to
support each are markedly different.
• Provide free electronic tax preparation and filing to
taxpayers through a public-private partnership and
disadvantaged populations.
• Provide a safe and trusted portal via IRS.gov for
taxpayers to link from IRS.gov to FFI software
companies. Deliver web content in clear and easy to
understand language.
• Provide a forms-based unbranded free federal
preparation alternative to taxpayers who don’t
qualify for the traditional FF software (Free File
Fillable Forms).
• Provide customer service support for the FF Fillable
Forms utility, including self-help options.
• Offer self-help interactive tools on IRS.gov to
reduce taxpayer burden and offer 24 X 7 services.
The tools enable a filer to self-identify corrective
action(s) necessary to successfully process the FF
Fillable Forms federal tax return.
• Convey FF information in a clear and easy to
understand language.
Source: Free File Program Management Plan v3
Two assumptions, as noted in IRS’s Program Management Approach for Free File, highlight the
parameters of the IRS program office’s support for TFF:
•
FFA will continue to provide management oversight of member companies
•
Each Free File member that provides free interview-based software will continue to
provide customer service
For FFFF, in contrast, the program office is responsible for all aspects of maintenance and
customer service support for the fillable form. Such duties include managing form requirements
and changes, creating/updating an in-depth user guide, updating line-by-line instructions,
training for W&I Accounts Management and EPPS staff, updating/testing an error search tool,
and managing a FFFF mailbox. Responsibility for the FFFF consumes the majority of the time of
two of the three full-time office staff. The program office team is focused on reducing errors and
improving customer experience for the FFFF.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
51
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
The principal driver for program office staff is planning and preparing for, and then executing, a
successful filing season for both TFF and FFFF. For TFF, each IRS Free File Program Office
employee completes the formal website review of four members (between September and
December), and works with member POCs to resolve any issues found. A high-level summary of
the primary duties and estimated percentage of time each Free File Program Office employee
spends on each program component is provided in Table 7.
Table 7. Free File Program Office Key Areas of Focus.
Employee
Team Lead
Key Areas of Focus*
Acts, in effect, as a national account manager for FFA,
managing the agency response to FFA’s industry partners
Estimated Time per
Component
~40% TFF
~60% FFFF
Plans program activities
Supports FFFF customer service
TFF Senior Program
Analyst
Reviews and supports team TFF activity
Supports program planning activities
100% TFF
Manages TTF requirements and company offers
FF Fillable Forms
Business Analyst
Ensures TFF program documents, reviews and activity are
completed timely
Supports FFFF customer service
~85% FFFF
Manages FFFF mailbox
~15% TFF
Supports TFF requirements in the Fall
*Areas of focus list is not comprehensive, but rather highlights key duties; estimated time per component is based on
interviews with program staff.
Given the nature of managing a public-private partnership, and the results evident in MITRE’s
assessment of member compliance, including the empirical success of the program as a PPP (see
Section 4.4), IRS’s current program resource allocation—for relationship management, planned
periodic reviews, and direct, transparent collaboration with FFA members—is appropriate for the
current level of work required by the Free File program.
As noted above, the Free File program is not a traditional contractual arrangement and success
for the program as a PPP calls for joint responsibility and decision making, where oversight and
compliance duties are shared. Evident from interviews with both FFA and IRS stakeholders,
much of the successful operational performance of the TFF program to date is due to the wellestablished relationships that IRS program staff, particularly the program manager and team lead,
maintain with FFA leadership, members, and contractors. Some relationships have spanned
decades and the level of trust, professional respect, and cooperation that exists is central to the
program’s accomplishments.
Given key personnel within the program office are currently retirement eligible, the IRS must
ensure, perhaps more so than in a traditional program, that knowledge transfer and succession
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
52
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
planning are deliberate and well-planned. In addition to gaining technical expertise, new FF staff
must have the time and space to build trust and establish collaborative working relationships.
Moreover, any additional or more in-depth technical reviews of member websites or significant
compliance process changes would require a careful assessment of both the nature and required
time of the new duties to ensure the workload and required skill sets are appropriate for current
staff resources. And, more importantly, the IRS must ensure the new duties support and do not
jeopardize the high level of relationship management and trust that is required for such a PPP—
and that the changes do not impact the spirit or the letter of the program.
Program Funding
Currently, the Free File program does not have a marketing budget. As such, the Free File
program office works with W&I Communications and Liaison to utilize the existing
communication and promotion vehicles available to all IRS programs, e.g., press releases before
and during filing season and links on the IRS.gov website. As a part of IRS’s larger, annual
campaigns, promotion of Free File in news releases, tax tips, and fact sheets is not necessarily
specifically targeted to the Free File population. A 2018 redesign of the IRS.gov main page
added a “File Your Taxes for Free” button at the top of the page to simplify navigation to Free
File. There is also presently a link to Free File on page one of the Form 1040 instructions.
IRS also employs social media to promote Free File, actively posting on Facebook, LinkedIn,
Instagram and @IRSNews and @IRSenEspanol. A comprehensive review of IRS’s recent FF
outreach activities can be found in the taxpayer journey IRS.gov analysis in Appendix F.2
From 2006 through 2009, the Free File program conducted a Free File Customer Satisfaction
Survey, utilizing the firm Russell Research. This telephone survey of a nationally representative
sampling of Free File users provided the program with insight into users’ perceptions of
Free File and areas for program improvement. The survey provided information not available
from IRS data routinely collected and enabled a more in-depth understanding of the Free File
customer experience. Drivers for Free File usage and returning to the program were explored; in
the 2009 survey, ease of navigation and likelihood of using the same provider were assessed.
2009 was the last year the survey was conducted.
Given paragraph 4.36 of the Free File MOU specifies that promotion of the Free File program is
the IRS’s responsibility, the IRS has relied on unpaid vehicles and standard outreach tactics to
fulfill this provision and increase awareness of the program. Past external reviews of the program
recommended the IRS consider member companies assistance in promoting the program;
however, several constraints need to be addressed to support such a pursuit, e.g., does such
action align with the spirit of the PPP, and how would the IRS and FFA ensure such promotion
would not result in the exclusive benefit to specific members.
Based on MITRE’s web forensic analyses (see Section 4.2.2), paid web advertisements have a
five-fold more effective result than organic web search hits. However, before pursuing the need
for paid marketing and promotion, which would likely require legislative action, the IRS would
benefit from better understanding the current level of awareness of the Free File program as an
option. Low participation does not always indicate a lack of taxpayer awareness as other factors,
not only cost, drive a taxpayer’s choice of how to file. The last awareness study was conducted
in 2011 and conducting another such study would allow the IRS to baseline the current level of
awareness and would inform strategic goal setting for the program.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
53
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
In addition to assessing taxpayers’ current level of awareness, resurrecting the use of a taxpayer
survey, not just for Free File users but also for those taxpayers eligible but choosing not to use
the Free File program, could provide valuable insight into what other options taxpayers are using
and what influences taxpayers to choose them. The results of a satisfaction survey, combined
with demographic and migration data provided by IRS Research, can provide the specific
targeted populations to make the Free File outreach and promotions more effective, and give a
clearer understanding of the true population for Free File. Provision 4.28 of the MOU specifies
that FFA members “will provide necessary support to accomplish a customer satisfaction
survey”; hence, the IRS may request and negotiate assistance from FFA members in the
administration of such a survey.
If, based on future program direction, unpaid outreach continues to be the primary avenue to
promote the program and increase awareness, the IRS could look to leverage related IRS
programs (and other federal sites) to expand the Free File program’s reach, as they currently do
with VITA, e.g., EITC, Low-Income Tax Clinics (LITC), other agencies (Department of
Veterans Affairs, Department of Defense), universities, and parents of first-time filers.
Program Governance
In assessing the Free File program’s governance, MITRE considered both the Free File
program’s internal monitoring to ensure the program is operating effectively and supporting
IRS’s strategic goals, and the shared governance of the program as required by a PPP.
Internal Program Monitoring
Like most long-standing, established programs, the Free File Program Office provides program
status in routine, as needed updates to IRS leadership and there is no prescribed program
performance review.
The office monitors and reports on the accurate and timely completion of tasks against its
schedule to ensure a successful kick off and execution of filing season, and collects and monitors
program data, such as weekly volume of accepted returns during FS. The office annually runs a
demographics report of Free File returns to obtain insight into the Free File user population and
conducts ad hoc migration analysis to assess movement of taxpayers into and out of the Free File
program.
Oversight and advisory boards that assess and support the program office’s performance include
the Electronic Tax Administration Advisory Committee, IRSAC, TAS, TIGTA, and the U.S.
Government Accountability Office (GAO). A crosswalk of MITRE recommendations with
reports from IRSAC and TAS is included in Appendix B.
IRS/FFA Partnership Management
Given the Free File program is a public-private partnership and not a traditional government
program, true governance of the program and its performance is shared between the IRS and
FFA. As noted in the IRS’s Free File Program Management document, the program “is governed
by W&I Submissions Processing and the FF Inc.” As such, the IRS meets with FFA leadership
and FFA members in the spring and fall of each year to discuss status and proposed changes to
the program. For day-to-day operations, the IRS senior manager of the Free File program
maintains open communication and manages the relationship with the FFA executive director;
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
54
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
the IRS Free File program team members collaborate directly with FFA members to address
routine status and items of concern throughout the year.
As part of the management of the MOU between FFA and IRS, one year before the expiration of
the MOU, the IRS Free File team develops a compendium of potential improvements for the
program based on lessons learned and recommendations from oversight bodies. These
improvements are reviewed and prioritized by IRS CAS leadership and the list is used to develop
IRS’s negotiating options for a revised MOU. Those options are then vetted by the W&I
Commissioner. After negotiations with FFA are complete, FFA legal counsel and IRS General
Legal Services update the language of the MOU to reflect the renegotiation. As stated
previously, since the program began, updates to the MOU have been additive and have not been
part of a comprehensive review of the entire document.
Effective program governance is based on sufficient articulation of program goals and adequate
information on program performance. Evaluating the Free File program through the lens of a
PPP (which is outlined in detail in Section 4.4), and given the level of the program’s current
compliance with the MOU, IRS’s existing governance of the FF program is appropriate and
effectively allows the program to fulfill the IRS’s currently articulated goals for the program. To
help ensure the MOU remains an unambiguous and durable document by which to manage the
program, the IRS should encourage a periodic global review of the MOU to make certain it
accurately and clearly reflects program objectives.
Good governance also focuses on decision making and requires appropriate data and reporting to
inform those decisions. Currently, the Free File program office is able to provide an appropriate
level of information on program performance based on existing program goals. However, as
noted above, to enable a more accurate discussion of the program and help the IRS, as well as
interested stakeholders, better understand the impact of the Free File program more clearly, the
IRS would benefit from continued research to determine the true target population of Free File
users and eligible taxpayers’ preferences when filing (as referenced in Section 3.2).
While gathering data on the current population of Free File participants allows for more targeted
marketing and effective outreach to current users, if the IRS seeks to increase awareness of the
program to all those eligible and likely to use it, analysis of the larger population that is eligible
but not currently participating would be beneficial. Such data could help the program better
equip IRS leadership in program advocacy and ensure the appropriate level of resources continue
to support the partnership. More and detailed data to explain the program’s direction would
create enhanced clarity for the public and increased stakeholder understanding of the program.
The introduction of measures, metrics, or targets into a PPP brings with it a multitude of
challenges. Given the nature of the IRS/FFA partnership and current objectives of the program,
trying to measure the value and success of the program is difficult; any introduction of such
targets would need mutual agreement on what success looks like and would need to ensure that
all members remain free from attribution and bias, and proprietary data is respected.
4.3.3 Member Oversight and Compliance Activities
Consistent for a public-private partnership, Free File oversight and compliance activities are a
shared responsibility between the IRS and FFA. The Free File MOU requires the IRS and the
FFA ED to monitor and assess member compliance on an ongoing basis as part of program
oversight.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
55
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
Each year, FFA enters into an operating agreement with the FFA members. As part of the annual
application for membership, each member must complete and sign a questionnaire that certifies
compliance with select provisions of the MOU. This self-attestation is shared with the IRS as
evidence of compliance. The FFA also contracts with an independent auditor to conduct a
comprehensive audit of member sites to ensure member websites are in compliance with the
provisions of the MOU. In addition, due to the competitive forces at work among FFA members,
members are compelled to police each other regarding MOU compliance. Any issues they detect
are reported to the FFA ED.
As part of its efforts to ensure a successful filing season for the program, the IRS Free File
program office ensures the members’ offers are timely and complete, and their software and
website are ready for the new filing season. Employing a set of established procedures and tools
to assess member MOU compliance, the IRS conducts website reviews, evaluates member offer
coverage and monitors member acceptance rates. Specifically, the IRS Program Office:
• Confirms member offers fall within the agreed upon coverage of not less than 10
percent and not more than 50 percent of the eligible taxpayer population; the program
office forwards a calculator tool developed by IRS Research to assist members in
determining this coverage and developing their Free File offers.
• Ensures members meet a 75 percent acceptance rate for Free File returns
throughout the filing season; an IRS data analyst generates a report of accepted Free File
returns on a weekly basis and acceptance rate pass/fail is part of IRS’s mid-year website
review.
• Conducts two website reviews of each member company, one pre-filing season review
(in December/January) and one mid-year review (March-May). The program office uses a
website review guide, assessing each member site against 28 areas of the MOU with a
pass/fail rating. The mid-year review contains two additional elements focused on
acceptance rate and free extensions.
After conducting its member website reviews, the IRS program office works collaboratively with
the FFA independent auditor to adjudicate the findings. The FFA auditor separately evaluates all
items included in the IRS’s website review guide, in addition to conducting a significantly more
comprehensive audit of member sites, particularly in the area of web security, website links,
search results, and source code. All member problems that need to be addressed are captured and
tracked until they are remedied. The Program Office works directly with the FFA auditor and the
member POCs to address deficiencies as part of a formal resolution process. To date, there has
not been a case where a member issue was not adequately resolved. In addition to these formal
reviews, the IRS program office works directly with members and FFA leadership to jointly
address MOU compliance issues, if necessary, as they arise.
Figure 12 shows the key program management and oversight activities conducted by the IRS.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
56
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
Figure 12. Key IRS Free File Program Management and Oversight Activities for TFF.
Based on discussions with both the IRS and FFA representatives, and in light of the results of
this assessment, MITRE considers the IRS’s current compliance processes adequate and
effective to support the integrity of the program. IRS’s current oversight and compliance
activities are the result of the partnership’s continuous assessment and improvement of the
program as it has evolved. As evidenced in the history of the Free File Program Agreement
outlined in Section 4.1, the IRS program office has incorporated with each iteration of the MOU
additional or modified provisions to the compliance process to address potential deficiencies
identified by both parties and outside bodies. For example, the formal addition of a mid-year
review of member sites, added last year, reflects the program office’s considered response to
external recommendations.
As noted above, the recent suggestion for the IRS to perform additional or more extensive
compliance reviews must be carefully weighed against the true return of such a provision. As a
PPP, any threat to the integrity of the program should be solved through the collaborative work
of both FFA and IRS. Additional member requirements negotiated on behalf of the IRS may
reduce members’ incentives to participate and jeopardize the overall gains of a successful
partnership.
Finally, when the Free File program began, the program “led the charge,” as one IRS staff
member advised, on security requirements for e-file. Hence the original MOU was populated
with the detailed security and privacy requirements that the IRS was advancing as part of its
larger e-file efforts. Today, however, Free File members are held to the same security
requirements, and no more, as all e-file providers. Currently, member MOU-related security
compliance is monitored in part through self-attestation, in part through website review, and
more thoroughly during the FFA independent audit of members each year. The IRS receives all
companies’ PCI scans. Given all FFA member companies are authorized IRS Electronic Return
Originators (EROs), are subject to the assurance process of that role, and are members of the
Security Summit, the IRS and FFA may want to consider reviewing the MOU with a critical eye
to what security language is still relevant and warrants inclusion. To offset any risk, the program
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
57
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
may want to increase the level of corroboration with IRS’s larger e-file security assessment arm,
e.g., to ensure they are informed if any member is out of compliance with the e-file requirements.
4.4 Assessment of Public-Private Partnership
As described in Section 1.1.4, many media reports and congressional concerns about the
Free File program have focused on the number of taxpayers who take advantage of the program.
While it’s important to have metrics to determine whether a program is meeting its objectives,
other factors can also determine the success or failure of a program. The Free File program is a
unique entity that was created to provide a service to taxpayers that neither the government nor
private industry could address alone. As depicted in Figure 13, this partnership requires a balance
that serves the interests of the government and taxpayers, but also creates a value to the for-profit
industry that provides the service. Without that value, private companies may decline to
participate. As a piece of the overall assessment, MITRE looked at the partnership itself, separate
from the service it provides.
Memorandum of
Understanding
(MOU)
IRS
Free File, Inc.
Lower administrative and oversight
cost (including, tax processing and
efficiency costs due to increased efiling; Free File administrative and
oversight costs, etc.)
Increased business development
and revenue generation
opportunities.
Free File
Program
Taxpayers
Lower economic cost of tax
preparation and filing (time,
money, effort, etc.)
Figure 13. Public-Private Partnership: Balance of Value to Stakeholders.
The lack of a shared understanding of what a PPP is makes the process of evaluating whether a
PPP has been successful complex.65 In looking at the Free File program, MITRE uses existing
literature to provide a framework to assess the program’s performance as a PPP.
65
G. A. Hodge and C. Greve, "On Public–Private Partnership Performance: A Contemporary Review," Public Works
Management & Policy, vol. 22, no. 1, pp. 55-78, 2016.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
58
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
Research suggests using a broader set of criteria for assessing whether partnerships are
successful. For example, other key attributes of successful PPPs including democratic
responsiveness and process legitimacy, are considered to be equally important values for public
undertakings.66
In addition to interviews with each partner and review of literature and documentation regarding
the Free File program, MITRE used a comprehensive framework for assessment of cross-sector
collaborations67 to assess the Free File program PPP. The authors explain that the framework
encompasses dimensions from the disciplines of public management, administration, and
accountability. They suggest that PPPs can be evaluated in three categories of measures:
1. Democratic accountability, otherwise stated as the degree of stakeholder engagement
and support. Do both public and private leaders and partners clearly endorse and support
the work? Are partners participating and executing on their commitments? Do all partners
believe they are gaining the value through their support and participation?
2. Procedural legitimacy, otherwise stated as process measures. Are processes fair,
transparent, rational, and intentional? Process legitimacy is critical to measure because it
improves the chances that partners, sponsors, and others accept the results that PPPs
produce, and increases the likelihood that collaborations are managed and implemented
responsibly and effectively.
Substantive outcomes, which emphasizes effectiveness and efficiency, and also encompasses the attribute of
equity (i.e., who pays and who benefits from a collaboration’s work, and what is distributed.)
Table 8 summarizes the framework, and the attributes of value in each of the three categories
above. The third column identifies indicators demonstrating how the Free File program measures
against those attributes.
Table 8. Public-Private Partnership Assessment Criteria for Free File Program.
Democratic
Accountability
Attribute of Value
Definition
Vertical democratic
accountability
Extent to which PPP is responsive
to legal mandates, authorizers,
funders
Horizontal democratic
accountability
Extent to which PPP is responsive
to partners and other external
stakeholders
Free File Indicators
• Legislators, administrative officials,
senior agency officials show firm
support for the program
• MOU signed by agency
• Implicit or explicit support from
key stakeholders including
Congress, TIGTA, GAO, TAS
• Periodic MOU updates are
responsive to concerns from
partners and/or key stakeholders
66
R. D. Behn, Rethinking Democratic Accountability, Washington, D.C.: The Brookings Institution, 2001.
S. B. Page, M. M. Stone, J. M. Bryson and B. C. Crosby, "Public value creation by cross-sector collaborations: A framework
and challenges of assessment," Public Administration, vol. 93, no. 3, pp. 715-732, 2015
67
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
59
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
Substantive Outcomes
Procedural Legitimacy
Attribute of Value
68
Definition
Procedural rationality
Extent to which decisions are
based on technically and
administratively sound data,
analysis, and planning
Procedural justice
Extent to which stakeholders
perceive decisions and activities
to be fair and transparent
Operational control
Extent to which PPP uses
requirements, budgets, and
schedules to oversee projects and
activities
Effective performance
Extent to which collaboration
achieves its goals
Efficient performance
Extent to which collaboration
achieves its goals at reasonable
costs
Equity of benefits
Extent to which benefits of
collaboration are spread
appropriately among stakeholders
and the public
Free File Indicators
• MOU is valid for a specified period
and jointly negotiated at regular
intervals
• Eligibility requirements updated
annually based on data from both
IRS and individual members
• In addition to periodic
renegotiation, MOU provision
allows for annual updates based
on filing season data
• Long standing relationships have
created trust and transparency
between the partners
• Mature process for negotiations,
equal bargaining position for both
IRS and industry
• IRS program office works
collaboratively with each member
on compliance upfront
• IRS Program office has
documented procedures for
oversight of the program,
conducts compliance checks prior
to filing season
• FFI contracts yearly compliance
checks on each of the 12 members
• Because it is a partnership, both
IRS and FFI work collaboratively on
any findings from either party
• Based on the four objectives
articulated in the MOU, the parties
achieved their goal by providing
free filing to taxpayers and by
increasing the e-filing participation
to more than 90 percent
• The goals should be updated to
include new performance metrics
• The cost of the IRS providing a free
offering itself would be in the
hundreds of millions of dollars68.
By industry providing free access,
the IRS is able to minimize costs
with three FTE program office
staff. Partners receive benefits
cited in Appendix A, The
Economics of Free File.
• IRS benefitted from not having to
create its own e-file program,
while industry benefited from
keeping the IRS out of the market.
the IRS was able to fulfill the
President’s Management Agenda
at the time of inception and
The MITRE Corporation, "Advancing E-file Study - Phase 2 Report," Internal Revenue Service, Washington, D.C., 2010.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
60
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
Attribute of Value
Definition
Free File Indicators
•
•
Equity of payment
Extent to which costs of
collaboration are spread
appropriately among stakeholders
and the public
•
•
Problem-solving
capacity
New behaviors or norms that
increase the potential to address
complex problems; reflects lesstangible outcomes that affect how
partners do their work now and in
the future
•
•
•
provide citizens with the ability to
file electronically for free.
Industry has received benefits
(stated above)
Because the 12 members are of
various sizes, the 10/50 rule was
initiated to provide equity among
the members and prevent the two
large companies from drawing all
traffic
Regardless of size, each member
pays an equal amount to belong to
the Alliance. While MITRE did not
specifically ask each member
about whether they perceived this
to be fair, no negative comments
were made.
IRS maintains a small program
office to administer the program
IRS maintains a strong working
relationship and environment of
trust with the members
As technology and the
environment change each year,
the Alliance and the IRS work
collaboratively to address these
with changes to the MOU
The MOU provides opportunities
for innovation within the industry
Using the above framework, MITRE concludes that the Free File program is an effective PPP
given the value that the Free File program provides to the public as a venue for taxpayers to
prepare and electronically file their tax returns for free; to the IRS in providing that venue as a
government service; and to the industry, not only in terms of preserving their unique role of
preparing and filing taxes, but also in providing a potential economic benefit to members as
discussed in Appendix A.
4.5 MOU Renewal in 2021
While the IRS could use the MOU to negotiate stricter terms to alleviate taxpayer and public
concerns about FFA activities, there are limitations to this process. First, the MOU is ultimately a
two-way negotiation tool. Moreover, changing MOU provisions that restrict or dictate FFA’s
business practices may require other concessions to FFA members in return. Individual FFA
members may demand concessions that align with the members’ own economic and business
incentives adding another layer of complexity to the negotiation process.
Substantial changes to the MOU that either aim to ease or tighten restrictions on providers of
Free File can influence both the use and provision of Free File services. Easing restrictions on
FFI members would create opportunities of extracting additional revenue from Free File users
through commercial tactics. If providers impose commercial tactics on Free Filers, taxpayers
would be indifferent between Free File and comparable commercial software, potentially
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
61
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
reducing the use of Free File. This would be a considerable problem at high volumes of Free File
usage, because a large drop in the demand for Free File could affect overall e-filing numbers. In
the current state of affairs, this is not a significant concern as a drop in the current Free File
numbers would not affect overall e-filing figures enough to cause it to fall below the 80 percent
overall e-filing target of the IRS.
Monopolistic
Competition
• Greater opportunities of
economic profit for FFI
• Numerous companies are
likely to participate in
Free File
Regulated
Monopoly
MOU Restrictions on FFI Activities
• Limited opportunities of
economic profit for FFI
• Few companies are likely
to participate in Free File
• Existing FFI members
may leave the program
Figure 14. The MOU as a Negotiation Tool.
Figure 14 demonstrates how the provisions in the MOU can impact the provision of Free File
services. The number of firms that are willing to participate in the MOU depend on the extent to
which the IRS imposes restrictions on the business activities of tax software companies. Under
limited restrictions, firms have greater opportunities of extracting profits from taxpayers through
commercial business tactics. As restrictions on the FFA lessen, the Free File market approaches
characteristics of a more competitive industry with many potential service providers and
increased consumer choice.
Conversely, a restrictive MOU that impedes on business development and profit-making
activities of tax software companies would drive out firms from participating in Free File. As
more and more suppliers leave the program the market would approach characteristics of a
regulated monopoly in which both the number of providers as well as consumer choice is
limited. A lower number of providers may also impact the number of taxpayers that are eligible
to participate in the program under the limited eligibility criteria of a fewer firms.
The discussion above demonstrates that the IRS stands in a unique position of having to balance
taxpayer expectations from the program with the industry members’ outlook for economic gains.
From some stakeholders’ perspective, one Free File eligible customer paying for tax preparation
services is one too many. From FFA’s perspective, the provision of Free File member offerings
costs money, and the public-private partnership must leave room for recovering these costs and
potentially generate additional business. In other words, the cost of provision of the free services
to some customers must be subsidized by other paying customers.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
62
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
Taxpayer Experience with Free File
Taxpayers are, of course, the target population for the Free File program and its accompanying
systems. As such, examining the taxpayer experience is crucial to a holistic assessment of Free
File. MITRE approached the taxpayer experience from two complementary research
perspectives: the expert perspective, represented by the heuristic review; and the taxpayer
perspective, which MITRE investigated via usability testing interviews with prior Free Filers.
In many cases, the findings from the heuristic review and the usability testing mirror each other;
a major advantage of conducting these research streams in tandem is the ability to observe a
large set of potential usability issues for a wide range of systems in the heuristic review, and
then observe whether or not these potential issues manifest in testing with real end users
(taxpayers, in this case). Integrating the results from the taxpayer journey heuristic review and
the usability testing with taxpayers, the following high-level findings stand out:
•
Messaging and functionality on IRS.gov pages and within member systems to help
taxpayers determine eligibility and ineligibility for Free File needs improvement
•
Taxpayers lack understanding of what Free File is and the IRS’s relationship to the
program; use of consistent, plain language on IRS and member pages would help
mitigate this issue
•
Taxpayers experience difficulty navigating the Free File landing page and the
software offers page; they lack clear “calls to action” and are too text-heavy
•
The lookup tool and the Free File member systems could take more actions to
prevent users from making errors, and help them correct errors where committed
MITRE conducted two research activities in order to understand how taxpayers interact with and
experience the Free File system and to evaluate the usability of the Free File system components.
MITRE evaluated Free File experience aspects including searching for Free File online,
navigating IRS.gov web pages, selecting a Free File software offer, and filing a return with a
Free File member system.
MITRE approached the taxpayer experience from two complementary, yet distinct, angles:
1. Taxpayer journey—expert review of Free File member systems and IRS.gov
(heuristic review). In this activity, MITRE human factors engineers drew on established
industry best practices and expert knowledge of system usability to evaluate various
dimensions of usability and compliance with the MOU. MITRE identified potential
usability issues in a systematic, exhaustive fashion and made recommendations on the
basis of technical expertise.
2. Usability testing sessions with Free File taxpayers. In this activity, MITRE researchers
conducted in-person interviews with taxpayers who filed using Free File in TY 2018.
This activity allowed MITRE to observe usability issues encountered by real end users
and to collect qualitative data on taxpayers’ perspectives, opinions, and why aspects of
the system caused difficulty.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
63
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
This two-pronged approach to understanding the taxpayer experience allowed MITRE to
mitigate some of the limitations of each individual approach. Although the usability testing has
the advantage of greater external validity—the ability to generalize beyond the testing
situation—due to its focus on actual end users, usability testing is time-consuming and resourceintensive, and as a result, the MITRE interviews tested three of 12 Free File member systems. In
contrast, the heuristic review encompassed all 12 member systems. Additionally, combining the
heuristic review with usability testing presents the opportunity to observe which potential
usability issues identified in the heuristic review caused problems for taxpayers in interviews.
In the following sections, MITRE reports the key findings from the heuristic review and the key
findings from the usability testing. Connections between the two sets of findings, with
conclusions, are spelled out in Section 5.3.
5.1 Taxpayer Journey
This section of the report summarizes the findings from the expert usability review of the 12 Free
File member systems as well as of IRS.gov Free File pages (landing page, offer page, etc.).
In this review, MITRE human factors engineers applied three sets of criteria, known as
heuristics, to evaluate the usability of the systems. MITRE used three fictitious taxpayer
scenarios to navigate the Free File systems in the course of the review. The three types of
heuristics MITRE used to evaluate the member sites and IRS.gov’s Free File-related pages were:
1. Nielsen’s 10 usability heuristics. Factors used to evaluate overall system usability and
user experience. These factors were rolled up to determine a single “user experience”
score. These heuristics are detailed in Section 2.5.
2. MOU UX requirements. MITRE viewed the MOU requirements from a usability lens.
MITRE derived a set of criteria based on the MOU requirements that are related to user
experience, such as “transparency around ineligibility.” MITRE then rated the systems
against this list of criteria and rolled up these factors to determine a single UX MOU
adherence score. Note, this set of MOU-related heuristics is separate from the MOU
compliance assessment activities and should not be interpreted as evidence of member
compliance or noncompliance with the MOU. The heuristics are also detailed in Section
2.5.
3. Mobile optimization. MITRE evaluated to what extent the member systems were
optimized for use on mobile devices.
5.1.1 Heuristic Review of Member Free File Websites
MITRE evaluated the member sites against Nielsen’s 10 usability heuristics to derive the user
experience score, against the MOU-related heuristics to derive the UX MOU adherence score,
and evaluated mobile optimization to derive a mobile experience score. The severity ratings
MITRE used to determine these scores are displayed in Figure 15. All of the scores from the
heuristic review are rolled up into the summary-level dashboard shown in Figure 16. The
dashboard provides a quick view of the summary scores for user experience, mobile experience,
and MOU adherence from a UX perspective.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
64
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
Figure 15. Severity Level and Description.
The information that follows provides a more detailed breakout of the scores from the heuristic
review by each heuristic and grouping of MOU requirements. The numbers across the columns
identify the member systems; the heuristics sets are listed on the rows.
Figure 16. Heuristic Evaluation Dashboard Summary.
As shown in Figure 17, at a glance, the Free File member organizations adhere to many UX best
practices. Overall, they could improve the most in the areas of “error prevention” and “match
between system and real world” heuristics. The most significant UX MOU improvement to make
across the board is “transparency around ineligibility.”
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
65
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
Figure 17. Heuristic Evaluation Dashboard.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
66
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
The complete set of results from the heuristic assessment for individual Free File member
websites is found in Appendix F.1.
5.1.2 Heuristic Review of IRS.gov
In addition to assessing member’s Free File sites, MITRE also reviewed the user experience for
Free File-related pages on IRS.gov and other relevant IRS materials.
Specifically, in the heuristic review of IRS.gov, MITRE evaluated the user experience of
determining how to file taxes for free and identify a company to use within the Free File
program, including: 1) issues observed in web searches for Free File and related terms; 2)
IRS.gov internal search functionality; and 3) IRS.gov navigation. MITRE identified and
described potential user pain points along the overall experience.
Similar to the heuristics review of the member sites, the taxpayer journey from search to lookup
tool was evaluated against Nielsen’s 10 usability heuristics to derive a “user experience” score
for search, navigation, and the IRS Free File landing page and application. MITRE also provided
comments on examples of marketing materials shared by the IRS but were not included as part of
the heuristic review due to messaging appearing in familiar user interfaces, such as Twitter or
Facebook.
The majority of heuristics issues were on the IRS Free File landing page and application. For
example, the landing and software offers pages are text-heavy, difficult to scan, use unfamiliar
tax jargon, and do not clearly present the next step to users. Figure 18 provides a snapshot of the
IRS.gov review, specific to the IRS Free File landing page.
Figure 18. Samples from Heuristics Review for IRS.gov Landing Page.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
67
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
Similarly, the software offers page uses overly technical or potentially unfamiliar terms like
“lookup tool.” The list of vendors and their criteria is hard to scan, as users must read
inconsistently formatted bullets with conditional statements that are difficult to understand.
Issues such as these identified in the heuristic review were used to develop user interface design
recommendations. Figure 19 and Figure 20 offer two potential design alternatives intended to
improve scanning for FFA member offers.
Figure 19. Design Alternative #1 to Improve Scanning for FFA Member Offers.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
68
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
Figure 20. Design Alternative #2 to Improve Scanning for FFA Member Offers.
Other potential opportunities identified through the taxpayer journey analysis include:
• Free File awareness. Free File is referred to as a “known” program throughout the
messaging in the promotional materials. Given that so few taxpayers participate in the
program, it’s likely the awareness and understanding reading “Free File” would be low for
the average taxpayer. Explain what the Free File program is to people who have never
heard of it in very plain and simple language. Also, gather foundational knowledge in
support of further testing that could be done related to the value of name recognition on
the IRS website. Users rely on finding what they need based on terms they know (e.g.
Form 1040). In general, the Free File program is never clearly explained with its purpose,
requirements, and how to participate. The way it’s referred to throughout the site makes it
seem as though it’s very well advertised and explained, making users feel they may have
missed something and fumble through the action buttons presented to them without much
purpose.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
69
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
• Keep language easy to understand. Words like “software,” “e-file,” “AGI,” and “EITC”
are used repeatedly in relation to Free File when those words may not mean anything to
the average user. Avoid compliance-oriented language in favor of service-oriented
language by providing more plain language descriptions of tax terms and IRS programs.
Using the word “software” is misleading and outdated for users since it isn’t something
they need to install. Given that the programs are accessible from mobile devices,
recommend using words like “online” or “website” or “program.”
• Rethink the program with the novice in mind. Determining eligibility for the Free File
program may be intimidating to taxpayers. The vendor requirements are complex and the
lookup tool asks for information that is not common terminology. Where possible, attempt
to reduce the complexity of the vendor requirements for the program.
• Keep users within the task at hand. The lookup tool assumes taxpayers are
knowledgeable of their adjusted gross income and eligibility for the Earned Income Tax
Credit. For novice taxpayers, the help text is not written in plain language, and references
content and calculators outside of the tool. This forces users to leave their current task at
hand and work on what might feel like an unrelated task, increasing the chance they do not
return to participate in the Free File program. If an eligibility requirement includes
knowledge of a program or calculation that is not commonly known, provide a simple
calculator within the tool itself.
• Clearly outline Free File options upfront. Determine the primary business goal of the
page(s) and use it to drive layout and content development. For example: If the primary
goal is to move people towards exploring their eligibility and the options available, use a
task-oriented approach. Provide “Income below” and “Income above” boxes above the
fold, with buttons that clearly describe the next step in the process. The landing page
buries the actual “start now” process behind many clicks. Improve the overall readability
by developing content from “bite, to snack, to meal”69 that supports the primary business
goal of the page. Ensure that the primary conclusions of the content are summarized
within titles, headings, and initial content. Make sure distinct topics are separated on the
page (one sentence paragraphs are fine) and aim to use keywords (like state, income,
military) within the first few words of a sentence.
• Branding strategy. The IRS marketing, public release, and social media presence around
the Free File program does not have a consistent voice, purpose, or strategy. The design of
the visuals connected to the various forms of social media all follow different styles and
themes, making it difficult to thread together a coherent brand for a taxpayer. The IRS
loses credibility by disorienting the user with different fonts, pages, and calls to action.
Recommend creating an overarching communications strategy around Free File that
matches the style of the rest of IRS branding with specific calls-to-action and educational
materials to teach those unfamiliar with the program about how to participate.
• Continuous improvement. When determining the primary business goal of the Free File
page(s), capture baseline metrics from web analytics, usability testing, and overall
satisfaction scores. For example, in Appendix F.2 (page 20) shows that only 32 percent of
69
C. R. J. S. K. S. K. Jarrett, "Design to Read: Guidelines for People Who Do Not Read Easily.," 2010. [Online]. Available:
http://uxpamagazine.org/people_who_do_not_read_easily/. [Accessed 15 August 2019]
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
70
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
users click through to the application from the Free File landing page, in comparison to 76
percent from the refunds landing page to the application. While it is not a direct
comparison (a very different audience and goal), it poses two questions: are the right users
navigating to the landing page, and can the number of eligible users who navigate to the
application increase (by say, content and user interface updates)? Use captured baseline
metrics to compare the performance of future redesigns against program goals.
• Error prevention. Help users through the basic steps of determining an estimated AGI
and eligibility for EITC to prevent errors. For age include more descriptive labeling, or
update the question so it takes less thought (birthday versus age). Include a link to fillable
forms and back to the offers page. Incorporate primary site navigation such as the new
withholding tool.
• Support search. Identify new ways to help users pick IRS Free File out of the sea of
“free” offers in the most prominent search engines. Consider and test all available options:
from paid search advertisements to more task-oriented headers that include IRS within the
first one or two words (so that users can scan and immediately identify with IRS). To
improve search within IRS.gov, use a wider variety of search terms.
• Develop a design challenge to improve offers list. The list of vendors is difficult to scan
for eligibility criteria. If the eligibility requirements remain complex, consider exploring
alternative designs. See Appendix F.2 (page 42-43). Continue to conduct usability testing
to ensure it is easily understood and scannable.
The complete IRS.gov Free File assessment is found in Appendix F.2.
5.2 Taxpayer Experience Usability Findings
In the Taxpayer Experience Testing Report (Appendix G), MITRE reported the findings from the
taxpayer usability testing study. In July 2019, MITRE conducted 29 in-person testing sessions
with taxpayers identified in CDW as having used Free File to file their federal returns in TY
2018. Taxpayers were randomly selected to receive invitation letters to participate in the study
from a cluster of six Chicago zip codes.
During the usability interviews, taxpayers completed three usability tasks while a MITRE
researcher observed them. The three tasks were: 1) locate where to file a federal tax return for
free via a web browser; 2) navigate the Free File landing page and select a Free File member
offer, including with the software lookup tool; 3) file a tax return using one of three randomly
selected Free File systems. For all activities, participants used fictitious tax scenarios to protect
their privacy. Taxpayers also answered interview questions regarding their previous experience
using Free File and their perceptions of the program.
MITRE identified and reported all usability issues observed by the research team with each of
the three Free File usability tasks and provided suggested solutions for addressing each of these
issues individually. Drawing on the data collected from the usability tasks, analysis of qualitative
data gathered during debriefing and with semi-structured interview questions, the satisfaction
and user experience questionnaires, and metrics on taxpayer behaviors and selections during the
tasks, MITRE identified the high-level findings discussed in the following sections.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
71
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
5.2.1 General Free File Program Findings and Usability Issues
Taxpayers lack understanding of what the Free File program is and its relationship to the
IRS.
Despite the fact that MITRE specifically recruited taxpayers recorded in CDW as having
previously used Free File, many participants conflated the Free File program with other forms of
filing taxes for free, including free commercial software, and were unsure whether they had used
Free File or a free commercial version in the past. Some reported not having heard of Free File
until the past year. Taxpayers also seemed unclear on the relationship between the IRS and
Free File. A few never sought out IRS.gov and did not seem to realize that they should use IRS
resources to find the program, whereas others expressed surprise that Free File offers were all
provided by third-party vendors and not directly from the IRS. Evidence for these findings
include:
Usability Issues
• Seven taxpayers expressed surprise that Free File vendors are commercial companies, not
directly offered through the IRS
• Taxpayers mistook finding an e-file provider for finding a Free File software vendor
Qualitative/Debriefing Support
• Four taxpayers conflated free versions of FFA member software offerings with Free File
during debriefing interviews
• Five taxpayers reported not having heard about Free File before the prior year
• Three taxpayers suggested the IRS advertise Free File more
Eligibility Screening
• Of 36 taxpayers deemed eligible to participate in the study, 28 percent expressed
uncertainty as to whether they used Free File or a free version of commercial software
Taxpayer Pathways/Metrics
• Four taxpayers ended their Free File search on a commercial vendor website and never
sought IRS.gov, suggesting that they did not realize that Free File offers should be
accessed via IRS.gov. MITRE’s study design (e.g., recruitment letters and interview
procedures) may have raised awareness among participants about Free File and the IRS’s
involvement. The majority of search keywords used to find Free File in the interviews
involved some combination of “IRS” and “free.” Keywords with this combination tended
to lead taxpayers successfully to the IRS Free File landing page
5.2.2 Locating the Free File Program Online
Taxpayers struggled to find the Free File landing page, including through IRS.gov.
Most taxpayers approached finding Free File by starting with a search engine query, using
keywords such as “IRS,” “free tax filing,” and “free file.” This suggests that the IRS needs to
plan for taxpayers to enter the Free File system from search query results, rather than from the
IRS home page. Some ended up on commercial vendor websites; most were able to eventually
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
72
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
find the Free File landing page on their own, though through various routes that caused
confusion. Some taxpayers meandered through pages of IRS.gov looking for Free File and
expressed frustration that they could not find it.
Usability Issues
• Three taxpayers navigated to various IRS pages while attempting to find Free File. They
thought that Free File was hard to find, were not sure how they arrived at Free File once
they did, or never found it on their own at all.
• One taxpayer went to multiple pages on the IRS website including the free fillable forms,
find a tax professional, and the forms and instructions pages. This taxpayer did not find
the Free File page on their own.
• One taxpayer made it to the IRS Free File landing page but then had trouble proceeding to
the list of Free File software offers. The taxpayer veered away from the IRS Free File
landing page. They navigated from the IRS Free File landing page to the filing for
individuals page.
• One taxpayer navigated to the authorized e-file provider page. This taxpayer initially
found the Free File landing page but then navigated to the page with authorized e-file
providers.
Taxpayer Pathways/Metrics
• Twenty-four participants were able to find the IRS Free File landing page
• Of the 24 participants that ultimately found the IRS Free File landing page independently,
10 followed suboptimal paths, meaning that they were frustrated or confused along the
way and visited other, unnecessary pages in the course of their search
• Twenty-six taxpayers started their search for Free File with an internet search engine
query (e.g., free tax filing)
o The search terms that most frequently led to the IRS Free File landing page included
both “IRS” and “Free File”
5.2.3 Selecting a Free File Software Offer, Including with the Software Lookup Tool
Taxpayers experienced difficulties and confusion with selecting a software offering,
particularly when evaluating Earned Income Tax Credit (EITC) eligibility.
EITC eligibility requirements were confusing for taxpayers and a substantial proportion (about
one-third) committed errors when selecting a vendor using the lookup tool due to erroneously
answering the EITC eligibility question. Information available to taxpayers on IRS.gov regarding
EITC eligibility largely did not resolve the problem.
The lookup tool has room for improvements in usability. Common issues included taxpayers not
noticing the lookup tool, not understanding its purpose, and committing errors on EITC
eligibility.
Errors on the lookup tool result in vendors returned that taxpayers may not be eligible for;
taxpayers might only discover this down the road when they are midway through completing a
tax return. Incorrectly selecting yes for EITC eligibility in particular causes more results returned
from the lookup tool than are accurate. Evidence for these finding include:
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
73
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
Usability Issues
• Eight taxpayers selected “yes” on the EITC question in the lookup tool despite being
unsure of answer.
• Five taxpayers believed that the lookup tool would verify their responses, especially on
EITC eligibility.
• Six taxpayers misunderstood the information in the EITC modal popup.
• Two taxpayers were confused after using EITC assistant tool.
• Three taxpayers had preconceived notions regarding eligibility criteria and committed
errors as a result.
• Two did not know how to answer the EITC question and wanted to leave it blank or leave
the lookup tool.
Taxpayer Pathways/Metrics
• Twenty-nine percent of taxpayers committed an error on the software lookup tool; 100
percent of these errors were due to incorrect response on the EITC eligibility question
Taxpayers were overwhelmed by the Free File software options and by the information
provided along with options on IRS.gov Free File-related pages (e.g., Free File landing page
and Free File software offers page).
They were overwhelmed by the amount of information they saw when they reached the Free File
software offers page, including the paragraphs above the Free File software lookup tool box and
the Free File software offers list below the lookup tool box. Taxpayers also found that the lookup
tool returned too many options. Taxpayers found it difficult to select an offer due to the number
of choices and amount of information presented to them. Evidence for these findings include:
Usability Issues
•
Five taxpayers were overwhelmed by the amount of information on the Free File
software offers page, particularly the Free File software offers.
•
Three taxpayers perceived too many options returned from the lookup tool and had
trouble choosing an offer.
•
Five taxpayers did not understand if they needed to meet all or only some of the
eligibility criteria on the Free File software offers list page in order to qualify. They did
not understand if the criteria were “and” or “or” statements. This confusion resulted in
taxpayers not knowing which Free File software offers they were eligible for. Some
understood Free File software offers as being ONLY for military or EITC.
Qualitative/Debriefing Support
• Five participants felt the biggest difficulty was that there were too many options to choose
from on the IRS website
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
74
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
o “There are too many choices. If there are ten there and four qualify, then it’s like eenie
meenie miney mo.”
The majority of taxpayers reported reasons other than eligibility criteria for selecting Free
File software offers, in part because the eligibility criteria were long and complex.
Taxpayers rely on familiarity with company names, particularly the large software providers, and
previous experience with tax software. When in doubt, taxpayers drew on heuristic devices to
help themselves establish the legitimacy of software providers, such as keywords in software
providers’ names. In part, these strategies were necessitated by the sense that there were too
many options and it was difficult to distinguish between them. Some taxpayers mistook
commercial providers with similar keywords as legitimate IRS Free File program selections.
Evidence for these findings include:
Usability Issues
•
Taxpayers were overwhelmed by the number of options, both on the software offers page
and in the lookup tool results.
•
Taxpayers expressed a desire to see the eligibility requirements repeated on the lookup
tool results list, to help them choose from the list of Free File software offers.
Qualitative/Debriefing Support
• The majority of taxpayers cited reasons other than eligibility criteria for choosing a
vendor. Prior experience and name recognition were the top two reasons, with a total of 45
mentions.
• Eligibility criteria were mentioned as the reason for selection 13 times.
Taxpayer Pathways/Metrics
• Nine taxpayers navigated to a commercial website as part of the process while they were
searching for Free File. Four of the nine ultimately stayed on a commercial page, not
realizing that they had not selected a Free File offer.
5.2.4 Completing a Tax Return Using a Specific Free File Offer
General usability issues across multiple software providers included finding where to enter
student loan interest information, selecting a filing status, and navigating through the filing
systems.
Taxpayers were not sure where to enter student loan information and often found it where they
were not expecting. Taxpayers had a hard time deciding between selecting head of household or
single. They did not understand the flow, or sequence of pages and information, of various
aspects of the tax preparation systems. Evidence for these findings include:
Usability Issues
•
Seven taxpayers were not sure where to enter their student loan information from Form
1098-E while filing a return using two out of the three Free File software offers.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
75
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
•
Six taxpayers were uncertain or confused about either selecting head of household or
single as their filing status. Taxpayers expected more information within the system to
help them make the correct choice. Instead, they had to turn to an external source for
more information.
o If they chose head of household, some taxpayers expected to see additional
questions or prompts to confirm their selection but did not receive them.
o System prompts asking taxpayers if they wanted to change their filing status were
confusing and undermined taxpayer confidence
•
Three taxpayers did not understand the path or flow of information presented to them
while filing. They lost track of where they were in the filing process, had trouble
navigating menus, or did not see the pages they were expecting to see.
•
Three taxpayers were confused about what to do once they chose the quick file option.
They were not familiar enough with tax preparation to enter each of the relevant forms
they had for filing.
Overall, taxpayers rated the usability of the Free File system a “B.” The filing task was the
most difficult for participants, which is not surprising given that completing a tax return is
a more complex task than browsing the internet or navigating IRS.gov.
Usability scores varied by Free File software offer. There was some evidence that usability
scores varied by age and education level, though these results should be interpreted with caution
given the small sample size. Evidence for these findings include:
Questionnaire Scores/Analysis
• Participants rated task 3 as harder than tasks 1 and 2; in line with task 3 being inherently
more complex than tasks 1 and 2.
• Participants’ ratings of the Free File system overall varied by which Free File software
offer they used to file a return, suggesting that the software experience colored taxpayers’
perceptions of the entire experience, including IRS site interactions.
• Patterns in usability ratings by education were unclear:
o High school-equivalency group had lowest per-task ease for Tasks 1 and 2
(SEQ) but highest overall usability (UMUX)
• Youngest age tercile (20-35 years) reported the best usability across entire experience, but
there was not a conclusive age group pattern.
5.3 Integrated Look at Taxpayer Experience
MITRE conducted two distinct research activities to evaluate and understand the taxpayer
experience: the taxpayer journey heuristic review of IRS.gov Free File pages and all 12 current
Free File member systems, and usability testing interviews with 29 prior Free File users.
Identifying themes and integration points between these two activities highlights crucial areas for
improvement and recommendations moving forward.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
76
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
Improve functionality and messaging around taxpayer eligibility and ineligibility for Free
File, on both IRS.gov pages and within member systems.
Free File systems are mandated by the MOU to clearly communicate ineligibility for Free File;
additionally, selecting an offer for which they are eligible is one of the core tasks that a taxpayer
must perform in order to access Free File program services. As such, it is of central importance.
Findings from both the heuristic review and taxpayer usability testing show that this is an area
for improvement for IRS.gov and member sites. The heuristic review showed that the majority of
the 12 Free File member systems had severity ratings suggesting potential usability issues around
users not being able to quickly recognize ineligibility for Free File on a particular system. In the
taxpayer testing, MITRE observed that many taxpayers struggled to determine eligibility at the
outset; specifically, participants had trouble answering the EITC eligibility question used in the
software lookup tool and were confused by the many eligibility criteria presented on the software
offers page. Taxpayers largely relied on prior experience with vendors and name recognition to
select an offer. Taken together, there is risk that taxpayers will mistakenly select a vendor for
which they are not eligible, and not realize it until late in the return filing process.
Promote understanding of what the Free File program is throughout IRS.gov by using
consistent language and labels.
The usability testing with taxpayers revealed two issues that affected their ability to effectively
locate and use Free File: they had difficulty navigating the internet and IRS.gov to find Free File,
and they made comments during the interviews that reflected a misunderstanding of what the
Free File program is (e.g., expressing surprise that the IRS doesn’t have their own tax filing
software, or assuming they should go straight to a commercial website rather than accessing Free
File through the IRS). In parallel, the heuristic review identified similar-sounding links and
buttons throughout IRS.gov that taxpayers might try to follow to find Free File if they are not
familiar with the program (e.g., File > How to File).
In the usability study, MITRE observed multiple taxpayers follow various links in their pursuit
of Free File, often getting frustrated. Figure 21 shows the sub-optimal paths that 10 taxpayers
interviewed took to reach the Free File landing page, including accidentally landing on a
commercial site before finding IRS.gov, and reaching the Free File landing page but accidentally
navigating away from it by using various links other than the correct “Start Free File Now”
button. Additionally, MITRE observed that participants most successful in identifying Free File
quickly from a web browser search used a combination of terms that include “IRS” and “free
file” or “file taxes for free,” rather than relying only on “IRS” or “file taxes for free.” Together,
these findings point to the need to consistently brand Free File, use consistent terminology, and
raise awareness of this terminology. These steps should help taxpayers more quickly and easily
home in on the information they need.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
77
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
Figure 21. Observed Taxpayer Paths to Reach Free File.
Improve Free File landing page and Software Offers page.
Both the heuristic review of the Free File-related IRS.gov pages and the taxpayer usability
testing revealed areas for improvement. On the Free File landing page, some taxpayers had
trouble finding the button to start Free File or were attracted to the Fillable Forms section; the
heuristic review pointed out that this “call to action” is difficult to find and requires the user to
scroll down. The heuristic review found that the software offers page in particular was too heavy
on text, some of which was unfamiliar to taxpayers in context, especially “software lookup tool.”
The usability study validated this, as taxpayers commented on the heavy amount of text and
feeling that there were too many options on the software offers page. They also did not see the
software lookup tool or often did not understand what it was for.
Improve error prevention on IRS.gov and in Free File member systems.
The heuristic review determined room for improvement in helping prevent user errors and
helping users correct their errors. Many Free File member systems received scores on the “error
prevention” heuristic that reflect need for improving this capability; for example, some member
sites had warning and error messages that were difficult to tell apart and could leave users
wondering what action to take. The heuristic review of IRS.gov also noted that there is no
method of checks and balances on taxpayers’ self-determined AGI and EITC eligibility; the
taxpayer usability testing validated the importance of this finding, given that many taxpayers in
the usability study expressed the belief that the lookup tool would not let them report erroneous
EITC eligibility (“At some point it will check my work”). Taxpayers in the usability study also
struggled with the error prevention and messaging used for creating account names and
passwords, and some systems prompted users with alerts (e.g., to check their filing status) that
taxpayers interpreted as evidence that they had made an error.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
78
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
Recommendations
Recommendations
Based on this comprehensive review of the program, the interviews with the IRS and FFA,
and the findings from each of the workstreams, MITRE developed recommendations to
improve the integrity and effectiveness of the Free File program. The recommended actions
are categorized into recommended programmatic improvements to Free File, MOU-related
recommendations, and usability recommendations based on direct taxpayer experience. Some
key points for the IRS to consider when looking at the recommendations are:
•
While IRS has an abundance of data that can identify the demographics for those
in the eligible population who don’t use the program, what they don’t have is the
research to tell them why those taxpayers don’t use the program. It may be lack of
awareness, but it is likely that isn’t the only factor. Studies to determine the “why”
should be conducted before any other major decisions are made about the program.
•
Recommendations align across the different workstreams. While teams worked
independently on their approaches and subsequent recommendations for their
particular disciplines, when integrating the findings, many of the recommendations
were aligned and validated from the separate workstreams.
•
MITRE recognizes that any recommendations concerning changes to the MOU
must be negotiated with the FFA. While some of the recommendations may be
rejected outright by the industry, it doesn’t negate our position that they would
strengthen the program for the taxpayers using it.
.
6.1 Improvements to the Free File Program
Program recommendations address four primary issues that emerged from MITRE’s integrated
assessment:
1. Objectives of the Free File program have not been updated since the program’s inception.
2. If Free File objectives are revised to focus on taxpayer awareness, more needs to be
known about the target population and taxpayer behavior.
3. IRS Free File program is a long-standing public-private partnership, but there is a lack of
public awareness about what that means to the program.
4. Rules that were put into the MOU to promote fairness among FFA members and promote
choices for taxpayers translate into unclear presentation of offers and confusion for
taxpayers.
A description of each issue and MITRE’s recommended actions is provided in the sections
below.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
79
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
6.1.1 Define Updated Program Objectives and Create Corresponding Metrics
Objectives of the Free File program have not been updated since the program’s inception.
The IRS and FFA have not revisited the objectives of the program since its inception. As such,
the objectives (reflected in the MOU) are outdated and don’t reflect leadership’s current view of
program success.
The program objective of providing a venue for free tax filing for 70 percent of the population
has been met. The e-filing objective has been met. No other metrics currently exist by which the
program office can measure its performance.
Recommendation:
MITRE recommends that IRS leadership, in collaboration with the FFA, define new program
objectives and create metrics to determine the Program Office’s performance against those
objectives. Recognizing that the IRS and FFA will need to negotiate any formal changes to joint
program objectives, MITRE recommends that IRS develop internal objectives and metrics that
reflect its own priorities outside the MOU. While agreement between the IRS and FFA would be
the ideal, translating priorities into objectives and metrics should not be contingent on that
agreement.
For example, according to IRS leadership, its focus for Free File is less about a target number of
participants, and more about maximizing awareness to ensure people recognize it as an option.
Potential metrics could include:
• Number of companies participating, potentially targeting a specific market share
• Actual number of Free File taxpayer candidate pool/participants (e.g., xx% of the DIY
community minus RALs/RACs)
• Metrics associated with compliance (e.g., offering parameters.)
• Awareness numbers (see additional recommendations)
• Web analysis metrics (e.g., number of clicks on the Free File link on the IRS.gov home
page, percentage of select keyword searches that bring taxpayers to the IRS.gov landing
page, etc.)
6.1.2 Take Steps to Better Understand the Free File Target Taxpayer Population and
Behavior
If Free File objectives are revised to focus on taxpayer awareness, more needs to be known
about the target population and taxpayer behavior.
The IRS has collected demographic data on users of Free File and migration data of those users,
but they do not have demographics data of the potential pool of Free File users. As shown in
Section 3.3, this data would be helpful in targeting outreach.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
80
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
The total population pool of 103 million taxpayers who are eligible may not represent an
accurate picture of the pool of taxpayers who are candidates for Free File. Taxpayers make
choices that take them out of the candidate pool. About 9 percent of taxpayers who have not
taken themselves out of the candidate pool by (for example) using a paid preparer or choosing a
RAC or RAL, use Free File (not taking into consideration those who choose free commercial
offerings). There is no consensus on what would be an acceptable number.
In addition, while each individual member receives customer feedback in various ways for their
own Free File product, the IRS has not conducted a customer survey since 2009.
The IRS has no access to data that would indicate whether a DIY filer paid for a return or
received a free commercial offering. As the issue is whether a low-income taxpayer has to pay
for filing, knowing how many taxpayers receive free filings either through Free File or a
commercial product would give the IRS a better understanding of their target markets.
Leadership acknowledged that the number of returns submitted through Free File have not
increased in the same way that e-file numbers have and articulated a goal of increased awareness.
The last awareness study was conducted in 2011.
Paid advertising yields results. In MITRE’s web analysis of those searching for free tax filing, an
estimated 85 percent of the traffic followed a paid search result to member sites, while the other
15 percent of visits to member sites came from an organic search result. Searchers were more
than five times likely to click on a paid search result.
Recommendations:
To better understand the Free File target taxpayer population and behavior, MITRE recommends
the IRS:
•
Conduct data analysis of the demographics of the population who are prime candidates
for Free File but not using it.
•
Conduct a taxpayer behavior study to better understand the factors involved in a
taxpayer’s choice of filing methods. Understanding taxpayer motivation and choices
could help determine what would be an acceptable Free File number, if a number is
needed at all. This aligns with the IRS-related priorities of the internal MITRE research
program and will be proposed for a future research project.
•
Conduct a customer survey of Free File users specifically to the IRS experience of Free
File (not the software they filed their return through) to use as a baseline and create an
online survey for users to capture continuous feedback.
•
Propose, in the spirit of transparency, that the industry use an indicator on free
commercial filings or request that FFA provide them the aggregate number of taxpayers
who file their taxes using free commercial products each year. When developing metrics,
this could help with understanding a more accurate picture of the population of the Free
File candidate pool.
•
Commission an awareness study to baseline taxpayer awareness of Free File, specifically
of targeted demographics determined in the prior recommendation. Based on the results
of this study—and other recommended research—conduct a targeted awareness
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
81
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
campaign to the desired Free File demographics, and include other IRS-sponsored venues
such as low-income tax clinics (LITCs) and VITA sites, including IRS’ own website.
•
Determine whether there is a cost benefit to using paid advertising to reach taxpayers
searching for free tax filing. We recognize the IRS may have to seek statutory authority
and budget-specific funding for such advertising if it was deemed beneficial.
This is consistent with findings from taxpayer experience workstream that found taxpayers lack
understanding of what the Free File program is and its relationship to the IRS. See
recommendation in Section 6.3 to conduct research on Free File awareness, attending to
different demographic groups’ awareness.
6.1.3 Increase Awareness of the Free File as a Long-Standing PPP
IRS Free File Program is a long-standing public-private partnership, but there is a lack of
public awareness about what that means to the program.
Free File program is a partnership between the IRS and industry that requires cooperation to
achieve success. This partnership requires a balance that serves the interests of the government
and taxpayers, but also creates a value to the for-profit industry that provides the service.
The IRS Free File program could benefit from providing clarity and scope around the PPP, both
internally and externally. The purpose and scope of PPPs are not common, and awareness is key
to success.
Recommendation:
MITRE recommends that a health assessment of the PPP become part of the overall metrics for
the Free File program, using a generally accepted framework provided by external sources, such
as that used by MITRE in this assessment.
MITRE further recommends that the IRS Free File Program Office incorporate talking points
addressing the unique nature of the PPP in terms of joint governance and the value a PPP
provides to all partners as central themes when addressing critics or publicly discussing the
program to help address this lack of awareness.
This is consistent with findings from taxpayer experience workstream that found taxpayers
seemed unclear on the relationship between the IRS and Free File offers. See recommendation in
Section 6.3 to clarify the IRS’s relationship to the Free File program and the software offers.
6.1.4 Simplify Member Offerings to Reduce Taxpayer Confusion
Rules that were put into the MOU to promote fairness among FFA members and promote
choices for taxpayers translate into unclear presentation of offers and confusion for
taxpayers.
The FFA is obligated to provide Free File to eligible taxpayers through individual commercial
sites such that, when taken in aggregate, these services are offered to the lowest 70 percent of the
taxpayer population, calculated using AGI. Moreover, the MOU requires that each company
must offer services to at least 10 percent of total eligible taxpayers but to no more than 50
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
82
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
percent of taxpayers (10/50 rule)—a requirement set to level the playing field for smaller
providers.
While the 10/50 rule has been (and may continue to be) beneficial to FFA members to ensure
equity, it is possible that it has outlived its usefulness. MITRE did not have adequate time, nor
was it within scope, to do an analysis of how the 10/50 rule impacts Free Filed returns. However,
from a taxpayer perspective, the different company offerings create confusion as a taxpayer
navigates the various choices.
Recommendations:
MITRE recommends the IRS conduct an analysis/study of the 10/50 rule. This could include a
confidential market study of the members to help understand the economic incentive to assist in
preserving competition amongst the members and promoting taxpayer economic welfare; a small
pilot to determine the business impact of removing the rule; a consideration of whether differing
rules for the two large companies would benefit or create inequity; and/or an in-depth data
analysis of how the different offerings have impacted members’ free file numbers.
Even if the rule is not changed, MITRE recommends simplifying the display of members’
offering to reduce taxpayer confusion.
This is consistent with findings from taxpayer experience workstream that found taxpayers were
overwhelmed by the Free File software options and by the information provided along with the
options on IRS.gov Free File-related pages. See recommendation in Section 6.3 to take steps to
increase ease of understanding eligibility requirements.
6.2 Recommendations Related to the MOU
MOU-specific recommendations include suggested changes directly to the content of the MOU
document. Recommendations address the need for a comprehensive review and refresh of the
current agreement; inclusion of search exclusion techniques in future MOU negotiations; and the
results of a legal review of the document.
Discussion of each issue and MITRE’s recommended actions are included in the sections below.
6.2.1 Update MOU to Reflect Current Environment
The MOU is an evolving document that is added to with each new negotiation, resulting in
a piecemeal agreement that is out of date in some areas.
The MOU generally has provisions added with each revision, but few are taken out. Much of the
same language is used, not only in the objectives, but also in the technology portion, that has
become outdated (e.g., text CAPTCHA).
All but two of the members’ Free File offerings are the same as their commercial offerings. The
security/privacy rules of Free File are the same as e-File, and all Free File members are also
members of the Security Summit where the IRS works collaboratively with the industry to
improve security.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
83
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
MITRE’s review of the individual members’ processes for applying the Free File indicator
(without looking at the actual coding) indicated most members used the same process that should
place the indicator only on Free Filed returns. However, that is not always the case.
In MITRE’s data analysis of returns marked with the Free File indicator, several thousand
returns from one vendor indicated the presence of a RAC. The vendor noted that the IRS
informed them of this irregularity, and they discovered a coding error had mistakenly placed the
indicator on returns that had not been Free Filed, which was subsequently corrected. Another
vendor also had several hundred returns with the Free File indicator on returns with AGI of more
than $66,000.
Recommendations:
MITRE recommends that the IRS and the Alliance jointly:
•
Determine new, measurable objectives for the Free File program, and draft a new MOU,
pulling and using provisions from the current MOU that are relevant, discarding
provisions that are not, and creating new provisions based on the current environment.
•
Simplify the MOU in its security and privacy provisions by simply requiring members to
adhere to the e-File/Security Summit requirements and recommendations.
•
Add language to the MOU to specify conditions under which the indicator is applied to
ensure quality control.
Though not necessarily dependent on the MOU, MITRE also recommends the IRS develop a
quality control method that would allow the IRS to validate returns with the Free File indicator
were actually filed through the Free File program.
6.2.2 Determine the IRS Position on Search Exclusion and Provide Guidance
Search exclusion techniques are the latest new issue for consideration when entering into
the next round of MOU negotiations.
Five of the companies used a NOINDEX code to exclude their company’s Free File landing page
from organic searches. They reported that their belief was that this kept them compliant with the
MOU. However, while the MOU states the IRS will provide links to their site, it does not
prohibit taxpayers from accessing their sites directly. IRS press releases state that taxpayers can
ONLY access Free File sites through IRS.gov.
Recommendations:
MITRE recommends that if the IRS wants taxpayers to go to Free File sites only through
IRS.gov, the MOU should reflect that specifically as a provision, as opposed to the current
reference in the preamble.
MITRE further recommends the IRS determine their position on the use of NOINDEX and add a
provision to the MOU addressing that use.
This is related to findings from taxpayer experience workstream that found taxpayers struggled
to find the Free File landing page, including through IRS.gov. See recommendation in Section
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
84
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
6.3 to improve taxpayers’ ability to find the Free File landing page both from IRS.gov web pages
and directly from internet search results.
6.2.3 Legal Analysis of MOU
The following recommendations were made by MITRE’s general counsel based on a legal
review of the MOU in collaboration with outside counsel, and after consultation with IRS
counsel. A redline version of the current MOU is provided below.
MOU Provision
Article 1
Article 2
Section 4.1.1 (ii)
Section 4.1.3 (ii)
Section 4.2.4
Section 4.2.5
Section 4.3.1
Section 4.3.2
Section 4.3.3
Section 4.4
Section 4.5.3
Section 4.6.2
Section 4.6.6
Section 4.6.7
Section 4.7.1
Section 4.11
Section 4.12
Recommendation
Alphabetize all definitions; add new definitions for certain terms.
Add language to restate MOU compliance obligations of FFI members and revise
language to state that the federal government “declines” entering a marketplace, rather
than making a binding pledge in perpetuity.
Change qualifying criterion for listing from definitive number of online returns to a
minimum number of returns.
Make clear that non-discrimination applies only within the coverage (because the
program, by definition, discriminates among taxpayers based upon income level). Also,
include a covenant by members not to reject arbitrarily any taxpayer in the coverage.
Provide that it is the responsibility of each member to assure that certain service
providers adhere to established industry practices.
Include a provision obligating members to not utilize or engage in any deceptive or
unfair trade practices.
Insert a FFI self-regulation provision as a means of promoting truthfulness and accuracy
in advertising the services.
Provide an affirmative obligation for FFI and/or members to report to the IRS when
truthfulness and accuracy in advertising the services is not being met
Include a provision allowing the IRS to make referrals to the Federal Trade Commission if
a good faith belief is held that a member might be engaging or has engaged in unfair and
deceptive business practices.
Include a requirement in functionality that a member’s software programs provide both
user access and guidance to taxpayers in addition to other functions.
Require that members disclose prominently on their Free File landing page any
limitations in the forms and schedules that may be needed by a taxpayer to support the
member’s free offerings.
Clarify the date by which certain information regarding third-party security and privacy
certifications must be provided to the IRS and the executive director.
Require that members must not only implement, but also maintain in effect at all times,
a CAPTCHA program.
Require that access to a member’s program must be available to visually impaired
taxpayers without the use of special equipment.
Require shutdown of a member’s Free File website in the event of an unauthorized
disclosure of information only if determined necessary in the sole discretion of the
executive director or IRS in order to prevent any further unauthorized disclosure.
Clarify that a member will be both liable for and timely pay any IRS penalties and/or
interest resulting from an error in the member’s software program calculations.
Shift responsibility to members to assure that where servers or transmitters are located
outside the U.S., taxpayers using the services of the member agree and sign a form
consenting to the disclosure.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
85
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
Section 4.14
Section 4.19.2
Section 4.24
Section 4.29.2
Section 6.1.6
Section 6.1.7
Article 8
Require that a member remind a taxpayer upon his or her return to the member’s Free
File website of the terms for eligibility for current use of the member’s services.
Clarify sequencing (“waterfall”) of actions to be offered to a taxpayer following an
ineligibility notification.
Reserve to the IRS the right to delist a successor to a member if the successor is not in
compliance with the MOU, or to require the successor to reapply.
Add a post-filing season review for compliance requiring members to provide detailed
data regarding the use of the IRS Free File program by taxpayers on their website,
including paid services provided to such taxpayers.
Allow the IRS to delist a member upon the occurrence of any event that the IRS or the
executive director determines in its discretion either jeopardizes the integrity of the IRS
Free File program or the IRS Free File website.
Allow the IRS to delist a member upon the FTC acceptance of an IRS referral along with
the opening of an investigation by the FTC of a member’s alleged unfair trade or
deceptive practices.
Include a covenant that members comply with applicable antitrust laws with failure to
comply constituting grounds for delisting.
Throughout the MOU there are references to “clear links.” There should be a standard accepted
format for the links. For example, the reference in the MOU to links as “clear” might mean bold,
a different color when hovered over, a specific font style and height, etc. Currently, the phrase
“clear links” in the MOU may lead to different interpretations and varied judgements as to what
is clear to a specific individual. Recommend that clear links be defined.
If this does not currently exist, IRS leadership may want to consider a formal MOU relationship
with the FTC to provide mutual assistance in evaluating whether certain FFI member business
practices might be unfair and/or deceptive as it relates to the Free File program.
REDLINE EIGHTH
MEMORANDUM OF UNDERSTANDING ON SERVICE STANDARDS AND DISPUTES - MITRE 7-16-2019.docx
6.3 Taxpayer Experience-Specific Recommendations
The following recommendations are the result of the taxpayer experience workstreams. These
recommendations are strictly from the perspective of the taxpayer as users of the Free File
program. MITRE recognizes that some of these recommendations may be difficult or even
impractical to implement; however, it is important for the IRS to understand how they could best
serve the taxpayers, even if budget or other considerations might prevent them from doing so.
Because these originated from the Taxpayer Usability Report they both complement and may be
repetitive of the program-level recommendations. The fact that some of the same
recommendations came forth from different workstreams underscores their validity.
MITRE provides the following usability-centric, issue-level recommendations:
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
86
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
• Standardize how Free File is referenced on IRS platforms and communications.
Currently, different web pages at IRS.gov have different labels and titles for Free File,
such as buttons that say, “File Your Taxes for Free” versus “Free File.” Standardize all
references to one label or title and use that label/title in all communications about the
program. Encourage Free File members to follow suit. This should also help taxpayers
access Free File from web searches, as it may influence their search terms.
o Clarify the IRS’s relationship to the Free File program and the software offers
o Use visual cues like third-party logos on the IRS site to signal that taxpayers will go to
the software provider website
o Improve taxpayers’ ability to find the Free File landing page both from IRS.gov web
pages (e.g., the home page) and directly from internet search results
• Take steps to increase ease of understanding eligibility requirements. The current
volume and complexity of eligibility requirements may lead to taxpayer mistakes in
selecting a Free File software offer. Clearer, simplified eligibility requirements will help
taxpayers focus on the important information to choose a Free File software offer.
o Reduce the amount of information presented in individual boxes on the software offers
page
o Consider presenting eligibility criteria on the software offers page in a new format,
such as in a grid view where all criteria are presented consistently or via tooltips
o Include information on eligibility criteria in the results returned by the software lookup
tool
• Ensure that taxpayers understand that determining eligibility for a software offer is
ultimately their responsibility. Taxpayers expressed the belief that their eligibility would
be “checked” at some point along the way, for example, by the software lookup tool.
MITRE recommends that the IRS incorporate language on the Free File pages and lookup
tool to help taxpayers understand that misunderstanding or misreporting eligibility criteria
can lead to their selecting a Free File software offer for which they are not eligible.
• Eliminate or change the EITC eligibility question in the lookup tool and on the
software offers page. MITRE recommends that:
o If the IRS stated on the Free File site that all taxpayers eligible for EITC are eligible
for Free File, the question could be removed from the tool. MITRE recognizes this
would be a topic for negotiation between the IRS and FFA because of how that could
impact some companies’ appearance in the tool.
o If the IRS chooses to leave EITC in the lookup, MITRE recommends linking the
question to the EITC eligibility tool, but ONLY after conducting usability testing on
the EITC eligibility tool and addressing the results.
• Recommend suggested solutions be shared with FFA to increase usability of
members’ tax preparation software systems. MITRE recommends that the IRS share
the FFA member website suggested solutions within this report and findings identified in
the taxpayer journey study with the FFA. This can help member programs increase the
likelihood of taxpayers having a positive experience with Free File by improving look and
feel, communication of eligibility on member home pages, and how to best walk taxpayers
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
87
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
through their online tax return as they fill it out. Recognize that the experience of using a
particular member offer will affect how taxpayers perceive the Free File experience as a
whole.
• Conduct research on the usability of taxpayer access and use of the Free File system.
MITRE recommends that the IRS and Free File members conduct iterative testing to
evaluate system design and usability. MITRE recommends that this usability testing
include representation of different age groups, income groups, and education groups, to
ensure that the Free File system is maximally usable for all eligible taxpayers.
• Conduct research on Free File awareness, attending to different demographic
groups’ awareness. The present study was not designed to assess awareness of Free File
among the taxpayer population. MITRE recommends that the IRS conduct future research
to better understand whether taxpayers are aware of Free File, their understandings of
what the program is, and their reasons for using or not using Free File. MITRE
recommends including both previous Free Filers and eligible taxpayers who have not
previously used Free File in this research.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
88
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
Appendix A The Economics of IRS Free File
[Provided in supplemental file]
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
A-1
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
Appendix B Crosswalk of External Program Reviews
The embedded document provides a crosswalk of 2018 Free File program recommendations
from the IRS Advisory Committee (IRSAC) and Taxpayer Advocate Service (TAS) with MITRE
findings and recommendations.
[Provided in supplemental file]
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
B-1
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
Appendix C ProPublica Articles on IRS Free File Program
Huseman, J (2017, March 20). Filing Taxes Could Be Free and Simple. But H&R Block and
Intuit Are Still Lobbying Against It. Retrieved from https://www.propublica.org/article/filingtaxes-could-be-free-simple-hr-block-intuit-lobbying-against-it
Root, T (2018, June 18). Why Are Millions Paying Online Tax Preparation Fees When They
Don’t Need To? Retrieved from https://www.propublica.org/article/free-file-online-taxpreparation-fees-intuit-turbotax-h-r-block
Elliott, J (2019, April, 9). Congress Is About to Ban the Government From Offering Free Online
Tax Filing. Thank TurboTax. Retrieved from https://www.propublica.org/article/congress-isabout-to-ban-the-government-from-offering-free-online-tax-filing-thank-turbotax
Elliott, J (2019, April 10). Bill to Limit IRS’ Ability to Offer Free Tax Filing Service Is Getting
New Scrutiny. Retrieved from https://www.propublica.org/article/bill-to-limit-irs-ability-to-offerfree-tax-filing-service-is-getting-new-scrutiny
Elliott J. and Waldron L. (2019, April 22). Here’s How TurboTax Just Tricked You Into Paying
to File Your Taxes. Retrieved from https://www.propublica.org/article/turbotax-just-tricked-youinto-paying-to-file-your-taxes
Elliott, J. (2019, April 23). Updated: If You Paid TurboTax but Make Under $34,000, You Could
Get a Refund. Here’s How. Retrieved from https://www.propublica.org/article/how-to-getturbotax-refund
Elliott, J. (2019, April 26). TurboTax Deliberately Hid Its Free File Page From Search Engines.
Retrieved from https://www.propublica.org/article/turbotax-deliberately-hides-its-free-file-pagefrom-search-engines
Elliott, J. (2019, May 3). Elizabeth Warren and Other Senators Call for Refunds and
Investigations of TurboTax and H&R Block. Retrieved from
https://www.propublica.org/article/elizabeth-warren-and-other-senators-call-for-refunds-andinvestigations-of-turbotax-and-h-r-block
Elliott, J. (2019, May 6). Senior IRS Leaders Launch Review of Agency’s Partnership With
TurboTax and H&R Block. Retrieved from https://www.propublica.org/article/senior-irs-leaderslaunch-review-of-partnership-with-turbotax-and-h-r-block
Elliott, J. and Marco M. (2019, May 9). Listen to TurboTax Lie to Get Out of Refunding
Overcharged Customers. Retrieved from https://www.propublica.org/article/listen-to-turbotaxlie-to-get-out-of-refunding-overcharged-customers
Elliott, J. (2019, May 10). New York Regulator Launches Investigation Into TurboTax Maker
Intuit and H&R Block. Retrieved from https://www.propublica.org/article/turbotax-maker-intuith-r-block-new-york-regulator-launches-investigation
Elliott, J. and Tsutsumi, K. (2019, May 23). TurboTax Uses A “Military Discount” to Trick
Troops Into Paying to File Their Taxes. Retrieved from
https://www.propublica.org/article/turbotax-military-discount-trick-troops-paying-to-file-taxes
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
C-1
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
Elliott, J. (2019, June 5). Congress Scraps Provision to Restrict IRS From Competing With
TurboTax. Retrieved from https://www.propublica.org/article/congress-scraps-provision-torestrict-irs-from-competing-with-turbotax
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
C-2
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
Appendix D Initial Assessment: Overall Search Engine Results
As of July 2019, six Free File landing pages were not discoverable in Google and/or Bing. There
are cases where a member used more than one of the techniques described below.
The following are some of the most common techniques used to discourage discovery on organic
internet searches, descriptions of those techniques, and results from MITRE’s source code
analysis of members’ landing pages. Methods exist to block a URL from indexation that would
be impossible for a third-party to observe and are therefore not included in this assessment.
Additional information about how major web search engines work is provided in the Appendix
E.
• Robots.txt “Disallow” Statements [R]: a text file, usually located in the root folder level,
informing search bots which files and file folders can or cannot be crawled. For example,
if a member site wanted to prevent their Free File landing page from appearing in web
search engines, they could add the following instruction in their robots.txt file to prevent
bots from crawling the page: Disallow: /ffi-landing-page
Only one member explicitly used this technique to disallow search bots from crawling their Free
File landing page.
• Meta Robots NOINDEX [M]: a directive placed in the HEAD section of a web page
providing instructions to bots at the page level not to include that page in the search index.
For example, if a member site wanted to prevent their Free File landing page from
appearing in web search engines, they would add the following instruction in the HEAD
section of the landing page’s source code:
Five sites explicitly NOINDEX’ed their Free File landing pages. Some members have since
changed this and allow search engines to index.
• Rel “Canonical” tags [C]: a method of communicating which page among a set of
duplicate pages is preferred for indexation. A page canonicalized to another page will be
omitted from the search index. For example, if a member site has two very similar free tax
filing product pages—one complies with the MOU and another commercial free tax filing
product that does not comply, the member could add code in the compliant landing page
instructing search bots to view that page as a duplicate of the commercial page. This
would cause the search engines to exclude the MOU compliant landing page from the
searchable index.
Only one member used this technique and MITRE could not determine their intent. If the intent
were to purposely divert users, the target URL would have led users to a similar but commercial
product page. However, the member’s target URL was the home page. This may have
contributed to the desired consequence of keeping the FFI page out of the search index, but the
determination of intent remains inconclusive.
Two members did not use any of the aforementioned search avoidance techniques, however,
their Free File landing pages were not discoverable on Google or Bing. The methods used to
block URLs from the major web search engines—especially robots.txt and meta robots
NOINDEX—as shown in the report are the most commonly used techniques to prevent content
from showing up in the search indexes. However, there are other methods to prevent indexation
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
D-1
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
that are not specifically designed to block content but indirectly achieve that purpose. For
example:
• Using the rel canonical tag to inform the search engines that the page in question should
be considered a duplicate of another page is intended to solve a different problem, but it
too will prevent the page tagged as a duplicate from being indexed.
• Building a page entirely with JavaScript—or previously with Flash—would also have the
effect of making it very difficult, if not impossible, for the search bots to crawl the content
for inclusion in the search index.
• Both Google and Bing provide a mechanism through their respective webmaster tool
applications to temporarily block a URL from indexation for periods of 90 days at a time.
If website owners avail themselves of that feature, it would be impossible for someone
outside of that organization to know this was done—unless the information were shared.
MITRE cannot know with any degree of certainty whether these two members used this feature
to block the site on Bing or even on Google at some point in time. The only thing that can be said
with certainty is that the Free File landing page is not currently indexed in Bing, but the typical
methods used to prevent indexation were not used in this case.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
D-2
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
Appendix E How the Major Web Search Engines Work
Major search engines such as Google and Bing are comprised of a variety of different types of
search results (including image search, video search, product search, local business search, and a
few others), but the two primary types of search results are known in the industry as organic
search (the most important and by far the largest set of search results) and paid search. These two
types of search results conform to two different sets of rules that govern a web page’s inclusion
and ranking.
Paid Search vs Organic Search
Paid search70 results are advertisements for which the web publisher participates in a type of
auction that rewards high bid prices along with high relevance to the user’s search query. The
relevance factor ensures that the advertisements are also useful to the searcher. In this set of
results, publishers can exert greater control over their appearance in the results because the main
governing factor is a willingness to pay for placement. Assuming the advertisements being
purchased are relevant to the search queries associated with those advertisements, they need only
out-bid their competitors for the most prominent positions at the top of the search results page.
An important benefit of the paid search results section is that it appears at the very top of the list
of search results. It is common for the first four paid results to be displayed before the list of
organic search results begins. Any remaining paid results for a query will continue at the bottom
of the first 10 organic search results and will then continue at the top of next page of results.
Organic search results are considered free to website publishers in that they do not, and cannot,
pay to appear in the search results. Their content’s inclusion and ranking are governed by
complex algorithms that evaluate many factors71 that fall under a few high-level categories
including keyword relevance to the user’s search query; quality of user experience; and a concept
known as “authority” or “trust.” Websites with high authority or trust scores will out-rank sites
with lower authority when other factors such as relevance and user experience are comparable.
This helps ensure that content from the most credible sources of information are given
prominence in a search ahead of less trustworthy websites. The ability to measure trust in web
searches was first developed by Google in 1996 with their patented PageRank algorithm72 and
was a significant innovation in the web search industry.
The major search engines emphatically state that success in one of the two main forms of search
—paid and organic—does not in any way influence success in the other. For example, a
publisher that pays a high price for inclusion in the paid search results area cannot expect their
content to rank highly in the organic search results section because of those payments. This
“church and state” separation of paid and organic search is often described as being a firewall
that separates the two to provide a high degree of confidence to website publishers that the
organic results are driven by what is best for the searcher, not what is in the economic interest of
the search engine.
70
Search Enginer Watch, “A beginner’s guide to paid search,” 1 June 2018. [Online]. Available:
https://searchenginewatch.com/2018/06/01/a-beginners-guide-to-paid-search/
71
Google, “How Search algorithms work.” [Online]. Available: https://www.google.com/search/howsearchworks/algorithms/
72 Wikipedia, “Page Rank,” 10 September 2019. [Online]. Available: https://en.wikipedia.org/wiki/PageRank
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
E-1
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
A More Detailed Description of Organic Search
For any given search query, there could be hundreds, thousands or even millions of matching
web pages in the organic section of the search results. Yet searchers will rarely go past the
second page of search results and about 80 percent of the time, they do not go past the first
page.73
Given this demand for the highest quality results on the first page, search engines must
constantly work to provide only the highest quality, most relevant, most trusted and unduplicated search results within the first 10 or 20 results.
Well before a searcher performs begins, the search engine has been busy crawling the web
looking for new information to include in its searchable index and carefully inspecting many
details about this content that will help influence which queries this content will be considered a
relevant match and how well it will rank against other web pages that are similarly relevant.
• Crawling74 is a discovery process whereby the search engine is looking for available
content on the internet to make available to the searcher. However, discovery of content
does not guarantee inclusion in the searchable index.
• Indexing75 is the process of including web pages and making them searchable by users of
the search engine. It is a content curation process in which the search engine uses an
algorithmic approach to determine what should be made available to searchers and what
should be hidden or excluded.
Consider a local public library. This library is similar to a search index in that the available
books have gone through a curation process. While the library may contain many books, it is a
small fraction of the total number of books ever written.
Google has stated they are aware of trillions76 of web pages on the internet. Yet, only a fraction
of this number (tens or hundreds of billions of pages77) is discoverable in their search engine. The
rest of those discovered pages were intentionally excluded. Later, some of the reasons that
govern why a piece of content is excluded from the searchable index will be discussed.
The word “index” here is also analogous to the index you would find at the back of the book.
Like a book’s index, the information being sought is managed through the creation of a list of all
words found on the crawled web pages. These word lists are recorded in an enormous database
of all the places on the internet where those words exist.
73
Advanced Web Ranking, “Google Organic CTR History.” [Online]. Available:
https://www.advancedwebranking.com/ctrstudy/
74 Google, “How Search organizes information.” [Online]. Available: https://www.google.com/search/howsearchworks/crawlingindexing/
75 Google, “How Search organizes information.” [Online]. Available: https://www.google.com/search/howsearchworks/crawlingindexing/
76 Proficient/digital, “How Google’s Search Results Work: Crawling, Indexing, and Ranking,” Enge, Eric, 21 December 2016.
[Online]. Available: https://blogs.perficientdigital.com/2016/12/21/how-googles-search-results-work-crawling-indexing-andranking/
77
WorldWideWebSize.com, “The size of the World Wide Web (The Internet),” 11 September 2019. [Online]. Available:
https://www.worldwidewebsize.com/
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
E-2
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
How Search Engines Discover Content
Crawlers are software applications that are designed to perform two basic tasks: 1) ingest content
found on a web page for storage in the search engine’s database for inspection at a later time by
the indexer and 2) identification of hyperlinks on the web page, or in a website’s XML
Sitemaps* (lists of web pages, videos or images that the website publisher wants the crawler to
find, crawl, and index).
These link lists help guide the crawler to the web pages it should look at next. The processes of
content ingestion and link identification continue over-and-over again without end. When the
crawler runs out of web pages to crawl, the process is started again. Pages are re-crawled to see if
anything on the page has changed and make sure the page still exists. Given the enormity of the
internet, at any given moment in time, a search engine may have deployed tens of thousands of
crawlers simultaneously crawling the web.
Robots.txt and Meta Robots
Before a crawler begins its work at a website, one of the first things it does is look for a special
file called the robots.txt file78 (most commonly found at locations matching this structure:
https://www.nameofsite.com/robots.txt ). This file is intended to provide instructions to the
crawler regarding which web pages should not be crawled (these are pages the website publisher
does not want included in the searchable index or pages that would consume excessive crawl
resources for pages that should not be searchable such as a website’s site search results pages),
how much time must pass before the crawler can request another page and where the crawler can
find the site’s XML sitemaps79.
Once a web page is identified for crawling, the crawler will look for another instruction located
in the HEAD section of the web page (the HEAD is where meta information about the page is
stored and is not something the typical user ever sees). The specific instruction the crawler is
looking for is called the “meta robots” tag80 and is a string of text that may take the following
form:
In this example, the robots tag is instructing the crawler that it may not include this page in the
search engine’s searchable index (NOINDEX) and that the crawler should not include the links
found on that page in its crawl queue (NOFOLLOW). If a website publisher omits this meta tag,
the default condition is the same as explicitly allowing indexation and link following:
Other Reasons for Crawl and Index Exclusion
There are other reasons that would prevent a crawler from crawling a URL in its crawl queue. A
common reason is password protection. There is much content on the internet that is only
78
Robots.txt, “The Web Robots Pages.” [Online]. Available: https://www.robotstxt.org
Sitemaps.org, “What are Sitemaps?” 7 February 2008. [Online]. Available: https://www.sitemaps.org
80 Robots.txt, “About the Robots tag,” 2007. [Online]. Available: https://www.robotstxt.org/meta.html
79
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
E-3
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
accessible after successful authentication by a user. This can include many types of confidential
information such an individual’s health data, an organization’s financial data, or a government’s
most secure information. It is important that none of this information be discoverable by others
in web search engines.
Other reasons81 content may be omitted from the searchable index: content depends on web
technologies that the crawler is incapable of reading (once upon a time, content that was created
with Flash was difficult for crawlers to parse and was often excluded from search); content is too
slow to load (the crawler gives up waiting and moves on); or content is believed to be a duplicate
of another web page on the site).
Content Duplication: The Rel Canonical Tag
When a website publishes the same content to multiple places on the site, the search engines will
choose one to be the authoritative or preferred version (in web parlance, this page is referred to
as the “canonical” version) and the crawler will ignore the duplicates. A website publisher can
provide instructions to the search engine regarding which page the publisher wants to be treated
as the canonical version through the user of the “rel canonical” tag82. Canonical tags also appear
in the HEAD section of a web page. For example, suppose the crawler arrives at the following
URL:
https://www.example.com/subject-name/second-instance-of-article-1
The crawler finds the following instruction in the HEAD of that page:
This instruction is telling the crawler that the page “article 1” is canonical and the web page
“second instance of article 1” is a duplicate of “article 1.” The crawler will ignore the duplicate
in favor of what the website has designated its preferred version. If the crawler finds no “rel
canonical” tag, the default condition applies which is identical to explicitly setting a “selfreferential” canonical tag. For example, the canonical tag we expect to find on the page named
“article 1” above is:
When a web page points its canonical tag to itself, it is saying “I am the canonical version of this
content.”
Content Redirection: When Web Pages Change their Web Address (URL)
It is common for webpages to move to a new URL. To help users, and crawlers, find the new
location of the webpage, the website implements one of a variety of methods that very quickly
move the page’s visitor to the new address. These are called “redirects.” There are several types
of redirect but they fall into two major categories: server-side and client-side. Server-side
redirects are the more common and include the two most used redirect types: “301 Permanent”
81
Google, “Why is my page missing from Google Search?” 2019. [Online]. Available:
https://support.google.com/webmasters/answer/7474347
82 Google, “Consolidate duplicate URLs,” 2019. [Online]. Available: https://support.google.com/webmasters/answer/139066
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
E-4
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
Redirects and “302 Temporary” Redirects. The most common client-side redirect is the “Meta
Refresh.”
The 301 Permanent Redirect and Meta Refresh Redirect will both work to instruct the crawler to
replace the original URL in its searchable index with the new URL pointed to by the redirect
instruction. The 302 Temporary Redirect com, on the other hand, causes the crawler to keep the
old URL in its index and ignore the new URL on the other side of the redirect because the
redirection has been designated “temporary” in nature. The implication of this is that the older
URL is expected to return to its normal, “pre-redirect” state. In response, the search engine keeps
the older URL in its search index.
Index Exclusions Apply to All Queries
The directives described above such as robots.txt, meta robots, rel canonical, or redirects apply
equally regardless of search query. In other words, the page is either searchable or not searchable
for all queries. The presence of a web page in the index is not tied to any specific search queries.
Search Ranking: How Search Engines Determine the Order of Results
Attempting to describe all of the ranking factors that influence how search engines determine the
order of search results for any given query would be beyond the scope of a summary search
overview such as this one. But we can distill these factors down to the three main categories
described earlier: keyword relevance, site credibility, and quality of user experience.
Keyword relevance is perhaps the ranking factor most people intuitively understand about
search. Solving this problem is at the heart of search because it, more than anything else,
measures how well the search engine was able to help the user find what they were looking for.
The nature of what people are searching for have been described as belonging to one of three
types83: navigational, informational and transactional.
Site credibility is driven by the quantity and quality of hyperlinks that point to a given website. A
quality hyperlink is one that comes from another trusted website. An example of a high-quality
link would occur if a website such as NIH.gov linked to another website. In this case, NIH.gov is
considered a highly trusted source of health-related information. If they chose to link to another
website, the credibility of the site on the receiving end of that link would receive a big bump in
its own credibility score.
Quality of user experience is determined by a variety of signals encountered by the crawler that
it then translates into a user experience rating for the website. For example, if the crawler’s
request for a page takes a long time to complete, that would be a negative quality signal. Other
negative quality signals include things like pages cluttered with advertisements or text that is too
small to read. The search engines are motivated to help their searchers avoid these poor-quality
sites in favor of sites that deliver a much higher quality user experience.
83
IBM, “A taxonomy of web search,” Broder, Anthony, IBM, Fall 2002. [Online]. Available:
http://www.cis.upenn.edu/~nenkova/Courses/cis430/p3-broder.pdf.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
E-5
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
Types of Search Queries
Search queries are the words and phrases people type into a search box in order to pull up a list
of results. Below are commonly accepted types of search queries.
Navigational searches are used by searchers who know where they want to go and are using the
search engine to help get them there. For example, a user that wants to go to the IRS website
might go to a search engine and simply search for [ irs ]. The search engine would almost
certainly present the IRS.gov website as the very first search result (and quite possibly more
search results from IRS.gov right below that).
Informational searches can best be thought of someone performing some level of research about
a subject they want to learn more about. For example, a taxpayer that wants to learn more about
deductions they can claim on their tax return might perform a search for [ what deductions can i
claim ]. For searches such as these, the search engine must quickly identify which web pages it
has in its index that can best answer this question and then determine in what order to present the
results beginning with the most trusted sources of this information.
Transactional searches represent someone looking to perform a “web-mediated activity” such
as making a purchase, downloading media, or subscribing to a news source.
Within this universe of search activity, there is also another important distinction of search
queries: branded vs. non-branded. This distinction can be applied to informational and
transactional search queries. Navigational searches are by definition, branded searches.
Branded searches are those search queries in which the user has specifically identified with
which business or organization they prefer to achieve their search goal. This has the effect of
turning an informational or transactional query into a navigational query. For example, here are
two similar search queries, the first is non-branded, the second is branded.
[ streaming music ]
[ streaming music Spotify]
In both cases, the user is searching for a streaming music service (transactional) but the first one
leaves the door open for different competing services to appear, and the major streaming services
do appear. The user has not yet decided which service they will use. In the case of the second
search query, the user specifies a service which makes clear that they have decided where they
want to go.
As discussed earlier, the fact that organic search results are in a separate section from paid
results, it is important to notice that Spotify competitors appear prominently above the organic
results in the paid search section. Despite the user’s stated preference for Spotify as their music
streaming service, the competing music services are willing to pay for the opportunity to show
their search result to this searcher in the hopes of luring them away from a competitor and are
successful in obtaining a presence in this valuable area of the search results page.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
E-6
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
Appendix F
Taxpayer Journey
F.1 Heuristic Review of Free File Web Pages
[Provided in supplemental file]
F.2 Heuristic Review of IRS.gov
[Provided in supplemental file]
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
F-1
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
Appendix G Taxpayer Experience Testing Report
[Provided in supplemental file]
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
G-1
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
Appendix H Acronyms and Initials
Acronym
AC
AGI
C
CAS
CAPTCHA
CDW
CPA
DIY
ED
EITC
EPSS
ERO
ETARAS
FFA
FFFF
FFI
FFRDC
FY
FS
GAO
IC
ICANN
IRS
IRSAC
LITC
LLC
MC
MOU
MR
NC
OLS
OMB
P
PCI
PM
PMO
PPP
PTIN
Q
RAC
RAL
reCAPTCHA
Definition
Average Cost
Adjusted Gross Income
Compliant
Customer Account Services
Completely Automated Public Turing Test to Tell
Computers and Humans Apart
Compliance Data Warehouse
Certified Public Accountant
Do it yourself
Executive Director
Earned Income Tax Credit
Electronic Products Support & Services
Electronic Return Originator
Electronic Tax Administration Research and Analysis
System
Free File Alliance
Free File Fillable Form
Free File, Inc.
Federally Funded Research and Development Center
Fiscal Year
Filing Season
Government Accountability Office
Inconclusive
Internet Corporation for Assigned Names and Numbers
Internal Revenue Service
IRS Advisory Committee
Low-Income Tax Clinics
Limited Liability Company
Marginal Cost
Memorandum of Understanding
Marginal Revenue
Non-compliant
Office of Online Services
Office of Management and Budget
Price
Payment Card Industry
Program Management
Program Management Organization
Public-Private Partnership
Preparer Tax Identification Number
Quantity
Refund Anticipation Check
Refund Anticipation Loan
Google version of Completely Automated Public Turing
Test to Tell Computers and Humans Apart
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
H-1
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
RRA
TAS
TCE
TFF
TIGTA
TP
TY
URL
UX
VITA
W&I
Restructuring and Reform Act
Taxpayer Advocate Service
Tax Counseling for the Elderly
Traditional Free File
Treasury Inspector General for Tax Administration
Taxpayer
Tax Year
Uniform Resource Locator
User Experience
Volunteer Income Tax Assistance
Wage & Investment
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
H-2
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
Appendix I
References
Advanced Web Ranking, “Google Organic CTR History.” [Online]. Available:
https://www.advancedwebranking.com/ctrstudy/
Behn, R. D. 2001. Rethinking Democratic Accountability, Washington, D.C.: The Brookings
Institution.
Chu, M. S., and M. M. Kovalick. 2006. "An Analysis of the Free File Program," in National Tax
Associate Proceedings, 99th Annual Conference on Taxation, Boston.
Enge, E. 2016. “How Google’s Search Results Work: Crawling, Indexing, and Ranking,”, 21
December 2016. [Online]. Available: https://blogs.perficientdigital.com/2016/12/21/howgoogles-search-results-work-crawling-indexing-and-ranking/
Goldsmith, S. and W. D. Eggers. 2004. Governing by Network: The New Shape of the Public
Sector, Washington, D.C.: The Brookings Institution.
Google. “How Search organizes information.” [Online]. Available:
https://www.google.com/search/howsearchworks/crawling-indexing/
Google, “How Search algorithms work.” [Online]. Available:
https://www.google.com/search/howsearchworks/algorithms/
Google. 2019. “Consolidate duplicate URLs.” [Online]. Available:
https://support.google.com/webmasters/answer/139066
Google. 2019. “Why is my page missing from Google Search?” [Online]. Available:
https://support.google.com/webmasters/answer/7474347
Google Trends, 2019. [Online]. Available: https://trends.google.com. [Accessed week of 27 May
2019].
Hodge, G. A., and C. Greve. 2016. "On Public–Private Partnership Performance: A
Contemporary Review," Public Works Management & Policy, vol. 22, no. 1, pp. 55-78.
IBM. 2002. “A taxonomy of web search,” Broder, Anthony, IBM, Fall 2002. [Online].
Available: http://www.cis.upenn.edu/~nenkova/Courses/cis430/p3-broder.pdf.
Internal Revenue Manual, 12-28-2018.
Internal Revenue Service. 1998. Internal Revenue Service Restructuring and Reform Act of
1998, Pub.L. 105–206, 112 Stat. 685, enacted July 22, 1998.
Internal Revenue Service. 2008. "Findings From The Free File Migration Study, Publication
4684 (3-2008), Catalog Number 51580P," Internal Revenue Service, Washington, D.C.
Internal Revenue Service. 2008. "Free File Survey, Publication 4556 (11-2008), Catalog Number
48884," Internal Revenue Service, Washington, D.C.
Internal Revenue Service. 2010. "Report of Findings From the Free File Marketing Focus Group,
Publication 4833 (Rev 09-2010) Catalog Number 54427P," Internal Revenue Service,
Washington, D.C.
Internal Revenue Service. 2011. "Free File Demographics and Migration Analysis, Publication
4829 (Rev 11-2011), Catalog Number 54376," Internal Revenue Service, Washington, D.C.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
I-1
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
Internal Revenue Service. 2011. "Free File Awareness & Attitudinal Research, Findings From
Research Conducted Among Free File Non-Users With Focus On Three Specific Segments,
Publication 4928 (01-2011), Catalog Number 57580X," Internal Revenue Service, Washington,
D.C.
Internal Revenue Service. 2018. "Internal Revenue Service Advisory Council Public Report,
Publication 5316, Catalog Number 71824A," Internal Revenue Service, Washington, D.C.,
2018.; Taxpayer Advocate Service, "National Taxpayer Advocate Annual Report to Congress,"
Taxpayer Advocate Service, Washington, D.C.
Internal Revenue Service. 2019. "IRS, Free File Alliance announce changes to improve program;
Improved taxpayer options available for 2019 Free File program," Internal Revenue Service, 2
November 2018. [Online]. Available: https://www.irs.gov/newsroom/irs-free-file-allianceannounce-changes-to-improve-program-improved-taxpayer-options-available-for-2019-free-fileprogram. [Accessed 2 July 2019].
Internal Revenue Service. 2019. "Changes strengthen program; Free File can also help navigate
new tax law provisions," Internal Revenue Service, 11 january 2019. [Online]. Available:
https://www.irs.gov/newsroom/irs-free-file-opens-today-in-advance-of-tax-season. [Accessed 10
September 2019].
Internal Revenue Service. 2019. "Free File: About the Free File Alliance," Internal Revenue
Service, 25 March 2019. [Online]. Available: https://www.irs.gov/e-file-providers/about-thefree-file-alliance. [Accessed 2 June 2019].
Internal Revenue Service. 2019. "Free File: Do Your Federal Taxes for Free," Internal Revenue
Service, 24 June 2019. [Online]. Available: https://www.irs.gov/filing/free-file-do-your-federaltaxes-for-free. [Accessed 2 July 2019].
Internal Revenue Service Free File Program Office. No date. Draft Program Management
Approach for Free File, undated [Free File Program Management Plan.v3.docx.]
Internal Revenue Service and Free File Alliance, LLC. 2002. “Free On-Line Electronic Filing
Agreement”, Internal Revenue Service.
Internal Revenue Service and Free File, Incorporated. 2005. "Memorandum of Understanding on
Service Standards and Disputes," Internal Revenue Service.
Internal Revenue Service and Free File, Incorporated. 2018. "Eigth Memorandum of
Understanding on Service Standards and Disputes Between the Internal Revenue Service and
Free File, Incorporated," Internal Revenue Service.
Internet Archive, "Wayback Machine," Internet Archive, 31 December 2014. [Online].
Available: https://web.archive.org/. [Accessed 15 July 2019].
IRS Filing Season Statistics for Week Ending August 31, 2018, available at:
https://www.irs.gov/newsroom/filing-season-statistics-by-year.
Jarrett, C. R. J. S. K. S. K. 2010. "Design to Read: Guidelines for People Who Do Not Read
Easily.” [Online]. Available: http://uxpamagazine.org/people_who_do_not_read_easily/.
[Accessed 15 August 2019]
Marsilio, M., G. Cappellaro and C. Cuccurullo. 2011. "The Intellectual Structure Of Research
Into PPPs," Public Management Review, vol. 13, no. 6, pp. 763-782.
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
I-2
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
The MITRE Corporation. 2010. "Advancing E-file Study - Phase 2 Report," Internal Revenue
Service, Washington, D.C..
The MITRE Corporation. 2010. "Advancing E-file Study - Phase 2 Report," Internal Revenue
Service, Washington, D.C.
Nielsen, Jakob. 1994. Heuristic evaluation. In Nielsen, J., and Mack, R.L. (Eds.), Usability
Inspection Methods, John Wiley & Sons, New York, NY.
Page, S.B., M. M. Stone, J. Bryson and B. C. Crosby. 2015. "Public value creation by crosssector collaborations: A framework and challenges of assessment," Public Administration, vol.
93, no. 3, pp. 715-732.
ProPublica, "About Us," ProPublica, 2019. [Online]. Available:
https://www.propublica.org/about/. [Accessed 2 July 2019].
Robots.txt. “The Web Robots Pages.” [Online]. Available: https://www.robotstxt.org
Robots.txt. 2007. “About the Robots tag”. [Online]. Available:
https://www.robotstxt.org/meta.html
Russell, J. G. 2006. "2006 Tax Return Filing Season and the 2007 Budget Proposal for the
Internal Revenue Service: Hearing before the U.S. House Of Representatives Committee on
Ways and Means Subcommittee on Oversight (Statement of J. Russell George)," Department of
Treasury, Washington D.C.
Search Enginer Watch. 2018. “A beginner’s guide to paid search,” 1 June 2018. [Online].
Available: https://searchenginewatch.com/2018/06/01/a-beginners-guide-to-paid-search/
SEMrush Inc., "All-in-one Marketing Toolkit," SEMrush Inc., 2019. [Online]. Available:
https://www.semrush.com/. [Accessed week of 27 May 2019].
Sitemaps.org. 2008. “What are Sitemaps?” 7 February 2008. [Online]. Available:
https://www.sitemaps.org
Taxpayer Advocate Service. 2018. "National Taxpayer Advocate Annual Report to Congress:
Volume 1," Taxpayer Advocate Service, Washington, D.C.
Treasury Inspector General for Tax Administration. 2007. "Additional Action Is Needed to
Expand the Use and Improve the Administration of the Free File Program," U.S. Department of
Treasury, Washington, D.C.
U.S. Department of the Treasury, Office of Public Affairs. 2002. "Treasury, IRS Announce New
Efforts to Expand E-Filing," Washington D.C.
Wikipedia. 2019. “Page Rank,” 10 September 2019. [Online]. Available:
https://en.wikipedia.org/wiki/PageRank
WorldWideWebSize.com. 2019. “The size of the World Wide Web (The Internet),” 11
September 2019. [Online]. Available: https://www.worldwidewebsize.com/
PRE-DECISIONAL – FOR INTERNAL IRS W&I USE ONLY – NOT FOR PUBLIC RELEASE
I-3