Checklist for Determining whether a Tool or System
is an ADS/Agency ADS as defined by Local Law 49 (2018)
Question 1: Is the tool or program an “automated decision system” (ADS)?
YES ☐ NO ☐
1
Legal definition Automated decision system (ADS): Computerized implementations of algorithms,
Local Law 49 of including those derived from machine learning or other data processing or artificial
2018, § 1(a).
intelligence techniques, which are used to assist in making decisions.
In determining whether a tool or program constitutes an ADS as defined by Local Law 49 of
2018, answer “yes” or “no” to the following questions. If answering “yes” to questions
(1)(a) through (1)(c), check “yes” for Question #1 above, and proceed to Question #2.
YES ☐ NO ☐
(1)(a) Is it an algorithm?
•

A set of formal or informal rules, processes, or instructions for carrying out a
specified operation or solving a problem.

For informational purposes only, indicate whether the algorithm was derived from any of the
2
following:
•
Machine learning ☐

•

•

•

The algorithm derives from a process by which a software component or other
computerized implementation identifies patterns or regularities from input data. Key
types of machine learning may include, but are not limited to (in varying degrees of human
interaction):
o Supervised learning (where an algorithm uses data and human feedback to learn
relationships of given input to output);
o Unsupervised learning (where an algorithm explores input data without being given
explicit output variables); and
o Reinforcement learning (where an algorithm learns to perform a task by trying to
maximize rewards it receives for its actions).

Data processing ☐

The algorithm derives from an operation performed on data, such as collection, recording,
organization, storage, adaption or alteration, retrieval, consultation, use, disclosure,
transmission, dissemination or otherwise making available, alignment or combination,
blocking, erasure, or destruction.

Artificial intelligence (AI) techniques ☐

The algorithm derives from methods that simulate human thought or processes, language,
and intelligence. AI techniques may include, but are not limited to:
o Natural language processing (software or a computer understanding, interpreting,
and manipulating human language);
o Neural networks (using algorithms modeled after the human brain to recognize
patterns);
o Regression analysis (finding relationships between two or more variables of interest
by examining influence of one or more independent variables on a dependent
variable); or
o Markov decision process (decision model for situations in which outcomes are in
part random and in part under control of a decision maker).

Other ☐

1

For purposes of ADS Task Force’s work, clarification of relevant key terms that are included in Local Law 49 of 2018, but not defined in
the Law (i.e., underlined terms), is provided below.
2
Note, the language used in (1)(a) to describe the below terms in this subsection is provided solely as guidance to help clarify their
plain language meaning for informational purposes, recognizing that in the emerging ADS field, there may be alternate and/or more
technical descriptions.

1

 (1)(b)

Is it a computerized implementation of an algorithm?
YES ☐ NO ☐
• An algorithm that takes the form of, or is it otherwise partially or entirely operated or
controlled by, software or a computer.
o

(1)(c)

Is the algorithm used to make or assist in making a decision?
•

•

YES ☐ NO ☐

A determination made by a City official/employee which may impact the actions,
policies, procedures, services, programs, employment matters, contracting practices,
rulemaking, or budgeting/allocation of resources, that is implemented by the City or
an individual agency.
Decisions may be related to, but are not limited to:
o Granting or denying a permission or right;
o Assigning status or eligibility;
o Assigning the type, level, or amount of assistance, benefit or service an
individual, household, or population does/does not receive (e.g. housing, cash
assistance, medical assistance, etc.);
o Allocating resources to City programs or services;
o Furthering civil and criminal law enforcement activities (e.g., investigation,
surveillance, arrest, pre-trial detention, sentencing or corrections)
o Responding to public safety/public health concerns (e.g., through health
promotion or disease prevention techniques);
o Assigning value to assets (e.g. property valuation); or
o Auditing programs or entities.

If any of the above questions is answered “no”, then the system or tool under review is not
considered an “automated decision system (ADS)” as defined by Local Law 49 (2018).

2

 Question #2: Does the ADS constitute an “Agency ADS”?

YES ☐ NO ☐
Legal definition Automated decision system, agency (Agency ADS): An ADS used by an agency to
Local Law 49 of make or assist in making decisions 3 concerning rules, policies, or actions
2018, § 1(a).
implemented that impact the public.
In determining whether a tool or program constitutes an Agency ADS as defined by Local
Law 49 of 2018, answer “yes” or “no” to the following questions. If answering “yes” to
questions (2)(a) and (2)(b), check “yes” for Question #2 above.
(2)(a)

Is the ADS used by an agency to make or assist in making decisions concerning rules,
policies, or actions implemented by the City or a City agency?
YES ☐ NO ☐
• Rules refer to regulations promulgated by a City agency, either pursuant to
the City Administrative Procedure Act (CAPA), or by a federal, state or other
agency where the City agency is tasked with implementation.
• Policies refer to principles, procedures, or protocols adopted by an agency to
guide the agency’s practices.
• Actions refer to affirmative acts/steps the agency takes, as well as inaction by
an agency.

(2)(b) Do/will such decisions impact the public?
YES ☐ NO ☐
• The term “impact” is understood as having a material impact on an individual
member of the public or a population, in connection with a City agency’s use
of an ADS. In determining whether an impact is material, consider, among
other factors, whether the decision:
o Pertains to a procedural or substantive right granted under law,
regulation, or by City policy;
o Relates to an individual’s or population’s protected status;
o Relates to an individual’s or population’s eligibility for, or receipt or
denial of a City or agency program, service, or benefit;
o Affects whether an individual or population will, as a result of the
decision, be subject to a specific City program or activity; or
o Implicates judicial, administrative, or other forms of redress.
• Programs that solely perform calculations for purposes of administrative
convenience, such as adding numbers rather than performing manual addition, or
that simply organize information, such as spreadsheets, shall not be considered a
“computerized implementation” for purposes of ADS Task Force work.

If any of the above questions is answered “no”, then the ADS under review is not considered an
“Agency ADS” as defined by Local Law 49 (2018).

3

Refer to clarification on meaning of “Decisions” provided in Question #1, at (1)(d).

3