On April 6, 2023, New York City’s Department of Consumer and Worker Protection (“DCWP”) issued final rules governing Local Law 144 of 2021. That law prohibits employers in New York City from using automated employment decision tools (“AEDTs”) to screen candidates for hiring or promotion without satisfying certain requirements. Among other things, the employer must obtain an independent bias audit of the AEDT and notify affected applicants and employees of its use. The final rules clarify: (1) the definition of AEDTs; (2) the requirements of a bias audit; (3) the data requirements for the bias audit; and (4) the information employers must publish on their websites about the bias audit. The final rules also address some questions that remained from the earlier versions.

Which Tools Qualify as an AEDT

The law defines an AEDT to be a computational process derived from machine learning, statistical modeling, data analytics, or artificial intelligence that issues a simplified output, including a score, classification, or recommendation, and is used to substantially assist or replace discretionary decision-making for employment decisions impacting natural persons. The law expressly excludes tools that do not automate, support, substantially assist or replace discretionary decision-making processes and that do not materially impact natural persons (e.g., junk email filters, firewalls, antivirus software, calculators, spreadsheets, databases, data sets, and other compilations of data).

The second version of the proposed rules had interpreted “machine learning, statistical modeling, data analytics, or artificial intelligence” to mean mathematical, computer-based techniques:

1. that generate a prediction, meaning an expected outcome for an observation, such as an assessment of a candidate’s fit or likelihood of success, or that generate a classification, meaning an assignment of an observation to a group, such as categorizations based on skill sets or aptitude; and
2. for which a computer at least in part identifies the inputs, the relative importance placed on those inputs, and other parameters for the models in order to improve the accuracy of the prediction or classification; and
3. for which the inputs and parameters are refined through cross-validation or by using training and testing data.

The final rules eliminate the third requirement above, thereby broadening which tools qualify as an AEDT because they need not be cross-validated to come within the ambit of the law.

Bias Audit Requirements

Like previous versions, the final rules require the independent auditor to calculate the impact ratio—the scoring or selection rate, as applicable, of a given demographic category divided by the rate of the highest scoring or most selected category—for sex, race/ethnicity, and intersectional categories. The bias audit must also indicate the number of candidates not included in the impact ratio calculations because they fall within an unknown category. The independent auditor may exclude categories representing less than 2% of the data. Yet, the summary of results must still contain the number of applicants in the category and the scoring or selection rate, which is enough to calculate the impact ratio.

Notably, the final rules clarify that the employer’s bias audit summary of an AEDT that scores candidates must reflect the number of candidates subject to the AEDT. Previously, the rules only required employers to post sample-size data for AEDTs that selected, rather than scored, candidates.

Data Requirements

The final rules also clarify what types of data are required for the bias audit. In the first instance, employers must rely on historical data from the AEDT’s use, either from the employer’s own use of the AEDT, or historical data from other employers if (1) the employer provided historical data from its own use of the AEDT to the independent auditor, or (2) the employer has never used the AEDT before. Alternatively, the employer may rely on a bias audit using test data if insufficient historical data is available to conduct a statistically significant bias audit, so long as the summary of results explains why historical data was not used and describes how the test data was generated and obtained.

Summary of Results

Before using the AEDT, an employer must publish the distribution date (i.e., the date the employer first used the AEDT) and a summary of the results from the bias audit on the employer’s website. The final rules clarify that the summary must contain: “the source and explanation of the data used to conduct the bias audit, the number of individuals the AEDT assessed that fall within an unknown category, and the number of applicants or candidates, the selection or scoring rates, as applicable, and the impact ratios for all categories.” Previously, there was ambiguity regarding how specific the employer’s publicized summary of the bias audit must be.

---

The DCWP announced that it will not enforce the law until July 5, 2023. Employers using AEDTs should take this extra time to obtain a bias audit by an independent auditor. They should also prepare summaries for their websites once the audit is complete, as well as the required notices to candidates.