The NYC Local Law 144 &
the “hidden” business value
of AI Audits
If you’re working in the HR domain you probably have heard of New York City’s Local Law 144 (also known as New York City’s Algorithmic Hiring Law or Automated Employment Decision Tool – AEDT law). The goal of this law is to regulate the use of AED tools for candidates and employees within New York City. Essentially it requires that algorithm-based technologies for recruiting, hiring, or promotion be audited for bias before being used. This law is of local importance as it concerns employers or employment agencies in New York City and software providers that build these tools and sell/license/lease to the said employers and agencies.
The law is going to be in effect as of the 5th of July and in our opinion, its importance and gravity are far greater and crucial than just its local impact in the city of New York. It may set the blueprint on how to operationalise AI Audits in different (and most probably larger) regulatory contexts such as EU AI Act. This is based on the Law’s characteristics, which namely are:
- It is probably one of the first (or maybe the first) legislations that will be enforced very soon,
- It establishes the need for independent auditors to exercise objective and impartial judgement within the scope of the audit,
- It defines clear criteria as to what measurements shall be at a minimum conducted, in order to test whether an AED tool is discriminating against any protected categories, such as gender, sex, ethnicity, or race.
The law also is becoming more concrete by specifying and including within the scope of the audit, metrics such as:
- The selection rate or scoring rate for each category
- The Impact ratio for each category
- The Intersectional analysis of disparate impact
- Its definition about the machine learning and artificial intelligence terms, as it is not restricted solely to machine learning or artificial intelligence systems, but includes a wider variety of automated algorithmic systems such as those based on statistical modeling and data analytics and for which the need for a mathematical computer-based technique that allows for inputs and parameters to be refined through cross-validation or by using training and testing data was eliminated.
Most interestingly (maybe) is the public debate about what NYC Local Law
144 prescribes, that took place before its finalization. The NYC Department of Consumer and Worker
Protection (DCWP) adopted its final rules on April 6, 2023, following two
previous draft proposals, the first in September 2022 and the second in
December 2022 ; a significant volume of comments (including from employers,
employment agencies, law firms, AEDT developers and advocacy
organizations); and two public hearings. The debate seemed to be focusing in the following areas:
- The proposed definition of what an “Automated Employment Decision Tool” is and the ambiguities it might create. Currently and although from a technology point of view, Law 144 is wide enough (as stated above), certain extracts in the definition are narrowing down the scope of the Law having as criterion the gravity of the output of a given tool. In essence, advocates of stricter AI regulation, argue that those changes would render many tools exempt from the regulation as practically if someone claims that human judgment typically overrules the judgment of a given tool then it is not subject to an audit.
- The definition of an “Independent Auditor” which essentially requires third-party organizations to conduct the audit. HR software vendors and employers organizations argued that this practically prohibits them from conducting those audits in-house which they believe to be more efficient and effective than outsourcing audit responsibilities to individuals who are not familiar with the AEDT being used or employer practices.
As a company with experience in assessing and auditing large-scale software and AI based systems, we considered extensive discussion and even controversy around Local Law 144, as the first law of “its kind”, expected. The domain of Ethical (or Trustworthy) AI, on which we are working on for the past years, has only recently attracted general attention, given the recent advancements of AI technology and its implications, which are at their beginning of being understood and assessed. Also expected as a first reaction was the tendency of a part of affected stakeholders to push towards decreasing the scope of the Law.
However, the important issue of essence from a corporate aspect is how organisations can benefit from the mandate of the Law. You see, an audit can be either seen as unnecessary scrutiny or an added-value exercise in which a system, a team or an organisation can benefit from and become better.
Based on our experience, a value-adding audit consists of the following elements:
- It is time-boxed: It has a specific timeline or deadline to ensure that the overall audit process is efficient and does not drag on indefinitely. This helps to ensure that the project is completed in a timely manner and the recommendations can be implemented as soon as possible.
- It is as less intrusive as possible: The audit is conducted in a manner that helps to minimize disruption to daily operations of the organisation and ensures that it is conducted in a way that is respectful of their resources and staff.
- It is fact-based and grounded on accepted industry standards: This helps to ensure that the audit is rigorous, reliable and credible, and provides actionable recommendations that are relevant to the organisation’s needs. For instance, standards such as ISO 29119-11 and ISO 4358 can be employed for the purposes of such an audit.
- It doesn’t end with the diagnosis. Actually the diagnosis is the starting point for delivering value as the identified areas for improvement alongside practical recommendations will facilitate the organisation to achieve its goals.
- It provides insights and practical recommendations that can be implemented by the parties responsible for: The most important criterion for the provided recommendations is be specific, actionable, and tailored to the needs and priorities of the organisation. So, as any good advice, it should take into account the organisation’s specific context and challenges.
- It has a follow-up component in which the auditing team keeps track of the improvements and works next to the team implementing them by providing constant advice. This helps to ensure that the recommendations are being implemented effectively and that the organisation is making progress towards its goals.
Provided that the above are operationalised properly, the benefits for an employer organization or HR vendor can be the following:
- (Bias) Risks detection and mitigation: In general, an AI audit can help to identify potential risks and vulnerabilities in an organisation’s AI systems, processes, and data. The bias audits as defined by the Local Law 144 can be the first step for a full scale audit which can include other characteristics such as transparency (explainability analysis) and safety/security (e.g. evasion attacks, data poisoning and others).
- Increased trust over their products and services: By demonstrating that they have undergone a rigorous audit process and are committed to ensuring the fairness and quality of their AI systems, organisations can build trust with their customers, partners, and other stakeholders.
- Improved brand perception: By demonstrating a commitment to fairness, transparency and accountability, organisations can differentiate themselves from competitors and position themselves as leaders in the industry.
- Ability to diversify from the rest of the competition: By demonstrating that they have undergone a thorough audit process and have implemented best practices in AI development, organisations can stand out in a crowded marketplace and attract new customers and partners.
Finally, a short “note” about why an independent auditor is necessary to
perform the audit instead of an in-house solution: objectivity, expertise,
credibility. An independent auditor is the obvious choice for ensuring that an
audit of an AI system is conducted in a thorough, objective, and credible
manner, and that any issues or risks associated with the AI system are
identified and addressed appropriately.
In conclusion, when legislations such as NYC Law 144, are to be introduced, controversy and opposition shall be expected. However, it is in the hands of the legislators and most importantly in our peers in the domain of AI Audits to alleviate concerns and communicate clearly to those who will be subject to audits the benefits they can get and the opportunity they have to build and/or operate better and more trusted systems.