The use of algorithmic software and automated decision systems (ADS) for making workforce decisions, including the most sophisticated types of AI (AI), has been rapidly increasing in recent years. The promises of HR technology to increase productivity and efficiency, data-driven insights, and cost-reducing are undoubtedly appealing to businesses striving to streamline their operations, such as employment, promotion, performance assessment, compensation reviews, and termination of employment. However, as businesses increasingly rely on AI, algorithms, and automated decision-making tools (ADTs) to make high-stakes workforce decisions, they could unconsciously be exposed to serious legal risks, particularly based on the Civil Rights Act of 1964, the Employment Act (ADEA), Local Law (ADA), and the Federal Government’s Age Discrimination Title VII.
Quick Hit
Using automated technology to make workforce decisions presents important legal risks under existing anti-discrimination laws such as Title VII, ADEA, and ADA, as algorithmic bias can lead to allegations of discrimination. Algorithm HR software is extremely dangerous because it amplifies the scale of potential harm, unlike human judgment. A single biased algorithm can affect thousands of candidates or employees, and exponentially increases liability risk compared to biased individual human decisions. A proactive and privileged software audit is important for reducing legal risks and monitoring the effectiveness of AI in making workforce decisions.
What are automated technology tools and how is AI related?
In the context of employment, an algorithm or automatic HR tool refers to a software system that utilizes predefined rules to execute data through algorithms to assist in a variety of HR functions. These tools range from simple rule-based formula systems to more advanced generation AI-driven technologies. Unlike traditional algorithms that operate on fixed, explicit instructions for processing data and making decisions, the generator AI systems differ in that they are not limited to defined rules, but can learn from the data, adapt over time, and make autonomous adjustments.
Employers use these tools in a variety of ways to automate and enhance HR capabilities. Some examples:
Applicant Tracking Systems (ATS) often use algorithms to grade applicants, comparing applicants with position descriptions or rank resumes, by comparing applicants’ skills with each other. Skill-based search engines rely on algorithms to match job seekers with open positions based on resume qualifications, experience and keywords. The AI-powered interview platform evaluates candidate responses in video interviews, assesses facial expressions, tones and language to predict skills, fit, potential for success, and more. An automated performance rating system can analyze employee data such as productivity metrics and feedback to provide an assessment of individual performance. AI systems allow you to listen on the phone and grade employee-customer interactions. This is a feature that is commonly used in the customer service and sales industry. AI systems can analyze background check information as part of the employment process. Automatic technology can be incorporated into your compensation process to predict pay, assess market equity, and assess wage equity. An automated system is available to employers or candidates in the employment process for scheduling, note-taking, or other logistics. AI models can analyze historical employment and employee data to predict which candidates are most likely to succeed in roles, or which new hires are at risk of early turnover.
AI liability is at risk under current law
AI-led workforce decisions are subject to a variety of employment laws, with employers increasing numbers of institutional investigations and litigation related to the use of AI in employment. Key legal frameworks include:
Title VII: Title VII prohibits discrimination based on race, color, religion, gender, or national origin in employment practices. Under Title VII, employers can be held responsible for facial neutral practices that have an imbalance and negative impact on members of protected classes. This includes decisions made by the AI system. Even if the AI system is designed to be neutral, if there is a discriminatory effect on the protected class, employers can be held liable under different theory of impact. The current administration directs federal agencies to strip them of different theories of influence, but it remains a legal theory that is feasible under federal, state and local anti-discrimination laws. If AI systems provide assessments used by human decision makers as one of many factors, they can also contribute to the risks of different treatment discrimination. ADA: If the AI system screens individuals with disabilities, it may violate the Persons with Disabilities Act (ADA). It is also important that employers provide appropriately reasonable accommodations to ensure that AI-based systems are accessible and avoid discrimination against individuals with disabilities. ADEA: The Age Discrimination in Employment Act (ADEA) prohibits discrimination against applicants and employees over the age of 40. Equal pay law: AI tools that consider compensation and pay data tend to replicate past wage disparities. Employers using AI should ensure that the system does not create or perpetuate gender-based pay inequality, or that there is a risk of violating the Equal Pay Act. EU AI Law: This comprehensive law is designed to safely and ethically use artificial intelligence across the European Union. It treats employer’s use in the workplace as potentially high risk, imposes obligations for ongoing use, and potential penalties for violations. State and Local Law: While there are no federal AI laws yet, many states and regions have passed or proposed AI laws and regulations, covering topics such as video interviews, facial recognition software, automated employment decision tools (AEDTS) bias audits, robust notification and disclosure requirements. The Trump administration has overturned Biden-era guidance on AI and underscored the need for minimal barriers to fostering AI innovation, but states may step in to close regulatory gaps. Additionally, existing state and local anti-discrimination laws also create liability risks for employers. Data Privacy Law: AI also includes many other types of laws, including international, state, and local laws that govern data privacy.
The challenges of algorithm transparency and accountability
One of the most important challenges regarding the use of AI in workforce decisions is the lack of transparency in the way algorithms make decisions. Unlike human decision makers who can explain reasoning, generative AI systems act as “black boxes,” making it difficult for employers to understand or defend how decisions are reached.
This opacity creates significant legal risks. Without a clear understanding of how algorithms reach conclusions, it may be difficult to defend against claims of discrimination. If a company cannot provide clear evidence as to why the AI system made a particular decision, it could face regulatory action or legal liability.
Algorithmic systems generally apply the same equation to all candidates, creating relative consistency in comparison. With a generator AI system, more complexity is increased as judgments and standards change over time as the system absorbs more information. As a result, the decisions applied to one candidate or employee differ from those made at different times.
Reducing legal risks: AI auditing, labor analysis, bias detection
While potential legal risks are important, there are positive steps in which employers want to hire them to mitigate their exposure to algorithm bias and discrimination claims. These steps include:
Ensuring that there is a robust policy that manages issues related to AI use such as transparency, non-discrimination, and data privacy ensures that there is a robust policy that manages AI privacy so that AI vendors do due diligence and continue to continue to ensure that they continue to consider what the intended purpose and impacted HR, talent acquisition, and what the AI tools are ultimately considering. Bias audit requirements that provide reasonable audit requirements for regular monitoring of AI tools through privileged workforce analysis, ensure that they do not have a different impact on protection groups that create continuous monitoring programs to ensure human surveillance of impact, privacy, legal risks, and more.
Implementing routines and continuous audits under legal privileges is one of the most important steps to ensure that AI is being used in a legally defensible way. These audits may include monitoring algorithms to differently affect protected groups. If the employment algorithm disproportionately screens individuals in a protected group, the employer may want to take steps to correct these biases before it leads to a charge of discrimination or lawsuit. Given the risks associated with volume, companies may want to undertake these privilege audits on a daily basis (e.g. monthly, quarterly, etc.) basis to ensure corrective action as quickly as possible.
As AI landscapes are evolving rapidly, employers may want to continue tracking changing laws and regulations to implement policies and procedures to ensure safe, non-discriminatory use of AI in the workplace, and to mitigate risk by conducting privileged and aggressive analysis to assess bias’s AI tools.
