Consequential Decisions: Colorado's 2024 Law on High-Risk Artificial Intelligence Systems, May 2024

On May 17, 2024, Colorado's governor approved Senate Bill 24-205, "Concerning consumer protections in interactions with artificial intelligence systems." The act adds a new part, titled "Artificial Intelligence," to article 1 of title 6 of the Colorado Revised Statutes, and the duty of reasonable care it places on developers and deployers applies "on and after February 1, 2026." This article reports the act as it was signed. It does not report any later change to it. Parts of the act follow, not the whole.

What the Law Covers

A "high-risk artificial intelligence system" is any AI system that, when deployed, "makes, or is a substantial factor in making, a consequential decision." A consequential decision is one with "a material legal or similarly significant effect on the provision or denial to any consumer of, or the cost or terms of" a list of things: education, employment, "a financial or lending service," an essential government service, health-care services, housing, insurance, and "a legal service." A consumer is an individual who is a Colorado resident. Algorithmic discrimination, as the act defines it, is any condition in which the use of an AI system "results in an unlawful differential treatment or impact that disfavors an individual or group of individuals" on the basis of a protected classification, such as age, disability, race or sex.

The definition leaves some systems out. A system intended to perform "a narrow procedural task" is not high-risk, and neither is one intended to detect decision-making patterns, or deviations from them, that is not intended to replace or influence "a previously completed human assessment without sufficient human review." A list of technologies, among them anti-malware, calculators, databases and anti-fraud tools that do not use facial recognition, is excluded unless, when deployed, they make or are a substantial factor in making a consequential decision.

What a Deployer Must Do

A deployer, a person doing business in the state that uses a high-risk system, "shall use reasonable care to protect consumers from any known or reasonably foreseeable risks of algorithmic discrimination." In an enforcement action by the attorney general, a deployer that complied with the section and any rules the attorney general adopts is presumed, rebuttably, to have used reasonable care.

The deployer must implement a risk management policy and program that specifies and incorporates "the principles, processes, and personnel" it uses to identify, document and mitigate those risks. The program must be reasonable in light of, among other things, the latest version of the AI risk management framework published by the National Institute of Standards and Technology, the international standard ISO/IEC 42001, or another recognized framework substantially equivalent or more stringent. With some exceptions, a deployer, or a third party it contracts, must complete an impact assessment for a deployed high-risk system "at least annually and within ninety days after any intentional and substantial modification to the high-risk artificial intelligence system is made available," and "a single impact assessment may address a comparable set" of such systems. The program, the impact assessments and a public statement on a deployer's website do not apply to a deployer if, when it deploys the system and at all times while the system is deployed, it employs fewer than fifty full-time equivalent employees and does not use its own data to train the system; the system is used for the intended uses disclosed to the deployer and "continues learning based on data derived from sources other than the deployer's own data"; and the deployer makes available to consumers any impact assessment the developer has completed and provided to it that includes information "substantially similar" to the impact assessment the act requires.

What a Consumer Is Told

Before a consequential decision is made, the deployer must notify the consumer that a high-risk system is being used to make it, or to be a substantial factor in making it. If the decision is adverse, the consumer is owed "a statement disclosing the principal reason or reasons for the consequential decision," including "the degree to which, and manner in which, the high-risk artificial intelligence system contributed to the consequential decision," the type of data it processed, and the source of that data. The consumer also gets an opportunity to correct incorrect personal data, and an opportunity to appeal, "which appeal must, if technically feasible, allow for human review unless providing the opportunity for appeal is not in the best interest of the consumer."

A separate section reaches beyond high-risk systems. A deployer or developer that makes available an AI system "intended to interact with consumers" must ensure each consumer is told they are interacting with one. That disclosure is not required where "it would be obvious to a reasonable person that the person is interacting with an artificial intelligence system."

Enforcement, and Full Compliance for Some Firms

The attorney general "has exclusive authority to enforce" the new part, and a violation is an unfair trade practice. The act "does not provide the basis for, and is not subject to, a private right of action for violations of this part 17 or any other law." It gives an affirmative defense to a developer or deployer that discovers and cures a violation through feedback, adversarial testing or red teaming, or an internal review process, and is otherwise in compliance with the NIST framework and ISO/IEC 42001, or with an equivalent framework or one the attorney general designates. The developer or deployer bears the burden of showing it.

The act also treats certain insurers, banks and credit unions as in full compliance in certain conditions. An insurer, a fraternal benefit society, or a developer of an AI system used by an insurer is "in full compliance" if it is subject to section 10-3-1104.9 of the state's statutes and any rules the commissioner of insurance adopts under it. A bank, out-of-state bank, Colorado-chartered, federal or out-of-state credit union, or an affiliate or subsidiary of one, is "in full compliance" if it is subject to examination by a state or federal prudential regulator under guidance or regulations on high-risk AI systems that are "substantially equivalent to or more stringent than" the act's requirements, and that, at a minimum, require it to "regularly audit" its use of high-risk systems for compliance with state and federal anti-discrimination laws and to mitigate algorithmic discrimination.

For an adverse decision, the act requires a deployer to state the principal reasons, including the degree to which, and the manner in which, the system contributed. Where the decision concerns a financial, lending, insurance or legal service, would the deployer's own records let it answer that, decision by decision?