Skip to main content
Bureau of AI
Category: Trustworthy AI Principles

Human Agency

Also known as: Agency
Simply put

Human agency is the capacity of people to make their own choices, act on them, and shape the world around them. In the context of AI, it commonly refers to a person's ability to direct their own thinking and decisions rather than defer entirely to automated systems. The exact meaning varies across disciplines such as philosophy, sociology, and psychology.

Formal definition

Human agency, as commonly defined across the philosophical, sociological, and psychological literature, denotes the capacity of individuals to make intentional choices, enact those choices, and influence their environment and life trajectory. Philosophical treatments frame it as humans making and enacting decisions independent of debates over determinism versus free will, while sociological accounts emphasize the power to think and act in ways that shape one's experiences. The sources provided define the general concept but do not establish a single authoritative definition tied to AI governance frameworks; readers should note that regulatory and policy uses of the term (for example, in the context of human oversight of AI systems) may attach more specific meanings not covered by this evidence.

Why it matters

Human agency matters in AI governance because it names what is at stake when decisions are delegated to automated systems: the capacity of people to make their own intentional choices, act on them, and shape their circumstances. As the evidence digest describes it, agency is the everyday act of directing one's own thinking and deciding what to do, rather than deferring entirely to external direction. When AI systems increasingly mediate the information people see and the options presented to them, the concern is whether individuals retain the ability to think for themselves and enact their own decisions.

The practical significance of the term depends heavily on context. In philosophical and psychological literature, human agency is treated as a general capacity of individuals, framed independently of debates over determinism or free will. In AI governance and policy settings, the term is often invoked in connection with human oversight of automated systems, but the sources provided here do not establish that specific regulatory meaning. Professionals should therefore treat the general concept and any framework-specific usage as related but distinct, and avoid assuming that a single definition applies across all contexts.

Because the concept spans several disciplines and its application to AI governance is still evolving, readers should be cautious about attributing precise operational requirements to it. The evidence supports a general definition of human agency but does not, on its own, define how the concept is operationalized within any particular governance framework or legal instrument.

Who it's relevant to

AI Governance and Policy Specialists
Those designing organizational governance for AI systems encounter human agency as a concept that informs discussions of human oversight and the appropriate role of automated decision-making. Because the sources here define the general concept rather than a framework-specific meaning, specialists should identify how any particular instrument or policy defines the term before treating it as an operational requirement.
Compliance and Legal Professionals
Practitioners interpreting requirements that reference human oversight or the preservation of individual choice may need to distinguish the general philosophical and psychological meaning of human agency from any narrower usage embedded in a specific regulatory or policy context. The evidence supports only the general definition, so framework-specific obligations should be sourced separately.
Model Risk and Oversight Practitioners
Those responsible for ensuring that people, rather than automated systems, retain meaningful control over decisions may use human agency as a conceptual anchor for evaluating where and how humans direct their own reasoning versus deferring to model outputs. Practitioners should note that the concept as defined here is general and does not by itself specify particular controls or monitoring measures.
Researchers and Ethicists
Those studying the interaction between people and AI systems draw on human agency as a cross-disciplinary construct spanning philosophy, sociology, and psychology. They should be aware that its meaning varies across these disciplines and that its application to AI governance is still evolving, so no single definition should be treated as authoritative across all contexts.

Inside Human Agency

Human oversight
The capacity for people to monitor an AI system's operation and intervene where appropriate. In many governance frameworks this is distinguished by the degree of human involvement, ranging from human-in-the-loop (a person reviews or approves individual decisions), human-on-the-loop (a person supervises operation and can intervene), to human-in-command (a person retains overall authority to decide whether and how the system is used).
Meaningful choice and autonomy
The principle that AI systems should support, rather than subvert, a person's ability to make informed decisions. This typically includes avoiding manipulation, deception, or design patterns that unduly steer users, so that affected individuals retain the ability to act on their own judgment.
Ability to intervene and override
Mechanisms that allow a human to stop, correct, or reverse an AI system's action, including override controls and, where applicable, the ability to disengage or shut down the system. The appropriate level of intervention typically depends on the system's context and the risk it presents.
Contestability and recourse
Processes through which individuals affected by an AI-informed decision can question, seek review of, or obtain human reconsideration of an outcome. This is commonly treated as a component of preserving human agency where automated processing affects people.
Informed understanding
Providing affected individuals and operators with sufficient information about the system's role, limitations, and effects so their involvement is genuine rather than nominal. Note that this concerns whether humans can exercise agency, and is related to but distinct from the technical properties of explainability or interpretability.

Common questions

Answers to the questions practitioners most commonly ask about Human Agency.

Is human agency the same as having a human review every AI-generated output?
No. Human agency, as commonly framed, concerns preserving people's capacity to make autonomous, informed decisions and to retain meaningful control over AI systems that affect them. Mandatory review of every output is one possible operational control, but it is not equivalent to the underlying principle. In many frameworks, human agency can be supported through design choices, disclosure, opt-out mechanisms, and escalation pathways, not solely through blanket manual review. Treating the two as identical conflates a broad governance objective with one specific implementation technique.
Does maintaining human agency mean a human always makes the final decision?
Not necessarily. Human agency is often distinguished from strict human decision-making authority. It emphasizes that individuals retain the ability to understand, contest, and influence AI-driven outcomes, and that they are not unduly subordinated to automated processes. Depending on the framework and use case, this may involve human-in-the-loop, human-on-the-loop, or human-in-command arrangements rather than a human personally deciding each case. The appropriate configuration typically depends on the risk and context of the application, so equating agency with a universal final-human-decision rule is an oversimplification.
How can an organization operationalize human agency in a specific AI deployment?
Organizations typically start by identifying where the AI system affects individuals and what decisions or capacities could be constrained. From there, common measures include defining oversight roles, specifying escalation and override procedures, providing meaningful disclosure to affected people, and enabling mechanisms to question or contest outcomes. The design of these measures is usually calibrated to the assessed risk and context of the use case. These controls are intended to support and preserve human agency, not to eliminate all risk associated with the system.
Which roles or lines of defense are usually responsible for supporting human agency?
Responsibility is often distributed across the lines of defense. First-line functions that build and operate the system typically embed oversight and control features into workflows. Second-line functions, such as risk and compliance, may set policy expectations, review the adequacy of oversight arrangements, and challenge design choices. Third-line functions, such as internal audit, may assess whether the controls operate as intended. The precise allocation depends on an organization's governance structure and should be defined explicitly rather than assumed.
How can human agency measures be documented for governance and audit purposes?
Documentation commonly records where human oversight is required, who holds oversight responsibility, the conditions that trigger escalation or override, and how affected individuals can contest outcomes. It is also useful to capture the rationale for the chosen oversight configuration relative to the assessed risk. Clear documentation supports internal review and external examination, though what is sufficient can vary by framework, sector, and jurisdiction, so requirements should be confirmed against the applicable regime.
How can an organization tell whether its human agency controls are actually effective rather than nominal?
Effectiveness is often assessed by examining whether oversight roles have the information, authority, time, and competence to act, rather than by confirming that a control exists on paper. Indicators may include how often overrides or escalations occur and are acted upon, whether reviewers can meaningfully understand system outputs, and whether contestation mechanisms are used and resolved. Because these assessments involve judgment and context, organizations typically define evaluation criteria in advance and revisit them as systems and usage evolve.

Common misconceptions

Human agency is the same as human oversight, and any human-in-the-loop arrangement satisfies it.
Oversight is a mechanism that can support human agency, but they are not identical. A human placed nominally in the loop may lack the information, authority, or practical ability to intervene, in which case agency is not genuinely preserved. Agency also extends to the autonomy of affected individuals, not only the role of an operator.
Human agency is a single, uniformly defined legal requirement that applies the same way across all jurisdictions and sectors.
The concept appears in several instruments and frameworks with differing scope and legal status, and its treatment varies by jurisdiction and sector. Whether a given provision is binding law, guidance, or a voluntary standard, and what specifically it requires, depends on the applicable instrument, so it should not be treated as one universal rule.
Maintaining human agency eliminates the risk of harmful automated decisions.
Preserving human agency is a control that can reduce and help manage risk, not one that removes it. Human reviewers can be subject to automation bias, time pressure, or insufficient information, so agency measures need to be designed and monitored to be effective rather than assumed to work.

Best practices

Define the intended level of human involvement for each use case explicitly (for example human-in-the-loop, on-the-loop, or in-command) and document why that level is appropriate given the system's context and potential impact.
Ensure that humans assigned oversight roles have the authority, time, training, and information needed to intervene meaningfully, and test whether they can in fact override or halt the system in practice.
Provide affected individuals with clear information about when and how an AI system is used in decisions that concern them, and establish accessible channels to question or seek human review of outcomes.
Assess and mitigate automation bias and over-reliance by monitoring how often human reviewers defer to system outputs and by building in checks that prompt genuine review rather than rubber-stamping.
Identify the specific framework or instrument governing each deployment and confirm what it requires, rather than applying a single generic notion of human agency across all jurisdictions and sectors.
Treat human agency controls as ongoing measures subject to validation and monitoring, reviewing their effectiveness over time rather than assuming a one-time design decision remains adequate.