Tracking FLOPs for EU AI Act Compliance

Scope of This Guide

This guide helps your team implement floating-point operations (FLOPs) tracking during large language model fine-tuning to comply with the EU AI Act. You'll learn when tracking is necessary, how thresholds work, the role of automated tools, and where manual processes can introduce risk.

The focus is on organizations fine-tuning General-Purpose AI Models on Amazon SageMaker, though the threshold logic and compliance framework apply to any infrastructure.

Key Concepts and Definitions

FLOPs (Floating-Point Operations): This computational metric determines whether your fine-tuning activity reclassifies you from a downstream user to a General-Purpose AI Model provider under the EU AI Act. One FLOP represents a single arithmetic operation on floating-point numbers.

Downstream User: An organization using an existing model without substantial modification, facing limited obligations under the EU AI Act.

General-Purpose AI Model Provider: An organization legally responsible for a model's compliance. You must deliver Technical Documentation (Annex IV), disclose training data sources, and demonstrate EU copyright compliance.

The One-Third Rule: If your fine-tuning consumes more than 30% of the original pretraining compute, you've made substantial modifications. This threshold typically indicates significant behavioral changes, triggering full provider obligations.

Architecture-Based FLOPs: The primary compliance metric calculated using model parameters and training tokens.

Hardware-Based FLOPs: An upper-bound estimate using GPU utilization monitoring for conservative reporting.

Requirements Breakdown

The EU AI Act establishes three threshold scenarios based on your base model's pretraining compute. Determine which applies before you start tracking.

Scenario	Pretraining Compute	Threshold	Classification Trigger
Known, High-Compute Base	≥ 10²³ FLOPs	30% of actual pretraining	Relative to base model
Unknown or Low-Compute Base	< 10²³ FLOPs or undisclosed	3.3×10²² FLOPs	Fixed default
Systemic Risk Models	≥ 10²⁵ FLOPs	3.3×10²⁴ FLOPs	Fixed high threshold

Determining Your Scenario: Most organizations fall under scenario 2 because foundation model providers rarely publish exact training FLOPs. Without documented pretraining compute, apply the default threshold of 3.3×10²² FLOPs.

Triggering Provider Obligations: Exceeding your applicable threshold means you must provide detailed architecture disclosures, maintain a public list of training data sources, and demonstrate EU copyright compliance. Non-compliance can result in fines up to €15 million or 3% of global annual turnover.

Implementation Guidance

Pre-Training Estimation

Before launching a fine-tuning job, estimate expected FLOPs across training methods. Compare full fine-tuning against parameter-efficient approaches like Low-Rank Adaptation (LoRA) or Spectrum. This helps you gauge whether your training configuration will approach threshold limits.

Runtime Tracking

The architecture-based formula for parameter-efficient fine-tuning is:

F_ft = (4 × N_total + 2 × N_trainable) × tokens_processed

Where:

• 4 × N_total accounts for forward pass and backward gradient computation through all layers, including frozen ones.
• 2 × N_trainable accounts for gradient computation with respect to trainable weights only.

For full fine-tuning where all parameters are trainable, this reduces to 6 × N × D, matching the EU AI Act's standard formula for dense transformers.

Threshold Comparison Logic

Your tracking system should implement this decision tree:

1. Do you know pretraining FLOPs?

   • No → Use default threshold 3.3×10²²
   • Yes → Continue to step 2

2. Is pretraining compute ≥ 10²⁵ FLOPs?

   • Yes → Systemic risk threshold: 3.3×10²⁴
   • No → Continue to step 3

3. Is pretraining compute ≥ 10²³ FLOPs?

   • Yes → Relative threshold: 30% of actual pretraining
   • No → Default threshold: 3.3×10²²

Audit Trail Requirements

Your compliance documentation must capture:

• Total FLOPs consumed (architecture-based calculation)
• Hardware-based upper bound (for conservative reporting)
• Applicable threshold and determination logic
• Training configuration (model name, parameter counts, token counts)
• Timestamp and job identifier
• Threshold comparison result (exceeded/not exceeded)

Store these records in a structured format (JSON) with persistent storage. You'll need them for regulatory review.

Common Pitfalls

Using Hardware Metrics as Your Primary Compliance Number: The architecture-based calculation is your official metric. Hardware-based monitoring provides an upper bound for conservative reporting but isn't the compliance standard.

Forgetting Parameter-Efficient Methods: If you use LoRA or Spectrum and apply the standard 6 × P × D formula, you'll overestimate your FLOPs consumption. The enhanced formula accurately reflects that fewer parameters receive gradient updates.

Assuming You're a Downstream User Without Checking: Don't rely on intuition. A modest-looking fine-tuning job on a large base model can exceed thresholds quickly. Track every job.

Losing Audit Trail Data: Compliance reviews can happen months or years after training. Don't rely on ephemeral logs or local storage. Persist your FLOPs records to Amazon S3 or DynamoDB immediately after each job completes.

Ignoring the Systemic Risk Threshold: If you're fine-tuning a model pretrained with ≥ 10²⁵ FLOPs, you face a much higher threshold (3.3×10²⁴). Don't apply the default 3.3×10²² by mistake.

Manual Calculation Errors: FLOPs formulas differ by training method, and token counting must exclude padding. Manual spreadsheet tracking introduces error. Automate the calculation within your training pipeline.

Quick Reference Table

Task	Method	Tool/Code
Determine scenario	Check if pretraining FLOPs known and compare to 10²³, 10²⁵	Decision tree (see Implementation)
Calculate FLOPs (full fine-tuning)	6 × parameters × tokens	Architecture-based formula
Calculate FLOPs (LoRA/Spectrum)	(4 × N_total + 2 × N_trainable) × tokens	Enhanced formula
Enable tracking in SageMaker	Add compute_flops: true to recipe YAML	Configuration flag
Pass pretraining compute	Set PRETRAIN_FLOPS environment variable	Environment variable
Access audit records	Retrieve JSON from S3 or DynamoDB	Post-training storage
Conservative upper bound	N_gpus × duration × peak_FLOPs × utilization	Hardware-based (NVML)

IAM Prerequisites: Your SageMaker execution role needs AmazonSageMakerFullAccess and AmazonS3FullAccess. For production, scope these down to specific bucket and model registry resources.

Instance Requirements: Request quota for at least one ml.g5.4xlarge instance (1 × NVIDIA A10G GPU) via Service Quotas console before running training jobs with FLOPs tracking enabled.

By integrating automated FLOPs tracking tools like Amazon SageMaker's Fine-Tuning FLOPs Meter, your team can ensure compliance with the EU AI Act, especially when fine-tuning large language models.