What Happened
In January 2025, South Africa's High Court declared the automated vetting system for the Social Relief of Distress grant unconstitutional. This system, which transitioned to fully digital, online-only applications in April 2022, used automated bank account checks and government database queries to determine eligibility for the R370 monthly grant. The number of successful applicants dropped from 16 million at the program's 2020 inception to 5.6 million by 2022.
The Institute for Economic Justice and #PayTheGrants challenged the regulations authorizing this automated verification, arguing they violated constitutional obligations to progressively realize socioeconomic rights. The court agreed, invalidating key regulations. The government has appealed, and the case now moves to the Supreme Court of Appeal.
Timeline
May 2020: Social Relief of Distress grant launches digitally under the Disaster Management Act, reaching 16 million beneficiaries.
April 2022: New regulations shift the grant to the Social Assistance Act. Applications become online-only. Automated decision-making systems begin treating algorithmic verdicts as definitive in many cases.
By 2022: Successful applicants fall to 5.6 million. Government reduces grant budget by billions of rands.
July 2023: Institute for Economic Justice files court proceedings contesting the digitized process.
January 2025: High Court declares regulations authorizing automated bank verification and database checks unconstitutional and invalid.
Current: Government appeals. Supreme Court of Appeal hearing pending.
Which Controls Failed or Were Missing
The system exhibited three critical control failures that mirror common AI governance gaps:
Inadequate data quality controls. The automated system relied on government databases that were error-ridden and outdated. A single bank deposit triggered an income flag, regardless of whether that deposit represented a temporary loan or money held on behalf of someone else. The system couldn't distinguish between regular income and one-time transactions.
Missing human review mechanisms. The regulations treated algorithmic outputs as definitive. When the system flagged an applicant as ineligible, there was no meaningful human intervention to verify the decision before denial. The appeal process was also online-only, creating a closed loop where algorithmic errors couldn't be challenged effectively.
Absent fairness and equity assessments. No evidence suggests the agency conducted impact assessments before deployment or monitored for discriminatory outcomes during operation. The drastic reduction from 16 million to 5.6 million beneficiaries should have triggered immediate review. Instead, it enabled budget cuts.
What the Relevant Standards Require
ISO/IEC 42001's AI Management System framework requires organizations to identify and assess risks to individuals and society. For a system affecting constitutional rights, Section 6.1.3 demands that you identify risks related to fundamental rights and establish controls proportionate to those risks.
ISO/IEC 23894 specifies that data quality risks must be assessed throughout the AI system lifecycle. Your data sources need documented provenance, accuracy validation, and regular quality checks. When your system makes eligibility determinations affecting millions, you can't treat outdated government databases as ground truth without verification.
The EU AI Act's Article 9 (risk management system) and Article 14 (human oversight) provide relevant parallels, even outside EU jurisdiction. Article 9 requires that high-risk systems undergo testing to identify discriminatory impacts. Article 14 mandates human oversight measures, including the ability to override automated decisions when they affect fundamental rights.
NIST AI RMF calls for regular impact assessments and monitoring for unintended consequences. The Map function requires you to understand your system's context, including the population it serves and their access to digital infrastructure.
For public sector deployments affecting constitutional rights, you need:
Pre-deployment impact assessment. Document how the system could exclude eligible recipients. Test against representative edge cases: temporary deposits, informal income, database errors.
Ongoing outcome monitoring. Track approval rates by demographic group and geography. A 65% reduction in beneficiaries demands investigation, not celebration.
Human review for adverse decisions. When your algorithm denies someone's application, a qualified human must review the decision with access to the underlying data and reasoning.
Accessible appeals process. Online-only appeals exclude the digitally marginalized. You need offline channels with human decision-makers.
Data quality validation. Audit your source databases for accuracy and currency. Establish thresholds: if a database is more than X months old or has Y% error rate, flag all decisions relying on it for manual review.
Lessons and Action Items for Your Team
If you're deploying automated decision-making in high-stakes contexts, this case offers four concrete directives:
Map your failure modes to constitutional or regulatory obligations. Don't wait for a court to tell you that your system violates rights. Before deployment, document which rights your system touches and how errors could infringe them. For each failure mode, define the control that prevents it.
Treat drastic outcome shifts as red flags, not efficiencies. When your system's approval rate drops significantly, investigate immediately. A budget-friendly algorithm may be a discriminatory one. Establish baseline metrics before deployment and define acceptable variance bands.
Never make automated decisions definitive without human oversight. Your system can flag, recommend, or prioritize. It shouldn't be the final authority on life-altering decisions. Build human review into your process architecture, not as an afterthought appeal mechanism.
Audit your data sources like you audit your models. The South African system failed because its databases were unreliable. Your model validation process must include data provenance documentation, accuracy testing, and refresh cadences. If you can't verify your data quality, you can't trust your outputs.
This case matters because it's not unique. As automated systems spread into education, healthcare, and law enforcement, the same control gaps will produce the same failures. The difference is whether you identify and fix them before or after a court does.
