Global/UK: Artificial Intelligence Summit must not ignore severe rights harms of fraud-detection tech

Ahead of the UK’s AI Safety Summit that begins on Wednesday November 1, Damini Satija, Head of the Algorithmic Accountability Lab at Amnesty International said: 

“The UK’s Global Summit on AI safety must set the right example in the global race to regulate AI, and not be led by speculative discourses on ‘existential threats’ of these technologies, at the expense of those experiencing the sharpest human rights impacts of AI systems today.  

Where is the government’s acknowledgement of these harms of AI and where is the attention to regulating and protecting the most marginalised communities? The AI Summit’s agenda clearly favours interests of the tech industry, and this is concerning as the use of such tech tends to act against marginalised groups.  

Damini Satija, Head of the Algorithmic Accountability Lab

“Last week Prime Minister Rishi Sunak promoted the usage of AI tools to ‘clamp down on benefit fraudsters’ as a positive technological development, completely neglecting research which shows the discriminatory nature and frequent misuse of such systems. 

“Where is the government’s acknowledgement of these harms of AI and where is the attention to regulating and protecting the most marginalised communities? The AI Summit’s agenda clearly favours interests of the tech industry, and this is concerning as the use of such tech tends to act against marginalised groups.  

“In a previous experimentation of such tech in the UK, the Department of Work and Pensions targeted disabled individuals unfairly, subjecting them to benefits fraud investigations, pushing them into an endless bureaucratic maze, with no transparency or justice in sight. 

“In 2021, Amnesty International’s research exposed the devastating consequences of a discriminatory AI system used by the Dutch Tax Authorities to detect fraudulent applications for child-benefits, and its racialised impacts on the right to social security. Four years on, impacted individuals are still waiting for justice and accountability. 

“States tend to implement AI systems as ‘technical fixes’ to societal problems, but these are often in reality cost-cutting measures which exacerbate punitive policies against marginalised people. 

“Commitments made at the Summit must not focus on alarmist narratives around AI, but rather move us closer towards regulation which ensures that human rights and the voices of impacted communities are centered in our protections against AI’s harms.”  

Background:

Amnesty International in collaboration with over a 100 civil society organisations has urged the UK government to keep the communities and workers most affected by AI at the forefront of AI decision-making. 

Amnesty International, as part of a coalition of civil society organizations led by the European Digital Rights Network (EDRi), has been calling for EU artificial intelligence regulation that protects and promotes human rights, including rights of people on the move.