AI in Police Investigations: Automated Risk Scores Introduce Bias Into Law Enforcement

Artificial intelligence is becoming an increasingly important part of modern law enforcement. Police departments and investigative agencies can use AI-powered tools to analyse large volumes of records, identify patterns across criminal investigations, compare images, examine digital evidence, and estimate the likelihood of future incidents. These capabilities may help investigators process information more quickly and direct limited resources towards cases that require closer attention. However, when an automated system assigns a person a high risk score or identifies a neighbourhood as a likely crime hotspot, an important question arises: can these predictions introduce bias into law enforcement?

Automated risk scores are not the same as proof of criminal activity. They are statistical estimates generated from selected data, assumptions, and model designs. A system might classify an individual as more likely to reoffend, identify a location as having an elevated risk of crime, or flag a person’s behaviour as suspicious. If police officers treat such assessments as reliable evidence without examining their limitations, an uncertain prediction can influence surveillance, questioning, investigation priorities, or other consequential decisions.

The concern is particularly serious because law enforcement decisions can affect personal liberty, privacy, reputation, and public trust. A biased system may direct more police attention towards communities already subject to intensive monitoring. That additional attention may produce more recorded incidents, which can then reinforce the system’s original assessment. Even if an algorithm does not explicitly use a sensitive personal characteristic, it may rely on other variables that indirectly reflect social and historical inequalities.

At the same time, AI bias is not inevitable in every application. Carefully tested systems can help investigators identify patterns, reduce certain forms of inconsistency, and manage complex evidence. The challenge is to determine when automated risk assessment provides genuine investigative value, when it reproduces existing disadvantages, and what safeguards are necessary to prevent a statistical prediction from becoming an unfair judgement.

The central issue is therefore not simply whether police should use AI. It is whether these systems are sufficiently accurate, transparent, proportionate, and accountable to support decisions that may significantly affect people’s lives.

Understanding Automated Risk Scores in Police Investigations

An automated risk score is a numerical or categorical estimate generated by a model to indicate the likelihood of a particular outcome. In policing, that outcome might involve the probability of an incident occurring in a particular area, the likelihood that an individual will be involved in a future event, or the priority that an investigation should receive. The meaning of a score depends on the model’s purpose, its training data, and the way the organisation interprets the result.

Predictive policing systems commonly analyse historical incident reports, geographical information, and patterns in recorded activity to estimate where certain incidents may occur. Individual risk assessment tools, by contrast, may examine previous records and other legally permissible information to estimate a specific outcome. AI tools can also assist with facial recognition, digital forensics, evidence review, and the identification of connections between people, locations, and events. These applications are different from one another, and their risks should not be treated as identical.

For example, a system that predicts increased demand for police services in a commercial district may help with resource planning. A system that labels a named individual as a high-risk suspect can have much more serious consequences, particularly if officers use the score to justify intrusive monitoring or treat it as evidence of guilt.

The Organisation for Economic Co-operation and Development has highlighted inadequate or skewed data, privacy risks, lack of transparency, and excessive reliance on automated recommendations as significant concerns in AI-supported law enforcement. It also emphasises that algorithms estimate the likelihood of events rather than establishing what will happen or proving that a particular person has committed an offence.

This distinction must remain central to the use of risk scores. A model can help identify a question that deserves investigation, but the answer must be established through reliable evidence and lawful investigative procedures.

How Bias Can Enter AI-Powered Policing

Historical Data Can Reproduce Existing Inequalities

One of the most important sources of algorithmic bias is historical data. AI models learn patterns from the information used to develop and train them. If that information reflects unequal enforcement, inconsistent reporting, or historical discrimination, the model may reproduce those patterns even when its developers do not deliberately design it to discriminate.

Consider two neighbourhoods with similar levels of underlying unlawful activity but different histories of police deployment. If officers have traditionally patrolled one neighbourhood more frequently, they may have recorded more incidents there. An algorithm trained on those records could interpret the greater number of reports as evidence that the neighbourhood presents a higher risk. It may then recommend additional patrols, leading to even more recorded incidents.

This creates a feedback loop in which the system reinforces the conditions reflected in its own data. The resulting prediction may appear statistically justified because it accurately reflects historical police records, yet those records may not provide a complete or neutral picture of actual crime.

The problem is not limited to geographical predictions. An individual risk assessment model trained on historical arrests or convictions may learn from patterns influenced by differences in surveillance, access to legal representation, reporting practices, and enforcement priorities. If these factors are not properly understood, the system may treat a history of recorded contact with law enforcement as a stronger indication of future wrongdoing than the evidence warrants.

Research and policy analysis on algorithmic decision-making in criminal justice have repeatedly identified the risk that historical disparities can be reinforced when systems rely on unrepresentative or biased data. The United Kingdom’s Centre for Data Ethics and Innovation, for example, has highlighted how predictive tools may entrench existing patterns of over-policing when historical police data is used without adequate safeguards.

Proxy Variables Can Create Indirect Discrimination

Removing sensitive personal attributes from an algorithm does not necessarily eliminate bias. A model may rely on variables that indirectly reflect characteristics such as socioeconomic circumstances, ethnicity, or the level of surveillance a person experiences. These variables are sometimes called proxies because they can carry information associated with attributes that are not explicitly included in the model.

For example, a system might use residential location, previous police contacts, employment instability, or patterns of recorded activity. Some of these variables may be relevant for a narrowly defined purpose, but they can also reflect circumstances unrelated to an individual’s actual involvement in crime. If the model gives them excessive weight, people facing similar situations may receive different assessments because of the environments in which they live or the extent to which their activities have been recorded.

This does not mean that every socioeconomic or geographical variable must automatically be excluded. Investigators and developers need to examine why a variable is being used, whether it is genuinely relevant to the intended prediction, and what consequences follow from its inclusion. A model’s statistical relationship between a variable and a recorded outcome does not, by itself, establish that the variable provides a fair or causally meaningful basis for a law enforcement decision.

Facial Recognition and Identity Errors

AI-powered facial recognition can help investigators compare an image with a database of known identities. However, a match generated by the software is a candidate identification, not definitive proof that the person shown in an image is the individual being investigated.

Errors can arise from poor lighting, low-resolution CCTV footage, unusual camera angles, image compression, or differences between the image being searched and the photographs stored in the database. Performance may also vary across demographic groups and across different systems. The consequences can be severe if investigators treat a weak match as sufficient evidence to focus on an innocent person.

The OECD has documented concerns about inaccurate facial recognition and inappropriate investigative procedures, including the use of poor-quality images and other unsuitable inputs. It stresses the importance of clear rules governing the technology and of distinguishing a software match from sufficient grounds for an arrest.

These risks demonstrate why identity verification should involve independent evidence. Investigators should establish whether the person matches other details in the case, examine the quality of the original image, and follow applicable procedures for corroboration before taking consequential action.

Automation Bias Among Investigators

Bias can also arise from the way humans interpret AI outputs. Investigators may assume that a numerical score is more objective than their own judgement because it appears to be based on scientific analysis. This tendency, often called automation bias, can cause people to place excessive confidence in a system even when its reasoning is unclear or its evidence is incomplete.

An officer who receives a high-risk alert may begin to interpret subsequent information as confirmation of the system’s assessment. Neutral behaviour might be interpreted as suspicious, contradictory evidence may receive less attention, and investigators may concentrate on supporting the initial hypothesis rather than considering alternative explanations.

This problem is not unique to AI. Human investigators can also be influenced by stereotypes, initial suspicions, and confirmation bias. However, automated systems can give these assumptions an appearance of technical authority, making them harder to challenge. Effective human oversight must therefore involve critical evaluation of the recommendation, not merely approval of a computer-generated result.

The College of Policing in the United Kingdom advises investigators to remain impartial when gathering and evaluating evidence, to consider the assumptions they make, and to record decisions with clear supporting reasons. Its guidance notes that an initial belief about a suspect’s guilt can influence the interpretation of later evidence.

Can Automated Risk Scores Affect Investigative Fairness?

The most significant concern arises when a risk score influences how police treat an individual before sufficient evidence has established a connection to an offence. A person classified as high risk may face more surveillance, repeated questioning, or closer scrutiny than someone with a lower score. If the system’s assessment is inaccurate or biased, the individual may experience consequences without having committed any wrongdoing.

A risk score may also shape the direction of an investigation. Officers who are repeatedly directed towards a particular person or neighbourhood may spend less time exploring alternative leads. Witness statements and physical evidence can then be interpreted through the assumption that the model has already identified the most likely explanation. This can reduce the range of possibilities considered and make it more difficult to recognise evidence that contradicts the initial assessment.

The problem becomes particularly serious when the system’s operation is difficult to explain. Some AI models are proprietary or technically complex, limiting an investigator’s ability to understand why a person received a particular score. If an affected individual cannot meaningfully challenge the basis of an assessment, accountability and procedural fairness may be weakened.

The United States Department of Justice’s 2024 report on artificial intelligence and criminal justice identifies risks associated with inaccurate assessments, biased data, privacy, and the possibility that AI systems may contribute to mistaken arrests or disproportionate effects on particular communities. It also notes that risk assessment tools require appropriate validation and careful interpretation.

It is important, however, to distinguish between a tool that helps organise investigative information and one that determines how an individual is treated. The legal and ethical concerns depend on the system’s intended purpose, the decisions it influences, the quality of supporting evidence, and the protections available to the people affected by it.

The Difference Between Prediction and Evidence

A fundamental principle of a fair investigation is that a person should not be treated as guilty simply because a system predicts an elevated risk. Risk assessment estimates the likelihood of a future or uncertain outcome, whereas evidence is used to establish the facts of a particular case. Confusing these two functions can turn a statistical estimate into an unjustified presumption of wrongdoing.

Suppose an AI system assigns a person a high score because of patterns associated with previous cases. That score does not prove that the person committed the offence currently under investigation. It may indicate a statistical association, but the association could be weak, outdated, affected by biased records, or irrelevant to the specific circumstances. The score must therefore be evaluated alongside independently obtained evidence.

A responsible investigative process should ask whether the information is reliable, whether the system was validated for the relevant population and purpose, and whether alternative explanations have been considered. Investigators should also be able to explain what the score means, what its limitations are, and why any resulting action is proportionate.

This distinction matters because AI can produce both false positives and false negatives. A false positive occurs when a system incorrectly flags someone or something as presenting a risk. A false negative occurs when the system fails to identify a case that does present the relevant risk. Focusing only on overall accuracy can obscure these errors, especially if they affect different groups at different rates.

For law enforcement, the cost of an error cannot be measured solely in technical terms. A false positive may expose an innocent person to suspicion or intrusive investigation, while a false negative may cause investigators to overlook a genuine threat. Evaluating an AI tool therefore requires an assessment of its predictive performance, the severity of possible mistakes, and the safeguards available when its output is wrong.

What Legal and Ethical Safeguards Are Needed?

Independent Testing Before Deployment

Police agencies should not assume that a tool is suitable simply because a vendor describes it as accurate or effective. Before deployment, the system should be independently tested using data that reflects the conditions in which it will actually be used. Evaluation should examine the model’s intended purpose, data quality, error rates, limitations, and performance across relevant groups.

Testing should also assess whether the tool performs better than existing investigative methods. A complex system that does not provide a meaningful improvement over conventional analysis may introduce additional costs, privacy risks, and opportunities for error without delivering a clear public-safety benefit.

The European Parliament’s research service has highlighted rigorous pre-deployment testing, ongoing bias assessment, documentation, transparency, and human evaluation as important measures for responsible AI use in law enforcement. It also notes that fairness must be assessed in context rather than assumed from a single universal metric.

Meaningful Human Oversight

Human oversight should be more than a requirement to have an officer approve an automated recommendation. Officers need sufficient training to understand how the system works, what the score represents, and which conditions may cause it to fail. They must also have access to relevant supporting information and the authority to reject the output when it conflicts with the evidence.

For consequential decisions, the reviewer should independently examine the facts rather than merely accepting the system’s conclusion. If a high-risk score is used to guide an investigation, the officer should record the independent evidence that supports the next step and explain why the action is necessary and proportionate.

Human oversight should also include a mechanism for reconsideration. When new evidence contradicts the model’s assessment, investigators must be willing to revise their approach rather than continuing to follow the original prediction.

Transparency and Accountability

Law enforcement agencies should maintain clear records of which AI tools they use, what those tools are intended to do, and how their outputs affect investigative decisions. Documentation should include relevant limitations, validation results, error reports, and procedures for reviewing disputed outcomes.

Transparency does not necessarily require publishing every technical detail that could compromise a legitimate investigation or expose sensitive security information. It does, however, require sufficient disclosure for appropriate oversight, legal scrutiny, and public accountability. Where an AI system materially contributes to an investigative decision or forms part of the evidence relied upon in a case, applicable disclosure obligations and the rights of the accused must be respected.

Clear responsibility is equally important. Agencies should identify who authorises the use of a tool, who reviews its outputs, who monitors its performance, and who responds when an error is discovered. A technology vendor’s involvement should not become a reason for an agency to avoid explaining or defending its own investigative decisions.

Continuous Monitoring and the Right to Challenge Errors

A model that performs well during initial testing may become less reliable as data patterns, criminal behaviour, technology, or policing practices change. Regular monitoring is therefore necessary to identify deteriorating performance, unexpected disparities, and recurring false positives. Agencies should suspend or restrict a system when its risks cannot be adequately controlled.

Individuals affected by AI-supported decisions should also have appropriate routes to challenge errors. Depending on the legal framework and circumstances, this may involve requesting a review, correcting inaccurate records, challenging the reliability of evidence, or seeking judicial scrutiny. The precise rights and procedures vary by jurisdiction, but a meaningful review process should not depend solely on asking the same automated system to reconsider its original conclusion.

AI in Police Investigations: The Indian Context

In India, AI-supported policing must be considered alongside constitutional protections, criminal procedure, evidence requirements, privacy concerns, and the applicable rules governing the particular technology and decision. AI adoption does not remove the obligation to conduct lawful investigations or establish allegations through relevant evidence. Nor does an automated risk score independently establish criminal guilt.

Potential applications include analysing digital evidence, processing large collections of records, supporting identification, detecting patterns across cases, and helping agencies allocate resources. These capabilities may be useful in investigations involving cybercrime, financial fraud, missing persons, and other complex cases. However, the value of each application depends on the quality of its data, its suitability for the intended purpose, and the safeguards surrounding its use.

If an automated system flags a person because of an inaccurate identity match or a pattern drawn from unreliable records, investigators should verify the information before taking consequential action. Where an AI-generated report or analytical result becomes relevant to a legal proceeding, questions about its reliability, provenance, disclosure, and evidentiary status must be addressed under the applicable legal framework. The fact that a computer generated a result does not automatically make it accurate or legally sufficient.

Public accountability is also important. Clear policies can help explain which systems are being used, what decisions they support, how errors are reported, and how performance is independently assessed. Appropriate oversight can build public confidence while helping agencies identify whether a technology is improving investigative outcomes or simply reproducing existing patterns.

The broader objective should be to use AI as an investigative aid rather than as a substitute for lawful judgement. The technology may help officers identify leads, but it should not become a shortcut for deciding who deserves suspicion.

Can AI Make Policing Fairer?

Although algorithmic bias presents serious risks, AI should not automatically be considered less fair than human decision-making. Human investigators can be influenced by prejudice, fatigue, incomplete information, and inconsistent judgement. A properly designed and independently evaluated system may help identify overlooked connections, apply consistent analytical procedures, or reveal patterns that would otherwise remain difficult to detect.

AI can also support auditing by helping agencies examine whether investigative attention, error rates, or outcomes differ across locations and demographic groups. When used carefully, this analysis may expose disparities that would be harder to identify through manual review alone. The technology can therefore contribute to fairness if it is used to question existing practices rather than simply automate them.

However, a model does not become fair merely because it treats every input according to the same mathematical formula. If the underlying data is incomplete or the chosen outcome reflects unequal historical practices, consistent calculations may reproduce inconsistent treatment. Fairness must be evaluated through evidence about the system’s actual performance and consequences.

There are also trade-offs between different measures of model performance. Improving one error rate does not necessarily improve every other measure of fairness, and removing sensitive variables does not guarantee that a model will stop producing unequal outcomes. Agencies must consider the purpose of the tool, the risks associated with its mistakes, the groups affected, and the legal standards that apply.

The most responsible approach is therefore neither unconditional adoption nor automatic rejection. It is careful evaluation of whether a particular tool provides a demonstrated benefit, whether its risks can be controlled, and whether its use remains compatible with individual rights and public accountability.

Conclusion

AI-powered risk scores can introduce bias into law enforcement when they rely on incomplete historical data, reproduce existing patterns of unequal policing, use misleading proxy variables, or encourage investigators to trust automated recommendations without sufficient scrutiny. Facial recognition errors, opaque prediction models, and feedback loops can further increase the risk that innocent people or already over-policed communities will receive disproportionate attention.

Nevertheless, these risks do not mean that every AI system used in police investigations is inherently biased or ineffective. Carefully validated tools can help investigators process complex information and identify useful leads. Their value depends on whether they improve real investigative outcomes without creating unacceptable risks to fairness, privacy, and individual liberty.

Effective safeguards must include independent testing, continuous monitoring, meaningful human oversight, transparent documentation, and procedures for challenging incorrect assessments. Most importantly, an automated score must remain a tool for evaluating information rather than a substitute for evidence.

The future of AI in policing should be measured not by how many predictions a system generates, but by whether those predictions help investigations become more accurate, fair, and accountable. AI can support law enforcement, but it should never turn statistical suspicion into a presumption of guilt.

Online Internship with Certificate