Flock says its new tool will help identify police abuse, but hasn’t explained how it works

Some official comments provide some understanding of what Audit Assistance can do; it gets vague when trying to determine how it does it.

For instance, Flock’s head of trust and compliance Ashley Haber said in a video that the Audit Assistance can flag “what may look like an odd search history from a specific user,” helping customers understand how their users are operating the system. In another video, Flock co-founder Paige Todd said “Audit Assistance actually flags unusual patterns early. Think a user searching the same plate repetitively for more than 30 days.”

And Major Patrick Krieg of the Dunwoody, Georgia Police Department, a Flock customer, described the feature in a press release as “an algorithm that has notified us of any type of bias.”

Flock describes Audit Assistance as a feature that can “surface atypical activity early and review it quickly with a clear, documented workflow.”

What’s unclear — and TechCrunch has asked for answers about — is how this “algorithm” actually works, what data is it trained on, and if it was developed using an large language model. TechCrunch also asked what kind of patterns is it designed to surface and what exactly gets flagged as “abnormal activity”? 

According to Flock, Audit Assistance is not based on or AI. The company told TechCrunch that “It is a data tool that flags atypical search patterns for further review. One example that would trigger a flag is if a user searches for the same plate using multiple different case codes, as this could potentially indicate abuse. A flag does not mean that it is definitely abuse — it only means it warrants further investigation.” 

Numerous questions remain unanswered, however, including whether Flock has any statistics or data that shows how effective the tool is, what its false positive and flase negative rates are, and if it has been audited by a third party?

Privacy experts and Flock critics are not impressed by Audit Assistance. 

The American Civil Liberties Union senior policy counsel Chad Marlow wrote in response to Flock’s announcement that “there is no evidence that the tool works consistently” to address abuse. 

“While dozens of officers have recently been arrested, fired, or otherwise disciplined for misusing Flock for personal reasons, Flock claims these arrests are proof its auditing tool works,” Marlow wrote. “However, unless we know the number of officers misusing the system, we cannot conclude if Flock and its auditing tools are catching 95% of violators or 5%. Flock needs to have its auditing tool analyzed by an independent evaluator to determine its actual effectiveness. Until then, we don’t know if the tool is a real security measure or just window dressing.”

Cooper Quintin, a security researcher and senior public interest technologist at the Electronic Frontier Foundation, told TechCrunch that it may not even matter whether Audit Assistance works. 

“If it does work, law enforcement will figure out ways to get around it,” he said, adding that what matters in the end is whether abusers are caught and held accountable. “If there are no consequences for abuse and whoever reports to is the same police agency that is doing the abuse, it’s just a fig leaf for Flock. The best way to ensure accountability is to pass laws constraining the use of this technology and requiring a warrant for its use.”

Rachel Levinson-Waldman, the director of the Brennan Center’s Liberty and National Security Program, told TechCrunch that making Audit Assistance mandatory is a step in the right direction, and that “in theory” the tool is “a great idea.” But she also said the tool itself “needs to be audited” because there’s not enough information about how it works in practice.

Flock’s announcements come after years of reported incidents of its customers repeatedly abusing its technology. Earlier this week, three former Georgia sheriff’s deputies were arrested and accused of using Flock cameras to stalk people with whom they had a “personal relationship at the time,” as the Bibb County Sheriff’s Office said in a press release. In this case, the Bibb County sheriff David Davis specifically called out Flock’s new tool Audit Assistance for uncovering the misuse.