On Dec. 10, just two days before the United Kingdom went to the polls, some 74,000 political advertisements vanished from Facebook’s Ad Library, a website that serves as an archive of political and issue ads run on the platform. For a while, what the company described as a “bug” wiped 40% of all political Facebook ads in the UK from the public record.

Facebook has said it will not fact-check political ads or restrict the ability for campaigns to target people. Instead, it said it will provide transparency with tools like the Ad Library, the Ad Library report, and the Ad Library API, so the public, researchers, and journalists can monitor how elections play out on the platform. But that only works to the degree that those tools operate properly. It was only the news media’s reporting that brought the issue out into the open.

“The fact that they could have an outage like this that went up to the day before an election, and they didn’t really publicly communicate,” Laura Edelson, a computer scientist at NYU whose work involves using the API, told BuzzFeed News, “that’s just not how you treat a security system. That’s what this is — this is a security system for elections.”

A Facebook spokesperson defended the site’s conduct, telling BuzzFeed News, “We are committed to making sure our transparency tools work, especially around elections. As soon as we became aware of a bug in the lead-up to the UK election, our teams worked around the clock to fix it and communicated publicly about our progress until it was fixed.”

In the wake of the failure during the UK elections, Facebook said it had launched a review of how to prevent these issues, as well as how to communicate them more clearly.

But the events of Dec. 10 are not the first time Facebook’s Ad Library has failed since its launch in May 2018. The API, which is supposed to give researchers greater access to data than the library website, went live in March 2019 and ran into trouble within weeks of the European Parliament election in May. Researchers have been documenting a myriad of issues ever since.

The platform also drew the ire of researchers when it failed to deliver the data it promised as part of a partnership with the nonprofit Social Science Research Council and Social Science One, a for-profit initiative run by researchers — a project that was funded by several large US foundations. Facebook said it remains committed to providing data to researchers, but the SSRC and funders have begun withdrawing from the project due to the company’s delays.

In December, the cochairs of Social Science One’s European advisory committee released a public letter expressing frustration with the lack of data and “eternal delays.”

“This is not an acceptable situation for scientific knowledge. It is not an acceptable situation for our societies,” they wrote, highlighting the need for Facebook data in order to investigate the true impact of the platform on society.

The Ad Library API is the most powerful tool available to monitor political ad spending on Facebook. But it too has been a constant source of frustration for researchers. The API delivers data about all social issue, election, and political ads in the system in a way that makes it possible for researchers and journalists to run in-depth analysis on hundreds of thousands of ads at the same time.

Of all the transparency tools made available by the big social media platforms, Facebook’s ads API offers the most comprehensive view of political ads — when it’s working properly.

Edelson said there are “regular delays of up to a full day between when ads start running and when they show up in the archive.”

“That’s why the API was so important; it allowed us to deal with the sheer volume of ads,” Edelson said. In the UK, more than 200,000 issue-based and political ads have been shown to Facebook users since October 2018, according to Facebook’s Ad Library Report. In the US, there have been millions. During the US 2018 midterm elections, the API was not yet functional, so the Ad Library was the only option. And it too experienced problems, according to an analysis by Mozilla.

Mozilla — the nonprofit maker of the Firefox web browser and a leader in the open-source movement for transparency in software — criticized the Ad Library for failing to “provide the kind of data journalists and researchers require for honest oversight.”

The organization ultimately concluded that “Facebook’s tools aren’t very useful today, which means they won’t provide meaningful transparency before the midterm elections.”

Mozilla had planned on creating a tool to help Firefox users understand how they were being tracked and targeted by political advertisers, but because of a lack of data, they weren’t able to implement it, Jason Chuang, a researcher at Mozilla, told BuzzFeed News.

“In any case, Facebook later deployed software to disable our browser extension,” he added.

During the EU parliamentary elections in 2019, the Ad Library API faced various bugs, including an infinite loop — in which the library returned the same results over and over again without stopping. “The Facebook Ad Library API would often fail due to programming errors, and return incomplete data,” Chuang said.

The total number of ads in the library would sometimes fluctuate from minute to minute and “ads that had existed in the archive one day would disappear and could no longer be found on a later day,” according to Chuang.

The many errors and bugs afflicting the Facebook Ad Library API were meticulously documented by researchers at Mozilla during the EU parliamentary election cycle.

“Rather than providing a clear picture of political advertising on the platform, the bugs introduced more uncertainty and cast doubt on whether anything can be trusted,” Chuang said. “We had to constantly question whether the data — and any statistics, analysis, and reporting derived from the data — is complete, or whether the Facebook Ad Library API is buggy today?”

These issues were flagged to Facebook numerous times by researchers. But the platform didn’t fix the Ad Library API until Matthew Rosenberg of the New York Times, published a piece documenting how it was “broken and difficult to use.”

“Within a week of that piece being published, they fixed the infinite loop bug,” Edelson said.

Facebook’s implementation of the ad-monitoring tools came as a reaction to the Cambridge Analytica scandal as well as “a lot of nefarious things going on during the 2016 US election and the 2016 Brexit election,” Rosalynd Southern, a lecturer in political communication at the University of Liverpool, told BuzzFeed News.

By providing a host of new initiatives, including third-party fact-checkers and a suite of ad monitoring tools, Facebook sought to demonstrate that it was fighting electoral interference.

But the series of issues with the Ad Library and the API shows that “it keeps going wrong for them. And I think that this is something that’s going to impact that kind of trust in Facebook,” Southern added.

Edelson said Facebook deserves credit for sharing as much information as it has. But when the company is armed with some of the world’s best programmers, the continued bugginess of the Facebook Ad Library raises some heavy questions about how committed the platform is to the mission of transparency and how effective it is at delivering on its promises.

“There’s no transparency unless we actually have the data. Delivering a working API — a usable piece of software — is only the first step toward transparency. Such a task shouldn’t be hard for a tech company like Facebook,” Chuang said.

“This is to us — to you, to me, to a lot of people — this is a critical tool for how we’re monitoring election interference,” Edelson said. “This is a pretty vital security system for our election. And clearly to Facebook, it isn’t.”

Rory Smith is a Research Manager at First Draft. Email him at [email protected]