Unreadable files, wrongful redactions, outright refusal: The maddening responses to our FOI requests

A little over a year ago, my reporting partner, Robyn Doolittle, and I sat down to file 253 freedom of information requests.

This was our first salvo for Secret Canada, The Globe and Mail’s effort to examine Canada’s broken FOI systems. We filed the same request to every federal, provincial and territorial ministry and department in the country. Our request read:

We are conducting a study of how freedom of information requests are processed across the country. To that end, we are requesting information relating to how these requests are dealt with by your office. Please provide us with an electronic (that is, machine-readable) record of all requests received between January 1, 2021 and end of day December 31, 2021, whether completed or not. Please include fields containing the following information: request number, request summary, type of requester (academic, media, etc.), whether the request is sensitive or contentious, date received, date request completed, disclosure decision, reason for time extension, extension duration in days, exemptions applied, whether it was subject to an appeal, and number of pages disclosed. We recognize your system may not capture all of these fields. If thats the case, please give us the fields you do have. Please provide in an electronic format such as Excel or delimited text (e.g. CSV). Do not provide PDF or image files.

We ultimately filed another 192 requests to other public bodies to broaden the scope of the project and collect more information on crucial public bodies such as municipalities, police services, transit systems, government-run utility and power companies, school boards and hospitals.

The responses we received were all over the place. For instance: According to our FOI request tracking spreadsheet, roughly 53 per cent of institutions responded within their legislated deadline, meaning the other 47 per cent replied outside their legal window to do so – effectively breaking the law.

Other patterns emerged as we wrangled with governments for their FOI data. Below are a few of the highlights.

Spreadsheets take a back seat

Many, many public bodies insisted on sending us PDFs, despite our explicit request for files we could easily open and select from on a computer (“machine-readable,” in other words) in Excel or some other format. In many maddening cases, the files we were sent had clearly started as a spreadsheet and then been converted into a PDF.

Most FOI offices convert their responses into image-only PDFs (that is, with no selectable text) during the redaction process. While PDF documents are useful if a person has requested a memo, report, or e-mail chain, they are a poor format for data releases. The PDF-ification process destroys the information in spreadsheet files, turning the data into strings of letters and numbers that then has to be tediously converted back into spreadsheet form, often by hand.

Jurisdictions were also inconsistent. While many offices insisted they were not allowed to provide us with raw data files, other public bodies gladly sent us spreadsheet files. In many cases, we had to push offices to re-release the data in spreadsheet form, further complicating an already tiresome process.

Redaction by subtraction

Ontario’s Ministry of Transportation charged us $60 to provide information in PDF form, but the summary texts we received were cut off after the first 10 words or so. I sent the ministry multiple e-mails about this, but the FOI officer told us their system, which ran on software called Nordat, didn’t allow them to export the raw data.

“In order to provide you with the full text of each request, we would need to manually create a record,” the officer told me. “Under the Act, we are not obligated to create a record.”

In one very similar case, we managed to get some data, however. The London Police Service, which also uses Nordat, got the company behind the software to write custom code to export their data. We asked the Ministry of Transportation whether London police’s solution would work in this case, but they said that the London police’s solution was not compatible with the government’s version of the software. The ministry added that it’s now moved to a new tracking system using SharePoint, a Microsoft platform.

An unreadable file

Saskatchewan’s Ministry of Justice sent us a completely unreadable file, then took 11 months to send us a usable copy.

The ministry responded roughly within their time limit in June, 2022, but the file they sent me, an 11-page PDF of what seems to originally have been an Excel spreadsheet, had been saved at such a low resolution the text was unreadable. I began e-mailing the ministry every so often to ask them for a higher-quality file, but heard nothing of substance until late January. A week later, I had a Microsoft Teams meeting with the FOI officers where I shared my screen, opened the file and zoomed in to show them the file’s poor quality. On May 8, I finally received a new, higher-resolution PDF.

Mis-redaction

One public body sent us a file which, at first glance, looked to be redacted, with boxes drawn on top of excluded text. But during the data cleaning and ingestion process for our searchable database, we learned the redacted portions hadn’t been redacted at all; one could simply copy the text under the box, revealing people’s names. (Interestingly, many of the requesters were filing on behalf of their businesses. The public body not only redacted people’s names, but the business names, too – which is likely not allowed under that jurisdiction’s laws for personal information redactions.)

I called the body’s FOI office to let them know the file hadn’t been redacted properly and they said they’d send a new file we could use. They never did, so we ultimately did the redaction ourselves.

Tracking systems

We encountered many tracking systems during our work: AccessPro, Nordat and ATIPXpress, to name a few. While most bodies had some kind of software for tracking their FOI requests, others took a more bespoke approach.

Ontario’s Ministry of Economic Development, Job Creation and Trade seems to use a table within a Microsoft Word document, for instance, and the province’s Ministry of Tourism, Culture and Sport uses Excel.

Others said they had no tracking system whatsoever, including the City of Charlottetown, the Saint John Police Force and the Delta Police and Saanich Police departments, both in B.C. In those cases, we constructed the data ourselves by asking for acknowledgement and response letters from the public bodies. Those are official letters sent by the bodies at the beginning and end of the process: The first restates the request text and confirms that an FOI request has been made; the second, sent out during the final stage of the process, outlines what is being sent to the requester.

Refusing to respond

A handful of bodies simply refused to respond to our requests.

Both the municipality and police force for Lévis, Que. told us they weren’t able to extract the information and closed our request. The police force in Saint-Jean-sur-Richelieu, also in Quebec, told us they were not required to create a record and closed our request. The Calgary Police Service, meanwhile, told us that they no longer used a tracking system and couldn’t generate a report with the information we’d requested.

Several didn’t respond to our request or follow-up messages. These include the Winnipeg School Division, the Ottawa Police Service, and the Quebec police services for Châteauguay, Repentigny, Richelieu-Saint-Laurent, Roussillon and Terrebonne. Those are all violations of freedom of information laws in those jurisdictions.

The University of Saskatchewan, Saskatoon Public Schools and Regina Public Schools told us they’d ignored our mailed request because it hadn’t been made using their form. The Toronto District School Board also insisted we use their online form. While Saskatchewan’s law is vague on how FOI requests need to be submitted, Ontario’s simply states that requests be made “in writing” – meaning TDSB’s insistence on online filings is not compliant with the law.

We’d love to hear about how you’re using Secret Canada. Send us a note or use the hashtag #SecretCanada on social media. This information helps us grow the project.

To stay updated on FOI news, upcoming data releases and new features, sign up for The Globe's Secret Canada newsletter.