Why responsive records aren't included in the Secret Canada database

The Secret Canada team is running a user survey through the end of December to try and better understand who is using the website, what’s working well and what needs improvement.

(If you haven’t already filled the survey out, please do. It will take 3 minutes. We’d really appreciate it and we’ll even send you a thank you Secret Canada sticker!)

Several of the folks who have replied so far have asked whether we would be able to post the documents themselves on our website one day. 

“It would be very nice if the responsive records to each request were available for download (so as not to have to file piggyback requests),” one respondent from Toronto wrote.

The comment has come up so much we thought we should address it.

We would love to do this. When we began building SecretCanada.com in 2022, this was absolutely the plan. At the very least, we wanted to aggregate what was already available. Some public institutions actually do post the response records online. We figured we could download them and put them in one place. Unfortunately, it just isn’t practical, or frankly, feasible.

With respect to the idea of asking for copies of all records connected to completed requests, this is a non starter. For one thing, it would be astronomically expensive as public bodies would probably want to re-vet each file. Some of these institutions get thousands of FOI requests each year, meaning they would need to review thousands of responsive records. Even getting the request summaries was like pulling teeth at times. I can’t imagine if we asked for the documents themselves. We also review the wording of every request before it goes online, so we’d need to have eyes on every responsive record page. We have more than 300,000 FOI summaries on our site right now. Imagine each one came with 10 pages of documents …

Even aggregating already posted documents had to be scratched from our wishlist when we got back quotes on how much it would cost to store tens of thousands of PDFs. Moreover, that number would grow exponentially each year as we continue to update the database. 

So as with any project, we regrouped and decided that the next best thing was making it as easy as possible for people to get the documents themselves, hence the piggyback letter generator.

It’s not perfect, but it’s workable.

Keep those comments coming! 

We’d love to hear about how you’re using Secret Canada. Send us a note or use the hashtag #SecretCanada on social media. This information helps us grow the project.

To stay updated on FOI news, upcoming data releases and new features, sign up for The Globe's Secret Canada newsletter.