Which government agencies receive the most Freedom of Information (FOIA) requests? How many of those requests yield responsive documents? And of those that do not, what are the most commonly stated reasons?
Perhaps most importantly, does the Obama administration live up to its claim of being the "most transparent" administration ever? (Spoiler Alert: No)
Last month, federal government agencies released their 2015 Annual FOIA Reports, which include detailed stats on how they responded to each and every Freedom of Information request.
Here is a rundown of what they reported, starting with what the Associated Press found in its analysis.
What the AP found
The Obama administration set a record for the number of times its federal employees told disappointed citizens, journalists and others that despite searching they couldn't find a single page requested under the Freedom of Information Act
There is no doubt the AP's conclusion is factually correct. The government failed to find the requested records 129,825 times last year, a new high. And a time series plot of the percentage of requests granted (shown in this chart) shows an alarmingly steep downward trend.
In 2008, the last year of the Bush administration, 41% of FOIA requests were granted in full. Since Obama took office, that number has steadily declined, hitting a new low of just 23% in 2015.
Having now spent a few days digging through the data myself, I think the AP gets the big picture right: under this administration, the government has become less transparent.
However, contrary to what the high-level figures imply, agencies seem to be turning up as many records as they always have. As explained below, what's changed is the likelihood that those records arrive with portions erased or withheld.
To accomplish this new level of censorship, agencies have expanded their use of one particular, broadly-defined clause in the Freedom of Information Act: Exemption 7(e).
Building on what the AP reported, here is a deeper dive into the data.
A handful of agencies accounted for the majority of FOIA requests
- In total, federal agencies processed a record-high 769,133 Freedom of Information requests last year.
- The top 5 agencies handled 71% of all FOIA requests.
- The Dept of Homeland Security (DHS) alone handled 45% of all requests
Here is how the requests broke down by agency in 2015. Each square is sized relative to the number of requests processed by that agency.
The top 5 agencies: the Dept of Homeland Security (DHS), the Dept of Justice (DOJ), the Dept of Defense (DoD), the Dept of Health & Human Services (HHS), and the Dept of Veterans Affairs (VA) together accounted for 71% of all Freedom of Information requests.
Of those, the busiest agency by far was the Department of Homeland Security. Last year, it processed 348,878 Freedom of Information requests, about 45% of the government-wide total.
How federal agencies responded to FOIA requests, 2008-2015
- Though the government responded with "no records found" a record-high 129,825 times last year, when viewed in proportion to the number of requests that were made, that figure is more-or-less in line with previous years.
- Agencies are granting requests at the same rate as in 2008, but it has become more common for the documents to arrive with pieces withheld or removed (a "partial grant").
If you separate the requests into two categories, those that yielded documents and those that did not, here's how it shakes out.
Requests that returned documents: ~60% of cases
- The percentage of requests yielding full grants has fallen from 42% in 2008 to 23% in 2015.
- Partial grant responses have risen from 18% in 2008 to 40% in 2015.
Requests that returned nothing: ~40% of cases
- 15% of the time, agencies find no responsive documents.
- 5% of the time, they do find documents, but withhold them entirely as legally exempt from disclosure.
- 3% of the time, the request is withdrawn by the requester (read: got scared away after the agency quoted a high fee estimate).
- The remaining 17% includes overly broad requests, duplicate requests, referrals to another agency, and other miscellaneous outcomes.
Not more denials, more partial denials
As shown above, FOIA denial rates and "no records found" rates have remained constant since 2008. What has increased is the frequency of partial grants (which you could just as easily call partial denials). The trend is clear when you look at the stats in aggregate, and also when you look at each agency individually.
The table below shows the percentage of requests that received partial grants in 2008 and in 2015. Together, the 20 agencies shown accounted for 97% of all FOIA requests that were processed last year. And all but 3 of them are awarding more partial grants today than they were in 2008.
Partial grants by agency (% of FOIA responses)
What does it mean that agencies are awarding more partial grants?
It's hard to say. A partial grant could be as trivial as the removal of a few inconsequential words. At the other extreme it could be a bunch of entirely blacked-out pages. It could also be the withholding of entire documents.
For an entertaining look at some of the most egregious examples of "partial grants", check out Muckrock's redaction hall of shame.
There is nothing in the annual FOIA reports about how much information agencies are withholding from partial grants. But the reports do cover agencies' use of FOIA exemptions, which give some clue as to the type of information that is being withheld.
Use of FOIA exemptions: what kind of information are agencies withholding?
- In 2015, the government applied FOIA disclosure exemptions 829,058 times.
- 2015 was the first year in which the number of exemptions used (829,058) exceeded the number of requests that were processed (769,133). This is possible because a document may have several pieces withheld, and each piece may be justified by more than one exemption.
- The most commonly used exemptions were Ex. 6 & Ex. 7(c) (invasions of personal privacy) and Ex. 7(e) (information that would reveal law enforcement investigation techniques).
Agencies are only permitted to withhold / redact information if it falls under one of 14 legal exemptions (technically 9 exemptions, one of which has 6 separate sub-exemptions).
The chart below shows how many times each exemption was applied in 2015 as compared to 2008.
The most common justifications for withholding information: exemptions 6, 7(c), and 7(e)
Exemptions 6 and 7(c): "unwarranted invasions of privacy"
The two most commonly used exemptions, Ex. 6 and Ex. 7(c), both deal with unwarranted invasions of personal privacy (e.g. medical information, home phone numbers, etc). The difference between the exemptions is subtle. Ex. 7(c) protects against "unwarranted invasions of privacy" from the release of law enforcement records. Ex. 6 applies to all records, not just law enforcement records. And it protects against "clearly unwarranted invasions of privacy," a higher bar than Ex. 7(c).
Both of these exemptions are fairly specific. And the information that gets withheld is normally not even material. For example, when requesting an agency's FOIA logs (a list of requests that the agency has received), I don't think anyone really cares whether the document includes the requesters' home addresses.
Exemption 7(e): "techniques and procedures for law enforcement investigations"
The third most commonly used exemption, Ex 7(e), is more controversial. Here is how it's defined:
Exemption (7)(E) protects from disclosure information which would reveal techniques and procedures for law enforcement investigations or prosecutions or that would disclose guidelines for law enforcement investigations or prosecutions if disclosure of the information could reasonably be expected to risk circumvention of the law.
Most of the other exemptions do not allow agencies to withhold information unless they can point to a specific negative consequence that would result from public disclosure. In contrast, Ex. 7(e) does not require the government to show a particular determination of harm or risk of circumvention of law, giving agencies much more discretion over what they can withhold.
- Tax evasion
- Immigration enforcement
- Terrorist threats
- Emergency response procedures
- Aviation watch lists
- Financial crimes
- Domestic surveillance
- U.S. customs
The most transparent administration in history?
Since 2008, agencies have become increasingly aggressive with their censorship of documents. It has now become the norm for FOIA-ed documents to arrive with portions removed or erased. This trend is true not only of the government as a whole, but of nearly every agency individually.
What kind of information is being withheld today that wasn't being withheld in 2008? Based on the growing use of Exemption 7(e), it appears agencies have decided to become less transparent about their activity behind the scenes to enforce the law. Presumably, the growing secrecy is not motivated by anything malicious, but rather a desire to keep the population safe.
If the biggest question of today's government is whether greater safety is worth sacrificing freedom, the data gives a pretty good idea of which direction the the current administration is leaning.
More
If you'd like to dig through the data yourself, you can download the raw files from FOIA Mapper.
There are certainly many ways of interpreting the numbers, but if there is any doubt where the current administration stands on transparency, here are some recent stories that answer the question pretty clearly.
- It Took a FOIA Lawsuit to Uncover How the Obama Administration Killed FOIA Reform (Vice News)
- White House office to delete its FOIA regulations (USA Today)
- FOIA Is Broken: A Report (U.S. House of Representatives)
- The federal government no longer cares about disclosing public information (Washington Post)
- In Comments Criticizing the Media, Obama Ignores His Administration's History of Censoring Journalists (Truth in Media)
- How Merrick Garland Made It Harder For Obama To Be So Secretive (Huffington Post)