Exempt from FOIA, US legislative support agencies follow uneven transparency standards

Reading room in the Library of Congress Jefferson Building. Source: https://www.house.gov/the-house-explained/legislative-branch-partners
Reading room in the Library of Congress Jefferson Building. Source: House.gov

Around the world, over one hundred nations have adopted freedom of information (FOI) laws that give their publics a right to request records from their governments. While these statutes are riddled with exemptions, marked by political interference, and often light on sanctions for those that block them, FOI laws remain essential tools for democratic governance everywhere they exist.

FOI laws have the greatest impact on transparency and accountability in states and nations where press freedom is strong and independent FOI ombudsmen and courts provide an adversarial venue where requesters can make appeals and challenge denials.

Sweden’s FOI law is the oldest in the world, passed in 1766. It wasn’t until July 4, 1966 that President Lyndon Johnson reluctantly signed the Freedom of Information Act (FOIA) at his ranch in Texas, codifying into law the American public’s right to access information from government agencies in the executive branch. 

During the Trump administration, the number of FOIA requests, FOIA lawsuits, and records censored have all reached record levels, driven from a combination of non-responsive agencies, reduced proactive disclosure, and active litigation by civil society groups.

Continue reading “Exempt from FOIA, US legislative support agencies follow uneven transparency standards”

House Publishes Its Rules, Jefferson’s Manual, & More Online as Structured Data

Today the Government Publishing Office published the House Manual — which contains Rules of the House of Representatives, Jefferson’s Manual, and other important legislative documents — online in a structured data format on GitHub. GPO did so pursuant to direction from the House Rules Committee, which was acting in accordance with the rules package passed at the beginning of the 114th Congress, which declares:

The House shall continue efforts to broaden the availability of legislative documents in machine readable formats in the One Hundred Fourteenth Congress in furtherance of the institutional priority of improving public availability and use of legislative information produced by the House and its committees.

The online publication of key legislative documents as structured data is a welcome development. The Congressional Data Coalition has for a long time requested the enhancement, which empowers further analysis and reuse of the information in many different context. I, for one, will be glad to be able to automatically track revisions in the House Rules from Congress to Congress. I know others will find much more insightful uses.

All the offices and agencies involved with the project deserve congratulations, including: the House Rules Committee, the House Parliamentarian, the Clerk of the House, and the U.S. Government Publishing Office. (I’m sure there are more.)

The House Manual, as helpfully explained in the user guide to using the electronic version of the manual, includes:

  • the U.S. Constitution
  • Jefferson’s Manual
  • the Rules of the House of Representatives
  • Provisions of the Legislative Reorganization Acts
  • Congressional Budget Act laws
  • Legislative Procedures

All of the files can be found here. I have not had an opportunity to fully review what’s online — for example, I’m trying to find an unannotated version of the House Rules— but GPO has helpfully requested feedback on the GitHub page.

House Rules Committee Chairman Pete Sessions hailed the move in a statement:

Technology plays an important role in our daily lives, and it is necessary that the House keep up with the most efficient and effective ways to provide information about Congressional activities. As Chairman of the House Rules Committee, I am committed to the advancement of sharing legislative data online and am confident that our efforts will result in a better informed public.

GPO explained, in a press release, the value of publishing the documents in a more flexible format:

Making Government information available in XML permits data to be reused and repurposed not only for print output but for conversion into ebooks, mobile web applications, and other forms of content delivery, including data mashups and other analytical tools by third party providers, which contributes to openness and transparency in Government.

Good job. And thank you.

— Written by Daniel Schuman

2016 Legislative Data & Transparency Conference Set for June 21

The Committee on House Administration will host its fifth annual Legislative Data and Transparency Conference on June 21, from 9–4 in the U.S. Capitol.

Free registration is now open. Per the invite:

The #LDTC16 brings individuals from Legislative Branch agencies together with data users and transparency advocates to foster a conversation about the use of legislative data — addressing how agencies use technology well and how they can use it better in the future.

The conferences are an amazing opportunity to engage with internal and external congressional stakeholders on making Congress more transparent and opening up legislative information. Perhaps even more importantly, they have become a place where everyone works together to find a way to make Congress work better and become more effective.

We expect to co-host a happy hour after the conference, location TBD.

What was it like in previous years? Well, here are write-ups from previous conferences:

Cross-posted from the Congressional Data Coalition blog.

— Written by Daniel Schuman

A Guide for Appropriators on Opening Up Congressional Information and Making Congress Work Better

For the fifth year in a row, today members of the Congressional Data Coalitionsubmitted testimony to House Appropriators on ways to open up legislative information. The bipartisan coalition focused on tweaking congressional procedures and releasing datasets that, in the hands of third parties, will strengthen Congress’ capacity to govern.

The testimony took note of notable successes:

We commend the House of Representatives for its ongoing efforts to open up congressional information. We applaud the House of Representatives for publishing online and in a structured data format bill text, status, and summary information — and are pleased the Senate has joined the effort. We commend the ongoing work on the Amendment Impact Program and efforts to modernize how committee hearings are published. We look forward to the release of House Rules and House Statement of Disbursements in structured data formats.

We would also like to recognize the growing Member and Congressional staff public engagement around innovation, civic technology and public data issues. From the 18 Members and dozens of staff participating in last year’s nationwide series of #Hack4Congress civic hacking events to the Second Congressional Hackathon co-sponsored by House Majority Leader Kevin McCarthy and House Democratic Whip Steny Hoyer, there is a growing level of enthusiastic support inside the institution for building a better Congress with better technology and data. Moreover, the House Ethics Committee’s recent approval of open source software and the launch of the Congressional Open Source Caucus means good things are in store for 2016.

This groundswell of support cuts across all ages, geographic areas and demographics, both inside and outside Congress. We are excited for the House’s 2016 legislative data and transparency conference and appreciate the quarterly public meetings of the Bulk Data Task Force.

And made recommendations on where the House should focus next or what kinds of data should be released:

Extend and Broaden the Bulk Data Task Force
● Release the Digitized Historical Congressional Record and Publish Future Editions in XML
● Publish all Congress.gov Information in Bulk and in a Structured Data Format
● Include All Public Laws in Congress.gov
● Publish Calendar of Committee Activities in Congress.gov
● Complete and Auditable Bill Text
● CRS Annual Reports and Indices of CRS Reports
● House and Committee Rules
● Publish Bioguide in XML with a Change Log
● Constitution Annotated
● House Office and Support Agency Reports

Signatories included: Center for Data Innovation, Data Coalition, Demand Progress, Free Government Information, GovTrack.Us, New America’s Open Technology Institute, OpenGov Foundation, OpenTheGovernment.Org, R Street Institute, and the Sunlight Foundation.

Read the testimony here. A primer on the work of the Congressional Data Coalition and its testimony over the last half decade is here.

— Written by Daniel Schuman

Top 5 Federal OpenGov Efforts

Photo credit: SDOT photos

In recent years there has been a lot of talk about opengov at the state, local and international levels, but when it comes to the federal government people just shake their heads and mutter. That is unfortunate, because a lot is happening at the federal level.

Here are five areas where the federal government is making major strides.

5. Innovative uses of technology

When you think of how the government uses technology, innovation often is not the word that comes to mind. More often it’s thought of as clunky, slow, out-of-date, insecure and expensive. But the executive branch has taken a step towards addressing these issues by creating 18F.

18F bills itself as “building the 21st century digital government.” It is an inside-government consultancy that builds technology for government on a cost-recovery basis. Housed at the General Services Administration, 18F addresses the twin problems of outside contractors who build cruddy tools that cost a ton of money as well as underfunded government developers who must use inadequate tools in unfriendly environments. Private sector developers are brought into government in the equivalent of a technology startup to help agencies build new tools and change the way they engage with technology. Most importantly, they work to change the culture around government information technology.

18F connects to opengov because many of its projects are opengov-related and the tools it builds are developed in the open. Projects include cleaning up Federal Election Commission data, a consolidated FOIA request hub, making federal spending transparent and rethinking the portal MyUSA. 18F is changing the way government uses technology, which often results in better, faster disclosure of government information.

4. Open courts

Unfortunately, federal courts are awful about opengov. But I did not want to let the opportunity to praise some great work being done on the federal civil society side, notably CourtlistenerOyezScotusBlogand Cornell’s Legal Information Institute. Respectively, they provide alerts for and deep content regarding federal court decisions; publish audio and transcripts of Supreme Court decisions from 1955 forward; provide real-time reporting and context for current Supreme Court activities; and provide access to many Supreme Court opinions.

3. Improved efforts to provide access to executive branch information

Over the last six years, both Congress and the executive branch have made serious efforts in proactively and responsively releasing information to the public, at least in some (non-national security) arenas.

The most notable effort has been in legislation to fix the Freedom of Information Act. Significant FOIA legislation passed the House and Senatelast Congress, but in slightly different forms so it has yet to be signed into law. The Obama administration, most notably the Department of Justice and financial regulatory agencies, fought against much-needed efforts to improve FOIA because it dared codify a presumption of openness and would require the public’s interest be weighed when evaluating whether to release information the executive branch deemed privileged. The legislation slowly is moving towards passage this Congress.

Other notable efforts ongoing on the FOIA front include the establishment of a FOIA advisory committee, an effort to unify FOIA regulations across all agencies and the construction of a central online FOIA request portal.

On the proactive disclosure side, the Obama administration conducted a survey of all the datasets it held, and — after a FOIA lawsuit — has agreed to release the inventory to the public. It also continues to publish a log of many of the visitors to the White House. And the administration is engaging in a biannual open government planning process, where many agencies publish a plan for releasing information to the public and follow through on some of their commitments; there’s also an international process around domestic transparency commitments. The Data.gov website, which publishes some federal datasets, also is of some value.

Still, there is a long way to go.

Congress has considered (and in a few instances passed) other notable legislation, including the DATA Act discussed below. Also on the docket is a bill — the Access to Congressionally Mandated Reports Act — that would require all agency reports to Congress be published online. The Presidential Library Donations Reform Act, which requires disclosure of donations to presidential libraries, is poised for consideration by the full House and Senate. There are other smart bills being drafted and considered as well.

2. Publishing federal spending information

Last Congress, the DATA Act was signed into law. This bipartisan measure would make much federal spending information available to the public, and would have gone further if not for strong oppositionfrom the Office of Management and Budget. The regulation governing the law currently is jointly being written by OMB and Treasury. By incorporating unique identifiers, following federal spending at a great level of detail and pushing information into a central repository, the DATA Act holds out the promise of transforming our understanding of federal spending. New legislation to extend the DATA Act also has been introduced.

Federal responding requirements like the DATA Act can be transformative. Whatever the merits of the2009 economic stimulus bill, the transparency requirements around the $787 billion legislation have had surprising results: “spending transparency became institutionalized in some states.” In some cases, the availability of Recovery Act data marked the first time officials were able to monitor performance trends for federal contracts, grants and loans across all state agencies. If properly implemented, the DATA Act can have similar follow-on benefits.

1. Open legislative information

By far the most remarkable transformation has been in public access to legislative information. The work to make congressional information available to the public in a structured data format and in bulk has been transformative, and the House literally has changed the way it operates to make this happen. We are here only because of a bipartisan commitment by House leadership who have labored without great acclaim against high bureaucratic barriers to modernize congressional operations.

There’s too much to point out all of the changes, but here are the highlights.

This is no less than a revolution in how Congress makes information available to the public. In turn, it has empowered a huge federal civic technology community that transforms the new data and tools into new ways to communicate with Congress, analyze information and make government more efficient and effective. Literally millions of people each month are directly or indirectly accessing federal legislative information because of this process. The Senate is not as far down the path as the House, and the Library of Congress is notable for its foot-dragging, but we have seen real, tangible, important progress.

Concluding thoughts

What makes all this progress even remarkable is that advocates for opengov at the federal level, especially when it comes to legislative information, have received much less attention recently than advocates at other levels of government. A lot is happening at the federal level, and if we keep working at it the possibilities are endless.

Cross-posted at the Sunlight Foundation blog.

{ Liked this? You may also like There Oughta Be a Law }

— Written by Daniel Schuman

It’s Time for Congress to Publish CRS Reports

This morning the New York Times editorialized in favor of public access to Congressional Research Service reports.

“Given the extreme partisanship and gridlock in Congress, it’s more crucial than ever to have an informed electorate. Putting these reports in the public domain is an important step toward that goal.”

Over the years our coalition has submitted testimony in favor of public access to these reports, most recently in March. In summary, the reports explain current legislative issues in language that everyone can understand, are written by a federal agencies that receives more than $100 million annually, and there is strong public demand for access. A detailed description of the issues at play is available here.

This congress, two legislative efforts are underway to make CRS reports public. First, the bipartisan H. Res. 34, introduced by Reps. Leonard Lance (R-NY) and Mike Quigley (D-IL), would make all reports widely distributed in Congress available to the public, except confidential memoranda and advice provided by CRS at the request of a member. Second, Rep. Quigley offered an amendment to an appropriations bill that would have required CRS to make available an index of all of its reports. Similar legislation has been introduced in the Senate in prior years.

The issue of public access can be resolved by either chamber of Congress passing a simple resolution. Indeed, a single member of Congress has the right to publish as many reports as desired on his or her website. Legislative branch appropriators in either chamber or the Committee on House Administration or Senate Rules Committee all have jurisdiction and could move swiftly to release the reports to the public. At a minimum, a hearing would shed light on the underlying issues. (In 2011, I hosted a panel discussion on the future of CRS that discusses public access.)

For two decades, members of Congress have tried to make these reports available to the public on a systematic basis. It’s time to finally make it happen.

For an index of resources, go here.

{ Liked this? You may also like A Rough Guide for Librarians on OpenGov and Choosing the Next Librarian of Congress }

— Written by Daniel Schuman

Legislative Project Ideas for Coders and Non-Coders

I thought it would be useful to identify legislative data projects in advance of the House’s annual Legislative Data and Transparency Conference and #Hack4Congress, a congressional hackathon we are co-hosting with our friends the OpenGov Foundation. I have written about some ideas previously, and others are newly published or elaborations. Not all are mine, but I like them all.

{Update: a bunch more ideas are available here.}

Requiring tech savvy

  • A robust roll-call vote comparison tool (more: see vote comparison tool)
  • An easy-but-smart search of the Congressional Record for member statements on a topic (more: see congressional record search)
  • A customizable daily or weekly email with all congressional hearings and floor votes, including committees and subcommittees, which at a click of a button would be added to your calendar (more: see open up draft legislation; also GovTrack’s webpage)
  • Congressional Research Service: CRS Report freshness ratings (more)
  • Congressional Research Service: One-stop source for Congressional Research Service reports (portal to search web) (more)
  • Congressional Research Service: Build a tool that allows congressional offices to receive requests to publish CRS reports and to host the publication of those reports; bonus if it allows the Member office to publish a list of all reports that a constituent/the public could request
  • A means to transform draft (pre-introduction) bills from PDF to a format that supports easy comparison, markup, and public discussion (more: see open up draft legislation)
  • A user-friendly webpage for the federal law website LegisLink (more: see federal law online)
  • Build a dashboard that reports on all House and Senate personal and committee websites that indicates whether they are SSL (HTTPS) compliant
  • Expand Capitol Bells to the Senate (see: Ted Henderson about Capitol Bells)
  • Build a Congressional Correspondence Tracker that allows Member offices to track all communications, automatically publish and thread letters and responses to the public (with redactions as appropriate)
  • Transform the Congressional Record into structured data
  • Transform the Constitution Annotated into structured data (pull it out of PDFs) (more: see public the Constitution Annotated as data)
  • Transform the entirety of House and Senate expenditure reports into structured data (CSVs) (more: see Sunlight blogpost and 1-pager)

For everyone (coders and non-coders)

  • Transform the Rules of the House of Representatives from PDF to TXT (or DOCX); ideally set up to reflect the organization of the document (e.g. indentations) (more: House Rules)
  • Transform the Rules of the Senate from HTML to something with proper indentations
  • Transform House and Senate Committee Rules from PDFs to TXT (or DOCX); ideally set up to reflect the organization of the document (e.g. indentations) (see: Rules of the Senate)
  • Build a wiki that links to or contains all the non-PDF versions of Chamber and Committee Rules (more: House committeesSenate committees)
  • Check House and Senate Committee websites to see if they work as HTTPS and compile as a list on a wiki (make a table)

{ Liked this? You may also like Learning from #Hack4Congress and Electronic Toolbox for Congress }

— Written by Daniel Schuman