Evaluating Open Government Efforts

Yesterday Federal agencies were required to release their Open Government plans and I am sure that there are a number of people out there looking at how well they did – me included. First, I applaud the agency staff that developed these and got these out. During my time in the Federal government, I worked on a number of  agency performance plans, GPRA reports, annual plans, and budgets so I have a pretty good idea about how much work these take and I think it is amazing that they got all of this done in the time frame they did.

In the meantime, it will take me a while to dig through all of these plans and ferret out the details but as I went through the different agency sites something struck me – how transparent were the agencies themselves being? For example, is it easy to find the Open Government page from the agency homepage? Can the open government plan be accessed in a number of formats? How open and transparent are the agencies on who leads each agency open government team? Is it clear what data sets are currently available and what data sets will be available in the future? Is it easy to provide feedback or input to the agency?  These and more are just some of ways to evaluate how open agencies are becoming.

To better evaluate agencies, I am in the process of designing and defining a heuristic review methodology of agency open government efforts. The review methodology will be composed of a number of criteria that can all be evaluated on a simple four point scale to determine how transparent, participatory, and collaborative the agency is in their Open Government efforts. When it is completed, I will publish the criteria so others and the agency can use it to track their own efforts.

For example, one of my review criteria is “Is the Open Government Team listed on the site and is there a way to contact them?” I looked at each agency Open Government page to see what team members were along with their corresponding titles and if there was an easy way to contact them. The best outcome would be an agency that listed all of its team members, their titles, and more then one way to contact the team or individual team members.  The worst outcome would be an agency that didn’t list any of their team members and didn’t provide a way for citizens to contact the team.  I took results of my first look, converted them to the common ABC scale and posted the results below:

USAID – B – The USAID OG site lists the Director of Public Information on their site as a point of contact and provides an alias email address.

Department of Agriculture – C – The Agriculture OG site does not list a point of contact. They only list an alias email address.

Department of Commerce – B -The Commerce OG site lists the complete team and uses an alias email address for the team lead.

Department of Defense – B – The Defense site lists the team lead and uses an alias email address for the team lead.

Department of Education– B- The Education OG site lists the complete team and uses an alias email address for the team lead.

Department of Energy – C – The Energy site lists a single person (who I could not find on the org chat on the DOE site and don’t know her role) and uses an alias email address.

Department of Health and Human Services – D – The HHS site does not list the OG team nor does it provide an easy way to contact the team outside of commenting on the plan.

Department of Homeland Security – D – The DHS site also does not list the OG team nor does it provide an easy way to contact the team.

Department of Housing and Urban Development – D – The HUD site also does not list the OG team nor does it provide an easy way to contact the team.

Department of Justice – D – The Justice site does not list the OG team nor does it provide an easy way to contact the team.

Department of Labor – C – The Labor OG site does not list a point of contact and they only list an alias email address.

Department of State – B – The State site lists the CIO email and another email, but there is no list of who the team is.

Department of the Interior – C -The Interior OG site does not list a point of contact. They only list an alias email address.

Department of Treasury – C – The Treasury site does not list the OG team nor does it provide an easy way to contact the team.

Department of Transportation – B – The Transportation OG site lists two team members and uses an alias email address for the team lead.

Department of Veterans Affairs – D – The VA site does not list the OG team nor does it provide an easy way to contact the team.

Environmental Protection Agency – D – The EPA site does not list the OG team nor does it provide an easy way to contact the team.

General Services Administration – B – The GSA OG site lists three team members and uses an alias email address for the team lead.

Across the agencies reviewed, the average score for this criteria was a 1.89 which I will round up to a C. The agencies overall are doing okay when it comes to being transparent about who is on their Open Government team and how to contact them. This is also not representative of the overall efforts of each agency since it is only a single criteria.

I am going to continue to develop the criteria, evaluate agencies, and then publish the results of my findings here. What criteria do you think should be used to evaluate agency Open Government efforts? Please feel free to include them in a comment here and I will look at including your suggestions with attributions. Thanks!

Note – Thank you to Andy and the others over at GovLoop for putting the fantastic list of agency efforts together!