Towards Automated eGovernment Monitoring

September 26, 2011

Morten Goodwin’s Ph.D. thesis, with the title Towards Automated eGovernment Monitoring, is now available online.

Illustration photo of digital government

EGovernment solutions promise to deliver a number of benefits including increased citizen participation. To make sure that these services work as intended there is a need for better measurements. However, finding suitable approaches to distinguish the good eGovernment services from those which need improvement is difficult. To elucidate, many surveys measuring the availability and quality of eGovernment services are carried out today on local, national and international level.

Because the majority of the methodologies and corresponding tests rely on human judgment, eGovernment benchmarking is mostly carried out manually by expert testers. These tasks are error prone and time consuming, which in practice means that most eGovernment surveys either focus on a specific topic, small geographical area, or evaluate a small sample, such as few web pages per country. Due to the substantial resources needed, large scale surveys assessing government web sites are predominantly carried out by big organizations. Further, for most surveys neither the methodologies nor detailed result are publicly available, which prevents efficient use of the surveys results for practical improvements.

This thesis focuses on automatic and open approaches to measure government web sites.

The thesis uses the collaboratively developed eGovMon application as a basis for testing, and presents corresponding methods and reference implementations for deterministic accessibility testing based on the unified web evaluation methodology (UWEM). It addresses to what extent web sites are accessible for people with special needs and disabilities. This enables large scale web accessibility testing, on demand testing of single web sites and web pages, as well as testing for accessibility barriers of PDF documents.

Further, the thesis extends the accessibility testing framework by introducing classification algorithms to detect accessibility barriers. This method supplements and partly replaces tests that are typically carried out manually. Based on training data from municipality web sites, the reference implementation suggests whether alternative texts, which are intended to describe the image content to people who are unable to see the images, are in-accessible. The introduced classification algorithms reach an accuracy of 90%.

Most eGovernment surveys include whether governments have specific services and information available online. This thesis presents service location as an information retrieval problem which can be addressed by automatic algorithms. It solves the problem by an innovative colony inspired classification algorithm called the lost sheep. The lost sheep automatically locates services on web sites, and indicates whether it can be found by a real user. The algorithm is both substantially tested in synthetic environments, and shown to perform well with realistic tasks on locating services related to transparency. It outperforms all comparable algorithms both with increased accuracy and reduced number of downloaded pages.

The results from the automatic testing approaches part of this thesis could either be used directly, or for more in-depth accessibility analysis, the automatic approaches can be used to prioritize which web sites and tests should be part of a manual evaluation.

This thesis also analyses and compares results from automatic and manual accessibility evaluations. It shows that when the aim of the accessibility benchmarking is to produce a representative accessibility score of a web site, for example for comparing or ranking web sites, automatic testing is in most cases sufficient.

The thesis further presents results gathered by the reference implementations and correlates the result to social factors. The results indicate that web sites for national governments are much more accessible than regional and local government web sites in Norway. It further shows that countries with established accessibility laws and regulations, have much more accessible web sites. In contrast, countries who have signed the UN Convention on the Rights of Persons with Disabilities do not reach the same increased accessibility. The results also indicate that even though countries with financial wealth have the most accessible web sites, it is possible to make web sites accessible for all also in countries with smaller financial resources.

Full disclosure: I am the author of the thesis.

Advertisements

I paid a bribe

January 13, 2011

An innovative online service has recently been launched called I paid a bride.

In this service you can report if you:

  • Unwillingly paid a bribe,
  • Got a demand for a bribe but resisted or
  • Discovered improvements when it comes to corruption.

The aim is to use the collected data to learn the nature and patterns of bribes as a step towards fighting the corruption.

(Source: Sabina Panth blog post in blogs.worldbank.org)


Fighting corruption with e-Government

July 1, 2010

A very interesting study called E-government as an anti-corruption strategy showed that establishing e-Government reduces corruption. This should not be a surprise to anyone working with e-Government since it commonly believed that introduction of e-government diminishes the contact between corrupt officials and citizens, as well as increases the transparency and accountability. Unfortunately, hard evidence for these claims have been lacking (United Nations Development Programme, Fighting Corruption with e-Government Applications – APDIP e-NOTE 8, 2006).

The study is innovative as it uses a statistical approach to examine trends between e-Government and anti-corruption. Most other papers presenting quantitative data in the area do not use a statistical approach, which makes it more challenging to trust the results.

However, in this publication the author inspected, in a sound statistical way, the changes in corruption, using the control of corruption index presented by the World Bank, versus the changes in e-Government, using data from a Global e-Government Survey.

Unfortunately, for the OECD countries the author was not able to find any clear trends. This could be explained by less corruption in the OECD countries (compared to non-OECD countries), which means that the OECD countries had less to win, when it comens to anti-corruption, by introducing e-Government. Note that this is not evidence for absent of reduced corruption because of e-Government in OECD countries, just that the trends are not clearly visible in the data.

However, for the non-OECD countries, there are clear trends in the examined data. The results strongly imply that the introduction of e-Government has led to a significant reduction of corruption. Thus, supporting the view that e-Government is a very useful for reducing corruption – on a global scale.


Weighing Indices in the UN E-Government Survey

May 14, 2010

The United Nations E-government Survey index is a weighted combination of three indices:

  • Web Measure, which represents the sophistication level of online citizen services.
  • Human Capital, which represents the education level of a country.  This index is again weighted with two-third weight of adult literacy and one-third to weights to enrollment.
  • ICT Infrastructure, which represents the infrastructure in a country. This is again average weighted including  number of computers per person, telephone lines, mobile phones etc.

These three are all weighted equally contributing 1/3 to the score, which means that formally the e-readiness is as following:
E-readiness =

1/3 * Web Measure +

1/3 * Human Development +

1/3 * ICT Infrastructure

An interesting question that follows is what happens if we assign other weights to these indices For example, if we change the weights, can we also change the ranking a country gets?

Using Monaco as an example, it was ranked as member state number 112 in the UN e-readiness survey 2010. However, by adjusting weights of the three indices, we can change the ranking of Monaco from 112 up to 25, or down to 184.

In the following plot, possibile combination from 10% up to 80% of the three indicies are plotted and the corresponding ranking of Monaco.

Ranking of Monaco when weighting the indicies Web Measure, Human Capital and ICT Infrastructure differently

Similarly, the following graphs how the top five member states, according to the E-readiness ranking in the 2010 survey, would rank if different weights would be used.

Ranking of the top five countries with different weights

(Note that for reason of clarity some weightings have deliberatly been removed).

The question which naturally arises is:

Why does the current E-readiness index use equal weights, and is this any more correct than any weights?

Thanks do Deniz Susar for input on this idea.


United Nations Global E-Government Survey 2010

April 15, 2010

UN E-government 2010 reportThe United Nations Global E-government Survey 2010 is now available. This fifth UN E-government survey focuses on e-government at a time of financial and economic crises.

The first part of the report is a discussion on ways e-government can mitigate the effects of the financial crises on development. It sees e-government in the light of the following United Nations priorities:

  1. Stimulus funds, transparency and public trust. By October 2009 the financial stimulus packages summed up to about one third of the gross world income. At the same time, the trust in banks decreased.
  2. Roles of e-government in financial regulation and monitoring. Government deploy ICT as a response to the financial crises which has the potential to improve the policy making process.
  3. E-service delivery and how it relates to the millennium development goals including how e-government poverty eradication, education, gender-inclusive approach to service delivery.

The second part is the results from the global survey. As previously, this includes the e-government ranking of the United Nations member states, regions and comparisons to the previous survey. Additionally, the second part includes the e-participation ranking and a (superficial) methodology section.

Please see the official page for more details.


Is e-government leading to more accountable and transparent local governments?

March 19, 2010

Recently, a very interesting and solid paper titled “Is e-government leading to more accountable and transparent local governments? An overall view“, authored by Vicenta Pina, Lourdes Torres and Sonia Royo, has been published. I recommend anyone interested in e-government assessment and transparency to read this.

The paper focuses on to what degree introducing e-government has had a positive impact of the transparency of local governments.

The transparency measurements are carried out by assessing local government web sites with a methodology that rewards the presence of services and information. The underlying assumption is that, for example, a web site having contact information is more transparent than a web site where contact information is missing.

The survey includes five local government web sites (the web site of the capital and the four subsequent largest cities) from 15 European web sites.  It is easy to argue that this is not a representative sample since smaller municipalities are not at all included in the survey. We can expect substantial differences between web sites from smaller and larger municipalities.

According to the survey results, the most transparent local governments can be found in the United Kingdom, followed closely by Germany, the Netherlands and Sweden. On the other hand, according to the survey, Greece had most improvement potential.

In addition to transparency the authors have performed a survey on account interoperability, usability and web site maturity.


2009 Transparency International Global Corruption Index

November 19, 2009

Transparency International has annually since 1996 released a global corruption index aim at representing the level of public-sector corruption. The results from the 2009 Global Corruption Index has recently been released.

New Zealand, Denmark, and Singepore received the highest score (least corrupt), while Somalia, Afghanistan, and Myanmar have most improvement potential.

Good scores reflect political stability, long-established conflict of interest regulations and solid, functioning public institutions. Furthermore, the results  show that countries which are perceived to have the highest levels of public-sector corruption are also those plagued by long-standing conflicts, which have torn apart their governance infrastructure. (What is new in the 2009 Corruption Perceptions Index)

The corruption index is calculated based on results from survey questions posed to business leaders around the world (Transparency International Frequently Asked Questions)