Towards Automated eGovernment Monitoring

September 26, 2011

Morten Goodwin’s Ph.D. thesis, with the title Towards Automated eGovernment Monitoring, is now available online.

Illustration photo of digital government

EGovernment solutions promise to deliver a number of benefits including increased citizen participation. To make sure that these services work as intended there is a need for better measurements. However, finding suitable approaches to distinguish the good eGovernment services from those which need improvement is difficult. To elucidate, many surveys measuring the availability and quality of eGovernment services are carried out today on local, national and international level.

Because the majority of the methodologies and corresponding tests rely on human judgment, eGovernment benchmarking is mostly carried out manually by expert testers. These tasks are error prone and time consuming, which in practice means that most eGovernment surveys either focus on a specific topic, small geographical area, or evaluate a small sample, such as few web pages per country. Due to the substantial resources needed, large scale surveys assessing government web sites are predominantly carried out by big organizations. Further, for most surveys neither the methodologies nor detailed result are publicly available, which prevents efficient use of the surveys results for practical improvements.

This thesis focuses on automatic and open approaches to measure government web sites.

The thesis uses the collaboratively developed eGovMon application as a basis for testing, and presents corresponding methods and reference implementations for deterministic accessibility testing based on the unified web evaluation methodology (UWEM). It addresses to what extent web sites are accessible for people with special needs and disabilities. This enables large scale web accessibility testing, on demand testing of single web sites and web pages, as well as testing for accessibility barriers of PDF documents.

Further, the thesis extends the accessibility testing framework by introducing classification algorithms to detect accessibility barriers. This method supplements and partly replaces tests that are typically carried out manually. Based on training data from municipality web sites, the reference implementation suggests whether alternative texts, which are intended to describe the image content to people who are unable to see the images, are in-accessible. The introduced classification algorithms reach an accuracy of 90%.

Most eGovernment surveys include whether governments have specific services and information available online. This thesis presents service location as an information retrieval problem which can be addressed by automatic algorithms. It solves the problem by an innovative colony inspired classification algorithm called the lost sheep. The lost sheep automatically locates services on web sites, and indicates whether it can be found by a real user. The algorithm is both substantially tested in synthetic environments, and shown to perform well with realistic tasks on locating services related to transparency. It outperforms all comparable algorithms both with increased accuracy and reduced number of downloaded pages.

The results from the automatic testing approaches part of this thesis could either be used directly, or for more in-depth accessibility analysis, the automatic approaches can be used to prioritize which web sites and tests should be part of a manual evaluation.

This thesis also analyses and compares results from automatic and manual accessibility evaluations. It shows that when the aim of the accessibility benchmarking is to produce a representative accessibility score of a web site, for example for comparing or ranking web sites, automatic testing is in most cases sufficient.

The thesis further presents results gathered by the reference implementations and correlates the result to social factors. The results indicate that web sites for national governments are much more accessible than regional and local government web sites in Norway. It further shows that countries with established accessibility laws and regulations, have much more accessible web sites. In contrast, countries who have signed the UN Convention on the Rights of Persons with Disabilities do not reach the same increased accessibility. The results also indicate that even though countries with financial wealth have the most accessible web sites, it is possible to make web sites accessible for all also in countries with smaller financial resources.

Full disclosure: I am the author of the thesis.

Global Web Accessibility

April 7, 2011

Cover of Journal of Information Technology and Politics

A scientific publication titled Global Web Accessibility Analysis of National Government Portals and Ministry Web Sites (Morten Goodwin, Deniz Susar, Annika Nietzio, Mikael Snaprud, Christian S. Jensen) was recently published.

The publication presents web accessibility benchmarking methodology, and uses this methodology to present a survey on the accessibility of public web sites in the 192 United Nations Member States. It further identifies common properties of Member States that have accessible and inaccessible Web sites and shows that implementing antidisability discrimination laws is highly beneficial for the accessibility of Web sites, while signing the UN Rights and Dignity of Persons with Disabilities has had no such effect yet. The article also demonstrates that, despite the commonly held assumption to the contrary, mature, high-quality Web sites are more accessible than lower quality ones. Moreover, Web accessibility conformance claims by Web site owners are generally exaggerated.

The countries with web sites that receive the best accessibility scores are:

  1. Germany
  2. Portugal
  3. Spain

The survey also shows that the economy of a country influences the accessibility of web sites, so that, not surprisingly, wealthy countries have more accessible web sites than poor countries. However, the study shows that accessibility laws have more impact than the financial status. Thus, it is not necessarily costly to make web sites accessible. It is however important to have well established accessibility laws which are actively followed up.

(Full disclosure: I am a co-author of the paper)
Morten Goodwin

Is financial wealth leading to high quality government services?

August 6, 2010

It is natural to assume that financial wealth leads to better government. It is further reasonable to expect that wealthy countries have higher quality of the e-government services compared to countries with less financial wealth. But how much does the finances alone influence quality e-government services? This short study gives a peek of how finances affects e-government services.

UN E-government 2010 report

In this study the data used for quality of e-government services is the E–Government Development Index (E-readiness score) from the United Nations E-Government Survey 2010. Thus, it is directly assumed that a government with high quality e-government services will receive a high score, and visa versa. The remaining data used is from the World Bank Data Catalog.

The following figure presents a box plot of the differences between the E–Government Development Index of Developing and Developed countries. The plot shows that developing countries have in average score of 0.4 while developed countries have an average score of about 0.7. Furthermore, all developing countries have scores less than 0.7, while all the developed countries have a score higher than 0.5. Thus based on the United Nations E–Government Development Index score it is, not surprisingly, significant difference between e-government services in developing and developed countries.

Developing countries have in average of 0.4 while developed countries have an average of about 0.7. All the developing countries have e-readiness score less than 0.7 while all the developed countres have a score higher than 0.5.

E-readiness score versus developing and developed countires.

Thus, the quality is clearly dependant on the finances, but how much of the quality e-Government services are influenced by finances alone?

The development of government services is complex procedure shaped by many factors. There exists no general conclusion of which factors influence the quality of the government service. It is however possible to determine to what extent data from the financial situation in a country can be used to predict the e-readiness score.

The following graph presents the plot between E–Government Development Index and GNI per capita. The graph also includes a regression, which can be used to calculate the E–Government Development Index based on the GNI per capita alone.

A dotplot showing the trends between E-readiness and GNI per capita.

E-readiness versus GNI per capita

The trends in the data are clearly visible. The regression can be seen as the black line, the mean response is shows as a green dashed line while the prediction interval is presented as the blue dashed line.

The regression line (black line) shows the relationship between the E–Government Development Index and GNI per capita. If no correlation existed between the two data sets, the line would be completely horizontall. The regression line can be used to predict the E–Government Development Index using only the GNI per capita. The graphs shows us that the relationship is not linear, but more complex.
The mean response interval (green dashed line) tells the estimated mean of the data.
The prediction interval (blue dashed line) tells where future data is expected be located (similar to confidence interval).

The data shows that the mean response interval and prediction interval changes as the GNI per capita increases. Generally, we are more certain of the prediction when these intervals are small. From this we can draw the following conclusion. It is relatively easy to predict the E-readiness score when a country has a low GNI per capita. In contrast, to predict the E-readiness score based on the GNI Per Capita alone for wealthy countries is a lot less precise. I.e. lack of finances generally means low quality services, while wealth alone is not sufficient to ensure quality in e-government.

Fighting corruption with e-Government

July 1, 2010

A very interesting study called E-government as an anti-corruption strategy showed that establishing e-Government reduces corruption. This should not be a surprise to anyone working with e-Government since it commonly believed that introduction of e-government diminishes the contact between corrupt officials and citizens, as well as increases the transparency and accountability. Unfortunately, hard evidence for these claims have been lacking (United Nations Development Programme, Fighting Corruption with e-Government Applications – APDIP e-NOTE 8, 2006).

The study is innovative as it uses a statistical approach to examine trends between e-Government and anti-corruption. Most other papers presenting quantitative data in the area do not use a statistical approach, which makes it more challenging to trust the results.

However, in this publication the author inspected, in a sound statistical way, the changes in corruption, using the control of corruption index presented by the World Bank, versus the changes in e-Government, using data from a Global e-Government Survey.

Unfortunately, for the OECD countries the author was not able to find any clear trends. This could be explained by less corruption in the OECD countries (compared to non-OECD countries), which means that the OECD countries had less to win, when it comens to anti-corruption, by introducing e-Government. Note that this is not evidence for absent of reduced corruption because of e-Government in OECD countries, just that the trends are not clearly visible in the data.

However, for the non-OECD countries, there are clear trends in the examined data. The results strongly imply that the introduction of e-Government has led to a significant reduction of corruption. Thus, supporting the view that e-Government is a very useful for reducing corruption – on a global scale.