Towards Automated eGovernment Monitoring

September 26, 2011

Morten Goodwin’s Ph.D. thesis, with the title Towards Automated eGovernment Monitoring, is now available online.

Illustration photo of digital government

EGovernment solutions promise to deliver a number of benefits including increased citizen participation. To make sure that these services work as intended there is a need for better measurements. However, finding suitable approaches to distinguish the good eGovernment services from those which need improvement is difficult. To elucidate, many surveys measuring the availability and quality of eGovernment services are carried out today on local, national and international level.

Because the majority of the methodologies and corresponding tests rely on human judgment, eGovernment benchmarking is mostly carried out manually by expert testers. These tasks are error prone and time consuming, which in practice means that most eGovernment surveys either focus on a specific topic, small geographical area, or evaluate a small sample, such as few web pages per country. Due to the substantial resources needed, large scale surveys assessing government web sites are predominantly carried out by big organizations. Further, for most surveys neither the methodologies nor detailed result are publicly available, which prevents efficient use of the surveys results for practical improvements.

This thesis focuses on automatic and open approaches to measure government web sites.

The thesis uses the collaboratively developed eGovMon application as a basis for testing, and presents corresponding methods and reference implementations for deterministic accessibility testing based on the unified web evaluation methodology (UWEM). It addresses to what extent web sites are accessible for people with special needs and disabilities. This enables large scale web accessibility testing, on demand testing of single web sites and web pages, as well as testing for accessibility barriers of PDF documents.

Further, the thesis extends the accessibility testing framework by introducing classification algorithms to detect accessibility barriers. This method supplements and partly replaces tests that are typically carried out manually. Based on training data from municipality web sites, the reference implementation suggests whether alternative texts, which are intended to describe the image content to people who are unable to see the images, are in-accessible. The introduced classification algorithms reach an accuracy of 90%.

Most eGovernment surveys include whether governments have specific services and information available online. This thesis presents service location as an information retrieval problem which can be addressed by automatic algorithms. It solves the problem by an innovative colony inspired classification algorithm called the lost sheep. The lost sheep automatically locates services on web sites, and indicates whether it can be found by a real user. The algorithm is both substantially tested in synthetic environments, and shown to perform well with realistic tasks on locating services related to transparency. It outperforms all comparable algorithms both with increased accuracy and reduced number of downloaded pages.

The results from the automatic testing approaches part of this thesis could either be used directly, or for more in-depth accessibility analysis, the automatic approaches can be used to prioritize which web sites and tests should be part of a manual evaluation.

This thesis also analyses and compares results from automatic and manual accessibility evaluations. It shows that when the aim of the accessibility benchmarking is to produce a representative accessibility score of a web site, for example for comparing or ranking web sites, automatic testing is in most cases sufficient.

The thesis further presents results gathered by the reference implementations and correlates the result to social factors. The results indicate that web sites for national governments are much more accessible than regional and local government web sites in Norway. It further shows that countries with established accessibility laws and regulations, have much more accessible web sites. In contrast, countries who have signed the UN Convention on the Rights of Persons with Disabilities do not reach the same increased accessibility. The results also indicate that even though countries with financial wealth have the most accessible web sites, it is possible to make web sites accessible for all also in countries with smaller financial resources.

Full disclosure: I am the author of the thesis.


Digitizing Public Services in Europe: Putting Ambition into Action

February 23, 2011

A report entitled Digitizing Public Services in Europe: Putting Ambition into Action was recently released by Capgemini for the European Commission. The report takes the pulse of eGovernment in Europe and is the ninth measurements of digital services of its kind.

The main focuses in the report includes how well digital services meet the the European i2010 action plan; How well the available eGovernment services are efficient, are able to include easy access to online services for all citizens, implement high impact services and strengthen participation and democracy.

The presented data shows that Ireland, Malta, Austria and Portugal rank best of the European countries on online sophistication.

Current and future challenges

Even though the report recognizes that the basic 20 services are available for almost all evaluated countries, it shows that the online sophistication levels significantly differs between national regional levels. Not surprisingly, the online national services score better than regional services, and online services in cities score better than in non-urban areas. A conclusion to be drawn from this is that even though eGovernment are mature on a national level in Europe, much work is left on regional levels.

Further challenges include take-up and impact. Even though services exists only 42% of individuals aged 16 to 74 use the Internet for interaction with public authorities. Another challenge is efficient trans-European interoperability.

Morten Goodwin


A collaborative approach for improving local government web sites

July 30, 2010

A publication on how to facilitate collaboration between local government and vendors entitled Accessibility of eGovernment web sites: Towards a collaborative retrofitting approach (Nietzio, Olsen, Eibegger, Snaprud) has recently been published.

Changing a local government web site is often a long process which normally includes vendors, editors and specialists in local regulations and legal enforcements. Results from benchmarking studies are often a good facilitators, but the results alone are of limited use when it comes to updates in practice. This is especially true if the web site updates are relatively small such as removing accessibility barriers. Thus, the paper presents an approach for rapid accessibility updates of government web sites. The approach uses benchmarking results together with forums and online checkers.

Collaborative process between municipalities, vendors and eGovMon. Vendors and municipalities collaborate through the eGovMon forum and through physical discussions. eGovMon organizes workshops and seminars for vendors and municipalities respectively.

Collaboration process between municipalities, vendors and eGovMon

The approach, visualised in the figure above, is applied to a group of Norwegian municipalities who want to improve the accessibility of their web site.

Accessibility benchmarking often fail to have an impact. This may be because of the following reasons:

  • The results are not detailed enough to be used for implementation purposes.
  • It is not clear what part of the publication chain the problem is located (in the CMS or introduced by the editor).
  • The maintainers do not have the technical knowledge to fix the problem.
  • The barriers are fixed in a one-off effort. However, since there are no quality process in place to detect if newly added content is in-accessible.
  • The benchmarking is carried out as a one-off study so that progress cannot be evaluated.

The presented approach includes three areas:

  1. Regular Benchmarking reports: Bi-monthly benchmarking reports of all municipality web sites. In these reports the editors of the local web sites can see how any web site updates affects accessibility.
  2. Online accessibility checkers: An interactive environment where editors and developers can instantly check their web pages and web sites. This allows for developers to incrementally remove accessibility barriers. (Blog post on Web Accessibility Checking)
  3. Online forum: Often times, it is clear where in the production chain an accessibility barrier is located. For example, when the logo of a web site is missing an alternative text, this is likely to be a problem caused by the CMS. However, if an individual image in a document is missing an alternative text, it could be because the editor did not provide this. Such discrepancies could lead to the situation where editors blame the CMS for accessibility problems, while the vendors claim that the editors are not using the CMS correctly. In the forum, editors can ask how to fix a specific barrier for a given CMS should be fixed, and the vendors can reply.

This approach allows for local web site editors to use e-government benchmarking results together with an online forum to fix any accessibility issues with the web site. Furthermore, the editors gets knowledge of which issues they cannot fix themselves, but has to be carried out by updates of the CMS software or web site template. Even though this collaborative concept was applied to web accessibility barriers, it may be useful for other areas of local e-government as well.

(Full disclosure: I’m a co-author of the paper)


Is e-government leading to more accountable and transparent local governments?

March 19, 2010

Recently, a very interesting and solid paper titled “Is e-government leading to more accountable and transparent local governments? An overall view“, authored by Vicenta Pina, Lourdes Torres and Sonia Royo, has been published. I recommend anyone interested in e-government assessment and transparency to read this.

The paper focuses on to what degree introducing e-government has had a positive impact of the transparency of local governments.

The transparency measurements are carried out by assessing local government web sites with a methodology that rewards the presence of services and information. The underlying assumption is that, for example, a web site having contact information is more transparent than a web site where contact information is missing.

The survey includes five local government web sites (the web site of the capital and the four subsequent largest cities) from 15 European web sites.  It is easy to argue that this is not a representative sample since smaller municipalities are not at all included in the survey. We can expect substantial differences between web sites from smaller and larger municipalities.

According to the survey results, the most transparent local governments can be found in the United Kingdom, followed closely by Germany, the Netherlands and Sweden. On the other hand, according to the survey, Greece had most improvement potential.

In addition to transparency the authors have performed a survey on account interoperability, usability and web site maturity.