Introduction

Objectives

Objective 1: Quantitative measurement and evidence-based online service improvement

New Zealanders' expectations from the State Services are both increasing and becoming more complex. The online channel holds out the promise of being both more efficient and effective. Yet, compared to other channels such as in-person or mail, it is relatively immature and less well understood.

Measuring customer experiences and perceptions lets us see how we're doing, both at a point in time and trends over time. Measuring at a whole of government level, to complement measurements at an individual agency level, lets us see how the State Services are doing overall.

There are several international studies that measure New Zealand's performance against that of other countries. For example, the 2008 United Nations survey  puts New Zealand at 18th place in a ranking of the "e-government readiness" of countries. This rank for New Zealand is broadly consistent with other international e-government studies.

However, these studies mostly evaluate the potential for use rather than people's actual use and satisfaction. For example, the United Nations survey measures "e-government readiness" which includes factors such as service sophistication, telecommunications infrastructure, and population literacy. These measures provide little actionable information to government agencies seeking to use the online channel more effectively.

Recently, several other countries and independent analysts have started moving towards measuring actual use and satisfaction with government's online services. A notable example is the Australian Government which has recently completed its fourth annual study  into Australians' use and satisfaction with e government services. It notes that, "The studies show that the way people interact with government has changed significantly in a relatively short period of time."

The State Services Commission has previously evaluated New Zealander's use and satisfaction with government's online services qualitatively as a part of measuring progress of e-government. There was "Achieving e-government 2004: A report on progress toward the E-government Strategy" and "New Zealand E-government 2007: Progress Towards Transformation". In addition, there have been other earlier efforts such as E-awareness , case studies , etc.

Quantitative measurements help complete the picture of how the State Services is doing.

Objective 2: Networked State Services Development Goal

Another objective of this survey is to inform the work on measuring progress that the State Services are making towards the 2010 milestone of the Networked State Services Development Goal.

The Development Goals for the State Services  provide a framework for delivering the next generation of State Services. The goals are aspirations for how the State Services will be configured and perform - "A system of world class professional State Services serving the government of the day and meeting the needs of New Zealanders."

The Networked State Services Development Goal aspires to "Use technology to transform the provision of services for New Zealanders." The 2010 milestone of "Government shared infrastructure is used to deliver user-centred services, and support joint results" will be measured by progress on three indicators :

Indicator 1

Grouping of services/transactions that apply technology to allow an individual - from one place at the same time - to access multiple programmes.

Indicator 2

Channel synchronisation of government transactions - within an agency or across government.

Indicator 3

The extent to which technology supports a user having to give the same information to government only once.

To better assess progress towards the 2010 milestone of the Networked State Services Development Goal, a useful input is to get New Zealanders' perceptions of how government is doing on each of the three indicators. This perspective will complement others, such as an assessment of the progress that individual agencies are making, in measuring progress.

Objective 3: Others

In doing this survey, the State Services Commission took the opportunity to also ask people their opinions about a range of issues in relation to its current work on the E-government Strategy . For example, whether people had heard of or visited the New Zealand Government website newzealand.govt.nz and how they find government information online.

Other relevant studies

There are three other studies that have some relevance in building the picture of New Zealanders' use and satisfaction with government's online services:

• Kiwis Count was the first all-of-government national survey to ask New Zealanders about their perceptions and experiences of public services as a whole. The report "Public Satisfaction with Service Quality 2007: The Kiwis Count Survey" highlights "where public services are doing well and where they can be improved, in relation to the drivers of satisfaction and trust."

Kiwis Count includes results for different ways ("channels") that people can get government's services, such as in-person or over the phone or online. Kiwis Count has some quantitative information about government's online services; however the focus is on service delivery across multiple channels.

This survey, Kiwis & Government Online Survey (2008), looks at only one channel (online) in a more fine-grained way across the three dimensions of information, transactions, and participation.

The next Kiwis Count survey will be in September/October 2009. Results will be available in 2010 including information on channel use and preferences.

• The World Internet Project New Zealand , Auckland University of Technology; and
• Household Use of Information and Communication Technology (ICT) Survey , Statistics New Zealand.

Limitations

There are two major limitations of the survey that need to be kept in mind.

First, the term "government".

The term means different things to different people. Also, various studies use the term to mean different things making it difficult to reconcile the results of various studies that have measured government's online services. To that extent, the term "government"s online services" is itself misleading as it depends on how questions are worded and how individual respondents interpret the term "government".

For example, some people may consider the Executive Branch of Government (Ministers) as part of "government" while others may not. Similarly, for the Legislative Branch, local government, Tertiary Educational Institutions, State Owned Enterprises, etc.

The scope of this survey, i.e. the definition of "government", is the State Services as it is the span of the State Services Commission's role. Broadly, this consists of the core Public Service and Crown Entities (other than Tertiary Educational Institutions) .

As can be seen from the questionnaire (Appendix B), an attempt was made in several places to try and convey what we meant by the term "government". This included being explicit about what was not in scope, e.g. "This research is not about your opinion of politicians, or the services provided by your local council."

Examples were also chosen to convey what was in scope, e.g. "using a savings calculator at Sorted" was chosen as, based on the work done by the Retirement Commission, many people are unaware that Sorted  is a service from the Retirement Commission (which is a part of the State Services and therefore in scope of the survey).

Pre-testing of the questionnaire confirmed that people generally did respond to questions based on the intended definition but it is quite likely that, at the margin, respondents may have had a different interpretation of the term "government" in mind when answering questions.

Second, in extending the results to the New Zealand population.

The survey undertaken for this survey used an online panel of New Zealand Internet users to find out their experience and expectations of using government online services. A total of 1,013 valid completed questionnaires were obtained. The panel sample demographics were broadly aligned with New Zealand population demographics. The online panel and questionnaire administration were provided by an independent commercial research company. More details about the methodology are provided in Appendix A.

In interpreting the results of this survey, it is important to keep in mind that:

• Being an online survey of Internet users, the survey results cannot be interpreted to represent the perspectives of New Zealanders who do not use the Internet.
• Although the sample demographics were broadly similar to those of the New Zealand population, the survey sample was not selected randomly from the general population. Conclusions from the data should be considered as indicative of the New Zealand population, rather than as being statistically valid.
• Given these limitations, the conclusions drawn from this survey have focused on those survey results for which there were a large and definitive proportion of respondents indicating a particular perspective. In those cases where the proportions were marginally different between different perspectives, this survey treated them as being undifferentiated.
• The confidence interval is approximately ± 3.1% for the total sample of 1,013 at a 95% confidence level and approximately ± 3.4% for the subset of 816 respondents who had used a government online service in the last 12 months.
• The survey achieved a response rate of about 18%.

[ Next ]