Skip to content.
|Networking government in New Zealand.
 

How to use e-government components to develop an NZGLS

3.1 Introduction

This Guide focuses on the components that are core to the current govt.nz portal, and therefore of most relevance to developers wishing to develop a subject portal based on the use of the same metadata.

For each component, as applicable, it discusses:

  • what it is

  • how it would be used in your portal

  • technical requirements

  • who to contact

  • conditions of use

  • further EGU documents to read

3.2 How to get started

The first steps towards using components for your portal is for your agency to:

  • define its business requirements

  • see which of the components will be most useful for the project you have in mind

  • contact EGU to get more information on the required systems and some estimates of the incremental costs of usage

  • decide whether to proceed

  • enter into negotiations on the Terms of Use MoU (Memorandum of Understanding) that will govern rights, responsibilities and cost-sharing.

Once the commercial arrangements are agreed, both your agency and EGU can begin activating the systems in accordance with your agency's project plan.

In addition to reading this Guide, the EGU recommends that you read the paper, 'better/faster/cheaper - the e-government component architecture'.

3.3 General conditions of use

Some components have conditions of use applying to them specifically. There are other conditions of use that apply more universally, as follows:

  1. EGU acquires the components and ensures they are compatible with the appropriate standards, but your agency has the autonomy to decide how, where and if it will use them;

  2. The components are freely available as a matter of right to the core Public Service agencies. In general, they will also be available to any central or local government organisation that is not purely commercial in nature. In some cases there may be additional licensing or other commercial arrangements to ensure the Government meets all its obligations to the external third parties that provide software and other componentry;

  3. While the components are freely available, they are not free. All services are charged on an incremental cost recovery basis; that is, your agency is responsible for the marginal costs associated with its use of various systems, over and above the base configuration funded by EGU;

  4. While costs are recovered, it's important to note that EGU is not a profit centre, nor is there any recovery of indirect costs associated with running core EGU infrastructure or systems recovered;

  5. A Memorandum of Understanding will generally govern the relationship between your agency and EGU. The intention is to articulate the rights and obligations of both parties in a manner that is clear, easily understood, and has been agreed by all concerned;

  6. EGU holds master agreements for software licenses, system hosting, bandwidth and the like. Your agency is free to leverage these existing commercial relationships, both within the context of the component architecture and for other external purposes, although there is no obligation to do so. EGU can provide guidance as to how an intended usage can be managed within the existing contracts;

  7. EGU is willing to assist your agency with developing and deploying its e-government strategy. EGU will not take on a project management role, but is willing to assist and advise on a case-by-case basis;

  8. Datacom Systems Ltd hosts all of the servers comprising the govt.nz portal production infrastructure (including Metalogue). The terms and conditions of use applying to the EGU are fully transferable to your agency.

3.4 Component-based subject portal architecture

The following diagram shows the interrelationships between a subject portal and its component parts, from an information flow perspective.

Component-based subject portal architecture.

3.5 Core policies and standards

Even if you choose not to use any of the components, there are several policies and standards that most government agencies are expected to adhere to when developing a government portal. These are:

If you're unsure whether these polices and standards apply to your agency, the web pages noted above will guide you as to their jurisdiction.

3.6 Web Guidelines

The Web Guidelines establish a standard for public sector websites in New Zealand. The major subject of the Guidelines is accessibility - removing impediments to online access to government. Accessible websites can be used by people regardless of disability, use of the latest technology or the availability of fast Internet connections.

Find out more at http://www.e-government.govt.nz/web-guidelines/.

3.7 e-GIF

The New Zealand e-Government Interoperability Framework (e-GIF) is a set of policies, technical standards, and guidelines (recommended practices) that outline the Government's policy on how public sector organisations should achieve electronic "interoperability" (i.e. the ability to share information and technology through using common policies and standards, to achieve 'joined-up service delivery').

Find out more at http://www.e-government.govt.nz/interoperability/.

3.8 NZGLS metadata standard

3.8.1 What is the NZGLS metadata standard?

The NZGLS metadata standard is the official New Zealand Government standard for creating discovery-level metadata (see Cabinet Circular CO (02) 3). The standard prescribes the use of data elements to create a record that describes a government service or information resource.

NZGLS is an acronym for the New Zealand Government Locator Service, a method of classifying and categorising New Zealand government organisations' information and services (both online and offline). The service is implemented through the NZGLS metadata standard and its associated thesauri.

The standard is based closely on two well-established standards: the Dublin Core Metadata Element Set and the Australian Government Locator Service.

NZGLS is now commonly used to refer to the metadata standard itself, and will be used this way throughout this document.

3.8.2 What will NZGLS be used for in your subject portal?

NZGLS will be used as the standard for any government information and services metadata created for, or used by, your subject portal. Use of this standard makes it possible for you to:

  • create metadata in a form compatible with e-government components (designed to handle NZGLS-compliant data), and usable by other government organisations

  • to interpret other organisations' NZGLS metadata, which you may use in your portal

  • to use other e-government components, such as Metalogue, both as a repository for your own metadata and as a source of metadata from other organisations.

3.8.3 Technical requirements

Application of the NZGLS standard will require tools to create, store and retrieve metadata records. The E-government programme offers the Metalogue repository for this purpose. Requirements for use of Metalogue are described in a later section of this document.

3.8.4 Who to contact

Web

http://www.e-government.govt.nz/nzgls/standard/

Email

nzgls@portal.govt.nz

Phone

Portal Information Manager 499-6600

3.9 Thesauri (FONZ & SONZ)

3.9.1 What are the NZGLS Thesauri?

The NZGLS Thesauri are sets of standard terms, cross-referenced to non-standard variations, which are used in the creation of NZGLS metadata.

There are two thesauri, one of which is used to control terms used in the NZGLS.Function field, the other to control terms used in the NZGLS.Subject field. Use of these thesauri is mandatory in conjunction with the NZGLS standard.

Functions of New Zealand (FONZ) is a hierarchically organised thesaurus of the functions performed by New Zealand government organisations. This thesaurus was directly derived from agency annual reports, mission statements, post-electoral briefings to Ministers, etc. Government services can thus be categorised according to which of these functions they are fulfilling, and documents can be categorised by the function to which their content relates.

The Subjects of New Zealand (SONZ) thesaurus is largely based on the New Zealand Parliamentary Library thesaurus, a mature and well-proven adaptation of the British House of Commons thesaurus.

Both thesauri are maintained and kept current by the National Library, in consultation with an advisory group composed of officials from local and central government, and in response to portal usage and agency feedback.

The thesauri operate synergistically, in that the combination of a term from each thesaurus provides a precise description of the resource, e.g.

Name of Service (NZGLS.Title)

FONZ term (NZGLS.Function)

SONZ term (NZGLS.Subject)

Obtain approval to export wine

Registering

Alcohol

Find information about programmes to reduce the harm caused by alcohol

Promoting Good Health

Alcohol

3.9.2 What will the NZGLS Thesauri be used for in your subject portal?

Your subject portal should use the NZGLS thesauri:

  • when creating metadata, to ensure that valid terms are entered in the NZGLS Function and Subject elements

  • when interpreting search terms entered by users, so that user terms can be translated into terms that match those used in the metadata, e.g. 'dole' translates to 'unemployment benefits'

Thesaurus terms can also be used to suggest or execute further searches for material, related to the original search.

The method followed by the govt.nz portal to use thesauri terms has been carefully thought out and refined. It is based on the following knowledge:

  • a full understanding of the intent of the NZGLS standard

  • the principles of the thesauri design

  • the actual real world practices of metadata creators

Do not take the decision to do something different, lightly.

EGU plans to develop an additional component (Text Search Builder) that will add thesaurus terms to your portal's searches in the right way, i.e. it will provide out-of-the-box compliance with the portal methodology. This component is expected in Q2 2003.

In addition to using the NZGLS Thesauri, the EGU recommends you consider using (or creating) a supplementary thesaurus that specialises in the subject matter of your subject portal. This will enable your portal to recognise and deal with the more granular search terms that could be expected on a subject portal. For example, the SONZ thesaurus has no narrower terms for 'astronomy', but a science portal thesaurus could include terms such as 'stars', 'galaxies', 'planets', 'radio telescope' etc.

3.9.3 Technical requirements

If you wish to host your own copy of the thesauri, you will need software capable of handling Z39.50-compliant thesauri.

Stand-alone versions of the thesauri can be obtained in several formats, but you can avoid the need to host your own versions by:

  • using Metalogue as your metadata repository (a basic 'search and select' interface is provided to the thesauri when the user is entering Function and Subject terms)

  • processing user terms by accessing the Search Service, enabling you to run various standard checks against the thesauri

If you use Metalogue as your data repository, you will automatically have use of the thesauri for selection and validation purposes.

If you wish to use the EGU-hosted thesauri in your searches, you will need to use the Search Service (unless the EGU develops the additional component described above). See the Search Service section for details.

You can read exactly how the govt.nz portal uses the Search Service to interrogate the thesauri, and how it uses the results, by reading:

Portal Search Specification http://www.e-government.govt.nz/docs/portal-search-spec/.

The EGU can provide further documentation about using the Search Service if requested.

3.9.4 Conditions of use

Requests by other portals to add or change FONZ or SONZ thesaurus terms should go through the normal channels, i.e. to the Metalogue Help Desk (metalogue@portal.govt.nz).

3.9.5 Who to contact

Web

http://www.e-government.govt.nz/nzgls/thesauri/

Email

metalogue@portal.govt.nz

Phone

Metalogue Help Desk: 04 495 2846

3.9.6 Further documentation

Further information is available as follows:

This document...

describes..

Portal Search Specification

http://www.e-government.govt.nz/docs/portal-search-spec/

how the portal search logic uses the thesauri (including calls to the Search Service)

(ask the EGU what can be provided)

how to write Search Service queries, used by the portal search logic to access the thesauri

3.10 Metalogue

3.10.1 What is Metalogue?

Metalogue is the NZGLS metadata repository used by the govt.nz portal.

The data is stored in a SQL Server database, hosted by Datacom Systems Ltd.

Agencies enter and edit their metadata by means of a web browser-based front-end.

Metalogue is supported by the EGU's Metalogue Help Desk, which:

  • administers the system

  • guides metadata creators in the mechanics of using the system

  • advises on principles and best practices of metadata creation

Metalogue enforces compliance with the NZGLS standard, and with additional requirements of the govt.nz portal

3.10.2 What to use Metalogue for in your subject portal

You can use Metalogue for two purposes in your subject portal:

  • as a validator and store for your own organisation's metadata

  • as a source of other organisations' metadata

3.10.3 Technical requirements

Entering/editing

Metadata is entered into Metalogue one NZGLS record at a time, via a web-based browser interface. Access is by userid and password.

To store metadata in Metalogue currently requires the data creator to have internet access and run the Internet Explorer web browser. The next version of Metalogue (due Q2 2003) will be certified to support the following browsers:

  • Internet Explorer v. 5.x and 6.x

  • Mozilla v. 1.x

  • Netscape v. 6.x

In future, independent creation and validation of metadata will also be possible using an XML Gateway.

Searching

You can search Metalogue metadata in one of two ways:

  • run Search Service queries against the metadata

  • send your own custom-built queries direct to the Autonomy search engine (bypassing the Search Service)

3.10.4 Conditions of use

For Searching, see 'Search Service & Autonomy' section.

Principles

  • The metadata asset should be built on and reused by as many portals as possible to drive down costs of new portals, increase the value to agencies in having created the metadata in the first instance, and reinforce consistent information management principles and practices across government.

  • The E-government Unit (EGU) sets the standard for metadata in accordance with the NZGLS (New Zealand Government Locator Service).

  • Archives New Zealand, the Custodian for the NZGLS, will administer use of the standard to catalogue government services and information. They will also ensure it stays aligned and contributes to the development of related international standards.

  • National Library of New Zealand, the Custodian for the thesauri used with NZGLS, Subjects of New Zealand and Functions of New Zealand, will work with government organisations to ensure they use the thesauri consistently and that the set of controlled terms continue to meet their needs as services evolve.

  • Metadata that appears on govt.nz must abide by portal implementation and quality assurance rules.

  • All metadata which has been passed through to public (i.e. can be viewed on govt.nz or another portal), must reside in the Metalogue database.

  • While agencies are custodians of their own metadata, where there are unresolved issues, (e.g. relating to scope, granularity, audience) between agencies whose metadata is sought by a portal and the other portal business owners, the EGU will assist by facilitating finding the best result. If a result cannot be reached, EGU will make the final decision.

Principles and procedures derived from 'Principles & procedures for govt.nz and other portals working together' (contact EGU to request SI-3-2-10, document no. 279052v2)

Protocols

  • The Metalogue Helpdesk is the first port of call for all agency queries relating to metadata servicing any portal.

  • Before approaching an agency with a request to create or change their metadata, the other portal should contact the Metalogue Helpdesk and discuss the request that they're going to make and a brief impact analysis carried out.

  • All other portal communications via email with agencies (once the issue(s) have been discussed with Metalogue helpdesk) should have Metalogue Helpdesk CCed into them.

  • Requests by other portals to add or change NZGLS elements should go through the normal channels e.g. to the custodian of NZGLS at Archives New Zealand and the NZGLS Working Group.

  • Requests to change information architecture or for Metalogue enhancements to go to Metalogue Helpdesk, who will then channel it to the appropriate role.

  • EGU and other portals will receive an automated alert when agencies delete any metadata records from Metalogue. [ This doesn't currently exist, but will be a Metalogue enhancement.]

3.10.5 Who to contact

Web

http://www.e-government.govt.nz/nzgls/management/

Email

For metadata creator access: metalogue@portal.govt.nz

For search access: refer to 'Who to contact' under 'Search Service & Autonomy'

Phone

Metalogue Help Desk: 04 495 2846

3.10.6 Further documentation

Further information is available as follows:

This document...

describes..

Portal Search Specification

http://www.e-government.govt.nz/docs/portal-search-spec/

how the portal search logic uses the metadata (including calls to the Search Service)

(ask the EGU what can be provided)

how to write Search Service queries, used by the portal search logic to access the metadata

3.11 Search Service and Autonomy

3.11.1 What is the Search Service and Autonomy?

The Search Service is used by the govt.nz portal as a unified interface to search:

  • Agency, Service and Document metadata

  • a full text index of Government websites

  • the NZGLS thesauri that control the metadata

Search Service encapsulates the expert knowledge built up by the portal designers about what information needs to be retrieved to satisfy various requirements. For example, the Thesaurus Search follows the relationships from any matched term to its preferred counterpart, and broader/narrower/related terms.

The search result interfaces share a common document structure so that these can be processed easily in the requesting application.

Search Service uses Autonomy to search indexes of metadata, thesauri and spidered web pages. It uses SQL Server queries to retrieve individual metadata records, search on non-indexed fields etc.

The Autonomy search engine is used to index both the NZGLS metadata in Metalogue, and government web pages. The Search Service uses Autonomy to execute any search that uses these indexes (the majority of searches). Autonomy (http://www.autonomy.com/) is one of the world's leading search engine tools, noted for its pioneering use of probabilistic concept modelling.

The EGU is providing a direct link through to Autonomy, thus enabling the Search Service to be bypassed if appropriate.

Other Autonomy services are likely to be available downstream, e.g. the ability to index sound files using speech recognition.

Autonomy is also available for agency use for other purposes beyond portal development, e.g. as an intranet or website search engine.

3.11.2 What to use the Search Service or Autonomy for in your subject portal

Search Service and Autonomy will be the standard mechanisms for your subject portal to search Metalogue data and the govt.nz portal's index of spidered text.

3.11.2.1 Querying metadata

The EGU does not generally recommend bypassing the Search Service when querying the metadata. In our opinion, there is little or no advantage to be gained from trying to use other Autonomy features to do this job, and plenty of risk of losing the learning reflected in the Search Service and its indexes. The Search Service uses Autonomy in a deterministic mode that suits the structured nature of the metadata.

The EGU strongly recommends that developers familiarise themselves with the logic the govt.nz portal uses to process queries, especially free-text searches.

The logic for handling free-text searches has been carefully developed and refined to:

  1. 'Build the Search', which consists of:

    • intelligently pre-processing (editing) the query string

    • identifying the right thesaurus terms to add to the query string, that ensures the search gets what the user wants, yet doesn't flood them with irrelevant material

  2. 'Query the metadata', which consists of:

    • processing the 'Built' search with the Search Service in such a way that users get the results they would intuitively expect, especially based on their experiences with other search engines such as Google.

Developers should not depart from this logic without fully understanding the consequences. The logic is detailed in the Portal Search Specification (see 'Further Documentation' below).

To make adhering to the portal free-text search logic easy for developers, the EGU plans to create an additional component, a Text Search Builder. This component will accept a text search string from a portal, and return a fully pre-processed version of the same string, i.e. string tidied up and thesaurus terms correctly added. This will provide out-of-the-box compliance with the 'Build the Search' portion of the portal methodology, saving developers from having to manually emulate the logic in the specification. Availability is expected in Q2 2003.

Future direction

The EGU plans to experiment with various techniques to supplement and transform the metadata sufficiently to make it suitable for querying using Autonomy's natural language searches. Achieving this will benefit portals with faster response times and a simpler search interface.

When and if results are achieved of comparable quality to those currently obtained from portal search logic in combination with Search Service, then EGU will promote natural language metadata searches as an alternative method. An indicative timeframe for this is Q4 2003.

3.11.2.2 Querying spidered text

The govt.nz portal currently uses the Search Service to query its spidered web pages index. The govt.nz portal's primary emphasis has been on finding government services supported by metadata. The web results have been treated as secondary. The govt.nz web results are deterministically correct, i.e. they contain either the words searched for, or the thesaurus-supplied equivalents, but the relevance rankings are not based on whether the documents themes match the words searched on. There is also a lot of clutter, resulting from repeated hits on document series, e.g. monthly journals and newsletters. The EGU regards a switch to natural language searching as a key part of its strategy to improve the govt.nz web results.

Developers constrained by time may wish, nevertheless, to use the Search Service for queries of the spidered web pages. If the developers are placing a low emphasis on spidered data, then the advantage of an existing templated solution may mean that the inherent limitations are acceptable, at least initially.

Based on our experiences as recounted above, EGU recommends the direct use of Autonomy's natural language searching for querying the full-text index of spidered government web pages, i.e. bypassing Search Service. This is because the Search Service is not utilising many of Autonomy's capabilities that are likely to prove valuable for document content indexes, most notably, its ability to create and use concept maps as a basis for searching.

Concept mapping is Autonomy's ability to interpret and categorise documents based on the content themes, and to support searches on these themes. It means that the disadvantage that we face with spidered web pages - a lack of metadata - can be more than compensated for by the ability to automatically categorise based on the much greater bulk of information present in a document.

Concept mapping will only work effectively if portal developers spend the time 'training' Autonomy to recognise concepts, by providing it with a representative set of model documents. EGU will assist agencies to source the necessary expertise.

To summarise - EGU is recommending:

  • use of Search Service for querying metadata, and

  • direct use of Autonomy natural language search to query spidered text.

3.11.3 Technical requirements

The Search Service is provided thorough a request/response messaging model based on a synchronous exchange of XML documents through HTTP POST.

Autonomy's natural language search can be accessed via HTTP GET.

3.11.4 Conditions of use

Usage of Search Service and Autonomy is authorised on the basis of a Terms of Use MoU (Memorandum of Understanding). Charging will be according to usage, on a cost-recovery basis.

3.11.5 Who to contact

Email

technical@portal.govt.nz

Phone

Portal Technical Support 495 6600

3.11.6 Further documentation

Further information is available as follows:

This document...

describes..

Portal Search Specification

http://www.e-government.govt.nz/docs/portal-search-spec/

how the portal search logic uses the Search Service to query the metadata and the spidered text, and how it handles results

(ask the EGU what can be provided)

queries available through the Search Service, and how to write them

Autonomy technical manuals

(available on request from EGU)

how to use the Autonomy natural language search

3.12 Content management

3.12.1 What is a content management system (CMS)?

The scope of content management is evolving with the technology, but in the context of the govt.nz portal, we use the term 'content management system' to refer to the software that manages creation and retrieval of a website's static content and hyperlink structure. The static content includes media releases.

The EGU is not providing a CMS as part of its components offering. The govt.nz portal uses a CMS based on the open source Midgard Application Server, but the EGU is not including it in the components offering because it has been difficult to use.

There are other Midgard-based options that may, however, be suitable for your needs (see http://www.midgard-project.org/), or you could consider other open source products (e.g. http://sourceforge.net/projects/phpwebsite/), or the wide range of commercial off-the-shelf products.

3.12.2 What to use content management for in your subject portal?

Your subject portal will need a mechanism for managing its static content and hyperlink structures

3.12.3 Technical requirements

It is possible to create your own CMS using back-end tools, but a more likely scenario for most agencies will be to purchase a commercial product, or use a free open-source product.

According to whatis.com (http://www.whatis.com/):

"Typical features of a content management system are

1. The Web-based publishing feature allows individuals to use a template or a set of templates approved by the organization, as well as wizards and other tools to create or modify Web content.

2. The format management feature allows documents including legacy electronic documents and scanned paper documents to be formatted into HTML or Portable Document Format (PDF) for the Web site.

3. The revision control feature allows content to be updated to a newer version or restored to a previous version. Revision control also tracks any changes made to files by individuals.

4. An additional feature is indexing, search, and retrieval. A CMS system indexes all data within an organization [EGU note: typically indexes only what's in the CMS]. Individuals can then search for data using keywords, which the CMS system retrieves. "

Some additional comments:

Publishing

The construction of the publishing templates and the architecture they fit into is of paramount importance. If your agency has a poor level of enforcement of styles for word processing documents, there are likely to be significant consistency issues when converting these for online use.

You should also be alert to the dangers of using Word/FrontPage to prepare your material for the Web. Word 2000 and above save a lot of material hidden in the MS-XML file format, including deleted text.

Revision Control

Tracking changes is important, for accountability. Revision control for restoration is hardly worth the effort it takes to implement it, i.e. you hardly ever have the opportunity or need to go back a complete version. However, this function will assist your agency in meeting auditing requirements.

High-end solutions will have many more capabilities than those listed above, e.g. full integration between print and web formats.

You will need to have HTML and technical web skills to deploy a CMS. Don't expect that a turnkey solution can completely shield you from this.

If you have the skills, and a starting set of templates (perhaps supplied by the EGU, though we don't have a standard set to offer at this point), you should allow:

  • a week to design a portal and adapt the templates to the CMS you have chosen

  • another week to install and test.

If you have little or no experience, count on needing at least a month.

3.12.4 Further documentation

There is an excellent short article at: http://webdesign.about.com//cs/contentmgmt/bb/aab-cms.htm.

This article summarises the types of solutions available, and what situation each is best suited to. You can then make your own judgment about what you want for your own agency. Bear in mind that your CMS will have uses beyond your portal (e.g. for your intranet and your agency website) - this consideration may influence you to seek a higher level of capability than that needed solely for your portal.

3.13 LDAP Directory (future)

3.13.1 What is an LDAP Directory?

An LDAP Directory is a structured repository, supporting the Lightweight Directory Access Protocol, which enables a user to obtain location/contact details for organisations, roles, individuals and other resources. It is organised in a tree hierarchy format, which supports the modelling of organisation structures, e.g. organisation-division-divisional subunit-individual resource (can be a role or person, but can also be a telephone, file etc)

The EGU is currently re-engineering the Metalogue database to migrate contact/availability data out of individual data records and into an LDAP Directory. This will remove the need for a lot of duplication. It will also enable the information to be used for other, non-portal applications. The LDAP Directory is expected to be available for portal usage in Q3 2003.

3.13.2 What to use LDAP Directory for in your subject portal?

When directory-related data is migrated from Metalogue to the Directory, the process will be transparent to users of the Search Service. The Search Service will be modified to get the data from a different source, and will continue to provide the same results.

However, subject portal designers may wish to take advantage of this separate source of data and write their own LDAP queries for other purposes.

3.13.3 Technical requirements

The EGU will publish the Directory schema once the Directory becomes available. Queries can be submitted to it using standard LDAP access software.

3.13.4 Conditions of use

To be supplied.

3.13.5 Who to contact

Not yet available.

3.13.6 Further documentation

Not yet available.


[ Previous | Next ]