Integrity Policies
Policy: Authenticity Integrity and Retrievability
Data and business documents will be managed to preserve and demonstrate their authenticity, integrity, and retrievability to meet business and statutory requirements.
Scope and Interpretation
This policy covers both the logical and physical integrity of data and document stores and their contents.
In order to present consistent information both internally and externally, document and data stores must be managed within agencies as a coherent whole. This means:
- All stores are known
- Duplication of content between stores is minimised and controlled
- Content is presented whole even if parts are stored on different physical media
- The original content, context, and structure of documents is preserved
- Authorised activities are permitted
- Unauthorised activities are prevented
- Relevant events are logged
- Content is retrievable in a usable format
Demonstrating the logical integrity of data at a single item level means ensuring that the content in a physical database or file matches the business (logical) definitions for those data elements.
Demonstrating the logical integrity of a business document means proving that the technology for managing the content of a document store performs to specifications.
Demonstrating authenticity means showing that systems for collection and storage were used in the manner intended, by reference to standard procedures.
Physical integrity must be guaranteed by the service provider acting as Physical Custodian. This will involve all aspects of managing the physical systems to maintain, store and deliver content from data and document stores.
Supporting Standards: Referential Integrity; Integrity of application software; Integrity of configuration; Integrity of content; Integrity of process; Skills and training; Version control; Document templates; Retrievability
Rationale
Agency data and business document resources lose their value in proportion to any loss in their perceived and actual integrity, authenticity and reliability. Any degradation in the value of Crown data and business document resource must be avoided wherever possible.
Policy: Auditability
Data elements and business documents must be defined in a consistent manner and stored in a consistent format across all stores and, where required by the Business Custodian, changes to form or content must be recorded in an audit trail.
Scope and Interpretation
The general requirement that data be defined consistently within each agency and between agencies is necessarily a long-term goal. The aim is to make sure that users can always compare "apples with apples", regardless of where the data came from.
The most obvious examples are classification schemes, which should follow international standards or generally accepted guidelines e.g. country codes or job types. Increasing importance will also need to be given to person identification details which may be used for access to government services through a common portal based on the internet.
Consistency in the metadata elements used to describe business documents is also an important goal for every government agency. This includes attributes of individual documents e.g. title, version etc. as well as agency-wide metadata, e.g. a directory tree structure or document classification scheme. Consistent storage allows consistent retrieval and the ability to follow an audit trail.
Change control over data structures, document templates, and systems that maintain content is essential to ensure that neither current nor historical material is corrupted or made invalid.
Comparison of the physical content of database fields with the logical definition is an audit task that should be included, where required, as part of a system data audit. It should also be checked as part of the internal audit procedures used by service providers.
The intention of a system audit trail is to track and report system access and data changes. It helps protect against the possibility of unauthorised change to critical or sensitive data elements by logging details of who made the change.
Data collected by machine e.g. scientific observations, or data with no person-related content might not need special audit provisions, depending on usage requirements.
With personal data, systems must not only track changes but must also allow the recording of any changes requested by the individual concerned which were not implemented. This is required under the Privacy Act.
Supporting Standards: Change control; Version control; Audit trail
Rationale
Auditing is the means whereby the integrity of Crown data and document assets is checked and verified. A clear audit report is easiest to achieve when data and document stores are well organised.
Consistent definition and storage is also important for the eventual harmonisation of definitions for key data elements throughout government, where these are held in common. This also aids in the compilation of statistics across government, which in turn assists operational and policy development.
Policy: Interchange, Replication and Interfaces
Within legislative provisions and access protocols, information may be interchanged between agencies. The preferred method is via a defined electronic system interface to the appropriate data or document stores.
Scope and Interpretation
At its most sophisticated a system interface may involve direct automatic database links, or hyperlinks to documents and data records with authentication via digital certificate. At its most humble, it may be a letter requesting a document as part of a formal manual procedure.
Interchange may involve a physical transfer one record at a time or in bulk, or establishing a temporary or permanent link between records on different systems. In general, interfaces should be designed to move the minimum amount of data to achieve the result required.
Interfaces for data matching purposes are controlled via the Privacy Act and must meet stringent criteria. All such arrangements must be subject to approval by the Privacy Commissioner unless specified in separate legislation.
Replication should normally be reserved for development of a data warehouse, high availability (hot) back-up sites, or where technological limitations require data to be duplicated at multiple sites. Where replication is undertaken the following principles will apply:
- One system for each data element or document will be identified as containing the master copy
- An audit trail will be retained to prove that the target matched the source at each transfer
- Data transformation between systems will be kept to a minimum and fully documented where required
Supporting Standards: Interchange; Replication; Interfaces; Migration
Rationale
To facilitate the delivery of government services, on-line government data and document stores need defined electronic interfaces to allow appropriate access. These can be used both externally by the public and internally between agencies to increase efficiency and improve service delivery.
Policy: Retention
Data and business documents will be managed within a defined retention process.
Scope and Interpretation
Parameters of the process such as retention time will be governed by legislation applying to particular agencies, business needs, or by provisions of the Archives Act, the Official Information Act or the Privacy Act.
Storage and back-up systems must ensure that data and documents are available in time to meet business requirements.
Supporting Standards: Media-independent classification of documents; Back-up, recovery and restore; Storage media; Disaster recovery; Retention requirements; Transfer between agencies; Destruction protocols
Rationale
Retention plans will allow agencies to retain stores of valuable business documents and data and dispose of unwanted material at the right time.
