Combined Annotated document



Foreword

The OAIS Reference Model contained a roadmap which included the need for a certification standard. The initial work was to be carried out outside CCSDS and then brought back into CCSDS to take into the standard. In 2003, RLG and the National Archives and Records Administration created a joint task force to specifically address digital repository certification. The task force published the Trusted Repository Audit & Certification: Criteria and Checklist (TRAC) document which forms the basis of this standard.

Introduction

A decade ago, the Task Force on Archiving of Digital Information (1996) declared, “a critical component of digital archiving infrastructure is the existence of a sufficient number of trusted organizations capable of storing, migrating, and providing access to digital collections.” The task force saw that “trusted” or trustworthy organizations could not simply identify themselves. To the contrary, the task force declared, “a process of certification for digital archives is needed to create an overall climate of trust about the prospects of preserving digital information.” The task force stopped short of articulating the details of such a certification process. Certainly one obstacle was that though some archives and repositories existed at the time, there was no organized “digital preservation community” with common, consensus-driven practices, let alone standards. Each archive or repository conducted digital preservation in its own manner and to the level that seemed to address funding and user community needs.

Work in articulating responsible digital archiving infrastructure was furthered by the development of the Open Archival Information System (OAIS) Reference Model (ISO 14721:2002). Designed to create a consensus on “what is required for an archive to provide permanent or indefinite long-term preservation of digital information,” the OAIS addressed fundamental questions regarding the long-term preservation of digital materials that cut across domain-specific implementations. The reference model provides a common conceptual framework describing the environment, functional components, and information objects within a system responsible for the long-term preservation of digital materials. Long before it became an approved standard in 2002, many in the cultural heritage community had adopted OAIS as a model to better understand what would be needed from digital preservation systems.

Institutions began to declare themselves “OAIS-compliant” to underscore the trustworthiness of their digital repositories, but there was no established understanding of “OAIS-compliance” beyond meeting the high-level responsibilities defined by the standard. There were certainly no criteria for measuring compliance.

Claims of trustworthiness are easy to make but are thus far difficult to justify or objectively prove. As Clifford Lynch has stated, “Stewardship is easy and inexpensive to claim; it is expensive and difficult to honor, and perhaps it will prove to be all too easy to later abdicate” (Lynch 2003). Establishing more clear criteria detailing what a trustworthy repository is and is not has become vital. In 2002, RLG and OCLC jointly published Trusted Digital Repositories: Attributes and Responsibilities (TDR), which further articulated a framework of attributes and responsibilities for trusted, reliable, sustainable digital repositories capable of handling the range of materials held by large and small cultural heritage and research institutions. The framework was broad enough to accommodate different situations, technical architectures, and institutional responsibilities while providing a basis for the expectations of a trusted repository. The document has proven to be useful for institutions grappling with the long-term preservation of cultural heritage resources and has been used in combination with the OAIS as a digital preservation planning tool. As a framework, this document concentrated on high-level organizational and technical attributes and discussed potential models for digital repository certification. It refrained from being prescriptive about the specific nature of rapidly emerging digital repositories and archives and instead reiterated the call for certification of digital repositories, recommending the development of certification program and articulation of auditable criteria.

The RLG and NARA later published the Trusted Repository Audit & Certification: Criteria and Checklist (TRAC) document which combined ideas from OAIS and TDR. TRAC formed the basis of the metrics in this standard.

Process Approach (Perhaps this should be in the "Requirements for Auditors" document)

This International Standard should be used within a process approach for establishing, implementing, operating, monitoring, reviewing, maintaining and improving an organization's trustworthiness, see for example ISO 27001:2005.

An organization needs to identify and manage many activities in order to function effectively. Any activity using resources and managed in order to enable the transformation of inputs into outputs can be considered to be a process. Often the output from one process directly forms the input to the next process. The application of a system of processes within an organization, together with the identification and interactions of these processes, and their management, can be referred to as a “process approach”.

The process approach for trustworthiness presented in this International Standard encourages its users to emphasize the importance of:

  • understanding an organization’s requirements and the need to establish policy and objectives for trustworthiness;
  • implementing and operating controls to manage an organization's preservation risks in the context of the organization’s overall business risks;
  • monitoring and reviewing the performance and effectiveness of the activities which support its trustworthiness; and
  • continual improvement based on objective measurement.

References (Probably should be at the end of the docuemnt)

  • Consultative Committee for Space Data Systems (CCSDS). 2002. Reference Model for an Open Archival Information System. (ISO Standard 14721). http://www.ccsds.org/publications/archive/650x0b1.pdf
  • CCSDS 2003. Producer-Archive Interface Methodology Abstract Standard. (ISO Standard 20652). http://www.ccsds.org/publications/archive//651x0b1.pdf
  • CCSDS May 15, 2006. XML Formatted Data Unit (XFDU) Structure and Construction Rules. http://sindbad.gsfc.nasa.gov/xfdu/pdfdocs/iprwbv2a.pdf
  • Cornell University Libraries. Digital Preservation Management: Implementing Short-term Strategies for Longterm Problems. 2004. www.library.cornell.edu/iris/tutorial/dpm/index.html
  • ISO 9000:2000 Quality management systems—Fundamentals and vocabulary. Geneva, Switzerland: International Organization for Standardization.
  • ISO/IEC 17799:2005 Information technology—Security techniques—Code of practice for information security management. Geneva, Switzerland: International Organization for Standardization.
  • ISO 27001:2005 Information technology - Security techniques - Information security management systems - Requirements
  • Lynch, Clifford A. February 2003. “Institutional Repositories: Essential Infrastructure for Scholarship in the Digital Age.” ARL BiMonthly Report 226. http://www.arl.org/newsltr/226/ir.html
  • Metadata Encoding and Transmission Standard (METS) version 1.4. 2005.Washington, DC: Digital Library Federation. http://www.loc.gov/standards/mets
  • Minnesota Historical Society, State Archives Department. 2002. Trustworthy Information Systems Handbook. http://www.mnhs.org/preserve/records/tis/tis.html
  • National Institute of Standards and Technology. 2001. Security Self-Assessment Guide for Information Technology Systems (NIST Special Publication 800-26). Washington, DC: NIST. http://csrc.nist.gov/publications/nistpubs/800-26/sp800-26.pdf
  • National Institute of Standards and Technology. April 2005. Revised NIST SP 800-26 System Questionnaire with NIST SP 800-53 References and Associated Security Control Mappings. Washington, DC: NIST. http://csrc.nist.gov/publications/drafts/Draft-sp800-26Rev1.pdf
  • Nestor Working Group on Trusted Repositories Certification. June 2006. Catalogue of Criteria for Trusted Digital Repositories. Version 1 (draft for public comment). English translation December 2006.urn:nbn:de:0008-2006060703. edoc.hu-berlin.de/series/nestor-materialien/8en/PDF/8en.pdf
  • PREMIS. May 2005. Data Dictionary for Preservation Metadata: Final Report of the PREMIS Working Group. Dublin, Ohio and Mountain View, CA: OCLC and RLG. www.oclc.org/research/projects/pmwg/premis-final.pdf
  • Task Force on Archiving of Digital Information. 1996. Preserving Digital Information. Washington, DC, and Mountain View, CA: Commission on Preservation and Access and the Research Libraries Group. http://www.rlg.org/legacy/ftpd/pub/archtf/final-report.pdf
  • Trusted Digital Repositories: Attributes and Responsibilities. May 2002. Mountain View, CA: RLG. http://www.rlg.org/en/pdfs/repositories.pdf
  • Trusted Repository Audit & Certification: Criteria and Checklist (TRAC), 2007. http://www.crl.edu/PDF/trac.pdf

-- DavidGiaretta - 20 Apr 2009

Section A: Organisational Infrastructure (BR & MEW)

A1. Governance & organizational viability

The TRAC Checklist identifies and defines the criteria that must be met by a given repository to be considered capable of managing, maintaining and preserving digital materials over time (Please note that division of the TRAC Checklist into three sections has been done for organizational purposes, and does not suggest that any single section can stand alone or should be applied without a review of all three sections). Section A addresses the administrative and structural aspects of the organization responsible for providing repository services. Criteria in this section address how the organization is governed, managed, staffed, and sustained financially, and describe activities that contribute to the organization's long term viability. The criteria in Section A relate to all operations of the repository, as robust policy infrastructure and attentiveness to the legal and financial environments in which a repository operates are essential to the implementation of adequate systems and technologies.

A1.1 Repository shall have a mission statement that reflects a commitment to the long-term retention of, management of, and access to digital information.

Supporting Text
The long-term preservation and management of digital content, and, where appropriate, provision of access to that information, should be integral parts of the repository's mission. The ideal locus for such work is within an organization that by mission is devoted to the continuity of information and other resources. This is necessary in order to ensure that the commitment at the highest organizational level, for preservation and access.
Bruce Ambacher 16April2009 - Does "highest level" refer to the organizational hierarchy or to the largest financial commitment possible for preservation activities?
DG 20090418 I assumed this is organizational aspects

Financial committment is a big part of it, but it isn't everything. mew

BA 20April2009 - This was changed per the webchat of 20April2009.
The mission shall
-Bruce Ambacher 16April2009 - "should" is used here and above and in Discussion. Is this appropriate language for ISO or should it be "must"?
DG 20090418 - need to check Pub Manual
provide a foundation for appropriate and focused repository Preservation Policy development in support of long-term planning and resource allocation.
Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Mission statement or charter of the repository or its parent organization that specifically addresses or implicitly calls for the preservation of information and/or other resources under its purview; a legal, statutory, or government regulatory mandate applicable to the repository that specifically addresses or implicitly requires the preservation of information and/or other resources under its purview.
Discussion
The repository or its parent organization's mission statement should explicitly address preservation. If preservation is not among the primary purposes of an organization that houses a digital repository then preservation may not be essential to the organization's mission. In some instances a repository pursues its preservation mission as an outgrowth of the larger goals of an organization in which it is housed, such as a university or a government agency, and its narrower mission may be formalized through policies explicitly adopted and approved by the larger organization. Government agencies and other organization's may have legal mandates that require they preserve materials, in this case these mandates can be substituted for mission statements, as they define the purpose of the organization. Mission statements should be kept up to date and continue to reflect the common goals and practices for preservation.

A1.2 Repository shall have a Preservation Strategic Plan that defines the approach the repository will take in the long-term support of its mission.

Discussion
A Strategic planning should be a printable document which define's a repository's long-term plans in support of its mission. This is necessary in order to help the repository make administrative choices, shape policies and allocate resources in order to achieve long-term preservation.
Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
One definition of a strategic plan is, "a disciplined effort to produce decisions and actions that guide and shape what the organization is, what it does, and why it does it"[1]. A strategic plan for a repository can address such issues as, the overall direction the repository is taking and its preservation goals. The plan needs to tie into the organization's established mission, and define the values, vision, and goals. Strategic plans typically cover a particular finite time period, normally in the 3-5 year range.---[1] Bryson, J. M. ""Strategic planning for public and nonprofit organizations: A guide to strengthening and sustaining Organizational achievement."" Rev. ed.. San Francisco: Jossey-Bass Publishers. 1995

A1.2.1 Repository shall have an appropriate, formal succession plan, contingency plans, and/or escrow arrangements in place in case the repository ceases to operate or the governing or funding institution substantially changes its scope.

Supporting Text
These plans, and/or escrow arrangement’s are meant to address what actions a repository takes when failure occurs due to closure or substantial changes in its funding, scope, or mission. Failure will lead to threats to the long-term sustainability of a repository’s information content. This is necessary in order to preserve the information content entrusted to the repository.
Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Succession plan(s); escrow plan(s); escrow of critical code, software, and metadata sufficient to enable reconstitution of the repository and its content with a trusted, independent agent or entity that is authorized to maintain and provide access to same in the event of repository failure; escrow and/or reserve funds set aside for contingencies; insurance policies indemnifying the repository for recovery of critical systems in the event of system failure; explicit agreements with successor organizations documenting the intent to ensure continuity of the repository, and the measures to be taken to ensure the complete and formal transfer of responsibility for the repository’s digital content and related assets, and granting the requisite rights necessary to ensure continuity of the content and repository services; documents outlining crisis management plans, disaster plans, business continuity plan etc); depositor agreements
Discussion
It is not sufficient for the repository to have an informal plan or policy regarding where its data goes should a failure occur. A formal, printable plan must be in place with identified procedures needs to be in place.

A1.2.2 The Repository shall monitor its organizational environment to determine when to execute its formal succession plan, contingency plans, and/or escrow arrangements

Supporting Text
The repository must recognize the conditions under which it may need to execute these plans (succession plan, contingency plans, and/or escrow arrangements), such as the possibility of gaps between funding and the costs of meeting the repository's commitments to its stakeholders. This is necessary because even with effective business planning procedures in place, any repository with long-term commitments will likely face some kind of resource gap in the future. If the repository cannot bridge these gaps its viability is threatened.
Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Fiscal and fiduciary policies, procedures, protocols, requirements; budgets and financial analysis documents; fiscal calendars; business plan(s); any evidence of active monitoring and changes in policys and preparedness.
Discussion
The intent of this metric is that the repository shows it is monitoring and responding to risks and bridging funding gaps before they have reached the crisis point. The respository must demonstrate it is implementing sensible solutions which protect the long term viability of the repository. This continual monitoring and revising of the repository's environment helps to avoid situations where there is a need to invoke a succession plan, such as the repository ceasing operations permanently.

A1.3 Repository shall have a Collection Policy or other document that specifies the type of information to be preserved.

Supporting Text
A collection policy guides the development and management of the digital content to be preserved by the repository. This is necessary in order to ensure the repository is collecting materials which support its mission. The collection policy is written and may address any or all of the following areas: selection criteria (data formats, subject matter, languages, producers, etc), acquisition criteria (rights issues, access limitations, data quality, cost thresholds for purchase of digital content, etc.)
Bruce Ambacher 16April2009 - I question the need or desirability of putting "cost threshholds" and "access policies" in the Collection Pplicy. This is about collection policy not other repository procedures such as data purchase and/or acwho can access the data
DG 20090418 - no strong feelings about this but accept Bruce's view

MEW-I have come across repository's that sdo incldue this as part of their collection policy because it is an important way to explain why somethign never made it into the policy, the above are only examples, and I think it helps to illustrate the diversity of collection fevelopment policy's
protocols for selecting digital content for inclusion in the repository, access policies, benefits of depositing in the repository, and a definition of the designated community who will produce the information. The policy should also specify under what circumstances and how often the policy is to be reviewed.
Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
A collection policy and supporting documents, preservation policy,
Bruce Ambacher 16April2009 - Again I question whether preservation policy should be part of the Collection Policy and how it is evidence of the Collection Policy.
DG 20090418 - OK

MEW-Took it out.
mission, goals and vision of the repository, and definition of the designated community are necessary in order to define what the repository preserves and how the needs of the designated community are to be met by repository activities.
Discussion
The Collection policy is one of the most important documents the repository produces because it defines what the repository is storing and why. The Collection Policy supports the broader mission of the repository. Without such a policy the repository is likely to collect in a haphazard manner, or store large amounts of low-value digital content. The Collection Policy helps the organization identify what digital content it will and will not accept for ingestion. In an organization with a broader mission than preservation of digital content the collection policy helps to define the role of the repository within the larger organizational context. The Collection Policy should be reviewed with new digital content producers to ensure the repository is following its policies and that the producer understands the requirements for the repository’s designated community. The policy should be useable in order to understand what the repository holds, what it does not hold, and why. There are differences in the criteria for selection of data content. For example, if an object is in a language or subject area in which the repository is collecting it will be eligible for ingestion, but if it costs too much, or the data quality is bad it may not be possible to ingest it. Selection criteria are the "does it fit into the collection criteria" while acquisition criteria are about "is it practical for the object to be included in the collection." The distinction is important for explaining why something wasn’t ingested into the repository.

A2 Organizational structure & staffing

Discussion
Staffing of the repository must be by professionals with the required training and skills to carry out the activities of the repository. The repository must be able to document through development plans, organizational charts, job descriptions and related policies and procedures that the repository is defining and maintaining the skills and roles that are required for the sustained operation of the repository.

A2.1 Repository shall identify and establish the duties that it needs to perform and appointed staff with adequate skills and experience to fulfil these duties.

Discussion
Staffing of the repository must be by professionals with the required training and skills to carry out the activities of the repository. The repository must be able to document through development plans, organizational charts, job descriptions and related policies and procedures that the repository is defining and maintaining the skills and roles that are required for the sustained operation of the repository.

A2.1.1 Repository shall identify and establish the duties their staff need to perform.

Supporting Text
The repository shall identify the competencies and skill sets required to carry out its activities over time and must demonstrate that the staff and consultants have the appropriate range of requisite skills—e.g., archival training, technical skills, and legal expertise. This is necessary in order to ensure that the repository can complete all tasks associated with the long-term preservation and management of the data objects.
Bruce Ambacher 16April2009 - Some word(s) is missing here and after "that".
DG 20090418 - "that" should probably be replaced by "the data objects"

MEW-changed"
and provide appropriate access to that . Preservation depends upon a range of activities from maintaining hardware and software to migrating content and storage media to negotiating intellectual property rights agreements. In order to ensure long-term sustainability, a repository must be aware of all required activities and demonstrate that it can successfully complete them.
*Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
A staffing plan; competency definitions; job descriptions; staff professional development plans; certificates of training and accreditation; plus evidence that the repository reviews and maintains these documents as requirements evolve.
Discussion Preservation depends upon a range of activities from maintaining hardware and software to migrating content and storage media to negotiating intellectual property rights agreements. In order to ensure long-term sustainability, a repository must be aware of all required activities and demonstrate that it can successfully complete them.

A2.1.2 Repository shall have the appropriate number of staff to support all functions and services.

Supporting Text
The repository must maintain staffing that is adequate for the scope and mission of preservation.
Bruce Ambacher 16April2009 - I prefer "preservation" over "archiving".
DG 20090418 - agree

mew-done
program. This is necessary in order to ensure repository staffing levels are adequate for preserving the digital content and providing a secure, quality repository.
Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Organizational charts; definitions of roles and responsibilities; comparison of staffing levels to industry benchmarks and standards.
Discussion
Technology and general practices for digital preservation will continue to change, so the repository must ensure that its staff skill sets evolve, as will the requirements of its designated community(ies). The repository should determine the appropriate number and level of staff that corresponds to requirements and commitments through reference to established industry benchmarks. The repository should also demonstrate how it evaluates staff effectiveness and suitability to support its functions and services.
-- KatiaThomaz - 10 May 2007 - It lacks %u201Crepository evaluate staff effectiveness and suitability%u201D (see DCC/DPE DRAMBORA R24).

A2.1.3 Repository shall have an active professional development program in place that provides staff with skills and expertise development opportunities.

Supporting Text
Technology and general practices for digital preservation will continue to change, so the repository must ensure that its staff skill sets evolve, as will the requirements of its designated community(ies)
Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Professional development plans and reports; training requirements and training budgets, documentation of training expenditures (amount per staff); performance goals and documentation of staff assignments and achievements, copies of certificates awarded.
Discussion
Ideally the repository will meet this requirement through a lifelong learning approach to developing and retaining staff.

A3 Procedural accountability & Preservation Policy framework

Documentation assures stakeholders (consumers, producers, and contributors of digital content) that the repository is meeting its requirements and fully performing its role as a trusted digital repository. A repository must create documentation that reflects its Mission and Strategic plan and captures its habitual activities. This entails documenting all repository processes, decision-making, and goal setting. Documentation is provided so that the activities of the repository will be understood by stakeholders and management. It ensures that repository policies and procedures are carried out in approved, consistent ways, resulting in long-term preservation and access to digital content in its care. Certification, the clearest indicator of a repository’s sound and standards-based practice, is facilitated by procedural accountability and documentation.

A3.1 Repository shall define its designated community(ies) and associated knowledge base(s) and has these definitions appropriately accessible .

Supporting Text
It is the repository’s responsibility to determine and document the community(ies) to be served and the requirements for service. This is necessary in order to meet the needs of the designated community.
Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
A written definition, in a standalone document or contained in the Collection policy; listing or register of contributors, subscribers.
Discussion
The designated community is defined as “a repository’s identified group of potential consumers who should be able to understand a particular set of information. The designated community may be composed of multiple user communities.”(TRAC Glossary) The producers or contributors of the content held by a repository may also be considered a designated community.

A.3.2 Repository shall have Preservation Policies in place to describe how its Preservation Strategic Plan will be met.

Supporting Text
The repository must have Preservation Policies in place that define its technical infrastructure, its services, and the expected level of understandability for each of its designated community(ies) for each Archival Information Product. This is necessary in order to ensure that each designated community knows the operational definition of understandability for its community and the knowledge base each user must possess in order to understand the preserved
Bruce Ambacher 16April2009 - I prefer "preserved" over "archived"
DG 20090418 - agree

MEW-done
content. This allows the repository to provide appropriate levels of service to each designated community while keeping the necessary resource commitment to a minimum.
Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Mission statement; written definitions of the designated community(ies); documented policies; protocols, rules, manuals, handbooks, and workflows; service-level agreements.
Discussion
The repository's level of service can vary from one submission to another, just as the definition of understandability that establishes the repository’s responsibility in this area may vary. This may range from no responsibility, if bits are only to be preserved, to the maintenance of a particular level of use, if understanding by the members of the designated community(ies) is determined outside the repository, to a responsibility for ensuring a given level of designated community(ies) human understanding, requiring appropriate Representation Information. The documentation of understandability will typically include a definition of the applications the designated community(ies) will use with the information, possibly after transformation by repository services. For example, if a designated community is defined as readers of English with access to widely available document rendering tools, and if this definition is clearly associated with a given set of Content Information and Preservation Description Information, then the requirement is met. Examples of designated community definitions include: General English-reading public educated to high school and above, with access to a Web Browser (HTML 4.0 capable). For GIS data: GIS researchers%u2014undergraduates and above%u2014 having an understanding of the concepts of Geographic data and having access to current (2005, USA) GIS tools/computer software, e.g., ArcInfo? (2005). Astronomer (undergraduate and above) with access to FITS software such as FITSIO, familiar with astronomical spectrographic instruments. Student of Middle English with an understanding of TEI encoding and access to an XML rendering environment. Variant 1: Cannot understand TEI Variant 2: Cannot understand TEI and no access to XML rendering environment Variant 3: No understanding of Middle English but does understand TEI and XML Two groups: the publishers of scholarly journals and their readers, each of whom have different rights to access material and different services offered to them. ...
-- KatiaThomaz - 28 Apr 2008 - It lacks %u201Crepository has defined the external parties%u201D (see ISO27001 A.6.2), and %u201Crepository has defined its assets, owners and uses%u201D (see ISO27001 A.7).

A3.2.1 Repository shall have mechanisms for reviewing, updating, and ongoing development of its Preservation Policies as the repository grows and as technology and community practice evolve.

Supporting Text
The repository must have up-to-date, complete policies and procedures in place that reflect the current requirements and practices of its community(ies) for preservation. These policies must be understandable by the repository staff. Preservation Policies and Strategic Plans are high level documents that capture organizational commitments and intents for staffing, security and other preservation-related concerns. Preservation Implementation Plans address such preservation activities and practices as transfer, submission, quality control, storage management, metadata management, and access and rights management.
Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Current and past written documentation in the form of Preservation Policies, Preservation Strategic Plans and Preservation Implementation Plans, procedures, protocols, and workflows; specifications of review cycles for documentation; documentation detailing reviews, surveys and feedback. If documentation is embedded in system logic, functionality should demonstrate the implementation of policies and procedures.
Discussion
The repository must manage all versions of these preservation documents (e.g., outdated versions are clearly identified and maintained in some organized way) and qualified staff and peers must be involved in reviewing, updating, and expanding these documents. Older versions must be preserved in order to document the results of monitoring for new developments; responsiveness to prevailing standards and practice, emerging requirements, and standards that are specific to the domain, if appropriate; and similar developments. Preservation Policies and procedures must be demonstrated to be understandable and implementable. Appendix 3 explains the Minimum Required Documents.

A3.2.2 Repository shall maintain policies that specify the nature of any legal permissions required to preserve digital content over time, and repository can demonstrate that these permissions have been acquired when needed.

Supporting Text
The repositories must have written policies and agreements with depositors that specify and/or transfer certain rights to the repository enabling appropriate and necessary preservation actions to take place on the digital objects within the repository. This is necessary in order to maintain and modify digital objects to keep them accessible over time, a right often restricted by law to the creator.
Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Deposit agreements; licenses and written grants and/or assignments of rights; records schedule; digital preservation policies; records legislation and policies; service agreements.
Discussion
A repository should obtain the rights necessary to limit if not eliminate liability or legal exposure stemming from the repository’s preservation activities. Rights are best obtained at or before the time of ingest since it is often difficult to secure them later when the authors may no longer be available
-- KatiaThomaz - 28 Apr 2008 - Does it include IPR, legislative requirements, and regulatory requirements (see DCC/DPE DRAMBORA R15, R17, R18 and ISO27001 A15)?

A3.3 (A3.6) Repository shall have a documented history of the changes to its operations, procedures, software, and hardware.

Supporting Text
The repository must document all activities and developments over time that may have affected its management and preservation. This should include decisions about the organizational and technical infrastructure. This is necessary in order to provide an “audit trail” through which stakeholders can identify and trace decisions made by the repository.
Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Capital equipment inventories; documentation of the acquisition, implementation, update, and retirement of critical repository software and hardware file retention and disposal schedules and policies.
Discussion
Documentation should be in printable form with some explanation of local practices given by appropriate staff.

A3.4 (A3.7) Repository shall commit to transparency and accountability in all actions supporting the operation and management of the repository that affect the preservation of digital content over time.

Supporting Text
The repository must show a commitment to transparency and accountability through providing reasonable access to its content and to whatever documentation gives information about the development, implementation, evolution, preservation and performance of the repository. This is necessary because transparency is the best assurance that the repository operates in accordance with accepted standards and practices.
Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Reports of financial and technical audits and certifications; disclosure of governance documents, independent program reviews, and contracts and agreements with providers of funding and critical services.
Discussion
If the repository uses software to document its history, it should be able to demonstrate these tracking tools. Where appropriate, the history is linked to relevant preservation strategies and describes potential effects on preserving digital content. This requirement does not mean that the organization must make information which would make it vulnerable to competitors available, but rather that the organization commits to disclosing its methods for preserving digital content at least to the designated community or other appropriate stakeholder in order to demonstrate that is meeting all current preservation requirements.

A3.5 (A3.8) Repository shall commit to defining, collecting, tracking, and appropriately providing its information integrity measurements.

Supporting Text
The repository must provide documentation that it has developed or adapted appropriate measures for ensuring the integrity of its holdings. This is necessary in order to ensure that documentation exists regarding how the loss of content or compromise of the integrity of the repository is prevented.
Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Written definition or specification of the repository's integrity measures (for example, computed checksum or hash value); documentation of the procedures and mechanisms for monitoring integrity measurements and for responding to results of integrity measurements that indicate digital content is at risk; an audit process for collecting, tracking and presenting integrity measurements; Preservation Policy and workflow documentation.
Discussion
We know that the mechanisms to measure integrity will evolve as technology evolves. If protocols, rules and mechanisms are embedded in the repository software, there should be some way to demonstrate the implementation of integrity measures.

A3.6 (A3.9) Repository shall commit to a regular schedule of self-assessment and certification.

Supporting Text
The repository must commit to regular, periodic self-assessment and certification as a way of systematically and objectively verifying that it fulfills its preservation requirements. This is necessary in order to ensure the repository is trustworthy
Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Completed, dated checklists from self-assessments and/or third-party audits; certificates awarded for compliance with relevant ISO standards; timetables and evidence of adequate budget allocations for future certification.

A4 Financial sustainability

A4.1 The Repository shall have short- and long-term business planning processes in place to sustain the repository over time.

Supporting Text
The repository must demonstrate that it has formal, cyclical (at minimum, annual), and proactive business planning processes in place. This is necessary in order to ensure the viability of the repository over the period of time it has promised to provide access to its contents for its designated community(ies).
Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Up-to-date, multi-year strategic, operating and/or business plans; audited annual financial statements; financial forecasts with multiple budget scenarios; contingency plans; market analysis.

A4.2 (A4.3) The Repository shall have financial practices and procedures which are transparent, compliant with relevant accounting standards and practices, and audited by third parties in accordance with territorial legal requirements.

Supporting Text
The repository cannot simply claim transparency, but must show that it adjusts its business practices to keep them transparent, compliant, and auditable. This is necessary in order to guard against malfeasance or other untoward activity that might threaten the economic viability of the repository.
Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Demonstrated dissemination requirements for business planning and practices; citations to and/or examples of accounting and audit requirements, standards, and practice; audited annual financial statements.
Bruce Ambacher 16April2009 - This entire text is a repeat of the previous. with the addition of Discussion

A4.3 (A4.4) The Repository shall have an ongoing commitment to analyze and report on risk, benefit, investment, and expenditure (including assets, licenses, and liabilities).

Supporting Text
The repository must commit to at least these categories of analysis and reporting, with the goal, of maintaining an appropriate balance between risk and benefits, investment and return. This is necessary in order to demonstrate that the repository has identified and documented these categories, and actively manages them, including identifying and responding to risks, describing and leveraging benefits, specifying and balancing investments, and anticipating and preparing for expenditures.
Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Risk management documents that identify perceived and potential threats and planned or implemented responses (a risk register); technology infrastructure investment planning documents; cost/benefit analyses; financial investment documents and portfolios; requirements for and examples of licenses, contracts, and asset management; evidence of revision based on risk.

A5 Contracts, licenses, & liabilities

A5.1 The Repository shall have and maintain appropriate contracts or deposit agreements for digital materials that it manages, preserves, and/or to which it provides access.

Supporting Text
The repository must be able to produce/execute and adhere to
Bruce Ambacher 16April2009 - I suggest replacing "produce" with "execute and adhere to"
DG 20090418 - could argue that "produce" is needed here because archive must be able to show them - BUT probably should also need to be able to show evidence that the contracts have been followed
relevant contracts, licenses, and/or deposit agreements that express rights, responsibilities, and expectations of each party for digital materials that the repository manages and preserves and to which it provides access. This is necessary in order to ensure that the repository has the rights and authorizations needed to enable it to collect and preserve digital content over time, make that information available to its designated community(ies), and defend those rights when challenged.
Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Properly signed and executed deposit agreements and licenses; Preservation Policies on third-party deposit arrangements; definitions of service levels and permitted uses; repository policies on the treatment of “orphan works” and copyright dispute resolution; reports of independent risk assessments of these policies; procedures for regularly reviewing and maintaining agreements, contracts, and licenses.
Discussion
Repositories, especially those with third-party deposit arrangements, should guarantee that relevant contracts, licenses, or deposit agreements express rights, responsibilities, and expectations of each party. The repository must show evidence that the contracts have been followed. Contracts and formal deposit agreements should be countersigned and current. When the relationship between depositor and repository is less formal (i.e., a faculty member depositing work in an academic institution's preservation repository), documentation articulating the repository's capabilities and commitments should be provided to each depositor. Repositories engaged in Web archiving may find this requirement difficult because of how Web-based information is harvested/captured for long-term preservation. This kind of data is rarely acquired with contracts or deposit agreements. By its very nature, digital information on the Web is perceived to belong to "everyone and no one." Some repositories capture, manage, and preserve access to this material without written permission from the content creators. Others go through the very time-consuming and costly process of contacting content owners before capturing and ingesting information. Ideally, these agreements will be tracked, linked, managed, and made accessible in a contracts database.

A5.1.1 The Repository shall have contracts or deposit agreements which specify and transfer all necessary preservation rights, and these transferred rights shall be documented.

Supporting Text
The repository shall possess at least minimal preservation rights for the objects that it accepts. This is necessary in order to have sufficient control of the information and limit the repository’s exposure to liability or legal and financial harm that might threaten its viability.
Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Contracts, deposit agreements; specification(s) of rights transferred for different types of digital content (if applicable); policy statements on requisite preservation rights.
Discussion
Because the right to change or alter digital information is often restricted by law to the creator, it is important that digital repository contracts and agreements address the need to be able to work with and potentially modify digital objects to keep them accessible. Repository agreements with depositors must specify and/or transfer certain rights to the repository enabling appropriate and necessary preservation actions for the digital objects within the repository (this requirement is linked to A3.3.) Because legal negotiations can take time, potentially preventing or slowing the ingest of digital objects at risk, it is acceptable for a digital repository to take in or accept digital objects even with only minimal preservation rights using an open-ended agreement and then deal with expanding to detailed rights later.

A5.1.2 Repository shall have specified all appropriate aspects of acquisition, maintenance, access, and withdrawal in written agreements with depositors and other relevant parties.

Supporting Text
The repository must possess written agreements with producers and contributors of digital content it accepts that specify appropriate responsibilities for the acquisition, maintenance, access, and withdrawal of such information or that demonstrate that it does not need such agreements. This is necessary in order to ensure that the respective roles of repository, producers and contributors in the depositing of digital content and transfer of responsibility for preservation are understood and accepted by all parties
Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Properly executed submission agreements, deposit agreements, and deeds of gift; written standard operating procedures
Discussion
The deposit agreement specifies all aspects of these issues that are necessary for the repository to carry out its function. There may be a single agreement covering all deposits, or specific agreements for each deposit, or a standard agreement supplemented by special conditions for some deposits. These special conditions may add to the standard agreement or override some aspects of the standard agreement. Agreements may need to cover restrictions on access and will need to cover all property rights in the digital objects. Agreements may place responsibilities on depositors, such as ensuring that Submission Information Packages (SIPs) conform to some pre-agreed standards, and may allow repositories to refuse SIPs that do not meet these standards. Other repositories may take responsibility for fixing errors in SIPs. The division of responsibilities must always be clear. Agreements, written or otherwise, may not always be necessary. The burden of proof is on the repository to demonstrate that it does not need such agreements for instance, because it has a legal mandate for its activities. An agreement should include, at a minimum, property rights, access rights, conditions for withdrawal, level of security, level of finding aids, SIP definitions, time, volume, and content of transfers. One example of a standard to follow for this is the CCSDS/ISO Producer- Archive Interface Methodology Abstract Standard.

A5.1.3 (was B1.8) The Repository shall have written policies that indicate when it accepts preservation responsibility for contents of each set of submitted data objects

Supporting Text
The repository must ensure that the point at which it accepts responsibility for the preservation of digital content is evident to all producers and depositors. This is necessary in order to avoid misunderstandings between the repository and producer/depositor as to when and how the transfer of responsibility for the digital content occurs.
Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Properly executed submission agreements, deposit agreements, and deeds of gift; confirmation receipt sent back to producer/depositor.
Discussion
If this requirement is not met, there is a risk that, for example, the original is erased before the repository has taken responsibility for the submitted data objects. Without the understanding that the repository has already taken preservation responsibility for the SIP, there is the risk that the producer/depositor may make changes to the data and these would not be properly archived since they had already been ingested by the repository. For example, for convenience the repository could receive a copy of raw science data from the instrument at the same time the science team gets it, but the science team would have responsibility for it until they turn over responsibility to the final repository. Repositories that report back to their depositors generally will mark this acceptance with some form of notification (for example, confirmation receipts) to the depositor. (This may depend on repository responsibilities as designated in the depositor agreement.) A repository may mark the transfer by sending a formal document, often a final signed copy of the transfer agreement, back to the depositor signifying the completion of the transformation from SIP to AIP process. Other approaches are equally acceptable. Brief daily updates may be generated by a repository that only provides annual formal transfer reports.

A5.1.4 The repository shall have Policies in place to address liability and challenges to ownership/rights.

Supporting Text
The repository must possess Preservation Policies and Preservation Implementation Plans and mechanisms that have been vetted by the appropriate institutional authorities and/or legal experts. This is necessary in order to minimize potential liability and challenges to the rights of the repository
Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
A definition of rights, licenses and permissions to be obtained from producers and contributors of digital content; citations to relevant laws and regulations; policy on responding to challenges; documented track record for responding to challenges in ways that do not inhibit preservation; records of relevant legal advice sought and received.
Discussion
The repository's Preservation Policies and Preservation Implementation Plans and mechanisms must be vetted by appropriate institutional authorities and/or legal experts to ensure that responses to challenges adhere to relevant laws and requirements, and that the potential liability for the repository is minimized.

A5.2 The repository shall track and manage intellectual property rights and restrictions on use of repository content as required by deposit agreement, contract, or license.

Supporting Text
The repository must have mechanisms that are sufficient for the institution to track, act on, and verify rights and restrictions related to the use of the digital objects within the repository. This is necessary in order to determine the rights and restrictions for the use of the digital objects within the repository.
Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
A Preservation Policy statement that defines and specifies the repository’s requirements and process for managing intellectual property rights; depositor agreements; samples of agreements and other documents that specify and address intellectual property rights; documentation of monitoring by repository over time of changes in status and ownership of intellectual property in digital content held by the repository ; results from monitoring, metadata that captures rights information.
Discussion

-- MarkConrad - 10 Sep 2007 Need to include a reference to the law as well. Have to be able to track legal requirements that impact the jurisdiction of the repository and the depositors (These may be a different set.).
-- The repository should have a mechanism for tracking licenses and contracts to which it is obligated. Whatever the format of the tracking system, it must be sufficient for the institution to track, act on, and verify rights and restrictions related to the use of the digital objects within the repository. -RD
Bruce Ambacher 16April2009 - I suggest the RD comment be moved up into Supporting Text.
DG 20090418 - looks as if this metric needs to be reviewed


Section B: Digital Object Management

B1 Ingest: acquisition of content (RF)

B1.1 Repository identifies the Content Information and the properties of that information that the repository will preserve.


Supporting Text
The repository must define explicitly what properties of Content Information that or information content which must be preserved over the long term. This is necessary in order to make it clear to funders, depositors and users what responsibilities the repository is taking on and what aspects are excluded. It is also a necessary step in defining the information which is needed from the information producers or depositors.
Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Mission statement; submission agreements/deposit agreements/deeds of gift; workflow and Preservation Policy documents, including written definition of properties as agreed in the deposit agreement/deed of gift; written processing procedures; documentation of properties to be preserved.
Discussion
This process begins in general with the repository's mission statement and may be further specified in pre-accessioning agreements with producers or depositors (e.g., producer-archive agreements) and made very specific in deposit or transfer agreements for specific digital objects and their related documentation. For example, one repository may only commit to preserving the textual content of a document and not its exact appearance on a screen. Another may wish to preserve the exact appearance and layout of textual documents, while others may choose to keep the units of the measurement of data fields and to normalize the data during the ingest process. If unique identifiers are associated with digital objects before ingest, they may also be significant properties that need to be preserved.

B1.1.1 The repository has a procedure(s) for identifying those properties that it will preserve.


Supporting Text
The repository must explicity define the procedure(s) it uses to determine the properties of Content Information that it will preserve. This is necessary to establish a clear understanding with depositors, funders, and the repository's designated communities how the repository determines what the characteristics and properties of preserved items will be over the long term. These procedures will be necessary to confirm provenance or to identify erroneous provenance of the preserved digital record.
Examples of Ways the Repository Can Demonstrate It Is Meeting This Requirement
Submission agreements/deposit agreements, Preservation Policies, written processing procedures, workflow documentation.
Discussion
These procedure(s) document the methods and factors a repository uses to determine the aspects of different types of Content Information for which it accepts preservation responsibility to its designated communities. For example, a repository's
BA 16April2009 - Insert "practice" here
may be to use file formats in order to determine the properties it will preserve unless otherwise specified in a deposit agreement. In this case, the repository would be able to demonstrate provenance for objects that may have been in the same file format when received but are preserved differently over the long term.

B1.1.2 The repository has a record of the Content Information and the properties of that information that it will preserve.


Supporting Text
The repository identifies in writing the Content Information of the records for which it has taken preservation responsibility and the properties it has committed to preserve for those records based on their Content Information.
Examples of Ways the Repository Can Demonstrate It Is Meeting This Requirement
Preservation Policies, processing manuals, collection inventories or surveys, logs of Content Information types acquired, preservation strategies and action plans.
Discussion
The repository must demonstrate that it establishes and maintains an understanding of its digital collections sufficient to carry out the preservation necessary to persist the properties to which it has committed. The repository can use this information to determine the effectiveness of its preservation activities over time.

B1.2 Repository clearly specifies the information that needs to be associated with specific Content Information at the time of its deposit.


Supporting Text
The repository must explicitly specify what information is needed from the content provider.
Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Transfer requirements; producer-archive agreements. Workflow plans to produce the AIP.
Discussion
For most types of digital objects to be ingested, the repository should have written criteria, prepared by the repository on its own or in conjunction with other parties, that specify exactly what digital object(s) are transferred, what documentation is associated with the object(s), and any restrictions on access, whether technical, regulatory, or donor-imposed. This criteria documents what information the repository and its designated communities may expect for digital object(s) upon deposit. Note that the depositor may be a harvesting process created by the repository. The level of precision in these specifications will vary with the nature of the repository's collection policy and its relationship with creators. For instance, repositories engaged in Web harvesting, or those that rescue digital materials long after their creators have abandoned them, cannot impose conditions on the creators of material, since they are not "depositors" in the usual sense of the word. But Web harvesters can, for instance, decide which metadata elements from the HTTP transactions that captured a site are to be preserved along with the site's files, and this still constitutes "information associated with the digital material." They may also choose to record the information or decisions -whether taken by humans or by automated algorithms- that led to the site being captured. The repository can check what it receives from the producer based on the specifications.

B1.3 Repository has specifications enabling recognition and parsing of the SIPs.


Supporting Text
The repository has explicit written specifications for the file types and/or object types that are transferred so that it can identify the construction of ingested SIPs and verify the correctness and completeness of the SIP components and verify that the SIP overall contains the Content Information declared by the producer/depositor.
Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Documented file format specifications; published data standards; documentation of valid object construction.
Discussion
The repository must be able to determine what the contents of a SIP are with regard to the technical construction of its components. For example, the repository needs to be able to recognize a TIFF file and confirm that it is not simply a file with a filename ending in "TIFF." Another example, would be a website for which the repository would need to be able to recognize and test the validity of the variety of file types (e.g., HTML, images, audio, video, CSS, etc.) that are part of the website. This is necessary in order to confirm: 1) the SIP is what the repository expected; 2) the Content Information is correctly identified; and 3) the properties of the Content Information to be preserved have been appropriately selected.

B1.4 Repository has mechanisms to appropriately verify the depositor of all materials.


Supporting Text
The repository must ensure that the sources of the objects it intends to preserve are who/what they claim to be. This is necessary in order to avoid providing erroneous provenance to the information which is preserved.
Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Legally binding submission agreements/deposit agreements/deeds of gift, evidence of appropriate technological measures; logs from procedures and authentications, legally binding submission agreements/deposit agreements/deeds of gift
Discussion
The repository's written standard operating procedures and actual practices must ensure the digital objects are obtained from the expected sourcedepositor. Examples of a depositor include persons, organizations, corporate entities, or harvesting processes.Written procedures and practices are necessary to demonstrate that the provenance that has been maintained prior to submission. Confirmation can use various means including, but not limited to, digital processing and data verification and validation, and through exchange of appropriate instruments of ownership (e.g., legally binding submission agreements/deposit agreement/deed of gift). Different repositories will adopt different levels of proof needed; the Designated Community should have the opportunity to review the evidence.
-- KatiaThomaz - 20 Mar 2008 - What does "source of all materials" really mean? A physical or corporate person responsible for issuing the materials?

B1.5 Repository's ingest process verifies each SIP for completeness and correctness.


Supporting Text
The repository must verify, as far as it can, that each SIP is complete and correct. This is necessary in order to detect and correct potential transmission errors between the depositor and the repository.
Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Evidence that the repository checks the information that needs to be associated with digital material at the time of its deposit against the SIP. Appropriate Preservation Policy and Preservation Implementation Plan documents and system log files from system(s) performing ingest procedure(s); formal or informal "acquisitions register"logs or registers of files received during the transfer and ingest process; workflow, documentation of standard operating procedures, detailed procedures, and/or workflows<; format registries; definitions of completeness and correctness, probably incorporated in Preservation Policy documents.
Discussion
Information collected during the ingest process must be compared with information from some other source --the producer or the repository's own expectations-- to verify the correctness of the data transfer and ingest process. Other sources will include technical and descriptive metadata obtained prior to ingest and may also include expectations set by the depositor, the object producer, a format registry, or the repository's own expectations. The extent to which a repository can determine correctness will depend on what it knows about the SIP and what tools are available for verifying correctness. It can mean simply checking that file formats are what they claim to be (TIFF files are valid TIFF format, for instance), or can imply checking the content. This might involve human checking in some cases, such as confirming that the description of a picture matches the image. This allows the repository to demonstrate that its preserved objects have completeness and correctness, having originated from complete and correct SIPs. It also allows the repository to document reasons for other SIP related actions such as Repositories should have established procedures for handling incomplete SIPs. These can range from rejecting the transfer, to suspending processing until the missing information is received, orto simply reporting the errors. Similarly, the definition of "completeness" should be appropriate to a repository's activities. If an inventory of files was provided by a producer as part of pre-ingest negotiations, one would expect checks to be carried out against that inventory. But for some activities such as Web harvesting, "complete" may simply mean "whatever we could capture in the harvest session." Whatever checks are carried out must be consistent with the repository's own documented definition and understanding of completeness and correctness. One thing that a repository might want to do is check for network drop out or other corruption during the transmission process. %NOTE% B1.2 does not specify everything about completeness - only what needs to accompany the deposited info. %ENDNOTE%
-- JohnGarrett - 31 Mar 2008 Should this note be here or in B1.2? ---Bruce Ambacher - Does this function check the SIP against the metadata? The Evidence
Discussion
includes other aspects beyond those stated.

B1.6 Repository obtains sufficient control over the Digital Objects to preserve them.


Supporting Text
The repository must have adequate control over the bits which make up the digital objects. This is necessary in order to ensure that the most basic type of preservation, namely bit preservation, is assured.

**Note: We might want to come back to this. (First discussed 3/29/08 meeting)
Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Documents showing the level of physical control the repository actually has. A separate database/metadata catalog listing all of the digital objects in the repository and metadata sufficient to validate the integrity of those objects (file size, checksum, hash, location, number of copies, etc.) Documentation of the level(s) of access staff, contractors, and/or similar persons/organizations have to storage media and systems containing the Digital Objects.
Discussion
The repository must obtain complete control of the bits of the digital objects conveyed with each SIP. For example, some SIPs may only reference digital objects and in such cases the repository must get the referenced digital objects if they constitute part of the object that the repository has committed to conservepreserve. It is not always the case that referenced digital objects are preserved. For example a decision needs to be made if just an email message is to be the preserved object or if it is the email message with the attachments. In the latter case, the repository might, for example, need to go to a separate directory and pick up the attachment also.

Bruce Ambacher - Katia's question is very pertinent. Under Examples we should consider adding the crossed out examples to "documents . . .such as ..." In the
Discussion
the insert should read "It is not always the case"
Ricc Ferrante - I wonder if Katia's question got lost in reformatting...

B1.7 Repository provides producer/depositor with appropriate responses at agreed points during the ingest processes.


Supporting Text
The repository must provide responses to the producer/depositor at agreed points. This is necessary in order to ensure that the producer can verify that there is no inadvertent lapses in communications which might otherwise allow loss of SIPs.
Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Submission agreements/deposit agreements/deeds of gift; workflow documentation; standard operating procedures; evidence of "reporting back" such as reports, correspondence, memos, or emails.
*Discussion*
Based on the initial processing plan and agreement between the repository and the producer/depositor, the repository must provide the producer/depositor, if it is appropriate to have such a plan, with progress reports at agreed points throughout the ingest process. Responses can include initial ingest receipts, or receipts that confirm that the AIP has been created and stored. Repository responses can range from nothing at all to predetermined, periodic reports of the ingest completeness and correctness, error reports and any final transfer of custody document. Producers/Depositors can request further information on an ad hoc basis when the previously agreed upon reports are insufficient.

B1.8 (was B1.9) Repository has contemporaneous records of actions and administration processes that are relevant to content acquisition.


Supporting Text
The repository must document relevant events as they happen. This is necessary in order to avoid such documentation, which might be evidence in an audit, from being omitted or erroneous or of questionable authenticity. This is necessary to ensure that documentation which may be needed in an audit is captured and is accurate.
Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Written documentation of decisions and/or action taken; preservation metadata logged, stored, and linked to pertinent digital objects, confirmation receipts sent back to providers.
Discussion
These records must be created on or about the time of the actions they refer to and are related to actions taken during the Ingest:
BA 16April2009 - I suggest inserting "e.g."
DG 20090418 - OK

content acquisition process. The records may be automated or may be written by individuals, depending on the nature of the actions described. Where community or international standards are used, the repository must demonstrate that all relevant actions are carried through.

B2 Ingest: creation of the AIP (BA)

B2.1 Repository has an associated, printable definition for each AIP or class of AIPs preserved by the repository that is adequate to fit long-term preservation needs.

Supporting Text
This is necessary to ensure that the AIP and its associated definition can always be found and managed within the archives.

B2.1.1 The repository must be able to demonstrate which definition applies to which AIP.

Supporting Text
This is necessary to ensure each AIP can be properly parsed/interpreted.

Examples of Ways the Repository can Demonstrate it is Meeting this Requirement

Documentation clearly linking each definition to a specific AIP or class of AIPs and linking that AIP(s) back to the SIP(s) from which it was created.

Supporting Text

It is necessary to create definitions for each AIP and to demonstrate the clear persistent link between the definition, the SIP(s), and the AIP(s) that were created from the SIP(s)

B2.1.2 The repository must be able to demonstrate that the definition of each AIP is adequate for long term preservation by demonstrating that it has all the required components, each of which can be maintained over time.

Supporting Text
This is necessary in order to explicitly show that the AIPs are fit for purpose, that each component of an AIP has been adequately thought through and the plans for the maintenance of each AIP are in place. (See B.3 Preservation planning, below)

Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Documentation identifying each class of AIP and describing how each is implemented within the repository. Implementations may, for example, involve some combination of files, databases, and/or documents. Documentation that relates the AIP component%u2019s contents to the related preservation needs of the repository, with enough detail for the repository's providers and consumers to be confident that the significant properties of AIPs will be preserved. It should be clear how AIP components such as Representation Information and Provenance can be managed and kept up to date. It should be clear when new versions of AIPs need to be created in order to keep them fit for purpose. The external dependencies of the AIP should also be recorded.


Discussion
It is necessary that definitions exist for each AIP, or class of AIP if there are many instances of the same type. Repositories that store a wide variety of object types may need a specific definition for each AIP they hold, but it is expected that most repositories will establish class descriptions that apply to many AIPs. It must be possible to determine which definition applies to which AIP. It may also be necessary for the definitions to say something about the semantics or intended use of the AIPs if this could affect long-term preservation decisions. For example, say two repositories both only preserve digital still images, both using multi- image TIFF files as their preservation format. Repository 1 consists entirely of real-world photographic images intended for viewing by people and has a single definition covering all of its AIPs. (The definition may refer to a local or external definition of the TIFF format.) Repository 2 contains some images, such as medical x-rays, that are intended for computer analysis rather than viewing by the human eye, and other images that are like those in Repository 1. Repository 2 should perhaps define two classes of AIPs, even though it only uses one storage format for both. A future preservation action may depend on the intended use of the image%u2014an action that changes the bit-depth of the image in a way that is not perceivable to the human eye may be satisfactory for real-world photographs but not for medical images, for example. An AIP contains these key components: the primary data object to be preserved, its supporting Representation Information (format and meaning of the format elements), and the various categories of Preservation Description Information (PDI) that also need to be associated with the primary data object: Fixity, Provenance, Context, and Reference. There should be a definition of how these categories of information are linked.

B2.2 Repository has a description of how AIPs are constructed from SIPs.

Supporting Text
The repository must be able to show how the AIP(s) is constructed from the SIP(s). This is necessary in order to ensure that the AIP(s) adequately represents the information in the SIP(s).

Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Process description documents; documentation of SIP relationship to AIP; clear documentation of how AIPs are derived from SIPs; documentation of standard/process against which normalization occurs; documentation of normalization outcome and how outcome is different from SIP.


Discussion
In some cases, the AIP and SIP will be almost identical apart from packaging and location, and the repository need only state this. In other cases, complex transformations (e.g., data normalization) may be applied to objects during the ingest process, and a precise description of these actions may be necessary to reflect how the AIP(s) has been adequately transformed from the information in the SIP(s). The AIP construction description should include documentation that gives a detailed description of the ingest process for each SIP to AIP transformation, typically consisting of an overview of general processing being applied to all such transformations, augmented with description of different classes of such processing and, when applicable, with special transformations that were needed.

Some repositories may need to produce these complex descriptions case by case, in which case diaries or logs of actions taken to produce each AIP will be needed. In these cases, documentation needs to be mapped to individual AIPs, and the mapping needs to be available for examination. Other repositories that can run a more production-line approach may have a description for how each class of incoming object is transformed to produce the AIP. It must be clear which definition applies to which AIP. If, to take a simple example, two separate processes each produce a TIFF file, it must be clear which process was applied to produce a particular TIFF file.

B2.3 Repository can demonstrate that all accepted SIPs are either incorporated into one or more AIPs or otherwise disposed of in a recorded fashion.

In particular the following aspect must be checked.

B2.3.1 The repository must follow documented procedures in the event that a SIP is discarded.

Supporting Text
This is necessary in order to ensure that the SIPs received have been dealt with appropriately, and in particular have not been accidentally lost.


Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
System processing files; disposal records; donor or depositor agreements/deeds of gift; provenance tracking system; system log files. Process description documents; documentation of SIP relationship to AIP; clear documentation of how AIPs are derived from SIPs; documentation of standard/process against which normalization occurs; documentation of normalization outcome and how the resulting AIP is different from the SIP(s).


Discussion
The timescale of this process will vary between repositories from seconds to many months, but SIPs must not remain in an unprocessed limbo-like state forever. The accessioning procedures and the internal processing and audit logs should maintain records of all internal transformations of SIPs to demonstrate that they either become AIPs (or part of AIPs) or are disposed of. Appropriate descriptive information should also document the provenance of all digital objects.

B2.4 Repository has and uses a convention that generates persistent, unique identifiers for all AIPs.

In particular the following aspects must be checked.

B2.4.1 The repository must be able to show how any AIP can be uniquely identified within the repository.

B2.4.1.1 The repository must be able to demonstrate that the identifiers are unique.

B2.4.1.2 Documentation must show how the persistent identifiers of the AIP and its components are assigned and maintained so as to be unique within the context of the repository.

B2.4.1.3 Documentation must also describe any processes used for changes to such identifiers.

B2.4.1.4 The repository must be able to provide a complete list of all such identifiers and do spot checks for duplications.

B2.4.1.5 The system of identifiers must be seen to fit the repository%u2019s current and foreseeable future requirements for things like numbers of objects.
Supporting Text
These requirements are necessary in order to ensure that each AIP can be unambiguously found in the future. They also are necessary to ensure that each AIP can be distinguished from all other AIPs in the repository.

B2.4.2 The repository must have a system of reliable linking/resolution services in order to find the uniquely identified object, regardless of its physical location.

Supporting Text
This is so that actions relating to AIPs can be traced over time, over system changes, and over storage changes.


Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Documentation describing naming convention and physical evidence of its application (e.g., logs)


Discussion
A repository needs to ensure that an accepted, standard naming convention is in place that identifies its materials uniquely and persistently for use both in and outside the repository. The %u201Cvisibility%u201D requirement here means %u201Cvisible%u201D to repository managers and auditors. It does not imply that these unique identifiers need to be visible to end users or that they serve as the primary means of access to digital objects. Ideally, the unique ID lives as long as the AIP; if it does not, there must be traceability. Note that B2.1 requires that the components of an AIP be suitably bound and identified for long-term management, but places no restrictions on how AIPs are identified with files. Thus, in the general case, an AIP may be distributed over many files, or a single file may contain more than one AIP. Therefore identifiers and filenames may not necessarily correspond to each other. Documentation must represent these relationships.

B2.5 Repository demonstrates that it has access to necessary tools and resources to provide authoritative Representation Information for all of the digital objects it contains.

In particular the following aspects must be checked.

B2.5.1 The repository must have tools or methods to identify the file type of all submitted Data Objects.

B2.5.2 The repository must have tools or methods to determine what Representation Information is necessary to make each Data Object understandable to the Designated Communit(ies).

B2.5.3 The repository must ensure that it has access to the requisite Representation Information.

B2.5.4 The repository must have tools or methods to ensure that the requisite Representation Information is persistently associated with the relevant Data Objects.

Supporting Text
These are necessary in order to ensure that the repository's digital objects are understandable to the Designated Communit(ies).


Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Subscription or access to such registries; association of unique identifiers to registries of Representation Information (including format registries); viewable records in local registries (with persistent links to digital objects); database records that include Representation Information and a persistent link to relevant digital objects.


Discussion
These tools and resources can be held internally or can be shared via, for example, a trusted set of registries. However this requirement does not demand that each repository has such tools and resources, merely that it has access to them. The Global Digital Format Registry (GDFR), the UK National Archives' file format registry PRONOM, and the UK Digital Curation Centre's Registry Repository of Representation Information (RRORI) are three emerging examples of external registries a repository might adopt. Any such registry is a specialized type of repository, which itself must be certified/trusted. The repository may use these types of standardized, authoritative information sources to identify and/or verify the Representation Information components of Content Information and PDI. This will reduce the long-term maintenance costs to the repository and improve quality control. Sometimes there is both general representation information (e.g. format information) and also specific representation information (e.g., meanings of individual fields within a dataset). Often the general information will be available in an external repository, but the local repository may need to maintain the instance specific information. It is likely that many repositories would wish to keep local copies of relevant Representation Information; however this may not be practical in all cases. Even where a repository strives to keep all such information locally there may be, for example, a schedule of updates which means that until an update is performed, the local Representation Information is incomplete. This may be regarded as a kind of local caching of, for example, the Representation Information held in registries. Alternatively one may say that in these cases, the use of international registries is not meant to replace local registries but instead serve as a resource to verify or obtain independent, authoritative information about any and all Representation Information. Good practice suggests that any locally held Representation Information should also be made available to other repositories via a trusted registry. In addition any item of Representation Information should itself have adequate Representation Information to ensure that the Designated Community can understand and use the data object being preserved.

B2.6 Repository has documented processes for acquiring Preservation Description Information (PDI) for its associated Content Information and acquires PDI in accordance with the documented processes.

In particular the following aspects must be checked.

B2.6.1 The repository has documented processes for acquiring PDI.

B2.6.2 The repository must execute its documented processes for acquiring PDI.

B2.6.3 The repository must ensure that the PDI is persistently associated with the relevant Content Information.

Supporting Text
These requirements are necessary in order to ensure that an auditable trail to support claims of authenticity is available, that unauthorized changes to the digital holdings can be detected, and that the digital objects can be identified and placed in their appropriate context.


Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Standard operating procedures; manuals describing ingest procedures; viewable documentation on how the repository acquires and manages Preservation Description Information (PDI); creation of checksums or digests, consulting with designated community about Context.


Discussion
PDI is needed not only by the repository to help ensure the Content Information is not corrupted (Fixity) and is findable (Reference Information), but to help ensure the Content Information is adequately understandable by providing a historical perspective (Provenance Information) and by providing relationships to other information (Context Information). The extent of such information needs is best addressed by members of the designated community(ies). The PDI must be permanently associated with Content Information.

B2.7 Repository ensures that Content Information of the AIPs is understandable for their Designated Communities at the time of creation of the AIP.

In particular the following aspects must be checked.

B2.7.1 Repository has a documented process for testing understandability for their Designated Communities of the Content Information of the AIPs at their creation.

B2.7.2 The repository must execute the testing process for each class of Content Information of the AIPs.

B2.7.3 The Repository must bring the Content Information up to the required level of understandability when the AIP fails understandability testing.

Supporting Text
These requirements are necessary in order to ensure that one of the primary tests of preservation, namely that the digital holdings are understandable by their Designated Communit(ies), can be met. See B3 for additional requirements for understandability beyond ingest.


Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Test procedures to be run against the digital holdings to ensure their understandability to the defined Designated Communities and their knowledge bases; records of such tests being performed and evaluated; evidence of gathering or identifying Representation Information to fill any intelligibility gaps which have been found; Retention of individuals with the discipline expertise.


Discussion
These requirements are concerned with the understandability of the AIP. If the ingested material is not understandable, the repository needs to ingest or make available additional information to make sure that the AIPs are understandable to the designated comminut(ies). For example, if documents are written in a dying language and the Designated Communit(ies) are no longer able to understand the language the documents are written in, the repository would need to provide additional documentation that would allow the Designated Communit(ies) to understand the documents (e.g., translations of the documents in a language the Designated Communit(ies) could understand or dictionaries that would allow the Designated Communit(ies) to translate the documents into a language its members understand.)

B2.8 Repository verifies each AIP for completeness and correctness at the point it is created.

Supporting Text
The repository must be sure that the AIPs it creates are as they are expected to be by checking them against the associated definition for each AIP or class of AIP (see B2.1) and the description of how AIPs are constructed from SIPs (see B.2.3.). This is necessary in order to ensure that what is maintained over the long term is as it should be and can be traced to the information provided by the Producers.


Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Description of the procedure that verifies completeness and correctness of the AIPs; logs of the procedure.


Discussion
If the repository has a standard process to verify SIPs for both completeness and correctness and a demonstrably correct process for transforming SIPs into AIPs (see B2.2), then it simply needs to demonstrate that the initial checks were carried out successfully and that the transformation process was carried out without indicating errors. On the other hand repositories that must create unique processes for many of their AIPs will also need to generate unique methods for validating the completeness and correctness of AIPs. This may include performing tests of some sort on the content of the AIP that can be compared with tests on the SIP. Such tests might be simple (counting the number of records in a file, or performing some simple statistical measure), but they might be complex. Documentation should describe how the completeness and correctness of AIPs is ensured, starting with receipt from the producer and continuing through AIP creation and supporting long-term preservation. Example approaches include the use of checksums, testing that checksums are still correct at various points during ingest and preservation, logs that such checks have been made, and any special tests that may be required for a particular AIP instance or class.

B2.9 Repository provides a mechanism for verifying the integrity of the repository collection/content.


Supporting Text
The repository must provide a way to independently demonstrate the completeness and correctness of its collections and their contents. This is necessary to enable the audit of the integrity of the collection as a whole. It is the responsibility of the repository to choose the appropriate form of such a mechanism.


Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Documentation provided for B2.1 through B2.4; documented agreements negotiated between the producer and the repository (see B 1.1-B1.9); logs of material received and associated action (receipt, action, etc.) dates; logs of periodic checks.


Discussion
In general, it is likely that a repository that meets all the previous criteria will satisfy this one without needing to demonstrate anything more. As a separate requirement, it demonstrates the importance of being able to audit the integrity of the collection as a whole. For example, if a repository claims to have all e-mail sent or received by The Yoyodyne Corporation between 1985 and 2005, it has been required to show that:

  • The content it holds came from Yoyodyne%u2019s e-mail servers.
  • It is all correctly transformed into a preservation format.
  • Each monthly SIP of e-mail has been correctly preserved, including original unique identifiers such as Message-IDs.
However it may still have no way of showing whether this really represents all of Yoyodyne's email. For example, if there is a three-day period with no messages in the repository, is this because Yoyodyne was shut down for those three days, or was the e-mail lost before the SIP was constructed? This case could be resolved by the repository amending its description of the collection, but other cases may not be so straightforward. A familiar mechanism from the world of traditional materials in libraries and archives is an accessions or acquisitions register that is independent of other catalog metadata. A repository should be able to show, for each item in its accessions register, which AIP(s) contain content from that item. Alternatively, it may need to show that there is no AIP for an item, either because ingest is still in progress, or because the item was rejected for some reason. Conversely, any AIP should be able to be related to an entry in the acquisitions register.

B2.10 Repository has contemporaneous records of actions and administration processes that are relevant to AIP creation.


Supporting Text
The repository must create records of its preservation related activities essentially as they happen. This is necessary in order to be sure that nothing relevant is omitted from the record that might be needed to provide an independent means to verify that all AIPs have been properly created in accord with the documented procedures (see B.2.1. through B.2.9). It is the responsibility of the repository to justify its practice in this respect.


Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Written documentation of decisions and/or action taken; preservation metadata logged, stored, and linked to pertinent digital objects.


Discussion
These records must be created on or about the time of the actions they refer to and are related to actions associated with AIP creation. The records may be automated or may be written by individuals, depending on the nature of the actions described. Where community or international standards are used, the repository must demonstrate that all relevant actions are carried through.

B3 Preservation planning (BS)

In this chapter several similar expressions are used, which does not make it clear to the reader. I have: preservation strategies, preservation policies, preservation plans, preservation implementation plan, preservation planning, preservation strategic plan, preservation issue. I would suggest the following:

  • preservation policies are the guiding principles on a high level
  • a preservation issue is a risk that was identified and need some action, I would suggest we call it a “preservation risk”
  • a preservation (implementation) plan describes the decision to mitigate the risk, which preservation actions will be performed to mitigate the risk, in relation to the context of the objects in danger
  • a preservation strategy is a suggested/ performed (range of ) actions that mitigates the risk

B3.1 Repository has documented preservation strategies relevant to its holdings.


Supporting Text
The repository must have current, sound, and documented preservation strategies which describes how the repository will act upon identified risks.. This isThese documented preservation strategies are necessary in order to make the information available and usable for future generations and to provide a means to check and validate the preservation work of the repository.
Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Documentation identifying each preservation issuerisk identified and the strategy for dealing with that issuerisk.
Discussion
These preservation strategies will typically address the degradation of storage media, the obsolescence of media drives, and the obsolescence of Representation Information (including formats), safeguards against accidental or intentional digital corruption. For example, if migration is the chosen approach to some of these issues, there also needs to be Preservation Implementation Plans on; what triggers a migration and what types of migration are expected forto solve the solution of each preservation issuerisk identified.

B3.2 Repository has mechanisms in place for monitoring its preservation environment.

Repository has mechanisms in place for monitoring and notification when Representation Information (including formats) approaches obsolescence or is no longer viable.
Supporting Text
The repository must show that it has some active mechanism to ensure that the preserved information remains understandable and usable by the designated community(ies). The repository must show that it has some active mechanism to warn of impending obsolescence. Obsolescence is determined largely in terms of the knowledge base of the designated community(ies). This is necessary in order to ensure that the preserved information remains understandable and usable by the designated community(ies).
Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
sSubscription to a format registry service service which registers information about file format obsolescence and/or notifies if this occurs; subscription to a technology watch service; surveys in the Designed Community of the repository.
Discussion
For most repositories, the concern will be with the Representation Information used to preserve information, which may include information on how to deal with a file format or software that can be used to render or process it. Sometimes the format needs to change because the repository can no longer deal with it. Sometimes the format is retained and the information about what software is needed to process it needs to change. If the mechanism depends on an external registry, the repository must demonstrate how it uses the information from that registry. For most repositories, the concern will be with the Representation Information used to preserve information, which may include information on how to deal with a file format or software that can be used to render or process it. Sometimes the format needs to change because the repository can no longer deal with it. Sometimes the format is retained and the information about what software is needed to process it needs to change. If the mechanism depends on an external registry, the repository must demonstrate how it uses the information from that registry.

B3.2.1 Repository has mechanisms in place for monitoring and notification when Representation Information is inadequate for the Designated Community(ies) to understand the data holdings. Repository has mechanisms in place for monitoring and notification when Representation Information (including formats) approaches obsolescence or is no longer viable.


Supporting Text
The repository must show that it has some active mechanism to ensure that the preserved information remains understandable and usable by the designated community(ies). The repository must show that it has some active mechanism to warn of impending obsolescence. Obsolescence is determined largely in terms of the knowledge base of the designated community(ies). This is necessary in order to ensure that the preserved information remains understandable and usable by the designated community(ies).
Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
subscription to a format registry service; subscription to a technology watch service, surveys amongst its designated community members, relevant working processes to deal with this information.
Discussion
For most repositories, the concern will be with the Representation Information used to preserve information, which may include information on how to deal with a file format or software that can be used to render or process it. Sometimes the format needs to change because the repository can no longer deal with it. Sometimes the format is retained and the information about what software is needed to process it needs to change. If the mechanism depends on an external registry, the repository must demonstrate how it uses the information from that registry.see Discussion for 3.2

B3.3 Repository has mechanisms to change its preservation plans as a result of its monitoring activities.


Supporting Text
The repository must demonstrate or describe how it reacts to information from monitoring, which sometimes requires a repository to change how it deals with the material it holds in ways that could not have been anticipated at an earlier stage. This is necessary in order for the repository to be prepared for changes in the external environment that may make its current preservation plans a bad choice as the time to implement draws near.
Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Preservation Strategic Plans tied to formal or informal technology watch(es); preservation planning or processes that are timed to shorter intervals (e.g., not more than five years); proof of frequent Preservation Policies and Preservation Strategic Plans updates; sections of Preservation Policies that address how plans may be updated and that address how often the plans are required to be reviewed and reaffirmed or updated.
Discussion
Plans as simple as migrating from format X to format Y when the registries show that format X is no longer supported are not sufficiently flexible; other events may have made format Y a bad choice. The repository must periodically review it's preservation plans and the technology environment and, if necessary, makes changes to those plans to ensure their continued effectiveness. Another possible response to information gathered by monitoring is for the repository to update and create additional Representation Information and/or PDI.

B3.3.1 (was B3.2) Repository has mechanisms for creating, identifying or gathering any extra Representation Information required. Repository has mechanisms in place for monitoring and notification when Representation Information (including formats) approaches obsolescence or is no longer viable.


Supporting Text
The repository must show that it has some active mechanism to ensure that the preserved information remains understandable and usable by the designated community(ies). The repository must show that it has some active mechanism to warn of impending obsolescence. Obsolescence is determined largely in terms of the knowledge base of the designated community(ies). Repository has mechanisms in place for monitoring and notification when Representation Information (including formats) approaches obsolescence or is no longer viable and must show that it has mechanisms to incorporate these findings in its workflows. This is necessary in order to ensure that the preserved information remains understandable and usable by the designated community(ies).
Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
sSubscription to a format registry service; subscription to a technology watch service; preservation plans.
Discussion
For most repositories, the concern will be with the Representation Information used to preserve information, which may include information on how to deal with a file format or software that can be used to render or process it. Sometimes the format needs to change because the repository can no longer deal with it. Sometimes the format is retained and the information about what software is needed to process it needs to change. If the mechanism depends on an external registry, the repository must demonstrate how it uses the information from that registry. NOTE THIS IS THE SAME TEXT AS 3.2, DOES THAT MAKE SENSE?

B3.4 Repository can provide evidence of the effectiveness of its preservation activities.


Supporting Text
The repository must be able to demonstrate the continued preservation, including understandability, of its holdings. This is necessary in order to assure the Designated Community(ies) that the repository will be able to make the information available and usable over the mid-to-long-term.
Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Collection of appropriate preservation metadata; proof of usability of randomly selected digital objects held within the system; demonstrable track record for retaining usable digital objects over time; Designated Community polls.
Discussion
This could be evaluated at a number of degrees and depends on the specificity of the Designated Community(ies). If a Designated Community is fairly broad, an auditor could represent the test subject in the evaluation. More specific Designated Communities could require significant efforts.
-- MarkConrad - 25 Jun 2008 We still need to go over David's cross-walk of Appendix 5 of the TRAC document and the requirements in B.3. to ensure that we have captured all of the requirements that should be listed in this section.
-- MarkConrad - 26 Jun 2008 In B.2.10. we inserted a reference to a B.3.x as a needed requirement to continue to test the understandability of AIPs beyond ingest. Does B.3.2. adequately meet this need or do we need another requirement in B.3?
-- JohnGarrett - 07 Jul 2008 - Following text was originally from B4.2, but today's meeting determined that it should be included in B3.
-- JohnGarrett - 07 Jul 2008 - I don't see a need to require strategies that are not yet employed. But I do think there is a need to be able to include those new strategies. Could the sentence above be changed to something like "The repository must demonstrate that it has documented preservation strategies and the strategy document must include a means to update and add to the strategies employed." This is necessary in order for the repository to be prepared for changes in the external environment that may make its current preservation plans a bad choice as the time to implement draws near. Minimally, documentation of Preservation Implementation Plans must be included in repository policies and practices. This is necessary in order to assure funders, producers, and users - and allow them to verify this
-- MarkConrad - 25 Jun 2008 It is not clear what the distinction is between preservation policies, preservation plans and preservation strategies. If there is one we need to make it explicit. [End of text moved from B4.1]


B4 AIP preservation (JGG)

B4.1 Repository must have specifications of how the AIPs are stored down to the bit level.

Supporting Text

The repository must specify the format (with representation information down to the bit level) of each AIP component and must specify how the separate components are packaged together.

This is necessary in order to ensure that the information can be extracted from the AIP over the long-term.

Examples of Ways the Repository can Demonstrate it is Meeting this Requirement Documentation of the format of AIPs;

EAST and DEDSL descriptions of the data components; <br. BA 16April2009 - There are spacing and bold issues here. I also think EAST and DEDSL are specific to the space discipline and not of general understanding. Other examples should be listed such as: "the repository's internal of designated community's standard AIP storage standards; system processing logs; validation and verification records."

DG 20090418: EAST and DEDSL can be applied to many types of data. However additional examples would be useful

Discussion

A description of the format (Representation Information in OAIS terms) must be available for each AIP and must be appropriately linked to the AIP. Often, repositories are tempted to describe AIP content only down to a level where a program will then be used to convert the information to a form understandable to their Designated Communities. However, if those programs ever fail to operate, then the information would be lost in all the AIPs that relied on that program.

B4.1.1 Repository must preserve the Content Information of AIPs.

Supporting Text
The repository must be able to demonstrate that the AIPs faithfully reflect the information was captured during ingest and that any subsequent or future planned transformations will continue to preserve all the Significant Properties of the Content Information. This is necessary because it is the fundamental mission of a repository to preserve the Content Information for its Designated Communities.


Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Preservation workflow procedure documentation;

workflow procedure documentation;

Preservation Policy documents specifying treatment of AIPs and under what circumstances they may ever be deleted;

Ability to demonstrate the sequence of conversions for an AIP for any particular digital object or group of objects ingested;

Documentation linking ingested objects and the current AIPs.


Discussion
One approach to this requirement assumes that the repository has a policy specifying that AIPs cannot be deleted at any time. This particularly simple and robust implementation preserves links between what was originally ingested, as well as new versions that have been transformed or changed in any way. Depending upon implementation, these newer objects may be completely new AIPs or merely updated AIPs. Either way, persistent links between the ingested object and the AIP should be maintained.

B4.1.2 Repository must actively monitor the integrity of AIPs.

Supporting Text
In OAIS terminology, this means that the repository must have Fixity Information for AIPs and must make some use of it. This is necessary in order to protect the integrity of the archival objects over time.

A repository should have logs that show actions taken to check the integrity of archival objects. This is necessary in order to assure funders, producers, and users - and to allow them to audit/validate - that the repository is taking the necessary steps to ensure the long-term integrity of the digital objects.
[JGG Note: I think this should be in the introduction to this document]

This is necessary to ensure that AIPs have not been changed. documentation that integrity checks are carried out on a regular basis and to allow interested parties to verify that this is the case.
DG 20090418 - the "documentation that integrity checks are carried out " should probably be "by being able to produce proof that that integrity checks are carried out"

AIP integrity also needs to be monitored at a higher level, ensuring that all AIPs that should exist actually do exist, and that the repository does not possess AIPs it is not meant to. This is necessary to ensure that the repository has not lost any AIPs and that any AIPs that are in the repository meet the criteria and standards expected of the repository.

A repository should have logs that show actions taken to check the integrity of the repository as a whole. This is necessary to document that integrity checks are carried out on a regular basis and to allow interested parties to verify that this is the case.

This is necessary in order to validate the integrity of the repository as a whole.


Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Fixity information (e.g., checksums) for each ingested digital object/AIP;

Logs of fixity checks;

Documentation of how AIPs and Fixity information are kept separate;

Documentation of how AIPs and accession registers are kept separate.

.
Discussion
At present, most repositories deal with this at the level of individual information objects by using a checksum of some form, such as MD5. In this case, the repository must be ablemay want to demonstrate that the Fixity Information (checksums, and the information that ties them to AIPs) are stored separately or protected separately from the AIPs themselves, so that accidental alteration of the AIP would not also damage the Fixity Information. Also someone who can maliciously alter an AIP would not likely be able toas easily alter the Fixity Information as well.

B4.2 Repository must have contemporaneous records of actions and administration processes that are relevant to storage and preservation of the AIPs.

Supporting Text
The repository must create records of actions associated with archival storage and must create those records on or about the time of the actions to which they refer. This is necessary in order to ensure documentation is not omitted or erroneous or of questionable authenticity.


Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Written documentation of decisions and/or action taken;

Preservation metadata logged, stored, and linked to pertinent digital objects.


Discussion
The records may be automated or may be written by individuals, depending on the nature of the actions described. Where community or international standards are used, the repository must demonstrate that all relevant actions are appropriately performed.

B4.2.1 Repository must have specifications for all actions taken on AIPs.

Supporting Text

The repository must have written documentation describing all actions that can be performed against an AIP. This is necessary in order to ensure that any actions performed against an AIP do not alter the AIP information in a manner unacceptable to its Designated Communities.

Examples of Ways the Repository can Demonstrate it is Meeting this Requirement

Written documentation describing all actions that can be performed against an AIP.


Discussion

This documentation is normally created during design of the repository. It should detail the normal handling of AIPs, all actions that can be performed against the AIP, including success and failure conditions and details of how these processes can be monitored.

B4.2.2 Repository demonstrates that any actions taken on AIPs were compliant with the specification of those actions

Supporting Text

The repository must have documentation that all actions taken against an AIP are the actions allowed and documented in the design of the archive. This is necessary in order to ensure that any actions performed against an AIP do not alter the AIP information in a manner unacceptable to its Designated Communities.

Examples of Ways the Repository can Demonstrate it is Meeting this Requirement

Preservation metadata logged, stored, and linked to pertinent digital objects and documentation of that action;

Procedural audits of repository showing that all actions conform to the documented processes.


Discussion

Successful preservation of information in the archive is strongly linked to following established and documented procedures to complete any actions that affect the repository data. The more often “special handling” of repository data and the more often this “special handling” is not overseen in a consistent manner, the more likely that the repository held data will be compromised. When procedures are regularly followed, any deviation from procedures that would be likely to cause an alteration in the data will more likely be noticed or if not noticed may be more likely able to be corrected or the timing and likely change could be identified in the future.

B5 Information management (RD)

B5.1 Repository specifies minimum Descriptive Information requirements to enable the designated community(ies) to discover and identify material of interest.


Supporting Text
The repository must be able to deal with the types of requests that will come from a typical user from the designated community(ies). This is necessary in order to enable discovery of the repository's holdings.

Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Retrieval and Descriptive Information, discovery metadata, such as Dublin Core, and other documentation describing the object.

Discussion
A repository does not necessarily have to satisfy every possible request. Retrieval metadata is distinct from Descriptive Information that describes what has been found. For example, in a library we might say that a book's title is mandatory, but the name of the publisher is not mandatory, because people generally search on the title.

B5.2 Repository captures or creates minimum Descriptive Information and ensures that it is associated with the AIP.


Supporting Text
The repository must show that it associates with each AIP, minimum Descriptive Information that was either received from the producer or created by the repository. This is required in order to make it clear to producers and users that Descriptive Information is associated with the AIP.

Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Descriptive metadata; internal or external persistent, unique identifier or locator that is associated with the AIP (see also B2.4 about persistent, unique identifier); system documentation and technical architecture; depositor agreements; metadata policy documentation, incorporating details of metadata requirements and a statement describing where responsibility for its procurement falls; process workflow documentation.

Discussion
Associating the metadata with the object is important, although it does not require one to one correspondence, and metadata may not necessarily be stored with the AIP. Hierarchical schemes of description can allow some descriptive elements to be associated with many items. The association should be unbreakable - it must not be lost even if other associations are created.

B5.3 Repository can demonstrate that all AIPs are associated with their descriptive information and there is descriptive information for each AIP.


Supporting Text
The repository must ensure that every AIP has some descriptive information and that all descriptive information must point to at least one AIP. This is necessary to validate the integrity of an AIP and to ensure that all AIPs can be located and retrieved.

Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Descriptive metadata; unique, persistent identifier or locator associated with the AIP; documented relationship between the AIP and its metadata; system documentation and technical architecture; process workflow documentation.

Discussion
Repositories must implement procedures to capture or create descriptive information for each AIP.

B5.3.1 Repository can demonstrate that it maintains the associations between its AIPs and their descriptive information.


Supporting Text
The repository must ensure that the linkages between AIPs and their descriptive information are maintained over time. This is necessary to ensure that all AIPs can continue to be located and retrieved.

Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Log detailing ongoing maintenance or checking of the integrity of the data and its relationships to the associated descriptive information, especially following repair or modification of the AIP; legacy descriptive information; persistence of identifier or locator; documented relationship between AIP and its descriptive information; system documentation and technical architecture; process workflow documentation.

Discussion
Repositories must implement procedures that let them know when the relationship between the data and the associated descriptive information is temporarily broken to ensure that it can be restored.

B6 Access management (SL)

The term “access” has a number of different senses, including access by users to the repository system—for example, physical security and user authentication), and the different stages of accessing records (making a request, verifing the rights of the requester, and preparing and sending a Dissemination Information Package (DIP). This section is concerned with all of these. It is divided into two main requirements: one concerned with the existence and implementation of access policies, and one with the capacity of the repository to provide demonstrably authentic objects as DIPs. Thus the first requirement relates to requests initiated by a user and how the repository handles them to ensure that rights and agreements are respected, that security is monitored, that requests are fulfilled, etc. The second requirement relates to what is delivered to the accessor of the repository and the trust that can be placed in it.

It must be understood that the capabilities and sophistication of the access system will vary depending on the repository’s designated community(ies) and the access mandates of the repository. Because of the variety of repositories, archives, and access mandates, these criteria may be subject to questions about applicability and interpretation at a local level. For in-depth discussion of access management issues, see Appendix 6: Understanding Digital Repositories & Access Functionality.

Repositories with a mandate to provide current access must be able to produce Dissemination Information Packages that meet the needs of their users or are appropriate to the levels of access they offer. “Dark” archives or national archives that may have mandates restricting access for a certain number of years will produce most DIPs for internal requirements, such as performing migrations, rather than for access. In any case, any repository must be able to produce a DIP, however primitive and whatever its purpose.

B6.1 Repository can demonstrate compliance with Access Policies.

Supporting Text

The repository must demonstrate that a complete access management system, with all access policies, is implemented. This is necessary in order to ensure the repository has fully addressed all aspects of usage which might effect the trustworthiness of the repository, particularly with reference to support of the user community.

Examples of Ways the Repository can Demonstrate it is Meeting this Requirement

Statements of policies that are available to the user communities; information about user capabilities (authentication matrices); logs and audit trails of access requests;explicit tests of some types of access.

Discussion

Depending on the nature of the repository, the Access Policies may cover:

  • Statements of what is accessible to which community, and on what conditions
  • Requirements for authentication and authorisation of accessors
  • Enforcement of agreements applivable to access conditions
  • Recording access actions

Access may be managed partly by computers and partly by humans—checking passports, for instance, before issuing a user ID and password may be an appropriate part of access management for some institutions.

B6.1.1 Repository logs all access management failures, and staff review inappropriate incidents.

Supporting Text

The repository must demonstrate that there is regular review of anomalous or unusual usage and access management failures. This is necessary in order to identify security threats and access management system failures.

Examples of Ways the Repository can Demonstrate it is Meeting this Requirement

Access logs, capability of the system to use automated analysis/monitoring tools and generate problem/error messages; notes of reviews undertaken or action taken as a result of reviews.

Discussion

A repository should have some automated mechanism to note anomalous or unusual denials and use them to identify either security threats or failures in the access management system, such as valid users being denied access. This does not mean looking at every denied access.

This requirement does not apply to repositories with unrestricted access.

B6.2 Repository follows policies that enable the dissemination of copies of that are traceable to the originals, with evidence supporting their authenticity.

The repository must demonstrate that it can trace in some auditable way the DIP it disseminates back to the original object(s). The trace must provide evidence of the authenticity of the DIP. This is necessary to ensure the DIP that is disseminated can be regarded as an authentic copy of the original object(s) (AIPs).

Examples of Ways the Repository can Demonstrate it is Meeting this Requirement

System design documents; work instructions (if DIPs involve manual processing); process walkthroughs; production of a sample copy with evidence of authenticity; documentation of community requirements for evidence of authenticity.

Discussion

Authenticity is not an “all or nothing” concept, but is a matter of degree, judged on the basis of evidence. Thus the adequacy of the evidence is of key importance in assessing this requirement.

It must also be remembered that objects are not always disseminated in the same way, or in the same groupings, as they are deposited. For example, a database, which originally had subsets of its rows, columns, and tables, may be disseminated without the original formatting, so that the phrase “authentic copy” has little meaning. Ingest and preservation actions may change the formats of files, or may group and split the original objects deposited. The degree of authenticity might also arise when transformation processes are applied. For instance, a repository that stores digital audio from radio broadcasts may disseminate derived text that is constructed by automated voice recognition from the digital audio stream. Derived text may be imperfect but useful to many users, but clearly has a lesser degree of authenticity than either delivering the original audio stream or getting a human to verify and correct the transcript against the stored audio.

This requirement ensures that ingest, preservation, and transformation actions do not lose information that would support an auditable trail of authenticity between the original deposited object and the eventual disseminated object.

For compliance, the chain of authenticity need only reach as far back as ingest, though some communities, such as those dealing with legal records, may require chains of authenticity that reach back further.

A repository should be able to demonstrate the processes to construct the DIP from the relevant AIP(s). This is a key part of establishing that DIPs reflect the content of AIPs, and hence of original material, in a trustworthy and consistent fashion. DIPs may simply be a copy of AIPs, or may result from a simple format transformation of an AIP. But in other cases, they may be derived in complex ways from a large set of AIPs. A user may request a DIP consisting of the title pages from all e-books published in a given period, for instance, which will require these to be extracted from many different AIPs. A repository that allows requests for such complex DIPs will need to put more effort into demonstrating how it meets this requirement than a repository that only allows requests for DIPs that correspond to an entire AIP.

A repository is not required to show that every DIP it provides can be verified as authentic at a later date; it must show that it can do this when it is required at the time of production of the DIP. The level of authenticity required is to be determined by the designated community(ies). This requirement is meant to enable demosntration of high levels of authenticity, not to impose it on all copies, since it may be an expensive process.

B6.2.1 Repository can demonstrate that any problem reports about errors in data or responses from users are recorded and acted on.

Supporting Text

The objective of access management is to ensure that user receives a usable and correct version of the digital object(s) (i.e., DIP) that they requested. A repository must show that any problems that do occur and are brought to its attention are investigated and acted on. This is necessary in order for the repository to be considered trustworthy.

Examples of Ways the Repository can Demonstrate it is Meeting this Requirement

System design documents; work instructions (if DIPs involve manual processing); process walkthroughs; logs of orders and DIP production; documentation of error reports and the actions taken.

Discussion

This requirement is intended to cover the correctness of any transformations that may be applied to generate the DIP. A simple example is that if the repository stores TIFF images but delivers JPEGS, the users should receive that format, be able to read it and have it correspond to what they expect. In general it is difficult, if not impossible to prove that this is the case, so the ability to respond to error reports is essential.

Section C: Infrastructure and Security Risk Management

To fill in the template for a requirement, simply click on the "Edit" tab at the top of the page and type text to replace the "..." where they appear under each requirement. When you have finished, click on "Save" at the bottom of the edit page.

C1 Technical infrastructure risk management (RM)

C1.1 Repository identifies and manages the risks to its preservation operations and goals associated with system infrastructure.

Supporting Text
The repository must conduct or contract assessments of the risks related to hardware and software infrastructure, and operational procedures. The repository must provide mechanisms that minimize risk from dependencies on proprietary or obsolete system infrastructure. The repository must provide mechanisms to minimize risk from operational error. This is necessary to ensure a secure and trustworthy infrastructure.

Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Infrastructure inventory of system components; periodic technology assessments; estimates of system component lifetime; export of authentic records to an independent system; use of strongly community supported software .e.g., Apache, iRODS, Fedora); re-creation of archives from backups.

Discussion
The degree of support required relates to the criticality of the subsystem(s) involved in long- term preservation. The repository should maintain a system that is scalable (e.g. able to handle anticipated future volumes of both bytes and files) without a major disruption of the system. The repository should maintain a system that is evolvable. That is, the system should be designed in such a way that major components of the system can be replaced with newer technologies without major disruption of the system as a whole. The repository system should be extensible. That is, the system should be designed to accommodate future formats (media and files) without major disruption of the system as a whole. The repository should be able to export its holdings to a future custodian. The repository should be able to re-create the archives after an operational error that overwrites or deletes digital holdings.

C1.1.1 Repository performs technology watch.

Supporting Text
The repository must employ technology watches or other technology monitoring notification systems to track when hardware or software components will become obsolete. This is necessary to track when migration is needed to new infrastructure.

Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Management of periodic technology assessment reports. Comparison of existing technology to each new assessment.

Discussion
The objective is to understand when any sub-system poses a risk of obsolescence, and enable planning migration to new technology before interoperability mechanisms are no longer available. This can be driven by proprietary software dependencies (the vendor no longer supports the sub-system component), and by emergence of new protocols (the mechanism for accessing the system has become obsolete and is no longer supported).

C1.1.1.1 Repository has hardware technologies appropriate to the services it provides to its designated communities.

Supporting Text
The repository must be aware of the types of storage, file management, preservation and access services expected by its designated community(ies), including where applicable, the types of media to be delivered, and needs to make sure its hardware capabilities can support these services. This is necessary to provide expected, contracted, secure, and persistent levels of service including: ease of ingest and dissemination through appropriate depositor and user interfaces and technologies such as upload mechanisms; on-going digital object management; preservation approaches and solutions, such as migration; and system security.

Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Maintenance of up-to-date designated community technology, expectations, and use profiles; Provision of bandwidth adequate to support ingest and use demands; systematic elicitation of feedback regarding hardware and service adequacy; maintenance of a current hardware inventory.

Discussion
The objective is to track when changes in service requirements by the designated communities require a corresponding change in the hardware technology, when changes in ingestion policies require expanded capabilities, and when changes in preservation policies require new preservation capabilities. This can be driven by changes in capacity requirements (the time needed to read all media is longer than the media lifetime), by changes in delivery mechanisms (new clients for displaying authentic records), and changes in the number and size of archived records.

C1.1.1.2 Repository has procedures in place to monitor and receive notifications when hardware technology changes are needed.

Supporting Text
The repository must conduct or contract frequent environmental scans regarding hardware status, sources of failure, and inter-operability among hardware components. The repository must also be in contact with its hardware vendors regarding technology updates, points of likely failure, and how new components may affect system integration and performance. This is necessary to ensure expected, contracted, secure, and persistent levels of service.

Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Audits of capacity versus actual usage; audits of observed error rates; audits of performance bottlenecks that limit ability to meet user community access requirements; documentation of technology watch assessments; documentation of technology updates from vendors.

Discussion
The objective is to track when changes in service requirements by the designated communities require a corresponding change in the hardware technology, when changes in ingestion policies require expanded capabilities, and when changes in preservation policies require new preservation capabilities. This can be driven by changes in capacity requirements (the time needed to read all media is longer than the media lifetime), by changes in delivery mechanisms (new clients for displaying authentic records), and changes in the number and size of archived records.

C1.1.1.3 Repository has procedures in place to evaluate when changes are needed to current hardware.

Supporting Text
Given information from technology watches or other technology monitoring notification systems, the repository must have procedures and expertise to evaluate this data and make sound decisions regarding the need for new hardware. This is necessary to ensure that the repository has the capacity to make informed and timely decisions when information indicates the need for new hardware.

Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Evaluation procedures in place; documented staff expertise in each technology sub-system.

Discussion
The objective is to track when technology providers have developed sub-systems that minimize risk, or that minimize cost, or that improve performance. This is necessary to track emerging technologies, and plan for upgrades before capacity limits occur. The evaluation should identify when the risk of using new technology outweighs the expected benefit, and when the new technology is sufficiently mature to minimize risk.

C1.1.1.4 Repository has procedures to replace hardware when evaluation indicates the need to do so.

Supporting Text
The repository must have a commitment and a financial escrow plan or certain funding streams by which it can demonstrate the capacity to purchase new hardware when evaluation indicates the need to do so. This is necessary to ensure hardware replacement in a timely fashion so as to avert system failure or performance inadequacy. Without such a commitment, and more importantly, without escrowed financial resources or a secure funding stream, technology watches and notifications are of little value. The repository must have mechanisms for evaluating the efficacy of the new systems before implementation in the production system.

Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Statement of commitment to provide expected and contracted levels of service; evidence of ongoing financial assets set aside for hardware procurement; demonstration of cost savings through amortized cost of new system.

Discussion
The objective is to demonstrate that the repository has the ability to incorporate new technology, both financially through funding commitments or cost reduction, and operationally through verification of the capabilities of the new systems.

C1.1.1.5 Repository has software technologies appropriate to the services it provides to its designated communities.

Supporting Text
The repository must be aware of the types of storage, file management, preservation and access services expected by its designated community(ies), including where applicable, the types of media to be delivered, and needs to make sure its software capabilities can support these services and be compatible with repository hardware. This is necessary to provide expected, contracted, secure, and persistent levels of service including: ease of ingest and dissemination through appropriate depositor and user interfaces and technologies such as upload mechanisms; on-going digital object management; preservation approaches and solutions, such as migration; and system security.

Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Maintenance of up-to-date designated community technology, expectations, and use profiles; Provision of software systems adequate to support ingest and use demands; systematic elicitation of feedback regarding software and service adequacy; maintenance of a current software inventory.

Discussion
The objective is to track when changes in service requirements by the designated communities require a corresponding change in the software components, when changes in ingestion policies require support for new data formats and when changes in software technology require new format migration capabilities. This can be driven by changes in access requirements (new clients become preferred that require new data formats), by changes in delivery mechanisms (new data transfer mechanisms), and changes in the number and size of archived records that require more scalable software.

C1.1.1.6 Repository has procedures in place to monitor and receive notifications when software changes are needed.

Supporting Text
The repository must conduct or contract frequent environmental scans regarding software evolution, likely points of failure, and inter-operability among the software and hardware components. The repository must also be in contact with its software vendors regarding technology updates, points of likely failure, and how new programs may affect system integration and performance. This is necessary to ensure expected, contracted, secure, and persistent levels of service.

Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Audits of capacity versus actual usage; audits of observed error rates; audits of performance bottlenecks that limit ability to meet user community access requirements; documentation of technology watch assessments; documentation of software updates from vendors.

Discussion
The objective is to track when changes in service requirements by the designated communities require a corresponding change in the hardware technology, when changes in ingestion policies require expanded capabilities, and when changes in preservation policies require new preservation capabilities. This can be driven by security updates (vendor supplied corrections to newly identified vulnerablities), by changes in delivery mechanisms (new software clients for displaying authentic records), and changes in the number and size of archived records (expanded database requirements).

C1.1.1.7 Repository has procedures in place to evaluate when changes are needed to current software.


Supporting Text
Given information from technology watches or other technology monitoring notification systems, the repository must have procedures and expertise to evaluate this data and make sound decisions regarding the need for new software. This is necessary to ensure that the repository has the capacity to make informed and timely decisions when information indicates the need for new software.

Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Evaluation procedures in place; documented staff expertise in each software technology sub-system.

Discussion
The objective is to track when technology providers have developed software infrastructure that minimizes risk, or that minimizes cost, or that improves performance. This is necessary to track emerging technologies, and plan for upgrades before capacity limits occur. The evaluation should identify when the risk of using new technology outweighs the expected benefit, and when the new technology is sufficiently mature to minimize risk.

C1.1.1.8 Repository has procedures to replace software when evaluation indicates the need to do so.

Supporting Text
The repository must have a commitment and a financial escrow plan or certain funding streams by which it can demonstrate the capacity to purchase new software when evaluation indicates the need to do so. This is necessary to ensure software replacement in a timely fashion so as to avert system failure or performance inadequacy. Without such a commitment, and more importantly, without escrowed financial resources or a secure funding stream, technology watches and notifications are of little value. The repository must have mechanisms for evaluating the efficacy of the new systems before implementation in the production system.

Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Statement of commitment to provide expected and contracted levels of service; evidence of ongoing financial assets set aside for software procurement; demonstration of cost savings through amortized cost of new system.

Discussion
The objective is to demonstrate that the repository has the ability to incorporate new technology, both financially through funding commitments or cost reduction, and operationally through verification of the capabilities of the new systems.

C1.1.2 Repository ensures that it has adequate hardware and software support for backup functionality sufficient for preserving the repository content and tracking repository functions.

Supporting Text
The repository must be able to demonstrate the adequacy of the processes, hardware and software for its backup systems. This is necessary in order to ensure continued access to and tracking of preservation functions applied to the digital objects in their custody.

Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Documentation of what is being backed up and how often; audit log/inventory of backups; validation of completed backups; disaster recovery plan, policy and documentation; fire drills;
BA 16April2009 - Do you literally mean "fire drills" or is this generic for a disaster planning practice exercise?
testing of backups; support contracts for hardware and software for backup mechanisms; demonstrated preservation of system metadata such as access controls, location of replicas, audit trails, checksum values.

Discussion
Repositories must be able to demonstrate the full range of ingest, preservation, and dissemination functions required of a repository entrusted with long-term preservation. Simple backup mechanisms must preserve not only the repository main content, but also the system metadata generated by the preservation functions. Repositories need to develop backup plans that ensure their continuity of operations across all failure modes.

C1.1.3 Repository has effective mechanisms to detect bit corruption or loss.


Supporting Text
The repository must be able to demonstrate that all the AIP's and accompanying metadata are uncorrupted. This is necessary in order to ensure that any data losses are detected and fall within the tolerances levels established by repository policy (see A3.5).

Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Documents that specify bit error detection and correction mechanisms used; risk analysis; error reports; threat analysis; periodic analysis of the integrity of repository holdings.

Discussion
The objective is a comprehensive treatment of the sources of data loss and their real world complexity. Any data or metadata that is (temporarily) lost should be recoverable from backups. Routine systematic failures must not be allowed to accumulate and cause data loss beyond the tolerances established by the repository policies. Mechanisms such as checksums (MD5 signatures) or digital signatures should be recognized for their effectiveness in detecting bit loss and incorporated into the overall approach of the repository for validating integrity.

C1.1.3.1 Repository reports to its administration all incidents of data corruption or loss, and steps taken to repair/replace corrupt or lost data.

Supporting Text
The repository must record, report, and repair as soon as possible all violations of data integrity. This is necessary in order to ensure the repository administration is being kept informed of incidents and recovery actions, and to enable identification of sources of data corruption or loss.

Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Procedures related to reporting incidents to administrators; Preservation metadata (e.g., PDI) records; comparison of error logs to reports to administration; escalation procedures related to data loss; tracking of sources of incidents; remediation actions taken to remove sources of incidents.

Discussion
Having effective mechanisms to detect bit corruption and loss within a repository system is critical but it is only the initial step of a larger process. In addition to recording, reporting and repairing as soon as possible all violations of data integrity, these incidents and the recovery actions and their results must be reported to administrators and made available to all relevant staff. Given identification of the sources of data loss, an assessment of revisions to software and hardware systems, or operational procedures, or management policies is needed to minimize future risk of data loss.

C1.1.4 Repository has a process to react to the availability of new security updates based on a risk-benefit assessment.


Supporting Text
The repository must maintain a record of all security updates containing an evaluation of the risk of applying the update vs. not doing so, and a record of which security updates were applied. This is necessary in order to protect the integrity of the archival objects from unauthorized changes or deletions.

Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Risk register (list of all patches available and risk documentation analysis); evidence of update processes (e.g., server update manager daemon); documentation related to the update installations.

Discussion
Decisions to apply security updates are likely to be the outcome of a risk-benefit assessment; security patches are frequently responsible for upsetting alternative aspects of system functionality or performance. It may not be necessary for a repository to implement all software patches, and the application of any must be carefully considered. Each security update implemented by the repository must be documented with details about how it is completed; both automated and manual updates are acceptable. Significant security updates might pertain to software other than core operating systems, such as database applications and Web servers, and these should also be documented. Security updates are not limited to software security updates. Updates to actual hardware or to the hardware system's firmware are included. Over time it is likely that security updates will also be needed for the repository processes and for its physical security. Although security updates can be considered as a part of the change control, they are identified separately here because there are often outside services that compile and circulate information on security issues and updates. At a minimum, repositories should be monitoring these services to ensure that repository held data is not subject to compromise by identified threats.

C1.1.5 Repository has defined processes for storage media and/or hardware change (e.g., refreshing, migration).

Supporting Text
The repository must identify the expected lifetime for each type of storage media (and its supporting hardware) in use at the repository and identify a process to copy the data off of the media prior to it reaching that point. This is necessary in order to ensure that data is not lost when either the media fail or the supporting hardware can no longer be used to access the data.

Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Documentation of migration processes; policies related to hardware support, maintenance, and replacement; documentation of hardware manufacturer's expected support life cycles; policies related to migration of records to alternate hardware systems.

Discussion
The repository should have estimates of the access speed and the quantity of information for each type of storage media. Then with estimates of the reliable lifetime of the storage media and information of system loading, etc., the repository can estimate the time required for storage media migration, or refreshing or copying between media without reformatting the bitstream. The repository can then set triggers for initiating the action at an appropriate time so the actions will be completed before data is lost. Copying large quantities of data can take a long time and can affect other system performance metrics. Repositories should also consider the obsolescence of any and all hardware components within the repository system as potential trigger events for migration. Increasingly, long-term, appropriate support for system hardware components is difficult to obtain, exposing repositories to risks and liabilities should they chose to continue to operate the hardware beyond the manufacturer or third-party support warranties. Repositories will likely need to perform media migration off of some types of media onto better supported media based on the estimated lifetime of hardware support rather than on the longer life expected from the media. It is important that the process includes a check that the copying has happened correctly. (See B4.2.2)

C1.1.6 Repository has identified and documented critical processes that affect its ability to comply with its mandatory responsibilities.

Supporting Text
The repository must ensure that each process that is applied to meet the mandatory preservation responsibilities (authenticity, integrity, chain of custody) is identified. This is necessary in order to ensure that the critical processes can be monitored to ensure that they continue to meet the mandatory responsibilities and to ensure that any changes to those processes are examined and tested.

Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Traceability matrix between processes and mandatory requirements.

Discussion
Examples of critical processes include data management, access, archival storage, ingest, and security processes. Traceability makes it possible to understand which repository processes are required to meet each of the mandatory responsibilities.

C1.1.6.1 Repository has a documented change management process that identifies changes to critical processes that potentially affect the repository's ability to comply with its mandatory responsibilities.

Supporting Text
The repository must ensure that changes to the preservation and management processes are tracked. This is necessary in order to ensure that the repository can specify not only the current processes, but the prior processes that were applied to the repository holdings.

Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Documentation of change management process; assessment of risk associated with a process change; analysis of the exptected impact of a process change; comparison of logs of actual changes to processes versus associated analyses of their impact and criticality.

Discussion
Examples of this would include changes to processes for data management, access, archival storage, ingest, and security. The really important thing is to be able to know what changes were made and when they were made. Traceability makes it possible to understand what was affected by particular changes to the systems. If unintended consequences are later discovered, then having this record may make it possible to reverse the changes or at least to document the changes that were introduced. Change management is a component of the broader topic of configuration mangement described by ISO 10007:2003 which includes configuration management planning, configuration identification, change control, configuration status accounting and configuration audit. Configuration Management efforts should result in a complete audit trail of decisions and design modifications.

C1.1.6.1.1 Repository has a process for testing the effect of changes to the repository's critical processes.

Supporting Text
The repository must test and evaluate the impact of every change to any critical process. This is necessary in order to protect the integrity of the repository's critical processes such that they continue in their ability to meet the repository's mandatory requirements.

Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Documented testing procedures; documentation of results from prior tests and proof of changes made as a result of tests; analysis of the impact of a process change.

Discussion
Changes to critical systems should be, where possible, pre-tested separately, the expected behaviors documented, and roll-back procedures prepared. After changes, the systems should be monitored for unexpected and unacceptable behavior. If such behavior is discovered the changes and their consequences should be reversed. Whole-system testing or unit testing can address this requirement; complex safety-type tests are not required. Testing can be very expensive, but there should be some recognition of the fact that a completely open regime where no changes are ever evaluated or tested will have problems.

C1.2 Repository manages the number and location of copies of all digital objects.

Supporting Text
The repository system must be able to identify all stored digital objects, including the location of each object (all copies and all versions). This is necessary in order to assert that the repository is providing an authentic copy of a particular digital object.

Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Random retrieval tests; validation of object existence for each registered location; validation of a registered location for each object on storage systems; provenance and fixity checking (see C1.5)
BA 16April2009 - There is no C1.5.

DG 20090419 - suggest we delete this reference

information; location register/log of digital objects compared to the expected number and location of copies of particular objects; .

Discussion
A repository can have different preservation policies for different classes of objects, depending on factors such as the producer, the information type, or its value. Repositories may require adifferent number of copies for each class, or manage versions needed to meet access requirements. There may be additional identification requirements if the data integrity mechanisms use alternative copies to replace failed copies. The location of each digital object must be described such that the object can be located precisely, without ambiguity. The location can be an absolute physical location or a logical location within a storage media or a storage subsystem. Provenance information about copying and moving the data, must be maintained/updated, including the identification of those responsible. This is necessary in order to track chain of custody and assert that the repository is providing an authentic copy of a particular digital object. The repository must be able to distinguish between versions of objects or copies and identical copies. This is necessary in order that a repository can assert that it is providing an authentic copy of the correct version of an object.

C1.2.1 Repository has mechanisms in place to ensure any/multiple copies of digital objects are synchronized.


Supporting Text
The repository must have a mechanism to ensure that intentional changes to an object are propagated to all copies of that object within a time established as acceptable by the repository. This is necessary in order to ensure that multiple copies of a digital object remain identical, and that a copy can be used to replace a corrupted copy of the object.

Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Synchronization workflows; system analysis of how long it takes for copies to synchronize; procedures/documentation of synchronization processes.

Discussion
The disaster recovery plan should address what to do should a disaster and an update coincide. For example, if one copy of an object is altered and a disaster occurs while the second is being updated, there needs to be a mechanism to assure that the copy will be updated at the first available opportunity. The mechanisms to synchronize copies of digital objects should be able to detect bit corruption, and validate fixity checks before synchronization is attempted.

C2 Security risk management (HT)

C2.1 Repository maintains a systematic analysis of security risk factors associated with data, systems, personnel, and physical plant.


Supporting Text
The repository must conduct regular risk assessments and maintain adequate security protection in order to provide expected and contracted levels of service, following codes of practice such as ISO 27000.

This is necessary to ensure ongoing and uninterrupted service to the designated community(ies).

Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Repository employs the codes of practice found in the ISO 27000 series of standards system control list; risk, threat, or control analysis.

Discussion
“System” here refers to more than IT systems, such as hardware, software, communications equipment and facilities, and firewalls. Fire protection and flood detection systems are also significant, as are means to assess personnel, management and administration procedures, resources, as well as operations and service delivery. Loss of income, budget and reputation are significant threats to overall operations as is loss of mandate. On-going internal and external evaluation should be conducted to assess quality of service and relevance to user community served and periodic financial audits should be secured to ascertain ethical and legal practice and maintenance of required operating funds. Intellectual property rights practices should also be reviewed regularly as well as the repository’s liability for regulatory non-compliance as applicable. The repository should assess its staff’s skills against those required in the evolving digital repository environment and ensure acquisition of new staff or retraining of existing staff as necessary. Regular risk assessment should also address external threats and denial of service attacks and loss of or unacceptable quality of third party services. Repository may conduct overall risk assessments with tools such as DRAMBORA.

C2.2 Repository has implemented controls to adequately address each of the defined security risks.


Supporting Text
The repository must show how it has dealt with its security requirements. If some types of material are more likely to be attacked, the repository will need to provide more protection, for instance. This is necessary in order to ensure that controls are in place to meet the security needs of the repository.

Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Repository employs the codes of practice found in the ISO 27000 series of standards; system control list; risk, threat, or control analyses; and addition of controls based on ongoing risk detection and assessment. Repository maintains ISO 17799 certification.

Discussion
Repositories that have experienced incidents could record such instances, including the times when systems or content were affected and describe procedures that have been put in place to prevent similar occurrences in the future. Repositories may also conduct a variety of disaster drills that may involve their parent organization or the community at large. Contingency plans are especially important and need to be tested, updated, and revised on a regular basis.

C2.3 Repository staff has delineated roles, responsibilities, and authorizations related to implementing changes within the system.


Supporting Text
The repository must assign individuals to be responsible for implementing changes within the system and provide them with the authority and resources to implement such changes. This is necessary in order to ensure that individuals have the authority to implement changes, that adequate resources have been assigned for the effort, and that the responsible individuals will be accountable for implementing such changes.

Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Repository employs the codes of practice found in the ISO 27000 series of standards; organizational chart; system authorization documentation. Repository maintains ISO 17799 certification.

Discussion
Authorizations are about who can do what: who can add users, who has access to change metadata, who can access audit logs. It is important that authorizations are justified, that staff understand what they are authorized to do, that staff have required skills associated with various roles and authorizations, and that there is a consistent view of this across the organization.

C2.4 Repository has suitable written disaster preparedness and recovery plan(s), including at least one off-site backup of all preserved information together with an offsite copy of the recovery plan(s).


Supporting Text
The repository must have a written plan with some approval process for what happens in specific types of disaster (fire, flood, system compromise, etc.) and for who has responsibility for actions. This is necessary in order to ensure that sufficient backup and recovery capabilities are in place to facilitate continuing preservation of and access to systems and their content with limited disruption of services.

Examples of Ways the Repository can Demonstrate it is Meeting this Requirement
Repository employs the codes of practice found in the ISO 27000 series of standards; disaster and recovery plans; information about and proof of at least one off-site copy of preserved information; service continuity plan; documentation linking roles with activities; local geological, geographical, or meteorological data or threat assessments. Repository maintains ISO 17799 certification.

Discussion
The level of detail in a disaster plan, and the specific risks addressed need to be appropriate to the repository's location and service expectations. Fire is an almost universal concern, but earthquakes may not require specific planning at all locations. The disaster plan must, however, deal with unspecified situations that would have specific consequences, such as lack of access to a building or widespread illness among critical staff. In the event of a disaster at the repository, the repository may want to contact local and/or national disaster recovery bodies for assistance. Repositories may also conduct a variety of disaster drills that may involve their parent organization or the community at large.

Edit | Attach | Watch | Print version | History: r2 < r1 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r2 - 2009-04-20 - DavidGiaretta
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2018 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback