Notes from Megameeting 7th December 2009

Attendees

Summary

There was a discussion of the implications for the surveillance audit regime of the sizes and budgets of repositories, particularly whether the requirements might be prohibitively expensive. This has a bearing on how much ISO 17021 is followed or varied in our own document.

Actions

• All to go through the requirements of ISO 17021 and indicate which of the

Transcript of chat

Mark Conrad >> (All): Was there a meeting last week?
SimonLambert >> (All): I don't think so - neither David nor I was available and 
I didn't see any notes from anyone else.
Mark Conrad >> (All): Ok. An David and Bruce won't be here today, either.
Mark Conrad >> (All): Looking at David's proposed text and trying to load the 
referenced standards.
SimonLambert >> (All): The key addition I think is in 8.1.1: "on a periodic 
basis agreed with the initial audit committee" as the possibility to vary the 
stipulations of the parent standards.
Mark Conrad >> (All): We will follow the referenced standards except when we 
choose to ignore them?
SimonLambert >> (All): I think that all sections do say "The requirements from 
XXX apply".
SimonLambert >> (All): And then we amplify them or - in this case - vary them.
Mark Conrad >> (All): Thus my statement.
JohnGarrett >> (All): There was  a few of us who were here last week, but we 
didn't really make any progress since the ones sensitive to current issues
weren't here last week.
Mark Conrad >> (All): Were there any notes from last week?
JohnGarrett >> (All): No, we didn't save them.  We didn't discuss this since you 
and David and SImon and Bruce were all gone.
Mark Conrad >> (All): David and Bruce will not be here again today. How should 
we proceed?
JohnGarrett >> (All): If possible, I would like to discuss this recertification 
issue and come up with a proposal.   We need to resolve this to move on.
JohnGarrett >> (All): I don't think people will have much respect for a 
certification that isn't renewed fairly often.
Mark Conrad >> (All): I don't think too many people will want to pay the costs 
necessary to maintain the kind of recertification regime in the referenced 
standards.
RobertDowns >> (All): We might specify the time period for which a certified 
repository remains certified and the monitoring and reporting necessary for 
renewal.
Mark Conrad >> (All): That is specified in the referenced standards.
JohnGarrett >> (All): You're absolutely right.  No one will want to pay for it.  
But on the other hand, I think they can get funders to pay for if the funders 
think it is worthwhile. 
Mark Conrad >> (All): Who are these magical funders and where do I find one?
RobertDowns >> (All): We should be careful to ensure that less funded 
repositories can maintain certification as well as the well-funded repositories.
JohnGarrett >> (All): For NARA it is congress.  I could see legislation 
requiring them to be certified.  Of course it would be difficult to see the 
costs in that budget.
Mark Conrad >> (All): I don't see Congress taking up that legislation.
JohnGarrett >> (All): For smaller repositories there is usually another 
organization upstream in the organization or some other grant provider that 
could expect it.
Mark Conrad >> (All): John, Most smaller repositories are having trouble getting 
enough money to keep their current programs going.
JohnGarrett >> (All): Mark, OK, I'm sure you have a better idea of what would 
happen.  Is it likely that NARA would ask any of the other agencies that send 
things to it to  be certified?
Mark Conrad >> (All): No.
JohnGarrett >> (All): Mark, I'm getting the impression that you don't think 
anyone will use this standard.
Mark Conrad >> (All): I think folks will use this standard if the costs of 
obtaining and maintaining certification are not too costly.
Mark Conrad >> (All): I don't think we can require the number and level of 
follow-on audits in the referenced standards.
Mark Conrad >> (All): From ISO 17021: 9.3.2.2 Surveillance audits shall be 
conducted at least once a year. The date of the first surveillance 
auditfollowing initial certification shall not be more than 12 months from the 
last day of the stage 2 audit.
JohnGarrett >> (All): What is the price point for organizations being willing to 
maintain certification?
Mark Conrad >> (All): For most repositories I would guess no more than 5K over 5 
years. And that would stretch many repositories.
JohnGarrett >> (All): I see the current requirements for a moderate sized 
repository to be the cost of a several day trip for maybe three people for the 
initial audit and the cost of a single day trip for a single person for the 
annual audits.
RobertDowns >> (All): There also is the cost of preparing for these audits.
Mark Conrad >> (All): Travel would be about $5600 for that level of onsite 
auditing for five years.
RobertDowns >> (All): Would that include the time for the people who are 
involved?
JohnGarrett >> (All): OK, so we have to figure out a way to cut costs some then. 
The current parent standards would be more than $5K over 5 years.  But a cost of 
only $1K per year is really really small.
Mark Conrad >> (All): Robert, no.
Mark Conrad >> (All): Really really small for who?
RobertDowns >> (All): We should find a way to reduce the costs of maintaining 
certification
RobertDowns >> (All): so that it is not prohibitive for small repositories
TerryLongstreth >> (All): How big is a moderate sized repository? Do we want 
auditors to take repository size into account when tailoring the certification 
schedules?
Mark Conrad >> (All): My proposal would be to de-couple our document from the 
currently-referenced ISO standards or explicitly state that we will apply a less 
rigorous regime for maintaining certification than is proposed in the referenced 
standards.
JohnGarrett >> (All): I think we do take repository size into consideration.  I 
think recertification schedule should be the same for all repositories, but the 
amount of effort and the number of people involved would be larger for larger 
repositories.
RobertDowns >> (All): Mark's proposal makes sense for ensuring that less-funded 
repositories can participate in the continuing certification process.
Mark Conrad >> (All): Terry, In the archival world, very few archives even have 
digital repositories. The ones that do tend to have small collections ~1-2 TB.
TerryLongstreth >> (All): Recertification schedule probably should be a function 
of the dynamics of the repository; the more volatile the collections, the more 
auditing needed
Mark Conrad >> (All): John, ISO 17021 requires an annual on-site surveillance 
audit.
JohnGarrett >> (All): Most of our general requirements are being picked up from 
17021 in the same way as virtually every ISO certification standard does.  If we 
start over completely not using it, this effort would be hampered greatly.  If 
we want, we should continue to reference it for most, but specify different 
recertification periods.
JohnGarrett >> (All): What recertification periods do we think are acceptable?
Mark Conrad >> (All): I would suggest that everyone read the referenced 
standards before we decide whether or not to incorporate them by reference or 
not.
TerryLongstreth >> (All): I think there should be an annual "touch", but the 
scope and intensity of the interaction could vary with the size and activity of 
the repository
JohnGarrett >> (All): I don't think digital repositories with only 1-2 TB of 
data will ever pay anything to be certified.  
Mark Conrad >> (All): Ok who does that leave?
JohnGarrett >> (All): I hope that they would self-certify themselves like very 
small organization do for ISO9000, but only decent sized organizations are 
generally willing to pay to be ISO 9000 certified.
Mark Conrad >> (All): Do we have a mechanism for self-certification?
TerryLongstreth >> (All): I'm with John.. The auditor's role for small 
repositories, is reviewing and perhaps critiquing the self-cert reports
RobertDowns >> (All): How would we certify self-certified repositories to ensure 
that they are not left out
JohnGarrett >> (All): I think in general only organizaitons approaching 100 
employees would be pay for ISO 9000 certification (and then only when required 
for a contract, but then they build the costs into their bid)
Mark Conrad >> (All): Other than National Archives, I do not know too many 
archives with anywhere near that number of employees.
JohnGarrett >> (All): I don't think we certify very small digital repositories.  
I think it is very much like ISO 9000.  Small companies can claim that the 
follow ISO 9000 practices, but the are not registered as being certified by any 
outside organization.
RobertDowns >> (All): Repositories are not funded like ISO 9000 corporations
TerryLongstreth >> (All): So, for the smaller repositories, the audit point of 
presence (website?) would list who has submitted reports and whether the reports 
met with audit approval
Mark Conrad >> (All): John, Can you name some repositories that you think will 
seek certification? Everything that you have said about potential users of the 
standard doesn't sound like any archives I know about.
Mark Conrad >> (All): Terry, Then we would have to drastically change the audit 
document and de-couple from the regimes described in the referenced standards.
JohnGarrett >> (All): Realistically, I think only larger organizations will be 
willing to be certified.  National, state archives, National libraries, space 
projects, oil companies, banks, large organizational or university repositories.
Mark Conrad >> (All): Most state archives and university repositories would not 
be classified as large organizations.
TerryLongstreth >> (All): I'm trying to offer suggestions that soften the blow, 
and encourage participation.  The site visit requirement may be impossible to 
meet anyway. If that's part of 17021,  then let's provide an entry level step 
that meets the form but avoids the full cost.  Many home systems have two 
terabytes, so there's a huge potential for participation if we could lower the 
entry barriers.
SimonLambert >> (All): I notice that section 9.3.1.2 starts "As a minimum, 
surveillance shall review the following" - by defining this minimum, can we 
adapt to our needs here?
JohnGarrett >> (All): I think an organization our size, 50-ish employees, might 
in good times be able to convince the people up the funding chain to commit $10K 
over 3 years to get the political cover to have the archives certified.
Mark Conrad >> (All): Many state archives are furloughing their employees 
several days a month.
JohnGarrett >> (All): Maybe we can have two levels of certification.  One that 
includes site visits and one that doesn't.
Mark Conrad >> (All): Then we have to say we are not fully following ISO 17021 - 
which calls for annual on-site surveillance audits.
Mark Conrad >> (All): Simon, I don't see what you are citing at 9.3.1.2.
JohnGarrett >> (All): That's right.  If we're not going to fully follow ISO 
17021, then we have to say that.  I would like to follow it as much as possible, 
so we don't have to document too many exceptions and so we don't have to justify 
to many exceptions when we get comments back from the review.  We do want to get 
it passed as an ISO Standard.
SimonLambert >> (All): Mark - it's the title of that section itself.
Mark Conrad >> (All): Simon, This is what I see at 9.3.1.2.: 9.3.1.2 
Surveillance activities shall include on-site audits assessing the certified 
client's managementsystem's fulfilment of specified requirements with respect to 
the standard to which the certification is granted.Other surveillance activities 
may include
TerryLongstreth >> (All): But 17021 is about 'management systems', we could set 
up to manage the Audit Authority as a management system ( i.e, how we manage 
audits and auditors), but not as a requirement for their clients
Mark Conrad >> (All): I would suggest that we all read the referenced standards 
before our next meeting. There are lots of other requirements in them that I do 
not think will work for TDR audit.
SimonLambert >> (All): That's strange - I wonder if the numbering has got out of 
synch somehow :-(  I am looking at the wiki page http://wiki.digitalrepositoryauditandcertification.org/bin/view/Main/AuditorGuidelinesIssues
Mark Conrad >> (All): Simon, I am looking at ISO 17021:2006.
SimonLambert >> (All): Mark - OK, I thought that the section numbers 
corresponded 1:1 but maybe they don't.
JohnGarrett >> (All): The major sections correspond one-to-one, but the 
subsections may not.
JohnGarrett >> (All): The text in our standard indicates what text is being 
picked up in ISO 17021
RobertDowns >> (All): Following up on Terry's observation, we need to be clear 
whether we are specifying requirements for the auditing body or for the 
repositories being audited. 
Mark Conrad >> (All): John, Our document says: This International Standard 
specifies requirements and provides guidance for bodies providing audit and 
certification of a trusted digital repository (TDR), IN ADDITION to the 
requirements contained within ISO/IEC 17021 and <ISO XXXXX - RAC Document>. 
JohnGarrett >> (All): Yes, this standard is for the auditing body.  But there 
are a lot of requirements in there that specify how the auditing body interacts 
with the repositories.
RobertDowns >> (All): Can we propose that, during monitoring for re-
certification,  the auditing body reviews reports received from certified TDRs 
to determine eligibility for renewal?
Mark Conrad >> (All): Our document section 8.1. says: b)   surveillance and 
recertification audits of a client organization's TDR in accordance with ISO 
19011 and ISO/IEC 17021 on a periodic basis for continuing conformity with 
relevant requirements and for verifying and recording that a client organization 
takes corrective action on a timely basis to correct all nonconformities.
JohnGarrett >> (All): Mark, correct, we generally say that throughout the 
document.  Each section usually says that.   For the section on auditing 
periods, we would have to say something to specifically say that we do not 
follow the periods given in ISO 17021 and instead do not require on-site annual 
reviews.
TerryLongstreth >> (All): We're not inspecting repositories for compliance as 
management systems
TerryLongstreth >> (All): We shouldn't have to say we're not following 17921 if 
it's not relevant to the goals of the audit
JohnGarrett >> (All): I don't know, I think we are looking at repositories as 
management systems for digital data.
Mark Conrad >> (All): The way our document is currently written, it says that we 
are only adding requirements IN ADDITION to those found in ISO 17021.
TerryLongstreth >> (All): Mark: then our document should probably be a profile 
of 17021
Mark Conrad >> (All): I really think that we need to read 17021 and decide if we 
want to reference it in our document.
Mark Conrad >> (All): Terry, What is a profile of a standard?
JohnGarrett >> (All): As homework for next week, why don't we ask everyone who 
hasn't done so to go through the requirements of 17021 and indicate which of the 
requirements they think we should require and which ones we should leave out.
JohnGarrett >> (All): As an aide, each section of the Word document includes the 
text from 17021 in boxes immediately followed by the adaptations that we are 
making to it.
TerryLongstreth >> (All): Yes, John. I thought we were using 17021 as a 
template, but it's clear now that it's being posited as normative. 
Mark Conrad >> (All): John, I would suggest reading ISO 17021 in its entirety.
JohnGarrett >> (All): OK, I think it is worth doing that.
JohnGarrett >> (All): Terry, I think the document we used as a template was ISO 
21006 which was for security systems.  That standard also referenced ISO 17021. 
JohnGarrett >> (All): Bye
TerryLongstreth >> (All): Ok, I was confused (it happens fairly often). But if 
17021 is normative, we need to see it all.

-- SimonLambert - 07 Dec 2009