Notes from Megameeting 19th October 2009

Attendees

Summary

The review of the "Requirements for Bodies ..." document continued from section 7.2.1.3. There was a debate about the appropriate level of qualifications and experience in the early stages, and how to bootstrap the process. It was generally agreed that a lead audit committee is needed, but how to express that in the document is not quite clear.

Specific agreements:

• 7.2.1.3.1 (b) should be re-expressed as requiring experience in "experience in information technology, data management, libraries or archives" - or something similar.

• Omit 7.2.1.3.1 (f).

• It will be necessary to specify familiarity with OAIS and the metrics document.

• For 7.2.1.3.2 (b), two audits is a more sensible requirement than three.

To look at next time:

• Start with section 9.2.1, but note that we will need to return to 7.2.1.1 and 7.2.1.3

The next meeting will be on Monday October 26, and probably Wednesday 28 as well.

Transcript of chat

David Giaretta >> (All): The notes say that we were continuing from 7.2.1.1 - 
were people looking at John's document or the WIki.page?
SimonLambert >> (All): We were looking at John's doc as it has the context from 
ISO 17021
David Giaretta >> (All): Do it was agree to delete 7.2.1.1 (d) - is that right?
RobertDowns >> (All): Last week, it was proposed that we drop 7.2.1.1 d and h
RobertDowns >> (All): I have been looking at the following page: http://wiki.digitalrepositoryauditandcertification.org/bin/view/Main/AuditorGuidelinesIssues
David Giaretta >> (All): John's doc is at http://wiki.digitalrepositoryauditandcertification.org/pub/Main/ReqtsForAuditors/AuditorGuidelines-rev2009Sep09w17021InBoxes.doc
David Giaretta >> (All): he extracted text from the "parent" docs and places 
them in context so we could understand what was going on better. Those boxes 
will be deleted in the final version
JohnGarrett >> (All): So are we back to looking at 7.2.1.3?
David Giaretta >> (All): Who was leading last week?
David Giaretta >> (All): OK 7.2.1.3 - pretty key
David Giaretta >> (All): (d) is key - they others are fairly general
SimonLambert >> (All): How does this allow the newly qualified auditor to gain 
experience?
SimonLambert >> (All): Do we perhaps want a broader statement about the team as 
a whole?
David Giaretta >> (All): Simon - gains experience by accompanying an audit team
JohnGarrett >> (All): I think d) is too stringent for the first issue of this 
recommendation.  I think 5 audits and 20 days of auditing is too much.  Maybe in 
issue 2 after 5 years, we could update it to that.
RobertDowns >> (All): We could change "should" to "could".
RobertDowns >> (All): We also might allow for education and other experiences to 
qualify an individual.
David Giaretta >> (All): I'll take a look at some other ISO standards and see if 
how it was dealt with there, but I like John's suggestion. Robert's suggestions 
are good too but we need to be careful with phrasing
David Giaretta >> (All): I was also wondering about the first audit committee - 
for which (d) could not apply
JohnGarrett >> (All): It looks to me like it is mostly experience based now.  a) 
only requires secondary (pre-university) education and c) is 5 days of training 
(which maybe we would drop in the first issue also until programs can be set up)
David Giaretta >> (All): ...we could mention it as a special case and then say 
auditors are approved by it
RobertDowns >> (All): The experience that (e) refers to should be specified. 
David Giaretta >> (All): John (c) - yes we would have to be sure such training 
exists - and again the first audit committee could approve it
David Giaretta >> (All): Same with (e)
JohnGarrett >> (All): I would contract the d) requirements to participation in 1 
or 2 audits.  Then if many of us participate in a audits before it is approved 
as a standard, then we will have the first class of lead auditors to boot up the 
others.
David Giaretta >> (All): I wonder if it would be sensible to specify that first 
audit committee and make things depend on that for the first few years and then 
with Issue 2 change things
David Giaretta >> (All): John - yes
JohnGarrett >> (All): Yes, I think that is the way to go.  Have a fairly low bar 
set up for issue 1 and raise the bar in subsequent issues based on what the 
community can support.
SimonLambert >> (All): Just a point of procedure - do we mention this thinking 
in the document (that we expect things to change in the 2nd issue), or just keep 
it to ourselves for now?
JohnGarrett >> (All): I think we need to specify what requirements the 1st group 
needs to meet and not who they are.  I don't think we would ever get approval 
for the latter.
David Giaretta >> (All): John - yes indeed
Mark Conrad >> (All): b) bothers me. Most folks I know who have IT in their 
title are not concerned with digital preservation and most of us that are 
concerned with digital preservation do not have IT in our job titles.
JohnGarrett >> (All): Actually, I think a note indicating that requirements may 
increase in future years might be a good idea.  It would encourage others to do 
audits early and to get their auditors to the level needed.
SimonLambert >> (All): Mark - that was inherited from the base standard I think.
David Giaretta >> (All): Simon - not sure - I guess a footnote may do
JohnGarrett >> (All): As with any standard, the more people who use and 
recognize the standard, the better and more effective it is.
David Giaretta >> (All): Mark - I agree - we need some other words - maybe 
libraries/archives/IT things
RobertDowns >> (All): Mark - we could refer to data management experience.
David Giaretta >> (All): Good point
SimonLambert >> (All): Does (f) say anything useful :-)
Mark Conrad >> (All): Robert - That might work.
JohnGarrett >> (All): I agree with Mark.  I think b) wording was inherited from 
the IT security standard.  I was going to suggest that we change b) to be IT or 
archival/preservation background.  
Mark Conrad >> (All): Simon - No.
Mark Conrad >> (All): John - I like Robert's suggestion better.
David Giaretta >> (All): How about "experience in data management, libraries, 
archives"
BarbaraSierman >> (All): better
David Giaretta >> (All): Ooops "experience in data management, libraries or 
archives"
JohnGarrett >> (All): I like having the IT perspective because many archival 
types don't have the digital perspective.  But most IT types won't have the 
archival perspective.  I would allow either one as a background and the total 
set of these requirements add enough that the wider view is also supported.
David Giaretta >> (All): John the following sentence says "at least two years 
are in a role or function relating to digital preservation"
BarbaraSierman >> (All): the mix of the team need to represent these qualities
David Giaretta >> (All): ...but we could add IT to the list I suggested
RobertDowns >> (All): rather than referring to specific units, where someone 
could be engaged in limited tasks that have nothing to do with data management 
or digital preservation. Perhaps we should specify areas of responsibility, 
instead. 
BarbaraSierman >> (All): david, we will always miss areas, like museums
RobertDowns >> (All): That's why we should not list units and list 
responsibilities.
David Giaretta >> (All): Yes, these are examples I guess - we should make it 
clear
JohnGarrett >> (All): OK, I like Robert's suggestion also.  Also want to keep 
the coming from the IT side open.
RobertDowns >> (All): How about data management and digital preservation.
Mark Conrad >> (All): Somewhere in here we need to have something about 
familarity with the OAIS, don't we?
David Giaretta >> (All): Good point - also the Metrics doc
David Giaretta >> (All): Robert - looks good
David Giaretta >> (All): Can I move to 7.2.1.3.2 -  what about (b)
David Giaretta >> (All): ...can we shorten that to 2 audits?
RobertDowns >> (All): For 7.2.1.3.2 (b) two audits makes sense for Issue 1.
BarbaraSierman >> (All): i think that is more realistic, is auditing on a 
voluntary (non-paid) basis?
David Giaretta >> (All): I guess some might be but this involves people's time - 
several days each - so someone would have to pay eventually
Mark Conrad >> (All): How do they conduct these two audits, "under guidance and 
supervision".
BarbaraSierman >> (All): someone should have an overall control?
David Giaretta >> (All): Mark - yes I guess it;s implicit that they were not 
audit leaders for those two, but we could state it explicitly
David Giaretta >> (All): Barbara - what do you mean -control of what?
RobertDowns >> (All): Before there is a certification process, they would have 
to be internal, but they could have oversight from organizational authorities.
BarbaraSierman >> (All): David- the one with the supervision
Mark Conrad >> (All): My question is who is providing the guidance and 
supervision for the two audits?
David Giaretta >> (All): It has to be the "first audit committee" initially 
until others gain experience to become leads
Mark Conrad >> (All): How does the first audit committee get their experience, 
etc?
JohnGarrett >> (All): I think auditing experience should count whether it is 
paid or not.  It is the experience that is important.
David Giaretta >> (All): SO the first audit cttee are defined as having enough 
recognition for the requirements of 7.2.1.3 not to apply 
David Giaretta >> (All): John - good point - it is not specified in the doc.
RobertDowns >> (All): the first auditing committee could obtain their experience 
throught internal audits
David Giaretta >> (All): And test audits
David Giaretta >> (All): ...of willing 3rd parties
David Giaretta >> (All): That was one of the aims of the test audits we planned
David Giaretta >> (All): but essentially the first cttee is something special 
and we may need to define it explicitly
David Giaretta >> (All): ...ie in terms of international recognition rather than 
names of course
Mark Conrad >> (All): There has to be a big bang - a starting point. We seem to 
be starting several days later.
David Giaretta >> (All): Yes, but I think we need the standards in place before 
we can set up the cttee - chicken and egg!
JohnGarrett >> (All): How about we add a note to the first issue indicating that 
it is expected that the pool of qualified auditors will grow over time, but at 
the beginning it may be limited.  And add that the Technical Committee 
responsible for this standard (i.e. our group) can help locate qualified 
auditors or individuals with recognized authority/experience.
Mark Conrad >> (All): It would be helpful to know how this has been done for 
other standards - e.g., ISO 9000 certification.
David Giaretta >> (All): Mark - I agree - they may have done it as John suggests 
OR via some other mechanism in ISO. We don't necessarily have to put everything 
in this doc
RobertDowns >> (All): Rather than "recognized" authority/experience, which might 
be considered ambiguous, we might refer to documented or demonstrated 
experience.
David Giaretta >> (All): But documented to whom?
JohnGarrett >> (All): I suspect the communities for many of the other standards 
are both wider communities and the experience was already in place.  Certainly 
there were people auditing quality (even without a formal standard) before the 
first issue of ISO 9000.  And there have been audits of IT security before the 
ISO standard.
RobertDowns >> (All): The experience could be documented by those who designate 
an individual as an auditor.
JohnGarrett >> (All): Robert, yes I agree, it should be documented or 
demonstrated rather than recognized.  And it is the responsibility of the 
organization performing the audit to get that documentation and ensure it is 
adequate.  For now we are not setting up an organization to give certifications 
to individuals.
David Giaretta >> (All): But the body offering the international certification 
has to have a first audit cttee - I thought that is what we are talking about 
JohnGarrett >> (All): BY the way, there is another base ISO standard for 
organizations certifying individuals.  It is amazing how many standards and 
guides ISO has for certification activities.  Must be about 40 different 
standards for each piece of the certification activities from the group that 
brought us the ISO 17021 base standard we are using. 
David Giaretta >> (All): Anyway the plan was to get this doc finished and then 
to set up the organisation while it is undergoing review
JohnGarrett >> (All): I think that is a good approach.  Anyway, I think if we 
work on getting test audits performed by our first  class of lead auditors, we 
can claim that this group meets the initial criteria.
David Giaretta >> (All): We certainly need to do the test audits
Mark Conrad >> (All): Here is something interesting from the ISO 9000 website: 
The ISO 9001 Auditing Practices Group is constituted as an informal group of 
quality management system (QMS) experts, auditors and practitioners, drawn from 
the ISO Technical Committee 176 Quality Management and Quality Assurance (ISO/TC 
176) and the International Accreditation Forum (IAF).
RobertDowns >> (All): Can we state something similar regarding this group.
David Giaretta >> (All): Is it a group from whence the certification comes?
David Giaretta >> (All): Not sure if we want to jump ahead but the next section 
in Simon's WIki page about Issues is section 9.2.1 "Audit team competence"
David Giaretta >> (All): Would it be sensible to suggest we start on 9.2.1 next 
week unless someone sees something more urgent that should be dealt with?
JohnGarrett >> (All): OK with me.  
David Giaretta >> (All): I think there was an action on everyone to think about 
the definition of the first audit cttee
Mark Conrad >> (All): Ok. So we have not reached resolution on 7.2.1.1. or 
7.2.1.3.?
JohnGarrett >> (All): Will there be a meeting on Monday?  I'll be at the CCSDS 
meeting and hopefully will be connected by then  and we will also be having a 
megameeting on Wednesday?
David Giaretta >> (All): Mark - I though we had ground to a halt on 7 - but if 
not then we should continue on it.
RobertDowns >> (All): I believe that we have been making progress on 7. If we do 
not continue with it, we will have to come back to it.
David Giaretta >> (All): John - yes hope so - it would be 16:30 ESTEC time
Mark Conrad >> (All): David - I think we have ground to a halt. I think coming 
up with a definintion/concept for the first auditing committee will help resolve 
some of the issues with 7.2.1.1. and 7.2.1.3.
RobertDowns >> (All): I agree, that definition will help.
Mark Conrad >> (All): I just wanted to be clear that we were not saying we were 
done with 7.2.1.1. and 7.2.1.3.
David Giaretta >> (All): Should we attempt to propose something on Simon's Issues 
WIki page?
David Giaretta >> (All): Yes, I agree we are not done yet - just the first pass!
David Giaretta >> (All): John's version with the boxes helps a lot
SimonLambert >> (All): We could use John's version as the reference and start 
editing the wiki version to record the decisions.
David Giaretta >> (All): OK
SimonLambert >> (All): I can put in the decisons from previous weeks and this 
week
Mark Conrad >> (All): So are we meeting next Monday? Wednesday, both?
David Giaretta >> (All): Certainly Monday. Probably Wednesday if enough people 
can attend in MegaMeeting
JohnGarrett >> (All): OK, see you then.

-- SimonLambert - 19 Oct 2009