Notes from Megameeting 29th March 2010

Attendees

DavidGiaretta STFC
JohnGarrett GSFC
MarkConrad NARA
SimonLambert STFC
TerryLongstreth  

Summary

Discussion resumed of the document http://wiki.digitalrepositoryauditandcertification.org/pub/Main/ReqtsForAuditors/AuditorGuidelines-CCSDS-format-3.doc. It was agreed to adopt a minimalist approach, and discard inherited text if there is not a clear need for it in the TDR context.

Specific decisions:

  • 5.2.1 - remove bullet (b)
  • Delete all context of 7.1 apart from "The requirements from ISO/IEC 17021:2006, Clause 7.1 apply"
  • Move 7.2.1.1. up to 7.2.1, change a) to read ISO-RAC XXXX instead of TDR, put digital preservation back into b)

There was a discussion of whether it is needed to mention ISO 19011. This is fundamental to ISO 17021 and so does not need to be repeated here.

Mark Conrad >> (All): What is the relation ship of the document at http://wiki.digitalrepositoryauditandcertification.org/bin/view/Main/AuditorGuidelinesIssues to the document at 
http://wiki.digitalrepositoryauditandcertification.org/pub/Main/ReqtsForAuditors/AuditorGuidelines-CCSDS-format-3.doc?
SimonLambert >> (All): The Word doc is now the master
Mark Conrad >> (All): Ok. So that is the one we should be using for discussion 
and comment?
David Giaretta2 >> (All): Yes
David Giaretta2 >> (All): In the notes from last week, what was the problem is 
5.2.1?
TerryLongstreth >> (All): The numbering and its relationship to 17021 were 
ambiguous. Also, there was some feeling of discrepancies between it and the base 
17021 document.
Mark Conrad >> (All): I am pulling up 17021 so I can compare the two documents.
Mark Conrad >> (All): Some of the exceptions in 5.2.1. appear to be in conflict 
with 17026 clauses 5.2.6. and 5.2.7. 
David Giaretta2 >> (All): Curious - we did not write those bullet points - just 
copied I think
David Giaretta2 >> (All): Mark - did you mean in conflict with 17021 clauses?
Mark Conrad >> (All): Yes. Sorry!
JohnGarrett >> (All): Sorry, I don't see much that conflicts.  Maybe I"m just 
reading something else into the points than other people.
David Giaretta2 >> (All): Mark - can you point to a particular bullet?
JohnGarrett >> (All): a) to me seems pretty worthless and unneeded.  Mostly 
saying you can do this audit., but I don't see any harm in keeping it.  But I 
wouldn't mind deleting it.
Mark Conrad >> (All): 17021 Clause 5.2.6. The certification body shall not 
certify a management system on which it provided internalaudits within two years 
following the end of the internal audits. This also applies to that part of 
government identified as the certification body.
David Giaretta2 >> (All): What does that clash with in 5.2.1?
Mark Conrad >> (All): b and d
Mark Conrad >> (All): See also: 5.2.7 The certification body shall not certify a 
management system on which a client has received management system consultancy 
or internal audits, where the relationship between the consultancy organization 
and the certification body poses an unacceptable threat to the impartiality of 
the certification body.
JohnGarrett >> (All): For b) I didn't see a teaching a class being the same as 
an internal audit or consulting
Mark Conrad >> (All): Teaching a class is a.
JohnGarrett >> (All): SOrry, I'm switching between versions and the old a) has 
been eliminated.
David Giaretta2 >> (All): I can see that (b) could be read as a clash but I did 
not read it that way - we could easily delete it
Mark Conrad >> (All): What is the upside to leaving 5.2.1. in our document?
David Giaretta2 >> (All): ...(d) on the other hand - 2nd sentence seems to 
restate 17021. The point I read was that it made clear that the improvement plan 
was not a conflict of interest
JohnGarrett >> (All): b) then is the one that might be a problem, but I saw it 
as an allowed extension of the audit that allowed both sides to see what was 
involved in the actual audit.
Mark Conrad >> (All): In 17021 that second sentence applies to all of the 
bullets.
David Giaretta2 >> (All): Mark - I guess if we cut (b) then that only leaves the 
three bullets.
Mark Conrad >> (All): Sorry, I mean 27006
JohnGarrett >> (All): d) I think just says that during the audit, the team can 
say "this area is shaky and may need improvement" (and they may or may not pass 
that area)
Mark Conrad >> (All): Is there value in leaving in the three bullets?
David Giaretta2 >> (All): Mark - I guess we would not lose much if we deleted 
them. I would have soad they were all pretty obviously NOT a conflict of 
interest.
JohnGarrett >> (All): I think they do clarify things a bit, but if they cause 
other people problems, I don't think 17021 disallows any of them.  So it doesn't 
really matter if they are here.
JohnGarrett >> (All): On the other hand, if others think these activities are 
disallowed by 17021, they then need to stay to clarify what is allowed.
Mark Conrad >> (All): My thought is to keep this document as simple as possible. 
The more we specify what we don't really need, the more we lock in what the 
initial audit committee can and cannot do.
TerryLongstreth >> (All): I agree with Mark. When in doubt.>>>>
David Giaretta2 >> (All): It looks to me as if there were in the doc we copied 
from as a defense against criticism. A few weeks back I though we should inherit 
as much of these as we can but now I would agree with Mark - let's keep it 
simple.
TerryLongstreth >> (All): I think we need to see some piloting results 
David Giaretta2 >> (All): Yes, that was the plan - start the pilots before the 
docs finish in ISO so we can insert RIDS and make changes
Mark Conrad >> (All): What is the current status of the RAC document in terms of 
ISO?
TerryLongstreth >> (All): In particular, any restrictions on the activities of 
the auditors may drive up cost, i.e. hiring additional experts
David Giaretta2 >> (All): Mark - I keep asking that question to the CCSDS 
Secretariat
Mark Conrad >> (All): David, Just curious. Hadn't heard anything in some time.
David Giaretta2 >> (All): Quite right - I am worried it is sitting on someone's 
desk!
Mark Conrad >> (All): Have been on travel the past two weeks. At both sets of 
meetings folks were referencing the TRAC document and I told them that an ISO 
standard was coming.
JohnGarrett >> (All): And on the other hand preventing them from doing things 
like providing courses to groups they audit also will drive up costs for the 
audits.
David Giaretta2 >> (All): Yes, I do the same
Mark Conrad >> (All): I will be at the ASIST Digital Preservation Summit in two 
weeks and I have been asked specifically to give an update on the work of this 
group.
TerryLongstreth >> (All): Yes, that's what I've been saying.  I get the 
impression 17021 is for BIG organizations. We need to show how it can be safely 
relaxed to make audits affordable.
TerryLongstreth >> (All): --that was @John
Mark Conrad >> (All): If this document is silent on teaching classes, doesn't 
the initial audit committee get to determine whether or not such teaching is a 
conflict?
Mark Conrad >> (All): If we remove b) I can live with the other three bullets if 
folks see value in having them there.
David Giaretta2 >> (All): OK with me
JohnGarrett >> (All): Yes, OK with me
TerryLongstreth >> (All): Is it time to move on?
David Giaretta2 >> (All): OK - 9?
Mark Conrad >> (All): 7?
David Giaretta2 >> (All): Lead on
Mark Conrad >> (All): 7.2.1.1. b) Why was digital preservation deleted?
TerryLongstreth >> (All): 7.1 TDR should be candidate repository
Mark Conrad >> (All): Terry 7.1?
TerryLongstreth >> (All): in the phrase 'TDR-specific'
Mark Conrad >> (All): I disagree. It should say ISO-RAC or TDR- specific.
David Giaretta2 >> (All): 7.2.1.1 (d) - could be my mistake - I'd need to check 
back - but I seems sensible to leave it in
David Giaretta2 >> (All): OOps (b)
TerryLongstreth >> (All): Earlier, there had been general consensus that TDR was 
only applicable to Trusted (i.e  successfully audited) Repositories
Mark Conrad >> (All): Terry, This is referring to the requirements in the 
metrics document - the ISO-RAC. At this point in the text it does need to be TDR 
specific.
David Giaretta2 >> (All): Terry - we use the same phrase in just about all 
sections
David Giaretta2 >> (All): .... at least several of them
David Giaretta2 >> (All): Mark I see the comemnt says that we deleted 7.2.1.1 on 
the Wiki - but I cannot see why
David Giaretta2 >> (All): ..7.2.1.1 (b)
Mark Conrad >> (All): David, I don't know why it was deleted on the wiki.
Mark Conrad >> (All): 7.1.1.1 b) needs to be rephrased. It is not clear how it 
applies to TDRs or candidate TDRS
Mark Conrad >> (All): I don't really understand what it is trying to say at all.
TerryLongstreth >> (All): Still gnawing on this bone:  8March mtg, 
"BruceAmbacher >> (All): Can we globally accept (with a little caution) the 
substitute for TDR where it is not yet certified? "  ... David Giaretta >> 
(All): Bruce - I think we did discuss this and we forgot to correct things in 
the doc "
JohnGarrett >> (All): My recollection is that we eliminated 7.2.1.1 because we 
thought that 17021 already required that auditors be competent to do the audit.  
We didn't think we should also require the auditing body to have training 
requirements (that the auditing body would need to keep documented)
Mark Conrad >> (All): Terry, This is one of the places where the global replace 
is not appropriate.
David Giaretta2 >> (All): I must admit that with a minimalist hat on I can see 
taht we could just delete 7.1.1 and 7.1.1.1
Mark Conrad >> (All): These seem to be copied from 27006 and I am not sure that 
they add much value over what is in 17021.
David Giaretta2 >> (All): Exactly
JohnGarrett >> (All): Yes, I agree.  With a minimalist hat on most of this can 
be eliminated.  And simpler is better.
Mark Conrad >> (All): Somewhere we might want to say specifically that the 
members of the audit team should be thoroughly familiar with the ISO-RAC 
standard.
David Giaretta2 >> (All): That's in 7.2.1.1
TerryLongstreth >> (All): On that, I agree with Mark. 
TerryLongstreth >> (All): ....rather than TDR
Mark Conrad >> (All): So are we getting rid of 7.1?
TerryLongstreth >> (All): 7.2.1.1 replace TDR with ISO-RAC
Mark Conrad >> (All): Terry, I agree.
David Giaretta2 >> (All): Looks like it - just leave the bit saying it applies
Mark Conrad >> (All): David, 7.1. right?
David Giaretta2 >> (All): Yes - just leave "The requirements from ISO/IEC 
17021:2006, Clause 7.1 apply"
Mark Conrad >> (All): Ok by me.
TerryLongstreth >> (All): Can we move the first bullet in 7.2.1.1 up to 7.2?
TerryLongstreth >> (All): ISO-RAC knowledge isn't just a training issue, is it?
David Giaretta2 >> (All): Training "that ensures knowledge..." 
Mark Conrad >> (All): Terry, good point! 7.2.1.?
TerryLongstreth >> (All): Yes, our very own 7.2.1
David Giaretta2 >> (All): OK
Mark Conrad >> (All): So, move 7.2.1.1. up to 7.2.1. Change a) to read ISO-RAC 
XXXX instead of TDR. Put digital preservation back into b)
TerryLongstreth >> (All): some of that sentiment was deleted in earlier edit of 
7.2.1, but with 7.1 changes, we might should put it back.
Mark Conrad >> (All): Terry, I still think the deleted material is adequately 
covered in 17021.
Mark Conrad >> (All): When in doubt...
TerryLongstreth >> (All): I meant only that part related to digital repositories 
and preservation
TerryLongstreth >> (All): c/related/relevant/
Mark Conrad >> (All): I think 17021 clauses 7.2.1. through 7.2.12 covers the 
bases adequately
TerryLongstreth >> (All): Ok, I guess the new 7.2.1 gets my oar in the water
Mark Conrad >> (All): Do we want to keep f)? That is, do we want all of our 
auditors to be specifically trained in 19011?
David Giaretta2 >> (All): Seems pretty fundamental
TerryLongstreth >> (All): 19011 is not a broadly accepted auditing standard, 
even if 17021 thinks it is.  I'd prefer to keep 17021 as our sole normative 
prereq
Mark Conrad >> (All): From the ISO site: It is applicable to all organizations 
needing to conduct internal or external audits of quality and/or environmental 
management systems or to manage an audit programme. The application of ISO 19011 
to other types of audits is possible in principle provided that special 
consideration is paid to identifying the competence needed by the audit team 
members in such cases.
Mark Conrad >> (All): See especially the last sentence. I am not sure that we 
want to tie this around our necks.
TerryLongstreth >> (All): concur
TerryLongstreth >> (All): We could include it in the informative references 
section of a bibliography
Mark Conrad >> (All): 8.1. also references 19011.
Mark Conrad >> (All): It is also listed in the references at 1.9.
Mark Conrad >> (All): At 1.9 it indicates that 19011 is included in this 
standard by reference.
David Giaretta2 >> (All): Yes, it's pretty fundamnetal to ISO auditing
JohnGarrett >> (All): What do we object to that is in 19011?
Mark Conrad >> (All): Requiring all of our auditors to be trained in it?
TerryLongstreth >> (All): John are these from 27006?
David Giaretta2 >> (All): So when you say "19011 is not a broadly accepted 
auditing standard" I think taht all ISO based auditing uses it - but probably 
don't mention it on their certificates
Mark Conrad >> (All): So we all have to become 19011 experts to carry out TDR 
audits?
JohnGarrett >> (All): From section 3 most of what is in here originated in 
27006, but just as a template for what another group found not clear enough in 
17021.  We have no connection to 27006 in this current document.
TerryLongstreth >> (All): Perhaps Quality and environmental management systems, 
which are not technology based, but I'd be surprised if CMMI or any of the US 
DoD or DCAA audits include it
David Giaretta2 >> (All): Terry - but we are aiming for ISO audits so  I don't 
think we have a choice!
Mark Conrad >> (All): David,  Doesn't being silent on this issue give us greater 
flexibility?
TerryLongstreth >> (All): IF 17021 mandates it, then we need specific 
justification to leave it out.. I'd like to know what it says that's important 
to our context.
JohnGarrett >> (All): I think 17021 already references it, so we are subject to 
it regardless.   Is there anything in it that we object to?
Mark Conrad >> (All): From 17021: 7.2.5 The certification body shall have a 
process to achieve and demonstrate effective auditing, including the use of 
auditors and audit team leaders possessing generic auditing skills and 
knowledge, as well as skills and knowledge appropriate for auditing in specific 
technical areas. This process shall be defined indocumented requirements drawn 
up in accordance with the relevant guidance provided in ISO 19011.
Mark Conrad >> (All): 7.2.11 The documented monitoring procedures for auditors 
shall include a combination of on-site observation,review of audit reports and 
feedback from clients or from the market and shall be defined in 
documented requirements drawn up in accordance with the relevant guidance 
provided in ISO 19011. This monitoring shallbe designed in such a way as to 
minimize disturbance to the normal processes of certification, especially 
from the client's viewpoint.
JohnGarrett >> (All): Even if DOD and DCAA doesn't require it, I think they 
follow the same general principals and they are likely more stringent and based 
in US law and not in voluntary consensus standard.
Mark Conrad >> (All): 9.1.2 The certification body shall ensure that an audit 
plan is established for each audit to provide the basis for agreement regarding 
the conduct and scheduling of the audit activities. This audit plan shall be 
based on documented requirements of the certification body, drawn up in 
accordance with the relevant guidance provided in ISO 19011. 9.1.3 The 
certification body shall have a process for selecting and appointing the audit 
team, including the audit team leader, taking into account the competence needed 
to achieve the objectives of the audit. This process shall be based on 
documented requirements, drawn up in accordance with the relevant 
guidance provided in ISO 19011.
Mark Conrad >> (All): 9.1.9 The certification body shall have a process for 
conducting on-site audits defined in documented requirements drawn up in 
accordance with the relevant guidance provided in ISO 19011.
David Giaretta2 >> (All): Mark - your extracts give the strong message taht 
19011 are written through 17021 like the writing in a stick of rock!
Mark Conrad >> (All): It seems pretty prevalent in 17021. I do not understand 
the stick of rock reference.
David Giaretta2 >> (All): SOrry - UK specific maybe - sticks of candy with 
writing all the way through it
David Giaretta2 >> (All): ...so anywhere you break it along the long axis you 
see the writing
Mark Conrad >> (All): Given its prevalence in 17021 do we need another reference 
to 19011 here?
TerryLongstreth >> (All): I'd prefer not to repeat 17021 provisions, if we can 
avoid it.  We could make 1.9 specifically mention the pass thru of 19011 from 
17021
David Giaretta2 >> (All): Mark - I'd agree that we don't need that specific 
reference
Mark Conrad >> (All): Thanks for the explanation of the stick of rock. That was 
a new one for me.
JohnGarrett >> (All): Me also
Mark Conrad >> (All): We have been at this for almost two hours. Enough for one 
day?
TerryLongstreth >> (All): What's next week?
JohnGarrett >> (All): Good point,  I lost track of time.  Need to go.
TerryLongstreth >> (All): I'll be out for Spring break
David Giaretta2 >> (All): I've been doing the edits as we went - I'll put it on 
the Wiki.
David Giaretta2 >> (All): Bye all
Mark Conrad >> (All): So we pick up here next week or the following week

-- SimonLambert - 29 Mar 2010

Topic revision: r1 - 2010-03-29 - SimonLambert
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2018 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback