Notes from Megameeting 10th May 2010

Attendees

BruceAmbacher UM
JohnGarrett GSFC
MarkConrad NARA
TerryLongstreth  

Summary

The discussion focussed on the relation between the guidelines and the ISO 17021 standard, specifically whether the guidelines may admit exceptions or variations to ISO 17021, particularly in the case of the audit timetable. It was agreed that participants should come prepared to make a final decision about strict adherence to 17021 next week.

Transcript of chat

TerryLongstreth >> (All): Hi Bruce
BruceAmbacher >> (All): Hi.  As you saw we may not have enough for a "quorum"
TerryLongstreth >> (All): Looks like were short at least 3.  Simon asked us to 
capture the Chat session
Mark Conrad >> (All): Hello.
TerryLongstreth >> (All): but that requires host authority
Mark Conrad >> (All): No. I screen scape the chat all of the time.
TerryLongstreth >> (All): Ok. Can you do it for us?
Mark Conrad >> (All): Yes.
BruceAmbacher >> (All): Mark, You just volunteered
Mark Conrad >> (All): Yep.
Mark Conrad >> (All): Morning, John.
JohnGarrett >> (All): Good Morning!
BruceAmbacher >> (All): What version are we working from today and where in 9?
Mark Conrad >> (All): I think format-6-20100426 is the most recent version. We 
left off at 9.1. last week.
JohnGarrett >> (All): The last version I see on the web is the 04-26 version
RobertDowns >> (All): Good morning, all Mark Conrad >> (All): Good morning Dr. 
Downs!
BruceAmbacher >> (All): Are we starting with 9.2?  If so, I am not sure about 
David's embedded comment that the requirements are not applicable to an initial 
audit.
JohnGarrett >> (All): Yes, I also didn't understand that comment
Mark Conrad >> (All): I guess my first question goes back to 9.1. and raises the 
same question you are raising here. Are we adopting the audit schedule in 17021 
or not?
BruceAmbacher >> (All): I believe David's operating principle is that we adopt 
the audit schedule, do the site visits, then adjust the time frames based on the 
results, the vulnerabilities, the scale of the risk, etc.
BruceAmbacher >> (All): So we officially commit to the 3 year cycle but stretch 
it for most certified repositories into a 5 year cycle.
Mark Conrad >> (All): Then we cannot say that these requirements are "in 
addition" to the guidance in 17021. We have to say that they modify 17021.
BruceAmbacher >> (All): Agreed, but David (I think) senses that we should agree 
to timetable then find our TDR environment allows us to stretch timetable 
through annual interim reports.  Do you agree with my interpretation of David's 
views?
Mark Conrad >> (All): I agree that those are David's view. I think the way this 
is currently written will make no sense to an outside reader.
RobertDowns >> (All): It is a bit confusing
JohnGarrett >> (All): What would you suggest should be the wording?
BruceAmbacher >> (All): Are we discussing 9.1 or 9.2?  And waht is the 
requirements and timetable in 17021?
Mark Conrad >> (All): Bruce, We are discussing both. See my comment on 9.1. for 
the timetable.
Mark Conrad >> (All): John, I would say, "The requirements from 17021 apply with 
the following exceptions and additions."
JohnGarrett >> (All): That wording is OK with me.
RobertDowns >> (All): Me too
TerryLongstreth >> (All): Concur
Mark Conrad >> (All): Bruce?
BruceAmbacher >> (All): I think we all agree with the concept of stretching the 
timetable but what is politically possible in the ISO realm when saying our 
audit is based on 17021?
BruceAmbacher >> (All): If the wording will pass ISO muster I also prefer 
getting our timetable out in the open.
Mark Conrad >> (All): So do we put it in and see if it will pass ISO muster or 
leave it out and confuse potential clients?
BruceAmbacher >> (All): But will we have a "one size fits all" timetable or set 
of timetables depending on the results of the initial audit?
TerryLongstreth >> (All): We need to explicitly state that TDRs are management 
systems in the sense meant in 17021, or that they are different.   Those 
differences are the launch points for exceptions to 17021
Mark Conrad >> (All): Terry, I agree.
BruceAmbacher >> (All): I thin of TDRs as the latter, as more extensive than a 
management system.
JohnGarrett >> (All): My personal preference would be that I would prefer to get 
a standard out as quickly as possible.  If that means, it is a bit more 
confusing for the user, but the confusion will be easily cleared up by 
contacting us, then let's be more sure that the standard passes through the ISO 
review.
TerryLongstreth >> (All): How do users contacts us?
Mark Conrad >> (All): Who is the POC between this working group and the ISO 
folks that will review this document? Can we get that person to ask the ISO 
folks what they would think of us having exceptions to 17021?
BruceAmbacher >> (All): I am not aware that we have any POC.
TerryLongstreth >> (All): Probably David or Simon
Mark Conrad >> (All): Terry, We would have to convene the initial audit 
committee (I forgot its new name!) and then advertise that it was open for 
business.
BruceAmbacher >> (All): I was just thinking about Terry's ? how users contact 
us.  We did not include any such clause in the RAC criteria.
TerryLongstreth >> (All): TAB - TDR A(udit/ssesment/djudication) Board
Mark Conrad >> (All): Bruce, Why would it have to be in the RAC as opposed to 
this document?
BruceAmbacher >> (All): Is that a single IAC or multiple based on locations (US, 
EU, , etc.)
TerryLongstreth >> (All): Mark- because this book is a handbook, offering help 
in interpreting RAC, but no new requirements
BruceAmbacher >> (All): Mark, It should/could be in both
Mark Conrad >> (All): Terry, David calls it the Primary Accreditation Board.
BruceAmbacher >> (All): Terry, calling this interpretative may be stretching it.
Mark Conrad >> (All): I don't think we want to be modifying the RAC at this late 
date.
TerryLongstreth >> (All): Yes, but he agreed to the TDR in place of Primary, to 
make it clear what we're accrediting
BruceAmbacher >> (All): Terry, lots of articles from various "communities" 
interpreting RAC and this will mushroom as they did after OAIS.
Mark Conrad >> (All): Terry, OK. I missed that discussion.
Mark Conrad >> (All): Back to the question...Do we leave the text as is or 
modify to indicate exceptions?
JohnGarrett >> (All): When we get the standard completed in CCSDS, then an 
administrative person in CCSDS enters the completed document into the ISO 
editors queue and they send in another document requesting review of the 
standard to become an ISO standard.  Some mysterious group of ISO adminstrative 
people decide then if the Standard overlaps any other ISO groups (of which there 
are probably thousands) and sends it to them for comment. Then the draft ISO 
standard is circulated to all national members of ISO for probably a 5 month 
review period and it is made available on the ISO web site for anyone to comment 
on.  All the comments are collected by ISO and returned to us and we need to 
respond to each comment we receive whether or not we accept it.  If we make 
significant updates then we go through another round of reviews at ISO. 
Then the national bodies vote to approve the document.  If the document receives 
enough favorable votes and few enough unfavorable votes then it becomes an ISO 
Standard.
BruceAmbacher >> (All): John, Where is RAC in that process?  Any developments 
from the recent meeting?
JohnGarrett >> (All): SO there is no one person  to ask at ISO what they think 
of our exceptions.  Probably better to try to fly under the radar.
JohnGarrett >> (All): If we ask the 17021 group, we are likely to raise their 
attention to this even more and then they are more likely to object.
TerryLongstreth >> (All): I think we should make it say what we want, and deal 
with comments that may generate in ISO review
JohnGarrett >> (All): I think that is fine if we choose that route, but it is 
more likely that we will get comments and that we will then still be working on 
this a year from now.
BruceAmbacher >> (All): If we go with Terry's approach (which I favor) does that 
mean we (or an editor) have to go bak to the beginning of this handbook and 
check it for consistency?
Mark Conrad >> (All): My concern is if we say we are following 17021 and then 
the TAB starts granting exceptions to the timetable the TAB may be seen as 
making ad hoc and possibly less than impartial exceptions.
BruceAmbacher >> (All): David certainly is in JOhn's camp on this
TerryLongstreth >> (All): Do you think we can avoid getting any comments? Does 
it take longer to deal with the additional comments that might be generated by a 
document with clearly stated positions?
JohnGarrett >> (All): The RAC Working Group is a CCSDS Working Group. 
It is at the beginning of the process and it is the group where all the comments 
will return.  And in our modified requirements, we are also the Primary Audit 
Board.
BruceAmbacher >> (All): Ah, the eternal struggle between honesty and expediency!
Mark Conrad >> (All): Another alternative would be to stick with the
17021 timetable, remove any text that contradicts that timetable, and see if the 
repositories are willing to put up with the expense of near constant monitoring.
BruceAmbacher >> (All): Another scenario would start that way, then base a 
stretched schedule on repository feedback.
JohnGarrett >> (All): I don't know that we can avoid getting comments, but 
without years of experience doing audits, I don't know that I would have a good 
idea of the right timetable.  We can submit updates to the standard anytime and 
are required to review it for updates at least
every 5 years.   My vote would be to get something passed as an  ISO
Standard with our less than perfect solution of letting the Primary Audit Board 
decide and then at the next review cycle, insert the timetable we find to be 
appropriate.
BruceAmbacher >> (All): John, I understand that approach but will a 3 year 
cycle, without any hint that it is stretchable, frighten away potential TDRs?
TerryLongstreth >> (All): John - You seem to be insistent on Primary against 
TDR.  Why is that?
Mark Conrad >> (All): John, How do we let the TAB decide without running afoul 
of 17021's requirements for impartiality?
RobertDowns >> (All): Would it help to be explicit about the conditions under 
which the TAB decides?
TerryLongstreth >> (All): Should we be reworking this as a proposal for a 
prototype process, to be field tested before we go for ISO approval?
JohnGarrett >> (All): Answering an earlier question, at the RAC meeting at CCSDS 
last week the main discussions were the same strategy questions that David has 
presented here before about the best way to get an organizations set up to do 
audits and to be the Primary Audit Board and to locate funding for such 
organizations and to get initial audits started. 
BruceAmbacher >> (All): Terry, taken to the full extent would that mean a 3 year 
test with the results being TDRs say the 3 year cycle is too tight, too onerous, 
too expensive?
BruceAmbacher >> (All): John, Where is RAC in the CCSDS-ISO cycles?
JohnGarrett >> (All): Terry, I'm not sure what you mean about my insistence on 
Primary vs. TDR can you elaborate?
JohnGarrett >> (All): Bruce, the 3 year cycle might frighten repositories.  On 
the other hand the large ones already do that for ISO 9000 certification if they 
have that.
JohnGarrett >> (All): And I'm OK specifying longer schedules, but we may or may 
not get comments about that.
BruceAmbacher >> (All): Good point.  But most TDRs will be small and not part of 
a 9000 organization TerryLongstreth >> (All): John - we're still looking for 
consensus on TAB vs. PAB JohnGarrett >> (All): My personal opinion is Yes, it 
would help to have conditions under which the TAB decides to lengthen the audit 
schedule. 
However that also ties our hands in allowing longer schedules.  And someone 
needs to describe those conditions.
BruceAmbacher >> (All): All right, anyone for returning to our editing as if we 
areadopting the 17021 cycle of 3 years?
TerryLongstreth >> (All): Would that give us space (kind of a holding
action) to start proto typing the process while the book's in ISO review?
JohnGarrett >> (All): Yes
Mark Conrad >> (All): It seems to me there are several decisions that we need to 
make before we can really proceed. 1. Are we going to stick with 17021's audit 
schedule - yes or no. If yes, we modify any text that contradicts the timetable 
in 17021. If no, we proceed to question 2.
BruceAmbacher >> (All): Terry, It certainly could.  We did test audits after the 
first draft of RAC
TerryLongstreth >> (All): I think the distinction between Management Systems and 
TDRs is significant, but we won't be able to show that without gathereing 
TerryLongstreth >> (All): without some real-world interactions with candidate 
TDRs
BruceAmbacher >> (All): I would think the test audits already done could be "re
-analyzed" to get that.
Mark Conrad >> (All): 1. Are we going to stick with 17021's audit schedule - yes 
or no?
TerryLongstreth >> (All): did those test audits cleave closely to 17021?
BruceAmbacher >> (All): Yes, keep 17021 schedule until we have contrary evidence/
testimony
JohnGarrett >> (All): The reality is that the 3 year schedule 
is a pain, but that most of the review materials are created at the initial 
review or will be generated as part of the normal processes of the archive. 
Recertification audits every 3 years will not be as hard on the TDR as the 
initial audit and will not be as hard for the same auditor to do a 
recertification audit. 
Mark Conrad >> (All): 1. Are we going to stick with 17021's audit schedule - yes 
or no? Others?
JohnGarrett >> (All): OK with me to stick with the 3 year schedule if others 
want that.
TerryLongstreth >> (All): John - those are the kinds of assertions that need to 
be tested.  Also, the issue of TDR scope being greater than management system 
scope
BruceAmbacher >> (All): we have to get this done.  The current schedule is 
expedient
Mark Conrad >> (All): Terry? Robert?
RobertDowns >> (All): If we are going to stick with the 3 year schedule, perhaps 
we can make it clear how the re-certification can be conducted in an efficient 
manner.
Mark Conrad >> (All): And the surveillance audits in years 1 and 2 as well?
BruceAmbacher >> (All): Terry, I think there is enough evidence of what a TDR's 
systems are to demonstrate it is more than a mgmt system.
TerryLongstreth >> (All): Robert - I don't think we know how to do it 
efficiently if we broaden scope beyond 17021
JohnGarrett >> (All): I agree that they need to be tested.  Either we create the 
Standard first and test that it works; or we test against this draft without 
making it a Standard and only create the Standard afterwards.  I think the 
community would do better and advance more quickly if the Standard comes first.
TerryLongstreth >> (All): so 17021 --> Mgmt audits, RAC --> repository audits :: 
repository audits will be incomplete if constrained by 17021
Mark Conrad >> (All): Terry, Are you suggesting reworking this document so that 
it is not based on 17021?
BruceAmbacher >> (All): I agree with JOhn, we have to get something out there 
and let the real world experience show its deficiencies.
JohnGarrett >> (All): The RAC metrics Standard is what we measure against for a 
TDR.  17021 is primarily a process that is used, but the particular metrics need 
to be identified so you know exactly what you are testing.  Knowing that we are 
using the RAC metrics determines that we are auditing the TDR system rather than 
the security audit (in 20006) that uses the 20001 security metrics as the 
testing criteria.
BruceAmbacher >> (All): Mark, Isn't Terry's position theone we have been taking 
in each section when we expand on 17021?
BruceAmbacher >> (All): John puts this in focus.
Mark Conrad >> (All): Bruce, I am not sure what exactly Terry's position is. 
Thus the question.
TerryLongstreth >> (All): Mark - that's exactly what I think should happen, but 
in the interests of driving toward a testable document, I'd say lets focus on 
te3 17021 solution, but add an introductions making it clear that RAC covers 
more than 17021,
TerryLongstreth >> (All): and that in parallel with early tests of the audit 
process, future versions of this document may relax  17021 constraints 
JohnGarrett >> (All): OK, I agree with that.  My view of what we have been doing 
is that we start with the generic 17021 process and in this document we are 
clarifying things we thought were not clear in 17021 and we are adding and 
modifying some specific activities so they apply better to TDRs.
Mark Conrad >> (All): Terry, Thanks for the explanation. I agree. I am not sure 
why we went down the 17021 road in the first place, but we may be in too deep to 
start over, now.
TerryLongstreth >> (All): We may be able to assist in evoluition of
17021 as well
Mark Conrad >> (All): John, We can't modify 17021 and say that our requirements 
are in addition to 17021. We can add. We can clarify, but we cannot add.
JohnGarrett >> (All): Yes, we can submit suggestions to 17021 to be modified.
Mark Conrad >> (All): sorry We cannot modify.
TerryLongstreth >> (All): We can even open a formal Liaison agreement with the 
17021 committee
TerryLongstreth >> (All): if we think that RAC needs to be better aligned with 
17021
Mark Conrad >> (All): Can we make a FINAL DECISION about the relationship 
between this document and 17021 so that we can make sure that the text of this 
document is aligned with that decision and move forward?
JohnGarrett >> (All): Mark,  right.  If we want the RAC auditing to be fully 
compliant to 17021, we cannot modify anything (except to make it stricter).  If 
we don't care if we are compliant to 17021, we can still take 17021 as our basis 
and loosen some requirements, but then other groups may be more likely to 
comment if they take the time to review our draft Standard in detail.
BruceAmbacher >> (All): We are a small group to make that binding decision but I 
say yes
Mark Conrad >> (All): Bruce, Do you want to put off the decision until next week 
to give everyone a chance to (re)read 17021?
JohnGarrett >> (All): I vote that the main base of our audit Standard should be 
17021 (whether or not we are fully compliant).  There is a huge amount of 
material there that I would not want to have to recreate in our standard.
TerryLongstreth >> (All): John - I agree with your statement of alternatives.  I 
think that we must take 17021 strictly in this edition
JohnGarrett >> (All): I am OK with taking 17021 strictly and being fully 
compliant with it , if that is the consensus of this group.
BruceAmbacher >> (All): Am I correct that JOhn has embedded 17021 into one 
version of our document?
BruceAmbacher >> (All): otherwise I do not have access to 17021
TerryLongstreth >> (All): but we must make it clear that later editions of our 
document will relax the 17021 dependencies
Mark Conrad >> (All): Bruce, I can send you a copy of 17021 that David loaned to 
me.
JohnGarrett >> (All): There is a version about 10 versions back that had
17021 in boxes in each section.
BruceAmbacher >> (All): I am not sure I will get to really do much this week as 
it is finals time - exams to read, grading,
Mark Conrad >> (All): I have to leave in a few minutes. Action items for next 
week?
Mark Conrad >> (All): Read 17021?
JohnGarrett >> (All): I guess we need to make a decision if we want to be fully 
compliant to 17021 or not.
Mark Conrad >> (All): Come prepared to make a final decision about strict 
adherence to 17021?
TerryLongstreth >> (All): We still haven't addressed the Risk management issues 
that I researched for 9.1.2.  Let's carry that over to next week
BruceAmbacher >> (All): My sense will not change - I think we are committed to 
using 17021 as our model. That dictates a minimum of changes.
Mark Conrad >> (All): Terry, OK.
JohnGarrett >> (All): OK.  I think it was a good discussion this week. 
Hopefully we can make the final decision next week.
TerryLongstreth >> (All): I will check 17021 for Risk Management
Mark Conrad >> (All): Anything else?
BruceAmbacher >> (All): We are all facing the classic Jeffersonian battle 
between our hearts and our heads.
JohnGarrett >> (All): I think some of the risk management for TDR's comes from 
having to comply with the TDR metrics.
BruceAmbacher >> (All): Mark, send 17021 please
Mark Conrad >> (All): Bruce, Will do.
BruceAmbacher >> (All): ok, next week.
JohnGarrett >> (All): OK, bye
RobertDowns >> (All): Bye
TerryLongstreth >> (All): John - but if we're going with strict 17021, we can't  
add it without reference back to 17021
TerryLongstreth >> (All): by all
Mark Conrad >> (All): Anything else, to add? I am about to collect the chat to 
send to Simon.

-- SimonLambert - 12 May 2010

Topic revision: r1 - 2010-05-12 - SimonLambert
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2018 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback