Notes from Megameeting 2nd February 2009


BruceAmbacher UM
DavidGiaretta STFC
HelenTibbo UNC
JohnGarrett GSFC
KatiaThomaz INPE
MarieWaltz Center for Research Libraries
MarkConrad NARA
RobertDowns CIESIN, Columbia University
SimonLambert STFC

Today's discussion was about arrangements for the face-to-face meeting February 11-13. The meeting will take place from 9 to 6 on all three days.

There were different opinions about how to proceed with the requirements for auditors (sometimes referred to as "guidelines"). It was agreed to clarify it by filling it in using some of the completed metrics from the working document.


  • SimonLambert and DavidGiaretta to fill in some illustrative examples in the requirements for auditors.
  • DavidGiaretta to put a snapshot of the main working document into Word based on the current state of the wiki.
  • DavidGiaretta to circulate copy of ISO/IEC 17021 in advance of the meeting.

Mark Conrad >> (All): David, Has the list of attendees been finalized for next 
David Giaretta >> (All): Mark : just putting things on the Wiki
David Giaretta >> (All): SO please check the wiki page
Mark Conrad >> (All): We discussed the agenda a little bit last week.
Marie Waltz >> (All): Hi David, I will be there all three days. Marie
Mark Conrad >> (All): We were concerned that there is too much time alloted to 
things other than working on the metrics.
Mark Conrad >> (All): Where are you posting these things on the wiki?
Mark Conrad >> (All): The agenda
KatiaThomaz >> (All): and also the material to consult
David Giaretta >> (All): material other than what is on the wiki?
David Giaretta >> (All): The agenda is at
Marie Waltz >> (All): Would it be useful to have a copy of the checklist as it 
is now with all the changes we have done?
Mark Conrad >> (All): As opposed to what is on the wiki?
BruceAmbacher >> (All): Marie, do you mean just the checklist or the entire 
Marie Waltz >> (All): Yes, without all the strike outs, etc.
Marie Waltz >> (All): Just the checklist
David Giaretta >> (All): I can create a Word document without and without 
Mark Conrad >> (All): What are you calling the checklist?
Marie Waltz >> (All): Sections, A. B, C
JohnGarrett >> (All): Am I right that we won't all have Internet at the meeting. 
 A paper copy of the checklist may then be useful at the meeting.
BruceAmbacher >> (All): The checklist is just the stated requirements - without 
any supporting text.
Marie Waltz >> (All): I see, I think with supporting text.
Mark Conrad >> (All): John, We will probably only have one Internet connection 
in the meeting room.
David Giaretta >> (All): I guess we cannot bring our own wireless hub?
Mark Conrad >> (All): Wireless is not allowed in the building.
David Giaretta >> (All): Wired hub?
RobertDowns >> (All): It would be useful if the wiki could be displayed on a 
large screen and edited during the meeting.
Mark Conrad >> (All): Robert, we will have that capability.
BruceAmbacher >> (All): The room is a mini theater
RobertDowns >> (All): Great! Thanks for arranging it Mark
David Giaretta >> (All): What about the agenda? ANy thing we should change?
Mark Conrad >> (All): If someone needs to urgently check their e-mail, I can 
escort them to the ERA Reseach lab to connect.
Marie Waltz >> (All): Can we have a tour? I've never been to NARA
KatiaThomaz >> (All): at the end of the meeting what should we give as a result?
David Giaretta >> (All): Is there any outstanding organisational thing we need 
to do?
Mark Conrad >> (All): In terms of the agenda, What is meant by "Finalisation of 
document drafts"?
Marie Waltz >> (All): What is the goal of the meeting? What do we have to get 
David Giaretta >> (All): I meant documents which are close to final so we can 
have confidence that we can be ready to submnit to ISO in the Spring
Mark Conrad >> (All): I will need a finalized list of the names of attendees no 
later than Monday, 9 February.
Mark Conrad >> (All): It says drafts - plural. What document other than the 
metrics are you referring to?
David Giaretta >> (All): The 2nd is the
David Giaretta >> (All): We need that 2nd document - mostly boilerplate - to set 
up the process for audit and certification
BruceAmbacher >> (All): This group has not had formal discussion about the 
auditor document.  That was discussed/developed at the CCSDS meeting in Berlin.  
Is it ripe for our consideration next week?
David Giaretta >> (All): If you look at it it is mostly boilerplate as far as I 
can see. Simon may have a view on this
SimonLambert >> (All): There's a version on the wiki though it still needs 
customising for audit/certification
David Giaretta >> (All): It is also possible that it will provide a better place 
for some of the more general text we have.
SimonLambert >> (All): The obvious words have been changed but I think it will 
need examining more closely - some areas are highlighted I think.
RobertDowns >> (All): It also might be useful to identify all of the sections 
that are needed for each document so that nothing is left out.
Mark Conrad >> (All): Given that most of us have not reviewed this document, and 
given that we still have a long way to go with the metrics. I would say that we 
should concentrate on the metrics during the precious little face to face time 
we have.
SimonLambert >> (All): Would it be possible to work in parallel?
David Giaretta >> (All): In other words we should talk about making the metrics 
document fairly sparse - mostly a bullet list - like ISO27001. The Guidelines 
would more naturally have the more general text.
Marie Waltz >> (All): Do we need to submit the requirements to ISO at the same 
time or can they come later?
Marie Waltz >> (All): Sorry, the audit requirements
David Giaretta >> (All): Mark - yes, I think the agenda does have that, and if 
we cannot agree on the metrics then we cannot make progress on the guidelines
David Giaretta >> (All): Marie as I understand it the guildelines could be later 
but it would be nice and consistent for them to come together
Mark Conrad >> (All): The agenda says finalize both drafts.
David Giaretta >> (All): ...and as I say, it may be better to transfer some text 
between the two.
BruceAmbacher >> (All): David, are you stripping text from the metrics and 
moving it to the audit document or duplicating?  I am concerned that repository 
staff will not look at the audit as much as at the metrics.  That was the logic 
for doing more than the checklist in the first place.
David Giaretta >> (All): Mark: I guess the agenda's an expression of hope rather 
than certainty
KatiaThomaz >> (All): simon, it is importat to transfer the all notes from the 
old document to the new document
David Giaretta >> (All): Bruce - that is what we should discuss - at the moment 
the metrics doc is not like ISO27001 - too much text.
Mark Conrad >> (All): Given the need to get a lot done in a very short time, it 
would be nice to have a fairly good idea of what we are planning to do before we 
arrive so we can hit the ground running.
David Giaretta >> (All): ..On the other hand the gudelines has appendices which 
might be a natural home for the more extensive text.
BruceAmbacher >> (All): We really have 3 things - metrics checklist, supporting 
text, audit guidelines.  If metrics and audit are sparse the supporting text can 
be designed to supplement both.
JohnGarrett >> (All): I didn't see that the auditor's guidelines had much 
discussion of the base document being audited.  I saw it as much more just a 
procedures document.
David Giaretta >> (All): Mark - that's why I suggested a brief talk about 
distribution of material on Wed am while we had some people missing.
JohnGarrett >> (All): I think that would be useful.
Mark Conrad >> (All): Not sure what you mean by distribution of material.
JohnGarrett >> (All): And I think it will provide some good direction for those 
that can meet again in April
BruceAmbacher >> (All): John, David, I am concerned about a disconnect between 
the weekly webchat and the CCSDS crew.  There is very little overlap compared to 
the OAIS development effort.
SimonLambert >> (All): The appendices of the guidelines for auditors cover 
things like estimating complexity and audit time - but that's just taken from 
ISO 27006 and we might want to put something different.
David Giaretta >> (All): Maybe I was wrong - but that is something I thought we 
could take a look at on Wed am looking at a screen
JohnGarrett >> (All): Yes, I agree we have to bring them more into agreement 
since this is a CCSDS and ISO document we are working on.  Perhaps we can 
megameeting in others who can't physically attend the CCSDS meeting
SimonLambert >> (All): The main body of the guidelines deals with things like 
competence of the auditors, the meaning of certification, the availability of 
documentation - fairly generic.
David Giaretta >> (All): Yes - mostly boilerplate
Mark Conrad >> (All): Let me start with a simpler (I hope!) question. What time 
are you planning to begin meeting on the 11th?
David Giaretta >> (All): I'll ne there at 9am if its open
David Giaretta >> (All): I'll be flying over Monday
Mark Conrad >> (All): Ok. 9 am it is!
Marie Waltz >> (All): And it ends on 5 on Friday?
David Giaretta >> (All): Hope so
Mark Conrad >> (All): We have the room until 6.
JohnGarrett >> (All): Yes, and much of it refers to an even more generic ISO 
Auditors Guidelines.  Most of the tailored content will need to be in the tables 
at the end that list qualifications of auditors and "amount of effort" types of 
BruceAmbacher >> (All): May  I suggest the first task next week is to review the 
checklist - are we comfortable with the logic, the flow, the development from 
item to item.  Is there duplication, missing parts?  Second, we focus on the 
supporting text that has not been addressed yet.  Third, we examine the audit 
document and allocate text to its best place.
SimonLambert >> (All): John - are you saying there is no place in the guidelines 
for explanatory text about how to apply the metrics?
Mark Conrad >> (All): Bruce, I like your suggestion. I don't think we will make 
it as far as your third bullet.
JohnGarrett >> (All): I didn't really see much in there about that.  
David Giaretta >> (All): Bruce - the only issue is whether we will have everyone 
together Wed am
Helen Tibbo >> (All): I will not be there Wed AM
JohnGarrett >> (All): I'm not sure I would approach it that way either.  Then 
implementers would need to read this extra document that is mostly not of 
interest to them.
BruceAmbacher >> (All): This discussion is similar to that on the TRAC document. 
We opted to include explanatory text and make the checklist Appendix A for those 
who wanted to start there.
David Giaretta >> (All): John - I was thinking about "Audit Review Guidance" 
KatiaThomaz >> (All): IŽll be there afternoon
RobertDowns >> (All): I expect to arrive early in the afternoon.
Marie Waltz >> (All): I'll be there Wed morning
David Giaretta >> (All): Bruce - in that case it would be fine to have very 
little in the guidelines doc 
David Giaretta >> (All): Bruce - Mark - I did not mean this to be a biggy - just 
something we needed to talk about.
JohnGarrett >> (All): I think audit review guidance is OK for a column if it 
applies mostly to auditors and not so much to implementers.
David Giaretta >> (All): That's the purpose of the doc
JohnGarrett >> (All): Agree
David Giaretta >> (All): John - maybe that is a good way of looking at it - we 
can duplicate the evidence in to that table
JohnGarrett >> (All): Yes, that may be the right way to go.
David Giaretta >> (All): OK - great - but we should just touch base next week 
about it.
JohnGarrett >> (All): Yes
BruceAmbacher >> (All): I remain more committed to quality than to rushing to a 
JohnGarrett >> (All): I thought Wed AM was a good time for that since several 
people weren't in yet.  Otherwise we can touch base on it for an hour at the end 
on Friday.
Mark Conrad >> (All): Bruce, I am with you on that.
David Giaretta >> (All): As long as the best is not the enemy of the good
David Giaretta >> (All): ... or something like that
BruceAmbacher >> (All): John, could you take one metric and fill out the 
corresponding auditor's table, placing the metric, evidence, supporting text, 
etc so we can envision how the auditor's workbook will look like?
Mark Conrad >> (All): If this requirements for auditors document is supposed to 
be a product of this working group, I would like to know some more about it.
JohnGarrett >> (All): Right and it is easier to get acceptance if we get 
something out that is good and work on perfecting it after others start 
accepting it.
Mark Conrad >> (All): We currently have documents that we cannot agree among 
ourselves what they mean. I would not classify them as "good".
JohnGarrett >> (All): Mark, it is on the Wiki and I thought you were at the CCSDS
meetings where we discussed it.  Some times though I lose track of who is 
there at what times.
Mark Conrad >> (All): I was not in Berlin.
David Giaretta >> (All): So most of the agenda would be on the metrics and 
support text with a short time devoted to test audits and guidelines
David Giaretta >> (All): Mark - I guess that's why I thought there should be 
some time to discuss it on the agenda
BruceAmbacher >> (All): John, the auditors' requirements do not have any of our 
metrics, let alone all fleshed out
Mark Conrad >> (All): David, I don't understand how we put requirements for 
auditors in place before we know what needs to be audited (i.e., what the 
metrics are.)
JohnGarrett >> (All): Right that is the current status.  I think that is why 
they can be discussed very briefly next week.
BruceAmbacher >> (All): What goes in evidence, what goes in auditor's review 
David Giaretta >> (All): Bruce - the way I think I understand it we will 
transcribe the metrics over to the gudelines doc - when we have agreed the 
emtrics and add in the extra info  which covers mostly where the evidence comes 
David Giaretta >> (All): Mark - yes, we put in the metrics after we finalise 
them, but I thought we could get an understanding of how the docs fit together.
David Giaretta >> (All): Again, as far as I understand it, we need the 
guidelines doc otherwise all we will have is the metrics.
Mark Conrad >> (All): A mapping between the two documents would be useful to 
BruceAmbacher >> (All): The auditor's handbook has to be more than a simple 
sorting of the document into set columns.  What else is required to "measure" 
the criteria and assign points
David Giaretta >> (All): We need the guidelines doc so that we can set up the 
actual audit process.
Mark Conrad >> (All): Most of us have not reviewed the requirements for auditors 
document so we will need to get up to speed to be able to contribute to the 
mapping of the two documents.
David Giaretta >> (All): Maybe I should write some more on the wiki of how the 2 
fit together and where there is necessary duplication and where there might be 
some transfer of text (if any)
Marie Waltz >> (All): Please
RobertDowns >> (All): It might be useful to have a copy of ISO/IEC 17021 
available during the meeting as a reference.
Mark Conrad >> (All): It would be very helpful to have that well in advance of 
the face to face so we can come prepared to the meeting.
David Giaretta >> (All): The 2 docs are complementary, but with some duplication 
(the bare metrics) so they can be joined 
BruceAmbacher >> (All): Let me repeat my suggestion to actually take a completed 
metric and show us how it would look in the auditor's handbook.  I am especially 
curious about what goes in "Auditor review guidance."
Mark Conrad >> (All): I second that.
Marie Waltz >> (All): I third
Helen Tibbo >> (All): Yes, that would be very instructive
David Giaretta >> (All): Robert - yes, I'll have some - I can probably email 
copies to individuals coming to the meeting since I don't really want to carry 
lots of copies with me!
David Giaretta >> (All): OK - I guess Simon and I can put somthing in the tables
Mark Conrad >> (All): We completed C.3.4. last week. Could you use that metric?
KatiaThomaz >> (All): i agree
BruceAmbacher >> (All): I want to gauge the work effort after the metrics, etc. 
are exported to the handbook.
Marie Waltz >> (All): That seems very useful to know too
Mark Conrad >> (All): Bruce, What is the handbook?
RobertDowns >> (All): Thanks, David, I am looking forward to receiving a copy of 
ISO/IEC 17021 to review before the meeting.
David Giaretta >> (All): OK - it would certainly be useful to all if Simon and I 
put together as complete a guidelines doc as we can before the meeting
BruceAmbacher >> (All): Mark - the Requirements for Auditors
Mark Conrad >> (All): Thanks, Bruce.
David Giaretta >> (All): Sorry - I keep using "guidelines"
BruceAmbacher >> (All): David, I am only looking for enough that I envision what 
a total document would look like and the effort it will require.
David Giaretta >> (All): Yes - we will not go overboard on it
Mark Conrad >> (All): Maybe load a couple "finalized" metrics from Sections B 
and C.
David Giaretta >> (All): It's mostly some cutting and pasting
BruceAmbacher >> (All): For example, there may be three types of evidence.  Is 
the auditor instructed to look for just one or must all three be present or can 
other parallel evidence be accepted
Mark Conrad >> (All): Bruce, The evidence section of the metrics is not 
mandatory. Will this have an impact?
BruceAmbacher >> (All): Am I right in assuming that is the type of information 
that would be in the audit review guidance column?
David Giaretta >> (All): My impression is that for other standards the docs are 
not sooo prescriptive
David Giaretta >> (All): The key lies in the auditor qualifications and the 
first audit committee
Mark Conrad >> (All): Are we meeting next Monday or will too many folks be in 
transit to the face-to-face?
David Giaretta >> (All): I'll be in transit
BruceAmbacher >> (All): But to have consistency there must be measures for each 
metric - right?
KatiaThomaz >> (All): please, what will be the schedule for the meeting 
(begin/end in the morning; begin/end in the afternoon)?
BruceAmbacher >> (All): I will be on a panel with Helen at the iconference
David Giaretta >> (All): WHile I remember - any outstanding organisational 
Mark Conrad >> (All): So I guess we will not be meeting next week.
David Giaretta >> (All): Also any further changes to the agenda - I'll just add 
start and end times
Mark Conrad >> (All): Katia, We will begin meeting on Wednesday February 11th at 
9 am. We have the room until 6 pm.
Mark Conrad >> (All): I do not know how long we will take for lunch?
KatiaThomaz >> (All): by th way, where will we take a lunch?
Mark Conrad >> (All): On Thursday and Friday do we want to start at 9 am as 
BruceAmbacher >> (All): Mark, does cafeteria still stop serving lunch at 2:00?
Mark Conrad >> (All): We can take lunch in the cafeteria in the Archives 2 
Mark Conrad >> (All): Bruce, yes.
KatiaThomaz >> (All): thanks
David Giaretta >> (All): 1 hour??
David Giaretta >> (All): We also need some organising for dinner.
KatiaThomaz >> (All): from 1 pm to 2 pm?
Mark Conrad >> (All): We will have the room all three days until 6 pm.
BruceAmbacher >> (All): I will not be available for dinner Wednesday.
Helen Tibbo >> (All): I will be available for dinner each day
Mark Conrad >> (All): In terms of dinner, there is a restaurant in the hotel and 
there is one restaurant within walking distance of the hotel (TGIFridays) All 
other restaurants will require securing transportation.
David Giaretta >> (All): We can add details to the wiki 
David Giaretta >> (All): My machine is very slow - my backup may have something 
to do with it!
Mark Conrad >> (All): So are we saying that we will be meeting from 9 - 6 all 
three days?
BruceAmbacher >> (All): See you all on the 11th.
JohnGarrett >> (All): See you then
KatiaThomaz >> (All): and stop for lunch from 1 to 2?
Mark Conrad >> (All): I will need to escort everyone from the lobby to the 
meeting room. I sent out the telephone number for the room. Be sure to bring 
that with you.
KatiaThomaz >> (All): Ok. IŽll try to see you  at 2 pm on the 11th
JohnGarrett >> (All): Yes, we will plan to stop for lunch at 1 to 2, but we will 
be flexible with the discussions going on and may break a bit earlier if it 
seems appropriate
David Giaretta >> (All): I've put some additional info on the wiki
Mark Conrad >> (All): We could stop for lunch from 1 until 2. The food selection 
will be somewhat diminished if we wait until 1 to eat.
David Giaretta >> (All): See you all next week - please update the wiki agenda 
Mark Conrad >> (All): What are we supposed to update on the wiki agenda?
David Giaretta >> (All): Any ideas or concerns and times of lunch and dinner
Mark Conrad >> (All): Ok. See you next week.

-- SimonLambert - 02 Feb 2009

Topic revision: r1 - 2009-02-02 - SimonLambert
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2018 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback