Notes from Megameeting 25th February 2008


BarbaraSierman Koninklijke Bibliotheek, Netherlands
BruceAmbacher UM
CandidaFenton HATII, Univ Glasgow
DavidGiaretta STFC
HelenTibbo UNC
JohnGarrett NASA/GSFC
KatiaThomaz INPE, Brazil
SimonLambert STFC

All the discussion at this meeting was conducted by chat, so the following transcript of the meeting (with a few typos corrected) is complete.

Helen Tibbo >> (All): Hi David. How are you?
David  >> (All): I'm fine - in Phoenix AZ at the Nuclear Waste Management conference
Helen Tibbo >> (All): Foote, Kenneth E. "To Remember and Forget: Archives, Memory, and Culture." American Archivist 53 (Summer 1990): 378-92.
Helen Tibbo >> (All): David, I just sent you the reference to the nuclear waste article.
David  >> (All): I've been trying to catch up on my actions - almost finished B2
David  >> (All): There is a CCSDS meeting the week after next i.e. week of March 10th
David  >> (All): It's in Washington
BarbaraSierman >> (All): David does that meeting affect these activities
David  >> (All): I see John is here - he suggested that anyone who is in the area could meet face to face in Washington
candida fenton >> (All): I've put a summary of some work I've done on standards up onto the wiki under Working Documents. I hope it is useful.
Helen Tibbo >> (All): What days of the week is it?
David  >> (All): The CCSDS meeting is all week - but John suggested Wednesday or Thursday for a face to face on RAC - is that right John?
BruceAmbacher >> (All): Where in the DC area is the meeting?
John Garrett >> (All): Actually I'd say probably Thursday would be best.  We'll be trying to get PAIMAS finished up on Wednesday.
David  >> (All): Marriott Courtyard Arlington Crystal City/Reagan National Airport hotel will be the primary meeting location,
David  >> (All): I should send out an email to the list with details
BruceAmbacher >> (All): I have a doctor's appointment at 7:30 but could meet after 8:45 am
David  >> (All): I was hoping to collect all the revised new template version for the meeting, to see where we are
David  >> (All): Also there was a meeting in Berlin a few months back with nestor etc but I have not seen the minutes - have you Barbara?
BarbaraSierman >> (All): No I have not seen minutes either
David  >> (All): I have the draft minutes Joy Davidson produced. The main point from my point of view was the agreement to use RAC as a clearing house for all the private discussions
David  >> (All): There has not been much activity on the Wiki in the last week or so
David  >> (All): I'm half was through B2 - should finish this morning, and candida has put up her document
Helen Tibbo >> (All): I've done some work but it's not up. I'm swamped.
David  >> (All): What is the state of the other parts -  Simon you may have the best idea?
SimonLambert >> (All): Section B is largely there I think.
David  >> (All): Do you want to look at what I put up for B2 so far? Nothing exciting but I'd be interested in peoiple's first impression
SimonLambert >> (All): What I summarised two weeks ago was: "It seems that A1.1 to A2.2, B1.*, B3.*, most of B4.*, B5.*, B6.* are done."
SimonLambert >> (All): Section C is untouched so far.
David  >> (All): I've put up B2.1-B2.6 - I'll put up the rest shortly
David  >> (All): I think it's looking better with this template
BruceAmbacher >> (All): What do we envision the standard to encompass - the entire document with all of its parts or just the audit checklist requirements with all of the text as supplemental?
BarbaraSierman >> (All): As section C is concerned with system infrastructure, I often hear people saying that they have their own standards for this part. We discussed this before, but may be we can save some effort there.
David  >> (All): We said that we reserved the option to split the docuemnt into 2 parts - (1) the metrics and (2) the discussion
David  >> (All): Perhaps Candida has a view on this split based on the work she has been doing?
John Garrett >> (All): CCSDS Standards Documents usually have a conformance section that says what parts need to be followed to comply with the standard.
David  >> (All): Yes John - and when we were doing TRAC we talked about a marking scheme.
David  >> (All): We have got a collection of the metrics we regard as "mandatory"
John Garrett >> (All): I think the document should probably include all the text.  I think we will say that you have to comply by having an acceptable answer to the checklist items.
David  >> (All): Other metrics are not "mandatory" and there are some in-between
BruceAmbacher >> (All): The audit teams had a problem getting some to read beyond the extracted requirements.
John Garrett >> (All): Is there any proposal for what the marking scheme will be? 
David  >> (All): Candida was going to make a start to integrate some of the Risk measures from DRAMBORA into this
candida fenton >> (All): Yes, I should start that this week
John Garrett >> (All): I think we need to identify which ones are mandatory and to comply you need to get a passing mark on those.
David  >> (All): I was hoing that we could have, say, the "mandatory" ones grouped together - and you had to get "full marks" for these (with some appropriate risk measure)
David  >> (All): The other non-mandatory ones could be less stringent
BruceAmbacher >> (All): I thought Mark Conrad's idea with the new template was to list only mandatory aspects of the requirements in the first section and the optional in the last section (discussion).
BarbaraSierman >> (All): May be it is best to have this discussion after the first complete draft?
John Garrett >> (All): I would also like to see the mandatory ones grouped together.  But since we are also grouping items by archive function, I didn't know how we would do that also.
David  >> (All): Barbara - yes, let's finish the new template text and then discuss the ordering later - remember in TRAC we had the spreadsheet so we could re-order the metrics in a variety of ways - it's just as a Word doc this is more difficult
BruceAmbacher >> (All): Wew could have a part A - Mandatory and a part b - Nonmandatory to each section.
John Garrett >> (All): I'm OK with just mandatory information first in the items, but will there be any items that include no mandatory portion?
David  >> (All): We clearly need a tool to track the mandatory/non-mandatory nature of each metric - and maybe also allow one to have different views on the metrics for different purposes.
BruceAmbacher >> (All): Section C may raise that as it is written with mandatory concepts but non mandatory applications as far as what specific measurres/systems/etc.
David  >> (All): I recall we looked at the metrics according to (1) Functional Model plus TDR (2) OAIS mandatory requirements and (3) current TRAC order - it was useful to see all three orderings
David  >> (All): Do people agree that we need some sort of a tool to help with the acceptance of the metrics?
BruceAmbacher >> (All): David, explain what you see the "tool" doing.
John Garrett >> (All): A tool could be useful.  Is this just the same Yes, No, Maybe for indicating each item again?
David  >> (All): I guess something which would allow one to deal with the risks . The alternative is to require a tick in each box - but what does a tick mean?
David  >> (All): This also comes back to the question of what auditors actually do
David  >> (All): ....especially since what we are talking about is guarding against future contingencies whereas things like ISO9000 and 27000 look at the here and now
John Garrett >> (All): Oh sorry are we talking about a tool for auditors or are we talking about a tool for standard writers deciding on content of the standard?
David  >> (All): John - I meant one for auditors (external and internal)
David  >> (All): John - I see the confusion - I was also meaning that when we were doing TRAC the ability to re-order things was useful in helping US to understand the interrelationships between metrics - and presumably the auditors would also benefit from that way of understanding things
Helen Tibbo >> (All): At the end of the day, auditors must evaluate the here and now and perhaps plans for the future. They can also take the past into consideration, i.e., how long a repository has been successful, but as we know with department stores this is not necessarily much of a metric.
David  >> (All): And therein lies the evaluation of risk I guess
BruceAmbacher >> (All): And thus the succession planning metric
Helen Tibbo >> (All): Yes, risk assessment is all we have for the future.
BarbaraSierman >> (All): Sorry I need to go, I'll read the minutes about next steps!
David  >> (All): Maybe that is another ordering - some metrics are essentially yes/no but others involve some evaluation of future risk
John Garrett >> (All): I see value in being able to reorder items and look at them in different ways.
Helen Tibbo >> (All): At one point I thought TRAC and DRAMBORA were competitors, but in reality, I think they are complementary and we need to blend both so as to take careof the here and now and the future to the extent we can.
David  >> (All): Helen - yes indeed. 
David  >> (All): To be fair to the TRAC work risk was  something we talked about doing but did not get around to in time
David  >> (All): ...its just that the discussion was hidden!
John Garrett >> (All): I've sort of seen the evaluation as something like ISO9000 with an iternal or external auditor looking at evidence they are provided and making  a determination for each item if they passed or not.  Perhaps we could have a few options for each.  
David  >> (All): I hope that since Candida is based in Glasgow she can help in the merge, as will the agreement in Berlin
candida fenton >> (All): I'll do what I can
David  >> (All): John - yes, but should we have a risk attached to say the succession planning?
David  >> (All): ...when everything depends on funding
BruceAmbacher >> (All): John, are you thinking of comments/grades such as exceeds requirement, meets requirement, needs improvement in . . ., fails?
BruceAmbacher >> (All): With funding an institution can only show the stability of its major funders, its endowment (if it hasone), and solid future income such as a grant.
David  >> (All): Does an "exceeds requirement" in one metric balance against "needs improvement" in another to average out at "meets requirement"? I guess that the one underlying question.
Helen Tibbo >> (All): I think this is a chain is only as strong as its weakest link.
Helen Tibbo >> (All): I don't think being above and beyond in some things and deficient in others evens out.
BruceAmbacher >> (All): Do we, or the future auditors, make that decision?  I would not support allowing an exceeds to wipe out/counteract a deficiency.
David  >> (All): I guess it was a rhetorical question!
David  >> (All): I remember that we talked about groupings - where one needed a pass mark in each group
David  >> (All): within the group one might be able to do some balancing
David  >> (All): ...not sure if that really made sense but there is something in that line of thought about mandatory and non-mandatory metrics
BruceAmbacher >> (All): Must you pass all items in a group or just pass the group overall?  And are there any  must pass items in a group?
David  >> (All): IN some ways getting the text of the metrics may be the easy part!
John Garrett >> (All): My question is how to do risk assessment and record it.  And regardless of what we do, it is an assessment at the time the audit is conducted.
David  >> (All): I was hoping we could some up with a couple of marking strategies AND DECIDE BETWEEN THEM
David  >> (All): ...oops sorry about the caps
BruceAmbacher >> (All): And it indicates the suditors think the repository is viable for some period of time into the future (3-5 years).
Helen Tibbo >> (All): That is true - we cannot actually predict the future, but then, neither do insurance companies - they just have more reliable statistics from which to work.
John Garrett >> (All): I think each item will stand on its own and exceeds requirement in one area will not wipe out deficient in another.
Helen Tibbo >> (All): I agree, John
BruceAmbacher >> (All): helen raises an interesting research question - how many, ifany, digital repositories have failed/gone out of existence?  And why.
David  >> (All): Loss of content is always a tricky question to get an answer to
Helen Tibbo >> (All): I think a confounding question is that collections have come and gone even when repositories have survived.
John Garrett >> (All): How about grades of Exceeds Requirement, Meets requirement, Meets requirement with comments, deficient, Fails Requirement?
John Garrett >> (All): Comments could be used to provide risk assessment that they will continue to meet the requirement or are approaching becoming deficient
David  >> (All): John - presumably one "Fail" means the whole audit is failed
BruceAmbacher >> (All): If a repository must meet all mandatories, does an audit stop with the first fail on a mandatory?  Or does the audit continue to the end, noting all scores?
Helen Tibbo >> (All): Folks, I need to run. Please email the info. about the March meeting. I may be able to attend. I'll work away on section A this week - I promise.
Helen Tibbo >> (All): I would say an audit should continue so the repository knows what the problems are.
David  >> (All): John - will you email details of the face to face RAC meeting please - you probably have a better feel of times and places.
John Garrett >> (All): In the same auditing sense that ISO 9000  uses, I think how audit would be handled would depend on what you wanted to get out of audit.
David  >> (All): Helen - yes I agree, especially for an internal audit
BruceAmbacher >> (All): That would be my thinking.  Then they could work on all deficiencies
Helen Tibbo >> (All): Bye
BruceAmbacher >> (All): I must go and get ready for the next meeting.
John Garrett >> (All): Usually all items are marked even if only one is needed to fail.  This will help you identify all deficiencies and correct them.
David  >> (All): I must go too. 
KatiaThomaz >> (All): I must go too. Bye. Have a nice week.
candida fenton >> (All): Bye
John Garrett >> (All): On other hand, you may pull resources for audit after you start failing and will get less information.
David  >> (All): Simon - can you cut and paste the minutes?
John Garrett >> (All): Bye for now
SimonLambert >> (All): Yes, sure.
SimonLambert >> (All): Bye for now

-- SimonLambert - 25 Feb 2008

Topic revision: r1 - 2008-02-25 - SimonLambert
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2018 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback