Notes from Megameeting 4th January 2010

Attendees

MarkConrad NARA
TerryLongstreth  
HelenTibbo  
DavidGiaretta  
BruceAmbacher  
JohnGarrett  

Summary Catch up meeting and introduction to the CCSDS format of the document

Actions

  • ACTION: DG: put stripped version of doc on wiki
  • Action: All to complete new call time poll
  • Action: all to read stripped version and comment on its coherence, comprehension
  • ACTION: ALL: propose summary text to replace the boxes?

Transcript of chat

TerryLongstreth >> (All): Financial Audit/or described in http://www.aicpa.org/Professional+Resources/Accounting+and+Auditing/Audit+and+Attest+Standards/Authoritative+Standards+and+Related+Guidance+for+Non-Issuers/auditing_standards.htm
David Giaretta2 >> (All): Hi Terry
TerryLongstreth >> (All): Good Morning.   I've been doing reseach about audits at the AICP websiet
David Giaretta2 >> (All): I take it that these are the operating docuemnts rather than the standards
TerryLongstreth >> (All): I think the thing that worried me most about our book was the issue of site visit.  The AICPA stuff makes it clearer for me.
David Giaretta2 >> (All): Which document talks about that?
TerryLongstreth >> (All): They call them standards.  They are the basis for certification of Certified Professional Accountants in the US
David Giaretta2 >> (All): Where is the reference to site visits?
BruceAmbacher >> (All): Hello all
David Giaretta2 >> (All): I guess it's at a different level to ISO standards. Is there any mention of the ISO approach e.g. ISO 19011?
David Giaretta2 >> (All): Hi Bruce
David Giaretta2 >> (All): Happy New Year
TerryLongstreth >> (All): Nothing speaks direclly, but analogically, there are several.  See   AU-00331 - Inventories, and AU-0901-Public Warehouses—Controls and AuditingProcedures for Goods Held
Helen Tibbo >> (All): Happy New Year to All!
David Giaretta2 >> (All): Hi Helen
David Giaretta2 >> (All): I rather belatedly put the CCSDS format of the doc on the WIki
TerryLongstreth >> (All): No mention of ISO - many of these were codified in the 70's, but that doesn't mean AICPA is unaware, just that I"m not a CPA
David Giaretta2 >> (All): Any thoughts about the topics we were discussing before the break?
Helen Tibbo >> (All): David, can we have a brief review? I have not been at the sessions for a while and many of us may have Holiday Head.
David Giaretta2 >> (All): Sure - me too. I think we were talking about  who would be likely to be subject to an audit
BruceAmbacher >> (All): I have taken a quick read of the document David posted and have some edit type issues I will share later.  My big concern is the audit schedule laid out - stage 1, stage 2, certification, surveillance year 1 and year 2, reaudit in year 3.  That seems too much of a burden on the repositories.
David Giaretta2 >> (All): ...this lead to discussions about whether or not we should write in the standard explicitly how to keep costs down or whether to allow some flexibility via the initial audit committee
David Giaretta2 >> (All): Bruce - yes - the schedule was this sort of point
David Giaretta2 >> (All): My thought was that repositories would not pay for the audit - their funders would
BruceAmbacher >> (All): TDRs will have to devote both management and staff time to preparing for each phase, to the detriment of normal operations.  Quite an overhead burden, time commitment and possibly lead to non participation.
David Giaretta2 >> (All): But if we remove ourselves from the ISO way of auditing then we lose our significance
David Giaretta2 >> (All): We can achieve the same ends, if we need to, by allowing some flexibility through the initial audit committee
David Giaretta2 >> (All): But we should not do that lightly
BruceAmbacher >> (All): External funding certainly is not established in the US.  Since TDR status would be voluntary, its positive value must be established.  Would there be a way to close down a failed repository?
BruceAmbacher >> (All): Could a negative report become the basis for additional funding?
David Giaretta2 >> (All): All repositories have some "external" funding surely - unless they live on the interest on their investments 
David Giaretta2 >> (All): ....I'm not sure we would just fail someone - just have a big list of areas where improvement is needed
Helen Tibbo >> (All): It seems to me that a negative report (and I suspect they will all be negative in the beginning) could be cast as a type of consulting report.
TerryLongstreth >> (All): The one's that are evolving online today live on ad revenue, subscription and item fees
Helen Tibbo >> (All): Companies pay bunches for consulting so this might not seem so onerous.
David Giaretta2 >> (All): Helen - yes that is how we need to start - prove we can provide a useful service to the funders
Helen Tibbo >> (All): However, to pay a lot in time and someone's money and then to get a less than stellar report is unlikely to sell in the US without some mandate driving it such as state governments requiring it of their repositories (unlikely to happen).
BruceAmbacher >> (All): Can we establish interim or provisional certification?  Or is it best to hold off until all deficiencies are corrected?  I support the latter.
David Giaretta2 >> (All): Terry - not many of those offer loing term preservation but it is possile such services might evolve
BruceAmbacher >> (All): And what do we do with commercial services such as OCLC's Digital Archive or Iron Mountain's service?
David Giaretta2 >> (All): Helen - but if the report just says everything is fine then it is not much use to funders - they need to know where improvements should be mase
Helen Tibbo >> (All): Well, we could give them some sort of score as restaurants are scored for cleanliness but I am guessing many repositories would rather see this as consultancy until they receive certification. That way there won't be any public embarrassment of not being certified.
TerryLongstreth >> (All): It must evolve if they're to stay in business (of course, e service companies fail all the time)
Helen Tibbo >> (All): Of course, the report would be useless unless it contains detailed discussion of the shortcomings (and the strengths).
BruceAmbacher >> (All): Helen, just as universities "advertise" job talks as research seminars.
David Giaretta2 >> (All): I guess the question in my mind is about how we start the process rolling.
David Giaretta2 >> (All): ...and what wording we need in the standard
David Giaretta2 >> (All): Helen - yes we must be able to show that we provide useful information about shortcomings and ways to improve
David Giaretta2 >> (All): The proposal we talked about before Xmas was to contact funders to see if they could fund some initial test audits so we could prove our worth - before we have a full standard
BruceAmbacher >> (All): I do not remember seeing any language in the latest draft about the audit being conducted in secrecy.  If not, how then to keep shortcomings (non-issuance of a certificate) from the community?  Would the auditors have to post lists of which repositories they audited?
TerryLongstreth >> (All): Secrecy shouldn't be a problem with corporate entities (just call the discussions proprietary/commercial-in-confidence
BruceAmbacher >> (All): David, is that a chicken and egg issue?  Would potential funders need to see the standards before agreeing?  Or would a draft audit process be sufficient?
TerryLongstreth >> (All): with government entities, it might be harder
David Giaretta2 >> (All): Section 8 discusses confidentiality
JohnGarrett >> (All): I suspect that maybe not for the initial audits, but I suspect that pretty quickly organizations will reach levels where they pass audits they undergo.
BruceAmbacher >> (All): Would the fact that CIA or MI5 were being audited be public?
David Giaretta2 >> (All): Bruce - section 8.5 discusses this - basically an agreement between the audit body and the client
TerryLongstreth >> (All): @bruce - in some circles
JohnGarrett >> (All): Most will do secret, internal audits of themselves until they feel they are ready to pass and will then have an external audit done.  Much the same process they use for ISO 9000 certification now.
Helen Tibbo >> (All): David, where are the documents you placed on the wiki?
David Giaretta2 >> (All): The Word doc is http://wiki.digitalrepositoryauditandcertification.org/pub/Main/ReqtsForAuditors/AuditorGuidelines-CCSDS-format-1.doc
Helen Tibbo >> (All): thanks
David Giaretta2 >> (All): ...it is what we had but in CCSDS format - and adding in the changes we made on the Wiki about the issues
BruceAmbacher >> (All): John, I agree.  Only a fool would start an audit blind, not having pre-tested itself
David Giaretta2 >> (All): Yes - I agree
BruceAmbacher >> (All): David, I sensed a real disconnect between sections 1-6 from 7.2 forward.  style, content,
David Giaretta2 >> (All): Bbruce - about the "chicken and egg issue" - yes it's a problem but we can either wait until everything is final or else we can try to do some audits - and perhaps put in our own RIDS before it is final
BruceAmbacher >> (All): David, the place to start may be the organizations that cooperated in the earlier rounds of test audits.  And possibly some large repositories known to have good practices.
David Giaretta2 >> (All): Bruce - I hope I did not mess things up. Should have just been a reformatting with a bit of top and tail
JohnGarrett >> (All): Obviously, we believe there is a huge need for pushing people to use the at least the metrics document if we think that most or all organizations would fail an audit at the present time.  If that is the case, we need to do something to ensure our heritage countries, organization, etc. materials are properly protected.
David Giaretta2 >> (All): Bruce - yes, but we'd have to get some funding from somewhere because there would be some international travel
BruceAmbacher >> (All): The style difference comes in the long bolded clause/condition followed by the supporting text.  That does not appear in sections 1-6.
David Giaretta2 >> (All): ... I mention funders because it seems to me they are more likely to be interested but we could ask anyone
David Giaretta2 >> (All): Bruce - do you mean the text in the boxes?
BruceAmbacher >> (All): section 4.1 uses language/terms inconsistent with our language.  I do not see TDR, producer, consumer, designated user.
JohnGarrett >> (All): There may not need to be too much international travel.  At least not once this is established.  I would expect each country to develop an auditing organizations within their own countries just to keep costs down.  Of course this wouldn't be until enough auditors get experience.
David Giaretta2 >> (All): Bruce - the text in the boxes is just copied from ISO 17021
David Giaretta2 >> (All): In the early sections all we are saying is that we agree with 17021. I the later sections where we get more specific we have some specific things to say - hence more text outside of the boxes
David Giaretta2 >> (All): ...the final version will have the boxes deleted - they are only there so we do not have to keep turning to ISO 17021
TerryLongstreth >> (All): I have to cut out early guys.   I'll review the chat session later
David Giaretta2 >> (All): Bye Terry
BruceAmbacher >> (All): David, we need to put our terms in after their terms.  Look at 4.2.1  I had ro struggle to see our naming conventions 
David Giaretta2 >> (All): Are we looking at the same doc? 4.2.1 is about impartiality
David Giaretta2 >> (All): .. of the audit process
BruceAmbacher >> (All): Sorry, I meant 4.1.2
David Giaretta2 >> (All): But that is just about people having confidence in the audit process rather than the repositories
BruceAmbacher >> (All): Is 4.1.2a -TDR?  Is 4.1.2b the consumer, designated use?  I did not see where producer fit in
David Giaretta2 >> (All): I guess the client must be whoever commissions the audit
BruceAmbacher >> (All): I think the language should incorporate, even if it is parentheses, the language of OAIS and TDR.
David Giaretta2 >> (All): But that text will not be in the final version of the doc. We can add text in the Introduction - section 1 or perhaps section 2.
BruceAmbacher >> (All): David, explain the mechanics of this part to me.  I assumed all of the text would be in this CCSDS Red Book draft and thus in the final companion ISO standard.
David Giaretta2 >> (All): ...so section 1.2 SCOPe or 1.3 APPLICABILItY - these are CCSDS required sections which I created
David Giaretta2 >> (All): Bruce - John inserted the text in the boxes to make our life easier and we said we would take it out in the final version.
Mark Conrad >> (All): Under 4. Principles the only text would be "The principles from ISO/IEC 17021:2006, Clause 4 apply."?
David Giaretta2 >> (All): Mark - yes 
Mark Conrad >> (All): Does that help Bruce?
David Giaretta2 >> (All): We were following the precedent of other ISO docs
David Giaretta2 >> (All): ..which inherited from 17021 etc
BruceAmbacher >> (All): It explains the mechanics.  I think I could see wider understanding and acceptance if the full text were there but I guess that is some copyright type isse.
David Giaretta2 >> (All): Exactly
David Giaretta2 >> (All): ISO wants to sell its standards
David Giaretta2 >> (All): but things produced by CCSDS are free from the CCSDS site
BruceAmbacher >> (All): So, full comprehension of the TDR process requires the purchase of the TDR standard, the TDR audit standard, and the ISO audit standard.
Mark Conrad >> (All): Which also references other standards
BruceAmbacher >> (All): with the first 2 available free as CCSDS redbooks
David Giaretta2 >> (All): Our two documents would be free from CCSDS but any pure ISO standards would cost money
David Giaretta2 >> (All): So we would expect repositories to be able to use the metrics doc stand alone. But they would not need the Requirements on bodies... doc
BruceAmbacher >> (All): Does our TDR audit document provide repositories with enough information that they could get y without purchasing the ISO audit standard?
David Giaretta2 >> (All): ....however they would probably need the comfort of knowing we follow the ISO audit tradition/methodology
David Giaretta2 >> (All): Bruce - yes the mETRICS DOC SHOULD BE ok STAND ALONE (WITH oais MAYBE - ALSO FREE)
David Giaretta2 >> (All): OOPS - sorry about the upper case
JohnGarrett >> (All): Remember the auditor guidelines is in the final respects for the auditor's organization not directly applicable to the archives and producer organizations.  Most material needed by archives and producers will be in contractual materials they have with the auditiing organization. 
David Giaretta2 >> (All): John - yes - and we would need some funding to help set up those contract templates
JohnGarrett >> (All): Yes
BruceAmbacher >> (All): John, yes but ... repositories need to be able to know the criteria they will be judged on.  That is the TDR standard.
David Giaretta2 >> (All):  - we as the initial audit committee that is
Mark Conrad >> (All): Except for the costs imposed upon the repository by the audit standards.
JohnGarrett >> (All): Yes that is the TDR metrics standard and it will be free from CCSDS
Mark Conrad >> (All): John, I was responding to your earlier post - Remember the auditor guidelines is in the final respects for the auditor's organization not directly applicable to the archives and producer organizations.  Most material needed by archives and producers will be in contractual materials they have with the auditiing organization. 
Mark Conrad >> (All): The audit standards have a very real cost impact on the repositories.
BruceAmbacher >> (All): How does ISO label our TDR audit document?  As a separate standard or as an extension of the existing audit standard?
David Giaretta2 >> (All): Bruce - it will have its own ISO standard number - just as OAIS is ISO 14721
JohnGarrett >> (All): Hi Mark, sorry out of sync here, my response was to Bruce regarding costs of obtaining standards.
BruceAmbacher >> (All): But OAIS is unique, not an extension/adaptation of an extisting standard.
BruceAmbacher >> (All): overall, a minor issue I know.
David Giaretta2 >> (All): Bruce - yes - my point was taht it would have its own number and we would advertise it as being an ISO audit standard in the full ISO audit tradition - the current doc thyat is
JohnGarrett >> (All): Mark, I agree costs imposed by auditor guidelines standard will be passed along to the audited archives and producers.  But the actual costs will be part of the contract between them.
David Giaretta2 >> (All): Onwards and upwards!
David Giaretta2 >> (All): What do we need to do now
JohnGarrett >> (All): This standard will have their own standard number.  Within the standards, you will see the reference section with the references to the other standards.
David Giaretta2 >> (All): I'd appreciate it if people could take a look at the doc on the Wiki
David Giaretta2 >> (All): and begin asking about possible funding for text audits
David Giaretta2 >> (All): ...test audits
BruceAmbacher >> (All): Do we need the stripped document as well to see that it stands alone and is coherent, flows well, etc,?
David Giaretta2 >> (All): That's easy to do - I can make a stripped out doc 
BruceAmbacher >> (All): Since it is the stripped standard that will circulate for review, comment and voting
David Giaretta2 >> (All): ...but it's hard to understand
BruceAmbacher >> (All): Agreed.
David Giaretta2 >> (All): OK I'll put a stripped version on the wiki 
BruceAmbacher >> (All): That is the crux of the problem - getting a stand alone text that also is understandable
David Giaretta2 >> (All): ACTION: DG: put stripped version of doc on wiki
BruceAmbacher >> (All): Action:  All to complete new call time poll
David Giaretta2 >> (All): Bruce we could write a summary of the boxed text but it's so terse it would be difficult to give many details.
BruceAmbacher >> (All): Action: all to read stripped version and comment on its coherence, comprehension
David Giaretta2 >> (All): How about ACTION: ALL: propose summary text to replace the boxes?
Mark Conrad >> (All): I missed several weeks. Was there any discussion of the document that I prepared identifying some of the troublesome areas of 17021?
BruceAmbacher >> (All): I have not looked at it yet.
David Giaretta2 >> (All): Yes - not addressing the individual points but the more general point of keeping within the ISO audit "tradition"
David Giaretta2 >> (All): I think I had a marked up version I did not circulate - I can do that
JohnGarrett >> (All): I spent a lot of time with the original security auditinig documents we started with and I think they are understandable if you go in with the belief that the underlying ISO auditing standard provides a good general base for all audits. 
JohnGarrett >> (All): If you need to know details of the underlying standard, then it will not be satisfactory.
David Giaretta2 >> (All): John - yes that's right - it is a matter of confidence in the rpocess
JohnGarrett >> (All): But all the ISO auditing standards are in the same format.
BruceAmbacher >> (All): We also need to make it clear that the audit will focus on the criteria in the TDR metrics standard
Helen Tibbo >> (All): He guys. Sorry. II need to move off to another meeting. Hitting the new year running. -Helen
David Giaretta2 >> (All): I put something about that in section 2 of the CCSDS version
David Giaretta2 >> (All): Bye Helen
Mark Conrad >> (All): The draft with the boxes still says 17021 is normative and that everything here is IN ADDITION to what is in 17021.
Mark Conrad >> (All): Are we meeting next week at the same time or will you call the next meeting based on the poll results?
David Giaretta2 >> (All): Mark - 17021 is general, and we provide the TDR specifics
David Giaretta2 >> (All): The idea was that the poll results would affect what happens after 21st Jan
David Giaretta2 >> (All): SO next week - same time, same day
BruceAmbacher >> (All): We have established a good number of action items for next week and I think we have re-established our areas of concern.  I propose we should focus on the coherence, and understandability of the stripped draft for next week.
David Giaretta2 >> (All): OK - good plan
BruceAmbacher >> (All): So next week is same time?
David Giaretta2 >> (All): OK - bye all
JohnGarrett >> (All): OK, Bye all
BruceAmbacher >> (All): I cannot make next week we are reviewing PhD applications; delayed due to snow
Mark Conrad >> (All): Ok. Bye.
Topic revision: r1 - 2010-01-05 - DavidGiaretta
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2018 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback