Difference between revisions of "IOCM FT Evaluation"

From IHE Wiki
Jump to navigation Jump to search
 
(13 intermediate revisions by 2 users not shown)
Line 1: Line 1:
−
Run down the following checklist for each profile:
+
[[Imaging Object Change Management]] has been nominated for advancement to Final Text. (Advocate: Kinson Ho)
  
−
Are all significant CPs against the profile "closed"?
+
Per the [[Final Text Process]], <font color="blue">Items in blue text</font> below warrant Committee discussion.
−
[Kinson] CP-242 is currently in ballot.
 
  
−
Are all significant CPs against the underlying standards "closed"?
+
==TC Checklist==
−
[Kinson] No pending CPs for underlying standards
 
  
−
Have all significant comments been CP'd or rejected?
+
* Are all significant CPs against the profile "closed"?
−
[Kinson] Yes.
+
:: Mostly. <font color="blue">Open CPs</font>:
 +
::: CP-241 - Clarify how to handle retention policy expiry with XDS-I manifest deletion
 +
::: CP-283 - Clarify object referencing in Rejection Note
 +
::: CP-296 - Clarify usage of Modalities in Study in IOCM
  
−
Have all open issues listed in the Supplement been closed?
+
* Are all significant CPs against the underlying standards "closed"?
−
[Kinson] Yes.
+
:: [Kinson] No pending CPs for underlying standards
  
−
Have all significant issues at Connectathon been dealt with?
+
* Have all significant comments been CP'd or rejected?  
−
[Kinson] Yes. All issues have been identified during or after Connecthaton have corresponding CPs created. Some of them have already been balloted and ready for final text. Some of them are still pending reviews before ready for ballot.
+
:: [Kinson] Yes.
  
−
Gather feedback from implementers via a formal questionnaire to Connectathon participants
+
*Have all open issues listed in the Supplement been closed?
−
[Kinson] ??
+
:: [Kinson] Yes.
  
−
Has the Connectathon Project Manager been queried and significant issues addressed?
+
* Have all significant issues at Connectathon been dealt with?  
−
[Kinson] Connectathon test results for IOCM is available [ftp://ftp.ihe.net/Radiology/RAD_Connectathon_Results_Summaries/ here].
+
:: [Kinson] Yes. All issues have been identified during or after Connecthaton have corresponding CPs created. Some of them have already been balloted and ready for final text. Some of them are still pending reviews before ready for ballot.
  
−
Debate checklist exceptions (failure of any of the above is cause for discussion)
+
* Gather feedback from implementers via a formal questionnaire to Connectathon participants
−
Record checklist findings and debate conclusion
+
:: [Kinson] Annecdotal feedback from Lynn and David are that the implementations had no particular technical issues (aside from needing to use the Content Tree for references which has a CP).  Some Image Display implementations were a little hokey with KIN in general.  But a number of the implementations were well thought out and mature.  The scenarios of supressing/exposing, etc. worked well.
 +
 
 +
* Has the Connectathon Project Manager been queried and significant issues addressed?
 +
:: [Kinson] Yes.  No issues. Connectathon test results for IOCM is available [ftp://ftp.ihe.net/Radiology/RAD_Connectathon_Results_Summaries/ here].
 +
 
 +
 
 +
===TC Conclusion===
 +
Looking ahead, there are significant issues of lack of testing of many actors in IOCM.
 +
Since the hub of IOCM '''has''' been getting tested, but there appears to be no interest in the "peripheral" actors, Tech Cmte proposes that a CP be written to reduce the scope of IOCM.  Cut out the untested actors (and the associated interactions with PDI, SWF, etc).
 +
 
 +
Kinson will submit such a CP.
 +
Upon review of the CP, the Tech Cmte will consider proposing the result to Final Text.
 +
 
 +
==PC Checklist==
 +
 
 +
* Put Final Text Decision on the planning committee agenda
 +
** Consider doing this a couple months before new TF version will be released so it can be incorporated.
 +
** It's helpful to assign an advocate for the supplement at this time to check/prepare the evidence for the upcoming checklist rather than go hunting for it during the meeting
 +
 
 +
 
 +
* Has the profile been through a Connectathon in at least two regions?
 +
:: No. EU (2014, 2013, 2012)
 +
:: But at least one of the Image Managers (Agfa) and one of the Change Requesters (GE) are global vendors so the committee decided to accept the EU-only testing.
 +
* Has the profile been successfully tested with all actors at least at one Connectathon?
 +
:: No, but the Final Text profile has been reduced in scope to include only the actors that were tested in multiple Connectathons. (Image Manager and Change Requester).
 +
:: The other untested (peripheral) actors have been extracted from the Final text version into a Trial Implementation draft called IOCM Extensions.  This allows the possibility that such actors may be implemented going forward (they seem like they could be useful) without holding back the functional core of the profile.
 +
* Have different implementations of each actor in the profile been tested?
 +
:: Yes.
 +
:: Image Manager (Agfa, INFINITT, Visbeon)
 +
:: Change Requester (Fujifilm, GE, Rogan-Delft, Visbeon)
 +
:: One implementation (Visbeon) has tested being able to both send and receive change requests.
 +
* Have all the options been tested successfully at at least one Connectathon?
 +
:: No options.
 +
* Is there IHE-provided software testing infrastructure that addresses all aspects of the profile?
 +
:: <font color="blue"> (Check with Lynn) </font>
 +
:: Yes, the committee reported that there are sample IOCM-KIN objects and the test plans check the required behaviors.
 +
* Have the standards underlying the profile been implemented? In similar use cases? In healthcare? In general IT?
 +
:: DICOM Key Object Selection objects have been implemented in Radiology before, however this is a new usage of KOS
 +
:: The committee felt this is sufficient.
 +
* (Do you have concrete reason to believe that this works robustly in the Real World) / (Are any products available for purchase that implement the profile?)
 +
:: There are no products in the product registry or found by Google search, however one company reports that it has a product in the works with test installations that are using IOCM successfully. (UK?) Some existing sites have proprietary solutions deployed which match the capabilities specified.
 +
:: The committee felt this is sufficient.
 +
* Have all issues that may have been raised about the profile been resolved?
 +
:: yes
 +
* Has there been sufficient interest in the profile to generate a one-page [[Profiles|overview of the profile]]
 +
:: Yes. [[Imaging Object Change Management]]
 +
 
 +
===PC Conclusion===
 +
: Move to Final Text

Latest revision as of 15:53, 21 July 2014

Imaging Object Change Management has been nominated for advancement to Final Text. (Advocate: Kinson Ho)

Per the Final Text Process, Items in blue text below warrant Committee discussion.

TC Checklist

  • Are all significant CPs against the profile "closed"?
Mostly. Open CPs:
CP-241 - Clarify how to handle retention policy expiry with XDS-I manifest deletion
CP-283 - Clarify object referencing in Rejection Note
CP-296 - Clarify usage of Modalities in Study in IOCM
  • Are all significant CPs against the underlying standards "closed"?
[Kinson] No pending CPs for underlying standards
  • Have all significant comments been CP'd or rejected?
[Kinson] Yes.
  • Have all open issues listed in the Supplement been closed?
[Kinson] Yes.
  • Have all significant issues at Connectathon been dealt with?
[Kinson] Yes. All issues have been identified during or after Connecthaton have corresponding CPs created. Some of them have already been balloted and ready for final text. Some of them are still pending reviews before ready for ballot.
  • Gather feedback from implementers via a formal questionnaire to Connectathon participants
[Kinson] Annecdotal feedback from Lynn and David are that the implementations had no particular technical issues (aside from needing to use the Content Tree for references which has a CP). Some Image Display implementations were a little hokey with KIN in general. But a number of the implementations were well thought out and mature. The scenarios of supressing/exposing, etc. worked well.
  • Has the Connectathon Project Manager been queried and significant issues addressed?
[Kinson] Yes. No issues. Connectathon test results for IOCM is available here.


TC Conclusion

Looking ahead, there are significant issues of lack of testing of many actors in IOCM. Since the hub of IOCM has been getting tested, but there appears to be no interest in the "peripheral" actors, Tech Cmte proposes that a CP be written to reduce the scope of IOCM. Cut out the untested actors (and the associated interactions with PDI, SWF, etc).

Kinson will submit such a CP. Upon review of the CP, the Tech Cmte will consider proposing the result to Final Text.

PC Checklist

  • Put Final Text Decision on the planning committee agenda
    • Consider doing this a couple months before new TF version will be released so it can be incorporated.
    • It's helpful to assign an advocate for the supplement at this time to check/prepare the evidence for the upcoming checklist rather than go hunting for it during the meeting


  • Has the profile been through a Connectathon in at least two regions?
No. EU (2014, 2013, 2012)
But at least one of the Image Managers (Agfa) and one of the Change Requesters (GE) are global vendors so the committee decided to accept the EU-only testing.
  • Has the profile been successfully tested with all actors at least at one Connectathon?
No, but the Final Text profile has been reduced in scope to include only the actors that were tested in multiple Connectathons. (Image Manager and Change Requester).
The other untested (peripheral) actors have been extracted from the Final text version into a Trial Implementation draft called IOCM Extensions. This allows the possibility that such actors may be implemented going forward (they seem like they could be useful) without holding back the functional core of the profile.
  • Have different implementations of each actor in the profile been tested?
Yes.
Image Manager (Agfa, INFINITT, Visbeon)
Change Requester (Fujifilm, GE, Rogan-Delft, Visbeon)
One implementation (Visbeon) has tested being able to both send and receive change requests.
  • Have all the options been tested successfully at at least one Connectathon?
No options.
  • Is there IHE-provided software testing infrastructure that addresses all aspects of the profile?
(Check with Lynn)
Yes, the committee reported that there are sample IOCM-KIN objects and the test plans check the required behaviors.
  • Have the standards underlying the profile been implemented? In similar use cases? In healthcare? In general IT?
DICOM Key Object Selection objects have been implemented in Radiology before, however this is a new usage of KOS
The committee felt this is sufficient.
  • (Do you have concrete reason to believe that this works robustly in the Real World) / (Are any products available for purchase that implement the profile?)
There are no products in the product registry or found by Google search, however one company reports that it has a product in the works with test installations that are using IOCM successfully. (UK?) Some existing sites have proprietary solutions deployed which match the capabilities specified.
The committee felt this is sufficient.
  • Have all issues that may have been raised about the profile been resolved?
yes
Yes. Imaging Object Change Management

PC Conclusion

Move to Final Text
Retrieved from "http://wiki.ihe.net/index.php?title=IOCM_FT_Evaluation&oldid=78473"