Difference between revisions of "MRRT FT Evaluation"

From IHE Wiki
Jump to navigation Jump to search
 
(17 intermediate revisions by 2 users not shown)
Line 19: Line 19:
  
 
* Have all open issues listed in the Supplement been closed?
 
* Have all open issues listed in the Supplement been closed?
::* Yes
+
::* Yes   '''TODO:'''  There are 2 open issues in MRRT.  Verify with kinson how they should be closed
  
 
* Have all significant issues at Connectathon been dealt with?
 
* Have all significant issues at Connectathon been dealt with?
::* Yes
+
::* Yes.  Project Manager confirmed that no issues were raised during Connectathon testing.
  
* Gather feedback from implementers via a [ftp://ftp.ihe.net/Connectathon/IHE%20Vendor%20Questionnaire%20V0.4.doc formal questionnaire to Connectathon participants]
+
* Gather feedback from implementers via a [ftp://ftp.ihe.net/Connectathon/IHE%20Vendor%20Questionnaire%20V0.4.doc formal questionnaire to Connectathon participants]   '''TODO:'''  Find whether it exists on google drive
 
::* radreport.org used MRRT format for its report template. The templates have significant number of downloads [[https://radreport.org/metrics]]
 
::* radreport.org used MRRT format for its report template. The templates have significant number of downloads [[https://radreport.org/metrics]]
 
:::* According to Dr. Charles Kahn, the counter only includes downloads of the revised website, excluding 5 million downloads already at the original site
 
:::* According to Dr. Charles Kahn, the counter only includes downloads of the revised website, excluding 5 million downloads already at the original site
Line 32: Line 32:
  
 
* Has the Connectathon Project Manager been queried and significant issues addressed?
 
* Has the Connectathon Project Manager been queried and significant issues addressed?
::* Yes
+
::* Yes.  No issues to address.
  
 
== Technical Committee Consensus==
 
== Technical Committee Consensus==
Line 49: Line 49:
 
::* No. Only Report Template Manager and Report Creator were tested, but no Report Template Creator.
 
::* No. Only Report Template Manager and Report Creator were tested, but no Report Template Creator.
 
:::* However, there exist T-REX at radreport.org which is a Report Template Creator.
 
:::* However, there exist T-REX at radreport.org which is a Report Template Creator.
 
  
 
* Have different implementations of each actor in the profile been tested?
 
* Have different implementations of each actor in the profile been tested?
Line 61: Line 60:
  
 
* Have the standards underlying the profile been implemented? In similar use cases? In healthcare? In general IT?
 
* Have the standards underlying the profile been implemented? In similar use cases? In healthcare? In general IT?
::* Yes, HTML5 and HTTP(S) are widely implemented.
+
::* Yes, outside healthcare, HTML5 is widely implemented.  In healthcare it's not clear (using google) that HTML5 is broadly used to create medical documents using a template.  However, radiology templates are a bit of a special case, and the implementation experience referenced above gives us confidence in our use of HTML5 in MRRT.  IHE RAD acknowledges that IHE SDC is using ISO-19763-13 and XML for a similar use case.
  
 
* (Do you have concrete reason to believe that this works robustly in the Real World) / (Are any products available for purchase that implement the profile?)
 
* (Do you have concrete reason to believe that this works robustly in the Real World) / (Are any products available for purchase that implement the profile?)
::* Yes
+
::* Yes.  As of 2020:
:::* '''TODO''' Prof. Mildenberger provides additional evidence of implementation.
+
:::* From Prof. Mildenberger:
 +
::::* one is actually implementing a solution (goal Q4/2020), which will provide all three actors and also will support the import/export of MRRT compliant templates. This company is already aiming for connectathon participation
 +
::::* one company does have a PoC (Proof of Concept) implementation supporting Report Creator, not yet transferred in a product
 +
::::* one company does actually using their proprietary structure, but is analyzing options for compatibility (all three actors in principle). Actually, a manual import for MRRT templates is supported. Participation at connectathon might be a topic for the future.
 +
::::* another company (international PACS provider with reporting solution): structured reporting will be integrated actually, MRRT compatibility planned (personal communication, e-mail not yet received)
 +
:::* From Dr. Kahn, SmartRadiology and Sectra has expressed interest in using MRRT (and Sectra has tested at Connectathon)
  
 
* Have all issues that may have been raised about the profile been resolved?
 
* Have all issues that may have been raised about the profile been resolved?
Line 77: Line 81:
 
* Has there been sufficient interest in the profile to generate a one-page [[Profiles|overview of the profile]]
 
* Has there been sufficient interest in the profile to generate a one-page [[Profiles|overview of the profile]]
 
:::* Yes, '''TODO''' see [https://wiki.ihe.net/index.php/Management_of_Radiology_Report_Templates]
 
:::* Yes, '''TODO''' see [https://wiki.ihe.net/index.php/Management_of_Radiology_Report_Templates]
 +
 +
'''DECISION''':  2021-11-18:  RAD Tech will vote to recommend moving this profile to Final text once one-page wiki overview is complete and open issues in the Supplement have been reviewed and resolved by RAD Tech.  Note that CP cleanup (rejection) is a TO DO but not a barrier to moving to Final Text.
 +
 +
'''TO DO''': Schedule RAD PLAN voting call to review the results of this checklist and ask RAD PLAN to formally approve MRRT for Final Text.
 +
 +
'''RATIONALE NOTES''':  Although Connectathon testing is not as complete as the guidelines state (e.g. testing in 2 regions), IHE RAD believes that the real-world usage experience described above compensates for that.  The profile would not really benefit from additional time in Trial Implementation.

Latest revision as of 15:00, 18 November 2021

Proposal

The Management of Radiology Report Template (MRRT) profile has been nominated for advancement to Final Text. (Advocate: Kinson Ho) Per the Final Text Process, Items in blue text below warrant Committee discussion.

Technical Committee Checklist

  • Are all significant CPs against the profile "closed"
  • Yes. Significant CPs identified and addressed. See [1]
  • Are all significant CPs against the underlying standards "closed"?
  • There are no significant CPs against the underlying standards (HTML5, HTTPS, Dublin Core)
  • Have all significant comments been CP'd or rejected?
  • Yes, specifically the following CPs are rejected as we decided to leave those advanced features out of MRRT. Alternative solutions like IHE QRPH Structured Data Capture (SDC) can be used that has already provided these advanced features
  • TODO CP-RAD-307 Add Conditional data-field-completion attributes in MRRT
  • TODO CP-RAD-308 Add Business Names to MRRT template format
  • The following CPs are rejected as identified out of scope of MRRT but implementation can support them
  • CP-RAD-412 Support optional use of javascript in MRRT
  • Have all open issues listed in the Supplement been closed?
  • Yes TODO: There are 2 open issues in MRRT. Verify with kinson how they should be closed
  • Have all significant issues at Connectathon been dealt with?
  • Yes. Project Manager confirmed that no issues were raised during Connectathon testing.
  • radreport.org used MRRT format for its report template. The templates have significant number of downloads [[2]]
  • According to Dr. Charles Kahn, the counter only includes downloads of the revised website, excluding 5 million downloads already at the original site
  • T-REX, a report template creator tool, is available at radreport.org to create radiology report template that are compliant with MRRT
  • According to Dr. Charles Kahn, T-REX is also actively being used to create new templates
  • Prof. Mildenberger published a paper regarding MRRT [[3]]
  • Has the Connectathon Project Manager been queried and significant issues addressed?
  • Yes. No issues to address.

Technical Committee Consensus

  • TODO: The Technical Committee agreed to continue with the Final Text Process and continue with an evaluation by the Planning Committee

Planning Committee Checklist

  • Has the profile been through a Connectathon in at least two regions?
  • No. There is one successful testing done in NA2016. Here are additional comments from Lynn
  • Siemens successfully tested the Report Creator with one Report Template Mgr partner (Sectra); this included [RAD-105] Query Img Report Templates, and [RAD-103] Retrieve Img Report Templates
  • Thus, Sectra successfully tested the Report Template Manager with Siemens for [RAD-103] and [RAD-105]. However, since there was no Report Template Creator test partner, Sectra was not able to test [RAD-104] Store Imaging Report Template.
  • We had no Report Template Creator test partner.
  • We have a set of MRRT Report templates that we used in the test scenario. They are now stored on the IHE Google Drive here. That test data has not been re-assessed for any updates that need to be done due to CPs processed since 2016.
  • Has the profile been successfully tested with all actors at least at one Connectathon?
  • No. Only Report Template Manager and Report Creator were tested, but no Report Template Creator.
  • However, there exist T-REX at radreport.org which is a Report Template Creator.
  • Have different implementations of each actor in the profile been tested?
  • No. Only one implementation for Report Template Manager and one implementation for Report Creator was tested.
  • Have all the options been tested successfully at at least one Connectathon?
  • N/A. No options available in MRRT
  • Are there IHE-provided software testing tools to address all aspects of the profile?
  • No testing tools available, besides the sample report templates mentioned above (extracted from radreport.org)
  • Have the standards underlying the profile been implemented? In similar use cases? In healthcare? In general IT?
  • Yes, outside healthcare, HTML5 is widely implemented. In healthcare it's not clear (using google) that HTML5 is broadly used to create medical documents using a template. However, radiology templates are a bit of a special case, and the implementation experience referenced above gives us confidence in our use of HTML5 in MRRT. IHE RAD acknowledges that IHE SDC is using ISO-19763-13 and XML for a similar use case.
  • (Do you have concrete reason to believe that this works robustly in the Real World) / (Are any products available for purchase that implement the profile?)
  • Yes. As of 2020:
  • From Prof. Mildenberger:
  • one is actually implementing a solution (goal Q4/2020), which will provide all three actors and also will support the import/export of MRRT compliant templates. This company is already aiming for connectathon participation
  • one company does have a PoC (Proof of Concept) implementation supporting Report Creator, not yet transferred in a product
  • one company does actually using their proprietary structure, but is analyzing options for compatibility (all three actors in principle). Actually, a manual import for MRRT templates is supported. Participation at connectathon might be a topic for the future.
  • another company (international PACS provider with reporting solution): structured reporting will be integrated actually, MRRT compatibility planned (personal communication, e-mail not yet received)
  • From Dr. Kahn, SmartRadiology and Sectra has expressed interest in using MRRT (and Sectra has tested at Connectathon)
  • Have all issues that may have been raised about the profile been resolved?
  • Yes, all significant issues have been addressed
  • The following issues are rejected
  • Conditional statements (CP-RAD-307) - defer to other alternatives such as IHE QRPH SDC
  • Support for javascript (CP-RAD-412) - Implementation details, cannot be standardized. Leave it out as implementation decision
  • Support for image link - discussed and decided to defer
  • Support for repeated statements - discussed and defer to other alternatives such as IHE QRPH SDC
  • Yes, TODO see [4]

DECISION: 2021-11-18: RAD Tech will vote to recommend moving this profile to Final text once one-page wiki overview is complete and open issues in the Supplement have been reviewed and resolved by RAD Tech. Note that CP cleanup (rejection) is a TO DO but not a barrier to moving to Final Text.

TO DO: Schedule RAD PLAN voting call to review the results of this checklist and ask RAD PLAN to formally approve MRRT for Final Text.

RATIONALE NOTES: Although Connectathon testing is not as complete as the guidelines state (e.g. testing in 2 regions), IHE RAD believes that the real-world usage experience described above compensates for that. The profile would not really benefit from additional time in Trial Implementation.