Rad Tech Minutes 2020-01-27-30

(Number in parentheses is cumulative total hours for week)

Monday, January 27, 2019: 8:30 am - 5:00 pm Central Time (CT)

IHE Radiology Technical Committee Roster

08:30-09:00: Breakfast, Welcome, Patent Disclosure Announcement, Agenda Review

• Lynn Felhofer, IHE Radiology Technical Project Manager
• Kevin O'Donnell, Canon Medical
• Jonathan Whitby, Canon Medical
• Wim Corbijn, Philips
• Kinson Ho, Change Healthcare
• Harald Zachmann, IBM Watson Health
• Chris Lindop, Citius Tech
• Brad Genereaux, SIIM
• Antje Schroeder, Siemens Healthineers
• David Kwan, CCO
• Yi-Hwa Liu, Yale
• Andrei Leontiev, Visage Imaging
• Steve Nichols, GE
• Charles Parisot, InteropEhealth
• Hamid Neshat, Agfa
• Nichole Knox, RSNA
  • Goal of meeting is to finalize documents by end of the meeting for Public Comment publication.
• Reviewed Patent Disclosure Announcement

09:00-10:30: AI Result (1.5)

• Read and agree document is okay for context
• Flag comments and highlight in yellow items to come back to for further review
• Action: Review new definition process; may need a new one for this document; add to DCC on either 1/28 or February call depending on outcome

10:30-12:00: AI Workflow (1.5)

12:00-13:00: Lunch

13:00-14:00: Technical Framework Maintenance (1) - Review Shareathon by Steve, CP Ballot Comments Review, SWF.b final text questionnaire

• Focused on Shareathon review by Steve

14:00-15:30: AI Result (3)

• Committee homework to review and return with questions and comments on 1/28

15:30-17:00: AI Workflow (3)

Tuesday, January 28, 2019: 8:30 am - 5:00 pm CT

IHE Radiology Technical Committee Roster

08:30-09:00: Breakfast, Welcome, Patent Disclosure Announcement, Agenda Review

• Lynn Felhofer, IHE Radiology Technical Project Manager
• Kevin O'Donnell, Canon Medical
• Jonathan Whitby, Canon Medical
• Wim Corbijn, Philips
• Kinson Ho, Change Healthcare
• Harald Zachmann, IBM Watson Health
• Chris Lindop, Citius Tech
• Brad Genereaux, SIIM
• Antje Schroeder, Siemens Healthineers
• David Kwan, CCO
• Andrei Leontiev, Visage Imaging
• Nichole Knox, RSNA

09:00-10:30: AI Result (4.5)

• Committee homework - finish reading the display transaction and concept section. If committee can accomplish this, we can discuss open issues.

10:30-12:00: AI Workflow (4.5)

12:00-13:00: Lunch

13:00-15:00: TF Maintenance (3)

• CP-RAD-258: Accepted with changes as discussed; move to Final Text)
• CP-RAD-356: Accepted w/editorial changes; move to Final Text
• CP-RAD-378: pending; may be able to review again later in the week
• CP-RAD-379: Accepted; move to Final Text
• CP-RAD-379: Accepted; move to Final Text
  • Due to number of people having problems accessing the IHE FTP site, Kinson is asking for a volunteer to maintain the FTP site, update the spreadsheet
• Discussion about TF Maintenance proposed breakdown of Radiology Technical Framework
  • Editor will be assigned by end of this meeting but the plan is listed in link above.
  • Goal is to finish in this cycle by June/July.

15:00-17:00: AI Workflow (6.5)

• Committee homework: Look at concept section. Pay particular attention to the 4.80.4.1.2.1.1. AI Service Request and the X.4.2.1 AI Workflow Clinical Example.

• Desired results for end of day on 1/29/2020
  • TF Maintenance: SWF.b final text, incoming CP review, Final Review of CP-378, Connectathon debrief and preview of EU Connectathon
  • AI Workflow morning: Review Concept Section (e.g. business logic separate from task management, consideration regarding Orchestrator and Performer, etc)
  • AI Results afternoon – Through whole document, start review side bar comments
  • AI Workflow afternoon – Vol 1 complete review, Attribute requests for Rad-80 Create Workitem

• Desired results for end of day on 1/30/2020
  • AI Result morning: Completed review most side bar comments, ready for Public Comment publication (remaining side bar comments will be in open questions)
  • AI Workflow morning: : Review Concept Section, in particular the reporting workflow integration, RAD-49 Instance Availability Notification
  • Scheduling and Evaluation Checkpoint
  • AI Workflow afternoon: Add references to UPS transactions to AIW-I and review those transactions to see if adjustments required.
  • TF Maintenance – Assign volume editor to start TF re-structure. Timeline for publication for final text (need to communicate to Mary)

Wednesday, January 29, 2019: 8:30 am - 5:00 pm CT

IHE Radiology Technical Committee Roster

08:30-08:30: Breakfast, Welcome, Patent Disclosure Announcement, Agenda Review

• Lynn Felhofer, IHE Radiology Technical Project Manager
• Kevin O'Donnell, Canon Medical
• Jonathan Whitby, Canon Medical
• Wim Corbijn, Philips
• Kinson Ho, Change Healthcare
• Harald Zachmann, IBM Watson Health
• Chris Lindop, Citius Tech
• Antje Schroeder, Siemens Healthineers
• Andrei Leontiev, Visage Imaging
• Teri Sippel Schmidt, Marquette University
• Brad Genereaux, SIIM
• Chris Carr, RSNA
• Nichole Knox, RSNA

08:30-10:00: TF Maintenance (4.5) - SWF.b final text, incoming CP review, Final review of CP-378, Connectathon debrief and preview of EU Connectathon

• IHE Rad Tech committee motioned to approve SWF.b for Final Text
  • Unanimously approved. Now goes to the planning committee. Kinson will send email to planning committee co-chairs with information on technical committee recommendation and offer to help.
• IHE USA Connectathon Radiology Recap

10:00-10:30: AI Results

10:30-12:00: AI Workflow (8) - Review Concept Section (e.g. business logic separate from task management, consideration regarding Orchestrator and Performer, etc.)

12:00-13:00: Lunch

13:00-13:30: Discuss communication strategy to promote the AI profiles to the AI communities

• AI Communication

• Activities
  • February / March
    • Public comments publication (February / March)
      • Target audiences
        • the RSNA AI vendors - get the list and contact from Chris Carr
        • Neil's group
        • DICOM WG-23 - Larry
      • Webinar during public comment - Brad, Kevin Chris Lindop (March 3 1pm CT - 1 hour)
      • Target specific sections in the profile - Rad Planning Committee, RSNA RIC - Chris Roth, David Mandelson
    • ECR (March) - Vienna - Plan to have DSI session, Peter is our evangelist
    • HIMSS? (March 9-13) - Orlando - during the HIMSS-SIIM meeting?
  • April
    • RBMA (April) - Interoperability session - Kinson
    • IHE Rad TC Trial Implementation Preparation F2F meeting
  • May
    • Trial Implementation publication
  • June
    • SIIM
      • SIIM Hackathon (June) - Mohannad
      • SIIM session
  • July / August
    • IHE Radiology Webinar
  • September
    • SIIM C-MIMI (September 13-14)
  • October
    • ACR Informatics Summit (October 24-25)
  • November
    • RSNA (Nov) - Advocacy meeting at the AI Showcase area
  • Other
    • SIIM webinar - Any time - Brad / Kevin

• Outreach opportunites
  • SIIM Outreach committee - Raym, Brad
  • SIIM machine learning committee
  • ACR DSI
  • RSNA RIC - Chris Roth

• AI Champions
  • Kevin
  • Chris Lindop
  • Brad the Ambassador
  • Brad's demo (to be open source in Github)

13:30-15:30: AI Results (6.5) - Through whole document, start review side bar comments

15:30-17:30: AI Workflow (10) - Vol 1 complete review, Attribute requests for RAD-80 Create Workitem

Thursday, January 30, 2019: 8.30 am - 3:00 pm CT

IHE Radiology Technical Committee Roster

• Lynn Felhofer, IHE Radiology Technical Project Manager
• Kevin O'Donnell, Canon Medical
• Jonathan Whitby, Canon Medical
• Wim Corbijn, Philips
• Kinson Ho, Change Healthcare
• Harald Zachmann, IBM Watson Health
• Chris Lindop, Citius Tech
• Antje Schroeder, Siemens Healthineers
• Andrei Leontiev, Visage Imaging
• David Kwan, CCO
• Chris Carr, RSNA
• Nichole Knox, RSNA

08:30-10:00: AI Result (7.5) - Completed review most side bar comments, ready for Public Comment publication (remaining side bar comments will be in open questions)

10:00-11:00: AI Workflow (9) - Review Concept Section, in particular the reporting workflow integration, RAD-49 Instance Availability Notification

11:00-12:00: Scheduling + Evaluation Checkpoint

12:00-12:30: Lunch

12:30-14:00: AI Workflow (10.5) - Add references to UPS transactions to AIW-I and review those transactions to see if adjustments required.

14:00-15:00: TF Maintenance (7.5) - Assign volume editor to start TF re-structure. Timeline for publication for final text (need to communicate to Mary)

Annex: PC-Prep Closing Assessments

• Profile Name: AI Workflow
  • Did we line-by-line the entire document

• No. Lots of editing still to do

• • How ready is it to go out for PC: Completely, Almost, Soonish, Hmmm

• Soonish?

• • Which open issues are risky, and why

• Haven't really populated the open issues section yet
• UPS adoption is risky

• • Are all open issues phrased to solicit the needed information to close them?

• <see above>

• • Which use cases need more input

• Would be nice to get DSI to describe whether they like push, pull and why.
• E.g. can algorithms recognize study characteristics and tasks that are applicable to them (pull) or do they depend on another system to tell them what they can work on (push)
• Chris - mostly the latter. The business logic is in the orchestrator which knows the detailed capabilities, strengths and weaknesses of all the performers it manages
• Wim - worried that the business logic expectations might not be aligned (Add to Risks)
• Related question is how much business logic is the Requester permitted to have. How does it collaborate with the orchestrator.

• • Which issues from the Kickoff Closing Assessment are still unresolved

• The Open Issues for the UPs were closed but might need to be reflected in the Closed Issues section
• Procedure Update for worklist modification is deprecated to Concepts Section
• UPS-RS was not completely put to bed. Redebated variety of details but we're done now(?)

• But DSI might want to re-open it again and use their API. And FHIR Task is also waiting in the wings.

• AIs have accepted using DICOM as input rather than bitmaps
• The mapping of products to actors will be in the Actor description section but isnt yet. (Clarity Risk)
• Proxy use case is intended to cover the cloud-based performers. Haven't specifically discussed any details of security or configuration etc. Basically the Retrieve and Store transactions with the Performer still work.

• • What significant debates in PC-prep were not anticipated in the Kickoff

• Debating many parts of UPS-RS again.
• Discussion of the splits in responsibility between the Requester, Manager, and Performer.

• Have clarified grouping a bit, but a certain amount that is being assumed in the Manager/Orchestrator

• Lots of time to explain that the UPS-RS Push workflow is a triggered Pull and the differences.
• Discussions about tracking edits and leveraging existing text.
• Debate on why Update Workitem is needed/useful

• • Review the "complexity points" in the evaluation. Did each get appropriate text coverage/resolution?

• Alternate protocols was "resolved" during evaluation, and again during Kickoff, and again during PC-prep so WELL covered
• Context information for the Requester has been reduced to just Instance Availability Notification. All other information about completion of exam, type of exam, patient demographics, admission info, prior history has been considered out of scope.
• Enumeration of details likely to be useful when processing or choosing whether to process a dataset - for processing the list was discussed but still needs to be added to the profile (informative)
• Definitely covered Push vs Pull (more than the allocated points)
• Standard code sets are mentioned in Concepts with some sources of codes and an open issue for how to deal with this.

• • Do the effort points in the evaluation still seem right?

• Spent more time on Use Cases than the EP-1 predicted
• Registration activity is still TBA that informs the Manager so it can identify relevant AI models to execute

- will be Concept Section - sliding out of scope for a technical solution - Open Channel just tells you existence.

• Allowing an AI given the minimum dataset to use API’s to retrieve “what it needs” is now out of scope. Not listing C-FIND or QIDO as optional for Performer and Manager proxying gets complex. Will add a Concept section.
• Didn't initially work from RRR-WF text but starting to do so now.

• • How does the scope feel in terms of being a useful chunk of work? (Needs more? Just right? More than enough?)

• Scope is workable. Might be nicer to be bigger (add FHIR resources, registration, etc), but the amount of work is prohibitive

• • How is the work fitting in the allocated bandwidth? (Time to spare? Just right? Things were left undone?)

• Struggling.

• • Did the Breakdown of Tasks accurately reflect the work? What extra tasks arose?

• No. BoT was too high level (Analyze issues, write vol1/2, review, update) need more concrete list next time.

• Should we make one now? A Todo list?
• Definitely next time follow the template, but don't worry about it for now. Keep a good todo.

• Planning of our advocacy (consider adding to the template)

• • Looking forward, if you had to reduce scope to hit TI, what would you drop

• (Reaching consensus)
• Nothing, will drop the entire document instead and ask for next year instead

• • Have the promised resources manifested

• Lots of advice, opinions. Might have been good to request more concrete contributions to the document text.

• • What vendors are engaged (for each actor)

• Requester - GE, IBM, Change, Canon/Vital, Visage
• Orchestrator - Siemens, GE, IBM, Change, Canon/Vital, Visage
• Performer - Siemens, GE, IBM, Change, Canon/Vital, Visage

• • When will we have sample data/objects

• (Brads demo?)

• • Who should specifically be targeted for Public Comment feedback

• ACR DSI, Radelement folks, AI Vendor list, Display vendors, WG-23, see Kinsons list

• • Was the profile where it needed to be at the start of the PC meeting (See "PC Prep Meeting" above), if not what was the gap

• No. Ended up going back to use cases and working forward.

• • Was the profile where it needed to be at the end of the PC meeting, if not what was the gap

• Not really. More concepts needed, line-by-line of most of document needed

• • How many tcons would you like between now and PC Publication

• 3 and maybe a fourth (all 2 hour)

• • Do you need any tcons before TI Prep Meeting

• At least 1

• Profile Name: AI Results
  • Did we line-by-line the entire document

• Yes (quickly - additional edits will be accepted by email before PC)

• • How ready is it to go out for PC: Completely, Almost, Soonish, Hmmm

• Almost

• • Which open issues are risky, and why

• Some interesting feedback hopefully, but don't see any risks to the profile

• • Are all open issues phrased to solicit the needed information to close them?

• Yes. Just checked.

• • Which use cases need more input

• None needed in the Use Case section per se, but would love feedback on the Filtering/Navigation/Display concepts.

• • Which issues from the Kickoff Closing Assessment are still unresolved

• Weren't really any unresolved issues. Still need to figure out examples/samples

• • What significant debates in PC-prep were not anticipated in the Kickoff

• No significant debates. Loads of little debates on many various details.

• • Review the "complexity points" in the evaluation. Did each get appropriate text coverage/resolution?

• Yes. (Filtering/navigation was probably more than 0.5 - more like 1.5)

• • Do the effort points in the evaluation still seem right?

• Mostly. Profile Review should have been a bit higher because of all the interesting details; the filtering discussion was also meatier; the transaction cloning was easier than expected.

• • How does the scope feel in terms of being a useful chunk of work? (Needs more? Just right? More than enough?)

• About right

• • How is the work fitting in the allocated bandwidth? (Time to spare? Just right? Things were left undone?)

• OK-ish, but tight. A few details moved to offline work.

• • Did the Breakdown of Tasks accurately reflect the work? What extra tasks arose?

• Basically yes. Should maybe have mentioned "architectural" discussions

• • Looking forward, if you had to reduce scope to hit TI, what would you drop

• Examples (but would work on them for later)

• • Have the promised resources manifested

• Mostly, got input from clinical and technical but still pulling some additional clinical input

• • What vendors are engaged (for each actor)

• Evidence Creators - IBM, Canon, Philips, Siemens, Visage, GE, CitiusTech, ...
• Image Display - IBM, Canon, Philips, Siemens, Visage, GE, CitiusTech, ...
• Doc Consumer - ?? haven't really explored

• • When will we have sample data/objects

• Excellent question :-) Work on coordinating with RSNA 2020 demo - looking at Stroke, incidental pulmonary nodules

• Good opportunity - should align examples to share work, ease uptake, etc.

• • Who should specifically be targeted for Public Comment feedback

• ACR DSI, Radelement folks, AI Vendor list, Display vendors, WG-23, see Kinsons list

• • Was the profile where it needed to be at the start of the PC meeting (See "PC Prep Meeting" above), if not what was the gap

• Close but not quite, had a mostly complete PC draft, still had a few open sections

• • Was the profile where it needed to be at the end of the PC meeting, if not what was the gap

• Almost. Still some homework to clean up

• • How many tcons would you like between now and PC Publication

• Plan to circulate for final clarity edit proposals
• Plan one tcon to formally approve for PC (could be combined with a Workflow Tcon)

• • Do you need any tcons before TI Prep Meeting

• Yes, can cancel if not needed. Might use for hand-grenades or to start edit reviews.