AI Results - Brief Proposal

From IHE Wiki
Jump to navigation Jump to search

1. Proposed Workitem: AI Results (AIR) Profile

  • Proposal Editor: Kevin O'Donnell
  • Editor: Kevin O'Donnell
  • Contributors: DICOM WG-23, David Clunie
  • Domain: Radiology

2. The Problem

Applying AI methods (deep learning, etc) to the analysis of medical images is an area of significant interest and activity, which raises related needs...

Interoperable results: The results of such AI analysis need to be presented to radiologists in their reading environment. This depends on interoperability between AI result generation products and radiology reading workstation systems/software.

Study Integrated results: Radiologists interpreting studies likely expect AI-generated results to supplement rather than replace traditional image analysis results and thus a given study will be composed of acquired images, AI results, & traditional clinical data.

Effective presentation: Effective use of AI results will hinge on how they are presented during the busy process of reading the study.

Convergence of result encoding: Many AI results are results a human could otherwise have produced, and those human results may be used as training data for the AI, and AI results may be used by other AIs in an adversarial network. AI and non-AI results need to be handled together. Also want to facilitate data pooling and sharing between sites.

For AI to productively live up to it's promise, results and data must be reliably, and conveniently, assembled, managed and presented.

3. Key Use Case

Goal: AI products store AI results which are then retrieved and presented consistently by a variety of imaging display systems.

The display presents the result in the context of the medical imaging study to which it applies.

To minimize implementation complexity for displays, and avoid needing different software for each new AI result, compose AI results from a reasonable set of primitives.

This fits the IHE model of profiles as tools for convergence.

  • Display products that support those primitives can declare they are “AI-Ready”
  • AI products that output results using those primitives know a variety of displays can present their results
  • Users motivate display vendors to be AI-Ready by requiring conformance
  • Users motivate AI vendors to conform since it simplifies deployment for the users

4. Standards and Systems

  • AI algorithms - running in PACS, cloud, processing servers, standalone workstations, etc. (Result Creators)
  • PACS, VNA, etc.
  • Reading workstations, Image Displays
  • Databases, clinical analysis, report creators (import results)

Standards

DICOM SR - TID 1500

DICOM Segmentations

DICOM Sup 219 Simplified SR in JSON for AI

  • David Clunie has a significant draft that will be presented at WG-06 in September. He has also been working on tooling to test/confirm the transcoding logic.
  • This work is what really makes this accessible to the AI community, and bridges into the existing radiology infrastructure

AIM - Annotation and Image Markup & DICOM PS3.21 transcoding of AIM to DICOM SR

RSNA CDE (Common Data Elements)

  • helps with longitudinal data and automation logic for reporting and decision support
  • http://radelement.org

IHE MAMMO - examples of defining converged content display behaviors, including CAD results

IHE CXCAD - brief text on presenting CAD results

IHE Evidence Documents Profile

  • Should review to see if there anything we should borrow/revive

IHE Query for Existing Data for Mobile

  • consider using this to query the EHR for diagnostic reports and observations about the patient

IHE Results Distribution

  • consider the critical results status flags since AI results may be used to prioritize reading worklists

5. Technical Approach

A basic store and retrieve Profile that establishes a baseline for encoding, display behaviors/conventions, and data management (where do AI results belong/where do you find/use them).

There are many other radiology applications of AI that are not about processing images. This proposal is image-centric.

Existing actors

  • Image Manager/Archive - store AI segmentations and measurements alongside human segmentations and measurements
  • Image Display - present AI results together with the associated images
  • Evidence Creator - AI algorithms running in PACS, cloud, processing servers, standalone workstations, etc.
  • Imaging Document Consumer - systems that consume AI results rather than display them (databases, clinical analysis, report creators)

New actors

None

Existing transactions

  • Store: Re-use [RAD-43] Store Evidence Documents
  • Query: Re-use [RAD-129] QIDO-RS Query & [RAD-44] Query Evidence Documents
  • Retrieve: Re-use [RAD-107] WADO-RS Retrieve & [RAD-45] Retrieve Evidence Documents

New transactions (standards used)

  • Display AI Result - technically a behavior spec. rather than a transaction
  • STOW AI Result - store JSON-coded result using REST

Impact on existing integration profiles

  • If SR-based then most other Radiology profiles apply - XDS-I, XCA-I, IRWF.b, PDI, IDEP, IOCM, WIA, ATNA-Rad, etc.

New integration profile

  • AIR - AI Results

Breakdown of tasks

The Tech Cmte will enumerate

  • Effort Points (EP) for volume of new pages to be written/reviewed,
  • Complexity Points (CP) if the pages will be hard to write/review, and
  • Uncertainty Points (UP) where debates/discussion can be expected.
  • Re-use the following transactions for DIMSE Store/Query/Retrieve
  • [RAD-43] Evidence Document Stored - add line for template 1500
  • [RAD-44] Query Evidence Document - note navigation question
  • [RAD-45] Retrieve Evidence Document - no change
EP 0.5, CP 0, UP 0 - slight effort and complexity
UP 0.5 - should we document using existing STOW to store existing JSON (or binary) of SR during Phase 1 (see below)
  • Develop use cases around how displays/users navigate available results
  • If mostly client side interactions, then existing patient/accession/TID filtering likely adequate
  • If server side filtering, add query fields to [RAD-44] & QIDO Query transactions
  • Consider "Filtering Strategies" text cloned from RAD-64 Query Dose Information
  • Discuss image references that link results to images, and tracking ids that link results to prior results
EP 1, CP 0.5, UP 1 (client or server side)
  • Draft/Review "transaction" for Display AI Result
  • Consider general baseline behavior; displays can augment and may pursue the varied preferences of their users
  • Require/Describe display of SR annotations/observations (categorizations (e.g. pneumothorax T/F), locations (CAD marks), measurements (lengths, areas)
  • Leverage existing MAMMO text for CAD-mark display
  • Require/Describe display of segmentations (2D and 3D)
  • Require/Describe display of parametric map images (heatmaps, probability maps or other floating point scoring)
  • Require fusion of parametric maps with source image? Option?
  • Require/Describe display of images (including secondary capture)
EP 3, CP 1, UP 1 (require all or nothing?, options?)
  • Consider informative annex
  • Show examples of encoding of a few "typical" AI results in the described primitives
  • Show examples of navigation/presentation of results (if not adequately covered in the use case and Display transaction)
(Not Min) EP 1
  • Draft/Review Profile
  • Should be fairly straightforward content profile (store, query, retrieve, display)
  • Use Cases might get interesting, but those are covered above
EP 2

(Potential "Phase II" depending on progress of Sup219 JSON SR)

  • Clone "transaction" for DICOMweb STOW AI Result based on RAD-63 Submit Dose Information
  • Uses STOW-RS to carry JSON-encoded SR payload
  • Reference DICOM Sup219 on Simplified SR in JSON
  • Reference DICOM TID 1500 for measurements/observations
  • Reference DICOM Segmentation IOD
(Not Min) EP 1, CP 1
  • Re-use the following transactions for DICOMweb Query/Retrieve
  • [RAD-129] QIDO-RS Query
  • [RAD-107] WADO-RS Retrieve - no change
(not Min) EP 0.5 (if new query tags)

6. Support & Resources

  • DICOM WG-23 is engaged on the topic of AI Results and has completed work useful to this activity
  • Need to reach out/identify those with an interest in implementing/prototyping the Profile if it is published this cycle.

7. Risks

  • The JSON SR supplement could get hung up on technical hitches.
  • Strategy: That primarily affects Stage II, could be moved to second cycle
  • Will the integration preferences of the report composition system (e.g. to the EMR) negatively influence adoption of presenting AI results by the display system
  • Debates on alternative standards
  • Scope creep - AI is a large space and there is much fun to be had
  • Strategy: Focus this work on a) results of image analysis that b) informs the radiologist during reading of the study
  • AI that analyzes things outside of imaging, and results that are applied elsewhere like worklist prioritization or patient risk stratification, are useful but will be out of scope for this particular effort.

8. Open Issues

<Point out any key issues or design problems. This will be helpful for estimating the amount of work and demonstrates thought has already gone into the candidate profile.>

  • Issue: How will displays/users navigate available results? (See above)
  • Issue: What are sensible default presentation behaviors for each type of primitive? (See above)

9. Tech Cmte Evaluation

Effort Evaluation (as a % of Tech Cmte Bandwidth):

  • EP 5.5, CP 1.5, UP 2.5 - Total 9.5 (min useful effort) - although arguably just doing the display behaviors would be a start
  • EP 2.5, CP 1.0, UP 0.0 - Total 3.5
  • Full 13
  • xx% for ...

Candidate Editor:

Kevin O'Donnell
Retrieved from "http://wiki.ihe.net/index.php?title=AI_Results_-_Brief_Proposal&oldid=117213"