Realtime Bidirectional Communication for Interactive Multimedia Reporting - Detailed

From IHE Wiki
Jump to navigation Jump to search

1. Proposed Workitem: Realtime Bidirectional Communication for Interactive Multimedia Reporting

  • Proposal Editor: Kinson Ho, David Vining, Peter O'Toole, Seth Berkowitz, David Kwan
  • Editor: Kinson Ho
  • Domain: Radiology


Standardized realtime bidirectional communication protocols do not exist to allow for rapid information flow between systems in order to support advanced use cases in the healthcare enterprise.

FHIRcast is an emerging standard that facilitates a general publication / subscription messaging mechanism. It defines the messaging protocol as well as the message payload.

With realtime bidirectional communication, data and/or commands can be conveyed between system to facilitate advanced integration. For example, an Interactive Multimedia Reporting (IMR) extension might require an Image Display to convey measurements made in realtime to a Report Creator so that measurements can be automatically displayed in context while a medical report is being generated.

HIMSS/SIIM published a whitepaper regarding the importance of IMR and a critical component was identified as being realtime bidirectional communication. HIMSS/SIIM IMR workgroup participants from multiple disciplines expressed strong interest pursuing this aim.

Several proprietary methods for point-to-point integration exist, but a standardized method for bidirectional communication is a fundamental unmet need that would be optimally orchestrated by the IHE. Creating a standards-based mechanism will promote interoperability and accelerate adoption of more advanced integrated workflows.

2. The Problem

As the health enterprise becomes more sophisticated with the use of multiple systems (i.e. EMR, PACS, reporting systems, artificial intelligence analysis, etc.) requiring information flow between each other to enable advanced integrated use cases, there is a growing need to support realtime bidirectional communication between systems. Solving this challenge has the potential to improve the workflow efficiency of radiologists and promote patient safety by eliminating human transfer of data by manual or speech means that is time consuming and susceptible to error.

3. Key Use Case

  1. Use Case 1: Interactive Multimedia Reporting

A key element for interactive multimedia reporting (IMR) is the ability to include multimedia content (e.g. links to images, tables/graphs of measurement data, etc.) in a radiology report. Traditionally, these annotations, markups, presentation states, and key images have been captured as DICOM objects (e.g. GSPS, SR, or KOS). These objects are designed to capture static evidence for long-term reference instead of real-time communication or composition. Most PACS will create DICOM objects at the end of a reporting session in order to record metrics and annotations created by an image-centric specialist within a single object, rather than create multiple unique evidence objects for each metric or annotation. These DICOM evidence objects are resources for subsequent retrieval when viewing an IMR, but not good payload candidates for real-time communication during a reporting session. As the image-centric specialist generates measurements, regions of interest, and other annotation data, the PACS should provide those data to the reporting system in real-time without unnecessary delays/interruptions or adding transitory content to the permanent record.

4. Standards and Systems


  • PACS (Image Manager and Image Display)
  • Reporting System (Report Creator)
  • AI Manager (Evidence Creator)


5. Technical Approach


New Actors:

  • Data Publisher
  • Data Subscriber
  • Data Hub

These are actors that provide the messaging mechanism. These actors are meant to be grouped with other actors that provides the message content.

Existing actors:

  • For the IMR use case
    • Report Creator is the Data Subscriber
    • Image Display / Evidence Creator is the Data Publisher


New transactions:

  • [RAD-X1] Subscribe Event - FHIRcast
  • [RAD-X2] Publish Event - FHIRcast
  • [RAD-X3] Notify Event - FHIRcast
  • [RAD-X4] Unsubscribe Event - FHIRcast

Content Modules:

  • IMR Measurements and image references - FHIR Observation

Out of Scope

  • [RAD-X5] Launch Application - OAuth2 / SMART on FHIR


  • New profile for RTBC-IMR

Note that once RAD-X1 through RAD-X5 are defined, new message payload can be defined to accommodate other use cases such as IID context switching.


  • IHE IID is currently blocked by oustanding open CP regarding not having a method to open or close the context of the viewer. This can be addressed by this proposal and move IID to final text
  • IHE is the optimal venue because realtime bidirectional communication is a common unmet need and currently there are only proprietary methods for point-to-point integration.
  • FHIRcast has been successfully tested at recent FHIR Connectathons, and there is a growing rate of adoption for new applications (e.g. one major reporting vendor is already using FHIRcast as its new API integrating with PACS).
  • FHIRcast has already defined a number of events (e.g. Patient-open/close, ImagingStudy-open/close) which are immediately applicable to the use cases highlighted above.
  • (Out of Scope) Adopting FHIRcast and SMART on FHIR enables new secure application integration and communication beyond traditional DICOM and HL7.
  • (Out of Scope) FHIRcast recommends SMART on FHIR as a method to convey the topic from the App Launcher to the Application. This should be an optional method as not all integrations leverage SMART on FHIR or OAuth2.

Task Breakdowns

  • Describe the use case for realtime bidirectional communication in the context of reporting between the PACS and Reporting System
  • Define the new transactions
  • Define the message content (e.g. using FHIR Observation) for IMR use case

Note: This profile can be done using either the traditional Word document or the FHIR IG Publisher. FHIRcast definition is not directly supported by the IG Publisher, thus the majority of the specification will be composed in regular text.

6. Support & Resources

HIMSS / SIIM IMR workgroup has been an ardent supporter of the IMR profile and this extension. Many of the active participants (e.g. David Vining and Peter O'Toole) are interested in helping the development of the profile, from the clinical expert perspective as well as vendor / implementation perspective.

7. Risks

  • HL7 FHIR is also developing a new SMART Web Messaging protocol which is aiming at browser applications. However, this is less mature than FHIRcast and is limited to browser applications.
  • Realtime Bidirectional Communication is a cross-cutting concern that has broad applications. One may argue that it is better to have a profile in the ITI domain first and then Radiology can adopt it by defining content modules. The counter-argument is that the need in Radiology is current with many rapid data exchange integration use cases exist in imaging.

8. Tech Cmte Evaluation

Effort Evaluation (as a % of Tech Cmte Bandwidth):

  • 40% for MUE
  • 50% for MUE + optional


Kinson Ho


  • David Vining (MD Anderson / VisionSR)
  • Peter O'Toole (mTuitive)
  • Seth Berkowitz, MD (BIDMC)