Difference between revisions of "Image Manager/Archive Content Migration Profile - Brief Proposal"
| Line 16: | Line 16: | ||
In practice this is often a difficult, time-consuming and expensive process, since the internal storage format of an IA is not standardized, and the extent to which it is filesystem-based and/or database-based varies between implementations, particularly with respect to how corrections that may have been made to the originally acquired data (e.g., patient identity corrections, study splits and merges, etc.) are stored. Nor is it necessarily desirable to standardize the "internal" format, since it may be optimized to satisfy the primary use-cases for the normal clinical use of the archive. | In practice this is often a difficult, time-consuming and expensive process, since the internal storage format of an IA is not standardized, and the extent to which it is filesystem-based and/or database-based varies between implementations, particularly with respect to how corrections that may have been made to the originally acquired data (e.g., patient identity corrections, study splits and merges, etc.) are stored. Nor is it necessarily desirable to standardize the "internal" format, since it may be optimized to satisfy the primary use-cases for the normal clinical use of the archive. | ||
| − | Though theoretically images and information in an IM/IA are accessible in an updated form through the standard DICOM query/retrieve interface, such an interface is often not optimized for migrating an entire archive in this manner and hence too slow, and doing so may degrade concurrent clinical performance when the archive cannot be taken out of service to perform a large migration. Further, when the old and the new archive are not physically co-located, the speed of migration may be severely constrained by available bandwidth. | + | Though theoretically images and information in an IM/IA are accessible in an updated form through the standard DICOM query/retrieve (Q/R) interface, such an interface is often not optimized for migrating an entire archive in this manner and hence too slow, and doing so may degrade concurrent clinical performance when the archive cannot be taken out of service to perform a large migration. Further, when the old and the new archive are not physically co-located, the speed of migration may be severely constrained by available bandwidth. The Q/R interface also does not expose the history of any changes. |
One solution, the subject of this proposal, is to define transactions for creation and importation of a complete replica of the entire content of an archive on off-line media (e.g., a low cost, high speed, transportable, reusable or disposable, external disk array) such that it can be created, detached, physically transported if necessary, then imported into a new archive, potentially including a history of all changes. | One solution, the subject of this proposal, is to define transactions for creation and importation of a complete replica of the entire content of an archive on off-line media (e.g., a low cost, high speed, transportable, reusable or disposable, external disk array) such that it can be created, detached, physically transported if necessary, then imported into a new archive, potentially including a history of all changes. | ||
Revision as of 09:39, 11 September 2011
1. Proposed Workitem: Image Manager/Archive Content Migration Profile
- Proposal Editor: David Clunie
- Editor: David Clunie
- Date: N/A (Wiki keeps history)
- Version: N/A (Wiki keeps history)
- Domain: Radiology
2. The Problem
Image Archives (and their associated Image Managers) need to be regularly replaced due to technical obsolescence and other factors, yet their content (images and associated information) needs to be available in the replacement implementation.
In practice this is often a difficult, time-consuming and expensive process, since the internal storage format of an IA is not standardized, and the extent to which it is filesystem-based and/or database-based varies between implementations, particularly with respect to how corrections that may have been made to the originally acquired data (e.g., patient identity corrections, study splits and merges, etc.) are stored. Nor is it necessarily desirable to standardize the "internal" format, since it may be optimized to satisfy the primary use-cases for the normal clinical use of the archive.
Though theoretically images and information in an IM/IA are accessible in an updated form through the standard DICOM query/retrieve (Q/R) interface, such an interface is often not optimized for migrating an entire archive in this manner and hence too slow, and doing so may degrade concurrent clinical performance when the archive cannot be taken out of service to perform a large migration. Further, when the old and the new archive are not physically co-located, the speed of migration may be severely constrained by available bandwidth. The Q/R interface also does not expose the history of any changes.
One solution, the subject of this proposal, is to define transactions for creation and importation of a complete replica of the entire content of an archive on off-line media (e.g., a low cost, high speed, transportable, reusable or disposable, external disk array) such that it can be created, detached, physically transported if necessary, then imported into a new archive, potentially including a history of all changes.
3. Key Use Case
- A single facility wants to replace its old IM/IA with a new IM/IA, preserving all images for all exams and all subjects with all corrections/merges/splits accounted for but without the need to retain a history of changes or the original images, and with all user annotations and key images included, with all objects in a standard DICOM form; process is to export a "snapshot" of everything at a point in time to a low cost disk array directly attached to the old IM/IA (whilst the old IM/IA is still live), detach it and directly re-attach it to the new IM/IA, import the snapshot; then repeat the process for all changes and additions since the initial (large) snapshot once the old IM/IA is taken offline and then import this (small) delta into the new IM/IA which is then brought live (with minimal downtime, during the delta import/export step only).
- A variant is for a single facility to replace its on site IM/IA with an off-site IM/IA from a service provider, such that the snapshot and deltas require physical transport to an off-site location (or the last small delta is handled over the network).
- A variant is when multiple facilities "merge" (literally or figuratively) and want to combine their local IM/IA archives into a single centrally provided one (or one facility is absorbed by another); this may require intermediate processing of the exported data to reconcile identifiers before import.
- A variant is when it is also required to preserve a record of changes made during corrections/splits/merges and that the originally acquired images also be accessible in the new IM/IA in addition to the up-to-date images, hence the snapshots and the deltas also contain the originals, change objects and replacements (a la IOCM) and the receiving IM/IA applies them appropriately.
- A variant when future migration is anticipated is to proactively export snapshots and deltas on a regular basis, physically storing them or a copy of them (powered down, off-site or on-site, on tape, etc.) until needed.
4. Standards and Systems
IHE PDI, IOCM, CPI, KIN
5. Discussion
IHE is a good venue to address this problem because though what is described is possible by assembling variations of existing standard components, for their to be reliable implementation between the exporter and the importer (and/or any intervening processor), constraints need to be specified on the content and completeness of the snapshots and the deltas (including which SOP Classes and Transfer Syntaxes are used), and hence a profile is appropriate. Further, users and product managers need a profile that clearly defines the functionality of the "feature".
Scope of content needs to be defined (e.g., what about reports, and what about DSS/OF "content").
Between export and import into the new archive, additional "cleanup" could be performed (e.g., by a third-party service provider or tool), for example, to transform one coding scheme to another (e.g., for identifiers or procedure codes) or strings relevant to new or different features in the new image manager or image displays (e.g., for hanging protocols).
Security may be a concern if the disk array is to be physically transported, and encryption of the data would likely be required.
The granularity of the organization of the exported fileset into one or more volumes, and the need for a manifest (like a DICOMDIR) for confirming completeness and integrity (+/- checksums, hashes, etc.) needs to be considered.
This profile is NOT intended to address the use-case of a so-called "vendor neutral archive" (VNA), since a VNA is typically described as being a live (active) resource with the additional cost and complexity that implies, whereas what is proposed, though by definition "vendor neutral" since it contains standard content in a standard format, is intended to be an offline resource, typically discarded once migration has been completed.
This profile is also not intended to address off-site backup, disaster recovery and business continuity use-cases, and though perhaps it could, those use-cases potentially introduce other requirements that are out of scope (such as longevity and durability and accessibility of media).