Difference between revisions of "XDS Implementation Notes"

From IHE Wiki
Jump to navigation Jump to search
Line 176: Line 176:
 
# Base 64 encoding IS appropriate (may be generated by the Document Source, must be accepted by the Document Repository) for the Provide and Register transaction (XDS.a and XDS.b).
 
# Base 64 encoding IS appropriate (may be generated by the Document Source, must be accepted by the Document Repository) for the Provide and Register transaction (XDS.a and XDS.b).
 
#Base 64 encoding IS NOT allowed in the Retrieve Document (HTTP GET) transaction.  
 
#Base 64 encoding IS NOT allowed in the Retrieve Document (HTTP GET) transaction.  
 +
 +
== Provide and Register content type ==
 +
 +
Updated: [[User:BillM|bill]] 13:20, 15 October 2009 (UTC)
 +
 +
Should the HTTP header Content-Type match the XDSDocumentEntry.mimeType attribute?
 +
 +
Each document in a Provide and Register transaction (and a Retrieve Document Set Response) is packaged in a separate Multipart Part which carries a Content-Type header.  One would expect this header to always match the XDSDocumentEntry.mimeType attribute.  This turns out to not always be true.
 +
 +
Frequently the message encoder building the Part gets mime types it does not understand.  In this case it is allowed to user application/octet-stream, a kind of super class for Content-Type value.  This frequently comes up with some software packages when they encode PDF files for transport.  One would expect to see application/pdf but instead you get application/octet-stream. This is allowable by the standards.
 +
 +
To properly decode the document byte stream, a Document Repository or Document Recipient should rely on the XDSDocumentEntry.mimeType attribute when it sees application/octet-stream in the Content-Type header.
  
 
==See Also==
 
==See Also==

Revision as of 12:24, 15 October 2009


This page documents implementation notes for general XDS, the issues that are shared by many or all XD* profiles (XDS/XDR/XCA).

What are Format Codes?

Updated: bill 14:10, 23 July 2009 (UTC)

The movement of documents in XDS/XDR/XCA is based on the Mime infrastructure of standards, more specifically the Mime Multipart format. This format specifies that each part of a multipart has a mime type which of course comes from a controlled vocabulary defined here. With some much content being defined on top of the XML standard, this has the possibility of having all documents in a registry labeled with the mime type text/xml. Document Consumer actor implementations sometimes need to differentiate content based on format. The XDSDocumentEntry.formatCode attribute was created to carry this information. The values for this attribute come from a vocabulary controlled by the local Affinity Domain (in XDS). Management of this attribute in XCA and XDR is undefined at this time. All IHE defined document formats are assigned formatCodes in the content profile documentation. It is expected that vendor products and local communities will define their own formatCodes to support the documents they manage. Instead of giving formatCodes a simple name which could conflict with the work of others in the future, the code value should be a unique OID value or name-spaced name instead. OIDs are preferable.

What are UUIDs?

Updated: bill 15:57, 22 July 2009 (UTC)

UUIDs are Universally Unique Identifiers defined by RFC 4122. Furthermore, values 10 through 15 shall be formatted in hexadecimal using lower case 'a'-'f'. An example of a properly formatted UUID is

urn:uuid:10b545ea-725c-446d-9b95-8aeb444eddf3

Ids in XDS

Updated: bill 15:57, 22 July 2009 (UTC)

This could have been called Object Identity in XDS.

XDS, according to its ebXML Registry heritage, assigns id attributes to all major and some minor objects in their XML representation. The objects used in XDS that have id attributes are ExtrinsicObject (DocumentEntry), RegistryPackage (Submission Set and Folder), Association, ExternalIdentifier, and Classification. Other minor objects like Name and Slot do not have id attributes according to the ebRIM Schema.

XDS/ebRIM specifies two kinds of ids: UUIDs and symbolic ids. UUIDs are permanent global identifiers that create identity for ebRIM objects. An id value that has the prefix urn:uuid: is assumed to be a UUID. An id value that has this prefix but is not otherwise valid (according to the RFC) is an error. Objects are assigned UUIDs for their id attributes before being stored in the Document Registry.

Symbolic ids are used in the same places as UUIDs but only for temporary use within a fixed context. An example is a Provide and Register Document Set transaction. An ExtrinsicObject (DocumentEntry) may have a symbolic id value for its id attribute. All references to that object within the submission use that symbolic id value. But, this value has no meaning outside the context of the submission. A reference to an object already in the Document Registry must use a UUID since that is global and permanent. An example of this kind of reference would be submitting an Association that adds a DocumentEntry to an existing Folder. The existing Folder, already in the registry, would have a permanent UUID value for its id attribute which would be coded into the new Association object.

In a submission in XDS, three different actors may assign UUIDs: Document Source, Document Repository, and Document Registry. The Document Registry must assign UUIDs to all remaining symbolic ids before saving the metadata. When symbolic ids are converted or compiled into UUIDs, this must happen in a consistent way. A DocumentEntry/ExtrinsicObject with a symbolic id gets a new UUID and all Associations (and other objects) in the submission that reference the DocumentEntry must be updated so that all copies of the original symbolic id are updated to have the new UUID.

Document Source bound to Document Consumer

Updated: bill 15:57, 22 July 2009 (UTC)

Actor binding is a common specification technique in IHE to indicate that a piece of software (or hardware) implements more than one actor and the actors are connected through the implementation giving the implementation benefits of both. This is usually discussed where each of the individual actors could be implemented separately but greater functionality is possible when implemented together. A good example of this in XDS is the binding of a Document Source to a Document Consumer.

By itself, a Document Source can compose and submit documents and metadata via the Provide and Register transaction via either the XDS or XDR profile. A Document Consumer can query the Document Registry for metadata and retrieve document contents from a Document Repository.

The binding of a Document Source with a Document Consumer gives the Document Source the ability to query the Document Registry for metadata. There are advanced types of submissions that require queries to formulate submissions. The two most common examples are to add a new document to an existing folder and the submission of a replacement document. In each case, the submission references some metadata objects already in the registry. Query is the typical way to discover their identity.

Document Source not bound to Document Consumer

Updated: bill 15:57, 22 July 2009 (UTC)

This is of course the opposite of having the Document Source bound to Document Consumer. The only way to perform Complex metadata operations is to pre-assign ids.

Edge system

Updated: bill 15:57, 22 July 2009 (UTC)

An edge system in XDS is an implementation of the Document Source or Document Consumer actors, actors implemented separate from the common infrastructure built up from Document Registry and Document Repository actors.

Complex metadata operations

Updated: bill 15:57, 22 July 2009 (UTC)

While there is no formal definition of a complex metadata operation, we use it here to mean a submission (Provide and Register or Register transaction) containing Association objects that link to existing registry contents. Examples are folder additions and document replacements. In these cases the Document Source must know the UUID of specific objects in the registry so that Association objects can be built.

There are two ways for a Document Source to know these ids. Either it submitted the original objects, pre-assigned the ids, and saved them for future use or it used a query to discover the ids from the registry.

Document Source assigning IDs

Updated: bill 15:57, 22 July 2009 (UTC)

Why should a Document Source want to pre-assign UUIDs to metdata? In the most commonly discussed use case, where an edge system implements both Document Source and Document Consumer actors, there is no value. The Document Registry actor will take care of this chore. The most common reason to pre-assign UUIDs to id attributes in the Document Source is that the Document Source wishes to perform more complex metadata operations without being bound to a Document Consumer. Without this actor binding, which enables queries, the Document Source must already know the ids of the objects in the registry it wishes to reference. The only way to know these ids (without using queries) is if the Document Source submitted the original objects, pre-assigned the ids, and saved them for future use.

This behavior in a Document Source is not without risk. There are no rules in XDS restricting a Document Source from submitting metadata that links to metadata from another Document Source. The troubling scenario is Doc Src A submits a document. Doc Src B replaces the document with another. Doc Src A attempts to replace its original document. This operation fails since Doc Src A's original document has been deprecated (replacement operation is a submission of new document; linked to an original document; and labeling the original as deprecated). The XDS profile (or any other IHE profile) does not give any advice or restrictions to help this scenario. If a Document Source behaves in this way it is either taking a chance or it knows more about the operation of the local environment than the base XDS profile.

There is only one UUID per UID, Right?

Updated: bill 15:57, 15 Sept 2009 (UTC)

No. It is possible to have multiple UUIDs for a single UID (UID is short hand for XDSDocumentEntry.uniqueId) . This is the same as saying you have multiple DocumentEntry objects in the Registry for a single document (as defined by a unique hash) in one or more Repositories. The multiple submission of the document contents to a Repository MAY reuse the UID. A Document can be submitted multiple times. The same UID can be used as long as the hash matches exactly. This can play out in different ways. In each case it is assumed that the same Document Registry is involved:

  • Document submitted twice to same Repository. Each DocumentEntry is assigned a different UUID (required) but since the hash is identical (same document contents) the same UID is (can be) used. The Document Repository stores two copies of document.
  • Same scenario but Repository only stores one copy of document (cannot tell the difference from the interface, just an implementation detail).
  • Document submitted to Repository A and later to Repository B. Each Repository stores a copy of the document contents.

But, every DocumentEntry object in the Registry (ExtrinsicObject) must have a unique UUID. In all the above cases, separate submissions of a single document, with a single UID, result in different UUIDs.

One way this scenario can happen is if documents are shared outside the XDS environment in a format that carries along the UID. Each user of the document could submit it as supporting material along with their other work.

Lifecycle operations when a document is submitted multiple times

Updated: bill 15:57, 15 Sept 2009 (UTC)

Since a document, as identified by its UID (UID is short hand for XDSDocumentEntry.uniqueId), can be registered multiple times with each registration generating a DocumentEntry with a unique UUID, XDS has a functionality gap in handling Lifecycle operations. Looking at Document Replacement. Two submissions of the same document, each time the document carries the same UID but different UUID. Later someone issues a document replacement. If they are real careful they will notice that there are two DocumentEntry object (UUIDs) for this document (both with the same UID) and perform the replace on both DocumentEntry objects in the Registry. But, maybe they don't notice or don't care. At this time XDS does not require a replace operation to operate on all copies of the document metadata. A later query leading to the display of the document for clinical use does not notice both copies of the document metadata. Does it display the correct document? Again, this is beyond the scope of XDS at this time.

All this leads to the suggestion that developers focus their Query and Lifecycle Management implementations on document UIDs (XDSDocumentEntry.unqiueId) instead of UUIDs (XDSDocumentEntry.entryUuid).

There is a harder to detect situation that can occur when the above scenario plays out but the duplicate submission does not carry the same UID. While it may be possible to detect, it is much harder than noticing multiple documents for a patient with the same UID. One would have to scan all documents for a patient and notice multiple documents with the same hash. Again, this is way beyond the requirements of the profile.

Folder Association - RPLC Interaction

Updated: bill 15:57, 22 July 2009 (UTC)

The following two rules interact:

  1. When a Document Source requests that a Document be replaced and that Document is present in one or more Folders, the replacement Document is automatically added to the Folders as a side-effect by the Document Registry.
  2. The Association between Folder and Document is annotated with a secondary Association that links it to the Submission Set thus documenting which submission act linked the Folder and Document.

These two rules interact within the Registry implementation as described in the following flow:

  • Registry detects Document Replacement (RPLC Association) in submission, submission must contain replacement document
  • Original Document is member of one or more Folders
  • Registry generates new HasMember Association between Folder and replacement Document based on rule #1 above
  • Registry also generates new HasMember Association between Submission Set (must be present in submission) and the above new Association based on rule #2 above

A naive observer, using the GetSubmissionSetAndContents Stored Query might conclude that the Document Source constructed the entire Submission Set as present in the registry.


Just a reminder...
A Document is added to a Folder by creating a HasMember Association between the Folder and the Document. This can happen at several potential times:

  • When the Folder is created
  • When the Document is created
  • When the Folder and Document are both created together
  • After the Document and Folder are created

Metadata Coding

Updated: bill 15:57, 22 July 2009 (UTC)

There are many nuances to coding XD* metadata and unfortunately the standard does not explain them in detail. Much of the detail in the Old XDS FAQ deals with such coding issues. Newer help is documented in this section. Make sure to check the XDS.a and XDS.b specific Implementation Notes pages for metadata version specific discussions.

Classifying RegistryPackage objects as Submission Set vs. Folder

Both Submission Set and Folder objects are based on the ebRIM RegistryPackage element. There are two ways of identifying a RegistryPackage as being a Submission Set vs. a Folder:

  1. Check for a Classification labeling the RegistryPackage object
  2. Identify an attribute within the RegistryPackage that is unique to either Submission Set or Folder. Patient ID or Unique ID are easy choices. They exist within both Submission Set and Folder objects but have different identificationScheme attributes which can be used to differentiate them.

While differentiating based on Classification is the technically correct approach, it is slightly harder than it should be since ebRIM 2.1/3.0 Schema allows the Classification object to be coded inside the RegistryPackage object or outside. Both of the following metadata snippets represent valid metadata coding:

Classification within RegistryPackage

<…>
    <RegistryPackage id="SS">
        <!-- Classify registry package SS as being an XDSSubmissionSet -->
        <Classification classificationNode="urn:uuid:a54d6aa5-d40d-43f9-88c5-b4633d873bdd" classifiedObject="SS"/>
    </RegistryPackage>
    <RegistryPackage id="Fol">
        <!-- Classify registry package Fol as being an XDSFolder -->
        <Classification classificationNode="urn:uuid:d9d542f3-6cc4-48b6-8870-ea235fbc94c2" classifiedObject="Fol"/>
    </RegistryPackage>
<…>

Classification outside RegistryPackage

<…>
    <RegistryPackage id="SS"/>
    <RegistryPackage id="Fol"/>
    <!-- Classify registry packages as XDSSubmissionSet/XDSFolder respectively -->
    <Classification classificationNode="urn:uuid:a54d6aa5-d40d-43f9-88c5-b4633d873bdd" classifiedObject="SS"/>
    <Classification classificationNode="urn:uuid:d9d542f3-6cc4-48b6-8870-ea235fbc94c2" classifiedObject="Fol"/>
<…>

All Transactions

Updated: bill 15:57, 22 July 2009 (UTC)

Duplicate Metadata

Document Source(s) can submit a document multiple times. This results in a single value of XDSDocumentEntry.uniqueId being present on multiple XDSDocumentEntry objects in the Registry. As long as the size and hash attributes are the same, this is considered a normal condition that a Document Consumer must be prepared for. This condition can occur for two other reasons. First a Provide and Register transaction can fail because the response message from the Repository back to the Document Source is not delivered. This can happen on a synchronous connection as well as with asynchronous web services as described in the new Async XDS Supplement. A natural response for the Document Source is to resubmit, again causing duplication.

A document replace (RPLC from LifeCycle Management option to XDS) has the known issue that a Document Source replacing a document is not required to find all copies of the document in the Registry. A replacement will be applied to only one copy of the document. Likewise the new topic of Metadata Versioning (currently a white paper) does not carry the requirement to find all copies of the document metadata.

Old XDS FAQ

While new material is to be added to this page, there is older material available from the original XDS FAQ

Use of base64 encoding

The use of base64 encoding in XDS.a has been poorly understood for a long time. After a review of relevant standards we have found that the relevant standards clearly specify what is acceptable.

  1. Base 64 encoding IS appropriate (may be generated by the Document Source, must be accepted by the Document Repository) for the Provide and Register transaction (XDS.a and XDS.b).
  2. Base 64 encoding IS NOT allowed in the Retrieve Document (HTTP GET) transaction.

Provide and Register content type

Updated: bill 13:20, 15 October 2009 (UTC)

Should the HTTP header Content-Type match the XDSDocumentEntry.mimeType attribute?

Each document in a Provide and Register transaction (and a Retrieve Document Set Response) is packaged in a separate Multipart Part which carries a Content-Type header. One would expect this header to always match the XDSDocumentEntry.mimeType attribute. This turns out to not always be true.

Frequently the message encoder building the Part gets mime types it does not understand. In this case it is allowed to user application/octet-stream, a kind of super class for Content-Type value. This frequently comes up with some software packages when they encode PDF files for transport. One would expect to see application/pdf but instead you get application/octet-stream. This is allowable by the standards.

To properly decode the document byte stream, a Document Repository or Document Recipient should rely on the XDSDocumentEntry.mimeType attribute when it sees application/octet-stream in the Content-Type header.

See Also

Updated: bill 15:57, 22 July 2009 (UTC)

The IT Infrastructure Domain manages this profile.

The ITI Technical Framework is the official master document for this Profile.