XDS Implementation Notes
This page documents implementation notes for general XDS, the issues that are shared by many or all XD* profiles (XDS/XDR/XCA).
I jokingly call it my blog because I tend to be the only one to add updates. (bill)
Metadata Update initial version number
Updated: bill 22:52, 13 December 2011 (UTC)
Q: What is the initial version of the Document when RegisterDocumentSet transaction takes place? In the Supplement for Metadata Update specification, the version starts with 1. Whereas, the schema is default this value to 1.1 in rim.xsd. So the question is there a set version to start with or is it depending on the Registry's implementor? Or the default value in the schema is not even considered?
A: If the Metadata Update option is supported by the Document Registry then the initial version is 1 per the Metadata Update supplement. If the Document Registry does not support the Metadata Update option then the version defaults to the schema default version of 1.1.
mustUnderstand
Updated: bill 22:52, 12 December 2011 (UTC)
The WS:Addressing spec says that if WS:Addressing headers are included then they must be labeled as mustUnderstand and the receiver must reject the message if they cannot process these headers. This basically means that if they are used, the WS:Addressing contain critical addressing information. In the WS world you specify this by adding a header mustUnderstand="1" or mustUnderstand="yes" to one of the elements in the WS:Addressing namespace. The base spec says either form is acceptable and the quoted (but the TF) WS-I spec specifies which for is to be used (don't remember off hand which one was selected).
That being said, any message carrying WS:Addressing headers (any IHE transaction referencing Appendix V) that does not include mustUnderstand does not meet the standard. Notice I didn't say profile. We are simply quoting the WS standards here.
IHE ITI Appendix V goes on to constrain this by saying exactly which WS:Addressing elements the mustUnderstand must be present on. This IHE requirement will be removed in the future. We did not intend to add any constraints to the underlying standards.
Now for the thorny part. Many of the toolkits do not properly implement the WS mustUnderstand feature. To translate this to IHE testing:
1) If an actor accepts an Appendix V compliant message (which is every message in the XD* world) that does not contain mustUnderstand=1 on at least one WS:Addressing header then that actor implementation is incorrect.
2) If an actor sends the same then it is incorrect.
In IHE testing we have overlooked a lot of this for several years for two reasons:
1) Awkward to test
2) Hoping the technology vendors would catch up with the standards.
We will soon start testing for this and failing systems in Pre-Connectathon and Connectathon for not being compliant with the standards. This will not happen in January. Now is the time to inspect your own systems and complain to your technology vendor if this feature of the WS standards is not handled properly.
Now looking at the Public Registry. It is based on the Apache Axis2 libraries. If you label a header as mustUnderstand and I don't have the proper module loaded then it will reject the message with a SOAP Fault. But, it doesn't fail a message that has mustUnderstand missing when required by the WS standards. I have left it this way since I knew many implementations could not yet do the right thing. Between this testing season and next I will be adding the checks. The result will be that when your actor implementation interacts with the Public Registry and doesn't properly handle the WS:Addressing mustUnderstand requirements, the test will fail. Of course that will feed into the Pre-Connectathon testing for next year.
Metadata Update and SubmitObjectsRequest
Updated: bill 22:52, 1 December 2011 (UTC)
Q: In the Metadata Update profile spec the request type for UpdateDocumentSet is specified as "lcm:SubmitObjectsRequest" ( and in examples given )
But if you look into the lcm.xsd you would find below comment UpdateObjectsRequest.
A: UpdateObjectsRequest supports the updating of object only. SubmitObjectsRequest supports submitting and updating. Technically, an XDS update is a combined submission and update. Consider the simplest update, a new version of a DocumentEntry:
SubmissionSet (new submission) DocumentEntry (update) HasMember Association (new submission)
If we used the UpdateObjectsRequest, we would not be able to include the new SubmissionSet and Association which would break the XDS info model. The ebRIM info model is expected to be constrained this way.
Metadata Update and Deprecated Folders
Updated: bill 22:52, 30 November 2011 (UTC)
Q: In the context of Metadata Update: step 1: Submitted a document and a folder in one submission. step 2 : Update the folder status to deprecated.
For GetSubmissionset,I use deprecated folder uuid and metadatalevel is 1 Now should I get back the submissionset in the result ?
A: Given that metadatalevel is 1, you are expecting the registry to behave as if it doesn't implement the metadata update supplement. A pre-metadata update Document Registry would never contain a folder with status of Deprecated. The use of Deprecated status on Folders and Associations was first introduced in the Metadata Update supplement. So the question has to be asked in the context of an older registry. The older registry would ignore or not have the Deprecated status and would indeed return the SubmissionSet object.
Problems with WSDL generated interfaces (RPC vs Document based Web Services)
Updated: bill 22:52, 23 August 2011 (UTC)
A typical follow-on to questions about WSDL usually goes something like this:
There seems to be a problem with the XXX profile. It does not document what the parameters are to transaction YYY.
There are two general styles to Web Service requests: RPC-style and Document-style.
With RPC-style there is a method/procedure name and a collection of parameters in the request and the response returns a value. Very RPC-ish. With Document-style the input is an XML document that is typically much more complex that a collection of parameters. The response is another XML document. XD* transactions are based on the Document-style Web Service. RPC-style web services always seem to be more familiar to developers and maybe the use of WSDL leads them to think they are used here. I am not a WSDL hack so I would not know.
All XD* transactions use Document-style web services.
Updated: bill 22:52, 23 August 2011 (UTC)
Officially WSDL is an informative part of IHE profiles, it is written by committee members, used by some, but never is it the final answer to any particular question. It is understood that developers who use some WS toolkits depend on WSDL heavily.
First, the Public Registry testing site answers to endpoint requests ending in ?wsdl and returns WSDL as expected. This WSDL is not that developed by committee. Instead it is generated by the Apache Axis2 web services implementation that the Public Registry implementation is based on. It may be correct or not. If I could easily turn it off I would since no one checks it and no one stands behind it. I have heard good and bad things about Axis2 generated WSDL. If it helps, please use it.
In developing tests for XD* and answering questions about the same, most WSDL questions start something like this: when I run your WSDL though WSDL tool xyz ... There is a big problem here. The question is frequently not about WSDL but instead about a particular tool's reaction to the WSDL. But we are not testing or recommending WDSL tools, it's just a technology like Java or C#. It has nothing to do with interoperability.
The developer that relies on WSDL to generate their interfaces should review the archives of the Google Group IHE XDS Implementers (ihe-xds-implementors@googlegroups.com) where this has been discussed quite a bit.
What are Format Codes?
Updated: bill 14:10, 23 July 2009 (UTC)
The movement of documents in XDS/XDR/XCA is based on the Mime infrastructure of standards, more specifically the Mime Multipart format. This format specifies that each part of a multipart has a mime type which of course comes from a controlled vocabulary defined here. With some much content being defined on top of the XML standard, this has the possibility of having all documents in a registry labeled with the mime type text/xml. Document Consumer actor implementations sometimes need to differentiate content based on format. The XDSDocumentEntry.formatCode attribute was created to carry this information. The values for this attribute come from a vocabulary controlled by the local Affinity Domain (in XDS). Management of this attribute in XCA and XDR is undefined at this time. All IHE defined document formats are assigned formatCodes in the content profile documentation. It is expected that vendor products and local communities will define their own formatCodes to support the documents they manage. Instead of giving formatCodes a simple name which could conflict with the work of others in the future, the code value should be a unique OID value or name-spaced name instead. OIDs are preferable.
What are UUIDs?
Updated: bill 15:57, 22 July 2009 (UTC)
UUIDs are Universally Unique Identifiers defined by RFC 4122. Furthermore, values 10 through 15 shall be formatted in hexadecimal using lower case 'a'-'f'. An example of a properly formatted UUID is
Ids in XDS
Updated: bill 15:57, 22 July 2009 (UTC)
This could have been called Object Identity in XDS.
XDS, according to its ebXML Registry heritage, assigns id attributes to all major and some minor objects in their XML representation. The objects used in XDS that have id attributes are ExtrinsicObject (DocumentEntry), RegistryPackage (Submission Set and Folder), Association, ExternalIdentifier, and Classification. Other minor objects like Name and Slot do not have id attributes according to the ebRIM Schema.
XDS/ebRIM specifies two kinds of ids: UUIDs and symbolic ids. UUIDs are permanent global identifiers that create identity for ebRIM objects. An id value that has the prefix urn:uuid: is assumed to be a UUID. An id value that has this prefix but is not otherwise valid (according to the RFC) is an error. Objects are assigned UUIDs for their id attributes before being stored in the Document Registry.
Symbolic ids are used in the same places as UUIDs but only for temporary use within a fixed context. An example is a Provide and Register Document Set transaction. An ExtrinsicObject (DocumentEntry) may have a symbolic id value for its id attribute. All references to that object within the submission use that symbolic id value. But, this value has no meaning outside the context of the submission. A reference to an object already in the Document Registry must use a UUID since that is global and permanent. An example of this kind of reference would be submitting an Association that adds a DocumentEntry to an existing Folder. The existing Folder, already in the registry, would have a permanent UUID value for its id attribute which would be coded into the new Association object.
In a submission in XDS, three different actors may assign UUIDs: Document Source, Document Repository, and Document Registry. The Document Registry must assign UUIDs to all remaining symbolic ids before saving the metadata. When symbolic ids are converted or compiled into UUIDs, this must happen in a consistent way. A DocumentEntry/ExtrinsicObject with a symbolic id gets a new UUID and all Associations (and other objects) in the submission that reference the DocumentEntry must be updated so that all copies of the original symbolic id are updated to have the new UUID.
Document Source bound to Document Consumer
Updated: bill 15:57, 22 July 2009 (UTC)
Actor binding is a common specification technique in IHE to indicate that a piece of software (or hardware) implements more than one actor and the actors are connected through the implementation giving the implementation benefits of both. This is usually discussed where each of the individual actors could be implemented separately but greater functionality is possible when implemented together. A good example of this in XDS is the binding of a Document Source to a Document Consumer.
By itself, a Document Source can compose and submit documents and metadata via the Provide and Register transaction via either the XDS or XDR profile. A Document Consumer can query the Document Registry for metadata and retrieve document contents from a Document Repository.
The binding of a Document Source with a Document Consumer gives the Document Source the ability to query the Document Registry for metadata. There are advanced types of submissions that require queries to formulate submissions. The two most common examples are to add a new document to an existing folder and the submission of a replacement document. In each case, the submission references some metadata objects already in the registry. Query is the typical way to discover their identity.
Document Source not bound to Document Consumer
Updated: bill 15:57, 22 July 2009 (UTC)
This is of course the opposite of having the Document Source bound to Document Consumer. The only way to perform Complex metadata operations is to pre-assign ids.
Edge system
Updated: bill 15:57, 22 July 2009 (UTC)
An edge system in XDS is an implementation of the Document Source or Document Consumer actors, actors implemented separate from the common infrastructure built up from Document Registry and Document Repository actors.
Complex metadata operations
Updated: bill 15:57, 22 July 2009 (UTC)
While there is no formal definition of a complex metadata operation, we use it here to mean a submission (Provide and Register or Register transaction) containing Association objects that link to existing registry contents. Examples are folder additions and document replacements. In these cases the Document Source must know the UUID of specific objects in the registry so that Association objects can be built.
There are two ways for a Document Source to know these ids. Either it submitted the original objects, pre-assigned the ids, and saved them for future use or it used a query to discover the ids from the registry.
Document Source assigning IDs
Updated: bill 15:57, 22 July 2009 (UTC)
Why should a Document Source want to pre-assign UUIDs to metdata? In the most commonly discussed use case, where an edge system implements both Document Source and Document Consumer actors, there is no value. The Document Registry actor will take care of this chore. The most common reason to pre-assign UUIDs to id attributes in the Document Source is that the Document Source wishes to perform more complex metadata operations without being bound to a Document Consumer. Without this actor binding, which enables queries, the Document Source must already know the ids of the objects in the registry it wishes to reference. The only way to know these ids (without using queries) is if the Document Source submitted the original objects, pre-assigned the ids, and saved them for future use.
This behavior in a Document Source is not without risk. There are no rules in XDS restricting a Document Source from submitting metadata that links to metadata from another Document Source. The troubling scenario is Doc Src A submits a document. Doc Src B replaces the document with another. Doc Src A attempts to replace its original document. This operation fails since Doc Src A's original document has been deprecated (replacement operation is a submission of new document; linked to an original document; and labeling the original as deprecated). The XDS profile (or any other IHE profile) does not give any advice or restrictions to help this scenario. If a Document Source behaves in this way it is either taking a chance or it knows more about the operation of the local environment than the base XDS profile.
There is only one UUID per UID, Right?
Updated: bill 15:57, 15 Sept 2009 (UTC)
No. It is possible to have multiple UUIDs for a single UID (UID is short hand for XDSDocumentEntry.uniqueId) . This is the same as saying you have multiple DocumentEntry objects in the Registry for a single document (as defined by a unique hash) in one or more Repositories. The multiple submission of the document contents to a Repository MAY reuse the UID. A Document can be submitted multiple times. The same UID can be used as long as the hash matches exactly. This can play out in different ways. In each case it is assumed that the same Document Registry is involved:
- Document submitted twice to same Repository. Each DocumentEntry is assigned a different UUID (required) but since the hash is identical (same document contents) the same UID is (can be) used. The Document Repository stores two copies of document.
- Same scenario but Repository only stores one copy of document (cannot tell the difference from the interface, just an implementation detail).
- Document submitted to Repository A and later to Repository B. Each Repository stores a copy of the document contents.
But, every DocumentEntry object in the Registry (ExtrinsicObject) must have a unique UUID. In all the above cases, separate submissions of a single document, with a single UID, result in different UUIDs.
One way this scenario can happen is if documents are shared outside the XDS environment in a format that carries along the UID. Each user of the document could submit it as supporting material along with their other work.
Lifecycle operations when a document is submitted multiple times
Updated: bill 15:57, 15 Sept 2009 (UTC)
Since a document, as identified by its UID (UID is short hand for XDSDocumentEntry.uniqueId), can be registered multiple times with each registration generating a DocumentEntry with a unique UUID, XDS has a functionality gap in handling Lifecycle operations. Looking at Document Replacement. Two submissions of the same document, each time the document carries the same UID but different UUID. Later someone issues a document replacement. If they are real careful they will notice that there are two DocumentEntry object (UUIDs) for this document (both with the same UID) and perform the replace on both DocumentEntry objects in the Registry. But, maybe they don't notice or don't care. At this time XDS does not require a replace operation to operate on all copies of the document metadata. A later query leading to the display of the document for clinical use does not notice both copies of the document metadata. Does it display the correct document? Again, this is beyond the scope of XDS at this time.
All this leads to the suggestion that developers focus their Query and Lifecycle Management implementations on document UIDs (XDSDocumentEntry.unqiueId) instead of UUIDs (XDSDocumentEntry.entryUuid).
There is a harder to detect situation that can occur when the above scenario plays out but the duplicate submission does not carry the same UID. While it may be possible to detect, it is much harder than noticing multiple documents for a patient with the same UID. One would have to scan all documents for a patient and notice multiple documents with the same hash. Again, this is way beyond the requirements of the profile.
Folder Association - RPLC Interaction
Updated: bill 15:57, 22 July 2009 (UTC)
The following two rules interact:
- When a Document Source requests that a Document be replaced and that Document is present in one or more Folders, the replacement Document is automatically added to the Folders as a side-effect by the Document Registry.
- The Association between Folder and Document is annotated with a secondary Association that links it to the Submission Set thus documenting which submission act linked the Folder and Document.
These two rules interact within the Registry implementation as described in the following flow:
- Registry detects Document Replacement (RPLC Association) in submission, submission must contain replacement document
- Original Document is member of one or more Folders
- Registry generates new HasMember Association between Folder and replacement Document based on rule #1 above
- Registry also generates new HasMember Association between Submission Set (must be present in submission) and the above new Association based on rule #2 above
A naive observer, using the GetSubmissionSetAndContents Stored Query might conclude that the Document Source constructed the entire Submission Set as present in the registry.
Just a reminder...
A Document is added to a Folder by creating a HasMember Association between the Folder and the Document. This can happen at several potential times:
- When the Folder is created
- When the Document is created
- When the Folder and Document are both created together
- After the Document and Folder are created
Metadata Coding
Updated: bill 15:57, 22 July 2009 (UTC)
There are many nuances to coding XD* metadata and unfortunately the standard does not explain them in detail. Much of the detail in the Old XDS FAQ deals with such coding issues. Newer help is documented in this section. Make sure to check the XDS.a and XDS.b specific Implementation Notes pages for metadata version specific discussions.
Classifying RegistryPackage objects as Submission Set vs. Folder
Both Submission Set and Folder objects are based on the ebRIM RegistryPackage element. There are two ways of identifying a RegistryPackage as being a Submission Set vs. a Folder:
- Check for a Classification labeling the RegistryPackage object
- Identify an attribute within the RegistryPackage that is unique to either Submission Set or Folder. Patient ID or Unique ID are easy choices. They exist within both Submission Set and Folder objects but have different identificationScheme attributes which can be used to differentiate them.
While differentiating based on Classification is the technically correct approach, it is slightly harder than it should be since ebRIM 2.1/3.0 Schema allows the Classification object to be coded inside the RegistryPackage object or outside. Both of the following metadata snippets represent valid metadata coding:
Classification within RegistryPackage
<…>
<RegistryPackage id="SS">
<!-- Classify registry package SS as being an XDSSubmissionSet -->
<Classification classificationNode="urn:uuid:a54d6aa5-d40d-43f9-88c5-b4633d873bdd" classifiedObject="SS"/>
</RegistryPackage>
<RegistryPackage id="Fol">
<!-- Classify registry package Fol as being an XDSFolder -->
<Classification classificationNode="urn:uuid:d9d542f3-6cc4-48b6-8870-ea235fbc94c2" classifiedObject="Fol"/>
</RegistryPackage>
<…>
Classification outside RegistryPackage
<…>
<RegistryPackage id="SS"/>
<RegistryPackage id="Fol"/>
<!-- Classify registry packages as XDSSubmissionSet/XDSFolder respectively -->
<Classification classificationNode="urn:uuid:a54d6aa5-d40d-43f9-88c5-b4633d873bdd" classifiedObject="SS"/>
<Classification classificationNode="urn:uuid:d9d542f3-6cc4-48b6-8870-ea235fbc94c2" classifiedObject="Fol"/>
<…>
All Transactions
Updated: bill 15:57, 22 July 2009 (UTC)
Duplicate Metadata
Document Source(s) can submit a document multiple times. This results in a single value of XDSDocumentEntry.uniqueId being present on multiple XDSDocumentEntry objects in the Registry. As long as the size and hash attributes are the same, this is considered a normal condition that a Document Consumer must be prepared for. This condition can occur for two other reasons. First a Provide and Register transaction can fail because the response message from the Repository back to the Document Source is not delivered. This can happen on a synchronous connection as well as with asynchronous web services as described in the new Async XDS Supplement. A natural response for the Document Source is to resubmit, again causing duplication.
A document replace (RPLC from LifeCycle Management option to XDS) has the known issue that a Document Source replacing a document is not required to find all copies of the document in the Registry. A replacement will be applied to only one copy of the document. Likewise the new topic of Metadata Versioning (currently a white paper) does not carry the requirement to find all copies of the document metadata.
Old XDS FAQ
While new material is to be added to this page, there is older material available from the original XDS FAQ
Use of base64 encoding
The use of base64 encoding in XDS.a has been poorly understood for a long time. After a review of relevant standards we have found that the relevant standards clearly specify what is acceptable.
- Base 64 encoding IS appropriate (may be generated by the Document Source, must be accepted by the Document Repository) for the Provide and Register transaction (XDS.a and XDS.b).
- Base 64 encoding IS NOT allowed in the Retrieve Document (HTTP GET) transaction.
Provide and Register content type
Updated: bill 13:20, 15 October 2009 (UTC)
Should the HTTP header Content-Type match the XDSDocumentEntry.mimeType attribute?
Each document in a Provide and Register transaction (and a Retrieve Document Set Response) is packaged in a separate Multipart Part which carries a Content-Type header. One would expect this header to always match the XDSDocumentEntry.mimeType attribute. This turns out to not always be true.
Frequently the message encoder building the Part gets mime types it does not understand. In this case it is allowed to user application/octet-stream, a kind of super class for Content-Type value. This frequently comes up with some software packages when they encode PDF files for transport. One would expect to see application/pdf but instead you get application/octet-stream. This is allowable by the standards.
To properly decode the document byte stream, a Document Repository or Document Recipient should rely on the XDSDocumentEntry.mimeType attribute when it sees application/octet-stream in the Content-Type header.
See Also
Updated: bill 15:57, 22 July 2009 (UTC)
The IT Infrastructure Domain manages this profile.
The ITI Technical Framework is the official master document for this Profile.