Difference between revisions of "XDS Test Kit 2009-2010 Test Requirements"

Latest revision as of 09:54, 18 March 2019

Please note that the specific requirements for which tests must be run for a given connectathon are found in gazelle under menu Connectathon/Pre-Connectathon testing. The list on this page is not specific to any IHE connectathon.

Previous year's tests are documented here.

Testing HowTos

Some important topics on testing XDS:

How to report test results

How to assign patientIDs and uniqueIDs for tests

How to use the Public Registry including how to allocate unique patientIDs for tests.

Key to Codes

R - Require
O - Optional
MD - Required for Multiple Document Submission Option
DR - Required for Document Replacement Option
LC - Used for Document Life Cycle Management Option
F - Used for Folder Management Option
OL - Required for Offline Mode Option
ESO - Required if Embedded Repository accepts Provide and Register transaction (External Source Option)
A - Required if ATNA option implemented
C - Optional for ACC
Z - A test which is not yet available. It will be available later in the testing season. This code is frequently coupled with other codes such as:

Z/R - Will be required, but the test is not ready yet

Z/O - Will be an optional test, but it's not ready yet

D - Deprecated
CM - As directed by the Connectathon Manager
EDS - Required for External Document Source option
EDR - Required for External Document Repository option
IR - Required for Internal Repository option
QO - Required for Query Registry transaction option
BPPC - Required for BPPC Option
XDR - Required for XDR profile
XDM - Required for XDM profile

The test numbers are hyperlinks to the test descriptions. Tests where the test number is not a hyperlink are not ready yet.

Note on ATNA mutual TLS

XDS and ATNA related tests have been separated. Each XDS transaction has a list of tests to support it. They are listed below. Under each transaction is a single mutual TLS related test which demonstrates the ability to perform that transaction over mutual TLS.

Basic Communications and Tools

Test #	Test Name	Doc Src	Doc Cons	Registry	Repository	Src/Rep
11710	Connection + IP Registration	D	D		D	D
10511	XDS Patient Feed: A01	See MESA testkit
10512	XDS Patient Feed: A04	See MESA testkit
10513	XDS Patient Feed: A05	See MESA testkit
10514	XDS Patient Feed: A08	See MESA testkit
10515	XDS Patient Feed: A40	See MESA testkit
11717	Verify Mutual TLS connection with NIST	D	D	D	D
11720	Metadata Validation Tool	D	D	D	D	D

Provide and Register Document Set (-a and -b)

XDS.a Test #	XDS.b Test #	Test Name	Doc Src	Doc Cons	Registry	Repository	Src/Rep
11742	xxxxx	Generate Basic Transport - Document Source	D
11744	xxxxx	Accept Basic Transport - Document Repository				D
11746	12049	Submit One Document	R				EDR
11743	12046	Provide and Register with mutual TLS	A
11747	12047	Submit Two Documents	MD				MD+EDR
11728	11969	Create Folder	F				F
11729	11970	Create Folder with Initial Document	F				F
11730	11971	Add New Document to Existing Folder	F				F+EDS
11972	11973	Add Existing Document to Existing Folder	F				F+EDS
11748	11974	Replace Existing Document	DR				DR
11750	11975	Submit Transformation for Existing Document	Z/O				Z/O
11976	11977	Submit Addendum for Existing Document	Z/O				Z/O
11737	11978	Submit Add By-Reference to Submission Set	Z/O				Z/O
11827	11966	Accept Document				R
11751	11979	Accept Two Documents				R	EDS
11982	11983	Reject submissions where metadata and documents do not match				R
12085	12086	Accept Document w mutual TLS				R
11980	11981	Accept Document with size, hash and URI attributes				D	Z/EDS
11886	11968	Pass Arbitrary Metadata				Z/R	Z/EDS
11749		Life-cycle Management	D				D
11736	11736	Submit Replace/Transformation for Existing Document	D				D
11752		Provide Example Document	D				D
11763		Submit example HL7 Lab	D				D
11764		Submit example PDF	D				D
11765		Submit example Wrapped PDF	D				D
11766		Submit example XDS-MS/CDA R2	D				D
11738		Submit via Offline Mode	D			D
11887	11986	Return Errors from Registry				R	Z/EDS
xxxxx	xxxxx	All times in metadata must be UTC	D				D
11952		Submit a Consent Document	D
11953		Submit a Clinical Document	D
	12369	RepositoryMetadataError				R

Register Document Set (-a and -b)

XDS.a Test #	XDS.b Test #	Test Name	Doc Src	Doc Cons	Registry	Repository	Src/Rep
11721		Basic Transport			D
11731		Register One Document				D	D
	12320	Registry SOAP Pattern			D
	12321	Repository SOAP Pattern				D
11732		Register Two Documents				D	D
11733	11990	Accept Register One Document			R
11735	11991	Accept Register Two Documents			R
11871	11992	Accept Document Replace			R
12048	12022	Accept Document Replace, Addendum or Transformation present			Z/R
11873	11993	Accept Document Addendum			R
11874	11994	Accept Document Transformation			R
11875	11995	Accept Document Replace with Transformation			R
11876	11996	Reject Submission of Invalid Patient ID			R
11877	11997	Reject Submission Set, Patient ID does not match Document			R
11878	11998	Reject Submission, Patient ID on Replacement Document does not match Original			R
11879	11999	Accept Create Folder			R
11880	12000	Accept Create Folder with Initial Document			R
11881	12001	Accept Add New Document to Folder			R
12325	12326	Accept Existing Document to Existing Folder			R
12322	12323	Manage Folder lastUpdateTime			R
11882	12002	Reject Add Document to Folder - Patient ID does not match			R
12324	12327	Accept Document Replace, Document in Folder			R
11883	12084	Submission Stored - All or Nothing			R
11884	12003	Accept Add By-Reference to Submission Set			Z/R
11885	12004	Reject Duplicate Document uniqueID with Different Hash			R
11740	12005	Accept Register with mutual TLS			R
11953		Submit a Clinical Document					D
11952		Submit a Consent Document					D
11955		Accept Clinical Document referencing Consent					D
12050	12051	Register One Document					R
12052	12053	Register One Document with mutual TLS					R
12054	12055	Register Two Documents					O
12056	12057	Replace Existing Document					Z/O
12058	12059	Transform Existing Document					Z/O
12060	12061	Submit Addendum to Existing Document					Z/O
12062	12063	Submit Add By-Reference to Submission Set					Z/O
12064	12065	Create Folder					Z/F
12066	12067	Create Folder with initial Document					Z/F
12068	12069	Add New Document to Folder					Z/F
12070	12071	Add Existing Document to Folder					Z/F
	12370	Association Documentation Classifications			R

Query Transaction

XDS.a Test #	XDS.b Test #	Test Name	Doc Src	Doc Cons	Registry	Repository	Src/Rep
11814	xxxxx	Basic Transport		D	D
11801	xxxxx	Query for ObjectRefs		O	O
11802	xxxxx	Query for LeafClass		O	O
11910	xxxxx	FindDocuments Query			QO
11911	xxxxx	FindSubmissionSets Query			QO
11912	xxxxx	FindFolders Query			QO
11913	xxxxx	GetAll Query			QO
11914	xxxxx	GetDocuments Query			QO
11915	xxxxx	GetFolders Query			QO
11916	xxxxx	GetAssociations Query			QO
11917	xxxxx	GetDocumentsAndAssociations Query			QO
11918	xxxxx	GetSubmissionSets Query			QO
11919	xxxxx	GetSubmissionSetAndContents Query			QO
11920	xxxxx	GetFolderAndContents Query			QO
11921	xxxxx	GetFoldersForDocument Query			QO
11922	xxxxx	GetRelatedDocuments Query			QO
11923	xxxxx	FindDocuments Query		O
11924	xxxxx	FindSubmissionSets Query		O
11925	xxxxx	FindFolders Query		O
11926	xxxxx	GetAll Query		O
11927	xxxxx	GetDocuments Query		O
11928	xxxxx	GetFolders Query		O
11929	xxxxx	GetAssociations Query		O
11930	xxxxx	GetDocumentsAndAssociations Query		O
11931	xxxxx	GetSubmissionSets Query		O
11932	xxxxx	GetSubmissionSetAndContents Query		O
11933	xxxxx	GetFolderAndContents Query		O
11934	xxxxx	GetFoldersForDocument Query		O
11935	xxxxx	GetRelatedDocuments Query		O
11803	xxxxx	FindDocuments Query		D	D
11804	xxxxx	FindSubmissionSets Query		D	D
11805	xxxxx	FindFolders Query		D	D
11806	xxxxx	GetAll Query		D	D
11893	xxxxx	GetDocuments Query		D	D
11888	xxxxx	GetFolders Query		D	D
11889	xxxxx	GetAssociations Query		D	D
11891	xxxxx	GetDocumentsAndAssociations Query		D	D
11892	xxxxx	GetSubmissionSets Query		D	D
11894	xxxxx	GetSubmissionSetAndContents Query		D	D
11895	xxxxx	GetFolderAndContents Query		D	D
11810	xxxxx	GetFoldersForDocument Query		D	D
11896	xxxxx	GetRelatedDocuments Query		D	D
11741	xxxxx	Query with mutual TLS		A	A
11807	xxxxx	GetDocument Query		D	D
11808	xxxxx	GetSubmissionSetContents Query		D	D
11809	xxxxx	GetFolderContents Query		D	D
11811	xxxxx	GetAddendums Query		D	D
11812	xxxxx	GetHistory Query		D	D
11813	xxxxx	GetTransformations Query		D	D

Stored Query Transaction

XDS.a Test #	XDS.b Test #	Test Name	Doc Src	Doc Cons	Registry	Repository	Src/Rep
11814	11814	Basic Transport		D	D
11949	11949	Stored Query for ObjectRefs		D	D
11950	11950	Stored Query for LeafClass		D	D
12347	11897	FindDocuments Stored Query			R
12348	11898	FindSubmissionSets Stored Query			R
12349	11899	FindFolders Stored Query			R
11900	11900	GetAll Stored Query			Z/R
12350	11901	GetDocuments Stored Query			R
12351	11902	GetFolders Stored Query			R
12352	11903	GetAssociations Stored Query			R
12353	11904	GetDocumentsAndAssociations Stored Query			R
12354	11905	GetSubmissionSets Stored Query			R
12355	11906	GetSubmissionSetAndContents Stored Query			R
12356	11907	GetFolderAndContents Stored Query			R
12357	11908	GetFoldersForDocument Stored Query			R
12358	11909	GetRelatedDocuments Stored Query			R
12363	12363	Demonstrate use of at least one Stored Query		R
11936	11936	FindDocuments Stored Query		O
11937	11937	FindSubmissionSets Stored Query		O
11938	11938	FindFolders Stored Query		O
11939	11939	GetAll Stored Query		O
11940	11940	GetDocuments Stored Query		O
11941	11941	GetFolders Stored Query		O
11942	11942	GetAssociations Stored Query		O
11943	11943	GetDocumentsAndAssociations Stored Query		O
11944	11944	GetSubmissionSets Stored Query		O
11945	11945	GetSubmissionSetAndContents Stored Query		O
11946	11946	GetFolderAndContents Stored Query		O
11947	11947	GetFoldersForDocument Stored Query		O
11948	11948	GetRelatedDocuments Stored Query		O
11741	11741	Query with mutual TLS		A	A
11954	11954	Query for a Consent protected Clinical Document		D
11957	11957	Accept Query requiring Consent filtering			D
11958	11958	Accept Query bypassing Consent filtering			D
11960	11960	Consent Revocation			D
11961	11961	Consent Expiration			D
	12342	Stored Query in the presence of XCA		R
	12368	Result not single patient error			R

Retrieve Document Transaction

XDS.a Test #	Test Name	Doc Src	Doc Cons	Registry	Repository	Src/Rep
11734	Retrieve Document		O
11739	Retrieve Document with TLS		O
12024	Accept Retrieve Document				R
12026	Accept Retrieve Document with mutual TLS				Z/A
12045	Accept Retrieve Document					R
12043	Accept Retrieve Document with mutual TLS					Z/A
12359	XDS.a Retrieve mimetype				R
12362	XDS.a vs XDS.b Retrieve				D

Retrieve Document Set Transaction

XDS.b Test #	Test Name	Doc Src	Doc Cons	Registry	Repository	Src/Rep
12023	Retrieve Documents		R
12020	Retrieve Documents with TLS		R
12029	Accept Retrieve Document Set – single document				R	R
12021	Accept Retrieve Document Set – two documents				R	R
12028	Accept Retrieve Document Set with Mutual TLS				A
12038	Accept Retrieve Document Set – single document					R
12037	Accept Retrieve Document Set – two documents					R
12083	Accept Retrieve Document Set with Mutual TLS					A
12343	Retrieve Documents in the presence of XCA		R
12360	XDS.b Retrieve mimetype				R
12362	XDS.a vs XDS.b Retrieve				R

Load Test Data

Test Data Description

XDS.a Test #	XDS.b Test #	Test Name	Doc Src	Doc Cons	Registry	Repository	Src/Rep
11890	12346	Registry Test data			R
12328	12345	Repository & Registry Test data				R

XDR

Document Source actor tests

Test	Status	Description	Purpose
Required
12049	Ready	Submit one document via XDS.b or XDR	Issue Provide and Register transaction to a Document Recipient.
12046	Ready	Submit one document via XDR over TLS	Repeat test 12049 over TLS.
Optional
12047	Ready	Submit two documents via XDR	Issue Provide and Register transaction to a Document Recipient containing two documents.
12378	Ready	Validate XDR Provide and Register transaction with Limited or Full Metadata against Message Validator	Validate XDR Provide and Register transaction with Limited or Full Metadata against Message Validator

Document Recipient actor tests

Test	Status	Description	Purpose
Required
12371	Ready	Accept one document via XDR
12372	Ready	Accept one document via XDR over TLS	Repeat test 12371 over TLS.
Optional
12373	Ready	Accept two documents via XDR	Accept Provide and Register transaction as a Document Recipient containing two documents.
Optional
12375	Ready	Accept XDR Provide and Register.b with Limited Metadata	Accept Provide and Register transaction as a Document Recipient. Metadata objects conform to Limited Metadata requirements and are properly labeled.
Optional
12376	Ready	Accept XDR Provide and Register.b with Limited Metadata but without Limited Metadata Label	Accept Provide and Register transaction as a Document Recipient. Metadata objects conform to Limited Metadata requirements and are not properly labeled.

XDM

Portable Media Creator actor tests

Test	Status	Description	Purpose
Required
12377	Ready	Validate XDM Zip format against Message Validator	Validate XDM Zip format against Message Validator

XCA

Tests for XCA are documented the following pages:

Initiating Gateway (XCA)

Test	Status	Description	Purpose
Required
12300	Ready	FindDocuments XGQ for LeafClass	Issue FindDocuments XGQ requesting LeafClass (full metadata) be returned. This test will return metadata for two documents, a small one and a large one.
12319	Not Ready	FindDocuments XGQ returns metadata from multiple Receiving Gateways	Issue FindDocuments XGQ requesting LeafClass (full metadata) be returned. The returned metadata represents content from multiple Gateways
12301	Ready	Cross Gateway Retrieve of a single document	Issue Cross Gateway Retrieve for a single document based on metadata returned in test 12300. There are two steps inside this test. The first one retrieves the small document from test 12300, the second retrieves the large one.
12302	Ready	FindDocuments XGQ over TLS	Repeat test 12300 over TLS
12303	Ready	Cross Gateway Retrieve over TLS	Repeat test 12301 over TLS
12304	Not Ready	Audit of FindDocuments XGQ	Generate audit log submission for FindDocuments XGQ
12305	Not Ready	Audit of Cross Gateway Retrieve	Generate audit log submission for Cross Gateway Retrieve
Optional
12306	Ready	FindDocuments XGQ for ObjectRef	Issue FindDocuments XGQ requesting ObjectRefs be returned
12307	Ready	GetDocuments XGQ for LeafClass	Issue GetDocuments XGQ using ObjectRefs returned in test 12306 to get full metadata for documents
12308	Ready	Retrieve multiple documents	Retrieve two documents based on metadata returned in test 12300

Responding Gateway (XCA)

Test	Status	Description	Purpose
Initialization
12318	Ready	Test initialization	Establish test data in systems behind Receiving Gateway
Required
12309	Ready	FindDocuments XGQ for ObjectRef	Accept FindDocuments Cross Gateway Query requesting ObjectRefs be returned. Depends on test 12318 to initialize the data.
12310	Ready	FindDocuments XGQ for LeafClass	Accept FindDocuments Cross Gateway Query requesting LeafClass (full metadata) be returned
12311	Ready	GetDocuments XGQ for LeafClass	Accept GetDocuments Cross Gateway Query using ObjectRefs returned in test 12309 to get full metadata for documents
12312	Ready	Retrieve single document	Accept request for the retrieval of a single document based on metadata returned in test 12311
12313	Ready	Retrieve multiple documents	Accept request for the retrieval of two documents based on metadata returned in test 12311
12314	Ready	FindDocuments XGQ over TLS	Repeat test 12309 over TLS
12315	Ready	Retrieve Single Document over TLS	Repeat test 12312 over TLS
12316	Not Ready	Audit of FindDocuments XGQ	Generate audit log submission for FindDocuments Cross Gateway Query
12317	Not Ready	Audit of Cross Gateway Retrieve	Generate audit log submission for Cross Gateway Retrieve

Async

Notes for asynchronous web services can be found here.

Tests

Actor	Transaction	Test number
Document Registry	Stored Query	12331
Document Registry	Register.b	12334
Document Repository	Provide and Register.b	12333
Document Repository	Register.b	12333
Document Repository	Retrieve Document Set	12335
Document Source	Provide and Register.b	12338
Document Consumer	Stored Query	12339
Responding Gateway	Cross-Gateway Query	12336
Responding Gateway	Cross-Gateway Retrieve	12337
Initiating Gateway	Cross-Gateway Query	12340
Initiating Gateway	Cross-Gateway Retrieve	12341

Multi-Patient Query

Actor	What	Test number
Document Registry	Load test data for FindDocumentsForMultiplePatients	12361
Document Registry	Test FindDocumentsForMultiplePatients	12364
Document Consumer	Test FindDocumentsForMultiplePatients	12366

Metadata Update

Actor tested	Test	Test number
Document Registry	Simple DocumentEntry Update	15800
Document Administrator	Simple DocumentEntry Update	15801
Document Registry	Update DocumentEntry Status	15802
Document Administrator	Update DocumentEntry Status	15803