Proposal Effort Evaluation

As part of the Profile Proposal Process, the Technical Committee estimates the effort required (in terms of TC Bandwidth) to develop each of the proposals. The effort estimate is considered by the Planning Committee when they prioritize and select which Profiles will be developed this cycle.

The TC in each domain chooses how it estimates. This page reflects some ideas being tried out in the IHE Radiology domain. The idea is to expand on the Breakdown of Tasks section in the Brief Proposal Template by estimating "effort points" for each task. Turns out Agile has already thought of everything cool :-), so the material below borrows from Story Points.

This is a pilot so all details are open to discussion/suggestions. The retrospective indicated we need to do a better job of flushing out issues early in the process, but we don't want to overburden. So we'll try this but try to keep it brisk. The evaluation call should note Debates, not presume to resolve them.

Method

Proposer Homework

After the Brief proposal has passed the Short List vote, the proposal editors should create a Detailed Proposal by copying the Delta Proposal Template into the bottom of your proposal page and editing appropriately. This includes adding a Breakdown of Tasks section.

Transactions

• Bullet each new/modified transaction

• Indicate if it will clone an existing transaction, be similar, or be new
• If new, note the protocol/mechanism
• If similar, confirm the protocol/mechanism
• If clone, note what will be changed/tweaked
• Don't list existing transactions you plan to use unchanged (as noted in Technical Approach)

Decisions/Topics

• Bullet things that

TC Review & Effort Estimation

Transactions

• Review the description and challenge any unreasonable assumptions

• E.g., if

• Effort points: 1 - "normal" size transaction; 2 - "large" transaction
• Complexity points: 0 - cloning another transaction; +1 - new API/mechanism; +1 - new context or domain
• Uncertainty points:

• Consider if any other new transactions should be added
• Consider if any existing transactions will need cloning/modification

Decisions/Topics

• Uncertainty points:

Result

Tally up the points. Do our usual "planning poker" to get a % estimate, but with the tally and above discussions in mind.

Next year we might have a "conversion rate" from points to %. At the very least, there should be a pseudo-linear relationship between the proposals. One with a significantly higher tally should have a significantly higher estimate.

Profile Draft

We're currently recommending that the revised Breakdown of Tasks with point estimates be migrated into the Profile Draft, likely in the first Open Issue row or thereabouts so we can track it and see how this works out.

Brainstorming Meeting

A key goal of the Brainstorming Meeting is to resolve the Decisions/Topics items and other uncertainty points.

The Profile Editors should work to prepare clear write ups of each of the decisions/topics (and perhaps a proposed resolution) before the meeting. Hold t-cons if you think it's warranted.

The last 15-30 minutes of time allocated to each profile at the Brainstorming meeting will be spent confirming/recording the resolutions for each. Any still unresolved are flagged as critical items to address by/during Public Comment. A couple modest unresolved items is fine. More risks failure of the Profile.

Retrospective

When we do the end-of-cycle retrospective it might be interesting for each profile to think about Open Issues that came up and topics that used a lot of time and see if we might have anticipated those during evaluation.