A Solution For Tracking and Monitoring Multi-Stream Processes Using Rational Team Concert

I'm going to assume you're already managing your plans and work items in Rational Team Concert (if not, read more here: https://jazz.net/products/rational-team-concert/).

Have you wanted to track the flow of work through a set of steps in a defined multi-stream process using RTC?

I've recently implemented an extension to RTC for a customer that does just this and I thought I'd write about that here. I'm not able to share the code for this (sorry), but hopefully these notes will be useful for anyone looking to implement their own solution.

Figure 1. - Simple example of a "defined multi-stream process"
In figure 1 we see a simple example "defined process" which exhibits the following characteristics:
  • With the exception of the first step, all other steps are dependent on their "upstream" steps completing before they can start.
  • Each step is owned by a team i.e. team members of that team are able to claim the step to work on it.
  • A team can own (be responsible for) multiple steps.
  • A step can be dependent on more than one "upstream" step e.g. step 4.
  • More than one step can be dependent on the same "upstream" step e.g. steps 3.1 and 3.2.
  • The process is completed once all of the steps in the process have been completed.
Let me start with an important observation: For many teams, this type of workflow is not required as they'd simply use the OOTB Scrum template and be done with it. In our case the team was using Scrum for their software development work, but also wanted to track certain work according to a defined process with parallel work occuring.

So lets now address the question of why you might need a custom extension to support this? Lets look at what RTC gives you out of the box and consider whether any of these options meet our needs.

The Workflow States Solution - And the "Who Owns Each Step?" Problem

Its easy enough to represent a process flow using a custom work item workflow describing the states in the process.

Figure 2 - Workflow States Solution
So job done?

Well lets remember that we really care about being able to assign various steps in the process to specific teams.

There is no way in this solution to pre-assign ownership for various steps in the process to specific teams.

Also things get a bit interesting around steps 3.1 and 3.2. In the process defined in figure 1 these steps can occur in parallel - hence referring to this as a "multi-stream" process. In the solution shown in figure 2 the steps can be done in either sequence, but not at the same time.

Lets also add a new requirement that we care about seeing how well our process is performing. How long are we spending in each of the tasks? Who is performing them well. Not all teams will care about tracking this information. But if you're performing continuous improvement (and why wouldn't you be), then read on!

Although the Jazz Report Builder will let us create awesome reports from our data warehouse, this solution doesn't make it easy to track who completed each step, how long we waited to start the step (wait time), and how long it took to complete the step (execution time). That data is simply not readily available in the data warehouse if we stick to the solution shown in figure 2.

So in summary, although this solution is pretty simple to understand, it turns out it doesn't meet our requirements very well.

The Work Item Template Solution - And the "Is It Workable?" Problem

Its easy enough to represent a process using a work item template that contains a parent work item that represents the process, along with child tasks that represent the steps of the process, organised end-to-end using Blocks/Depends On links. That will let you instantiate the process and track the flow of work using the linked set of tasks. Figure 3 outlines what this looks like.

Figure 3 - Work Item Template Solution
Note in figure 3 that the template contains a high-level Process work item to track overall completion of the process, as well as 5 child Task work items, one for each of the steps in figure 1, and with Blocks/Depends on links representing the process flows shown in figure 1.

The reason that the parent Process is required in the template is to act as a container for various information that needs to be captured that is relevant to the overall process.

Figure 4 - Workflows for the Work Item Template Solution
Now we have the following:
  • Each task in the process can be pre-assigned to a team.
  • By following the Blocks/Depends On links you will know the sequence that tasks should be completed in, and the assignment, start and completion of each task can be tracked.
So are we there yet?

Well lets also assume you would really like the steps of our process to be completed in strict sequence. In that light, there are a few things that this solution can't do. 
  1. If I use a plan to consume my work, there is no way to indicate on the plan that one of the process step tasks is ready to be worked on i.e. its "upstream" process step tasks are completed.
  2. When all of the process steps are completed, you need to remember to mark the parent "process" work item as completed.
To expand on what I mean in point 1 above lets refer back to figure 1. When the process work item template is instantiated, all of the steps are created as part of the template. Other than inspecting the Blocks/Depends On Links, there is no indication on a step as to whether someone can validly start working on it yet i.e. its previous step(s) is/are completed. So they will all appear to be available to work on as soon as they are created - which is not correct! It is possible to get creative with RTC queries to filter out those work items whose "upstream" dependency steps are not completed, but this will not help you in RTC plans. So work items will appear to be workable on plans even when they are not.

Solution Wish List

Each of the options evaluated thus far has fallen short on certain requirements. Lets organise those requirements we've noted as a wish list so we can fully understand what we'd like RTC to do.

Wish list
  1. I want to be able to track completion of the steps of a defined process in RTC.
  2. I want to be able to assign execution of process steps to specific teams/roles.
  3. Steps can be worked on in parallel.
  4. I want the process steps to be completed in strict sequence i.e. I don't want someone to be presented with a task unless its upstream tasks have been completed.
  5. I want to be able to track how well my processes is being performed. Both for the overall process, but also from the point of view of individual steps and teams.

The "Workable Date" Solution

The solution I came up with aims to be simple and therefore easy to understand.

It is based on the Work Item Template solution, but with some additions.

Key to the solution is adding a pair of new attribute to the Task work item type:
  • a timestamp called Workable Date
  • and a Boolean called Is Workable.
Figure 5 - New Attributes on the Task Work Item Type
Definitions: Workable Date is a timestamp that represents the moment when a process step becomes workable. A process step becomes workable when the process has been started, and all previous "upstream" process steps have been completed. Is Workable is a convenience attribute that returns true when workable date is set, and false when it is not set.

Now that we have these new attributes, we need automation to set their values.

This is provided through a custom Operation Participant deployed as a server-side RTC plugin, and triggered on a Work Item Save action. It automates setting/unsetting the workable date, and marking the overall process as completed once the final step in the process is completed.

Figure 6 - Workflow Manager Operation Participant
Setting/unsetting the Workable Date is done as follows:

1. When a process is started, any steps that have no open upstream steps - i.e. no Depends On links to an open step - are marked as workable.
2. If the process is stopped, then all workable steps are marked as not workable.
3. Do not allow a step to be started if it is not workable.
4. If a step is completed, then any downstream steps that are not workable are marked as workable, provided they don't have any other upstream steps that are not completed.

The automation also sets the Is Workable boolean at the same time as setting/unsettting the workable date.

With this combination of work item template + attributes + plugin allows us to do the following:

1. I can use Is Workable to filter in any queries or plans that present work to teams - essentially filtering out those work items where Is Workable = false. Figure 7 shows an example Kanban board, with the configuration shown in figure 8 where you can see the isWorkable:false exclusion. The effect of this is to only show those work items that are indeed workable. We can now ensure that work items that are not yet workable don't appear on plans!

Figure 7 - Is Workable Kanban Board
Figure 8 - Configuration For Is Workable Kanban Board
2. I can use the Workable Date timestamp to determine when a specific work item became workable, and therefore I can do reporting on:
  • How long it took for someone to start work on a work item after it became workable.
  • What the turnaround time is for the work item from when it became workable.
Some further reports examples are shown below. Figure 9 shows how Is Workable is used to filter out non-workable work items from a report. Figure 10 shows how Workable Date is used to calculate the average turnaround time of each process step.

Figure 9 - Advanced Report Builder Report Filtering Out Non-Workable Work

    Figure 10 - Advanced Report Builder Report Showing Average Step Turnaround Time

Summary

My solution to tracking the flow of work through a set of steps in a defined process using RTC consists of the following:

1. A work item template for a process and its steps - see figure 3.

2. The Workable Date and Is Workable attributes - see figure 5.

3. A custom Operation Participant that sets/unsets those attributes, enforces the strict sequence of the process, and marks the process as completed once the steps are all completed - see figure 6.

This solution has allowed my customer to successfully track seven unique processes across a team of a few hundred individuals, with detailed work item analytics showing the performance of the overall process, and each of the process steps and teams working on the process.

Although a fair amount of thought went into this solution to keep it simple for the users to understand, there are certainly other ways to solve this! Would love to hear some of your ideas/feedback in the comments thread. If there is enough interest I might add further posts that go into some of the technical detail behind the solution.

Thanks to Ralph Schoon and Sam Detweiler for their postings on the IBM CLM forums for providing pointers that helped me understand the ins-and-outs of the RTC API. Ralph's forum and blog postings were especially invaluable and I highly recommend his extensions workshop (see references).

One final thought to end on. Please follow @continualoop to hear more about my thoughts on improving software development and delivery. Let's improve IT!

References

1. Various postings from Ralph Schoon and Sam Detweiler on the IBM Jazz forum. https://jazz.net/forum/questions

2. Ralph Schoon's blog. https://rsjazz.wordpress.com

3. Rational Team Concert Extensions Workshop. https://jazz.net/library/article/1000

Copyright © Continualoop Blog 2017. All rights reserved.

Comments

Popular posts from this blog

DevOps Analytics: Five Steps To Visualise Your Jenkins/UrbanCode Deploy Delivery Pipeline In Action!

We All Care About The Loop!