The Full Assessment

This model originates from the time when I founded and operated Praqma. In this post, I’ll describe how I conducted this Continuous Delivery Metric Assessment at more then 25 companies

The name and concept “Continuous Delivery” is a deliberate reference to a lean or TPS inspired approach often used in manufacturing which takes offset in a one-piece flow that strives for quality built into the product as opposed to glued on - after it’s done.

In the modern world of software development we use the concepts Continuous Delivery and DevOps to be the equivalents in software to what lean means to manufacturing.

We strive for Lean Software Development.

Value Stream Mapping - the as-is picturePermalink

In Lean manufacturing there is a strict focus on Product Lifecycle Management – in software our focus is on Software Development Lifecycle Management (SDLC).

As in lean, we investigate and assess three different core phenomena that jeopardize an ideal delivery value stream:

• Muda: Wastefulness – things and processes that are excess, which don’t contribute to value.
• Muri: Unevenness – inefficient coordination of takt and workflows that build up stock or create wait-states.
• Mura: Overburdenness – bad utilization of resources, scope creep and poor prioritizations

Just as in a lean process we get the first impression of the current state from an engaging workshop exercise based on a Value Stream Mapping technique. We gather team members and stakeholders and facilitate them to draw a rich picture of the current (as-is) process. We mark the waste, the back-loops, the wait states, the excess manual processes, the communication conflicts etc.

The Value Stream Mapping exercise’s overall purpose is to understand current workflow and most importantly map lead times. We map the DORA metrics; Deployment Frequency, Mean Lead Time for Changes, Mean Time to Recovery, Change Failure Rate and reliability.

I’ve gathered a handful af pictures from some of the workshops and the first pile is from the initial part of the session where we do a Value Stream Map. It’s a rich picture we draw on the wall, everyone involved in the process has a pen, and we’re mapping what exactly goes on while the software is produced, validated and shipped.

When the Value Stream begins to take form we introduce the trouble markers which the participants use to mark where the process needs to be optimization in six different categories of troubles:

• Manual Work - dull, repetitive work done my humans
• Queues - processes where takt is off due to lack of resources
• Wait states - processes where flow is broken - often during handovers. Tasks are dropped between chairs. What is important to someone is unimportant to someone else
• Conflicts - parties fighting each other, serving different and contradicting agendas
• Unplanned work - prioritization and systemic approach is soo poorly adapted, that plans fall apart all the time. It’s fire fighting
• Needs repair - Processes are adapted to a certain situation, but the world has moved on. These processes needs to be updated

Some board examplesPermalink

When the Value stream is mapped and assessed we ask the team to prepare a walk-through of what goes on in the board. One person will start in the upper left corner and explain all the processes, troubles, connection, and back-loops.

It’s very seldom that there exists such one person that has the full overview of the entire process. During the walk-through new team members will have to take over. Most often no one grasps the full picture.

The Goal - get the full picture?Permalink

The Dreyfus brothers published a model for skill acquisition which contributed to the field of Cognitive Science as early as in the 80’s to propose an early model for understanding intelligence and expertise. This has inspired us to build a model, not necessarily – or not just – for maturity, but also for in-depth metrics.

In Continuous Delivery in context of software development we focus on different areas of knowledge or matter; Build, Test, Version control, DevOps, Architecture & Design and Organization & Culture. These areas can be molded and adapted to the individual client and assessment.

After the Value Stream mapping session we present the model and ask the team to score each card from a miraculous point of view:

Imagine that the principle described on the card was fully implemented in your process. How much would it affect throughput, feedback and payback — and how complex would the implementation be

The model suggests different approaches, techniques, tools or principles, which represent different levels of expertise and proficiency. We use the model to conduct a facilitated discussion with the team and stakeholders. Through this workshop we will debate every single card and assess how relevant this capability is in context of the quality of the product and the Software Development Lifecycle.

So we spread out in smaller teams and each team picks up a couple of cards of their own choice. Each card is scored for effect and cost of implementation. The processes continues until all cards are scored and assessed. The team must agree and debate the scores to get to some level on consensus.

Scoring the cards in the modelPermalink

This concludes the first day of the Value Steam Mapping - and Card scoring workshops.

Observations – digging in the dirtPermalink

Next stage is to get our fingers into the dirt and have various team members explain and demonstrate some of the tools and processes that we identify as a special point of interest. These sessions will be scheduled when we have the overview established from the initial workshops, so we can say for sure what these points of special interests are but typically we would do a combination of code inspections, static analysis, review tool utilizations, and we will study and observe the team’s common ways of working, branching strategies, deliveries, deploys, monitoring etc.

Lean Coffee WorkshopsPermalink

To prepare ourselves to go into solution mode and enable us to suggest a feasible and realistic roadmap towards the end goal we will organize a series of “lean coffee workshop” - typically one in each area in the model – and whatever else is needed. In these workshops we will invite team members and stakeholders with special expertise in the field.

A lean coffee workshop is a special workshop format that allows participants to help build that agenda of the workshop dynamically in such a way that dire problems or issues take higher priority. For each discussion topic we will facilitate that we capture:

• The root cause problem
• A possible (maybe more) solution to mitigate it
• The value each identified solution would yield to SDLC or product quality

The findings are then recorded and presented as a detailed and customized note on each card - see card “Automated Builds” as an example (these are real finding from some of the assessments, but anonymized and used here to serve as examples). Also notice on the “Automated Builds” card, that the scoring from the workshop is captured. The score is 7 and complexity is categorized as a group effort.

The reportPermalink

This concludes our onsite effort – from here we will take the elicited intelligence home to discuss internally among ourselves and we will write up a report and propose a prioritized list of all findings in. It’s a quite comprehensive report that can serve as a roadmap or backlog for the work ahead.

The report will include an executive summary.

If you are interested the model score, the findings and the prioritized recommendations can be delivered to you as a static website; your one version of the website that your’re currently at, which you can host securely on your own corporate inside – for every stakeholder to browse and reference.

This framework and model are inspired by lean principles, cognitive science, DORA metrics and DevOps and agile culture. Lars Kruse has been the lead on the development of this model and approach and has conducted more then 25 assessments based on this framework including:

Grundfos, Danfoss, Universal Robots, Volvo, Terma, Brüel & Kjær Sound & Vibration, Atmel (now Microchip), ATP, Alm. Brand, 3Shape, TetraPak, Novelda, Yxlon International, Oticon, Kamstrup, Napatech, Schneider Electrics, Schlumberger, Cryptera, DR, BK Ultrasound.