This post is about how the mindset and methodologies of our company affects the way we work with quality requirements. It further suggests a five step approach on how to work with quality requirements in an agile setting.

Since there seems to be a lot of different definitions and understandings about quality, we’ll start with a brief introduction to the terms.

What is quality?

System or software product quality is the “degree to which a software product satisfies stated and implied needs when used under specified conditions”, according to ISO 25010.

Each stakeholder has an interest in software quality. For example, the user wants the system to be safe, responsive and user-friendly, the developer wants good logging and the product owner wants the system to be easy to change in the future (even if they tend to prioritize more functionality over a maintainable solution, but then it is your job as a tech person to explain the business implications of that decision).

Software quality is categorized into eight characteristics by ISO 25010.

Related to these characteristics we define non-functional requirements, like numbers of requests per second, security level and response time. We often say that non-functional requirements defines how the system is supposed to be, whereas functional requirements defines what the system is supposed to do.

Some people further make a distinction between external quality and internal quality, where the external quality is experiences by the users and the internal quality is experienced by the developers.

If it’s not functional, then what is it?

“Non-functional requirements” is not a self-explanatory term. Non-functional is often interpreted as not related to functionality or even worse: technical. When you talk about non-functional requirements you will hear things like “I’m a functional tester/architect/[insert title here], so I don’t care about non-functional requirements”.

To save yourself the pain of having to convince everyone that non-functional requirements are important and actually can be related to functionality, talk about quality requirements instead. Everyone cares about quality. Or at least it’s harder to say “I don’t care about the quality of the product we are making”.

Quality benefits all stakeholders

Quality requirements are a means to deliver value to all stakeholders – not just the end-user. This is why you and everyone involved in software development should work with quality requirements early and continuously:

• The user expects user-friendlier, faster, more secure and reliable systems.
• A maintainable system speeds up lead time, bug fixing and time to onboard new team members.
• Early focus on quality needs helps you make better choices when it comes to technology, architecture and design.

The problem with not paying attention to quality occurs when we are heavily invested in an architecture or technology that simply does not support our needs. We risk spending too much effort on the wrong solution and are faced with expensive changes.

How mindset affects quality

The path to quality depends on your company’s mindset. Is your IT company managed like a production factory where one change takes numerous man hours and goes through several handovers between different activity-oriented teams consisting of resources i.e. people? Or do you work in a company where people are trusted, empowered and encouraged to be creative and practise DevOps with continuous deployment? Taking the same plan-driven approach to quality as you would in a large factory to a modern IT environment will most likely slow you down and create more resistance towards quality work than it will do any good.

How is our approach to quality in agile development different from that in a waterfall development?

The waterfall path to quality

For the sake of simplicity, we’ll look at waterfall and agile as two extremities, even though most tech companies probably deals with a mix of the two.

Back in the days when most of the IT industry were doing software development in a control-oriented linear fashion, quality requirements were carefully analyzed and specified up front. They were written down in a separate document and there were dedicated processes for handling quality. In a way, this is a more structured and easier way of working with quality.

• You dedicate time for gathering all known requirements up front.
• You write the requirements once and approve the quality requirements early in the process.
• You get the quality you specify up front ((unless the handovers are causing too much information loss) even if that turned out to be not what you actually needed).

Besides getting what you ask for and not necessarily what you need, the traditional approach to quality encourage exhaustive lists of requirements up-front. One consequence of this is that the development team are put in a brain-off mode, ticking off check-lists with requirements they don’t have any relation to. This might work well if we are building something predictable and concrete (in the “obvious” domain of the Cynefin framework), but not so well when in a creative and more abstract world where most of the requirements are unclear and changing.

The agile path to quality

In an agile and feedback oriented way of creating software we don’t always have the “luxury” of spending days and weeks on making exhaustive lists of requirements in advance or testing every aspect of the product before going live.

Two important distinctions from the waterfall approach:

• The quality is mostly built-in by iterating over the product based on feedback from real users and from running the software in production over time.
• The developers need to work closer with a variety of stakeholders in order to understand the requirements.

Agile doesn’t equal good quality

You might think that an iterative approach to system development with continuous delivery and stakeholder collaboration effortlessly leads you towards good quality. Besides, refactoring is a natural part of agile.

Sure, agile development (with an agile mindset – not just the rituals) can help us enhance quality. It can also do the opposite. From my experience, the way we practice agile does not always emphasize quality in architecture and design the same way it emphasize functionality.  There might be different reasons for this:

• We don’t make time for planning and documenting the design, because it’s hard to predict up front and will be up for change later.
• We are strongly driven by the amount of functionality we deliver to the end-user, but not the long-term quality as seen from other stakeholders (a natural consequence of the product owner i.e. customer representative prioritizing the backlog).
• Convincing the product owner we need to improve quality characteristics, like portability and interoperability, can be hard.

Five steps to quality

How do we ensure the right quality characteristics without wasting time on heavy processes, comprehensive documentation and long check-lists that will soon be outdated?

These are my key suggestions on how to work with quality requirements in a more agile way. I also find this approach useful for more non-agile projects.

1. Define a product quality model for your organization

The product quality model defined in ISO 25010 provides a set of useful quality characteristics and sub-characteristics. Adjust the model to fit your organization, and make it well-known among the different stakeholders. Project managers, test experts, testers, developers, UX and UI experts, architects, product owner and everyone who has an interest in the system should be familiar with the model. This will create awareness and help draw attention to delivering on quality, not just time and cost.

2. Prioritize quality characteristics

For each system, bring the necessary stakeholders together and prioritize the relevant characteristics for that system. The ATAM process can be a good technique for making tradeoffs. This is useful for giving everyone a shared view on what is the goal and purpose of the system.

3. Establish quality requirements

Based on the prioritized characteristics, define clear, verifiable requirements with criteria for satisfaction and corresponding measures. This can be done for both

• the system or company in general and
• to any specific change or projects.

For the first alternative you can make a general list of mandatory requirements like the ones you are required to follow by law. These are usually predictable and less likely to change over time. Keep the list short, up to date and only enforce it on changes where they are relevant.

Keep the requirements on a reasonable level when it comes to technical details. You don’t want to tell the experts how to do their job or take the responsibility away from the developement team.

4. Treat quality requirements the same way you treat functional requirements

This one is not obvious: When your test manager identify what types of tests that needs to be run, it often doesn’t matter if the requirements are technical, non-technical, functional, non-functional or whatnot. A requirement is a requirement. What matter is the types of tests that needs to be run. You can test quality attributes like security, logging and interoperability with functional testing, combining them with test of functional requirements. Often there is also an overlap between functional and quality requirements, e.g. when it comes to functional suitability.

Separating quality from functionality might be good for drawing attention to quality, but requires an extra effort in administration and defining processes and responsibilities. Architectural work can also be inconsistent when we separate quality from functionality. For the same reason, I do not differentiate between external and internal quality when working with quality requirements (please leave me a comment if you disagree).

Write user stories and make use of the tools and processes already in place. There is no need to invest in a fancy tool for managing quality requirements (maybe for executing the tests, but not for the requirements themselves).

5. Measure, monitor, evaluate and adjust the quality requirements 

As we gain more knowledge about our domain and as the world around us change, the quality requirements also change.  Make use of feedback from the real world to continuously improve the system and the requirements.

Conclusion

Agile is not a get out of jail free card when it comes to planning and documenting. We shouldn’t spend too much time planning, but we shouldn’t make assumptions when it comes to quality needs either. An early, continuous and systematic approach to quality, as defined above, can make your stakeholders more happy, save you from having to rewrite large part of the system later and even increase lead time.

From the 9th principle of The Agile Manifesto: “Continuous attention to technical excellence and good design enhances agility”.