Why Involve Test Practitioners in the software vendor selection process?
Very rarely are QA or Test practitioners involved in the software vendor selection process, often this activity takes place prior to the core project being mobilised for delivery.
Would a vendor selection process (such as an RFP or RFQ) benefit from Test engagement and if so, what would they look for?
What type of vendor would Test recommend?
All Test teams wear the scars of failed IT projects where external vendors or software has failed to integrate technically, or the mode of delivery is incompatible with their own process, culture or standards. What appears to be a robust e2e plan, soon becomes a long drawn out affair.
It is common to work with software vendors that may not invest or consider testing as a discipline which warrants much attention. Would a traditional software vendor selection process really tease items that would influence the likely success of the project delivery phase, such as how requirements will be traced through to releases, how compatible is the defect management process with existing tooling or whether the vendor performance tests using a stopwatch? These are just a few examples that can contribute to the outcome of the future engagement.
It’s likely, based on AheadMG observations in the market place, that Test teams would want to work with software vendors/suppliers who can demonstrate a high testing culture and willingness to collaborate, share and improve together to achieve common goals.
Nobody wants to work in an opaque and confrontational environment, where there is a disproportional need to inspect every last piece of evidence that comes out of the vendor through fear of quality issues. Worse still, the relationship breaks down to the point where the contract has to be dusted down to change or enforce ongoing sub-optimal vendor deliveries.
Test practice should consider endorsing, via a selection process, software vendors that possess the following key attributes:
Quality Standards
There are many quality standards and accreditations a software vendor can hold and actively practice, but which are important to your organisation and how would they interlock? As an organisation it is important to understand the software vendors QA approach in detail and if it is aligned with your expectations: Does the software vendor have the right approach to software and quality standards? Is there a documented and proven testing framework which has been used successfully in IT deliveries which are underpinned by established guidelines and patterns that mess with your own organisation?
Team Principles
The software vendors team structure and principles could differ from yours, which could be a good or a bad thing depending on your desired outcomes. Could the software vendor actually help plug known gaps in your own capabilities, maybe bring automation expertise or innovative working patterns?
Importantly, does the software vendor have specialist and dedicated independent test professionals in their team which is likely to increase the likelihood of quality outcomes? This will give you an indication if the company has invested in developing their core test capability including the more technical disciplines such as non-functional testing and automation. This will ultimately lead to having the right people available to be assigned to the right roles across the test phases for project success.
Communication and Collaboration
Communication and collaboration are key through the whole lifecycle (pre, during and post-project delivery). Is the software vendor open to collaborative working with the client, interlocking the delivery and handoffs? During the initial discussions, assessments, or contract development it is important to understand if you have a shared perspective in this area. The software vendor should show keenness to participate in joint planning, from defining roles and responsibilities across the testing phases, sharing test assets and test tooling which could be used jointly.
Co-operation and Flexibility
It is important to understand the software vendor’s ability to adapt. For example, does the software vendor have the resources and the ability to innovate, adapt and react to a sudden change? Do they have the resources to scale-up and make the right assessments to meet the client’s expectations without impacting on quality standards; there is an expectation that a software vendor is scalable, and a real-life scenario would reinforce this attribute.
Vision and Continuous Improvement
The software vendor’s vision and experience could differ to yours in a positive way due to their exposures to other clients and technologies. Does the software vendor have the appetite to innovate and initiate new approaches to meet the clients current or future needs? There are new and experimental trends in the software industry all the time; an ideal software vendor would explore and utilise these techniques to increasing efficiency and enhancing quality; which would potentially feed into your own service and process improvements. For example, the software vendor has used a new tool successfully that detects software/code vulnerabilities in early development stages. This could be the value added and be a critical factor in selecting the software vendor as it is looking forward and sharing the benefit of their innovation.
What are the overall benefits of selecting the right software vendor?
It is AheadMG view that introducing a QA voice into a software vendor selection process will provide an extra layer of insight for your company, underpinning any decision based on the likely outcomes that will be seen through delivery. Picking a software vendor who you can work with collaboratively and above all trust, will ensure a level of quality and above all a predictable delivery experience (which is likely to improve at each iteration).
Recap of the key benefits of QA Engagement and assessment:
- Establishing early relationships and providing an insight as to the likely delivery/quality experience the organisation will see if the software vendor is selected
- Independent QA representation and viewpoint; giving an indication of software vendor’s test capability including the more technical disciplines such as non-functional testing and automation
- Can assess if suppliers have the right approach to both software and quality; and if these standards are aligned to their own
- Predict and assess likely levels of collaboration and interlocked processes
- Identify risks that could manifest themselves after the software vendor is onboarded
- Start to build potential test strategies based on the knowledge of the software vendors holistic approach. Such as where gaps and synergies exist – e.g. shared API tests or access to knowledge repositories
- Understand likely level of customisation and change, which in turn could predict the level of defects, change requests, level of business involvement and overall test resources required on the client side
Summary
Involving Test practitioners in the software vendor selection process is likely to result in the selection of software vendors who demonstrate a high testing culture and willingness to collaborate, interlock processes, share and improve together and will also give an early insight into the key attributes such as ability, vision, culture, adaptability etc regarding test/quality requirements and much more.