People can wrap their minds around the concepts of writing code, designing the interface, talking to end users to find out what they need, and managing the project, but then there’s this group of people in charge of judging the quality of what is produced.
“Quality” is a bit of a nebulous concept that is hard to define on its own, let alone trying to convey the differences between the various roles.
The difference between a QA tester, QA analyst, and QA architect is the degree to which you understand the solution you’re testing, the problem the solution is attempting to solve, the code and the platform the code sits on, the factors that affect the project that may not be spelled out but should be taken into account, and the individual needs of the end users that will be using the application.
Quality assurance tester
At the most basic level of QA, someone tests an application to see what it does given different inputs. At this level of understanding, you can find obvious errors, display issues, and things like the application having different behaviour based on if you access it via Internet Explorer vs. Chrome vs. Safari on an iPhone.
What this level of understanding will not be able to tell you is:
- Does the solution do what it’s supposed to do? If you don’t know what it’s supposed to do, you can’t answer this question.
- Does it have any undesirable behaviors? Again, if you don’t know what the solution is supposed to do, you are equally clueless about what it should never do.
- How bad are the bugs that are there? Something that would be a huge problem in one project will be an acceptable limitation in another one.
To use an example from a project I worked on a few years ago, the application had a web part on the home page that said what time it was in the current user’s location, with a drop-down menu that allowed the user to see what time it was in other locations. A QA tester could get that application to the point where there are no obvious problems, but that’s as far as they can go.
In other words, a tester can tell you what they think is wrong with it, but if they don’t know that you wanted it in purple text and need it to support Sḵwx̱wú7mesh language characters, then they won’t recognize the fact that it doesn’t do either of those things as problems.
The quality of the solution depends on whether or not what it does addresses the problem it should be designed to solve. At this level of understanding, all you can do is tell stakeholders what the code does, and leave it up to the people who understand the concept better to figure out what to do with that information.
Quality assurance analyst
There are two approaches to being a QA analyst, which for the purpose of this article, I will refer to as top down and bottom up. We’ll start with top down.
At this level, you understand both the requirements and the solution. You have at least some basic context for the business problem that it’s attempting to solve, so you can offer opinions on how well the solution satisfies those requirements.
In my example above of the application that shows the time in different locations, the company it was designed for had offices all over the world, and people working in the Vancouver office may have to collaborate with someone working in the Santiago, Chile, office. Employees needed a way to see what time during business hours it was in that office. Someone who has this level of understanding of the functionality required will be able to tell you how closely the solution gets to fixing the problem it was designed to solve.
A QA analyst that takes a bottom up approach would have an understanding of the underlying code and platforms that are used. This will uncover problems that are much different from the QA analyst that is approaching it from the other perspective. In the “What Time Is It?” example, this person will know how the system will return that information if the source of truth is a SharePoint list vs. a third party application.
The greater the understanding this person has for the functions and limitations of the code that is being used to solve the problem, and the platform that the code sits on, the more precise their testing will be.
This is what elevates the level of testing from hacking and slashing blindly, to carving methodically, to a surgical strike that attacks the system at its most vulnerable points. This is the person that can tell you the Web Part breaks for time zones like Newfoundland (3.5 hours off of UTC) because the source of truth is passing in a decimal value and the Web Part is expecting a whole number.
Ideally, a QA analyst would combine both the top down view of looking at the solution from the point of view of the end user and the bottom up view of looking at it from the point of view of the code and platform.
A QA analyst that can see the whole problem and whole solution in this way will have a much more informed opinion on the quality of the system.
Quality assurance architect
As the understanding of the external factors that affect the project grows, you start to move into the realm of the QA architect. What things are not spelled out in the requirements, but will affect the team’s ability to solve the problem?
For the time Web Part, the QA architect will be raising concerns such as what impact daylight savings time will have on the solution. Since not all locations observe DST, and those that do might not necessarily observe it at the same time, what affect does that have on how we address this problem?
If we are using a SharePoint list to hold the names of the locations with offsets from UTC, that list will have to be updated at least once a year to add the start and end date for each location, and is that an acceptable limitation?
Conversely, if we are pulling that information from an external source, what impact will that have on the system performance if that call happens from the client side vs. the server side? If the call is client side, are there limits to the number of calls that can be made each day? Because 600 employees using the intranet every day with this Web Part on the home page would be thousands and thousands of calls. But if the call is server side, how often is that call being made and how is it being cached? And will this be a one-off solution, or should it be built so that it can be used on multiple projects?
QA architect will also structure the QA process for the project based on the customer requirements as far as deliverables and burdens of proof are concerned. A project for a customer that is ISO 13485 compliant will require much more exhaustive levels of documentation of what was tested, how, and by whom than a customer that just wants a thumbs up/thumbs down assessment of the solution.
A QA architect is basically someone who has reached this level of understanding of not only the requirements, solution, and platform, but also the things that need to be accounted for in the project plan as far as tools, training, and resourcing is concerned, before we can estimate how long it will take to deliver the project, and how much it will cost to do so.
So, does your project need a QA tester, a QA analyst, or a QA architect? I would need to know more about the problem that you’re trying to solve to be able to tell you that.
