James Bach on Questioning the Norms of Software Testing

Speed and agility reign supreme in software development. However, this drive can sideline the role of testing: Bugs slip through the cracks, security vulnerabilities go unnoticed, and user frustrations mount. Amidst this turmoil, James Bach stands as the voice of reason—and dissent. From his unconventional beginnings as a computer salesman to his rise as an authority in the testing field, James shares his journey in the industry and the necessity for testers to challenge the norm.

Edited transcription

James Bach doesn’t fit the mold of the typical software tester. A self-proclaimed “proud high school dropout,” his journey defies convention. Starting as a teenager programming video games, he got his first job as a computer salesman. “I never managed to sell a single computer and I broke one computer,” he recalls. Yet, James’ self-taught knowledge of assembly language landed him a job in software development by one of the store’s clients, despite lacking any formal experience in the industry. “If you don’t have a lot of experience in the field, you don’t have to worry too much because just be hungrier and be a little less expensive than the experienced people,” James advises.

“I’m a natural complainer; I look at everything as problems,” James admits, arguing that this contrarian perspective is precisely what makes him a good tester. “Serious projects doing serious software should want to have independent software testers, not just a bunch of cheerleaders,” he emphasizes.

The challenges of modern testing

Since James’ early days, the software landscape has undergone a seismic shift, and testing was not left unturned. “There are two main pressures on testers today,” he explains.

The first is the loss of management. Before, it was common to have test managers protecting testers’ autonomy, something often absent in agile environments. The modern trend of embedding testers directly within development teams, fueled by the idea of fostering collaboration, diminishes the independence of testers and their impartiality of judgment. “When testers work directly for developers,” he argues, “they end up working for people who may not fully understand the value and nuances of testing.” Tests might feel that their job of finding flaws in code negatively affects developers who have to fix it. 

The second pressure on testers stems from the online, continuous development nature of modern software. In the past, when internet updates were not the norm, projects had more breathing room before customer release, allowing testers time for thorough exploration. But today, the rush to deploy new features often clashes with rigorous testing. “Prioritizing constant deployment can inadvertently prioritize crashing deployment,” James says. While agility and speed are important, they should not come at the expense of thorough testing and careful judgment. Development focused on rapid release risks compromising quality. “You can put out bad software today, and people might just shrug and say, ‘It’s alright,’ ” James admits. However, effective testing goes beyond product quality; it can also mitigate major security risks. 

In like manner, the pressure to release faster fuels the trend toward automation, but James remains skeptical about its ability to fully replace human testers. “Automation is great for repetitive tasks,” he concedes, “but human judgment plays a massive role in testing: A machine can’t replicate the critical thinking and creativity needed to uncover truly unexpected issues.” 

Moving beyond quality

“Testing is not about quality,” James declares, shattering a fundamental tenet of the field. For him, it’s not about ticking boxes or meeting pre-defined standards, but a relentless pursuit of truth, a quest in which the tester has to uncover the hidden flaws and discrepancies between “what the software should be” and its current reality. “A lot of times what’s written down is obsolete,” James explains. Specifications evolve, user needs shift, and unexpected scenarios emerge. The “truth” on product development is dynamic, requiring a shift in focus from predefined quality metrics to a deeper understanding of the software’s true state.

Hence, due to software’s nature, James views testing as an “investigation” rather than a technical exercise. He encourages testers to adopt a detective’s mentality, actively looking for trouble, for the cracks in the facade of functionality. He believes issues are inherent, waiting to be exposed, and ignoring them creates a false sense of security.

Building a safe space is a requirement for this kind of investigative journey. Accountability, according to James, isn’t about blind adherence to authority but about fostering an environment where testers can speak their minds without fear of reprisal. He differentiates between “responsible testers,” who own the testing strategy, and “supporting testers,” who execute it. Both roles are crucial, but only the responsible tester holds the ultimate accountability for uncovering the truth. 

What’s more, James argues both testing roles should respond to a testing manager, who acts as an intermediary between them and their tasks and programmers, allowing them to concentrate on their work and filtering external feedback. To further keep testing and development separate, managers from both groups report to a shared business unit manager. “They can cooperate but maintain autonomy,” James reiterates, emphasizing the importance of independent judgment.

And this autonomy extends beyond technical expertise. James champions the value of non-technical testers, arguing that a monoculture of coding hinders diverse perspectives and ultimately inhibits excellence in testing. “We need diversity,” he proclaims, “if we’re going to have excellence.”

Rethinking testing education

To this end, James advocates for a “philosophy that encourages diversity” in testing and welcomes individuals with varied skill sets and backgrounds, believing they enrich the field with unique perspectives. In this regard, James recalls his experiences hiring a philosophy graduate and a lawyer with a CS degree, both thriving as testers thanks to their non-technical strengths. He even recounts mentoring a restaurant owner and former 911 call agent into a successful testing career, bypassing the traditional “qualifications.”

Still, broad perspectives and critical thinking are often excluded by the industry’s obsession with CS degrees and programming experience. His own brother serves as a poignant example of this exclusion. A successful tester at Microsoft in multiple roles, James’ brother’s lack of a computer science degree disqualifies him in the eyes of standardized recruitment processes.

Tech giants like Google and Facebook, James believes, are to blame for these “crazy ideologues” of pushing CS degrees instead of fostering true testing talent and product quality. Their claimed success, he suggests, rests more on monopolies than truly exceptional software quality.

Taking Testing Seriously

While other testing books “wave their hands” at automation, James affirms his upcoming release, Taking Testing Seriously, promises a refreshing dive into practical, hands-on skills. Rather than following the latest trends or buzzwords. James aims to create a resource for aspiring “excellent testers,” individuals who value learning and honing essential skills above all else.

But before diving into tactics, James acknowledges the need for a solid foundation. Key concepts like “responsibility,” “system,” “quality,” and “risk” all carry subtle nuances within his methodology, requiring clear definitions to establish a shared understanding. He throws shade at existing testing discourse for its “newly written code bias,” criticizing the neglect of integration challenges posed by pre-existing code from platforms like GitHub. Taking Testing Seriously, James guarantees, breaks free from this narrow focus and embraces the real-world complexities of testing across diverse contexts, from medical devices to embedded systems.

And it’s not a solo journey. Taking Testing Seriously will include guest chapters from fellow practitioners who utilize variations of the Rapid Software Testing (RST) approach. This diverse perspective ensures the book reflects the richness and adaptability of RST in action.

The bottom line

Taking Testing Seriously will be published by Wiley towards the end of 2024. Besides, James is currently penning two more volumes –one on rapid software testing and another detailing his unique methodology honed over two decades— with no release date yet.

Visit James’ website Satisfice and follow him on X (formerly Twitter), at @jamesmarcusbach, and on Linkedin.