Three Essential Black Box Test Design Techniques

Black-box testing is a method of software testing that examines the functionality of an application without peering into its internal structures or workings. This method of test can be applied virtually to every level of software testing: unit, integration, system and acceptance. (Wikipedia)

Ideally, you have an automated testing solution to handle repetitive manual testing, but to create those tests you still need to “poke around” and explore the system.  There are a wide variety of other black-box techniques that support test case design to consider, but where do you start? We've highlighted three of them below.

1. Error Guessing

The overall idea behind error guessing is to benefit from stakeholder experience with the same (or similar) systems as the one currently under test.  It should be used in supplement to structured test design techniques.  The execution of this method could be in the form of lengthy questionnaires given to all senior staff, or as simple as an informal interview over coffee with a grizzled implementation veteran.  The bottom line is, your team should attempt to avoid pitfalls experienced in previous software rollouts.

Error Guessing Example

A web design company has been tasked with upgrading a section of a client’s website where users can login and upload PDF scans of financial documents.  You meet with a project lead who is familiar with both the technical requirements of the login page and the subject matter, who advises the following:

  • Clients who try to login twice are logged out on the older systems
  • Uploading files with filenames that contain Chinese characters always causes a system crash
  • Uploading files over 1 gigabyte in size can result in an extreme slowdown with existing servers

These are test cases you should make sure are covered in your test plan.

2. Decision Tables

Decision tables are a great tool to match specific requirements with relatively complex business rules.  At a glance you can see all possible combinations of conditions and their expected results.  Conditions are usually expressed as true, false, and not applicable, though you should feel free to modify your table as needed.  Please note that a decision table is used to help design a test case and can be part of a complete test case, but it should not be considered a replacement for one.

This decision table was made for the client login mentioned in the “Error Guessing” example.  All possible combinations of logins are attempted for the username and password fields, along with the expected results.  We also took our project lead’s advice into account regarding users who log in twice being logged out in older versions, and so a row was added to check if a user who is already online can remain so if they login in again with valid credentials.

Decision Table Example

Valid PasswordTRUE
Already Logged InNYN/AN/AN/A
Expected ResultLogin SuccessLogin SuccessError: PasswordError: EmailError: Both

3. Equivalence Partitioning and Boundary Value Analysis

Both sound wordy, but together they can prove to be a powerful tool when creating test cases.  Equivalence partitioning is the process of dividing the test input data into ranges of values, and then selecting one representative value from each range to serve as the actual input.  The idea is that we want to pare down from a huge number of test cases, to a more manageable finite number while ensuring that the chosen representative inputs still cover all scenarios.  Boundary value analysis is used to test the boundary values of the ranges (or partitions), as failures tend to occur more frequently at the extremes.  Not only do these techniques help streamline the number of test cases, but they also make them more readable – which promotes collaboration.

Three-Value Boundary Analysis Example:

Expected ResultInvalidValidValidValidValidInvalid
Password Character Length345171819

Continuing our previous example, the client password field accepts a length of 4 to 18 characters - anything above or below results in a message advising the user of an incorrect password length.  We have three equivalency partitions:  Less than four, from four to eighteen, and above eighteen.  Since we don’t need to test every number in the range, a three-value boundary analysis was performed above by choosing the following input values:  The minimum - 1, the minimum itself, and the minimum + 1 and then we take the maximum – 1, the maximum, and the maximum +1.

These are 3 essential black box testing techniques you should add to your repertoire--happy testing!

This post was written by:

James Prior
Technical Pre-Sales Consultant

James has been working in software pre-sales and implementation since 2000, and has more recently settled into focusing on technical pre-sales. He takes care of our hands-on demonstrations, and eagerly awaits your request to see our Cycle test automation software in action. Drop him a line at: james.prior[at]tryonsolutions[dot]com.