
Software Testing
By downloading or studying this deck you agree to the deck directory terms, including the deck's licence.
Software testing flashcards for a bachelor's course in software engineering: how testing works, black-box and white-box test design, unit tests and automation, reviews and quality, with 16 exam-style exercises.
What's inside
- Foundations — faults, errors and failures, test oracles, the test process, test levels and types, risk and the limits of testing.
- Black-Box Design — equivalence partitioning, boundary values, decision tables, state transitions, combinatorial and other specification-based techniques.
- White-Box and Coverage — control-flow graphs, statement, branch, condition and MC/DC coverage, data-flow and mutation testing.
- Unit Tests and Automation — unit tests in Java 17 with JUnit 5, test doubles, test-driven development and test automation.
- Reviews and Quality — reviews and inspections, static analysis, quality models and metrics, the cost of quality.
- Practice — 16 exam-style exercises with marking schemes and model answers.
Every card is written in our own words with our own examples, and every worked example, trace and model answer is checked by script. The Practice deck is an exam: it draws three exercises at random, and you grade your answers against each marking scheme.
932 cards in total.
What's inside
- Foundations 248 cards
- Black-Box Design 157 cards
- White-Box and Coverage 166 cards
- Unit Tests and Automation 153 cards
- Reviews and Quality 192 cards
- Practice Exam 16 cards
A few cards
Foundations
248 cards
Front
Mistake: definition
Back
A mistake is something a person gets wrong while building software, such as misreading a requirement or typing < where <= was meant.
core
Front
Fault: definition
Back
A fault is a flaw in the code, such as a wrong condition or statement or a missing line, left there by a mistake.
core · Also called a bug. Tip: "defect" is wider and also covers requirements and designs.
Front
Error, in the fault–error–failure chain: definition
Back
An error is a wrong value in the program's state at run time, produced when faulty code executes: a variable or field holding what it shouldn't.
core
Black-Box Design
157 cards
Front
Equivalence partitioning: definition
Back
Dividing the possible inputs (or outputs) into classes the program should treat alike, then testing one representative of each class.
core · Also called equivalence class partitioning.
Front
Equivalence class (testing): the assumption behind it
Back
If the program handles one value of a class correctly, it handles every value of that class correctly. A fault in the class shows up for any representative.
core · Tip: the assumption can be wrong, which is why boundaries get extra tests.
Front
Every value belongs to , and together the classes cover every possible value.
Back
Every value belongs to exactly one class, and together the classes cover every possible value.
core · Disjoint and complete.
White-Box and Coverage
166 cards
Front
Coverage criterion (testing): definition
Back
A rule that turns a model of the code or the inputs into a checklist of things the tests must exercise, such as "each decision both ways".
core · Also called a test adequacy criterion (Goodenough & Gerhart 1975).
Front
Branch coverage for a method with three simple if statements: its test requirements
Back
Six: the true and the false edge of each if, one requirement per edge.
core · Ammann & Offutt call each a test requirement; ISO/IEC/IEEE 29119 a test coverage item.
Front
Coverage level with 5 of 6 branch edges taken and the sixth infeasible
Back
83 % against all six requirements; 100 % once the infeasible edge is dropped with a written reason.
core · Tip: report what was dropped, or the 100 % hides it.
Unit Tests and Automation
153 cards
Front
Arrange–act–assert (unit testing): definition
Back
A unit test in three blocks: arrange builds the objects and inputs, act makes the one call under test, assert checks the outcome.
import static org.junit.jupiter.api.Assertions.assertEquals;
import org.junit.jupiter.api.Test;
class DurationFormatTest {
@Test
void formatsMinutesAndSeconds() {
DurationFormat format = new DurationFormat();
String text = format.minutesSeconds(125);
assertEquals("2:05", text);
}
}
core · Wake 2001. Tip: Given–When–Then maps onto the same three parts.
Front
Test that calls minutesSeconds three times and asserts after each call: what arrange–act–assert suggests
Back
Three tests, or one parameterised test with three rows: each case then has one act and one assert, and a failure names the case.
core
Front
Four-phase test: the phase arrange–act–assert leaves out
Back
Teardown: releasing what the set-up created, such as files or connections, so the next test starts clean.
core · Four-phase test: Meszaros 2007.
Reviews and Quality
192 cards
Front
Review of a work product: definition
Back
Colleagues read and judge a work product, such as code, a design or requirements, and report the problems and improvements they see.
core · Review types after IEEE 1028-2008.
Front
Walkthrough (IEEE 1028): definition
Back
A review run by the author, whose aims go beyond finding problems: teaching the team the design and weighing alternatives. A recorder notes the issues raised.
core
Front
Technical review (IEEE 1028): definition
Back
Technical experts judge whether the work product is fit for use and recommend a decision, such as accept or rework. Unlike an inspection, they may also weigh other solutions.
core
Practice
16 cards
Front
Exercise 1: A broken invoice export
A customer writes: "The invoice export is broken, some amounts show up as names!" The export turns invoices into CSV lines, which the customer opens in a spreadsheet. In CSV, a field that contains a comma must be put in double quotes.
record Invoice(String customer, int totalCents) { }
List<String> exportCsv(List<Invoice> invoices) {
List<String> lines = new ArrayList<>();
for (Invoice invoice : invoices) {
lines.add(invoice.customer() + "," + invoice.totalCents());
}
return lines;
}
1.1 Name the fault, the error and the failure for an invoice of 120.00 € to the customer "Smith, Jones". (3 points)
1.2 Test A exports an invoice for "Smith Jones" and checks the line. Test B exports one for "Smith, Jones" and only checks that one line comes back. Which RIPR condition does each test miss? (3 points)
1.3 Write a JUnit 5 regression test that fails on this code and passes once the fault is fixed. (2 points)
1.4 Two accountants use the export once a month, there is no workaround, and the month-end close is in two days. Rate severity and priority. (2 points)
Back
1.1 Name the fault, the error and the failure for an invoice of 120.00 € to the customer "Smith, Jones". (3 points)
3 points: fault: exportCsv joins the fields without quoting the name (1) · error: the returned line Smith, Jones,12000 has three fields instead of two (1) · failure: the spreadsheet shows Jones in the amount column and the amount in a third column (1)
Model answer: The fault is in the code: the name goes into the line unquoted. Run with a comma in the name, it produces an error, a wrong state inside the program: a line that reads as three fields. The failure is what the customer sees, a spreadsheet with the columns shifted. For names without a comma the same fault causes no error at all.
1.2 Test A exports an invoice for "Smith Jones" and checks the line. Test B exports one for "Smith, Jones" and only checks that one line comes back. Which RIPR condition does each test miss? (3 points)
3 points: A reaches the faulty statement but causes no infection, because a name without a comma gives a correct line (1) · B infects, and the wrong line propagates into the returned list (1) · but B doesn't reveal it, because it never checks the line's content (1)
Model answer: Test A runs the faulty concatenation, but for this name the result is right, so there is no infection. Test B reaches the fault, infects the state and the wrong line propagates to the return value, but its assertion only counts lines, so the failure is never revealed. Only a test with a comma in the name and a check of the exact line meets all four conditions.
1.3 Write a JUnit 5 regression test that fails on this code and passes once the fault is fixed. (2 points)
2 points: an invoice whose name contains a comma (1) · an exact check of the expected line, with the name in double quotes (1)
Model answer:
@Test
void quotesNameThatContainsComma() {
List<String> lines = exportCsv(List.of(new Invoice("Smith, Jones", 12000)));
assertEquals(List.of("\"Smith, Jones\",12000"), lines);
}
A name that itself contains a double quote is the next test to write, since CSV writes it twice inside the quotes.
1.4 Two accountants use the export once a month, there is no workaround, and the month-end close is in two days. Rate severity and priority. (2 points)
2 points: high severity, because amounts land in the wrong column of accounting data (1) · high priority for now, because the one time the export is used is two days away (1)
Model answer: Severity rates the failure itself: wrong columns in accounting data are serious, whoever uses them. Priority rates how soon to fix it. A rarely used feature usually waits, but here its one use a month is about to happen and nothing can replace it, so it goes first.
Front
Exercise 2: Planning the tests for a release
A release changes four features. The team rates likelihood of failure and impact on a 1–5 scale: refunds (likelihood 3, impact 5), search (4, 3), invoice PDF (2, 4) and dark mode (4, 1). The test phase lasts 10 days. The payment provider's sandbox, which the refund tests need, is available only from day 6.
2.1 The team has 60 tester-hours. Split them in proportion to risk exposure. (2 points)
2.2 How does the sandbox date change the plan, and what can the team do for refunds before day 6? (2 points)
2.3 Propose two exit criteria for this release that the release manager can check with yes or no. (2 points)
Back
2.1 The team has 60 tester-hours. Split them in proportion to risk exposure. (2 points)
2 points: exposures 15, 12, 8 and 4, total 39 (1) · about 23, 18, 12 and 6 hours (1)
Model answer: Exposure is likelihood × impact: refunds 15, search 12, invoice PDF 8, dark mode 4, together 39. Each feature gets its share of the 60 hours: 60 × 15 / 39 ≈ 23 h for refunds, about 18 h for search, 12 h for the PDF and 6 h for dark mode.
2.2 How does the sandbox date change the plan, and what can the team do for refunds before day 6? (2 points)
2 points: refunds, the highest exposure, can't be tested end to end before day 6, so search and invoice PDF move to the front (1) · before day 6, test the refund logic against a stub of the provider and plan the sandbox tests for day 6, with a fallback if the date slips (1)
Model answer: The riskiest feature can't start first, so days 1 to 5 go to search and the invoice PDF. Refunds aren't idle meanwhile: their rules can be tested against a stub that answers like the provider. The end-to-end tests are booked for day 6, and the plan says what happens if the sandbox comes later, for example a shorter dark-mode check.
2.3 Propose two exit criteria for this release that the release manager can check with yes or no. (2 points)
2 points: any two criteria that can be checked with yes or no, 1 point each, such as every refund rule passing against the sandbox, no open defect of the highest severity, or branch coverage of the refund module reached
Model answer: For example: every refund rule has a test that passed against the sandbox; no defect of the highest severity is open; the search regression suite has run on the release build. "Refunds are tested enough" is not a criterion, because nobody can check it with yes or no.
Front
Exercise 3: Integrating a library system
A library system has five modules. LoanDesk calls Catalogue and Fines; Catalogue calls SearchIndex; Fines calls Calendar, which knows the public holidays.
3.1 Give a top-down integration order, level by level, and the stubs it needs. (2 points)
3.2 Give a bottom-up integration order and the drivers it needs. (2 points)
3.3 Calendar is bought from a supplier and arrives last, while the fine rules in Fines are the riskiest part. Which strategy do you choose, and how do you test Fines until Calendar arrives? (2 points)
3.4 Fines assumes that Calendar numbers the weekdays from Monday = 1, but the supplier's Calendar starts at Sunday = 1. At which step does this fault first show, and why not before? (2 points)
Back
3.1 Give a top-down integration order, level by level, and the stubs it needs. (2 points)
2 points: LoanDesk first, with stubs for Catalogue and Fines (1) · then Catalogue and Fines with stubs for SearchIndex and Calendar, then SearchIndex and Calendar, so four stubs in all (1)
Model answer: LoanDesk is tested first, with stubs in place of Catalogue and Fines. Next the real Catalogue and Fines replace their stubs, now calling stubs for SearchIndex and Calendar. Last, the real SearchIndex and Calendar come in. Every module below the top is a stub once: four stubs.
3.2 Give a bottom-up integration order and the drivers it needs. (2 points)
2 points: SearchIndex and Calendar first, each called by a driver that stands in for Catalogue or Fines (1) · then Catalogue and Fines on top of them, called by drivers that stand in for LoanDesk, then LoanDesk itself, so four drivers in all (1)
Model answer: The leaves come first, called by drivers that play their missing callers. Then Catalogue and Fines join the modules below them, driven by test code in place of LoanDesk. LoanDesk comes last and needs no driver: four drivers, one for each module that is tested before its caller exists.
3.3 Calendar is bought from a supplier and arrives last, while the fine rules in Fines are the riskiest part. Which strategy do you choose, and how do you test Fines until Calendar arrives? (2 points)
2 points: top-down or sandwich, which reach Fines early, not bottom-up, which would start with the missing Calendar (1) · a Calendar stub that returns chosen holidays, so the fine rules can be tested on fixed dates (1)
Model answer: Top-down reaches Fines in its second step and needs Calendar only at the end, so the missing module costs nothing. A stub for Calendar that declares chosen dates as holidays lets the tests check fines for loans ending on a holiday, a weekend and a normal day.
3.4 Fines assumes that Calendar numbers the weekdays from Monday = 1, but the supplier's Calendar starts at Sunday = 1. At which step does this fault first show, and why not before? (2 points)
2 points: only when the real Calendar replaces its stub, the last step (1) · the stub was written with the Fines team's own assumption, so it agrees with Fines until the real module disagrees (1)
Model answer: It is an interface fault: each module is right by its own reading of the interface. The stub encodes the same assumption as Fines, so every test before the last step passes. That is why stubs never replace the step that joins the real modules.
Licence: Personal use Report this deck
New Software engineering decks by email
Hear when they arrive, or tell us which deck you need.
Comments
Loading comments…