The Joy of testing

src: The Joy of Testing by Victor Rentea - 2024-10-09

Abstract

Why write tests?

• Prove it works today, including edge cases.
• Fast feedback to preserve focus.
• Refactor without fear tomorrow, if functional tests.
• Up-to-date spec running on CI, if functional tests.
• Clarify requirements, if test-early.
• Design feedback: hard to test → poor design, if test-early.

Qualities of a test:

• Sensible: detects changes in behavior.
• Functional: not coupled to implementation (resilient to refactoring).
• Specific: fails for one clear reason.
• Expressive: use domain terms with a high signal/ratio ratio.
• Automated: easy to run by anyone with no/minimal manual setup.
• Repeatable: produces the same outcome if rerun.
• Isolated vs other tests / external system state.
• Minimal: includes only relevant stuff.

Prefer branch coverage vs line coverage, as the latter will not tell you the whole story.

Consider using mutating testing, like PIT Mutation Testing, but:

• Can report false positives.
• Takes long time to run if applied on lots of code.
• Use it:
  • To learn.
  • On super-critical code (target a single package).

Breakpoint-debugging an issue is antisocial behavior as it requires deep focus and drains lots of brain energy. Instead, fix a bug like a pro:

• Reproduce the bug (twice).
• Write a test for it (it must fail!)
• log.debug instead of breakpoint.
• Find the cause > kill bug > Test becomes green.

Prefer explicit tests:

assertEquals("Greetings" + name, logic.getGreetings(name));
// Prefer this:
assertEquals("Greetings John", logic.getGreetings(name));

Use brain-friendly test names: Naming Test Classes and Methods | Codurance.

Use assertj instead of native junit assertions because it brings more context in the error messages.

You can use InvocationInterceptor interface to create a junit extension.

public class TimeExtension implements InvocationInterceptor {
  @Override
  public void interceptTestMethod(Invocation invocation, ReflectiveInvocation ref) {
    try (...) {
       invocation.proceed();
    }
  }
}
// Then, in your tests
@RegisterExtension
TimeExtension timeExtension = new TimeExtension("2023-12-25");

⚠️ This is not thread safe!

You can use assert softly to assert all assertions in one go. You can also inject SoftAssertions so you don’t need to create it for all tests:

@ExtendWith(SoftlyExtension.class)
class FoobarTest {
  @InjectSoftAssertions
  SoftAssertions soft;
  
  @Test
  void someTest() {
    soft.assertThat(....);
  }
}

Parameterized tests are powerful to reduce boilerplates. BUT do not abuse them, i.e. makes multiple argument. You can mitigate by create a class containing all the arguments in the test. Some best practices for parameterized tests:

• orthogonal parameters:
  • ex for 3 input params:
    • Good: a={1, 2, 3}, b={7, 8}, c={T, F} = 3*2*2 = 12 valid combinations
    • Bad if only 3 valid combinations
• Few parameters < 3-5.
  • More → group them in a class/record with named fields.
  • Complex input structure * many tests → consider file-based tests.
• Avoid boolean-riddled test.
  • if (param1) exception was thrown
  • if (param2) when(mock)
  • if(param3) verify(mock)

Testing styles:

• Output-based: assert the returned value.
  • Enough when tested code has no side effects, pure functions.
• State-based:
  • Assert fields of tested stateful object.
  • Assert fields of collaborator objects (e.g. passed as params).
  • Assert external state in DB/files.
• Communication-based:
  • Verify interaction with collaborators.

For legacy code testing practices:

• dark techniques to write temporary, ugly tests
  • break encapsulation of tested code (private → public)
  • partial mock (@Spy) internal methods
  • mock statics & control global state
• pre-testing refactoring (unguarded by tests)
  • exploratory refactor (e.g. 30 minute > undo!): look for seams (“fracture planes”)
  • tiny, safe baby-steps refactoring with IDE, pairing or mobbing
• extract and test code with a clear role = maintainable tests
• characterization tests: capture input/output as a black-box

Gherkin Language to express business logic by non-technical people. Use cucumber testing tool to use this feature file. Developer’s dream: business people will maintain the .feature files. But it never works. It could work if the dev works pair on them (e.g. after an event storming). When should you use a .feature?

• ✅ hesitating / conflicting requirement (CoverYourArse principle)
• ✅ complex rule, to better understand via more examples
• ✅ business-critical features to avoid expensive confusions
• ✅ table-like test data → scenario outline
• ❌ business disengages: trivial or too technical features?

Gherkin Best practices:

• Use ubiquitous language understood by tech and non-tech: PO, BA, QA.
• Focus on behavior (aka functional tests), hide details behind steps.
• Keep steps implementation simple; expose relevant parameters.
• UI scenarios (Selenium) can be fragile and slow → fire at API-level.
• DRY: use shared steps, Background:, Scenario Outline, and @tags.
• Mock external services for faster and more reliable tests.
• Share state between step classes via dependency injection.
• Prevent state leak: cleanup test environment after each scenario.
• Runs on CI (even if slow).

When only developers work with Gherkin files, consider building a test DSL instead. No need for an additional abstraction layer.

Use @Nested tests to separate behaviors, but it can grow. Instead, you can create hierarchical fixtures by creating subclasses.