05. Testing & Validation

How to Test Your Open Source Project the Right Way

Table of Contents
In: 05. Testing & Validation

Why This Stage Matters

Testing isn't about writing the most tests. It's about making sure your project actually works, reliably, accessibly, and as expected, for the people it's meant to help.

In Open Source, broken features mean lost trust. Confusing interfaces mean lost users. And tests written only for coverage metrics? They won’t save you when something important breaks in the real world.

This stage is about building confidence, in your code, in your decisions, and in your users' experience.


Step 1: Understand Why Testing Matters

When you test with purpose, you’re doing more than checking if code runs. You’re asking:

  • Does this do what I think it does?
  • Will it break in ways I didn’t expect?
  • Will it still work after I change something later?
  • Can someone else trust this enough to use it or contribute?

Good testing protects your users, your contributors, and your time.


Step 2: Know the Types of Tests (And When to Use Them)

You don’t need to use every kind of test right away, but it helps to know your options.

Unit Tests

Test small pieces of code in isolation (like a function or component).

  • Fast to run
  • Great for logic-heavy code
  • Example: formatDate("2025-01-01") returns "January 1, 2025"

Integration Tests

Test how different parts of your system work together.

  • Slower than unit tests
  • Useful for API calls, DB logic, or UI + backend flows
  • Example: Creating a user updates the database and sends an email

End-to-End (E2E) Tests

Test full user journeys, usually through a browser or CLI.

  • Simulate real behavior
  • Catches user-facing bugs
  • Example: A user signs in, creates a quiz, and downloads a PDF

Accessibility Tests

Check if your app works for people using assistive technology.

  • Can be automated (with tools like Axe)
  • Manual checks are still important
  • Example: Can this button be activated with a keyboard?

Each of these catches different issues, together, they give you real confidence.


Step 3: Write Fewer, Better Tests

It’s tempting to chase 100% test coverage. But not every line of code needs a test — and not all tests are useful.

Instead, aim for:

  • Testing the paths people actually use
  • Covering the features you’d be scared to break
  • Writing tests that describe the why, not just the what

If you're only testing what’s easy to test, you’re not testing what matters.


Step 4: Mix Automation With Real Feedback

Even great automated tests can miss real-world issues. Always combine testing with human validation:

  • Try using your app on a slow connection
  • Watch someone else try to complete a task
  • Use a screen reader or keyboard-only navigation for 5 minutes

This kind of manual testing uncovers usability, accessibility, and performance problems that automation can't catch.


Step 5: Test With Users in Mind

The goal of testing isn’t just correctness, it’s usefulness.

Ask yourself:

  • Will someone new to this project understand what to do?
  • Will someone with a disability be able to complete a task?
  • Will this feature break if used differently than I expected?

If the answer to any of these is “I’m not sure,” you’ve found something worth testing.


TL;DR

  • Don’t write tests just for coverage, write them to protect what matters
  • Use a mix of unit, integration, E2E, and accessibility testing
  • Focus on real-world scenarios, not just ideal cases
  • Combine automation with manual testing to catch what tools miss
  • Test like someone else’s experience depends on it, because it does
Written by
Cory Fail
Cory Fail leads the Software for Progress Foundation, helping developers build Open Source tools for education, accessibility, and social good through mentorship and community support.
Comments
More from Software for Progress Foundation
Great! You’ve successfully signed up.
Welcome back! You've successfully signed in.
You've successfully subscribed to Software for Progress Foundation.
Your link has expired.
Success! Check your email for magic link to sign-in.
Success! Your billing info has been updated.
Your billing was not updated.