Diff it

Verification

How to verify behavior changes when CI is green

Turn a pull request’s behavior claims into targeted checks for success, rejection, and failure paths, even when the existing CI suite passes.

Read what the green result actually covers

A passing CI run tells you that the checks it executed passed on that revision. To connect that result to a product claim, inspect which jobs ran, what they asserted, and which relevant checks were skipped.

“The user sees the saved review” can require several things to work together: authorization, a database lookup, response serialization, and rendering. A unit test of the lookup does not by itself establish the browser outcome.

Start from the claim in the PR, then find the narrowest useful observation that would establish it.

Turn the claim into a small check matrix

Write the trigger and expected result before choosing a test tool. For a change that restores saved reviews, the matrix might look like this:

Example checks for reopening a saved review
SituationObserve
Authorized member opens an existing runThe saved result appears; no fresh analysis starts.
A member of another repository opens the same linkAccess is rejected and the response contains no saved review content.
The run no longer existsA clear missing-result state appears, with a useful next action.
The storage service is unavailableThe failure is visible; it is not presented as an empty history.

These are example requirements, not a report of checks performed by Diff it. Adapt the matrix to the contract of the PR you are reviewing.

Check at the level where the risk appears

A pure transformation often needs only a unit test. Permission enforcement usually benefits from exercising the request boundary. A navigation or rendering claim may need a browser check. Pick the level that observes the promised result without replacing the important part with a mock.

When practical, run a regression check on the previous revision too. If it passes there unchanged, it may not distinguish the behavior the PR claims to fix. For compatibility changes, inspect both the new case and an existing success case.

Keep test data small and controlled. For network behavior, a local test server can let you observe the received request without depending on a third-party service. The gRPC redirect walkthrough shows this pattern in a public regression test.

Keep proposed checks separate from completed checks

A useful review note says which revision was checked, how it was exercised, and what was observed. If a browser check was suggested but never run, leave it marked as proposed. If a service was mocked, state that limitation where it affects the conclusion.

For example: “On this head commit, the request test rejects another repository’s user and returns no review content. Browser error rendering is still unverified.” That is more actionable than “looks safe” or “CI is green.”

Explore Diff it’s demo to see behavior explanations alongside check statuses. A proposed check is a starting point for verification; it becomes evidence only after execution produces a result you can inspect.