Testing with AI: the determinism problem

AI testing breaks the assumption every automated suite was built on, because the same input no longer produces the same output. Anthropic documents that temperature 0.0 is still not fully deterministic, and OpenAI's seed parameter only promises mostly reproducible results. What works instead is a layered belt of checks that catches regressions without tripping over ordinary variance.

November 11, 2025 4 min 234

Methodologies

Contract Testing with Pact for Microservices

With 30 or more microservices, end-to-end tests become slow, fragile and impractical. Pact implements consumer-driven contract testing: the consumer defines what it expects, the provider verifies it in its own CI pipeline, with no shared environment needed. The result is integration proof in seconds, not minutes.

July 22, 2023 5 min 244 4.6