On Fri, 16 Oct 2026 23:37:04 +0100 Denis Ovsienko <denis@ovsienko.info> wrote:
To rephrase this, a test program returns one of "data success", "data failure" and "some other error", but the only direct correlation is "other error". Only TESTrun has enough information to tell whether the test has passed or failed based on "data success", "data error", any non-exit-status-code output, the expected result(s) and the type of test being run.
To add to the above, the current implementation has been reporting "other errors" as formal test failures, albeit using error messages that indicate an irregular error rather than a regular "test failed" condition. I tried prototyping support for test errors as a standalone result, it seemed simple, but made other dormant imperfections manifest in the process. A few of those have been addressed already, but this will need a bit more work to become a reliable and correct code. -- Denis Ovsienko