nesquena/hermes-webui

Test robustness follow-up: test_invalidate_models_cache_direct has the same intra-file order dependency (#7100/#7101)

Open

#7,104 opened on Aug 17, 2026

 (1 comment) (0 reactions) (0 assignees)Python (2,399 forks)github user discovery
bughelp wantedsprint-candidate

Repository metrics

Stars
 (17,426 stars)
PR merge metrics
 (Avg merge 14h 31m) (314 merged PRs in 30d)

Description

Follow-up to #7100 / #7101. #7101 made test_mtime_invalidation and test_glm_5_3_in_models_payload_for_zai_provider order-independent, but a third test in the same file has the same intra-file ordering dependency and was not covered:

tests/test_ttl_cache.py::test_invalidate_models_cache_direct — fails when run in isolation (pytest tests/test_ttl_cache.py::test_invalidate_models_cache_direct) with an AssertionError, but passes when the whole file runs (pytest tests/test_ttl_cache.py → 5/5). It depends on cache state established by an earlier sibling test, so a shard that picks it up without its predecessors fails it. Verified: fails identically on clean origin/master in isolation, so it's not caused by any feature PR — it's the same shard-split hermeticity gap #7101 addressed for its siblings.

Fix: apply the same self-priming pattern #7101 used — have test_invalidate_models_cache_direct set up its own _available_models_cache precondition (call the priming path in the test body or a shared fixture) instead of relying on sibling execution order. It should pass in isolation and under sharding.

Not release-blocking (green in CI's normal full-file run), but it produces a spurious failure under SHARDED=1 gate runs.

Contributor guide