PR State:
merged
PR:
#47
Commit:
92bc1373119cf20682688a540456b1a2d76ed74a
[ADD] viin_ai_*: no-key AI/LLM test framework (helpers + test_viin_ai module + docs)
Give every viin_ai_* module a way to test LLM-dependent logic deterministically
without an API key and without network, and unlock a live no-key demo - without
weakening the BYOK invariant (no fabrication code reachable in a customer prod DB).
A code-grounded audit found 5 disconnected stub patterns across 15 test files
(3 _FakeResponse copies, 2 byte-identical fake_embed, 3 raw-SQL pgvector blocks,
no canonical response shape). See docs/ai/testing.md + ADR-016 for the rationale.
L1 - shared unit-test helpers in viin_ai_base/tests/ (not on the runtime path):
- fake_response.py: one canonical FakeResponse.
- builders.py: make_completion_result / make_llm_call_result / make_stream_chunks /
make_fake_embed_fn - single-sourced result shapes (Appendix A).
- common.py: AIStubMixin with _stub_llm / _stub_stream / _stub_provider context
managers patching the registry-class agent seam (_do_llm_call / _do_llm_stream),
plus seed_pgvector_rows. 10 legacy test files converge onto these; the duplicated
_FakeResponse / fake_embed / inline result dicts and raw-SQL blocks are deleted.
L2 - test_viin_ai: the cluster's TEST-ONLY module (Odoo test_* convention, cf.
test_pylint) the customer NEVER installs (installable but no auto_install, in no
product dependency closure, nothing depends on it; dev/demo/CI only -> customer prod
carries zero fabrication code). Two roles: (1) a gated mock LLM provider; (2) the home
for test cases that cannot live in a production module (cross-module / full-stack /
mock-dependent). It overrides one seam (viin.ai.provider.make_request) for
api_protocol='mock'; every protocol-keyed helper already falls through to the
openai_compat shape, so cost-cap, PII redaction, normalize, usage.log and trace all run
real - only the network call is faked (a local _MockResponse; models/ never imports the
tests-only FakeResponse). Fail-closed gate (SSOT in models/gate.py): an @api.constrains
on the vendor (a 'mock' vendor cannot persist outside test/demo) AND a refusal in
make_request (never fabricates when the gate is closed). Gate open iff --test-enable OR
registry test-mode OR ir.config_parameter test_viin_ai.enabled='1' (demo data sets it).
11 tests: 6 fail-closed gate proofs + 5 no-key full-stack proofs (real agent.run /
call_embedding -> mock -> usage.log written).
Also harden test_provider_inherits_mixin_and_thread: it asserted on
viin.ai.provider._inherit (which reports only the last-loaded module's own
declaration, so test_viin_ai legitimately flipped it) - rewritten to assert the
resolved MRO + the capabilities the bases contribute; still fails if a base is
genuinely removed.
Docs: docs/ai/testing.md (SSOT decision tree + helper API + coverage fence + @tagged
rule + Runbot lint/test commands with the W503/W504 trap called out), ADR-016
(decision + gated-placement rationale + L3 rejection), pointers from AGENTS.md §6,
docs/ai/README.md, docs/decisions/README.md.
Verified no-key: Community 376/376 + EE 304/304 (with viin_workflow_automation present),
0 failed 0 error; flake8 + pylint_odoo (exact Runbot config) clean, 10.00/10, no
lint suppressions. Live demo: a real sale-quotation approval confirm -> cron ->
agent.run -> mock provider yields a genuine advisory card (no network, no key, no
canned message_post).
Branch:
17.0