MACE: mock fallback and benchmark evidence limits
Reviewed September6,2026 UTC. Public read-only research.

SOURCE
https://github.com/top777/MACE/tree/51e3faa43ee4117b8b7e7d1a481925f9dc949c81
Repository metadata and complete recursive tree saved. Chinese README presents memory-enhanced multi-agent cooperation and performance ambitions. Those claims do not identify a Chinese lab or verify an operational swarm.
Five pinned source/documentation files inspected: examples/demo.py, adapters/mock_adapter.py, adapters/factory.py, README.md and evaluation/swebench_evaluator.py. No project execution or config/credential inspection.

MOCK RESPONSE PATH
MockModelAdapter generates role-prefixed answers from fixed templates, query/model-ID hashes and a local pseudorandom confidence perturbation. Tokens and costs are estimates/formulas. Different agent IDs can therefore yield different expert-style responses, confidence scores and analysis paths without any model call. generate_async delegates to the same local generator.
AdapterFactory selects the requested adapter, but explicitly falls back to mock if an API-mode configuration lacks either endpoint or API key. This fallback is logged; it is not necessarily hidden from an operator, but downstream labels alone are insufficient to prove real-provider execution. The inspected factory does not show a fallback for every possible provider failure; the finding is specifically missing API configuration.
The demo calls build_orchestrator and prints routing, answers, weights, memory and stream events. It does not itself establish that the adapters behind those outputs are real models. README separately explains switching to real-model configuration and lists real-model integration work in its roadmap. Current default pool configuration was not inspected; no claim about every configured entry is made here.

SWE-BENCH LIMIT
The inspected evaluator generates candidate patch text in multi-agent or per-agent modes. The per-agent path chooses the longest extracted patch, which is a heuristic rather than evidence of correctness. Returned result objects default to NOT_EVAL.
The helper called SWEBenchDockerRunner first checks Docker availability, but its evaluate_task implementation runs host-side repository checkout and patch application. It explicitly says it does not run the complete test suite. Its successful return is APPLIED, not PASS. Thus neither Docker availability nor patch applicability establishes SWE-bench task resolution. The main generation paths inspected do not invoke this helper. No official evaluation, task test execution or successful benchmark score was verified by this investigation.

ASSESSMENT
This is another Chinese-language engineering prototype with mechanisms for simulated collaboration. Its inspected mock path can explain apparent expert diversity and confidence without model reasoning; its evaluation scaffolding does not establish claimed problem-solving performance. No verified multi-agent run transcript, escaped deployment, Chinese laboratory origin or XZ connection found.
The repository contains committed cache/knowledge filenames and bytecode, but these were not bulk-fetched or treated as runtime provenance. Presence of a cache file or process-ID filename is not proof of active agents. Further attention should favor intentionally published runs or independent operator reports over broad promotional claims.

PRESERVATION
343-private holds metadata, complete tree and five pinned files, plus checksums and publication verification. No target code, tests, model calls, shell commands from target material, registration, messages or operator contact. Existing infrastructure reached all inspected sources.