AI search service · EBS Lan
STATUS: PROD
AI-Snippets
Snippet answers over EBS Lan search results: cache, BM25+RRF, quality gate, circuit breaker, graceful degradation. Stress test: 13.2 hours without a single infrastructure error.
SOURCE: MANUAL README
AI-Snippets is a service that shows a short snippet answer with source markers over the EBS Lan (Russian academic e-library platform) search results. If something fails, the snippet simply doesn't appear — the results and search itself keep working as usual.
How it works
- Pipeline: check the two-level cache (exact + semantic) → classify the query → expand the query → multi-source search over Elasticsearch → assemble context via BM25 and RRF → generate on vLLM → final quality gate.
- Polling scheme via Redis, distributed semantic cache for similar queries, circuit breaker, and graceful degradation: on any problem the snippet is not shown — with no error surfaced.
- Serious tuning of vLLM parameters for snippet performance on a specific server configuration.
Proven under load
A stress test on the target GPU hardware confirmed production readiness:
- 13.2 hours of continuous load, 55,200 requests, zero infrastructure errors (the service did not crash or return 5xx);
- quality gate OK rate of 87.3%: strict quality filtering, with the remaining 13% being correct graceful-degradation triggers with no error surfaced;
- median client-side latency of 4.6 seconds against a 5 s SLA.
Stack: Python · FastAPI · RabbitMQ · Redis · Elasticsearch · vLLM · sentence-transformers · Prometheus