Why most RAG systems fail

If you’ve tried to build an agentic RAG system that reliably answers real questions, you know the frustration. Typical pipelines vectorize a query, retrieve some ‘similar’ text chunks, and hope the model stitches together a useful reply. Often you get irrelevant snippets, hallucinations, or answers that ignore structure and context.

Elysia, an open-source Python framework from Weaviate, takes a different approach. Instead of piling on more general-purpose intelligence, it restructures how agents interact with data: controlled decision flows, adaptive data presentation, and upfront data understanding.

Note: Python 3.12 required

Elysia’s three core ideas

1) Decision trees for agent behavior

Rather than exposing every tool and letting the agent roam freely, Elysia wraps decision-making in explicit nodes. These nodes form a guided flowchart: each node records context, choices, and the rationale for the next step. That makes the agent’s path auditable — you can see the exact route taken and debug where the logic went wrong.

A practical behavior improvement is the ‘impossible flag’: when the agent recognizes a task it cannot perform (for example, searching car prices inside a makeup dataset), it marks it as impossible and stops wasting effort.

2) Smart data-source display

Elysia inspects the shape of your data first — fields, types, relationships — and picks an appropriate presentation from a small set of templates. Instead of dumping paragraphs, the framework can render product cards for e-commerce, ticket layouts for GitHub issues, or real tables for spreadsheets. The result is answers that not only read better but reflect the underlying structure of the data.

3) Data expertise before search

Before searching, Elysia analyzes the database: what fields exist, value ranges, relations between entities, and which queries are meaningful. It can summarize collections, generate metadata, and choose display formats based on that analysis. This upfront expertise helps it find and present relevant facts rather than noisy text fragments.

How Elysia improves results

Learning from feedback

Elysia uses user feedback to improve future responses, but it does so in a scoped way — feedback from one user doesn’t pollute others’ results. That lets teams use smaller, cheaper models while still benefiting from real-world successful answers.

On-demand chunking

Instead of chunking everything up front (which consumes storage and creates awkward splits), Elysia searches full documents first and only splits a document when necessary. Chunking decisions are driven by the user query, which reduces storage and yields more coherent retrievals.

Model routing by complexity

Not all tasks need the biggest model. Elysia routes simple tasks to smaller, cheaper models and reserves larger models for complex reasoning. This improves latency and lowers cost without sacrificing quality.

Getting started

Installation is straightforward:

pip install elysia-ai
elysia start

That launches both a web interface and the Python framework.

For developers who want to expose custom tools, here’s the example provided:

from elysia import tool, Tree

tree = Tree()

@tool(tree=tree)
async def add(x: int, y: int) -> int:
    return x + y

tree("What is the sum of 9009 and 6006?")

If you have a Weaviate collection, usage looks like this:

import elysia
tree = elysia.Tree()
response, objects = tree(
    "What are the 10 most expensive items in the Ecommerce collection?",
    collection_names = ["Ecommerce"]
)

Real-world use case: Glowe skincare chatbot

Glowe uses Elysia to handle nuanced product recommendations. Users can ask things like ‘What products work well with retinol but won’t irritate sensitive skin?’, and the system considers ingredient interactions, user preferences, and availability. This goes beyond keyword matching: it reasons about relationships between ingredients, user history, and product attributes.

Where Elysia fits

Elysia is Weaviate’s attempt to move beyond the standard ask-retrieve-generate pattern by combining transparent decision-tree agents, adaptive displays, and feedback-driven learning. It aims to give developers a foundation for building RAG applications that understand both the user’s intent and the structure of the data. The framework is still in beta, so real-world results will determine how significant the improvements are over existing RAG systems.

For more technical details and source code, check the project’s GitHub and the Weaviate blog post announcing Elysia.