Similarly,
Openrag: meritis wants facilitate test:
Meritis recently launched Openrag, an open source comparator in order to assess the performance of RAG (Retrieval Augmented Generation) architecture. Therefore, Yes. Moreover, the consulting company distinguishes “naive” RAG systems from advanced RAGs and graphrag, as it discerns different cases of use of technology in business.
Meritis believes that the most useful method must be identified. In addition, analyzing the technical, human costs, implementing it by ensuring that it maintains performance. Nevertheless, And this is not obvious, according to Benoît Joly, R&D engineer at Meritis.
“A RAG system is not” plug and play “,” he says. Nevertheless, “There is no miracle solution to its use case either. Meanwhile, There are things that can work, there are settings that can help improve the implementation of solutions, etc. In addition, But that it is not something magical, as Chatgpt could have been perceived, openrag: meritis wants facilitate test ”he adds.
Due to the many components. Furthermore, the expertise he claims, the deployment of a RAG mechanism would be an obstacle to the test tests.
“There are, in my opinion, two brakes on the adoption of AI solution in business,” says Benoît Joly. Similarly, “The first is the expensive entrance ticket. Similarly, […] The second brake is the lack of concrete evidence ”.
Hence the birth of Openrag, a toolbox and an application to assess research performance increased by generation. However, “We can do this pre -selection work of a certain number of Rags which seem relevant to us. Therefore, make them ultimately accessible,” adds Théodore Boullier, director of innovation at Meritis. Therefore, “Customers do not only want to test performance, assess the costs, but also the environmental impact.”
Not one, but rag architectures
The framework openrag: meritis wants facilitate test therefore makes it possible to deploy around twenty pipelines for 11 types of RAG.
“A naive RAG mechanism is the fact of connecting a documentary base to a conversational agent,” explains Benoît Joly. Moreover, “We cut the documents in chunks-from small paragraphs-, vector representations of these chunks (the famous embedding) are established,” he recalls. For example, “When the user asks a question. Nevertheless, an embedding of this question is also created in order to identify the closest paragraphs semantically in a vector space”.
The selected paragraphs are provided to the large language model that propels the conversational agent to respond to the request.
“This approach works well for simple and precise questions,” says the R&D engineer. Therefore, “On the other hand, it shows its limits when the question is complex or abstract”.
Very quickly, publishers and researchers recommended adding a Reranking function. Similarly, It is a question of deploying an openrag: meritis wants facilitate test algorithm which can sort in a larger number of chunks the most useful. “Reranking is an advanced rag method,” says Benoît Joly. “There is also the reformulation of questions which consists in making the question more specific by asking for an LLM to extend the request. Additionally, ” he adds.
The generation in a loop – using an LLM as a judge if information is lacking in the response of a chatbot coupled with a RAG -. The Contextual Retrieval which mixes annotation of paragraphs before their vectorization and hybrid research (BM25 and Embedding model); The Graphrag approach, combining vector representations and knowledge graphs, are some of the other techniques gauged by Meritis.
“There are many types of rag,” says Benoît Joly. “These methods were evaluated at the academic level. Problem, the resulting benchmarks are biased. The datasets have been carefully prepared so that the method presented in the article openrag: meritis wants facilitate test that accompanies it appears better than the others. ”he said.
“Openrag makes it possible to test these techniques on real use cases,” says R&D engineer. After installing the framework. the application, it is a question of loading its knowledge base, interacting with the interface to select the pipelines then use the LLM as a judge mechanism provided by Meritis in order to estimate the relevance of the issues delivered.
In addition to the results for each technique. companies can assess the number of tokens consumed, response times and obtain an estimate of the carbon impact. These indicators are displayed through a dashboard created with Streamlit.
Openrag: meritis wants facilitate test
A modular framework supposed to be robust. accessible
The tool would first serve to arbitrate technological choices, to validate RAG architecture, to test different approaches or openrag: meritis wants facilitate test even frame a project. “For example. this makes it possible to realize that a graphrag is expensive and that it is specific to a few use cases. Sometimes a naive RAG system is more than enough, ”illustrates the R&D engineer at Meritis. “Or the architects will go that a small Reranking model improves the results. that it drastically reduce the size of the LLM responsible for generating the answer at the end of the race”.
Openrag includes different tools to propel pipelines available from a GitHub deposit. For the heart of the engine, the database, the consulting firm has chosen Elasticsearch. There are a plethora of specialized databases. but those envisaged for a time – Chromadb and Milvus – were unstable with Docker. On the LLM side. Meritis is mainly based on Gemma 2-9b and Mistral Small as well as their embedding model which can be installed on a server openrag: meritis wants facilitate test or computer using VLLM and Ollama, two deployment frameworks. “It was an accessibility criterion. but we can very well call larger LLMs like those of Anthropic or Openai,” says Benoît Joly.
Each component of a RAG system can be personalized. For the moment. Openrag mainly makes it possible to modify the chunking strategy and modify the LLM torque – Embedding model.
“The differences between the way of cutting the paragraphs are enormous,” he justifies. “Some tools are only a mechanical division. while others take into account semantics in order to assimilate the end of a sentence or a segment. It is a parameter that must be correlated with the context window used: very large language models are therefore. better than the little ones who need these optimizations ”.
Other optimizations are put aside in Openrag. And for good reasons.
“Training (fine-tuner) an embedding model on its own data. provides openrag: meritis wants facilitate test undeniable performance gains,” said R&D engineer and doctor in mathematics. “On the other hand, how to do it automatically?” Most scientific articles on the subject do not contain important details. either because they are knowingly omitted, or because the article must be short to be accepted. ”
At the same time. if the web is full of resources to test the fine-tuning of embedding and LLM models, companies abandon due to lack of means and expertise.
Openrag: meritis wants facilitate test
The difficult transition from RAG architectures
“We are observing production cases on fairly simple subjects despite everything. ” says Théodore Boullier. “FAQ type IA agents, ONBOARDING AI agents, RAGs on specialized documentation for technicians accelerate information taking. These projects have proven the interest of coupling a chabot with a documentary base, ”he continues. “There are openrag: meritis wants facilitate test also many who realized that the transition to the intimate scale to rework the quality of the data. therefore to review the business processes. It is the nerve of war ”.
As for agent and multi -aging systems, they are still testing within Meritis customers. “We feel that there are still limits,” says Théodore Boullier. These first seek to industrialize and put RAG projects on the scale.
“Companies are wondering about the relevance of replacing their existing automation solutions, such as RPA. But technologies associated with LLM – A2A, MCP, etc. – are more reserved for experts. Especially since companies are demanding precision close to 100 % ”.
According to the manager, many enthusiasts in large companies appropriate technologies. “Again, we will deepen these subjects to help customers overcome obstacles,” he concludes.
Further reading: Imminent launch of the Huawei Pura 80 with a new version of the latest generation silicon – THE MINICAMÉRA INSTA360 GO 3S – The new JBL evening speaker is already 100 € less during sales at Boulanger – Baker takes out the big game for the sales by displaying this promo on the iPhone 16 Apple – 445 euros reduction not to be missed on Amazon.
