{"id":15982,"date":"2024-01-15T10:33:15","date_gmt":"2024-01-15T10:33:15","guid":{"rendered":"https:\/\/www.stratio.com\/blog\/?p=15982"},"modified":"2024-02-08T10:45:38","modified_gmt":"2024-02-08T10:45:38","slug":"stratio-business-semantic-data-layer-delivers-99-answer-accuracy-for-llms","status":"publish","type":"post","link":"https:\/\/www.stratio.com\/blog\/stratio-business-semantic-data-layer-delivers-99-answer-accuracy-for-llms\/","title":{"rendered":"Stratio Business Semantic Data Layer delivers 99% answer accuracy for LLMs"},"content":{"rendered":"\n

Generative AI is now ready for enterprise decisioning following benchmark test<\/strong><\/h2>\n\n\n\n

Click here<\/mark><\/a> to watch the on demand webinar “Stratio Business Semantic Data Layer delivers 99% answer accuracy for LLMs<\/em><\/strong>” and check how Stratio was able to achieve these results. Hosted by Rom\u00e1n Mart\u00edn Gonz\u00e1lez, Chief Technology Officer at Stratio BD. \u00a0<\/em>Spanish version available here<\/mark><\/a>.<\/strong><\/p>\n\n\n\n

The results, produced for the first time by linking ChatGPT analysis of independent datasets with Stratio BD\u2019s Business Semantic Data Layer<\/mark><\/a>, confirm that generative tools can now deliver reliable results from data analysis and interrogation. This means that businesses can rely on gen AI analysis to produce accurate answers, allowing employees with any level of technical expertise to ask data-related questions in their natural language, and receive answers with a high level of accuracy, immediately.<\/p>\n\n\n\n

The test conditions mirror a benchmark test<\/mark><\/a> previously conducted by data.world two months ago, which resulted in LLMs delivering three times greater efficiencies when connected to a knowledge graph. This test<\/mark><\/a> was subsequently validated in December 2023 by dbt labs, which saw the accuracy rate hit 69%. In this latest replication of the test, Stratio BD found that ChatGPT 4 was able to accurately answer data queries ran on large amounts of data, up to 99% of the time.<\/p>\n\n\n\n

Background<\/h2>\n\n\n\n

As readily-available Large Language Model (LLM) platforms like ChatGPT, Bard AI, Mistral, and Bing AI become more prevalent, business leaders are increasingly exploring the potential of Generative AI (Gen AI) technologies to transform their business operations. For instance, according to one study,  78%<\/a> <\/mark>of organizations plan to invest in AI platforms over the next 12 months. <\/p>\n\n\n\n

However, business owners need reassurance they\u2019re getting the most out of what this tech has to offer. LLMs, while capable of generating masses of human-quality text in milliseconds, may be unable to process technical data properly, giving rise to inaccurate responses (also known as \u2018hallucinations\u2019) and erroneous decision-making.
To avoid making wrong decisions, business users expect accurate results when using LLMs and Generative AI. Stratio Generative AI Data Fabric<\/mark><\/a> is able to deliver 99% answer accuracy for LLMs on the basis of a benchmark test in accordance with the methodology described in this document<\/mark><\/a>.<\/p>\n\n\n\n

Benchmark Execution<\/h2>\n\n\n\n

The following is a summary of the results obtained for each of the scenarios described below:<\/p>\n\n\n\n

To understand the results shown in the table, you need to take into consideration that:<\/p>\n\n\n\n