13.2 C
Casper
Friday, October 11, 2024

SingleStore Unveils Features Aimed at Enabling Real-time AI

Must read

The database vendor’s new vector search capabilities, compute layer, and integrations are all designed to enable customers to build and deploy real-time generative AI models.

SingleStore unveiled a broad set of features and integrations designed to enable organizations to analyze AI models using real-time data, including improved vector search capabilities and a new compute layer.

The vendor introduced the new capabilities during SingleStore Now, a user conference in San Francisco.

Based in San Francisco, SingleStore is a database vendor whose platform is focused on speed, enabling both cloud and on-premises users to quickly ingest data as well as rapidly process queries and transactions. Competitors include MongoDB and MariaDB, while tech giants such as AWS, Google, Microsoft, and Oracle offer database platforms.

In October 2022, SingleStore raised $146 million in Series F venture capital financing to bring its total funding to $464 million. The financing came when capital market conditions were unfavorable for technology vendors, with stock prices plummeting and capital investments difficult.

At the time of the funding, SingleStore said the funds were being earmarked largely for marketing and product development purposes. Now, about a year later, the vendor is producing features and integrations with AI as their focus.

Vector search

Vector search is an important enabler of generative AI.

Generative AI models are larger than traditional AI models and need to be trained on huge amounts of data to reduce the potential for incorrect responses — termed AI hallucinations — that generative AI models automatically produce when they don’t have the right data to respond to a query.

As a result, unlike traditional AI models that may be trained on only structured data, generative AI models are generally trained using structured data, such as financial records and transaction logs, and unstructured data, such as text, audio, and images.

That additional data gives the model more data to draw on, reducing its inclination to make up an answer should it not have the right data.

However, training a massive generative AI model with the most pertinent data is difficult, which is why vector search is gaining importance as more organizations look to build customized large language models (LLMs).

Vector searches enable data engineers and other model developers to discover data based on its similarity to other data, which reduces the time and effort it takes to search for and find all the relevant data for a particular generative AI model.

SingleStore has offered exact nearest neighbor vector searches, which enable customers to find exact matches within their data, since 2017.

Now, the vendor is adding an approximate nearest neighbor (ANN) search, which not only expands the parameters of a given search and its potential results but substantially increases the performance speed of searches and the accuracy of those searches, SingleStore said in a press release.

That increased speed, meanwhile, is significant for those developing generative AI models, typically using text-based JSON documents, according to Doug Henschen, an analyst at Constellation Research.

“Approximate nearest neighbor is much faster than exact nearest neighbor search. And it’s more appropriate to the semi-structured … data likely to be associated with generative AI,” he said.

Some SingleStore users may find the new compute layer a more significant addition, Henschen continued. But for those concerned largely with generative AI, ANN search in concert with other vector search improvements is important.

In addition to ANN search, SingleStore aims to improve the speed of vector searches by providing the following:

  • New algorithms for vector indexing that reduce the time it takes to build indexes within vector databases.
  • Combined vector-based semantic search with exact keyword search to speed retrieval-augmented generation time to foster real-time analysis.
  • A simple interface that enables customers to use SQL or MongoDB’s query language rather than query languages specific to a given vector database.

“The standout announcement is really in the eye of the beholder,” Henschen said. “For those entirely focused on generative AI innovation, [it is] the addition of approximate nearest neighbor vector search for faster performance … along with the combination of semantic and full-text search.”

Meanwhile, the impetus for developing the new vector search capabilities came from a combination of customer requests and SingleStore on its own recognizing a need, according to Madhukar Kumar, the vendor’s chief marketing officer.

“Every customer we talk to is thinking about [building] LLMs or has already started the process,” he said.

However, customers quickly realize that developing LLMs takes significant time, effort, and computing power, which leads to significant costs. Speed and efficiency reduce those costs.

“Cost and efficiency become a huge part of creating vectors and searching them in real-time,” Kumar said. “[Real time] was our vision going forward. But with LLMs, that has become front and center.”

More capabilities

Beyond improved vector search capabilities, SingleStore unveiled Aura, a new compute layer on the vendor’s database platform that lets users deploy GPUs and CPUs for AI and machine learning workloads near where their data resides.

The compute layer enables developers to build LLMs and machine learning models without having to move data outside the SingleStore platform, which not only saves time and effort by eliminating extract, transform, and load workloads but keeps data in a governed environment to ensure data security.

When organizations are forced to export data to combine it with LLM capabilities, they risk exposing their data to breaches. But when they can instead import the LLM capabilities of their data within a database, the risk of exposure is reduced.

In addition to Aura, the vendor launched SingleStore Notebooks, web-based Jupyter notebooks where developers can collaborate to build models and other data science assets using SQL or Python code.

Included in Notebooks is Spaces, a set of prebuilt notebooks that can help customers develop generative AI applications, recommendation engines, and real-time workflows.

Meanwhile, included in Aura and connected to Notebooks is Job Service, which enables customers to schedule SQL and Python jobs — including automating repeated tasks — from within SingleStore Notebooks.

Radian Replication Service is related to both Aura and Notebooks, a set of services that lets users replicate data from various sources into SingleStore.

Aura is in limited preview, while Job Service is in private preview. Both Notebooks and Radian Replication Service are generally available.

Together, Aura and its related features will prove even more significant for SingleStore customers than the improved vector search capabilities, according to Henschen.

“To me, the most important collection of new capabilities is the SingleStore Aura ‘intelligence layer’ and related Notebook, Job Service, and Radian data replication service announcements,” he said. “All of these things will work together to enable customers to do more with their data without copying it or moving it out of SingleStore.”

A new free pricing tier is also now available. SingleStore previously offered a free trial with $600 of credits. Now, however, the vendor’s SingleStoreDB Free level provides users with a sandbox environment for hosting data and testing workloads.

SingleStore otherwise is available in three tiers, with Standard starting at 80 cents per hour; Premium at $1.60 per hour, with support for mission-critical workloads; and Dedicated for an unlisted amount for enterprises with significant security concerns.

Finally, SingleStore unveiled new and improved integrations that enable developers to build generative AI models using the platform of their choice.

Among them is an extension of an existing integration with Google that now includes Vertex AI, a one-click feature so users can deploy LLMs on AWS using Amazon SageMaker; two integrations with IBM’s Watsonx; and numerous connectors to open source platforms, including OpenAI and Vercel.

While the integrations, vector search capabilities, and new compute layer are seemingly separate features, they work together to move models from development through deployment, according to Kumar.

Although vector search may not seem related to integration with SageMaker, vector search is what helps begin the process of developing an AI model that eventually gets put into production.

“Hybrid search that includes vectors, a layer of compute that makes it easier and faster to run workloads next to the data, and the ability to run in all the ecosystems is our overall platform,” he said.

Down the road

As SingleStore continues to invest in product development, AI will remain a priority, according to Kumar.

In particular, the vendor will focus on enabling customers to use their data to develop models that inform decisions and making the data informing those models as close to real-time as possible.

“AI is not useful — in fact, it’s gimmicky — unless it is aware of your data,” Kumar said. “What will be critical is the ability to query and curate that data in split seconds. Our vision is to give companies everything they need to contextualize that data in real-time.”

He added that customers have historically used SingleStore for transactions and analytics. The next step for the vendor will be to add contextualization capabilities.

Henschen, meanwhile, noted that SingleStore would be wise to continue investing in marketing. In particular, increasing awareness of its cloud-based capabilities is key.

“Anything SingleStore can do to promote broader recognition, adoption, and use of the SingleStore Helios cloud service will be to the long-term benefit of the vendor and its core product,” Henschen said. “Scale is everything, whether talking about numbers of customers and users, economies of cloud resources in use, or funds for future research and development.”

More articles

Latest posts