Product Update | May 2024

Henry Scott-Green

Jun 3, 2024 — 2 min read

Are you desperate to understand how people are using your LLM product, but don’t want to share your transcripts with a third party for analysis?

Good news! We now support self-hosted deployments for LLM user analytics. This allows you to run the Context.ai product within your own cloud instance without data leaving your control. Get in touch to learn more!

This month we’ve additionally improved transcript categorization to be significantly more accurate, and we support projects to allow separation of multiple LLM products within one team or company

Self-hosted deployment

Self-hosted deployment has long been requested by the most privacy and security conscious enterprises - and we now support it.

Deployment is available via Docker image for our first beta group of customers, please get in touch if you’d like to be included in the next wave!

Improved transcript categorisation

This month we’ve rebuilt transcript categorisation (previously known as topics) to be significantly more reliable and flexible.

We still allow for intent and semantic matching, but our architecture is much more flexible to allow you to run any LLM request over your messages to assign them to categories. Templates are available to make this easy to get started.

Projects

Larger companies often have many LLM products, and these can now be split apart using our new projects feature.

Select new projects in the upper left corner of the application and generate API keys for each. Then send transcripts for each of your LLM products using the API key for that specific project to ensure data from each product is kept separate.

What product experiences are enabled by multi-agent LLM frameworks?

It feels like everyone is excited about multi-agent frameworks - even though their performance isn’t yet ready for prime-time. These performance problems are improving with increasingly powerful models like Claude 3 and GPT-4o - and great things are expected from GPT-5, a launch that will likely make agentic workflows

Launching Custom Conversion Events - Product Update | July 2024

Today we’re launching support for custom conversion events 🧾 This addresses one of the biggest challenges in the LLM ecosystem - proving ROI 📈 Context.ai users can now log custom conversion events with their LLM conversation transcripts, indicating where users completed an action: a purchase, a link click, or even

Are your LLM Products Guardrails working?

How do you know if the guardrails on your LLM product are working? 🛡️🎯 Some people wait until they show up in the The New York Times - like McDonald's, Air Canada, or Chevrolet Conversational LLM products are a challenging consumer experience as users can ask an infinite number

Is LLM progress slowing?

LLMs haven’t significantly improved since GPT4: is progress slowing? 🐢 Dramatically more powerful model training clusters are being built: 15 of them, with 31 times more power than trained GPT4 This means models much more powerful than GPT4 are coming 🐇 SemiAnalysis did a phenomenal deep dive into this topic -