July was all about configurability and scale at Context.ai We’ve always allowed the creation of custom graphs in our workspace, but these are now dramatically more powerful with many more dimensions available to filter and to group by, as well as a new UI. Custom dashboards can now
It feels like everyone is excited about multi-agent frameworks - even though their performance isn’t yet ready for prime-time. These performance problems are improving with increasingly powerful models like Claude 3 and GPT-4o - and great things are expected from GPT-5, a launch that will likely make agentic workflows
Today we’re launching support for custom conversion events 🧾 This addresses one of the biggest challenges in the LLM ecosystem - proving ROI 📈 Context.ai users can now log custom conversion events with their LLM conversation transcripts, indicating where users completed an action: a purchase, a link click, or even
Our mission at Context.ai is to stop AI builders from flying blind. AI will have a staggering and positive impact over the next decade. We want to accelerate this transition and the enormous benefits it will bring - from healthcare to productivity. But building high performing, reliable, and safe
At Context.ai we recently completed a $3.5M first fundraise in a heavily oversubscribed round that you can read about here. I’ve since been asked for advice by many founders about how to approach an early stage fundraising process. We were lucky to have many mentors and friends
Today we’re announcing that Context.ai has raised $3.5 million in funding co-led by GV (Google Ventures) and Tomasz Tunguz at Theory Ventures. Context.ai provides product analytics for LLM-powered applications, helping businesses build better AI products.
As an AI startup founder in London, one of the most common questions I hear is “when are you moving to San Francisco?”. California is clearly a great place to start a startup, but in 2023 I think London is even better.
How do you know if the guardrails on your LLM product are working? 🛡️🎯 Some people wait until they show up in the The New York Times - like McDonald's, Air Canada, or Chevrolet Conversational LLM products are a challenging consumer experience as users can ask an infinite number
LLMs haven’t significantly improved since GPT4: is progress slowing? 🐢 Dramatically more powerful model training clusters are being built: 15 of them, with 31 times more power than trained GPT4 This means models much more powerful than GPT4 are coming 🐇 SemiAnalysis did a phenomenal deep dive into this topic -
How should you prove the ROI of your LLM product? This is key to securing resources for your team - and getting everyone promoted It’s also important to measure so you can identify and fix the weaker areas over time But what does success look like? Product teams use
How do the best tech companies find product market fit for their LLM products? 🔎 🎯 It starts with deeply understanding their users 💬🔬 Yet many people building in the LLM ecosystem still don’t know how or why people use their products. The phrase I always hear is that LLM products feel
Everyone’s talking about the Aschenbrenner AGI interview Have you seen it yet? If you haven’t got 4 hours to spare, these are my takeaways: Aschenbrenner graduated top of his class from Columbia at 19, then worked in Superalignment at OpenAI. He now runs an AI fund backed by
LLM product people need to step up - because product is the hardest problem in the ecosystem today 🪜 Challenges include deciding how to integrate LLMs into your product, proving ROI, and placing bets when it isn’t guaranteed you can build great experiences 🎰 And they’re all secondary to the
Building LLM products requires threading a needle. Product people trade off iteration speed, product quality, and safety risks - and the best PMs know how to do it 🧵🪡 ➡️ is the product going to fail embarrassingly? ➡️ when is quality good enough to launch? ➡️ are we getting enough (high quality) shots on
Are you desperate to understand how people are using your LLM product, but don’t want to share your transcripts with a third party for analysis? Good news! We now support self-hosted deployments for LLM user analytics. This allows you to run the Context.ai product within your own cloud
Today we’re launching the ecosystem’s best support for evaluating multi-call chains 🔎⛓️ This allows you to evaluate multi-stage workflows with many calls to LLMs and functions, and you can evaluate these both end-to-end and across any stage of the chain You can then debug exactly where errors are occuring
There’s a more powerful way to use LLMs in your applications: multi-agent frameworks. Most of us are familiar with making calls to an LLM directly, or using techniques like RAG to increase relevance and context that LLMs have. These approaches let you access human-like reasoning in your application, but
What did we ship in March? 🚢 Lots of improvements to evals! We now allow users to repeat LLM generations and evaluations to get more certainty in evaluation results, we version our custom evaluators, we’ve improved support for large test sets and added search, we now support Mistral models and
Context.ai now provides developers using Haystack with product analytics via the new Context.ai Haystack integration. This integration allows the easy export of transcripts from Haystack 2.0 to Context.ai for analysis. Analytics are then visible in Context.ai, allowing developers to monitor the performance of their LLM
