Sourcetable's GPT-5 Evaluation: Performance, Multi-Model Architecture, and User Impact

GPT-5 Performance for Sourcetable

• ⚠️ GPT-5 models did not exceed Sourcetable's existing solution in internal evaluations, leading to no immediate integration.
• 💡 This outcome was surprising given strong public benchmarks and positive expert reviews.
• 🎯 Sourcetable's current architecture already employs a multi-model approach, which is a key feature of GPT-5, reducing the delta of improvement for them.

Open-Source Model Advantages

• 🚀 GPT-OSS models provide a ~20% performance boost over alternatives like Llama and DeepSeek for granular, inline command-bar operations.
• 🛠️ These open-source models are also effective as a prompt router, helping to select the best model for a given task within Sourcetable.

Discrepancy in Model Evaluation

• 🔍 There's a disconnect between public benchmarks and internal evaluations, with Claude models sometimes outperforming GPT-5 on spreadsheet tasks.
• 🧠 The speaker suggests that OpenAI's consumer-focused revenue (compared to Anthropic's developer focus) might influence model quality and usability for application developers.

Multi-Model Architecture Benefits

• ✅ Multi-model architecture improves app quality, reduces latency, lowers LLM costs, and provides redundancy for service uptime.
• 📈 The speaker is bullish on multi-model toolchain solutions like NotDiamond and LiteLLM, which support this approach.

Winners of the GPT-5 Release

• 👏 The biggest winner is consumers, particularly ChatGPT users, who experience improved quality and better awareness of the model's limitations.
• 🚀 Application companies like Sourcetable, Windsurf, and Cursor also benefit from general model improvements and cost savings in the ecosystem.
• ⚠️ While developers didn't see a "decisive victory," the release is still a positive step forward despite "supernova" expectations.