Trust, but Verify: Knowledge Agents for Finance Workflows - Mike Conover

The Need for AI in Financial Workflows

• 💡 Manual financial analysis is highly demanding, time-intensive, and often leads to junior analysts being overwhelmed by vast amounts of data.
• 🎯 Brightwave addresses this by building research agents that digest large content corpuses for tasks like due diligence, competitive analysis, and mutual fund research.
• 🚀 These AI agents significantly accelerate efficiency and time-to-value in financial markets by automating complex data digestion.

Designing Knowledge Agent Systems

• 🧠 A key design challenge is revealing the agent's thought process and reasoning from thousands of pages of content in a useful and legible way.
• 🛠️ Effective autonomous agents mimic human decision-making by decomposing tasks like finding comparables, assessing content, distilling findings, and error-correcting.
• ✅ Verticalized product workflows are crucial as they specify intent and remove the burden of becoming a "prompting expert" from the user.

Technical Aspects and Model Limitations

• ⚠️ Non-reasoning models often perform greedy local search, leading to exponential error rates when chaining calls.
• 🔬 Winning systems will employ end-to-end Reinforcement Learning (RL) over tool use calls to achieve globally optimal outputs, even with locally suboptimal decisions.
• 🧩 Models have limitations in synthesis due to characteristic output lengths and low presence of recombinative reasoning in training data, making it hard to weave disparate facts from many documents.

Fostering Trust and User Interaction

• 🔍 Human oversight is paramount, allowing analysts to "nudge" the model and incorporate non-digitized information or "taste-making" insights.
• ⏳ The "latency trap" highlights that long feedback loops (e.g., 8-20 minutes) hinder a user's ability to develop faculty with the system.
• ✨ Product affordances like clickable citations, structured interactive outputs, and the ability to interrogate any passage of text provide an essential audit trail and build trust.

Brightwave's Product Vision

• 📈 Brightwave's reports offer details on demand, allowing users to click on citations for context or highlight text to ask for implications.
• 📊 The system provides an audit trail by laying out all findings, enabling users to drill into specific details like patent litigation or factory fires.
• 🚀 The final form factor for these agentic products is still evolving, emphasizing the ongoing innovation in this space.