Skip to main content

How Zerve Accelerates Data Science with Parallel Execution and AI Assistance

Super Data Science: ML & AI Podcast with Jon KrohnApril 16, 20258 min157 views
19 connections·26 entities in this video→

Zerve's New Features: Fleet and AI Assistant

  • πŸš€ The new 'fleet' feature enables massive parallelization of code execution using serverless technology, allowing tasks like making thousands of calls to a large language model simultaneously.
  • πŸ’‘ This parallelization is achieved by spinning up multiple serverless compute instances, which doesn't increase costs as it uses the same amount of compute, just concurrently.
  • πŸ€– Zerve's AI assistant helps users write code within the application, build blocks, and generally modernizes the coding experience, reflecting the impact of large language models on software development.

Understanding Zerve's DAG Architecture

  • 🎯 Zerve is a block-based platform where code is arranged as a Directed Acyclic Graph (DAG), with nodes representing code blocks and connections indicating data and memory flow.
  • πŸ”— A DAG ensures a directed flow of information without any loops, effectively outlining processes like data science, data modeling, or data engineering.
  • 🀝 The architecture allows multiple users to work on the same canvas simultaneously, writing code in various languages like Python, R, and SQL, fostering real-time collaboration and visibility.

Zerve vs. Traditional Collaboration Tools

  • πŸ’¬ Zerve offers a collaborative experience similar to Google Docs, contrasting sharply with the difficulties of sharing and collaborating on traditional Jupyter notebooks.
  • ⚠️ It aims to eliminate the version control nightmares and merge conflicts common with tools like Jupyter notebooks, where file naming conventions like 'final_final_really_final' become prevalent.

Empowering Code-First Data Teams

  • ⚑ Zerve empowers code-first data teams to significantly cut down model development cycle times, potentially by up to 10x.
  • πŸ’‘ While Zerve can integrate AI assistance, it is fundamentally a coding environment, not a low-code or no-code tool, recognizing that experts writing code are the primary value generators in data science.
Knowledge graph26 entities Β· 19 connections

How they connect

An interactive map of every person, idea, and reference from this conversation. Hover to trace connections, click to explore.

Hover Β· drag to explore
26 entities
Chapters4 moments

Key Moments

Transcript31 segments

Full Transcript

Topics13 themes

What’s Discussed

ZerveData ScienceParallel ExecutionServerless TechnologyLarge Language ModelsAI AssistantDirected Acyclic Graph (DAG)Code CollaborationJupyter NotebooksCode-First Data TeamsModel DevelopmentRAGRetrieval-Augmented Generation
Smart Objects26 Β· 19 links
CompaniesΒ· 4
ConceptsΒ· 13
ProductsΒ· 9