Building an AI-Powered Web Game: Living Canvas with Angular, Gemini, and Imagen

Introducing Living Canvas

• 💡 Living Canvas is a web-based puzzle game that leverages Generative AI to create dynamic gameplay experiences.
• 🎯 The core mechanic involves users drawing anything, which AI then analyzes to determine gameplay properties and integrate into the game world.
• 🚀 Users can also input text-based commands, which the AI interprets to trigger in-game functions.

AI-Powered Graphics Generation

• ✨ The game utilizes multiple AI models for graphics, including Gemini for image analysis and generation, and Imagen for higher-fidelity image creation.
• 🎬 VEO is employed for generating full video sequences from static images, which are then used to create animated sprite objects.
• 🎨 Players can choose from different artistic styles (realistic, cartoon, pixelated) for generated graphics.

Core Technologies and Integration

• 💻 The game is built using Angular for the front-end UI and PhaserJS as the web game engine.
• ☁️ Firebase App Hosting and Firebase Studio are used for prototyping, backend hosting, and code exploration.
• 🧠 Gemini plays a crucial role in image analysis, matching drawings to predefined properties, and dynamically assigning properties when images don't match.

Advanced AI Applications in Gameplay

• 🪄 Gemini interprets free-form text commands, transforming them into JSON representations for game engine functions, enabling features like "douse all fires."
• 🧩 For puzzle design, AI generates background graphics by interpreting black and white visual masks, creating distinct level environments.
• 🛠️ The integration of AI allows for novel gameplay experiences, such as dynamically attaching properties to any player-drawn image or enabling natural language interaction.

Technology Stack and Benefits

• ⚡ The solution demonstrates how to combine Google's cutting-edge AI technologies like Gemini, Imagen, and VEO with web development frameworks.
• 📈 Developers can explore trade-offs in cost and quality between different AI models for image and video generation.
• ✅ The use of AI unlocks new creative possibilities in game design, going beyond enhancements to enable entirely new types of experiences.