Skip to main content

AI Model Transparency Declines: Insights from the 2025 FMTI Report

[HPP] Percy LiangDecember 14, 202517 min
27 connections·39 entities in this video

Declining AI Transparency

  • 📉 The 2025 Foundation Model Transparency Index (FMTI) revealed a significant drop in average transparency scores for AI models, falling from 58 points in 2024 to 40 points.
  • ⚠️ This decline indicates a worrying trend of receding transparency, with companies increasingly concealing information, particularly regarding training data and computational resources.
  • 💡 Researchers express strong concern over this "return to opacity," noting instances where companies have removed previously public information.

Evaluation Methodology & Key Findings

  • 🔬 The study evaluated 13 foundational models using 100 indicators across three areas: upstream (model creation), model itself, and downstream (model usage).
  • 📊 Information on learning data sources, copyright handling, and computational costs for model development was found to be critically lacking.
  • 🎯 IBM's Granite model achieved the highest score at 95 points, demonstrating that transparency can be a competitive strategy, especially for B2B services.

Diverse Company Strategies

  • 🎭 Companies like IBM and Writer, serving enterprise clients, prioritize transparency, while consumer-facing giants like OpenAI (35 points), Google (41 points), and Amazon (39 points) cluster in the middle, disclosing minimal information.
  • 🚫 The concept of "open-weight" models (e.g., Meta's Llama) does not equate to transparency; Meta's score dropped from 60 to 31 as it ceased releasing detailed technical reports on training processes.
  • 💰 Cost disclosure is rare, but IBM revealed its Granite model cost approximately $10 million, with 40% allocated to data processing.

The Role of AI Agents & Environmental Impact

  • 🤖 The research experimentally used AI agents to gather information, finding they could discover an average of 13 additional pieces of information missed by humans, especially in scattered documents like terms of service.
  • 🧠 While AI agents excelled at finding specific clauses, humans remained superior in understanding the context of complex documents like model cards.
  • 🌍 A significant concern is the lack of environmental impact disclosure, with 10 out of 13 companies providing no information on energy consumption or environmental load.

Future Implications & User Action

  • ⚖️ Regulations like the EU AI Act are expected to drive future transparency, potentially mandating the disclosure of training data summaries.
  • 🗣️ Users play a crucial role by demanding transparency from AI service providers and considering FMTI scores, which can compel companies to disclose more information.
  • ✅ Transparency is not just an ethical issue but also a strategic factor in market competition and legal risk management, influencing the development of a sound AI ecosystem.
Knowledge graph39 entities · 27 connections

How they connect

An interactive map of every person, idea, and reference from this conversation. Hover to trace connections, click to explore.

Hover · drag to explore
39 entities
Chapters5 moments

Key Moments

Transcript55 segments

Full Transcript

Topics15 themes

What’s Discussed

Foundation Model Transparency Index (FMTI)AI model transparencyTraining dataComputational resourcesAI agentsEU AI ActOpen-weight modelsAI governanceCopyright issuesEnvironmental impactModel evaluationData processing costsB2B AI servicesStrategic information disclosureBlack-box problem
Smart Objects39 · 27 links
Companies· 15
Concepts· 16
Medias· 4
People· 2
Products· 2